This email from sgagrove@student.liverpool.ac.uk originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list to disable email stamping for this address.

 

Hi


Thanks for than. My proxy should be correct (I have used the same command to delete the another directory) however for this quite sizeable one is still throws a error. Ive included the debug for it as I don't understand it.

Is there a limit on the size which it will work?


Thanks

Andrew


[groves@gamma.ph.liv.ac.uk GRID]> dirac-dms-clean-directory /mu3e.org/prod/build4.4.1/BhabbaMichel -ddd
2020-12-18 10:31:56 UTC Framework [139810673047360] DEBUG: dirac.cfg should be at /user/agroves/dirac_ui/etc/dirac.cfg
2020-12-18 10:31:56 UTC Framework [139810673047360] DEBUG: CFG merged
2020-12-18 10:31:56 UTC Framework [139810673047360] DEBUG: Updating configuration internals
2020-12-18 10:31:56 UTC Framework [139810673047360] DEBUG: Updating configuration internals
Parsing command line
Trying to load Resources.LogBackends.StdoutBackend
Trying to load DIRAC.Resources.LogBackends.StdoutBackend
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Refreshing configuration...
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Refreshing from list ['dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server', 'dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server', 'dips://diracweb.grid.hep.ph.ic.ac.uk:9135/Configuration/Server']
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Randomized server list is dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server, dips://diracweb.grid.hep.ph.ic.ac.uk:9135/Configuration/Server, dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Already given a valid url dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG:  Trying to refresh from dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to connect to: dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Using proxy /user/agroves/.globus/x509up_u10233
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: CAs location is /cvmfs/grid.cern.ch/etc/grid-security/certificates
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Loaded 94 CAs [188 CRLs]
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Connection timeout set to:  1
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG:  Authenticated peer (/C=UK/O=eScience/OU=Imperial/L=Physics/CN=dirac02.grid.hep.ph.ic.ac.uk)
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: New session connecting to server at ('dirac02.grid.hep.ph.ic.ac.uk', 9135)
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Connected to: dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: New connection -> 2a0c:5bc0:c8:2:216:3eff:fe7f:1c0:9135
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Closing socket
2020-12-18 10:31:56 UTC dirac-dms-clean-directory [139810673047360] DEBUG: New version available Updating to version 2020-12-18 09:33:23.654869...
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Updating configuration internals
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Updated to version 2020-12-18 09:33:23.654869
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to load Resources.LogBackends.StdoutBackend
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to load WebAppDIRAC.Resources.LogBackends.StdoutBackend
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to load GridPPDIRAC.Resources.LogBackends.StdoutBackend
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to load VMDIRAC.Resources.LogBackends.StdoutBackend
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to load DIRAC.Resources.LogBackends.StdoutBackend
2020-12-18 10:31:57 UTC dirac-dms-clean-directory/FileCatalogFactory [139810673047360] DEBUG: Creating FileCatalog client of type FileCatalog
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to load Resources.Catalog.FileCatalogClient
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to load WebAppDIRAC.Resources.Catalog.FileCatalogClient
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to load GridPPDIRAC.Resources.Catalog.FileCatalogClient
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to load VMDIRAC.Resources.Catalog.FileCatalogClient
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to load DIRAC.Resources.Catalog.FileCatalogClient
2020-12-18 10:31:57 UTC dirac-dms-clean-directory/FileCatalogFactory [139810673047360] DEBUG: Loaded module FileCatalogClient
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] NOTICE: Cleaning directory '/mu3e.org/prod/build4.4.1/BhabbaMichel' ... 
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Discovering URL for service DataManagement/FileCatalog -> dips://dirac01.grid.hep.ph.ic.ac.uk:9197/DataManagement/FileCatalog
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to connect to: dips://dirac01.grid.hep.ph.ic.ac.uk:9197/DataManagement/FileCatalog
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Using proxy /user/agroves/.globus/x509up_u10233
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Connection timeout set to:  1
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG:  Authenticated peer (/C=UK/O=eScience/OU=Imperial/L=Physics/CN=dirac01.grid.hep.ph.ic.ac.uk)
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: New session connecting to server at ('dirac01.grid.hep.ph.ic.ac.uk', 9197)
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Connected to: dips://dirac01.grid.hep.ph.ic.ac.uk:9197/DataManagement/FileCatalog
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: New connection -> 2a0c:5bc0:c8:2:216:3eff:fe7f:1bf:9197
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Closing socket
2020-12-18 10:31:57 UTC dirac-dms-clean-directory/DataManager/__getCatalogDirectoryContents [139810673047360] DEBUG: Obtaining the catalog contents for 1 directories:
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Discovering URL for service DataManagement/FileCatalog -> dips://dirac01.grid.hep.ph.ic.ac.uk:9197/DataManagement/FileCatalog
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to connect to: dips://dirac01.grid.hep.ph.ic.ac.uk:9197/DataManagement/FileCatalog
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Using proxy /user/agroves/.globus/x509up_u10233
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Connection timeout set to:  1
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG:  Authenticated peer (/C=UK/O=eScience/OU=Imperial/L=Physics/CN=dirac01.grid.hep.ph.ic.ac.uk)
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: New session connecting to server at ('dirac01.grid.hep.ph.ic.ac.uk', 9197)
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Connected to: dips://dirac01.grid.hep.ph.ic.ac.uk:9197/DataManagement/FileCatalog
2020-12-18 10:31:57 UTC dirac-dms-clean-directory [139810673047360] DEBUG: New connection -> 2a0c:5bc0:c8:2:216:3eff:fe7f:1bf:9197
2020-12-18 10:32:37 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Closing socket
2020-12-18 10:32:37 UTC dirac-dms-clean-directory/DataManager/__getCatalogDirectoryContents [139810673047360] DEBUG: Found 124608 files
2020-12-18 10:32:38 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Discovering URL for service DataManagement/FileCatalog -> dips://dirac01.grid.hep.ph.ic.ac.uk:9197/DataManagement/FileCatalog
2020-12-18 10:32:38 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Trying to connect to: dips://dirac01.grid.hep.ph.ic.ac.uk:9197/DataManagement/FileCatalog
2020-12-18 10:32:38 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Using proxy /user/agroves/.globus/x509up_u10233
2020-12-18 10:32:38 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Connection timeout set to:  1
2020-12-18 10:32:38 UTC dirac-dms-clean-directory [139810673047360] DEBUG:  Authenticated peer (/C=UK/O=eScience/OU=Imperial/L=Physics/CN=dirac01.grid.hep.ph.ic.ac.uk)
2020-12-18 10:32:38 UTC dirac-dms-clean-directory [139810673047360] DEBUG: New session connecting to server at ('dirac01.grid.hep.ph.ic.ac.uk', 9197)
2020-12-18 10:32:38 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Connected to: dips://dirac01.grid.hep.ph.ic.ac.uk:9197/DataManagement/FileCatalog
2020-12-18 10:32:38 UTC dirac-dms-clean-directory [139810673047360] DEBUG: New connection -> 2a0c:5bc0:c8:2:216:3eff:fe7f:1bf:9197
2020-12-18 10:34:44 UTC dirac-dms-clean-directory [139810673047360] DEBUG: Closing socket
2020-12-18 10:34:44 UTC dirac-dms-clean-directory/DataManager/cleanLogicalDirectory [139810673047360] DEBUG: Failed to clean directory. /mu3e.org/prod/build4.4.1/BhabbaMichel FileCatalog error ( 1604 : Failed to perform isDirectory from any catalog)
2020-12-18 10:34:44 UTC dirac-dms-clean-directory [139810673047360] ERROR: Failed to clean folder '/mu3e.org/prod/build4.4.1/BhabbaMichel': FileCatalog error ( 1604 : Failed to perform isDirectory from any catalog)


From: gridpp-dirac-users-bounces@imperial.ac.uk <gridpp-dirac-users-bounces@imperial.ac.uk> on behalf of Simon Fayer <simon.fayer05@imperial.ac.uk>
Sent: 17 December 2020 17:04:17
To: gridpp-dirac-users@imperial.ac.uk
Subject: Re: [Gridpp-Dirac-Users] Questions on dirac file catolog
 
Hi Andrew,

On Thu, Dec 17, 2020 at 04:46:49PM +0000, Groves, Andrew wrote:
> Im trying to remove files on mass from the dirac file catolog (a whole directories worth) i've tried the command dirac-dms-clean-directory however I keep getting the error Failed to perform isDirectory from any catolog.
>
> Im trying dirac-dms-clean-directory /mu3e.org/prod/build4.4.1.

Hmm, I think this should work (I tried it out on a small test case and
verified the permissions on the directory you're trying to remove). Can you
please verify that your proxy is valid and in the correct group?

If the proxy looks OK, try adding the -ddd option and see if the extra
debug gives you any further clues as to what the problem may be.

> Also im trying to use dirac-dms-data-size, could I do this on the whole directory in one command or only on a list of files.

It can only take a list of files, but you can give it a file containing a
list of files (which you can generate with another command):

dirac-dms-find-lfns Path=/mu3e.org/... > files.lst
dirac-dms-data-size files.lst

Regards,
Simon


--
_______________________________________________
Gridpp-Dirac-Users mailing list
Gridpp-Dirac-Users@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/gridpp-dirac-users