******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Hi Everyone As far as I can tell, DIRAC does not like me uploading files that end in .cfg to the DFC. Below you can see my attempts at uploading a file with .cfg and how the problem is fixed if I move it to a .txt file. Both local and lfn have what.cfg - nothing happens - get no complaint, but if I check, the file has not uploaded king@heppc105:~ $ dirac-dms-add-file /t2k.org/user/k/sophie.king/what/what*.cfg* ./what*.cfg* CA-SFU-T21-disk Try again with debug - no error, just seems to stop, and still no upload king@heppc105:~ $ dirac-dms-add-file /t2k.org/user/k/sophie.king/what/what*.cfg* ./what*.cfg* CA-SFU-T21-disk -ddd 2020-04-12 19:27:16 UTC Framework [140440172357376] DEBUG: dirac.cfg should be at /.data/king/grid/dirac/dirac_ui/etc/dirac.cfg 2020-04-12 19:27:16 UTC Framework [140440172357376] DEBUG: CFG merged 2020-04-12 19:27:16 UTC Framework [140440172357376] DEBUG: Updating configuration internals 2020-04-12 19:27:16 UTC Framework [140440172357376] DEBUG: Updating configuration internals Parsing command line Loading file /t2k.org/user/k/sophie.king/what/what.cfg Could not load file /t2k.org/user/k/sophie.king/what/what.cfg: Can't load a cfg file '/t2k.org/user/k/sophie.king/what/what.cfg' Loading file ./what.cfg CFG merged Updating configuration internals Trying to load Resources.LogBackends.StdoutBackend Trying to load DIRAC.Resources.LogBackends.StdoutBackend 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Refreshing configuration... 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Refreshing from list ['dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server', 'dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server', 'dips://diracweb.grid.hep.ph.ic.ac.uk:9135/Configuration/Server'] 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Randomized server list is dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server, dips://diracweb.grid.hep.ph.ic.ac.uk:9135/Configuration/Server, dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Already given a valid url dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to refresh from dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to connect to: dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Using proxy /tmp/x509up_u32226 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: CAs location is /etc/grid-security/certificates 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Loaded 92 CAs [184 CRLs] 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Connection timeout set to: 1 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Authenticated peer (/C=UK/O=eScience/OU=Imperial/L=Physics/CN=dirac02.grid.hep.ph.ic.ac.uk) 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: New session connecting to server at ('dirac02.grid.hep.ph.ic.ac.uk', 9135) 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Connected to: dips://dirac02.grid.hep.ph.ic.ac.uk:9135/Configuration/Server 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: New connection -> 146.179.232.11:9135 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: Closing socket 2020-04-12 19:27:16 UTC dirac-dms-add-file [140440172357376] DEBUG: New version available Updating to version 2020-04-12 18:53:18.780475... 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Updating configuration internals 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Updated to version 2020-04-12 18:53:18.780475 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to load Resources.LogBackends.StdoutBackend 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to load WebAppDIRAC.Resources.LogBackends.StdoutBackend 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to load GridPPDIRAC.Resources.LogBackends.StdoutBackend 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to load VMDIRAC.Resources.LogBackends.StdoutBackend 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to load DIRAC.Resources.LogBackends.StdoutBackend 2020-04-12 19:27:17 UTC dirac-dms-add-file/FileCatalogFactory [140440172357376] DEBUG: Creating FileCatalog client of type FileCatalog 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to load Resources.Catalog.FileCatalogClient 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to load WebAppDIRAC.Resources.Catalog.FileCatalogClient 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to load GridPPDIRAC.Resources.Catalog.FileCatalogClient 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to load VMDIRAC.Resources.Catalog.FileCatalogClient 2020-04-12 19:27:17 UTC dirac-dms-add-file [140440172357376] DEBUG: Trying to load DIRAC.Resources.Catalog.FileCatalogClient 2020-04-12 19:27:17 UTC dirac-dms-add-file/FileCatalogFactory [140440172357376] DEBUG: Loaded module FileCatalogClient Keep local file as .cfg but change lfn to .txt - get an error king@heppc105:~ $ dirac-dms-add-file /t2k.org/user/k/sophie.king/what/what.txt ./what.cfg CA-SFU-T21-disk Traceback (most recent call last): File "/.data/king/grid/dirac/dirac_ui/DIRAC/DataManagementSystem/scripts/dirac-dms-add-file.py", line 71, in <module> lfns.append( getDict( args ) ) File "/.data/king/grid/dirac/dirac_ui/DIRAC/DataManagementSystem/scripts/dirac-dms-add-file.py", line 52, in getDict lfn_dict['SE'] = item_list[2] IndexError: list index out of range Ensure both the local file and the lfn have .txt - Finally, this does work king@heppc105:~ $ mv what.cfg what.txt king@heppc105:~ $ dirac-dms-add-file /t2k.org/user/k/sophie.king/what/what.txt ./what.txt CA-SFU-T21-disk Uploading /t2k.org/user/k/sophie.king/what/what.txt Successfully uploaded file to CA-SFU-T21-disk I can reproduce this problem with multiple tests. We can do a workaround of course, but I was wondering if this a known thing, that DIRAC doesnt like the local or LFN file to have .cfg ? And can it be fixed or is it the by-product of something deeper and we must just live with it? It would seem quite sensible/common that one may have some files on the grid, and for bookkeeping purposes want to preserve a file that ends in .cfg that was used as input to the software that created said files. Cheers Sophie