Hi Yongheng, First of all I need to warn you that a 114GB file is going to cause you all kind of unexpected problems. Are you sure all your results need to live in one big file ? To debug your acute problem, assuming you have a CentOS7 node with cvfms enabled, you could try uploading the file with palin gfal and registering it by hand. In a *clean* (no dirac!) window do: source /cvmfs/ grid.cern.ch/umd-c7ui-test/etc/profile.d/setup-c7-ui-example.sh voms-proxy-init --voms t2k.org gfal-copy -v -T 10800 -t 10800 run4a_FHC_combined.root srm:// lcg-t2kse1.sfu.computecanada.ca:8443/srm/managerv2?SFN=/nd280data/t2k.org/users/yoneheng.xu/run4a_FHC_combined.root (The timeout is in seconds, here it's meant to be 3 h.) Let me know if that works. Regards, Daniela On Thu, 12 Mar 2020 at 20:27, Xu, Yongheng (Student) <y.xu39@lancaster.ac.uk> wrote:
This email from y.xu39@lancaster.ac.uk originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list <https://spam.ic.ac.uk/SpamConsole/Senders.aspx> to disable email stamping for this address.
Hi grid users,
I am Yongheng Xu from Lancaster and I am having trouble uploading a single large root file (114GB) to CA-SFU-T21-disk using dirac. What I did was
dirac-dms-add-file /t2k.org/users/yoneheng.xu/run4a_FHC_combined.root ~/p6t_validation_samples/r4a.root CA-SFU-T21-disk -ddd
However, it runs for a while (~1 hour) then fails and gives the following error message:
2020-03-12 16:26:20 UTC dirac-dms-add-file [140363232319296] DEBUG: New session connecting to server at ('dirac01.grid.hep.ph.ic.ac.uk', 9133)
2020-03-12 16:26:20 UTC dirac-dms-add-file [140363232319296] DEBUG: Connected to: dips:// dirac01.grid.hep.ph.ic.ac.uk:9133/Accounting/DataStore
2020-03-12 16:26:20 UTC dirac-dms-add-file [140363232319296] DEBUG: New connection -> 146.179.232.10:9133
2020-03-12 16:26:20 UTC dirac-dms-add-file [140363232319296] DEBUG: Closing socket
2020-03-12 16:26:20 UTC dirac-dms-add-file/DataManager/putAndRegister [140363232319296] DEBUG: putAndRegister: Sending accounting took 0.0 seconds
2020-03-12 16:26:20 UTC dirac-dms-add-file/DataManager/putAndRegister [140363232319296] DEBUG: Failed to put file to Storage Element. /home/t2k/yxu/p6t_validation_samples/r4a.root: Connection timed out ( 110 : Failed to copy file /home/t2k/yxu/p6t_validation_samples/r4a.root to destination url srm:// lcg-t2kse1.sfu.computecanada.ca:8443/srm/managerv2?SFN=/nd280data/t2k.org/users/yoneheng.xu/run4a_FHC_combined.root: [110] [gfalt_copy_file][perform_copy][srm_plugin_filecopy][srm_do_transfer][gfalt_copy_file][perform_copy][perform_local_copy][streamed_copy][gfal_plugin_closeG][gfal_gridftp_closeG][gfal_gridftp_closeG] Operation timed out) 2020-03-12 16:26:20 UTC dirac-dms-add-file [140363232319296] ERROR: Error: failed to upload / t2k.org/users/yoneheng.xu/run4a_FHC_combined.root to CA-SFU-T21-disk
I have tested with smaller files and they can be uploaded without any problem. Does anyone know what might be the cause of this issue? The full log is attached to this email.
Many thanks,
Yongheng Xu -- _______________________________________________ Gridpp-Dirac-Users mailing list Gridpp-Dirac-Users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/gridpp-dirac-users
-- Sent from the pit of despair ----------------------------------------------------------- daniela.bauer@imperial.ac.uk HEP Group/Physics Dep Imperial College London, SW7 2BW Tel: +44-(0)20-75947810 http://www.hep.ph.ic.ac.uk/~dbauer/