******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Hi Daniela, Thanks for your reply. Unfortunately​, I get error messages and crashes when I try the gfal-copy command. (The first two commands work fine). I have opened a GGUS ticket. Thanks for all your help. Best Regards Aditya Upreti | Ph.D. Candidate / Research Assistant [Divider line] Department of Physics The University of Alabama<https://www.ua.edu/> 102 Gallalee Hall Tuscaloosa, AL 35404 Phone 205-886-2914 aupreti@crimson.ua.edu [Divider line] [The University of Alabama box A with stacked words logo]<https://www.ua.edu/> ________________________________ From: Daniela Bauer <daniela.bauer.grid@googlemail.com> Sent: Thursday, October 7, 2021 9:02 AM To: Aditya Upreti <aupreti@crimson.ua.edu> Cc: gridpp-dirac-users@imperial.ac.uk <gridpp-dirac-users@imperial.ac.uk>; Daniel Traynor <d.traynor@qmul.ac.uk>; Simon Fayer <simon.fayer05@imperial.ac.uk> Subject: Re: Issue regrading GRID Hi Aditya and anybody else who has seen such an error before, Step one should be to test your access to the directory on the SE in question outside DIRAC. For this you ideally need an SL7 machine with cvmfs access (a CERNVM will do). There you do the following (in a *clean* terminal, with neither Ganga nor DIRAC set up): source /cvmfs/grid.cern.ch/umd-c7ui-latest/etc/profile.d/setup-c7-ui-example.sh<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgrid.cern.ch%2Fumd-c7ui-latest%2Fetc%2Fprofile.d%2Fsetup-c7-ui-example.sh&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166499706%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GFzlKL33h%2BDiXQ5gpCDtk61LrSiKpaIZyEdWlzRueug%3D&reserved=0> voms-proxy-init --voms [your VO goes here] (here: voms-proxy-init --voms vo.moedal.org<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2Fdummy1.txt&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166509661%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=nm0FXWQIB9WTH6jOGmzmxTasqi%2BI7tEisLyqAPWsFdA%3D&reserved=0>) gfal-copy -vvv myfile.txt fullpath_it_fails_to_copy_to (here: gfal-copy -vvv dummy1.txt srm://se03.esc.qmul.ac.uk:8444/srm/managerv2?SFN=/vo.moedal.org/vo.moedal.org/user/a/aditya.upreti/dummy1.txt<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fse03.esc.qmul.ac.uk%3A8444%2Fsrm%2Fmanagerv2%3FSFN%3D%2Fvo.moedal.org%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2Fdummy1.txt&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166509661%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1%2Bmh5PakZ4BWw%2Bg5pEhu%2BTqEgvU4Hn20xO4FIORQWkI%3D&reserved=0> ) If this fails, please open a GGUS ticket (https://ggus.eu/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fggus.eu%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166519618%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=aPC8cccv26PeWWGCSQKBcV2INpvdDQ%2BBN5dA5d4jJNM%3D&reserved=0>). Please always try and state which site you are dealing with - or - you can notify the site directly under 'Routing information' (here it would be: UKI-LT2-QMUL). If this works, please get back to this list. I can't comment on the LocalFile option as I have never seen it. Please be aware that the DIRAC team has no experience with Ganga, though there might be people on this list who do. Regards, Daniela On Thu, 7 Oct 2021 at 02:20, Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>> wrote: Hi Daniela, Daniel, Thanks for your help in the matter, but I'm still experiencing issues with running simulations on GRID. The permissions issue isn't resolved yet. I'm still getting the following error message when I try to upload a file on UKI-LT2-QMUL2-disk. 2021-10-06 22:18:18 UTC dirac-dms-add-file/DataManager/putAndRegister [140605462013760] DEBUG: Failed to put file to Storage Element. dummy1.txt: Permission denied ( 13 : Failed to copy file dummy1.txt to destination url srm://se03.esc.qmul.ac.uk:8444/srm/managerv2?SFN=/vo.moedal.org/vo.moedal.org/user/a/aditya.upreti/dummy1.txt<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fse03.esc.qmul.ac.uk%3A8444%2Fsrm%2Fmanagerv2%3FSFN%3D%2Fvo.moedal.org%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2Fdummy1.txt&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166519618%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=C1%2B%2Bv%2F50XTcNfsPFVBemmNt87RQ2RbpKvn8bEepODqI%3D&reserved=0>: [13] [gfalt_copy_file][perform_copy][srm_plugin_filecopy][srm_do_transfer][gfalt_copy_file][perform_copy][perform_local_copy] Could not open destination: [streamed_copy][gfal_plugin_openG][gfal_gridftp_openG][gfal_gridftp_openG] globus_ftp_client: the server responded with an error 530 530-Login incorrect. : globus_gss_assist: Error invoking callout 530-globus_callout_module: The callout returned an error 530-an unknown error occurred 530 End. ) 2021-10-06 22:18:18 UTC dirac-dms-add-file [140605462013760] ERROR: Error: failed to upload /vo.moedal.org/user/a/aditya.upreti/dummy1.txt<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2Fdummy1.txt&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166529584%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9YatRycaO9xNxrIldgd6%2BgsYRygrKVDHh1sBxJwzP7U%3D&reserved=0> to UKI-LT2-QMUL2-disk The job fails with the same permissions issue when I use the following code (DiracFile to get the output). I tried this approach as an alternative to submitting it via a JDL script as I described earlier. j = Job() j.name<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fj.name%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166529584%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RIRUvkf4K1ngB3XTvmJruMoSX%2F5rLTzo8SZrAtiuLPM%3D&reserved=0> = job_name j.application = Executable() j.application.exe = File(sh_run_ganga_name) j.application.args = [] j.inputfiles = [LocalFile(cfg_path), LocalFile(cfg_ganga_pgun_name), LocalFile(input3Dpath+input3Dhist)] j.outputfiles =[DiracFile(outputfile)] j.backend = Dirac() j.submit() ('/vo.moedal.org/user/a/aditya.upreti/32164/32164451/MonopoleData_2_100_default_0.root<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F32164%2F32164451%2FMonopoleData_2_100_default_0.root&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166539536%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=HseIArlr4eV93mcS1tT%2BtCFVMfnYK8h2gq%2FUYzRIXRs%3D&reserved=0>','/scratch/condor/dir_11177/WcfLDmo4Uszn8FVDjqQVj3jqav4PzpABFKDm9FCSDmABFKDmtftwXo/DIRAC_YjF9xbpilot/32164451/MonopoleData_2_100_default_0.root','UKI-LT2-QMUL2-disk',guid='F11FBC28-DC8A-C4ED-2292-AF1995B2A826',catalog='[]', checksum = 'b8149d64') dm.putAndRegister failed with message Failed to put file to Storage Element. Permission denied ( 13 : Failed to copy file Then, I tried to get the output files locally using LocalFile instead of DiracFile, using the following lines of code j = Job() j.name<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fj.name%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166539536%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=j2W8VE25uxttDw%2FcxnDSUTqepgGU8ad%2Fjfm5eMgsrfk%3D&reserved=0> = job_name j.application = Executable() j.application.exe = File(sh_run_ganga_name) j.application.args = [] j.inputfiles = [LocalFile(cfg_path), LocalFile(cfg_ganga_pgun_name), LocalFile(input3Dpath+input3Dhist)] j.outputfiles =[LocalFile('/eos/experiment/moedal/aditya/GRID_outputs/test/'+outputfile)] j.backend = Dirac() j.submit() and it gives me the following error message in std.out. The simulation stops and job fails halfway. UTC dirac-jobexec/Subprocess ERROR: Maximum output buffer length reached 2021-10-05 22:30:19 UTC dirac-jobexec/Script ERROR: System call execution failed: Reached maximum allowed length (52428800 bytes) for called function return value for '['/scratch/condor/dir_7759/OGNNDmQ46rznOOVDjqUTj3jq6xrg1pABFKDm2f6LDmABFKDmuey6Zn/DIRAC__a2iHspilot/32155611/exe-script.py']' call 2021-10-05 22:30:19 UTC dirac-jobexec/Script ERROR: Non-zero status while executing /scratch/condor/dir_7759/OGNNDmQ46rznOOVDjqUTj3jq6xrg1pABFKDm2f6LDmABFKDmuey6Zn/DIRAC__a2iHspilot/32155611/exe-script.py exited with status 9 Please let me know if there's anything else I can provide to help us debug these errors. Best Regards Aditya Upreti | Ph.D. Candidate / Research Assistant [Divider line] Department of Physics The University of Alabama<https://www.ua.edu/> 102 Gallalee Hall Tuscaloosa, AL 35404 Phone 205-886-2914 aupreti@crimson.ua.edu [Divider line] [The University of Alabama box A with stacked words logo]<https://www.ua.edu/> ________________________________ From: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>> Sent: Monday, September 27, 2021 4:37 AM To: Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>> Cc: Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>>; Simon Fayer <simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk>> Subject: Re: Issue regrading GRID Hi Aditya, I don't know Ganga, but in DIRAC there is the concept of an OutputSandbox: This is for small files, and you can retrieve them via dirac-wms-job-get-output. For the actual data, you should use the various data handling mechanism within DIRAC which will upload your data to an SE. There is some documentation here: https://www.gridpp.ac.uk/wiki/DIRAC_Data_Handling_within_a_Job<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gridpp.ac.uk%2Fwiki%2FDIRAC_Data_Handling_within_a_Job&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166549488%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=OR6hnfoVA2eOWhVJoHSqVNj%2B4%2BHU0jxoROd2BFoBOwA%3D&reserved=0> Regards, Daniela On Mon, 27 Sept 2021 at 05:54, Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>> wrote: Hi Daniela, Sorry for the late reply. Okay, I can try writing to RALPP. Do I have to mention RALPP in the OutputSandbox instead of UKI-LT2-QMUL2-disk? Also, I am able to submit jobs via the Ganga interface using a python submit script and getting the output file locally using "j = Job() j.outputfiles = [LocalFile(outputfile)]" which outputs the file locally, but I'm not sure why I'm failing to do that with a JDL script directly, bypassing Ganga. (as with Ganga, I get some python dependency errors like 'no module named six'. etc, when submitting the jobs"). Best Regards Aditya Upreti | Ph.D. Candidate / Research Assistant [Divider line] Department of Physics The University of Alabama<https://www.ua.edu/> 102 Gallalee Hall Tuscaloosa, AL 35404 Phone 205-886-2914 aupreti@crimson.ua.edu [Divider line] [The University of Alabama box A with stacked words logo]<https://www.ua.edu/> ________________________________ From: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>> Sent: Thursday, September 23, 2021 4:20 AM To: Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>> Cc: Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>; Simon Fayer <simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk>> Subject: Re: Issue regrading GRID @Aditya Upreti<mailto:aupreti@crimson.ua.edu> : In the meantime, do you want to try and write to RALPP ? You should be able to copy the data later if needed. Regards, Daniela On Wed, 22 Sept 2021 at 18:38, Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>> wrote: not yet. * Dr Daniel Traynor, Grid cluster system manager * Tel +44(0)20 7882 6560, Particle Physics,QMUL ________________________________________ From: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>> Sent: 22 September 2021 18:33 To: Aditya Upreti Cc: Daniel Traynor; Simon Fayer Subject: Re: Issue regrading GRID This is a question for Dan - we need to work out where the permission denied comes from. @Dan: Any idea ? Cheers, Daniela On Tue, 21 Sept 2021 at 21:59, Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu><mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>> wrote: Hi all, Any resolution on this issue? Thanks! Best Regards Aditya Upreti | Ph.D. Candidate / Research Assistant [Divider line] Department of Physics The University of Alabama<https://www.ua.edu/> 102 Gallalee Hall Tuscaloosa, AL 35404 Phone 205-886-2914 aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu> [Divider line] [The University of Alabama box A with stacked words logo]<https://www.ua.edu/> ________________________________ From: Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu><mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>> Sent: Monday, September 20, 2021 10:53 AM To: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com><mailto:daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>>> Cc: Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk><mailto:d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>>>; Simon Fayer <simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk><mailto:simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk>>> Subject: Re: Issue regrading GRID Hi Daniela, I tried those commands and it still gives me the same error messages (attached). Best Regards Aditya Upreti | Ph.D. Candidate / Research Assistant [Divider line] Department of Physics The University of Alabama<https://www.ua.edu/> 102 Gallalee Hall Tuscaloosa, AL 35404 Phone 205-886-2914 aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu> [Divider line] [The University of Alabama box A with stacked words logo]<https://www.ua.edu/> ________________________________ From: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com><mailto:daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>>> Sent: Monday, September 20, 2021 10:42 AM To: Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu><mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>> Cc: Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk><mailto:d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>>>; Simon Fayer <simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk><mailto:simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk>>> Subject: Re: Issue regrading GRID Hi Aditya, This looks like a straight forward permissions error, so Dan needs to have a look to see why you wouldn't be allowed to access the disk. Could you please run one last test ? In a clean window (no ganga setup) can you please run: export VOMS_USERCONF=/cvmfs/grid.cern.ch/etc/grid-security/vomses<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgrid.cern.ch%2Fetc%2Fgrid-security%2Fvomses&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166549488%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=m8ZlC6O1HR3JZY%2BMt7SeLdDdUlwKG6WQlzBD9cHh%2Bms%3D&reserved=0><https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgrid.cern.ch%2Fetc%2Fgrid-security%2Fvomses&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735006265%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2B7w7RwLzWRK1VxCdBifcehs8C%2Fs2Yo93kgkyBZG0zAk%3D&reserved=0<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgrid.cern.ch%2Fetc%2Fgrid-security%2Fvomses&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166549488%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=m8ZlC6O1HR3JZY%2BMt7SeLdDdUlwKG6WQlzBD9cHh%2Bms%3D&reserved=0>> export X509_VOMSES=/cvmfs/grid.cern.ch/etc/grid-security/vomses<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgrid.cern.ch%2Fetc%2Fgrid-security%2Fvomses&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166559443%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GV1AqW8A3kfLPbB7vGIBuYM0w11EFEvjs8%2FVcgwTp4c%3D&reserved=0><https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgrid.cern.ch%2Fetc%2Fgrid-security%2Fvomses&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735016224%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UAN1oqkRXIHRpucTITosJv6HoL2K2p4GUBtDNS6zPj0%3D&reserved=0<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgrid.cern.ch%2Fetc%2Fgrid-security%2Fvomses&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166559443%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GV1AqW8A3kfLPbB7vGIBuYM0w11EFEvjs8%2FVcgwTp4c%3D&reserved=0>> source /cvmfs/dirac.egi.eu/dirac/bashrc_gridpp<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdirac.egi.eu%2Fdirac%2Fbashrc_gridpp&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166569400%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Rsq4B6s61blbXeEB7i3NGp4g9vmXi8S1B1VsEMQoxbE%3D&reserved=0><https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdirac.egi.eu%2Fdirac%2Fbashrc_gridpp&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735016224%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zw9tTNin1Xb%2B8u0ymD8I8yadbMvyWfqcusi9nPLXQq4%3D&reserved=0<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdirac.egi.eu%2Fdirac%2Fbashrc_gridpp&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166569400%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Rsq4B6s61blbXeEB7i3NGp4g9vmXi8S1B1VsEMQoxbE%3D&reserved=0>> dirac-proxy-init -g vo.moedal.org_user and the copy command again (maybe with a slightly different file name to avoid any potential override conflicts). The ganga DIRAC version is very old, and while I don't think it matters in this case, I want to make sure. Regards, Daniela On Mon, 20 Sept 2021 at 16:20, Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu><mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>> wrote: Hi Daniela, I used the command dirac-dms-add-file and I'm still unable to upload a file. Attached is the output in a text file. Best Regards Aditya Upreti | Ph.D. Candidate / Research Assistant [Divider line] Department of Physics The University of Alabama<https://www.ua.edu/> 102 Gallalee Hall Tuscaloosa, AL 35404 Phone 205-886-2914 aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu> [Divider line] [The University of Alabama box A with stacked words logo]<https://www.ua.edu/> ________________________________ From: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com><mailto:daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>>> Sent: Monday, September 20, 2021 9:58 AM To: Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk><mailto:d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>>> Cc: Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu><mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>>; Simon Fayer <simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk><mailto:simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk>>> Subject: Re: Issue regrading GRID Hi Dan, Aditya, @Aditya Upreti<mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>> Could you please try to upload a file using: dirac-proxy-info dirac-dms-file-add -ddd /vo.moedal.org/user/a/aditya.upreti/dummy1.txt<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2Fdummy1.txt&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166579362%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=j7ABxKoH8Z9ykr1AN2xO6gm%2BM9SEw27Y7X6P1PPZlHU%3D&reserved=0><https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2Fdummy1.txt&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735026179%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=j4ZoYVGkSdrkjPtarCj20yFtIj3fjOXaG%2B1q56iaE30%3D&reserved=0<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2Fdummy1.txt&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166579362%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=j7ABxKoH8Z9ykr1AN2xO6gm%2BM9SEw27Y7X6P1PPZlHU%3D&reserved=0>> dummy.txt UKI-LT2-QMUL2-disk and email the output back ? Just renewing your certificate should not make any difference. @Dan: Can you see a permissions error for this user somewhere ? Regards, Daniela On Mon, 20 Sept 2021 at 15:28, Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk><mailto:d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>>> wrote: We have user directories for moedal and i see attempts to writes the jobs output on disk there for several users e.g. ls /mnt/lustre_3/storm_3/vo.moedal.org/vo.moedal.org/user/a/aditya.upreti/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166589312%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CasvAchNMb16eGbbj5z7Kx%2FijvFtE0PY4YSjDfREIhw%3D&reserved=0><https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735026179%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3hxegJTr14RK0ili3pKZWh3qmwE7%2BudrYf24aADsrEU%3D&reserved=0<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166589312%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CasvAchNMb16eGbbj5z7Kx%2FijvFtE0PY4YSjDfREIhw%3D&reserved=0>> 30272 30348 31778 31780 31781 31783 31784 31793 31798 31805 31829 31861 GRID tmp but recent directories are empty. The last successful wite was on the 13th July (directory 30348) dan * Dr Daniel Traynor, Grid cluster system manager * Tel +44(0)20 7882 6560, Particle Physics,QMUL ________________________________________ From: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com><mailto:daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>>> Sent: 20 September 2021 14:21 To: Aditya Upreti Cc: Simon Fayer; Daniel Traynor Subject: Re: Issue regrading GRID Hi Aditya, I'll try to disentangle this. Looking at the JDL I can see on the server you seem to have your output files specified both in the OutputData and the OutputSandbox field. Listing them in OutputData means DIRAC will try to write it to a storage element (SE), which you need to specify. I think you initially specified QMUL. I've cc'ed the QMUL admin to check if moedal users are meant to use this SE (@Dan: Should they ?). If yes, you'd need to debug with him to see why you can't write there. If all you want to do is ship a couple of log files back, then just use OutputSandbox (but never in combination with OutputData), which you can retrieve with dirac-wms-job-get-output. Just be aware that output sandboxes are limited in size (< 10MB) and tend not to scale. If you have any question wrt DIRAC, I would encourage you to use the DIRAC mailing list: gridpp-dirac-users@imperial.ac.uk<mailto:gridpp-dirac-users@imperial.ac.uk><mailto:gridpp-dirac-users@imperial.ac.uk<mailto:gridpp-dirac-users@imperial.ac.uk>><mailto:gridpp-dirac-users@imperial.ac.uk<mailto:gridpp-dirac-users@imperial.ac.uk><mailto:gridpp-dirac-users@imperial.ac.uk<mailto:gridpp-dirac-users@imperial.ac.uk>>> as I am not always online and (as you will have seen), there can be quite a delay between you sending and me seeing this email. Hope that helps, Daiela On Fri, 17 Sept 2021 at 17:53, Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu><mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>><mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu><mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>>> wrote: ​Hi Daniela, Hope you are doing well. I am getting the following error message after my simulation completes running on Grid. Failed to upload output data file Encountered 1 errors Could not putAndRegister file std.out with LFN /vo.moedal.org/user/a/aditya.upreti/31829/31829313/std.out<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F31829%2F31829313%2Fstd.out&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166589312%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=dg3ADyRWTbJ7rkU2ptLua8f0%2BaKCXdypQ1xa0%2FrL2G8%3D&reserved=0><https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F31829%2F31829313%2Fstd.out&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735026179%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rB%2BUj5bFMRgprSiftesgzQgUVovZHQld5VzSRjUGr38%3D&reserved=0<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F31829%2F31829313%2Fstd.out&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166599278%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GL6TL3vwz%2FkXDEgfGxkqinJqsX5KblW4Rf3%2F%2Bp%2Busus%3D&reserved=0>><http://vo.moedal.org/user/a/aditya.upreti/31829/31829313/std.out<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F31829%2F31829313%2Fstd.out&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166599278%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GL6TL3vwz%2FkXDEgfGxkqinJqsX5KblW4Rf3%2F%2Bp%2Busus%3D&reserved=0><https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F31829%2F31829313%2Fstd.out&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735036136%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Y0tzOs96lKNi%2BxYZ6UFpFBkv02rxR8CKfscfQEIq%2BPE%3D&reserved=0<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F31829%2F31829313%2Fstd.out&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166609230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IMeEIsmYAS4LnIAR3s8t7LHXOqC83xPCXPEbrU2fpt8%3D&reserved=0>>> to UKI-LT2-QMUL2-disk with GUID E7D2E680-3F6A-B9AB-6969-CF91C330F037 trying failover storage No failover SEs defined for JobWrapper, cannot try to upload output file std.out anywhere else. Failed to upload OutputData JobWrapper failed to process output files Traceback (most recent call last): File "/scratch/tmp/OD6MDmdLVlzn5Wq3smovVnomg01fWmABFKDmqGGKDmABFKDmTORfJm/DIRAC_ZJLEU2pilot/job/Wrapper/Wrapper_31829313", line 197, in execute raise JobWrapperError(result['Message']) JobWrapperError: Failed to upload OutputData The output sandbox only contains the std.out and std.err files and not the output root files. Would you be able to help me resolve this error? Please let me know if you need more information/input files. Thank you! Best Regards Aditya Upreti | Ph.D. Candidate / Research Assistant [Divider line] Department of Physics The University of Alabama<https://www.ua.edu/> 102 Gallalee Hall Tuscaloosa, AL 35404 Phone 205-886-2914 aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu><mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>> [Divider line] [The University of Alabama box A with stacked words logo]<https://www.ua.edu/> -- Sent from my guinea pig enhanced living room ----------------------------------------------------------- daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk><mailto:daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk>><mailto:daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk><mailto:daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk>>> HEP Group/Physics Dep Imperial College London, SW7 2BW Tel: Working from home, please use email. http://www.hep.ph.ic.ac.uk/~dbauer/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166609230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=pVb9eoYrV8kYU81EbEckSsGkdDJcJNx32prORbygp8A%3D&reserved=0><https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735036136%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MEDYmE5FHRmY3eF2sDm5UJuBuVpP%2BlYTartFD0svRGw%3D&reserved=0<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166619187%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=B5JUiQ7iAV5UuPqWR1oYXWwY6ucQuxcEzwlrNix6neY%3D&reserved=0>> -- Sent from my guinea pig enhanced living room ----------------------------------------------------------- daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk><mailto:daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk>> HEP Group/Physics Dep Imperial College London, SW7 2BW Tel: Working from home, please use email. http://www.hep.ph.ic.ac.uk/~dbauer/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166619187%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=B5JUiQ7iAV5UuPqWR1oYXWwY6ucQuxcEzwlrNix6neY%3D&reserved=0><https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735046086%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NiQ%2FctQNqtrgzLLhJA9LGbIjzjsevmR%2FesRYO02tm0c%3D&reserved=0<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166629139%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=nvpQ%2FOV1kwODnIG9ViHpdGOwOy19mZ34xnscrr3Xzqc%3D&reserved=0>> -- Sent from my guinea pig enhanced living room ----------------------------------------------------------- daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk><mailto:daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk>> HEP Group/Physics Dep Imperial College London, SW7 2BW Tel: Working from home, please use email. http://www.hep.ph.ic.ac.uk/~dbauer/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166629139%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=nvpQ%2FOV1kwODnIG9ViHpdGOwOy19mZ34xnscrr3Xzqc%3D&reserved=0><https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735046086%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NiQ%2FctQNqtrgzLLhJA9LGbIjzjsevmR%2FesRYO02tm0c%3D&reserved=0<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166629139%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=nvpQ%2FOV1kwODnIG9ViHpdGOwOy19mZ34xnscrr3Xzqc%3D&reserved=0>> -- Sent from my guinea pig enhanced living room ----------------------------------------------------------- daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk><mailto:daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk>> HEP Group/Physics Dep Imperial College London, SW7 2BW Tel: Working from home, please use email. http://www.hep.ph.ic.ac.uk/~dbauer/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166639100%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=TM0xQhtxiRjNP9aPOoZD2j74P4N9idBT8WnLozOtCMU%3D&reserved=0> -- Sent from my guinea pig enhanced living room ----------------------------------------------------------- daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk> HEP Group/Physics Dep Imperial College London, SW7 2BW Tel: Working from home, please use email. http://www.hep.ph.ic.ac.uk/~dbauer/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166639100%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=TM0xQhtxiRjNP9aPOoZD2j74P4N9idBT8WnLozOtCMU%3D&reserved=0> -- Sent from my guinea pig enhanced living room ----------------------------------------------------------- daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk> HEP Group/Physics Dep Imperial College London, SW7 2BW Tel: Working from home, please use email. http://www.hep.ph.ic.ac.uk/~dbauer/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166649054%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=YVEvvK3%2FfUqLaE%2FBUNS3rEXdpsvhukRQEhw%2BeqBsezE%3D&reserved=0> -- Sent from my guinea pig enhanced living room ----------------------------------------------------------- daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk> HEP Group/Physics Dep Imperial College London, SW7 2BW Tel: Working from home, please use email. http://www.hep.ph.ic.ac.uk/~dbauer/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C4e211cf778d54d8e52f808d9899b4047%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637692122166649054%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=YVEvvK3%2FfUqLaE%2FBUNS3rEXdpsvhukRQEhw%2BeqBsezE%3D&reserved=0>

Hi Aditya, The "pilot not running" error indicates that the site killed the job: In these instances we don't get any further feedback from the site (short of contacting the admins to ask them what the local batch system logged). The biggest causes of this are: - Jobs using too much (>2-4GB) of memory, this is by far the most common cause. - Jobs that don't use enough CPU time for a given amount of real time. - Jobs that use too much CPU, i.e. multithreaded jobs that only request a single CPU slot. I suggest that you run one of the jobs that failed locally using /usr/bin/time, top or a similar tool that can track resource usage and see if you can work out which of the above possibilities is happening. Regards, Simon On Wed, Oct 27, 2021 at 11:29:04AM +0000, Aditya Upreti wrote:

Hi Daniela,

Thanks for all your help thus far, I am able to retrieve the output files from the storage elements. I wanted to bring another issue to your concern (sorry to bother again!).

I recently submitted several jobs to the GRID (~1000), a few of them are running but a lot of them show the status failed with the status "Job stalled: pilot not running". Would you know why this is happening? Could it be because of some allotted quota per user, if so, how can I increase the quota as I would have to run a large number of jobs simultaneously?

Thank you so much.

Best Regards

Aditya Upreti | Ph.D. Candidate / Research Assistant

[Divider line]

Department of Physics

The University of Alabama<https://www.ua.edu/>

102 Gallalee Hall

Tuscaloosa, AL 35404

Phone 205-886-2914

aupreti@crimson.ua.edu