| This email from aupreti@crimson.ua.edu originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list to disable email stamping for this address. | 
2021-10-06 22:18:18 UTC dirac-dms-add-file/DataManager/putAndRegister [140605462013760] DEBUG: Failed to put file to Storage Element. dummy1.txt: Permission denied ( 13 : Failed to copy file dummy1.txt to destination url srm://se03.esc.qmul.ac.uk:8444/srm/managerv2?SFN=/vo.moedal.org/vo.moedal.org/user/a/aditya.upreti/dummy1.txt: [13] [gfalt_copy_file][perform_copy][srm_plugin_filecopy][srm_do_transfer][gfalt_copy_file][perform_copy][perform_local_copy] Could not open destination: [streamed_copy][gfal_plugin_openG][gfal_gridftp_openG][gfal_gridftp_openG] globus_ftp_client: the server responded with an error 530 530-Login incorrect. : globus_gss_assist: Error invoking callout 530-globus_callout_module: The callout returned an error 530-an unknown error occurred 530 End. )
2021-10-06 22:18:18 UTC dirac-dms-add-file [140605462013760] ERROR: Error: failed to upload /vo.moedal.org/user/a/aditya.upreti/dummy1.txt to UKI-LT2-QMUL2-disk
j = Job()
j.name = job_name
j.application = Executable()
j.application.exe = File(sh_run_ganga_name)
j.application.args = []
j.inputfiles = [LocalFile(cfg_path), LocalFile(cfg_ganga_pgun_name), LocalFile(input3Dpath+input3Dhist)]
j.outputfiles =[DiracFile(outputfile)]
j.backend = Dirac()
j.submit()
j = Job()
j.name = job_name
j.application = Executable()
j.application.exe = File(sh_run_ganga_name)
j.application.args = []
j.inputfiles = [LocalFile(cfg_path), LocalFile(cfg_ganga_pgun_name), LocalFile(input3Dpath+input3Dhist)]
j.outputfiles =[LocalFile('/eos/experiment/moedal/aditya/GRID_outputs/test/'+outputfile)]
j.backend = Dirac()
j.submit()
Aditya Upreti | Ph.D. Candidate / Research Assistant
Department of Physics
102 Gallalee Hall
Tuscaloosa, AL 35404
Phone 205-886-2914
Hi Daniela,
Sorry for the late reply.Okay, I can try writing to RALPP. Do I have to mention RALPP in the OutputSandbox instead of UKI-LT2-QMUL2-disk?
Also, I am able to submit jobs via the Ganga interface using a python submit script and getting the output file locally using"j = Job()j.outputfiles = [LocalFile(outputfile)]"
which outputs the file locally, but I'm not sure why I'm failing to do that with a JDL script directly, bypassing Ganga. (as with Ganga, I get some python dependency errors like 'no module named six'. etc, when submitting the jobs").
Best RegardsAditya Upreti | Ph.D. Candidate / Research Assistant
Department of Physics
102 Gallalee Hall
Tuscaloosa, AL 35404
Phone 205-886-2914
From: Daniela Bauer <daniela.bauer.grid@googlemail.com>
Sent: Thursday, September 23, 2021 4:20 AM
To: Daniel Traynor <d.traynor@qmul.ac.uk>
Cc: Aditya Upreti <aupreti@crimson.ua.edu>; Simon Fayer <simon.fayer05@imperial.ac.uk>
Subject: Re: Issue regrading GRID@Aditya Upreti: In the meantime, do you want to try and write to RALPP ? You should be able to copy the data later if needed.
Regards,Daniela
On Wed, 22 Sept 2021 at 18:38, Daniel Traynor <d.traynor@qmul.ac.uk> wrote:
not yet.
* Dr Daniel Traynor, Grid cluster system manager
* Tel +44(0)20 7882 6560, Particle Physics,QMUL
________________________________________
From: Daniela Bauer <daniela.bauer.grid@googlemail.com>
Sent: 22 September 2021 18:33
To: Aditya Upreti
Cc: Daniel Traynor; Simon Fayer
Subject: Re: Issue regrading GRID
This is a question for Dan - we need to work out where the permission denied comes from.
@Dan: Any idea ?
Cheers,
Daniela
On Tue, 21 Sept 2021 at 21:59, Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>> wrote:
Hi all,
Any resolution on this issue?
Thanks!
Best Regards
Aditya Upreti | Ph.D. Candidate / Research Assistant
[Divider line]
Department of Physics
The University of Alabama<https://www.ua.edu/>
102 Gallalee Hall
Tuscaloosa, AL 35404
Phone 205-886-2914
aupreti@crimson.ua.edu
[Divider line]
[The University of Alabama box A with stacked words logo]<https://www.ua.edu/>
________________________________
From: Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>
Sent: Monday, September 20, 2021 10:53 AM
To: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>>
Cc: Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>>; Simon Fayer <simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk>>
Subject: Re: Issue regrading GRID
Hi Daniela,
I tried those commands and it still gives me the same error messages (attached).
Best Regards
Aditya Upreti | Ph.D. Candidate / Research Assistant
[Divider line]
Department of Physics
The University of Alabama<https://www.ua.edu/>
102 Gallalee Hall
Tuscaloosa, AL 35404
Phone 205-886-2914
aupreti@crimson.ua.edu
[Divider line]
[The University of Alabama box A with stacked words logo]<https://www.ua.edu/>
________________________________
From: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>>
Sent: Monday, September 20, 2021 10:42 AM
To: Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>
Cc: Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>>; Simon Fayer <simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk>>
Subject: Re: Issue regrading GRID
Hi Aditya,
This looks like a straight forward permissions error, so Dan needs to have a look to see why you wouldn't be allowed to access the disk.
Could you please run one last test ?
In a clean window (no ganga setup) can you please run:
export VOMS_USERCONF=/cvmfs/grid.cern.ch/etc/grid-security/vomses<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgrid.cern.ch%2Fetc%2Fgrid-security%2Fvomses&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735006265%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2B7w7RwLzWRK1VxCdBifcehs8C%2Fs2Yo93kgkyBZG0zAk%3D&reserved=0>
export X509_VOMSES=/cvmfs/grid.cern.ch/etc/grid-security/vomses<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgrid.cern.ch%2Fetc%2Fgrid-security%2Fvomses&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735016224%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UAN1oqkRXIHRpucTITosJv6HoL2K2p4GUBtDNS6zPj0%3D&reserved=0>
source /cvmfs/dirac.egi.eu/dirac/bashrc_gridpp<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdirac.egi.eu%2Fdirac%2Fbashrc_gridpp&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735016224%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zw9tTNin1Xb%2B8u0ymD8I8yadbMvyWfqcusi9nPLXQq4%3D&reserved=0>
dirac-proxy-init -g vo.moedal.org_user
and the copy command again (maybe with a slightly different file name to avoid any potential override conflicts).
The ganga DIRAC version is very old, and while I don't think it matters in this case, I want to make sure.
Regards,
Daniela
On Mon, 20 Sept 2021 at 16:20, Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>> wrote:
Hi Daniela,
I used the command dirac-dms-add-file and I'm still unable to upload a file.
Attached is the output in a text file.
Best Regards
Aditya Upreti | Ph.D. Candidate / Research Assistant
[Divider line]
Department of Physics
The University of Alabama<https://www.ua.edu/>
102 Gallalee Hall
Tuscaloosa, AL 35404
Phone 205-886-2914
aupreti@crimson.ua.edu
[Divider line]
[The University of Alabama box A with stacked words logo]<https://www.ua.edu/>
________________________________
From: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>>
Sent: Monday, September 20, 2021 9:58 AM
To: Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>>
Cc: Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>; Simon Fayer <simon.fayer05@imperial.ac.uk<mailto:simon.fayer05@imperial.ac.uk>>
Subject: Re: Issue regrading GRID
Hi Dan, Aditya,
@Aditya Upreti<mailto:aupreti@crimson.ua.edu> Could you please try to upload a file using:
dirac-proxy-info
dirac-dms-file-add -ddd /vo.moedal.org/user/a/aditya.upreti/dummy1.txt<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2Fdummy1.txt&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735026179%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=j4ZoYVGkSdrkjPtarCj20yFtIj3fjOXaG%2B1q56iaE30%3D&reserved=0> dummy.txt UKI-LT2-QMUL2-disk and email the output back ?
Just renewing your certificate should not make any difference.
@Dan: Can you see a permissions error for this user somewhere ?
Regards,
Daniela
On Mon, 20 Sept 2021 at 15:28, Daniel Traynor <d.traynor@qmul.ac.uk<mailto:d.traynor@qmul.ac.uk>> wrote:
We have user directories for moedal and i see attempts to writes the jobs output on disk there for several users e.g.
ls /mnt/lustre_3/storm_3/vo.moedal.org/vo.moedal.org/user/a/aditya.upreti/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735026179%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3hxegJTr14RK0ili3pKZWh3qmwE7%2BudrYf24aADsrEU%3D&reserved=0>
30272 30348 31778 31780 31781 31783 31784 31793 31798 31805 31829 31861 GRID tmp
but recent directories are empty. The last successful wite was on the 13th July (directory 30348)
dan
* Dr Daniel Traynor, Grid cluster system manager
* Tel +44(0)20 7882 6560, Particle Physics,QMUL
________________________________________
From: Daniela Bauer <daniela.bauer.grid@googlemail.com<mailto:daniela.bauer.grid@googlemail.com>>
Sent: 20 September 2021 14:21
To: Aditya Upreti
Cc: Simon Fayer; Daniel Traynor
Subject: Re: Issue regrading GRID
Hi Aditya,
I'll try to disentangle this.
Looking at the JDL I can see on the server you seem to have your output files specified both in the OutputData and the OutputSandbox field.
Listing them in OutputData means DIRAC will try to write it to a storage element (SE), which you need to specify. I think you initially specified QMUL. I've cc'ed the QMUL admin to check if moedal users are meant to use this SE (@Dan: Should they ?). If yes, you'd need to debug with him to see why you can't write there.
If all you want to do is ship a couple of log files back, then just use OutputSandbox (but never in combination with OutputData), which you can retrieve with dirac-wms-job-get-output. Just be aware that output sandboxes are limited in size (< 10MB) and tend not to scale.
If you have any question wrt DIRAC, I would encourage you to use the DIRAC mailing list: gridpp-dirac-users@imperial.ac.uk<mailto:gridpp-dirac-users@imperial.ac.uk><mailto:gridpp-dirac-users@imperial.ac.uk<mailto:gridpp-dirac-users@imperial.ac.uk>>
as I am not always online and (as you will have seen), there can be quite a delay between you sending and me seeing this email.
Hope that helps,
Daiela
On Fri, 17 Sept 2021 at 17:53, Aditya Upreti <aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu><mailto:aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>>> wrote:
Hi Daniela,
Hope you are doing well.
I am getting the following error message after my simulation completes running on Grid.
Failed to upload output data file Encountered 1 errors
Could not putAndRegister file std.out with LFN /vo.moedal.org/user/a/aditya.upreti/31829/31829313/std.out<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F31829%2F31829313%2Fstd.out&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735026179%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rB%2BUj5bFMRgprSiftesgzQgUVovZHQld5VzSRjUGr38%3D&reserved=0><http://vo.moedal.org/user/a/aditya.upreti/31829/31829313/std.out<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvo.moedal.org%2Fuser%2Fa%2Faditya.upreti%2F31829%2F31829313%2Fstd.out&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735036136%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Y0tzOs96lKNi%2BxYZ6UFpFBkv02rxR8CKfscfQEIq%2BPE%3D&reserved=0>> to UKI-LT2-QMUL2-disk with GUID E7D2E680-3F6A-B9AB-6969-CF91C330F037 trying failover storage
No failover SEs defined for JobWrapper, cannot try to upload output file std.out anywhere else.
Failed to upload OutputData
JobWrapper failed to process output files
Traceback (most recent call last):
File "/scratch/tmp/OD6MDmdLVlzn5Wq3smovVnomg01fWmABFKDmqGGKDmABFKDmTORfJm/DIRAC_ZJLEU2pilot/job/Wrapper/Wrapper_31829313", line 197, in execute
raise JobWrapperError(result['Message'])
JobWrapperError: Failed to upload OutputData
The output sandbox only contains the std.out and std.err files and not the output root files. Would you be able to help me resolve this error? Please let me know if you need more information/input files.
Thank you!
Best Regards
Aditya Upreti | Ph.D. Candidate / Research Assistant
[Divider line]
Department of Physics
The University of Alabama<https://www.ua.edu/>
102 Gallalee Hall
Tuscaloosa, AL 35404
Phone 205-886-2914
aupreti@crimson.ua.edu<mailto:aupreti@crimson.ua.edu>
[Divider line]
[The University of Alabama box A with stacked words logo]<https://www.ua.edu/>
--
Sent from my guinea pig enhanced living room
-----------------------------------------------------------
daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk><mailto:daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk>>
HEP Group/Physics Dep
Imperial College
London, SW7 2BW
Tel: Working from home, please use email.
http://www.hep.ph.ic.ac.uk/~dbauer/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735036136%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MEDYmE5FHRmY3eF2sDm5UJuBuVpP%2BlYTartFD0svRGw%3D&reserved=0>
--
Sent from my guinea pig enhanced living room
-----------------------------------------------------------
daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk>
HEP Group/Physics Dep
Imperial College
London, SW7 2BW
Tel: Working from home, please use email.
http://www.hep.ph.ic.ac.uk/~dbauer/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735046086%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NiQ%2FctQNqtrgzLLhJA9LGbIjzjsevmR%2FesRYO02tm0c%3D&reserved=0>
--
Sent from my guinea pig enhanced living room
-----------------------------------------------------------
daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk>
HEP Group/Physics Dep
Imperial College
London, SW7 2BW
Tel: Working from home, please use email.
http://www.hep.ph.ic.ac.uk/~dbauer/<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.hep.ph.ic.ac.uk%2F~dbauer%2F&data=04%7C01%7Caupreti%40crimson.ua.edu%7C820b3cbda3094ab4ba6f08d97c4d4ce7%7C2a00728ef0d040b4a4e8ce433f3fbca7%7C0%7C0%7C637677493735046086%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NiQ%2FctQNqtrgzLLhJA9LGbIjzjsevmR%2FesRYO02tm0c%3D&reserved=0>
--
Sent from my guinea pig enhanced living room
-----------------------------------------------------------
daniela.bauer@imperial.ac.uk<mailto:daniela.bauer@imperial.ac.uk>
HEP Group/Physics Dep
Imperial College
London, SW7 2BW
Tel: Working from home, please use email.
http://www.hep.ph.ic.ac.uk/~dbauer/
--
Sent from my guinea pig enhanced living room
-----------------------------------------------------------
daniela.bauer@imperial.ac.uk
HEP Group/Physics Dep
Imperial College
London, SW7 2BW
Tel: Working from home, please use email.
http://www.hep.ph.ic.ac.uk/~dbauer/