Hi Daniela,

Thanks so much for looking into this. As far as I understand, I would need the software/code to be on cvmfs (so I won't need to send it with the job), and data on DFC. This way, all sites will be able to see both software and data. Please let me know if this looks sensible to you.

Thanks again for your help.

Giuseppe


On 20/03/2017 11:59, Daniela Bauer wrote:
Hi Giuseppe,

There are two problems here:
a) there is something odd a ECDF. It fails to load one of the essential (for dirac that is) libraries. Simon and me need to debug this. We might set ECDF offline until we know what is going on

b) you are mixing VOs and cvmfs repos. This often works, but tends cause more harm than good. If you are using the gridpp VO for work, you code should go in the gridpp cvmfs repo (which, as it's used much more  frequently, tends to be available everywhere). I've c'ed Jeremy and Pete as the gridpp VO admins. I hope they will be able to arrange access for you.

Regards,
Daniela



On 18 March 2017 at 23:04, Giuseppe Congedo <giuseppe.congedo@ed.ac.uk> wrote:
Hi Simon,

Many thanks for all your useful tips. Indeed the file exists on CVMFS. Then I tried with

j.backend.settings['Destination'] = 'LCG.UKI-SCOTGRID-ECDF.uk'

as you suggested, and the job 2335584 went to the site as expected. Good suggestion! Unfortunately it was put on hold because of, again, a failure in the input sandbox, despite the paths were fine and the data stored in the same place, LCG.UKI-SCOTGRID-ECDF.uk.

Unexpectedly, though, another identical job, 2335585, I submitted afterwards without the setting above started running on LCG.UKI-LT2-QMUL.uk, which I suppose supports the CVMFS repo. This is officially my first running job on Dirac!

I wonder if it's possible to know what sites support the CVMFS repo, so I won't waste jobs, or if it makes sense to change the workflow so I don't use CVMFS any more and avoid these problems.

Best wishes
Giuseppe



On 18/03/2017 16:22, Simon Fayer wrote:
Hi Giuseppe,

When using Ganga, the peek stdout function on the website unfortunately
shows the wrapper output rather than the main (user) script output. The
trick to get around this is to select "Sandbox -> Get Output Sandbox" and
have a look in the returned zip file at the Ganga_Executable.log file. In
this case the error seems to be:

python: can't open file '/cvmfs/scotgrid.gridpp.ac.uk/euclid/gcongedo/.lensmc_env/lib/python2.7/site-packages/lensmc-1.1-py2.7-linux-x86_64.egg/lensmc/bulge_disc/shear_measurement.py': [Errno 2] No such file or directory

The file seems to be there in CVMFS, so I suspect the site it ran at
(VAC.UKI-NORTHGRID-MAN-HEP.uk) doesn't have that specific CVMFS repo
enabled. You could try forcing the job to go to LCG.UKI-SCOTGRID-ECDF.uk[*]
(as they almost certainly have the scotgrid repo enabled) and see if it
works there.

Regards,
Simon

[*]
I don't really know much about Ganga, but I think the syntax for this is
something like:
j.backend.settings['Destination'] = 'LCG.UKI-SCOTGRID-ECDF.uk'


On Sat, Mar 18, 2017 at 03:38:15PM +0000, Giuseppe Congedo wrote:
Hi Simon,

Thanks so much. That helped indeed, the job was correctly dispatched to a
site, and data correctly downloaded, which I think it's already good
progress!

But it ran into problems afterwards as it crashed a few minutes later. The
stdout and stderr don't say that much, so I'm not sure what happened. Also
peek in ganga is not very informative, showing only a warning about a
non-existent scratch directory. Any clues?

Thanks again for your help.

Giuseppe



==================================================================================
Last 8 lines of application output from JobWrapper on 2017-03-18 14:54:47.985686 :
CPU Total: 00:00:02 (h:m:s) Normalized CPU Total 24.7 s @ HEP'06
==================================================================================
2017-03-18 14:54:39 UTC dirac-jobexec   INFO: JobID: 2335477
2017-03-18 14:54:39 UTC dirac-jobexec   INFO: DIRAC JobID 2335477 is running at site VAC.UKI-NORTHGRID-MAN-HEP.uk
Executing StepInstance RunScriptStep1 of type ScriptStep1 ['ScriptStep1']
2017-03-18 14:54:40 UTC dirac-jobexec/Script   INFO: Command is: /scratch/plt00/2335477/exe-script.py
2017-03-18 14:54:40 UTC dirac-jobexec/Script  ERROR: Non-zero status while executing 2: /scratch/plt00/2335477/exe-script.py
2017-03-18 14:54:40 UTC dirac-jobexec/Script   INFO: Output written to Ganga_Executable.log, execution complete.
2017-03-18 14:54:40 UTC dirac-jobexec/Script  ERROR: 'exe-script.py' Exited With Status 2
2017-03-18 14:54:40 UTC dirac-jobexec/Script   INFO: ===== Terminating=====


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



--
_______________________________________________
Gridpp-Dirac-Users mailing list
Gridpp-Dirac-Users@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/gridpp-dirac-users



--
Sent from the pit of despair

-----------------------------------------------------------
daniela.bauer@imperial.ac.uk
HEP Group/Physics Dep
Imperial College
London, SW7 2BW
Tel: +44-(0)20-75947810
http://www.hep.ph.ic.ac.uk/~dbauer/