June 2022 - Gridpp-Dirac-Users

setting up a DIRAC client
by Giuseppe Congedo 07 Jul '22

07 Jul '22

******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Hello everyone, I am relatively new to gridPP, having obtained a certificate/VO membership only recently. I am writing because I have started experimenting with the Python DIRAC API (followed the official instructions https://dirac.readthedocs.io/en/latest/UserGuide/GettingStarted/InstallingC… ) Apart one lucky job (which I managed to submit via /cvmfs/ganga.cern.ch/dirac_ui/bashrc as opposed to the manual procedure above, but failed probably due to me deleting the proxy), unfortunately all my other attempts have failed so far. This is the error I always get: > Job submission failure Cannot get URL for > WorkloadManagement/JobManager in setup DIRAC-Certification: > RuntimeError('Option > /DIRAC/Setups/DIRAC-Certification/WorkloadManagement is not defined') I wonder if I am doing something wrong somewere. Unrelated from the above, I was also wondering what environment the code will see when it arrives at the remote node. Can I sandbox my Python package, cd into the directory and install it along with all dependencies all in a job step? For instance: > job.setExecutable('cd my_package_dir && pip install -r > requirements.txt --user && python setup.py install --user') > job.setExecutable('my_package_dir/my_package/my_script.py') > job.setInputSandbox([my_package_dir, other_files]) Also, are directories relative to the home? Many thanks for all your help Giuseppe -- Dr Giuseppe Congedo (Senior Researcher) Institute for Astronomy, University of Edinburgh Royal Observatory, Blackford Hill Edinburgh, EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

3 13

Limits on DIRAC jobs
by CURRIE Robert 28 Jun '22

28 Jun '22

******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Dear gridpp-dirac-users, I've recently been asked if there are any grid resources for running "long-lived jobs" within the UK grid? Given that proxy-certs are typically only valid for 48hr does anyone know. Is there any support for running ~7day jobs via the DIRAC system? Or, Is there any potential of this being supported in the future? * I had a quick look through the maxCPUTime entries in the DIRAC config and I see that Manchester advertises a very long (multi-year(?)) parameter here, but most appear to be around the 48hr walltime. As a worst-case scenario our back-end batch-system at Edinburgh supports running weeklong jobs. However, that means users having to split their job management between 2 systems and must become more involved in topics such as data-management, which I'd rather avoid if possible. Thanks for any help, Rob *PS: This isn't the first time I've heard this come up in a conversation when discussing grid-use with non-LHC communities so I'm not sure if there is a plan here. The style of workflow seems to be like what LSST users were seeing at some point in the past. Most jobs are well defined and short lived, but a small percentage of jobs, which can't be broken down smaller, take a very long time to process. When this is combined with the fact, they can't be identified ahead of time these prove difficult to handle. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th' ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

2 3

GridPP DIRAC upgrade
by Daniela Bauer 14 Jun '22

14 Jun '22

******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Dear All, This is an advance warning that we are planning the next major version (v7r2 -> v7r3) upgrade of the GridPP DIRAC instance ( https://dirac.gridpp.ac.uk) on Monday July 25th. What does this mean for you ? As this is a major upgrade we cannot guarantee that jobs that are in a non-final state will survive, though we will try. The upgrade will proceed in the following order: - Server upgrade - Web interface upgrade - cvmfs UI upgrade - cloud interface (in case of problems only) v7r3 is a python3 release. For backwards compatibility we will maintain a python2 UI on cvmfs as well as a python3 UI. If you are maintaining your own DIRAC UI you should try to upgrade soon thereafter, though the old UIs will probably keep working for quite a while. Instructions will be supplied nearer the time. Please let me know if you have any questions and/or if this date is the worst possible date for the upgrade we could have picked. Regards, Daniela -- ----------------------------------------------------------- daniela.bauer(a)imperial.ac.uk HEP Group/Physics Dep Imperial College London, SW7 2BW Tel: Working from home, please use email. http://www.hep.ph.ic.ac.uk/~dbauer/

1 0

Mailing list post
by Abderrazaq El Abassi 12 Jun '22

12 Jun '22

******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Hi, Here is my email : abderrazaq.el.abassi(a)cern.ch Cheers, Abderrazaq Get Outlook for Android<https://aka.ms/AAb9ysg>

1 0