Re: [Gridpp-Dirac-Users] Can't contact DIRAC CS

27 Mar 2018

      Hi Mark,

Thank you, but unfortunately, for the moment it isn't working, for uploading at least... I have just given a try, but the same error message emerged:

[gridpp@10 ganga]$ dirac-dms-add-file /vo.moedal.org/user/d/daniel.felea/unweighted_events.lhe unweighted_events.lhe UKI-LT2-QMUL2-disk

Uploading /vo.moedal.org/user/d/daniel.felea/unweighted_events.lhe
Error: failed to upload /vo.moedal.org/user/d/daniel.felea/unweighted_events.lhe to UKI-LT2-QMUL2-disk

Thanks and cheers,
Daniel

________________________________
From: gridpp-dirac-users-bounces@imperial.ac.uk [gridpp-dirac-users-bounces@imperial.ac.uk] on behalf of Mark Slater [mslater@cern.ch]
Sent: 27 March 2018 16:03
To: gridpp-dirac-users@imperial.ac.uk
Subject: Re: [Gridpp-Dirac-Users] Can't contact DIRAC CS

This sender failed our fraud detection checks and may not be who they appear to be. Learn about spoofing<http://aka.ms/LearnAboutSpoofing>
        Feedback<http://aka.ms/SafetyTipsFeedback>
Hi Daniel et al,

Just to say that I believe the CVMFS dirac issue is now solved (by me updating it :)) so you should be able issue:

source /cvmfs/ganga.cern.ch/dirac_ui/bashrc

in a clean shell to use the client, including the correct GFAL setup for down/uploading files to the DMS.

Please let me know if this is not the case - it seemed to work for me yesterday!

Thanks!

Mark

On 27/03/2018 14:55, santra.arka@gmail.com<mailto:santra.arka@gmail.com> wrote:
Hi Daniel,
  Thank you for this. I have modified my gangarc according to your instructions.
  Best.
  Arka

On Tue, Mar 27, 2018 at 3:38 PM, Daniel Felea <Daniel.Felea@cern.ch<mailto:Daniel.Felea@cern.ch>> wrote:

Hi Arka,

I looked a bit on your gangarc file.

Please try to update it according with :

https://gitlab.cern.ch/moedal/moedal-run-simulations

In short, I think you need all the below mentioned (you've got only the last one for the time being):

[Configuration]
...
RUNTIME_PATH = GangaDirac
...
[DIRAC]
...
DiracEnvSource = /cvmfs/ganga.cern.ch/dirac_ui/bashrc<http://ganga.cern.ch/dirac_ui/bashrc>
...
DiracLFNBase = /vo.moedal.org/user/y/your.name<http://vo.moedal.org/user/y/your.name>
...
[LCG]
...
GLITE_SETUP = /cvmfs/ganga.cern.ch/dirac_ui/bashrc<http://ganga.cern.ch/dirac_ui/bashrc>
...
[defaults_DiracFile]
...
defaultSE = <your SE of choice>
...
[defaults_DiracProxy]
...
group = vo.moedal.org_user
...

Then, just in case you used the CVMFS Dirac :

$ source /cvmfs/ganga.cern.ch/dirac_ui/bashrc<http://ganga.cern.ch/dirac_ui/bashrc>

one needs to use instead (in a clean environment) the Dirac UI, as Daniela advised us last week:

$ cd dirac_ui
$ source bashrc
$ dirac-proxy-init -g vo.moedal.org_user -M

till the CVMFS Dirac issue is solved.

Hope the aforementioned can help you...

Cheers,
Daniel

________________________________
From: gridpp-dirac-users-bounces@imperial.ac.uk<mailto:gridpp-dirac-users-bounces@imperial.ac.uk> [gridpp-dirac-users-bounces@imperial.ac.uk<mailto:gridpp-dirac-users-bounces@imperial.ac.uk>] on behalf of Arka Santra [santra.arka@gmail.com<mailto:santra.arka@gmail.com>]
Sent: 27 March 2018 15:18
To: rob.currie@ed.ac.uk<mailto:rob.currie@ed.ac.uk>
Cc: gridpp-dirac-users@imperial.ac.uk<mailto:gridpp-dirac-users@imperial.ac.uk>
Subject: Re: [Gridpp-Dirac-Users] Can't contact DIRAC CS

Hi Rob,
  I was using a tutorial made by Tom Whyntie for CERN (attached). Here section 4.3 deals with grid running with Ganga.
  Thanks,
  Arka

On Tue, Mar 27, 2018 at 2:55 PM, Robert Currie <rob.currie@ed.ac.uk<mailto:rob.currie@ed.ac.uk>> wrote:

Hi Arka,

These are configuration errors which are relatively harmless,
I suspect these are bad config entries in /cvmfs/ganga.cern.ch/Dirac.ini<http://ganga.cern.ch/Dirac.ini> but I've not investigated,

Glad to hear this is now working,

Just for future reference were you following a specific tutorial to get running on the grid?

Thanks,

Rob

On 27/03/2018 01:28 PM, Arka Santra wrote:

Hi Rob,
  This time grid jobs are being submitted without any problem. Thanks for your help.

  Now when Ganga starts, I see a new error message though:

*** Welcome to Ganga ***
Version: 7.0.1
Documentation and support: http://cern.ch/ganga
Type help() or help('index') for online help.

This is free software (GPL), and you are welcome to redistribute it
under certain conditions; type license() for details.

INFO     reading config file /afs/cern.ch/user/a/asantra/.gangarc<http://cern.ch/user/a/asantra/.gangarc>
INFO     reading config file /cvmfs/ganga.cern.ch/Dirac.ini<http://ganga.cern.ch/Dirac.ini>
ERROR    unknown options [Configuration]VirtualOrganisation,GLITE_SETUP
ERROR    unknown options [Configuration]VirtualOrganisation,GLITE_SETUP

I hope this will not be a problem since the jobs are running fine (at least upto now).

Thanks again,
Arka

On Tue, Mar 27, 2018 at 1:54 PM, Robert Currie <rob.currie@ed.ac.uk<mailto:rob.currie@ed.ac.uk>> wrote:

Hi Arka,

Can you make sure you have the exact lines (without quotes):

```
[defaults_DiracProxy]
group = vo.moedal.org_user
```
Ganga needs to know to apply the group argument to your `defaults_DiracProxy` config section, which hasn't yet been created,

It would be nice for ganga on initial launch to inspect any valid proxies and do the config automatically but this ability doesn't exist yet,

After this can you run `credential_store` again when ganga starts as you should see your DIRAC proxy.

If the proxy exists I would suggest running a simple `hello world` job again,Rob

On 27/03/2018 12:44 PM, Arka Santra wrote:

Hi Rob,
  I tried with: /cvmfs/ganga.cern.ch/runGanga-dirac.sh<http://ganga.cern.ch/runGanga-dirac.sh>

but this is the error I am getting:

*** Welcome to Ganga ***
Version: 7.0.1
Documentation and support: http://cern.ch/ganga
Type help() or help('index') for online help.

This is free software (GPL), and you are welcome to redistribute it
under certain conditions; type license() for details.

INFO     reading config file /afs/cern.ch/user/a/asantra/.gangarc<http://cern.ch/user/a/asantra/.gangarc>
INFO     reading config file /cvmfs/ganga.cern.ch/Dirac.ini<http://ganga.cern.ch/Dirac.ini>
ERROR    problems with loading Plugin /cvmfs/ganga.cern.ch/Ganga/install/7.0.1/ganga/GangaDirac<http://ganga.cern.ch/Ganga/install/7.0.1/ganga/GangaDirac>
ERROR    problems with loading Plugin /cvmfs/ganga.cern.ch/Ganga/install/7.0.1/ganga/GangaDirac<http://ganga.cern.ch/Ganga/install/7.0.1/ganga/GangaDirac>
ERROR    Reason: GangaValueError: DIRAC Proxy `group` is not set. Set this in ~/.gangarc in `[defaults_DiracProxy]/group`
ERROR    Reason: GangaValueError: DIRAC Proxy `group` is not set. Set this in ~/.gangarc in `[defaults_DiracProxy]/group`
PluginError: Failed to load plugin: /cvmfs/ganga.cern.ch/Ganga/install/7.0.1/ganga/GangaDirac<http://ganga.cern.ch/Ganga/install/7.0.1/ganga/GangaDirac>. Ganga will now shutdown to prevent job corruption.
(consider --debug option for more information)
INFO     Stopping Job processing before shutting down Repositories
INFO     Shutting Down Ganga Repositories
INFO     Registry Shutdown

I attach my gangarc file with this mail. It includes these lines:

#  Group for the proxy - either "group" or "group/subgroup"
#group = None
group = vo.moedal.org_user

so I do not understand the error quoted by Ganga.

Arka

On Tue, Mar 27, 2018 at 1:36 PM, Robert Currie <rob.currie@ed.ac.uk<mailto:rob.currie@ed.ac.uk>> wrote:

Hi Arka,

Can you try simply running:

/cvmfs/ganga.cern.ch/runGanga-dirac.sh<http://ganga.cern.ch/runGanga-dirac.sh>

I'm not sure why someone decided to add `source` to the documentation but it's not needed,

This may fail if you need to edit your ganga config,

Rob

On 27/03/2018 12:32 PM, Arka Santra wrote:

Hi Rob,
  To launch Ganga, I simply do this:

source /cvmfs/ganga.cern.ch/runGanga.sh<http://ganga.cern.ch/runGanga.sh>

Is this outdated and should I update it?
Thanks,
Arka

On Tue, Mar 27, 2018 at 1:27 PM, Robert Currie <rob.currie@ed.ac.uk<mailto:rob.currie@ed.ac.uk>> wrote:

Hi Arka,

OK this appears that you have the correct Dirac proxy created and registered with the VO,

How are you launching ganga?
I ask because you don't have the correct ganga configuration for working with the grid, the GangaDirac module is missing from your config and you cannot load this in ganga at runtime.

Rob

On 27/03/2018 12:22 PM, Arka Santra wrote:

Hi Rob,
  1. This is the output of:
dirac-proxy-init -g vo.moedal.org_user -M

Generating proxy...
Enter Certificate password:
Added VOMS attribute /vo.moedal.org<http://vo.moedal.org>
Uploading proxy for vo.moedal.org_user...
Proxy generated:
subject      : /DC=org/DC=terena/DC=tcs/C=ES/O=Consejo Superior de Investigaciones Cientificas/CN=ARKA SANTRA 859633@csic.es/CN=2970399870/CN=1069280930<http://859633@csic.es/CN=2970399870/CN=1069280930>
issuer       : /DC=org/DC=terena/DC=tcs/C=ES/O=Consejo Superior de Investigaciones Cientificas/CN=ARKA SANTRA 859633@csic.es/CN=2970399870<http://859633@csic.es/CN=2970399870>
identity     : /DC=org/DC=terena/DC=tcs/C=ES/O=Consejo Superior de Investigaciones Cientificas/CN=ARKA SANTRA 859633@csic.es<mailto:859633@csic.es>
timeleft     : 23:53:59
DIRAC group  : vo.moedal.org_user
rfc          : True
path         : /tmp/x509up_u41943
username     : arka.santra.csic.es<http://arka.santra.csic.es>
properties   : NormalUser, JobMonitor
VOMS         : True
VOMS fqan    : ['/vo.moedal.org<http://vo.moedal.org>']

Proxies uploaded:
 DN                                                                                                            | Group              | Until (GMT)
 /DC=org/DC=terena/DC=tcs/C=ES/O=Consejo Superior de Investigaciones Cientificas/CN=ARKA SANTRA 859633@csic.es<mailto:859633@csic.es> | vo.moedal.org_user | 2019/04/04 11:54

2. This is the output of `j=Job(backend=Dirac());j.submit()`:

ERROR    name 'Dirac' is not defined
ERROR    name 'Dirac' is not defined

3. This is the output of `credential_store`:

Type         | Location                 | Valid | Time left
------------ + ------------------------ + ----- + ---------------------
AfsTokenInfo | /tmp/krb5cc_41943_mBcgYg | True  | 1 day, 0:59:35.819471

I hope this helps.
Thanks,
Arka

On Tue, Mar 27, 2018 at 1:12 PM, Robert Currie <rob.currie@ed.ac.uk<mailto:rob.currie@ed.ac.uk>> wrote:

Hi Arka,

Can you submit a simple "hello world" style job in Ganga?

i.e. first run:

dirac-proxy-init -g vo.moedal.org_user -M

(What output do you get here?)

and within ganga:

`j=Job(backend=Dirac());j.submit()`

If this doesn't work can you share the output of `credential_store` from within ganga

Thanks,

Rob

On 27/03/2018 12:05 PM, Arka Santra wrote:

Hi,
  I wanted to submit a grid job on Ganga. Even though I can generate the proxy locally using this command:
dirac-proxy-init -g vo.moedal.org_user -M,

but when on the grid, I am getting this error:

  Ganga In [1]: execfile('make_dirac_lhe_job.py')
*
* Generating the MoEDAL simulation (LHE) configuration file for Ganga (DIRAC)
*
*
* Configuration filename: 'cfg_13TeV_PhotonFusion_myRun_betaIndependent_SpinZero_q10_run_05_default'
INFO     submitting job 50
INFO     job 50 status changed to "submitting"
INFO     Preparing Executable application.
INFO     Created shared directory: conf-2958b76e-93ea-4f5c-a338-5a99cef6564a
INFO     Preparing subjobs
Generating proxy...
Enter Certificate password:
Can't contact DIRAC CS: Reason(s):
Unauthorized query ( 1111 : Unauthorized query) (User possibly not registered with dirac server)

It seems I am not registered with the dirac server. May I know how I can do that and use the grid to submit jobs?

Thanks,
Arka

--
===============================================================
 Arka Santra
 Post Doctoral Researcher
 Instituto de Fisica Corpuscular
 Parque Cientifico
 C/Catedratico Jose Beltran, 2<https://maps.google.com/?q=C/Catedratico+Jose+Beltran,+2&entry=gmail&source=g>
 E-46980, Paterna
 Espana
===============================================================