Hi Daniela,
To make this DIRAC compatible we would have to move our existing files to the <VO-prefix>/<Full-LFN> structure so that the path looks something like the following (ie. duplicating the VO name): srm://kek2-se01.cc.kek.jp:8444/srm/managerv2?SFN=/t2k.org/t2k.org/nd280/raw/ND280/ND280 /00013000_00013999/nd280_00013512_0005.daq.mid.gz ?
Yes. We have scripts to move/help dealing with data previously registered elsewhere.
It is good to hear that there are tools available to help deal with this. Could I ask what these scripts are and are they available to be looked at?
So the file is where it should be. The rest is a bug :-( What do you need the srm path for ? Maybe there's a workaround, while we are looking into this bug.
I was trying to better understand what DIRAC was doing, and what sort of path would be necessary for the T2K to be used by DIRAC. No immediate need for the SRM path. Cheers Trevor
________________________________________ From: Simon Fayer [simon.fayer05@imperial.ac.uk] Sent: August 15, 2017 10:08 AM To: Stewart, Trevor (STFC,RAL,PPD) Cc: gridpp-dirac-users@imperial.ac.uk Subject: Re: [Gridpp-Dirac-Users] Adding a SE/Registering files from an SE not shown by dirac-dms-show-se-status
Hi Trevor,
Just to let you know that we found the bug in the BDII import module... There was a condition where the supported VO list wouldn't get created correctly if the site's xrootd info was incomplete.
The bug has now been patched and JP-KEK-CRC-02-disk appears correctly in the dirac-dms-show-se-status output for the t2k.org VO.
Regards, Simon
On Mon, Jul 24, 2017 at 11:29:02PM +0100, Simon Fayer wrote:
Hi Trevor,
The SE mappings in this DIRAC instance are automatically generated from the BDII (with a sequence number to prevent duplicates where a site has more than one SE), so the names can be a bit strange... The ones you're looking for are[1]:
kek2-se01.cc.kek.jp -> <Missing> srm-t2k.gridpp.rl.ac.uk -> RAL-LCG22-disk t2ksrm.nd280.org -> CA-TRIUMF-T2K1-disk
The KEK disk is half-registered (as JP-KEK-CRC-02-disk) for some reason, it doesn't have a VO name set, which is why it doesn't appear in the dirac-dms-show-se-status list. I'll open a ticket to check whether it's a code problem or something missing from the BDII[2].
One more thing to note is that DIRAC is quite fussy about the path names on the SE... Generally this means having filenames of <VO-prefix>/<Full-LFN>, in most cases this results in the VO name being duplicated (it was the only way to guarantee consistency in certain cases). You may find that registering existing files that don't match the expected pattern causes strange file-not-found errors.
p.s. Most of the DIRAC support people at Imperial are on holiday at the moment, so replies may be a bit delayed compared to normal.
Regards, Simon
[1] The DIRAC to SE host mappings can be found on the web-interface: https://dirac.gridpp.ac.uk under System -> Configuration -> Browse Remote Configuration None Configuration -> Resources -> StorageElements
[2] https://github.com/ic-hep/DIRAC/issues/85
On Mon, Jul 24, 2017 at 03:51:56PM +0000, trevor.stewart@stfc.ac.uk wrote:
Hi Raja, Thank you for the reply.
I'm following the guide at https://www.gridpp.ac.uk/userguide/example-workflow-grid/example-workflow-gr... where I supply the the t2k.org_user group to the dirac-proxy-init command below.
source /cvmfs/ganga.cern.ch/dirac_ui/bashrc dirac-proxy-init -g t2k.org_user -M dirac-dms-show-se-status False False t2k.org SE ReadAccess WriteAccess RemoveAccess CheckAccess ============================================================================ CA-TRIUMF-T2K1-disk Active Active Unknown Unknown GRIF-disk Active Active Unknown Unknown GridPPSandboxSE Active Active Unknown Unknown IFIC-LCG2-disk Active Active Unknown Unknown INFN-BARI1-disk Active Active Unknown Unknown Nebraska1-disk Active Active Unknown Unknown PIC-disk Active Active Unknown Unknown RAL-LCG22-disk Active Active Unknown Unknown UKI-LT2-IC-HEP-disk Active Active Unknown Unknown UKI-LT2-QMUL2-disk Active Active Unknown Unknown UKI-NORTHGRID-LANCS-HEP-disk Active Active Unknown Unknown UKI-NORTHGRID-LIV-HEP-disk Active Active Unknown Unknown UKI-NORTHGRID-MAN-HEP-disk Active Active Unknown Unknown UKI-NORTHGRID-SHEF-HEP-disk Active Active Unknown Unknown UKI-SOUTHGRID-OX-HEP-disk Active Active Unknown Unknown UKI-SOUTHGRID-RALPP-disk Active Active Unknown Unknown
Cheers Trevor
________________________________________ From: Raja Nandakumar [raja.nandakumar@cern.ch] Sent: July 24, 2017 4:41 PM To: gridpp-dirac-users@imperial.ac.uk; Stewart, Trevor (STFC,RAL,PPD) Subject: Re: [Gridpp-Dirac-Users] Adding a SE/Registering files from an SE not shown by dirac-dms-show-se-status
Hi Trevor,
From a brief look, I am able to see for example, kek2-se01.cc.kek.jp and I believe it is visible only for VO "t2k.org" (if I understood it correct).
Are you sure you used the proxy for the correct VO? I am guessing that DIRAC uses the information of your proxy to determine which storages are visible to you.
Cheers, Raja.
On 24/07/17 14:58, trevor.stewart@stfc.ac.uk wrote:
Hi, I've a real newbie question.
I've been investigating migrating the T2K grid software for files transfers/job submission from LCG/WMS to DIRAC. As part of this I was hoping to register a sub-section of our data files into the GridPP DIRAC instance to begin experimenting. I've read that this is possible via the CLI using the 'register file' command. The data is current stored at 3 locations, KEK, TRIUMF and RAL such that for a typical file the output of lcg-lr is:
srm://kek2-se01.cc.kek.jp/t2k.org/nd280/raw/ND280/ND280/00013000_00013999/nd280_00013512_0005.daq.mid.gz srm://srm-t2k.gridpp.rl.ac.uk/castor/ads.rl.ac.uk/prod/t2k.org/nd280/raw/ND280/ND280/00013000_00013999/nd280_00013512_0005.daq.mid.gz srm://t2ksrm.nd280.org/nd280data/raw/ND280/ND280/00013000_00013999/nd280_00013512_0005.daq.mid.gz
The output of dirac-dms-show-se-status does not appear to have kek2-se01.cc.kek.jp, srm-t2k.gridpp.rl.ac.uk or t2ksrm.nd280.org (or is this CA-TRIUMF-T2K1-disk?) as available SE.
Is it possible to add new SEs to the GridPP deployment of DIRAC?
Cheers Trevor Stewart
-- _______________________________________________ Gridpp-Dirac-Users mailing list Gridpp-Dirac-Users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/gridpp-dirac-users
-- Sent from the pit of despair ----------------------------------------------------------- daniela.bauer@imperial.ac.uk HEP Group/Physics Dep Imperial College London, SW7 2BW Tel: +44-(0)20-75947810 http://www.hep.ph.ic.ac.uk/~dbauer/ -- _______________________________________________ Gridpp-Dirac-Users mailing list Gridpp-Dirac-Users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/gridpp-dirac-users