This email from daniela.bauer.grid@googlemail.com originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list to disable email stamping for this address.

 

Hi Priyaa,

just to clarify, when working with DIRAC we do not recommend using gfal-copy and the DIRAC register mechanism separately. We only recommend registering data explicitly (as it tends to be error prone) if they are already on a storage element (or if there is no way to upload them with DIRAC).

dirac-dms-put-and-register-request is not quite what it seems. The help throws up an unfortunately not very clear warning:
warning: make sure the file you want to put is accessible from DIRAC production hosts,
          i.e. put file on network fs (AFS or NFS), otherwise operation will fail!!!
This looks like some CERN centric hangover to me. This would only work if the file is on the actual DIRAC server. Unfortunately it will probably hang around forever now.

The command you really want is dirac-dms-add-file - it will upload the file and register it in the DIRAC catalogue all in one go.

Regards,
Daniela

On Tue, 6 Oct 2020 at 15:20, Priyaa Thavasimani <priyaa.thavasimani@manchester.ac.uk> wrote:

Hi Daniela,

            I have a question regarding registering file without using dirac-dms-add-file. As you suggested, I have gfal-cop(y) ied a local file to pfn and registered using “register” command. It is working fine. But I also thought is there a dirac command that combines the functionality of gfal-copy and register? Would like to know the difference between dirac-dms-put-and-register-request and dirac-dms-add-file? I got to know about dirac-dms-put-register-request, and submitted the request, but keeps on waiting even for a small file(assuming it does the functionality, also there is no clear documentation about this as  far as I am aware). It would be great if you could clarify this too.

 

Here is the request status for dirac-dms-put-and-register-request:

 

dirac-rms-request req1

Request name='req1' ID=319813 Status='Waiting'

Created 2020-10-06 11:48:41, Updated 2020-10-06 11:48:41, NotBefore 2020-10-06 11:48:41

Owner: '/C=UK/O=eScience/OU=Manchester/L=HEP/CN=priyaa thavasimani', Group: skatelescope.eu_user

  [0] Operation Type='PutAndRegister' ID=290706 Order=0 Status='Waiting'

      TargetSE: UKI-NORTHGRID-MAN-HEP-disk - Created 2020-10-06 11:48:41, Updated 2020-10-06 11:48:41

    [01] ID=17727 LFN='/skatelescope.eu/user/p/priyaa.thavasimani/ramdom.txt' Status='Waiting' Checksum='01b700bc'

 

Thanks,

Priyaa

 

From: Priyaa Thavasimani <priyaa.thavasimani@manchester.ac.uk>
Date: Wednesday, 1 July 2020 at 12:53
To: Daniela Bauer <daniela.bauer@IMPERIAL.AC.UK>
Subject: Dirac register file in file catalogue

 

Hi Daniela,

    I am a Research Associate working in JBCA, University of Manchester. We have 1.3 TB data in our highmem machine to be registered in the file catalogue. Have seen few of the links below to do that. 

  1. https://dirac.readthedocs.io/en/latest/UserGuide/CommandReference/DataManagement/dirac-dms-filecatalog-cli.html
  2. https://groups.google.com/forum/#!topic/diracgrid-forum/sclcLrQBPFY

But the links are not so much helpful. It would be great if you could guide on this. I am wondering if it is possible register the file in file catalog without adding the file in LFN using "dirac-dms-add-file"? If so, could you please send me some links that could help me out to do the same.

 

Thanks,

Priyaa



--
Sent from my guinea pig enhanced living room

-----------------------------------------------------------
daniela.bauer@imperial.ac.uk
HEP Group/Physics Dep
Imperial College
London, SW7 2BW
Tel: Working from home, please use email.
http://www.hep.ph.ic.ac.uk/~dbauer/