Hi Jo, On Fri, Apr 14, 2023 at 01:26:32PM +0100, Jonathan Orpwood wrote:
Could you advise me on the best way to transfer a directory to DIRAC?
It's possible to use dirac-dms-directory-sync to upload (or download) an entire directory: dirac-dms-directory-sync local_path LFN_path SE The problem with doing that is that it's then difficult to reference a large selection of files in a job, although it may be viable if you only need a couple of the files in each job. If you need the whole (or most of the) reference set in every job, it would be better to create a single large .tar file containing them all and upload that with dirac-dms-add-file. You can then reference it in the input sandbox by LFN to get it copied to the WN (DIRAC will probably unpack this for your automatically in the job's default scratch location). If you're planning on running a large number of jobs, we may want to see if we can get the reference dataset copied onto a suitable CVMFS area instead. Regards, Simon