Lawrence Mitchell <wence@gmx.li> writes:
Hi Colin,
cc'ing the loopy mailing list.
On 6 Jun 2019, at 07:02, Cotter, Colin J <colin.cotter@imperial.ac.uk> wrote:
Hi folks, When running on ARCHER (national supercomputer) we need to set new paths for writing cache files because you have to build in /home but compute nodes are only allowed to write to /work. Since the loo.py changes it looks like we need to set paths for loo.py as well, otherwise I get the message at the bottom of this email. How do I set them?
It looks like loopy (via pytools) sets up these caches by asking python where the use cache directory is. This is controlled by the XDG_CACHE_HOME environment variable
Running your firedrake with:
export XDG_CACHE_HOME=$VIRTUAL_ENV/.cache
Should do the trick (once you've activated the virtualenv before setting the var).
Or try this patch?
diff --git a/firedrake_configuration/__init__.py b/firedrake_configuration/__init__.py index b64b1b52..c6a1566b 100644 --- a/firedrake_configuration/__init__.py +++ b/firedrake_configuration/__init__.py @@ -46,3 +46,5 @@ def setup_cache_dirs(): os.environ["PYOP2_CACHE_DIR"] = os.path.join(config["options"]["cache_dir"], "pyop2") if 'FIREDRAKE_TSFC_KERNEL_CACHE_DIR' not in os.environ: os.environ["FIREDRAKE_TSFC_KERNEL_CACHE_DIR"] = os.path.join(config["options"]["cache_dir"], "tsfc") + + os.environ["XDG_CACHE_HOME"] = config["options"]["cache_dir"]
@loopy folk, I guess you're being good citizens by asking python for the cache directory. But controlling this on supercomputers is potentially an issue: you need to set different things depending on whether you're on linux or not.
At the moment, Loopy follows the "XDG Base Directory Specification" [1]. This (and the appdirs package) is in principle Linux-specific, but both (the spec and the package) technically work on other OSs, which is what is happening. That was the only sane convention that I was able to find at the time the cache code got written. Arguably, even though this is a desktop-focused spec, Linux supercomputer centers would be well-advised to set $XDG_CACHE_HOME to something sane, which might make this problem go away. If you have another convention you'd like pytools to follow, I'm happy to consider it. Another thing that could easily be changed: Right now, a lot of this configuration gets enacted at module import, which potentially makes it hard to change programmatically. I'd be happy to move this to time of first use, so that Firedrake has a chance to set things up beforehand. [1] https://specifications.freedesktop.org/basedir-spec/latest/ HTH, Andreas