On 09/09/14 14:47, Eike Mueller wrote:
Dear all,
some more observations/issues encountered while running on ARCHER:
I think I have to set the path to petsc4py, ufl, fiat and ffc in firedrake.tpl, as otherwise it complains that it can't find the modules:
export PYTHONPATH=${FDRAKE_DIR}/petsc4py/lib/python2.7/site-packages:${PYTHONPATH} export PYTHONPATH=${FDRAKE_DIR}/ufl/lib/python2.7/site-packages/:${PYTHONPATH} export PYTHONPATH=${FDRAKE_DIR}/fiat/lib/python2.7/site-packages/:${PYTHONPATH} export PYTHONPATH=${FDRAKE_DIR}/ffc/lib/python2.7/site-packages/:${PYTHONPATH}
Still, even though I think the ufl path is correct, it can't load ufl.algorithms.compute_form_data:
File "firedrake_poisson.py", line 4, in <module> from firedrake import * File "/work/n02/n02/eike/firedrake/firedrake/firedrake/__init__.py", line 11, in <module> from bcs import * File "/work/n02/n02/eike/firedrake/firedrake/firedrake/bcs.py", line 9, in <module> import function File "/work/n02/n02/eike/firedrake/firedrake/firedrake/function.py", line 11, in <module> import projection File "/work/n02/n02/eike/firedrake/firedrake/firedrake/projection.py", line 6, in <module> import solving File "/work/n02/n02/eike/firedrake/firedrake/firedrake/solving.py", line 38, in <module> import ffc_interface File "/work/n02/n02/eike/firedrake/firedrake/firedrake/ffc_interface.py", line 13, in <module> from ufl.algorithms import compute_form_data, ReuseTransformer ImportError: cannot import name compute_form_data
Does it need a more recent version of ufl? The directory
/work/y07/y07/fdrake/ufl/lib/python2.7/site-packages/ufl/algorithms/
on ARCHER does not contain the file compute_form_data.py, but the installation on my laptop does, so looks like something is missing there.
I have now updated all the dependencies. Let me know if the issue persists.
Also (I believe this is an unrelated problem), I had to replace
module swap python anaconda
by the two lines
module unload python module add anaconda
If python is not loaded by default on the compute nodes, the swap command fails, and the firedrake python code later crashes with a segmentation fault since it can't load the anaconda library:
ModuleCmd_Switch.c(172):ERROR:152: Module 'python' is currently not loaded fdrake-python-env(30):ERROR:102: Tcl command execution failed: prepend-path LD_LIBRARY_PATH "$env(ANACONDA_LIB)"
Thanks, that is indeed the safer way of doing it. Some more news on the issues you reported earlier: * The https cloning issue I believe is resolved, at least I can't reproduce it anymore. * The PyOP2 installation issue related to NumPy is a bug [1] in setuptools 2.2 which comes with anaconda. For now, the best thing to do is to remove/comment line 138 in PyOP2/setup.py which declares the setup dependency on NumPy. * The anaconda/environment modules conflict is resolved: I have removed ANACONDA_LIB from the LD_LIBRARY_PATH and added it to LD_RUN_PATH instead. This means Python extension modules get an RPATH to that directory baked in and therefore can find the right libpython2.7.so without any LD_LIBRARY_PATH being set (which is preferable anyway). Unfortunately, Firedrake's setup.py manually sets an RPATH for PETSc and therefore the LD_RUN_PATH is ignored. This patch is required: diff --git a/setup.py b/setup.py index 00838f8..350d51d 100644 --- a/setup.py +++ b/setup.py @@ -70,6 +70,7 @@ setup(name='firedrake', include_dirs=include_dirs, libraries=["petsc"], extra_link_args=["-L%s/lib" % d for d in petsc_dirs] + - ["-Wl,-rpath,%s/lib" % d for d in petsc_dirs]), + ["-Wl,-rpath,%s/lib" % d for d in petsc_dirs] + + ["-Wl,-rpath,%s/lib" % sys.prefix]), Extension('evtk.cevtk', evtk_sources, include_dirs=[np.get_include()])]) For FFC it's more tricky, still trying to figure out what's needed. Keep us posted on how things are progressing! Florian
I will go ahead and edit the wiki.
Eike
On 8 Sep 2014, at 15:53, Eike Mueller <E.Mueller@bath.ac.uk <mailto:E.Mueller@bath.ac.uk>> wrote:
Hi Florian,
thanks a lot for your reply and for updating the wiki, I got a bit further now.
You're exactly right again: it seems that anaconda contains a version of the TCL libraries which are incompatible with those use by the environment modules. I'm not sure how to work around this, since we need the anaconda libs (as you figured out later) - will open a ticket.
ok, I will use the environment script as it is and not worry about the fact that the anaconda library breaks the module command at the moment. Let's see if the support can figure something out.
I've changed the wiki page to clone via https, thanks!
It works with the updated instructions on the webpage. Strangely, if I clone a repository into a directory, delete this directory and then try to clone again with exactly the same command, nothing happens.
This seems to happen because (for reasons I can't figure out) setuptools doesn't detect that NumPy is already installed and builds it from source. The primary cause of the error you're seeing is this: https://stackoverflow.com/a/21621493
I will ignore this then, and just call the setup.py script twice in the PyOP2 directory.
This is because you've removed the anaconda lib path, which also contains the Python libraries themselves, from LD_LIBRARY_PATH
I tried a simpler hello-world script where I swap python -> anaconda and run a python script. This does work. Still trying to run the poisson example from the benchmark repository, but the ARCHER queues seem to be terribly slow, even if I just run on one node on the debug queue :-(
Absolutely, feel free to edit the wiki if you find more errors or things that are unclear.
I think I already have access to the repository, so will edit if necessary.
Thanks a lot,
Eike
Florian
Cheers,
Eike
*** content of firedrake.env ***
module swap PrgEnv-cray PrgEnv-gnu module unload python module add anaconda
# Add fdrake module path export FDRAKE_DIR=/work/y07/y07/fdrake module use "$FDRAKE_DIR/modules"
# Load build environment module load fdrake-build-env module load fdrake-python-env
LD_LIBRARY_PATH=`echo $LD_LIBRARY_PATH | sed 's/\/work\/y07\/y07\/cse\/anaconda\/1.9.2\/lib//g'`
export PYTHONPATH=$WORK/firedrake-bench:$WORK/firedrake-bench/pybench:$WORK/firedrake:$WORK/PyOP2:$PYTHONPATH
--
Dr Eike Hermann Mueller Research Associate (PostDoc)
Department of Mathematical Sciences University of Bath Bath BA2 7AY, United Kingdom
+44 1225 38 5803 e.mueller@bath.ac.uk <mailto:e.mueller@bath.ac.uk> <mailto:e.mueller@bath.ac.uk> http://people.bath.ac.uk/em459/
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake