> Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters.
I believe that is correct, yes.
I did try loading that netCDF4 module (in my submission script) prior to activating the Firedrake virtual environment, however, it then throws a version error at me.
I have checked the list of available modules on the cluster and it does not include the netCDF4 version installed by the Firedrake installer (as below). Therefore possibly this is the problem.
Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters.
I am currently checking your other question - previously we found that the code only worked when we explicitly passed those environment variables to mpirun.
Wait, what do you need the PYTHONPATH for? And why don't you load netCDF?
Yes, if you do that then python does run from within the Firedrake virtual environment...but this is essentially what we do when we submit a job to the scheduler:
#!/bin/sh
#PBS -d .
#PBS -q ptq
#PBS -l walltime=00:05:00
#PBS -A Research_Project-183035
#PBS -l nodes=1:ppn=16
#PBS -m e -M p.burns2@exeter.ac.uk
echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>'
echo PBS_O_HOST = $PBS_O_HOST
echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT
echo PBS_NODEFILE = $PBS_NODEFILE
echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>'
module purge
#module load OpenMPI/2.0.0-GCC-4.8.5-torque
#module load Python/2.7.11-foss-2016a
module load OpenMPI/1.10.2-GCC-4.9.3-2.25
module load Python/3.5.1-foss-2016a
#module load netCDF/4.4.0-foss-2016a
. ~/firedrake-20171026/bin/activate
source ~/.bash_profile
#exec_address="./examples/boussinesq_2d_lab.py"
exec_address="./examples/sk_nonlinear.py"
echo -e 'Running Firedrake...'
mpirun \
-x FIREDRAKE_TSFC_KERNEL_CACHE_DIR \
-x PYOP2_CACHE_DIR \
-x LIBRARY_PATH \
-x LD_LIBRARY_PATH \
-x CPATH \
-x VIRTUAL_ENV \
-x PATH \
-x PKG_CONFIG_PATH \
-x PYTHONHOME \
-x PYTHONPATH \
-np 16 python $exec_address > log
So maybe the problem is that the python path is incorrect...
P
OK, now what if you load that Python module, and then activate the Firedrake venv, and then run
$ python
?
Confirmed - python does not run from within the virtual environment.
Note that for your second test to make any sense, you first would need to load a python module. So I did:
module load Python/3.5.1-foss-2016a
(foobar) [pb412@login02 ~]$ python
Python 3.5.1 (default, Jul 24 2017, 16:03:47)
[GCC 4.9.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
You don't even have a running Python in your venv! To confirm, can you just run
$ python
while still the Firedrake venv is still _active_?
And outside the Firedrake venv, try this:
$ python3 -mvenv foobar
$ . foobar/bin/activate
$ python
This is what happens when I execute the first line (from within virtual environment):
(firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4
/gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory
Paul
What if you try to reinstall netCDF4 from source inside the virtualenv? That is,
$ pip uninstall netCDF4
$ pip uninstall netCDF4
$ pip uninstall netCDF4
$ pip install --no-binary netCDF4 netCDF4
They are indeed syntax errors. Running Python 2 code under Python 3.
So the first important error message I can see in the log is:
-----------------------------------------------------------------------------------------------------
checking for library 'lmpe' ...
/gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o
/gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest
/gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
building 'mpe' dylib library
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/src
creating build/temp.linux-x86_64-3.5/src/lib-pmpi
/gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o
creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi
...
- To me it looks like the installer managed to work around the error but I am not certain.....?
Then further down the log there are a number of syntax-type errors. Here are the fist few:
-----------------------------------------------------------------------------------------------------
Removing source in /tmp/pip-build-b2dkectq/wrapt
*** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'...
File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7
def func(): print 'yo'
^
SyntaxError: invalid syntax
*** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'...
File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0
SyntaxError: unknown encoding: lala
*** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'...
File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32
except ValueError, ex:
^
SyntaxError: invalid syntax
*** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'...
File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78
exec 'c = 3'
^
SyntaxError: Missing parentheses in call to 'exec'
P
OK, great.
A quick google of the import error suggests this could be either a cython issue or a python version issue.
I wonder if it would be better to start from the errors that the usual firedrake install log gives?
[I'm at the end of my expertise on such matters now, by the way! What do others think?]
Jemma
By passing the flag '--install gusto' to the installer...
P
Hi Paul,
How did you install Gusto?
Jemma