Yes, if you do that then python does run from within the Firedrake virtual environment...but this is essentially what we do when we submit a job to the scheduler:
#!/bin/sh
#PBS -d .
#PBS -q ptq
#PBS -l walltime=00:05:00
#PBS -A Research_Project-183035
#PBS -l nodes=1:ppn=16
#PBS -m e -M p.burns2@exeter.ac.uk
echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>'
echo PBS_O_HOST = $PBS_O_HOST
echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT
echo PBS_NODEFILE = $PBS_NODEFILE
echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>'
module purge
#module load OpenMPI/2.0.0-GCC-4.8.5-torque
#module load Python/2.7.11-foss-2016a
module load OpenMPI/1.10.2-GCC-4.9.3-2.25
module load Python/3.5.1-foss-2016a
#module load netCDF/4.4.0-foss-2016a
. ~/firedrake-20171026/bin/activate
source ~/.bash_profile
#exec_address="./examples/boussinesq_2d_lab.py"
exec_address="./examples/sk_nonlinear.py"
echo -e 'Running Firedrake...'
mpirun \
-x FIREDRAKE_TSFC_KERNEL_CACHE_DIR \
-x PYOP2_CACHE_DIR \
-x LIBRARY_PATH \
-x LD_LIBRARY_PATH \
-x CPATH \
-x VIRTUAL_ENV \
-x PATH \
-x PKG_CONFIG_PATH \
-x PYTHONHOME \
-x PYTHONPATH \
-np 16 python $exec_address > log
So maybe the problem is that the python path is incorrect...
P
OK, now what if you load that Python module, and then activate the Firedrake venv, and then run
$ python
?
Confirmed - python does not run from within the virtual environment.
Note that for your second test to make any sense, you first would need to load a python module. So I did:
module load Python/3.5.1-foss-2016a
(foobar) [pb412@login02 ~]$ python
Python 3.5.1 (default, Jul 24 2017, 16:03:47)
[GCC 4.9.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
You don't even have a running Python in your venv! To confirm, can you just run
$ python
while still the Firedrake venv is still _active_?
And outside the Firedrake venv, try this:
$ python3 -mvenv foobar
$ . foobar/bin/activate
$ python
This is what happens when I execute the first line (from within virtual environment):
(firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4
/gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory
Paul
What if you try to reinstall netCDF4 from source inside the virtualenv? That is,
$ pip uninstall netCDF4
$ pip uninstall netCDF4
$ pip uninstall netCDF4
$ pip install --no-binary netCDF4 netCDF4
They are indeed syntax errors. Running Python 2 code under Python 3.
So the first important error message I can see in the log is:
-----------------------------------------------------------------------------------------------------
checking for library 'lmpe' ...
/gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o
/gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest
/gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
building 'mpe' dylib library
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/src
creating build/temp.linux-x86_64-3.5/src/lib-pmpi
/gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o
creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi
...
- To me it looks like the installer managed to work around the error but I am not certain.....?
Then further down the log there are a number of syntax-type errors. Here are the fist few:
-----------------------------------------------------------------------------------------------------
Removing source in /tmp/pip-build-b2dkectq/wrapt
*** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'...
File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7
def func(): print 'yo'
^
SyntaxError: invalid syntax
*** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'...
File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0
SyntaxError: unknown encoding: lala
*** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'...
File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32
except ValueError, ex:
^
SyntaxError: invalid syntax
*** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'...
File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78
exec 'c = 3'
^
SyntaxError: Missing parentheses in call to 'exec'
P
OK, great.
A quick google of the import error suggests this could be either a cython issue or a python version issue.
I wonder if it would be better to start from the errors that the usual firedrake install log gives?
[I'm at the end of my expertise on such matters now, by the way! What do others think?]
Jemma
By passing the flag '--install gusto' to the installer...
P
Hi Paul,
How did you install Gusto?
Jemma