Yes, if you do that then python does run from within the Firedrake virtual environment...but this is essentially what we do when we submit a job to the scheduler:


#!/bin/sh

#PBS -d .

#PBS -q ptq 

#PBS -l walltime=00:05:00 

#PBS -A Research_Project-183035 

#PBS -l nodes=1:ppn=16 

#PBS -m e -M p.burns2@exeter.ac.uk


echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>'

echo PBS_O_HOST      = $PBS_O_HOST

echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT

echo PBS_NODEFILE    = $PBS_NODEFILE

echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>'


module purge

#module load OpenMPI/2.0.0-GCC-4.8.5-torque

#module load Python/2.7.11-foss-2016a


module load OpenMPI/1.10.2-GCC-4.9.3-2.25

module load Python/3.5.1-foss-2016a

#module load netCDF/4.4.0-foss-2016a


~/firedrake-20171026/bin/activate

source ~/.bash_profile


#exec_address="./examples/boussinesq_2d_lab.py"

exec_address="./examples/sk_nonlinear.py"


echo -e 'Running Firedrake...'


mpirun \

-x FIREDRAKE_TSFC_KERNEL_CACHE_DIR \

-x PYOP2_CACHE_DIR \

-x LIBRARY_PATH \

-x LD_LIBRARY_PATH \

-x CPATH \

-x VIRTUAL_ENV \

-x PATH \

-x PKG_CONFIG_PATH \

-x PYTHONHOME \

-x PYTHONPATH \

-np 16 python $exec_address > log


So maybe the problem is that the python path is incorrect...


P


From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk>
Sent: 27 October 2017 13:18:56
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 

OK, now what if you load that Python module, and then activate the Firedrake venv, and then run

$ python

?


From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk>
Sent: 27 October 2017 13:06:08
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 

Confirmed - python does not run from within the virtual environment.


Note that for your second test to make any sense, you first would need to load a python module.  So I did:


module load Python/3.5.1-foss-2016a

python3 -mvenv foobar
. foobar/bin/activate
python

After executing the last line python does run:

(foobar) [pb412@login02 ~]$ python

Python 3.5.1 (default, Jul 24 2017, 16:03:47) 

[GCC 4.9.3] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>>       


From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk>
Sent: 27 October 2017 12:49:59
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 

You don't even have a running Python in your venv! To confirm, can you just run

$ python

while still the Firedrake venv is still _active_?


And outside the Firedrake venv, try this:

$ python3 -mvenv foobar

$ . foobar/bin/activate

$ python


From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk>
Sent: 27 October 2017 12:38:02
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 

This is what happens when I execute the first line (from within virtual environment):


(firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4

/gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory


Paul


From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk>
Sent: 27 October 2017 12:19:54
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 

What if you try to reinstall netCDF4 from source inside the virtualenv? That is,


$ pip uninstall netCDF4

$ pip uninstall netCDF4

$ pip uninstall netCDF4

$ pip install --no-binary netCDF4 netCDF4



From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk>
Sent: 27 October 2017 11:59:39
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 

They are indeed syntax errors. Running Python 2 code under Python 3.


From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk>
Sent: 27 October 2017 11:06:47
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 

So the first important error message I can see in the log is:


-----------------------------------------------------------------------------------------------------

   checking for library 'lmpe' ...

    /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o

    /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest

    /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe

    collect2: error: ld returned 1 exit status

    failure.

    removing: _configtest.c _configtest.o

    building 'mpe' dylib library

    creating build/temp.linux-x86_64-3.5

    creating build/temp.linux-x86_64-3.5/src

    creating build/temp.linux-x86_64-3.5/src/lib-pmpi

    /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o

    creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi

...

-----------------------------------------------------------------------------------------------------

- To me it looks like the installer managed to work around the error but I am not certain.....?



Then further down the log there are a number of syntax-type errors.  Here are the fist few:

-----------------------------------------------------------------------------------------------------

 Removing source in /tmp/pip-build-b2dkectq/wrapt


  *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'...

    File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7

      def func(): print 'yo'

                           ^

  SyntaxError: invalid syntax

  

  *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'...

    File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0

  SyntaxError: unknown encoding: lala

  

  *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'...

    File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32

      except ValueError, ex:

                       ^

  SyntaxError: invalid syntax

  

  *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'...

    File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78

      exec 'c = 3'

                 ^

  SyntaxError: Missing parentheses in call to 'exec'

-----------------------------------------------------------------------------------------------------





P





From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk>
Sent: 27 October 2017 10:39:31
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 
Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence:

  Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl" in the cache
  No cache entry available
  "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771
  Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB)
  Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl#md5=d4660a6dea3062578fc9cfec6b243949 (from https://pypi.python.org/simple/netcdf4/)
  Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl"
  Caching due to etag
Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1))
 
Installing collected packages: netCDF4


It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4:

netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6

I will have another look at the install log and search for errors.  See attached the install log - I am not an expert in this stuff.


P


From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk>
Sent: 26 October 2017 18:06:25
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 

OK, great.


A quick google of the import error suggests this could be either a cython issue or a python version issue.


I wonder if it would be better to start from the errors that the usual firedrake install log gives?


[I'm at the end of my expertise on such matters now, by the way! What do others think?]


Jemma


From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk>
Sent: 26 October 2017 17:55:20
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 

By passing the flag '--install gusto' to the installer...


P


From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk>
Sent: 26 October 2017 15:14:50
To: firedrake
Subject: Re: [firedrake] Firedrake installation on cluster
 

Hi Paul,


How did you install Gusto?


Jemma


From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk>
Sent: 26 October 2017 15:10:14
To: firedrake
Subject: [firedrake] Firedrake installation on cluster
 
Dear Sir/Madam,

I am trying to get Firedrake up and running on the Isca cluster at Exeter.  The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install.

However, when I try and run a simulation on Gusto I get the following error: 

Traceback (most recent call last):
  File "./examples/boussinesq_2d_lab.py", line 1, in <module>
    from gusto import *
  File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module>
    from gusto.state import *           # noqa
  File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module>
    from netCDF4 import Dataset
  File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module>
    from ._netCDF4 import *
 
ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet


So it looks like there is some problem with the netCDF4 library.

Please advise.

Regards,
Paul