Firedrake installation on cluster
Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
You don't even have a running Python in your venv! To confirm, can you just run $ python while still the Firedrake venv is still _active_? And outside the Firedrake venv, try this: $ python3 -mvenv foobar $ . foobar/bin/activate $ python ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 12:38:02 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
Confirmed - python does not run from within the virtual environment. Note that for your second test to make any sense, you first would need to load a python module. So I did: module load Python/3.5.1-foss-2016a python3 -mvenv foobar . foobar/bin/activate python After executing the last line python does run: (foobar) [pb412@login02 ~]$ python Python 3.5.1 (default, Jul 24 2017, 16:03:47) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information.
________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:49:59 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster You don't even have a running Python in your venv! To confirm, can you just run $ python while still the Firedrake venv is still _active_? And outside the Firedrake venv, try this: $ python3 -mvenv foobar $ . foobar/bin/activate $ python ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 12:38:02 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
OK, now what if you load that Python module, and then activate the Firedrake venv, and then run $ python ? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:06:08 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Confirmed - python does not run from within the virtual environment. Note that for your second test to make any sense, you first would need to load a python module. So I did: module load Python/3.5.1-foss-2016a python3 -mvenv foobar . foobar/bin/activate python After executing the last line python does run: (foobar) [pb412@login02 ~]$ python Python 3.5.1 (default, Jul 24 2017, 16:03:47) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information.
________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:49:59 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster You don't even have a running Python in your venv! To confirm, can you just run $ python while still the Firedrake venv is still _active_? And outside the Firedrake venv, try this: $ python3 -mvenv foobar $ . foobar/bin/activate $ python ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 12:38:02 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
Yes, if you do that then python does run from within the Firedrake virtual environment...but this is essentially what we do when we submit a job to the scheduler: #!/bin/sh #PBS -d . #PBS -q ptq #PBS -l walltime=00:05:00 #PBS -A Research_Project-183035 #PBS -l nodes=1:ppn=16 #PBS -m e -M p.burns2@exeter.ac.uk echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' echo PBS_O_HOST = $PBS_O_HOST echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT echo PBS_NODEFILE = $PBS_NODEFILE echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' module purge #module load OpenMPI/2.0.0-GCC-4.8.5-torque #module load Python/2.7.11-foss-2016a module load OpenMPI/1.10.2-GCC-4.9.3-2.25 module load Python/3.5.1-foss-2016a #module load netCDF/4.4.0-foss-2016a . ~/firedrake-20171026/bin/activate source ~/.bash_profile #exec_address="./examples/boussinesq_2d_lab.py" exec_address="./examples/sk_nonlinear.py" echo -e 'Running Firedrake...' mpirun \ -x FIREDRAKE_TSFC_KERNEL_CACHE_DIR \ -x PYOP2_CACHE_DIR \ -x LIBRARY_PATH \ -x LD_LIBRARY_PATH \ -x CPATH \ -x VIRTUAL_ENV \ -x PATH \ -x PKG_CONFIG_PATH \ -x PYTHONHOME \ -x PYTHONPATH \ -np 16 python $exec_address > log So maybe the problem is that the python path is incorrect... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:18:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, now what if you load that Python module, and then activate the Firedrake venv, and then run $ python ? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:06:08 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Confirmed - python does not run from within the virtual environment. Note that for your second test to make any sense, you first would need to load a python module. So I did: module load Python/3.5.1-foss-2016a python3 -mvenv foobar . foobar/bin/activate python After executing the last line python does run: (foobar) [pb412@login02 ~]$ python Python 3.5.1 (default, Jul 24 2017, 16:03:47) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information.
________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:49:59 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster You don't even have a running Python in your venv! To confirm, can you just run $ python while still the Firedrake venv is still _active_? And outside the Firedrake venv, try this: $ python3 -mvenv foobar $ . foobar/bin/activate $ python ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 12:38:02 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
Wait, what do you need the PYTHONPATH for? And why don't you load netCDF? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:29:41 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Yes, if you do that then python does run from within the Firedrake virtual environment...but this is essentially what we do when we submit a job to the scheduler: #!/bin/sh #PBS -d . #PBS -q ptq #PBS -l walltime=00:05:00 #PBS -A Research_Project-183035 #PBS -l nodes=1:ppn=16 #PBS -m e -M p.burns2@exeter.ac.uk echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' echo PBS_O_HOST = $PBS_O_HOST echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT echo PBS_NODEFILE = $PBS_NODEFILE echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' module purge #module load OpenMPI/2.0.0-GCC-4.8.5-torque #module load Python/2.7.11-foss-2016a module load OpenMPI/1.10.2-GCC-4.9.3-2.25 module load Python/3.5.1-foss-2016a #module load netCDF/4.4.0-foss-2016a . ~/firedrake-20171026/bin/activate source ~/.bash_profile #exec_address="./examples/boussinesq_2d_lab.py" exec_address="./examples/sk_nonlinear.py" echo -e 'Running Firedrake...' mpirun \ -x FIREDRAKE_TSFC_KERNEL_CACHE_DIR \ -x PYOP2_CACHE_DIR \ -x LIBRARY_PATH \ -x LD_LIBRARY_PATH \ -x CPATH \ -x VIRTUAL_ENV \ -x PATH \ -x PKG_CONFIG_PATH \ -x PYTHONHOME \ -x PYTHONPATH \ -np 16 python $exec_address > log So maybe the problem is that the python path is incorrect... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:18:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, now what if you load that Python module, and then activate the Firedrake venv, and then run $ python ? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:06:08 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Confirmed - python does not run from within the virtual environment. Note that for your second test to make any sense, you first would need to load a python module. So I did: module load Python/3.5.1-foss-2016a python3 -mvenv foobar . foobar/bin/activate python After executing the last line python does run: (foobar) [pb412@login02 ~]$ python Python 3.5.1 (default, Jul 24 2017, 16:03:47) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information.
________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:49:59 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster You don't even have a running Python in your venv! To confirm, can you just run $ python while still the Firedrake venv is still _active_? And outside the Firedrake venv, try this: $ python3 -mvenv foobar $ . foobar/bin/activate $ python ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 12:38:02 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
I did try loading that netCDF4 module (in my submission script) prior to activating the Firedrake virtual environment, however, it then throws a version error at me. I have checked the list of available modules on the cluster and it does not include the netCDF4 version installed by the Firedrake installer (as below). Therefore possibly this is the problem. Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters. I am currently checking your other question - previously we found that the code only worked when we explicitly passed those environment variables to mpirun. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:33:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Wait, what do you need the PYTHONPATH for? And why don't you load netCDF? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:29:41 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Yes, if you do that then python does run from within the Firedrake virtual environment...but this is essentially what we do when we submit a job to the scheduler: #!/bin/sh #PBS -d . #PBS -q ptq #PBS -l walltime=00:05:00 #PBS -A Research_Project-183035 #PBS -l nodes=1:ppn=16 #PBS -m e -M p.burns2@exeter.ac.uk echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' echo PBS_O_HOST = $PBS_O_HOST echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT echo PBS_NODEFILE = $PBS_NODEFILE echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' module purge #module load OpenMPI/2.0.0-GCC-4.8.5-torque #module load Python/2.7.11-foss-2016a module load OpenMPI/1.10.2-GCC-4.9.3-2.25 module load Python/3.5.1-foss-2016a #module load netCDF/4.4.0-foss-2016a . ~/firedrake-20171026/bin/activate source ~/.bash_profile #exec_address="./examples/boussinesq_2d_lab.py" exec_address="./examples/sk_nonlinear.py" echo -e 'Running Firedrake...' mpirun \ -x FIREDRAKE_TSFC_KERNEL_CACHE_DIR \ -x PYOP2_CACHE_DIR \ -x LIBRARY_PATH \ -x LD_LIBRARY_PATH \ -x CPATH \ -x VIRTUAL_ENV \ -x PATH \ -x PKG_CONFIG_PATH \ -x PYTHONHOME \ -x PYTHONPATH \ -np 16 python $exec_address > log So maybe the problem is that the python path is incorrect... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:18:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, now what if you load that Python module, and then activate the Firedrake venv, and then run $ python ? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:06:08 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Confirmed - python does not run from within the virtual environment. Note that for your second test to make any sense, you first would need to load a python module. So I did: module load Python/3.5.1-foss-2016a python3 -mvenv foobar . foobar/bin/activate python After executing the last line python does run: (foobar) [pb412@login02 ~]$ python Python 3.5.1 (default, Jul 24 2017, 16:03:47) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information.
________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:49:59 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster You don't even have a running Python in your venv! To confirm, can you just run $ python while still the Firedrake venv is still _active_? And outside the Firedrake venv, try this: $ python3 -mvenv foobar $ . foobar/bin/activate $ python ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 12:38:02 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters.
I believe that is correct, yes. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 30 October 2017 09:07 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster I did try loading that netCDF4 module (in my submission script) prior to activating the Firedrake virtual environment, however, it then throws a version error at me. I have checked the list of available modules on the cluster and it does not include the netCDF4 version installed by the Firedrake installer (as below). Therefore possibly this is the problem. Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters. I am currently checking your other question - previously we found that the code only worked when we explicitly passed those environment variables to mpirun. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:33:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Wait, what do you need the PYTHONPATH for? And why don't you load netCDF? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:29:41 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Yes, if you do that then python does run from within the Firedrake virtual environment...but this is essentially what we do when we submit a job to the scheduler: #!/bin/sh #PBS -d . #PBS -q ptq #PBS -l walltime=00:05:00 #PBS -A Research_Project-183035 #PBS -l nodes=1:ppn=16 #PBS -m e -M p.burns2@exeter.ac.uk echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' echo PBS_O_HOST = $PBS_O_HOST echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT echo PBS_NODEFILE = $PBS_NODEFILE echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' module purge #module load OpenMPI/2.0.0-GCC-4.8.5-torque #module load Python/2.7.11-foss-2016a module load OpenMPI/1.10.2-GCC-4.9.3-2.25 module load Python/3.5.1-foss-2016a #module load netCDF/4.4.0-foss-2016a . ~/firedrake-20171026/bin/activate source ~/.bash_profile #exec_address="./examples/boussinesq_2d_lab.py" exec_address="./examples/sk_nonlinear.py" echo -e 'Running Firedrake...' mpirun \ -x FIREDRAKE_TSFC_KERNEL_CACHE_DIR \ -x PYOP2_CACHE_DIR \ -x LIBRARY_PATH \ -x LD_LIBRARY_PATH \ -x CPATH \ -x VIRTUAL_ENV \ -x PATH \ -x PKG_CONFIG_PATH \ -x PYTHONHOME \ -x PYTHONPATH \ -np 16 python $exec_address > log So maybe the problem is that the python path is incorrect... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:18:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, now what if you load that Python module, and then activate the Firedrake venv, and then run $ python ? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:06:08 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Confirmed - python does not run from within the virtual environment. Note that for your second test to make any sense, you first would need to load a python module. So I did: module load Python/3.5.1-foss-2016a python3 -mvenv foobar . foobar/bin/activate python After executing the last line python does run: (foobar) [pb412@login02 ~]$ python Python 3.5.1 (default, Jul 24 2017, 16:03:47) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information.
________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:49:59 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster You don't even have a running Python in your venv! To confirm, can you just run $ python while still the Firedrake venv is still _active_? And outside the Firedrake venv, try this: $ python3 -mvenv foobar $ . foobar/bin/activate $ python ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 12:38:02 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
Hi, So I discussed this with the Isca support team. They think that as long as we enable the Firedrake Python virtual environment to 'fire up', by loading the relevant Python module, before trying to activate the virtual environment, then the correctly activated virtual environment should then take care of the rest of the required software. The exception is that we also need to load an OpenMPI module, as in the submission script. So they don't think we need to load a netCDF4 module - they think this should be sorted out by the Firedrake virtual environment. To answer the other question more completely - we found that we had to pass the environment variables to mpirun because otherwise the slave MPI jobs were missing the virtual environment and using the system Python instead. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 30 October 2017 10:22:03 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster
Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters.
I believe that is correct, yes. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 30 October 2017 09:07 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster I did try loading that netCDF4 module (in my submission script) prior to activating the Firedrake virtual environment, however, it then throws a version error at me. I have checked the list of available modules on the cluster and it does not include the netCDF4 version installed by the Firedrake installer (as below). Therefore possibly this is the problem. Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters. I am currently checking your other question - previously we found that the code only worked when we explicitly passed those environment variables to mpirun. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:33:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Wait, what do you need the PYTHONPATH for? And why don't you load netCDF? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:29:41 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Yes, if you do that then python does run from within the Firedrake virtual environment...but this is essentially what we do when we submit a job to the scheduler: #!/bin/sh #PBS -d . #PBS -q ptq #PBS -l walltime=00:05:00 #PBS -A Research_Project-183035 #PBS -l nodes=1:ppn=16 #PBS -m e -M p.burns2@exeter.ac.uk echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' echo PBS_O_HOST = $PBS_O_HOST echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT echo PBS_NODEFILE = $PBS_NODEFILE echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' module purge #module load OpenMPI/2.0.0-GCC-4.8.5-torque #module load Python/2.7.11-foss-2016a module load OpenMPI/1.10.2-GCC-4.9.3-2.25 module load Python/3.5.1-foss-2016a #module load netCDF/4.4.0-foss-2016a . ~/firedrake-20171026/bin/activate source ~/.bash_profile #exec_address="./examples/boussinesq_2d_lab.py" exec_address="./examples/sk_nonlinear.py" echo -e 'Running Firedrake...' mpirun \ -x FIREDRAKE_TSFC_KERNEL_CACHE_DIR \ -x PYOP2_CACHE_DIR \ -x LIBRARY_PATH \ -x LD_LIBRARY_PATH \ -x CPATH \ -x VIRTUAL_ENV \ -x PATH \ -x PKG_CONFIG_PATH \ -x PYTHONHOME \ -x PYTHONPATH \ -np 16 python $exec_address > log So maybe the problem is that the python path is incorrect... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:18:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, now what if you load that Python module, and then activate the Firedrake venv, and then run $ python ? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:06:08 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Confirmed - python does not run from within the virtual environment. Note that for your second test to make any sense, you first would need to load a python module. So I did: module load Python/3.5.1-foss-2016a python3 -mvenv foobar . foobar/bin/activate python After executing the last line python does run: (foobar) [pb412@login02 ~]$ python Python 3.5.1 (default, Jul 24 2017, 16:03:47) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information.
________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:49:59 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster You don't even have a running Python in your venv! To confirm, can you just run $ python while still the Firedrake venv is still _active_? And outside the Firedrake venv, try this: $ python3 -mvenv foobar $ . foobar/bin/activate $ python ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 12:38:02 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
They think that as long as we enable the Firedrake Python virtual environment to 'fire up', by loading the relevant Python module, before trying to activate the virtual environment, then the correctly activated virtual environment should then take care of the rest of the required software.
I'm not sure I can fully believe that. I mean, the Firedrake virtual environment is self-contained with respect to Python, so it should take care of all the Python dependencies. We also take care of PETSc, and some other compiled libraries, but some compiled libraries are considered "system". For example, we rely on whatever compilers and MPI implementation the host system provides, we only install Cython and mpi4py ourselves in the Python virtualenv. I don't know what the deal with NetCDF is, perhaps some gusto people know the answer: is compiled NetCDF expected to be present as a pre-installed library, or does Gusto have an installer that takes care of that? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 30 October 2017 12:22:26 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi, So I discussed this with the Isca support team. They think that as long as we enable the Firedrake Python virtual environment to 'fire up', by loading the relevant Python module, before trying to activate the virtual environment, then the correctly activated virtual environment should then take care of the rest of the required software. The exception is that we also need to load an OpenMPI module, as in the submission script. So they don't think we need to load a netCDF4 module - they think this should be sorted out by the Firedrake virtual environment. To answer the other question more completely - we found that we had to pass the environment variables to mpirun because otherwise the slave MPI jobs were missing the virtual environment and using the system Python instead. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 30 October 2017 10:22:03 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster
Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters.
I believe that is correct, yes. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 30 October 2017 09:07 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster I did try loading that netCDF4 module (in my submission script) prior to activating the Firedrake virtual environment, however, it then throws a version error at me. I have checked the list of available modules on the cluster and it does not include the netCDF4 version installed by the Firedrake installer (as below). Therefore possibly this is the problem. Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters. I am currently checking your other question - previously we found that the code only worked when we explicitly passed those environment variables to mpirun. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:33:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Wait, what do you need the PYTHONPATH for? And why don't you load netCDF? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:29:41 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Yes, if you do that then python does run from within the Firedrake virtual environment...but this is essentially what we do when we submit a job to the scheduler: #!/bin/sh #PBS -d . #PBS -q ptq #PBS -l walltime=00:05:00 #PBS -A Research_Project-183035 #PBS -l nodes=1:ppn=16 #PBS -m e -M p.burns2@exeter.ac.uk echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' echo PBS_O_HOST = $PBS_O_HOST echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT echo PBS_NODEFILE = $PBS_NODEFILE echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' module purge #module load OpenMPI/2.0.0-GCC-4.8.5-torque #module load Python/2.7.11-foss-2016a module load OpenMPI/1.10.2-GCC-4.9.3-2.25 module load Python/3.5.1-foss-2016a #module load netCDF/4.4.0-foss-2016a . ~/firedrake-20171026/bin/activate source ~/.bash_profile #exec_address="./examples/boussinesq_2d_lab.py" exec_address="./examples/sk_nonlinear.py" echo -e 'Running Firedrake...' mpirun \ -x FIREDRAKE_TSFC_KERNEL_CACHE_DIR \ -x PYOP2_CACHE_DIR \ -x LIBRARY_PATH \ -x LD_LIBRARY_PATH \ -x CPATH \ -x VIRTUAL_ENV \ -x PATH \ -x PKG_CONFIG_PATH \ -x PYTHONHOME \ -x PYTHONPATH \ -np 16 python $exec_address > log So maybe the problem is that the python path is incorrect... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:18:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, now what if you load that Python module, and then activate the Firedrake venv, and then run $ python ? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:06:08 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Confirmed - python does not run from within the virtual environment. Note that for your second test to make any sense, you first would need to load a python module. So I did: module load Python/3.5.1-foss-2016a python3 -mvenv foobar . foobar/bin/activate python After executing the last line python does run: (foobar) [pb412@login02 ~]$ python Python 3.5.1 (default, Jul 24 2017, 16:03:47) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information.
________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:49:59 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster You don't even have a running Python in your venv! To confirm, can you just run $ python while still the Firedrake venv is still _active_? And outside the Firedrake venv, try this: $ python3 -mvenv foobar $ . foobar/bin/activate $ python ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 12:38:02 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
netCDF4 is in gusto/requirements.txt so I think that means it is taken care of by firedrake. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 30 October 2017 12:52:34 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster
They think that as long as we enable the Firedrake Python virtual environment to 'fire up', by loading the relevant Python module, before trying to activate the virtual environment, then the correctly activated virtual environment should then take care of the rest of the required software.
I'm not sure I can fully believe that. I mean, the Firedrake virtual environment is self-contained with respect to Python, so it should take care of all the Python dependencies. We also take care of PETSc, and some other compiled libraries, but some compiled libraries are considered "system". For example, we rely on whatever compilers and MPI implementation the host system provides, we only install Cython and mpi4py ourselves in the Python virtualenv. I don't know what the deal with NetCDF is, perhaps some gusto people know the answer: is compiled NetCDF expected to be present as a pre-installed library, or does Gusto have an installer that takes care of that? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 30 October 2017 12:22:26 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi, So I discussed this with the Isca support team. They think that as long as we enable the Firedrake Python virtual environment to 'fire up', by loading the relevant Python module, before trying to activate the virtual environment, then the correctly activated virtual environment should then take care of the rest of the required software. The exception is that we also need to load an OpenMPI module, as in the submission script. So they don't think we need to load a netCDF4 module - they think this should be sorted out by the Firedrake virtual environment. To answer the other question more completely - we found that we had to pass the environment variables to mpirun because otherwise the slave MPI jobs were missing the virtual environment and using the system Python instead. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 30 October 2017 10:22:03 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster
Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters.
I believe that is correct, yes. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 30 October 2017 09:07 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster I did try loading that netCDF4 module (in my submission script) prior to activating the Firedrake virtual environment, however, it then throws a version error at me. I have checked the list of available modules on the cluster and it does not include the netCDF4 version installed by the Firedrake installer (as below). Therefore possibly this is the problem. Presumably we need to load all modules required by the Firedrake virtual environment prior to activating the virtual environment. I guess this is just a quirk of using clusters. I am currently checking your other question - previously we found that the code only worked when we explicitly passed those environment variables to mpirun. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:33:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Wait, what do you need the PYTHONPATH for? And why don't you load netCDF? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:29:41 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Yes, if you do that then python does run from within the Firedrake virtual environment...but this is essentially what we do when we submit a job to the scheduler: #!/bin/sh #PBS -d . #PBS -q ptq #PBS -l walltime=00:05:00 #PBS -A Research_Project-183035 #PBS -l nodes=1:ppn=16 #PBS -m e -M p.burns2@exeter.ac.uk echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' echo PBS_O_HOST = $PBS_O_HOST echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT echo PBS_NODEFILE = $PBS_NODEFILE echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' module purge #module load OpenMPI/2.0.0-GCC-4.8.5-torque #module load Python/2.7.11-foss-2016a module load OpenMPI/1.10.2-GCC-4.9.3-2.25 module load Python/3.5.1-foss-2016a #module load netCDF/4.4.0-foss-2016a . ~/firedrake-20171026/bin/activate source ~/.bash_profile #exec_address="./examples/boussinesq_2d_lab.py" exec_address="./examples/sk_nonlinear.py" echo -e 'Running Firedrake...' mpirun \ -x FIREDRAKE_TSFC_KERNEL_CACHE_DIR \ -x PYOP2_CACHE_DIR \ -x LIBRARY_PATH \ -x LD_LIBRARY_PATH \ -x CPATH \ -x VIRTUAL_ENV \ -x PATH \ -x PKG_CONFIG_PATH \ -x PYTHONHOME \ -x PYTHONPATH \ -np 16 python $exec_address > log So maybe the problem is that the python path is incorrect... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 13:18:56 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, now what if you load that Python module, and then activate the Firedrake venv, and then run $ python ? ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 13:06:08 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Confirmed - python does not run from within the virtual environment. Note that for your second test to make any sense, you first would need to load a python module. So I did: module load Python/3.5.1-foss-2016a python3 -mvenv foobar . foobar/bin/activate python After executing the last line python does run: (foobar) [pb412@login02 ~]$ python Python 3.5.1 (default, Jul 24 2017, 16:03:47) [GCC 4.9.3] on linux Type "help", "copyright", "credits" or "license" for more information.
________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:49:59 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster You don't even have a running Python in your venv! To confirm, can you just run $ python while still the Firedrake venv is still _active_? And outside the Firedrake venv, try this: $ python3 -mvenv foobar $ . foobar/bin/activate $ python ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 12:38:02 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster This is what happens when I execute the first line (from within virtual environment): (firedrake-20171026) [pb412@login02 ~]$ pip uninstall netCDF4 /gpfs/ts0/home/pb412/firedrake-20171026/bin/python: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 12:19:54 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster What if you try to reinstall netCDF4 from source inside the virtualenv? That is, $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip uninstall netCDF4 $ pip install --no-binary netCDF4 netCDF4 ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Homolya, Miklós <m.homolya14@imperial.ac.uk> Sent: 27 October 2017 11:59:39 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster They are indeed syntax errors. Running Python 2 code under Python 3. ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 11:06:47 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster So the first important error message I can see in the log is: ----------------------------------------------------------------------------------------------------- checking for library 'lmpe' ... /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c _configtest.c -o _configtest.o /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc _configtest.o -llmpe -o _configtest /gpfs/ts0/shared/software/binutils/2.25-GCCcore-4.9.3/bin/ld.gold: error: cannot find -llmpe collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/src creating build/temp.linux-x86_64-3.5/src/lib-pmpi /gpfs/ts0/shared/software/OpenMPI/1.10.2-GCC-4.9.3-2.25/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fPIC -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-3.5/src/lib-pmpi/mpe.o creating build/lib.linux-x86_64-3.5/mpi4py/lib-pmpi ... ----------------------------------------------------------------------------------------------------- - To me it looks like the installer managed to work around the error but I am not certain.....? Then further down the log there are a number of syntax-type errors. Here are the fist few: ----------------------------------------------------------------------------------------------------- Removing source in /tmp/pip-build-b2dkectq/wrapt *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/all.py", line 7 def func(): print 'yo' ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/invalid_encoding.py", line 0 SyntaxError: unknown encoding: lala *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module.py", line 32 except ValueError, ex: ^ SyntaxError: invalid syntax *** Error compiling '/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py'... File "/tmp/pip-build-b2dkectq/astroid/astroid/tests/testdata/python2/data/module2.py", line 78 exec 'c = 3' ^ SyntaxError: Missing parentheses in call to 'exec' ----------------------------------------------------------------------------------------------------- P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 27 October 2017 10:39:31 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Previously I did search the firedrake-install.log file for pattern 'netcdf4'...this is the last occurrence: Looking up "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." in the cache No cache entry available "GET /packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25edb3a144af71f88a2dfcb69a/netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl HTTP/1.1" 200 6163771 Downloading netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_64.whl (6.2MB) Downloading from URL https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e... (from https://pypi.python.org/simple/netcdf4/) Updating cache with response from "https://pypi.python.org/packages/c0/06/c59f37661ede4f9ae1af893e08f9c214bc25e..." Caching due to etag Requirement already satisfied: numpy>=1.7 in /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages (from netCDF4->-r gusto/requirements.txt (line 1)) Installing collected packages: netCDF4 It looks like it struggled to find a compatible version of netcdf4 but in the end was successful and installed this version of netcdf4: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I will have another look at the install log and search for errors. See attached the install log - I am not an expert in this stuff. P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 18:06:25 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster OK, great. A quick google of the import error suggests this could be either a cython issue or a python version issue. I wonder if it would be better to start from the errors that the usual firedrake install log gives? [I'm at the end of my expertise on such matters now, by the way! What do others think?] Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 17:55:20 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster By passing the flag '--install gusto' to the installer... P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Shipton, Jemma <j.shipton@imperial.ac.uk> Sent: 26 October 2017 15:14:50 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi Paul, How did you install Gusto? Jemma ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 26 October 2017 15:10:14 To: firedrake Subject: [firedrake] Firedrake installation on cluster Dear Sir/Madam, I am trying to get Firedrake up and running on the Isca cluster at Exeter. The standard install method (i.e., executing 'python3 firedrake-install') does not work, however, we have managed to install Firedrake by developing an install script that loads modules manually, that sets a number of petsc options, and that passes a number of extra options to firedrake-install. However, when I try and run a simulation on Gusto I get the following error: Traceback (most recent call last): File "./examples/boussinesq_2d_lab.py", line 1, in <module> from gusto import * File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/__init__.py", line 11, in <module> from gusto.state import * # noqa File "/gpfs/ts0/home/pb412/firedrake-20171026/src/gusto/gusto/state.py", line 3, in <module> from netCDF4 import Dataset File "/gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * ImportError: /gpfs/ts0/home/pb412/firedrake-20171026/lib/python3.5/site-packages/netCDF4/_netCDF4.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet So it looks like there is some problem with the netCDF4 library. Please advise. Regards, Paul
On 30 Oct 2017, at 12:57, Shipton, Jemma <j.shipton@imperial.ac.uk> wrote:
netCDF4 is in gusto/requirements.txt so I think that means it is taken care of by firedrake.
well, we just "pip install" it. Plausibly it ships with a wheel which links against a netcdf version that doesn't match whatever the cluster provides? Lawrence
Hi, As stated below we did manage to install the Firedrake Python virtual environment on Isca. However, I thought it might be useful to post our install script here in case someone can see something in it that might be leading to the problems encountered when trying to run the model. I thought it also might be of use to other people trying to install Firedrake on a cluster. Here is the install script: #!/bin/sh #PBS -d . #PBS -q sq #PBS -l walltime=01:00:00 #PBS -A Research_Project-183035 #PBS -l nodes=1:ppn=1 #PBS -m e -M p.burns2@exeter.ac.uk echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' echo PBS_O_HOST = $PBS_O_HOST echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT echo PBS_NODEFILE = $PBS_NODEFILE echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' VENV="firedrake-`date +%Y%m%d`" # Commands used to build firedrake on ISCA user home space. (firedrake does not play well with easybuild) module purge module load Automake/1.15-foss-2016a module load libtool/2.4.6-foss-2016a module load CMake/3.5.2-foss-2016a module load Python/3.5.1-foss-2016a module load Torque/6.0.2-foss-2016a-devel export PETSC_CONFIGURE_OPTIONS="--download-chaco --download-metis --download-parmetis --download-scalapack --download-hypre --download-mumps --download-netcdf --download-hdf5 --download-exodusii --download-eigen --download-fblaslapack=1" unset PYTHONPATH curl -O https://raw.githubusercontent.com/firedrakeproject/firedrake/master/scripts/... python3 ./firedrake-install --no-package-manager --disable-ssh --venv-name $VENV --install gusto Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Lawrence Mitchell <lawrence.mitchell@imperial.ac.uk> Sent: 30 October 2017 13:00:41 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster
On 30 Oct 2017, at 12:57, Shipton, Jemma <j.shipton@imperial.ac.uk> wrote:
netCDF4 is in gusto/requirements.txt so I think that means it is taken care of by firedrake.
well, we just "pip install" it. Plausibly it ships with a wheel which links against a netcdf version that doesn't match whatever the cluster provides? Lawrence _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi, Jemma mentioned that you think it might be a good idea to try pre-loading the relevant netCDF4 module before activating the Firedrake virtual environment on Isca? Presumably, to use the 'module load' command we would need to ask the Isca team to add a new version of netCDF4 to their central library of software. I have checked and Isca does not include the version of netCDF4 installed by the Firedrake installer. The Firedrake installer seemed to install: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6 I did try pre-loading another version of netCDF4, however, when I ran the model it then threw a version-type error at me. Can I check if you think there is any other software (except for OpenMPI) that needs to be pre-loaded prior to activating the virtual env? Thank you, P ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Burns, Paul <P.Burns2@exeter.ac.uk> Sent: 01 November 2017 13:20:03 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster Hi, As stated below we did manage to install the Firedrake Python virtual environment on Isca. However, I thought it might be useful to post our install script here in case someone can see something in it that might be leading to the problems encountered when trying to run the model. I thought it also might be of use to other people trying to install Firedrake on a cluster. Here is the install script: #!/bin/sh #PBS -d . #PBS -q sq #PBS -l walltime=01:00:00 #PBS -A Research_Project-183035 #PBS -l nodes=1:ppn=1 #PBS -m e -M p.burns2@exeter.ac.uk echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' echo PBS_O_HOST = $PBS_O_HOST echo PBS_ENVIRONMENT = $PBS_ENVIRONMENT echo PBS_NODEFILE = $PBS_NODEFILE echo '<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>' VENV="firedrake-`date +%Y%m%d`" # Commands used to build firedrake on ISCA user home space. (firedrake does not play well with easybuild) module purge module load Automake/1.15-foss-2016a module load libtool/2.4.6-foss-2016a module load CMake/3.5.2-foss-2016a module load Python/3.5.1-foss-2016a module load Torque/6.0.2-foss-2016a-devel export PETSC_CONFIGURE_OPTIONS="--download-chaco --download-metis --download-parmetis --download-scalapack --download-hypre --download-mumps --download-netcdf --download-hdf5 --download-exodusii --download-eigen --download-fblaslapack=1" unset PYTHONPATH curl -O https://raw.githubusercontent.com/firedrakeproject/firedrake/master/scripts/... python3 ./firedrake-install --no-package-manager --disable-ssh --venv-name $VENV --install gusto Paul ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Lawrence Mitchell <lawrence.mitchell@imperial.ac.uk> Sent: 30 October 2017 13:00:41 To: firedrake Subject: Re: [firedrake] Firedrake installation on cluster
On 30 Oct 2017, at 12:57, Shipton, Jemma <j.shipton@imperial.ac.uk> wrote:
netCDF4 is in gusto/requirements.txt so I think that means it is taken care of by firedrake.
well, we just "pip install" it. Plausibly it ships with a wheel which links against a netcdf version that doesn't match whatever the cluster provides? Lawrence _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
On 02/11/17 13:38, Burns, Paul wrote:
Hi,
Jemma mentioned that you think it might be a good idea to try pre-loading the relevant netCDF4 module before activating the Firedrake virtual environment on Isca?
Presumably, to use the 'module load' command we would need to ask the Isca team to add a new version of netCDF4 to their central library of software. I have checked and Isca does not include the version of netCDF4 installed by the Firedrake installer. The Firedrake installer seemed to install: netCDF4-1.3.0-cp35-cp35m-manylinux1_x86_6
OK, one thing that may be happening here is that the netcdf package that pip installs comes with a mismatching netcdf library. Can you do pip uninstall netCDF4 Then load all the modules you normally do, then use pip install --no-binary netCDF4 netCDF4 Hopefully then the install will link against the system-provided netcdf. Lawrence
participants (4)
-
Burns, Paul
-
Homolya, Miklós
-
Lawrence Mitchell
-
Shipton, Jemma