Problem to install Firedrake on Ubuntu 18.04.2 LTS
Hi, My name is Simone Puel, a graduate student at the University of Texas at Austin. I would like to use Firedrake for my research, but once installed I run the test suggested to check that the Firedrake installation is fully functional, but I got the following error. I am using Ubuntu-Mate distribution with ubuntu bionic 18.04.2 LTS with Python 3.6.8 in Anaconda environment. Could you help me in the installation, please? Thank you so much in advance. Have a wonderful day. Best regards, Simone ERROR: simone@simone:~$ source firedrake/bin/activate (firedrake) simone@simone:~$ cd firedrake/src/firedrake/ (firedrake) simone@simone:~/firedrake/src/firedrake$ make alltest Building extension modules Linting firedrake codebase Linting firedrake test suite Linting firedrake scripts Running all regression tests ============================================================ test session starts ============================================================= platform linux -- Python 3.6.6, pytest-4.4.0, py-1.8.0, pluggy-0.9.0 rootdir: /home/simone/firedrake/src/firedrake, inifile: setup.cfg plugins: xdist-1.27.0, forked-1.0.2 collected 0 items / 1 errors =================================================================== ERRORS =================================================================== _______________________________________________________ ERROR collecting test session ________________________________________________________ ../../lib/python3.6/site-packages/_pytest/config/__init__.py:435: in _importconftest return self._conftestpath2mod[conftestpath] E KeyError: local('/home/simone/firedrake/src/firedrake/tests/extrusion/conftest.py') During handling of the above exception, another exception occurred: ../h5py/h5py/__init__.py:26: in <module> from . import _errors E ImportError: libhdf5.so.101: cannot open shared object file: No such file or directory During handling of the above exception, another exception occurred: ../../lib/python3.6/site-packages/_pytest/config/__init__.py:441: in _importconftest mod = conftestpath.pyimport() ../../lib/python3.6/site-packages/py/_path/local.py:701: in pyimport __import__(modname) ../../lib/python3.6/site-packages/_pytest/assertion/rewrite.py:299: in load_module six.exec_(co, mod.__dict__) tests/extrusion/conftest.py:2: in <module> from firedrake import ExtrudedMesh, UnitSquareMesh, UnitIntervalMesh firedrake/__init__.py:48: in <module> from firedrake.checkpointing import * firedrake/checkpointing.py:3: in <module> from firedrake import hdf5interface as h5i firedrake/hdf5interface.pyx:1: in init firedrake.hdf5interface import h5py ../h5py/h5py/__init__.py:30: in <module> raise ImportError("You cannot import h5py from inside the install directory.\nChange to another directory first.") E ImportError: You cannot import h5py from inside the install directory. E Change to another directory first. During handling of the above exception, another exception occurred: ../../lib/python3.6/site-packages/py/_path/common.py:377: in visit for x in Visitor(fil, rec, ignore, bf, sort).gen(self): ../../lib/python3.6/site-packages/py/_path/common.py:418: in gen dirs = self.optsort([p for p in entries ../../lib/python3.6/site-packages/py/_path/common.py:419: in <listcomp> if p.check(dir=1) and (rec is None or rec(p))]) ../../lib/python3.6/site-packages/_pytest/main.py:660: in _recurse ihook = self.gethookproxy(dirpath) ../../lib/python3.6/site-packages/_pytest/main.py:481: in gethookproxy my_conftestmodules = pm._getconftestmodules(fspath) ../../lib/python3.6/site-packages/_pytest/config/__init__.py:419: in _getconftestmodules mod = self._importconftest(conftestpath.realpath()) ../../lib/python3.6/site-packages/_pytest/config/__init__.py:458: in _importconftest raise ConftestImportFailure(conftestpath, sys.exc_info()) E _pytest.config.ConftestImportFailure: (local('/home/simone/firedrake/src/firedrake/tests/extrusion/conftest.py'), (<class 'ImportError'>, ImportError('You cannot import h5py from inside the install directory.\nChange to another directory first.',), <traceback object at 0x7fe2527a1848>)) !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ========================================================== 1 error in 4.85 seconds =========================================================== Makefile:64: recipe for target 'test' failed make: *** [test] Error 2 -- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
Dear Simone,
On 17 Jun 2019, at 17:46, Simone Puel <spuel@utexas.edu> wrote:
Hi,
My name is Simone Puel, a graduate student at the University of Texas at Austin. I would like to use Firedrake for my research, but once installed I run the test suggested to check that the Firedrake installation is fully functional, but I got the following error. I am using Ubuntu-Mate distribution with ubuntu bionic 18.04.2 LTS with Python 3.6.8 in Anaconda environment. Could you help me in the installation, please? Thank you so much in advance. Have a wonderful day.
This error suggests that somehow there were some problems in the installation. Can you send the contents of firedrake-install.log? We can also check if importing h5py, which is the issue here, on its own works. After activating the virtual environment, can you send the output of python -c "import h5py" ? Thanks, Lawrence
Dear Lawrence, Thank you for your email. Yes, of course. Attached there is the "firedrake-install.log" while the output: (firedrake) simone@simone:~$ python -c "import h5py" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/simone/firedrake/src/h5py/h5py/__init__.py", line 36, in <module> from ._conv import register_converters as _register_converters File "h5py/h5r.pxd", line 21, in init h5py._conv File "h5py/_objects.pxd", line 12, in init h5py.h5r File "h5py/_objects.pyx", line 1, in init h5py._objects ImportError: /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so: undefined symbol: H5Pget_dxpl_mpio Thank you so much, have a wonderful day. Best regards, Simone On Tue, Jun 18, 2019 at 11:14 AM Lawrence Mitchell <wencel@gmail.com> wrote:
Dear Simone,
On 17 Jun 2019, at 17:46, Simone Puel <spuel@utexas.edu> wrote:
Hi,
My name is Simone Puel, a graduate student at the University of Texas at Austin. I would like to use Firedrake for my research, but once installed I run the test suggested to check that the Firedrake installation is fully functional, but I got the following error. I am using Ubuntu-Mate distribution with ubuntu bionic 18.04.2 LTS with Python 3.6.8 in Anaconda environment. Could you help me in the installation, please? Thank you so much in advance. Have a wonderful day.
This error suggests that somehow there were some problems in the installation. Can you send the contents of firedrake-install.log?
We can also check if importing h5py, which is the issue here, on its own works.
After activating the virtual environment, can you send the output of
python -c "import h5py"
?
Thanks,
Lawrence
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
On Tue, Jun 18, 2019 at 1:25 PM Simone Puel <spuel@utexas.edu> wrote:
Dear Lawrence,
Thank you for your email. Yes, of course. Attached there is the "firedrake-install.log" while the output:
(firedrake) simone@simone:~$ python -c "import h5py" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/simone/firedrake/src/h5py/h5py/__init__.py", line 36, in <module> from ._conv import register_converters as _register_converters File "h5py/h5r.pxd", line 21, in init h5py._conv File "h5py/_objects.pxd", line 12, in init h5py.h5r File "h5py/_objects.pyx", line 1, in init h5py._objects ImportError: /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so: undefined symbol: H5Pget_dxpl_mpio
Hi Simone, I think this can happen if you have 2 copies of HDF5 floating around, where one is compiled with MPI turned on and one isn't. Thanks, Matt
Thank you so much, have a wonderful day.
Best regards, Simone
On Tue, Jun 18, 2019 at 11:14 AM Lawrence Mitchell <wencel@gmail.com> wrote:
Dear Simone,
On 17 Jun 2019, at 17:46, Simone Puel <spuel@utexas.edu> wrote:
Hi,
My name is Simone Puel, a graduate student at the University of Texas at Austin. I would like to use Firedrake for my research, but once installed I run the test suggested to check that the Firedrake installation is fully functional, but I got the following error. I am using Ubuntu-Mate distribution with ubuntu bionic 18.04.2 LTS with Python 3.6.8 in Anaconda environment. Could you help me in the installation, please? Thank you so much in advance. Have a wonderful day.
This error suggests that somehow there were some problems in the installation. Can you send the contents of firedrake-install.log?
We can also check if importing h5py, which is the issue here, on its own works.
After activating the virtual environment, can you send the output of
python -c "import h5py"
?
Thanks,
Lawrence
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811 _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On Wed, 19 Jun 2019 at 08:44, Matthew Knepley <knepley@gmail.com> wrote:
On Tue, Jun 18, 2019 at 1:25 PM Simone Puel <spuel@utexas.edu> wrote:
Dear Lawrence,
Thank you for your email. Yes, of course. Attached there is the "firedrake-install.log" while the output:
(firedrake) simone@simone:~$ python -c "import h5py" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/simone/firedrake/src/h5py/h5py/__init__.py", line 36, in <module> from ._conv import register_converters as _register_converters File "h5py/h5r.pxd", line 21, in init h5py._conv File "h5py/_objects.pxd", line 12, in init h5py.h5r File "h5py/_objects.pyx", line 1, in init h5py._objects ImportError: /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so: undefined symbol: H5Pget_dxpl_mpio
Hi Simone,
I think this can happen if you have 2 copies of HDF5 floating around, where one is compiled with MPI turned on and one isn't.
Ah. I wonder if you have a system install. We ask petsc to build hdf5 for us and then try and link against that. Can you provide the output of ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so Thanks, Lawrence
Hi Lawrence and Matt, Thank you so much for your email. Thanks Matt, what can I do? I was wondering that the problem might be related to have installed most of the packages with both anaconda and directly on my computer. This is the output: simone@simone:~$ ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so linux-vdso.so.1 (0x00007fffd29f4000) libhdf5.so.103 => /home/simone/anaconda3/lib/libhdf5.so.103 (0x00007f7380c31000) libhdf5_hl.so.100 => /home/simone/anaconda3/lib/libhdf5_hl.so.100 (0x00007f738118c000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7380a12000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7380621000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f7380419000) libz.so.1 => /home/simone/anaconda3/lib/./libz.so.1 (0x00007f7380202000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f737fffe000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f737fc60000) /lib64/ld-linux-x86-64.so.2 (0x00007f7380fc7000) Thanks, have a wonderful day. Best regards, Simone On Wed, Jun 19, 2019 at 2:53 AM Lawrence Mitchell <wence@gmx.li> wrote:
On Wed, 19 Jun 2019 at 08:44, Matthew Knepley <knepley@gmail.com> wrote:
On Tue, Jun 18, 2019 at 1:25 PM Simone Puel <spuel@utexas.edu> wrote:
Dear Lawrence,
Thank you for your email. Yes, of course. Attached there is the "firedrake-install.log" while the output:
(firedrake) simone@simone:~$ python -c "import h5py" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/simone/firedrake/src/h5py/h5py/__init__.py", line 36, in <module> from ._conv import register_converters as _register_converters File "h5py/h5r.pxd", line 21, in init h5py._conv File "h5py/_objects.pxd", line 12, in init h5py.h5r File "h5py/_objects.pyx", line 1, in init h5py._objects ImportError: /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so: undefined symbol: H5Pget_dxpl_mpio
Hi Simone,
I think this can happen if you have 2 copies of HDF5 floating around, where one is compiled with MPI turned on and one isn't.
Ah. I wonder if you have a system install. We ask petsc to build hdf5 for us and then try and link against that.
Can you provide the output of
ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so
Thanks,
Lawrence
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
On Wed, Jun 19, 2019 at 9:50 AM Simone Puel <spuel@utexas.edu> wrote:
Hi Lawrence and Matt,
Thank you so much for your email. Thanks Matt, what can I do? I was wondering that the problem might be related to have installed most of the packages with both anaconda and directly on my computer. This is the output:
I think the easiest thing to do is not let Firedrake see the Anaconda libraries. I think this should be easy by just removing those directories from the environment variables. Does this make sense? Thanks, Matt
simone@simone:~$ ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so linux-vdso.so.1 (0x00007fffd29f4000) libhdf5.so.103 => /home/simone/anaconda3/lib/libhdf5.so.103 (0x00007f7380c31000) libhdf5_hl.so.100 => /home/simone/anaconda3/lib/libhdf5_hl.so.100 (0x00007f738118c000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7380a12000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7380621000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f7380419000) libz.so.1 => /home/simone/anaconda3/lib/./libz.so.1 (0x00007f7380202000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f737fffe000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f737fc60000) /lib64/ld-linux-x86-64.so.2 (0x00007f7380fc7000)
Thanks, have a wonderful day.
Best regards, Simone
On Wed, Jun 19, 2019 at 2:53 AM Lawrence Mitchell <wence@gmx.li> wrote:
On Wed, 19 Jun 2019 at 08:44, Matthew Knepley <knepley@gmail.com> wrote:
On Tue, Jun 18, 2019 at 1:25 PM Simone Puel <spuel@utexas.edu> wrote:
Dear Lawrence,
Thank you for your email. Yes, of course. Attached there is the "firedrake-install.log" while the output:
(firedrake) simone@simone:~$ python -c "import h5py" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/simone/firedrake/src/h5py/h5py/__init__.py", line 36, in <module> from ._conv import register_converters as _register_converters File "h5py/h5r.pxd", line 21, in init h5py._conv File "h5py/_objects.pxd", line 12, in init h5py.h5r File "h5py/_objects.pyx", line 1, in init h5py._objects ImportError: /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so: undefined symbol: H5Pget_dxpl_mpio
Hi Simone,
I think this can happen if you have 2 copies of HDF5 floating around, where one is compiled with MPI turned on and one isn't.
Ah. I wonder if you have a system install. We ask petsc to build hdf5 for us and then try and link against that.
Can you provide the output of
ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so
Thanks,
Lawrence
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
Hi Matt, Thank you so much for your email. Do you mean remove the entire Anaconda, or removing some directories in the firedrake folder? Thanks in advance. Have a wonderful day. Best regards, Simone On Thu, Jun 20, 2019 at 2:40 AM Matthew Knepley <knepley@gmail.com> wrote:
On Wed, Jun 19, 2019 at 9:50 AM Simone Puel <spuel@utexas.edu> wrote:
Hi Lawrence and Matt,
Thank you so much for your email. Thanks Matt, what can I do? I was wondering that the problem might be related to have installed most of the packages with both anaconda and directly on my computer. This is the output:
I think the easiest thing to do is not let Firedrake see the Anaconda libraries. I think this should be easy by just removing those directories from the environment variables. Does this make sense?
Thanks,
Matt
simone@simone:~$ ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so linux-vdso.so.1 (0x00007fffd29f4000) libhdf5.so.103 => /home/simone/anaconda3/lib/libhdf5.so.103 (0x00007f7380c31000) libhdf5_hl.so.100 => /home/simone/anaconda3/lib/libhdf5_hl.so.100 (0x00007f738118c000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7380a12000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7380621000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f7380419000) libz.so.1 => /home/simone/anaconda3/lib/./libz.so.1 (0x00007f7380202000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f737fffe000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f737fc60000) /lib64/ld-linux-x86-64.so.2 (0x00007f7380fc7000)
Thanks, have a wonderful day.
Best regards, Simone
On Wed, Jun 19, 2019 at 2:53 AM Lawrence Mitchell <wence@gmx.li> wrote:
On Wed, 19 Jun 2019 at 08:44, Matthew Knepley <knepley@gmail.com> wrote:
On Tue, Jun 18, 2019 at 1:25 PM Simone Puel <spuel@utexas.edu> wrote:
Dear Lawrence,
Thank you for your email. Yes, of course. Attached there is the "firedrake-install.log" while the output:
(firedrake) simone@simone:~$ python -c "import h5py" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/simone/firedrake/src/h5py/h5py/__init__.py", line 36, in <module> from ._conv import register_converters as _register_converters File "h5py/h5r.pxd", line 21, in init h5py._conv File "h5py/_objects.pxd", line 12, in init h5py.h5r File "h5py/_objects.pyx", line 1, in init h5py._objects ImportError: /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so: undefined symbol: H5Pget_dxpl_mpio
Hi Simone,
I think this can happen if you have 2 copies of HDF5 floating around, where one is compiled with MPI turned on and one isn't.
Ah. I wonder if you have a system install. We ask petsc to build hdf5 for us and then try and link against that.
Can you provide the output of
ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so
Thanks,
Lawrence
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
On 20 Jun 2019, at 14:24, Simone Puel <spuel@utexas.edu> wrote:
Hi Matt,
Thank you so much for your email. Do you mean remove the entire Anaconda, or removing some directories in the firedrake folder? Thanks in advance. Have a wonderful day.
Your anaconda environment has installed a version of HDF5 in: /home/simone/anaconda3/lib/libhdf5.so.103 When we build h5py, we attempt to link it agains the (parallel) version of HDF5 built by PETSc. Looking back at your firedrake-install.log I see: gcc -pthread -shared -B /home/simone/anaconda3/compiler_compat -L/home/simone/anaconda3/lib -Wl,-rpath=/home/simone/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/home/simone/firedrake/src/h5py/h5py/h5ac.o -L/home/simone/firedrake/lib/python3.6/site-packages/petsc/lib -Wl,-R/home/simone/firedrake/lib/python3.6/site-packages/petsc/lib -lhdf5 -lhdf5_hl -o /home/simone/firedrake/src/h5py/h5py/h5ac.cpython-36m-x86_64-linux-gnu.so We (firedrake-install) arranged for this bit: -L/home/simone/firedrake/lib/python3.6/site-packages/petsc/lib -Wl,-R/home/simone/firedrake/lib/python3.6/site-packages/petsc/lib Which is the "right" place. But anaconda (I guess) has added. -L/home/simone/anaconda3/lib -Wl,-rpath=/home/simone/anaconda3/lib Which is what is being used to resolve libhdf5.so I don't know enough about anaconda to know why it adds these extra link lines. Perhaps you should start in a new conda "environment" that doesn't have hdf5 installed? Thanks, Lawrence
Best regards, Simone
On Thu, Jun 20, 2019 at 2:40 AM Matthew Knepley <knepley@gmail.com> wrote: On Wed, Jun 19, 2019 at 9:50 AM Simone Puel <spuel@utexas.edu> wrote: Hi Lawrence and Matt,
Thank you so much for your email. Thanks Matt, what can I do? I was wondering that the problem might be related to have installed most of the packages with both anaconda and directly on my computer. This is the output:
I think the easiest thing to do is not let Firedrake see the Anaconda libraries. I think this should be easy by just removing those directories from the environment variables. Does this make sense?
On Thu, Jun 20, 2019 at 9:24 AM Simone Puel <spuel@utexas.edu> wrote:
Hi Matt,
Thank you so much for your email. Do you mean remove the entire Anaconda, or removing some directories in the firedrake folder? Thanks in advance. Have a wonderful day.
I mean that you do not want Firedrake to _see_ Anaconda. I believe you can do this by taking any Anaconda directory out of your environment. Something like env | grep conda and taking it out of all those. Thanks, Matt
Best regards, Simone
On Thu, Jun 20, 2019 at 2:40 AM Matthew Knepley <knepley@gmail.com> wrote:
On Wed, Jun 19, 2019 at 9:50 AM Simone Puel <spuel@utexas.edu> wrote:
Hi Lawrence and Matt,
Thank you so much for your email. Thanks Matt, what can I do? I was wondering that the problem might be related to have installed most of the packages with both anaconda and directly on my computer. This is the output:
I think the easiest thing to do is not let Firedrake see the Anaconda libraries. I think this should be easy by just removing those directories from the environment variables. Does this make sense?
Thanks,
Matt
simone@simone:~$ ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so linux-vdso.so.1 (0x00007fffd29f4000) libhdf5.so.103 => /home/simone/anaconda3/lib/libhdf5.so.103 (0x00007f7380c31000) libhdf5_hl.so.100 => /home/simone/anaconda3/lib/libhdf5_hl.so.100 (0x00007f738118c000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7380a12000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7380621000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f7380419000) libz.so.1 => /home/simone/anaconda3/lib/./libz.so.1 (0x00007f7380202000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f737fffe000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f737fc60000) /lib64/ld-linux-x86-64.so.2 (0x00007f7380fc7000)
Thanks, have a wonderful day.
Best regards, Simone
On Wed, Jun 19, 2019 at 2:53 AM Lawrence Mitchell <wence@gmx.li> wrote:
On Wed, 19 Jun 2019 at 08:44, Matthew Knepley <knepley@gmail.com> wrote:
On Tue, Jun 18, 2019 at 1:25 PM Simone Puel <spuel@utexas.edu> wrote:
Dear Lawrence,
Thank you for your email. Yes, of course. Attached there is the "firedrake-install.log" while the output:
(firedrake) simone@simone:~$ python -c "import h5py" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/simone/firedrake/src/h5py/h5py/__init__.py", line 36, in <module> from ._conv import register_converters as _register_converters File "h5py/h5r.pxd", line 21, in init h5py._conv File "h5py/_objects.pxd", line 12, in init h5py.h5r File "h5py/_objects.pyx", line 1, in init h5py._objects ImportError: /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so: undefined symbol: H5Pget_dxpl_mpio
Hi Simone,
I think this can happen if you have 2 copies of HDF5 floating around, where one is compiled with MPI turned on and one isn't.
Ah. I wonder if you have a system install. We ask petsc to build hdf5 for us and then try and link against that.
Can you provide the output of
ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so
Thanks,
Lawrence
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
Hi Simone, For what it's worth, anaconda usually adds its activation commands in your ~/.bashrc file, should be something like: # >>> conda initialize >>> [... some commands here...] # <<< conda initialize <<< If you comment out those lines (and open a new shell), you should be able to install and run firedrake without conflicts. (To check that anaconda has indeed been disabled, the command 'conda' should return 'command not found' error). If you copy those commands into another file, for example, 'activate_anaconda.sh', you can then activate anaconda by saying . path/to/activate_anaconda.sh whenever you need it. Cheers, Tuomas On 6/20/19 4:52 PM, Matthew Knepley wrote:
On Thu, Jun 20, 2019 at 9:24 AM Simone Puel <spuel@utexas.edu <mailto:spuel@utexas.edu>> wrote:
Hi Matt,
Thank you so much for your email. Do you mean remove the entire Anaconda, or removing some directories in the firedrake folder? Thanks in advance. Have a wonderful day.
I mean that you do not want Firedrake to _see_ Anaconda. I believe you can do this by taking any Anaconda directory out of your environment. Something like
env | grep conda
and taking it out of all those.
Thanks,
Matt
Best regards, Simone
On Thu, Jun 20, 2019 at 2:40 AM Matthew Knepley <knepley@gmail.com <mailto:knepley@gmail.com>> wrote:
On Wed, Jun 19, 2019 at 9:50 AM Simone Puel <spuel@utexas.edu <mailto:spuel@utexas.edu>> wrote:
Hi Lawrence and Matt,
Thank you so much for your email. Thanks Matt, what can I do? I was wondering that the problem might be related to have installed most of the packages with both anaconda and directly on my computer. This is the output:
I think the easiest thing to do is not let Firedrake see the Anaconda libraries. I think this should be easy by just removing those directories from the environment variables. Does this make sense?
Thanks,
Matt
simone@simone:~$ ldd /home/simone/firedrake/src/h5py/h5py/defs.cpython-36m-x86_64-linux-gnu.so <http://defs.cpython-36m-x86_64-linux-gnu.so> linux-vdso.so.1 (0x00007fffd29f4000) libhdf5.so.103 => /home/simone/anaconda3/lib/libhdf5.so.103 (0x00007f7380c31000) libhdf5_hl.so.100 => /home/simone/anaconda3/lib/libhdf5_hl.so.100 (0x00007f738118c000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7380a12000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7380621000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f7380419000) libz.so.1 => /home/simone/anaconda3/lib/./libz.so.1 (0x00007f7380202000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f737fffe000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f737fc60000) /lib64/ld-linux-x86-64.so.2 (0x00007f7380fc7000)
Thanks, have a wonderful day.
Best regards, Simone
On Wed, Jun 19, 2019 at 2:53 AM Lawrence Mitchell <wence@gmx.li <mailto:wence@gmx.li>> wrote:
On Wed, 19 Jun 2019 at 08:44, Matthew Knepley <knepley@gmail.com <mailto:knepley@gmail.com>> wrote:
On Tue, Jun 18, 2019 at 1:25 PM Simone Puel <spuel@utexas.edu <mailto:spuel@utexas.edu>> wrote:
Dear Lawrence,
Thank you for your email. Yes, of course. Attached there is the "firedrake-install.log" while the output:
(firedrake) simone@simone:~$ python -c "import h5py" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/simone/firedrake/src/h5py/h5py/__init__.py", line 36, in <module> from ._conv import register_converters as _register_converters File "h5py/h5r.pxd", line 21, in init h5py._conv File "h5py/_objects.pxd", line 12, in init h5py.h5r File "h5py/_objects.pyx", line 1, in init h5py._objects ImportError: /home/simone/firedrake/src/h5py/h5py/defs.cpython-36m-x86_64-linux-gnu.so <http://defs.cpython-36m-x86_64-linux-gnu.so>: undefined symbol: H5Pget_dxpl_mpio
Hi Simone,
I think this can happen if you have 2 copies of HDF5 floating around, where one is compiled with MPI turned on and one isn't.
Ah. I wonder if you have a system install. We ask petsc to build hdf5 for us and then try and link against that.
Can you provide the output of
ldd /home/simone/firedrake/src/h5py/h5py/defs.cpython-36m-x86_64-linux-gnu.so <http://defs.cpython-36m-x86_64-linux-gnu.so>
Thanks,
Lawrence
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu <mailto:spuel@utexas.edu> +1 (737)-230-6811
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu <mailto:spuel@utexas.edu> +1 (737)-230-6811
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi Tuomas, Matt and Lawrence, Thank you so much for your emails. Yes, I did so (it was the anaconda in my .baschrc file). I can run the tests but some of them are not successful. Attached there is all the summary of the tests. Would you know why I have still have these issues. Thank you so much in advance, I really appreciate you suggestion and help. Have a wonderful day. Best regards, Simone On Thu, Jun 20, 2019 at 2:34 PM Tuomas Karna <tuomas.karna@gmail.com> wrote:
Hi Simone,
For what it's worth, anaconda usually adds its activation commands in your ~/.bashrc file, should be something like:
# >>> conda initialize >>> [... some commands here...] # <<< conda initialize <<<
If you comment out those lines (and open a new shell), you should be able to install and run firedrake without conflicts. (To check that anaconda has indeed been disabled, the command 'conda' should return 'command not found' error).
If you copy those commands into another file, for example, 'activate_anaconda.sh', you can then activate anaconda by saying . path/to/activate_anaconda.sh whenever you need it.
Cheers, Tuomas
On 6/20/19 4:52 PM, Matthew Knepley wrote:
On Thu, Jun 20, 2019 at 9:24 AM Simone Puel <spuel@utexas.edu> wrote:
Hi Matt,
Thank you so much for your email. Do you mean remove the entire Anaconda, or removing some directories in the firedrake folder? Thanks in advance. Have a wonderful day.
I mean that you do not want Firedrake to _see_ Anaconda. I believe you can do this by taking any Anaconda directory out of your environment. Something like
env | grep conda
and taking it out of all those.
Thanks,
Matt
Best regards, Simone
On Thu, Jun 20, 2019 at 2:40 AM Matthew Knepley <knepley@gmail.com> wrote:
On Wed, Jun 19, 2019 at 9:50 AM Simone Puel <spuel@utexas.edu> wrote:
Hi Lawrence and Matt,
Thank you so much for your email. Thanks Matt, what can I do? I was wondering that the problem might be related to have installed most of the packages with both anaconda and directly on my computer. This is the output:
I think the easiest thing to do is not let Firedrake see the Anaconda libraries. I think this should be easy by just removing those directories from the environment variables. Does this make sense?
Thanks,
Matt
simone@simone:~$ ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so linux-vdso.so.1 (0x00007fffd29f4000) libhdf5.so.103 => /home/simone/anaconda3/lib/libhdf5.so.103 (0x00007f7380c31000) libhdf5_hl.so.100 => /home/simone/anaconda3/lib/libhdf5_hl.so.100 (0x00007f738118c000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7380a12000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7380621000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f7380419000) libz.so.1 => /home/simone/anaconda3/lib/./libz.so.1 (0x00007f7380202000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f737fffe000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f737fc60000) /lib64/ld-linux-x86-64.so.2 (0x00007f7380fc7000)
Thanks, have a wonderful day.
Best regards, Simone
On Wed, Jun 19, 2019 at 2:53 AM Lawrence Mitchell <wence@gmx.li> wrote:
On Wed, 19 Jun 2019 at 08:44, Matthew Knepley <knepley@gmail.com> wrote:
On Tue, Jun 18, 2019 at 1:25 PM Simone Puel <spuel@utexas.edu> wrote:
> Dear Lawrence, > > Thank you for your email. Yes, of course. > Attached there is the "firedrake-install.log" while the output: > > (firedrake) simone@simone:~$ python -c "import h5py" > Traceback (most recent call last): > File "<string>", line 1, in <module> > File "/home/simone/firedrake/src/h5py/h5py/__init__.py", line 36, > in <module> > from ._conv import register_converters as _register_converters > File "h5py/h5r.pxd", line 21, in init h5py._conv > File "h5py/_objects.pxd", line 12, in init h5py.h5r > File "h5py/_objects.pyx", line 1, in init h5py._objects > ImportError: /home/simone/firedrake/src/h5py/h5py/ > defs.cpython-36m-x86_64-linux-gnu.so: undefined symbol: > H5Pget_dxpl_mpio >
Hi Simone,
I think this can happen if you have 2 copies of HDF5 floating around, where one is compiled with MPI turned on and one isn't.
Ah. I wonder if you have a system install. We ask petsc to build hdf5 for us and then try and link against that.
Can you provide the output of
ldd /home/simone/firedrake/src/h5py/h5py/ defs.cpython-36m-x86_64-linux-gnu.so
Thanks,
Lawrence
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
_______________________________________________ firedrake mailing listfiredrake@imperial.ac.ukhttps://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
Hi Simone,
On 21 Jun 2019, at 04:55, Simone Puel <spuel@utexas.edu> wrote:
Hi Tuomas, Matt and Lawrence,
Thank you so much for your emails. Yes, I did so (it was the anaconda in my .baschrc file). I can run the tests but some of them are not successful. Attached there is all the summary of the tests. Would you know why I have still have these issues. Thank you so much in advance, I really appreciate you suggestion and help. Have a wonderful day.
Looking at some of those errors and staring at my crystal ball, I wonder if something is bad with the MPI. Let's try some tests: After activating the virtual environment, can you show the output of: mpiexec -n 4 python -c "from mpi4py import MPI; print(MPI.COMM_WORLD.rank)" mpiexec -n 4 python -c "from petsc4py import PETSc; print(PETSc.COMM_WORLD.rank)" mpiexec -n 4 python -c "from firedrake import *; mesh = UnitSquareMesh(3, 3); mesh.init(); print(mesh.comm.rank)" Thanks, Lawrence
Hi Lawrence, Thank you so much for your help. Here are the output of the commands: (firedrake) simone@simone:~/firedrake/src/firedrake$ mpiexec -n 4 python -c "from mpi4py import MPI; print(MPI.COMM_WORLD.rank)" 1 3 2 0 (firedrake) simone@simone:~/firedrake/src/firedrake$ mpiexec -n 4 python -c "from petsc4py import PETSc; print(PETSc.COMM_WORLD.rank)" 0 3 1 2 (firedrake) simone@simone:~/firedrake/src/firedrake$ mpiexec -n 4 python -c "from firedrake import *; mesh = UnitSquareMesh(3, 3); mesh.init(); print(mesh.comm.rank)" 0 1 2 3 Thank you so much, have a wonderful day. Best regards, Simone On Fri, Jun 21, 2019 at 8:13 AM Lawrence Mitchell <wence@gmx.li> wrote:
Hi Simone,
On 21 Jun 2019, at 04:55, Simone Puel <spuel@utexas.edu> wrote:
Hi Tuomas, Matt and Lawrence,
Thank you so much for your emails. Yes, I did so (it was the anaconda in my .baschrc file). I can run the tests but some of them are not successful. Attached there is all the summary of the tests. Would you know why I have still have these issues. Thank you so much in advance, I really appreciate you suggestion and help. Have a wonderful day.
Looking at some of those errors and staring at my crystal ball, I wonder if something is bad with the MPI. Let's try some tests:
After activating the virtual environment, can you show the output of:
mpiexec -n 4 python -c "from mpi4py import MPI; print(MPI.COMM_WORLD.rank)"
mpiexec -n 4 python -c "from petsc4py import PETSc; print(PETSc.COMM_WORLD.rank)"
mpiexec -n 4 python -c "from firedrake import *; mesh = UnitSquareMesh(3, 3); mesh.init(); print(mesh.comm.rank)"
Thanks,
Lawrence
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
On 21 Jun 2019, at 14:40, Simone Puel <spuel@utexas.edu> wrote:
Hi Lawrence,
Thank you so much for your help. Here are the output of the commands:
(firedrake) simone@simone:~/firedrake/src/firedrake$ mpiexec -n 4 python -c "from mpi4py import MPI; print(MPI.COMM_WORLD.rank)" 1 3 2 0 (firedrake) simone@simone:~/firedrake/src/firedrake$ mpiexec -n 4 python -c "from petsc4py import PETSc; print(PETSc.COMM_WORLD.rank)" 0 3 1 2 (firedrake) simone@simone:~/firedrake/src/firedrake$ mpiexec -n 4 python -c "from firedrake import *; mesh = UnitSquareMesh(3, 3); mesh.init(); print(mesh.comm.rank)" 0 1 2 3
OK, so some minimal things work fine. How about: pytest -s -m parallel tests/regression/test_identity.py and pytest -s -m 'not parallel' tests/regression/test_identity.py Thanks, Lawrence
Hi Lawrence, It seems to work fine as well. Here is the output: (firedrake) simone@simone:~/firedrake/src/firedrake$ pytest -s -m parallel tests/regression/test_identity.py =========================================================== test session starts ============================================================ platform linux -- Python 3.6.8, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 rootdir: /home/simone/firedrake/src/firedrake, inifile: setup.cfg plugins: xdist-1.29.0, forked-1.0.2 collected 8 items / 4 deselected / 4 selected tests/regression/test_identity.py ... [100%] [100%] 1 passed in 27.72 seconds 1 passed in 27.72 seconds1 passed in 27.72 seconds ... [100%] 1 passed in 11.00 seconds 1 passed in 11.00 seconds ... [100%] 1 passed in 6.08 seconds 1 passed in 6.08 seconds ... [100%] 1 passed in 6.14 seconds1 passed in 6.14 seconds . ================================================= 4 passed, 4 deselected in 59.03 seconds ================================================== (firedrake) simone@simone:~/firedrake/src/firedrake$ pytest -s -m 'not parallel' tests/regression/test_identity.py =========================================================== test session starts ============================================================ platform linux -- Python 3.6.8, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 rootdir: /home/simone/firedrake/src/firedrake, inifile: setup.cfg plugins: xdist-1.29.0, forked-1.0.2 collected 8 items / 4 deselected / 4 selected tests/regression/test_identity.py .... ================================================= 4 passed, 4 deselected in 16.31 seconds ================================================== Thank you so much, have a wonderful day. Best regards, Simone On Fri, Jun 21, 2019 at 9:16 AM Lawrence Mitchell <wence@gmx.li> wrote:
On 21 Jun 2019, at 14:40, Simone Puel <spuel@utexas.edu> wrote:
Hi Lawrence,
Thank you so much for your help. Here are the output of the commands:
(firedrake) simone@simone:~/firedrake/src/firedrake$ mpiexec -n 4 python -c "from mpi4py import MPI; print(MPI.COMM_WORLD.rank)" 1 3 2 0 (firedrake) simone@simone:~/firedrake/src/firedrake$ mpiexec -n 4 python -c "from petsc4py import PETSc; print(PETSc.COMM_WORLD.rank)" 0 3 1 2 (firedrake) simone@simone:~/firedrake/src/firedrake$ mpiexec -n 4 python -c "from firedrake import *; mesh = UnitSquareMesh(3, 3); mesh.init(); print(mesh.comm.rank)" 0 1 2 3
OK, so some minimal things work fine.
How about:
pytest -s -m parallel tests/regression/test_identity.py
and
pytest -s -m 'not parallel' tests/regression/test_identity.py
Thanks,
Lawrence
-- *Simone Puel* Ph.D. student in Geological Sciences at the University of Texas at Austin Jackson School of Geosciences & Institute for Geophysics (UTIG) JGB 5.332 & ROC 2.116L spuel@utexas.edu +1 (737)-230-6811
participants (5)
-
Lawrence Mitchell
-
Lawrence Mitchell
-
Matthew Knepley
-
Simone Puel
-
Tuomas Karna