firedrake's open-mpi dependency
Hi, all I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. I already have impi installed on my machine. What should I do to make this change happen ? Thanks, Tao
Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.
On 10 Feb 2016, at 22:28, Tao Wang <twang15@ncsu.edu> wrote:
Hi, all
I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. I already have impi installed on my machine.
What should I do to make this change happen ?
Thanks, Tao _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
I failed to install it on CentOS with --no-package-manager option Please see this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> for more details. Could you answer my questions one by one ? Thanks a lot for your help. On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya <m.homolya14@imperial.ac.uk> wrote:
Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.
On 10 Feb 2016, at 22:28, Tao Wang <twang15@ncsu.edu> wrote:
Hi, all
I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. I already have impi installed on my machine.
What should I do to make this change happen ?
Thanks, Tao _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Since firedrake depends on mpi4py, which depends on open-mpi or mpich. But the server I used does not have them, only having intel mpi. The following is a test. python -c "from mpi4py import MPI" ====*Error msg *==== Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so: undefined symbol: ompi_mpi_char Then, I run nm /opt/apps/intel15/impi/5.0.2.044/intel64/lib/libmpi.so.12 <http://5.0.2.36/intel64/lib/libmpi.so.12> | grep ompi_mpi_char Nothing is returned, it seems there is no such symbol in Intel MPI implementation. Does this suggest that firedrake cannot work with intel mpi implementation ? On Wed, Feb 10, 2016 at 8:43 PM, Tao Wang <twang15@ncsu.edu> wrote:
I failed to install it on CentOS with --no-package-manager option
Please see this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> for more details. Could you answer my questions one by one ?
Thanks a lot for your help.
On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya < m.homolya14@imperial.ac.uk> wrote:
Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.
On 10 Feb 2016, at 22:28, Tao Wang <twang15@ncsu.edu> wrote:
Hi, all
I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. I already have impi installed on my machine.
What should I do to make this change happen ?
Thanks, Tao _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi Tao, May I ask which machine you are using? Stampede for instance has mvapich2 available which has mpi4py, and works with PETSc/firedrake. - Tuomas On 02/10/2016 06:26 PM, Tao Wang wrote:
Since firedrake depends on mpi4py, which depends on open-mpi or mpich. But the server I used does not have them, only having intel mpi. The following is a test.
python -c "from mpi4py import MPI"
====*Error msg *==== Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so: undefined symbol: ompi_mpi_char
Then, I run nm /opt/apps/intel15/impi/5.0.2.044/intel64/lib/libmpi.so.12 <http://5.0.2.36/intel64/lib/libmpi.so.12> | grep ompi_mpi_char
Nothing is returned, it seems there is no such symbol in Intel MPI implementation.
Does this suggest that firedrake cannot work with intel mpi implementation ?
On Wed, Feb 10, 2016 at 8:43 PM, Tao Wang <twang15@ncsu.edu <mailto:twang15@ncsu.edu>> wrote:
I failed to install it on CentOS with --no-package-manager option
Please see this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> for more details. Could you answer my questions one by one ?
Thanks a lot for your help.
On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya <m.homolya14@imperial.ac.uk <mailto:m.homolya14@imperial.ac.uk>> wrote:
Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.
> On 10 Feb 2016, at 22:28, Tao Wang <twang15@ncsu.edu <mailto:twang15@ncsu.edu>> wrote: > > Hi, all > > I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. > I already have impi installed on my machine. > > What should I do to make this change happen ? > > Thanks, > Tao > _______________________________________________ > firedrake mailing list > firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> > https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi, Tuomas It is Stampede. (firedrake) c559-703.stampede(20)$ ldd /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so linux-vdso.so.1 => (0x00007fff6c47f000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaf89bd3000) libpython2.7.so.1.0 => /home1/03915/taoncsu/tools/python/lib/libpython2.7.so.1.0 (0x00002aaf89dd7000) *libmpi.so.12 => /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12* (0x00002aaf8a1b9000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaf8abe6000) libc.so.6 => /lib64/libc.so.6 (0x00002aaf8ae03000) /lib64/ld-linux-x86-64.so.2 (0x0000003267200000) libutil.so.1 => /lib64/libutil.so.1 (0x00002aaf8b197000) libm.so.6 => /lib64/libm.so.6 (0x00002aaf8b39b000) libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00002aaf8b61f000) libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00002aaf8b82b000) libibmad.so.5 => /opt/ofed/lib64/libibmad.so.5 (0x00002aaf8bb7e000) librdmacm.so.1 => /opt/ofed/lib64/librdmacm.so.1 (0x00002aaf8bd95000) libibumad.so.3 => /opt/ofed/lib64/libibumad.so.3 (0x00002aaf8bf9d000) libibverbs.so.1 => /opt/ofed/lib64/libibverbs.so.1 (0x00002aaf8c1a4000) librt.so.1 => /lib64/librt.so.1 (0x00002aaf8c3b2000) libifport.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifport.so.5 (0x00002aaf8c5ba000) libifcore.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifcore.so.5 (0x00002aaf8c7e8000) libimf.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libimf.so (0x00002aaf8cb1a000) libsvml.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libsvml.so (0x00002aaf8cfd5000) libintlc.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libintlc.so.5 (0x00002aaf8dea9000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aaf8e104000) libz.so.1 => /work/03915/taoncsu/linuxbrew/lib/libz.so.1 (0x00002aaf8e31b000) Then, I run nm /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12 | grep ompi_mpi_char Nothing found. Thus, I supposed ompi_mpi_char is not in /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12 Any ideas ? On Wed, Feb 10, 2016 at 10:48 PM, Tuomas Karna <tuomas.karna@gmail.com> wrote:
Hi Tao,
May I ask which machine you are using?
Stampede for instance has mvapich2 available which has mpi4py, and works with PETSc/firedrake.
- Tuomas
On 02/10/2016 06:26 PM, Tao Wang wrote:
Since firedrake depends on mpi4py, which depends on open-mpi or mpich. But the server I used does not have them, only having intel mpi. The following is a test.
python -c "from mpi4py import MPI"
====*Error msg *==== Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so: undefined symbol: ompi_mpi_char
Then, I run nm /opt/apps/intel15/impi/5.0.2.044/intel64/lib/libmpi.so.12 <http://5.0.2.36/intel64/lib/libmpi.so.12> | grep ompi_mpi_char
Nothing is returned, it seems there is no such symbol in Intel MPI implementation.
Does this suggest that firedrake cannot work with intel mpi implementation ?
On Wed, Feb 10, 2016 at 8:43 PM, Tao Wang <twang15@ncsu.edu> wrote:
I failed to install it on CentOS with --no-package-manager option
Please see this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> for more details. Could you answer my questions one by one ?
Thanks a lot for your help.
On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya < <m.homolya14@imperial.ac.uk>m.homolya14@imperial.ac.uk> wrote:
Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.
On 10 Feb 2016, at 22:28, Tao Wang < <twang15@ncsu.edu> twang15@ncsu.edu> wrote:
Hi, all
I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. I already have impi installed on my machine.
What should I do to make this change happen ?
Thanks, Tao _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing listfiredrake@imperial.ac.ukhttps://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
The mpi4py was installed by me with "pip install mpi4py" If I uninstall it, then run "which mpi4py" or "pydoc mpi4py", nothing is found. I don't understand *"**Stampede for instance has mvapich2 available which has mpi4py". * What do you mean and how can I enable it ? On Thu, Feb 11, 2016 at 12:12 AM, Tao Wang <twang15@ncsu.edu> wrote:
Hi, Tuomas
It is Stampede.
(firedrake) c559-703.stampede(20)$ ldd /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so linux-vdso.so.1 => (0x00007fff6c47f000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaf89bd3000) libpython2.7.so.1.0 => /home1/03915/taoncsu/tools/python/lib/libpython2.7.so.1.0 (0x00002aaf89dd7000) *libmpi.so.12 => /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12* (0x00002aaf8a1b9000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaf8abe6000) libc.so.6 => /lib64/libc.so.6 (0x00002aaf8ae03000) /lib64/ld-linux-x86-64.so.2 (0x0000003267200000) libutil.so.1 => /lib64/libutil.so.1 (0x00002aaf8b197000) libm.so.6 => /lib64/libm.so.6 (0x00002aaf8b39b000) libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00002aaf8b61f000) libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00002aaf8b82b000) libibmad.so.5 => /opt/ofed/lib64/libibmad.so.5 (0x00002aaf8bb7e000) librdmacm.so.1 => /opt/ofed/lib64/librdmacm.so.1 (0x00002aaf8bd95000) libibumad.so.3 => /opt/ofed/lib64/libibumad.so.3 (0x00002aaf8bf9d000) libibverbs.so.1 => /opt/ofed/lib64/libibverbs.so.1 (0x00002aaf8c1a4000) librt.so.1 => /lib64/librt.so.1 (0x00002aaf8c3b2000) libifport.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifport.so.5 (0x00002aaf8c5ba000) libifcore.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifcore.so.5 (0x00002aaf8c7e8000) libimf.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libimf.so (0x00002aaf8cb1a000) libsvml.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libsvml.so (0x00002aaf8cfd5000) libintlc.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libintlc.so.5 (0x00002aaf8dea9000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aaf8e104000) libz.so.1 => /work/03915/taoncsu/linuxbrew/lib/libz.so.1 (0x00002aaf8e31b000)
Then, I run
nm /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12 | grep ompi_mpi_char
Nothing found. Thus, I supposed ompi_mpi_char is not in /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12
Any ideas ?
On Wed, Feb 10, 2016 at 10:48 PM, Tuomas Karna <tuomas.karna@gmail.com> wrote:
Hi Tao,
May I ask which machine you are using?
Stampede for instance has mvapich2 available which has mpi4py, and works with PETSc/firedrake.
- Tuomas
On 02/10/2016 06:26 PM, Tao Wang wrote:
Since firedrake depends on mpi4py, which depends on open-mpi or mpich. But the server I used does not have them, only having intel mpi. The following is a test.
python -c "from mpi4py import MPI"
====*Error msg *==== Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so: undefined symbol: ompi_mpi_char
Then, I run nm /opt/apps/intel15/impi/5.0.2.044/intel64/lib/libmpi.so.12 <http://5.0.2.36/intel64/lib/libmpi.so.12> | grep ompi_mpi_char
Nothing is returned, it seems there is no such symbol in Intel MPI implementation.
Does this suggest that firedrake cannot work with intel mpi implementation ?
On Wed, Feb 10, 2016 at 8:43 PM, Tao Wang <twang15@ncsu.edu> wrote:
I failed to install it on CentOS with --no-package-manager option
Please see this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> for more details. Could you answer my questions one by one ?
Thanks a lot for your help.
On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya < <m.homolya14@imperial.ac.uk>m.homolya14@imperial.ac.uk> wrote:
Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.
On 10 Feb 2016, at 22:28, Tao Wang < <twang15@ncsu.edu> twang15@ncsu.edu> wrote:
Hi, all
I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. I already have impi installed on my machine.
What should I do to make this change happen ?
Thanks, Tao _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing listfiredrake@imperial.ac.ukhttps://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi Tao, Also on Stampede, I have loaded the following modules: $module list Currently Loaded Modules: 1) xalt/0.6 3) intel/15.0.2 5) phdf5/1.8.16 7) netcdf/4.3.3.1 (m) 2) TACC 4) mvapich2/2.1 6) python/2.7.9 8) pmetis/4.0.2 Python picks up the right mpi4py (virtualenv is not active): $pip list ... mpi4py (1.3.1) ... $pip show mpi4py --- Metadata-Version: 1.1 Name: mpi4py Version: 1.3.1 Summary: MPI for Python Home-page: http://mpi4py.googlecode.com/ Author: Lisandro Dalcin Author-email: dalcinl@gmail.com License: BSD Location: /opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages Requires: $python -c "import mpi4py; print mpi4py.__path__" ['/opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages/mpi4py'] $which python /opt/apps/intel15/python/2.7.9/bin/python Could you try if you can reproduce these? You might be using wrong python for instance. Cheers, Tuomas On 02/10/2016 09:21 PM, Tao Wang wrote:
The mpi4py was installed by me with "pip install mpi4py"
If I uninstall it, then run "which mpi4py" or "pydoc mpi4py", nothing is found. I don't understand *"**Stampede for instance has mvapich2 available which has mpi4py". *
What do you mean and how can I enable it ?
On Thu, Feb 11, 2016 at 12:12 AM, Tao Wang <twang15@ncsu.edu <mailto:twang15@ncsu.edu>> wrote:
Hi, Tuomas
It is Stampede.
(firedrake) c559-703.stampede(20)$ ldd /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so linux-vdso.so.1 => (0x00007fff6c47f000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaf89bd3000) libpython2.7.so.1.0 => /home1/03915/taoncsu/tools/python/lib/libpython2.7.so.1.0 (0x00002aaf89dd7000) *libmpi.so.12 => /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12* (0x00002aaf8a1b9000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaf8abe6000) libc.so.6 => /lib64/libc.so.6 (0x00002aaf8ae03000) /lib64/ld-linux-x86-64.so.2 (0x0000003267200000) libutil.so.1 => /lib64/libutil.so.1 (0x00002aaf8b197000) libm.so.6 => /lib64/libm.so.6 (0x00002aaf8b39b000) libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00002aaf8b61f000) libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00002aaf8b82b000) libibmad.so.5 => /opt/ofed/lib64/libibmad.so.5 (0x00002aaf8bb7e000) librdmacm.so.1 => /opt/ofed/lib64/librdmacm.so.1 (0x00002aaf8bd95000) libibumad.so.3 => /opt/ofed/lib64/libibumad.so.3 (0x00002aaf8bf9d000) libibverbs.so.1 => /opt/ofed/lib64/libibverbs.so.1 (0x00002aaf8c1a4000) librt.so.1 => /lib64/librt.so.1 (0x00002aaf8c3b2000) libifport.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifport.so.5 (0x00002aaf8c5ba000) libifcore.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifcore.so.5 (0x00002aaf8c7e8000) libimf.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libimf.so (0x00002aaf8cb1a000) libsvml.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libsvml.so (0x00002aaf8cfd5000) libintlc.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libintlc.so.5 (0x00002aaf8dea9000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aaf8e104000) libz.so.1 => /work/03915/taoncsu/linuxbrew/lib/libz.so.1 (0x00002aaf8e31b000)
Then, I run
nm /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12 | grep ompi_mpi_char
Nothing found. Thus, I supposed ompi_mpi_char is not in /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12
Any ideas ?
On Wed, Feb 10, 2016 at 10:48 PM, Tuomas Karna <tuomas.karna@gmail.com <mailto:tuomas.karna@gmail.com>> wrote:
Hi Tao,
May I ask which machine you are using?
Stampede for instance has mvapich2 available which has mpi4py, and works with PETSc/firedrake.
- Tuomas
On 02/10/2016 06:26 PM, Tao Wang wrote:
Since firedrake depends on mpi4py, which depends on open-mpi or mpich. But the server I used does not have them, only having intel mpi. The following is a test.
python -c "from mpi4py import MPI"
====*Error msg *==== Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so: undefined symbol: ompi_mpi_char
Then, I run nm /opt/apps/intel15/impi/5.0.2.044/intel64/lib/libmpi.so.12 <http://5.0.2.36/intel64/lib/libmpi.so.12> | grep ompi_mpi_char
Nothing is returned, it seems there is no such symbol in Intel MPI implementation.
Does this suggest that firedrake cannot work with intel mpi implementation ?
On Wed, Feb 10, 2016 at 8:43 PM, Tao Wang <twang15@ncsu.edu <mailto:twang15@ncsu.edu>> wrote:
I failed to install it on CentOS with --no-package-manager option
Please see this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> for more details. Could you answer my questions one by one ?
Thanks a lot for your help.
On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya <m.homolya14@imperial.ac.uk <mailto:m.homolya14@imperial.ac.uk>> wrote:
Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.
> On 10 Feb 2016, at 22:28, Tao Wang <twang15@ncsu.edu <mailto:twang15@ncsu.edu>> wrote: > > Hi, all > > I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. > I already have impi installed on my machine. > > What should I do to make this change happen ? > > Thanks, > Tao > _______________________________________________ > firedrake mailing list > firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> > https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi, Tuomas The following is the output: c559-703.stampede(33)$ module list Currently Loaded Modules: 1) intel/15.0.2 2) mvapich2/2.1 3) xalt/0.6 4) TACC c559-703.stampede(34)$ pip list apipkg (1.4) Cython (0.23.4) decorator (4.0.6) execnet (1.4.1) flake8 (2.5.1) mccabe (0.3.1) *mpi4py (2.0.0)* networkx (1.10) numpy (1.10.4) pep8 (1.7.0) petsc4py (3.6.0) pip (8.0.2) pudb (2015.4.1) py (1.4.31) pyflakes (1.0.0) Pygments (2.1) pytest (2.8.5) pytest-xdist (1.13.1) setuptools (19.4) triangle (2015.12.14) urwid (1.3.1) virtualenv (14.0.1) wheel (0.26.0) c559-703.stampede(35)$ pip show mpi4py --- Metadata-Version: 2.0 Name: mpi4py Version: 2.0.0 Summary: Python bindings for MPI Home-page: https://bitbucket.org/mpi4py/mpi4py/ Author: Lisandro Dalcin Author-email: dalcinl@gmail.com License: BSD Location: */home1/03915/taoncsu/tools/python/lib/python2.7/site-packages* Requires: c559-703.stampede(36)$ $python -c "import mpi4py; print mpi4py.__path__" -bash: -c: command not found c559-703.stampede(37)$ python -c "import mpi4py; print mpi4py.__path__" ['/home1/03915/taoncsu/tools/python/lib/python2.7/site-packages/mpi4py'] c559-703.stampede(38)$ which python *~/tools/python/bin/python* Apparently, the mpy4py and python are not the same to yours. I am curious about what makes these differences. It seems something is wrong with my .bashrc On Thu, Feb 11, 2016 at 12:36 AM, Tuomas Karna <tuomas.karna@gmail.com> wrote:
Hi Tao,
Also on Stampede, I have loaded the following modules: $module list
Currently Loaded Modules: 1) xalt/0.6 3) intel/15.0.2 5) phdf5/1.8.16 7) netcdf/4.3.3.1 (m) 2) TACC 4) mvapich2/2.1 6) python/2.7.9 8) pmetis/4.0.2
Python picks up the right mpi4py (virtualenv is not active): $pip list ... mpi4py (1.3.1) ...
$pip show mpi4py --- Metadata-Version: 1.1 Name: mpi4py Version: 1.3.1 Summary: MPI for Python Home-page: http://mpi4py.googlecode.com/ Author: Lisandro Dalcin Author-email: dalcinl@gmail.com License: BSD Location: /opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages Requires:
$python -c "import mpi4py; print mpi4py.__path__"
['/opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages/mpi4py']
$which python /opt/apps/intel15/python/2.7.9/bin/python
Could you try if you can reproduce these? You might be using wrong python for instance.
Cheers,
Tuomas
On 02/10/2016 09:21 PM, Tao Wang wrote:
The mpi4py was installed by me with "pip install mpi4py"
If I uninstall it, then run "which mpi4py" or "pydoc mpi4py", nothing is found. I don't understand *"**Stampede for instance has mvapich2 available which has mpi4py". *
What do you mean and how can I enable it ?
On Thu, Feb 11, 2016 at 12:12 AM, Tao Wang <twang15@ncsu.edu> wrote:
Hi, Tuomas
It is Stampede.
(firedrake) c559-703.stampede(20)$ ldd /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so linux-vdso.so.1 => (0x00007fff6c47f000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaf89bd3000) libpython2.7.so.1.0 => /home1/03915/taoncsu/tools/python/lib/libpython2.7.so.1.0 (0x00002aaf89dd7000) *libmpi.so.12 => /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12* (0x00002aaf8a1b9000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaf8abe6000) libc.so.6 => /lib64/libc.so.6 (0x00002aaf8ae03000) /lib64/ld-linux-x86-64.so.2 (0x0000003267200000) libutil.so.1 => /lib64/libutil.so.1 (0x00002aaf8b197000) libm.so.6 => /lib64/libm.so.6 (0x00002aaf8b39b000) libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00002aaf8b61f000) libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00002aaf8b82b000) libibmad.so.5 => /opt/ofed/lib64/libibmad.so.5 (0x00002aaf8bb7e000) librdmacm.so.1 => /opt/ofed/lib64/librdmacm.so.1 (0x00002aaf8bd95000) libibumad.so.3 => /opt/ofed/lib64/libibumad.so.3 (0x00002aaf8bf9d000) libibverbs.so.1 => /opt/ofed/lib64/libibverbs.so.1 (0x00002aaf8c1a4000) librt.so.1 => /lib64/librt.so.1 (0x00002aaf8c3b2000) libifport.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifport.so.5 (0x00002aaf8c5ba000) libifcore.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifcore.so.5 (0x00002aaf8c7e8000) libimf.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libimf.so (0x00002aaf8cb1a000) libsvml.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libsvml.so (0x00002aaf8cfd5000) libintlc.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libintlc.so.5 (0x00002aaf8dea9000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aaf8e104000) libz.so.1 => /work/03915/taoncsu/linuxbrew/lib/libz.so.1 (0x00002aaf8e31b000)
Then, I run
nm /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12 | grep ompi_mpi_char
Nothing found. Thus, I supposed ompi_mpi_char is not in /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12
Any ideas ?
On Wed, Feb 10, 2016 at 10:48 PM, Tuomas Karna < <tuomas.karna@gmail.com> tuomas.karna@gmail.com> wrote:
Hi Tao,
May I ask which machine you are using?
Stampede for instance has mvapich2 available which has mpi4py, and works with PETSc/firedrake.
- Tuomas
On 02/10/2016 06:26 PM, Tao Wang wrote:
Since firedrake depends on mpi4py, which depends on open-mpi or mpich. But the server I used does not have them, only having intel mpi. The following is a test.
python -c "from mpi4py import MPI"
====*Error msg *==== Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so: undefined symbol: ompi_mpi_char
Then, I run nm /opt/apps/intel15/impi/5.0.2.044/intel64/lib/libmpi.so.12 <http://5.0.2.36/intel64/lib/libmpi.so.12> | grep ompi_mpi_char
Nothing is returned, it seems there is no such symbol in Intel MPI implementation.
Does this suggest that firedrake cannot work with intel mpi implementation ?
On Wed, Feb 10, 2016 at 8:43 PM, Tao Wang < <twang15@ncsu.edu> twang15@ncsu.edu> wrote:
I failed to install it on CentOS with --no-package-manager option
Please see this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> for more details. Could you answer my questions one by one ?
Thanks a lot for your help.
On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya < <m.homolya14@imperial.ac.uk>m.homolya14@imperial.ac.uk> wrote:
Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.
On 10 Feb 2016, at 22:28, Tao Wang < <twang15@ncsu.edu> twang15@ncsu.edu> wrote:
Hi, all
I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. I already have impi installed on my machine.
What should I do to make this change happen ?
Thanks, Tao _______________________________________________ firedrake mailing list <firedrake@imperial.ac.uk>firedrake@imperial.ac.uk <https://mailman.ic.ac.uk/mailman/listinfo/firedrake> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing listfiredrake@imperial.ac.ukhttps://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing listfiredrake@imperial.ac.ukhttps://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi, Tuomas, I remember this python(*~/tools/python/bin/python)* was installed by me. Because, the first I login to Stampede, I found `which python` gave me "/usr/bin/python", Python 2.6.6. It is too old for me to install firedrake. So I build one and install it under my home. I was not aware there is a python (/opt/apps/intel15/python/2.7.9/bin/python) new enough for firedrake until you showed me. It is easy to set python to /opt/apps/intel15/python/2.7.9/bin/python. But, I failed to make "pip list" to find mpi4py after just set the $PATH= /opt/apps/intel15/python/2.7.9/bin:$PATH I tried another way module load python/2.7.9 The both python and mpi4py shows as yours. But, the "make test" under firedrake directory fails. I guess firedrake has to be rebuilt from scratch. I will update On Thu, Feb 11, 2016 at 1:25 AM, Tao Wang <twang15@ncsu.edu> wrote:
Hi, Tuomas The following is the output:
c559-703.stampede(33)$ module list
Currently Loaded Modules: 1) intel/15.0.2 2) mvapich2/2.1 3) xalt/0.6 4) TACC
c559-703.stampede(34)$ pip list apipkg (1.4) Cython (0.23.4) decorator (4.0.6) execnet (1.4.1) flake8 (2.5.1) mccabe (0.3.1) *mpi4py (2.0.0)* networkx (1.10) numpy (1.10.4) pep8 (1.7.0) petsc4py (3.6.0) pip (8.0.2) pudb (2015.4.1) py (1.4.31) pyflakes (1.0.0) Pygments (2.1) pytest (2.8.5) pytest-xdist (1.13.1) setuptools (19.4) triangle (2015.12.14) urwid (1.3.1) virtualenv (14.0.1) wheel (0.26.0) c559-703.stampede(35)$ pip show mpi4py --- Metadata-Version: 2.0 Name: mpi4py Version: 2.0.0 Summary: Python bindings for MPI Home-page: https://bitbucket.org/mpi4py/mpi4py/ Author: Lisandro Dalcin Author-email: dalcinl@gmail.com License: BSD Location: */home1/03915/taoncsu/tools/python/lib/python2.7/site-packages* Requires: c559-703.stampede(36)$ $python -c "import mpi4py; print mpi4py.__path__" -bash: -c: command not found c559-703.stampede(37)$ python -c "import mpi4py; print mpi4py.__path__" ['/home1/03915/taoncsu/tools/python/lib/python2.7/site-packages/mpi4py'] c559-703.stampede(38)$ which python *~/tools/python/bin/python*
Apparently, the mpy4py and python are not the same to yours. I am curious about what makes these differences. It seems something is wrong with my .bashrc
On Thu, Feb 11, 2016 at 12:36 AM, Tuomas Karna <tuomas.karna@gmail.com> wrote:
Hi Tao,
Also on Stampede, I have loaded the following modules: $module list
Currently Loaded Modules: 1) xalt/0.6 3) intel/15.0.2 5) phdf5/1.8.16 7) netcdf/4.3.3.1 (m) 2) TACC 4) mvapich2/2.1 6) python/2.7.9 8) pmetis/4.0.2
Python picks up the right mpi4py (virtualenv is not active): $pip list ... mpi4py (1.3.1) ...
$pip show mpi4py --- Metadata-Version: 1.1 Name: mpi4py Version: 1.3.1 Summary: MPI for Python Home-page: http://mpi4py.googlecode.com/ Author: Lisandro Dalcin Author-email: dalcinl@gmail.com License: BSD Location: /opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages Requires:
$python -c "import mpi4py; print mpi4py.__path__"
['/opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages/mpi4py']
$which python /opt/apps/intel15/python/2.7.9/bin/python
Could you try if you can reproduce these? You might be using wrong python for instance.
Cheers,
Tuomas
On 02/10/2016 09:21 PM, Tao Wang wrote:
The mpi4py was installed by me with "pip install mpi4py"
If I uninstall it, then run "which mpi4py" or "pydoc mpi4py", nothing is found. I don't understand *"**Stampede for instance has mvapich2 available which has mpi4py". *
What do you mean and how can I enable it ?
On Thu, Feb 11, 2016 at 12:12 AM, Tao Wang <twang15@ncsu.edu> wrote:
Hi, Tuomas
It is Stampede.
(firedrake) c559-703.stampede(20)$ ldd /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so linux-vdso.so.1 => (0x00007fff6c47f000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaf89bd3000) libpython2.7.so.1.0 => /home1/03915/taoncsu/tools/python/lib/libpython2.7.so.1.0 (0x00002aaf89dd7000) *libmpi.so.12 => /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12* (0x00002aaf8a1b9000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaf8abe6000) libc.so.6 => /lib64/libc.so.6 (0x00002aaf8ae03000) /lib64/ld-linux-x86-64.so.2 (0x0000003267200000) libutil.so.1 => /lib64/libutil.so.1 (0x00002aaf8b197000) libm.so.6 => /lib64/libm.so.6 (0x00002aaf8b39b000) libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00002aaf8b61f000) libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00002aaf8b82b000) libibmad.so.5 => /opt/ofed/lib64/libibmad.so.5 (0x00002aaf8bb7e000) librdmacm.so.1 => /opt/ofed/lib64/librdmacm.so.1 (0x00002aaf8bd95000) libibumad.so.3 => /opt/ofed/lib64/libibumad.so.3 (0x00002aaf8bf9d000) libibverbs.so.1 => /opt/ofed/lib64/libibverbs.so.1 (0x00002aaf8c1a4000) librt.so.1 => /lib64/librt.so.1 (0x00002aaf8c3b2000) libifport.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifport.so.5 (0x00002aaf8c5ba000) libifcore.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifcore.so.5 (0x00002aaf8c7e8000) libimf.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libimf.so (0x00002aaf8cb1a000) libsvml.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libsvml.so (0x00002aaf8cfd5000) libintlc.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libintlc.so.5 (0x00002aaf8dea9000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aaf8e104000) libz.so.1 => /work/03915/taoncsu/linuxbrew/lib/libz.so.1 (0x00002aaf8e31b000)
Then, I run
nm /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12 | grep ompi_mpi_char
Nothing found. Thus, I supposed ompi_mpi_char is not in /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12
Any ideas ?
On Wed, Feb 10, 2016 at 10:48 PM, Tuomas Karna < <tuomas.karna@gmail.com>tuomas.karna@gmail.com> wrote:
Hi Tao,
May I ask which machine you are using?
Stampede for instance has mvapich2 available which has mpi4py, and works with PETSc/firedrake.
- Tuomas
On 02/10/2016 06:26 PM, Tao Wang wrote:
Since firedrake depends on mpi4py, which depends on open-mpi or mpich. But the server I used does not have them, only having intel mpi. The following is a test.
python -c "from mpi4py import MPI"
====*Error msg *==== Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so: undefined symbol: ompi_mpi_char
Then, I run nm /opt/apps/intel15/impi/5.0.2.044/intel64/lib/libmpi.so.12 <http://5.0.2.36/intel64/lib/libmpi.so.12> | grep ompi_mpi_char
Nothing is returned, it seems there is no such symbol in Intel MPI implementation.
Does this suggest that firedrake cannot work with intel mpi implementation ?
On Wed, Feb 10, 2016 at 8:43 PM, Tao Wang < <twang15@ncsu.edu> twang15@ncsu.edu> wrote:
I failed to install it on CentOS with --no-package-manager option
Please see this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> for more details. Could you answer my questions one by one ?
Thanks a lot for your help.
On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya < <m.homolya14@imperial.ac.uk>m.homolya14@imperial.ac.uk> wrote:
Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.
> On 10 Feb 2016, at 22:28, Tao Wang < <twang15@ncsu.edu> twang15@ncsu.edu> wrote: > > Hi, all > > I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. > I already have impi installed on my machine. > > What should I do to make this change happen ? > > Thanks, > Tao > _______________________________________________ > firedrake mailing list > <firedrake@imperial.ac.uk>firedrake@imperial.ac.uk > <https://mailman.ic.ac.uk/mailman/listinfo/firedrake> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing listfiredrake@imperial.ac.ukhttps://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing listfiredrake@imperial.ac.ukhttps://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi Tao, On 02/10/2016 10:59 PM, Tao Wang wrote:
Hi, Tuomas,
I remember this python(*~/tools/python/bin/python)* was installed by me. Because, the first I login to Stampede, I found `which python` gave me "/usr/bin/python", Python 2.6.6. It is too old for me to install firedrake. So I build one and install it under my home. I was not aware there is a python (/opt/apps/intel15/python/2.7.9/bin/python) new enough for firedrake until you showed me.
It is easy to set python to /opt/apps/intel15/python/2.7.9/bin/python. But, I failed to make "pip list" to find mpi4py after just set the $PATH=/opt/apps/intel15/python/2.7.9/bin:$PATH
I tried another way
module load python/2.7.9 The both python and mpi4py shows as yours.
But, the "make test" under firedrake directory fails. I guess firedrake has to be rebuilt from scratch. I will update
OK, great! We've made some progress. Yes you'll need to rebuild firedrake. I think this worked for me earlier: export PETSC_CONFIGURE_OPTIONS="--with-blas-lapack-dir=$MKLROOT" python firedrake-install --developer --no-package-manager --log The first line tells petsc to use intel mkl blas and lapack, $MKLROOT should point to the right dir.
On Thu, Feb 11, 2016 at 1:25 AM, Tao Wang <twang15@ncsu.edu <mailto:twang15@ncsu.edu>> wrote:
Hi, Tuomas The following is the output:
c559-703.stampede(33)$ module list
Currently Loaded Modules: 1) intel/15.0.2 2) mvapich2/2.1 3) xalt/0.6 4) TACC
c559-703.stampede(34)$ pip list apipkg (1.4) Cython (0.23.4) decorator (4.0.6) execnet (1.4.1) flake8 (2.5.1) mccabe (0.3.1) *mpi4py (2.0.0)* networkx (1.10) numpy (1.10.4) pep8 (1.7.0) petsc4py (3.6.0) pip (8.0.2) pudb (2015.4.1) py (1.4.31) pyflakes (1.0.0) Pygments (2.1) pytest (2.8.5) pytest-xdist (1.13.1) setuptools (19.4) triangle (2015.12.14) urwid (1.3.1) virtualenv (14.0.1) wheel (0.26.0) c559-703.stampede(35)$ pip show mpi4py --- Metadata-Version: 2.0 Name: mpi4py Version: 2.0.0 Summary: Python bindings for MPI Home-page: https://bitbucket.org/mpi4py/mpi4py/ Author: Lisandro Dalcin Author-email: dalcinl@gmail.com <mailto:dalcinl@gmail.com> License: BSD Location: */home1/03915/taoncsu/tools/python/lib/python2.7/site-packages* Requires: c559-703.stampede(36)$ $python -c "import mpi4py; print mpi4py.__path__" -bash: -c: command not found c559-703.stampede(37)$ python -c "import mpi4py; print mpi4py.__path__" ['/home1/03915/taoncsu/tools/python/lib/python2.7/site-packages/mpi4py'] c559-703.stampede(38)$ which python *~/tools/python/bin/python*
Apparently, the mpy4py and python are not the same to yours. I am curious about what makes these differences. It seems something is wrong with my .bashrc
On Thu, Feb 11, 2016 at 12:36 AM, Tuomas Karna <tuomas.karna@gmail.com <mailto:tuomas.karna@gmail.com>> wrote:
Hi Tao,
Also on Stampede, I have loaded the following modules: $module list
Currently Loaded Modules: 1) xalt/0.6 3) intel/15.0.2 5) phdf5/1.8.16 7) netcdf/4.3.3.1 <http://4.3.3.1> (m) 2) TACC 4) mvapich2/2.1 6) python/2.7.9 8) pmetis/4.0.2
Python picks up the right mpi4py (virtualenv is not active): $pip list ... mpi4py (1.3.1) ...
$pip show mpi4py --- Metadata-Version: 1.1 Name: mpi4py Version: 1.3.1 Summary: MPI for Python Home-page: http://mpi4py.googlecode.com/ Author: Lisandro Dalcin Author-email: dalcinl@gmail.com <mailto:dalcinl@gmail.com> License: BSD Location: /opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages Requires:
$python -c "import mpi4py; print mpi4py.__path__" ['/opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages/mpi4py']
$which python /opt/apps/intel15/python/2.7.9/bin/python
Could you try if you can reproduce these? You might be using wrong python for instance.
Cheers,
Tuomas
On 02/10/2016 09:21 PM, Tao Wang wrote:
The mpi4py was installed by me with "pip install mpi4py"
If I uninstall it, then run "which mpi4py" or "pydoc mpi4py", nothing is found. I don't understand *"**Stampede for instance has mvapich2 available which has mpi4py". *
What do you mean and how can I enable it ?
On Thu, Feb 11, 2016 at 12:12 AM, Tao Wang <twang15@ncsu.edu <mailto:twang15@ncsu.edu>> wrote:
Hi, Tuomas
It is Stampede.
(firedrake) c559-703.stampede(20)$ ldd /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so linux-vdso.so.1 => (0x00007fff6c47f000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaf89bd3000) libpython2.7.so.1.0 => /home1/03915/taoncsu/tools/python/lib/libpython2.7.so.1.0 (0x00002aaf89dd7000) *libmpi.so.12 => /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12* (0x00002aaf8a1b9000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaf8abe6000) libc.so.6 => /lib64/libc.so.6 (0x00002aaf8ae03000) /lib64/ld-linux-x86-64.so.2 (0x0000003267200000) libutil.so.1 => /lib64/libutil.so.1 (0x00002aaf8b197000) libm.so.6 => /lib64/libm.so.6 (0x00002aaf8b39b000) libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00002aaf8b61f000) libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00002aaf8b82b000) libibmad.so.5 => /opt/ofed/lib64/libibmad.so.5 (0x00002aaf8bb7e000) librdmacm.so.1 => /opt/ofed/lib64/librdmacm.so.1 (0x00002aaf8bd95000) libibumad.so.3 => /opt/ofed/lib64/libibumad.so.3 (0x00002aaf8bf9d000) libibverbs.so.1 => /opt/ofed/lib64/libibverbs.so.1 (0x00002aaf8c1a4000) librt.so.1 => /lib64/librt.so.1 (0x00002aaf8c3b2000) libifport.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifport.so.5 (0x00002aaf8c5ba000) libifcore.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifcore.so.5 (0x00002aaf8c7e8000) libimf.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libimf.so (0x00002aaf8cb1a000) libsvml.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libsvml.so (0x00002aaf8cfd5000) libintlc.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libintlc.so.5 (0x00002aaf8dea9000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aaf8e104000) libz.so.1 => /work/03915/taoncsu/linuxbrew/lib/libz.so.1 (0x00002aaf8e31b000)
Then, I run
nm /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12 | grep ompi_mpi_char
Nothing found. Thus, I supposed ompi_mpi_char is not in /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12
Any ideas ?
On Wed, Feb 10, 2016 at 10:48 PM, Tuomas Karna <tuomas.karna@gmail.com <mailto:tuomas.karna@gmail.com>> wrote:
Hi Tao,
May I ask which machine you are using?
Stampede for instance has mvapich2 available which has mpi4py, and works with PETSc/firedrake.
- Tuomas
On 02/10/2016 06:26 PM, Tao Wang wrote:
Since firedrake depends on mpi4py, which depends on open-mpi or mpich. But the server I used does not have them, only having intel mpi. The following is a test.
python -c "from mpi4py import MPI"
====*Error msg *==== Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so: undefined symbol: ompi_mpi_char
Then, I run nm /opt/apps/intel15/impi/5.0.2.044/intel64/lib/libmpi.so.12 <http://5.0.2.36/intel64/lib/libmpi.so.12> | grep ompi_mpi_char
Nothing is returned, it seems there is no such symbol in Intel MPI implementation.
Does this suggest that firedrake cannot work with intel mpi implementation ?
On Wed, Feb 10, 2016 at 8:43 PM, Tao Wang <twang15@ncsu.edu <mailto:twang15@ncsu.edu>> wrote:
I failed to install it on CentOS with --no-package-manager option
Please see this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> for more details. Could you answer my questions one by one ?
Thanks a lot for your help.
On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya <m.homolya14@imperial.ac.uk <mailto:m.homolya14@imperial.ac.uk>> wrote:
Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.
> On 10 Feb 2016, at 22:28, Tao Wang <twang15@ncsu.edu <mailto:twang15@ncsu.edu>> wrote: > > Hi, all > > I notice firedrake has dependency on open-mpi. But, I want to use other mpi implementation. > I already have impi installed on my machine. > > What should I do to make this change happen ? > > Thanks, > Tao > _______________________________________________ > firedrake mailing list > firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> > https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi, Tuomas The build and run succeed now ! Though, it still needs the extra workaround about libspatialindex as recorded in this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> . Thanks for your great help ! Best, Tao On Thu, Feb 11, 2016 at 2:14 AM, Tuomas Karna <tuomas.karna@gmail.com> wrote:
Hi Tao,
On 02/10/2016 10:59 PM, Tao Wang wrote:
Hi, Tuomas,
I remember this python(*~/tools/python/bin/python)* was installed by me. Because, the first I login to Stampede, I found `which python` gave me "/usr/bin/python", Python 2.6.6. It is too old for me to install firedrake. So I build one and install it under my home. I was not aware there is a python (/opt/apps/intel15/python/2.7.9/bin/python) new enough for firedrake until you showed me.
It is easy to set python to /opt/apps/intel15/python/2.7.9/bin/python. But, I failed to make "pip list" to find mpi4py after just set the $PATH= /opt/apps/intel15/python/2.7.9/bin:$PATH
I tried another way
module load python/2.7.9 The both python and mpi4py shows as yours.
But, the "make test" under firedrake directory fails. I guess firedrake has to be rebuilt from scratch. I will update
OK, great! We've made some progress. Yes you'll need to rebuild firedrake.
I think this worked for me earlier:
export PETSC_CONFIGURE_OPTIONS="--with-blas-lapack-dir=$MKLROOT" python firedrake-install --developer --no-package-manager --log
The first line tells petsc to use intel mkl blas and lapack, $MKLROOT should point to the right dir.
On Thu, Feb 11, 2016 at 1:25 AM, Tao Wang <twang15@ncsu.edu> wrote:
Hi, Tuomas The following is the output:
c559-703.stampede(33)$ module list
Currently Loaded Modules: 1) intel/15.0.2 2) mvapich2/2.1 3) xalt/0.6 4) TACC
c559-703.stampede(34)$ pip list apipkg (1.4) Cython (0.23.4) decorator (4.0.6) execnet (1.4.1) flake8 (2.5.1) mccabe (0.3.1) *mpi4py (2.0.0)* networkx (1.10) numpy (1.10.4) pep8 (1.7.0) petsc4py (3.6.0) pip (8.0.2) pudb (2015.4.1) py (1.4.31) pyflakes (1.0.0) Pygments (2.1) pytest (2.8.5) pytest-xdist (1.13.1) setuptools (19.4) triangle (2015.12.14) urwid (1.3.1) virtualenv (14.0.1) wheel (0.26.0) c559-703.stampede(35)$ pip show mpi4py --- Metadata-Version: 2.0 Name: mpi4py Version: 2.0.0 Summary: Python bindings for MPI Home-page: https://bitbucket.org/mpi4py/mpi4py/ Author: Lisandro Dalcin Author-email: dalcinl@gmail.com License: BSD Location: */home1/03915/taoncsu/tools/python/lib/python2.7/site-packages* Requires: c559-703.stampede(36)$ $python -c "import mpi4py; print mpi4py.__path__" -bash: -c: command not found c559-703.stampede(37)$ python -c "import mpi4py; print mpi4py.__path__" ['/home1/03915/taoncsu/tools/python/lib/python2.7/site-packages/mpi4py'] c559-703.stampede(38)$ which python *~/tools/python/bin/python*
Apparently, the mpy4py and python are not the same to yours. I am curious about what makes these differences. It seems something is wrong with my .bashrc
On Thu, Feb 11, 2016 at 12:36 AM, Tuomas Karna < <tuomas.karna@gmail.com> tuomas.karna@gmail.com> wrote:
Hi Tao,
Also on Stampede, I have loaded the following modules: $module list
Currently Loaded Modules: 1) xalt/0.6 3) intel/15.0.2 5) phdf5/1.8.16 7) netcdf/4.3.3.1 (m) 2) TACC 4) mvapich2/2.1 6) python/2.7.9 8) pmetis/4.0.2
Python picks up the right mpi4py (virtualenv is not active): $pip list ... mpi4py (1.3.1) ...
$pip show mpi4py --- Metadata-Version: 1.1 Name: mpi4py Version: 1.3.1 Summary: MPI for Python Home-page: http://mpi4py.googlecode.com/ Author: Lisandro Dalcin Author-email: dalcinl@gmail.com License: BSD Location: /opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages Requires:
$python -c "import mpi4py; print mpi4py.__path__"
['/opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages/mpi4py']
$which python /opt/apps/intel15/python/2.7.9/bin/python
Could you try if you can reproduce these? You might be using wrong python for instance.
Cheers,
Tuomas
On 02/10/2016 09:21 PM, Tao Wang wrote:
The mpi4py was installed by me with "pip install mpi4py"
If I uninstall it, then run "which mpi4py" or "pydoc mpi4py", nothing is found. I don't understand *"**Stampede for instance has mvapich2 available which has mpi4py". *
What do you mean and how can I enable it ?
On Thu, Feb 11, 2016 at 12:12 AM, Tao Wang < <twang15@ncsu.edu> twang15@ncsu.edu> wrote:
Hi, Tuomas
It is Stampede.
(firedrake) c559-703.stampede(20)$ ldd /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so linux-vdso.so.1 => (0x00007fff6c47f000) libdl.so.2 => /lib64/libdl.so.2 (0x00002aaf89bd3000) libpython2.7.so.1.0 => /home1/03915/taoncsu/tools/python/lib/libpython2.7.so.1.0 (0x00002aaf89dd7000) *libmpi.so.12 => /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12* (0x00002aaf8a1b9000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaf8abe6000) libc.so.6 => /lib64/libc.so.6 (0x00002aaf8ae03000) /lib64/ld-linux-x86-64.so.2 (0x0000003267200000) libutil.so.1 => /lib64/libutil.so.1 (0x00002aaf8b197000) libm.so.6 => /lib64/libm.so.6 (0x00002aaf8b39b000) libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00002aaf8b61f000) libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00002aaf8b82b000) libibmad.so.5 => /opt/ofed/lib64/libibmad.so.5 (0x00002aaf8bb7e000) librdmacm.so.1 => /opt/ofed/lib64/librdmacm.so.1 (0x00002aaf8bd95000) libibumad.so.3 => /opt/ofed/lib64/libibumad.so.3 (0x00002aaf8bf9d000) libibverbs.so.1 => /opt/ofed/lib64/libibverbs.so.1 (0x00002aaf8c1a4000) librt.so.1 => /lib64/librt.so.1 (0x00002aaf8c3b2000) libifport.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifport.so.5 (0x00002aaf8c5ba000) libifcore.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifcore.so.5 (0x00002aaf8c7e8000) libimf.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libimf.so (0x00002aaf8cb1a000) libsvml.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libsvml.so (0x00002aaf8cfd5000) libintlc.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libintlc.so.5 (0x00002aaf8dea9000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aaf8e104000) libz.so.1 => /work/03915/taoncsu/linuxbrew/lib/libz.so.1 (0x00002aaf8e31b000)
Then, I run
nm /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12 | grep ompi_mpi_char
Nothing found. Thus, I supposed ompi_mpi_char is not in /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12
Any ideas ?
On Wed, Feb 10, 2016 at 10:48 PM, Tuomas Karna < <tuomas.karna@gmail.com>tuomas.karna@gmail.com> wrote:
Hi Tao,
May I ask which machine you are using?
Stampede for instance has mvapich2 available which has mpi4py, and works with PETSc/firedrake.
- Tuomas
On 02/10/2016 06:26 PM, Tao Wang wrote:
Since firedrake depends on mpi4py, which depends on open-mpi or mpich. But the server I used does not have them, only having intel mpi. The following is a test.
python -c "from mpi4py import MPI"
====*Error msg *==== Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so: undefined symbol: ompi_mpi_char
Then, I run nm /opt/apps/intel15/impi/5.0.2.044/intel64/lib/libmpi.so.12 <http://5.0.2.36/intel64/lib/libmpi.so.12> | grep ompi_mpi_char
Nothing is returned, it seems there is no such symbol in Intel MPI implementation.
Does this suggest that firedrake cannot work with intel mpi implementation ?
On Wed, Feb 10, 2016 at 8:43 PM, Tao Wang < <twang15@ncsu.edu> twang15@ncsu.edu> wrote:
I failed to install it on CentOS with --no-package-manager option
Please see this issue <https://github.com/firedrakeproject/firedrake/issues/680#issuecomment-175511215> for more details. Could you answer my questions one by one ?
Thanks a lot for your help.
On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya < <m.homolya14@imperial.ac.uk>m.homolya14@imperial.ac.uk> wrote:
> Make sure you have all the other dependencies installed and then use > the install script with the —no-package-manager option. > > > On 10 Feb 2016, at 22:28, Tao Wang < <twang15@ncsu.edu> > twang15@ncsu.edu> wrote: > > > > Hi, all > > > > I notice firedrake has dependency on open-mpi. But, I want to use > other mpi implementation. > > I already have impi installed on my machine. > > > > What should I do to make this change happen ? > > > > Thanks, > > Tao > > _______________________________________________ > > firedrake mailing list > > <firedrake@imperial.ac.uk>firedrake@imperial.ac.uk > > <https://mailman.ic.ac.uk/mailman/listinfo/firedrake> > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > _______________________________________________ > firedrake mailing list > <firedrake@imperial.ac.uk>firedrake@imperial.ac.uk > <https://mailman.ic.ac.uk/mailman/listinfo/firedrake> > https://mailman.ic.ac.uk/mailman/listinfo/firedrake >
_______________________________________________ firedrake mailing listfiredrake@imperial.ac.ukhttps://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing listfiredrake@imperial.ac.ukhttps://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing listfiredrake@imperial.ac.ukhttps://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
participants (3)
-
Miklós Homolya
-
Tao Wang
-
Tuomas Karna