Hi Tao,

Also on Stampede, I have loaded the following modules:
$module list

Currently Loaded Modules:
  1) xalt/0.6   3) intel/15.0.2   5) phdf5/1.8.16   7) netcdf/4.3.3.1 (m)
  2) TACC       4) mvapich2/2.1   6) python/2.7.9   8) pmetis/4.0.2

Python picks up the right mpi4py (virtualenv is not active):
$pip list
...
mpi4py (1.3.1)
...

$pip show mpi4py
---
Metadata-Version: 1.1
Name: mpi4py
Version: 1.3.1
Summary: MPI for Python
Home-page: http://mpi4py.googlecode.com/
Author: Lisandro Dalcin
Author-email: dalcinl@gmail.com
License: BSD
Location: /opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages
Requires:

$python -c "import mpi4py; print mpi4py.__path__"
['/opt/apps/intel15/mvapich2_2_1/python/2.7.9/lib/python2.7/site-packages/mpi4py']

$which python
/opt/apps/intel15/python/2.7.9/bin/python

Could you try if you can reproduce these? You might be using wrong python for instance.

Cheers,

Tuomas

On 02/10/2016 09:21 PM, Tao Wang wrote:

The mpi4py was installed by me with  "pip install mpi4py"

If I uninstall it, then run "which mpi4py" or "pydoc mpi4py", nothing is found.

I don't understand "Stampede for instance has mvapich2 available which has mpi4py". 

What do you mean and how can I enable it ?

On Thu, Feb 11, 2016 at 12:12 AM, Tao Wang <twang15@ncsu.edu> wrote:

Hi, Tuomas

It is Stampede.  

(firedrake) c559-703.stampede(20)$ ldd /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so

linux-vdso.so.1 =>  (0x00007fff6c47f000)

libdl.so.2 => /lib64/libdl.so.2 (0x00002aaf89bd3000)

libpython2.7.so.1.0 => /home1/03915/taoncsu/tools/python/lib/libpython2.7.so.1.0 (0x00002aaf89dd7000)

libmpi.so.12 => /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12 (0x00002aaf8a1b9000)

libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaf8abe6000)

libc.so.6 => /lib64/libc.so.6 (0x00002aaf8ae03000)

/lib64/ld-linux-x86-64.so.2 (0x0000003267200000)

libutil.so.1 => /lib64/libutil.so.1 (0x00002aaf8b197000)

libm.so.6 => /lib64/libm.so.6 (0x00002aaf8b39b000)

libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00002aaf8b61f000)

libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00002aaf8b82b000)

libibmad.so.5 => /opt/ofed/lib64/libibmad.so.5 (0x00002aaf8bb7e000)

librdmacm.so.1 => /opt/ofed/lib64/librdmacm.so.1 (0x00002aaf8bd95000)

libibumad.so.3 => /opt/ofed/lib64/libibumad.so.3 (0x00002aaf8bf9d000)

libibverbs.so.1 => /opt/ofed/lib64/libibverbs.so.1 (0x00002aaf8c1a4000)

librt.so.1 => /lib64/librt.so.1 (0x00002aaf8c3b2000)

libifport.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifport.so.5 (0x00002aaf8c5ba000)

libifcore.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libifcore.so.5 (0x00002aaf8c7e8000)

libimf.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libimf.so (0x00002aaf8cb1a000)

libsvml.so => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libsvml.so (0x00002aaf8cfd5000)

libintlc.so.5 => /opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64/libintlc.so.5 (0x00002aaf8dea9000)

libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aaf8e104000)

libz.so.1 => /work/03915/taoncsu/linuxbrew/lib/libz.so.1 (0x00002aaf8e31b000)

Then, I run

nm /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12 | grep ompi_mpi_char

Nothing found.

Thus, I supposed ompi_mpi_char is not in /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so.12

Any ideas ?

On Wed, Feb 10, 2016 at 10:48 PM, Tuomas Karna <tuomas.karna@gmail.com> wrote:

Hi Tao,

May I ask which machine you are using?

Stampede for instance has mvapich2 available which has mpi4py, and works with PETSc/firedrake.

- Tuomas

On 02/10/2016 06:26 PM, Tao Wang wrote:

Since firedrake depends on mpi4py, which depends on open-mpi or mpich. But the server I used does not have them, only having intel mpi.  The following is a test.

python -c "from mpi4py import MPI"

====Error msg ====

Traceback (most recent call last):

  File "<string>", line 1, in <module>

ImportError: /work/03915/taoncsu/firedrake.impi/firedrake/lib/python2.7/site-packages/mpi4py/MPI.so: undefined symbol: ompi_mpi_char

Then, I run

nm /opt/apps/intel15/impi/5.0.2.044/intel64/lib/libmpi.so.12 | grep ompi_mpi_char

Nothing is returned, it seems there is no such symbol in Intel MPI implementation.

Does this suggest that firedrake cannot work with intel mpi implementation ?

On Wed, Feb 10, 2016 at 8:43 PM, Tao Wang <twang15@ncsu.edu> wrote:

I failed to install it on CentOS with --no-package-manager option

Please see this issue for more details.

Could you answer my questions one by one ?

Thanks a lot for your help.

On Wed, Feb 10, 2016 at 5:50 PM, Miklós Homolya <m.homolya14@imperial.ac.uk> wrote:

Make sure you have all the other dependencies installed and then use the install script with the —no-package-manager option.

> On 10 Feb 2016, at 22:28, Tao Wang <twang15@ncsu.edu> wrote:
>
> Hi, all
>
> I notice firedrake has dependency on open-mpi.  But, I want to use other mpi implementation.
> I already have impi installed on my machine.
>
> What should I do to make this change happen ?
>
> Thanks,
> Tao

> _______________________________________________
> firedrake mailing list
> firedrake@imperial.ac.uk
> https://mailman.ic.ac.uk/mailman/listinfo/firedrake


_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake

_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake

_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake

_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake

_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake