Yes run-pyop2.py contains exactly those two lines

> mpiexec -n 1 python run-pyop2.py 

init pyop2

forking

init mpi

--------------------------------------------------------------------------

An MPI process has executed an operation involving a call to the

"fork()" system call to create a child process.  Open MPI is currently

operating in a condition that could result in memory corruption or

other system errors; your MPI job may hang, crash, or produce silent

data corruption.  The use of fork() (or system() or other calls that

create child processes) is strongly discouraged.  


The process that invoked fork was:


  Local host:          compute-0-0 (PID 52215)

  MPI_COMM_WORLD rank: 0


If you are *absolutely sure* that your application will successfully

and correctly survive a call to fork(), you may disable this warning

by setting the mpi_warn_on_fork MCA parameter to 0.

--------------------------------------------------------------------------


So I think I am getting exactly what you're getting except I am still getting the fork error. And when I try your latest test:

>mpiexec -n 2 python manual-fork.py 

forking

forking

init pyop2

init mpi

init pyop2

init mpi

--------------------------------------------------------------------------

An MPI process has executed an operation involving a call to the

"fork()" system call to create a child process.  Open MPI is currently

operating in a condition that could result in memory corruption or

other system errors; your MPI job may hang, crash, or produce silent

data corruption.  The use of fork() (or system() or other calls that

create child processes) is strongly discouraged.  


The process that invoked fork was:


  Local host:          compute-0-0 (PID 52345)

  MPI_COMM_WORLD rank: 1


If you are *absolutely sure* that your application will successfully

and correctly survive a call to fork(), you may disable this warning

by setting the mpi_warn_on_fork MCA parameter to 0.

--------------------------------------------------------------------------

2

2

[compute-0-0.local:52342] 1 more process has sent help message help-mpi-runtime.txt / mpi_init:warn-fork

[compute-0-0.local:52342] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages


On Thu, Aug 6, 2015 at 9:55 AM, Lawrence Mitchell <lawrence.mitchell@imperial.ac.uk> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/08/15 15:42, Justin Chang wrote:
>> mpiexec -n 2 python run-pyop2.py
>
> --------------------------------------------------------------------------
>
>  An MPI process has executed an operation involving a call to the
>
> "fork()" system call to create a child process.  Open MPI is
> currently
>
> operating in a condition that could result in memory corruption or
>
> other system errors; your MPI job may hang, crash, or produce
> silent
>
> data corruption.  The use of fork() (or system() or other calls
> that
>
> create child processes) is strongly discouraged.
>
Huh, OK.

I was expecting this to work!  To confirm, run-pyop2.py just contains:

from pyop2 import op2
op2.init()

Can you edit some files in PyOP2 just to check what's going on:

In PyOP2/pyop2/__init__.py

Replace:

from pyop2_utils import enable_mpi_prefork

with:

print 'init pyop2'
from pyop2_utils import enable_mpi_prefork

In PyOP2/pyop2/mpi.py

Replace:

from decorator import decorator

with:

print 'init mpi'
from decorator import decorator

In PyOP2/pyop2_utils/__init__.py

Replace:

        prefork.enable_prefork()

with:

        print 'forking'
        prefork.enable_prefork()


And then try running run-pyop2.py on just one MPI process.

I see:

init pyop2
forking
loading mpi


But maybe you get something else?

And then let's test if the following works:

cat > manual-fork.py << EOF
from pyop2_utils import enable_mpi_prefork
enable_mpi_prefork()
from firedrake import *
from mpi4py import MPI
print MPI.COMM_WORLD.size
EOF
mpiexec -n 2 python manual-fork.py

Cheers,

Lawrence

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJVw3VgAAoJECOc1kQ8PEYv6vMH/0axxqZd+KqdQ5OMIjdBuF9g
OIJC6YcoCCcigYh/RccSAVpzTuAvGWQdRJqMMlEfukrZrn6NSsw+TT+kzR+SPneg
D66OVIsO2vfKfia/9yflEE4gKkVHfzo067dzstDZ8HnG0tVEqAexJdKzfK7nUE6M
LJovg+YbJ5g/50BPbfCYJNRCtYAfjHjZWNOfYlLRITH55O7oIlAo4d2Pvk3wb+ok
lDJI9G1gdHOIz8U+uuQ8/dKbutUSRfH7a3NK4wbjf/Q05BCS16k2T/Ouim1wkUql
9/4lKDkKsNTfMILXxNf3STlHJJlF5JRpDRJ+N462UjfJyTI9qTeZIv5GD9lH/x8=
=afxk
-----END PGP SIGNATURE-----

_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake