Dear Niall,

It seems to me the following line is critical:

src/dynload.c:5:10: fatal error: Python.h: No such file or directory

You want to have an appropriate package loaded. If you have the `module` command available,

you could try `module search python3-dev` or (`module search python-dev`), followed by `module load package_name`.

We can also see that you have openmpi loaded (`module list` will show you openmpi), but it is known to cause some issues, so you probably want to unload openmpi and load mpich.

Though it is not directly related to the error you are having right now, petsc usually requires some additional parameterswhen configuring on clusters (--with-batch --known-mpi-shared-libraries=0) (https://www.mcs.anl.gov/petsc/documentation/installation.html ),

which we don't see in your previous email.

Were you able to test petsc on a compute node? If it worked on a computed node, I think it is fine.

Regards,

Koki

Hi Lawrence,

Thanks for the pointer on PETSc. I was able to make and test it without problems.

Then I set PETSC_DIR and PETSC_ARCH,

and ran

python3 firedrake-install --disable-ssh --no-package-manager --honour-petsc-dir

It fails at

subprocess.CalledProcessError: Command '['/MYOB/r/libs/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py,randomgen', '-vvv', '-r', 'PyOP2/requirements-ext.txt']' returned non-zero exit status 1.

But

/MYOB/r/libs/firedrake/bin/python -m pip install --no-binary mpi4py -vvv -r ./src/PyOP2/requirements-ext.txt

returns numerous "Requirement already satisfied:" messages.

Gzip'ed log file attached.

Thanks,

Niall.

From: Lawrence Mitchell <wence@gmx.li>
Sent: Thursday 1 August 2019 15:53
To: Madden, Niall <niall.madden@nuigalway.ie>
Cc: Firedrake Project <firedrake@imperial.ac.uk>
Subject: Re: [firedrake] Installing firedrake on a HPC system without package manager

Hi Niall, (adding in firedrake cc again)

> On 1 Aug 2019, at 15:43, Madden, Niall <niall.madden@nuigalway.ie> wrote:
>
> Hi Lawrence,
> Thanks for the swift reply. I don't want to reply to the whole list with too much specific information on the system

Please do reply with the list. That way others who might be able to help will see things.

>> Can you give us some details on this system? Is it a cray, for example?
> The system is ICHEC's "kay" - Ireland's national supercomputer for academic purposes:
> https://www.ichec.ie/about/infrastructure/kay
>
>> Normally we find when building on supercomputers (contrary to the one-shot "firedrake-install" unfortunately),
>> that it is best to build PETSc by hand. One then sets the appropriate environment variables and runs
>> firedrake-install --honour-petsc-dir. This might be the best step forward.
>
> Thanks for the tip. I tried compiling PETSc myself:
>
> git clone https://github.com/firedrakeproject/petsc.git (but should I be cloning fromhttps://bitbucket.org/petsc/petsc ?)

No, cloning from the firedrakeproject "fork" is correct. We keep a (reasonably up to date) version of petsc that is known good with all Firedrake components there: it isn't otherwise special.

> ./configure --with-c2html=0 --download-chaco --download-metis --download-parmetis --download-scalapack --download-hypre --download-mumps --download-netcdf --download-hdf5 --download-pnetcdf --download-exodusii --download-mpich --download-suitesparse=1 --download-petsc4py=1 --download-mpi4py=1 --download-fblaslapack --download-zlib
>
> Then I set PETSC_DIR and PETSC_ARCH, and PYTHONPATH
>
> Then I try to install Firedrake with
> python3 firedrake-install --no-package-manager -v --disable-ssh --honour-petsc-dir --honour-pythonpath
>
> But it eventually fails compiling libspatialindex
> for reasons that seem to be related to the mpicxx , but it is a little opaque:
> undefined reference to `psm2_info_query@PSM2_1.0'

Can you attach the contents of firedrake-install.log (maybe gzip it since it is likely to be large).

Thanks,

Lawrence