Hi Lawrence,
Thanks for the pointer on PETSc. I was able to make and test it without problems.
Then I set PETSC_DIR and PETSC_ARCH,
and ran
python3 firedrake-install --disable-ssh --no-package-manager --honour-petsc-dir
It fails at
subprocess.CalledProcessError: Command '['/MYOB/r/libs/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py,randomgen', '-vvv', '-r', 'PyOP2/requirements-ext.txt']' returned non-zero exit status 1.

But
   /MYOB/r/libs/firedrake/bin/python -m pip install --no-binary mpi4py -vvv -r ./src/PyOP2/requirements-ext.txt
returns numerous "Requirement already satisfied:" messages.

Gzip'ed log file attached.

Thanks,
Niall.
From: Lawrence Mitchell <wence@gmx.li>
Sent: Thursday 1 August 2019 15:53
To: Madden, Niall <niall.madden@nuigalway.ie>
Cc: Firedrake Project <firedrake@imperial.ac.uk>
Subject: Re: [firedrake] Installing firedrake on a HPC system without package manager
 
Hi Niall, (adding in firedrake cc again)

> On 1 Aug 2019, at 15:43, Madden, Niall <niall.madden@nuigalway.ie> wrote:
>
> Hi Lawrence,
> Thanks for the swift reply. I don't want to reply to the whole list with too much specific information on the system

Please do reply with the list. That way others who might be able to help will see things.


>> Can you give us some details on this system? Is it a cray, for example?
> The system is ICHEC's "kay" - Ireland's national supercomputer for academic purposes:
> https://www.ichec.ie/about/infrastructure/kay
>
>> Normally we find when building on supercomputers (contrary to the one-shot "firedrake-install" unfortunately),
>> that it is best to build PETSc by hand. One then sets the appropriate environment variables and runs
>> firedrake-install --honour-petsc-dir. This might be the best step forward.
>
> Thanks for the tip. I tried compiling PETSc myself:
>
> git clone https://github.com/firedrakeproject/petsc.git  (but should I be cloning fromhttps://bitbucket.org/petsc/petsc ?)

No, cloning from the firedrakeproject "fork" is correct. We keep a (reasonably up to date) version of petsc that is known good with all Firedrake components there: it isn't otherwise special.

> ./configure --with-c2html=0 --download-chaco --download-metis --download-parmetis --download-scalapack --download-hypre --download-mumps --download-netcdf --download-hdf5 --download-pnetcdf --download-exodusii  --download-mpich --download-suitesparse=1 --download-petsc4py=1 --download-mpi4py=1 --download-fblaslapack  --download-zlib
>
> Then I set PETSC_DIR and PETSC_ARCH, and PYTHONPATH
>
> Then I try to install Firedrake with
> python3 firedrake-install --no-package-manager -v --disable-ssh --honour-petsc-dir --honour-pythonpath
>
> But it eventually fails compiling libspatialindex
> for reasons that seem to be related to the mpicxx , but it is a little opaque:
> undefined reference to `psm2_info_query@PSM2_1.0'

Can you attach the contents of firedrake-install.log (maybe gzip it since it is likely to be large).

Thanks,

Lawrence