When you compiled PETSc by hand, which configure options did you use? We depend on PETSc having downloaded quite a few things for us, in particular including hdf5 . It looks like you didn’t build
PETSc with that option.
If you run:
firedrake-install --show-petsc-options
the installer will list the options it would have passed to PETSc’s configure. You should ensure that you at least pass those options.
Regards,
David
From:
"Madden, Niall" <niall.madden@nuigalway.ie>
Date: Thursday, 22 August 2019 at 22:57
To: "Ham, David A" <david.ham@imperial.ac.uk>, Lawrence Mitchell <wence@gmx.li>
Cc: "Sagiyama, Koki" <k.sagiyama@imperial.ac.uk>, firedrake <firedrake@imperial.ac.uk>
Subject: Re: [firedrake] Installing firedrake on a HPC system without package manager
Hi guys,
Thanks for the replies.
Lawrence:
> Given you started with a clean conda environment, did you run into
> issues just running firedrake-install inside that?
Yes. Fails when compiling Petsc.
Since I can compile Petsc by hand, I tried that passed --honour-petsc-dir when trying to build firedrake.
But then it fails to compile h5py
The log reports "/MYOB/firedrake/src/h5py/h5py/api_compat.h(27): catastrophic error: cannot open source file "hdf5.h""
David:
> One more piece of the puzzle for you: the activate script is part of Python venv. One of the first things that the install script
does is
> create a Firedrake venv into which Firedrake is installed. It is the creation of the venv which creates the activate script.
I see. Thanks.
> I would reiterate what Lawrence says: installing by hand like this is very, very error-prone.
So too is not installing by hand it seems. At least for my setup.
> In particular, installing by hand and then hoping that the update script will work sounds very optimistic indeed.
I share your pessimism. I hadn't expressed that hope, I just noted that that file hadn't been created.
I guess I could try building Python, as David suggested earlier. (Or going back to FEniCS, which installed without problems - but I do like quads :-)
Niall.
From: Ham, David A <david.ham@imperial.ac.uk>
Sent: Thursday 22 August 2019 08:48
To: Lawrence Mitchell <wence@gmx.li>; Madden, Niall <niall.madden@nuigalway.ie>
Cc: Sagiyama, Koki <k.sagiyama@imperial.ac.uk>; firedrake <firedrake@imperial.ac.uk>
Subject: Re: [firedrake] Installing firedrake on a HPC system without package manager
Hi Niall,
One more piece of the puzzle for you: the activate script is part of Python venv. One of the first things that the install script does is create a Firedrake venv into which Firedrake is installed. It is the creation of the venv which creates the activate script.
I would reiterate what Lawrence says: installing by hand like this is very, very error-prone. It would almost certainly be better to run the install script. In particular, installing by hand and then hoping that the update script will work sounds very optimistic
indeed.
Regards,
David
On 21/08/2019, 16:22, "Lawrence Mitchell" <wence@gmx.li> wrote:
Hi Niall,
On Wed, 21 Aug 2019 at 15:04, Madden, Niall <niall.madden@nuigalway.ie> wrote:
>
> Hi all,
> I've returned to this task, and tried to follow David's suggestion
> of building on a clean conda environment. This time, I used the intel compiler (mpiicc).
>
> Trying to follow the steps in the install script, we cloned
> firedrake, ufl, fiat, PyOP2, COFFEE, loopy, petsc, supermesh, spatialindex, etc.
>
> Then we can successfully install all these. In particular, we can
> configure/make/make install petsc
>
> Finally, we run, in firedrake,
> python setup.py install
> with LDSHARED, CFLAGS, CC, CXX, and prefix set appropriately.
[...]
> That is, if I try
> echo "import firedrake" | python
> I get
>
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/MYOB/r/libs/firedrake/firedrake/__init__.py", line 5, in <module>
> if "PETSC_DIR" in os.environ and not config["options"]["honour_petsc_dir"]:
> TypeError: 'NoneType' object is not subscriptable
[...]
> I note that, the "activate" script is not in firedrake/bin
> Neither, as it happens, is firedrake-update, but firedrake-clean, firedrake-install, and firedrake-zenodo are.
This is an idiosyncrasy of the firedrake-install process.
"firedrake-update" is actually just a copy of "firedrake-install" with
a different name (so it knows it's in update mode).
firedrake-{clean/install/zenodo} are created in the bin directory by
setup.py, but firedrake-update is copied _by the firedrake-install
process_. Hence it not existing because you did everything by hand.
You can "fix" this by just copying firedrake-install to
firedrake-update in the bin directory.
The issue with the TypeError above is related. As well as creating the
firedrake-update script, firedrake-install also writes a little
configuration file that remembers the options you installed firedrake
with (so that firedrake-update maintains them). Again, because you
didn't run firedrake-install, this file doesn't exist.
The relevant file to be created is `configuration.json` in the
firedrake_configuration subdirectory of the firedrake source tree.
Mine looks like:
{"options": {"package_manager": false,
"minimal_petsc": false,
"mpicc": null,
"mpicxx": null,
"mpif90": null,
"mpiexec": null,
"disable_ssh": false,
"honour_petsc_dir": true,
"show_petsc_configure_options": false,
"slepc": true,
"packages": [],
"honour_pythonpath": false,
"petsc_int_type": "int32",
"cache_dir": "/home/vtdb72/Documents/work/src/firedrake/.cache"},
"environment": {}, "additions": []}
Editing this by hand is, however, rather error prone. Additionally,
I'm not necessarily sure that firedrake-update will run as normal.
Given you started with a clean conda environment, did you run into
issues just running firedrake-install inside that?
Thanks,
Lawrence