When you compiled PETSc by hand, which configure options did you use? We depend on PETSc having downloaded quite a few things for us, in particular including hdf5 . It looks like you didn’t build PETSc with that option. If you run: firedrake-install --show-petsc-options the installer will list the options it would have passed to PETSc’s configure. You should ensure that you at least pass those options. Regards, David From: "Madden, Niall" <niall.madden@nuigalway.ie> Date: Thursday, 22 August 2019 at 22:57 To: "Ham, David A" <david.ham@imperial.ac.uk>, Lawrence Mitchell <wence@gmx.li> Cc: "Sagiyama, Koki" <k.sagiyama@imperial.ac.uk>, firedrake <firedrake@imperial.ac.uk> Subject: Re: [firedrake] Installing firedrake on a HPC system without package manager Hi guys, Thanks for the replies. Lawrence:
Given you started with a clean conda environment, did you run into issues just running firedrake-install inside that? Yes. Fails when compiling Petsc. Since I can compile Petsc by hand, I tried that passed --honour-petsc-dir when trying to build firedrake. But then it fails to compile h5py The log reports "/MYOB/firedrake/src/h5py/h5py/api_compat.h(27): catastrophic error: cannot open source file "hdf5.h""
David:
One more piece of the puzzle for you: the activate script is part of Python venv. One of the first things that the install script does is create a Firedrake venv into which Firedrake is installed. It is the creation of the venv which creates the activate script. I see. Thanks.
I would reiterate what Lawrence says: installing by hand like this is very, very error-prone. So too is not installing by hand it seems. At least for my setup.
In particular, installing by hand and then hoping that the update script will work sounds very optimistic indeed. I share your pessimism. I hadn't expressed that hope, I just noted that that file hadn't been created.
I guess I could try building Python, as David suggested earlier. (Or going back to FEniCS, which installed without problems - but I do like quads :-) Niall. ________________________________ From: Ham, David A <david.ham@imperial.ac.uk> Sent: Thursday 22 August 2019 08:48 To: Lawrence Mitchell <wence@gmx.li>; Madden, Niall <niall.madden@nuigalway.ie> Cc: Sagiyama, Koki <k.sagiyama@imperial.ac.uk>; firedrake <firedrake@imperial.ac.uk> Subject: Re: [firedrake] Installing firedrake on a HPC system without package manager Hi Niall, One more piece of the puzzle for you: the activate script is part of Python venv. One of the first things that the install script does is create a Firedrake venv into which Firedrake is installed. It is the creation of the venv which creates the activate script. I would reiterate what Lawrence says: installing by hand like this is very, very error-prone. It would almost certainly be better to run the install script. In particular, installing by hand and then hoping that the update script will work sounds very optimistic indeed. Regards, David On 21/08/2019, 16:22, "Lawrence Mitchell" <wence@gmx.li> wrote: Hi Niall, On Wed, 21 Aug 2019 at 15:04, Madden, Niall <niall.madden@nuigalway.ie> wrote: > > Hi all, > I've returned to this task, and tried to follow David's suggestion > of building on a clean conda environment. This time, I used the intel compiler (mpiicc). > > Trying to follow the steps in the install script, we cloned > firedrake, ufl, fiat, PyOP2, COFFEE, loopy, petsc, supermesh, spatialindex, etc. > > Then we can successfully install all these. In particular, we can > configure/make/make install petsc > > Finally, we run, in firedrake, > python setup.py install > with LDSHARED, CFLAGS, CC, CXX, and prefix set appropriately. [...] > That is, if I try > echo "import firedrake" | python > I get > > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/MYOB/r/libs/firedrake/firedrake/__init__.py", line 5, in <module> > if "PETSC_DIR" in os.environ and not config["options"]["honour_petsc_dir"]: > TypeError: 'NoneType' object is not subscriptable [...] > I note that, the "activate" script is not in firedrake/bin > Neither, as it happens, is firedrake-update, but firedrake-clean, firedrake-install, and firedrake-zenodo are. This is an idiosyncrasy of the firedrake-install process. "firedrake-update" is actually just a copy of "firedrake-install" with a different name (so it knows it's in update mode). firedrake-{clean/install/zenodo} are created in the bin directory by setup.py, but firedrake-update is copied _by the firedrake-install process_. Hence it not existing because you did everything by hand. You can "fix" this by just copying firedrake-install to firedrake-update in the bin directory. The issue with the TypeError above is related. As well as creating the firedrake-update script, firedrake-install also writes a little configuration file that remembers the options you installed firedrake with (so that firedrake-update maintains them). Again, because you didn't run firedrake-install, this file doesn't exist. The relevant file to be created is `configuration.json` in the firedrake_configuration subdirectory of the firedrake source tree. Mine looks like: {"options": {"package_manager": false, "minimal_petsc": false, "mpicc": null, "mpicxx": null, "mpif90": null, "mpiexec": null, "disable_ssh": false, "honour_petsc_dir": true, "show_petsc_configure_options": false, "slepc": true, "packages": [], "honour_pythonpath": false, "petsc_int_type": "int32", "cache_dir": "/home/vtdb72/Documents/work/src/firedrake/.cache"}, "environment": {}, "additions": []} Editing this by hand is, however, rather error prone. Additionally, I'm not necessarily sure that firedrake-update will run as normal. Given you started with a clean conda environment, did you run into issues just running firedrake-install inside that? Thanks, Lawrence