Hi guys,
Thanks for the replies.

Lawrence:
>    Given you started with a clean conda environment, did you run into
>    issues just running firedrake-install inside that?
Yes. Fails when compiling Petsc.
Since I can compile Petsc by hand, I tried that passed --honour-petsc-dir when trying to build firedrake.
But then it fails to compile h5py
The log reports  "/MYOB/firedrake/src/h5py/h5py/api_compat.h(27): catastrophic error: cannot open source file "hdf5.h""

David:
>   One more piece of the puzzle for you: the activate script is part of Python venv. One of the first things that the install script does is
>   create a Firedrake venv into which Firedrake is installed. It is the creation of the venv which creates the activate script.
I see. Thanks.

> I would reiterate what Lawrence says: installing by hand like this is very, very error-prone.
So too is not installing by hand it seems. At least for my setup. 

> In particular, installing by hand and then hoping that the update script will work sounds very optimistic indeed.
I share your pessimism. I hadn't expressed that hope, I just noted that that file hadn't been created.

I guess I could try building Python, as David suggested earlier. (Or going back to FEniCS, which installed without problems - but I do like quads :-)

Niall.
From: Ham, David A <david.ham@imperial.ac.uk>
Sent: Thursday 22 August 2019 08:48
To: Lawrence Mitchell <wence@gmx.li>; Madden, Niall <niall.madden@nuigalway.ie>
Cc: Sagiyama, Koki <k.sagiyama@imperial.ac.uk>; firedrake <firedrake@imperial.ac.uk>
Subject: Re: [firedrake] Installing firedrake on a HPC system without package manager
 
Hi Niall,

One more piece of the puzzle for you: the activate script is part of Python venv. One of the first things that the install script does is create a Firedrake venv into which Firedrake is installed. It is the creation of the venv which creates the activate script.

I would reiterate what Lawrence says: installing by hand like this is very, very error-prone. It would almost certainly be better to run the install script. In particular, installing by hand and then hoping that the update script will work sounds very optimistic indeed.

Regards,

David

On 21/08/2019, 16:22, "Lawrence Mitchell" <wence@gmx.li> wrote:

    Hi Niall,
   
    On Wed, 21 Aug 2019 at 15:04, Madden, Niall <niall.madden@nuigalway.ie> wrote:
    >
    > Hi all,
    > I've returned to this task, and tried to follow David's suggestion
    > of building on a clean conda environment. This time, I used the intel compiler (mpiicc).
    >
    > Trying to follow the steps in the install script, we cloned
    > firedrake, ufl, fiat, PyOP2, COFFEE, loopy, petsc, supermesh, spatialindex, etc.
    >
    > Then we can successfully install all these. In particular, we can
    > configure/make/make install petsc
    >
    > Finally, we run, in firedrake,
    > python setup.py install
    > with LDSHARED, CFLAGS, CC, CXX, and prefix set appropriately.
   
    [...]
   
    > That is, if I try
    > echo "import firedrake" | python
    > I get
    >
    > Traceback (most recent call last):
    >   File "<stdin>", line 1, in <module>
    >   File "/MYOB/r/libs/firedrake/firedrake/__init__.py", line 5, in <module>
    >     if "PETSC_DIR" in os.environ and not config["options"]["honour_petsc_dir"]:
    > TypeError: 'NoneType' object is not subscriptable
   
    [...]
   
    > I note that, the "activate" script is not in firedrake/bin
    > Neither, as it happens, is  firedrake-update, but firedrake-clean, firedrake-install, and firedrake-zenodo are.
   
    This is an idiosyncrasy of the firedrake-install process.
    "firedrake-update" is actually just a copy of "firedrake-install" with
    a different name (so it knows it's in update mode).
    firedrake-{clean/install/zenodo} are created in the bin directory by
    setup.py, but firedrake-update is copied _by the firedrake-install
    process_. Hence it not existing because you did everything by hand.
    You can "fix" this by just copying firedrake-install to
    firedrake-update in the bin directory.
   
    The issue with the TypeError above is related. As well as creating the
    firedrake-update script, firedrake-install also writes a little
    configuration file that remembers the options you installed firedrake
    with (so that firedrake-update maintains them). Again, because you
    didn't run firedrake-install, this file doesn't exist.
   
    The relevant file to be created is `configuration.json` in the
    firedrake_configuration subdirectory of the firedrake source tree.
    Mine looks like:
   
    {"options": {"package_manager": false,
                 "minimal_petsc": false,
                 "mpicc": null,
                 "mpicxx": null,
                 "mpif90": null,
                 "mpiexec": null,
                 "disable_ssh": false,
                 "honour_petsc_dir": true,
                 "show_petsc_configure_options": false,
                 "slepc": true,
                 "packages": [],
                 "honour_pythonpath": false,
                 "petsc_int_type": "int32",
                 "cache_dir": "/home/vtdb72/Documents/work/src/firedrake/.cache"},
     "environment": {}, "additions": []}
   
    Editing this by hand is, however, rather error prone. Additionally,
    I'm not necessarily sure that firedrake-update will run as normal.
    Given you started with a clean conda environment, did you run into
    issues just running firedrake-install inside that?
   
    Thanks,
   
    Lawrence