Yeah - the 'tensorflow' dependency was listed in pyadjoint/requirements.txt. So I've restarted the build without '--install pyadjoint' option - and this works.. Satish ------ (firedrake) balay@asterix /home/balay/soft/test/firedrake/src/firedrake (master=) $ make alltest Building extension modules Linting firedrake codebase Linting firedrake test suite Linting firedrake scripts Running all regression tests ============================================================================================== test session starts =============================================================================================== platform linux -- Python 3.7.1, pytest-3.10.0, py-1.7.0, pluggy-0.8.0 benchmark: 3.1.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /home/balay/soft/test/firedrake/src/firedrake, inifile: setup.cfg plugins: xdist-1.24.0, forked-0.2, benchmark-3.1.1 collected 4442 items tests/test_0init.py ... [ 0%] tests/test_tsfc_interface.py .......... [ 0%] tests/benchmarks/test_assembly_overheads.py .......................... [ 0%] tests/benchmarks/test_solver_overheads.py ............ [ 1%] tests/demos/test_demos_run.py ....s........s...... [ 1%] tests/extrusion/test_2d_cohomology.py ...... [ 1%] tests/extrusion/test_annulus.py .. [ 1%] tests/extrusion/test_assembly.py ......................... <still running> On Mon, 5 Nov 2018, Wallwork, Joe wrote:

Hi Satish,

Please could you try installing without pyadjoint? Just to see if the problem is with Firedrake or with Pyadjoint.

Best wishes,

Joe

_______________________________________________________________________________________________________________________________________________________________________________________________________ From: Balay, Satish Sent: 05 November 2018 17:16:43 To: firedrake@imperial.ac.uk Cc: Wallwork, Joe; Smith, Barry F. Subject: firedrake build failure.   Hi,

I'm attempted a build of firedrake - and get a failure.

Attaching the log.

thanks, Satish

-------------

balay@asterix /home/balay/soft $ python3 firedrake-install --install pyadjoint --no-package-manager Running firedrake-install --install pyadjoint --no-package-manager apt-get not found or disabled. Proceeding on the rash assumption that your compiled dependencies are in place. If this is not the case, please install the following and try again: * A C and C++ compiler (for example gcc/g++ or clang), GNU make * A Fortran compiler (for PETSc) * MPI * Blas and Lapack * Git, Mercurial * Python version >=3.5 * The Python headers * autoconf, automake, libtool * CMake * zlib Creating firedrake venv in '/home/balay/soft/firedrake'. Cloning firedrake

Successfully cloned repository firedrake Checking out branch master Successfully checked out branch master Cloning the dependencies of firedrake Cloning ufl

Successfully cloned repository ufl Checking out branch master Successfully checked out branch master Cloning fiat

Successfully cloned repository fiat Checking out branch master Successfully checked out branch master Cloning FInAT

Successfully cloned repository FInAT Checking out branch master Successfully checked out branch master Cloning tsfc

Successfully cloned repository tsfc Checking out branch master Successfully checked out branch master Cloning PyOP2

Successfully cloned repository PyOP2 Checking out branch master Successfully checked out branch master Cloning the dependencies of PyOP2 Cloning petsc

Successfully cloned repository petsc Checking out branch firedrake Successfully checked out branch firedrake Cloning petsc4py

Successfully cloned repository petsc4py Checking out branch firedrake Successfully checked out branch firedrake Cloning COFFEE

Successfully cloned repository COFFEE Checking out branch master Successfully checked out branch master Cloning pyadjoint

Successfully cloned repository pyadjoint Checking out branch master Successfully checked out branch master Cloning the dependencies of pyadjoint Installing petsc/ Downloading Eigen from 'https://bitbucket.org/eigen/eigen/get/3.3.3.tar.gz' to 'eigen-3.3.3.tgz' Checking Eigen tarball integrity Eigen tarball hash valid Building PETSc. Depending on your platform, may take between a few minutes and an hour or more to build! Installing pip dependencies for petsc4py No dependencies found. Skipping. Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for firedrake Installing pip dependencies for pyadjoint Traceback (most recent call last):   File "firedrake-install", line 1124, in <module>     pip_requirements(p)   File "firedrake-install", line 614, in pip_requirements     run_pip_install(["-r", "%s/requirements.txt" % package])   File "firedrake-install", line 597, in run_pip_install     check_call(pipinstall + pipargs)   File "firedrake-install", line 435, in check_call     log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())   File "/usr/lib64/python3.7/subprocess.py", line 389, in check_output     **kwargs).stdout   File "/usr/lib64/python3.7/subprocess.py", line 481, in run     output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/home/balay/soft/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '-vvv', '-r', 'pyadjoint/requirements.txt']' returned non-zero exit status 1.

Install log saved in firedrake-install.log balay@asterix /home/balay/soft

Dear Barry, Satish, Joe, it indeed looks like tensorflow (which is an optional dependency of pyadjoint) is not compatible with python 3.7 (which is what you both have), hence there are no wheels available and pip doesn't install it. Unfortunately, this failure is occurring because we unconditionally install all of the (potentially optional) requirements of pyadjoint. Let us think about how to fix this problem at our end. For now, as Joe says, you can run firedrake-install without the --pyadjoint flag. Hopefully by the time you need pyadjoint for the tutorial we will have a fix deployed via firedrake-update. Cheers, Lawrence

On Tue, 6 Nov 2018, Lawrence Mitchell wrote:

Dear all,

...

On 6 Nov 2018, at 10:17, Lawrence Mitchell <wence@gmx.li> wrote:

Dear Barry, Satish, Joe,

it indeed looks like tensorflow (which is an optional dependency of pyadjoint) is not compatible with python 3.7 (which is what you both have), hence there are no wheels available and pip doesn't install it.

Unfortunately, this failure is occurring because we unconditionally install all of the (potentially optional) requirements of pyadjoint.

Let us think about how to fix this problem at our end. For now, as Joe says, you can run firedrake-install without the --pyadjoint flag.

Hopefully by the time you need pyadjoint for the tutorial we will have a fix deployed via firedrake-update.

I believe I have now fixed this.

If you have a firedrake installation that you generated without "--install pyadjoint" you should be able to update it to include pyadjoint with:

firedrake-update --install pyadjoint

(after activating the firedrake virtual env)

If you're starting from scratch, I believe "firedrake-install --install pyadjoint" should now work.

Please get in touch if there are any problems.

Thanks! It works now. Satish ------------- (firedrake) balay@asterix /home/balay/soft $ firedrake-update --install pyadjoint Running /home/balay/soft/test/firedrake/bin/firedrake-update --install pyadjoint Updating the git repository for firedrake Creating firedrake-update script. Running ../bin/firedrake-update --no-update-script --install pyadjoint apt-get not found or disabled. Proceeding on the rash assumption that your compiled dependencies are in place. If this is not the case, please install the following and try again: * A C and C++ compiler (for example gcc/g++ or clang), GNU make * A Fortran compiler (for PETSc) * MPI * Blas and Lapack * Git, Mercurial * Python version >=3.5 * The Python headers * autoconf, automake, libtool * CMake * zlib Finding dependencies of PyOP2 Finding dependencies of firedrake Finding dependencies of pyadjoint Updating the git repository for petsc Updating the git repository for petsc4py Updating the git repository for COFFEE Updating the git repository for ufl Updating the git repository for fiat Updating the git repository for FInAT Updating the git repository for tsfc Updating the git repository for PyOP2 Updating the git repository for pyadjoint pyadjoint missing, cloning anew. Cloning pyadjoint Successfully cloned repository pyadjoint Checking out branch master Successfully checked out branch master Updating the git repository for firedrake Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for pyadjoint Installing pip dependencies for firedrake Installing h5py Updating the git repository for h5py No need to rebuild h5py Installing libspatialindex Updating the git repository for libspatialindex No need to rebuild libspatialindex Cleaning COFFEE Installing COFFEE/ Cleaning ufl Installing ufl/ Cleaning fiat Installing fiat/ Cleaning FInAT Installing FInAT/ Cleaning tsfc Installing tsfc/ Cleaning pyadjoint Installing pyadjoint/ Cleaning PyOP2 Installing PyOP2/ Cleaning firedrake Installing firedrake/ Configuration saved to configuration.json Clearing just in time compilation caches. Removing cached TSFC kernels from /home/balay/soft/test/firedrake/.cache/tsfc Removing 6827 cached libraries from /home/balay/soft/test/firedrake/.cache/pyop2 Successfully updated Firedrake. Update log saved in /home/balay/soft/test/firedrake/src/firedrake-update.log (firedrake) balay@asterix /home/balay/soft $ cd firedrake/src/firedrake (firedrake) balay@asterix /home/balay/soft/firedrake/src/firedrake (master=) $ make alltest Building extension modules Linting firedrake codebase Linting firedrake test suite Linting firedrake scripts Running all regression tests ============================================================================= test session starts ============================================================================== platform linux -- Python 3.7.1, pytest-3.10.0, py-1.7.0, pluggy-0.8.0 benchmark: 3.1.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /home/balay/soft/test/firedrake/src/firedrake, inifile: setup.cfg plugins: xdist-1.24.0, forked-0.2, benchmark-3.1.1 collected 4442 items tests/test_0init.py ... [ 0%] tests/test_tsfc_interface.py .......... [ 0%] tests/benchmarks/test_assembly_overheads.py ............