Re: [firedrake] Installation with my own PETSc repository
Line 5406 _configtest.c:2:10: fatal error: 'mpi.h' file not found On Wed, 31 Oct 2018 at 20:27, Matthew Knepley <knepley@gmail.com> wrote:
Thanks folks! I made it farther:
:/PETSc3/fem$ ~/MacSoftware/bin/python3 ./firedrake-install --honour-petsc-dir --verbose --no-package-manager
Running ./firedrake-install --honour-petsc-dir --verbose --no-package-manager
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Creating firedrake venv in '/PETSc3/fem/firedrake'.
Cloning firedrake
Successfully cloned repository firedrake
Checking out branch master
Successfully checked out branch master
Cloning the dependencies of firedrake
Cloning ufl
Successfully cloned repository ufl
Checking out branch master
Successfully checked out branch master
Cloning fiat
Successfully cloned repository fiat
Checking out branch master
Successfully checked out branch master
Cloning FInAT
Successfully cloned repository FInAT
Checking out branch master
Successfully checked out branch master
Cloning tsfc
Successfully cloned repository tsfc
Checking out branch master
Successfully checked out branch master
Cloning PyOP2
Successfully cloned repository PyOP2
Checking out branch master
Successfully checked out branch master
Cloning the dependencies of PyOP2
Using existing petsc installation
Cloning petsc4py
Successfully cloned repository petsc4py
Checking out branch firedrake
Successfully checked out branch firedrake
Cloning COFFEE
Successfully cloned repository COFFEE
Checking out branch master
Successfully checked out branch master
Installing pip dependencies for petsc4py
No dependencies found. Skipping.
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Traceback (most recent call last):
File "./firedrake-install", line 1124, in <module>
pip_requirements(p)
File "./firedrake-install", line 607, in pip_requirements
run_pip_install(["-r", "%s/requirements-ext.txt" % package])
File "./firedrake-install", line 597, in run_pip_install
check_call(pipinstall + pipargs)
File "./firedrake-install", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '-vvv', '-r', 'PyOP2/requirements-ext.txt']' returned non-zero exit status 1.
Install log saved in firedrake-install.log
1:/PETSc3/fem$ vi firedrake-install.log
It looks like mpi4py is failing, but I do not know where to look for the actual error message. Log attached.
Thanks,
Matt
On Tue, Oct 30, 2018 at 12:51 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 30 Oct 2018, at 16:30, Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Oct 26, 2018 at 11:26 AM Matthew Knepley <knepley@gmail.com> wrote: Dear Firedrake,
Long-time listener, first time mailer.
I would like to build using my own repository for PETSc. I am reading this page:
https://www.firedrakeproject.org/download.html
but it is not clear to me how to tell the installer to use mine, rather than install its own. Do I just point to it using PETSC_DIR?
Its not going to install itself by osmosis.
Aha, you're not subscribed and people don't automatically CC. You would normally get a bounce notification asking you to subscribe, but I think someone in the past (probably me) added you to a list of addresses that get accepted anyway.
Answers from David:
Nearly: there’s a small piece of idiot-proofing to prevent the unwary new user from accidentally using the wrong PETSc. You need to pass –honour-petsc-dir (note UK spelling) to firedrake-install
You also need to check the install script to make sure your petsc has our minimal set of options. I apologise that you have to RTFS for that. From memory, the absolute minimum is that you need –download-hdf5 and –download-eigen
Nicolas said:
You need to define PETSC_DIR and PETSC_ARCH, then run the install
script with the --honour-petsc-dir option.
If you call the script with --help, you may also discover a couple of
options you might want to use (or not) such as --disable-ssh or --no-package-manager
And Miklos pointed out:
You don’t actually need to read the script,
firedrake-install --show-petsc-configure-options
tells you which options the Firedrake install script passes to PETSc’s configure script. It is normally safe add things, but removing things may result in harmless test failures (e.g. some solver packages), or in broken Firedrake functionality.
This last option doesn't exit the installer after showing them, so it's not quite as useful as it seems.
FWIW, we by default build with:
--download-chaco --download-metis --download-parmetis --download-scalapack --download-hypre --download-mumps --with-zlib --download-netcdf --download-hdf5 --download-pnetcdf --download-exodusii --with-debugging=0 --with-shared-libraries=1
Cheers,
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On 31 Oct 2018, at 22:34, Andrew McRae <andrew.mcrae@physics.ox.ac.uk> wrote:
Line 5406 _configtest.c:2:10: fatal error: 'mpi.h' file not found
A bit more detail. mpi4py's setup.py attempts to determine what compilers to use to compile and link MPI programs. It looks like it failed to do so for you somehow:
copying src/mpi4py/include/mpi4py/mpi4py.i -> build/lib.macosx-10.5-x86_64-3.6/mpi4py/include/mpi4py copying src/mpi4py/include/mpi4py/mpi.pxi -> build/lib.macosx-10.5-x86_64-3.6/mpi4py/include/mpi4py running build_clib MPI configuration: [mpi] from 'mpi.cfg' checking for library 'lmpe' ...
If I run "pip install -vvv mpi4py" I see:
running build_clib MPI configuration: [mpi] from 'mpi.cfg' MPI C compiler: /usr/local/bin/mpicc MPI C++ compiler: /usr/local/bin/mpicxx MPI F compiler: /usr/local/bin/mpifort MPI F90 compiler: /usr/local/bin/mpif90 MPI F77 compiler: /usr/local/bin/mpif77 checking for library 'lmpe' ...
I notice that your environment sets "mpiexec" to something in $PETSC_DIR/$PETSC_ARCH/bin But that directory is not in your PATH. Is that also where mpicc etc.. live? You can say: MPICC=...; MPICXX=...; pip install -vvv mpi4py to see if that works. In theory you can say "firedrake-install --mpicc=/path/to/mpicc --mpicxx=... --mpif90=..." but I am not 100% confident that that sets things up correctly for the mpi4py installation (only the PETSc build). Since (unfortunately) recovering from a part-failed install is a "remove and try again" process, to speed up pinning down the issue can we try: $ python3 -m venv foo $ . foo/bin/activate $ pip install -vvv --no-binary mpi4py mpi4py And see what happens? Cheers, Lawrence
On Thu, Nov 1, 2018 at 5:25 AM Lawrence Mitchell <wence@gmx.li> wrote:
On 31 Oct 2018, at 22:34, Andrew McRae <andrew.mcrae@physics.ox.ac.uk> wrote:
Line 5406 _configtest.c:2:10: fatal error: 'mpi.h' file not found
A bit more detail. mpi4py's setup.py attempts to determine what compilers to use to compile and link MPI programs.
It looks like it failed to do so for you somehow:
copying src/mpi4py/include/mpi4py/mpi4py.i ->
build/lib.macosx-10.5-x86_64-3.6/mpi4py/include/mpi4py
copying src/mpi4py/include/mpi4py/mpi.pxi ->
build/lib.macosx-10.5-x86_64-3.6/mpi4py/include/mpi4py
running build_clib MPI configuration: [mpi] from 'mpi.cfg' checking for library 'lmpe' ...
If I run "pip install -vvv mpi4py" I see:
running build_clib MPI configuration: [mpi] from 'mpi.cfg' MPI C compiler: /usr/local/bin/mpicc MPI C++ compiler: /usr/local/bin/mpicxx MPI F compiler: /usr/local/bin/mpifort MPI F90 compiler: /usr/local/bin/mpif90 MPI F77 compiler: /usr/local/bin/mpif77 checking for library 'lmpe' ...
I notice that your environment sets "mpiexec" to something in $PETSC_DIR/$PETSC_ARCH/bin
But that directory is not in your PATH. Is that also where mpicc etc.. live?
You can say:
MPICC=...; MPICXX=...; pip install -vvv mpi4py to see if that works.
In theory you can say "firedrake-install --mpicc=/path/to/mpicc --mpicxx=... --mpif90=..." but I am not 100% confident that that sets things up correctly for the mpi4py installation (only the PETSc build).
Since (unfortunately) recovering from a part-failed install is a "remove and try again" process, to speed up pinning down the issue can we try:
$ python3 -m venv foo $ . foo/bin/activate $ pip install -vvv --no-binary mpi4py mpi4py
And see what happens?
The env variables worked. I like that trick for building just one section. Restarting the full build... Thanks, Matt
Cheers,
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On Thu, Nov 1, 2018 at 9:32 PM Matthew Knepley <knepley@gmail.com> wrote:
On Thu, Nov 1, 2018 at 5:25 AM Lawrence Mitchell <wence@gmx.li> wrote:
On 31 Oct 2018, at 22:34, Andrew McRae <andrew.mcrae@physics.ox.ac.uk> wrote:
Line 5406 _configtest.c:2:10: fatal error: 'mpi.h' file not found
A bit more detail. mpi4py's setup.py attempts to determine what compilers to use to compile and link MPI programs.
It looks like it failed to do so for you somehow:
copying src/mpi4py/include/mpi4py/mpi4py.i ->
build/lib.macosx-10.5-x86_64-3.6/mpi4py/include/mpi4py
copying src/mpi4py/include/mpi4py/mpi.pxi ->
build/lib.macosx-10.5-x86_64-3.6/mpi4py/include/mpi4py
running build_clib MPI configuration: [mpi] from 'mpi.cfg' checking for library 'lmpe' ...
If I run "pip install -vvv mpi4py" I see:
running build_clib MPI configuration: [mpi] from 'mpi.cfg' MPI C compiler: /usr/local/bin/mpicc MPI C++ compiler: /usr/local/bin/mpicxx MPI F compiler: /usr/local/bin/mpifort MPI F90 compiler: /usr/local/bin/mpif90 MPI F77 compiler: /usr/local/bin/mpif77 checking for library 'lmpe' ...
I notice that your environment sets "mpiexec" to something in $PETSC_DIR/$PETSC_ARCH/bin
But that directory is not in your PATH. Is that also where mpicc etc.. live?
You can say:
MPICC=...; MPICXX=...; pip install -vvv mpi4py to see if that works.
In theory you can say "firedrake-install --mpicc=/path/to/mpicc --mpicxx=... --mpif90=..." but I am not 100% confident that that sets things up correctly for the mpi4py installation (only the PETSc build).
Since (unfortunately) recovering from a part-failed install is a "remove and try again" process, to speed up pinning down the issue can we try:
$ python3 -m venv foo $ . foo/bin/activate $ pip install -vvv --no-binary mpi4py mpi4py
And see what happens?
The env variables worked. I like that trick for building just one section. Restarting the full build...
Well crap. h5py failed, and it does not look like a simple env fix. It is complaining about an SSL certificate. Is there a way to point to something that uses HTTP? Thanks, Matt
Thanks,
Matt
Cheers,
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On 2 Nov 2018, at 01:37, Matthew Knepley <knepley@gmail.com> wrote:
Well crap. h5py failed, and it does not look like a simple env fix. It is complaining about an SSL certificate. Is there a way to point to something that uses HTTP?
Hmmph, this is very odd. Because that failed checking the certificate for a host it had previously downloaded from (and passed the certificate checks). This failed at a state late enough in the game that we can try building "in place" in the virtualenv: $ . firedrake/bin/activate $ cd firedrake/src/h5py $ export CC=mpicc # I assume you got PETSc to build hdf5 $ export HDF5_DIR=$PETSC_DIR/$PETSC_ARCH $ export HDF5_MPI=ON $ pip install . Lawrence
On Fri, Nov 2, 2018 at 12:54 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 01:37, Matthew Knepley <knepley@gmail.com> wrote:
Well crap. h5py failed, and it does not look like a simple env fix. It is complaining about an SSL certificate. Is there a way to point to something that uses HTTP?
Hmmph, this is very odd. Because that failed checking the certificate for a host it had previously downloaded from (and passed the certificate checks).
This failed at a state late enough in the game that we can try building "in place" in the virtualenv:
$ . firedrake/bin/activate $ cd firedrake/src/h5py $ export CC=mpicc # I assume you got PETSc to build hdf5 $ export HDF5_DIR=$PETSC_DIR/$PETSC_ARCH $ export HDF5_MPI=ON $ pip install .
Okay, this line pip install . fails with (firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Complete output from command python setup.py egg_info: Download error on https://pypi.org/simple/pkgconfig/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833) -- Some packages may not be found! Couldn't find index page for 'pkgconfig' (maybe misspelled?) Download error on https://pypi.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833) -- Some packages may not be found! No local packages or working download links found for pkgconfig Traceback (most recent call last): File "<string>", line 1, in <module> File "/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-build-vy7d9wuj/setup.py", line 168, in <module> cmdclass = CMDCLASS, File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in setup _install_setup_requires(attrs) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/__init__.py", line 134, in _install_setup_requires dist.fetch_build_eggs(dist.setup_requires) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs replace_conflicting=True, File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 777, in resolve replace_conflicting=replace_conflicting File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1060, in best_match return self.obtain(req, installer) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1072, in obtain return installer(requirement) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/dist.py", line 581, in fetch_build_egg return cmd.easy_install(req) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 670, in easy_install raise DistutilsError(msg) distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pkgconfig') ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-build-vy7d9wuj/ but I messed it up initially using pip install h5py which succeeds with (firedrake) 1firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install h5py Collecting h5py Downloading https://files.pythonhosted.org/packages/44/81/50a0560aac57a33c2a624d9e160735... (6.0MB) 100% |████████████████████████████████| 6.1MB 677kB/s Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py) (1.11.0) Requirement already satisfied: numpy>=1.7 in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py) (1.15.3) Installing collected packages: h5py Successfully installed h5py-2.8.0 Can I keep going from here? How do I restart the Firedrake install? Thanks, Matt
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On 2 Nov 2018, at 16:59, Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 12:54 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 01:37, Matthew Knepley <knepley@gmail.com> wrote:
Well crap. h5py failed, and it does not look like a simple env fix. It is complaining about an SSL certificate. Is there a way to point to something that uses HTTP?
Hmmph, this is very odd. Because that failed checking the certificate for a host it had previously downloaded from (and passed the certificate checks).
This failed at a state late enough in the game that we can try building "in place" in the virtualenv:
$ . firedrake/bin/activate $ cd firedrake/src/h5py $ export CC=mpicc # I assume you got PETSc to build hdf5 $ export HDF5_DIR=$PETSC_DIR/$PETSC_ARCH $ export HDF5_MPI=ON $ pip install .
Okay, this line
pip install .
fails with
(firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Complete output from command python setup.py egg_info: Download error on https://pypi.org/simple/pkgconfig/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833) -- Some packages may not be found! Couldn't find index page for 'pkgconfig' (maybe misspelled?) Download error on https://pypi.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833) -- Some packages may not be found! No local packages or working download links found for pkgconfig Traceback (most recent call last): File "<string>", line 1, in <module> File "/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-build-vy7d9wuj/setup.py", line 168, in <module> cmdclass = CMDCLASS, File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in setup _install_setup_requires(attrs) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/__init__.py", line 134, in _install_setup_requires dist.fetch_build_eggs(dist.setup_requires) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs replace_conflicting=True, File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 777, in resolve replace_conflicting=replace_conflicting File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1060, in best_match return self.obtain(req, installer) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1072, in obtain return installer(requirement) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/dist.py", line 581, in fetch_build_egg return cmd.easy_install(req) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 670, in easy_install raise DistutilsError(msg) distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pkgconfig')
---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-build-vy7d9wuj/
OK, this is the same error as before, but we can hopefully debug it in isolation. It's failing certificate checks on downloading pkgconfig. So in the activated virtualenv, try: $ pip install -vvv pkgconfig And see what we get.
but I messed it up initially using
pip install h5py
which succeeds with
(firedrake) 1firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install h5py Collecting h5py Downloading https://files.pythonhosted.org/packages/44/81/50a0560aac57a33c2a624d9e160735... (6.0MB) 100% |████████████████████████████████| 6.1MB 677kB/s Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py) (1.11.0) Requirement already satisfied: numpy>=1.7 in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py) (1.15.3) Installing collected packages: h5py Successfully installed h5py-2.8.0
Can I keep going from here? How do I restart the Firedrake install?
This won't work, unfortunately. Or at least, I don't trust it will work. The h5py wheel pacakges come with prebuilt HDF5 libraries inside them and that will almost certainly break things at runtime, because the compiled parts of Firedrake will now be linked against a different HDF5 from the PETSc one. And this gets messy. Lawrence
On Fri, Nov 2, 2018 at 1:11 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 16:59, Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 12:54 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 01:37, Matthew Knepley <knepley@gmail.com> wrote:
Well crap. h5py failed, and it does not look like a simple env fix. It is complaining about an SSL certificate. Is there a way to point to something that uses HTTP?
Hmmph, this is very odd. Because that failed checking the certificate for a host it had previously downloaded from (and passed the certificate checks).
This failed at a state late enough in the game that we can try building "in place" in the virtualenv:
$ . firedrake/bin/activate $ cd firedrake/src/h5py $ export CC=mpicc # I assume you got PETSc to build hdf5 $ export HDF5_DIR=$PETSC_DIR/$PETSC_ARCH $ export HDF5_MPI=ON $ pip install .
Okay, this line
pip install .
fails with
(firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Complete output from command python setup.py egg_info: Download error on https://pypi.org/simple/pkgconfig/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833) -- Some packages may not be found! Couldn't find index page for 'pkgconfig' (maybe misspelled?) Download error on https://pypi.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833) -- Some packages may not be found! No local packages or working download links found for pkgconfig Traceback (most recent call last): File "<string>", line 1, in <module> File "/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-build-vy7d9wuj/setup.py", line 168, in <module> cmdclass = CMDCLASS, File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in setup _install_setup_requires(attrs) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/__init__.py", line 134, in _install_setup_requires dist.fetch_build_eggs(dist.setup_requires) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs replace_conflicting=True, File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 777, in resolve replace_conflicting=replace_conflicting File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1060, in best_match return self.obtain(req, installer) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1072, in obtain return installer(requirement) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/dist.py", line 581, in fetch_build_egg return cmd.easy_install(req) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 670, in easy_install raise DistutilsError(msg) distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pkgconfig')
---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-build-vy7d9wuj/
OK, this is the same error as before, but we can hopefully debug it in isolation. It's failing certificate checks on downloading pkgconfig.
So in the activated virtualenv, try:
$ pip install -vvv pkgconfig
And see what we get.
That seems to have worked. Looks like it had stale URLs? (firedrake) :/PETSc3/fem/firedrake$ pip install -vvv pkgconfig Created temporary directory: /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-ephem-wheel-cache-fjx4plz3 Created temporary directory: /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-tracker-f08je7xv Created requirements tracker '/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-tracker-f08je7xv' Created temporary directory: /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-install-ftkb2slk Collecting pkgconfig 1 location(s) to search for versions of pkgconfig: * https://pypi.org/simple/pkgconfig/ Getting page https://pypi.org/simple/pkgconfig/ Looking up "https://pypi.org/simple/pkgconfig/" in the cache Request header has "max_age" as 0, cache bypassed Starting new HTTPS connection (1): pypi.org:443 https://pypi.org:443 "GET /simple/pkgconfig/ HTTP/1.1" 200 1261 Updating cache with response from "https://pypi.org/simple/pkgconfig/" Caching due to etag Analyzing links from page https://pypi.org/simple/pkgconfig/ Found link https://files.pythonhosted.org/packages/78/b5/b1ca52c3a4e2847f5408d776c8e513... (from https://pypi.org/simple/pkgconfig/), version: 0.1.0 Found link https://files.pythonhosted.org/packages/6f/0b/1d3498423cfcac1db48eca7fc867d4... (from https://pypi.org/simple/pkgconfig/), version: 1.0.0 Found link https://files.pythonhosted.org/packages/87/35/4af9634270c00e3411cf951b7e0ea7... (from https://pypi.org/simple/pkgconfig/), version: 1.1.0 Found link https://files.pythonhosted.org/packages/ec/1c/70f81f8400e767277c11770ce5eb38... (from https://pypi.org/simple/pkgconfig/), version: 1.2.0 Found link https://files.pythonhosted.org/packages/1f/90/4ce89b250a9fdf87a73d109a496cbe... (from https://pypi.org/simple/pkgconfig/), version: 1.2.1 Found link https://files.pythonhosted.org/packages/9d/ba/80910bbed2b4e646a6adab4474d2e5... (from https://pypi.org/simple/pkgconfig/), version: 1.2.2 Skipping link https://files.pythonhosted.org/packages/fe/1b/e36407744b0118efd020f0aceaa198... (from https://pypi.org/simple/pkgconfig/); unsupported archive format: .egg Found link https://files.pythonhosted.org/packages/b8/f9/0930d35b5035201865118cbb79cf4d... (from https://pypi.org/simple/pkgconfig/), version: 1.3.0 Found link https://files.pythonhosted.org/packages/94/ea/b6cc7007afb696d6a3c7bc0e62142f... (from https://pypi.org/simple/pkgconfig/), version: 1.3.1 Found link https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... (from https://pypi.org/simple/pkgconfig/) (requires-python:>=2.6, !=3.0.*, !=3.1.*, !=3.2.*), version: 1.4.0 Found link https://files.pythonhosted.org/packages/17/b7/b6e2b5798eeadb6bfbebf6c332c898... (from https://pypi.org/simple/pkgconfig/) (requires-python:>=2.6, !=3.0.*, !=3.1.*, !=3.2.*), version: 1.4.0 Using version 1.4.0 (newest of versions: 0.1.0, 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.3.0, 1.3.1, 1.4.0) Created temporary directory: /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-unpack-2acg6mhp Looking up " https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a..." in the cache No cache entry available Starting new HTTPS connection (1): files.pythonhosted.org:443 https://files.pythonhosted.org:443 "GET /packages/e1/8a/c205fad800317a59e8d218981a8a7a3a00ec69c7fd55eca58dd08e6b0a18/pkgconfig-1.4.0-py2.py3-none-any.whl HTTP/1.1" 200 7247 Downloading https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... Downloading from URL https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... (from https://pypi.org/simple/pkgconfig/) (requires-python:>=2.6, !=3.0.*, !=3.1.*, !=3.2.*) Ignoring unknown cache-control directive: immutable Updating cache with response from " https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... " Caching due to etag Added pkgconfig from https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... to build tracker '/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-tracker-f08je7xv' Removed pkgconfig from https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... from build tracker '/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-tracker-f08je7xv' Installing collected packages: pkgconfig Successfully installed pkgconfig-1.4.0 Cleaning up... Removed build tracker '/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-tracker-f08je7xv' I went on to cd src/h5py pip install . and it that worked (firedrake) :/PETSc3/fem/firedrake$ pushd src/h5py/ /PETSc3/fem/firedrake/src/h5py /PETSc3/fem/firedrake /PETSc3/fem (firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Requirement already satisfied: numpy>=1.7 in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.15.3) Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.11.0) Installing collected packages: h5py Found existing installation: h5py 2.8.0 Uninstalling h5py-2.8.0: Successfully uninstalled h5py-2.8.0 Running setup.py install for h5py ... done Successfully installed h5py-2.8.0.post0 How do I restart the whole Firedrake now? Thanks, Matt
but I messed it up initially using
pip install h5py
which succeeds with
(firedrake) 1firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install h5py Collecting h5py Downloading https://files.pythonhosted.org/packages/44/81/50a0560aac57a33c2a624d9e160735... (6.0MB) 100% |████████████████████████████████| 6.1MB 677kB/s Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py) (1.11.0) Requirement already satisfied: numpy>=1.7 in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py) (1.15.3) Installing collected packages: h5py Successfully installed h5py-2.8.0
Can I keep going from here? How do I restart the Firedrake install?
This won't work, unfortunately. Or at least, I don't trust it will work. The h5py wheel pacakges come with prebuilt HDF5 libraries inside them and that will almost certainly break things at runtime, because the compiled parts of Firedrake will now be linked against a different HDF5 from the PETSc one. And this gets messy.
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On 2 Nov 2018, at 17:16, Matthew Knepley <knepley@gmail.com> wrote:
I went on to
cd src/h5py pip install .
and it that worked
(firedrake) :/PETSc3/fem/firedrake$ pushd src/h5py/ /PETSc3/fem/firedrake/src/h5py /PETSc3/fem/firedrake /PETSc3/fem (firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Requirement already satisfied: numpy>=1.7 in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.15.3) Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.11.0) Installing collected packages: h5py Found existing installation: h5py 2.8.0 Uninstalling h5py-2.8.0: Successfully uninstalled h5py-2.8.0 Running setup.py install for h5py ... done Successfully installed h5py-2.8.0.post0
Great!
How do I restart the whole Firedrake now?
This is unsupported, but I think it should work: $ cp firedrake-install firedrake/bin/firedrake-update $ chmod u+x firedrake/bin/firedrake-update $ . firedrake/bin/activate $ rm -rf firedrake/src/libspatialindex $ firedrake-update Cheers, Lawrence
On Fri, Nov 2, 2018 at 1:20 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 17:16, Matthew Knepley <knepley@gmail.com> wrote:
I went on to
cd src/h5py pip install .
and it that worked
(firedrake) :/PETSc3/fem/firedrake$ pushd src/h5py/ /PETSc3/fem/firedrake/src/h5py /PETSc3/fem/firedrake /PETSc3/fem (firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Requirement already satisfied: numpy>=1.7 in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.15.3) Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.11.0) Installing collected packages: h5py Found existing installation: h5py 2.8.0 Uninstalling h5py-2.8.0: Successfully uninstalled h5py-2.8.0 Running setup.py install for h5py ... done Successfully installed h5py-2.8.0.post0
Great!
How do I restart the whole Firedrake now?
This is unsupported, but I think it should work:
$ cp firedrake-install firedrake/bin/firedrake-update $ chmod u+x firedrake/bin/firedrake-update $ . firedrake/bin/activate $ rm -rf firedrake/src/libspatialindex $ firedrake-update
I get (firedrake) :/PETSc3/fem$ cp firedrake-install firedrake/bin/firedrake-update (firedrake) :/PETSc3/fem$ chmod u+x firedrake/bin/firedrake-update (firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex (firedrake) :/PETSc3/fem$ firedrake-update Running /PETSc3/fem/firedrake/bin/firedrake-update Traceback (most recent call last): File "/PETSc3/fem/firedrake/bin/firedrake-update", line 277, in <module> import firedrake_configuration ModuleNotFoundError: No module named 'firedrake_configuration' Thanks, Matt Cheers,
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On Fri, Nov 2, 2018 at 1:27 PM Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 1:20 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 17:16, Matthew Knepley <knepley@gmail.com> wrote:
I went on to
cd src/h5py pip install .
and it that worked
(firedrake) :/PETSc3/fem/firedrake$ pushd src/h5py/ /PETSc3/fem/firedrake/src/h5py /PETSc3/fem/firedrake /PETSc3/fem (firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Requirement already satisfied: numpy>=1.7 in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.15.3) Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.11.0) Installing collected packages: h5py Found existing installation: h5py 2.8.0 Uninstalling h5py-2.8.0: Successfully uninstalled h5py-2.8.0 Running setup.py install for h5py ... done Successfully installed h5py-2.8.0.post0
Great!
How do I restart the whole Firedrake now?
This is unsupported, but I think it should work:
$ cp firedrake-install firedrake/bin/firedrake-update $ chmod u+x firedrake/bin/firedrake-update $ . firedrake/bin/activate $ rm -rf firedrake/src/libspatialindex $ firedrake-update
I get
(firedrake) :/PETSc3/fem$ cp firedrake-install firedrake/bin/firedrake-update
(firedrake) :/PETSc3/fem$ chmod u+x firedrake/bin/firedrake-update
(firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex
(firedrake) :/PETSc3/fem$ firedrake-update
Running /PETSc3/fem/firedrake/bin/firedrake-update
Traceback (most recent call last):
File "/PETSc3/fem/firedrake/bin/firedrake-update", line 277, in <module>
import firedrake_configuration
ModuleNotFoundError: No module named 'firedrake_configuration'
Also note that I have this module (firedrake) 1:/PETSc3/fem/firedrake$ ls src/firedrake/firedrake_configuration/ __init__.py but it has no configuration.json in it, so I think it would still fail if the import succeeds. Thanks, Matt Thanks,
Matt
Cheers,
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On 2 Nov 2018, at 17:27, Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 1:20 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 17:16, Matthew Knepley <knepley@gmail.com> wrote:
I went on to
cd src/h5py pip install .
and it that worked
(firedrake) :/PETSc3/fem/firedrake$ pushd src/h5py/ /PETSc3/fem/firedrake/src/h5py /PETSc3/fem/firedrake /PETSc3/fem (firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Requirement already satisfied: numpy>=1.7 in
/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.15.3)
Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.11.0) Installing collected packages: h5py Found existing installation: h5py 2.8.0 Uninstalling h5py-2.8.0: Successfully uninstalled h5py-2.8.0 Running setup.py install for h5py ... done Successfully installed h5py-2.8.0.post0
Great!
How do I restart the whole Firedrake now?
This is unsupported, but I think it should work:
$ cp firedrake-install firedrake/bin/firedrake-update $ chmod u+x firedrake/bin/firedrake-update $ . firedrake/bin/activate $ rm -rf firedrake/src/libspatialindex $ firedrake-update
I get
(firedrake) :/PETSc3/fem$ cp firedrake-install firedrake/bin/firedrake-update (firedrake) :/PETSc3/fem$ chmod u+x firedrake/bin/firedrake-update (firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex (firedrake) :/PETSc3/fem$ firedrake-update Running /PETSc3/fem/firedrake/bin/firedrake-update Traceback (most recent call last): File "/PETSc3/fem/firedrake/bin/firedrake-update", line 277, in <module> import firedrake_configuration ModuleNotFoundError: No module named 'firedrake_configuration'
Ah, ok, we have to do this manually then. First: $ cat > firedrake/src/firedrake/firedrake_configuration/configuration.json <<EOF {"options": {"package_manager": false, "minimal_petsc": false, "mpicc": "mpicc", "mpicxx": "mpicxx", "mpif90": "mpif90", "disable_ssh": false, "honour_petsc_dir": true, "show_petsc_configure_options": false, "slepc": false, "slope": false, "packages": [], "honour_pythonpath": false, "petsc_int_type": "int32", "cache_dir": "/PETSc3/fem/firedrake/.cache"}, "environment": {}, "additions": []} EOF This makes the configuration file firedrake-install would have done. Now let's pretend we installed the firedrake_configuration module: $ cat > firedrake/lib/python3.6/site-packages/firedrake.egg-link << EOF /PETSc3/fem/firedrake/src/firedrake . EOF Now try doing: $ python -c "import firedrake_configuration; print(firedrake_configuration.get_config())" If that works, try again with "firedrake-update" Cheers, Lawrence
On Fri, Nov 2, 2018 at 1:36 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 17:27, Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 1:20 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 17:16, Matthew Knepley <knepley@gmail.com> wrote:
I went on to
cd src/h5py pip install .
and it that worked
(firedrake) :/PETSc3/fem/firedrake$ pushd src/h5py/ /PETSc3/fem/firedrake/src/h5py /PETSc3/fem/firedrake /PETSc3/fem (firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Requirement already satisfied: numpy>=1.7 in
/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.15.3)
Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.11.0) Installing collected packages: h5py Found existing installation: h5py 2.8.0 Uninstalling h5py-2.8.0: Successfully uninstalled h5py-2.8.0 Running setup.py install for h5py ... done Successfully installed h5py-2.8.0.post0
Great!
How do I restart the whole Firedrake now?
This is unsupported, but I think it should work:
$ cp firedrake-install firedrake/bin/firedrake-update $ chmod u+x firedrake/bin/firedrake-update $ . firedrake/bin/activate $ rm -rf firedrake/src/libspatialindex $ firedrake-update
I get
(firedrake) :/PETSc3/fem$ cp firedrake-install firedrake/bin/firedrake-update (firedrake) :/PETSc3/fem$ chmod u+x firedrake/bin/firedrake-update (firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex (firedrake) :/PETSc3/fem$ firedrake-update Running /PETSc3/fem/firedrake/bin/firedrake-update Traceback (most recent call last): File "/PETSc3/fem/firedrake/bin/firedrake-update", line 277, in <module> import firedrake_configuration ModuleNotFoundError: No module named 'firedrake_configuration'
Ah, ok, we have to do this manually then.
First:
$ cat > firedrake/src/firedrake/firedrake_configuration/configuration.json <<EOF {"options": {"package_manager": false, "minimal_petsc": false, "mpicc": "mpicc", "mpicxx": "mpicxx", "mpif90": "mpif90", "disable_ssh": false, "honour_petsc_dir": true, "show_petsc_configure_options": false, "slepc": false, "slope": false, "packages": [], "honour_pythonpath": false, "petsc_int_type": "int32", "cache_dir": "/PETSc3/fem/firedrake/.cache"}, "environment": {}, "additions": []} EOF
This makes the configuration file firedrake-install would have done.
Now let's pretend we installed the firedrake_configuration module:
$ cat > firedrake/lib/python3.6/site-packages/firedrake.egg-link << EOF /PETSc3/fem/firedrake/src/firedrake . EOF
Now try doing:
$ python -c "import firedrake_configuration; print(firedrake_configuration.get_config())"
If that works, try again with "firedrake-update"
Okay, we get farther. I had to set PYTHONPATH in order to see the configuration file: (firedrake) :/PETSc3/fem$ export PYTHONPATH=$PWD/firedrake/src/firedrake (firedrake) :/PETSc3/fem$ python -c "import firedrake_configuration; print(firedrake_configuration.get_config())" {'options': {'package_manager': False, 'minimal_petsc': False, 'mpicc': 'mpicc', 'mpicxx': 'mpicxx', 'mpif90': 'mpif90', 'disable_ssh': False, 'honour_petsc_dir': True, 'show_petsc_configure_options': False, 'slepc': False, 'slope': False, 'packages': [], 'honour_pythonpath': False, 'petsc_int_type': 'int32', 'cache_dir': '/PETSc3/fem/firedrake/.cache'}, 'environment': {}, 'additions': []} (firedrake) :/PETSc3/fem$ firedrake_update -bash: firedrake_update: command not found (firedrake) 127:/PETSc3/fem$ firedrake-update Running /PETSc3/fem/firedrake/bin/firedrake-update The PYTHONPATH environment variable is set. This is probably an error. If you really want to use your own Python packages, please run again with the --honour-pythonpath option. Update log saved in firedrake-update.log (firedrake) 1:/PETSc3/fem$ firedrake-update --honour-pythonpath Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath Updating the git repository for firedrake Creating firedrake-update script. Running ../bin/firedrake-update --no-update-script --honour-pythonpath Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. Finding dependencies of PyOP2 Finding dependencies of firedrake Updating the git repository for petsc4py Updating the git repository for COFFEE Updating the git repository for ufl Updating the git repository for fiat Updating the git repository for FInAT Updating the git repository for tsfc Updating the git repository for PyOP2 Updating the git repository for firedrake Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for firedrake Installing h5py Updating the git repository for h5py No need to rebuild h5py Installing libspatialindex Cloning libspatialindex Successfully cloned repository libspatialindex Checking out branch master Successfully checked out branch master Traceback (most recent call last): File "../bin/firedrake-update", line 1207, in <module> build_and_install_libspatialindex() File "../bin/firedrake-update", line 808, in build_and_install_libspatialindex check_call(["./autogen.sh"]) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['./autogen.sh']' returned non-zero exit status 1. Update log saved in firedrake-update.log Here is the log. Thanks, Matt Cheers,
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On Fri, Nov 2, 2018 at 1:49 PM Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 1:36 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 17:27, Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 1:20 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 17:16, Matthew Knepley <knepley@gmail.com> wrote:
I went on to
cd src/h5py pip install .
and it that worked
(firedrake) :/PETSc3/fem/firedrake$ pushd src/h5py/ /PETSc3/fem/firedrake/src/h5py /PETSc3/fem/firedrake /PETSc3/fem (firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Requirement already satisfied: numpy>=1.7 in
/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.15.3)
Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.11.0) Installing collected packages: h5py Found existing installation: h5py 2.8.0 Uninstalling h5py-2.8.0: Successfully uninstalled h5py-2.8.0 Running setup.py install for h5py ... done Successfully installed h5py-2.8.0.post0
Great!
How do I restart the whole Firedrake now?
This is unsupported, but I think it should work:
$ cp firedrake-install firedrake/bin/firedrake-update $ chmod u+x firedrake/bin/firedrake-update $ . firedrake/bin/activate $ rm -rf firedrake/src/libspatialindex $ firedrake-update
I get
(firedrake) :/PETSc3/fem$ cp firedrake-install firedrake/bin/firedrake-update (firedrake) :/PETSc3/fem$ chmod u+x firedrake/bin/firedrake-update (firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex (firedrake) :/PETSc3/fem$ firedrake-update Running /PETSc3/fem/firedrake/bin/firedrake-update Traceback (most recent call last): File "/PETSc3/fem/firedrake/bin/firedrake-update", line 277, in <module> import firedrake_configuration ModuleNotFoundError: No module named 'firedrake_configuration'
Ah, ok, we have to do this manually then.
First:
$ cat > firedrake/src/firedrake/firedrake_configuration/configuration.json <<EOF {"options": {"package_manager": false, "minimal_petsc": false, "mpicc": "mpicc", "mpicxx": "mpicxx", "mpif90": "mpif90", "disable_ssh": false, "honour_petsc_dir": true, "show_petsc_configure_options": false, "slepc": false, "slope": false, "packages": [], "honour_pythonpath": false, "petsc_int_type": "int32", "cache_dir": "/PETSc3/fem/firedrake/.cache"}, "environment": {}, "additions": []} EOF
This makes the configuration file firedrake-install would have done.
Now let's pretend we installed the firedrake_configuration module:
$ cat > firedrake/lib/python3.6/site-packages/firedrake.egg-link << EOF /PETSc3/fem/firedrake/src/firedrake . EOF
Now try doing:
$ python -c "import firedrake_configuration; print(firedrake_configuration.get_config())"
If that works, try again with "firedrake-update"
Okay, we get farther. I had to set PYTHONPATH in order to see the configuration file:
(firedrake) :/PETSc3/fem$ export PYTHONPATH=$PWD/firedrake/src/firedrake
(firedrake) :/PETSc3/fem$ python -c "import firedrake_configuration; print(firedrake_configuration.get_config())"
{'options': {'package_manager': False, 'minimal_petsc': False, 'mpicc': 'mpicc', 'mpicxx': 'mpicxx', 'mpif90': 'mpif90', 'disable_ssh': False, 'honour_petsc_dir': True, 'show_petsc_configure_options': False, 'slepc': False, 'slope': False, 'packages': [], 'honour_pythonpath': False, 'petsc_int_type': 'int32', 'cache_dir': '/PETSc3/fem/firedrake/.cache'}, 'environment': {}, 'additions': []}
(firedrake) :/PETSc3/fem$ firedrake_update
-bash: firedrake_update: command not found
(firedrake) 127:/PETSc3/fem$ firedrake-update
Running /PETSc3/fem/firedrake/bin/firedrake-update
The PYTHONPATH environment variable is set. This is probably an error.
If you really want to use your own Python packages, please run again with the
--honour-pythonpath option.
Update log saved in firedrake-update.log
(firedrake) 1:/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Cloning libspatialindex
Successfully cloned repository libspatialindex
Checking out branch master
Successfully checked out branch master
Traceback (most recent call last):
File "../bin/firedrake-update", line 1207, in <module>
build_and_install_libspatialindex()
File "../bin/firedrake-update", line 808, in build_and_install_libspatialindex
check_call(["./autogen.sh"])
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['./autogen.sh']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
Ah ha! You need autotools, but do not check up front :) We get farther (firedrake) 127:/PETSc3/fem$ export PATH=${PATH}:${HOME}/MacSoftware/bin (firedrake) :/PETSc3/fem$ which libtoolize /Users/knepley/MacSoftware/bin/libtoolize (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath Updating the git repository for firedrake Creating firedrake-update script. Running ../bin/firedrake-update --no-update-script --honour-pythonpath Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. Finding dependencies of PyOP2 Finding dependencies of firedrake Updating the git repository for petsc4py Updating the git repository for COFFEE Updating the git repository for ufl Updating the git repository for fiat Updating the git repository for FInAT Updating the git repository for tsfc Updating the git repository for PyOP2 Updating the git repository for firedrake Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for firedrake Installing h5py Updating the git repository for h5py No need to rebuild h5py Installing libspatialindex Updating the git repository for libspatialindex No need to rebuild libspatialindex Cleaning COFFEE Installing COFFEE/ Cleaning ufl Installing ufl/ Cleaning fiat Installing fiat/ Cleaning FInAT Installing FInAT/ Cleaning tsfc Installing tsfc/ Cleaning PyOP2 Traceback (most recent call last): File "../bin/firedrake-update", line 1216, in <module> clean(p) File "../bin/firedrake-update", line 633, in clean check_call(["python", "setup.py", "clean"]) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['python', 'setup.py', 'clean']' returned non-zero exit status 1. Update log saved in firedrake-update.log Here is the log. Thanks, Matt Thanks,
Matt
Cheers,
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On Fri, Nov 2, 2018 at 1:52 PM Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 1:49 PM Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 1:36 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 17:27, Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 1:20 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 17:16, Matthew Knepley <knepley@gmail.com> wrote:
I went on to
cd src/h5py pip install .
and it that worked
(firedrake) :/PETSc3/fem/firedrake$ pushd src/h5py/ /PETSc3/fem/firedrake/src/h5py /PETSc3/fem/firedrake /PETSc3/fem (firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Requirement already satisfied: numpy>=1.7 in
/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.15.3)
Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.11.0) Installing collected packages: h5py Found existing installation: h5py 2.8.0 Uninstalling h5py-2.8.0: Successfully uninstalled h5py-2.8.0 Running setup.py install for h5py ... done Successfully installed h5py-2.8.0.post0
Great!
How do I restart the whole Firedrake now?
This is unsupported, but I think it should work:
$ cp firedrake-install firedrake/bin/firedrake-update $ chmod u+x firedrake/bin/firedrake-update $ . firedrake/bin/activate $ rm -rf firedrake/src/libspatialindex $ firedrake-update
I get
(firedrake) :/PETSc3/fem$ cp firedrake-install firedrake/bin/firedrake-update (firedrake) :/PETSc3/fem$ chmod u+x firedrake/bin/firedrake-update (firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex (firedrake) :/PETSc3/fem$ firedrake-update Running /PETSc3/fem/firedrake/bin/firedrake-update Traceback (most recent call last): File "/PETSc3/fem/firedrake/bin/firedrake-update", line 277, in <module> import firedrake_configuration ModuleNotFoundError: No module named 'firedrake_configuration'
Ah, ok, we have to do this manually then.
First:
$ cat > firedrake/src/firedrake/firedrake_configuration/configuration.json <<EOF {"options": {"package_manager": false, "minimal_petsc": false, "mpicc": "mpicc", "mpicxx": "mpicxx", "mpif90": "mpif90", "disable_ssh": false, "honour_petsc_dir": true, "show_petsc_configure_options": false, "slepc": false, "slope": false, "packages": [], "honour_pythonpath": false, "petsc_int_type": "int32", "cache_dir": "/PETSc3/fem/firedrake/.cache"}, "environment": {}, "additions": []} EOF
This makes the configuration file firedrake-install would have done.
Now let's pretend we installed the firedrake_configuration module:
$ cat > firedrake/lib/python3.6/site-packages/firedrake.egg-link << EOF /PETSc3/fem/firedrake/src/firedrake . EOF
Now try doing:
$ python -c "import firedrake_configuration; print(firedrake_configuration.get_config())"
If that works, try again with "firedrake-update"
Okay, we get farther. I had to set PYTHONPATH in order to see the configuration file:
(firedrake) :/PETSc3/fem$ export PYTHONPATH=$PWD/firedrake/src/firedrake
(firedrake) :/PETSc3/fem$ python -c "import firedrake_configuration; print(firedrake_configuration.get_config())"
{'options': {'package_manager': False, 'minimal_petsc': False, 'mpicc': 'mpicc', 'mpicxx': 'mpicxx', 'mpif90': 'mpif90', 'disable_ssh': False, 'honour_petsc_dir': True, 'show_petsc_configure_options': False, 'slepc': False, 'slope': False, 'packages': [], 'honour_pythonpath': False, 'petsc_int_type': 'int32', 'cache_dir': '/PETSc3/fem/firedrake/.cache'}, 'environment': {}, 'additions': []}
(firedrake) :/PETSc3/fem$ firedrake_update
-bash: firedrake_update: command not found
(firedrake) 127:/PETSc3/fem$ firedrake-update
Running /PETSc3/fem/firedrake/bin/firedrake-update
The PYTHONPATH environment variable is set. This is probably an error.
If you really want to use your own Python packages, please run again with the
--honour-pythonpath option.
Update log saved in firedrake-update.log
(firedrake) 1:/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Cloning libspatialindex
Successfully cloned repository libspatialindex
Checking out branch master
Successfully checked out branch master
Traceback (most recent call last):
File "../bin/firedrake-update", line 1207, in <module>
build_and_install_libspatialindex()
File "../bin/firedrake-update", line 808, in build_and_install_libspatialindex
check_call(["./autogen.sh"])
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['./autogen.sh']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
Ah ha! You need autotools, but do not check up front :) We get farther
(firedrake) 127:/PETSc3/fem$ export PATH=${PATH}:${HOME}/MacSoftware/bin
(firedrake) :/PETSc3/fem$ which libtoolize
/Users/knepley/MacSoftware/bin/libtoolize
(firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Updating the git repository for libspatialindex
No need to rebuild libspatialindex
Cleaning COFFEE
Installing COFFEE/
Cleaning ufl
Installing ufl/
Cleaning fiat
Installing fiat/
Cleaning FInAT
Installing FInAT/
Cleaning tsfc
Installing tsfc/
Cleaning PyOP2
Traceback (most recent call last):
File "../bin/firedrake-update", line 1216, in <module>
clean(p)
File "../bin/firedrake-update", line 633, in clean
check_call(["python", "setup.py", "clean"])
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['python', 'setup.py', 'clean']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
I see. My petsc4py is not visible in this venv. WIll fix that. Thanks, Matt
Thanks,
Matt
Thanks,
Matt
Cheers,
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On 2 Nov 2018, at 17:53, Matthew Knepley <knepley@gmail.com> wrote: ...
Successfully cloned repository libspatialindex Checking out branch master Successfully checked out branch master Traceback (most recent call last): File "../bin/firedrake-update", line 1207, in <module> build_and_install_libspatialindex() File "../bin/firedrake-update", line 808, in build_and_install_libspatialindex check_call(["./autogen.sh"]) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['./autogen.sh']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
Ah ha! You need autotools, but do not check up front :) We get farther
When you first ran "firedrake-install" you said --no-package-manager, so we assumed that you had all the things we need.
(firedrake) 127:/PETSc3/fem$ export PATH=${PATH}:${HOME}/MacSoftware/bin (firedrake) :/PETSc3/fem$ which libtoolize /Users/knepley/MacSoftware/bin/libtoolize (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath Updating the git repository for firedrake Creating firedrake-update script. Running ../bin/firedrake-update --no-update-script --honour-pythonpath Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py Updating the git repository for COFFEE Updating the git repository for ufl Updating the git repository for fiat Updating the git repository for FInAT Updating the git repository for tsfc Updating the git repository for PyOP2 Updating the git repository for firedrake Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for firedrake Installing h5py Updating the git repository for h5py No need to rebuild h5py Installing libspatialindex Updating the git repository for libspatialindex No need to rebuild libspatialindex Cleaning COFFEE Installing COFFEE/ Cleaning ufl Installing ufl/ Cleaning fiat Installing fiat/ Cleaning FInAT Installing FInAT/ Cleaning tsfc Installing tsfc/ Cleaning PyOP2 Traceback (most recent call last): File "../bin/firedrake-update", line 1216, in <module> clean(p) File "../bin/firedrake-update", line 633, in clean check_call(["python", "setup.py", "clean"]) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['python', 'setup.py', 'clean']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
I see. My petsc4py is not visible in this venv. WIll fix that.
We should have installed petsc4py, but obviously didn't and then the firedrake-update has some logic to not rebuild it. Just "pip install -U --no-deps firedrake/src/petsc4py" Lawrence
On Fri, Nov 2, 2018 at 1:56 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 17:53, Matthew Knepley <knepley@gmail.com> wrote: ...
Successfully cloned repository libspatialindex Checking out branch master Successfully checked out branch master Traceback (most recent call last): File "../bin/firedrake-update", line 1207, in <module> build_and_install_libspatialindex() File "../bin/firedrake-update", line 808, in build_and_install_libspatialindex check_call(["./autogen.sh"]) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['./autogen.sh']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
Ah ha! You need autotools, but do not check up front :) We get farther
When you first ran "firedrake-install" you said --no-package-manager, so we assumed that you had all the things we need.
(firedrake) 127:/PETSc3/fem$ export PATH=${PATH}:${HOME}/MacSoftware/bin (firedrake) :/PETSc3/fem$ which libtoolize /Users/knepley/MacSoftware/bin/libtoolize (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath Updating the git repository for firedrake Creating firedrake-update script. Running ../bin/firedrake-update --no-update-script --honour-pythonpath Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py Updating the git repository for COFFEE Updating the git repository for ufl Updating the git repository for fiat Updating the git repository for FInAT Updating the git repository for tsfc Updating the git repository for PyOP2 Updating the git repository for firedrake Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for firedrake Installing h5py Updating the git repository for h5py No need to rebuild h5py Installing libspatialindex Updating the git repository for libspatialindex No need to rebuild libspatialindex Cleaning COFFEE Installing COFFEE/ Cleaning ufl Installing ufl/ Cleaning fiat Installing fiat/ Cleaning FInAT Installing FInAT/ Cleaning tsfc Installing tsfc/ Cleaning PyOP2 Traceback (most recent call last): File "../bin/firedrake-update", line 1216, in <module> clean(p) File "../bin/firedrake-update", line 633, in clean check_call(["python", "setup.py", "clean"]) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['python', 'setup.py', 'clean']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
I see. My petsc4py is not visible in this venv. WIll fix that.
We should have installed petsc4py, but obviously didn't and then the firedrake-update has some logic to not rebuild it.
Just "pip install -U --no-deps firedrake/src/petsc4py"
That worked. We are almost there. However, I cannot interpret the error here: (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath Updating the git repository for firedrake Creating firedrake-update script. Running ../bin/firedrake-update --no-update-script --honour-pythonpath Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. Finding dependencies of PyOP2 Finding dependencies of firedrake Updating the git repository for petsc4py Updating the git repository for COFFEE Updating the git repository for ufl Updating the git repository for fiat Updating the git repository for FInAT Updating the git repository for tsfc Updating the git repository for PyOP2 Updating the git repository for firedrake Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for firedrake Installing h5py Updating the git repository for h5py No need to rebuild h5py Installing libspatialindex Updating the git repository for libspatialindex No need to rebuild libspatialindex Cleaning COFFEE Installing COFFEE/ Cleaning ufl Installing ufl/ Cleaning fiat Installing fiat/ Cleaning FInAT Installing FInAT/ Cleaning tsfc Installing tsfc/ Cleaning PyOP2 Installing PyOP2/ Cleaning firedrake Installing firedrake/ Traceback (most recent call last): File "../bin/firedrake-update", line 1217, in <module> install(p+"/") File "../bin/firedrake-update", line 625, in install run_pip_install(["-e", package]) File "../bin/firedrake-update", line 597, in run_pip_install check_call(pipinstall + pipargs) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'firedrake/']' returned non-zero exit status 1. Update log saved in firedrake-update.log Here is the log. Thanks, Matt Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On 02/11/2018 20:42, Matthew Knepley wrote:
On Fri, Nov 2, 2018 at 1:56 PM Lawrence Mitchell <wence@gmx.li <mailto:wence@gmx.li>> wrote:
> On 2 Nov 2018, at 17:53, Matthew Knepley <knepley@gmail.com <mailto:knepley@gmail.com>> wrote: > ...
> Successfully cloned repository libspatialindex > Checking out branch master > Successfully checked out branch master > Traceback (most recent call last): > File "../bin/firedrake-update", line 1207, in <module> > build_and_install_libspatialindex() > File "../bin/firedrake-update", line 808, in build_and_install_libspatialindex > check_call(["./autogen.sh"]) > File "../bin/firedrake-update", line 435, in check_call > log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output > **kwargs).stdout > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run > output=stdout, stderr=stderr) > subprocess.CalledProcessError: Command '['./autogen.sh']' returned non-zero exit status 1. > > > Update log saved in firedrake-update.log > > Here is the log. > > Ah ha! You need autotools, but do not check up front :) We get farther
When you first ran "firedrake-install" you said --no-package-manager, so we assumed that you had all the things we need.
> (firedrake) 127:/PETSc3/fem$ export PATH=${PATH}:${HOME}/MacSoftware/bin > (firedrake) :/PETSc3/fem$ which libtoolize > /Users/knepley/MacSoftware/bin/libtoolize > (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath > Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath > Updating the git repository for firedrake > Creating firedrake-update script. > Running ../bin/firedrake-update --no-update-script --honour-pythonpath > Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. > Finding dependencies of PyOP2 > > Finding dependencies of firedrake > > Updating the git repository for petsc4py > Updating the git repository for COFFEE > Updating the git repository for ufl > Updating the git repository for fiat > Updating the git repository for FInAT > Updating the git repository for tsfc > Updating the git repository for PyOP2 > Updating the git repository for firedrake > Installing pip dependencies for COFFEE > Installing pip dependencies for ufl > No dependencies found. Skipping. > Installing pip dependencies for fiat > No dependencies found. Skipping. > Installing pip dependencies for FInAT > Installing pip dependencies for tsfc > Installing pip dependencies for PyOP2 > Installing pip dependencies for firedrake > Installing h5py > Updating the git repository for h5py > No need to rebuild h5py > Installing libspatialindex > Updating the git repository for libspatialindex > No need to rebuild libspatialindex > Cleaning COFFEE > Installing COFFEE/ > Cleaning ufl > Installing ufl/ > Cleaning fiat > Installing fiat/ > Cleaning FInAT > Installing FInAT/ > Cleaning tsfc > Installing tsfc/ > Cleaning PyOP2 > Traceback (most recent call last): > File "../bin/firedrake-update", line 1216, in <module> > clean(p) > File "../bin/firedrake-update", line 633, in clean > check_call(["python", "setup.py", "clean"]) > File "../bin/firedrake-update", line 435, in check_call > log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output > **kwargs).stdout > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run > output=stdout, stderr=stderr) > subprocess.CalledProcessError: Command '['python', 'setup.py', 'clean']' returned non-zero exit status 1. > > > Update log saved in firedrake-update.log > > Here is the log. > > I see. My petsc4py is not visible in this venv. WIll fix that. > We should have installed petsc4py, but obviously didn't and then the firedrake-update has some logic to not rebuild it.
Just "pip install -U --no-deps firedrake/src/petsc4py"
That worked. We are almost there. However, I cannot interpret the error here:
(firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Updating the git repository for libspatialindex
No need to rebuild libspatialindex
Cleaning COFFEE
Installing COFFEE/
Cleaning ufl
Installing ufl/
Cleaning fiat
Installing fiat/
Cleaning FInAT
Installing FInAT/
Cleaning tsfc
Installing tsfc/
Cleaning PyOP2
Installing PyOP2/
Cleaning firedrake
Installing firedrake/
Traceback (most recent call last):
File "../bin/firedrake-update", line 1217, in <module>
install(p+"/")
File "../bin/firedrake-update", line 625, in install
run_pip_install(["-e", package])
File "../bin/firedrake-update", line 597, in run_pip_install
check_call(pipinstall + pipargs)
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'firedrake/']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
Thanks,
Matt
The log seems to say:
firedrake/spatialindex.c:614:10: fatal error: 'spatialindex/capi/sidx_api.h' file not found #include "spatialindex/capi/sidx_api.h"
which suggests libspatialindex wasn't installed (properly/at all) in your virtual env. Notably you should have a include/spatialindex in your virtual env. I'm not sure how to fix that. If you don't want to wait for Lawrence or David, I would try reinstalling like in the script (cf below), but without any guarantee. Otherwise restarting the installation from scratch is usually a good idea when things get too messed up. If you use the --venv-name option with the firedrake-install script, you can install Firedrake in a different directory so you can come back to your broken install later! def build_and_install_libspatialindex(): log.info("Installing libspatialindex") if os.path.exists("libspatialindex"): log.info("Updating the git repository for libspatialindex") with directory("libspatialindex"): check_call(["git", "fetch"]) git_sha = check_output(["git", "rev-parse", "HEAD"]) git_sha_new = check_output(["git", "rev-parse", "@{u}"]) libspatialindex_changed = git_sha != git_sha_new if libspatialindex_changed: check_call(["git", "reset", "--hard"]) check_call(["git", "pull"]) else: git_clone("git+https://github.com/firedrakeproject/libspatialindex.git") libspatialindex_changed = True if libspatialindex_changed: with directory("libspatialindex"): # Clean source directory check_call(["git", "reset", "--hard"]) check_call(["git", "clean", "-f", "-x", "-d"]) # Patch Makefile.am to skip building test check_call(["sed", "-i", "-e", "/^SUBDIRS/s/ test//", "Makefile.am"]) # Build and install check_call(["./autogen.sh"]) check_call(["./configure", "--prefix=" + firedrake_env, "--enable-shared", "--disable-static"]) check_call(["make"]) check_call(["make", "install"]) else: log.info("No need to rebuild libspatialindex") -- Nicolas
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/%7Eknepley/>
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- Nicolas Barral Dept. of Earth Science and Engineering Imperial College London Royal School of Mines - Office 4.88 London SW7 2AZ
On Fri, Nov 2, 2018 at 6:17 PM Nicolas Barral <n.barral@imperial.ac.uk> wrote:
On 02/11/2018 20:42, Matthew Knepley wrote:
On Fri, Nov 2, 2018 at 1:56 PM Lawrence Mitchell <wence@gmx.li <mailto:wence@gmx.li>> wrote:
> On 2 Nov 2018, at 17:53, Matthew Knepley <knepley@gmail.com <mailto:knepley@gmail.com>> wrote: > ...
> Successfully cloned repository libspatialindex > Checking out branch master > Successfully checked out branch master > Traceback (most recent call last): > File "../bin/firedrake-update", line 1207, in <module> > build_and_install_libspatialindex() > File "../bin/firedrake-update", line 808, in build_and_install_libspatialindex > check_call(["./autogen.sh"]) > File "../bin/firedrake-update", line 435, in check_call > log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output > **kwargs).stdout > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run > output=stdout, stderr=stderr) > subprocess.CalledProcessError: Command '['./autogen.sh']' returned non-zero exit status 1. > > > Update log saved in firedrake-update.log > > Here is the log. > > Ah ha! You need autotools, but do not check up front :) We get farther
When you first ran "firedrake-install" you said --no-package-manager, so we assumed that you had all the things we need.
> (firedrake) 127:/PETSc3/fem$ export PATH=${PATH}:${HOME}/MacSoftware/bin > (firedrake) :/PETSc3/fem$ which libtoolize > /Users/knepley/MacSoftware/bin/libtoolize > (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath > Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath > Updating the git repository for firedrake > Creating firedrake-update script. > Running ../bin/firedrake-update --no-update-script --honour-pythonpath > Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. > Finding dependencies of PyOP2 > > Finding dependencies of firedrake > > Updating the git repository for petsc4py > Updating the git repository for COFFEE > Updating the git repository for ufl > Updating the git repository for fiat > Updating the git repository for FInAT > Updating the git repository for tsfc > Updating the git repository for PyOP2 > Updating the git repository for firedrake > Installing pip dependencies for COFFEE > Installing pip dependencies for ufl > No dependencies found. Skipping. > Installing pip dependencies for fiat > No dependencies found. Skipping. > Installing pip dependencies for FInAT > Installing pip dependencies for tsfc > Installing pip dependencies for PyOP2 > Installing pip dependencies for firedrake > Installing h5py > Updating the git repository for h5py > No need to rebuild h5py > Installing libspatialindex > Updating the git repository for libspatialindex > No need to rebuild libspatialindex > Cleaning COFFEE > Installing COFFEE/ > Cleaning ufl > Installing ufl/ > Cleaning fiat > Installing fiat/ > Cleaning FInAT > Installing FInAT/ > Cleaning tsfc > Installing tsfc/ > Cleaning PyOP2 > Traceback (most recent call last): > File "../bin/firedrake-update", line 1216, in <module> > clean(p) > File "../bin/firedrake-update", line 633, in clean > check_call(["python", "setup.py", "clean"]) > File "../bin/firedrake-update", line 435, in check_call > log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output > **kwargs).stdout > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run > output=stdout, stderr=stderr) > subprocess.CalledProcessError: Command '['python', 'setup.py', 'clean']' returned non-zero exit status 1. > > > Update log saved in firedrake-update.log > > Here is the log. > > I see. My petsc4py is not visible in this venv. WIll fix that. > We should have installed petsc4py, but obviously didn't and then the firedrake-update has some logic to not rebuild it.
Just "pip install -U --no-deps firedrake/src/petsc4py"
That worked. We are almost there. However, I cannot interpret the error here:
(firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Updating the git repository for libspatialindex
No need to rebuild libspatialindex
Cleaning COFFEE
Installing COFFEE/
Cleaning ufl
Installing ufl/
Cleaning fiat
Installing fiat/
Cleaning FInAT
Installing FInAT/
Cleaning tsfc
Installing tsfc/
Cleaning PyOP2
Installing PyOP2/
Cleaning firedrake
Installing firedrake/
Traceback (most recent call last):
File "../bin/firedrake-update", line 1217, in <module>
install(p+"/")
File "../bin/firedrake-update", line 625, in install
run_pip_install(["-e", package])
File "../bin/firedrake-update", line 597, in run_pip_install
check_call(pipinstall + pipargs)
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'firedrake/']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
Thanks,
Matt
The log seems to say:
firedrake/spatialindex.c:614:10: fatal error: 'spatialindex/capi/sidx_api.h' file not found #include "spatialindex/capi/sidx_api.h"
which suggests libspatialindex wasn't installed (properly/at all) in your virtual env. Notably you should have a include/spatialindex in your virtual env.
Good hint! I did (firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex/ (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath Updating the git repository for firedrake Creating firedrake-update script. Running ../bin/firedrake-update --no-update-script --honour-pythonpath Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. Finding dependencies of PyOP2 Finding dependencies of firedrake Updating the git repository for petsc4py Updating the git repository for COFFEE Updating the git repository for ufl Updating the git repository for fiat Updating the git repository for FInAT Updating the git repository for tsfc Updating the git repository for PyOP2 Updating the git repository for firedrake Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for firedrake Installing h5py Updating the git repository for h5py No need to rebuild h5py Installing libspatialindex Cloning libspatialindex Successfully cloned repository libspatialindex Checking out branch master Successfully checked out branch master Traceback (most recent call last): File "../bin/firedrake-update", line 1207, in <module> build_and_install_libspatialindex() File "../bin/firedrake-update", line 811, in build_and_install_libspatialindex check_call(["make"]) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['make']' returned non-zero exit status 2. Update log saved in firedrake-update.log This seems to be an honest-to-goodness bug in the configure for libspatialindex. You can't assume that my crappy g++ on Apple can do thread-local storage. Here is the log. Thanks, Matt I'm not sure how to fix that. If you don't want to wait for Lawrence or
David, I would try reinstalling like in the script (cf below), but without any guarantee. Otherwise restarting the installation from scratch is usually a good idea when things get too messed up. If you use the --venv-name option with the firedrake-install script, you can install Firedrake in a different directory so you can come back to your broken install later!
def build_and_install_libspatialindex(): log.info("Installing libspatialindex") if os.path.exists("libspatialindex"): log.info("Updating the git repository for libspatialindex") with directory("libspatialindex"): check_call(["git", "fetch"]) git_sha = check_output(["git", "rev-parse", "HEAD"]) git_sha_new = check_output(["git", "rev-parse", "@{u}"]) libspatialindex_changed = git_sha != git_sha_new if libspatialindex_changed: check_call(["git", "reset", "--hard"]) check_call(["git", "pull"]) else:
git_clone("git+https://github.com/firedrakeproject/libspatialindex.git") libspatialindex_changed = True
if libspatialindex_changed: with directory("libspatialindex"): # Clean source directory check_call(["git", "reset", "--hard"]) check_call(["git", "clean", "-f", "-x", "-d"]) # Patch Makefile.am to skip building test check_call(["sed", "-i", "-e", "/^SUBDIRS/s/ test//", "Makefile.am"]) # Build and install check_call(["./autogen.sh"]) check_call(["./configure", "--prefix=" + firedrake_env, "--enable-shared", "--disable-static"]) check_call(["make"]) check_call(["make", "install"]) else: log.info("No need to rebuild libspatialindex")
-- Nicolas
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/%7Eknepley/>
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- Nicolas Barral
Dept. of Earth Science and Engineering Imperial College London Royal School of Mines - Office 4.88 London SW7 2AZ
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
Okay, I restarted today and I get a compile problem with PyOP2, but I cannot see a compile error. (firedrake) 1:/PETSc3/fem$ firedrake-update --honour-pythonpath Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath Updating the git repository for firedrake Creating firedrake-update script. Running ../bin/firedrake-update --no-update-script --honour-pythonpath Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. Finding dependencies of PyOP2 Finding dependencies of firedrake Updating the git repository for petsc4py Updating the git repository for COFFEE Updating the git repository for ufl Updating the git repository for fiat Updating the git repository for FInAT Updating the git repository for tsfc Updating the git repository for PyOP2 Updating the git repository for firedrake Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for firedrake Installing h5py Updating the git repository for h5py No need to rebuild h5py Installing libspatialindex Updating the git repository for libspatialindex No need to rebuild libspatialindex Cleaning COFFEE Installing COFFEE/ Cleaning ufl Installing ufl/ Cleaning fiat Installing fiat/ Cleaning FInAT Installing FInAT/ Cleaning tsfc Installing tsfc/ Cleaning PyOP2 Installing PyOP2/ Traceback (most recent call last): File "../bin/firedrake-update", line 1217, in <module> install(p+"/") File "../bin/firedrake-update", line 625, in install run_pip_install(["-e", package]) File "../bin/firedrake-update", line 597, in run_pip_install check_call(pipinstall + pipargs) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'PyOP2/']' returned non-zero exit status 1. Update log saved in firedrake-update.log Thanks, Matt On Sat, Nov 3, 2018 at 8:27 AM Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 6:17 PM Nicolas Barral <n.barral@imperial.ac.uk> wrote:
On 02/11/2018 20:42, Matthew Knepley wrote:
On Fri, Nov 2, 2018 at 1:56 PM Lawrence Mitchell <wence@gmx.li <mailto:wence@gmx.li>> wrote:
> On 2 Nov 2018, at 17:53, Matthew Knepley <knepley@gmail.com <mailto:knepley@gmail.com>> wrote: > ...
> Successfully cloned repository libspatialindex > Checking out branch master > Successfully checked out branch master > Traceback (most recent call last): > File "../bin/firedrake-update", line 1207, in <module> > build_and_install_libspatialindex() > File "../bin/firedrake-update", line 808, in build_and_install_libspatialindex > check_call(["./autogen.sh"]) > File "../bin/firedrake-update", line 435, in check_call > log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output > **kwargs).stdout > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run > output=stdout, stderr=stderr) > subprocess.CalledProcessError: Command '['./autogen.sh']' returned non-zero exit status 1. > > > Update log saved in firedrake-update.log > > Here is the log. > > Ah ha! You need autotools, but do not check up front :) We get farther
When you first ran "firedrake-install" you said --no-package-manager, so we assumed that you had all the things we need.
> (firedrake) 127:/PETSc3/fem$ export PATH=${PATH}:${HOME}/MacSoftware/bin > (firedrake) :/PETSc3/fem$ which libtoolize > /Users/knepley/MacSoftware/bin/libtoolize > (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath > Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath > Updating the git repository for firedrake > Creating firedrake-update script. > Running ../bin/firedrake-update --no-update-script --honour-pythonpath > Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. > Finding dependencies of PyOP2 > > Finding dependencies of firedrake > > Updating the git repository for petsc4py > Updating the git repository for COFFEE > Updating the git repository for ufl > Updating the git repository for fiat > Updating the git repository for FInAT > Updating the git repository for tsfc > Updating the git repository for PyOP2 > Updating the git repository for firedrake > Installing pip dependencies for COFFEE > Installing pip dependencies for ufl > No dependencies found. Skipping. > Installing pip dependencies for fiat > No dependencies found. Skipping. > Installing pip dependencies for FInAT > Installing pip dependencies for tsfc > Installing pip dependencies for PyOP2 > Installing pip dependencies for firedrake > Installing h5py > Updating the git repository for h5py > No need to rebuild h5py > Installing libspatialindex > Updating the git repository for libspatialindex > No need to rebuild libspatialindex > Cleaning COFFEE > Installing COFFEE/ > Cleaning ufl > Installing ufl/ > Cleaning fiat > Installing fiat/ > Cleaning FInAT > Installing FInAT/ > Cleaning tsfc > Installing tsfc/ > Cleaning PyOP2 > Traceback (most recent call last): > File "../bin/firedrake-update", line 1216, in <module> > clean(p) > File "../bin/firedrake-update", line 633, in clean > check_call(["python", "setup.py", "clean"]) > File "../bin/firedrake-update", line 435, in check_call > log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output > **kwargs).stdout > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run > output=stdout, stderr=stderr) > subprocess.CalledProcessError: Command '['python', 'setup.py', 'clean']' returned non-zero exit status 1. > > > Update log saved in firedrake-update.log > > Here is the log. > > I see. My petsc4py is not visible in this venv. WIll fix that. > We should have installed petsc4py, but obviously didn't and then the firedrake-update has some logic to not rebuild it.
Just "pip install -U --no-deps firedrake/src/petsc4py"
That worked. We are almost there. However, I cannot interpret the error here:
(firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Updating the git repository for libspatialindex
No need to rebuild libspatialindex
Cleaning COFFEE
Installing COFFEE/
Cleaning ufl
Installing ufl/
Cleaning fiat
Installing fiat/
Cleaning FInAT
Installing FInAT/
Cleaning tsfc
Installing tsfc/
Cleaning PyOP2
Installing PyOP2/
Cleaning firedrake
Installing firedrake/
Traceback (most recent call last):
File "../bin/firedrake-update", line 1217, in <module>
install(p+"/")
File "../bin/firedrake-update", line 625, in install
run_pip_install(["-e", package])
File "../bin/firedrake-update", line 597, in run_pip_install
check_call(pipinstall + pipargs)
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'firedrake/']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
Thanks,
Matt
The log seems to say:
firedrake/spatialindex.c:614:10: fatal error: 'spatialindex/capi/sidx_api.h' file not found #include "spatialindex/capi/sidx_api.h"
which suggests libspatialindex wasn't installed (properly/at all) in your virtual env. Notably you should have a include/spatialindex in your virtual env.
Good hint! I did
(firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex/
(firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Cloning libspatialindex
Successfully cloned repository libspatialindex
Checking out branch master
Successfully checked out branch master
Traceback (most recent call last):
File "../bin/firedrake-update", line 1207, in <module>
build_and_install_libspatialindex()
File "../bin/firedrake-update", line 811, in build_and_install_libspatialindex
check_call(["make"])
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['make']' returned non-zero exit status 2.
Update log saved in firedrake-update.log
This seems to be an honest-to-goodness bug in the configure for libspatialindex. You can't assume that my crappy g++ on Apple can do thread-local storage.
Here is the log.
Thanks,
Matt
I'm not sure how to fix that. If you don't want to wait for Lawrence or
David, I would try reinstalling like in the script (cf below), but without any guarantee. Otherwise restarting the installation from scratch is usually a good idea when things get too messed up. If you use the --venv-name option with the firedrake-install script, you can install Firedrake in a different directory so you can come back to your broken install later!
def build_and_install_libspatialindex(): log.info("Installing libspatialindex") if os.path.exists("libspatialindex"): log.info("Updating the git repository for libspatialindex") with directory("libspatialindex"): check_call(["git", "fetch"]) git_sha = check_output(["git", "rev-parse", "HEAD"]) git_sha_new = check_output(["git", "rev-parse", "@{u}"]) libspatialindex_changed = git_sha != git_sha_new if libspatialindex_changed: check_call(["git", "reset", "--hard"]) check_call(["git", "pull"]) else:
git_clone("git+https://github.com/firedrakeproject/libspatialindex.git") libspatialindex_changed = True
if libspatialindex_changed: with directory("libspatialindex"): # Clean source directory check_call(["git", "reset", "--hard"]) check_call(["git", "clean", "-f", "-x", "-d"]) # Patch Makefile.am to skip building test check_call(["sed", "-i", "-e", "/^SUBDIRS/s/ test//", "Makefile.am"]) # Build and install check_call(["./autogen.sh"]) check_call(["./configure", "--prefix=" + firedrake_env, "--enable-shared", "--disable-static"]) check_call(["make"]) check_call(["make", "install"]) else: log.info("No need to rebuild libspatialindex")
-- Nicolas
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/%7Eknepley/>
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- Nicolas Barral
Dept. of Earth Science and Engineering Imperial College London Royal School of Mines - Office 4.88 London SW7 2AZ
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
We saw this: error: command 'mpicc' failed with exit status 1 when build a c module in pyop2. Can you: cd firedrake/src/PyOP2 make and post the results. Output goes to make.log Lawrence
On 6 Nov 2018, at 08:07, Matthew Knepley <knepley@gmail.com> wrote:
Okay, I restarted today and I get a compile problem with PyOP2, but I cannot see a compile error.
(firedrake) 1:/PETSc3/fem$ firedrake-update --honour-pythonpath Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath Updating the git repository for firedrake Creating firedrake-update script. Running ../bin/firedrake-update --no-update-script --honour-pythonpath Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py Updating the git repository for COFFEE Updating the git repository for ufl Updating the git repository for fiat Updating the git repository for FInAT Updating the git repository for tsfc Updating the git repository for PyOP2 Updating the git repository for firedrake Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for firedrake Installing h5py Updating the git repository for h5py No need to rebuild h5py Installing libspatialindex Updating the git repository for libspatialindex No need to rebuild libspatialindex Cleaning COFFEE Installing COFFEE/ Cleaning ufl Installing ufl/ Cleaning fiat Installing fiat/ Cleaning FInAT Installing FInAT/ Cleaning tsfc Installing tsfc/ Cleaning PyOP2 Installing PyOP2/ Traceback (most recent call last): File "../bin/firedrake-update", line 1217, in <module> install(p+"/") File "../bin/firedrake-update", line 625, in install run_pip_install(["-e", package]) File "../bin/firedrake-update", line 597, in run_pip_install check_call(pipinstall + pipargs) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'PyOP2/']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Thanks,
Matt
On Sat, Nov 3, 2018 at 8:27 AM Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 6:17 PM Nicolas Barral <n.barral@imperial.ac.uk> wrote: On 02/11/2018 20:42, Matthew Knepley wrote:
On Fri, Nov 2, 2018 at 1:56 PM Lawrence Mitchell <wence@gmx.li <mailto:wence@gmx.li>> wrote:
> On 2 Nov 2018, at 17:53, Matthew Knepley <knepley@gmail.com <mailto:knepley@gmail.com>> wrote: > ...
> Successfully cloned repository libspatialindex > Checking out branch master > Successfully checked out branch master > Traceback (most recent call last): > File "../bin/firedrake-update", line 1207, in <module> > build_and_install_libspatialindex() > File "../bin/firedrake-update", line 808, in build_and_install_libspatialindex > check_call(["./autogen.sh"]) > File "../bin/firedrake-update", line 435, in check_call > log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output > **kwargs).stdout > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run > output=stdout, stderr=stderr) > subprocess.CalledProcessError: Command '['./autogen.sh']' returned non-zero exit status 1. > > > Update log saved in firedrake-update.log > > Here is the log. > > Ah ha! You need autotools, but do not check up front :) We get farther
When you first ran "firedrake-install" you said --no-package-manager, so we assumed that you had all the things we need.
> (firedrake) 127:/PETSc3/fem$ export PATH=${PATH}:${HOME}/MacSoftware/bin > (firedrake) :/PETSc3/fem$ which libtoolize > /Users/knepley/MacSoftware/bin/libtoolize > (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath > Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath > Updating the git repository for firedrake > Creating firedrake-update script. > Running ../bin/firedrake-update --no-update-script --honour-pythonpath > Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. > Finding dependencies of PyOP2 > > Finding dependencies of firedrake > > Updating the git repository for petsc4py > Updating the git repository for COFFEE > Updating the git repository for ufl > Updating the git repository for fiat > Updating the git repository for FInAT > Updating the git repository for tsfc > Updating the git repository for PyOP2 > Updating the git repository for firedrake > Installing pip dependencies for COFFEE > Installing pip dependencies for ufl > No dependencies found. Skipping. > Installing pip dependencies for fiat > No dependencies found. Skipping. > Installing pip dependencies for FInAT > Installing pip dependencies for tsfc > Installing pip dependencies for PyOP2 > Installing pip dependencies for firedrake > Installing h5py > Updating the git repository for h5py > No need to rebuild h5py > Installing libspatialindex > Updating the git repository for libspatialindex > No need to rebuild libspatialindex > Cleaning COFFEE > Installing COFFEE/ > Cleaning ufl > Installing ufl/ > Cleaning fiat > Installing fiat/ > Cleaning FInAT > Installing FInAT/ > Cleaning tsfc > Installing tsfc/ > Cleaning PyOP2 > Traceback (most recent call last): > File "../bin/firedrake-update", line 1216, in <module> > clean(p) > File "../bin/firedrake-update", line 633, in clean > check_call(["python", "setup.py", "clean"]) > File "../bin/firedrake-update", line 435, in check_call > log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output > **kwargs).stdout > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run > output=stdout, stderr=stderr) > subprocess.CalledProcessError: Command '['python', 'setup.py', 'clean']' returned non-zero exit status 1. > > > Update log saved in firedrake-update.log > > Here is the log. > > I see. My petsc4py is not visible in this venv. WIll fix that. > We should have installed petsc4py, but obviously didn't and then the firedrake-update has some logic to not rebuild it.
Just "pip install -U --no-deps firedrake/src/petsc4py"
That worked. We are almost there. However, I cannot interpret the error here:
(firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Updating the git repository for libspatialindex
No need to rebuild libspatialindex
Cleaning COFFEE
Installing COFFEE/
Cleaning ufl
Installing ufl/
Cleaning fiat
Installing fiat/
Cleaning FInAT
Installing FInAT/
Cleaning tsfc
Installing tsfc/
Cleaning PyOP2
Installing PyOP2/
Cleaning firedrake
Installing firedrake/
Traceback (most recent call last):
File "../bin/firedrake-update", line 1217, in <module>
install(p+"/")
File "../bin/firedrake-update", line 625, in install
run_pip_install(["-e", package])
File "../bin/firedrake-update", line 597, in run_pip_install
check_call(pipinstall + pipargs)
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'firedrake/']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
Thanks,
Matt
The log seems to say:
firedrake/spatialindex.c:614:10: fatal error: 'spatialindex/capi/sidx_api.h' file not found #include "spatialindex/capi/sidx_api.h"
which suggests libspatialindex wasn't installed (properly/at all) in your virtual env. Notably you should have a include/spatialindex in your virtual env.
Good hint! I did
(firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex/ (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath Updating the git repository for firedrake Creating firedrake-update script. Running ../bin/firedrake-update --no-update-script --honour-pythonpath Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py Updating the git repository for COFFEE Updating the git repository for ufl Updating the git repository for fiat Updating the git repository for FInAT Updating the git repository for tsfc Updating the git repository for PyOP2 Updating the git repository for firedrake Installing pip dependencies for COFFEE Installing pip dependencies for ufl No dependencies found. Skipping. Installing pip dependencies for fiat No dependencies found. Skipping. Installing pip dependencies for FInAT Installing pip dependencies for tsfc Installing pip dependencies for PyOP2 Installing pip dependencies for firedrake Installing h5py Updating the git repository for h5py No need to rebuild h5py Installing libspatialindex Cloning libspatialindex
Successfully cloned repository libspatialindex Checking out branch master Successfully checked out branch master Traceback (most recent call last): File "../bin/firedrake-update", line 1207, in <module> build_and_install_libspatialindex() File "../bin/firedrake-update", line 811, in build_and_install_libspatialindex check_call(["make"]) File "../bin/firedrake-update", line 435, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['make']' returned non-zero exit status 2.
Update log saved in firedrake-update.log
This seems to be an honest-to-goodness bug in the configure for libspatialindex. You can't assume that my crappy g++ on Apple can do thread-local storage.
Here is the log.
Thanks,
Matt
I'm not sure how to fix that. If you don't want to wait for Lawrence or David, I would try reinstalling like in the script (cf below), but without any guarantee. Otherwise restarting the installation from scratch is usually a good idea when things get too messed up. If you use the --venv-name option with the firedrake-install script, you can install Firedrake in a different directory so you can come back to your broken install later!
def build_and_install_libspatialindex(): log.info("Installing libspatialindex") if os.path.exists("libspatialindex"): log.info("Updating the git repository for libspatialindex") with directory("libspatialindex"): check_call(["git", "fetch"]) git_sha = check_output(["git", "rev-parse", "HEAD"]) git_sha_new = check_output(["git", "rev-parse", "@{u}"]) libspatialindex_changed = git_sha != git_sha_new if libspatialindex_changed: check_call(["git", "reset", "--hard"]) check_call(["git", "pull"]) else:
git_clone("git+https://github.com/firedrakeproject/libspatialindex.git") libspatialindex_changed = True
if libspatialindex_changed: with directory("libspatialindex"): # Clean source directory check_call(["git", "reset", "--hard"]) check_call(["git", "clean", "-f", "-x", "-d"]) # Patch Makefile.am to skip building test check_call(["sed", "-i", "-e", "/^SUBDIRS/s/ test//", "Makefile.am"]) # Build and install check_call(["./autogen.sh"]) check_call(["./configure", "--prefix=" + firedrake_env, "--enable-shared", "--disable-static"]) check_call(["make"]) check_call(["make", "install"]) else: log.info("No need to rebuild libspatialindex")
-- Nicolas
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/%7Eknepley/>
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- Nicolas Barral
Dept. of Earth Science and Engineering Imperial College London Royal School of Mines - Office 4.88 London SW7 2AZ
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <firedrake-update.log> _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
On Tue, Nov 6, 2018 at 3:14 AM Lawrence Mitchell <wence@gmx.li> wrote:
We saw this:
error: command 'mpicc' failed with exit status 1
when build a c module in pyop2. Can you:
cd firedrake/src/PyOP2
make
and post the results. Output goes to make.log
Ah, mpicc was not in the path. Fixed that, and now the real problem is libspatialindex: (firedrake) :/PETSc3/fem$ pushd firedrake/src/libspatialindex/ /PETSc3/fem/firedrake/src/libspatialindex /PETSc3/fem (firedrake) master *:/PETSc3/fem/firedrake/src/libspatialindex$ make Making all in src Making all in storagemanager make[2]: Nothing to be done for `all'. Making all in spatialindex make[2]: Nothing to be done for `all'. Making all in rtree make[2]: Nothing to be done for `all'. Making all in mvrtree make[2]: Nothing to be done for `all'. Making all in tprtree make[2]: Nothing to be done for `all'. Making all in tools make[2]: Nothing to be done for `all'. Making all in capi depbase=`echo sidx_api.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\ /bin/sh ../../libtool --tag=CXX --mode=compile g++ -DPACKAGE_NAME=\"spatialindex\" -DPACKAGE_TARNAME=\"spatialindex-src\" -DPACKAGE_VERSION=\"1.8.6\" -DPACKAGE_STRING=\"spatialindex\ 1.8.6\" -DPACKAGE_BUGREPORT=\"mhadji@gmail.com\" -DPACKAGE_URL=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DPACKAGE=\"spatialindex-src\" -DVERSION=\"1.8.6\" -DHAVE_FCNTL_H=1 -DHAVE_UNISTD_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_PTHREAD_H=1 -DHAVE_SYS_RESOURCE_H=1 -DHAVE_SYS_TIME_H=1 -DHAVE_STDINT_H=1 -DHAVE_GETTIMEOFDAY=1 -DHAVE_MEMSET=1 -DHAVE_MEMCPY=1 -DHAVE_BCOPY=1 -DHAVE_SRAND48=1 -I. -I../../include -I../../include/capi -Wall -Wno-long-long -pedantic -std=c++98 -O2 -DNDEBUG -MT sidx_api.lo -MD -MP -MF $depbase.Tpo -c -o sidx_api.lo sidx_api.cc &&\ mv -f $depbase.Tpo $depbase.Plo libtool: compile: g++ -DPACKAGE_NAME=\"spatialindex\" -DPACKAGE_TARNAME=\"spatialindex-src\" -DPACKAGE_VERSION=\"1.8.6\" "-DPACKAGE_STRING=\"spatialindex 1.8.6\"" -DPACKAGE_BUGREPORT=\" mhadji@gmail.com\" -DPACKAGE_URL=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DPACKAGE=\"spatialindex-src\" -DVERSION=\"1.8.6\" -DHAVE_FCNTL_H=1 -DHAVE_UNISTD_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_PTHREAD_H=1 -DHAVE_SYS_RESOURCE_H=1 -DHAVE_SYS_TIME_H=1 -DHAVE_STDINT_H=1 -DHAVE_GETTIMEOFDAY=1 -DHAVE_MEMSET=1 -DHAVE_MEMCPY=1 -DHAVE_BCOPY=1 -DHAVE_SRAND48=1 -I. -I../../include -I../../include/capi -Wall -Wno-long-long -pedantic -std=c++98 -O2 -DNDEBUG -MT sidx_api.lo -MD -MP -MF .deps/sidx_api.Tpo -c sidx_api.cc -fno-common -DPIC -o .libs/sidx_api.o *sidx_api.cc:45:8: **error: **thread-local storage is unsupported for the current target* static __thread struct * ^* 1 error generated. make[2]: *** [sidx_api.lo] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all-recursive] Error 1 Thanks, Matt Lawrence
On 6 Nov 2018, at 08:07, Matthew Knepley <knepley@gmail.com> wrote:
Okay, I restarted today and I get a compile problem with PyOP2, but I cannot see a compile error.
(firedrake) 1:/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Updating the git repository for libspatialindex
No need to rebuild libspatialindex
Cleaning COFFEE
Installing COFFEE/
Cleaning ufl
Installing ufl/
Cleaning fiat
Installing fiat/
Cleaning FInAT
Installing FInAT/
Cleaning tsfc
Installing tsfc/
Cleaning PyOP2
Installing PyOP2/
Traceback (most recent call last):
File "../bin/firedrake-update", line 1217, in <module>
install(p+"/")
File "../bin/firedrake-update", line 625, in install
run_pip_install(["-e", package])
File "../bin/firedrake-update", line 597, in run_pip_install
check_call(pipinstall + pipargs)
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'PyOP2/']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Thanks,
Matt
On Sat, Nov 3, 2018 at 8:27 AM Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 6:17 PM Nicolas Barral <n.barral@imperial.ac.uk> wrote:
On 02/11/2018 20:42, Matthew Knepley wrote:
On Fri, Nov 2, 2018 at 1:56 PM Lawrence Mitchell <wence@gmx.li <mailto:wence@gmx.li>> wrote:
> On 2 Nov 2018, at 17:53, Matthew Knepley <knepley@gmail.com <mailto:knepley@gmail.com>> wrote: > ...
> Successfully cloned repository libspatialindex > Checking out branch master > Successfully checked out branch master > Traceback (most recent call last): > File "../bin/firedrake-update", line 1207, in <module> > build_and_install_libspatialindex() > File "../bin/firedrake-update", line 808, in build_and_install_libspatialindex > check_call(["./autogen.sh"]) > File "../bin/firedrake-update", line 435, in check_call > log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output > **kwargs).stdout > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run > output=stdout, stderr=stderr) > subprocess.CalledProcessError: Command '['./autogen.sh']' returned non-zero exit status 1. > > > Update log saved in firedrake-update.log > > Here is the log. > > Ah ha! You need autotools, but do not check up front :) We get farther
When you first ran "firedrake-install" you said --no-package-manager, so we assumed that you had all the things we need.
> (firedrake) 127:/PETSc3/fem$ export PATH=${PATH}:${HOME}/MacSoftware/bin > (firedrake) :/PETSc3/fem$ which libtoolize > /Users/knepley/MacSoftware/bin/libtoolize > (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath > Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath > Updating the git repository for firedrake > Creating firedrake-update script. > Running ../bin/firedrake-update --no-update-script --honour-pythonpath > Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place. > Finding dependencies of PyOP2 > > Finding dependencies of firedrake > > Updating the git repository for petsc4py > Updating the git repository for COFFEE > Updating the git repository for ufl > Updating the git repository for fiat > Updating the git repository for FInAT > Updating the git repository for tsfc > Updating the git repository for PyOP2 > Updating the git repository for firedrake > Installing pip dependencies for COFFEE > Installing pip dependencies for ufl > No dependencies found. Skipping. > Installing pip dependencies for fiat > No dependencies found. Skipping. > Installing pip dependencies for FInAT > Installing pip dependencies for tsfc > Installing pip dependencies for PyOP2 > Installing pip dependencies for firedrake > Installing h5py > Updating the git repository for h5py > No need to rebuild h5py > Installing libspatialindex > Updating the git repository for libspatialindex > No need to rebuild libspatialindex > Cleaning COFFEE > Installing COFFEE/ > Cleaning ufl > Installing ufl/ > Cleaning fiat > Installing fiat/ > Cleaning FInAT > Installing FInAT/ > Cleaning tsfc > Installing tsfc/ > Cleaning PyOP2 > Traceback (most recent call last): > File "../bin/firedrake-update", line 1216, in <module> > clean(p) > File "../bin/firedrake-update", line 633, in clean > check_call(["python", "setup.py", "clean"]) > File "../bin/firedrake-update", line 435, in check_call > log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output > **kwargs).stdout > File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run > output=stdout, stderr=stderr) > subprocess.CalledProcessError: Command '['python', 'setup.py', 'clean']' returned non-zero exit status 1. > > > Update log saved in firedrake-update.log > > Here is the log. > > I see. My petsc4py is not visible in this venv. WIll fix that. > We should have installed petsc4py, but obviously didn't and then the firedrake-update has some logic to not rebuild it.
Just "pip install -U --no-deps firedrake/src/petsc4py"
That worked. We are almost there. However, I cannot interpret the error here:
(firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Updating the git repository for libspatialindex
No need to rebuild libspatialindex
Cleaning COFFEE
Installing COFFEE/
Cleaning ufl
Installing ufl/
Cleaning fiat
Installing fiat/
Cleaning FInAT
Installing FInAT/
Cleaning tsfc
Installing tsfc/
Cleaning PyOP2
Installing PyOP2/
Cleaning firedrake
Installing firedrake/
Traceback (most recent call last):
File "../bin/firedrake-update", line 1217, in <module>
install(p+"/")
File "../bin/firedrake-update", line 625, in install
run_pip_install(["-e", package])
File "../bin/firedrake-update", line 597, in run_pip_install
check_call(pipinstall + pipargs)
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'firedrake/']' returned non-zero exit status 1.
Update log saved in firedrake-update.log
Here is the log.
Thanks,
Matt
The log seems to say:
firedrake/spatialindex.c:614:10: fatal error: 'spatialindex/capi/sidx_api.h' file not found #include "spatialindex/capi/sidx_api.h"
which suggests libspatialindex wasn't installed (properly/at all) in your virtual env. Notably you should have a include/spatialindex in your virtual env.
Good hint! I did
(firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex/
(firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath
Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
Updating the git repository for firedrake
Creating firedrake-update script.
Running ../bin/firedrake-update --no-update-script --honour-pythonpath
Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.
Finding dependencies of PyOP2
Finding dependencies of firedrake
Updating the git repository for petsc4py
Updating the git repository for COFFEE
Updating the git repository for ufl
Updating the git repository for fiat
Updating the git repository for FInAT
Updating the git repository for tsfc
Updating the git repository for PyOP2
Updating the git repository for firedrake
Installing pip dependencies for COFFEE
Installing pip dependencies for ufl
No dependencies found. Skipping.
Installing pip dependencies for fiat
No dependencies found. Skipping.
Installing pip dependencies for FInAT
Installing pip dependencies for tsfc
Installing pip dependencies for PyOP2
Installing pip dependencies for firedrake
Installing h5py
Updating the git repository for h5py
No need to rebuild h5py
Installing libspatialindex
Cloning libspatialindex
Successfully cloned repository libspatialindex
Checking out branch master
Successfully checked out branch master
Traceback (most recent call last):
File "../bin/firedrake-update", line 1207, in <module>
build_and_install_libspatialindex()
File "../bin/firedrake-update", line 811, in build_and_install_libspatialindex
check_call(["make"])
File "../bin/firedrake-update", line 435, in check_call
log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['make']' returned non-zero exit status 2.
Update log saved in firedrake-update.log
This seems to be an honest-to-goodness bug in the configure for libspatialindex. You can't assume that my crappy g++ on Apple can do thread-local storage.
Here is the log.
Thanks,
Matt
I'm not sure how to fix that. If you don't want to wait for Lawrence or
David, I would try reinstalling like in the script (cf below), but without any guarantee. Otherwise restarting the installation from scratch is usually a good idea when things get too messed up. If you use the --venv-name option with the firedrake-install script, you can install Firedrake in a different directory so you can come back to your broken install later!
def build_and_install_libspatialindex(): log.info("Installing libspatialindex") if os.path.exists("libspatialindex"): log.info("Updating the git repository for libspatialindex") with directory("libspatialindex"): check_call(["git", "fetch"]) git_sha = check_output(["git", "rev-parse", "HEAD"]) git_sha_new = check_output(["git", "rev-parse", "@{u}"]) libspatialindex_changed = git_sha != git_sha_new if libspatialindex_changed: check_call(["git", "reset", "--hard"]) check_call(["git", "pull"]) else:
git_clone("git+https://github.com/firedrakeproject/libspatialindex.git") libspatialindex_changed = True
if libspatialindex_changed: with directory("libspatialindex"): # Clean source directory check_call(["git", "reset", "--hard"]) check_call(["git", "clean", "-f", "-x", "-d"]) # Patch Makefile.am to skip building test check_call(["sed", "-i", "-e", "/^SUBDIRS/s/ test//", "Makefile.am"]) # Build and install check_call(["./autogen.sh"]) check_call(["./configure", "--prefix=" + firedrake_env, "--enable-shared", "--disable-static"]) check_call(["make"]) check_call(["make", "install"]) else: log.info("No need to rebuild libspatialindex")
-- Nicolas
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/%7Eknepley/>
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- Nicolas Barral
Dept. of Earth Science and Engineering Imperial College London Royal School of Mines - Office 4.88 London SW7 2AZ
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
<firedrake-update.log>
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On 6 Nov 2018, at 08:27, Matthew Knepley <knepley@gmail.com> wrote:
sidx_api.cc:45:8: error: thread-local storage is unsupported for the current target static __thread struct
Hmm. The internet suggests that various Xcode versions removed (and then later reinstated) thread local storage in apple's clang. Eg here: https://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c... Seems like Xcode 8 and newer do support it. But you have a very old Xcode? Lawrence
On Tue, Nov 6, 2018 at 3:38 AM Lawrence Mitchell <wencel@gmail.com> wrote:
On 6 Nov 2018, at 08:27, Matthew Knepley <knepley@gmail.com> wrote:
*sidx_api.cc:45 <http://sidx_api.cc:45>:8: **error: **thread-local storage is unsupported for the current target*
static __thread struct
Hmm. The internet suggests that various Xcode versions removed (and then later reinstated) thread local storage in apple's clang. Eg here: https://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c...
Seems like Xcode 8 and newer do support it. But you have a very old Xcode?
Yep, the old good Xcode is an old Xcode :) Can we make __thread a #define which is checked by configure? Thanks, Matt
Lawrence _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
From: <firedrake-bounces@imperial.ac.uk> on behalf of Matthew Knepley <knepley@gmail.com> Reply-To: firedrake <firedrake@imperial.ac.uk> Date: Tuesday, 6 November 2018 at 08:44 To: firedrake <firedrake@imperial.ac.uk> Subject: Re: [firedrake] Installation with my own PETSc repository On Tue, Nov 6, 2018 at 3:38 AM Lawrence Mitchell <wencel@gmail.com<mailto:wencel@gmail.com>> wrote: On 6 Nov 2018, at 08:27, Matthew Knepley <knepley@gmail.com<mailto:knepley@gmail.com>> wrote: sidx_api.cc:45<http://sidx_api.cc:45>:8: error: thread-local storage is unsupported for the current target static __thread struct Hmm. The internet suggests that various Xcode versions removed (and then later reinstated) thread local storage in apple's clang. Eg here: https://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c... Seems like Xcode 8 and newer do support it. But you have a very old Xcode? Yep, the old good Xcode is an old Xcode :) Can we make __thread a #define which is checked by configure? In principle, yes. Though this means we get further into maintaining a libspatialindex fork. Do we really need this? Thanks, Matt Lawrence _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>
On Tue, Nov 6, 2018 at 4:54 AM Ham, David A <david.ham@imperial.ac.uk> wrote:
*From: *<firedrake-bounces@imperial.ac.uk> on behalf of Matthew Knepley < knepley@gmail.com> *Reply-To: *firedrake <firedrake@imperial.ac.uk> *Date: *Tuesday, 6 November 2018 at 08:44 *To: *firedrake <firedrake@imperial.ac.uk> *Subject: *Re: [firedrake] Installation with my own PETSc repository
On Tue, Nov 6, 2018 at 3:38 AM Lawrence Mitchell <wencel@gmail.com> wrote:
On 6 Nov 2018, at 08:27, Matthew Knepley <knepley@gmail.com> wrote:
*sidx_api.cc:45 <http://sidx_api.cc:45>:8: error: thread-local storage is unsupported for the current target*
static __thread struct
Hmm. The internet suggests that various Xcode versions removed (and then later reinstated) thread local storage in apple's clang. Eg here: https://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c...
Seems like Xcode 8 and newer do support it. But you have a very old Xcode?
Yep, the old good Xcode is an old Xcode :)
Can we make __thread a #define which is checked by configure?
In principle, yes. Though this means we get further into maintaining a libspatialindex fork. Do we really need this?
I am a fan of working around compiler deficiencies, rather than requiring users to change compilers. Its not a broken thing, but rather an unsupported thing. This is in the spirit of PETSc supporting old version of Python, old MPI, etc. If you tell me where it goes, I will do it. Matt
Thanks,
Matt
Lawrence
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/> _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On 6 Nov 2018, at 14:18, Matthew Knepley <knepley@gmail.com> wrote:
On Tue, Nov 6, 2018 at 4:54 AM Ham, David A <david.ham@imperial.ac.uk> wrote:
From: <firedrake-bounces@imperial.ac.uk> on behalf of Matthew Knepley <knepley@gmail.com> Reply-To: firedrake <firedrake@imperial.ac.uk> Date: Tuesday, 6 November 2018 at 08:44 To: firedrake <firedrake@imperial.ac.uk> Subject: Re: [firedrake] Installation with my own PETSc repository
On Tue, Nov 6, 2018 at 3:38 AM Lawrence Mitchell <wencel@gmail.com> wrote:
On 6 Nov 2018, at 08:27, Matthew Knepley <knepley@gmail.com> wrote:
sidx_api.cc:45:8: error: thread-local storage is unsupported for the current target static __thread struct
Hmm. The internet suggests that various Xcode versions removed (and then later reinstated) thread local storage in apple's clang. Eg here: https://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c...
Seems like Xcode 8 and newer do support it. But you have a very old Xcode?
Yep, the old good Xcode is an old Xcode :)
Can we make __thread a #define which is checked by configure?
In principle, yes. Though this means we get further into maintaining a libspatialindex fork. Do we really need this?
I am a fan of working around compiler deficiencies, rather than requiring users to change compilers. Its not a broken thing, but rather an unsupported thing. This is in the spirit of PETSc supporting old version of Python, old MPI, etc. If you tell me where it goes, I will do it.
libspatialindex has a configure script (generated via autoconf), we maintain a "known good" fork here: github.com/firedrakeproject/libspatialindex. I don't understand autoconf very well, but I think the branch at configure-thread-local DTRT. Can you checkout that branch and try building? Cheers, Lawrence
On Tue, Nov 6, 2018 at 9:33 AM Lawrence Mitchell <wencel@gmail.com> wrote:
On 6 Nov 2018, at 14:18, Matthew Knepley <knepley@gmail.com> wrote:
On Tue, Nov 6, 2018 at 4:54 AM Ham, David A <david.ham@imperial.ac.uk> wrote:
From: <firedrake-bounces@imperial.ac.uk> on behalf of Matthew Knepley < knepley@gmail.com> Reply-To: firedrake <firedrake@imperial.ac.uk> Date: Tuesday, 6 November 2018 at 08:44 To: firedrake <firedrake@imperial.ac.uk> Subject: Re: [firedrake] Installation with my own PETSc repository
On Tue, Nov 6, 2018 at 3:38 AM Lawrence Mitchell <wencel@gmail.com> wrote:
On 6 Nov 2018, at 08:27, Matthew Knepley <knepley@gmail.com> wrote:
sidx_api.cc:45:8: error: thread-local storage is unsupported for the current target static __thread struct
Hmm. The internet suggests that various Xcode versions removed (and then later reinstated) thread local storage in apple's clang. Eg here: https://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c...
Seems like Xcode 8 and newer do support it. But you have a very old Xcode?
Yep, the old good Xcode is an old Xcode :)
Can we make __thread a #define which is checked by configure?
In principle, yes. Though this means we get further into maintaining a libspatialindex fork. Do we really need this?
I am a fan of working around compiler deficiencies, rather than requiring users to change compilers. Its not a broken thing, but rather an unsupported thing. This is in the spirit of PETSc supporting old version of Python, old MPI, etc. If you tell me where it goes, I will do it.
libspatialindex has a configure script (generated via autoconf), we maintain a "known good" fork here: github.com/firedrakeproject/libspatialindex. I don't understand autoconf very well, but I think the branch at configure-thread-local DTRT.
Can you checkout that branch and try building?
Okay, I did this and failed on std::shared_ptr. It seems that libspatialindex is just not going to build at all with my C++ compiler. Matt
Cheers,
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
On Tue, 6 Nov 2018, Matthew Knepley wrote:
On Tue, Nov 6, 2018 at 3:38 AM Lawrence Mitchell <wencel@gmail.com> wrote:
On 6 Nov 2018, at 08:27, Matthew Knepley <knepley@gmail.com> wrote:
*sidx_api.cc:45 <http://sidx_api.cc:45>:8: **error: **thread-local storage is unsupported for the current target*
static __thread struct
Hmm. The internet suggests that various Xcode versions removed (and then later reinstated) thread local storage in apple's clang. Eg here: https://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c...
Seems like Xcode 8 and newer do support it. But you have a very old Xcode?
Yep, the old good Xcode is an old Xcode :)
And EOLed :) time for an update to latest osx/xcode - as old osx/Xcode is causing multiple problems for you.. Satish
Can we make __thread a #define which is checked by configure?
Thanks,
Matt
Lawrence _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
participants (7)
-
Andrew McRae
-
Balay, Satish
-
Ham, David A
-
Lawrence Mitchell
-
Lawrence Mitchell
-
Matthew Knepley
-
Nicolas Barral