On Fri, Nov 2, 2018 at 1:11 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 16:59, Matthew Knepley <knepley@gmail.com> wrote:
On Fri, Nov 2, 2018 at 12:54 PM Lawrence Mitchell <wence@gmx.li> wrote:
On 2 Nov 2018, at 01:37, Matthew Knepley <knepley@gmail.com> wrote:
Well crap. h5py failed, and it does not look like a simple env fix. It is complaining about an SSL certificate. Is there a way to point to something that uses HTTP?
Hmmph, this is very odd. Because that failed checking the certificate for a host it had previously downloaded from (and passed the certificate checks).
This failed at a state late enough in the game that we can try building "in place" in the virtualenv:
$ . firedrake/bin/activate $ cd firedrake/src/h5py $ export CC=mpicc # I assume you got PETSc to build hdf5 $ export HDF5_DIR=$PETSC_DIR/$PETSC_ARCH $ export HDF5_MPI=ON $ pip install .
Okay, this line
pip install .
fails with
(firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Complete output from command python setup.py egg_info: Download error on https://pypi.org/simple/pkgconfig/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833) -- Some packages may not be found! Couldn't find index page for 'pkgconfig' (maybe misspelled?) Download error on https://pypi.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833) -- Some packages may not be found! No local packages or working download links found for pkgconfig Traceback (most recent call last): File "<string>", line 1, in <module> File "/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-build-vy7d9wuj/setup.py", line 168, in <module> cmdclass = CMDCLASS, File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in setup _install_setup_requires(attrs) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/__init__.py", line 134, in _install_setup_requires dist.fetch_build_eggs(dist.setup_requires) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs replace_conflicting=True, File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 777, in resolve replace_conflicting=replace_conflicting File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1060, in best_match return self.obtain(req, installer) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1072, in obtain return installer(requirement) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/dist.py", line 581, in fetch_build_egg return cmd.easy_install(req) File "/PETSc3/fem/firedrake/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 670, in easy_install raise DistutilsError(msg) distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pkgconfig')
---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-build-vy7d9wuj/
OK, this is the same error as before, but we can hopefully debug it in isolation. It's failing certificate checks on downloading pkgconfig.
So in the activated virtualenv, try:
$ pip install -vvv pkgconfig
And see what we get.
That seems to have worked. Looks like it had stale URLs? (firedrake) :/PETSc3/fem/firedrake$ pip install -vvv pkgconfig Created temporary directory: /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-ephem-wheel-cache-fjx4plz3 Created temporary directory: /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-tracker-f08je7xv Created requirements tracker '/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-tracker-f08je7xv' Created temporary directory: /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-install-ftkb2slk Collecting pkgconfig 1 location(s) to search for versions of pkgconfig: * https://pypi.org/simple/pkgconfig/ Getting page https://pypi.org/simple/pkgconfig/ Looking up "https://pypi.org/simple/pkgconfig/" in the cache Request header has "max_age" as 0, cache bypassed Starting new HTTPS connection (1): pypi.org:443 https://pypi.org:443 "GET /simple/pkgconfig/ HTTP/1.1" 200 1261 Updating cache with response from "https://pypi.org/simple/pkgconfig/" Caching due to etag Analyzing links from page https://pypi.org/simple/pkgconfig/ Found link https://files.pythonhosted.org/packages/78/b5/b1ca52c3a4e2847f5408d776c8e513... (from https://pypi.org/simple/pkgconfig/), version: 0.1.0 Found link https://files.pythonhosted.org/packages/6f/0b/1d3498423cfcac1db48eca7fc867d4... (from https://pypi.org/simple/pkgconfig/), version: 1.0.0 Found link https://files.pythonhosted.org/packages/87/35/4af9634270c00e3411cf951b7e0ea7... (from https://pypi.org/simple/pkgconfig/), version: 1.1.0 Found link https://files.pythonhosted.org/packages/ec/1c/70f81f8400e767277c11770ce5eb38... (from https://pypi.org/simple/pkgconfig/), version: 1.2.0 Found link https://files.pythonhosted.org/packages/1f/90/4ce89b250a9fdf87a73d109a496cbe... (from https://pypi.org/simple/pkgconfig/), version: 1.2.1 Found link https://files.pythonhosted.org/packages/9d/ba/80910bbed2b4e646a6adab4474d2e5... (from https://pypi.org/simple/pkgconfig/), version: 1.2.2 Skipping link https://files.pythonhosted.org/packages/fe/1b/e36407744b0118efd020f0aceaa198... (from https://pypi.org/simple/pkgconfig/); unsupported archive format: .egg Found link https://files.pythonhosted.org/packages/b8/f9/0930d35b5035201865118cbb79cf4d... (from https://pypi.org/simple/pkgconfig/), version: 1.3.0 Found link https://files.pythonhosted.org/packages/94/ea/b6cc7007afb696d6a3c7bc0e62142f... (from https://pypi.org/simple/pkgconfig/), version: 1.3.1 Found link https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... (from https://pypi.org/simple/pkgconfig/) (requires-python:>=2.6, !=3.0.*, !=3.1.*, !=3.2.*), version: 1.4.0 Found link https://files.pythonhosted.org/packages/17/b7/b6e2b5798eeadb6bfbebf6c332c898... (from https://pypi.org/simple/pkgconfig/) (requires-python:>=2.6, !=3.0.*, !=3.1.*, !=3.2.*), version: 1.4.0 Using version 1.4.0 (newest of versions: 0.1.0, 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.3.0, 1.3.1, 1.4.0) Created temporary directory: /private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-unpack-2acg6mhp Looking up " https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a..." in the cache No cache entry available Starting new HTTPS connection (1): files.pythonhosted.org:443 https://files.pythonhosted.org:443 "GET /packages/e1/8a/c205fad800317a59e8d218981a8a7a3a00ec69c7fd55eca58dd08e6b0a18/pkgconfig-1.4.0-py2.py3-none-any.whl HTTP/1.1" 200 7247 Downloading https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... Downloading from URL https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... (from https://pypi.org/simple/pkgconfig/) (requires-python:>=2.6, !=3.0.*, !=3.1.*, !=3.2.*) Ignoring unknown cache-control directive: immutable Updating cache with response from " https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... " Caching due to etag Added pkgconfig from https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... to build tracker '/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-tracker-f08je7xv' Removed pkgconfig from https://files.pythonhosted.org/packages/e1/8a/c205fad800317a59e8d218981a8a7a... from build tracker '/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-tracker-f08je7xv' Installing collected packages: pkgconfig Successfully installed pkgconfig-1.4.0 Cleaning up... Removed build tracker '/private/var/folders/hk/tc1pd0g57l78rpt0lttc1sfh0000gn/T/pip-req-tracker-f08je7xv' I went on to cd src/h5py pip install . and it that worked (firedrake) :/PETSc3/fem/firedrake$ pushd src/h5py/ /PETSc3/fem/firedrake/src/h5py /PETSc3/fem/firedrake /PETSc3/fem (firedrake) firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install . Processing /PETSc3/fem/firedrake/src/h5py Requirement already satisfied: numpy>=1.7 in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.15.3) Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py==2.8.0.post0) (1.11.0) Installing collected packages: h5py Found existing installation: h5py 2.8.0 Uninstalling h5py-2.8.0: Successfully uninstalled h5py-2.8.0 Running setup.py install for h5py ... done Successfully installed h5py-2.8.0.post0 How do I restart the whole Firedrake now? Thanks, Matt
but I messed it up initially using
pip install h5py
which succeeds with
(firedrake) 1firedrake:/PETSc3/fem/firedrake/src/h5py$ pip install h5py Collecting h5py Downloading https://files.pythonhosted.org/packages/44/81/50a0560aac57a33c2a624d9e160735... (6.0MB) 100% |████████████████████████████████| 6.1MB 677kB/s Requirement already satisfied: six in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py) (1.11.0) Requirement already satisfied: numpy>=1.7 in /PETSc3/fem/firedrake/lib/python3.6/site-packages (from h5py) (1.15.3) Installing collected packages: h5py Successfully installed h5py-2.8.0
Can I keep going from here? How do I restart the Firedrake install?
This won't work, unfortunately. Or at least, I don't trust it will work. The h5py wheel pacakges come with prebuilt HDF5 libraries inside them and that will almost certainly break things at runtime, because the compiled parts of Firedrake will now be linked against a different HDF5 from the PETSc one. And this gets messy.
Lawrence
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>