We saw this:

    error: command 'mpicc' failed with exit status 1

when build a c module in pyop2. Can you:

cd firedrake/src/PyOP2

make

and post the results. Output goes to make.log

Lawrence

On 6 Nov 2018, at 08:07, Matthew Knepley <knepley@gmail.com> wrote:

Okay, I restarted today and I get a compile problem with PyOP2, but I cannot see a compile error.

(firedrake)  1:/PETSc3/fem$ firedrake-update --honour-pythonpath 

Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath

Updating the git repository for firedrake

Creating firedrake-update script.

Running ../bin/firedrake-update --no-update-script --honour-pythonpath

Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.

Finding dependencies of PyOP2

Finding dependencies of firedrake

Updating the git repository for petsc4py

Updating the git repository for COFFEE

Updating the git repository for ufl

Updating the git repository for fiat

Updating the git repository for FInAT

Updating the git repository for tsfc

Updating the git repository for PyOP2

Updating the git repository for firedrake

Installing pip dependencies for COFFEE

Installing pip dependencies for ufl

No dependencies found. Skipping.

Installing pip dependencies for fiat

No dependencies found. Skipping.

Installing pip dependencies for FInAT

Installing pip dependencies for tsfc

Installing pip dependencies for PyOP2

Installing pip dependencies for firedrake

Installing h5py

Updating the git repository for h5py

No need to rebuild h5py

Installing libspatialindex

Updating the git repository for libspatialindex

No need to rebuild libspatialindex

Cleaning COFFEE

Installing COFFEE/

Cleaning ufl

Installing ufl/

Cleaning fiat

Installing fiat/

Cleaning FInAT

Installing FInAT/

Cleaning tsfc

Installing tsfc/

Cleaning PyOP2

Installing PyOP2/

Traceback (most recent call last):

  File "../bin/firedrake-update", line 1217, in <module>

    install(p+"/")

  File "../bin/firedrake-update", line 625, in install

    run_pip_install(["-e", package])

  File "../bin/firedrake-update", line 597, in run_pip_install

    check_call(pipinstall + pipargs)

  File "../bin/firedrake-update", line 435, in check_call

    log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())

  File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output

    **kwargs).stdout

  File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run

    output=stdout, stderr=stderr)

subprocess.CalledProcessError: Command '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'PyOP2/']' returned non-zero exit status 1.

Update log saved in firedrake-update.log

  Thanks,

    Matt

On Sat, Nov 3, 2018 at 8:27 AM Matthew Knepley <knepley@gmail.com> wrote:

On Fri, Nov 2, 2018 at 6:17 PM Nicolas Barral <n.barral@imperial.ac.uk> wrote:

On 02/11/2018 20:42, Matthew Knepley wrote:
> On Fri, Nov 2, 2018 at 1:56 PM Lawrence Mitchell <wence@gmx.li
> <mailto:wence@gmx.li>> wrote:
>
>
>
>      > On 2 Nov 2018, at 17:53, Matthew Knepley <knepley@gmail.com
>     <mailto:knepley@gmail.com>> wrote:
>      > ...
>
>      > Successfully cloned repository libspatialindex
>      > Checking out branch master
>      > Successfully checked out branch master
>      > Traceback (most recent call last):
>      >   File "../bin/firedrake-update", line 1207, in <module>
>      >     build_and_install_libspatialindex()
>      >   File "../bin/firedrake-update", line 808, in
>     build_and_install_libspatialindex
>      >     check_call(["./autogen.sh"])
>      >   File "../bin/firedrake-update", line 435, in check_call
>      >     log.debug(subprocess.check_output(arguments,
>     stderr=subprocess.STDOUT, env=env).decode())
>      >   File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py",
>     line 336, in check_output
>      >     **kwargs).stdout
>      >   File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py",
>     line 418, in run
>      >     output=stdout, stderr=stderr)
>      > subprocess.CalledProcessError: Command '['./autogen.sh']'
>     returned non-zero exit status 1.
>      >
>      >
>      > Update log saved in firedrake-update.log
>      >
>      > Here is the log.
>      >
>      > Ah ha! You need autotools, but do not check up front :) We get
>     farther
>
>     When you first ran "firedrake-install" you said
>     --no-package-manager, so we assumed that you had all the things we need.
>
>
>      > (firedrake)  127:/PETSc3/fem$ export
>     PATH=${PATH}:${HOME}/MacSoftware/bin
>      > (firedrake) :/PETSc3/fem$ which libtoolize
>      > /Users/knepley/MacSoftware/bin/libtoolize
>      > (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath
>      > Running /PETSc3/fem/firedrake/bin/firedrake-update
>     --honour-pythonpath
>      > Updating the git repository for firedrake
>      > Creating firedrake-update script.
>      > Running ../bin/firedrake-update --no-update-script
>     --honour-pythonpath
>      > Xcode and homebrew installation disabled. Proceeding on the rash
>     assumption that packaged dependencies are in place.
>      > Finding dependencies of PyOP2
>      >
>      > Finding dependencies of firedrake
>      >
>      > Updating the git repository for petsc4py
>      > Updating the git repository for COFFEE
>      > Updating the git repository for ufl
>      > Updating the git repository for fiat
>      > Updating the git repository for FInAT
>      > Updating the git repository for tsfc
>      > Updating the git repository for PyOP2
>      > Updating the git repository for firedrake
>      > Installing pip dependencies for COFFEE
>      > Installing pip dependencies for ufl
>      > No dependencies found. Skipping.
>      > Installing pip dependencies for fiat
>      > No dependencies found. Skipping.
>      > Installing pip dependencies for FInAT
>      > Installing pip dependencies for tsfc
>      > Installing pip dependencies for PyOP2
>      > Installing pip dependencies for firedrake
>      > Installing h5py
>      > Updating the git repository for h5py
>      > No need to rebuild h5py
>      > Installing libspatialindex
>      > Updating the git repository for libspatialindex
>      > No need to rebuild libspatialindex
>      > Cleaning COFFEE
>      > Installing COFFEE/
>      > Cleaning ufl
>      > Installing ufl/
>      > Cleaning fiat
>      > Installing fiat/
>      > Cleaning FInAT
>      > Installing FInAT/
>      > Cleaning tsfc
>      > Installing tsfc/
>      > Cleaning PyOP2
>      > Traceback (most recent call last):
>      >   File "../bin/firedrake-update", line 1216, in <module>
>      >     clean(p)
>      >   File "../bin/firedrake-update", line 633, in clean
>      >     check_call(["python", "setup.py", "clean"])
>      >   File "../bin/firedrake-update", line 435, in check_call
>      >     log.debug(subprocess.check_output(arguments,
>     stderr=subprocess.STDOUT, env=env).decode())
>      >   File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py",
>     line 336, in check_output
>      >     **kwargs).stdout
>      >   File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py",
>     line 418, in run
>      >     output=stdout, stderr=stderr)
>      > subprocess.CalledProcessError: Command '['python', 'setup.py',
>     'clean']' returned non-zero exit status 1.
>      >
>      >
>      > Update log saved in firedrake-update.log
>      >
>      > Here is the log.
>      >
>      > I see. My petsc4py is not visible in this venv. WIll fix that.
>      >
>     We should have installed petsc4py, but obviously didn't and then the
>     firedrake-update has some logic to not rebuild it.
>
>     Just "pip install -U --no-deps firedrake/src/petsc4py"
>
>
> That worked. We are almost there. However, I cannot interpret the error
> here:
>
> (firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath
>
> Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath
>
> Updating the git repository for firedrake
>
> Creating firedrake-update script.
>
> Running ../bin/firedrake-update --no-update-script --honour-pythonpath
>
> Xcode and homebrew installation disabled. Proceeding on the rash
> assumption that packaged dependencies are in place.
>
> Finding dependencies of PyOP2
>
>
> Finding dependencies of firedrake
>
>
> Updating the git repository for petsc4py
>
> Updating the git repository for COFFEE
>
> Updating the git repository for ufl
>
> Updating the git repository for fiat
>
> Updating the git repository for FInAT
>
> Updating the git repository for tsfc
>
> Updating the git repository for PyOP2
>
> Updating the git repository for firedrake
>
> Installing pip dependencies for COFFEE
>
> Installing pip dependencies for ufl
>
> No dependencies found. Skipping.
>
> Installing pip dependencies for fiat
>
> No dependencies found. Skipping.
>
> Installing pip dependencies for FInAT
>
> Installing pip dependencies for tsfc
>
> Installing pip dependencies for PyOP2
>
> Installing pip dependencies for firedrake
>
> Installing h5py
>
> Updating the git repository for h5py
>
> No need to rebuild h5py
>
> Installing libspatialindex
>
> Updating the git repository for libspatialindex
>
> No need to rebuild libspatialindex
>
> Cleaning COFFEE
>
> Installing COFFEE/
>
> Cleaning ufl
>
> Installing ufl/
>
> Cleaning fiat
>
> Installing fiat/
>
> Cleaning FInAT
>
> Installing FInAT/
>
> Cleaning tsfc
>
> Installing tsfc/
>
> Cleaning PyOP2
>
> Installing PyOP2/
>
> Cleaning firedrake
>
> Installing firedrake/
>
> Traceback (most recent call last):
>
> File "../bin/firedrake-update", line 1217, in <module>
>
> install(p+"/")
>
> File "../bin/firedrake-update", line 625, in install
>
> run_pip_install(["-e", package])
>
> File "../bin/firedrake-update", line 597, in run_pip_install
>
> check_call(pipinstall + pipargs)
>
> File "../bin/firedrake-update", line 435, in check_call
>
> log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT,
> env=env).decode())
>
> File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336,
> in check_output
>
> **kwargs).stdout
>
> File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418,
> in run
>
> output=stdout, stderr=stderr)
>
> subprocess.CalledProcessError: Command
> '['/PETSc3/fem/firedrake/bin/python', '-m', 'pip', 'install',
> '--no-binary', 'mpi4py', '--no-deps', '-vvv', '-e', 'firedrake/']'
> returned non-zero exit status 1.
>
>
>
> Update log saved in firedrake-update.log
>
> Here is the log.
>
>    Thanks,
>
>      Matt
>

The log seems to say:
 >firedrake/spatialindex.c:614:10: fatal error:
'spatialindex/capi/sidx_api.h' file not found
 >    #include "spatialindex/capi/sidx_api.h"

which suggests libspatialindex wasn't installed (properly/at all) in
your virtual env. Notably you should have a include/spatialindex in your
virtual env.

Good hint! I did

(firedrake) :/PETSc3/fem$ rm -rf firedrake/src/libspatialindex/

(firedrake) :/PETSc3/fem$ firedrake-update --honour-pythonpath

Running /PETSc3/fem/firedrake/bin/firedrake-update --honour-pythonpath

Updating the git repository for firedrake

Creating firedrake-update script.

Running ../bin/firedrake-update --no-update-script --honour-pythonpath

Xcode and homebrew installation disabled. Proceeding on the rash assumption that packaged dependencies are in place.

Finding dependencies of PyOP2

Finding dependencies of firedrake

Updating the git repository for petsc4py

Updating the git repository for COFFEE

Updating the git repository for ufl

Updating the git repository for fiat

Updating the git repository for FInAT

Updating the git repository for tsfc

Updating the git repository for PyOP2

Updating the git repository for firedrake

Installing pip dependencies for COFFEE

Installing pip dependencies for ufl

No dependencies found. Skipping.

Installing pip dependencies for fiat

No dependencies found. Skipping.

Installing pip dependencies for FInAT

Installing pip dependencies for tsfc

Installing pip dependencies for PyOP2

Installing pip dependencies for firedrake

Installing h5py

Updating the git repository for h5py

No need to rebuild h5py

Installing libspatialindex

Cloning libspatialindex

Successfully cloned repository libspatialindex

Checking out branch master

Successfully checked out branch master

Traceback (most recent call last):

  File "../bin/firedrake-update", line 1207, in <module>

    build_and_install_libspatialindex()

  File "../bin/firedrake-update", line 811, in build_and_install_libspatialindex

    check_call(["make"])

  File "../bin/firedrake-update", line 435, in check_call

    log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode())

  File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 336, in check_output

    **kwargs).stdout

  File "/Users/knepley/MacSoftware/lib/python3.6/subprocess.py", line 418, in run

    output=stdout, stderr=stderr)

subprocess.CalledProcessError: Command '['make']' returned non-zero exit status 2.

Update log saved in firedrake-update.log

 

This seems to be an honest-to-goodness bug in the configure for libspatialindex. You can't assume that my

crappy g++ on Apple can do thread-local storage.

Here is the log.

  Thanks,

     Matt

I'm not sure how to fix that. If you don't want to wait for Lawrence or
David, I would try reinstalling like in the script (cf below), but
without any guarantee.
Otherwise restarting the installation from scratch is usually a good
idea when things get too messed up. If you use the --venv-name option
with the firedrake-install script, you can install Firedrake in a
different directory so you can come back to your broken install later!

def build_and_install_libspatialindex():
     log.info("Installing libspatialindex")
     if os.path.exists("libspatialindex"):
         log.info("Updating the git repository for libspatialindex")
         with directory("libspatialindex"):
             check_call(["git", "fetch"])
             git_sha = check_output(["git", "rev-parse", "HEAD"])
             git_sha_new = check_output(["git", "rev-parse", "@{u}"])
             libspatialindex_changed = git_sha != git_sha_new
             if libspatialindex_changed:
                 check_call(["git", "reset", "--hard"])
                 check_call(["git", "pull"])
     else:

git_clone("git+https://github.com/firedrakeproject/libspatialindex.git")
         libspatialindex_changed = True

     if libspatialindex_changed:
         with directory("libspatialindex"):
             # Clean source directory
             check_call(["git", "reset", "--hard"])
             check_call(["git", "clean", "-f", "-x", "-d"])
             # Patch Makefile.am to skip building test
             check_call(["sed", "-i", "-e", "/^SUBDIRS/s/ test//",
"Makefile.am"])
             # Build and install
             check_call(["./autogen.sh"])
             check_call(["./configure", "--prefix=" + firedrake_env,
                         "--enable-shared", "--disable-static"])
             check_call(["make"])
             check_call(["make", "install"])
     else:
         log.info("No need to rebuild libspatialindex")



--
Nicolas

>     Lawrence
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/%7Eknepley/>
>
>
> _______________________________________________
> firedrake mailing list
> firedrake@imperial.ac.uk
> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
>


--
Nicolas Barral

Dept. of Earth Science and Engineering
Imperial College London
Royal School of Mines - Office 4.88
London SW7 2AZ

--

What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/

--

What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/

<firedrake-update.log>

_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake

_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake