- firedrake - imperial.ac.uk

Re: [firedrake] Interacting with Gusto
by David Ham 03 Apr '17

03 Apr '17

Hi all, please keep discussions on list to spread the support load and ensure everyone has the chance to learn from the answers. Regards, David On Sun, 2 Apr 2017 at 15:58, Shipton, Jemma <j.shipton(a)imperial.ac.uk> wrote: > Hi Joy, > > > Yes, sorry, I'm afraid that there isn't much documentation on Gusto at the > moment - I hope to write some soon. > > > We don't have a straightforward way to interact with "physics" just yet, > although we are working on implementing moist physics. > > > I'll email you off-list to chat more about what you need... > > > Thanks! > > > Jemma > ------------------------------ > *From:* firedrake-bounces(a)imperial.ac.uk <firedrake-bounces(a)imperial.ac.uk> > on behalf of Joy merwin monteiro <joy.merwin(a)gmail.com> > *Sent:* 02 April 2017 15:44:21 > *To:* firedrake > *Subject:* [firedrake] Interacting with Gusto > > Hello, > > I have recently heard about Firedrake and Gusto, and am looking > forward to using it! > > I am working on a project named climt ( > http://climt.readthedocs.io/en/latest/readme.html) > and I was wondering whether I could get Gusto/Firedrake to work with the > components already available there. > > my question was what interface does Gusto provide for interaction with > "physics", like say, a radiative transfer code? > > In other words, is there a straightforward way to access model arrays, > calculate physics tendencies, and provide them back to the dynamical > core? > > I tried to look for more documentation about Gusto, but could not find any. > I would be grateful if you could point me in the right direction. > > TIA, > Joy > > -- > The best ruler, when he finishes his > tasks and completes his affairs, > the people say > “It all happened naturally” > > - Te Tao Ch'ing > -- Dr David Ham Department of Mathematics Imperial College London

3 4

New build test system
by David Ham 03 Apr '17

03 Apr '17

Dear Firedrakers, I am happy to announce that the new build test system is up and operational. We have full testing of the Firedrake stack on Ubuntu using Jenkins. We also have new build hardware which results in very fast build times. Currently the entire build and test cycle runs in around 40 minutes. We also have new Mac hardware and will be bringing it online as soon as we iron out a few network issues. What does this mean for you? 1. Please rebase your branches on master (or merge master into your branches). There are changes in Jenkinsfile and firedrake-install which are needed for the build test system. 2. Build test fails are real. If you had slipped into the habit of assuming that build test fails were spurious because Drone was flaky, this is no longer true. Please pay attention to build test failures. 3. The new build test architecture is not being used to full capacity. In my copious free time I will set up Thetis and Gusto to use the new system too. Firedrake components which have quick build test cycles can stay on Travis-CI. Regards, David -- Dr David Ham Department of Mathematics Imperial College London

1 0

Interacting with Gusto
by Joy merwin monteiro 02 Apr '17

02 Apr '17

Hello, I have recently heard about Firedrake and Gusto, and am looking forward to using it! I am working on a project named climt ( http://climt.readthedocs.io/en/latest/readme.html) and I was wondering whether I could get Gusto/Firedrake to work with the components already available there. my question was what interface does Gusto provide for interaction with "physics", like say, a radiative transfer code? In other words, is there a straightforward way to access model arrays, calculate physics tendencies, and provide them back to the dynamical core? I tried to look for more documentation about Gusto, but could not find any. I would be grateful if you could point me in the right direction. TIA, Joy -- The best ruler, when he finishes his tasks and completes his affairs, the people say “It all happened naturally” - Te Tao Ch'ing

2 1

Re: [firedrake] Parameter estimation and optimal experimental design with Firedrake
by David Ham 02 Apr '17

02 Apr '17

Hi Henrik, OK, for this you need Dolfin-adjoint which, despite its name, also works with Firedrake. Dolfin-adjoint solves the adjoint to a PDE solved with Firedrake, including the time-dependent case, with respect to a user-defined functional which can be a function of both space and time. Dolfin-adjoint also wraps several popular optimisation packages to provide a high level interface to solve the optimisation problem, You install Dolfin-adjoint in Firedrake by running: firedrake-update --adjoint This will install dolfin-adjoint and libadjoint in the Firedrake virtualenv, and future updates will update dolfin-adjoint as well as the rest of Firedrake. Once you've installed, dolfin-adjoint is documented at dolfin-adjoint.org . Most of the documentation there refers to FEniCS, but working with Firedrake is essentially identical. There is an example optimisation problem in the Firedrake Jupyter Notebooks ( http://firedrakeproject.org/notebooks.html) and you can also look at the Firedrake tests in dolfin-adjoint, which will be installed in src/dolfin-adjoint/tests_firedrake Cheers, David On Fri, 31 Mar 2017 at 20:48 Buesing, Henrik <HBuesing(a)eonerc.rwth-aachen.de> wrote: > Dear all, > > > > How would I do parameter estimation (and optimal experimental design) in > Firedrake? > > > > I have a time-dependent forward model consisting of a PDE system and > depending on certain parameters. I also have time-dependent > measurements/data. Now, I would like to minimize some functional, which > measures the distance from the forward model to the recorded data. > > > > How would I do this in Firedrake? Is there an example for this? > > > > In a next step, I would like to do optimal experimental design, where one > maximizes the Fisher information (e.g. minimize the trace of the inverse of > the information matrix) with respect to position. > > > > Any ideas on how to approach this? > > Thank you for your help! > Henrik > > > > > > -- > > Dipl.-Math. Henrik Büsing > > Institute for Applied Geophysics and Geothermal Energy > > E.ON Energy Research Center > > RWTH Aachen University > > > > Mathieustr. | Tel +49 (0)241 80 49907 <+49%20241%208049907> > > 52074 Aachen, Germany | Fax +49 (0)241 80 49889 <+49%20241%208049889> > > > > http://www.eonerc.rwth-aachen.de/GGE > > hbuesing(a)eonerc.rwth-aachen.de > > > -- Dr David Ham Department of Mathematics Imperial College London

2 1

Parameter estimation and optimal experimental design with Firedrake
by Buesing, Henrik 31 Mar '17

31 Mar '17

Dear all, How would I do parameter estimation (and optimal experimental design) in Firedrake? I have a time-dependent forward model consisting of a PDE system and depending on certain parameters. I also have time-dependent measurements/data. Now, I would like to minimize some functional, which measures the distance from the forward model to the recorded data. How would I do this in Firedrake? Is there an example for this? In a next step, I would like to do optimal experimental design, where one maximizes the Fisher information (e.g. minimize the trace of the inverse of the information matrix) with respect to position. Any ideas on how to approach this? Thank you for your help! Henrik -- Dipl.-Math. Henrik Büsing Institute for Applied Geophysics and Geothermal Energy E.ON Energy Research Center RWTH Aachen University Mathieustr. | Tel +49 (0)241 80 49907 52074 Aachen, Germany | Fax +49 (0)241 80 49889 http://www.eonerc.rwth-aachen.de/GGE hbuesing(a)eonerc.rwth-aachen.de

1 0

Re: [firedrake] close to fixing my nonlinear QG solver
by David Ham 31 Mar '17

31 Mar '17

Pull requests always welcome ;) On Thu, 30 Mar 2017 at 13:24 Francis Poulin <fpoulin(a)uwaterloo.ca> wrote: > I am sorry that I didn't follow the emails more closely last week. I > could have saved myself, and everyone, some time for debugging. > > When trying things out I did try help(split), thinking maybe that was the > problem. What I found is copied below. It is, let's say, terse. > > I agree with Colin, if someone could insert something on the docs spelling > out the difference that could be very helpful for other people who might > very well come across the same issue. > > Cheers, Francis > > P.S. I will go back and reread old emails to understand why this is the > correct syntax. > > > Help on function split in module ufl.split_functions: > > split(v) > UFL operator: If v is a Coefficient or Argument in a mixed space, > returns > a tuple with the function components corresponding to the subelements. > > > > > ------------------ > Francis Poulin > Associate Professor > Department of Applied Mathematics > University of Waterloo > > email: fpoulin(a)uwaterloo.ca > Web: https://uwaterloo.ca/poulin-research-group/ > Telephone: +1 519 888 4567 x32637 <(519)%20888-4567> > > ------------------------------ > *From:* firedrake-bounces(a)imperial.ac.uk [firedrake-bounces(a)imperial.ac.uk] > on behalf of Colin Cotter [colin.cotter(a)imperial.ac.uk] > *Sent:* Thursday, March 30, 2017 7:59 AM > *To:* firedrake > > *Subject:* Re: [firedrake] close to fixing my nonlinear QG solver > It's a classic gotcha. We are kind of stuck with it due to wanting to stay > close to the Fenics interface, but maybe we should have a specific bit in > the docs about it [or a more highlighted one, if it already exists]. > > all the best > --cjc > > On 30 March 2017 at 12:57, Francis Poulin <fpoulin(a)uwaterloo.ca> wrote: > > Houston, we have convergence! > > Thank you for the help Lawrence. > > I did see there were emails about the split passed around last week, which > I clearly should have read more closely. > > Now that I have convergence on a course grid I will test a finder grid and > let you know how robust this is. > > Cheers, Francis > > ------------------ > Francis Poulin > Associate Professor > Department of Applied Mathematics > University of Waterloo > > email: fpoulin(a)uwaterloo.ca > Web: https://uwaterloo.ca/poulin-research-group/ > Telephone: +1 519 888 4567 x32637 > > > ________________________________________ > From: firedrake-bounces(a)imperial.ac.uk [firedrake-bounces(a)imperial.ac.uk] > on behalf of Lawrence Mitchell [lawrence.mitchell(a)imperial.ac.uk] > Sent: Thursday, March 30, 2017 7:14 AM > To: firedrake(a)imperial.ac.uk > Subject: Re: [firedrake] close to fixing my nonlinear QG solver > > Hi Francis, > > sorry, I now saw your problem. > > You do: > > soln1.assign(soln0) > > psi1, u1, q1 = soln1.split() > > Gnon = F(psi1, u1, q1) > > You've been bitten by the "wrong split" bug!! ARGH. > > Only use function.split() for output. If you're going to use the > split values in a form, you need to use "split(function)" > > If you replace your lines: > > psi1, u1, q1 = soln1.split() > Gnon = ... > > with: > > psi1, u1, q1 = split(soln1) > Gnon = ... > psi1, u1, q1 = soln1.split() > > > Everything works. > > Lawrence > > > _______________________________________________ > firedrake mailing list > firedrake(a)imperial.ac.uk > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > > > -- > http://www.imperial.ac.uk/people/colin.cotter > > www.cambridge.org/9781107663916 > > > -- Dr David Ham Department of Mathematics Imperial College London

3 7

Re: [firedrake] installing firedrake on a cluster
by David Ham 31 Mar '17

31 Mar '17

Hi Francis and thanks Justin for the good advice! First, you almost certainly want to: unset PYTHONPATH instead of telling Firedrake to honour it. This is because it's unlikely that there is something in your PYTHONPATH which you want the virtualenv to see (this is why having PYTHONPATH set triggers this error). Next, assuming that you managed to install pip using Justin's instructions (or by having the sysadmin do it), you can try running: pip install --user virtualenv yourself. If that works, try running the install script again. If it fails, then show us the error message and we might be able to understand what's gone wrong. Regards, David On Fri, 31 Mar 2017 at 03:43 Justin Chang <jychang48(a)gmail.com> wrote: > Francis, I assume you tried get-pip.py > <https://bootstrap.pypa.io/get-pip.py>? > > If you simply run the above with system python (e.g., 'python get-pip.py') > it will return an error because it is trying to install pip into the root > directory which I am assuming you have no access to. What you need to do is > install pip locally. I did this on my local cluster through 'python > get-pip.py --user' which will install pip into $HOME/.local. I can't > remember what I did next but I think I had to ensure that this .local > directory was visible through PATH: > > export PATH=$HOME/.local:$HOME/.local/bin:$PATH > > and then pip worked. > > Justin > > On Thu, Mar 30, 2017 at 9:32 PM, Francis Poulin <fpoulin(a)uwaterloo.ca> > wrote: > > Thanks again. > > It would appear that I don't have pip installed. (see below) > > I found instructions on their site on how to do that and of course it > doesn't work for some reason. I will get to to him me set up pip and then > try again tomorrow and hope I can get a bit further. I will let you know > things progress, when they progress that is. > > Cheers, Francis > > > > [fpoulin@orc129 software]$ apt-get --version > -bash: apt-get: command not found > [fpoulin@orc129 software]$ python firedrake-install --honour-pythonpath > --no_package_manager > apt-get not found or disabled. Proceeding on the rash assumption that your > compiled dependencies are in place. > If this is not the case, please install the following and try again: > * A C and C++ compiler (for example gcc/g++ or clang), GNU make > * A Fortran compiler (for PETSc) > * MPI > * Blas and Lapack > * Git, Mercurial > * Python version 2.7 > * pip and the Python headers > * autoconf, automake, libtool > * CMake > * zlib > Installing virtualenv. > Collecting virtualenv > Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after > connection broken by 'ProtocolError('Connection aborted.', error(101, > 'Network is unreachable'))': /simple/virtualenv/ > Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after > connection broken by 'ProtocolError('Connection aborted.', error(101, > 'Network is unreachable'))': /simple/virtualenv/ > Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after > connection broken by 'ProtocolError('Connection aborted.', error(101, > 'Network is unreachable'))': /simple/virtualenv/ > Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after > connection broken by 'ProtocolError('Connection aborted.', error(101, > 'Network is unreachable'))': /simple/virtualenv/ > Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after > connection broken by 'ProtocolError('Connection aborted.', error(101, > 'Network is unreachable'))': /simple/virtualenv/ > Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after > connection broken by 'ProtocolError('Connection aborted.', error(101, > 'Network is unreachable'))': /simple/virtualenv/ > Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after > connection broken by 'ProtocolError('Connection aborted.', error(101, > 'Network is unreachable'))': /simple/virtualenv/ > Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after > connection broken by 'ProtocolError('Connection aborted.', error(101, > 'Network is unreachable'))': /simple/virtualenv/ > Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after > connection broken by 'ProtocolError('Connection aborted.', error(101, > 'Network is unreachable'))': /simple/virtualenv/ > Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after > connection broken by 'ProtocolError('Connection aborted.', error(101, > 'Network is unreachable'))': /simple/virtualenv/ > Could not find a version that satisfies the requirement virtualenv (from > versions: ) > No matching distribution found for virtualenv > Traceback (most recent call last): > File "firedrake-install", line 1172, in <module> > check_call(["pip", "install", "--user", "virtualenv"]) > File "firedrake-install", line 369, in check_call > subprocess.check_call(arguments, env=env) > File "/opt/sharcnet/python/2.7.8/intel/lib/python2.7/subprocess.py", > line 540, in check_call > raise CalledProcessError(retcode, cmd) > subprocess.CalledProcessError: Command '['pip', 'install', '--user', > 'virtualenv']' returned non-zero exit status 1 > > > > _______________________________________________ > firedrake mailing list > firedrake(a)imperial.ac.uk > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > -- Dr David Ham Department of Mathematics Imperial College London

2 1

Fwd: [petsc-maint] Weak scaling test: Fieldsplit questions
by Matthew Knepley 31 Mar '17

31 Mar '17

What is going on here? It looks like you are using subprocess. Why would you do that on a cluster rather than MPI? Matt ---------- Forwarded message ---------- From: Buesing, Henrik <HBuesing(a)eonerc.rwth-aachen.de> Date: Thu, Mar 30, 2017 at 8:25 AM Subject: AW: [petsc-maint] Weak scaling test: Fieldsplit questions To: Matthew Knepley <knepley(a)gmail.com> Cc: Barry Smith <bsmith(a)mcs.anl.gov>, Hong <hzhang(a)mcs.anl.gov>, " petsc-maint(a)mcs.anl.gov" <petsc-maint(a)mcs.anl.gov> In my opinion, there is some kind of race condition in Firedrake when running on more than one node. Thus, until this is fixed it is very unlikely for me to get the 64 cores case running. Hmm, we are running Firedrake in parallel with no problems here. What is the error? *[Buesing, Henrik] *See [1] for the error message and the attached three logs (for the 32 core case this was a 2/5 running and 3/5 crashing ). This is just for running the compiled code. During the compile stage I had problems, too. What I did is the following: 1) Run Firedrake on 1 node (this works). Now all the *.so files are in place. 2) Run Firedrake on more than one node. This crashes more often the more processes I use. I’m guessing for a race condition, because on 17 cores (1 node + 1 core) my problem runs fine. On 32 cores it sometimes runs. And on 64 cores it, up to now, never runs. But if you are not having these problems, and if the provided code reproduces the MatCreateSubMats problem, then you can do tests on your own. Well, a lot of ifs, but better than nothing. Thank you! Henrik [1] Traceback (most recent call last): File "/work/hb111949/Firedrake/twophase/2pDrake/2pinjection.py", line 228, in <module> solver = NonlinearVariationalSolver(problem,options_prefix="") File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ src/firedrake/firedrake/variational_solver.py", line 156, in __init__ pre_function_callback=pre_f_callback) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ src/firedrake/firedrake/solving_utils.py", line 260, in __init__ form_compiler_parameters=fcp) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ src/firedrake/firedrake/assemble.py", line 143, in create_assembly_callable collect_loops=True) File "<decorator-gen-279>", line 2, in _assemble File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ src/firedrake/firedrake/utils.py", line 62, in wrapper return f(*args, **kwargs) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ src/firedrake/firedrake/assemble.py", line 192, in _assemble kernels = tsfc_interface.compile_form(f, "form", parameters=form_compiler_parameters, inverse=inverse) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ src/firedrake/firedrake/tsfc_interface.py", line 193, in compile_form number_map).kernels File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/caching.py", line 200, in __new__ obj = make_obj() File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/caching.py", line 190, in make_obj obj.__init__(*args, **kwargs) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ src/firedrake/firedrake/tsfc_interface.py", line 121, in __init__ kernels.append(KernelInfo(kernel=Kernel(ast, ast.name, opts=opts), File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/caching.py", line 200, in __new__ obj = make_obj() File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/caching.py", line 190, in make_obj obj.__init__(*args, **kwargs) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/base.py", line 3843, in __init__ self._code = self._ast_to_c(self._ast, opts) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ src/PyOP2/pyop2/sequential.py", line 73, in _ast_to_c ast_handler.plan_cpu(self._opts) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/COFFEE/coffee/plan.py", line 121, in plan_cpu loop_opt.rewrite(rewrite) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ src/COFFEE/coffee/optimizer.py", line 117, in rewrite ew.sharing_graph_rewrite() File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ src/COFFEE/coffee/rewriter.py", line 619, in sharing_graph_rewrite prob.solve(ilp.GLPK(msg=0)) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ lib/python2.7/site-packages/pulp/pulp.py", line 1651, in solve status = solver.actualSolve(self, **kwargs) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ lib/python2.7/site-packages/pulp/solvers.py", line 383, in actualSolve rc = subprocess.call(proc, stdout = pipe, stderr = pipe) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ lib/python2.7/site-packages/subprocess32.py", line 578, in call p = Popen(*popenargs, **kwargs) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ lib/python2.7/site-packages/subprocess32.py", line 825, in __init__ restore_signals, start_new_session) File "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/ lib/python2.7/site-packages/subprocess32.py", line 1574, in _execute_child raise child_exception_type(errno_num, err_msg) OSError: [Errno 14] Bad address Thanks, Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

4 10

installing firedrake on a cluster
by Francis Poulin 31 Mar '17

31 Mar '17

Hello, I started trying to install firedrake on a cluster and I have some rather silly problems. First, it won't install because PYTHONPATH is installed. That's easy to resolve since I can ask it to honour the python path and then it goes forward. Second, it notices that there is a PETSc directory. I am pretty sure that I won't the install script to build petsc and not use what's there. IS there a way to get it to ignore both of these? I tried PYTHONPATH='' but that didn't seem to help for the first part so I don't imagine it will help much for the second. The good news is that the default python is 2.7.8 Any advice would be greatly appreciated to get by the python and petsc path's already set? Cheers, Francis ------------------ Francis Poulin Associate Professor Department of Applied Mathematics University of Waterloo email: fpoulin(a)uwaterloo.ca Web: https://uwaterloo.ca/poulin-research-group/ Telephone: +1 519 888 4567 x32637

2 5

Re: [firedrake] Fwd: [petsc-maint] Weak scaling test: Fieldsplit questions
by David Ham 30 Mar '17

30 Mar '17

Hi Matt, Are you sure? To me it looks like pulp is using subprocess to fork the linear program solver. Nothing to do with parallel. David On Thu, 30 Mar 2017 at 14:30 Matthew Knepley <knepley(a)gmail.com> wrote: > What is going on here? It looks like you are using subprocess. Why would > you do that on a cluster rather than MPI? > > Matt > > ---------- Forwarded message ---------- > From: *Buesing, Henrik* <HBuesing(a)eonerc.rwth-aachen.de> > Date: Thu, Mar 30, 2017 at 8:25 AM > Subject: AW: [petsc-maint] Weak scaling test: Fieldsplit questions > To: Matthew Knepley <knepley(a)gmail.com> > Cc: Barry Smith <bsmith(a)mcs.anl.gov>, Hong <hzhang(a)mcs.anl.gov>, " > petsc-maint(a)mcs.anl.gov" <petsc-maint(a)mcs.anl.gov> > > > In my opinion, there is some kind of race condition in Firedrake when > running on more than one node. Thus, until this is fixed it is very > unlikely for me to get the 64 cores case running. > > > Hmm, we are running Firedrake in parallel with no problems here. What is > the error? > > > > *[Buesing, Henrik] *See [1] for the error message and the attached three > logs (for the 32 core case this was a 2/5 running and 3/5 crashing ). > > > > This is just for running the compiled code. During the compile stage I had > problems, too. What I did is the following: 1) Run Firedrake on 1 node > (this works). Now all the *.so files are in place. 2) Run Firedrake on more > than one node. This crashes more often the more processes I use. > > > > I’m guessing for a race condition, because on 17 cores (1 node + 1 core) > my problem runs fine. On 32 cores it sometimes runs. And on 64 cores it, up > to now, never runs. > > But if you are not having these problems, and if the provided code > reproduces the MatCreateSubMats problem, then you can do tests on your own. > Well, a lot of ifs, but better than nothing. > > > > Thank you! > Henrik > > > > [1] > > > > Traceback (most recent call last): > > File "/work/hb111949/Firedrake/twophase/2pDrake/2pinjection.py", line > 228, in <module> > > solver = NonlinearVariationalSolver(problem,options_prefix="") > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/firedrake/firedrake/variational_solver.py", > line 156, in __init__ > > pre_function_callback=pre_f_callback) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/firedrake/firedrake/solving_utils.py", > line 260, in __init__ > > form_compiler_parameters=fcp) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/firedrake/firedrake/assemble.py", > line 143, in create_assembly_callable > > collect_loops=True) > > File "<decorator-gen-279>", line 2, in _assemble > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/firedrake/firedrake/utils.py", > line 62, in wrapper > > return f(*args, **kwargs) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/firedrake/firedrake/assemble.py", > line 192, in _assemble > > kernels = tsfc_interface.compile_form(f, "form", > parameters=form_compiler_parameters, inverse=inverse) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/firedrake/firedrake/tsfc_interface.py", > line 193, in compile_form > > number_map).kernels > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/caching.py", > line 200, in __new__ > > obj = make_obj() > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/caching.py", > line 190, in make_obj > > obj.__init__(*args, **kwargs) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/firedrake/firedrake/tsfc_interface.py", > line 121, in __init__ > > kernels.append(KernelInfo(kernel=Kernel(ast, ast.name, opts=opts), > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/caching.py", > line 200, in __new__ > > obj = make_obj() > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/caching.py", > line 190, in make_obj > > obj.__init__(*args, **kwargs) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/base.py", > line 3843, in __init__ > > self._code = self._ast_to_c(self._ast, opts) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/PyOP2/pyop2/sequential.py", > line 73, in _ast_to_c > > ast_handler.plan_cpu(self._opts) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/COFFEE/coffee/plan.py", > line 121, in plan_cpu > > loop_opt.rewrite(rewrite) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/COFFEE/coffee/optimizer.py", > line 117, in rewrite > > ew.sharing_graph_rewrite() > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/src/COFFEE/coffee/rewriter.py", > line 619, in sharing_graph_rewrite > > prob.solve(ilp.GLPK(msg=0)) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/lib/python2.7/site-packages/pulp/pulp.py", > line 1651, in solve > > status = solver.actualSolve(self, **kwargs) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/lib/python2.7/site-packages/pulp/solvers.py", > line 383, in actualSolve > > rc = subprocess.call(proc, stdout = pipe, stderr = pipe) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/lib/python2.7/site-packages/subprocess32.py", > line 578, in call > > p = Popen(*popenargs, **kwargs) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/lib/python2.7/site-packages/subprocess32.py", > line 825, in __init__ > > restore_signals, start_new_session) > > File > "/rwthfs/rz/cluster/work/hb111949/Firedrake/firedrake/lib/python2.7/site-packages/subprocess32.py", > line 1574, in _execute_child > > raise child_exception_type(errno_num, err_msg) > > OSError: [Errno 14] Bad address > > Thanks, > > > > Matt > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- Dr David Ham Department of Mathematics Imperial College London

1 0