- firedrake - imperial.ac.uk

problem solving a linear system
by Francis Poulin 21 Jun '17

21 Jun '17

Hello, We are trying to solve a 2-layer Quasi-Geostrophic model to determine a steady linear solution, basically building on a demo that we already wrote. When we try running it we get the following error, copied below. It seems to have a problem with the nonlinear solver, even though we are solving a linear problem. Can anyone give us some advice on how we might be able to fix this? Code is attached in case you wanted to see the details. Cheers, Francis (firedrake) fpoulin@domlt32:~/Dropbox/Research_Keegan/Code$ python test.py Traceback (most recent call last): File "test.py", line 91, in <module> soln0 = steadylinearQG2d(Ztrial, Ztest, Gwinds, bcs) File "test.py", line 46, in steadylinearQG2d linear_solver.solve() File "/home/fpoulin/software/firedrake/src/firedrake/firedrake/variational_solver.py", line 223, in solve solving_utils.check_snes_convergence(self.snes) File "/home/fpoulin/software/firedrake/src/firedrake/firedrake/solving_utils.py", line 233, in check_snes_convergence %s""" % (snes.getIterationNumber(), msg)) firedrake.exceptions.ConvergenceError: Nonlinear solve failed to converge after 0 nonlinear iterations. Reason: Inner linear solve failed to converge after 0 iterations with reason: unknown reason (petsc4py enum incomplete?), try with -snes_converged_reason and -ksp_converged_reason ------------------ Francis Poulin Associate Professor Department of Applied Mathematics University of Waterloo email: fpoulin(a)uwaterloo.ca Web: https://uwaterloo.ca/poulin-research-group/ Telephone: +1 519 888 4567 x32637

1 0

Problems with spud- Firedrake-fluids
by Sara Tomasoni 18 Jun '17

18 Jun '17

Dear all, I am trying to install the Firedrake-Fluids module. First of all, I installed Spud, but I have some problems with the source code. Since I work on Ubuntu, I tried to install the dependencies adding the Launchpad repository but, when I update and type *sudo apt-get build-dep spud* I receive the error *unable to find a source package for spud* I tried to build it and install from source via bzr repository an it seems to work, but when I try to make the Firedrake-Fluids tests, I have *No module named libspud* Can anybody help me with this problem? Many thanks, Sara

2 2

Re: [firedrake] parallel solving
by David Ham 17 Jun '17

17 Jun '17

Sounds like we need the ability to fire a Function from one communicator to another. We need to think about how to do this... On Tue, 13 Jun 2017 at 14:26 Cotter, Colin J <colin.cotter(a)imperial.ac.uk> wrote: > Hi David, > (1) Jemma can provide details but each elliptic problem fits comfortably > on one core (we are doing shallow-water). I don't think our productivity > can be addressed by keeping the elliptic problems serial but using spatial > parallelism. > (2) We are using a fairly complicated setup - GMRES on a 4x4 block, > fieldsplit into 2x2 blocks and multiplicative, each of the 2x2 diagonal > blocks is solved by hybridisation. > (3) Jemma can provide details, but I was able to fit ~40 solvers into > memory, but then we needed to go up by a factor of 10-20. The individual > solves are quite cheap but we need to do a lot of them. The idea is you do > a large number of concurrent solves and then take a big timestep. > > all the best > --cjc > > On 13 June 2017 at 13:19, David Ham <David.Ham(a)imperial.ac.uk> wrote: > >> Hi All, >> >> Can you provide the basic characteristics of this problem please: >> >> * How many DoFs per realisation. >> * What solvers are you using. >> * When you say "hammered", how much memory are you using per solver, how >> much time is creating a solver taking. >> >> At the moment you haven't provided enough information for us to >> understand where you are in parameter space. >> >> Regards, >> >> David >> >> On Tue, 13 Jun 2017 at 14:14 Cotter, Colin J <colin.cotter(a)imperial.ac.uk> >> wrote: >> >>> Dear Firedrakers, >>> I hope that everyone is enjoying Fenics 17. The weather is glorious >>> here back in London! >>> >>> Jemma and I are working some more on this REXI+averaging approach. >>> Basically the way it works is you solve a lot of independent (but with >>> different coefficients) elliptic problems with the same RHS, and then >>> compute a weighted sum of the solutions (with different weights for each >>> elliptic problem) which you then use to progress the solution. Eventually >>> we want to make a proper parallel implementation of this, but right now we >>> just need to see what works, and are just running over the loop in serial >>> and getting hammered either by memory to store all of the solvers, or by >>> time if we rebuild the solver for each elliptic problem. Is there a >>> quick-and-dirty way to implement this parallelism so that we can make >>> progress? Our starting point is that we don't have a clue about MPI >>> programming! >>> >>> all the best >>> --cjc >>> >>> -- >>> http://www.imperial.ac.uk/people/colin.cotter >>> >>> www.cambridge.org/9781107663916 >>> >>> >>> -- >> Dr David Ham >> Department of Mathematics >> Imperial College London >> >> _______________________________________________ >> firedrake mailing list >> firedrake(a)imperial.ac.uk >> https://mailman.ic.ac.uk/mailman/listinfo/firedrake >> >> > > > -- > http://www.imperial.ac.uk/people/colin.cotter > > www.cambridge.org/9781107663916 > > > -- Dr David Ham Department of Mathematics Imperial College London

2 2

Building firedrake using my own mpicc compiler
by Justin Chang 17 Jun '17

17 Jun '17

Hi all, I am building firedrake using the system provided "mpicc" command. On the NERSC systems, the mpicc command is "cc", so I: 1) built my own firedrake fork of PETSc using the appropriate options 2) passed in "--mpicc cc" to the firedrake-install command. Everything works until I get to the PyOP2 stage. Upon inspection of firedrake-install.log, it seems the pip install for pyop2 still defaults to the "epic" command (which does not exist on the NERSC system) and not "cc" which I told the firedrake script to use. 2017-06-16 12:31:27,681 INFO Installing PyOP2/ 2017-06-16 12:31:27,681 DEBUG /global/u1/j/jychang/Software/firedrake/bin/pip install --no-binary mpi4py --no-deps -vvv -e PyOP2/ 2017-06-16 12:31:31,251 DEBUG Obtaining file:///global/u1/j/jychang/Software/firedrake/src/PyOP2 Running setup.py (path:/global/u1/j/jychang/Software/firedrake/src/PyOP2/setup.py) egg_info for package from file:///global/u1/j/jychang/Software/firedrake/src/PyOP2 Running command python setup.py egg_info /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_requires' warnings.warn(msg) running egg_info creating PyOP2.egg-info writing requirements to PyOP2.egg-info/requires.txt writing PyOP2.egg-info/PKG-INFO writing top-level names to PyOP2.egg-info/top_level.txt writing dependency_links to PyOP2.egg-info/dependency_links.txt writing manifest file 'PyOP2.egg-info/SOURCES.txt' reading manifest file 'PyOP2.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching '*.c' under directory 'pyop2' writing manifest file 'PyOP2.egg-info/SOURCES.txt' Source in ./PyOP2 has version 0.13.0+299.g200780c, which satisfies requirement PyOP2==0.13.0+299.g200780c from file:///global/u1/j/jychang/Software/firedrake/src/PyOP2 Installing collected packages: PyOP2 Running setup.py develop for PyOP2 Running command /global/u1/j/jychang/Software/firedrake/bin/python -c "import setuptools, tokenize;__file__='/global/u1/j/jychang/Software/firedrake/src/PyOP2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_requires' warnings.warn(msg) running develop running egg_info writing requirements to PyOP2.egg-info/requires.txt writing PyOP2.egg-info/PKG-INFO writing top-level names to PyOP2.egg-info/top_level.txt writing dependency_links to PyOP2.egg-info/dependency_links.txt reading manifest file 'PyOP2.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching '*.c' under directory 'pyop2' writing manifest file 'PyOP2.egg-info/SOURCES.txt' running build_ext cythoning pyop2/sparsity.pyx to pyop2/sparsity.cpp building 'pyop2.sparsity' extension creating build creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/pyop2 mpicc -fno-strict-aliasing -fmessage-length=0 -grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2 -funwind-tables -fasynchronous-unwind-tables -g -DNDEBUG -fmessage-length=0 -grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2 -funwind-tables -fasynchronous-unwind-tables -g -DOPENSSL_LOAD_CONF -fPIC -Ipyop2 -I/global/u1/j/jychang/Software/firedrake/lib/python2.7/site-packages/numpy/core/include -I/global/u1/j/jychang/Software/firedrake/lib/python2.7/site-packages/petsc4py/include -I/global/homes/j/jychang/Software/petsc-fd/include -I/global/homes/j/jychang/Software/petsc-fd/arch-cori-c-opt/include -I/usr/include/python2.7 -c pyop2/sparsity.cpp -o build/temp.linux-x86_64-2.7/pyop2/sparsity.o unable to execute 'mpicc': No such file or directory error: command 'mpicc' failed with exit status 1 Cleaning up... Command "/global/u1/j/jychang/Software/firedrake/bin/python -c "import setuptools, tokenize;__file__='/global/u1/j/jychang/Software/firedrake/src/PyOP2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps" failed with error code 1 in /global/u1/j/jychang/Software/firedrake/src/PyOP2/ Exception information: Traceback (most recent call last): File "/global/u1/j/jychang/Software/firedrake/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main status = self.run(options, args) File "/global/u1/j/jychang/Software/firedrake/lib/python2.7/site-packages/pip/commands/install.py", line 342, in run prefix=options.prefix_path, File "/global/u1/j/jychang/Software/firedrake/lib/python2.7/site-packages/pip/req/req_set.py", line 784, in install **kwargs File "/global/u1/j/jychang/Software/firedrake/lib/python2.7/site-packages/pip/req/req_install.py", line 845, in install install_options, global_options, prefix=prefix) File "/global/u1/j/jychang/Software/firedrake/lib/python2.7/site-packages/pip/req/req_install.py", line 1007, in install_editable show_stdout=False) File "/global/u1/j/jychang/Software/firedrake/lib/python2.7/site-packages/pip/utils/__init__.py", line 707, in call_subprocess % (command_desc, proc.returncode, cwd)) InstallationError: Command "/global/u1/j/jychang/Software/firedrake/bin/python -c "import setuptools, tokenize;__file__='/global/u1/j/jychang/Software/firedrake/src/PyOP2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps" failed with error code 1 in /global/u1/j/jychang/Software/firedrake/src/PyOP2/ 2017-06-16 12:31:31,253 INFO Install log saved in firedrake-install.log I feel like there's a way to circumvent (or fix) this issue. What is the best way to do this? Thanks, Justin

1 2

matfree
by Colin Cotter 15 Jun '17

15 Jun '17

Dear Firedrakers [mostly Lawrence but asking here means it is archived and might be useful to someone], I'm using a matfree solver, for the same equation with changing coefficients, but always the same preconditioner. Is there are way to avoid rebuilding the preconditioner but updating the matrix entries? I'm hoping that this actually happens by default, because of the matfree setup, but am not sure. cheers --cjc -- http://www.imperial.ac.uk/people/colin.cotter www.cambridge.org/9781107663916

2 3

Re: [firedrake] parallel solving
by David Ham 13 Jun '17

13 Jun '17

Hi All, Can you provide the basic characteristics of this problem please: * How many DoFs per realisation. * What solvers are you using. * When you say "hammered", how much memory are you using per solver, how much time is creating a solver taking. At the moment you haven't provided enough information for us to understand where you are in parameter space. Regards, David On Tue, 13 Jun 2017 at 14:14 Cotter, Colin J <colin.cotter(a)imperial.ac.uk> wrote: > Dear Firedrakers, > I hope that everyone is enjoying Fenics 17. The weather is glorious here > back in London! > > Jemma and I are working some more on this REXI+averaging approach. > Basically the way it works is you solve a lot of independent (but with > different coefficients) elliptic problems with the same RHS, and then > compute a weighted sum of the solutions (with different weights for each > elliptic problem) which you then use to progress the solution. Eventually > we want to make a proper parallel implementation of this, but right now we > just need to see what works, and are just running over the loop in serial > and getting hammered either by memory to store all of the solvers, or by > time if we rebuild the solver for each elliptic problem. Is there a > quick-and-dirty way to implement this parallelism so that we can make > progress? Our starting point is that we don't have a clue about MPI > programming! > > all the best > --cjc > > -- > http://www.imperial.ac.uk/people/colin.cotter > > www.cambridge.org/9781107663916 > > > -- Dr David Ham Department of Mathematics Imperial College London

2 1

parallel solving
by Colin Cotter 13 Jun '17

13 Jun '17

Dear Firedrakers, I hope that everyone is enjoying Fenics 17. The weather is glorious here back in London! Jemma and I are working some more on this REXI+averaging approach. Basically the way it works is you solve a lot of independent (but with different coefficients) elliptic problems with the same RHS, and then compute a weighted sum of the solutions (with different weights for each elliptic problem) which you then use to progress the solution. Eventually we want to make a proper parallel implementation of this, but right now we just need to see what works, and are just running over the loop in serial and getting hammered either by memory to store all of the solvers, or by time if we rebuild the solver for each elliptic problem. Is there a quick-and-dirty way to implement this parallelism so that we can make progress? Our starting point is that we don't have a clue about MPI programming! all the best --cjc -- http://www.imperial.ac.uk/people/colin.cotter www.cambridge.org/9781107663916

1 0

mesh hierarchy inject function issue - possible bug
by Pan, Wei 09 Jun '17

09 Jun '17

Dear team, In the following code, I call inject twice, each time on separate sets of variables {lag_f, lag_c} and {psi0, psi0_c}, where lag_f and psi0 are defined on the fine mesh, and lag_c and psi0_c are defined on the coarse mesh. It seems that the first inject call on {psi0, psi0_c} changes the result of the second inject call on {lag_f, lag_c} which is not what I would expect as psi0 and lag are unrelated. I have highlighted the relevant definitions in blue, and problem bits in red. I should add that the problem goes away if in the VectorFunctionSpace definitions of vfs_f and vfs_c I change "CG" to "DG". But I'm worried about the result of the CG inject. Many thanks in advance. Kind regards Wei from firedrake import * import numpy as np L = 1.0 # Zonal length n0 = 50 # Spatial resolution mesh_base = SquareMesh(n0, n0, L) mesh_hierarchy = MeshHierarchy(mesh_base, 1) mesh = mesh_hierarchy[1] # fine mesh mesh_c = mesh_hierarchy[0] # coarse mesh Vdg = FunctionSpace(mesh,"DG",1) # DG elements for Potential Vorticity (PV) Vcg = FunctionSpace(mesh,"CG",1) # CG elements for Streamfunction vfs_f = VectorFunctionSpace(mesh,"CG",1) vfs_c = VectorFunctionSpace(mesh_c,"CG",1) vxf, vyf = SpatialCoordinate(vfs_f) x = SpatialCoordinate(mesh) q0 = Function(Vdg).interpolate(0.1*sin(x[0])*sin(x[1])) q1 = Function(Vdg) psi0 = Function(Vcg) # Streamfunctions for different time steps psi = TrialFunction(Vcg) # Test function phi = TestFunction(Vcg) # Trial function Apsi = (inner(grad(psi),grad(phi)))*dx Lpsi = -q1*phi*dx bc1 = DirichletBC(Vcg, 0., 'on_boundary') psi_problem = LinearVariationalProblem(Apsi,Lpsi,psi0,bcs=bc1) psi_solver = LinearVariationalSolver(psi_problem, solver_parameters={ 'ksp_type':'cg', 'pc_type':'sor' }) gradperp = lambda u: as_vector((-u.dx(1), u.dx(0))) lag_f = Function(vfs_f).interpolate(as_vector((vxf, vyf))) if 1: print "-------------------------------------------------------------" q1.assign(q0) psi_solver.solve() psi0_coarse = Function(FunctionSpace(mesh_c, "CG", 1)) inject(psi0, psi0_coarse) print "psi0 norm ", norm(psi0) print "psi0_coarse norm ", norm(psi0_coarse) print "psi0_coarse relative error ", np.absolute(norm(psi0_coarse) - norm(psi0)) / norm(psi0) # relative error 0.000583086120876 print "-------------------------------------------------------------" lag_c = Function(vfs_c) inject(lag_f, lag_c) print "analytical value ", np.sqrt(2./3.) # 0.816496580928 print "lag0_f norm ", norm(lag_f) print "lag0_c norm ", norm(lag_c) # this is the correct value of 0.816496580928 if inject(psi0, psi0_coarse) is not called # but if inject(psi0, psi0_coarse) is called then # this gives 0.825399087311 which is a relative error of 0.019032990351 print "lag relative error ", np.absolute(norm(lag_c) - norm(lag_f)) / norm(lag_f)

2 4

firedrake-update failed
by William Booker 07 Jun '17

07 Jun '17

Dear Firedrakers, Attached the log for a failed firedrake update. What should I do to work around this? thanks Will

2 3

Re: [firedrake] error norm convergence issues for RT/BDM
by Colin Cotter 01 Jun '17

01 Jun '17

Hi Justin, Yes, I think this is because of the p dependence in the u term - it messes up the "improved estimates" that you get for H(div) discretisations for elliptic problems. all the best --cjc On 1 June 2017 at 03:28, Justin Chang <jychang48(a)gmail.com> wrote: > Hi all, > > I am playing around with the RT/BDM L2 error norm convergence for the > Darcy equation: > > mu/K*v + grad(p) = rhob > div(v) = 0 > > Where mu is viscosity, K is permeability tensor, and rhob is specific body > force. The analytical solutions for a bi-unit square are: > > p(x,y) = sin(pi*x)*sin(pi*y) > v_x(x,y) = sin(pi*x)*cos(pi*x) > v_y(x,y) = -cos(pi*x)*sin(pi*y) > > And mu = K = 1. Theory states that RT0 has velocity/pressure slope of 1.0 > and that BDM has velocity slope of 2.0. I get these or better. > > Now if I extend the Darcy formulation to include pressure-dependence for > viscosity: > > mu(p)/K*v + grad(p) = rhob > div(v) = f > > where > > mu(p) = mu*(1 + beta*p) > > So we now have a nonlinear formulation. By employing the same analytical > solutions, the only thing that changes is rhob, but the velocity L2 > errornorm slopes for both RT and BDM worsen. For example, BDM goes from 2.0 > to 1.0. Pressure slopes remains unchanged. > > Why is this happening? Does this have something to do with the DG0 > pressure functionspace that is now included in the mass matrix term? If I > used a stabilized equal-order formulation for velocity/pressure, no such > slope differences were observed. These trends were also observed with > FEniCS. > > Attached is the code y'all can play around with: Run it as follows: > > python H_conv_barus.py Darcy RT 10 > python H_conv_barus.py Darcy RT 20 > python H_conv_barus.py Darcy RT 40 > ... > python H_conv_barus.py Darcy BDM 10 > python H_conv_barus.py Darcy BDM 20 > python H_conv_barus.py Darcy BDM 40 > ... > python H_conv_barus.py Barus RT 10 > python H_conv_barus.py Barus RT 20 > python H_conv_barus.py Barus RT 40 > ... > python H_conv_barus.py Barus BDM 10 > python H_conv_barus.py Barus BDM 20 > python H_conv_barus.py Barus BDM 40 > ... > > I am trying to understand whether these H(div) elements are suitable for > nonlinear mixed formulations or if there is a better way of implementing > what I have. > > Thanks! > Justin > -- http://www.imperial.ac.uk/people/colin.cotter www.cambridge.org/9781107663916

2 1