February 2015 - firedrake - imperial.ac.uk

Re: [firedrake] Nullspace & VectorSpaceBasis
by Andrew McRae 29 Apr '15

29 Apr '15

Four comments: 1) If I understand correctly, PETSc insists that the diagonal entries of the sparse matrix are filled in. Usually, the LHS form will cause this to happen; however, that obviously isn't the case for your form! 2) It should be possible to build the nullspace you want, but I'm clueless on the actual syntax. This wouldn't get around (1) without further fixes from our end. 3) Could the equations given not be included in the weak form for the rest of the problem? 4) If not, there might be a hacky "add-a-weighted-mass-matrix-to-the-LHS" fix you could use?.... Andrew On 11 December 2014 at 12:46, Anna Kalogirou <a.kalogirou(a)leeds.ac.uk> wrote: > Dear all, > > I am trying to solve a simple problem (linear water wave equations), > where I have the Laplace equation in a square domain with Neumann > boundary conditions on the three sides and on the top boundary two > equations are satisfied. > > I have the following code: > -------------------------------------- > V = FunctionSpace(mesh,"CG",1) > > eta0 = Function(V).interpolate(Expression("cos(2*pi*x[0])")) > phi0 = Function(V) > eta1 = Function(V) > phi1 = Function(V) > eta = TrialFunction(V) > phi = TrialFunction(V) > v = TestFunction(V) > > aeta = v*eta*ds(4) > Leta = v*eta0*ds(4) + dt*inner(grad(v),grad(phi0))*dx > > eta_problem = LinearVariationalProblem(aeta,Leta,eta1) > eta_solver = LinearVariationalSolver(eta_problem) > > aphi = v*phi*ds(4) > Lphi = (v*phi0 - g*dt*v*eta1)*ds(4) > > phi_problem = LinearVariationalProblem(aphi,Lphi,phi1) > phi_solver = LinearVariationalSolver(phi_problem) > -------------------------------------- > > The error I get tells me that Object is in wrong state, and Matrix is > missing diagonal entry 39. > > I thought this had to do with the problem being singular, so I tried to > build the nullspace but I don't know how to define the VectorSpaceBasis. > Any help would be appreciated. > > Thank you very much. > > Regards, Anna. > > _______________________________________________ > firedrake mailing list > firedrake(a)imperial.ac.uk > https://mailman.ic.ac.uk/mailman/listinfo/firedrake >

3 6

Re: [firedrake] Firedrake Installation
by Andrew McRae 06 Apr '15

06 Apr '15

Hi all, Suet [cc'd], one of Colin's project students, is encoutering the following error when using the PyOP2 'quick install' script. Can anyone help? IIRC, this is running in a (32-bit?) Ubuntu 14.04 virtual machine. (Last night, she was getting an mpi4py error, but I believe this <https://github.com/OP2/PyOP2/commit/e15b3ee8d72369f10c9a84d944a3ecbf06f7e927> fixed that). Cheers, Andrew On 16 November 2014 13:28, Lee, Suet <suet.lee11(a)imperial.ac.uk> wrote: > I ran the install mpi4py commange but I have a new error now: > > > mosh@ubuntu:~$ wget -O - > https://github.com/OP2/PyOP2/raw/master/install.sh | sudo bash > --2014-11-16 13:20:14-- > https://github.com/OP2/PyOP2/raw/master/install.sh > Resolving github.com (github.com) 192.30.252.131 > Connecting to github.com (github.com)|192.30.252.131|:443... connected. > HTTP request sent, awaiting response... 302 Found > Location: https://raw.githubusercontent.com/OP2/PyOP2/master/install.sh > [following] > --2014-11-16 13:20:16-- > https://raw.githubusercontent.com/OP2/PyOP2/master/install.sh > Resolving raw.githubusercontent.com (raw.githubusercontent.com) > 185.31.18.133 > Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.31.18.133|:443... > connected. > HTTP request sent, awaiting response... 200 OK > Length: 3667 (3.6K) [text/plain] > Saving to: ‘STDOUT’ > > 100%[======================================>] 3,667 --.-K/s in > 0s > > 2014-11-16 13:20:16 (40.6 MB/s) - written to stdout [3667/3667] > > PyOP2 installation started at Sun Nov 16 13:20:16 GMT 2014 > on Linux ubuntu 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:08:14 UTC > 2014 i686 i686 i686 GNU/Linux > > *** Privileged installation *** > Running unprivileged commands as mosh > > *** Preparing system *** > > *** Installing dependencies *** > > *** Installing PETSc *** > > *** Installing PyOP2 *** > > [ubuntu:31603] *** Process received signal *** > [ubuntu:31603] Signal: Floating point exception (8) > [ubuntu:31603] Signal code: Integer divide-by-zero (1) > [ubuntu:31603] Failing at address: 0xb74d6da0 > [ubuntu:31603] [ 0] [0xb779a40c] > [ubuntu:31603] [ 1] /usr/lib/i386-linux-gnu/libhwloc.so.5(+0x2cda0) > [0xb74d6da0] > [ubuntu:31603] [ 2] /usr/lib/i386-linux-gnu/libhwloc.so.5(+0x2e71c) > [0xb74d871c] > [ubuntu:31603] [ 3] /usr/lib/i386-linux-gnu/libhwloc.so.5(+0x2ea8b) > [0xb74d8a8b] > [ubuntu:31603] [ 4] /usr/lib/i386-linux-gnu/libhwloc.so.5(+0x98f6) > [0xb74b38f6] > [ubuntu:31603] [ 5] > /usr/lib/i386-linux-gnu/libhwloc.so.5(hwloc_topology_load+0x1c6) > [0xb74b48ec] > [ubuntu:31603] [ 6] /usr/lib/libopen-rte.so.4(orte_odls_base_open+0x7b1) > [0xb76fa881] > [ubuntu:31603] [ 7] /usr/lib/openmpi/lib/openmpi/mca_ess_hnp.so(+0x2445) > [0xb7488445] > [ubuntu:31603] [ 8] /usr/lib/libopen-rte.so.4(orte_init+0x1cf) [0xb76cfb3f] > [ubuntu:31603] [ 9] /usr/lib/libopen-rte.so.4(orte_daemon+0x256) > [0xb76ec1c6] > [ubuntu:31603] [10] orted() [0x80485b3] > [ubuntu:31603] [11] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3) > [0xb7504a83] > [ubuntu:31603] [12] orted() [0x80485f8] > [ubuntu:31603] *** End of error message *** > [ubuntu:31601] [[INVALID],INVALID] ORTE_ERROR_LOG: Unable to start a > daemon on the local node in file ess_singleton_module.c at line 343 > [ubuntu:31601] [[INVALID],INVALID] ORTE_ERROR_LOG: Unable to start a > daemon on the local node in file ess_singleton_module.c at line 140 > [ubuntu:31601] [[INVALID],INVALID] ORTE_ERROR_LOG: Unable to start a > daemon on the local node in file runtime/orte_init.c at line 128 > -------------------------------------------------------------------------- > It looks like orte_init failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during orte_init; some of which are due to configuration or > environment problems. This failure appears to be an internal failure; > here's some additional information (which may only be relevant to an > Open MPI developer): > > orte_ess_set_name failed > --> Returned value Unable to start a daemon on the local node (-128) > instead of ORTE_SUCCESS > -------------------------------------------------------------------------- > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: orte_init failed > --> Returned "Unable to start a daemon on the local node" (-128) instead > of "Success" (0) > -------------------------------------------------------------------------- > [ubuntu:31601] *** An error occurred in MPI_Init_thread > [ubuntu:31601] *** on a NULL communicator > [ubuntu:31601] *** Unknown error > [ubuntu:31601] *** MPI_ERRORS_ARE_FATAL: your MPI job will now abort > -------------------------------------------------------------------------- > An MPI process is aborting at a time when it cannot guarantee that all > of its peer processes in the job will be killed properly. You should > double check that everything has shut down cleanly. > > Reason: Before MPI_INIT completed > Local host: ubuntu > PID: 31601 > -------------------------------------------------------------------------- > PyOP2 installation failed > See /home/mosh/PyOP2/pyop2_install.log for details > > > Any idea what's going on? > > > Suet >

6 14

Perturbed extruded meshes
by Eike Mueller 07 Mar '15

07 Mar '15

Dear firdrakers, how can I perturb an extruded mesh? I want to add 'mountains' (i.e. bumps) to my spherical icosahedral mesh, so I want to change the radius according to something like r = r(theta) = R_earth+H*exp(-0.5*theta^2/D^2) Can I do this by moving around the underlying 2d host grid and then extruding that? How can I perturb the 2d grid? Thanks a lot, Eike

5 9

FEniCS '15 @ Imperial College London 29 June - 1 July 2015
by David Ham 25 Feb '15

25 Feb '15

The 2015 FEniCS workshop will be held at Imperial College London from 29 June - 1 July. This is the annual opportunity for developers, users and anyone else interested in FEniCS, and its relatives such as dolfin-adjoint and Firedrake, to communicate results and ideas formally and informally, face to face. Registration and abstract submission are now open via the workshop webpage: http://firedrakeproject.org/fenics_15 The abstract deadline (yes, this year we're having one of these!) is 30 April. The early registration deadline is 31 May. I look forward to seeing you all in London! Cheers, David -- Dr David Ham Departments of Mathematics and Computing Imperial College London http://www.imperial.ac.uk/people/david.ham

1 0

Fwd: [CMSE-Membership] Research talk - boundary integral equation problems - Timo Betcke - Thu Feb 26
by David Ham 24 Feb '15

24 Feb '15

This AMMP seminar may be of interest to a broader CMSE audience: Timo Betcke (UCL): BEM++ - Efficient solution of boundary integral equation problems The BEM++ boundary element library is a software project that was started in 2010 at University College London to provide an open-source general purpose BEM library for a variety of application areas. In this talk we introduce the underlying design concepts of the library and discuss several applications, including high-frequency preconditioning for ultrasound applications, the solution of time-domain problems via convolution quadrature, light-scattering from ice crystals, and the solution of coupled FEM/BEM problems with FEniCS and BEM++. Date: 26 Feb 2015 Time: 16:00 - 17:00 Venue: Huxley 340 Campus: South Kensington Campus Speaker: Timo Betcke, UCL Contact: Pavel Berloff -- Dr David Ham Departments of Mathematics and Computing Imperial College London http://www.imperial.ac.uk/people/david.ham

1 0

PETSc error when projecting onto DG space
by Eike Mueller 19 Feb '15

19 Feb '15

Dear firedrakers, having just updated both petsc and petsc4py on my laptop (pulled latest revision of mapdes/firedrake branch in both cases, and I use the origin/multigrid-extrusion branch of firedrake), I get the error message below when I try to project an expression onto a DG function: r_p = Function(W3) expression = Expression('exp(-0.5*(x[0]*x[0]+x[1]*x[1])/(0.25*0.25))') r_p.project(expression) This issue only came up with the PETSc update I did, never seen it before. Thanks a lot, Eike Error message: Number of cells on finest grid = 5120 dx = 364.458 km, dt = 2429.717 s Traceback (most recent call last): File "driver.py", line 485, in <module> main(parameter_filename) File "driver.py", line 288, in main r_p.project(expression) File "/Users/eikemueller/PostDocBath/EllipticSolvers/firedrake/firedrake/function.py", line 135, in project return projection.project(b, self, *args, **kwargs) File "/Users/eikemueller/PostDocBath/EllipticSolvers/firedrake/firedrake/projection.py", line 94, in project form_compiler_parameters=form_compiler_parameters) File "/Users/eikemueller/PostDocBath/EllipticSolvers/firedrake/firedrake/solving.py", line 119, in solve _solve_varproblem(*args, **kwargs) File "/Users/eikemueller/PostDocBath/EllipticSolvers/firedrake/firedrake/solving.py", line 143, in _solve_varproblem solver.solve() File "<string>", line 2, in solve File "/Users/eikemueller/PostDocBath/EllipticSolvers/PyOP2/pyop2/profiling.py", line 197, in wrapper return f(*args, **kwargs) File "/Users/eikemueller/PostDocBath/EllipticSolvers/firedrake/firedrake/variational_solver.py", line 292, in solve self.snes.solve(None, v) File "SNES.pyx", line 516, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:153742) File "petscsnes.pxi", line 251, in petsc4py.PETSc.SNES_Function (src/petsc4py.PETSc.c:30058) File "/Users/eikemueller/PostDocBath/EllipticSolvers/firedrake/firedrake/variational_solver.py", line 138, in form_function v.array[:] = X_.array[:] File "Vec.pyx", line 815, in petsc4py.PETSc.Vec.array.__get__ (src/petsc4py.PETSc.c:91905) File "arraynpy.pxi", line 67, in petsc4py.PETSc.asarray (src/petsc4py.PETSc.c:7333) File "petscvec.pxi", line 469, in petsc4py.PETSc._Vec_buffer.__getreadbuffer__ (src/petsc4py.PETSc.c:19630) File "petscvec.pxi", line 459, in petsc4py.PETSc._Vec_buffer.getbuffer (src/petsc4py.PETSc.c:19460) petsc4py.PETSc.Error: error code 73 [0] SNESSolve() line 3832 in /Users/eikemueller/PostDocBath/EllipticSolvers/petsc/src/snes/interface/snes.c [0] SNESSolve_KSPONLY() line 24 in /Users/eikemueller/PostDocBath/EllipticSolvers/petsc/src/snes/impls/ksponly/ksponly.c [0] SNESComputeFunction() line 2035 in /Users/eikemueller/PostDocBath/EllipticSolvers/petsc/src/snes/interface/snes.c [0] VecGetArray() line 1646 in /Users/eikemueller/PostDocBath/EllipticSolvers/petsc/src/vec/vec/interface/rvector.c [0] Object is in wrong state [0] Vec is locked read only, argument # 1 -- Dr Eike Hermann Mueller Research Associate (PostDoc) Department of Mathematical Sciences University of Bath Bath BA2 7AY, United Kingdom +44 1225 38 5803 e.mueller(a)bath.ac.uk http://people.bath.ac.uk/em459/

3 8

Re: [firedrake] Crash when running at higher order on ARCHER: resolved now
by Fabio Luporini 16 Feb '15

16 Feb '15

I, however, recommend to always run with COFFEE on, because that'd be useful not just for performance, but also (and, at this stage, especially) for bug fixing. Then of course if you see something weird is going on, just turn off COFFEE. Thanks -- Fabio 2015-02-16 10:33 GMT+00:00 Homolya, Miklós <m.homolya14(a)imperial.ac.uk>: > Hi Eike, > > I think, in general, when you want to disable COFFEE, probably the best > place to do so is very soon after "from firedrake import *". > Or, alternatively, you may merge caffeine_withdrawal branch with whatever > branch you are using, and use that version of Firedrake. > > Regards, > Miklos > > ------------------------------ > *From:* firedrake-bounces(a)imperial.ac.uk [firedrake-bounces(a)imperial.ac.uk] > on behalf of Eike Mueller [e.mueller(a)bath.ac.uk] > *Sent:* 14 February 2015 10:53 > *To:* firedrake > *Subject:* Re: [firedrake] Crash when running at higher order on ARCHER: > resolved now > > Dear firedrakers, > > I finally got to the bottom of this. It turns out that I had set > parameters[“COFFEE”][“O2”]= False around the parloop which executes the > kernel, but not around the bit of code which actually compiles the UFL > form. Stupid mistake… So this caused a horrible segfault, since the kernel > expected data of size A[8][20], but it was passed A[6][18]. It was quite > tricky to find this kind of bug, though, since it only segfaults without > much information. I finally managed to run interactively on ARCHER, > inspected the core dump with gdb and looked at the generated c-code. I was > wondering whether this kind of issue can be detected when you generate the > wrapper code? Don’t you know both the signature of the function and the > passed data at this point? > > Or has the COFFEE optimisation issue been resolved? I pulled the latest > version of COFFEE, though. > > Thanks, > > Eike > > -- > > Dr Eike Hermann Mueller > Research Associate (PostDoc) > > Department of Mathematical Sciences > University of Bath > Bath BA2 7AY, United Kingdom > > +44 1225 38 5803 > e.mueller(a)bath.ac.uk > http://people.bath.ac.uk/em459/ > > On 5 Feb 2015, at 16:28, Eike Mueller <E.Mueller(a)bath.ac.uk> wrote: > > Thanks, I tried the atp and also inspected the core dump with > > gdb python core > > There is no backtrace in the core dump, and ATP does not generate any > information either. > > I still only get the segfault in my output file. I hope I can localise > this a bit more tomorrow. > > Eike > > On 05/02/15 15:23, Patrick Farrell wrote: > > On 05/02/15 14:37, Lawrence Mitchell wrote: > > > Number of cells on finest grid = 5120 dx = 364.458 km, dt = > 2429.717 s _pmiu_daemon(SIGCHLD): [NID 01160] [c6-0c0s2n0] [Thu Feb > 5 14:22:05 2015] PE RANK 11 exit signal Segmentation fault [NID > 01160] 2015-02-05 14:22:05 Apid 12880356: initiated application > termination Application 12880356 exit codes: 139 Application > 12880356 resources: utime ~31s, stime ~19s, Rss ~318352, inblocks > ~104428, outblocks ~788 Finished atThu Feb 5 14:22:10 GMT 2015 > > > Hmm, that's not a lot of useful information. > > > Try running again with > > module load atp > export ATP_ENABLED=1 > > Sometimes it gives useful information about abnormal terminations; > http://www.archer.ac.uk/documentation/best-practice-guide/debug.php > > Patrick > > _______________________________________________ > firedrake mailing list > firedrake(a)imperial.ac.uk > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > _______________________________________________ > firedrake mailing list > firedrake(a)imperial.ac.uk > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > >

1 0

Re: [firedrake] Problems running on more than one node (ARCHER)
by Andrew McRae 16 Feb '15

16 Feb '15

Ah, my .env file was missing some magic lines. PYOP2_CACHE_DIR=$WORK/pyop2-cache export PYOP2_CACHE_DIR FIREDRAKE_FFC_KERNEL_CACHE_DIR=$WORK/firedrake-ffc-cache export FIREDRAKE_FFC_KERNEL_CACHE_DIR On 15 February 2015 at 23:27, Rathgeber, Florian < florian.rathgeber(a)imperial.ac.uk> wrote: > Don't use /tmp on ARCHER! It's mounted to ramfs, so you *really* > shouldn't write anything there. See also > > http://archer.ac.uk/documentation/user-guide/resource_management.php#sec-3.… > > On 2/15/2015 11:01 PM, Miklos Homolya wrote: > > Uh, is /tmp globally visible on ARCHER, or is it local to each node? > > Is it just rank 0, who writes the generated kernel to file? > > > > If the answers are local and /tmp, and rank 0 writing the kernel, then > it’s easy to see why it fails. > > > >> On 15 Feb 2015, at 22:42, Andrew McRae <a.mcrae12(a)imperial.ac.uk> > wrote: > >> > >> I have a script which runs fine on 1 node (24 cores) but fails on 2 or > more. > >> > >> It seems to die somewhere during the extruded mesh generation, with > messages such as > >> > >> OSError: /tmp/pyop2-cache-uid15427/9d8ebf65631aa626aeb3cd8098e2662f.so: > cannot open shared object file: No such file or directory > >> > >> Full output and error files attached. > >

1 0

Crash when running at higher order on ARCHER
by Eike Mueller 16 Feb '15

16 Feb '15

Dear firedrakers, I still get a crash when running at higher order on ARCHER (but it runs fine at lowest order). The output is below. This is odd, since the problem appears to be in the mesh generation, which should be the same for lowest- and higher order? This time I chose the problem size small enough (problems of this size run on my laptop, and I use a full node on ARCHER). I use the knepley/fix-plex-1d-refinement branch of PETSc and the firedrake branch of petsc4py On my laptop (where it works) I use dmplex-1d-refinement branch of petsc. Thanks, Eike eike@eslogin002 $ cat helmholtz_2692059.sdb/output.log Running helmholtz PBS_JOBID = 2692059.sdb Started atThu Feb 5 08:39:45 GMT 2015 +---------------------------+ ! Mixed Gravity wave solver ! +---------------------------+ Running on 24 MPI processes *** Parameters *** General: solve_matrixfree = True solve_petsc = True warmup_run = True orography = False nu_cfl = 10.0 speed_N = 0.01 speed_c = 300.0 Output: savetodisk = False output_dir = output Grid: nlayer = 64 ref_count_coarse = 0 nlevel = 3 thickness = 10000.0 Mixed system: higher_order = True verbose = 2 schur_diagonal_only = False maxiter = 20 tolerance = 1e-05 ksp_type = gmres Pressure solve: maxiter = 3 tolerance = 1e-14 verbose = 1 ksp_type = cg Multigrid: mu_relax = 0.8 n_postsmooth = 1 n_coarsesmooth = 1 n_presmooth = 1 Traceback (most recent call last): File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 486, in <module> Traceback (most recent call last): File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 486, in <module> Traceback (most recent call last): File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 486, in <module> Traceback (most recent call last): File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 486, in <module> main(parameter_filename) File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 256, in main main(parameter_filename) File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 256, in main param_grid['thickness']) File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 146, in build_mesh_hierarchy param_grid['thickness']) File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 146, in build_mesh_hierarchy refinement_level=ref_count_coarse) refinement_level=ref_count_coarse) main(parameter_filename) File "/work/n02/n02/eike/git_workspace/firedrake/firedrake/utility_meshes.py", line 560, in IcosahedralSphereMesh File "/work/n02/n02/eike/git_workspace/firedrake/firedrake/utility_meshes.py", line 560, in IcosahedralSphereMesh File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 256, in main param_grid['thickness']) File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 146, in build_mesh_hierarchy main(parameter_filename) refinement_level=ref_count_coarse) File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 256, in main File "/work/n02/n02/eike/git_workspace/firedrake/firedrake/utility_meshes.py", line 560, in IcosahedralSphereMesh m = mesh.Mesh(plex, dim=3, reorder=reorder) m = mesh.Mesh(plex, dim=3, reorder=reorder) File "<string>", line 2, in Mesh File "<string>", line 2, in Mesh m = mesh.Mesh(plex, dim=3, reorder=reorder) File "<string>", line 2, in Mesh param_grid['thickness']) File "/work/n02/n02/eike//git_workspace/firedrake-helmholtzsolver/source/driver.py", line 146, in build_mesh_hierarchy refinement_level=ref_count_coarse) File "/work/n02/n02/eike/git_workspace/firedrake/firedrake/utility_meshes.py", line 560, in IcosahedralSphereMesh m = mesh.Mesh(plex, dim=3, reorder=reorder) File "<string>", line 2, in Mesh File "/work/n02/n02/eike/git_workspace/PyOP2/pyop2/profiling.py", line 197, in wrapper File "/work/n02/n02/eike/git_workspace/PyOP2/pyop2/profiling.py", line 197, in wrapper File "/work/n02/n02/eike/git_workspace/PyOP2/pyop2/profiling.py", line 197, in wrapper File "/work/n02/n02/eike/git_workspace/PyOP2/pyop2/profiling.py", line 197, in wrapper return f(*args, **kwargs) return f(*args, **kwargs) File "/work/n02/n02/eike/git_workspace/firedrake/firedrake/mesh.py", line 210, in Mesh File "/work/n02/n02/eike/git_workspace/firedrake/firedrake/mesh.py", line 210, in Mesh return f(*args, **kwargs) File "/work/n02/n02/eike/git_workspace/firedrake/firedrake/mesh.py", line 210, in Mesh return f(*args, **kwargs) File "/work/n02/n02/eike/git_workspace/firedrake/firedrake/mesh.py", line 210, in Mesh cell_facets = plex.getConeSize(cStart) cell_facets = plex.getConeSize(cStart) cell_facets = plex.getConeSize(cStart) cell_facets = plex.getConeSize(cStart) File "PETSc/DMPlex.pyx", line 140, in petsc4py.PETSc.DMPlex.getConeSize (src/petsc4py.PETSc.c:203913) File "PETSc/DMPlex.pyx", line 140, in petsc4py.PETSc.DMPlex.getConeSize (src/petsc4py.PETSc.c:203913) File "PETSc/DMPlex.pyx", line 140, in petsc4py.PETSc.DMPlex.getConeSize (src/petsc4py.PETSc.c:203913) File "PETSc/DMPlex.pyx", line 140, in petsc4py.PETSc.DMPlex.getConeSize (src/petsc4py.PETSc.c:203913) AssertionError AssertionError AssertionError AssertionError Rank 13 [Thu Feb 5 08:40:23 2015] [c7-0c2s11n1] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 13 Rank 19 [Thu Feb 5 08:40:23 2015] [c7-0c2s11n1] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 19 Rank 1 [Thu Feb 5 08:40:23 2015] [c7-0c2s11n1] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1 Rank 7 [Thu Feb 5 08:40:23 2015] [c7-0c2s11n1] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 7 _pmiu_daemon(SIGCHLD): [NID 01517] [c7-0c2s11n1] [Thu Feb 5 08:40:23 2015] PE RANK 1 exit signal Aborted [NID 01517] 2015-02-05 08:40:23 Apid 12877131: initiated application termination Application 12877131 exit codes: 134 Application 12877131 resources: utime ~1s, stime ~15s, Rss ~59684, inblocks ~104518, outblocks ~790 Finished atThu Feb 5 08:40:25 GMT 2015

5 8

Problems running on more than one node (ARCHER)
by Andrew McRae 15 Feb '15

15 Feb '15

I have a script which runs fine on 1 node (24 cores) but fails on 2 or more. It seems to die somewhere during the extruded mesh generation, with messages such as OSError: /tmp/pyop2-cache-uid15427/9d8ebf65631aa626aeb3cd8098e2662f.so: cannot open shared object file: No such file or directory Full output and error files attached.

3 2