- firedrake - imperial.ac.uk

Re: [firedrake] aborted run
by David Ham 22 May '16

22 May '16

Additionally, that's something like half a million cells, so you should be able to go faster by using more cores. Depending on your discretisation, 10-20 cores probably makes sense for that problem. Regards, David On Sun, 22 May 2016 at 08:24 Cotter, Colin J <colin.cotter(a)imperial.ac.uk> wrote: > Hi Francis, > This is because the solver inside the projection is using ilu and it is > running out of memory. More scalable solvers are available but in this case > I would suggest making the velocity space into a DG vector function space > large enough to contain your grad perp (same degree as the pv space works) > and then using interpolate instead of project, which produces the same > result mathematically but can then be computed locally. This wasn't > previously possible which is why we put project. > > All the best > Cjc > > On Sunday, 22 May 2016, Francis Poulin <fpoulin(a)uwaterloo.ca> wrote: > >> Hello, >> >> First, sorry to trouble you on the weekend, if you are reading this on >> the weekend that is. Second, Jemma and I have had some success with our 3D >> QG model. You can seem the result of the instability of a barotropic jet a >> very course 50x50x50 run at the following link. The movie needs a lot of >> fixing up but energy and enstrophy are nearly conserved so somethings are >> going pretty well. >> >> https://www.youtube.com/watch?v=CbqdA3a9DZg >> >> I decide to try a finer resolution case, 100x100x50, in hopes of having a >> better data set to work with. >> >> I am using a time step of 0.01 and it does 1 time unit on 2 cores on my >> laptop in over an hour. Not sure if that is very fast or not but it was >> working away up until it died. Below you see the last few outputs of the >> energies before things went all pear shaped. >> >> I have a fixed time step and so can believe that maybe the CFL condition >> was violated but for a 2D case I did a 200x200 run with the same time step >> so I'm not convinced that the CFL condition is violated. >> >> Given the output, does anyone have any ideas on what might have gone >> wrong? >> >> It would be great to restart from the last output and see whether this >> happens every time. Unfortunately we haven't gotten to that stage yet but >> we really should do that next. >> >> Cheers, Francis >> >> >> ... >> 33.98 16.504104156 39.1507283386 55.6548324946 33.3509321074 >> 33.99 16.504103109 39.1507164876 55.6548195966 33.3509288412 >> 34.0 16.5041020578 39.1507045844 55.6548066422 33.3509255645 >> Traceback (most recent call last): >> File "qg3d_jet.py", line 151, in <module> >> v.project(gradperp_h(psi0)) >> File >> "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/function.py", >> line 277, in project >> return projection.project(b, self, *args, **kwargs) >> File >> "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/projection.py", >> line 101, in project >> form_compiler_parameters=form_compiler_parameters) >> File >> "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", >> line 118, in solve >> _solve_varproblem(*args, **kwargs) >> File >> "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", >> line 145, in _solve_varproblem >> solver.solve() >> File >> "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/variational_solver.py", >> line 185, in solve >> self.snes.solve(None, v) >> File "PETSc/SNES.pyx", line 520, in petsc4py.PETSc.SNES.solve >> (src/petsc4py.PETSc.c:168132) >> Traceback (most recent call last): >> File "qg3d_jet.py", line 151, in <module> >> v.project(gradperp_h(psi0)) >> File >> "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/function.py", >> line 277, in project >> return projection.project(b, self, *args, **kwargs) >> File >> "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/projection.py", >> line 101, in project >> form_compiler_parameters=form_compiler_parameters) >> File >> "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", >> line 118, in solve >> _solve_varproblem(*args, **kwargs) >> File >> "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", >> line 145, in _solve_varproblem >> solver.solve() >> File >> "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/variational_solver.py", >> line 185, in solve >> self.snes.solve(None, v) >> File "PETSc/SNES.pyx", line 520, in petsc4py.PETSc.SNES.solve >> (src/petsc4py.PETSc.c:168132) >> petsc4py.PETSc.Errorpetsc4py.PETSc.Error: error code 1 >> [0] SNESSolve() line 4008 in >> /tmp/pip-EZZoo0-build/src/snes/interface/snes.c >> [0] SNESSolve_KSPONLY() line 40 in >> /tmp/pip-EZZoo0-build/src/snes/impls/ksponly/ksponly.c >> [0] KSPSolve() line 600 in >> /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c >> [0] KSPSetUpOnBlocks() line 220 in >> /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c >> [0] PCSetUpOnBlocks() line 1001 in >> /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c >> [0] PCSetUpOnBlocks_BJacobi_Singleblock() line 650 in >> /tmp/pip-EZZoo0-build/src/ksp/pc/impls/bjacobi/bjacobi.c >> [0] KSPSetUp() line 390 in >> /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c >> [0] PCSetUp() line 968 in >> /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c >> [0] PCSetUp_ILU() line 212 in >> /tmp/pip-EZZoo0-build/src/ksp/pc/impls/factor/ilu/ilu.c >> [0] MatILUFactorSymbolic() line 6594 in >> /tmp/pip-EZZoo0-build/src/mat/interface/matrix.c >> [0] MatILUFactorSymbolic_SeqBAIJ() line 370 in >> /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c >> [0] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in >> /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c >> [0] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in >> /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c >> [0] Out of memory. Allocated: 0, Used by process: 2962321408 >> [0] Memory requested 584000046 >> : error code 1 >> [1] SNESSolve() line 4008 in >> /tmp/pip-EZZoo0-build/src/snes/interface/snes.c >> [1] SNESSolve_KSPONLY() line 40 in >> /tmp/pip-EZZoo0-build/src/snes/impls/ksponly/ksponly.c >> [1] KSPSolve() line 600 in >> /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c >> [1] KSPSetUpOnBlocks() line 220 in >> /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c >> [1] PCSetUpOnBlocks() line 1001 in >> /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c >> [1] PCSetUpOnBlocks_BJacobi_Singleblock() line 650 in >> /tmp/pip-EZZoo0-build/src/ksp/pc/impls/bjacobi/bjacobi.c >> [1] KSPSetUp() line 390 in >> /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c >> [1] PCSetUp() line 968 in >> /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c >> [1] PCSetUp_ILU() line 212 in >> /tmp/pip-EZZoo0-build/src/ksp/pc/impls/factor/ilu/ilu.c >> [1] MatILUFactorSymbolic() line 6594 in >> /tmp/pip-EZZoo0-build/src/mat/interface/matrix.c >> [1] MatILUFactorSymbolic_SeqBAIJ() line 370 in >> /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c >> [1] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in >> /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c >> [1] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in >> /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c >> [1] Out of memory. Allocated: 0, Used by process: 2943778816 >> [1] Memory requested 584000046 >> -------------------------------------------------------------------------- >> MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD >> with errorcode 1. >> >> NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. >> You may or may not see output from other processes, depending on >> exactly when Open MPI kills them. >> -------------------------------------------------------------------------- >> >> >> ------------------ >> Francis Poulin >> Associate Professor >> Department of Applied Mathematics >> University of Waterloo >> >> email: fpoulin(a)uwaterloo.ca >> Web: https://uwaterloo.ca/poulin-research-group/ >> Telephone: +1 519 888 4567 x32637 >> >> > > -- > http://www.imperial.ac.uk/people/colin.cotter > > www.cambridge.org/9781107663916 > > > >

1 0

receipt for airport train
by Colin Cotter 22 May '16

22 May '16

Dear Firedrakers, For those of you who used a credit card to swipe in for the airport train and need a receipt, (1) follow this link: http://www.flytoget.no and flip to English if you don't read Norwegian (2) click on "need a receipt" (3) create an account, being sure to enter the same card details as the card you used to swipe in (4) your receipt will appear cheers --cjc -- http://www.imperial.ac.uk/people/colin.cotter www.cambridge.org/9781107663916

1 0

Re: [firedrake] aborted run
by Colin Cotter 22 May '16

22 May '16

Hi Francis, This is because the solver inside the projection is using ilu and it is running out of memory. More scalable solvers are available but in this case I would suggest making the velocity space into a DG vector function space large enough to contain your grad perp (same degree as the pv space works) and then using interpolate instead of project, which produces the same result mathematically but can then be computed locally. This wasn't previously possible which is why we put project. All the best Cjc On Sunday, 22 May 2016, Francis Poulin <fpoulin(a)uwaterloo.ca> wrote: > Hello, > > First, sorry to trouble you on the weekend, if you are reading this on the > weekend that is. Second, Jemma and I have had some success with our 3D QG > model. You can seem the result of the instability of a barotropic jet a > very course 50x50x50 run at the following link. The movie needs a lot of > fixing up but energy and enstrophy are nearly conserved so somethings are > going pretty well. > > https://www.youtube.com/watch?v=CbqdA3a9DZg > > I decide to try a finer resolution case, 100x100x50, in hopes of having a > better data set to work with. > > I am using a time step of 0.01 and it does 1 time unit on 2 cores on my > laptop in over an hour. Not sure if that is very fast or not but it was > working away up until it died. Below you see the last few outputs of the > energies before things went all pear shaped. > > I have a fixed time step and so can believe that maybe the CFL condition > was violated but for a 2D case I did a 200x200 run with the same time step > so I'm not convinced that the CFL condition is violated. > > Given the output, does anyone have any ideas on what might have gone wrong? > > It would be great to restart from the last output and see whether this > happens every time. Unfortunately we haven't gotten to that stage yet but > we really should do that next. > > Cheers, Francis > > > ... > 33.98 16.504104156 39.1507283386 55.6548324946 33.3509321074 > 33.99 16.504103109 39.1507164876 55.6548195966 33.3509288412 > 34.0 16.5041020578 39.1507045844 55.6548066422 33.3509255645 > Traceback (most recent call last): > File "qg3d_jet.py", line 151, in <module> > v.project(gradperp_h(psi0)) > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/function.py", > line 277, in project > return projection.project(b, self, *args, **kwargs) > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/projection.py", > line 101, in project > form_compiler_parameters=form_compiler_parameters) > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", > line 118, in solve > _solve_varproblem(*args, **kwargs) > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", > line 145, in _solve_varproblem > solver.solve() > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/variational_solver.py", > line 185, in solve > self.snes.solve(None, v) > File "PETSc/SNES.pyx", line 520, in petsc4py.PETSc.SNES.solve > (src/petsc4py.PETSc.c:168132) > Traceback (most recent call last): > File "qg3d_jet.py", line 151, in <module> > v.project(gradperp_h(psi0)) > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/function.py", > line 277, in project > return projection.project(b, self, *args, **kwargs) > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/projection.py", > line 101, in project > form_compiler_parameters=form_compiler_parameters) > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", > line 118, in solve > _solve_varproblem(*args, **kwargs) > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", > line 145, in _solve_varproblem > solver.solve() > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/variational_solver.py", > line 185, in solve > self.snes.solve(None, v) > File "PETSc/SNES.pyx", line 520, in petsc4py.PETSc.SNES.solve > (src/petsc4py.PETSc.c:168132) > petsc4py.PETSc.Errorpetsc4py.PETSc.Error: error code 1 > [0] SNESSolve() line 4008 in > /tmp/pip-EZZoo0-build/src/snes/interface/snes.c > [0] SNESSolve_KSPONLY() line 40 in > /tmp/pip-EZZoo0-build/src/snes/impls/ksponly/ksponly.c > [0] KSPSolve() line 600 in > /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c > [0] KSPSetUpOnBlocks() line 220 in > /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c > [0] PCSetUpOnBlocks() line 1001 in > /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c > [0] PCSetUpOnBlocks_BJacobi_Singleblock() line 650 in > /tmp/pip-EZZoo0-build/src/ksp/pc/impls/bjacobi/bjacobi.c > [0] KSPSetUp() line 390 in > /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c > [0] PCSetUp() line 968 in > /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c > [0] PCSetUp_ILU() line 212 in > /tmp/pip-EZZoo0-build/src/ksp/pc/impls/factor/ilu/ilu.c > [0] MatILUFactorSymbolic() line 6594 in > /tmp/pip-EZZoo0-build/src/mat/interface/matrix.c > [0] MatILUFactorSymbolic_SeqBAIJ() line 370 in > /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c > [0] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in > /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c > [0] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in > /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c > [0] Out of memory. Allocated: 0, Used by process: 2962321408 > [0] Memory requested 584000046 > : error code 1 > [1] SNESSolve() line 4008 in > /tmp/pip-EZZoo0-build/src/snes/interface/snes.c > [1] SNESSolve_KSPONLY() line 40 in > /tmp/pip-EZZoo0-build/src/snes/impls/ksponly/ksponly.c > [1] KSPSolve() line 600 in > /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c > [1] KSPSetUpOnBlocks() line 220 in > /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c > [1] PCSetUpOnBlocks() line 1001 in > /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c > [1] PCSetUpOnBlocks_BJacobi_Singleblock() line 650 in > /tmp/pip-EZZoo0-build/src/ksp/pc/impls/bjacobi/bjacobi.c > [1] KSPSetUp() line 390 in > /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c > [1] PCSetUp() line 968 in > /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c > [1] PCSetUp_ILU() line 212 in > /tmp/pip-EZZoo0-build/src/ksp/pc/impls/factor/ilu/ilu.c > [1] MatILUFactorSymbolic() line 6594 in > /tmp/pip-EZZoo0-build/src/mat/interface/matrix.c > [1] MatILUFactorSymbolic_SeqBAIJ() line 370 in > /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c > [1] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in > /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c > [1] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in > /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c > [1] Out of memory. Allocated: 0, Used by process: 2943778816 > [1] Memory requested 584000046 > -------------------------------------------------------------------------- > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > with errorcode 1. > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > You may or may not see output from other processes, depending on > exactly when Open MPI kills them. > -------------------------------------------------------------------------- > > > ------------------ > Francis Poulin > Associate Professor > Department of Applied Mathematics > University of Waterloo > > email: fpoulin(a)uwaterloo.ca > <javascript:_e(%7B%7D,'cvml','fpoulin(a)uwaterloo.ca');> > Web: https://uwaterloo.ca/poulin-research-group/ > Telephone: +1 519 888 4567 x32637 > > -- http://www.imperial.ac.uk/people/colin.cotter www.cambridge.org/9781107663916

1 0

aborted run
by Francis Poulin 22 May '16

22 May '16

Hello, First, sorry to trouble you on the weekend, if you are reading this on the weekend that is. Second, Jemma and I have had some success with our 3D QG model. You can seem the result of the instability of a barotropic jet a very course 50x50x50 run at the following link. The movie needs a lot of fixing up but energy and enstrophy are nearly conserved so somethings are going pretty well. https://www.youtube.com/watch?v=CbqdA3a9DZg I decide to try a finer resolution case, 100x100x50, in hopes of having a better data set to work with. I am using a time step of 0.01 and it does 1 time unit on 2 cores on my laptop in over an hour. Not sure if that is very fast or not but it was working away up until it died. Below you see the last few outputs of the energies before things went all pear shaped. I have a fixed time step and so can believe that maybe the CFL condition was violated but for a 2D case I did a 200x200 run with the same time step so I'm not convinced that the CFL condition is violated. Given the output, does anyone have any ideas on what might have gone wrong? It would be great to restart from the last output and see whether this happens every time. Unfortunately we haven't gotten to that stage yet but we really should do that next. Cheers, Francis ... 33.98 16.504104156 39.1507283386 55.6548324946 33.3509321074 33.99 16.504103109 39.1507164876 55.6548195966 33.3509288412 34.0 16.5041020578 39.1507045844 55.6548066422 33.3509255645 Traceback (most recent call last): File "qg3d_jet.py", line 151, in <module> v.project(gradperp_h(psi0)) File "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/function.py", line 277, in project return projection.project(b, self, *args, **kwargs) File "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/projection.py", line 101, in project form_compiler_parameters=form_compiler_parameters) File "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", line 118, in solve _solve_varproblem(*args, **kwargs) File "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", line 145, in _solve_varproblem solver.solve() File "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/variational_solver.py", line 185, in solve self.snes.solve(None, v) File "PETSc/SNES.pyx", line 520, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:168132) Traceback (most recent call last): File "qg3d_jet.py", line 151, in <module> v.project(gradperp_h(psi0)) File "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/function.py", line 277, in project return projection.project(b, self, *args, **kwargs) File "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/projection.py", line 101, in project form_compiler_parameters=form_compiler_parameters) File "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", line 118, in solve _solve_varproblem(*args, **kwargs) File "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/solving.py", line 145, in _solve_varproblem solver.solve() File "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/variational_solver.py", line 185, in solve self.snes.solve(None, v) File "PETSc/SNES.pyx", line 520, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:168132) petsc4py.PETSc.Errorpetsc4py.PETSc.Error: error code 1 [0] SNESSolve() line 4008 in /tmp/pip-EZZoo0-build/src/snes/interface/snes.c [0] SNESSolve_KSPONLY() line 40 in /tmp/pip-EZZoo0-build/src/snes/impls/ksponly/ksponly.c [0] KSPSolve() line 600 in /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c [0] KSPSetUpOnBlocks() line 220 in /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c [0] PCSetUpOnBlocks() line 1001 in /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c [0] PCSetUpOnBlocks_BJacobi_Singleblock() line 650 in /tmp/pip-EZZoo0-build/src/ksp/pc/impls/bjacobi/bjacobi.c [0] KSPSetUp() line 390 in /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c [0] PCSetUp() line 968 in /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c [0] PCSetUp_ILU() line 212 in /tmp/pip-EZZoo0-build/src/ksp/pc/impls/factor/ilu/ilu.c [0] MatILUFactorSymbolic() line 6594 in /tmp/pip-EZZoo0-build/src/mat/interface/matrix.c [0] MatILUFactorSymbolic_SeqBAIJ() line 370 in /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c [0] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c [0] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c [0] Out of memory. Allocated: 0, Used by process: 2962321408 [0] Memory requested 584000046 : error code 1 [1] SNESSolve() line 4008 in /tmp/pip-EZZoo0-build/src/snes/interface/snes.c [1] SNESSolve_KSPONLY() line 40 in /tmp/pip-EZZoo0-build/src/snes/impls/ksponly/ksponly.c [1] KSPSolve() line 600 in /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c [1] KSPSetUpOnBlocks() line 220 in /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c [1] PCSetUpOnBlocks() line 1001 in /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c [1] PCSetUpOnBlocks_BJacobi_Singleblock() line 650 in /tmp/pip-EZZoo0-build/src/ksp/pc/impls/bjacobi/bjacobi.c [1] KSPSetUp() line 390 in /tmp/pip-EZZoo0-build/src/ksp/ksp/interface/itfunc.c [1] PCSetUp() line 968 in /tmp/pip-EZZoo0-build/src/ksp/pc/interface/precon.c [1] PCSetUp_ILU() line 212 in /tmp/pip-EZZoo0-build/src/ksp/pc/impls/factor/ilu/ilu.c [1] MatILUFactorSymbolic() line 6594 in /tmp/pip-EZZoo0-build/src/mat/interface/matrix.c [1] MatILUFactorSymbolic_SeqBAIJ() line 370 in /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c [1] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c [1] MatILUFactorSymbolic_SeqBAIJ_ilu0() line 276 in /tmp/pip-EZZoo0-build/src/mat/impls/baij/seq/baijfact2.c [1] Out of memory. Allocated: 0, Used by process: 2943778816 [1] Memory requested 584000046 -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 1. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. -------------------------------------------------------------------------- ------------------ Francis Poulin Associate Professor Department of Applied Mathematics University of Waterloo email: fpoulin(a)uwaterloo.ca Web: https://uwaterloo.ca/poulin-research-group/ Telephone: +1 519 888 4567 x32637

1 0

Matrix-free operator on mixed function space
by Anna Kalogirou 20 May '16

20 May '16

Dear all, I am writing regarding the following problem: https://bitbucket.org/annakalog/buoy2d/src/3ee44ca3db164d8955d363b7d07f6223… I have previously defined a matrix-free operator as described in your documentation to solve a linear system, say A*u = b, where A=M+Q*Q^T. I now need to solve a similar system but in a mixed function space with two unknowns, i.e. A*u + B*v = b, B*u + C*v = 0. Here, matrices B, C have known forms that are similar to a mass matrix. The matrix A is the same as before. How would I solve the above linear system? Kind regards, Anna. -- Dr Anna Kalogirou Research Fellow School of Mathematics University of Leeds http://www1.maths.leeds.ac.uk/~matak/

2 1

Set vector component BC on extruded mesh
by Tianjiao Sun 19 May '16

19 May '16

Hi team, I'm running into "NotImplemented" exception when trying to set component of a vector field using DirichletBC(), on extruded mesh. Basically I'm trying to set the vertical component of the field to zero throughout the domain. Is there anyway around this? Many thanks, -TJ P.S. I think my function space below is perhaps not ideal, it has more DoFs than necessary. But I'm having difficulty in getting something like below to work # V_h = FiniteElement("CG", "quadrilateral", 1) # V_v = FiniteElement("DG", "interval", 0) # V_elt = HDivElement(TensorProductElement(V_h, V_v)) # V = FunctionSpace(mesh, V_elt) Code skeleton --------------------------------------------------------------------------- m = RectangleMesh(10,10,10,10,quadrilateral=True) mesh = ExtrudedMesh(m, layers=1) V = VectorFunctionSpace(mesh, "CG", 1) bc0 = DirichletBC(V.sub(2), Constant(0.0), "top") bc1 = DirichletBC(V.sub(2), Constant(0.0), "bottom") Error message --------------------------------------------------------------------------- NotImplementedError Traceback (most recent call last) <ipython-input-176-0fe88457c2f0> in <module>() ----> 1 bc0 = DirichletBC(V.sub(2), Constant(0.0), "top") 2 bc1 = DirichletBC(V.sub(2), Constant(0.0), "bottom") /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/firedrake/bcs.pyc in __init__(self, V, g, sub_domain, method) 60 61 if V.extruded and V.component is not None: ---> 62 raise NotImplementedError("Indexed VFS bcs not implemented on extruded meshes") 63 64 @property NotImplementedError: Indexed VFS bcs not implemented on extruded meshes

1 0

Re: [firedrake] mpirun acting up
by David Ham 19 May '16

19 May '16

You mistyped. dof_dset not dot_dset David On Thu, 19 May 2016 at 15:17 Francis Poulin <fpoulin(a)uwaterloo.ca> wrote: > Hello David and Lawrence, > > I specified Uj it like a I would a regular python variable, maybe a bad > idea. > > You can see the code below. > > https://github.com/francispoulin/firedrakeQG/blob/master/qg3d_jet.py > > I tried to take David's advance and modified the following > > q0 = Function(Vdg, > name='pv').interpolate(Expression("-2.*Uj/Lj*tanh((x[1]-yc)/Lj)/pow(cosh((x[1]-yc)/Lj),2)",Uj=Uj, > Lj=Lj, yc=yc)) > V = q0.function_space() > q0.dat.data[:] += 0.01*Uj*np.random.randn(V.dot_dset.size, *V.shape) > > but that didn't seem to work. Now I get the following error, > > > Traceback (most recent call last): > File "qg3d_jet.py", line 39, in <module> > q0.dat.data[:] += 0.01*Uj*np.random.randn(V.dot_dset.size, *V.shape) > File > "/home/fpoulin/software/firedrake/local/lib/python2.7/site-packages/firedrake/functionspaceimpl.py", > line 202, in __getattr__ > return getattr(self.topological, name) > AttributeError: 'FunctionSpace' object has no attribute 'dot_dset' > > > Sorry if I misunderstood the advice. > > Francis > > > > > Francis Poulin > Associate Professor > Department of Applied Mathematics > University of Waterloo > > email: fpoulin(a)uwaterloo.ca > Web: https://uwaterloo.ca/poulin-research-group/ > Telephone: +1 519 888 4567 x32637 > > ------------------------------ > *From:* firedrake-bounces(a)imperial.ac.uk [firedrake-bounces(a)imperial.ac.uk] > on behalf of David Ham [David.Ham(a)imperial.ac.uk] > *Sent:* Thursday, May 19, 2016 9:03 AM > *To:* firedrake > *Subject:* Re: [firedrake] mpirun acting up > > How is Uj set? It looks like Uj has halos attached, while the others do > not. > > Regards, > > David > > On Thu, 19 May 2016 at 14:55 Francis Poulin <fpoulin(a)uwaterloo.ca> wrote: > >> Hello, >> >> I am trying to run something with mpiand for some reason I am getting an >> error. This is both on my mac and ubuntu machines. Any ideas what might >> be the problem? I was able to do this before the upgrade and thought it >> should work in the same way but maybe something has change? Or maybe I'm >> doing something very silly. >> >> Cheers, Francis >> >> fpoulin@fpoulin-Gazelle:~/Research/Firedrake/QG/firedrakeQG$ mpirun -np >> 2 ~/software/firedrake/bin/python qg3d_jet.py >> Discontinuous Lagrange element requested on quadrilateral, creating DQ >> element. >> Discontinuous Lagrange element requested on quadrilateral, creating DQ >> element. >> Discontinuous Lagrange element requested on quadrilateral, creating DQ >> element. >> Discontinuous Lagrange element requested on quadrilateral, creating DQ >> element. >> COFFEE finished in 0.00156093 seconds (flops: 0 -> 0) >> Discontinuous Lagrange element requested on None, creating DQ element. >> COFFEE finished in 0.00156093 seconds (flops: 0 -> 0) >> Discontinuous Lagrange element requested on None, creating DQ element. >> Discontinuous Lagrange element requested on None, creating DQ element. >> Discontinuous Lagrange element requested on None, creating DQ element. >> COFFEE finished in 0.00232983 seconds (flops: 0 -> 0) >> Traceback (most recent call last): >> File "qg3d_jet.py", line 38, in <module> >> q0.dat.data[:] += 0.01*Uj*np.random.randn(*q0.dat.shape) >> ValueError: operands could not be broadcast together with shapes >> (500000,) (520000,) (500000,) >> Traceback (most recent call last): >> File "qg3d_jet.py", line 38, in <module> >> q0.dat.data[:] += 0.01*Uj*np.random.randn(*q0.dat.shape) >> ValueError: operands could not be broadcast together with shapes >> (500000,) (520000,) (500000,) >> COFFEE finished in 0.00234294 seconds (flops: 0 -> 0) >> -------------------------------------------------------------------------- >> MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD >> with errorcode 1. >> >> NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. >> You may or may not see output from other processes, depending on >> exactly when Open MPI kills them. >> >> >> >> ------------------ >> Francis Poulin >> Associate Professor >> Department of Applied Mathematics >> University of Waterloo >> >> email: fpoulin(a)uwaterloo.ca >> Web: https://uwaterloo.ca/poulin-research-group/ >> Telephone: +1 519 888 4567 x32637 >> >>

2 1

Re: [firedrake] mpirun acting up
by David Ham 19 May '16

19 May '16

How is Uj set? It looks like Uj has halos attached, while the others do not. Regards, David On Thu, 19 May 2016 at 14:55 Francis Poulin <fpoulin(a)uwaterloo.ca> wrote: > Hello, > > I am trying to run something with mpiand for some reason I am getting an > error. This is both on my mac and ubuntu machines. Any ideas what might > be the problem? I was able to do this before the upgrade and thought it > should work in the same way but maybe something has change? Or maybe I'm > doing something very silly. > > Cheers, Francis > > fpoulin@fpoulin-Gazelle:~/Research/Firedrake/QG/firedrakeQG$ mpirun -np 2 > ~/software/firedrake/bin/python qg3d_jet.py > Discontinuous Lagrange element requested on quadrilateral, creating DQ > element. > Discontinuous Lagrange element requested on quadrilateral, creating DQ > element. > Discontinuous Lagrange element requested on quadrilateral, creating DQ > element. > Discontinuous Lagrange element requested on quadrilateral, creating DQ > element. > COFFEE finished in 0.00156093 seconds (flops: 0 -> 0) > Discontinuous Lagrange element requested on None, creating DQ element. > COFFEE finished in 0.00156093 seconds (flops: 0 -> 0) > Discontinuous Lagrange element requested on None, creating DQ element. > Discontinuous Lagrange element requested on None, creating DQ element. > Discontinuous Lagrange element requested on None, creating DQ element. > COFFEE finished in 0.00232983 seconds (flops: 0 -> 0) > Traceback (most recent call last): > File "qg3d_jet.py", line 38, in <module> > q0.dat.data[:] += 0.01*Uj*np.random.randn(*q0.dat.shape) > ValueError: operands could not be broadcast together with shapes (500000,) > (520000,) (500000,) > Traceback (most recent call last): > File "qg3d_jet.py", line 38, in <module> > q0.dat.data[:] += 0.01*Uj*np.random.randn(*q0.dat.shape) > ValueError: operands could not be broadcast together with shapes (500000,) > (520000,) (500000,) > COFFEE finished in 0.00234294 seconds (flops: 0 -> 0) > -------------------------------------------------------------------------- > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > with errorcode 1. > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > You may or may not see output from other processes, depending on > exactly when Open MPI kills them. > > > > ------------------ > Francis Poulin > Associate Professor > Department of Applied Mathematics > University of Waterloo > > email: fpoulin(a)uwaterloo.ca > Web: https://uwaterloo.ca/poulin-research-group/ > Telephone: +1 519 888 4567 x32637 > >

2 1

mpirun acting up
by Francis Poulin 19 May '16

19 May '16

Hello, I am trying to run something with mpiand for some reason I am getting an error. This is both on my mac and ubuntu machines. Any ideas what might be the problem? I was able to do this before the upgrade and thought it should work in the same way but maybe something has change? Or maybe I'm doing something very silly. Cheers, Francis fpoulin@fpoulin-Gazelle:~/Research/Firedrake/QG/firedrakeQG$ mpirun -np 2 ~/software/firedrake/bin/python qg3d_jet.py Discontinuous Lagrange element requested on quadrilateral, creating DQ element. Discontinuous Lagrange element requested on quadrilateral, creating DQ element. Discontinuous Lagrange element requested on quadrilateral, creating DQ element. Discontinuous Lagrange element requested on quadrilateral, creating DQ element. COFFEE finished in 0.00156093 seconds (flops: 0 -> 0) Discontinuous Lagrange element requested on None, creating DQ element. COFFEE finished in 0.00156093 seconds (flops: 0 -> 0) Discontinuous Lagrange element requested on None, creating DQ element. Discontinuous Lagrange element requested on None, creating DQ element. Discontinuous Lagrange element requested on None, creating DQ element. COFFEE finished in 0.00232983 seconds (flops: 0 -> 0) Traceback (most recent call last): File "qg3d_jet.py", line 38, in <module> q0.dat.data[:] += 0.01*Uj*np.random.randn(*q0.dat.shape) ValueError: operands could not be broadcast together with shapes (500000,) (520000,) (500000,) Traceback (most recent call last): File "qg3d_jet.py", line 38, in <module> q0.dat.data[:] += 0.01*Uj*np.random.randn(*q0.dat.shape) ValueError: operands could not be broadcast together with shapes (500000,) (520000,) (500000,) COFFEE finished in 0.00234294 seconds (flops: 0 -> 0) -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 1. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. ------------------ Francis Poulin Associate Professor Department of Applied Mathematics University of Waterloo email: fpoulin(a)uwaterloo.ca Web: https://uwaterloo.ca/poulin-research-group/ Telephone: +1 519 888 4567 x32637

2 1

Re: [firedrake] Expression on extruded mesh
by David Ham 19 May '16

19 May '16

Hi TJ, Can you use the direct UFL interpolation feature instead? It's documented in the manual. Cheers, David On Wed, 18 May 2016 at 20:48 Sun, Tianjiao <tianjiao.sun14(a)imperial.ac.uk> wrote: > Hi team, > > I run into a "NotImplemented" exception when trying to assign values to > an function on extruded mesh of 1 layer. I'm wondering is it really not > implemented or (more likely) just my bad usage? The situation is that I > have this function on DG(0) x CG(1), and I just need to assign values to > the top and bottom surfaces. > > Many thanks! > -TJ > > code: > --------------------------------------------------------------------------- > m = RectangleMesh(5,5,5,5,quadrilateral=True) > mesh = ExtrudedMesh(m, layers=1) > W_h = FiniteElement("DG", "quadrilateral", 0) > W_v = FiniteElement("CG", "interval", 1) > W_elt = TensorProductElement(W_h, W_v) > W = FunctionSpace(mesh, W_elt) > > # This Expression reads values from an array > class Ex_intensity(Expression): > def eval(self, value, X): > # this is not quite completed, there will be more logic here > x0 = int(X[0]) > x1 = int(X[1]) > value[:] = p0[x1,x0] > > I = Function(W) > I.interpolate(Ex_intensity()) > I.at(2.5,2.5,0) > > Error message: > --------------------------------------------------------------------------- > NotImplementedError Traceback (most recent call last) > <ipython-input-12-055ffdeb8731> in <module>() > ----> 1 I.at(2.5,2.5,0) > > > /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/firedrake/function.pyc > in at(self, arg, *args, **kwargs) > 492 else: > 493 p_result = np.empty(value_shape, dtype=float) > --> 494 single_eval(points[i:i+1], p_result) > 495 l_result.append((i, p_result)) > 496 except PointNotInDomainError: > > > /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/firedrake/function.pyc > in single_eval(x, buf) > 470 def single_eval(x, buf): > 471 """Helper function to evaluate at a single point.""" > --> 472 err = self._c_evaluate(self._ctypes, > 473 x.ctypes.data_as(POINTER(c_double)), > 474 buf.ctypes.data_as(POINTER(c_double))) > > > /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/pyop2/utils.pyc > in __get__(self, obj, cls) > 62 if obj is None: > 63 return self > ---> 64 obj.__dict__[self.__name__] = result = self.fget(obj) > 65 return result > 66 > > > /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/firedrake/function.pyc > in _ctypes(self) > 406 else: > 407 c_function.n_layers = 1 > --> 408 c_function.coords = > coordinates.dat.data.ctypes.data_as(POINTER(c_double)) > 409 c_function.coords_map = > coordinates_space.cell_node_list.ctypes.data_as(POINTER(c_int)) > 410 c_function.f = > self.dat.data.ctypes.data_as(POINTER(c_double)) > > <decorator-gen-136> in data(self) > > > /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/pyop2/versioning.pyc > in modifies(method, self, *args, **kwargs) > 113 _force_copies(self) > 114 > --> 115 retval = method(self, *args, **kwargs) > 116 > 117 self._version_bump() > > > /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/pyop2/base.pyc > in data(self) > 1759 > 1760 """ > -> 1761 _trace.evaluate(set([self]), set([self])) > 1762 if self.dataset.total_size > 0 and self._data.size == 0 > and self.cdim > 0: > 1763 raise RuntimeError("Illegal access: no data > associated with this Dat!") > > > /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/pyop2/base.pyc > in evaluate(self, reads, writes) > 169 to_run = fuse('from_trace', to_run, 0) > 170 for comp in to_run: > --> 171 comp._run() > 172 > 173 > > > /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/pyop2/base.pyc > in _run(self) > 4073 > 4074 def _run(self): > -> 4075 return self.compute() > 4076 > 4077 def prepare_arglist(self, iterset, *args): > > > /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/pyop2/base.pyc > in compute(self) > 4115 arglist = self.prepare_arglist(iterset, *self.args) > 4116 fun = self._jitmodule > -> 4117 self._compute(iterset.core_part, fun, *arglist) > 4118 self.halo_exchange_end() > 4119 self._compute(iterset.owned_part, fun, *arglist) > > > /home/ts2914/projects/firedrake/local/lib/python2.7/site-packages/pyop2/pyparloop.pyc > in _compute(self, part, *arglist) > 106 def _compute(self, part, *arglist): > 107 if part.set._extruded: > --> 108 raise NotImplementedError > 109 subset = isinstance(self._it_space._iterset, base.Subset) > 110 > > NotImplementedError: > > > > _______________________________________________ > firedrake mailing list > firedrake(a)imperial.ac.uk > https://mailman.ic.ac.uk/mailman/listinfo/firedrake >

1 0