- firedrake - imperial.ac.uk

Re: [firedrake] .assign() with a function from a mixed function space
by Hiroe Yamazaki 26 Nov '14

26 Nov '14

Dear Colin and Lawrence, switching off assembly caching worked for me too. Does this mean that 1. the error was all a matter of assemble() and nothing related to .assign() and/or .split()? 2. I had better use "parameters['assembly_cache']['enabled'] = False" as a default setting to make sure assemble() works as expected? Thanks, Hiroe 2014-11-25 21:06 GMT+00:00 Cotter, Colin J <colin.cotter(a)imperial.ac.uk>: > Dear all, > > Just to clarify that the file to execute the test is called "test_assign.py". For ease I've pasted it here: > https://gist.github.com/colinjcotter/4b7eb6731a6cb388efee > > Switching off assembly caching does indeed fix this for me, i.e. > parameters['assembly_cache']['enabled'] = False > > at the top of the file. > > all the best > --cjc > > ________________________________________ > From: firedrake-bounces(a)imperial.ac.uk [firedrake-bounces(a)imperial.ac.uk] on behalf of Hiroe Yamazaki [h.yamazaki(a)imperial.ac.uk] > Sent: 25 November 2014 19:03 > To: firedrake > Subject: [firedrake] .assign() with a function from a mixed function space > > Dear all, > > I got confused with .assign() with a function from a mixed function > space and I would appreciate some suggestions. The test code is in > bitbucket.org/colinjcotter/slicemodels/branch/tidy revision f87e865. > > I expect a non-zero kinetic energy is printed out in line 54, but the > result is still 0. To get an updated kinetic energy, I need line 57. > > Am I misunderstanding what .assign() and/or .split() do? > > All the best, > Hiroe > > _______________________________________________ > firedrake mailing list > firedrake(a)imperial.ac.uk > https://mailman.ic.ac.uk/mailman/listinfo/firedrake

2 1

.assign() with a function from a mixed function space
by Hiroe Yamazaki 25 Nov '14

25 Nov '14

Dear all, I got confused with .assign() with a function from a mixed function space and I would appreciate some suggestions. The test code is in bitbucket.org/colinjcotter/slicemodels/branch/tidy revision f87e865. I expect a non-zero kinetic energy is printed out in line 54, but the result is still 0. To get an updated kinetic energy, I need line 57. Am I misunderstanding what .assign() and/or .split() do? All the best, Hiroe

2 1

tracking write accesses to dats/functions
by Eike Mueller 24 Nov '14

24 Nov '14

Dear firedrakers, can I somehow check whether a dat/function has been written to since I last performed a given operation on it? I have a banded matrix A stored in each cell on mesh, and whenever I solve the equation A.x = b (separately in each column, of course) I want to do this in two steps: (1) calculate the LU decomposition (lapack dgbtrf) (2) solve based on that LU decomposition (lapack dgbtrs) I have to do a lot of solves, so for efficiency I want to only do (1) if my matrix A has changed. So when I do a solve, I check if a flag lu_decomp is True (and this flag is set to False upon matrix creation). If it is, then I go straight to (2), otherwise I do (1) first. But I need to make sure that whenever my banded matrix has been written to lu_decomp is set back to False. Thanks, Eike

4 4

firedrake broken on laptop
by Shipton, Jemma 20 Nov '14

20 Nov '14

Hello all, I mentioned this in the chat earlier - I thought it might help to give a summary here... I have updated firedrake and it no longer passes its tests. PyOP2 does pass make test. I have pip installed (-u --no-deps) everything I can think of: fiat, ffc, ufl, six, sympy, instant, petsc4py and I have pip installed petsc as per the instructions on the website. In the firedrake directory I have done a make clean all and I have run scripts/firedrake-clean I have attached the output from the first failing test: tests/test_ffc_interface.py in the hope that that will show what is wrong. If there is anything else I can send to help diagnose the problem then please let me know. Thanks for all your help, Jemma

2 2

Re: [firedrake] parallel weirdness
by Hiroe Yamazaki 20 Nov '14

20 Nov '14

Hi Lawrence, I updated the files with the exact settings I see the problem, so could you run test_linear.py again with the current HEAD of the branch "periodic_parallel" (267de84)? With this setting I got results that looked different at t = 600*dt in serial and parallel. I've attached the picture for your reference (left: serial, right: parallel(2 cores), serial run gives a symmetric result but parallel run results in slightly asymmetric). All the best, Hiroe 2014-11-19 11:10 GMT+00:00 Mitchell, Lawrence <lawrence.mitchell(a)imperial.ac.uk>: > > On 18 Nov 2014, at 21:35, Hiroe Yamazaki <h.yamazaki(a)imperial.ac.uk> wrote: > >> Hi Lawrence, >> >> you can reproduce the problem by runing test_linear.py at >> bitbucket.org/colinjcotter/slicemodels/branch/periodic_parallel. With >> 1 and 2 cores, it gives different results. >> >> The code Colin is describing with line numbers is an old-version of >> slicemodels.py, which is found here: >> >> https://gist.github.com/anonymous/c308131558ff54fec9c4 >> >> In this version I added this term >> >> - (-dt*div(w)*pbar)*dx >> >> in line 270, which was previously added to the RHS in line 139. >> With this change, the parallelization problem seemed to be fixed and I >> got the same results with 1 and 2 cores, but but it makes no sense to >> us. > > I cannot reproduce this problem with the current HEAD of that branch (6ce1ffb02). The output vtus look the same in the eyeball norm, and once I fix the printing of the deltap (Colin, in parallel you can't look at the sum of the local entries. Do function.dat.norm to get the l2 norm or norm(function) for the L2 norm). In particular, if I specify jacobi PCs for /all/ solvers I get basically identical values in serial and parallel (up to solver tolerance) > > So I don't really know where to look. Can you give a precise sequence of steps for me to follow to reproduce the problem and observe it? > > Lawrence

4 4

Re: [firedrake] parallel weirdness
by Hiroe Yamazaki 20 Nov '14

20 Nov '14

Thanks, Lawrence. I see the residual behavior and changed 'nits' (the number of iterations during the each time step) from 2 to 3, then the parallelization problem appears to go away without moving terms. Would that make sense for you (I still haven't understood why the residual is reduced less fast in parallel runs)? Anyway this change makes the calculation time a bit longer, but the parallel efficiency seems to be good in larger problems, so it would be still highly beneficial. Cheers, Hiroe 2014-11-19 17:23 GMT+00:00 Mitchell, Lawrence <lawrence.mitchell(a)imperial.ac.uk>: > > On 19 Nov 2014, at 13:01, Hiroe Yamazaki <h.yamazaki(a)imperial.ac.uk> wrote: > >> Hi Lawrence, >> >> I updated the files with the exact settings I see the problem, so >> could you run test_linear.py again with the current HEAD of the branch >> "periodic_parallel" (267de84)? With this setting I got results that >> looked different at t = 600*dt in serial and parallel. I've attached >> the picture for your reference (left: serial, right: parallel(2 >> cores), serial run gives a symmetric result but parallel run results >> in slightly asymmetric). > > Thanks, I see the problem. I note that if I dial the solver tolerances up to 1e-10, then the problem appears to go away, but other than that I've got no idea. I note that by moving this one term, you'll change the subspaces the Krylov solves explore. If I monitor the convergence of the U solver, I see that the true residual is reduced less fast than the preconditioned residual, so is the problem that you somehow haven't killed the errors well enough? Does this give any useful clues? > > Cheers, > > Lawrence

1 0

Re: [firedrake] NonlinearVariationalSolver
by David Ham 19 Nov '14

19 Nov '14

Hi Anna, The linear variational solves you are seeing are the projection of your solution fields into the bilinear function space for output purposes. This is because direct output of quadratic functions to VTU is not supported. This won't change the answer, but obviously it will have some impact on the visualisation of the answer. Regards, David On 19 November 2014 19:45, Anna Kalogirou <a.kalogirou(a)leeds.ac.uk> wrote: > Hello, > > Can someone of you please check the way I use NonlinearVariationalSolver > in the code attached? The code runs but the result is a bit odd. In > addition, in the screen it writes > > pyop2:INFO Solving linear variational problem... > > and I am suspecting it might be solving the corresponding linear problem > instead of the nonlinear one. > > Thank you very much. > > Best, Anna. > > > > On 11/11/14 12:31, Lawrence Mitchell wrote: > > Hi Anna, > > > > On 10/11/14 20:25, Lawrence Mitchell wrote: > >> Hi Anna, > >> > >> On 10 Nov 2014, at 19:39, Anna Kalogirou <a.kalogirou(a)leeds.ac.uk> > >> wrote: > >> > >>> Hi Lawrence, > >>> > >>> Does it matter that I am solving a linear problem with the > >>> nonlinear solver? I am just trying to reproduce the linear one that > >>> works first, before introducing the nonlinear terms. I tried also > >>> adding the parameter {'snes_type': 'ksponly'} and the code works, > >>> but the result is not as good. > >>> > >>> Do you have any idea what might be happening? > >>> > >>> P.S. The linear code can be found attached. > >> > >> Thanks, I think I have found the problem, it is not in your code. > >> The problem appears to be a subtle issue in which we interact with > >> PETSc, which means that it is not computing the initial residual > >> correctly. I will prepare a fix tomorrow and let you know when > >> things are fixed. > > I've now fixed the problem. If you reinstall PyOP2 (I guess with pip, > > if you followed the website instructions) your nonlinear problem should > > now run fine. > > > > Let us know if you have any problems. > > > > Cheers, > > > > Lawrence > > -- > > Dr Anna Kalogirou > Research Fellow > School of Mathematics > University of Leeds > > -- Dr David Ham Departments of Mathematics and Computing Imperial College London http://www.imperial.ac.uk/people/david.ham

1 0

Re: [firedrake] Error - NonlinearVariationalSolver
by Lawrence Mitchell 19 Nov '14

19 Nov '14

Failed with sending to the list, so copying below. Lawrence On 10 Nov 2014, at 19:02, Lawrence Mitchell <lawrence.mitchell(a)imperial.ac.uk> wrote: > On 10 Nov 2014, at 18:48, Anna Kalogirou <a.kalogirou(a)leeds.ac.uk> wrote: > >> Dear Lawrence, >> >> Thank you very much for the reply. Very useful. It now gives the error: >> >> ====================== >> Traceback (most recent call last): >> File "nonlinear_swe.py", line 73, in <module> >> phi_solver.solve() >> File "<string>", line 2, in solve >> File "/opt/firedrake/2.0/lib/python2.7/site-packages/PyOP2-0.11.0_117_gcfe592d-py2.7-linux-x86_64.egg/pyop2/profiling.py", line 197, in wrapper >> return f(*args, **kwargs) >> File "/opt/firedrake/2.0/lib/python2.7/site-packages/firedrake/solving.py", line 336, in solve >> %s""" % (self.snes.getIterationNumber(), msg)) >> RuntimeError: Nonlinear solve failed to converge after 0 nonlinear iterations. >> Reason: >> DIVERGED_LINE_SEARCH >> ====================== >> >> Do you have any idea what that might be? Do I need do give any arguments in the phi_solver.solve()? >> > > So this tells me that the nonlinear solver (line search newton) failed because the line search failed. You can try running with > > solver_parameters={'snes_monitor': True, 'ksp_monitor': True, 'snes_linesearch_monitor': True} > > (add this as an argument to the construction of the nonlinearvariationalsolver) > > to see what's going on. > > If I make what I /think/ are the correct changes to your initial script, I see that the phi solver has the same diverged line search. This looks to be because at the second time step, the residual is already 1e-14 before we start. But I don't know if that's expected, or if I changed bits correctly. > > Lawrence

2 4

Re: [firedrake] parallel weirdness
by Hiroe Yamazaki 19 Nov '14

19 Nov '14

Hi Lawrence, you can reproduce the problem by runing test_linear.py at bitbucket.org/colinjcotter/slicemodels/branch/periodic_parallel. With 1 and 2 cores, it gives different results. The code Colin is describing with line numbers is an old-version of slicemodels.py, which is found here: https://gist.github.com/anonymous/c308131558ff54fec9c4 In this version I added this term - (-dt*div(w)*pbar)*dx in line 270, which was previously added to the RHS in line 139. With this change, the parallelization problem seemed to be fixed and I got the same results with 1 and 2 cores, but but it makes no sense to us. All the best, Hiroe 2014-11-18 20:46 GMT+00:00 Mitchell, Lawrence <lawrence.mitchell(a)imperial.ac.uk>: > Hello, > > On 18 Nov 2014, at 18:57, Colin Cotter <colin.cotter(a)imperial.ac.uk> wrote: > >> Dear all, >> We have a rather weird parallel bug, and Hiroe and I would appreciate some suggestions. The relevant code is in bitbucket.org/colinjcotter/slicemodels/branch/periodic_parallel revision 622fb24. >> We were observing different results on 1 and 2 cores, which Hiroe fixed by moving some code, but it makes no sense to us! >> In line 270, you can see an extra term, >> - (-dt*div(w)*pbar)*dx, >> which was previously added to the RHS in line 139 >> so, this changes implementation but not the algorithm >> either, this extra term is solved into a contribution of self.ures in L142, in which case it pops up as part of self.ures in l270, or we write it explicitly in the form there in general, we don't wish to write it explicitly in the form though, and I want to understand what is going on! >> >> Some further investigation showed that the problem seems to occur in the pressure solve on line 393, resulting in a nonsense value of self.Deltap occurring. This is just inverting a DG mass system so it ought to be straightforward, and the inputs are (approximately) the same. >> >> Do let me know if you have any ideas, > > > Can you describe what I need to do to reproduce the problem? I have looked at the repository, but don't know where to begin. > > Lawrence

2 1

parallel weirdness
by Colin Cotter 18 Nov '14

18 Nov '14

Dear all, We have a rather weird parallel bug, and Hiroe and I would appreciate some suggestions. The relevant code is in bitbucket.org/colinjcotter/slicemodels/branch/periodic_parallel revision 622fb24. We were observing different results on 1 and 2 cores, which Hiroe fixed by moving some code, but it makes no sense to us! In line 270, you can see an extra term, - (-dt*div(w)*pbar)*dx, which was previously added to the RHS in line 139 so, this changes implementation but not the algorithm either, this extra term is solved into a contribution of self.ures in L142, in which case it pops up as part of self.ures in l270, or we write it explicitly in the form there in general, we don't wish to write it explicitly in the form though, and I want to understand what is going on! Some further investigation showed that the problem seems to occur in the pressure solve on line 393, resulting in a nonsense value of self.Deltap occurring. This is just inverting a DG mass system so it ought to be straightforward, and the inputs are (approximately) the same. Do let me know if you have any ideas, cheers --cjc

2 1