Re: [firedrake] [petsc-users] pcfieldsplitting not working for VI
Okay, switched over to the firedrake list On Tue, Nov 29, 2016 at 2:29 AM, Lawrence Mitchell < lawrence.mitchell@imperial.ac.uk> wrote:
I suspect this is not a petsc issue per se. Shall we migrate this to the firedrake list?
Lawrence
On 29 Nov 2016, at 07:40, Justin Chang <jychang48@gmail.com> wrote:
FWIW I ended up doing this:
rs_solver.setDM(W._dm())
and was left with this error:
Traceback (most recent call last): File "sphere.py", line 180, in <module> virs() File "sphere.py", line 163, in virs rs_solver.solve(None,sol_vec) File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:170012) petsc4py.PETSc.Error: error code 75 [0] SNESSolve() line 4060 in /private/var/folders/92/ 1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/snes/ interface/snes.c [0] SNESSolve_VINEWTONRSLS() line 424 in /private/var/folders/92/ 1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/snes/ impls/vi/rs/virs.c [0] MatGetSubMatrix() line 7929 in /private/var/folders/92/ 1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/mat/ interface/matrix.c [0] MatGetSubMatrix_Nest() line 451 in /private/var/folders/92/ 1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/mat/ impls/nest/matnest.c [0] MatNestFindSubMat() line 424 in /private/var/folders/92/ 1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/mat/ impls/nest/matnest.c [0] MatNestFindIS() line 363 in /private/var/folders/92/ 1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/mat/ impls/nest/matnest.c [0] Arguments are incompatible [0] Could not find index set
I must be doing something wrong here.
Justin
On Tue, Nov 29, 2016 at 1:24 AM, Justin Chang <jychang48@gmail.com> wrote:
Lawrence,
I added the following line:
rs_solver.setDM(a._dm)
And it gives this error:
Traceback (most recent call last): File "sphere.py", line 128, in <module> rs_solver.setDM(a._dm) AttributeError: 'Form' object has no attribute '_dm'
Thanks, Justin
On Tue, Nov 29, 2016 at 1:12 AM, Lawrence Mitchell < lawrence.mitchell@imperial.ac.uk> wrote:
On 29 Nov 2016, at 02:02, Justin Chang <jychang48@gmail.com> wrote:
Why is this happening? It seems to me VINEWTONRSLS is not understanding that I have a two-field formulation.
For the built in solvers, we set up a dmshell on the snes that knows how to do field splitting. Because you set your snes up by hand, it isn't available. You'll need to call snes.setDM. The mixed finctionspace provides the correct dm via a ._dm property I think.
Lawrence
On 29 Nov 2016, at 08:31, Justin Chang <jychang48@gmail.com> wrote:
Okay, switched over to the firedrake list
I wonder. Normally this appears if you didn't get the right splittings for a three or more field problem. I'm on my phone right now so can't debug. I'll have a look a bit later. Lawrence
On Tue, Nov 29, 2016 at 2:29 AM, Lawrence Mitchell <lawrence.mitchell@imperial.ac.uk> wrote: I suspect this is not a petsc issue per se. Shall we migrate this to the firedrake list?
Lawrence
On 29 Nov 2016, at 07:40, Justin Chang <jychang48@gmail.com> wrote:
FWIW I ended up doing this:
rs_solver.setDM(W._dm())
and was left with this error:
Traceback (most recent call last): File "sphere.py", line 180, in <module> virs() File "sphere.py", line 163, in virs rs_solver.solve(None,sol_vec) File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:170012) petsc4py.PETSc.Error: error code 75 [0] SNESSolve() line 4060 in /private/var/folders/92/1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/snes/interface/snes.c [0] SNESSolve_VINEWTONRSLS() line 424 in /private/var/folders/92/1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/snes/impls/vi/rs/virs.c [0] MatGetSubMatrix() line 7929 in /private/var/folders/92/1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/mat/interface/matrix.c [0] MatGetSubMatrix_Nest() line 451 in /private/var/folders/92/1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/mat/impls/nest/matnest.c [0] MatNestFindSubMat() line 424 in /private/var/folders/92/1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/mat/impls/nest/matnest.c [0] MatNestFindIS() line 363 in /private/var/folders/92/1kh0g4kn2z50fnwsmf8s269w0000gn/T/pip-gYEj0p-build/src/mat/impls/nest/matnest.c [0] Arguments are incompatible [0] Could not find index set
I must be doing something wrong here.
Justin
On Tue, Nov 29, 2016 at 1:24 AM, Justin Chang <jychang48@gmail.com> wrote: Lawrence,
I added the following line:
rs_solver.setDM(a._dm)
And it gives this error:
Traceback (most recent call last): File "sphere.py", line 128, in <module> rs_solver.setDM(a._dm) AttributeError: 'Form' object has no attribute '_dm'
Thanks, Justin
On Tue, Nov 29, 2016 at 1:12 AM, Lawrence Mitchell <lawrence.mitchell@imperial.ac.uk> wrote:
On 29 Nov 2016, at 02:02, Justin Chang <jychang48@gmail.com> wrote:
Why is this happening? It seems to me VINEWTONRSLS is not understanding that I have a two-field formulation.
For the built in solvers, we set up a dmshell on the snes that knows how to do field splitting. Because you set your snes up by hand, it isn't available. You'll need to call snes.setDM. The mixed finctionspace provides the correct dm via a ._dm property I think.
Lawrence
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi Justin, On 29/11/16 08:38, Lawrence Mitchell wrote:
On 29 Nov 2016, at 08:31, Justin Chang <jychang48@gmail.com <mailto:jychang48@gmail.com>> wrote:
Okay, switched over to the firedrake list
I wonder. Normally this appears if you didn't get the right splittings for a three or more field problem. I'm on my phone right now so can't debug. I'll have a look a bit later.
There are a few issues here. 1. The VIRS active set method creates the Jacobian by calling MatGetSubMatrix with arbitrary index sets. MatNest matrices don't support extracting these. I see you were setting a parameter to turn these off, but we changed the way you do this recently. Either set: parameters["default_matrix_type"] = "aij" or set it per-matrix with assemble(..., mat_type="aij") 2. With that out the way, we run into a bug in PETSc (which Barry has a branch fixing) that PCReset on a fieldsplit PC didn't clear everything out. 3. Cherry-picking that fix, we then run into a further issue in that the DM I provide you doesn't know anything about the currently active set, and therefore spits back ISes that define the unconstrained fields, so we can no longer split correctly. In short, it looks like maybe when I said this was a firedrake problem, it's not? To me it looks like there's currently no support in PETSc for using fieldsplit PCs with the VIRS if you're using a DM to define the splits, but I could be mistaken. There's special case code to handle interpolation and restriction, but I think one needs more special-casing for DMCreateFieldDecomposition and friends. Do you have a pure PETSc example that breaks? I guess we could support this stuff if I knew what to do. Lawrence
Lawrence, Thanks for your help. Unfortunately we do not have a pure PETSc example, although we possibly could attempt to modify Matt's DMPlex examples since all we need are CG1 finite elements. Our problems are relatively small so the direct solver is enough for our results, but if we do encounter bigger problems we will give this another go. Thanks again, Justin On Tue, Nov 29, 2016 at 5:28 AM, Lawrence Mitchell < lawrence.mitchell@imperial.ac.uk> wrote:
Hi Justin,
On 29/11/16 08:38, Lawrence Mitchell wrote:
On 29 Nov 2016, at 08:31, Justin Chang <jychang48@gmail.com <mailto:jychang48@gmail.com>> wrote:
Okay, switched over to the firedrake list
I wonder. Normally this appears if you didn't get the right splittings for a three or more field problem. I'm on my phone right now so can't debug. I'll have a look a bit later.
There are a few issues here.
1. The VIRS active set method creates the Jacobian by calling MatGetSubMatrix with arbitrary index sets. MatNest matrices don't support extracting these. I see you were setting a parameter to turn these off, but we changed the way you do this recently.
Either set:
parameters["default_matrix_type"] = "aij"
or set it per-matrix with
assemble(..., mat_type="aij")
2. With that out the way, we run into a bug in PETSc (which Barry has a branch fixing) that PCReset on a fieldsplit PC didn't clear everything out.
3. Cherry-picking that fix, we then run into a further issue in that the DM I provide you doesn't know anything about the currently active set, and therefore spits back ISes that define the unconstrained fields, so we can no longer split correctly.
In short, it looks like maybe when I said this was a firedrake problem, it's not?
To me it looks like there's currently no support in PETSc for using fieldsplit PCs with the VIRS if you're using a DM to define the splits, but I could be mistaken. There's special case code to handle interpolation and restriction, but I think one needs more special-casing for DMCreateFieldDecomposition and friends.
Do you have a pure PETSc example that breaks? I guess we could support this stuff if I knew what to do.
Lawrence
participants (2)
-
Justin Chang
-
Lawrence Mitchell