Hi Justin,

On 29/11/16 08:38, Lawrence Mitchell wrote:
>
>
> On 29 Nov 2016, at 08:31, Justin Chang <jychang48@gmail.com
> <mailto:jychang48@gmail.com>> wrote:
>
>> Okay, switched over to the firedrake list
>
> I wonder. Normally this appears if you didn't get the right splittings
> for a three or more field problem.
> I'm on my phone right now so can't debug. I'll have a look a bit later.

There are a few issues here.

1. The VIRS active set method creates the Jacobian by calling
MatGetSubMatrix with arbitrary index sets. MatNest matrices don't
support extracting these.  I see you were setting a parameter to turn
these off, but we changed the way you do this recently.

Either set:

parameters["default_matrix_type"] = "aij"

or set it per-matrix with

assemble(..., mat_type="aij")

2. With that out the way, we run into a bug in PETSc (which Barry has
a branch fixing) that PCReset on a fieldsplit PC didn't clear
everything out.

3. Cherry-picking that fix, we then run into a further issue in that
the DM I provide you doesn't know anything about the currently active
set, and therefore spits back ISes that define the unconstrained
fields, so we can no longer split correctly.

In short, it looks like maybe when I said this was a firedrake
problem, it's not?

To me it looks like there's currently no support in PETSc for using
fieldsplit PCs with the VIRS if you're using a DM to define the
splits, but I could be mistaken.  There's special case code to handle
interpolation and restriction, but I think one needs more
special-casing for DMCreateFieldDecomposition and friends.

Do you have a pure PETSc example that breaks?  I guess we could
support this stuff if I knew what to do.

Lawrence