On 14/01/16 08:24, Justin Chang wrote:
Ah I figured.
Is it possible to one day add a feature where one could achieve BAIJ format if the fields all belong to same function space? I think there was a way to do this in DMPlex last I spoke with Matt
Probably. So what we do is this: Build a dofmap for each of the individual spaces (via dmplex). The mixed space is then just a bag of individual spaces and dofmaps. When assembling into the matrices we use MatCreateLocalSubMatrix. See the comment here https://github.com/OP2/PyOP2/blob/master/pyop2/petsc_base.py#L157 describing the field ordering (for simplicity I use the /same/ field ordering for the monolithic and the nested matrices). If one wants a different field ordering for monolithic matrices, these lgmaps need changing. But, that's not all, a Function on a mixed space is actually just a bag of Functions on the subspaces, so the data is stored separately. When we pass a Vec over to petsc, we need to perform a (process-local) vecscatter (https://github.com/OP2/PyOP2/blob/master/pyop2/petsc_base.py#L271). This, obviously, uses the same field ordering as the matrices. So that would need changing. Finally, changing the field ordering will change the allocation pattern of the monolithic matrix (in particular the number of nonzeros on diagonal and offdiagonal blocks in parallel), so that code would need to be updated as well. FWIW, if you build a matrix on a single VectorFunctionSpace, then you do get a baij matrix. So if all your fields are the same, you may have luck reformulating your problem in that manner. Cheers, Lawrence