Your FunctionSpace W has roughly 388*584*2 = 450k degrees of freedom. Your VectorFunctionSpace V has roughly 2*450k = 900k degrees of freedom. Then you construct a MixedFunctionSpace Q = W*W*V, which will have roughly 1.8 million degrees of freedom.

LU causes fill-in, so (unless there's something very special about your PDE/discretisation) each row of the matrix will have O(sqrt(N)) [I think] non-zero entries which must be stored. And 1.8 million * O(sqrt(1.8 million)) * (8 bytes/double + 4 bytes/index) is probably more than 8 GB.

Using MUMPS rather than the inbuilt LU might help? I.e., add the option "pc_factor_mat_solver_package": "mumps" to your solver_parameters dict.

On 3 June 2016 at 23:15, Tianjiao Sun <tianjiao.sun14@imperial.ac.uk> wrote:

Hi team,

I have the attached python code which is a mixed formulation on extruded
mesh. It is possible that there are other bugs in my formulation, but
I'm running into problems using the non-linear solver. My intent is to
use direct solver in the non-linear iteration, and my parameters are:

solver_parameters={'ksp_type': 'preonly', 'pc_type': 'lu',
'snes_atol':1e-10, 'ksp_atol':1e-10})

Then PETSc complains that I couldn't use direct solver for nested
matrix. Google guides me to a post by Christian and Lawrence

https://github.com/firedrakeproject/firedrake/issues/431

So I added nest=False in solve(), but then the system run out of memory:

[0] Out of memory. Allocated: 0, Used by process: 5061259264
[0] Memory requested 8589934188

I guess it's not quite likely to use 8 GB of RAM for this problem size
(388 x 584 x 2 layers). Lawrence did say that "it doesn't work with
vector function spaces" earlier in the post, I'm not sure if that is
related?

My mixed space is [W, W, V]
where W = DG(0) x CG(1), V is vector space of dim=2.

Many thanks,
-TJ