-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 26/04/15 13:53, Eike Mueller wrote:
Hi,
At higher order, I don’t see to be able to run the matrix-free solver in the original configuration any more: it works on one core (with a reduced problem size), but not on a full node. I did check on my laptop that I can still run in parallel, but the results of a 1 core run and a 4 core run are not exactly identical at higher order, so something is not quite right there. Hang on, thinking about this again, if I use BJACOBI+ILU as the preconditioner for the velocity mass inverse, isn’t the result naturally dependent on the number of processors, since by default a block corresponds to the dofs stored on one process? At lowest order I use the inverse of the diagonal lumped mass matrix, so there I don’t see this.
Yes, that's right, bjacobi+ilu first decouples the processes and then does ILU on the local block, so it will be different on different processes.
However, if I change BJACOBI+ILU -> JACOBI I still get a difference between the results on 1 and on 4 processors, so I must be missing something else.
In this situation I would recommend checking that the operator is indeed the same on 1 and 4 processors. Computing some of the leading eigenvalues is probably good enough, you can do this with SLEPc: pip install git+https://wence@bitbucket.org/slepc/slepc.git#egg=slepc pip install git+https://wence@bitbucket.org/slepc/slepc4py.git#egg=slepc4py A = assemble(form) petsc_mat = A.M.handle from slepc4py import SLEPc eps = SLEPc.EPS().create() eps.setOperators(petsc_mat) eps.setWhichEigenpairs(eps.Which.LARGEST_MAGNITUDE) eps.solve() nev = eps.getConverged() for i in range(nev): evalue = eps.getEigenvalue(i) print evalue Cheers, Lawrence -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVPf+JAAoJECOc1kQ8PEYv650H/3ZGkCy/6G/BivFhSMyKNaCz HS3P4KO/sOcNEJSGum1YSktWvWa5YkRWX9xACZWRisO3TWKgTpBiqhS4aGuOn2Xh QrO3Uhyoj5dx1S5o69qQiZppsMjdPViRnw4/O3YEAw7xUiKbbJqBmb30yP2fLGKe q2Ce/kwPG8DfVf0Xl6Z4muKHHquG38YwqDCSMc6s4QbIQwi8PQI+fAWcBwbZ2Wag vuB9TOIJUXmBMMpdlk8YPr903iu1Qd4koxbP8kN0kHsORdJJzxOsYfotEEk5Fdm4 de0UjSUSKFP1Kj45ZbZLbXFrJFIrFlSaWRFQmQEmpck2H8AMYDsF8TMviH4dE4w= =RMZi -----END PGP SIGNATURE-----