Hi Lawrence,

as an additional test I just tried to re-run PETSc’s ex6 on 24 cores with Euclid. Here it does seem to work, the command I use is:

mpirun -n 24 ./ex6 -f -f helmholtz-sphere.dat  -ksp_type cg -ksp_convergence_test skip -ksp_max_it 2  -ksp_monitor -pc_hypre_type boomeramg -pc_type hypre -pc_hypre_boomeramg_smooth_type Euclid -log_summary -ksp_view

As a sanity check, is there a way of writing out the pressure matrix for the system that I currently solve with the fieldsplit preconditioner and run it through ex6 exactly like above?

Thanks,

Eike

--

Dr Eike Hermann Mueller
Lecturer in Scientific Computing

Department of Mathematical Sciences
University of Bath
Bath BA2 7AY, United Kingdom

+44 1225 38 6241
e.mueller@bath.ac.uk
http://people.bath.ac.uk/em459/





On 13 May 2015, at 12:34, Eike Mueller <E.Mueller@bath.ac.uk> wrote:

Hi Lawrence,

that's what I feared, an issue somewhere deep inside hypre/PETSc... I just reran sequentially on ARCHER with exactly the same setup as on my laptop, and then it does work. So maybe the problem is too large, I will try to reproduce the issue on my laptop.

Thanks a lot anyway,

Eike
________________________________________
From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Lawrence Mitchell <lawrence.mitchell@imperial.ac.uk>
Sent: 13 May 2015 11:21
To: firedrake@imperial.ac.uk
Subject: Re: [firedrake] EUCLID BoomerAMG smoother on ARCHER

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 13/05/15 11:00, Eike Mueller wrote:
Hi,


I just tried to run the PETSc fieldsplit solver with the EUCLID
smoother for the BoomerAMG preconditioner on ARCHER, and I get this
error message:


============= error stack trace ==================== [18] ERROR:
global index 112 not found in map

Numbering_dhGlobalToLocal  file= Numbering_dh.c  line= 175

[18] called from: Factor_dhSolveSetup  file= Factor_dh.c  line=
555 [18] called from: Euclid_dhSetup  file= Euclid_dh.c  line= 282
[18] called from: HYPRE_EuclidSetup  file= HYPRE_parcsr_Euclid.c
line= 278


any idea what might cause this?


I can run this without problems on my laptop.

This looks like it's happened deep inside hypre itself.  So I'm sort
of out of ideas.  Does this run, at the same scale, on your laptop
successfully?

Lawrence

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJVUyWlAAoJECOc1kQ8PEYvdz0H/11HfQy0p7LO1DpGyoXPGtEx
25rhNltEiK/Myv//IjSvgjiApgvCSTNLcgqJUtj86OMiSPmATadT+uVKfEk8wG/G
yv2bm11GRQxOX79EV4skfIdba6X2/Oiekb5ktmF0GL8iav3tYUVOHbRLYBmJExT1
YMihip/CFuvdzuoUCbFGGlcni9IJAArmADyJEdVmEUEV258eMVtPYiGFEdi8ajM9
H5X+f5SVRuW77lKg+PhSzgjaRRvJUfLeDE0CECHS0sLpWT5EZF+KjhBq5g6a7rIq
4iE4QE7TMPjGtL8tzqXEPJhqb9NMG+29BI+V+SYcT/aJBky2pOTC65rBm6kXT8g=
=wTeg
-----END PGP SIGNATURE-----

_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake

_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake