Hi Lawrence, I never heard back from the hypre developers, but I now use Block Jacobi + ILU instead of PILUT in the Euclid smoother for BoomerAMG, and then it works in parallel. As far as I can tell the number of iterations is the same (comparing PILUT and Block Jacobi+ILU on 1 processor). I had tried that change before, but due to a stupid bug that I created in my changes to the PETSc hypre interface that option did not get set properly (I called the wrong hypre method). I pushed a change to my PETSc branch (but I haven’t managed to create a pull-request to get those changes into the PETSc repository). I’ve now tested it on ARCHER as well. The bottom line is still that the geometric MG and BoomerAMG times are very similar both at lowest- and at higher-order. That’s an improvement compared to the numbers I showed in Edinburgh. Cheers, Eike
On 20 May 2015, at 17:21, Eike Mueller <E.Mueller@bath.ac.uk> wrote:
Hi Lawrence,
thanks, I had already sent them an email last week, and just tried to follow up on it again. I’m using 2 refinement levels, so there should be enough cells, besides I guess when I write the matrix out and then read it from petsc it will repartition anyway.
Thanks,
Eike
On 19 May 2015, at 15:31, Lawrence Mitchell <lawrence.mitchell@imperial.ac.uk> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 19/05/15 14:28, Eike Mueller wrote:
Hi Lawrence,
I just rebuilt PETSc with the debugging=1 option, and indeed It get a more verbose error message, see below. It crashes somewhere in HYPRE_SetupXXX.
Still, that only tells me that something is going wrong in the hypre setup AMG routine.
Yep, so I'm not sure what the problem is, I wonder if it's worthwhile reporting to the hypre team.
Cheers,
Eike
eikemueller@dhcppar3 $ mpirun -n 24 ./ex6 -f /Users/eikemueller/PostDocBath/EllipticSolvers/Firedrake_workspace/firedrake-helmholtzsolver/source/pressure_operator.dat
- -ksp_type cg -ksp_convergence_test skip -ksp_max_it 2 -ksp_monitor
-pc_hypre_type boomeramg -pc_type hypre -pc_hypre_boomeramg_smooth_type Euclid [21]PETSC ERROR: ------------------------------------------------------------------------
[21]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
probably memory access out of range [21]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger [21]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind [21]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors [21]PETSC ERROR: likely location of problem given in stack below [21]PETSC ERROR: --------------------- Stack Frames ------------------------------------ [21]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, [21]PETSC ERROR: INSTEAD the line number of the start of the function [21]PETSC ERROR: is given. [21]PETSC ERROR: [21] HYPRE_SetupXXX line
So this is calling HYPRE_BoomerAMGSetup, which is a hypre call. So I'm again not sure what's going wrong, but something inside hypre again I guess.
Lawrence -----BEGIN PGP SIGNATURE----- Version: GnuPG v1
iQEcBAEBAgAGBQJVWzsoAAoJECOc1kQ8PEYv/DQH/iiys8rClF52h1LnjD4gEKae XOARXGCA9w9K0BaKvOVqceWxGavOq3QZEHlfVhG9mfUcKV/BQB0cDtvk8rQiZ9d6 uWVQp8k7CJecGK9+FQh+m5Bt2nV9/ij1uSo87tpl1WesZSlQEt5GHXIzfc4RVJsF DTlZ0B7H53X4T7DTjcq5nAYRpht8fbb01QSBmlKTtTisyLcGhv/12yEEmiT3iPk+ qdKKZaWYOVQN+E/rcrF97I56ONXFoj/XRs0g8DlM1u4iaK8RNx34X9cOSfEAu93j afkPdaQ3tPMgyTG+TIbdylbjpIkxeyjkMLQjsWvsbIK/2/fkpBZqSiu1PRSp2cc= =Ibvj -----END PGP SIGNATURE-----
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake