On 13/10/14 08:58, Eike Mueller wrote:
Hi Florian,
thanks, that's interesting, is this for the weak scaling in Fig. 3? For
the strong scaling (Fig. 2) the number of iterations and levels should
stay constant (sorry, I should have been a bit more clear in my previous
email). If yes, then I suspect that hypre uses a direct (or iterative)
coarse grid solver, which will not scale (algorithmically) since, for a
constant number of levels of ~20 the global problem size on the coarsest
level will grow.
These were for P3 strong scaling. For P1 weak scaling with 1k DOFs/core
we get:
Levels:
1: 4
3: 6
6: 7
12: 9
24: 10
48: 11
96: 12
192: 13
384: 13
768: 15
1536: 16
KSP iterations:
1: 7
3: 8
6: 9
12: 10
24: 11
48: 12
96: 13
192: 13
384: 15
768: 15
1536: 16
For the strong scaling, the bottleneck will be the parallel scalability
of the coarse grid solver.
The problem at the coarsest grid is tiny (4 rows on 1536 cores). I had
experimented with restricting the number of levels but found it hardly
made a difference and was an awfully hard to tune parameter.
Florian
Eike
On 12/10/14 21:31, Florian Rathgeber wrote:
On 12/10/14 11:29, Eike Mueller wrote:
Hi Florian,
in section 8.2.2 on the Poisson solve, do you know how much of the
non-perfect scaling of the solver can be attributed to an increase in
the number of CG iterations? In theory the number of iterations should
stay constant with a multigrid preconditioner, but I think this is only
true if you use geometric MG with a FMG cycle and solve to a tolerance
which depends on the grid resolution. I would suspect that the number of
iterations grows, so would it be worth mentioning that part of the
reason for the poorer scaling of the solver is algorithmic (and say how
much the number of iterations increases)? Since the parallel AMG is
algorithmically not identical to the sequential version, algorithmic and
parallel scalability can only be partly separated, but it might still be
worth mentioning it.
The number of iterations varies slightly, but not significantly:
1: 21
3: 21
6: 22
12: 22
24: 22
48: 22
96: 22
192: 22
384: 21
768: 22
1536: 22
The number of level in the AMG preconditioner is:
1: 20
3: 19
6: 19
12: 20
24: 20
48: 20
96: 20
192: 19
384: 20
768: 20
1536: 20
Thanks for that explanation, I'll add a sentence or 2.
Florian
Thanks,
Eike
On 11 Oct 2014, at 11:27, Florian Rathgeber
<florian.rathgeber@imperial.ac.uk
<mailto:florian.rathgeber@imperial.ac.uk>> wrote:
I'd still be interested in feedback on this!
On 04/10/14 09:12, Florian Rathgeber wrote:
Dear all,
I finished a first draft of the results section for the Firedrake paper.
Any feedback gratefully received!
If you don't have access to the repository you can get a PDF from
https://wwwhomes.doc.ic.ac.uk/~fr710/paper.pdf
Cheers,
Florian
_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake