Re: [firedrake] Question about the efficiency of the Navier-Stokes solver in the Firedrake

13 Apr 2021

      Hi David,

Thanks for your reply! Do you have any idea of which inner solver can do a
much better job? I tried to replace lu with hypre, but it's still very slow
for large 3d problems, like solving on a 40x40x40 grid.

Best,
Zizhou

On Tue, Apr 13, 2021 at 6:23 PM Ham, David A <david.ham@imperial.ac.uk>
wrote:
...
Hi Zizhou,
That page uses lu for all the inner solves, because it’s just a
small-scale demo. You almost certainly want to use something more scalable
than that.
*From: *firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk>
on behalf of Zizhou Huang <zizhou@nyu.edu>
*Date: *Tuesday, 13 April 2021 at 03:14
*To: *Matthew Knepley <knepley@gmail.com>
*Cc: *firedrake <firedrake@imperial.ac.uk>
*Subject: *Re: [firedrake] Question about the efficiency of the
Navier-Stokes solver in the Firedrake
I'm using exactly the same solver as this website (
https://www.firedrakeproject.org/demos/navier_stokes.py.html). It seems
that they're using fgmres with the PCDPC preconditioner.
The logs are as follows.
N = 5
0 SNES Function norm 3.873711890948e+00
  1 SNES Function norm 7.100383245197e-02
  2 SNES Function norm 3.993316741711e-03
  3 SNES Function norm 3.990405864913e-03
  4 SNES Function norm 3.990405862838e-03
  0 SNES Function norm 3.873711890948e+00
    Residual norms for firedrake_1_ solve.
    0 KSP unpreconditioned resid norm 3.873711890948e+00 true resid norm
3.873711890948e+00 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 2.922269177394e-02 true resid norm
2.922269177394e-02 ||r(i)||/||b|| 7.543847502500e-03
    2 KSP unpreconditioned resid norm 1.837681883991e-03 true resid norm
1.837681883991e-03 ||r(i)||/||b|| 4.743981833769e-04
    3 KSP unpreconditioned resid norm 1.837657797525e-03 true resid norm
1.837657797525e-03 ||r(i)||/||b|| 4.743919654476e-04
    4 KSP unpreconditioned resid norm 1.837564856663e-03 true resid norm
1.837564856663e-03 ||r(i)||/||b|| 4.743679727336e-04
    5 KSP unpreconditioned resid norm 1.837330879480e-03 true resid norm
1.837330880004e-03 ||r(i)||/||b|| 4.743075715821e-04
    6 KSP unpreconditioned resid norm 5.396256497007e-04 true resid norm
5.396449573863e-04 ||r(i)||/||b|| 1.393095234179e-04
    7 KSP unpreconditioned resid norm 9.179035795321e-07 true resid norm
1.140070292524e-05 ||r(i)||/||b|| 2.943095213631e-06
  1 SNES Function norm 7.026786066650e-02
    Residual norms for firedrake_1_ solve.
    0 KSP unpreconditioned resid norm 7.026786066650e-02 true resid norm
7.026786066650e-02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 1.995090800849e-02 true resid norm
1.995090800849e-02 ||r(i)||/||b|| 2.839265037993e-01
    2 KSP unpreconditioned resid norm 1.115563390429e-03 true resid norm
1.115563390429e-03 ||r(i)||/||b|| 1.587586956323e-02
    3 KSP unpreconditioned resid norm 1.115554835302e-03 true resid norm
1.115554835302e-03 ||r(i)||/||b|| 1.587574781302e-02
    4 KSP unpreconditioned resid norm 1.115554764006e-03 true resid norm
1.115554764006e-03 ||r(i)||/||b|| 1.587574679839e-02
    5 KSP unpreconditioned resid norm 1.115548853894e-03 true resid norm
1.115548853906e-03 ||r(i)||/||b|| 1.587566269024e-02
    6 KSP unpreconditioned resid norm 7.632184921082e-05 true resid norm
7.632265261003e-05 ||r(i)||/||b|| 1.086167301610e-03
    7 KSP unpreconditioned resid norm 1.422983940334e-08 true resid norm
4.141122043271e-08 ||r(i)||/||b|| 5.893337300996e-07
  2 SNES Function norm 8.778069005998e-06
    Residual norms for firedrake_1_ solve.
    0 KSP unpreconditioned resid norm 8.778069005998e-06 true resid norm
8.778069005998e-06 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 3.525945734318e-06 true resid norm
3.525945734318e-06 ||r(i)||/||b|| 4.016766935768e-01
    2 KSP unpreconditioned resid norm 7.092855329067e-07 true resid norm
7.092855329067e-07 ||r(i)||/||b|| 8.080200012349e-02
    3 KSP unpreconditioned resid norm 7.092827495949e-07 true resid norm
7.092827495949e-07 ||r(i)||/||b|| 8.080168304786e-02
    4 KSP unpreconditioned resid norm 7.092826142571e-07 true resid norm
7.092826142574e-07 ||r(i)||/||b|| 8.080166763017e-02
    5 KSP unpreconditioned resid norm 8.490734432004e-08 true resid norm
8.490728815768e-08 ||r(i)||/||b|| 9.672661276604e-03
    6 KSP unpreconditioned resid norm 3.681976478361e-12 true resid norm
9.310211086004e-12 ||r(i)||/||b|| 1.060621769963e-06
  3 SNES Function norm 9.754530371801e-12
N = 10
0 SNES Function norm 3.857209122044e+00
  1 SNES Function norm 2.174942016837e-02
  2 SNES Function norm 7.117856370756e-04
  3 SNES Function norm 7.102760546744e-04
  4 SNES Function norm 7.102760536032e-04
  0 SNES Function norm 3.857209122044e+00
    Residual norms for firedrake_1_ solve.
    0 KSP unpreconditioned resid norm 3.857209122044e+00 true resid norm
3.857209122044e+00 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 1.386167872908e-01 true resid norm
1.386167872908e-01 ||r(i)||/||b|| 3.593706820265e-02
    2 KSP unpreconditioned resid norm 4.475586255809e-02 true resid norm
4.475586255809e-02 ||r(i)||/||b|| 1.160317243426e-02
    3 KSP unpreconditioned resid norm 1.722695448041e-02 true resid norm
1.722695448041e-02 ||r(i)||/||b|| 4.466170730012e-03
    4 KSP unpreconditioned resid norm 5.231768306662e-03 true resid norm
5.231768306662e-03 ||r(i)||/||b|| 1.356361073804e-03
    5 KSP unpreconditioned resid norm 2.041434254281e-03 true resid norm
2.041434254281e-03 ||r(i)||/||b|| 5.292516401597e-04
    6 KSP unpreconditioned resid norm 5.875484274903e-04 true resid norm
5.875484274903e-04 ||r(i)||/||b|| 1.523247531829e-04
    7 KSP unpreconditioned resid norm 2.354394338936e-04 true resid norm
2.354394338936e-04 ||r(i)||/||b|| 6.103880459787e-05
    8 KSP unpreconditioned resid norm 1.414423189101e-04 true resid norm
1.414423189101e-04 ||r(i)||/||b|| 3.666960085254e-05
    9 KSP unpreconditioned resid norm 1.272526161323e-04 true resid norm
1.272526161323e-04 ||r(i)||/||b|| 3.299085222139e-05
   10 KSP unpreconditioned resid norm 1.256180185176e-04 true resid norm
1.256180185176e-04 ||r(i)||/||b|| 3.256707493500e-05
   11 KSP unpreconditioned resid norm 1.253360959842e-04 true resid norm
1.253360959842e-04 ||r(i)||/||b|| 3.249398516348e-05
   12 KSP unpreconditioned resid norm 1.252954886770e-04 true resid norm
1.252954886770e-04 ||r(i)||/||b|| 3.248345752397e-05
   13 KSP unpreconditioned resid norm 1.252778828690e-04 true resid norm
1.252778828690e-04 ||r(i)||/||b|| 3.247889313364e-05
   14 KSP unpreconditioned resid norm 1.252729271101e-04 true resid norm
1.252729271101e-04 ||r(i)||/||b|| 3.247760832934e-05
   15 KSP unpreconditioned resid norm 1.252709541672e-04 true resid norm
1.252709541672e-04 ||r(i)||/||b|| 3.247709683442e-05
   16 KSP unpreconditioned resid norm 1.252698774325e-04 true resid norm
1.252698774325e-04 ||r(i)||/||b|| 3.247681768576e-05
   17 KSP unpreconditioned resid norm 1.252693026661e-04 true resid norm
1.252693026661e-04 ||r(i)||/||b|| 3.247666867482e-05
   18 KSP unpreconditioned resid norm 1.252691813938e-04 true resid norm
1.252691813938e-04 ||r(i)||/||b|| 3.247663723438e-05
   19 KSP unpreconditioned resid norm 1.252690631661e-04 true resid norm
1.252690631661e-04 ||r(i)||/||b|| 3.247660658329e-05
   20 KSP unpreconditioned resid norm 1.252682040589e-04 true resid norm
1.252682040589e-04 ||r(i)||/||b|| 3.247638385562e-05
   21 KSP unpreconditioned resid norm 1.252666235238e-04 true resid norm
1.252666235237e-04 ||r(i)||/||b|| 3.247597409427e-05
   22 KSP unpreconditioned resid norm 1.252663551040e-04 true resid norm
1.252663551038e-04 ||r(i)||/||b|| 3.247590450512e-05
   23 KSP unpreconditioned resid norm 1.252657099958e-04 true resid norm
1.252657099961e-04 ||r(i)||/||b|| 3.247573725784e-05
   24 KSP unpreconditioned resid norm 1.252222519823e-04 true resid norm
1.252222519982e-04 ||r(i)||/||b|| 3.246447056306e-05
   25 KSP unpreconditioned resid norm 1.248107616721e-04 true resid norm
1.248107618126e-04 ||r(i)||/||b|| 3.235778975510e-05
   26 KSP unpreconditioned resid norm 1.197171374954e-04 true resid norm
1.197171390035e-04 ||r(i)||/||b|| 3.103724356537e-05
   27 KSP unpreconditioned resid norm 9.324077689908e-05 true resid norm
9.324078248863e-05 ||r(i)||/||b|| 2.417312091163e-05
   28 KSP unpreconditioned resid norm 3.902477473206e-05 true resid norm
3.902477892599e-05 ||r(i)||/||b|| 1.011736146297e-05
   29 KSP unpreconditioned resid norm 1.307801475003e-05 true resid norm
1.307801741838e-05 ||r(i)||/||b|| 3.390538859724e-06
  1 SNES Function norm 2.171186815034e-02
    Residual norms for firedrake_1_ solve.
    0 KSP unpreconditioned resid norm 2.171186815034e-02 true resid norm
2.171186815034e-02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 5.755446049476e-03 true resid norm
5.755446049476e-03 ||r(i)||/||b|| 2.650829495474e-01
    2 KSP unpreconditioned resid norm 1.819729526723e-03 true resid norm
1.819729526723e-03 ||r(i)||/||b|| 8.381266476578e-02
    3 KSP unpreconditioned resid norm 5.868270016474e-04 true resid norm
5.868270016474e-04 ||r(i)||/||b|| 2.702793686771e-02
    4 KSP unpreconditioned resid norm 1.891824694427e-04 true resid norm
1.891824694427e-04 ||r(i)||/||b|| 8.713320665581e-03
    5 KSP unpreconditioned resid norm 7.091162609666e-05 true resid norm
7.091162609666e-05 ||r(i)||/||b|| 3.266030615406e-03
    6 KSP unpreconditioned resid norm 5.637701208830e-05 true resid norm
5.637701208830e-05 ||r(i)||/||b|| 2.596598860030e-03
    7 KSP unpreconditioned resid norm 5.383999628854e-05 true resid norm
5.383999628854e-05 ||r(i)||/||b|| 2.479749596660e-03
    8 KSP unpreconditioned resid norm 5.369249277053e-05 true resid norm
5.369249277053e-05 ||r(i)||/||b|| 2.472955915113e-03
    9 KSP unpreconditioned resid norm 5.368343313838e-05 true resid norm
5.368343313838e-05 ||r(i)||/||b|| 2.472538648755e-03
   10 KSP unpreconditioned resid norm 5.368324507663e-05 true resid norm
5.368324507663e-05 ||r(i)||/||b|| 2.472529987052e-03
   11 KSP unpreconditioned resid norm 5.367688316238e-05 true resid norm
5.367688316238e-05 ||r(i)||/||b|| 2.472236971536e-03
   12 KSP unpreconditioned resid norm 5.364660764375e-05 true resid norm
5.364660764375e-05 ||r(i)||/||b|| 2.470842548982e-03
   13 KSP unpreconditioned resid norm 5.363778445858e-05 true resid norm
5.363778445858e-05 ||r(i)||/||b|| 2.470436172843e-03
   14 KSP unpreconditioned resid norm 5.363146760267e-05 true resid norm
5.363146760267e-05 ||r(i)||/||b|| 2.470145232613e-03
   15 KSP unpreconditioned resid norm 5.363036199206e-05 true resid norm
5.363036199206e-05 ||r(i)||/||b|| 2.470094310665e-03
   16 KSP unpreconditioned resid norm 5.362586041467e-05 true resid norm
5.362586041467e-05 ||r(i)||/||b|| 2.469886978097e-03
   17 KSP unpreconditioned resid norm 5.362298140611e-05 true resid norm
5.362298140610e-05 ||r(i)||/||b|| 2.469754377413e-03
   18 KSP unpreconditioned resid norm 5.362041508854e-05 true resid norm
5.362041508851e-05 ||r(i)||/||b|| 2.469636178574e-03
   19 KSP unpreconditioned resid norm 5.362023682045e-05 true resid norm
5.362023682040e-05 ||r(i)||/||b|| 2.469627967945e-03
   20 KSP unpreconditioned resid norm 5.362021431482e-05 true resid norm
5.362021431475e-05 ||r(i)||/||b|| 2.469626931385e-03
   21 KSP unpreconditioned resid norm 5.362021307632e-05 true resid norm
5.362021307624e-05 ||r(i)||/||b|| 2.469626874342e-03
   22 KSP unpreconditioned resid norm 5.362020945699e-05 true resid norm
5.362020945697e-05 ||r(i)||/||b|| 2.469626707646e-03
   23 KSP unpreconditioned resid norm 5.360919215998e-05 true resid norm
5.360919217038e-05 ||r(i)||/||b|| 2.469119276111e-03
   24 KSP unpreconditioned resid norm 5.352778280091e-05 true resid norm
5.352778288656e-05 ||r(i)||/||b|| 2.465369746902e-03
   25 KSP unpreconditioned resid norm 5.256642319528e-05 true resid norm
5.256642407158e-05 ||r(i)||/||b|| 2.421091713877e-03
   26 KSP unpreconditioned resid norm 4.729160583219e-05 true resid norm
4.729161069155e-05 ||r(i)||/||b|| 2.178145628192e-03
   27 KSP unpreconditioned resid norm 2.668806077499e-05 true resid norm
2.668806954502e-05 ||r(i)||/||b|| 1.229192686701e-03
   28 KSP unpreconditioned resid norm 1.020558332703e-05 true resid norm
1.020558761616e-05 ||r(i)||/||b|| 4.700464992463e-04
   29 KSP unpreconditioned resid norm 3.451738103984e-06 true resid norm
3.451741888987e-06 ||r(i)||/||b|| 1.589794975304e-04
   30 KSP unpreconditioned resid norm 1.120801314646e-06 true resid norm
1.120801314646e-06 ||r(i)||/||b|| 5.162159731651e-05
   31 KSP unpreconditioned resid norm 6.657758394387e-07 true resid norm
6.657758394387e-07 ||r(i)||/||b|| 3.066414344582e-05
   32 KSP unpreconditioned resid norm 1.791100894652e-07 true resid norm
1.791100894652e-07 ||r(i)||/||b|| 8.249409411708e-06
  2 SNES Function norm 1.208714436027e-06
    Residual norms for firedrake_1_ solve.
    0 KSP unpreconditioned resid norm 1.208714436027e-06 true resid norm
1.208714436027e-06 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 7.009704990544e-07 true resid norm
7.009704990544e-07 ||r(i)||/||b|| 5.799306090514e-01
    2 KSP unpreconditioned resid norm 2.928942580645e-07 true resid norm
2.928942580645e-07 ||r(i)||/||b|| 2.423188218279e-01
    3 KSP unpreconditioned resid norm 1.900700846501e-07 true resid norm
1.900700846501e-07 ||r(i)||/||b|| 1.572497845520e-01
    4 KSP unpreconditioned resid norm 1.694415211872e-07 true resid norm
1.694415211872e-07 ||r(i)||/||b|| 1.401832526665e-01
    5 KSP unpreconditioned resid norm 1.670108100598e-07 true resid norm
1.670108100598e-07 ||r(i)||/||b|| 1.381722639210e-01
    6 KSP unpreconditioned resid norm 1.668707304029e-07 true resid norm
1.668707304029e-07 ||r(i)||/||b|| 1.380563724807e-01
    7 KSP unpreconditioned resid norm 1.668026114154e-07 true resid norm
1.668026114154e-07 ||r(i)||/||b|| 1.380000159208e-01
    8 KSP unpreconditioned resid norm 1.668025617944e-07 true resid norm
1.668025617944e-07 ||r(i)||/||b|| 1.379999748681e-01
    9 KSP unpreconditioned resid norm 1.667959318665e-07 true resid norm
1.667959318665e-07 ||r(i)||/||b|| 1.379944897612e-01
   10 KSP unpreconditioned resid norm 1.667495881623e-07 true resid norm
1.667495881623e-07 ||r(i)||/||b|| 1.379561484435e-01
   11 KSP unpreconditioned resid norm 1.667472366297e-07 true resid norm
1.667472366297e-07 ||r(i)||/||b|| 1.379542029611e-01
   12 KSP unpreconditioned resid norm 1.667421748942e-07 true resid norm
1.667421748942e-07 ||r(i)||/||b|| 1.379500152595e-01
   13 KSP unpreconditioned resid norm 1.667405719095e-07 true resid norm
1.667405719095e-07 ||r(i)||/||b|| 1.379486890697e-01
   14 KSP unpreconditioned resid norm 1.667313112753e-07 true resid norm
1.667313112754e-07 ||r(i)||/||b|| 1.379410275130e-01
   15 KSP unpreconditioned resid norm 1.667263791233e-07 true resid norm
1.667263791233e-07 ||r(i)||/||b|| 1.379369470190e-01
   16 KSP unpreconditioned resid norm 1.666960300587e-07 true resid norm
1.666960300590e-07 ||r(i)||/||b|| 1.379118384711e-01
   17 KSP unpreconditioned resid norm 1.666924323014e-07 true resid norm
1.666924323014e-07 ||r(i)||/||b|| 1.379088619553e-01
   18 KSP unpreconditioned resid norm 1.666835606480e-07 true resid norm
1.666835606492e-07 ||r(i)||/||b|| 1.379015222132e-01
   19 KSP unpreconditioned resid norm 1.666784002237e-07 true resid norm
1.666784002274e-07 ||r(i)||/||b|| 1.378972528659e-01
   20 KSP unpreconditioned resid norm 1.665390347333e-07 true resid norm
1.665390347759e-07 ||r(i)||/||b|| 1.377819523057e-01
   21 KSP unpreconditioned resid norm 1.662598119876e-07 true resid norm
1.662598121924e-07 ||r(i)||/||b|| 1.375509444058e-01
   22 KSP unpreconditioned resid norm 1.625064404254e-07 true resid norm
1.625064423607e-07 ||r(i)||/||b|| 1.344456866875e-01
   23 KSP unpreconditioned resid norm 1.391707373331e-07 true resid norm
1.391707472513e-07 ||r(i)||/||b|| 1.151394763752e-01
   24 KSP unpreconditioned resid norm 7.505751761619e-08 true resid norm
7.505754230672e-08 ||r(i)||/||b|| 6.209700163211e-02
   25 KSP unpreconditioned resid norm 2.799964315999e-08 true resid norm
2.799965549375e-08 ||r(i)||/||b|| 2.316482260755e-02
   26 KSP unpreconditioned resid norm 9.970551878855e-09 true resid norm
9.970561618435e-09 ||r(i)||/||b|| 8.248897606624e-03
   27 KSP unpreconditioned resid norm 3.252551635867e-09 true resid norm
3.252558544160e-09 ||r(i)||/||b|| 2.690923883437e-03
   28 KSP unpreconditioned resid norm 1.136728449942e-09 true resid norm
1.136739747171e-09 ||r(i)||/||b|| 9.404535209387e-04
   29 KSP unpreconditioned resid norm 3.672093573536e-10 true resid norm
3.672155322102e-10 ||r(i)||/||b|| 3.038066902032e-04
   30 KSP unpreconditioned resid norm 1.234864719987e-10 true resid norm
1.234864719987e-10 ||r(i)||/||b|| 1.021634790800e-04
   31 KSP unpreconditioned resid norm 6.510725158420e-11 true resid norm
6.510725158421e-11 ||r(i)||/||b|| 5.386487464998e-05
   32 KSP unpreconditioned resid norm 1.805042777062e-11 true resid norm
1.805042777059e-11 ||r(i)||/||b|| 1.493357507164e-05
   33 KSP unpreconditioned resid norm 7.492522181774e-12 true resid norm
7.492522181758e-12 ||r(i)||/||b|| 6.198752954739e-06
  3 SNES Function norm 7.512220214011e-12
N=20
0 SNES Function norm 3.857209122044e+00
  1 SNES Function norm 2.174942016837e-02
  2 SNES Function norm 7.117856370756e-04
  3 SNES Function norm 7.102760546744e-04
  4 SNES Function norm 7.102760536032e-04
  0 SNES Function norm 3.857209122044e+00
    Residual norms for firedrake_1_ solve.
    0 KSP unpreconditioned resid norm 3.857209122044e+00 true resid norm
3.857209122044e+00 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 1.386167872908e-01 true resid norm
1.386167872908e-01 ||r(i)||/||b|| 3.593706820265e-02
    2 KSP unpreconditioned resid norm 4.475586255809e-02 true resid norm
4.475586255809e-02 ||r(i)||/||b|| 1.160317243426e-02
    3 KSP unpreconditioned resid norm 1.722695448041e-02 true resid norm
1.722695448041e-02 ||r(i)||/||b|| 4.466170730012e-03
    4 KSP unpreconditioned resid norm 5.231768306662e-03 true resid norm
5.231768306662e-03 ||r(i)||/||b|| 1.356361073804e-03
    5 KSP unpreconditioned resid norm 2.041434254281e-03 true resid norm
2.041434254281e-03 ||r(i)||/||b|| 5.292516401597e-04
    6 KSP unpreconditioned resid norm 5.875484274903e-04 true resid norm
5.875484274903e-04 ||r(i)||/||b|| 1.523247531829e-04
    7 KSP unpreconditioned resid norm 2.354394338936e-04 true resid norm
2.354394338936e-04 ||r(i)||/||b|| 6.103880459787e-05
    8 KSP unpreconditioned resid norm 1.414423189101e-04 true resid norm
1.414423189101e-04 ||r(i)||/||b|| 3.666960085254e-05
    9 KSP unpreconditioned resid norm 1.272526161323e-04 true resid norm
1.272526161323e-04 ||r(i)||/||b|| 3.299085222139e-05
   10 KSP unpreconditioned resid norm 1.256180185176e-04 true resid norm
1.256180185176e-04 ||r(i)||/||b|| 3.256707493500e-05
   11 KSP unpreconditioned resid norm 1.253360959842e-04 true resid norm
1.253360959842e-04 ||r(i)||/||b|| 3.249398516348e-05
   12 KSP unpreconditioned resid norm 1.252954886770e-04 true resid norm
1.252954886770e-04 ||r(i)||/||b|| 3.248345752397e-05
   13 KSP unpreconditioned resid norm 1.252778828690e-04 true resid norm
1.252778828690e-04 ||r(i)||/||b|| 3.247889313364e-05
   14 KSP unpreconditioned resid norm 1.252729271101e-04 true resid norm
1.252729271101e-04 ||r(i)||/||b|| 3.247760832934e-05
   15 KSP unpreconditioned resid norm 1.252709541672e-04 true resid norm
1.252709541672e-04 ||r(i)||/||b|| 3.247709683442e-05
   16 KSP unpreconditioned resid norm 1.252698774325e-04 true resid norm
1.252698774325e-04 ||r(i)||/||b|| 3.247681768576e-05
   17 KSP unpreconditioned resid norm 1.252693026661e-04 true resid norm
1.252693026661e-04 ||r(i)||/||b|| 3.247666867482e-05
   18 KSP unpreconditioned resid norm 1.252691813938e-04 true resid norm
1.252691813938e-04 ||r(i)||/||b|| 3.247663723438e-05
   19 KSP unpreconditioned resid norm 1.252690631661e-04 true resid norm
1.252690631661e-04 ||r(i)||/||b|| 3.247660658329e-05
   20 KSP unpreconditioned resid norm 1.252682040589e-04 true resid norm
1.252682040589e-04 ||r(i)||/||b|| 3.247638385562e-05
   21 KSP unpreconditioned resid norm 1.252666235238e-04 true resid norm
1.252666235237e-04 ||r(i)||/||b|| 3.247597409427e-05
   22 KSP unpreconditioned resid norm 1.252663551040e-04 true resid norm
1.252663551038e-04 ||r(i)||/||b|| 3.247590450512e-05
   23 KSP unpreconditioned resid norm 1.252657099958e-04 true resid norm
1.252657099961e-04 ||r(i)||/||b|| 3.247573725784e-05
   24 KSP unpreconditioned resid norm 1.252222519823e-04 true resid norm
1.252222519982e-04 ||r(i)||/||b|| 3.246447056306e-05
   25 KSP unpreconditioned resid norm 1.248107616721e-04 true resid norm
1.248107618126e-04 ||r(i)||/||b|| 3.235778975510e-05
   26 KSP unpreconditioned resid norm 1.197171374954e-04 true resid norm
1.197171390035e-04 ||r(i)||/||b|| 3.103724356537e-05
   27 KSP unpreconditioned resid norm 9.324077689908e-05 true resid norm
9.324078248863e-05 ||r(i)||/||b|| 2.417312091163e-05
   28 KSP unpreconditioned resid norm 3.902477473206e-05 true resid norm
3.902477892599e-05 ||r(i)||/||b|| 1.011736146297e-05
   29 KSP unpreconditioned resid norm 1.307801475003e-05 true resid norm
1.307801741838e-05 ||r(i)||/||b|| 3.390538859724e-06
  1 SNES Function norm 2.171186815034e-02
    Residual norms for firedrake_1_ solve.
    0 KSP unpreconditioned resid norm 2.171186815034e-02 true resid norm
2.171186815034e-02 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 5.755446049476e-03 true resid norm
5.755446049476e-03 ||r(i)||/||b|| 2.650829495474e-01
    2 KSP unpreconditioned resid norm 1.819729526723e-03 true resid norm
1.819729526723e-03 ||r(i)||/||b|| 8.381266476578e-02
    3 KSP unpreconditioned resid norm 5.868270016474e-04 true resid norm
5.868270016474e-04 ||r(i)||/||b|| 2.702793686771e-02
    4 KSP unpreconditioned resid norm 1.891824694427e-04 true resid norm
1.891824694427e-04 ||r(i)||/||b|| 8.713320665581e-03
    5 KSP unpreconditioned resid norm 7.091162609666e-05 true resid norm
7.091162609666e-05 ||r(i)||/||b|| 3.266030615406e-03
    6 KSP unpreconditioned resid norm 5.637701208830e-05 true resid norm
5.637701208830e-05 ||r(i)||/||b|| 2.596598860030e-03
    7 KSP unpreconditioned resid norm 5.383999628854e-05 true resid norm
5.383999628854e-05 ||r(i)||/||b|| 2.479749596660e-03
    8 KSP unpreconditioned resid norm 5.369249277053e-05 true resid norm
5.369249277053e-05 ||r(i)||/||b|| 2.472955915113e-03
    9 KSP unpreconditioned resid norm 5.368343313838e-05 true resid norm
5.368343313838e-05 ||r(i)||/||b|| 2.472538648755e-03
   10 KSP unpreconditioned resid norm 5.368324507663e-05 true resid norm
5.368324507663e-05 ||r(i)||/||b|| 2.472529987052e-03
   11 KSP unpreconditioned resid norm 5.367688316238e-05 true resid norm
5.367688316238e-05 ||r(i)||/||b|| 2.472236971536e-03
   12 KSP unpreconditioned resid norm 5.364660764375e-05 true resid norm
5.364660764375e-05 ||r(i)||/||b|| 2.470842548982e-03
   13 KSP unpreconditioned resid norm 5.363778445858e-05 true resid norm
5.363778445858e-05 ||r(i)||/||b|| 2.470436172843e-03
   14 KSP unpreconditioned resid norm 5.363146760267e-05 true resid norm
5.363146760267e-05 ||r(i)||/||b|| 2.470145232613e-03
   15 KSP unpreconditioned resid norm 5.363036199206e-05 true resid norm
5.363036199206e-05 ||r(i)||/||b|| 2.470094310665e-03
   16 KSP unpreconditioned resid norm 5.362586041467e-05 true resid norm
5.362586041467e-05 ||r(i)||/||b|| 2.469886978097e-03
   17 KSP unpreconditioned resid norm 5.362298140611e-05 true resid norm
5.362298140610e-05 ||r(i)||/||b|| 2.469754377413e-03
   18 KSP unpreconditioned resid norm 5.362041508854e-05 true resid norm
5.362041508851e-05 ||r(i)||/||b|| 2.469636178574e-03
   19 KSP unpreconditioned resid norm 5.362023682045e-05 true resid norm
5.362023682040e-05 ||r(i)||/||b|| 2.469627967945e-03
   20 KSP unpreconditioned resid norm 5.362021431482e-05 true resid norm
5.362021431475e-05 ||r(i)||/||b|| 2.469626931385e-03
   21 KSP unpreconditioned resid norm 5.362021307632e-05 true resid norm
5.362021307624e-05 ||r(i)||/||b|| 2.469626874342e-03
   22 KSP unpreconditioned resid norm 5.362020945699e-05 true resid norm
5.362020945697e-05 ||r(i)||/||b|| 2.469626707646e-03
   23 KSP unpreconditioned resid norm 5.360919215998e-05 true resid norm
5.360919217038e-05 ||r(i)||/||b|| 2.469119276111e-03
   24 KSP unpreconditioned resid norm 5.352778280091e-05 true resid norm
5.352778288656e-05 ||r(i)||/||b|| 2.465369746902e-03
   25 KSP unpreconditioned resid norm 5.256642319528e-05 true resid norm
5.256642407158e-05 ||r(i)||/||b|| 2.421091713877e-03
   26 KSP unpreconditioned resid norm 4.729160583219e-05 true resid norm
4.729161069155e-05 ||r(i)||/||b|| 2.178145628192e-03
   27 KSP unpreconditioned resid norm 2.668806077499e-05 true resid norm
2.668806954502e-05 ||r(i)||/||b|| 1.229192686701e-03
   28 KSP unpreconditioned resid norm 1.020558332703e-05 true resid norm
1.020558761616e-05 ||r(i)||/||b|| 4.700464992463e-04
   29 KSP unpreconditioned resid norm 3.451738103984e-06 true resid norm
3.451741888987e-06 ||r(i)||/||b|| 1.589794975304e-04
   30 KSP unpreconditioned resid norm 1.120801314646e-06 true resid norm
1.120801314646e-06 ||r(i)||/||b|| 5.162159731651e-05
   31 KSP unpreconditioned resid norm 6.657758394387e-07 true resid norm
6.657758394387e-07 ||r(i)||/||b|| 3.066414344582e-05
   32 KSP unpreconditioned resid norm 1.791100894652e-07 true resid norm
1.791100894652e-07 ||r(i)||/||b|| 8.249409411708e-06
  2 SNES Function norm 1.208714436027e-06
    Residual norms for firedrake_1_ solve.
    0 KSP unpreconditioned resid norm 1.208714436027e-06 true resid norm
1.208714436027e-06 ||r(i)||/||b|| 1.000000000000e+00
    1 KSP unpreconditioned resid norm 7.009704990544e-07 true resid norm
7.009704990544e-07 ||r(i)||/||b|| 5.799306090514e-01
    2 KSP unpreconditioned resid norm 2.928942580645e-07 true resid norm
2.928942580645e-07 ||r(i)||/||b|| 2.423188218279e-01
    3 KSP unpreconditioned resid norm 1.900700846501e-07 true resid norm
1.900700846501e-07 ||r(i)||/||b|| 1.572497845520e-01
    4 KSP unpreconditioned resid norm 1.694415211872e-07 true resid norm
1.694415211872e-07 ||r(i)||/||b|| 1.401832526665e-01
    5 KSP unpreconditioned resid norm 1.670108100598e-07 true resid norm
1.670108100598e-07 ||r(i)||/||b|| 1.381722639210e-01
    6 KSP unpreconditioned resid norm 1.668707304029e-07 true resid norm
1.668707304029e-07 ||r(i)||/||b|| 1.380563724807e-01
    7 KSP unpreconditioned resid norm 1.668026114154e-07 true resid norm
1.668026114154e-07 ||r(i)||/||b|| 1.380000159208e-01
    8 KSP unpreconditioned resid norm 1.668025617944e-07 true resid norm
1.668025617944e-07 ||r(i)||/||b|| 1.379999748681e-01
    9 KSP unpreconditioned resid norm 1.667959318665e-07 true resid norm
1.667959318665e-07 ||r(i)||/||b|| 1.379944897612e-01
   10 KSP unpreconditioned resid norm 1.667495881623e-07 true resid norm
1.667495881623e-07 ||r(i)||/||b|| 1.379561484435e-01
   11 KSP unpreconditioned resid norm 1.667472366297e-07 true resid norm
1.667472366297e-07 ||r(i)||/||b|| 1.379542029611e-01
   12 KSP unpreconditioned resid norm 1.667421748942e-07 true resid norm
1.667421748942e-07 ||r(i)||/||b|| 1.379500152595e-01
   13 KSP unpreconditioned resid norm 1.667405719095e-07 true resid norm
1.667405719095e-07 ||r(i)||/||b|| 1.379486890697e-01
   14 KSP unpreconditioned resid norm 1.667313112753e-07 true resid norm
1.667313112754e-07 ||r(i)||/||b|| 1.379410275130e-01
   15 KSP unpreconditioned resid norm 1.667263791233e-07 true resid norm
1.667263791233e-07 ||r(i)||/||b|| 1.379369470190e-01
   16 KSP unpreconditioned resid norm 1.666960300587e-07 true resid norm
1.666960300590e-07 ||r(i)||/||b|| 1.379118384711e-01
   17 KSP unpreconditioned resid norm 1.666924323014e-07 true resid norm
1.666924323014e-07 ||r(i)||/||b|| 1.379088619553e-01
   18 KSP unpreconditioned resid norm 1.666835606480e-07 true resid norm
1.666835606492e-07 ||r(i)||/||b|| 1.379015222132e-01
   19 KSP unpreconditioned resid norm 1.666784002237e-07 true resid norm
1.666784002274e-07 ||r(i)||/||b|| 1.378972528659e-01
   20 KSP unpreconditioned resid norm 1.665390347333e-07 true resid norm
1.665390347759e-07 ||r(i)||/||b|| 1.377819523057e-01
   21 KSP unpreconditioned resid norm 1.662598119876e-07 true resid norm
1.662598121924e-07 ||r(i)||/||b|| 1.375509444058e-01
   22 KSP unpreconditioned resid norm 1.625064404254e-07 true resid norm
1.625064423607e-07 ||r(i)||/||b|| 1.344456866875e-01
   23 KSP unpreconditioned resid norm 1.391707373331e-07 true resid norm
1.391707472513e-07 ||r(i)||/||b|| 1.151394763752e-01
   24 KSP unpreconditioned resid norm 7.505751761619e-08 true resid norm
7.505754230672e-08 ||r(i)||/||b|| 6.209700163211e-02
   25 KSP unpreconditioned resid norm 2.799964315999e-08 true resid norm
2.799965549375e-08 ||r(i)||/||b|| 2.316482260755e-02
   26 KSP unpreconditioned resid norm 9.970551878855e-09 true resid norm
9.970561618435e-09 ||r(i)||/||b|| 8.248897606624e-03
   27 KSP unpreconditioned resid norm 3.252551635867e-09 true resid norm
3.252558544160e-09 ||r(i)||/||b|| 2.690923883437e-03
   28 KSP unpreconditioned resid norm 1.136728449942e-09 true resid norm
1.136739747171e-09 ||r(i)||/||b|| 9.404535209387e-04
   29 KSP unpreconditioned resid norm 3.672093573536e-10 true resid norm
3.672155322102e-10 ||r(i)||/||b|| 3.038066902032e-04
   30 KSP unpreconditioned resid norm 1.234864719987e-10 true resid norm
1.234864719987e-10 ||r(i)||/||b|| 1.021634790800e-04
   31 KSP unpreconditioned resid norm 6.510725158420e-11 true resid norm
6.510725158421e-11 ||r(i)||/||b|| 5.386487464998e-05
   32 KSP unpreconditioned resid norm 1.805042777062e-11 true resid norm
1.805042777059e-11 ||r(i)||/||b|| 1.493357507164e-05
   33 KSP unpreconditioned resid norm 7.492522181774e-12 true resid norm
7.492522181758e-12 ||r(i)||/||b|| 6.198752954739e-06
  3 SNES Function norm 7.512220214011e-12
Best,
Zizhou
On Tue, Apr 13, 2021 at 10:04 AM Matthew Knepley <knepley@gmail.com>
wrote:
On Mon, Apr 12, 2021 at 9:45 PM Zizhou Huang <zizhou@nyu.edu> wrote:
Hi Matthew,
Thanks for your advice!
I did some basic tests with the Navier-Stokes solver inside the Firedrake.
For the lid-driven cavity problem on a NxNxN grid, it takes 23sec for N=5,
594sec for N=10, and 7424sec for N=20. For N=40 it almost takes forever,
but it's really not a dense mesh in practice. I had the same experience
with my own implementation of fem, using Pardiso to solve the linear system
directly, and I've been finding efficient linear solvers to get rid of this
problem, so I'm asking if the Firedrake team has any better options for the
linear solver.
What solver did you use and how many iterates did it take? Also, posting
the logs makes things easier.
Thanks
Matt
Best,
Zizhou
On Mon, Apr 12, 2021 at 11:21 PM Matthew Knepley <knepley@gmail.com>
wrote:
On Mon, Apr 12, 2021 at 11:00 AM Zizhou Huang <zizhou@nyu.edu> wrote:
To one whom it may concern,
I'm a PhD student at NYU. I'm working on a project on solving the
Navier-Stokes equation with FEM, so I'm really interested in the amazing
demo of solving the Navier-Stokes equation on the website.  (
https://www.firedrakeproject.org/demos/navier_stokes.py.html)
I modified the code to solve the 3d lid-driven cavity problem with a
uniform grid of size 40x40x40, and found that it takes extremely long time
and large memory. I had the same issue with my code of similar sizes, which
uses Pardiso to solve the linear system. I'm wondering if fem is indeed
unsolvable for a large problem like this? Do you have any experience to
make it faster by changing the parameters in the linear solver?
When you want to understand how a solver is scaling, start with a small
problem, and look at the performance with -log_view. Then start increasing
the problem size, and look at how the number of iterates, subiterates, and
event times scale. Once you have that data, getting help is much easier.
Thanks,
Matt
Looking forward to hearing from you.
Best,
Zizhou
_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/
<http://www.cse.buffalo.edu/~knepley/>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/
<http://www.cse.buffalo.edu/~knepley/>