On 11 May 2016, at 10:39, Shipton, Jemma <j.shipton@imperial.ac.uk> wrote:
PS In fact, it runs just fine if you delete the solver_parameters={blah} option.
Beware! This is hiding a problem I think. The default solver parameters are to use GMRES, not CG. If you run with GMRES and 'ksp_monitor_true_residual': True, I get the following output. 0 KSP preconditioned resid norm 1.402486418742e-02 true resid norm 2.145922987616e-02 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 6.859114126588e-03 true resid norm 8.235483003565e-03 ||r(i)||/||b|| 3.837734648956e-01 1 KSP preconditioned resid norm 6.859114126588e-03 true resid norm 8.235483003565e-03 ||r(i)||/||b|| 3.837734648956e-01 2 KSP preconditioned resid norm 4.372980631318e-03 true resid norm 4.523115353529e-03 ||r(i)||/||b|| 2.107771518191e-01 2 KSP preconditioned resid norm 4.372980631318e-03 true resid norm 4.523115353529e-03 ||r(i)||/||b|| 2.107771518191e-01 3 KSP preconditioned resid norm 3.083878237363e-03 true resid norm 3.181122331413e-03 ||r(i)||/||b|| 1.482402840070e-01 3 KSP preconditioned resid norm 3.083878237363e-03 true resid norm 3.181122331413e-03 ||r(i)||/||b|| 1.482402840070e-01 4 KSP preconditioned resid norm 2.328339850318e-03 true resid norm 2.482928092561e-03 ||r(i)||/||b|| 1.157044361280e-01 4 KSP preconditioned resid norm 2.328339850318e-03 true resid norm 2.482928092561e-03 ||r(i)||/||b|| 1.157044361280e-01 5 KSP preconditioned resid norm 1.850645396168e-03 true resid norm 2.076376448198e-03 ||r(i)||/||b|| 9.675913162685e-02 5 KSP preconditioned resid norm 1.850645396168e-03 true resid norm 2.076376448198e-03 ||r(i)||/||b|| 9.675913162685e-02 6 KSP preconditioned resid norm 1.505245304176e-03 true resid norm 1.869875298565e-03 ||r(i)||/||b|| 8.713617913390e-02 6 KSP preconditioned resid norm 1.505245304176e-03 true resid norm 1.869875298565e-03 ||r(i)||/||b|| 8.713617913390e-02 7 KSP preconditioned resid norm 1.228163738742e-03 true resid norm 1.716825645339e-03 ||r(i)||/||b|| 8.000406609403e-02 7 KSP preconditioned resid norm 1.228163738742e-03 true resid norm 1.716825645339e-03 ||r(i)||/||b|| 8.000406609403e-02 8 KSP preconditioned resid norm 9.762580464613e-04 true resid norm 1.604598260859e-03 ||r(i)||/||b|| 7.477427056418e-02 8 KSP preconditioned resid norm 9.762580464613e-04 true resid norm 1.604598260859e-03 ||r(i)||/||b|| 7.477427056418e-02 9 KSP preconditioned resid norm 7.187761512558e-04 true resid norm 1.509128698387e-03 ||r(i)||/||b|| 7.032538945228e-02 9 KSP preconditioned resid norm 7.187761512558e-04 true resid norm 1.509128698387e-03 ||r(i)||/||b|| 7.032538945228e-02 10 KSP preconditioned resid norm 4.857940540733e-04 true resid norm 1.430687141460e-03 ||r(i)||/||b|| 6.667001330972e-02 10 KSP preconditioned resid norm 4.857940540733e-04 true resid norm 1.430687141460e-03 ||r(i)||/||b|| 6.667001330972e-02 11 KSP preconditioned resid norm 3.400753386006e-04 true resid norm 1.370869815346e-03 ||r(i)||/||b|| 6.388252622562e-02 11 KSP preconditioned resid norm 3.400753386006e-04 true resid norm 1.370869815346e-03 ||r(i)||/||b|| 6.388252622562e-02 12 KSP preconditioned resid norm 2.724827673558e-04 true resid norm 1.357871715518e-03 ||r(i)||/||b|| 6.327681484164e-02 12 KSP preconditioned resid norm 2.724827673558e-04 true resid norm 1.357871715518e-03 ||r(i)||/||b|| 6.327681484164e-02 13 KSP preconditioned resid norm 2.023603200214e-04 true resid norm 1.348266511796e-03 ||r(i)||/||b|| 6.282921239844e-02 13 KSP preconditioned resid norm 2.023603200214e-04 true resid norm 1.348266511796e-03 ||r(i)||/||b|| 6.282921239844e-02 14 KSP preconditioned resid norm 1.405713028843e-04 true resid norm 1.342655679836e-03 ||r(i)||/||b|| 6.256774765844e-02 14 KSP preconditioned resid norm 1.405713028843e-04 true resid norm 1.342655679836e-03 ||r(i)||/||b|| 6.256774765844e-02 15 KSP preconditioned resid norm 9.092448503001e-05 true resid norm 1.337633948725e-03 ||r(i)||/||b|| 6.233373501493e-02 15 KSP preconditioned resid norm 9.092448503001e-05 true resid norm 1.337633948725e-03 ||r(i)||/||b|| 6.233373501493e-02 16 KSP preconditioned resid norm 5.750986093948e-05 true resid norm 1.335928348383e-03 ||r(i)||/||b|| 6.225425404790e-02 16 KSP preconditioned resid norm 5.750986093948e-05 true resid norm 1.335928348383e-03 ||r(i)||/||b|| 6.225425404790e-02 17 KSP preconditioned resid norm 3.802056816287e-05 true resid norm 1.335095336743e-03 ||r(i)||/||b|| 6.221543570986e-02 17 KSP preconditioned resid norm 3.802056816287e-05 true resid norm 1.335095336743e-03 ||r(i)||/||b|| 6.221543570986e-02 18 KSP preconditioned resid norm 2.223795106365e-05 true resid norm 1.334942349117e-03 ||r(i)||/||b|| 6.220830648727e-02 18 KSP preconditioned resid norm 2.223795106365e-05 true resid norm 1.334942349117e-03 ||r(i)||/||b|| 6.220830648727e-02 19 KSP preconditioned resid norm 1.199219664791e-05 true resid norm 1.334849886266e-03 ||r(i)||/||b|| 6.220399771888e-02 19 KSP preconditioned resid norm 1.199219664791e-05 true resid norm 1.334849886266e-03 ||r(i)||/||b|| 6.220399771888e-02 20 KSP preconditioned resid norm 6.045970214995e-06 true resid norm 1.334822540473e-03 ||r(i)||/||b|| 6.220272340509e-02 20 KSP preconditioned resid norm 6.045970214995e-06 true resid norm 1.334822540473e-03 ||r(i)||/||b|| 6.220272340509e-02 21 KSP preconditioned resid norm 3.357136102289e-06 true resid norm 1.334814998460e-03 ||r(i)||/||b|| 6.220237194734e-02 21 KSP preconditioned resid norm 3.357136102289e-06 true resid norm 1.334814998460e-03 ||r(i)||/||b|| 6.220237194734e-02 22 KSP preconditioned resid norm 2.109412994583e-06 true resid norm 1.334823835708e-03 ||r(i)||/||b|| 6.220278376303e-02 22 KSP preconditioned resid norm 2.109412994583e-06 true resid norm 1.334823835708e-03 ||r(i)||/||b|| 6.220278376303e-02 23 KSP preconditioned resid norm 1.697698874597e-06 true resid norm 1.334823113503e-03 ||r(i)||/||b|| 6.220275010828e-02 23 KSP preconditioned resid norm 1.697698874597e-06 true resid norm 1.334823113503e-03 ||r(i)||/||b|| 6.220275010828e-02 24 KSP preconditioned resid norm 1.512292251273e-06 true resid norm 1.334824567918e-03 ||r(i)||/||b|| 6.220281788403e-02 24 KSP preconditioned resid norm 1.512292251273e-06 true resid norm 1.334824567918e-03 ||r(i)||/||b|| 6.220281788403e-02 25 KSP preconditioned resid norm 1.250309415572e-06 true resid norm 1.334825572707e-03 ||r(i)||/||b|| 6.220286470715e-02 25 KSP preconditioned resid norm 1.250309415572e-06 true resid norm 1.334825572707e-03 ||r(i)||/||b|| 6.220286470715e-02 26 KSP preconditioned resid norm 7.983659954253e-07 true resid norm 1.334829784399e-03 ||r(i)||/||b|| 6.220306097201e-02 26 KSP preconditioned resid norm 7.983659954253e-07 true resid norm 1.334829784399e-03 ||r(i)||/||b|| 6.220306097201e-02 27 KSP preconditioned resid norm 5.434753011846e-07 true resid norm 1.334830275105e-03 ||r(i)||/||b|| 6.220308383892e-02 27 KSP preconditioned resid norm 5.434753011846e-07 true resid norm 1.334830275105e-03 ||r(i)||/||b|| 6.220308383892e-02 28 KSP preconditioned resid norm 3.583899393830e-07 true resid norm 1.334831849286e-03 ||r(i)||/||b|| 6.220315719574e-02 28 KSP preconditioned resid norm 3.583899393830e-07 true resid norm 1.334831849286e-03 ||r(i)||/||b|| 6.220315719574e-02 29 KSP preconditioned resid norm 2.531881383802e-07 true resid norm 1.334830569064e-03 ||r(i)||/||b|| 6.220309753738e-02 29 KSP preconditioned resid norm 2.531881383802e-07 true resid norm 1.334830569064e-03 ||r(i)||/||b|| 6.220309753738e-02 30 KSP preconditioned resid norm 1.983419151306e-07 true resid norm 1.334831092617e-03 ||r(i)||/||b|| 6.220312193496e-02 30 KSP preconditioned resid norm 1.983419151306e-07 true resid norm 1.334831092617e-03 ||r(i)||/||b|| 6.220312193496e-02 31 KSP preconditioned resid norm 1.685855333188e-07 true resid norm 1.334830361112e-03 ||r(i)||/||b|| 6.220308784684e-02 31 KSP preconditioned resid norm 1.685855333188e-07 true resid norm 1.334830361112e-03 ||r(i)||/||b|| 6.220308784684e-02 32 KSP preconditioned resid norm 1.289738938190e-07 true resid norm 1.334830636894e-03 ||r(i)||/||b|| 6.220310069824e-02 32 KSP preconditioned resid norm 1.289738938190e-07 true resid norm 1.334830636894e-03 ||r(i)||/||b|| 6.220310069824e-02 33 KSP preconditioned resid norm 9.235117500873e-08 true resid norm 1.334830368242e-03 ||r(i)||/||b|| 6.220308817906e-02 33 KSP preconditioned resid norm 9.235117500873e-08 true resid norm 1.334830368242e-03 ||r(i)||/||b|| 6.220308817906e-02 34 KSP preconditioned resid norm 6.503935947986e-08 true resid norm 1.334830235572e-03 ||r(i)||/||b|| 6.220308199666e-02 34 KSP preconditioned resid norm 6.503935947986e-08 true resid norm 1.334830235572e-03 ||r(i)||/||b|| 6.220308199666e-02 35 KSP preconditioned resid norm 4.003439088177e-08 true resid norm 1.334830289372e-03 ||r(i)||/||b|| 6.220308450375e-02 35 KSP preconditioned resid norm 4.003439088177e-08 true resid norm 1.334830289372e-03 ||r(i)||/||b|| 6.220308450375e-02 36 KSP preconditioned resid norm 2.933016192914e-08 true resid norm 1.334830350834e-03 ||r(i)||/||b|| 6.220308736789e-02 36 KSP preconditioned resid norm 2.933016192914e-08 true resid norm 1.334830350834e-03 ||r(i)||/||b|| 6.220308736789e-02 37 KSP preconditioned resid norm 2.457785557100e-08 true resid norm 1.334830341474e-03 ||r(i)||/||b|| 6.220308693167e-02 37 KSP preconditioned resid norm 2.457785557100e-08 true resid norm 1.334830341474e-03 ||r(i)||/||b|| 6.220308693167e-02 38 KSP preconditioned resid norm 2.003045563251e-08 true resid norm 1.334830452035e-03 ||r(i)||/||b|| 6.220309208385e-02 38 KSP preconditioned resid norm 2.003045563251e-08 true resid norm 1.334830452035e-03 ||r(i)||/||b|| 6.220309208385e-02 39 KSP preconditioned resid norm 1.592346775337e-08 true resid norm 1.334830443617e-03 ||r(i)||/||b|| 6.220309169157e-02 39 KSP preconditioned resid norm 1.592346775337e-08 true resid norm 1.334830443617e-03 ||r(i)||/||b|| 6.220309169157e-02 40 KSP preconditioned resid norm 1.285777369303e-08 true resid norm 1.334830523051e-03 ||r(i)||/||b|| 6.220309539319e-02 40 KSP preconditioned resid norm 1.285777369303e-08 true resid norm 1.334830523051e-03 ||r(i)||/||b|| 6.220309539319e-02 41 KSP preconditioned resid norm 9.550269214853e-09 true resid norm 1.334830544472e-03 ||r(i)||/||b|| 6.220309639139e-02 41 KSP preconditioned resid norm 9.550269214853e-09 true resid norm 1.334830544472e-03 ||r(i)||/||b|| 6.220309639139e-02 42 KSP preconditioned resid norm 6.311572416881e-09 true resid norm 1.334830559098e-03 ||r(i)||/||b|| 6.220309707295e-02 42 KSP preconditioned resid norm 6.311572416881e-09 true resid norm 1.334830559098e-03 ||r(i)||/||b|| 6.220309707295e-02 43 KSP preconditioned resid norm 3.652454149992e-09 true resid norm 1.334830540583e-03 ||r(i)||/||b|| 6.220309621017e-02 43 KSP preconditioned resid norm 3.652454149992e-09 true resid norm 1.334830540583e-03 ||r(i)||/||b|| 6.220309621017e-02 44 KSP preconditioned resid norm 2.037777932328e-09 true resid norm 1.334830540790e-03 ||r(i)||/||b|| 6.220309621981e-02 44 KSP preconditioned resid norm 2.037777932328e-09 true resid norm 1.334830540790e-03 ||r(i)||/||b|| 6.220309621981e-02 45 KSP preconditioned resid norm 1.128203860755e-09 true resid norm 1.334830532154e-03 ||r(i)||/||b|| 6.220309581738e-02 45 KSP preconditioned resid norm 1.128203860755e-09 true resid norm 1.334830532154e-03 ||r(i)||/||b|| 6.220309581738e-02 So you can see that although the preconditioned norm is converging, the unpreconditioned norm is stagnating. If you provide a nullspace to the Krylov method and use GMRES, then the solution will be the minimum residual in the least squares sense in the space orthogonal to the nullspace. However, your solution may still have a large component in the nullspace (which is what I think is going on here). I'm not sure what the correct fix is though, unfortunately. Lawrence