OK, this is the problem then - you need to find an appropriate preconditioner. You need to start by finding an appropriate preconditioner for the system without the R space part. Lawrence: once that is done, is there anything special to do to incorporate it into the system with the R space part [asking because you seem to have thought about this previously], or do we just recommend block preconditioning in this case? cheers --cjc On 7 October 2016 at 14:39, Anna Kalogirou <A.Kalogirou@leeds.ac.uk> wrote:
Hi Lawrence,
I don’t use any preconditioning in either of the two sets of code.
Anna.
On 7 Oct 2016, at 14:33, Lawrence Mitchell <lawrence.mitchell@imperial. ac.uk> wrote:
Hi Anna,
On 6 Oct 2016, at 18:42, Anna Kalogirou <A.Kalogirou@leeds.ac.uk> wrote:
Sorry for the delay, I was away from my desk all day today. I attach the two log files.
Best, Anna.
<log_file_mixed.txt><log_file.txt>
OK great. We can see that both options have quite a large room for improvement in the linear solver.
The original code does ~6000 KSPSolve calls, but calls PCApply ~395000 times. This tells me you're taking around 65 iterations per solve. Which is probably quite a lot unless your problem is very stiff.
The new code (mixed system) is worse, you do only ~2000 KSPSolve calls, but call PCApply almost ~3000000 times, or around ~1500 iterations per solve. This is terrible :).
What preconditioning, if any, are you using for these solves (in both cases?). As Colin says, we need to figure out what the linear system looks like to choose an appropriate preconditioner.
Cheers,
Lawrence
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- http://www.imperial.ac.uk/people/colin.cotter www.cambridge.org/9781107663916
On 10 Oct 2016, at 18:19, Colin Cotter <colin.cotter@imperial.ac.uk> wrote:
OK, this is the problem then - you need to find an appropriate preconditioner.
You need to start by finding an appropriate preconditioner for the system without the R space part.
Lawrence: once that is done, is there anything special to do to incorporate it into the system with the R space part [asking because you seem to have thought about this previously], or do we just recommend block preconditioning in this case?
I think you could use block first. It's easier. Otherwise I think the firedrake webpages sketch how to do sherman-morrison. Lawrence
Hi Lawrence, How do I precondition a mixed space with 4 fields? Following the instructions in the Firedrake website for 3 fields, I used: parameters = {"pc_type": "fieldsplit", "pc_fieldsplit_type": "schur", "pc_fieldsplit_0_fields": "0, 1, 2", # first split contains first three fields, second contains the fourth "pc_fieldsplit_1_fields": "3", # Multiplicative fieldsplit for first field "fieldsplit_0_pc_type": "fieldsplit", "fieldsplit_0_pc_fieldsplit_type": "multiplicative", "fieldsplit_0_fieldsplit_0_pc_type": "lu", # LU on each field "fieldsplit_0_fieldsplit_1_pc_type": "lu", "fieldsplit_0_fieldsplit_2_pc_type": "lu", "fieldsplit_1_pc_type": "ilu"} # ILU on the schur complement block but I get the error below. It seems to be related to the RealFunctionSpaces, but I might also be doing something wrong. Thanks, Anna. Traceback (most recent call last): File "/e/data1/users/matak/opt/firedrake/Ship/Modules/Inequality constraint/Mixed system/solvers.py", line 176, in solvers_SV F_solver.solve() # Solve for phi0_5, eta1, lambda0_5, mu0_5 File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/firedrake/firedrake/variational_solver.py", line 201, in solve self.snes.solve(None, v) File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:170012) File "PETSc/petscdmshell.pxi", line 285, in petsc4py.PETSc.DMSHELL_CreateFieldDecomposition (src/petsc4py.PETSc.c:48877) File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/firedrake/firedrake/functionspaceimpl.py", line 744, in create_field_decomp dms = [V._dm for V in W] File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/PyOP2/pyop2/utils.py", line 64, in __get__ obj.__dict__[self.__name__] = result = self.fget(obj) File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/firedrake/firedrake/functionspaceimpl.py", line 318, in _dm dm.setDefaultSection(self._shared_data.global_numbering) AttributeError: 'RealFunctionSpace' object has no attribute '_shared_data' On 10/10/16 18:29, Lawrence Mitchell wrote:
On 10 Oct 2016, at 18:19, Colin Cotter <colin.cotter@imperial.ac.uk> wrote:
OK, this is the problem then - you need to find an appropriate preconditioner.
You need to start by finding an appropriate preconditioner for the system without the R space part.
Lawrence: once that is done, is there anything special to do to incorporate it into the system with the R space part [asking because you seem to have thought about this previously], or do we just recommend block preconditioning in this case? I think you could use block first. It's easier. Otherwise I think the firedrake webpages sketch how to do sherman-morrison.
Lawrence _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi Anna, This looks like a bug at our end. I reworked some of this stuff while David was doing the real space and some of the assumptions I make are probably not valid in that case. I am moving house this week. Let me try and get to it next week. Lawrence
On 12 Oct 2016, at 17:31, Anna Kalogirou <a.kalogirou@leeds.ac.uk> wrote:
Hi Lawrence,
How do I precondition a mixed space with 4 fields? Following the instructions in the Firedrake website for 3 fields, I used:
parameters = {"pc_type": "fieldsplit",
"pc_fieldsplit_type": "schur",
"pc_fieldsplit_0_fields": "0, 1, 2", # first split contains first three fields, second contains the fourth
"pc_fieldsplit_1_fields": "3",
# Multiplicative fieldsplit for first field
"fieldsplit_0_pc_type": "fieldsplit",
"fieldsplit_0_pc_fieldsplit_type": "multiplicative",
"fieldsplit_0_fieldsplit_0_pc_type": "lu", # LU on each field
"fieldsplit_0_fieldsplit_1_pc_type": "lu",
"fieldsplit_0_fieldsplit_2_pc_type": "lu",
"fieldsplit_1_pc_type": "ilu"} # ILU on the schur complement block
but I get the error below. It seems to be related to the RealFunctionSpaces, but I might also be doing something wrong.
Thanks,
Anna.
Traceback (most recent call last):
File "/e/data1/users/matak/opt/firedrake/Ship/Modules/Inequality constraint/Mixed system/solvers.py", line 176, in solvers_SV
F_solver.solve() # Solve for phi0_5, eta1, lambda0_5, mu0_5
File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/firedrake/firedrake/variational_solver.py", line 201, in solve
self.snes.solve(None, v)
File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:170012)
File "PETSc/petscdmshell.pxi", line 285, in petsc4py.PETSc.DMSHELL_CreateFieldDecomposition (src/petsc4py.PETSc.c:48877)
File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/firedrake/firedrake/functionspaceimpl.py", line 744, in create_field_decomp
dms = [V._dm for V in W]
File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/PyOP2/pyop2/utils.py", line 64, in __get__
obj.__dict__[self.__name__] = result = self.fget(obj)
File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/firedrake/firedrake/functionspaceimpl.py", line 318, in _dm
dm.setDefaultSection(self._shared_data.global_numbering)
AttributeError: 'RealFunctionSpace' object has no attribute '_shared_data'
On 10/10/16 18:29, Lawrence Mitchell wrote:
On 10 Oct 2016, at 18:19, Colin Cotter <colin.cotter@imperial.ac.uk> wrote:
OK, this is the problem then - you need to find an appropriate preconditioner.
You need to start by finding an appropriate preconditioner for the system without the R space part.
Lawrence: once that is done, is there anything special to do to incorporate it into the system with the R space part [asking because you seem to have thought about this previously], or do we just recommend block preconditioning in this case? I think you could use block first. It's easier. Otherwise I think the firedrake webpages sketch how to do sherman-morrison.
Lawrence _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
Hi Lawrence, Any update regarding the bug reported below? Best, Anna. On 13 Oct 2016, at 15:13, Lawrence Mitchell <lawrence.mitchell@imperial.ac.uk<mailto:lawrence.mitchell@imperial.ac.uk>> wrote: Hi Anna, This looks like a bug at our end. I reworked some of this stuff while David was doing the real space and some of the assumptions I make are probably not valid in that case. I am moving house this week. Let me try and get to it next week. Lawrence On 12 Oct 2016, at 17:31, Anna Kalogirou <a.kalogirou@leeds.ac.uk<mailto:a.kalogirou@leeds.ac.uk>> wrote: Hi Lawrence, How do I precondition a mixed space with 4 fields? Following the instructions in the Firedrake website for 3 fields, I used: parameters = {"pc_type": "fieldsplit", "pc_fieldsplit_type": "schur", "pc_fieldsplit_0_fields": "0, 1, 2", # first split contains first three fields, second contains the fourth "pc_fieldsplit_1_fields": "3", # Multiplicative fieldsplit for first field "fieldsplit_0_pc_type": "fieldsplit", "fieldsplit_0_pc_fieldsplit_type": "multiplicative", "fieldsplit_0_fieldsplit_0_pc_type": "lu", # LU on each field "fieldsplit_0_fieldsplit_1_pc_type": "lu", "fieldsplit_0_fieldsplit_2_pc_type": "lu", "fieldsplit_1_pc_type": "ilu"} # ILU on the schur complement block but I get the error below. It seems to be related to the RealFunctionSpaces, but I might also be doing something wrong. Thanks, Anna. Traceback (most recent call last): File "/e/data1/users/matak/opt/firedrake/Ship/Modules/Inequality constraint/Mixed system/solvers.py", line 176, in solvers_SV F_solver.solve() # Solve for phi0_5, eta1, lambda0_5, mu0_5 File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/firedrake/firedrake/variational_solver.py", line 201, in solve self.snes.solve(None, v) File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:170012) File "PETSc/petscdmshell.pxi", line 285, in petsc4py.PETSc.DMSHELL_CreateFieldDecomposition (src/petsc4py.PETSc.c:48877) File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/firedrake/firedrake/functionspaceimpl.py", line 744, in create_field_decomp dms = [V._dm for V in W] File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/PyOP2/pyop2/utils.py", line 64, in __get__ obj.__dict__[self.__name__] = result = self.fget(obj) File "/e/data1/users/matak/opt/firedrake-installation/firedrake/src/firedrake/firedrake/functionspaceimpl.py", line 318, in _dm dm.setDefaultSection(self._shared_data.global_numbering) AttributeError: 'RealFunctionSpace' object has no attribute '_shared_data' On 10/10/16 18:29, Lawrence Mitchell wrote: On 10 Oct 2016, at 18:19, Colin Cotter <colin.cotter@imperial.ac.uk<mailto:colin.cotter@imperial.ac.uk>> wrote: OK, this is the problem then - you need to find an appropriate preconditioner. You need to start by finding an appropriate preconditioner for the system without the R space part. Lawrence: once that is done, is there anything special to do to incorporate it into the system with the R space part [asking because you seem to have thought about this previously], or do we just recommend block preconditioning in this case? I think you could use block first. It's easier. Otherwise I think the firedrake webpages sketch how to do sherman-morrison. Lawrence _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
On 24 Oct 2016, at 11:13, Anna Kalogirou <A.Kalogirou@leeds.ac.uk> wrote:
Hi Lawrence,
Any update regarding the bug reported below?
This appears to be more involved than I had first thought. So no luck yet, sorry! David and I need to find some time together to work out the differences/corner cases. Cheers, Lawrence
participants (4)
-
Anna Kalogirou
-
Anna Kalogirou
-
Colin Cotter
-
Lawrence Mitchell