tracking write accesses to dats/functions
Dear firedrakers, can I somehow check whether a dat/function has been written to since I last performed a given operation on it? I have a banded matrix A stored in each cell on mesh, and whenever I solve the equation A.x = b (separately in each column, of course) I want to do this in two steps: (1) calculate the LU decomposition (lapack dgbtrf) (2) solve based on that LU decomposition (lapack dgbtrs) I have to do a lot of solves, so for efficiency I want to only do (1) if my matrix A has changed. So when I do a solve, I check if a flag lu_decomp is True (and this flag is set to False upon matrix creation). If it is, then I go straight to (2), otherwise I do (1) first. But I need to make sure that whenever my banded matrix has been written to lu_decomp is set back to False. Thanks, Eike
On 23/11/14 12:11, Eike Mueller wrote:
Dear firedrakers,
can I somehow check whether a dat/function has been written to since I last performed a given operation on it?
I have a banded matrix A stored in each cell on mesh, and whenever I solve the equation A.x = b (separately in each column, of course) I want to do this in two steps:
(1) calculate the LU decomposition (lapack dgbtrf) (2) solve based on that LU decomposition (lapack dgbtrs)
I have to do a lot of solves, so for efficiency I want to only do (1) if my matrix A has changed. So when I do a solve, I check if a flag lu_decomp is True (and this flag is set to False upon matrix creation). If it is, then I go straight to (2), otherwise I do (1) first. But I need to make sure that whenever my banded matrix has been written to lu_decomp is set back to False.
I'm not exactly sure I understand what you're trying to do. Are those matrices represented by firedrake.Matrix/pyop2.Mat? If so, they're only ever written to when you call assemble on the Matrix object. You can check the assembled property to see whether the Matrix is assembled and _needs_reassembly to check whether it needs to be reassembled. If you're interested whether or not PETSc has written to the matrix during solve I'm not sure. I don't think an assembled matrix is written to at all, but maybe Lawrence can clarify. Hope this helps, Florian
Thanks,
Eike
Hi Florian, this is for an extruded mesh and I’m always working in a horizontal DG space, so that the assembled matrix has a block-structure (i.e. there are no couplings between different columns). In each block (i.e. column) the matrix is banded. Mathematically, I can invert the matrix in each column independently without assembling the global matrix. I need this for my multigrid smoother. In each column I create a dat which stores the banded matrix and assemble it locally by hand. I then write my own kernels (looping over the 2d grid) to, for example, apply the local banded matrix or do a LU decomposition and solve with LAPACK. So in summary, I do not use the PyOP2 Mat() class, but represent my matrices by Functions (I create a DG0 space of size bandwidth*n on the base mesh). Thanks, Eike -- Dr Eike Hermann Mueller Research Associate (PostDoc) Department of Mathematical Sciences University of Bath Bath BA2 7AY, United Kingdom +44 1225 38 5803 e.mueller@bath.ac.uk http://people.bath.ac.uk/em459/
On 23 Nov 2014, at 17:19, Florian Rathgeber <florian.rathgeber@imperial.ac.uk> wrote:
On 23/11/14 12:11, Eike Mueller wrote:
Dear firedrakers,
can I somehow check whether a dat/function has been written to since I last performed a given operation on it?
I have a banded matrix A stored in each cell on mesh, and whenever I solve the equation A.x = b (separately in each column, of course) I want to do this in two steps:
(1) calculate the LU decomposition (lapack dgbtrf) (2) solve based on that LU decomposition (lapack dgbtrs)
I have to do a lot of solves, so for efficiency I want to only do (1) if my matrix A has changed. So when I do a solve, I check if a flag lu_decomp is True (and this flag is set to False upon matrix creation). If it is, then I go straight to (2), otherwise I do (1) first. But I need to make sure that whenever my banded matrix has been written to lu_decomp is set back to False.
I'm not exactly sure I understand what you're trying to do. Are those matrices represented by firedrake.Matrix/pyop2.Mat? If so, they're only ever written to when you call assemble on the Matrix object. You can check the assembled property to see whether the Matrix is assembled and _needs_reassembly to check whether it needs to be reassembled.
If you're interested whether or not PETSc has written to the matrix during solve I'm not sure. I don't think an assembled matrix is written to at all, but maybe Lawrence can clarify.
Hope this helps, Florian
Thanks,
Eike
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake <https://mailman.ic.ac.uk/mailman/listinfo/firedrake>
On 23 Nov 2014, at 12:11, Eike Mueller <eike.h.mueller@googlemail.com> wrote:
Dear firedrakers,
can I somehow check whether a dat/function has been written to since I last performed a given operation on it?
I have a banded matrix A stored in each cell on mesh, and whenever I solve the equation A.x = b (separately in each column, of course) I want to do this in two steps:
(1) calculate the LU decomposition (lapack dgbtrf) (2) solve based on that LU decomposition (lapack dgbtrs)
I have to do a lot of solves, so for efficiency I want to only do (1) if my matrix A has changed. So when I do a solve, I check if a flag lu_decomp is True (and this flag is set to False upon matrix creation). If it is, then I go straight to (2), otherwise I do (1) first. But I need to make sure that whenever my banded matrix has been written to lu_decomp is set back to False.
So there are two things you can do here. 1. keep track of the flag yourself, on the assumption that the state will only be modified by your public API functions. I.e., when you do the LU decomposition, set a flag to indicate that the decomposition is valid. Then when you call an assemble (or similar) into your banded matrix invalidate that flag. 2. Not part of the public API, but Dats (and indeed Mats) are versioned objects. You can inspect the Dat._version slot. This is a state counter that gets bumped every time the Dat is modified (either by using the public data accessor, or through a par_loop of some kind). So in this case, when you do the LU decomp you record the _version slot in your object, next time through you check if this has changed. Cheers, Lawrence
Hi Lawrence, thanks, option 2 seems to be more fool-proof since I don’t have to remember to track all writes myself. I have implemented this now and it works (all changes pushed to firedrake-helmholtzsolver). Cheers, Eike -- Dr Eike Hermann Mueller Research Associate (PostDoc) Department of Mathematical Sciences University of Bath Bath BA2 7AY, United Kingdom +44 1225 38 5803 e.mueller@bath.ac.uk http://people.bath.ac.uk/em459/
On 23 Nov 2014, at 21:01, Lawrence Mitchell <lawrence.mitchell@imperial.ac.uk> wrote:
On 23 Nov 2014, at 12:11, Eike Mueller <eike.h.mueller@googlemail.com <mailto:eike.h.mueller@googlemail.com>> wrote:
Dear firedrakers,
can I somehow check whether a dat/function has been written to since I last performed a given operation on it?
I have a banded matrix A stored in each cell on mesh, and whenever I solve the equation A.x = b (separately in each column, of course) I want to do this in two steps:
(1) calculate the LU decomposition (lapack dgbtrf) (2) solve based on that LU decomposition (lapack dgbtrs)
I have to do a lot of solves, so for efficiency I want to only do (1) if my matrix A has changed. So when I do a solve, I check if a flag lu_decomp is True (and this flag is set to False upon matrix creation). If it is, then I go straight to (2), otherwise I do (1) first. But I need to make sure that whenever my banded matrix has been written to lu_decomp is set back to False.
So there are two things you can do here.
1. keep track of the flag yourself, on the assumption that the state will only be modified by your public API functions. I.e., when you do the LU decomposition, set a flag to indicate that the decomposition is valid. Then when you call an assemble (or similar) into your banded matrix invalidate that flag.
2. Not part of the public API, but Dats (and indeed Mats) are versioned objects. You can inspect the Dat._version slot. This is a state counter that gets bumped every time the Dat is modified (either by using the public data accessor, or through a par_loop of some kind). So in this case, when you do the LU decomp you record the _version slot in your object, next time through you check if this has changed.
Cheers,
Lawrence _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake <https://mailman.ic.ac.uk/mailman/listinfo/firedrake>
participants (4)
-
Eike Mueller
-
Eike Mueller
-
Florian Rathgeber
-
Lawrence Mitchell