How do you detect that a user has got a solve inside a timestepping loop”

Dr David Ham

Department of Mathematics

Imperial College London

https://www.imperial.ac.uk/people/david.ham

From: <firedrake-bounces@imperial.ac.uk> on behalf of Andrew McRae <andrew.mcrae@physics.ox.ac.uk>
Date: Monday, 18 March 2019 at 15:59
To: firedrake <firedrake@imperial.ac.uk>
Subject: Re: [firedrake] Firedrake Cache files

As a way of mitigating this, I don't think this can possibly happen when LVP (or NVP) is used.

So, as a suggestion, don't use the raw solve(....) functionality for anything that is run each timestep? (Obviously, don't recreate an LVP/NVP each timestep either)

Andrew

On Fri, 15 Mar 2019 at 12:01, Stephan Kramer <s.kramer@imperial.ac.uk> wrote:

I understand this is how it currently works, but this is an easy
"mistake" to make especially in a complex model with lots of forcing
terms. Would it not be possible to always treat a float as a parameter
in the form, i.e. don't include the value of the float in the hash for
the compiled code, and substitute the right value at run-time? The form
itself based on a specific float, could still be treated as immutable
everywhere else in firedrake. Admittedly, I have no idea how easy it is
to do this

Cheers
Stephan

On 14/03/2019 20:10, Andrew McRae wrote:
> The usual cause is having a UFL form which contains a rapidly-changing
> floating point number.
>
> This is fine:
> solve(inner(grad(u), grad(v))*dx = 2.0*f*v*dx, out, ...) # solve
> -div(grad(u)) = 2*f just once
>
> This is a bad idea:
> t = 0.0; T = 500.0; dt = 0.1
> while t < T:
>     solve(inner(grad(u), grad(v))*dx = sin(t)*f*v*dx, out, ...) #
> multiply forcing by sin(t)
>     t += dt
>
> This would generate and compile 5000 different C files, with the values
> t = 0.0, 0.1, 0.2, 0.3, ... each "hardcoded" into the C code of one form.
>
> A better solution is to use a Constant object inside the form, and
> update this within the loop:
> t = 0.0; T = 500.0; dt = 0.1
> tc = Constant(t)
> while t < T:
>     solve(inner(grad(u), grad(v))*dx = sin(tc)*f*v*dx, out, ...) #
> multiply forcing by sin(t)
>     t += dt
>     tc.assign(t)
>
> This will generate a single compiled form where the current time is
> passed in via a pointer, rather than being hardcoded into the C code.
>
> Note: if you are repeatedly solving a problem like this, it's a bit
> better to use the LinearVariationalProblem/LinearVariationalSolver
> objects, as described at
> https://www.firedrakeproject.org/demos/DG_advection.py.html, but this is
> ultimately separate to Constant vs hard-coded floating point numbers.
>
> On Thu, 14 Mar 2019 at 18:57, Ziad Boutanios <ziad.boutanios@gmail.com
> <mailto:ziad.boutanios@gmail.com>> wrote:
>
> Hi,
>
> I'm a new user running on my own machine, not the Imperial clusters.
> Just noticed that I have almost half a Gig of compiled code and whatnot
> in my .cache generated in just a few hours of tutorials. I would be
> really interested in how to minimize this output. Could you please
> email
> the recommendations to this list?
>
> Thanks
>
> Ziad
>
> On 2019-03-14 11:21 a.m., Miklós Homolya wrote:
> > My crystal ball says that people don't know how to use Constant,
> so they
> > instead hard-code things that change at each iteration, and that will
> > generate a lot of compiled code. (But it's just a guess...)
> >
> > "Cregan, Bob" <b.cregan@imperial.ac.uk
> <mailto:b.cregan@imperial.ac.uk>> writes:
> >> Hi
> >>
> >> We noticed that some firedrake users on the Imperial
> clusters have
> >> a large ( between 0.5 M and 1.3M) numbers of files in their cache
> >> (usually in firedrake/.cache/pyop2) .
> >>
> >> Is this cache size configurable? If so can you let people know
> how to
> >> limit it. It has two bad effects; firstly no filesystem is
> happy with
> >> that many files in a single directory and it will slow things
> down to
> >> the point it is useless as cache. Also each user is limited to 10M
> >> files and a very large cache will effectively reduce their quota to
> >> well below the 1TB default (they will run out of files before
> they run
> >> out of blocks).
> >>
> >> Thanks
> >>
> >> Bob
> >>
> >>
> >>
> >> Bob Cregan
> >> HPC Systems Analyst
> >>
> >> Information & Communication Technologies
> >>
> >> Imperial College London,
> >> South Kensington Campus London, SW7 2AZ
> >> T: 07712388129
> >> E: b.cregan@imperial.ac.uk <mailto:b.cregan@imperial.ac.uk>
> >>
> >> W: www.imperial.ac.uk/ict/rcs
> <http://www.imperial.ac.uk/ict/rcs><http://www.imperial.ac.uk/ict/rcs>
> >>
> >> [1505984389175_twitter.png] @imperialRCS @imperialRSE
> >>
> >> [1505983882959_Imperial-RCS.png]
> >>
> >> _______________________________________________
> >> firedrake mailing list
> >> firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk>
> >> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
> > _______________________________________________
> > firedrake mailing list
> > firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk>
> > https://mailman.ic.ac.uk/mailman/listinfo/firedrake
>
> _______________________________________________
> firedrake mailing list
> firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk>
> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
>
>
> _______________________________________________
> firedrake mailing list
> firedrake@imperial.ac.uk
> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
>

_______________________________________________
firedrake mailing list
firedrake@imperial.ac.uk
https://mailman.ic.ac.uk/mailman/listinfo/firedrake