How do you detect that a user has got a solve inside a timestepping loop” -- Dr David Ham Department of Mathematics Imperial College London https://www.imperial.ac.uk/people/david.ham From: <firedrake-bounces@imperial.ac.uk> on behalf of Andrew McRae <andrew.mcrae@physics.ox.ac.uk> Date: Monday, 18 March 2019 at 15:59 To: firedrake <firedrake@imperial.ac.uk> Subject: Re: [firedrake] Firedrake Cache files As a way of mitigating this, I don't think this can possibly happen when LVP (or NVP) is used. So, as a suggestion, don't use the raw solve(....) functionality for anything that is run each timestep? (Obviously, don't recreate an LVP/NVP each timestep either) Andrew On Fri, 15 Mar 2019 at 12:01, Stephan Kramer <s.kramer@imperial.ac.uk<mailto:s.kramer@imperial.ac.uk>> wrote: I understand this is how it currently works, but this is an easy "mistake" to make especially in a complex model with lots of forcing terms. Would it not be possible to always treat a float as a parameter in the form, i.e. don't include the value of the float in the hash for the compiled code, and substitute the right value at run-time? The form itself based on a specific float, could still be treated as immutable everywhere else in firedrake. Admittedly, I have no idea how easy it is to do this Cheers Stephan On 14/03/2019 20:10, Andrew McRae wrote:
The usual cause is having a UFL form which contains a rapidly-changing floating point number.
This is fine: solve(inner(grad(u), grad(v))*dx = 2.0*f*v*dx, out, ...) # solve -div(grad(u)) = 2*f just once
This is a bad idea: t = 0.0; T = 500.0; dt = 0.1 while t < T: solve(inner(grad(u), grad(v))*dx = sin(t)*f*v*dx, out, ...) # multiply forcing by sin(t) t += dt
This would generate and compile 5000 different C files, with the values t = 0.0, 0.1, 0.2, 0.3, ... each "hardcoded" into the C code of one form.
A better solution is to use a Constant object inside the form, and update this within the loop: t = 0.0; T = 500.0; dt = 0.1 tc = Constant(t) while t < T: solve(inner(grad(u), grad(v))*dx = sin(tc)*f*v*dx, out, ...) # multiply forcing by sin(t) t += dt tc.assign(t)
This will generate a single compiled form where the current time is passed in via a pointer, rather than being hardcoded into the C code.
Note: if you are repeatedly solving a problem like this, it's a bit better to use the LinearVariationalProblem/LinearVariationalSolver objects, as described at https://www.firedrakeproject.org/demos/DG_advection.py.html, but this is ultimately separate to Constant vs hard-coded floating point numbers.
On Thu, 14 Mar 2019 at 18:57, Ziad Boutanios <ziad.boutanios@gmail.com<mailto:ziad.boutanios@gmail.com> <mailto:ziad.boutanios@gmail.com<mailto:ziad.boutanios@gmail.com>>> wrote:
Hi,
I'm a new user running on my own machine, not the Imperial clusters. Just noticed that I have almost half a Gig of compiled code and whatnot in my .cache generated in just a few hours of tutorials. I would be really interested in how to minimize this output. Could you please email the recommendations to this list?
Thanks
Ziad
On 2019-03-14 11:21 a.m., Miklós Homolya wrote: > My crystal ball says that people don't know how to use Constant, so they > instead hard-code things that change at each iteration, and that will > generate a lot of compiled code. (But it's just a guess...) > > "Cregan, Bob" <b.cregan@imperial.ac.uk<mailto:b.cregan@imperial.ac.uk> <mailto:b.cregan@imperial.ac.uk<mailto:b.cregan@imperial.ac.uk>>> writes: >> Hi >> >> We noticed that some firedrake users on the Imperial clusters have >> a large ( between 0.5 M and 1.3M) numbers of files in their cache >> (usually in firedrake/.cache/pyop2) . >> >> Is this cache size configurable? If so can you let people know how to >> limit it. It has two bad effects; firstly no filesystem is happy with >> that many files in a single directory and it will slow things down to >> the point it is useless as cache. Also each user is limited to 10M >> files and a very large cache will effectively reduce their quota to >> well below the 1TB default (they will run out of files before they run >> out of blocks). >> >> Thanks >> >> Bob >> >> >> >> Bob Cregan >> HPC Systems Analyst >> >> Information & Communication Technologies >> >> Imperial College London, >> South Kensington Campus London, SW7 2AZ >> T: 07712388129 >> E: b.cregan@imperial.ac.uk<mailto:b.cregan@imperial.ac.uk> <mailto:b.cregan@imperial.ac.uk<mailto:b.cregan@imperial.ac.uk>> >> >> W: www.imperial.ac.uk/ict/rcs<http://www.imperial.ac.uk/ict/rcs> <http://www.imperial.ac.uk/ict/rcs><http://www.imperial.ac.uk/ict/rcs> >> >> [1505984389175_twitter.png] @imperialRCS @imperialRSE >> >> [1505983882959_Imperial-RCS.png] >> >> _______________________________________________ >> firedrake mailing list >> firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> <mailto:firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk>> >> https://mailman.ic.ac.uk/mailman/listinfo/firedrake > _______________________________________________ > firedrake mailing list > firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> <mailto:firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk>> > https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> <mailto:firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk>> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake