Hi all, I have another variant on this to suggest. How about we just put the check in the firedrake import (on MPI rank 0). We check for having what seems to be an excessive number of cached files (say, over 10k) and spew a warning: "Your Firedrake disk caches contain n files. You should consider clearing them using: firedrake-clean If this issue recurs frequently, it may indicate a problem with the way you are specifying your differential equations in UFL. Please see <url> for more details." What do people think of that? David On 17/03/2019, 12:21, "firedrake-bounces@imperial.ac.uk on behalf of Miklós Homolya" <firedrake-bounces@imperial.ac.uk on behalf of miklos.homolya@tum.de> wrote: An improved version of the logging idea would be this: Stay silent for the first, say, few hundred to a thousand invocations of compiling generated code, but then start emitting visible warnings at each invocation. The message of the warning would warn for the potential issue of generating code in each time stepping iteration, and would reference the corresponding FAQ entry of the documentation, which would describe how to verify if that is the case, how to fix the problem, or to configure the number of swallowed invocations (for very complex models), or even to disable these warnings completely (not recommended). Miklós Homolya <miklos.homolya@tum.de> writes: > Arguably one could let some logging through when just-in-time > compilation happens, although that typically evokes the wrong reaction > from users: "firedrake keeps printing all this crap! how to disable it?" > > I find it a pretty aggressive move to convert all floats to Constants, > since it easily leads to a perfusion of kernel arguments. Though, it > may still be the safest default behaviour, but I think one should at > least be able to disable it. > > Stephan Kramer <s.kramer@imperial.ac.uk> writes: >> I understand this is how it currently works, but this is an easy >> "mistake" to make especially in a complex model with lots of forcing >> terms. Would it not be possible to always treat a float as a parameter >> in the form, i.e. don't include the value of the float in the hash for >> the compiled code, and substitute the right value at run-time? The form >> itself based on a specific float, could still be treated as immutable >> everywhere else in firedrake. Admittedly, I have no idea how easy it is >> to do this >> >> Cheers >> Stephan >> >> >> On 14/03/2019 20:10, Andrew McRae wrote: >>> The usual cause is having a UFL form which contains a rapidly-changing >>> floating point number. >>> >>> This is fine: >>> solve(inner(grad(u), grad(v))*dx = 2.0*f*v*dx, out, ...) # solve >>> -div(grad(u)) = 2*f just once >>> >>> This is a bad idea: >>> t = 0.0; T = 500.0; dt = 0.1 >>> while t < T: >>> solve(inner(grad(u), grad(v))*dx = sin(t)*f*v*dx, out, ...) # >>> multiply forcing by sin(t) >>> t += dt >>> >>> This would generate and compile 5000 different C files, with the values >>> t = 0.0, 0.1, 0.2, 0.3, ... each "hardcoded" into the C code of one form. >>> >>> A better solution is to use a Constant object inside the form, and >>> update this within the loop: >>> t = 0.0; T = 500.0; dt = 0.1 >>> tc = Constant(t) >>> while t < T: >>> solve(inner(grad(u), grad(v))*dx = sin(tc)*f*v*dx, out, ...) # >>> multiply forcing by sin(t) >>> t += dt >>> tc.assign(t) >>> >>> This will generate a single compiled form where the current time is >>> passed in via a pointer, rather than being hardcoded into the C code. >>> >>> Note: if you are repeatedly solving a problem like this, it's a bit >>> better to use the LinearVariationalProblem/LinearVariationalSolver >>> objects, as described at >>> https://www.firedrakeproject.org/demos/DG_advection.py.html, but this is >>> ultimately separate to Constant vs hard-coded floating point numbers. >>> >>> On Thu, 14 Mar 2019 at 18:57, Ziad Boutanios <ziad.boutanios@gmail.com >>> <mailto:ziad.boutanios@gmail.com>> wrote: >>> >>> Hi, >>> >>> I'm a new user running on my own machine, not the Imperial clusters. >>> Just noticed that I have almost half a Gig of compiled code and whatnot >>> in my .cache generated in just a few hours of tutorials. I would be >>> really interested in how to minimize this output. Could you please >>> email >>> the recommendations to this list? >>> >>> Thanks >>> >>> Ziad >>> >>> On 2019-03-14 11:21 a.m., Miklós Homolya wrote: >>> > My crystal ball says that people don't know how to use Constant, >>> so they >>> > instead hard-code things that change at each iteration, and that will >>> > generate a lot of compiled code. (But it's just a guess...) >>> > >>> > "Cregan, Bob" <b.cregan@imperial.ac.uk >>> <mailto:b.cregan@imperial.ac.uk>> writes: >>> >> Hi >>> >> >>> >> We noticed that some firedrake users on the Imperial >>> clusters have >>> >> a large ( between 0.5 M and 1.3M) numbers of files in their cache >>> >> (usually in firedrake/.cache/pyop2) . >>> >> >>> >> Is this cache size configurable? If so can you let people know >>> how to >>> >> limit it. It has two bad effects; firstly no filesystem is >>> happy with >>> >> that many files in a single directory and it will slow things >>> down to >>> >> the point it is useless as cache. Also each user is limited to 10M >>> >> files and a very large cache will effectively reduce their quota to >>> >> well below the 1TB default (they will run out of files before >>> they run >>> >> out of blocks). >>> >> >>> >> Thanks >>> >> >>> >> Bob >>> >> >>> >> >>> >> >>> >> Bob Cregan >>> >> HPC Systems Analyst >>> >> >>> >> Information & Communication Technologies >>> >> >>> >> Imperial College London, >>> >> South Kensington Campus London, SW7 2AZ >>> >> T: 07712388129 >>> >> E: b.cregan@imperial.ac.uk <mailto:b.cregan@imperial.ac.uk> >>> >> >>> >> W: www.imperial.ac.uk/ict/rcs >>> <http://www.imperial.ac.uk/ict/rcs><http://www.imperial.ac.uk/ict/rcs> >>> >> >>> >> [1505984389175_twitter.png] @imperialRCS @imperialRSE >>> >> >>> >> [1505983882959_Imperial-RCS.png] >>> >> >>> >> _______________________________________________ >>> >> firedrake mailing list >>> >> firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> >>> >> https://mailman.ic.ac.uk/mailman/listinfo/firedrake >>> > _______________________________________________ >>> > firedrake mailing list >>> > firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> >>> > https://mailman.ic.ac.uk/mailman/listinfo/firedrake >>> >>> _______________________________________________ >>> firedrake mailing list >>> firedrake@imperial.ac.uk <mailto:firedrake@imperial.ac.uk> >>> https://mailman.ic.ac.uk/mailman/listinfo/firedrake >>> >>> >>> _______________________________________________ >>> firedrake mailing list >>> firedrake@imperial.ac.uk >>> https://mailman.ic.ac.uk/mailman/listinfo/firedrake >>> >> >> >> >> _______________________________________________ >> firedrake mailing list >> firedrake@imperial.ac.uk >> https://mailman.ic.ac.uk/mailman/listinfo/firedrake _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake