March 2019 - firedrake - imperial.ac.uk

Re: [firedrake] Firedrake Cache files
by Andrew McRae 28 Mar '19

28 Mar '19

The usual cause is having a UFL form which contains a rapidly-changing floating point number. This is fine: solve(inner(grad(u), grad(v))*dx = 2.0*f*v*dx, out, ...) # solve -div(grad(u)) = 2*f just once This is a bad idea: t = 0.0; T = 500.0; dt = 0.1 while t < T: solve(inner(grad(u), grad(v))*dx = sin(t)*f*v*dx, out, ...) # multiply forcing by sin(t) t += dt This would generate and compile 5000 different C files, with the values t = 0.0, 0.1, 0.2, 0.3, ... each "hardcoded" into the C code of one form. A better solution is to use a Constant object inside the form, and update this within the loop: t = 0.0; T = 500.0; dt = 0.1 tc = Constant(t) while t < T: solve(inner(grad(u), grad(v))*dx = sin(tc)*f*v*dx, out, ...) # multiply forcing by sin(t) t += dt tc.assign(t) This will generate a single compiled form where the current time is passed in via a pointer, rather than being hardcoded into the C code. Note: if you are repeatedly solving a problem like this, it's a bit better to use the LinearVariationalProblem/LinearVariationalSolver objects, as described at https://www.firedrakeproject.org/demos/DG_advection.py.html, but this is ultimately separate to Constant vs hard-coded floating point numbers. On Thu, 14 Mar 2019 at 18:57, Ziad Boutanios <ziad.boutanios(a)gmail.com> wrote: > Hi, > > I'm a new user running on my own machine, not the Imperial clusters. > Just noticed that I have almost half a Gig of compiled code and whatnot > in my .cache generated in just a few hours of tutorials. I would be > really interested in how to minimize this output. Could you please email > the recommendations to this list? > > Thanks > > Ziad > > On 2019-03-14 11:21 a.m., Miklós Homolya wrote: > > My crystal ball says that people don't know how to use Constant, so they > > instead hard-code things that change at each iteration, and that will > > generate a lot of compiled code. (But it's just a guess...) > > > > "Cregan, Bob" <b.cregan(a)imperial.ac.uk> writes: > >> Hi > >> > >> We noticed that some firedrake users on the Imperial clusters have > >> a large ( between 0.5 M and 1.3M) numbers of files in their cache > >> (usually in firedrake/.cache/pyop2) . > >> > >> Is this cache size configurable? If so can you let people know how to > >> limit it. It has two bad effects; firstly no filesystem is happy with > >> that many files in a single directory and it will slow things down to > >> the point it is useless as cache. Also each user is limited to 10M > >> files and a very large cache will effectively reduce their quota to > >> well below the 1TB default (they will run out of files before they run > >> out of blocks). > >> > >> Thanks > >> > >> Bob > >> > >> > >> > >> Bob Cregan > >> HPC Systems Analyst > >> > >> Information & Communication Technologies > >> > >> Imperial College London, > >> South Kensington Campus London, SW7 2AZ > >> T: 07712388129 > >> E: b.cregan(a)imperial.ac.uk > >> > >> W: www.imperial.ac.uk/ict/rcs<http://www.imperial.ac.uk/ict/rcs> > >> > >> [1505984389175_twitter.png] @imperialRCS @imperialRSE > >> > >> [1505983882959_Imperial-RCS.png] > >> > >> _______________________________________________ > >> firedrake mailing list > >> firedrake(a)imperial.ac.uk > >> https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > _______________________________________________ > > firedrake mailing list > > firedrake(a)imperial.ac.uk > > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > _______________________________________________ > firedrake mailing list > firedrake(a)imperial.ac.uk > https://mailman.ic.ac.uk/mailman/listinfo/firedrake >

5 7

Build issue on Imperial clusters/Intel CC
by Cregan, Bob 25 Mar '19

25 Mar '19

Hi We've been experiencing a problem with the firedrake build using intel compilers/Intel MPI on the Imperial clusters. We have a workaround: see attached. However it would be nice if the build was fixed. The workaround basically builds petsc separately so that we can put a couple of symlinks into the petsc/include dir. They are links to mpi.h and mpio.h. If we don't do that then PyOP2 fails to find the headers in the intel mpi environment. The includes are set in the mpi module but seem to be lost when it comes to building PyOP2. Thanks Bob Bob Cregan HPC Systems Analyst Information & Communication Technologies Imperial College London, South Kensington Campus London, SW7 2AZ T: 07712388129 E: b.cregan(a)imperial.ac.uk W: www.imperial.ac.uk/ict/rcs<http://www.imperial.ac.uk/ict/rcs> [1505984389175_twitter.png] @imperialRCS @imperialRSE [1505983882959_Imperial-RCS.png]

2 5

schrodinger equation in firedrake
by Francis Poulin 21 Mar '19

21 Mar '19

Hello, I have students who want to modify code that I wrote to solve the diffusion equation to solve the schrodinger equation. This is fairly straightforward but I have two questions. 1. Can we use complex constants in firedrake? 2. Has someone else done this already? Thank you, Francis ------------------ Francis Poulin Associate Dean, Undergraduate Studies Professor Department of Applied Mathematics University of Waterloo email: fpoulin(a)uwaterloo.ca Web: https://uwaterloo.ca/poulin-research-group/ Telephone: +1 519 888 4567 x32637 [X]

1 0

Firedrake Cache files
by Cregan, Bob 20 Mar '19

20 Mar '19

Hi We noticed that some firedrake users on the Imperial clusters have a large ( between 0.5 M and 1.3M) numbers of files in their cache (usually in firedrake/.cache/pyop2) . Is this cache size configurable? If so can you let people know how to limit it. It has two bad effects; firstly no filesystem is happy with that many files in a single directory and it will slow things down to the point it is useless as cache. Also each user is limited to 10M files and a very large cache will effectively reduce their quota to well below the 1TB default (they will run out of files before they run out of blocks). Thanks Bob Bob Cregan HPC Systems Analyst Information & Communication Technologies Imperial College London, South Kensington Campus London, SW7 2AZ T: 07712388129 E: b.cregan(a)imperial.ac.uk W: www.imperial.ac.uk/ict/rcs<http://www.imperial.ac.uk/ict/rcs> [1505984389175_twitter.png] @imperialRCS @imperialRSE [1505983882959_Imperial-RCS.png]

5 8

Re: [firedrake] Firedrake Cache files
by Andrew McRae 18 Mar '19

18 Mar '19

This was a suggestion relating to users' coding practices, not a proposal that Firedrake should perform this analysis automatically. On Mon, 18 Mar 2019 at 16:02, Ham, David A <david.ham(a)imperial.ac.uk> wrote: > How do you detect that a user has got a solve inside a timestepping loop” > > > > -- > > Dr David Ham > > Department of Mathematics > > Imperial College London > > https://www.imperial.ac.uk/people/david.ham > > > > *From: *<firedrake-bounces(a)imperial.ac.uk> on behalf of Andrew McRae < > andrew.mcrae(a)physics.ox.ac.uk> > *Date: *Monday, 18 March 2019 at 15:59 > *To: *firedrake <firedrake(a)imperial.ac.uk> > *Subject: *Re: [firedrake] Firedrake Cache files > > > > As a way of mitigating this, I don't *think* this can possibly happen > when LVP (or NVP) is used. > > > > So, as a suggestion, don't use the raw solve(....) functionality for > anything that is run each timestep? (Obviously, don't recreate an LVP/NVP > each timestep either) > > > > Andrew > > > > On Fri, 15 Mar 2019 at 12:01, Stephan Kramer <s.kramer(a)imperial.ac.uk> > wrote: > > I understand this is how it currently works, but this is an easy > "mistake" to make especially in a complex model with lots of forcing > terms. Would it not be possible to always treat a float as a parameter > in the form, i.e. don't include the value of the float in the hash for > the compiled code, and substitute the right value at run-time? The form > itself based on a specific float, could still be treated as immutable > everywhere else in firedrake. Admittedly, I have no idea how easy it is > to do this > > Cheers > Stephan > > > On 14/03/2019 20:10, Andrew McRae wrote: > > The usual cause is having a UFL form which contains a rapidly-changing > > floating point number. > > > > This is fine: > > solve(inner(grad(u), grad(v))*dx = 2.0*f*v*dx, out, ...) # solve > > -div(grad(u)) = 2*f just once > > > > This is a bad idea: > > t = 0.0; T = 500.0; dt = 0.1 > > while t < T: > > solve(inner(grad(u), grad(v))*dx = sin(t)*f*v*dx, out, ...) # > > multiply forcing by sin(t) > > t += dt > > > > This would generate and compile 5000 different C files, with the values > > t = 0.0, 0.1, 0.2, 0.3, ... each "hardcoded" into the C code of one form. > > > > A better solution is to use a Constant object inside the form, and > > update this within the loop: > > t = 0.0; T = 500.0; dt = 0.1 > > tc = Constant(t) > > while t < T: > > solve(inner(grad(u), grad(v))*dx = sin(tc)*f*v*dx, out, ...) # > > multiply forcing by sin(t) > > t += dt > > tc.assign(t) > > > > This will generate a single compiled form where the current time is > > passed in via a pointer, rather than being hardcoded into the C code. > > > > Note: if you are repeatedly solving a problem like this, it's a bit > > better to use the LinearVariationalProblem/LinearVariationalSolver > > objects, as described at > > https://www.firedrakeproject.org/demos/DG_advection.py.html, but this > is > > ultimately separate to Constant vs hard-coded floating point numbers. > > > > On Thu, 14 Mar 2019 at 18:57, Ziad Boutanios <ziad.boutanios(a)gmail.com > > <mailto:ziad.boutanios@gmail.com>> wrote: > > > > Hi, > > > > I'm a new user running on my own machine, not the Imperial clusters. > > Just noticed that I have almost half a Gig of compiled code and > whatnot > > in my .cache generated in just a few hours of tutorials. I would be > > really interested in how to minimize this output. Could you please > > email > > the recommendations to this list? > > > > Thanks > > > > Ziad > > > > On 2019-03-14 11:21 a.m., Miklós Homolya wrote: > > > My crystal ball says that people don't know how to use Constant, > > so they > > > instead hard-code things that change at each iteration, and that > will > > > generate a lot of compiled code. (But it's just a guess...) > > > > > > "Cregan, Bob" <b.cregan(a)imperial.ac.uk > > <mailto:b.cregan@imperial.ac.uk>> writes: > > >> Hi > > >> > > >> We noticed that some firedrake users on the Imperial > > clusters have > > >> a large ( between 0.5 M and 1.3M) numbers of files in their cache > > >> (usually in firedrake/.cache/pyop2) . > > >> > > >> Is this cache size configurable? If so can you let people know > > how to > > >> limit it. It has two bad effects; firstly no filesystem is > > happy with > > >> that many files in a single directory and it will slow things > > down to > > >> the point it is useless as cache. Also each user is limited to > 10M > > >> files and a very large cache will effectively reduce their quota > to > > >> well below the 1TB default (they will run out of files before > > they run > > >> out of blocks). > > >> > > >> Thanks > > >> > > >> Bob > > >> > > >> > > >> > > >> Bob Cregan > > >> HPC Systems Analyst > > >> > > >> Information & Communication Technologies > > >> > > >> Imperial College London, > > >> South Kensington Campus London, SW7 2AZ > > >> T: 07712388129 > > >> E: b.cregan(a)imperial.ac.uk <mailto:b.cregan@imperial.ac.uk> > > >> > > >> W: www.imperial.ac.uk/ict/rcs > > <http://www.imperial.ac.uk/ict/rcs>< > http://www.imperial.ac.uk/ict/rcs> > > >> > > >> [1505984389175_twitter.png] @imperialRCS @imperialRSE > > >> > > >> [1505983882959_Imperial-RCS.png] > > >> > > >> _______________________________________________ > > >> firedrake mailing list > > >> firedrake(a)imperial.ac.uk <mailto:firedrake@imperial.ac.uk> > > >> https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > _______________________________________________ > > > firedrake mailing list > > > firedrake(a)imperial.ac.uk <mailto:firedrake@imperial.ac.uk> > > > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > > _______________________________________________ > > firedrake mailing list > > firedrake(a)imperial.ac.uk <mailto:firedrake@imperial.ac.uk> > > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > > > > _______________________________________________ > > firedrake mailing list > > firedrake(a)imperial.ac.uk > > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > > > > _______________________________________________ > firedrake mailing list > firedrake(a)imperial.ac.uk > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > >

1 0

Re: [firedrake] Firedrake Cache files
by Andrew McRae 18 Mar '19

18 Mar '19

As a way of mitigating this, I don't *think* this can possibly happen when LVP (or NVP) is used. So, as a suggestion, don't use the raw solve(....) functionality for anything that is run each timestep? (Obviously, don't recreate an LVP/NVP each timestep either) Andrew On Fri, 15 Mar 2019 at 12:01, Stephan Kramer <s.kramer(a)imperial.ac.uk> wrote: > I understand this is how it currently works, but this is an easy > "mistake" to make especially in a complex model with lots of forcing > terms. Would it not be possible to always treat a float as a parameter > in the form, i.e. don't include the value of the float in the hash for > the compiled code, and substitute the right value at run-time? The form > itself based on a specific float, could still be treated as immutable > everywhere else in firedrake. Admittedly, I have no idea how easy it is > to do this > > Cheers > Stephan > > > On 14/03/2019 20:10, Andrew McRae wrote: > > The usual cause is having a UFL form which contains a rapidly-changing > > floating point number. > > > > This is fine: > > solve(inner(grad(u), grad(v))*dx = 2.0*f*v*dx, out, ...) # solve > > -div(grad(u)) = 2*f just once > > > > This is a bad idea: > > t = 0.0; T = 500.0; dt = 0.1 > > while t < T: > > solve(inner(grad(u), grad(v))*dx = sin(t)*f*v*dx, out, ...) # > > multiply forcing by sin(t) > > t += dt > > > > This would generate and compile 5000 different C files, with the values > > t = 0.0, 0.1, 0.2, 0.3, ... each "hardcoded" into the C code of one form. > > > > A better solution is to use a Constant object inside the form, and > > update this within the loop: > > t = 0.0; T = 500.0; dt = 0.1 > > tc = Constant(t) > > while t < T: > > solve(inner(grad(u), grad(v))*dx = sin(tc)*f*v*dx, out, ...) # > > multiply forcing by sin(t) > > t += dt > > tc.assign(t) > > > > This will generate a single compiled form where the current time is > > passed in via a pointer, rather than being hardcoded into the C code. > > > > Note: if you are repeatedly solving a problem like this, it's a bit > > better to use the LinearVariationalProblem/LinearVariationalSolver > > objects, as described at > > https://www.firedrakeproject.org/demos/DG_advection.py.html, but this > is > > ultimately separate to Constant vs hard-coded floating point numbers. > > > > On Thu, 14 Mar 2019 at 18:57, Ziad Boutanios <ziad.boutanios(a)gmail.com > > <mailto:ziad.boutanios@gmail.com>> wrote: > > > > Hi, > > > > I'm a new user running on my own machine, not the Imperial clusters. > > Just noticed that I have almost half a Gig of compiled code and > whatnot > > in my .cache generated in just a few hours of tutorials. I would be > > really interested in how to minimize this output. Could you please > > email > > the recommendations to this list? > > > > Thanks > > > > Ziad > > > > On 2019-03-14 11:21 a.m., Miklós Homolya wrote: > > > My crystal ball says that people don't know how to use Constant, > > so they > > > instead hard-code things that change at each iteration, and that > will > > > generate a lot of compiled code. (But it's just a guess...) > > > > > > "Cregan, Bob" <b.cregan(a)imperial.ac.uk > > <mailto:b.cregan@imperial.ac.uk>> writes: > > >> Hi > > >> > > >> We noticed that some firedrake users on the Imperial > > clusters have > > >> a large ( between 0.5 M and 1.3M) numbers of files in their cache > > >> (usually in firedrake/.cache/pyop2) . > > >> > > >> Is this cache size configurable? If so can you let people know > > how to > > >> limit it. It has two bad effects; firstly no filesystem is > > happy with > > >> that many files in a single directory and it will slow things > > down to > > >> the point it is useless as cache. Also each user is limited to > 10M > > >> files and a very large cache will effectively reduce their quota > to > > >> well below the 1TB default (they will run out of files before > > they run > > >> out of blocks). > > >> > > >> Thanks > > >> > > >> Bob > > >> > > >> > > >> > > >> Bob Cregan > > >> HPC Systems Analyst > > >> > > >> Information & Communication Technologies > > >> > > >> Imperial College London, > > >> South Kensington Campus London, SW7 2AZ > > >> T: 07712388129 > > >> E: b.cregan(a)imperial.ac.uk <mailto:b.cregan@imperial.ac.uk> > > >> > > >> W: www.imperial.ac.uk/ict/rcs > > <http://www.imperial.ac.uk/ict/rcs>< > http://www.imperial.ac.uk/ict/rcs> > > >> > > >> [1505984389175_twitter.png] @imperialRCS @imperialRSE > > >> > > >> [1505983882959_Imperial-RCS.png] > > >> > > >> _______________________________________________ > > >> firedrake mailing list > > >> firedrake(a)imperial.ac.uk <mailto:firedrake@imperial.ac.uk> > > >> https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > _______________________________________________ > > > firedrake mailing list > > > firedrake(a)imperial.ac.uk <mailto:firedrake@imperial.ac.uk> > > > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > > _______________________________________________ > > firedrake mailing list > > firedrake(a)imperial.ac.uk <mailto:firedrake@imperial.ac.uk> > > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > > > > _______________________________________________ > > firedrake mailing list > > firedrake(a)imperial.ac.uk > > https://mailman.ic.ac.uk/mailman/listinfo/firedrake > > > > > > _______________________________________________ > firedrake mailing list > firedrake(a)imperial.ac.uk > https://mailman.ic.ac.uk/mailman/listinfo/firedrake >

2 1

Re: [firedrake] local stuff in UFL
by Lawrence Mitchell 09 Mar '19

09 Mar '19

> On 28 Feb 2019, at 10:36, Cotter, Colin J <colin.cotter(a)imperial.ac.uk> wrote: > > Thanks! > > I also need to stop pullbacks from being applied, how do I do that? > > What we actually need is to be able to replace J uhat/det J with J P(uhat)/det J where > P is the projection onto a vector in local coordinates, probably it is easiest to just hand > write the whole expression as it should appear after pullback. I you write everything after pullback, you might be able to get away with editing tsfc/ufl_utils.py to set do_apply_function_pullbacks=False. I'm hope that will work. Lawrence

3 3

Firedrake in Physics World
by Ham, David A 08 Mar '19

08 Mar '19

Dear Firedrakers, A brief bit of blowing our own trumpet to end the week: Firedrake gets a mention in this Physics World article. https://physicsworld.com/a/the-third-pillar-of-science/ Have a nice weekend, David Dr David Ham Department of Mathematics Imperial College London

1 0

firedrake install not working
by Shipton, Jemma 07 Mar '19

07 Mar '19

Hi All, I've just tried to install firedrake on my laptop and it's failing with errors in libsupermesh: Failed to clone libsupermesh using ssh, falling back to https. Successfully cloned repository libsupermesh. Checking out branch master Successfully checked out branch master Traceback (most recent call last): File "firedrake-install", line 1173, in <module> build_and_install_libsupermesh() File "firedrake-install", line 823, in build_and_install_libsupermesh check_call(["make"]) File "firedrake-install", line 438, in check_call log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=env).decode()) File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['make']' returned non-zero exit status 2. I've attached the log file. Any suggestions much appreciated! Jemma

6 6

PVTU files
by Floriane Gidel [RPG] 06 Mar '19

06 Mar '19

Dear all, I consider a big mesh which is made of several subdomains (basically a grid, cf attached). My issue is that I do not want to define the subdomains from the big mesh but the contrary: merge all small meshes to obtain the big mesh (because I want all subdomains to be defined on the exact same mesh). My first question is therefore: is there a way to merge several small meshes to one big mesh ? If not, I would like to at least be able to open the solutions on Paraview from one unique file, to avoid opening and setting hundreds of files when I have hundreds of subdomains. I though that creating a pvtu file from all my vtu files should work, since solving my equations on each small mesh is similar to parallel programming on the big mesh. Can you advise me on how to create this pvtu file out of my vtu files (I attach an example of my vtu file on one small mesh)? Thank you very much, Floriane

2 1