Hi Simon
            2k is fine. It is when you get to the many 10s of K that things start to go awry.

Bob


Bob Cregan
HPC Systems Analyst

Information & Communication Technologies

Imperial College London, 
South Kensington Campus London, SW7 2AZ
T: 07712388129
E: b.cregan@imperial.ac.uk

W: www.imperial.ac.uk/ict/rcs

1505984389175_twitter.png @imperialRCS @imperialRSE

1505983882959_Imperial-RCS.png



From: Warder, Simon C
Sent: 20 March 2019 11:22
To: Cregan, Bob; firedrake
Subject: Re: [firedrake] Firedrake Cache files
 
Hello,

I am one such person. However, the guilty directory (with ~350k files) was in an old version of Firedrake, last used a year ago (and last updated even longer ago). For reasons I can't recall, I'd made a fresh Firedrake directory but kept the old one.

The new one has ~2k files in firedrake/.cache/pyop2. I'm a Thetis user, so I assume this is a reasonable number.

Simon

From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Cregan, Bob <b.cregan@imperial.ac.uk>
Sent: 19 March 2019 13:06
To: firedrake
Subject: Re: [firedrake] Firedrake Cache files
 
Hi,
      Sorry for the delay. I've emailed two people who are affected so they should get in touch with you fairly soon.


Bob


Bob Cregan
HPC Systems Analyst

Information & Communication Technologies

Imperial College London, 
South Kensington Campus London, SW7 2AZ
T: 07712388129
E: b.cregan@imperial.ac.uk

W: www.imperial.ac.uk/ict/rcs

1505984389175_twitter.png @imperialRCS @imperialRSE

1505983882959_Imperial-RCS.png



From: Miklós Homolya <miklos.homolya@tum.de>
Sent: 14 March 2019 15:21
To: Cregan, Bob; firedrake
Subject: Re: [firedrake] Firedrake Cache files
 
My crystal ball says that people don't know how to use Constant, so they
instead hard-code things that change at each iteration, and that will
generate a lot of compiled code.  (But it's just a guess...)

"Cregan, Bob" <b.cregan@imperial.ac.uk> writes:
> Hi
>
>     We noticed that some firedrake users on the Imperial clusters have
> a large ( between 0.5 M and 1.3M) numbers of files in their cache
> (usually in firedrake/.cache/pyop2) .
>
> Is this cache size configurable? If so can you let people know how to
> limit it.  It has two bad effects; firstly no filesystem is happy with
> that many files in a single directory and it will slow things down to
> the point it is useless as cache.  Also each user is limited to 10M
> files and a very large cache will effectively reduce their quota to
> well below the 1TB default (they will run out of files before they run
> out of blocks).
>
> Thanks
>
> Bob
>
>
>
> Bob Cregan
> HPC Systems Analyst
>
> Information & Communication Technologies
>
> Imperial College London,
> South Kensington Campus London, SW7 2AZ
> T: 07712388129
> E: b.cregan@imperial.ac.uk
>
> W: www.imperial.ac.uk/ict/rcs<http://www.imperial.ac.uk/ict/rcs>
>
> [1505984389175_twitter.png] @imperialRCS @imperialRSE
>
> [1505983882959_Imperial-RCS.png]
>
> _______________________________________________
> firedrake mailing list
> firedrake@imperial.ac.uk
> https://mailman.ic.ac.uk/mailman/listinfo/firedrake