On 11 Oct 2014, at 15:16, Patrick Farrell <patrick.farrell@maths.ox.ac.uk> wrote:
I'm surprised that MATNEST doesn't make as much difference, I thought it would do more. It would be nice to see the memory usage too: I'm guessing that's where MATNEST would make a bigger difference. At scale (to billions of DOFs) I only run with 2/24 cores per node because of memory limitations, probably because of all the damn copies.
I'd expect those copies to be about the cost of a few matvecs in time, as you say, the memory problems are probably bigger. Note as well we haven't run really big problems (nothing like billions of dofs), so arguably we're not at "extreme scale" yet. ...
Do you guys run into problems with starting the Python interpreter on many cores? Chris Richardson's been doing some work on that, and has had some partial success with zipping the files the Python interpreter loads; if you've solved this problem, or ran into it, it would be good to mention it in the paper.
I'm pretty sure we don't have a solution to the problem. We've had other problems running at scale which have possibly been masking the python load problem (now fixed), so we haven't really been looking at it. But maybe Florian can comment otherwise. Lawrence