-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 16/07/15 14:32, Justin Chang wrote:
Lawrence,
I have attached the code I am working with. It's basically the one you sent me a few weeks ago, but I am only working with selfp. Attached are the log files with 1, 2, and 4 processors on our local HPC machine (Intel Xeon E5-2680v2 2.8 GHz)
1) I wrapped the PyPAPI calls around solver.solve(). I guess this is doing what I want. Right now I am estimating the arithmetic intensity by documenting the FLOPS, Loads, and Stores. When i compare the measured FLOPS with the PETSc manual FLOP count it seems papi over counts by a factor of 2 (which I suppose is expected coming from a new Intel machine). Anyway, in terms of computing the FLOPS and AI this is what I want, I just wanted to make sure these don't account for the DMPlex initialization and stuff because:
So note that inside SNESSolve, petsc attributes zero flops to forming the residual and jacobian (since that's a user function that it knows nothing about). We could actually do a reasonable job of adding a PetscLogFlops call, since we can inspect the kernel and make a reasonable guess at the number of flops it does, but we don't currently do that. This may explain the difference in flop counts.
2) According to the attached log_summaries it seems DMPlexDistribute and MeshMigration still consume a significant portion of the time. By significant I mean that the %T doesn't reduce as I increase the number of processors. I remember seeing Michael Lange's presentations (from PETSc-20 and the webinar) that mentioned something about this?
Yes, for more details on what scales and doesn't, see this paper: http://arxiv.org/abs/1506.06194
3) Bonus question: how do I also use PAPI_flops(&real_time, &proc_time, &flpins, &mflops)? I see there's the flops() function, but in my limited PAPI experience, I seem to have issues whenever I try to put both that and PAPI_start_counters into the same program, but I could be wrong.
I'm by no means a PAPI expert, but can you not just obtain the result of PAPI_flops by measuring the PAPI_FP_INS counter? Cheers, Lawrence -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVp7XVAAoJECOc1kQ8PEYvrU0IALr8aXhyfb+uVkQuS07s8Wov hn1a8i2Fu1ERNk/0W8NYXNckY0g+HP0zfgPpo/vsVv4b4W3l3uqsCqvXzzUG8AH7 GYgKIfBTqr9d6OvFN2niZNnrogbbpsq1u6RxVzqYCQCKgXkJ++BaGStHsQIyg++M 8zFoJ97HWEUdEgcjsNvuugqf14M/2PfZnMFrJJghr7xf4W37w47Ya4bizzAH2NNh yIefY5DFldPmfBgbEfDGhjEUig8wkuwTinVo8NnXW4yXJsqqu+THgdncQagiQwDo ugV6oABIdkX/JNHt0spADarL3vX+lk7aMZ7Zyibj7L+65YXCJlhgv8vXc/KQyzM= =WY+c -----END PGP SIGNATURE-----