Hi Ed, Allow me to be more specific. One part of my code which I would like to do only once is a part where I call on the separate module Sympy to calculate a few derivatives and then send the result to Firedrake to process. Another part is when I calculate the total energy of the system - but for this I think I can do what you said in your example and only write to the .txt file when mesh.comm.rank == 0. But how do you think I could do the Sympy thing? Basically I need to compute everything in Sympy and then convert it into UFL code and then interpolate it - this already works well in in serial. This is really a minor setback if anything though, because usually it takes less than 20 seconds to do this part on any given core. Andrew From: Ed Bueler <elbueler@alaska.edu> Sent: Monday, April 5, 2021 6:54 PM To: Andrew Hicks <ahick17@lsu.edu> Cc: firedrake <firedrake@imperial.ac.uk> Subject: Re: [firedrake] Parallelism question Andrew --
My question is, am I using parallelism correctly when I write Paraview files? Because when I create a new File object and run the .write() method, for each time step it seems to create a number of VTU files equal to the number of cores used, as well as a PVTU file. Is this correct?
Yes, that is expected. One .vtu per process. A single .pvtu and a single .pvd. Open the .pvd with Paraview.
My other question is, is there a way to tell Python to only use the first core when doing other tasks? For example, I have a Python script in my code which writes a simple text file. What I really don't want is for Python to write to this text file 22 times when it only needs to do it once.
It depends on what data you want to write to the file. There is no magic "do this task over all data, and save the result from rank 0" command in general. One generally needs to decide on and explicitly communicate data from the processes to rank 0. (E.g. PETSc VecScatter: https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PetscSF/VecScatterCreateToZero.html<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mcs.anl.gov%2Fpetsc%2Fpetsc-current%2Fdocs%2Fmanualpages%2FPetscSF%2FVecScatterCreateToZero.html&data=04%7C01%7Cahick17%40lsu.edu%7C839b7e64497d4cdcf3c308d8f88e0f42%7C2d4dad3f50ae47d983a09ae2b1f466f8%7C0%7C0%7C637532636325978191%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=htAsXNzc4PNplmTKTKaVO53JvaaIJQCOc1Ck5hPa6nQ%3D&reserved=0>.) But for printing something like a scalar norm, which uses an all-to-all reduction already, you can just print on rank 0: from firedrake import * mesh = UnitSquareMesh(9, 9) W = FunctionSpace(mesh, 'Lagrange', degree=1) u = Function(W) # do stuff to u normu = sqrt(assemble(dot(u, u) * dx)) # every process knows normu if mesh.comm.rank == 0: print('the norm is %f' % normu) You may want to be more specific about what you want to write. Is it a vector, e.g. a solution, or something which is already stored on every processor? Ed On Mon, Apr 5, 2021 at 2:27 PM Andrew Hicks <ahick17@lsu.edu<mailto:ahick17@lsu.edu>> wrote: Dear all, I am testing out using parallelism with Firedrake now that I have access to a computer with multiple cores. I have successfully used parallelism on a simple Laplace problem in 3D, and have also read the tutorial on using PETSc's Sys.Print() function to print output and using the "with" statement to compute the L-infinity norm. My question is, am I using parallelism correctly when I write Paraview files? Because when I create a new File object and run the .write() method, for each time step it seems to create a number of VTU files equal to the number of cores used, as well as a PVTU file. Is this correct? My other question is, is there a way to tell Python to only use the first core when doing other tasks? For example, I have a Python script in my code which writes a simple text file. What I really don't want is for Python to write to this text file 22 times when it only needs to do it once. Thanks for the help, Andrew Hicks _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.ic.ac.uk%2Fmailman%2Flistinfo%2Ffiredrake&data=04%7C01%7Cahick17%40lsu.edu%7C839b7e64497d4cdcf3c308d8f88e0f42%7C2d4dad3f50ae47d983a09ae2b1f466f8%7C0%7C0%7C637532636325988142%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=FAxHazSgt4T%2BvYk%2BomKKnSYNOq2WEVoGByGKLJoKlaE%3D&reserved=0> -- Ed Bueler Dept of Mathematics and Statistics University of Alaska Fairbanks Fairbanks, AK 99775-6660 306C Chapman