Dear Koki, Thanks a lot. Your suggestion did indeed speed up the simulation drastically. I also have not faced any problems after incorporating gc.collect() -- Abhishek Venketeswaran NETL Research Associate - ORISE National Energy Technology Laboratory Department of Energy abhishek.venketeswaran@netl.doe.gov Work: 412-386-4833 Mobile: 716-507-7890 www.netl.doe.gov<http://www.netl.doe.gov> [cid:8b48acc8-c39d-4249-9c43-7eea298696db] ________________________________ From: Sagiyama, Koki <k.sagiyama@imperial.ac.uk> Sent: Friday, January 21, 2022 12:05 PM To: Venketeswaran, Abhishek (FN) <Abhishek.Venketeswaran@netl.doe.gov>; firedrake <firedrake@imperial.ac.uk> Subject: [EXTERNAL] Re: Elastic Wave Propagation in Pipe Simulation Failing Dear Abhishek, The issue you found is actually from petsc4py and Jack has been working to resolve it: please see https://gitlab.com/petsc/petsc/-/merge_requests/4619 for problem description. In PipeExptrac.py, you call `solve()` function inside the time loop. This not only makes your code slow but also causes the above issue to happen more easily. So I suggest that you construct the solver object once before the time loop as: ``` problem = LinearVariationalProblem(a, r, u_np1, bcs=[bc], constant_jacobian=True) solver = LinearVariationalSolver(problem, solver_parameters=params) ``` and replace the `solve(a==r,...)` with `solver.solve()` in the time loop. See also: https://nbviewer.org/github/firedrakeproject/firedrake/blob/master/docs/note... https://www.firedrakeproject.org/firedrake.html#firedrake.variational_solver... Thanks, Koki ________________________________ From: Venketeswaran, Abhishek (FN) <Abhishek.Venketeswaran@netl.doe.gov> Sent: Friday, January 21, 2022 3:58 PM To: Sagiyama, Koki <k.sagiyama@imperial.ac.uk>; firedrake <firedrake@imperial.ac.uk> Subject: Re: Elastic Wave Propagation in Pipe Simulation Failing This email from Abhishek.Venketeswaran@netl.doe.gov originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list<https://spam.ic.ac.uk/SpamConsole/Senders.aspx> to disable email stamping for this address. Dear Koki AS of now I was able to run the elastic wave propagation simulation for longer time duration and using multiple nodes by adding the gc.disable() at the start and gc.collect() after every solve. Is this a robust solution? Could it trigger some other problems in the future for this particular simulation? I am trying out the same solution for the KMV lumped mass elements. I will reach out to you if I run into some trouble. -- Abhishek Venketeswaran NETL Research Associate - ORISE National Energy Technology Laboratory Department of Energy abhishek.venketeswaran@netl.doe.gov Work: 412-386-4833 Mobile: 716-507-7890 www.netl.doe.gov<http://www.netl.doe.gov> [cid:5b3f872a-04c7-43a1-af8b-f652d61152c5] ________________________________ From: Venketeswaran, Abhishek (FN) <Abhishek.Venketeswaran@netl.doe.gov> Sent: Wednesday, January 19, 2022 12:22 PM To: Sagiyama, Koki <k.sagiyama@imperial.ac.uk>; firedrake <firedrake@imperial.ac.uk> Subject: Re: Elastic Wave Propagation in Pipe Simulation Failing Dear Koki Reducing the time step merely delays the problem. Petsc runs into error (error code 55: Out of memory) after some time steps are executed. Please find attached the revised code. I have reduced the time step to 10 ns and try to run the simulation for around 30000 time steps. I would really appreciate your help on this. The simulation essentially should trigger elastic waves that travel along the pipe. That does not seem to happen with the results obtained for the converged time steps. P.S. I tried to provide an initial value for the solver as the solution from the previous time step. That also delays the convergence issue but does not solve the problem. -- Abhishek Venketeswaran NETL Research Associate - ORISE National Energy Technology Laboratory Department of Energy abhishek.venketeswaran@netl.doe.gov Work: 412-386-4833 Mobile: 716-507-7890 www.netl.doe.gov<http://www.netl.doe.gov> [cid:7a3634a2-bfea-4c78-95a5-9c0f0c62684e] ________________________________ From: Sagiyama, Koki <k.sagiyama@imperial.ac.uk> Sent: Friday, January 14, 2022 6:01 AM To: Venketeswaran, Abhishek (FN) <Abhishek.Venketeswaran@netl.doe.gov>; firedrake <firedrake@imperial.ac.uk> Subject: [EXTERNAL] Re: Elastic Wave Propagation in Pipe Simulation Failing Hi Abhishek, I think the explicit time integrator (central difference for acceleration + evaluation at $u_n$ for stress) requires a small enough timestep for stability. If I run PipeKMVExptrac.py with mus=1e-6 and Nsteps = 40, I see the solution diverge, but if I use mus=5e-7 and Nsteps = 80, I do not. Please let us know if it does not resolve the issue. Thanks, Koki ________________________________ From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Venketeswaran, Abhishek (FN) <Abhishek.Venketeswaran@netl.doe.gov> Sent: Wednesday, January 12, 2022 10:31 PM To: firedrake <firedrake@imperial.ac.uk> Subject: [firedrake] Elastic Wave Propagation in Pipe Simulation Failing Hi, I have spent considerable time trying to debug my code and am not able to find the bug in the code. I am essentially trying to simulate 3D elastic wave propagation in a pipe fixed on both ends and having an ultrasonic collar triggering ultrasonic impulse on a portion of the outer pipe segment. I would ideally like to use the KMV elements (similar to the spyro package). I generated a tetrahedral mesh of the pipe using GMsh and labeled the ends of the pipe using tag "1" and the collar segment as "2". Next I wrote the UFL expressions for 2nd order central difference for acceleration term and the elasticity as well as traction terms. The code runs but the displacement at certain nodes diverges to infinity and results in nonsensical results. I have tried multiple things including 1. specifying the collar load as displacement as well as traction 2. trying out KMV and CG element 3. trying out different KSP methods and preconditioners 4. Formulating the problem using semi-implicit HHT as well as second order explicit difference method 5. Ensuring the Gamma value of the tetrahedral mesh is atleast 0.2 I would really appreciate it if someone could help me out with this. -- Abhishek Venketeswaran NETL Research Associate - ORISE National Energy Technology Laboratory Department of Energy abhishek.venketeswaran@netl.doe.gov Work: 412-386-4833 Mobile: 716-507-7890 www.netl.doe.gov<http://www.netl.doe.gov> [cid:abfe3d17-ce71-45f1-8175-8e0cd802eaee] ******************************************************************** This message does not originate from a known Department of Energy email system. Use caution if this message contains attachments, links or requests for information. ******************************************************************** ******************************************************************** This message does not originate from a known Department of Energy email system. Use caution if this message contains attachments, links or requests for information. ********************************************************************
