July 2023 - Nektar-users - imperial.ac.uk

Unable to Restart SImulation on Different Cluster
by Isaac Rosin 31 Jul '23

31 Jul '23

******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Hello Nektar, I am unable to restart a simulation after moving the case files to a new cluster. I began the simulation on one cluster and ran it to the end of the transient flow stage. Once finished, took the final .fld file, put it on another cluster and tried to run it from that time step. My expansion section is <EXPANSIONS> <F VAR="u,v,w,p" FILE="session.fld" /> </EXPANSIONS> and my initial conditions are <FUNCTION NAME="InitialConditions"> <F VAR="u,v,w,p" FILE="session.fld" /> </FUNCTION> When I try to start the simulation on the new cluster, I get the same CG iterations problem every time (see output file as text at the bottom of this email). The new cluster uses different CPUs, so I thought this could have something to do with it, but I am still getting the same problem when I try different CPUs. The only thing I tried which had any effect was changing the time step size. This shouldn't have been necessary because the one I was using was already keeping the CFL low (~0.5) during the transient flow. Decreasing CFL only lowered the size of the error on the CG iterations made... line displayed before the Level 0 assertion violation, and not by much. I would be grateful for your assistance. Regards, Isaac ======================================================================= EquationType: UnsteadyNavierStokes Session Name: session Spatial Dim.: 3 Max SEM Exp. Order: 5 Num. Processes: 208 Expansion Dim.: 3 Projection Type: Continuous Galerkin Advect. advancement: explicit Diffuse. advancement: implicit Time Step: 0.004 No. of Steps: 187500 Checkpoints (steps): 63 Integration Type: IMEX Splitting Scheme: Velocity correction (strong press. form) Dealiasing: spectral/hp Smoothing-SpecHP: SVV (spectral/hp DG Kernel (diff coeff = 1*Uh/p)) ======================================================================= Initial Conditions: - Field u: from file session.fld - Field v: from file session.fld - Field w: from file session.fld - Field p: from file session.fld CG iterations made = 5001 using tolerance of 1e-09 (error = 9.57894e-07, rhs_mag = 15.6227) Fatal : Level 0 assertion violation Exceeded maximum number of iterations -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec detected that one or more processes exited with non-zero status, thus causing the job to be terminated. The first process to do so was: Process name: [[2602,1],4] Exit code: 1 --------------------------------------------------------------------------

4 6

Manipulating .fld Files With Python
by Isaac Rosin 31 Jul '23

31 Jul '23

******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Hello Nektar, Where can I find the indexing convention for field data and how it translates to spatial coordinates, and do I even need this information for postprocessing .fld files with python? I would like to read my velocity field, dealiase it by the cell boundaries, and save it in a new .fld file which I can then convert to .dat or .plt. I had looked at NekPy for this but could only find information about preprocessing when I am interested in postprocessing. So far I have been using the PyTables library to read the field file and .nekg mesh file. I have gained some useful insight from this but I can see it taking a lot of time and effort to decode the indexing and there are several other issues I will encounter along the way before I have full control over the fields. Before I continue down this path, could anyone advise me if this is the best way to go about my postprocessing? I would like to continue using python if possible but am open to other suggestions. Also, how is the data arranged within cells when NUMMODES>2? Thank you, Isaac

1 0

Re: [Nektar-users] problem with changing time-step
by Ehsan Asgari 30 Jul '23

30 Jul '23

Hi Alexandra, I have attached the output log of the simulation with dt=0.00025 with verbosity. Do you spot anything unusual? I guess by error, you meant the one associated with the tolerance. Kind regards syavash On Fri, Jul 28, 2023 at 1:32 PM Alexandra Liosi <liosi.alex(a)gmail.com> wrote: > Hi Syavash, > > Thank you for sharing your log files with me. There are 2 things that I > noticed: > 1. In both cases, the element with ID 31453 has the largest CFL value, I > would check how the mesh looks around this element. > 2. I would recommend to run with verbosity activated to see the error from > the Conjugate Gradient solver. To activate the verbosity, you need to use > the -v option after your solver. For example, the command would look like > this: > IncNavierStokesSolver mesh.xml session.xml -v > results > Could repeat these tests with verbosity on to see what is happening with > the conjugate gradient solver ? > > Kind regards, > Alexandra > > On Fri, 28 Jul 2023 at 12:46, Alexandra Liosi <liosi.alex(a)gmail.com> > wrote: > >> >> >> ---------- Forwarded message --------- >> From: Ehsan Asgari <eh.asgari(a)gmail.com> >> Date: Fri, 28 Jul 2023 at 09:49 >> Subject: Re: [Nektar-users] problem with changing time-step >> To: Alexandra Liosi <liosi.alex(a)gmail.com> >> >> >> Hi Alexandra >> >> Following your comments, I made an attempt to compare two cases with >> different time steps, one with dt=0.0005 and the other with dt=0.00025. >> Both runs started from the same initial condition. >> I have attached the log files of both simulations. It is very strange, as >> the case with lower time-step is taking significantly more time to >> converge!! >> I really have no clue how this can be explained. I run simulations on a >> pure hex mesh with wall-resolved boundary layer. >> Do you have any suggestions? >> >> Kind regards >> syavash >> >> >> >> >> On Wed, Jul 26, 2023 at 10:10 PM Alexandra Liosi <liosi.alex(a)gmail.com> >> wrote: >> >>> Hi Syavash, >>> >>> Thank you for the explanation! >>> >>> Is there a reason why you choose to start from a higher CFL and then >>> reduce it? Also, where do you notice the oscillations? Would it be possible >>> for your to rerun the case even for a couple of time-steps to print the >>> error of the CG iterations? In my simulations using the incompressible >>> solver, I have noticed that the forces were oscillating because the >>> pressure and velocity tolerances for the conjugate gradient(CG) solver were >>> very relaxed. There could be other reasons as well, hence my list of >>> questions. >>> >>> Kind regards, >>> Alexandra >>> >>> On Wed, 26 Jul 2023 at 11:40, Ehsan Asgari <eh.asgari(a)gmail.com> wrote: >>> >>>> Hi Alexandra >>>> >>>> Thanks for your kind reply. >>>> I let the simulation run for a while with CFL=0.4 for instance. Then I >>>> modify the time-step size so that I have CFL=0.2 and restart the simulation >>>> from the latest field file. >>>> >>>> I use the following session file: >>>> >>>> <NEKTAR> >>>>> <EXPANSIONS> >>>>> <E COMPOSITE="C[30011]" NUMMODES="6" FIELDS="u,v,w" >>>>> TYPE="MODIFIED" /> >>>>> <E COMPOSITE="C[30011]" NUMMODES="6" FIELDS="p" TYPE="MODIFIED" /> >>>>> </EXPANSIONS> >>>>> <CONDITIONS> >>>>> <FUNCTION NAME="InitialConditions"> >>>>> <F VAR="u,v,w,p" FILE="step98_Cubic7_restartFieldHdf5.fld" >>>>> /> >>>>> </FUNCTION> >>>>> <SOLVERINFO> >>>>> >>>> VALUE="VelocityCorrectionScheme"/> >>>>> >>>> VALUE="UnsteadyNavierStokes" /> >>>>> >>>> /> >>>>> >>>> /> >>>>> >>>> /> >>>>> >>>>> >>>> VALUE="IterativeStaticCond" /> >>>>> >>>> /> >>>>> </SOLVERINFO> >>>>> <GLOBALSYSSOLNINFO> >>>>> <V VAR="u,v,w"> >>>>> >>>> VALUE="IterativeStaticCond" /> >>>>> >>>>> >>>>> </V> >>>>> <V VAR="p"> >>>>> >>>> VALUE="IterativeStaticCond" /> >>>>> >>>> VALUE="FullLinearSpaceWithLowEnergyBlock"/> >>>>> >>>>> </V> >>>>> </GLOBALSYSSOLNINFO> >>>> >>>> >>>> I don't have the log file as it was overwritten. >>>> >>>> Kind regards >>>> syavash >>>> >>>> On Wed, Jul 26, 2023 at 12:00 PM Alexandra Liosi <liosi.alex(a)gmail.com> >>>> wrote: >>>> >>>>> Hi Syavash, >>>>> >>>>> Thank you for your email. Are you using as an initialisation field the >>>>> one that you received with the higher time-step? How large are your >>>>> starting CFL and the CFL using the new time-step? Could you provide the log >>>>> file and the session file that you use? >>>>> >>>>> Kind regards, >>>>> Alexandra >>>>> >>>>> On Wed, 26 Jul 2023 at 05:25, Ehsan Asgari <eh.asgari(a)gmail.com> >>>>> wrote: >>>>> >>>>>> This email from eh.asgari(a)gmail.com originates from outside >>>>>> Imperial. Do not click on links and attachments unless you recognise the >>>>>> sender. If you trust the sender, add them to your safe senders list >>>>>> <https://spam.ic.ac.uk/SpamConsole/Senders.aspx> to disable email >>>>>> stamping for this address. >>>>>> >>>>>> >>>>>> Dear All, >>>>>> >>>>>> When I change the time-step size to reduce the CFL and restart the >>>>>> simulation, I get oscillations in both pressure and velocity that won't go >>>>>> away. >>>>>> Is there any explanation for this? >>>>>> >>>>>> Kind regards >>>>>> syavash >>>>>> _______________________________________________ >>>>>> Nektar-users mailing list >>>>>> Nektar-users(a)imperial.ac.uk >>>>>> https://mailman.ic.ac.uk/mailman/listinfo/nektar-users >>>>>> >>>>>

1 0

Re: [Nektar-users] problem with changing time-step
by Ehsan Asgari 28 Jul '23

28 Jul '23

Alexandra, I will do as you suggest. However I don't get what you mean by the error. I don't see any in the log files. Would you clarify that for me? Kind regards syavash On Fri, Jul 28, 2023, 13:32 Alexandra Liosi <liosi.alex(a)gmail.com> wrote: > Hi Syavash, > > Thank you for sharing your log files with me. There are 2 things that I > noticed: > 1. In both cases, the element with ID 31453 has the largest CFL value, I > would check how the mesh looks around this element. > 2. I would recommend to run with verbosity activated to see the error from > the Conjugate Gradient solver. To activate the verbosity, you need to use > the -v option after your solver. For example, the command would look like > this: > IncNavierStokesSolver mesh.xml session.xml -v > results > Could repeat these tests with verbosity on to see what is happening with > the conjugate gradient solver ? > > Kind regards, > Alexandra > > On Fri, 28 Jul 2023 at 12:46, Alexandra Liosi <liosi.alex(a)gmail.com> > wrote: > >> >> >> ---------- Forwarded message --------- >> From: Ehsan Asgari <eh.asgari(a)gmail.com> >> Date: Fri, 28 Jul 2023 at 09:49 >> Subject: Re: [Nektar-users] problem with changing time-step >> To: Alexandra Liosi <liosi.alex(a)gmail.com> >> >> >> Hi Alexandra >> >> Following your comments, I made an attempt to compare two cases with >> different time steps, one with dt=0.0005 and the other with dt=0.00025. >> Both runs started from the same initial condition. >> I have attached the log files of both simulations. It is very strange, as >> the case with lower time-step is taking significantly more time to >> converge!! >> I really have no clue how this can be explained. I run simulations on a >> pure hex mesh with wall-resolved boundary layer. >> Do you have any suggestions? >> >> Kind regards >> syavash >> >> >> >> >> On Wed, Jul 26, 2023 at 10:10 PM Alexandra Liosi <liosi.alex(a)gmail.com> >> wrote: >> >>> Hi Syavash, >>> >>> Thank you for the explanation! >>> >>> Is there a reason why you choose to start from a higher CFL and then >>> reduce it? Also, where do you notice the oscillations? Would it be possible >>> for your to rerun the case even for a couple of time-steps to print the >>> error of the CG iterations? In my simulations using the incompressible >>> solver, I have noticed that the forces were oscillating because the >>> pressure and velocity tolerances for the conjugate gradient(CG) solver were >>> very relaxed. There could be other reasons as well, hence my list of >>> questions. >>> >>> Kind regards, >>> Alexandra >>> >>> On Wed, 26 Jul 2023 at 11:40, Ehsan Asgari <eh.asgari(a)gmail.com> wrote: >>> >>>> Hi Alexandra >>>> >>>> Thanks for your kind reply. >>>> I let the simulation run for a while with CFL=0.4 for instance. Then I >>>> modify the time-step size so that I have CFL=0.2 and restart the simulation >>>> from the latest field file. >>>> >>>> I use the following session file: >>>> >>>> <NEKTAR> >>>>> <EXPANSIONS> >>>>> <E COMPOSITE="C[30011]" NUMMODES="6" FIELDS="u,v,w" >>>>> TYPE="MODIFIED" /> >>>>> <E COMPOSITE="C[30011]" NUMMODES="6" FIELDS="p" TYPE="MODIFIED" /> >>>>> </EXPANSIONS> >>>>> <CONDITIONS> >>>>> <FUNCTION NAME="InitialConditions"> >>>>> <F VAR="u,v,w,p" FILE="step98_Cubic7_restartFieldHdf5.fld" >>>>> /> >>>>> </FUNCTION> >>>>> <SOLVERINFO> >>>>> >>>> VALUE="VelocityCorrectionScheme"/> >>>>> >>>> VALUE="UnsteadyNavierStokes" /> >>>>> >>>> /> >>>>> >>>> /> >>>>> >>>> /> >>>>> >>>>> >>>> VALUE="IterativeStaticCond" /> >>>>> >>>> /> >>>>> </SOLVERINFO> >>>>> <GLOBALSYSSOLNINFO> >>>>> <V VAR="u,v,w"> >>>>> >>>> VALUE="IterativeStaticCond" /> >>>>> >>>>> >>>>> </V> >>>>> <V VAR="p"> >>>>> >>>> VALUE="IterativeStaticCond" /> >>>>> >>>> VALUE="FullLinearSpaceWithLowEnergyBlock"/> >>>>> >>>>> </V> >>>>> </GLOBALSYSSOLNINFO> >>>> >>>> >>>> I don't have the log file as it was overwritten. >>>> >>>> Kind regards >>>> syavash >>>> >>>> On Wed, Jul 26, 2023 at 12:00 PM Alexandra Liosi <liosi.alex(a)gmail.com> >>>> wrote: >>>> >>>>> Hi Syavash, >>>>> >>>>> Thank you for your email. Are you using as an initialisation field the >>>>> one that you received with the higher time-step? How large are your >>>>> starting CFL and the CFL using the new time-step? Could you provide the log >>>>> file and the session file that you use? >>>>> >>>>> Kind regards, >>>>> Alexandra >>>>> >>>>> On Wed, 26 Jul 2023 at 05:25, Ehsan Asgari <eh.asgari(a)gmail.com> >>>>> wrote: >>>>> >>>>>> This email from eh.asgari(a)gmail.com originates from outside >>>>>> Imperial. Do not click on links and attachments unless you recognise the >>>>>> sender. If you trust the sender, add them to your safe senders list >>>>>> <https://spam.ic.ac.uk/SpamConsole/Senders.aspx> to disable email >>>>>> stamping for this address. >>>>>> >>>>>> >>>>>> Dear All, >>>>>> >>>>>> When I change the time-step size to reduce the CFL and restart the >>>>>> simulation, I get oscillations in both pressure and velocity that won't go >>>>>> away. >>>>>> Is there any explanation for this? >>>>>> >>>>>> Kind regards >>>>>> syavash >>>>>> _______________________________________________ >>>>>> Nektar-users mailing list >>>>>> Nektar-users(a)imperial.ac.uk >>>>>> https://mailman.ic.ac.uk/mailman/listinfo/nektar-users >>>>>> >>>>>

1 0

problem with changing time-step
by Ehsan Asgari 26 Jul '23

26 Jul '23

******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Dear All, When I change the time-step size to reduce the CFL and restart the simulation, I get oscillations in both pressure and velocity that won't go away. Is there any explanation for this? Kind regards syavash

2 1

(no subject)
by Zhipeng Yu 23 Jul '23

23 Jul '23

******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Hello everyone, I'm solving the interaction system of an incompressible fluid flowing past a flexible moving bodies and encontoured some problems. I wander whether the SolverType of VCSMapping support the mixed CG-DG projection, such as BDFImplicitOrder2 in the TimeIntegrationMethod? Since I want to apply the Substepping scheme in my simultiaon, the code experiences error when the TimeIntegrationMethod is set as BDFImplicitOrder2. The solverinfo is shown as follows: Kind regards Zhipeng Yu

1 0

Use of equispacedoutput in Quasi-3D Simulations
by İlteber Özdemir 14 Jul '23

14 Jul '23

******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Hi, Hope everybody is having a nice summer vacation. But the simulations must go on! I am running a quasi-3D simulation. In my test case I have a tetra 2D mesh and 8 Fourier planes. So the elements are triangular prisms. As discussed in many earlier posts, use of equispacedoutput normally decreases the output file size. However, when I use equispacedoutput it not only interpolates my 2D mesh, but also creates triangles in z-direction. Thus, the final output is not comprised of triangular prisms but tetrahedrons. Although decreasing the file size of the 2D mesh, this increases the total file size, because I basically have double the amount of elements along z-direction compared to prior. With the hope to solve this problem, I tried tetonly flag. I am unsure if this is the correct option to use. I get the attached error message, when I run FieldConvert with this option: FieldConvert -f -v -m equispacedoutput:tetonly timeSteps/out.xml timeSteps/out_3.chk/ timeSteps/fcFiles/out_3.plt Here out.xml is my mesh, out_3.chk is the time step I want to convert, and out_3.plt is the name of the output. I know that FieldConvert fails specifically when I use tetonly, as it works when I don't use it. I would very much appreciate any help to solve this problem or a suggestion for a workaround. Thank you in advance for your help. With kind regards, Ilteber -- İlteber R. Özdemir

1 0