Bug in IncNavierStokes solver terminal output for SVV with MPI
******************* This email originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address. ******************* Hello everyone, I noticed some odd behavior of the IncNavierStokesSolvers terminal output regarding SVV when using MPI for Quasi-3D cases. When I try to use SVV in the spectral/hp planes only, i.e. without SVV along the homogeneous direction, the terminal output sometimes includes the SVV-Homo1D info, eventhough it's not specified under the SOLVERINFO tag in the session file! When I re-run the same case, it will randomly disappear as you can see here for two consecutive runs with identical settings: nektar@0d6fdc64e227:~/host_pwd/run$ mpirun -n 16 IncNavierStokesSolver mesh.xml session.xml ======================================================================= EquationType: UnsteadyNavierStokes Session Name: CylReD3900_quad_coarse2_Pgeo3 Spatial Dim.: 3 Max SEM Exp. Order: 4 Num. Processes: 16 Quasi-3D: Homogeneous in z-direction Expansion Dim.: 3 Num. Hom. Modes (z): 64 Hom. length (LZ): 3.14159 FFT Type: FFTW Projection Type: Continuous Galerkin Advect. advancement: explicit Diffuse. advancement: implicit Time Step: 0.004 No. of Steps: 1 Checkpoints (steps): 1 Integration Type: IMEX Splitting Scheme: Velocity correction (strong press. form) Dealiasing: Homogeneous1D + spectral/hp Smoothing-SpecHP: SVV (spectral/hp DG Kernel (diff coeff = 0.10000000000000001*Uh/p)) ======================================================================= nektar@0d6fdc64e227:~/host_pwd/run$ mpirun -n 16 IncNavierStokesSolver -f mesh.xml session.xml ======================================================================= EquationType: UnsteadyNavierStokes Session Name: CylReD3900_quad_coarse2_Pgeo3 Spatial Dim.: 3 Max SEM Exp. Order: 4 Num. Processes: 16 Quasi-3D: Homogeneous in z-direction Expansion Dim.: 3 Num. Hom. Modes (z): 64 Hom. length (LZ): 3.14159 FFT Type: FFTW Projection Type: Continuous Galerkin Advect. advancement: explicit Diffuse. advancement: implicit Time Step: 0.004 No. of Steps: 1 Checkpoints (steps): 1 Integration Type: IMEX Splitting Scheme: Velocity correction (strong press. form) Dealiasing: Homogeneous1D + spectral/hp Smoothing-SpecHP: SVV (spectral/hp DG Kernel (diff coeff = 0.10000000000000001*Uh/p)) Smoothing-Homo1D: SVV (Homogeneous1D - Exp Kernel(cut-off = 0.75, diff coeff = 0.10000000000000001))======================================================================= The first run does not output the "Smoothing-Homo1D" info, while the second one includes it, which makes it impossible to know whether it is activated or not. It'll use the SVV parameters that are set for the spectral/hp SVV if no parameters for the Homo1D-SVV are specified under the PARAMETERS tag. Otherwise the values will be overwritten, eventhough - as I said before - the feature is turned off under the SOLVERINFO tag. Running on one CPU shows the expected behavior. So it would be interesting to know whether this only concerns the terminal output or if the feature really is activated despite not being defined by the user. All the best Alex Sicher versendet mit [Proton Mail](https://proton.me/).
Hi Alex, I have had a quick look, the output you are reporting is generated by a call to EquationSystem::PrintSummary which then calls v_GenerateSummary. The function is only called if rank == 0. So, I am not sure why you would see different behaviour for MPI or serial execution. If you are keen to take a look yourself. The parent function is in library/SolverUtils/EquationSystem.cpp::GenerateSummary and for the VelocityCorrectionScheme you would have virtual calls from child to parent in the files: * solvers/IncNavierStokesSolver/VelocityCorrectionScheme.cpp * (library/SolverUtils/AdvectionSystem.cpp) - no definition here * library/SolverUtils/UnsteadySystem.cpp * library/SolverUtils/EquationSystem.cpp Could you share the session file for your case? (without the .xml ending as outlook filters those) Best wishes, Henrik ________________________________ Von: nektar-users-bounces@imperial.ac.uk <nektar-users-bounces@imperial.ac.uk> im Auftrag von Alexander Schukmann <alexander.schukmann@protonmail.com> Gesendet: Dienstag, 27. August 2024 18:02 An: nektar-users@imperial.ac.uk <nektar-users@imperial.ac.uk> Betreff: [Nektar-users] Bug in IncNavierStokes solver terminal output for SVV with MPI This email from alexander.schukmann@protonmail.com originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list<https://spam.ic.ac.uk/SpamConsole/Senders.aspx> to disable email stamping for this address. Hello everyone, I noticed some odd behavior of the IncNavierStokesSolvers terminal output regarding SVV when using MPI for Quasi-3D cases. When I try to use SVV in the spectral/hp planes only, i.e. without SVV along the homogeneous direction, the terminal output sometimes includes the SVV-Homo1D info, eventhough it's not specified under the SOLVERINFO tag in the session file! When I re-run the same case, it will randomly disappear as you can see here for two consecutive runs with identical settings: nektar@0d6fdc64e227:~/host_pwd/run$ mpirun -n 16 IncNavierStokesSolver mesh.xml session.xml ======================================================================= EquationType: UnsteadyNavierStokes Session Name: CylReD3900_quad_coarse2_Pgeo3 Spatial Dim.: 3 Max SEM Exp. Order: 4 Num. Processes: 16 Quasi-3D: Homogeneous in z-direction Expansion Dim.: 3 Num. Hom. Modes (z): 64 Hom. length (LZ): 3.14159 FFT Type: FFTW Projection Type: Continuous Galerkin Advect. advancement: explicit Diffuse. advancement: implicit Time Step: 0.004 No. of Steps: 1 Checkpoints (steps): 1 Integration Type: IMEX Splitting Scheme: Velocity correction (strong press. form) Dealiasing: Homogeneous1D + spectral/hp Smoothing-SpecHP: SVV (spectral/hp DG Kernel (diff coeff = 0.10000000000000001*Uh/p)) ======================================================================= nektar@0d6fdc64e227:~/host_pwd/run$ mpirun -n 16 IncNavierStokesSolver -f mesh.xml session.xml ======================================================================= EquationType: UnsteadyNavierStokes Session Name: CylReD3900_quad_coarse2_Pgeo3 Spatial Dim.: 3 Max SEM Exp. Order: 4 Num. Processes: 16 Quasi-3D: Homogeneous in z-direction Expansion Dim.: 3 Num. Hom. Modes (z): 64 Hom. length (LZ): 3.14159 FFT Type: FFTW Projection Type: Continuous Galerkin Advect. advancement: explicit Diffuse. advancement: implicit Time Step: 0.004 No. of Steps: 1 Checkpoints (steps): 1 Integration Type: IMEX Splitting Scheme: Velocity correction (strong press. form) Dealiasing: Homogeneous1D + spectral/hp Smoothing-SpecHP: SVV (spectral/hp DG Kernel (diff coeff = 0.10000000000000001*Uh/p)) Smoothing-Homo1D: SVV (Homogeneous1D - Exp Kernel(cut-off = 0.75, diff coeff = 0.10000000000000001)) ======================================================================= The first run does not output the "Smoothing-Homo1D" info, while the second one includes it, which makes it impossible to know whether it is activated or not. It'll use the SVV parameters that are set for the spectral/hp SVV if no parameters for the Homo1D-SVV are specified under the PARAMETERS tag. Otherwise the values will be overwritten, eventhough - as I said before - the feature is turned off under the SOLVERINFO tag. Running on one CPU shows the expected behavior. So it would be interesting to know whether this only concerns the terminal output or if the feature really is activated despite not being defined by the user. All the best Alex Sicher versendet mit Proton Mail<https://proton.me/>.
Hi Alex, It would also be helpful if you can provide the session files (please compressed them in a zip file as outlook prevent downloading attached file with a .xml extension). Cheers, Jacques ________________________________ From: nektar-users-bounces@imperial.ac.uk <nektar-users-bounces@imperial.ac.uk> on behalf of Henrik Wüstenberg <henrik.wuestenberg@hotmail.de> Sent: 30 August 2024 09:41 To: Alexander Schukmann <alexander.schukmann@protonmail.com>; nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Bug in IncNavierStokes solver terminal output for SVV with MPI Hi Alex, I have had a quick look, the output you are reporting is generated by a call to EquationSystem::PrintSummary which then calls v_GenerateSummary. The function is only called if rank == 0. So, I am not sure why you would see different behaviour for MPI or serial execution. If you are keen to take a look yourself. The parent function is in library/SolverUtils/EquationSystem.cpp::GenerateSummary and for the VelocityCorrectionScheme you would have virtual calls from child to parent in the files: * solvers/IncNavierStokesSolver/VelocityCorrectionScheme.cpp * (library/SolverUtils/AdvectionSystem.cpp) - no definition here * library/SolverUtils/UnsteadySystem.cpp * library/SolverUtils/EquationSystem.cpp Could you share the session file for your case? (without the .xml ending as outlook filters those) Best wishes, Henrik ________________________________ Von: nektar-users-bounces@imperial.ac.uk <nektar-users-bounces@imperial.ac.uk> im Auftrag von Alexander Schukmann <alexander.schukmann@protonmail.com> Gesendet: Dienstag, 27. August 2024 18:02 An: nektar-users@imperial.ac.uk <nektar-users@imperial.ac.uk> Betreff: [Nektar-users] Bug in IncNavierStokes solver terminal output for SVV with MPI This email from alexander.schukmann@protonmail.com originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list<https://spam.ic.ac.uk/SpamConsole/Senders.aspx> to disable email stamping for this address. Hello everyone, I noticed some odd behavior of the IncNavierStokesSolvers terminal output regarding SVV when using MPI for Quasi-3D cases. When I try to use SVV in the spectral/hp planes only, i.e. without SVV along the homogeneous direction, the terminal output sometimes includes the SVV-Homo1D info, eventhough it's not specified under the SOLVERINFO tag in the session file! When I re-run the same case, it will randomly disappear as you can see here for two consecutive runs with identical settings: nektar@0d6fdc64e227:~/host_pwd/run$ mpirun -n 16 IncNavierStokesSolver mesh.xml session.xml ======================================================================= EquationType: UnsteadyNavierStokes Session Name: CylReD3900_quad_coarse2_Pgeo3 Spatial Dim.: 3 Max SEM Exp. Order: 4 Num. Processes: 16 Quasi-3D: Homogeneous in z-direction Expansion Dim.: 3 Num. Hom. Modes (z): 64 Hom. length (LZ): 3.14159 FFT Type: FFTW Projection Type: Continuous Galerkin Advect. advancement: explicit Diffuse. advancement: implicit Time Step: 0.004 No. of Steps: 1 Checkpoints (steps): 1 Integration Type: IMEX Splitting Scheme: Velocity correction (strong press. form) Dealiasing: Homogeneous1D + spectral/hp Smoothing-SpecHP: SVV (spectral/hp DG Kernel (diff coeff = 0.10000000000000001*Uh/p)) ======================================================================= nektar@0d6fdc64e227:~/host_pwd/run$ mpirun -n 16 IncNavierStokesSolver -f mesh.xml session.xml ======================================================================= EquationType: UnsteadyNavierStokes Session Name: CylReD3900_quad_coarse2_Pgeo3 Spatial Dim.: 3 Max SEM Exp. Order: 4 Num. Processes: 16 Quasi-3D: Homogeneous in z-direction Expansion Dim.: 3 Num. Hom. Modes (z): 64 Hom. length (LZ): 3.14159 FFT Type: FFTW Projection Type: Continuous Galerkin Advect. advancement: explicit Diffuse. advancement: implicit Time Step: 0.004 No. of Steps: 1 Checkpoints (steps): 1 Integration Type: IMEX Splitting Scheme: Velocity correction (strong press. form) Dealiasing: Homogeneous1D + spectral/hp Smoothing-SpecHP: SVV (spectral/hp DG Kernel (diff coeff = 0.10000000000000001*Uh/p)) Smoothing-Homo1D: SVV (Homogeneous1D - Exp Kernel(cut-off = 0.75, diff coeff = 0.10000000000000001)) ======================================================================= The first run does not output the "Smoothing-Homo1D" info, while the second one includes it, which makes it impossible to know whether it is activated or not. It'll use the SVV parameters that are set for the spectral/hp SVV if no parameters for the Homo1D-SVV are specified under the PARAMETERS tag. Otherwise the values will be overwritten, eventhough - as I said before - the feature is turned off under the SOLVERINFO tag. Running on one CPU shows the expected behavior. So it would be interesting to know whether this only concerns the terminal output or if the feature really is activated despite not being defined by the user. All the best Alex Sicher versendet mit Proton Mail<https://proton.me/>.
participants (3)
-
Alexander Schukmann
-
Henrik Wüstenberg
-
Xing, Jacques