Problem 3D CompressibleFlowSolver
Hello, i need your feedback about the simulation of a 3 dimensional flow around a cylinder or a NACA-profile with the CompressibleFlowSolver of the NavierStokesCFE or the EulerCFE. In every case the simulation starts, but always ends at the point of "Writing: "file_0.chk" " due to time-limit of the Cluster (limited to 1 day). I tried several times. The parameters, Solver Info and so on were shown (see below). In 2D there is not such a problem. So my question is, if it depends on the calculation capacity (CPU etc.) or what the reason for this? Actually iam calculating on a Cluster with 10 nodes and 120 CPUs. I think, that the calculation normally should be faster and another time-step should be done after one day or is it normal to a 3D-simulation with the compressible solver? So please can you give me an advice about the Problem? For example the 3D-problem of the cylinder with the CompressibleFlowSolver with the NavierStokesCFE produces this output after one day of calculation: (The example with the cylinder is exact the same you published at this page: http://www.nektar.info/compressible-flow/ ) Parameters: ALPHA = 0 CINF = 294.827 CP = 1004.7 FINTIME = 2 GAMMA = 1.4 GASCONSTANT = 287.058 IO_CHECKSTEPS = 50000 IO_INFOSTEPS = 1 KAPPA = 0.1 L = 1 MACH = 0.1 MU = 0.00275172 MU0 = 0.5 NUMSTEPS = 2e+06 PINF = 22600 PR = 0.72 RE = 3900 RHOINF = 0.364 SKAPPA = -2.3 THERMALCONDUCTIVITY = 3.83981 TIMESTEP = 1e-06 TINF = 216.29 TWALL = 216.29 UINF = 29.4827 VINF = 0 WINF = 0 Solver Info: ADVECTIONTYPE = WeakDG DIFFUSIONTYPE = LDGNS DRIVER = Standard EQTYPE = NavierStokesCFE EVOLUTIONOPERATOR = Nonlinear GLOBALMATRIXSTORAGETYPE = SmvBSR GLOBALSYSSOLN = IterativeStaticCond LINEARPRECONSOLVER = Xxt LOCALMATRIXSTORAGESTRATEGY = Sparse PRECONDITIONER = Diagonal PROBLEMTYPE = General PROJECTION = DisContinuous TIMEINTEGRATIONMETHOD = RungeKutta2_ImprovedEuler UPWINDTYPE = ExactToro VISCOSITYTYPE = Variable gs_setup: 240784 unique labels shared pairwise times (avg, min, max): 0.000148432 0.000120687 0.000157881 crystal router : 0.000947791 0.000929117 0.000967693 used all_to_all method: pairwise handle bytes (avg, min, max): 241216 152740 313476 buffer bytes (avg, min, max): 160523 101632 208640 gs_setup: 1187072 unique labels shared pairwise times (avg, min, max): 0.000483757 0.000410008 0.000513387 crystal router : 0.0109781 0.0108864 0.01108 used all_to_all method: pairwise handle bytes (avg, min, max): 1.18751e+06 656164 1481220 buffer bytes (avg, min, max): 791381 437248 987136 gs_setup: 240784 unique labels shared pairwise times (avg, min, max): 0.000113803 9.51052e-05 0.000123191 crystal router : 0.000868803 0.000849199 0.000890398 used all_to_all method: pairwise handle bytes (avg, min, max): 241216 152740 313476 buffer bytes (avg, min, max): 160523 101632 208640 gs_setup: 1187072 unique labels shared pairwise times (avg, min, max): 0.000463862 0.000412512 0.000496101 crystal router : 0.00953256 0.00941629 0.0096302 used all_to_all method: pairwise handle bytes (avg, min, max): 1.18751e+06 656164 1481220 buffer bytes (avg, min, max): 791381 437248 987136 gs_setup: 240784 unique labels shared pairwise times (avg, min, max): 0.000115333 9.5892e-05 0.000125098 crystal router : 0.00325584 0.0032357 0.0032764 used all_to_all method: pairwise handle bytes (avg, min, max): 241216 152740 313476 buffer bytes (avg, min, max): 160523 101632 208640 gs_setup: 1187072 unique labels shared pairwise times (avg, min, max): 0.00103487 0.00097959 0.00105419 crystal router : 0.00855757 0.00837069 0.0108672 used all_to_all method: pairwise handle bytes (avg, min, max): 1.18751e+06 656164 1481220 buffer bytes (avg, min, max): 791381 437248 987136 gs_setup: 240784 unique labels shared pairwise times (avg, min, max): 0.000114435 8.49009e-05 0.000136399 crystal router : 0.00204957 0.0020185 0.00207429 used all_to_all method: pairwise handle bytes (avg, min, max): 241216 152740 313476 buffer bytes (avg, min, max): 160523 101632 208640 gs_setup: 1187072 unique labels shared pairwise times (avg, min, max): 0.000471148 0.0004215 0.000498414 crystal router : 0.00977638 0.0085747 0.0109815 used all_to_all method: pairwise handle bytes (avg, min, max): 1.18751e+06 656164 1481220 buffer bytes (avg, min, max): 791381 437248 987136 gs_setup: 240784 unique labels shared pairwise times (avg, min, max): 0.000113707 9.59873e-05 0.000121403 crystal router : 0.000872626 0.000852704 0.00089221 used all_to_all method: pairwise handle bytes (avg, min, max): 241216 152740 313476 buffer bytes (avg, min, max): 160523 101632 208640 gs_setup: 1187072 unique labels shared pairwise times (avg, min, max): 0.000467315 0.000390005 0.000493503 crystal router : 0.010846 0.0107322 0.0109444 used all_to_all method: pairwise handle bytes (avg, min, max): 1.18751e+06 656164 1481220 buffer bytes (avg, min, max): 791381 437248 987136 ======================================================================= EquationType: NavierStokesCFE Session Name: Cylinder_NS_3D Spatial Dim.: 3 Max SEM Exp. Order: 4 Expansion Dim.: 3 Riemann Solver: ExactToro Advection Type: Projection Type: Discontinuous Galerkin Diffusion Type: Advection: explicit AdvectionType: WeakDG Diffusion: explicit Time Step: 1e-06 No. of Steps: 2000000 Checkpoints (steps): 50000 Integration Type: RungeKutta2_ImprovedEuler Problem Type: General ======================================================================= Initial Conditions: - Field rho: rhoInf - Field rhou: rhoInf * uInf - Field rhov: rhoInf * vInf - Field rhow: rhoInf * wInf - Field E: pInf / (Gamma - 1) + 0.5 * rhoInf * (uInf * uInf + vInf * vInf + wInf * wInf) Writing: "Cylinder_NS_3D_0.chk" slurmstepd: *** JOB 930941 CANCELLED AT 2016-07-06T03:27:36 DUE TO TIME LIMIT *** I run the solver by: "mpirun -np 120 CompressibleFlowSolver -v 3D_cylinder_mesh.xml 3D_cylinder_conditions.xml" Thanks for any hint. If you need more information, please let me know . Best regards Fabian Selbach University of Siegen
Hi Fabian, Do you have some details regarding the mesh - e.g. number of element, etc.? Regards, Gianmarco Sent from my Samsung device -------- Original message -------- From: "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Date: 06/07/2016 17:18 (GMT+00:00) To: nektar-users <nektar-users@imperial.ac.uk> Subject: [Nektar-users] Problem 3D CompressibleFlowSolver Hello, i need your feedback about the simulation of a 3 dimensional flow around a cylinder or a NACA-profile with the CompressibleFlowSolver of the NavierStokesCFE or the EulerCFE. In every case the simulation starts, but always ends at the point of "Writing: "file_0.chk" " due to time-limit of the Cluster (limited to 1 day). I tried several times. The parameters, Solver Info and so on were shown (see below). In 2D there is not such a problem. So my question is, if it depends on the calculation capacity (CPU etc.) or what the reason for this? Actually iam calculating on a Cluster with 10 nodes and 120 CPUs. I think, that the calculation normally should be faster and another time-step should be done after one day or is it normal to a 3D-simulation with the compressible solver? So please can you give me an advice about the Problem? For example the 3D-problem of the cylinder with the CompressibleFlowSolver with the NavierStokesCFE produces this output after one day of calculation: (The example with the cylinder is exact the same you published at this page: http://www.nektar.info/compressible-flow/ ) Parameters: ALPHA = 0 CINF = 294.827 CP = 1004.7 FINTIME = 2 GAMMA = 1.4 GASCONSTANT = 287.058 IO_CHECKSTEPS = 50000 IO_INFOSTEPS = 1 KAPPA = 0.1 L = 1 MACH = 0.1 MU = 0.00275172 MU0 = 0.5 NUMSTEPS = 2e+06 PINF = 22600 PR = 0.72 RE = 3900 RHOINF = 0.364 SKAPPA = -2.3 THERMALCONDUCTIVITY = 3.83981 TIMESTEP = 1e-06 TINF = 216.29 TWALL = 216.29 UINF = 29.4827 VINF = 0 WINF = 0 Solver Info: ADVECTIONTYPE = WeakDG DIFFUSIONTYPE = LDGNS DRIVER = Standard EQTYPE = NavierStokesCFE EVOLUTIONOPERATOR = Nonlinear GLOBALMATRIXSTORAGETYPE = SmvBSR GLOBALSYSSOLN = IterativeStaticCond LINEARPRECONSOLVER = Xxt LOCALMATRIXSTORAGESTRATEGY = Sparse PRECONDITIONER = Diagonal PROBLEMTYPE = General PROJECTION = DisContinuous TIMEINTEGRATIONMETHOD = RungeKutta2_ImprovedEuler UPWINDTYPE = ExactToro VISCOSITYTYPE = Variable gs_setup: 240784 unique labels shared pairwise times (avg, min, max): 0.000148432 0.000120687 0.000157881 crystal router : 0.000947791 0.000929117 0.000967693 used all_to_all method: pairwise handle bytes (avg, min, max): 241216 152740 313476 buffer bytes (avg, min, max): 160523 101632 208640 gs_setup: 1187072 unique labels shared pairwise times (avg, min, max): 0.000483757 0.000410008 0.000513387 crystal router : 0.0109781 0.0108864 0.01108 used all_to_all method: pairwise handle bytes (avg, min, max): 1.18751e+06 656164 1481220 buffer bytes (avg, min, max): 791381 437248 987136 gs_setup: 240784 unique labels shared pairwise times (avg, min, max): 0.000113803 9.51052e-05 0.000123191 crystal router : 0.000868803 0.000849199 0.000890398 used all_to_all method: pairwise handle bytes (avg, min, max): 241216 152740 313476 buffer bytes (avg, min, max): 160523 101632 208640 gs_setup: 1187072 unique labels shared pairwise times (avg, min, max): 0.000463862 0.000412512 0.000496101 crystal router : 0.00953256 0.00941629 0.0096302 used all_to_all method: pairwise handle bytes (avg, min, max): 1.18751e+06 656164 1481220 buffer bytes (avg, min, max): 791381 437248 987136 gs_setup: 240784 unique labels shared pairwise times (avg, min, max): 0.000115333 9.5892e-05 0.000125098 crystal router : 0.00325584 0.0032357 0.0032764 used all_to_all method: pairwise handle bytes (avg, min, max): 241216 152740 313476 buffer bytes (avg, min, max): 160523 101632 208640 gs_setup: 1187072 unique labels shared pairwise times (avg, min, max): 0.00103487 0.00097959 0.00105419 crystal router : 0.00855757 0.00837069 0.0108672 used all_to_all method: pairwise handle bytes (avg, min, max): 1.18751e+06 656164 1481220 buffer bytes (avg, min, max): 791381 437248 987136 gs_setup: 240784 unique labels shared pairwise times (avg, min, max): 0.000114435 8.49009e-05 0.000136399 crystal router : 0.00204957 0.0020185 0.00207429 used all_to_all method: pairwise handle bytes (avg, min, max): 241216 152740 313476 buffer bytes (avg, min, max): 160523 101632 208640 gs_setup: 1187072 unique labels shared pairwise times (avg, min, max): 0.000471148 0.0004215 0.000498414 crystal router : 0.00977638 0.0085747 0.0109815 used all_to_all method: pairwise handle bytes (avg, min, max): 1.18751e+06 656164 1481220 buffer bytes (avg, min, max): 791381 437248 987136 gs_setup: 240784 unique labels shared pairwise times (avg, min, max): 0.000113707 9.59873e-05 0.000121403 crystal router : 0.000872626 0.000852704 0.00089221 used all_to_all method: pairwise handle bytes (avg, min, max): 241216 152740 313476 buffer bytes (avg, min, max): 160523 101632 208640 gs_setup: 1187072 unique labels shared pairwise times (avg, min, max): 0.000467315 0.000390005 0.000493503 crystal router : 0.010846 0.0107322 0.0109444 used all_to_all method: pairwise handle bytes (avg, min, max): 1.18751e+06 656164 1481220 buffer bytes (avg, min, max): 791381 437248 987136 ======================================================================= EquationType: NavierStokesCFE Session Name: Cylinder_NS_3D Spatial Dim.: 3 Max SEM Exp. Order: 4 Expansion Dim.: 3 Riemann Solver: ExactToro Advection Type: Projection Type: Discontinuous Galerkin Diffusion Type: Advection: explicit AdvectionType: WeakDG Diffusion: explicit Time Step: 1e-06 No. of Steps: 2000000 Checkpoints (steps): 50000 Integration Type: RungeKutta2_ImprovedEuler Problem Type: General ======================================================================= Initial Conditions: - Field rho: rhoInf - Field rhou: rhoInf * uInf - Field rhov: rhoInf * vInf - Field rhow: rhoInf * wInf - Field E: pInf / (Gamma - 1) + 0.5 * rhoInf * (uInf * uInf + vInf * vInf + wInf * wInf) Writing: "Cylinder_NS_3D_0.chk" slurmstepd: *** JOB 930941 CANCELLED AT 2016-07-06T03:27:36 DUE TO TIME LIMIT *** I run the solver by: "mpirun -np 120 CompressibleFlowSolver -v 3D_cylinder_mesh.xml 3D_cylinder_conditions.xml" Thanks for any hint. If you need more information, please let me know . Best regards Fabian Selbach University of Siegen _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
Hello Fabian, This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done. Cheers, Rodrigo .
Hi, I've been having an identical problem and would appreciate knowledge of the bypass On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote:
Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- *Amitvikram Dutta* *MASc Candidate* *Graduate Research Assistant * *Okanagan CFD Laboratory* *University of British Columbia | Okanagan Campus*
Hi guys, Yes, this is a known issue that we are trying to fix permanently! We have a temporary fix for it at the moment. Rodrigo should be able to help. Regards, Gianmarco Sent from my Samsung device -------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver Hi, I've been having an identical problem and would appreciate knowledge of the bypass On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk<mailto:r.moura13@imperial.ac.uk>> wrote: Hello Fabian, This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done. Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk<mailto:Nektar-users@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus
Hi all, I am guessing this is down to us using ExtractTracePhys() inside the boundary condition functions? I thought this had been patched out but it seems not. Cheers, Dave
On 6 Jul 2016, at 20:25, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi guys,
Yes, this is a known issue that we are trying to fix permanently!
We have a temporary fix for it at the moment. Rodrigo should be able to help.
Regards, Gianmarco
Sent from my Samsung device
-------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi,
I've been having an identical problem and would appreciate knowledge of the bypass
On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote: Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
Hi Dave, Yep, indeed, this is the case! We should finally patch it out! ;) Cheers, Gian Sent from my Samsung device -------- Original message -------- From: David Moxey <d.moxey@imperial.ac.uk> Date: 06/07/2016 20:32 (GMT+00:00) To: "Mengaldo, Gianmarco" <g.mengaldo11@imperial.ac.uk> Cc: Amitvikram Dutta <amitvdutta23@gmail.com>, "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de>, nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver Hi all, I am guessing this is down to us using ExtractTracePhys() inside the boundary condition functions? I thought this had been patched out but it seems not. Cheers, Dave
On 6 Jul 2016, at 20:25, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi guys,
Yes, this is a known issue that we are trying to fix permanently!
We have a temporary fix for it at the moment. Rodrigo should be able to help.
Regards, Gianmarco
Sent from my Samsung device
-------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi,
I've been having an identical problem and would appreciate knowledge of the bypass
On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote: Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey> Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
Hi Gian, In fact it seems I have a branch sat in my repository that fixes this but I apparently forgot to submit a merge request..! I'll tidy it up and submit a request shortly, and then perhaps Amitvikram and Fabian, you can test this for us? Cheers, Dave
On 6 Jul 2016, at 20:51, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi Dave,
Yep, indeed, this is the case! We should finally patch it out! ;)
Cheers, Gian
Sent from my Samsung device
-------- Original message -------- From: David Moxey <d.moxey@imperial.ac.uk> Date: 06/07/2016 20:32 (GMT+00:00) To: "Mengaldo, Gianmarco" <g.mengaldo11@imperial.ac.uk> Cc: Amitvikram Dutta <amitvdutta23@gmail.com>, "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de>, nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi all,
I am guessing this is down to us using ExtractTracePhys() inside the boundary condition functions? I thought this had been patched out but it seems not.
Cheers,
Dave
On 6 Jul 2016, at 20:25, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi guys,
Yes, this is a known issue that we are trying to fix permanently!
We have a temporary fix for it at the moment. Rodrigo should be able to help.
Regards, Gianmarco
Sent from my Samsung device
-------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi,
I've been having an identical problem and would appreciate knowledge of the bypass
On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote: Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
Sure! On Wed, Jul 6, 2016, 12:55 PM David Moxey <d.moxey@imperial.ac.uk> wrote:
Hi Gian,
In fact it seems I have a branch sat in my repository that fixes this but I apparently forgot to submit a merge request..!
I'll tidy it up and submit a request shortly, and then perhaps Amitvikram and Fabian, you can test this for us?
Cheers,
Dave
On 6 Jul 2016, at 20:51, Mengaldo, Gianmarco < g.mengaldo11@imperial.ac.uk> wrote:
Hi Dave,
Yep, indeed, this is the case! We should finally patch it out! ;)
Cheers, Gian
Sent from my Samsung device
-------- Original message -------- From: David Moxey <d.moxey@imperial.ac.uk> Date: 06/07/2016 20:32 (GMT+00:00) To: "Mengaldo, Gianmarco" <g.mengaldo11@imperial.ac.uk> Cc: Amitvikram Dutta <amitvdutta23@gmail.com>, "Moura, Rodrigo C" < r.moura13@imperial.ac.uk>, "Selbach, Fabian" < fabian.selbach@student.uni-siegen.de>, nektar-users < nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi all,
I am guessing this is down to us using ExtractTracePhys() inside the boundary condition functions? I thought this had been patched out but it seems not.
Cheers,
Dave
On 6 Jul 2016, at 20:25, Mengaldo, Gianmarco < g.mengaldo11@imperial.ac.uk> wrote:
Hi guys,
Yes, this is a known issue that we are trying to fix permanently!
We have a temporary fix for it at the moment. Rodrigo should be able to help.
Regards, Gianmarco
Sent from my Samsung device
-------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" < fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi,
I've been having an identical problem and would appreciate knowledge of the bypass
On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote: Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
--
*Amitvikram Dutta* *MASc Candidate* *Graduate Research Assistant * *Okanagan CFD Laboratory* *University of British Columbia | Okanagan Campus*
Hi all, Okay, I have now submitted a MR which should hopefully fix this issue: https://gitlab.nektar.info/nektar/nektar/merge_requests/657 To test it you'll need to check out the source, which is in my local fork. You can check this out using the following commands: git remote add dmoxey https://gitlab.nektar.info/dmoxey/nektar.git git fetch dmoxey git checkout -t dmoxey/fix/cfs-dgopt Cheers, Dave
On 6 Jul 2016, at 21:21, Amitvikram Dutta <amitvdutta23@gmail.com> wrote:
Sure!
On Wed, Jul 6, 2016, 12:55 PM David Moxey <d.moxey@imperial.ac.uk> wrote: Hi Gian,
In fact it seems I have a branch sat in my repository that fixes this but I apparently forgot to submit a merge request..!
I'll tidy it up and submit a request shortly, and then perhaps Amitvikram and Fabian, you can test this for us?
Cheers,
Dave
On 6 Jul 2016, at 20:51, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi Dave,
Yep, indeed, this is the case! We should finally patch it out! ;)
Cheers, Gian
Sent from my Samsung device
-------- Original message -------- From: David Moxey <d.moxey@imperial.ac.uk> Date: 06/07/2016 20:32 (GMT+00:00) To: "Mengaldo, Gianmarco" <g.mengaldo11@imperial.ac.uk> Cc: Amitvikram Dutta <amitvdutta23@gmail.com>, "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de>, nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi all,
I am guessing this is down to us using ExtractTracePhys() inside the boundary condition functions? I thought this had been patched out but it seems not.
Cheers,
Dave
On 6 Jul 2016, at 20:25, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi guys,
Yes, this is a known issue that we are trying to fix permanently!
We have a temporary fix for it at the moment. Rodrigo should be able to help.
Regards, Gianmarco
Sent from my Samsung device
-------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi,
I've been having an identical problem and would appreciate knowledge of the bypass
On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote: Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
Hi Dave, this sounds good, thanks! As soon as Amitvikram and Fabian confirm that everything works fine, we can proceed to merge it. Cheers, Gian Gianmarco Mengaldo Department of Aeronautics and Mathematics Imperial College London SW7 2AZ, London, UK *Currently at ECMWF ________________________________ From: David Moxey <d.moxey@imperial.ac.uk> Sent: 06 July 2016 21:36:59 To: Amitvikram Dutta Cc: Mengaldo, Gianmarco; Moura, Rodrigo C; Selbach, Fabian; nektar-users Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver Hi all, Okay, I have now submitted a MR which should hopefully fix this issue: https://gitlab.nektar.info/nektar/nektar/merge_requests/657 To test it you'll need to check out the source, which is in my local fork. You can check this out using the following commands: git remote add dmoxey https://gitlab.nektar.info/dmoxey/nektar.git git fetch dmoxey git checkout -t dmoxey/fix/cfs-dgopt Cheers, Dave
On 6 Jul 2016, at 21:21, Amitvikram Dutta <amitvdutta23@gmail.com> wrote:
Sure!
On Wed, Jul 6, 2016, 12:55 PM David Moxey <d.moxey@imperial.ac.uk> wrote: Hi Gian,
In fact it seems I have a branch sat in my repository that fixes this but I apparently forgot to submit a merge request..!
I'll tidy it up and submit a request shortly, and then perhaps Amitvikram and Fabian, you can test this for us?
Cheers,
Dave
On 6 Jul 2016, at 20:51, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi Dave,
Yep, indeed, this is the case! We should finally patch it out! ;)
Cheers, Gian
Sent from my Samsung device
-------- Original message -------- From: David Moxey <d.moxey@imperial.ac.uk> Date: 06/07/2016 20:32 (GMT+00:00) To: "Mengaldo, Gianmarco" <g.mengaldo11@imperial.ac.uk> Cc: Amitvikram Dutta <amitvdutta23@gmail.com>, "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de>, nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi all,
I am guessing this is down to us using ExtractTracePhys() inside the boundary condition functions? I thought this had been patched out but it seems not.
Cheers,
Dave
On 6 Jul 2016, at 20:25, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi guys,
Yes, this is a known issue that we are trying to fix permanently!
We have a temporary fix for it at the moment. Rodrigo should be able to help.
Regards, Gianmarco
Sent from my Samsung device
-------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi,
I've been having an identical problem and would appreciate knowledge of the bypass
On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote: Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey> Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
Hello, thank you! Sure i will try it, but first i need to install a git repository version of Nektar++ on my computer. After that i will let you know about it. @Gianmarco: the mesh size is about 1800000 points. The expansion is shown in the code below. @Rodrigo: i could not try a serial-run right now, but i will do. Best regards Fabian ________________________________________ Von: Mengaldo, Gianmarco [g.mengaldo11@imperial.ac.uk] Gesendet: Donnerstag, 7. Juli 2016 11:05 An: Moxey, David; Amitvikram Dutta Cc: Moura, Rodrigo C; Selbach, Fabian; nektar-users Betreff: Re: [Nektar-users] Problem 3D CompressibleFlowSolver Hi Dave, this sounds good, thanks! As soon as Amitvikram and Fabian confirm that everything works fine, we can proceed to merge it. Cheers, Gian Gianmarco Mengaldo Department of Aeronautics and Mathematics Imperial College London SW7 2AZ, London, UK *Currently at ECMWF ________________________________ From: David Moxey <d.moxey@imperial.ac.uk> Sent: 06 July 2016 21:36:59 To: Amitvikram Dutta Cc: Mengaldo, Gianmarco; Moura, Rodrigo C; Selbach, Fabian; nektar-users Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver Hi all, Okay, I have now submitted a MR which should hopefully fix this issue: https://gitlab.nektar.info/nektar/nektar/merge_requests/657 To test it you'll need to check out the source, which is in my local fork. You can check this out using the following commands: git remote add dmoxey https://gitlab.nektar.info/dmoxey/nektar.git git fetch dmoxey git checkout -t dmoxey/fix/cfs-dgopt Cheers, Dave
On 6 Jul 2016, at 21:21, Amitvikram Dutta <amitvdutta23@gmail.com> wrote:
Sure!
On Wed, Jul 6, 2016, 12:55 PM David Moxey <d.moxey@imperial.ac.uk> wrote: Hi Gian,
In fact it seems I have a branch sat in my repository that fixes this but I apparently forgot to submit a merge request..!
I'll tidy it up and submit a request shortly, and then perhaps Amitvikram and Fabian, you can test this for us?
Cheers,
Dave
On 6 Jul 2016, at 20:51, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi Dave,
Yep, indeed, this is the case! We should finally patch it out! ;)
Cheers, Gian
Sent from my Samsung device
-------- Original message -------- From: David Moxey <d.moxey@imperial.ac.uk> Date: 06/07/2016 20:32 (GMT+00:00) To: "Mengaldo, Gianmarco" <g.mengaldo11@imperial.ac.uk> Cc: Amitvikram Dutta <amitvdutta23@gmail.com>, "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de>, nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi all,
I am guessing this is down to us using ExtractTracePhys() inside the boundary condition functions? I thought this had been patched out but it seems not.
Cheers,
Dave
On 6 Jul 2016, at 20:25, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi guys,
Yes, this is a known issue that we are trying to fix permanently!
We have a temporary fix for it at the moment. Rodrigo should be able to help.
Regards, Gianmarco
Sent from my Samsung device
-------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi,
I've been having an identical problem and would appreciate knowledge of the bypass
On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote: Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey> Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
Hello, sorry for my late response, but actually i tested a few things and still got some problems. @Rodrigo: I tested the serial-run. It works on our Cluster and my own computer in the 3D-case! By the way is the bug-fix by David you mentioned, when you wrote how to fix the problem? Or do you have another idea how to fix this bug? So it must be really a problem of parallelization due to openmpi in connection with Nektar++. After i tested the bug-fix (by David) on my computer, it runs in parallel-version! But actually i can't run it on our Cluster, because there is not installed an actually git-Version of Nektar++ with this bug-fix. So will there be a next Version of Nektar++ in the near future, which fixes this bug? I even realized, that the parallel-run is possible in 3D by the Nektar++ 4.3.2 (without the bug-fix) on my own Netbook with an old version of openmpi (version 1.6), but why? Best regards Fabian ________________________________________ Von: nektar-users-bounces@imperial.ac.uk [nektar-users-bounces@imperial.ac.uk]" im Auftrag von "Selbach, Fabian [fabian.selbach@student.uni-siegen.de] Gesendet: Donnerstag, 7. Juli 2016 12:04 An: Mengaldo, Gianmarco; Moxey, David; Amitvikram Dutta Cc: nektar-users Betreff: Re: [Nektar-users] Problem 3D CompressibleFlowSolver Hello, thank you! Sure i will try it, but first i need to install a git repository version of Nektar++ on my computer. After that i will let you know about it. @Gianmarco: the mesh size is about 1800000 points. The expansion is shown in the code below. @Rodrigo: i could not try a serial-run right now, but i will do. Best regards Fabian ________________________________________ Von: Mengaldo, Gianmarco [g.mengaldo11@imperial.ac.uk] Gesendet: Donnerstag, 7. Juli 2016 11:05 An: Moxey, David; Amitvikram Dutta Cc: Moura, Rodrigo C; Selbach, Fabian; nektar-users Betreff: Re: [Nektar-users] Problem 3D CompressibleFlowSolver Hi Dave, this sounds good, thanks! As soon as Amitvikram and Fabian confirm that everything works fine, we can proceed to merge it. Cheers, Gian Gianmarco Mengaldo Department of Aeronautics and Mathematics Imperial College London SW7 2AZ, London, UK *Currently at ECMWF ________________________________ From: David Moxey <d.moxey@imperial.ac.uk> Sent: 06 July 2016 21:36:59 To: Amitvikram Dutta Cc: Mengaldo, Gianmarco; Moura, Rodrigo C; Selbach, Fabian; nektar-users Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver Hi all, Okay, I have now submitted a MR which should hopefully fix this issue: https://gitlab.nektar.info/nektar/nektar/merge_requests/657 To test it you'll need to check out the source, which is in my local fork. You can check this out using the following commands: git remote add dmoxey https://gitlab.nektar.info/dmoxey/nektar.git git fetch dmoxey git checkout -t dmoxey/fix/cfs-dgopt Cheers, Dave
On 6 Jul 2016, at 21:21, Amitvikram Dutta <amitvdutta23@gmail.com> wrote:
Sure!
On Wed, Jul 6, 2016, 12:55 PM David Moxey <d.moxey@imperial.ac.uk> wrote: Hi Gian,
In fact it seems I have a branch sat in my repository that fixes this but I apparently forgot to submit a merge request..!
I'll tidy it up and submit a request shortly, and then perhaps Amitvikram and Fabian, you can test this for us?
Cheers,
Dave
On 6 Jul 2016, at 20:51, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi Dave,
Yep, indeed, this is the case! We should finally patch it out! ;)
Cheers, Gian
Sent from my Samsung device
-------- Original message -------- From: David Moxey <d.moxey@imperial.ac.uk> Date: 06/07/2016 20:32 (GMT+00:00) To: "Mengaldo, Gianmarco" <g.mengaldo11@imperial.ac.uk> Cc: Amitvikram Dutta <amitvdutta23@gmail.com>, "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de>, nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi all,
I am guessing this is down to us using ExtractTracePhys() inside the boundary condition functions? I thought this had been patched out but it seems not.
Cheers,
Dave
On 6 Jul 2016, at 20:25, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi guys,
Yes, this is a known issue that we are trying to fix permanently!
We have a temporary fix for it at the moment. Rodrigo should be able to help.
Regards, Gianmarco
Sent from my Samsung device
-------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi,
I've been having an identical problem and would appreciate knowledge of the bypass
On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote: Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey> Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK. _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
Hi Fabian, This is a bit of an irritating bug since, as you say, it _can_ work sometimes. Let me try to briefly explain the bug to give context and why you might be seeing this work fine on your laptop but not the cluster. It is basically due to the boundary conditions. Due to a recent optimisation we needed to add a small communication call into a routine we use to extract edge values from elements, which is used in the boundary conditions to calculate the necessary values for the Riemann solver. When running in parallel, we partition the mesh and the boundaries of the domain therefore lie on different processors. Depending on how many processors you use, some processors may have partitions that contain elements connected to the boundary, and others will have elements solely on the interior. This bug is due to us putting the call to that extraction function in the wrong place: inside the boundary condition calculation for each type of boundary condition. As long as you use sufficiently few processors, every partition will contain at least some elements connected to the boundary, and this communication will work without any problem. However past a certain point, you start to create partitions that don't have any elements connected to the boundary. They will happily carry on through the communication and cause the process to deadlock. Hope that explains the situation! I expect we'll be doing a patch release at the end of the month that will contain this fix. Cheers, Dave
On 11 Jul 2016, at 22:29, Selbach, Fabian <fabian.selbach@student.uni-siegen.de> wrote:
Hello,
sorry for my late response, but actually i tested a few things and still got some problems.
@Rodrigo: I tested the serial-run. It works on our Cluster and my own computer in the 3D-case! By the way is the bug-fix by David you mentioned, when you wrote how to fix the problem? Or do you have another idea how to fix this bug?
So it must be really a problem of parallelization due to openmpi in connection with Nektar++. After i tested the bug-fix (by David) on my computer, it runs in parallel-version! But actually i can't run it on our Cluster, because there is not installed an actually git-Version of Nektar++ with this bug-fix. So will there be a next Version of Nektar++ in the near future, which fixes this bug?
I even realized, that the parallel-run is possible in 3D by the Nektar++ 4.3.2 (without the bug-fix) on my own Netbook with an old version of openmpi (version 1.6), but why?
Best regards
Fabian ________________________________________ Von: nektar-users-bounces@imperial.ac.uk [nektar-users-bounces@imperial.ac.uk]" im Auftrag von "Selbach, Fabian [fabian.selbach@student.uni-siegen.de] Gesendet: Donnerstag, 7. Juli 2016 12:04 An: Mengaldo, Gianmarco; Moxey, David; Amitvikram Dutta Cc: nektar-users Betreff: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hello,
thank you! Sure i will try it, but first i need to install a git repository version of Nektar++ on my computer. After that i will let you know about it.
@Gianmarco: the mesh size is about 1800000 points. The expansion is shown in the code below. @Rodrigo: i could not try a serial-run right now, but i will do.
Best regards
Fabian
________________________________________ Von: Mengaldo, Gianmarco [g.mengaldo11@imperial.ac.uk] Gesendet: Donnerstag, 7. Juli 2016 11:05 An: Moxey, David; Amitvikram Dutta Cc: Moura, Rodrigo C; Selbach, Fabian; nektar-users Betreff: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi Dave,
this sounds good, thanks!
As soon as Amitvikram and Fabian confirm that everything works fine, we can proceed to merge it.
Cheers,
Gian
Gianmarco Mengaldo
Department of Aeronautics and Mathematics
Imperial College London
SW7 2AZ, London, UK
*Currently at ECMWF
________________________________ From: David Moxey <d.moxey@imperial.ac.uk> Sent: 06 July 2016 21:36:59 To: Amitvikram Dutta Cc: Mengaldo, Gianmarco; Moura, Rodrigo C; Selbach, Fabian; nektar-users Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi all,
Okay, I have now submitted a MR which should hopefully fix this issue:
https://gitlab.nektar.info/nektar/nektar/merge_requests/657
To test it you'll need to check out the source, which is in my local fork. You can check this out using the following commands:
git remote add dmoxey https://gitlab.nektar.info/dmoxey/nektar.git git fetch dmoxey git checkout -t dmoxey/fix/cfs-dgopt
Cheers,
Dave
On 6 Jul 2016, at 21:21, Amitvikram Dutta <amitvdutta23@gmail.com> wrote:
Sure!
On Wed, Jul 6, 2016, 12:55 PM David Moxey <d.moxey@imperial.ac.uk> wrote: Hi Gian,
In fact it seems I have a branch sat in my repository that fixes this but I apparently forgot to submit a merge request..!
I'll tidy it up and submit a request shortly, and then perhaps Amitvikram and Fabian, you can test this for us?
Cheers,
Dave
On 6 Jul 2016, at 20:51, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi Dave,
Yep, indeed, this is the case! We should finally patch it out! ;)
Cheers, Gian
Sent from my Samsung device
-------- Original message -------- From: David Moxey <d.moxey@imperial.ac.uk> Date: 06/07/2016 20:32 (GMT+00:00) To: "Mengaldo, Gianmarco" <g.mengaldo11@imperial.ac.uk> Cc: Amitvikram Dutta <amitvdutta23@gmail.com>, "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de>, nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi all,
I am guessing this is down to us using ExtractTracePhys() inside the boundary condition functions? I thought this had been patched out but it seems not.
Cheers,
Dave
On 6 Jul 2016, at 20:25, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi guys,
Yes, this is a known issue that we are trying to fix permanently!
We have a temporary fix for it at the moment. Rodrigo should be able to help.
Regards, Gianmarco
Sent from my Samsung device
-------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi,
I've been having an identical problem and would appreciate knowledge of the bypass
On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote: Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
_______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
Hi all, Just to follow up, the fix for this is now in the latest master. Cheers, Dave
On 12 Jul 2016, at 08:50, David Moxey <d.moxey@imperial.ac.uk> wrote:
Hi Fabian,
This is a bit of an irritating bug since, as you say, it _can_ work sometimes. Let me try to briefly explain the bug to give context and why you might be seeing this work fine on your laptop but not the cluster. It is basically due to the boundary conditions.
Due to a recent optimisation we needed to add a small communication call into a routine we use to extract edge values from elements, which is used in the boundary conditions to calculate the necessary values for the Riemann solver.
When running in parallel, we partition the mesh and the boundaries of the domain therefore lie on different processors. Depending on how many processors you use, some processors may have partitions that contain elements connected to the boundary, and others will have elements solely on the interior.
This bug is due to us putting the call to that extraction function in the wrong place: inside the boundary condition calculation for each type of boundary condition. As long as you use sufficiently few processors, every partition will contain at least some elements connected to the boundary, and this communication will work without any problem. However past a certain point, you start to create partitions that don't have any elements connected to the boundary. They will happily carry on through the communication and cause the process to deadlock.
Hope that explains the situation! I expect we'll be doing a patch release at the end of the month that will contain this fix.
Cheers,
Dave
On 11 Jul 2016, at 22:29, Selbach, Fabian <fabian.selbach@student.uni-siegen.de> wrote:
Hello,
sorry for my late response, but actually i tested a few things and still got some problems.
@Rodrigo: I tested the serial-run. It works on our Cluster and my own computer in the 3D-case! By the way is the bug-fix by David you mentioned, when you wrote how to fix the problem? Or do you have another idea how to fix this bug?
So it must be really a problem of parallelization due to openmpi in connection with Nektar++. After i tested the bug-fix (by David) on my computer, it runs in parallel-version! But actually i can't run it on our Cluster, because there is not installed an actually git-Version of Nektar++ with this bug-fix. So will there be a next Version of Nektar++ in the near future, which fixes this bug?
I even realized, that the parallel-run is possible in 3D by the Nektar++ 4.3.2 (without the bug-fix) on my own Netbook with an old version of openmpi (version 1.6), but why?
Best regards
Fabian ________________________________________ Von: nektar-users-bounces@imperial.ac.uk [nektar-users-bounces@imperial.ac.uk]" im Auftrag von "Selbach, Fabian [fabian.selbach@student.uni-siegen.de] Gesendet: Donnerstag, 7. Juli 2016 12:04 An: Mengaldo, Gianmarco; Moxey, David; Amitvikram Dutta Cc: nektar-users Betreff: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hello,
thank you! Sure i will try it, but first i need to install a git repository version of Nektar++ on my computer. After that i will let you know about it.
@Gianmarco: the mesh size is about 1800000 points. The expansion is shown in the code below. @Rodrigo: i could not try a serial-run right now, but i will do.
Best regards
Fabian
________________________________________ Von: Mengaldo, Gianmarco [g.mengaldo11@imperial.ac.uk] Gesendet: Donnerstag, 7. Juli 2016 11:05 An: Moxey, David; Amitvikram Dutta Cc: Moura, Rodrigo C; Selbach, Fabian; nektar-users Betreff: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi Dave,
this sounds good, thanks!
As soon as Amitvikram and Fabian confirm that everything works fine, we can proceed to merge it.
Cheers,
Gian
Gianmarco Mengaldo
Department of Aeronautics and Mathematics
Imperial College London
SW7 2AZ, London, UK
*Currently at ECMWF
________________________________ From: David Moxey <d.moxey@imperial.ac.uk> Sent: 06 July 2016 21:36:59 To: Amitvikram Dutta Cc: Mengaldo, Gianmarco; Moura, Rodrigo C; Selbach, Fabian; nektar-users Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi all,
Okay, I have now submitted a MR which should hopefully fix this issue:
https://gitlab.nektar.info/nektar/nektar/merge_requests/657
To test it you'll need to check out the source, which is in my local fork. You can check this out using the following commands:
git remote add dmoxey https://gitlab.nektar.info/dmoxey/nektar.git git fetch dmoxey git checkout -t dmoxey/fix/cfs-dgopt
Cheers,
Dave
On 6 Jul 2016, at 21:21, Amitvikram Dutta <amitvdutta23@gmail.com> wrote:
Sure!
On Wed, Jul 6, 2016, 12:55 PM David Moxey <d.moxey@imperial.ac.uk> wrote: Hi Gian,
In fact it seems I have a branch sat in my repository that fixes this but I apparently forgot to submit a merge request..!
I'll tidy it up and submit a request shortly, and then perhaps Amitvikram and Fabian, you can test this for us?
Cheers,
Dave
On 6 Jul 2016, at 20:51, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi Dave,
Yep, indeed, this is the case! We should finally patch it out! ;)
Cheers, Gian
Sent from my Samsung device
-------- Original message -------- From: David Moxey <d.moxey@imperial.ac.uk> Date: 06/07/2016 20:32 (GMT+00:00) To: "Mengaldo, Gianmarco" <g.mengaldo11@imperial.ac.uk> Cc: Amitvikram Dutta <amitvdutta23@gmail.com>, "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de>, nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi all,
I am guessing this is down to us using ExtractTracePhys() inside the boundary condition functions? I thought this had been patched out but it seems not.
Cheers,
Dave
On 6 Jul 2016, at 20:25, Mengaldo, Gianmarco <g.mengaldo11@imperial.ac.uk> wrote:
Hi guys,
Yes, this is a known issue that we are trying to fix permanently!
We have a temporary fix for it at the moment. Rodrigo should be able to help.
Regards, Gianmarco
Sent from my Samsung device
-------- Original message -------- From: Amitvikram Dutta <amitvdutta23@gmail.com> Date: 06/07/2016 19:22 (GMT+00:00) To: "Moura, Rodrigo C" <r.moura13@imperial.ac.uk>, "Selbach, Fabian" <fabian.selbach@student.uni-siegen.de> Cc: nektar-users <nektar-users@imperial.ac.uk> Subject: Re: [Nektar-users] Problem 3D CompressibleFlowSolver
Hi,
I've been having an identical problem and would appreciate knowledge of the bypass
On Wed, Jul 6, 2016, 11:20 AM Rodrigo Moura <r.moura13@imperial.ac.uk> wrote: Hello Fabian,
This might be an issue related to parallelization due to some recent modifications in the master or release branches. Does your case run in serial? If it runs, I can show you how to bypass this problem and run in parallel before an official fix is done.
Cheers, Rodrigo . _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users -- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus _______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
-- Amitvikram Dutta MASc Candidate Graduate Research Assistant Okanagan CFD Laboratory University of British Columbia | Okanagan Campus
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey<http://www.imperial.ac.uk/people/d.moxey>
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
_______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey
Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
_______________________________________________ Nektar-users mailing list Nektar-users@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/nektar-users
-- David Moxey (Research and Teaching Fellow) d.moxey@imperial.ac.uk | www.imperial.ac.uk/people/d.moxey Room 364, Department of Aeronautics, Imperial College London, London, SW7 2AZ, UK.
participants (5)
-
Amitvikram Dutta
-
David Moxey
-
Mengaldo, Gianmarco
-
Rodrigo Moura
-
Selbach, Fabian