- Nektar-users - imperial.ac.uk

Compiling nektar on Blue Gene / Q
by Fatih Ertinaz 23 Aug '18

23 Aug '18

Hello everyone I'm trying to compile nektar++-4.4.1 on a BG/Q cluster. So far my attempts are unsuccessful. I found this post quite helpful however still ended up with some errors: https://www.nektar.info/nektar-on-mira-cluster/ I've updated "$srcdir/cmake/ThirdPartyZlib.cmake" as it is written in that post. Then from $builddir I am running the following cmake: > > cmake $srcdir \ > > -DNEKTAR_USE_MPI=ON \ > > -DTHIRDPARTY_BUILD_ZLIB=OFF \ > > -DNEKTAR_USE_SYSTEM_BLAS_LAPACK=ON \ > > -DCMAKE_INSTALL_PREFIX=$prf > zlib and lapack are already installed on this cluster therefore I want to use those versions. This is the cmake output: > > -- The C compiler identification is GNU 4.8.1 > > -- The CXX compiler identification is GNU 4.8.1 > > -- Check for working C compiler: /....../gcc/4.8.1/mpi/bin/mpicc > > -- Check for working C compiler: /....../gcc/4.8.1/mpi/bin/mpicc -- works > > -- Detecting C compiler ABI info > > -- Detecting C compiler ABI info - done > > -- Check for working CXX compiler: /....../gcc/4.8.1/mpi/bin/mpicxx > > -- Check for working CXX compiler: /....../gcc/4.8.1/mpi/bin/mpicxx -- > > works > > -- Detecting CXX compiler ABI info > > -- Detecting CXX compiler ABI info - done > > -- Looking for a Fortran compiler > > -- Looking for a Fortran compiler - /....../gcc/4.8.1/mpi/bin/mpif90 > > -- The Fortran compiler identification is GNU > > -- Check for working Fortran compiler: /....../gcc/4.8.1/mpi/bin/mpif90 > > -- Check for working Fortran compiler: > > /....../gcc/4.8.1/mpi/bin/mpif90 -- works > > -- Detecting Fortran compiler ABI info > > -- Detecting Fortran compiler ABI info - done > > -- Checking whether /....../gcc/4.8.1/mpi/bin/mpif90 supports Fortran 90 > > -- Checking whether /....../gcc/4.8.1/mpi/bin/mpif90 supports Fortran > > 90 -- yes > > -- Build TinyXML: > > > /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/nektar++-4.4.1/libtinyxml.a > > -- Build Loki: > /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/include > > -- Build Metis: > > > /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/nektar++-4.4.1/libmetis.a > *> -- Found Zlib: /usr/lib64/libz.so (version 1.2.3)* > > -- Searching for Boost: > > -- -- Pre-installed Boost thread library not found > > -- -- Pre-installed Boost iostreams library not found > > -- -- Pre-installed Boost date_time library not found > > -- -- Pre-installed Boost filesystem library not found > > -- -- Pre-installed Boost system library not found > > -- -- Pre-installed Boost program_options library not found > > -- -- Pre-installed Boost regex library not found > > -- -- Pre-installed Boost timer library not found > > -- -- Pre-installed Boost chrono library not found > > -- Build boost libs: thread, iostreams, date_time, filesystem, system, > > program_options, regex, timer, chrono > > -- Looking for include file mpi.h > > -- Looking for include file mpi.h - found > > -- Looking for MPI_Send > > -- Looking for MPI_Send - found > > -- Found MPI: built in > > -- Build GSMPI: > > /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/libgsmpi.a > > -- Build XXT: > > /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/libxxt.a > > -- Build blas: > > > /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/nektar++-4.4.1/libblas.a > > -- Build lapack: > > > /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/nektar++-4.4.1/lapack.a > > CMake Warning (dev) at library/LibUtilities/CMakeLists.txt:422 > > (TARGET_LINK_LIBRARIES): > > Link library type specifier "debug" is followed by specifier > "optimized" > > instead of a library name. The first specifier will be ignored. > > This warning is for project developers. Use -Wno-dev to suppress it. > > > > CMake Error at library/LibUtilities/CMakeLists.txt:422 > > (TARGET_LINK_LIBRARIES): > > The "optimized" argument must be followed by a library. > > > > -- Configuring incomplete, errors occurred! > > See also > "/....../nektar/nektar++-4.4.1-build/CMakeFiles/CMakeOutput.log". > I've added the cmake.log as well. First thing I am confused is this line: > > -- Build lapack: > > > /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/nektar++-4.4.1/lapack.a > I've specified: > > -DNEKTAR_USE_SYSTEM_BLAS_LAPACK=ON \ > but it looks like somehow cmake is configured to use the third-party version. I can see both libblas.a and liblapack.a (created a symlink lapack.a too) are in my environment. Do they have to be shared libs? Should I update my LD_LIBRARY_PATH as well? Another thing is at the end of the cmake output above: > > CMake Error at library/LibUtilities/CMakeLists.txt:422 > > (TARGET_LINK_LIBRARIES): > > The "optimized" argument must be followed by a library. > I don't see any files in "nektar++-4.4.1-build/library/LibUtilities/CMakeFiles/". Therefore I have no idea why it says optimized argument must be followed by a library. When I search for optimized, this is all I get: > > bgqdev-fen1-$ grep -rn "optimized" * > > CMakeCache.txt:105:// /Ot /Oy /Gs will produce slightly less optimized > > but smaller > > CMakeCache.txt:125:// /Ot /Oy /Gs will produce slightly less optimized > > but smaller > > CMakeCache.txt:163:// /Ot /Oy /Gs will produce slightly less optimized > > but smaller > So, I don't know how to proceed any further at the moment. One last thing. Scaling achieved on Mira cluster looks impressive: https://www.nektar.info/nektar-strong-scaling-performance-on-mira/ Was it obtained using gcc or xl compilers? Any help is highly appreciated. // Fatih Ertinaz

1 0

Dong Outflow BCs
by Amitvikram Dutta 20 Aug '18

20 Aug '18

Hi all, I have been trying to implement the stabilised outflow BCs as given by DOng and included in the User guide as follows <R VAR="u" USERDEFINEDTYPE="HOutflow" VALUE="0" PRIMCOEFF="D0/TimeStep"/> <R VAR="v" USERDEFINEDTYPE="HOutflow" VALUE="0" PRIMCOEFF="D0/TimeStep"/> <R VAR="p" USERDEFINEDTYPE="HOutflow" VALUE="0" PRIMCOEFF="1.0/(D0*Kinvis)"/> What does the parameter Do represent? Is it a flow quantity?Also, how do I set it within the xml file? Sincerely, -- *Amitvikram Dutta* Graduate Research Assistant Fluid Mechanics Research Lab Multi-Physics Interaction Lab University of Waterloo

2 1

Extracting data from a plane over multiple timesteps
by Amitvikram Dutta 09 Aug '18

09 Aug '18

Hi all, I'm postprocessing a simulation and require data from a plane within my computational domain. I'm currently using the *FieldConvert -m interppoints* command to extract data from individual .chk/.fld files. Is there a variation of this command that allows a user to extract data from multiple .chk/.fld files thus obtaining the required data over a range of timesteps? Or is the simplest solution to write a bash script that executes this command multiple times in a loop over the desired range of filenames? Note, I'm also aware of the history points option, but since the number of points on the plane i require is quite large (over 1500), I have discounted this option for the present. Sincerely, -- *Amitvikram Dutta* Graduate Research Assistant Fluid Mechanics Research Lab Multi-Physics Interaction Lab University of Waterloo

1 0

help
by Rubini R. 01 Aug '18

01 Aug '18

Good morning, I'm currently using nektar++, as a user, for my research. In the near future I would like to implement some extra functionalities in the already existing library. In order not to spam anyone I would like to know if this is the right mailing list where to ask questions about the developing aspects of the software. Thanks for any info! Have a good day! Best regards, Riccardo

1 0

Reading Boundary conditions from a customized file
by wasim sarwar 30 Jul '18

30 Jul '18

Hi all, Is it possible in nektar++ to read the time-dependent boundary conditions from a file. Actually, I am simulating Vortex induced Vibration (VIV) of cylinder with blowing and suction from the surface. The motion of the cylinder is defined by the flag "userdefinedtype"=movingwall which accounts for the structural dynamics under the action of fluid forces. In order to implement blowing and suction, I need to update the value of velocity field at the cylinder surface at each time step. I cannot use the flag "userdefinedtype" twice in the same boundary condition tag to introduce this time-dependence. The way i could think of to implement this idea is to the read this time-dependent boundary condition from the file. I would also need to overwrite the content of the boundary file at each time step so that i donot need to write all file corresponding to the time steps. So, my question is, is there a way to generate the time-dependent boundary condition through a given function or by any other means, and then read the same time-dependent boundary condition from a given file for the next time step. cheers

2 1

Incompressible Navier-Stokes solver result problem
by Csaba Biró 15 Jul '18

15 Jul '18

Dear all, I am trying to solve an edge tone configuration related problem. I am using the Incompressible Navier.Stokes Solver to calculate the effects of blowing a jet flow, that impinges a wedge-shaped object. My problems are, that there seems to be an insufficient amount of outflow at the outlet ("u" velocity doesn't change), as if the airflow got stuck and that the errors shown after the calculation is complete are very high. Changing the time step or the number of modes did not decrease these errors considerably. Following the link you can see the used xml file (inc.xml). I attached a picture about the resulting "u" velocity and the geo+mesh. https://drive.google.com/open?id=1PYMXzPJo26wcIEns4OA71oI1lwWMCoYH I am only simulating half of the the problem. (hence the symmetry axis). I think there should be a much greater change in the "u" velocity overall. I double checked my boundary conditions. Please give me your opinion. Best regards, Csaba Biró

2 1

Problem in vtk fields
by Asım Önder 15 Jul '18

15 Jul '18

Dear all, I am postprocessing turbulent channel flow data from a 3D1H simulation, in which I used a coordinate mapping to impose a waviness on the bottom wall and specified NUMMODES="9" TYPE="GLL_LAGRANGE_SEM" in inhomogeneous directions. To analyze the data, I calculate the gradient (FieldConvert -m mapping -m gradient) of instantaneous fields and write the output in vtu format. After the conversion to physical coordinates in Paraview, I extract a horizontal plane from the resulting vtk fields where I see some spurious peaks on element borders in the streamwise derivative fields (u_x, v_x, w_x). I attached a figure showing an example of this, where on the left the problematic "u_x" is shown and on the right "u_x" derived from the continuity equation is shown, i.e., "-v_y-w_z". It can be seen that the fields are the same except some artifacts on element borders in direct "u_x". Is this maybe a bug? Thanks for any feedback. Regards, Asim

2 1

Calling Fortran modules in compressible flow solver
by Aidyn Aitzhan 15 Jul '18

15 Jul '18

Hi all, I am planning to call an external Fortran module in the compressible flow solver of Nektar++ 4.4.1. Could you please guide me about how I should modify the make (CMakeLists.txt) files? Basically, I have Fortran subroutines that allow me to compute temperature for non-constant heat capacities, so I want to use them. Thanks in advance, Aidyn

2 1

Vortex induced Vibration
by wasim sarwar 08 Jul '18

08 Jul '18

Hi all, I am interested to simulate flow past cylinder with vortex induced vibration. I went through a tutorial where finite-strip theory was used to model VIV of flexible cable. In the xml file of the tutorial they have already prescribed the motion of the cable via analytic expressions. Is it possible to compute the displacements and velocities of the cable arising from fluid forces, i.e. lift and drag rather than giving an explicit expression beforehand? Regards

2 1

Optimize solver settings for half billion mesh
by Sandeep Pandey 28 Jun '18

28 Jun '18

Hello all, I am running a 3DH1D turbulent channel flow at Re_tau=550 (MKM590, larger domain) which has a resolution of ~0.5 billion DoF and streamwise direction is the homogeneous direction with FFT. Our target is to achieve the solver setting to have fastest computation time on Cray- XC40 machine. We have tried a plenty of options e.g. Direct Solver, Iterative solver with 1D FFT and different pre-conditioners. But our CPU Time per time step is ~5 seconds, which is quite slow as per our experience. - We compare "DirectMultiLevelStaticCond" and "DirectStaticCond" solver and finds that the first one is slightly faster than the second one. It was due to 1D decomposition of the direct solver (--npz). We cannot utilize a large number of processors (e.g. 480 processors for 960 HomModeZ). Is 2D decomposition of direct solver exist? - If we use FFTW together with the iterative solver, can I expect better performance than the direct solver? What would be the optimal choice for the solver/preconditioner combination? Would the options in PETCS help? - According to my experience in full 3D DNS case (50 mio. mesh), around 0.2 sec/time step can be expected. Although this requires a large number of cores. Can I achieve something similar with 3DH1D cases? Part of xml file: <EXPANSIONS> <E COMPOSITE="C[0]" NUMMODES="7" FIELDS="u,v,w" TYPE="MODIFIED" /> <E COMPOSITE="C[0]" NUMMODES="7" FIELDS="p" TYPE="MODIFIED" /> </EXPANSIONS> <CONDITIONS> <SOLVERINFO> </SOLVERINFO> <PARAMETERS> TimeStep = 10e-05 FinalTime = 10.0 NumSteps = FinalTime/TimeStep NumSteps = 1000000 IO_CheckSteps = 2000 IO_InfoSteps = 5 IO_CFLSteps = IO_InfoSteps Re = 10000 Kinvis = 1.0/544.0 HomModesZ = 960 LZ = 8.0*PI SVVCutoffRatio = 0.7 SVVDiffCoeff = 0.1 </PARAMETERS> Thank you! Sandeep

2 1