Hello everyone

I'm trying to compile nektar++-4.4.1 on a BG/Q cluster. So far my
attempts are unsuccessful.

I found this post quite helpful however still ended up with some errors:
https://www.nektar.info/nektar-on-mira-cluster/

I've updated "$srcdir/cmake/ThirdPartyZlib.cmake" as it is written in
that post. Then from $builddir I am running the following cmake:

> cmake $srcdir \
>   -DNEKTAR_USE_MPI=ON \
>   -DTHIRDPARTY_BUILD_ZLIB=OFF \
>   -DNEKTAR_USE_SYSTEM_BLAS_LAPACK=ON \
>   -DCMAKE_INSTALL_PREFIX=$prf

zlib and lapack are already installed on this cluster therefore I want
to use those versions. This is the cmake output:

> -- The C compiler identification is GNU 4.8.1
> -- The CXX compiler identification is GNU 4.8.1
> -- Check for working C compiler: /....../gcc/4.8.1/mpi/bin/mpicc
> -- Check for working C compiler: /....../gcc/4.8.1/mpi/bin/mpicc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working CXX compiler: /....../gcc/4.8.1/mpi/bin/mpicxx
> -- Check for working CXX compiler: /....../gcc/4.8.1/mpi/bin/mpicxx --
> works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Looking for a Fortran compiler
> -- Looking for a Fortran compiler - /....../gcc/4.8.1/mpi/bin/mpif90
> -- The Fortran compiler identification is GNU
> -- Check for working Fortran compiler: /....../gcc/4.8.1/mpi/bin/mpif90
> -- Check for working Fortran compiler:
> /....../gcc/4.8.1/mpi/bin/mpif90  -- works
> -- Detecting Fortran compiler ABI info
> -- Detecting Fortran compiler ABI info - done
> -- Checking whether /....../gcc/4.8.1/mpi/bin/mpif90 supports Fortran 90
> -- Checking whether /....../gcc/4.8.1/mpi/bin/mpif90 supports Fortran
> 90 -- yes
> -- Build TinyXML:
> /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/nektar++-4.4.1/libtinyxml.a
> -- Build Loki: /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/include
> -- Build Metis:
> /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/nektar++-4.4.1/libmetis.a
> -- Found Zlib: /usr/lib64/libz.so (version 1.2.3)
> -- Searching for Boost:
> -- -- Pre-installed Boost thread library not found
> -- -- Pre-installed Boost iostreams library not found
> -- -- Pre-installed Boost date_time library not found
> -- -- Pre-installed Boost filesystem library not found
> -- -- Pre-installed Boost system library not found
> -- -- Pre-installed Boost program_options library not found
> -- -- Pre-installed Boost regex library not found
> -- -- Pre-installed Boost timer library not found
> -- -- Pre-installed Boost chrono library not found
> -- Build boost libs: thread, iostreams, date_time, filesystem, system,
> program_options, regex, timer, chrono
> -- Looking for include file mpi.h
> -- Looking for include file mpi.h - found
> -- Looking for MPI_Send
> -- Looking for MPI_Send - found
> -- Found MPI: built in
> -- Build GSMPI:
> /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/libgsmpi.a
> -- Build XXT:
> /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/libxxt.a
> -- Build blas:
> /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/nektar++-4.4.1/libblas.a
> -- Build lapack:
> /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/nektar++-4.4.1/lapack.a
> CMake Warning (dev) at library/LibUtilities/CMakeLists.txt:422
> (TARGET_LINK_LIBRARIES):
>   Link library type specifier "debug" is followed by specifier "optimized"
>   instead of a library name.  The first specifier will be ignored.
> This warning is for project developers.  Use -Wno-dev to suppress it.
>
> CMake Error at library/LibUtilities/CMakeLists.txt:422
> (TARGET_LINK_LIBRARIES):
>   The "optimized" argument must be followed by a library.
>
> -- Configuring incomplete, errors occurred!
> See also "/....../nektar/nektar++-4.4.1-build/CMakeFiles/CMakeOutput.log".

I've added the cmake.log as well.

First thing I am confused is this line:

> -- Build lapack:
> /....../nektar/nektar++-4.4.1-build/ThirdParty/dist/lib/nektar++-4.4.1/lapack.a

I've specified:

>   -DNEKTAR_USE_SYSTEM_BLAS_LAPACK=ON \

but it looks like somehow cmake is configured to use the third-party
version. I can see both libblas.a and liblapack.a (created a symlink
lapack.a too) are in my environment. Do they have to be shared libs?
Should I update my LD_LIBRARY_PATH as well?

Another thing is at the end of the cmake output above:

> CMake Error at library/LibUtilities/CMakeLists.txt:422
> (TARGET_LINK_LIBRARIES):
>   The "optimized" argument must be followed by a library.

I don't see any files in
"nektar++-4.4.1-build/library/LibUtilities/CMakeFiles/". Therefore I
have no idea why it says optimized argument must be followed by a library.

When I search for optimized, this is all I get:

> bgqdev-fen1-$ grep -rn "optimized" *
> CMakeCache.txt:105:// /Ot /Oy /Gs will produce slightly less optimized
> but smaller
> CMakeCache.txt:125:// /Ot /Oy /Gs will produce slightly less optimized
> but smaller
> CMakeCache.txt:163:// /Ot /Oy /Gs will produce slightly less optimized
> but smaller

So, I don't know how to proceed any further at the moment.

One last thing. Scaling achieved on Mira cluster looks impressive:
https://www.nektar.info/nektar-strong-scaling-performance-on-mira/

Was it obtained using gcc or xl compilers?

Any help is highly appreciated.

// Fatih Ertinaz