Hi Andrew, On Fri, Aug 06, 2021 at 05:36:47PM +0100, Andrew S Naylor wrote:
FATAL: while extracting hyper_lite__0_1.sif: root filesystem extraction
The LZ cloud site uses singularity for payload isolation: It's still possible to run a "nested" singularity instance, but there are a few extra limitations.
failed: extract command failed: FATAL: container creation failed:
It looks like you're using a .sif image file: We generally suggest using an extracted image directory instead as it works more reliably (this is one of the limitations of nested singularity).
Also by default i'm using `/cvmfs/ oasis.opensciencegrid.org/mis/singularity/current/bin/singularity` for the singularity binary, unless user namespaces aren't enabled on that system and then it uses the local singularity instead.
Yes, that's a sensible approach. If you use the version from CVMFS, add the --userns option too, which (along with avoiding the .sif format) should fix the problem. We wrote a script encapsulating all of the singularity workarounds we know about for LZ: I've included that below for reference (there are also some quirks around the current working directory at some other sites). p.s. You're not actually subscribed to this mailing list, you may want to consider joining it; it's rather low traffic. This sign-up page is: https://mailman.ic.ac.uk/mailman/listinfo/gridpp-dirac-users Regards, Simon #!/bin/bash # LZ Grid Container script # v1.0 ICHEP June 2021 - Initial version # v1.1 ICHEP June 2021 - Removed sysctl call and fixed /srv bug # Detect singularity location and run-mode SINGULARITY='/cvmfs/oasis.opensciencegrid.org/mis/singularity/current/bin/singularity' CONTAINER_OPT="" if [ -f /usr/bin/singularity ]; then # Use locally installed version SINGULARITY='/usr/bin/singularity' fi # Detect user namespace support and enable it if needed USERNS=$(awk '{ print $NF }' < /proc/sys/user/max_user_namespaces) if [ ${USERNS} -gt 200 ]; then CONTAINER_OPTS="--userns" fi # Uses the standard LZ container from CVMFS. CONTAINER='/cvmfs/singularity.opensciencegrid.org/luxzeplin/base_os:centos7' # Copy the proxy to the CWD if [ ! -z "${X509_USER_PROXY}" ]; then cp -p "${X509_USER_PROXY}" "${PWD}/user.proxy" elif [ -f "/tmp/x509up_u$(id -u)" ]; then # Proxy is using default location cp -p "/tmp/x509up_u$(id -u)" "${PWD}/user.proxy" fi # Prepare the environment for inside the conainter # Variables prepended with SINGULARITYENV_ are automatically included # even with the --cleanenv set. # Proxy inside the container needs to use the absolute /srv path export SINGULARITYENV_X509_USER_PROXY="/srv/user.proxy" export SINGULARITYENV_X509_CERT_DIR=/cvmfs/grid.cern.ch/etc/grid-security/certificates export SINGULARITYENV_LZ_SETUP_DBI=false # Mount the local path as /srv in the container and CVMFS CONTAINER_OPTS="${CONTAINER_OPTS} -B /cvmfs -B ${PWD}:/srv --pwd /srv --cleanenv" exec ${SINGULARITY} exec ${CONTAINER_OPTS} ${CONTAINER} "${@}"