Strumenti Utente

Strumenti Sito

Traccia:
calcoloscientifico:userguide:apptainer

Differenze

Queste sono le differenze tra la revisione selezionata e la versione attuale della pagina.

Link a questa pagina di confronto

Entrambe le parti precedenti la revisioneRevisione precedente
Prossima revisione		Revisione precedente
calcoloscientifico:userguide:apptainer [02/05/2022 18:07] – fabio.spatarocalcoloscientifico:userguide:apptainer [14/01/2025 13:00] (versione attuale) – [Apptainer] fabio.spataro
Linea 1: Linea 1:
-==== AppTainer ====+===== Apptainer =====
  
-[[https://apptainer.org|AppTainer]]+  * [[https://apptainer.org|Apptainer]] 
 +  * [[https://github.com/apptainer/apptainer|Apptainer on GitHub]]
  
-AppTainer is already available on the HPC cluster.+Apptainer is already available on the HPC cluster.
  
 Users are encouraged to use it on their system. Users are encouraged to use it on their system.
  
-=== Install a binary package ===+==== Install a binary package ====
  
-These instructions are intended for users who wish to install AppTainer on their Linux system.+These instructions are intended for users who wish to install Apptainer on their Linux system.
  
-On a **RedHat Enterpise Linux** system AppTainer can be installed by the administrator with the following command:+On a **RedHat Enterpise Linux** system Apptainer can be installed by the administrator with the following command:
  
 <code> <code>
-yum install https://github.com/apptainer/apptainer/releases/download/v1.0.1/apptainer-1.0.1-1.x86_64.rpm+yum install https://github.com/apptainer/apptainer/releases/download/v1.1.8/apptainer-1.1.8-1.x86_64.rpm
 </code> </code>
  
-On a **Debian/Ubuntu** system AppTainer can be installed by the administrator with the following command:+On a **Debian/Ubuntu** system Apptainer can be installed by the administrator with the following command:
  
 <code> <code>
-sudo apt install https://github.com/apptainer/apptainer/releases/download/v1.0.1/apptainer_1.0.1_amd64.deb+wget -qc https://github.com/apptainer/apptainer/releases/download/v1.1.8/apptainer_1.1.8_amd64.deb 
 +sudo dpkg -i apptainer_1.1.8_amd64.deb
 </code> </code>
  
-=== Extract NVIDIA driver ===+==== Extract NVIDIA driver ====
  
 The following commands The following commands
Linea 34: Linea 36:
 were executed on worker nodes with GPU by the administrator in a directory that contains the [[https://www.nvidia.it/Download/index.aspx|NVIDIA driver]] setup executable (for example ''NVIDIA-Linux-x86_64-510.47.03.run''). were executed on worker nodes with GPU by the administrator in a directory that contains the [[https://www.nvidia.it/Download/index.aspx|NVIDIA driver]] setup executable (for example ''NVIDIA-Linux-x86_64-510.47.03.run'').
  
-Users who want to install AppTainer on their Linux system will be able to change the NVIDIA driver version and the installation path:+Users who want to install Apptainer on their Linux system will be able to change the NVIDIA driver version and the installation path:
  
 <code> <code>
Linea 41: Linea 43:
 </code> </code>
  
-=== AppTainer on a worker node with CPU ===+==== Apptainer on a worker node with CPU ====
  
 On the login node of the HPC cluster run the following command: On the login node of the HPC cluster run the following command:
Linea 52: Linea 54:
  
 <code bash> <code bash>
-module load apptainer+module load apptainer/1.0
  
 container='/hpc/share/applications/amber/20/amber-20-cpu' container='/hpc/share/applications/amber/20/amber-20-cpu'
Linea 63: Linea 65:
 </code> </code>
  
-Inside the AppTainer container:+Inside the Apptainer container:
  
 <code> <code>
Linea 70: Linea 72:
 </code> </code>
  
-Users who want to try AppTainer on their system will have to copy the AppTainer image+Users who want to try Apptainer on their system will have to copy the Apptainer image
  
 <code> <code>
Linea 80: Linea 82:
 Users who have installed a binary package do not have to load the ''apptainer'' module. Users who have installed a binary package do not have to load the ''apptainer'' module.
  
-=== AppTainer on a worker node with GPU ===+=== Apptainer on a worker node with GPU ===
  
 On the login node of the HPC cluster run the following command: On the login node of the HPC cluster run the following command:
Linea 91: Linea 93:
  
 <code bash> <code bash>
-module load apptainer+module load apptainer/1.0
  
 container='/hpc/share/applications/amber/20/amber-20-gpu' container='/hpc/share/applications/amber/20/amber-20-gpu'
Linea 103: Linea 105:
 </code> </code>
  
-Inside the AppTainer container:+Inside the Apptainer container:
  
 <code> <code>
Linea 118: Linea 120:
 </code> </code>
  
-Users who want to try AppTainer on their system will have to copy the AppTainer image+Users who want to try Apptainer on their system will have to copy the Apptainer image
  
 <code> <code>
Linea 128: Linea 130:
 Users who have installed a binary package do not have to load the ''apptainer'' module. Users who have installed a binary package do not have to load the ''apptainer'' module.
  
-----+===== Apptainer containers =====
  
-<note> +==== Bacterial Genomics software collection ====
-The following section needs to be revised by replacing **singularity** with **apptainer** +
-</note>+
  
-=== CityChrone ===+''slurm-bactgen.sh'' script to get list of packages present in ''bactgen'' environment and help from ''tormes'' on 1 node (1 task, 4 CPUs per task):
  
-[[http://project-osrm.org|CityChrone Project]] +<code bash slurm-bactgen.sh>
- +
-== CityChrone from Rocky 8.5 docker == +
- +
-Singularity Definition File: +
- +
-<code text citychrone-rocky-8.5.def> +
-BootStrap: docker +
-From: rockylinux:8.5 +
- +
-%environment +
-export PATH=/miniconda3/bin:$PATH +
- +
-%runscript +
-exec vcontact "$@" +
- +
-%post +
-dnf -y update +
-dnf -y install scl-utils +
-dnf -y install gcc-toolset-9 +
-scl enable gcc-toolset-9 bash +
-dnf -y install git cmake3 zlib-devel wget +
- +
-# Install miniconda +
-wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh +
-bash Miniconda3-latest-Linux-x86_64.sh -b -f -p /miniconda3/ +
-rm Miniconda3-latest-Linux-x86_64.sh +
- +
-# pull the conda functions in . /miniconda3/etc/profile.d/conda.sh and make pip, etc. available while in %post +
-export PATH="/miniconda3/bin:$PATH" +
- +
-# CONDA install +
-conda install \ +
---yes \ +
---channel conda-forge \ +
---strict-channel-priority \ +
-pandas matplotlib folium gdal jupyter numba colorama geopy shapely tqdm pymongo geojson protobuf pyproj +
- +
-# Help conda resolving Python "import" +
-conda update --all +
- +
-# PIP install +
-pip install \ +
---no-deps \ +
-gtfs-realtime-bindings pyspark mpire +
- +
-# OSRM +
-git clone https://github.com/Project-OSRM/osrm-backend.git +
-cd osrm-backend +
-mkdir -p build +
-cd build +
-cmake3 .. -DENABLE_MASON=ON -DCMAKE_CXX_COMPILER=/opt/rh/gcc-toolset-9/root/usr/bin/g++ +
-make +
-make install +
-</code> +
- +
-Singularity Build script: +
- +
-<code bash singularity-build-rocky-8.5.sh>+
 #!/bin/bash #!/bin/bash
 +#SBATCH --job-name=bactgen
 +#SBATCH --output=%x.o%j
 +#SBATCH --error=%x.e%j
 +#SBATCH --nodes=1
 +#SBATCH --ntasks-per-node=1
 +#SBATCH --cpus-per-task=4
 +#SBATCH --time=0-01:00:00
 +#SBATCH --mem=8G
 +#SBATCH --partition=cpu
 +#SBATCH --qos=cpu
 +#SBATCH --account=<account>
  
-module load singularity/3.8.6+module load apptainer 
 +module load bactgen
  
-export      SINGULARITY_CACHEDIR="/node/$USER/singularity/.singularity_cache" +apptainer run "$CONTAINER" micromamba list 
-export    SINGULARITY_PULLFOLDER="/node/$USER/singularity/.singularity_images" +echo '─────────────────────────────────────────────────────────────────────────────────────────' 
-export        SINGULARITY_TMPDIR="/node/$USER/singularity/.singularity_tmp" +apptainer run "$CONTAINER" tormes --help 
-export SINGULARITY_LOCALCACHEDIR="/node/$USER/singularity/.singularity_localcache" +echo '─────────────────────────────────────────────────────────────────────────────────────────'
-export                    TMPDIR="/node/$USER/singularity/.tmp" +
- +
-mkdir -p "$SINGULARITY_CACHEDIR" +
-mkdir -p "$SINGULARITY_PULLFOLDER" +
-mkdir -p "$SINGULARITY_TMPDIR" +
-mkdir -p "$SINGULARITY_LOCALCACHEDIR" +
-mkdir -p "$TMPDIR" +
- +
-singularity build --fakeroot "/node/$USER/singularity/citychrone-rocky-8.5.sif" citychrone-rocky-8.5.def +
-mv "/node/$USER/singularity/citychrone-rocky-8.5.sif" . +
-</code> +
- +
-Singularity Run script: +
- +
-<code bash singularity-run-rocky-8.5.sh> +
-#!/bin/bash +
- +
-module load singularity/3.8.6 +
- +
-singularity shell citychrone-rocky-8.5.sif +
-</code> +
- +
-Interactive session: +
- +
-<code bash> +
-srun --nodes=1 --ntasks-per-node=2 --partition=cpu --mem=8G --time=02:00:00 --pty bash +
-</code> +
- +
-Launch Singularity Build script: +
- +
-<code bash> +
-bash singularity-build-rocky-8.5.sh +
-</code> +
- +
-Launch Singularity Run script: +
- +
-<code bash> +
-bash singularity-run-rocky-8.5.sh +
-</code> +
- +
-== CityChrone from Ubuntu Focal (20.04 LTS) docker == +
- +
-Singularity Definition File: +
- +
-<code text citychrone-ubuntu-focal.def> +
-BootStrap: docker +
-From: ubuntu:focal +
- +
-%environment +
-export PATH=/miniconda3/bin:$PATH +
-export DEBIAN_FRONTEND=noninteractive +
-export TZ='Europe/Rome' +
- +
-%runscript +
-exec vcontact "$@" +
- +
-%post +
-DEBIAN_FRONTEND=noninteractive +
-TZ='Europe/Rome' +
-ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone +
-apt-get update && \ +
-apt-get install -y automake build-essential bzip2 wget git default-jre unzip \ +
-build-essential git cmake pkg-config \ +
-libbz2-dev libstxxl-dev libstxxl1v5 libxml2-dev \ +
-libzip-dev libboost-all-dev lua5.2 liblua5.2-dev libtbb-dev +
- +
-# Install miniconda +
-wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh +
-bash Miniconda3-latest-Linux-x86_64.sh -b -f -p /miniconda3/ +
-rm Miniconda3-latest-Linux-x86_64.sh +
- +
-# pull the conda functions in . /miniconda3/etc/profile.d/conda.sh and make pip, etc. available while in %post +
-export PATH="/miniconda3/bin:$PATH" +
- +
-# CONDA install +
-conda install \ +
---yes \ +
---channel conda-forge \ +
---strict-channel-priority \ +
-pandas matplotlib folium gdal jupyter numba colorama geopy shapely tqdm pymongo geojson protobuf pyproj +
- +
-# Help conda resolving Python "import" +
-conda update --all +
- +
-# PIP install +
-pip install \ +
---no-deps \ +
-gtfs-realtime-bindings pyspark mpire +
- +
-# OSRM +
-git clone https://github.com/Project-OSRM/osrm-backend.git +
-cd osrm-backend +
-mkdir -p build +
-cd build +
-cmake .. -DCMAKE_BUILD_TYPE=Release +
-cmake --build . +
-cmake --build . --target install +
-</code> +
- +
-Singularity Build script:  +
- +
-<code bash singularity-build-ubuntu-focal.sh> +
-#!/bin/bash +
- +
-module load singularity/3.8.6 +
- +
-export      SINGULARITY_CACHEDIR="/node/$USER/singularity/.singularity_cache" +
-export    SINGULARITY_PULLFOLDER="/node/$USER/singularity/.singularity_images" +
-export        SINGULARITY_TMPDIR="/node/$USER/singularity/.singularity_tmp" +
-export SINGULARITY_LOCALCACHEDIR="/node/$USER/singularity/.singularity_localcache" +
-export                    TMPDIR="/node/$USER/singularity/.tmp" +
- +
-mkdir -p "$SINGULARITY_CACHEDIR" +
-mkdir -p "$SINGULARITY_PULLFOLDER" +
-mkdir -p "$SINGULARITY_TMPDIR" +
-mkdir -p "$SINGULARITY_LOCALCACHEDIR" +
-mkdir -p "$TMPDIR" +
- +
-singularity build --fakeroot "/node/$USER/singularity/citychrone-ubuntu-focal.sif" citychrone-ubuntu-focal.def +
-mv "/node/$USER/singularity/citychrone-ubuntu-focal.sif" . +
-</code> +
- +
-Singularity Run script: +
- +
-<code bash singularity-run-ubuntu-focal.sh> +
-#!/bin/bash +
- +
-module load singularity/3.8.6 +
- +
-singularity shell citychrone-ubuntu-focal.sif +
-</code> +
- +
-Interactive session: +
- +
-<code bash> +
-srun --nodes=1 --ntasks-per-node=2 --partition=cpu --mem=8G --time=02:00:00 --pty bash +
-</code> +
- +
-Launch Singularity Build script: +
- +
-<code bash> +
-bash singularity-build-ubuntu-focal.sh+
 </code> </code>
  
-Launch Singularity Run script:+Edit the ''slurm-bactgen.sh'' script and submit it with the following command:
  
 <code bash> <code bash>
-bash singularity-run-ubuntu-focal.sh+sbatch slurm-bactgen.sh
 </code> </code>
  
calcoloscientifico/userguide/apptainer.1651507676.txt.gz · Ultima modifica: 02/05/2022 18:07 da fabio.spataro
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki