Singularity []

Questa pagina è in sola lettura. Puoi visualizzare il sorgente, ma non puoi modificarlo. Contatta l'amministratore se pensi che ci sia un errore.
==== Singularity ====

Singularity has been replaced by [[calcoloscientifico:userguide:apptainer|Apptainer]].

<note>
The following section contains **outdated** information
</note>

=== CityChrone ===

[[http://project-osrm.org|CityChrone Project]]

== CityChrone from Rocky 8.5 docker ==

Singularity Definition File:

<code text citychrone-rocky-8.5.def>
BootStrap: docker
From: rockylinux:8.5

%environment
export PATH=/miniconda3/bin:$PATH

%runscript
exec vcontact "$@"

%post
dnf -y update
dnf -y install scl-utils
dnf -y install gcc-toolset-9
scl enable gcc-toolset-9 bash
dnf -y install git cmake3 zlib-devel wget

# Install miniconda
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -f -p /miniconda3/
rm Miniconda3-latest-Linux-x86_64.sh

# pull the conda functions in . /miniconda3/etc/profile.d/conda.sh and make pip, etc. available while in %post
export PATH="/miniconda3/bin:$PATH"

# CONDA install
conda install \
--yes \
--channel conda-forge \
--strict-channel-priority \
pandas matplotlib folium gdal jupyter numba colorama geopy shapely tqdm pymongo geojson protobuf pyproj

# Help conda resolving Python "import"
conda update --all

# PIP install
pip install \
--no-deps \
gtfs-realtime-bindings pyspark mpire

# OSRM
git clone https://github.com/Project-OSRM/osrm-backend.git
cd osrm-backend
mkdir -p build
cd build
cmake3 .. -DENABLE_MASON=ON -DCMAKE_CXX_COMPILER=/opt/rh/gcc-toolset-9/root/usr/bin/g++
make
make install
</code>

Singularity Build script:

<code bash singularity-build-rocky-8.5.sh>
#!/bin/bash

module load singularity/3.8.6

export      SINGULARITY_CACHEDIR="/node/$USER/singularity/.singularity_cache"
export    SINGULARITY_PULLFOLDER="/node/$USER/singularity/.singularity_images"
export        SINGULARITY_TMPDIR="/node/$USER/singularity/.singularity_tmp"
export SINGULARITY_LOCALCACHEDIR="/node/$USER/singularity/.singularity_localcache"
export                    TMPDIR="/node/$USER/singularity/.tmp"

mkdir -p "$SINGULARITY_CACHEDIR"
mkdir -p "$SINGULARITY_PULLFOLDER"
mkdir -p "$SINGULARITY_TMPDIR"
mkdir -p "$SINGULARITY_LOCALCACHEDIR"
mkdir -p "$TMPDIR"

singularity build --fakeroot "/node/$USER/singularity/citychrone-rocky-8.5.sif" citychrone-rocky-8.5.def
mv "/node/$USER/singularity/citychrone-rocky-8.5.sif" .
</code>

Singularity Run script:

<code bash singularity-run-rocky-8.5.sh>
#!/bin/bash

module load singularity/3.8.6

singularity shell citychrone-rocky-8.5.sif
</code>

Interactive session:

<code bash>
srun --nodes=1 --ntasks-per-node=2 --partition=cpu --mem=8G --time=02:00:00 --pty bash
</code>

Launch Singularity Build script:

<code bash>
bash singularity-build-rocky-8.5.sh
</code>

Launch Singularity Run script:

<code bash>
bash singularity-run-rocky-8.5.sh
</code>

== CityChrone from Ubuntu Focal (20.04 LTS) docker ==

Singularity Definition File:

<code text citychrone-ubuntu-focal.def>
BootStrap: docker
From: ubuntu:focal

%environment
export PATH=/miniconda3/bin:$PATH
export DEBIAN_FRONTEND=noninteractive
export TZ='Europe/Rome'

%runscript
exec vcontact "$@"

%post
DEBIAN_FRONTEND=noninteractive
TZ='Europe/Rome'
ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
apt-get update && \
apt-get install -y automake build-essential bzip2 wget git default-jre unzip \
build-essential git cmake pkg-config \
libbz2-dev libstxxl-dev libstxxl1v5 libxml2-dev \
libzip-dev libboost-all-dev lua5.2 liblua5.2-dev libtbb-dev

# Install miniconda
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -f -p /miniconda3/
rm Miniconda3-latest-Linux-x86_64.sh

# pull the conda functions in . /miniconda3/etc/profile.d/conda.sh and make pip, etc. available while in %post
export PATH="/miniconda3/bin:$PATH"

# CONDA install
conda install \
--yes \
--channel conda-forge \
--strict-channel-priority \
pandas matplotlib folium gdal jupyter numba colorama geopy shapely tqdm pymongo geojson protobuf pyproj

# Help conda resolving Python "import"
conda update --all

# PIP install
pip install \
--no-deps \
gtfs-realtime-bindings pyspark mpire

# OSRM
git clone https://github.com/Project-OSRM/osrm-backend.git
cd osrm-backend
mkdir -p build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
cmake --build .
cmake --build . --target install
</code>

Singularity Build script: 

<code bash singularity-build-ubuntu-focal.sh>
#!/bin/bash

module load singularity/3.8.6

export      SINGULARITY_CACHEDIR="/node/$USER/singularity/.singularity_cache"
export    SINGULARITY_PULLFOLDER="/node/$USER/singularity/.singularity_images"
export        SINGULARITY_TMPDIR="/node/$USER/singularity/.singularity_tmp"
export SINGULARITY_LOCALCACHEDIR="/node/$USER/singularity/.singularity_localcache"
export                    TMPDIR="/node/$USER/singularity/.tmp"

mkdir -p "$SINGULARITY_CACHEDIR"
mkdir -p "$SINGULARITY_PULLFOLDER"
mkdir -p "$SINGULARITY_TMPDIR"
mkdir -p "$SINGULARITY_LOCALCACHEDIR"
mkdir -p "$TMPDIR"

singularity build --fakeroot "/node/$USER/singularity/citychrone-ubuntu-focal.sif" citychrone-ubuntu-focal.def
mv "/node/$USER/singularity/citychrone-ubuntu-focal.sif" .
</code>

Singularity Run script:

<code bash singularity-run-ubuntu-focal.sh>
#!/bin/bash

module load singularity/3.8.6

singularity shell citychrone-ubuntu-focal.sif
</code>

Interactive session:

<code bash>
srun --nodes=1 --ntasks-per-node=2 --partition=cpu --mem=8G --time=02:00:00 --pty bash
</code>

Launch Singularity Build script:

<code bash>
bash singularity-build-ubuntu-focal.sh
</code>

Launch Singularity Run script:

<code bash>
bash singularity-run-ubuntu-focal.sh
</code>