Strumenti Utente

Strumenti Sito


calcoloscientifico:cluster:softwareapplicativo:conda_user

CONDA user defined environments

Users or groups can build their own environments.

General template

General template that can be used to create personal or group environments:

hpc-make-condaenv-template.sh
#!/bin/bash
 
set -e
 
module load miniconda3
 
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
 
CONDAENV_NAME='template'
CONDAENV_VERSION=$(date '+%Y.%m.%d')
PYTHON_VERSION='3.8'
 
CONDAENV="${CONDAENV_NAME}-${CONDAENV_VERSION}"
 
if [ $UID = 0 ]; then
    CONDAENV_PREFIX="/hpc/share/tools/miniconda3/envs/$CONDAENV"
    CONDAENV_SYMLNK="/hpc/share/tools/miniconda3/envs/$CONDAENV_NAME"
else
    if [ -z "$CONDA_ENVS_PATH" ]; then
        echo 'Error: the CONDA_ENVS_PATH envarionment variable is not set' 1>&2
        exit 1
    fi
    if [ -z "$CONDA_PKGS_DIRS" ]; then
        echo 'Error: the CONDA_PKGS_DIRS envarionment variable is not set' 1>&2
        exit 1
    fi
    echo "CONDA_ENVS_PATH: $CONDA_ENVS_PATH"
    echo "CONDA_PKGS_DIRS: $CONDA_PKGS_DIRS"
    CONDAENV_PREFIX="$CONDA_ENVS_PATH/$CONDAENV"
    CONDAENV_SYMLNK="$CONDA_ENVS_PATH/$CONDAENV_NAME"
fi
 
if [ $UID = 0 ]; then
    conda --version
    conda update conda
    conda --version
    echo
fi
 
echo "CONDAENV_PREFIX: $CONDAENV_PREFIX"
echo "CONDAENV_SYMLNK: $CONDAENV_SYMLNK"
echo
 
if [ -d "$CONDAENV_PREFIX" ]; then
    echo -n "Should we remove the already existing condaenv '$CONDAENV_NAME' (yes/no)? "
 
    read ans
 
    case "$ans" in
        [yY]|[yY][Ee][Ss])
        conda env remove --name "$CONDAENV"
        ;;
        [nN]|[nN][oO])
        ;;
        *)
        echo
        exec "$0"
        exit
        ;;
    esac
fi
 
if [ -d "$CONDAENV_PREFIX" ]; then
    echo -n "Should we proceed with the modification of the condaenv '$CONDAENV_NAME' (yes/no)? "
else
    echo -n "Should we proceed with the creation of the condaenv '$CONDAENV_NAME' (yes/no)? "
fi
 
read ans
 
case "$ans" in
    [yY]|[yY][Ee][Ss])
    ;;
    [nN]|[nN][oO])
    exit 1
    ;;
    *)
    echo
    exec "$0"
    exit
    ;;
esac
 
conda info
 
if [ ! -d "$CONDAENV_PREFIX" ]; then
    conda create --yes --prefix "$CONDAENV_PREFIX" python=$PYTHON_VERSION
fi
 
conda activate "$CONDAENV"
 
conda install --yes --name "$CONDAENV" \
--channel conda-forge \
xz
 
conda list || true
 
conda deactivate
 
ln -sf "$CONDAENV_PREFIX" "$CONDAENV_SYMLNK"
ls -la "$CONDAENV_SYMLNK"*

This script is located in

/hpc/share/tools/conda/bin/hpc-make-condaenv-template

Customize your script by editing CONDAENV_NAME, PYTHON_VERSION and the "conda install" command.

The "conda install" command can be repeated multiple times with different options and parameters.

It is also possible to install python packages using the "pip" command.

Creating a user environment

Create a script from the template and run it.

Creating a group environment

Change group:

newgrp <GROUP>

Create a script from the template and run it.

Working with conda

With conda, you can create, export, list, remove, and update that have different versions of Python and/or packages installed in them. Switching or moving between environments is called activating the environment.

Interactive work session

If you want to work at group level change group otherwise skip this step:

newgrp <GROUP>

Typical work session with conda:

[<USER>@ui01 ~]$ module load miniconda3

[<USER>@ui01 ~]$ source "$CONDA_PREFIX/etc/profile.d/conda.sh"

[<USER>@ui01 ~]$ conda --help

[<USER>@ui01 ~]$ conda --version
conda 4.9.2

[<USER>@ui01 ~]$ conda info

[<USER>@ui01 ~]$ conda activate

(base) [<USER>@ui01 ~]$ which python
/hpc/share/tools/miniconda3/bin/python

(base) [<USER>@ui01 ~]$ python -V
Python 3.7.4

(base) [<USER>@ui01 ~]$ conda deactivate

[<USER>@ui01 ~]$
conda search sklearn

Alternate channels

To search for alternate channels that may provide the conda package you're looking for, navigate to anaconda.org and use the search bar at the top of the page.

To add a new channel:

conda config --add channels conda-forge

Remove unused packages and caches

To remove unused packages and caches

conda clean --all

Environments list

To see a list of all available environments run:

conda env list

References

calcoloscientifico/cluster/softwareapplicativo/conda_user.txt · Ultima modifica: 22/04/2021 14:16 da fabio.spataro