CONDA user defined environments []

Questa pagina è in sola lettura. Puoi visualizzare il sorgente, ma non puoi modificarlo. Contatta l'amministratore se pensi che ci sia un errore.
===== CONDA user defined environments =====

Users or groups can build their own environments.

==== General template ====

General template that can be used to create personal or group environments:

<code bash hpc-make-condaenv-template.sh>
#!/bin/bash

set -e

module load miniconda3

source "$CONDA_PREFIX/etc/profile.d/conda.sh"

CONDAENV_NAME='template'
CONDAENV_VERSION=$(date '+%Y.%m.%d')
PYTHON_VERSION='3.9'

CONDAENV="${CONDAENV_NAME}-${CONDAENV_VERSION}"

if [ $UID = 0 ]; then
    CONDAENV_PREFIX="/hpc/share/tools/miniconda3/envs/$CONDAENV"
    CONDAENV_SYMLNK="/hpc/share/tools/miniconda3/envs/$CONDAENV_NAME"
else
    if [ -z "$CONDA_ENVS_PATH" ]; then
        echo 'Error: the CONDA_ENVS_PATH envarionment variable is not set' 1>&2
        exit 1
    fi
    if [ -z "$CONDA_PKGS_DIRS" ]; then
        echo 'Error: the CONDA_PKGS_DIRS envarionment variable is not set' 1>&2
        exit 1
    fi
    echo "CONDA_ENVS_PATH: $CONDA_ENVS_PATH"
    echo "CONDA_PKGS_DIRS: $CONDA_PKGS_DIRS"
    CONDAENV_PREFIX="$CONDA_ENVS_PATH/$CONDAENV"
    CONDAENV_SYMLNK="$CONDA_ENVS_PATH/$CONDAENV_NAME"
fi

if [ $UID = 0 ]; then
    conda --version
    conda update conda
    conda --version
    echo
fi

echo "CONDAENV_PREFIX: $CONDAENV_PREFIX"
echo "CONDAENV_SYMLNK: $CONDAENV_SYMLNK"
echo

if [ -d "$CONDAENV_PREFIX" ]; then
    echo -n "Should we remove the already existing condaenv '$CONDAENV_NAME' (yes/no)? "

    read ans

    case "$ans" in
        [yY]|[yY][Ee][Ss])
        conda env remove --name "$CONDAENV"
        ;;
        [nN]|[nN][oO])
        ;;
        *)
        echo
        exec "$0"
        exit
        ;;
    esac
fi

if [ -d "$CONDAENV_PREFIX" ]; then
    echo -n "Should we proceed with the modification of the condaenv '$CONDAENV_NAME' (yes/no)? "
else
    echo -n "Should we proceed with the creation of the condaenv '$CONDAENV_NAME' (yes/no)? "
fi

read ans

case "$ans" in
    [yY]|[yY][Ee][Ss])
    ;;
    [nN]|[nN][oO])
    exit 1
    ;;
    *)
    echo
    exec "$0"
    exit
    ;;
esac

conda info

if [ ! -d "$CONDAENV_PREFIX" ]; then
    conda create --yes --prefix "$CONDAENV_PREFIX" python${PYTHON_VERSION:+=$PYTHON_VERSION}
fi

conda activate "$CONDAENV"

conda install --yes --name "$CONDAENV" \
    --channel conda-forge \
    xz

conda list || true

conda deactivate

ln -sf "$CONDAENV_PREFIX" "$CONDAENV_SYMLNK"
ls -la "$CONDAENV_SYMLNK"*
</code>

This script is located in

<code>
/hpc/share/tools/conda/bin/hpc-make-condaenv-template
</code>

Customize your script by editing CONDAENV_NAME, PYTHON_VERSION and the "conda install" command.

The "conda install" command can be repeated multiple times with different options and parameters.

It is also possible to install python packages using the "pip" command.


==== Creating a user environment ====

Create a script from the template and run it.

==== Creating a group environment ====

Change group:

<code>
newgrp <GROUP>
</code>

Create a script from the template and run it.

==== Working with conda ====

With conda, you can create, export, list, remove, and update that have different versions of Python and/or packages installed in them. Switching or moving between environments is called activating the environment.

=== Interactive work session ===

If you want to work at group level change group otherwise skip this step:

<code>
newgrp <GROUP>
</code>

Typical work session with conda (CPU):

<code>
[<USER>@ui01 ~]$ srun --nodes=1 --ntasks-per-node=2 --partition=cpu --qos=cpu --mem=8G --pty bash

[<USER>@wn80 ~]$ module load miniconda3

[<USER>@wn80 ~]$ source "$CONDA_PREFIX/etc/profile.d/conda.sh"

[<USER>@wn80 ~]$ conda --help

[<USER>@wn80 ~]$ conda --version
conda 25.7.0

[<USER>@wn80 ~]$ conda info

[<USER>@wn80 ~]$ conda activate

(base) [<USER>@wn80 ~]$ which python
/hpc/share/tools/miniconda3/bin/python

(base) [<USER>@wn80 ~]$ python -V
Python 3.9.18

(base) [<USER>@wn80 ~]$ conda create --name my_conda_env [...]
(base) [<USER>@wn80 ~]$ conda activate my_conda_env

(my_conda_env) [<USER>@wn80 ~]$ [...]
(my_conda_env) [<USER>@wn80 ~]$ conda deactivate

(base) [<USER>@wn80 ~]$ conda deactivate

[<USER>@wn80 ~]$ exit

[<USER>@ui01 ~]$
</code>

Typical work session with conda (GPU):

<code>
[<USER>@ui01 ~]$ srun --nodes=1 --ntasks-per-node=2 --partition=gpu --qos=gpu --gres=gpu:p100:1 --mem=8G --pty bash

[<USER>@wn41 ~]$ module load miniconda3

[...]

[<USER>@wn41 ~]$ exit

[<USER>@ui01 ~]$
</code>

=== Package search ===

<code>
conda search sklearn
</code>

=== Alternate channels ===

To search for alternate channels that may provide the conda package you're looking for, navigate to [[https://anaconda.org|anaconda.org]] and use the search bar at the top of the page.

To add a new channel:

<code>
conda config --add channels conda-forge
</code>

=== Remove unused packages and caches ===

To remove unused packages and caches

<code>
conda clean --all
</code>

=== Environments list ===

To see a list of all available environments run:

<code>
conda env list
</code>

==== References ====

[[https://docs.conda.io/projects/conda/en/4.6.1/user-guide/tasks/manage-environments.html|Managing environments]]