Conda []

Questa pagina è in sola lettura. Puoi visualizzare il sorgente, ma non puoi modificarlo. Contatta l'amministratore se pensi che ci sia un errore.
===== Conda =====

Conda is an open source, cross-platform, language-agnostic package manager and environment management system. It is released under the Berkeley Software Distribution License by Continuum Analytics.

Conda allows users to easily install different versions of binary software packages and any required libraries appropriate for their computing platform. Also, it allows users to switch between package versions and download and install updates from a software repository.

Conda is written in the Python programming language, but can manage projects containing code written in other languages (e.g., R), including multi-language projects. Conda can install the Python programming language, while similar Python-based cross-platform package managers (such as wheel or pip) cannot.


A set of Conda [[https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#building-identical-conda-environments |environments]] has been defined on the HPC system.

[[https://www.hpc.unipr.it/dokuwiki/doku.php?id=calcoloscientifico:priv:cluster:conda | Conda configuration on the HPC cluster]] (private area)

==== Virtualenv list ====

To see a list of all available environments run:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda info --envs
</code>

==== Virtualenv "base" ====

Enable the "base" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate base
</code>

Disable the "base" environment:

<code>
conda deactivate
</code>

==== Virtualenv "statistical-genetics" ====

Available versions of the "statistical-genetics" environment:

^ condaenv                         ^ **build date**  ^
| statistical-genetics             | 2020.06.29      |
| statistical-genetics-2020.03.14  | 2020.03.14      |
| statistical-genetics-2020.06.29  | 2020.06.29      |

Enable the default version of the "statistical-genetics" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate statistical-genetics
</code>

Enable the 2020.03.14 version of the "statistical-genetics" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate statistical-genetics-2020.03.14
</code>

Packages in the "statistical-genetics" environment (any version):

<code>
conda list
</code>

Disable the "statistical-genetics" environment (any version):

<code>
conda deactivate
</code>

Now we will see how to run the scripts that you find on the [[calcoloscientifico:userguide:r|R]] page.

Script ''slurm-statistical-genetics-vrt.sh'':

<code bash slurm-statistical-genetics-vrt.sh>
#!/bin/bash
#SBATCH --job-name=hello
#SBATCH --output=%x.o%j
#SBATCH --error=%x.e%j
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --mem=512M
#SBATCH --partition=vrt
#SBATCH --qos=vrt
#SBATCH --time=0-00:05:00
##SBATCH --account=<account>

module load miniconda3

source "$CONDA_PREFIX/etc/profile.d/conda.sh"

conda activate statistical-genetics

Rscript --no-save hello.R

conda deactivate
</code>

Script ''slurm-statistical-genetics-cpu.sh'':

<code bash slurm-statistical-genetics-cpu.sh>
#!/bin/bash
#SBATCH --job-name=loop
#SBATCH --output=%x.o%j
#SBATCH --error=%x.e%j
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=4
#SBATCH --mem=512M
#SBATCH --partition=cpu
#SBATCH --qos=cpu
#SBATCH --time=0-00:05:00
##SBATCH --account=<account>

module load miniconda3

source "$CONDA_PREFIX/etc/profile.d/conda.sh"

conda activate statistical-genetics

Rscript --no-save loop.R ${SLURM_NTASKS_PER_NODE:-1}

conda deactivate
</code>

==== Virtualenv "machine-learning-cuda" ====

Main packages included in 2022 condaenvs:

<code>
conda install:
 - cudatoolkit
 - pip
 - mpi4py
 - h5py
 - keras
 - opencv
 - opencv-python-headless
 - tensorflow
 - torchvision
 - easydict
 - pytorch 
 - statsmodels
 - flax
 - transformers

pip install:
 - keras.datasets
 - sklearn
</code>

Available versions of the "machine-learning-cuda" environment:

^ condaenv                                 ^ build date  ^
| machine-learning-cuda-11.6.0             | 2022.05.20  |
| machine-learning-cuda-11.6.0-2022.05.20  | 2022.05.20  |
| machine-learning-cuda-11.6.0-2022.03.09  | 2022.03.09  |
| machine-learning-cuda-11.4.2             | 2022.05.20  |
| machine-learning-cuda-11.4.2-2022.05.20  | 2022.05.20  |
| machine-learning-cuda-11.4.2-2022.03.09  | 2022.03.09  |

Main packages included in 2021 condaenvs:

<code>
conda install:
 - cudatoolkit
 - pip
 - mpi4py
 - h5py
 - keras
 - opencv
 - opencv-python-headless
 - tensorflow
 - torchvision
 - easydict
 - pytorch 

pip install:
 - keras.datasets
 - sklearn
</code>

Available versions of the "machine-learning-cuda" environment:

^ condaenv                                 ^ build date  ^
| machine-learning-cuda-11.2.2             | 2021.10.23  |
| machine-learning-cuda-11.2.2-2021.10.23  | 2021.10.23  |
| machine-learning-cuda-11.2.1             | 2021.10.23  |
| machine-learning-cuda-11.2.1-2021.10.23  | 2021.10.23  |
| machine-learning-cuda-11.2.0             | 2021.10.23  |
| machine-learning-cuda-11.2.0-2021.10.23  | 2021.10.23  |
| machine-learning-cuda-11.1               | 2021.10.22  |
| machine-learning-cuda-11.1-2021.10.22    | 2021.10.22  |
| machine-learning-cuda-11.0               | 2021.10.23  |
| machine-learning-cuda-11.0-2021.10.23    | 2021.10.23  |
| machine-learning-cuda-10.2               | 2021.10.22  |
| machine-learning-cuda-10.2-2021.10.22    | 2021.10.22  |
| machine-learning-cuda-10.1               | 2021.10.23  |
| machine-learning-cuda-10.1-2021.10.23    | 2021.10.23  |
| machine-learning-cuda-10.0               | 2021.10.23  |
| machine-learning-cuda-10.0-2021.10.23    | 2021.10.23  |
| machine-learning-cuda-9.0                | 2019.10.22  |
| machine-learning-cuda-9.0-2019.10.22     | 2019.10.22  |
| machine-learning-cuda-8.0                | 2019.10.22  |
| machine-learning-cuda-8.0-2019.10.22     | 2019.10.22  |

Creation scripts:

<code>
/hpc/share/tools/conda/bin/hpc-make-condaenv-machine-learning-cuda-10.0
/hpc/share/tools/conda/bin/hpc-make-condaenv-machine-learning-cuda-10.1
/hpc/share/tools/conda/bin/hpc-make-condaenv-machine-learning-cuda-10.2
/hpc/share/tools/conda/bin/hpc-make-condaenv-machine-learning-cuda-11.0
/hpc/share/tools/conda/bin/hpc-make-condaenv-machine-learning-cuda-11.1
/hpc/share/tools/conda/bin/hpc-make-condaenv-machine-learning-cuda-11.2.0
/hpc/share/tools/conda/bin/hpc-make-condaenv-machine-learning-cuda-11.2.1
/hpc/share/tools/conda/bin/hpc-make-condaenv-machine-learning-cuda-11.2.2
</code>

Enable the default version of the "machine-learning-cuda-10.1" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate machine-learning-cuda-10.1
</code>

Packages in the "machine-learning-cuda-XX" environment (any version):

<code>
conda list
</code>

Disable the "machine-learning-cuda" environment (any version):

<code>
conda deactivate
</code>

Script ''slurm-machine-learning-cuda.sh'':

<code bash slurm-machine-learning-cuda.sh>
#!/bin/bash
#SBATCH --job-name=conda
#SBATCH --output=%x.o%j
#SBATCH --error=%x.e%j
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --gres=gpu:p100:1
#SBATCH --partition=gpu
#SBATCH --qos=gpu
#SBATCH --mem=16G
#SBATCH --time=0-00:30:00
##SBATCH --account=<account>

module load miniconda3

source "$CONDA_PREFIX/etc/profile.d/conda.sh"

conda activate machine-learning-cuda-10.1

#...

conda deactivate
</code>

==== Virtualenv "cryolo-cpu" ====

Available versions of the "cryolo-cpu" environment:

| **condaenv**           ^ build date  ^
| cryolo-cpu             | 2020.09.18  |
| cryolo-cpu-2020.09.18  | 2020.09.18  |

Enable the default version of the "cryolo-cpu" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate cryolo-cpu
</code>

Enable the 2020.09.18 version of the "cryolo-cpu" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate cryolo-cpu-2020.09.18
</code>

Packages in the "cryolo-cpu" environment (any version):

<code>
conda list
</code>

Disable the "cryolo-cpu" environment (any version):

<code>
conda deactivate
</code>

Script ''slurm-cryolo-vrt.sh'':

<code bash slurm-cryolo-vrt.sh>
#!/bin/bash
#SBATCH --job-name=cryolo
#SBATCH --output=%x.o%j
#SBATCH --error=%x.e%j
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --mem=1G
#SBATCH --partition=vrt
#SBATCH --qos=vrt
#SBATCH --time=0-01:00:00
##SBATCH --account=<account>

module load miniconda3

source "$CONDA_PREFIX/etc/profile.d/conda.sh"

conda activate cryolo-cpu

cryolo_gui.py config config_cryolo.json 220 \
--filter LOWPASS \
--low_pass_cutoff 0.1 \
--num_cpu ${SLURM_NTASKS_PER_NODE:-1}

conda deactivate
</code>

Script ''slurm-cryolo-cpu.sh'':

<code bash slurm-cryolo-cpu.sh>
#!/bin/bash
#SBATCH --job-name=cryolo
#SBATCH --output=%x.o%j
#SBATCH --error=%x.e%j
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=4
#SBATCH --mem=4G
#SBATCH --partition=cpu
#SBATCH --qos=cpu
#SBATCH --time=0-01:00:00
##SBATCH --account=<account>

module load miniconda3

source "$CONDA_PREFIX/etc/profile.d/conda.sh"

conda activate cryolo-cpu

cryolo_gui.py config config_cryolo.json 220 \
--filter LOWPASS \
--low_pass_cutoff 0.1 \
--num_cpu ${SLURM_NTASKS_PER_NODE:-1}

conda deactivate
</code>

==== Virtualenv "cryolo-gpu" ====

Available versions of the "cryolo-gpu" environment:

^ condaenv               ^ build date  ^
| cryolo-gpu             | 2020.09.18  |
| cryolo-gpu-2020.09.18  | 2020.09.18  |

Enable the default version of the "cryolo-gpu" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate cryolo-gpu
</code>

Enable the 2020.09.18 version of the "cryolo-gpu" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate cryolo-gpu-2020.09.18
</code>

Packages in the "cryolo-gpu" environment (any version):

<code>
conda list
</code>

Disable the "cryolo-gpu" environment (any version):

<code>
conda deactivate
</code>

Script ''slurm-cryolo-gpu.sh'':

<code bash slurm-cryolo-gpu.sh>
#!/bin/bash
#SBATCH --job-name=cryolo
#SBATCH --output=%x.o%j
#SBATCH --error=%x.e%j
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --gres=gpu:1
#SBATCH --mem=1G
#SBATCH --partition=gpu
#SBATCH --time=0-01:00:00
##SBATCH --account=<account>

module load miniconda3

source "$CONDA_PREFIX/etc/profile.d/conda.sh"

conda activate cryolo-gpu

cryolo_gui.py config config_cryolo.json 220 \
--filter LOWPASS \
--low_pass_cutoff 0.1 \
--num_cpu ${SLURM_NTASKS_PER_NODE:-1}

conda deactivate
</code>

==== Virtualenv "mdanalysis" ====

Available versions of the "mdanalysis" environment:

^ condaenv          ^ **build date**  ^
| mdanalysis        | 2023.03.20      |
| mdanalysis-2.3.0  | 2023.03.20      |

Enable the default version of the "mdanalysis" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate mdanalysis
</code>

Enable the 2.3.0 version of the "mdanalysis" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate mdanalysis-2.3.0
</code>

Packages in the "mdanalysis" environment (any version):

<code>
conda list
</code>

Disable the "mdanalysis" environment (any version):

<code>
conda deactivate
</code>

Script ''slurm-mdanalysis-cpu.sh'':

<code bash slurm-mdanalysis-cpu.sh>
#!/bin/bash
#SBATCH --job-name=mdanalysis
#SBATCH --output=%x.o%j
#SBATCH --error=%x.e%j
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=4
#SBATCH --mem=20G
#SBATCH --qos=cpu
#SBATCH --partition=cpu
#SBATCH --time=0-01:00:00
##SBATCH --account=<account>

module load gnu7/7.3.0
module load python/3.7.2

module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate mdanalysis-2.3.0

python --version
python3 Dihedral_analysis-MDA.py

conda deactivate
</code>


Script ''Dihedral_analysis-MDA.py'':

<code pyton Dihedral_analysis-MDA.py>
#!/usr/bin/env python3


import numpy as np
import MDAnalysis as MDA
from matplotlib import pyplot as plt
#from MDAnalysis import analysis as MDAa
from MDAnalysis.analysis.dihedrals import Dihedral as MDAaD
import MDAnalysis.lib.distances as MDAld


## EDIT HERE the resname and traj-file-name(no dcd)
resn='CBP'  # MCB # MCN # PCZ
trajfn='1atm_300k_CBP_512_mainrun_2'   #'pCBP_shorttraj' ## change for others
# number of frames to skip 
nskip=500

# loading trajectory
u=MDA.Universe(resn+'-512-new.psf',trajfn+'.dcd')
#u=MDA.Universe('MCB-512-new.psf','mCBP_shorttraj.dcd')

nmols=512
nframes=len(u.trajectory)

#print('u.trajectory')
#for i, ttr in enumerate(u.trajectory[::2]):
#    print(i,ttr)

if resn=='CBP':
    dh_atoms=[["C39", "N32", "C36", "C35"], ["C8", "N1", "C5", "C4"], ["C34", "C33", "C2", "C3"]]
elif resn=='MCB':
    dh_atoms=[["C43", "N42", "C35", "C33"], ["C7", "N8", "C22", "C23"], ["C23", "C25", "C32", "C33"]]
elif resn=='MCN':
    dh_atoms=[["C7", "N8", "C22", "C23"], ["C43", "N42", "C35", "C33"], ["C23", "C25", "C32", "C33"] ]
# add PCZ case here
    
ndhs=len(dh_atoms)
print(ndhs)

fig, ax = plt.subplots(ndhs,1)

print('qua')
print(len(u.trajectory[::nskip]))

dhs=np.zeros( (len(u.trajectory[::nskip]),nmols,ndhs) )

bindh=np.linspace(-180, 180, 361)


for kkd, a4dh in enumerate(dh_atoms):

    dA1=[res.atoms.select_atoms("name "+a4dh[0]) for res in u.residues[:nmols]]
    dA2=[res.atoms.select_atoms("name "+a4dh[1]) for res in u.residues[:nmols]]
    dA3=[res.atoms.select_atoms("name "+a4dh[2]) for res in u.residues[:nmols]]
    dA4=[res.atoms.select_atoms("name "+a4dh[3]) for res in u.residues[:nmols]]

    for i1, ts in enumerate(u.trajectory[::nskip]):
        #for d11 in d1:
            #xa1,xa2,xa3,xa4=np.split(d11.positions,4)
            #print(xa1[0])
            #print(dC39)
            #x=Dihedrals(xa4[0],xa1[0],xa3[0],xa2[0])
        for jj in range(nmols):
            x1=MDAld.calc_dihedrals(dA1[jj],dA2[jj],dA3[jj],dA4[jj])*180./np.pi
            dhs[i1,jj,kkd]=x1
        #MDAdh(d11.positions[:3],d11.positions[3:6], d11.positions[6:9], d11.positions[9:])

    #print(dhs[:,:,kkd])
    
    ax[kkd].hist(dhs[:,:,kkd].flatten(), label=" ".join(a4dh), bins=bindh, alpha=0.6)
    ax[kkd].legend()




np.savetxt('dhs_'+resn+'_long_3.dat',dhs.reshape(-1,ndhs))

print('fffffff')
print(dhs)
print()
print('qqqqqq')
print(dhs.reshape(-1,ndhs))
dhsnew = []
dhsresh = dhs.reshape(-1,ndhs)
print(len(dhsresh[:,0]))
for i in range(0,len(dhsresh[:,0])):
    if dhsresh[i,0] != 0.0:
        dhsnew = np.append(dhsnew,dhsresh[i,:])

dhsnew = np.reshape(dhsnew,(int(len(dhsnew)/ndhs),ndhs))
#print(len(dhsnew))

fig.suptitle(resn, fontsize=16)
fig.savefig('dhs_'+resn+'_long_3.png')

</code>

==== Virtualenv "bioconda-ngs" ====

Available versions of the "bioconda-ngs" environment:

^ condaenv          ^ **build date**  ^
| bioconda-ngs         | 2023.05.31      |
| bioconda-ngs-py3.11  | 2023.05.31      |

Enable the default version of the "bioconda-ngs" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate bioconda-ngs
</code>

Enable the py3.11 version of the "bioconda-ngs" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate bioconda-ngs-py3.11
</code>

Packages in the "bioconda-ngs" environment (any version):

<code>
conda list
</code>

Disable the "bioconda-ngs" environment (any version):

<code>
conda deactivate
</code>

Script ''slurm-bioconda-ngs-cpu.sh'':

<code bash slurm-bioconda-ngs-cpu.sh>
#!/bin/bash
#SBATCH --job-name=mdanalysis
#SBATCH --output=%x.o%j
#SBATCH --error=%x.e%j
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=4
#SBATCH --mem=20G
#SBATCH --qos=cpu
#SBATCH --partition=cpu
#SBATCH --time=0-01:00:00
##SBATCH --account=<account>

module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate bioconda-ngs

conda list

conda deactivate
</code>

==== Virtualenv "pennylane" ====

Available versions of the "pennylane" environment:

^ condaenv        ^ **build date**  ^
| pennylane       | 2023.11.03      |
| pennylane-gpu   | 2023.11.03      |

Enable the default version of the "pennylane" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate pennylane
</code>

Enable the gpu version of the "pennylane" environment:

<code>
module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate pennylane-gpu
</code>

Packages in the "pennylane" environment (any version):

<code>
conda list
</code>

Disable the "pennylane" environment (any version):

<code>
conda deactivate
</code>

Script ''slurm-pennylane-gpu.sh'':

<code bash slurm-pennylane-gpu.sh>
#!/bin/bash
#SBATCH --job-name=plExample
#SBATCH --output=%x.o%j
#SBATCH --error=%x.e%j
#SBATCH --nodes=1
#SBATCH --partition=gpu
#SBATCH --qos=gpu
#SBATCH --gres=gpu:a100_40g:1
#SBATCH --mem=8G
#SBATCH --time=0-10:00:00
#SBATCH --ntasks-per-node 4
#SBATCH --account=<account>

module load miniconda3
source "$CONDA_PREFIX/etc/profile.d/conda.sh"
conda activate pennylane-gpu

PYTHON_VERSION=$(python -V | awk '{ print $2 }')
echo $PYTHON_VERSION

python -u ./plExample.py

conda deactivate
</code>


Script ''plExample.py'':

<code pyton plExample.py>
#!/usr/bin/env python3

# Simple example where we train a parameterized quantum circuit
# to minimize the expectation value of a Pauli-Z operator.
import pennylane as qml
from pennylane import numpy as np

# Define a quantum device with one qubit
dev = qml.device("default.qubit", wires=1)

# Define a variational quantum circuit
@qml.qnode(dev)
def circuit(params):
    qml.RX(params[0], wires=0)  # Apply a rotation around the X-axis
    qml.RY(params[1], wires=0)  # Apply a rotation around the Y-axis
    return qml.expval(qml.PauliZ(0))

# Define the cost function to minimize
def cost(params):
    return circuit(params)

# Initialize parameters
init_params = np.array([0.01, 0.01], requires_grad=True)

# Set up the optimizer
opt = qml.GradientDescentOptimizer(stepsize=0.1)

# Number of optimization steps
steps = 100

# Optimize the circuit parameters
params = init_params
for i in range(steps):
    params = opt.step(cost, params)
    if (i + 1) % 10 == 0:
        print(f"Step {i + 1}: Cost = {cost(params):.6f}, Params = {params}")

print(f"Optimized Parameters: {params}")
print(f"Final Cost: {cost(params)}")

</code>