Using custom conda environments on UCloud

Set up your “own” conda

  • Set up your own conda (instead of the default conda), following this link
    • The reason is because that the system default conda is cleared every time.
  • After the correct set up, you can verify by: which conda
    • If it returns /work/miniconda3/bin/conda, it’s correctly setted up.

Create a conda environment

  • The conda should be in a custom place: conda create --prefix "/work/INFIMM Public/miniconda3/envs/scverse" python=3.12

  • A separate conda environment for each analysis. For example, in the case below, sc_gpu is used for GPU computation of single cell analysis and scverse is used for CPU comutation of single cell analysis.

sc_gpu                   /work/miniconda3/envs/sc_gpu
scverse                  /work/miniconda3/envs/scverse

Mount a conda env

When submitting the job, mount the conda folder, in our case /work/INFIMM Public/miniconda3/. This path will be accessible in the application as /work/miniconda3.

(For the first time) Execute the following command

  • Start the terminal interface by conda init
  • Restart the terminal interface
  • Activate the conda env by conda activate "/work/INFIMM Public/miniconda3/envs/scverse"
  • Install ipykernel conda install ipykernel
  • Add the environment as a Jupyter kernal python -m ipykernel install --user --name cellxgene --display-name "Python (cellxgene)"

(Wrap the set up in an) Initiation script

#!/bin/bash
eval "$(/work/miniconda3/bin/conda shell.bash hook)"
conda init
conda activate scverse
python -m ipykernel install --user --name scverse --display-name "Python (scverse)"

Additional infomration for creating a GPU accelerated sc analysis environment

Set up (only run once)

  • Use the custom conda. eval "$(/work/miniconda3/bin/conda shell.bash hook)

  • The conda environment should be in a custom place: conda create --prefix /work/miniconda3/envs/sc_gpu python=3.12

  • Activate the environment: conda activate sc_gpu

  • Install PyTorch and JAX

    • Identify the CUDA version (such as 12.4) by
      • nvidia-smi or
      • nvcc --version or
      • cat /usr/local/cuda/version.txt or
      • dpkg -l | grep cuda
    • Install the PyTorch with the appropriate command. conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia
    • Install JAX pip install -U "jax[cuda12]"

  • Install scvi conda install scvi-tools -c conda-forge

  • Install scanpy conda install -c conda-forge scanpy python-igraph leidenalg

  • Install ipykernel conda install ipykernel

  • Add the environment as a Jupyter kernal python -m ipykernel install --user --name sc_gpu --display-name "Python (sc_gpu)"