Custom Software Installation¶

This page explains how to access the software stack and how to install additional software inside the secure sciCORE+ environment.

Software modules¶

sciCORE+ provides the EESSI software stack. You can find the complete list of available software in the EESSI documentation.

First time use delays

Software modules are downloaded on demand, so the first time you use a module it can take some minutes to download the required data. Subsequent runs use the local cache and are much faster.

To use the software modules, first run enable-software-stack-eessi and then use the module command as you would in the regular sciCORE cluster, e.g.

$ enable-software-stack-eessi
$ module avail
$ module load SAMtools

Tip: SLURM scripts

In SLURM batch scripts, run source /etc/profile.d/soft_stacks.sh before enable-software-stack-eessi. See the SLURM batch jobs example.

Conda environments¶

You can create conda environments in your home folder and install any conda package using Miniforge:

$ wget https://nexus.scicoreplus.unibas.ch/repository/miniforge-installers/latest/download/Miniforge3-Linux-x86_64.sh
$ bash Miniforge3-Linux-x86_64.sh
$ source ~/.bashrc
$ conda create --name myenv
$ conda activate myenv
(myenv) $ conda install samtools

Python packages¶

You can install Python packages from pypi.org in a virtual environment.

Using Python from the software modules¶

$ enable-software-stack-eessi
$ module load Python
$ python3 -m venv venvtest
$ source venvtest/bin/activate
(venvtest) $ pip install numpy

Using Python installed as a conda package¶

First install Miniforge as described in Conda environments. Then create an environment with Python and install packages with pip:

$ conda create --name mypython python
$ conda activate mypython
(mypython) $ pip install numpy

R packages¶

You can install R packages from CRAN and Bioconductor.

Using R from the software modules¶

$ enable-software-stack-eessi
$ module load R
$ R
> install.packages("BiocManager")
> BiocManager::install("stringr")

The software modules also provide ready-to-use Bioconductor installations:

$ enable-software-stack-eessi
$ module spider R-bundle-Bioconductor
$ module load R-bundle-Bioconductor

Using R installed as a conda package¶

First install Miniforge as described in Conda environments. Then create an environment with R and install packages from R:

$ conda create --name myr r-base
$ conda activate myr
(myr) $ R
> install.packages("BiocManager")
> BiocManager::install("stringr")

Installing R packages from GitHub¶

GitHub is not accessible from sciCORE+, so if a package is not available in CRAN or Bioconductor you must download it on your local computer, copy it to sciCORE+ and install it from the local file.

On your local computer, download the package source from GitHub as a .tar.gz archive, e.g. for the main branch of repository <user>/<repo>:
```
$ wget https://github.com/<user>/<repo>/archive/refs/heads/main.tar.gz \
    -O <repo>.tar.gz
```
You can also download a release archive from the “Releases” page of the repository.
Copy the archive to your home folder in sciCORE+ using SFTP (see SFTP for how to connect):
```
$ sftp <username>@<login-node-hostname>
sftp> put <repo>.tar.gz
sftp> exit
```
Inside sciCORE+, load R (from the software modules or from your conda environment) and install the package from the local file:
```
$ R
> install.packages("~/<repo>.tar.gz", repos = NULL, type = "source")
```

Installing dependencies

When installing from a local file, R does not install the package dependencies automatically. Install any missing dependencies from CRAN or Bioconductor first, as shown above.

Containers with Apptainer¶

You can use Apptainer to download Docker images from public registries.

The Apptainer version to use

Use the local Apptainer installation in /usr/bin/apptainer. The Apptainer version provided by the software modules lacks some functionality and should not be used.

These Docker registries are accessible:

https://hub.docker.com/
https://quay.io/
https://gcr.io
https://cloud.google.com/container-registry/

The registries are accessed through a proxy, so you must add the proxy URL (nexus.scicoreplus.unibas.ch:444) to your pull command.

Downloading a Docker image from Docker Hub¶

If you had unrestricted Internet access you would download the BioContainers SAMtools image with apptainer pull docker://biocontainers/samtools:v1.9-4-deb_cv1. In sciCORE+ use the proxy URL instead:

$ apptainer pull docker://nexus.scicoreplus.unibas.ch:444/biocontainers/samtools:v1.9-4-deb_cv1

Downloading BioContainers images¶

The BioContainers project provides a curated list of container images with many bioinformatics tools:

$ apptainer pull docker://nexus.scicoreplus.unibas.ch:444/biocontainers/blast:2.2.31

Downloading big Docker images¶

If your image is too big to fit on the local disk while pulling it, set the APPTAINER_TMPDIR and APPTAINER_CACHEDIR environment variables to a folder in your home directory:

$ mkdir -p $HOME/tmp/cache
$ APPTAINER_TMPDIR=$HOME/tmp/ APPTAINER_CACHEDIR=$HOME/tmp/cache apptainer pull docker://nexus.scicoreplus.unibas.ch:444/freesurfer/freesurfer:7.3.1