Specific Application How-Tos¶

Here is a collection of instructions to run some applications of interest in the cluster.

Tmux for persistent shell sessions¶

Tmux is a “terminal multiplexer” which allows multiple terminal sessions to be accessed simultaneously in a single window. Processes can be detached from their controlling terminals, allowing remote sessions to remain active without being visible. This is particularly useful to protect programs running on a remote server (e.g. a sciCORE login node) from connection drops (e.g. when going through a tunnel on the train ).

This software is already installed on all sciCORE login nodes. We recommend checking the official manual, and provide here a quick cheat sheet of common operations in Tmux:

tmux new -s <name>  # create new session

tmux ls # list active tmux sessions

tmux a -t <name>  # attach target

ctrl-b  # base action call

ctrl-b c # create new tab in session (asterisk indicates the active tab)

ctrl-b n # next tab
ctrl-b p # previous tab

ctrl-b [ # navigation mode indicated by [0/199], use q to quit

ctrl-b &  # kills the currently active tab

ctrl-b d # detach and leave tabs running

AlphaFold¶

AlphaFold is a protein structure prediction database and program designed by DeepMind using deep learning techniques.

You can run the inference pipeline of AlphaFold in the sciCORE cluster. To explore the AlphaFold versions available run:

ml spider AlphaFold

Once you choose a version and load it with ml AlphaFold/<version>, two scripts will be available to you, run_alphafold.sh and run_alphafold_multimer.sh. For instance:

$ ml AlphaFold/2.3.2
$ which run_alphafold.sh
/scicore/soft/easybuild/apps/AlphaFold/2.3.2/bin/run_alphafold.sh

Under the hood, these scripts will run AlphaFold as if it was run via docker container. See the official documentation for information on the parameters you can pass to the script.

The monomer is the original model used at CASP14 with no ensembling. Here’s an example SLURM script to run AlphaFold with a monomer:

#!/bin/bash
#SBATCH --job-name=AlphaFold-Monomer-Example
#SBATCH --time=01:00:00
#SBATCH --mem=64G
#SBATCH --cpus-per-task=8
#SBATCH --partition=a100
#SBATCH --gres=gpu:1

module load AlphaFold/2.3.2

run_alphafold.sh \
    --fasta_paths /scicore/home/group/user/path/to/fasta/fasta.fa \
    --output_dir /scicore/home/group/user/path/to/output/ \
    --max_template_date 2021-09-01

export DB_PRESET="reduced_dbs"
run_alphafold.sh <remaining parameters>

The multimer is AlphaFold’s Multimer model. Here’s an example SLURM script to run AlphaFold with a multimer:

#!/bin/bash
#SBATCH --job-name=AlphaFold-Monomer-Example
#SBATCH --time=01:00:00
#SBATCH --mem=64G
#SBATCH --cpus-per-task=8
#SBATCH --partition=a100
#SBATCH --gres=gpu:1

module load AlphaFold/2.2.0

run_alphafold_multimer.sh \
    --fasta_paths /scicore/home/group/user/path/to/fasta/fasta.fa \
    --output_dir /scicore/home/group/user/path/to/output/ \
    --max_template_date 2021-09-01

Alphafold3¶

AlphaFold is a protein structure prediction database and program designed by DeepMind using deep learning techniques.

--max_template_date=$(date +%F)

As a software module¶

We have a built an apptainer container and companion software module that you can use with the following instructions:

$> ml AlphaFold3/3.0.1
# Important! you need to provide the path to the proprietary model parameters via --model_dir
#
# Usage:
#   run_alphafold.sh --model_dir=<path to model params> --output_dir=<path to output> --input_dir=<path to input>
# or:
#   run_alphafold.sh --model_dir=<path to model params> --output_dir=<path to output> --json_path=<path to json>
#
# For help run:
#   run_alphafold.sh --help

Directly within sbatch¶

The software module takes care of setting up some required environment variables for ease of use, but they can be set by manually and used within sbatch if needed:

• Setting up required environment variables (makes running/editing the command simpler)

# MODEL_DIR_PATH: Path for the AF3 model parameters you need to provide after requesting them
export MODEL_DIR_PATH="/path_to_model_parameters/af3.bin.zst"

# ALPHAFOLD3_PUBLIC_DATABASES: Path to the downloaded databases, we have them downloaded centrally so no need to re-download them
export ALPHAFOLD3_PUBLIC_DATABASES="/scicore/data/managed/AlphaFold3_DBs/frozen_250702T000000/"

# ALPHAFOLD3_APPTAINER_IMAGE: Path to the alphafold3 singularity image, in the cluster is
export ALPHAFOLD3_APPTAINER_IMAGE="/scicore/soft/containers/apptainer/alphafold/alphafold_3.0.1.sif"

• If you plan to work with large models that could require more than the available memory in a GPU (A100 40GB) you should also set the following, as described in the performance guide (remember to request enough memory for your job as well).

# Set the following vars for problems where the memory of a GPU might not be enough
export APPTAINERENV_TF_FORCE_UNIFIED_MEMORY=true
export APPTAINERENV_XLA_PYTHON_CLIENT_MEM_FRACTION="3.2"
export APPTAINERENV_XLA_PYTHON_CLIENT_PREALLOCATE=false

• To run Alphafold inside a slurm job you should then use something like this, replacing the placeholders with your parameters and extending flags as needed:

export MODEL_DIR_PATH="/path_to_model_parameters" # you need to request
export ALPHAFOLD3_PUBLIC_DATABASES="/scicore/data/managed/AlphaFold3_DBs/frozen_250702T000000/"
export ALPHAFOLD3_APPTAINER_IMAGE="/scicore/soft/containers/apptainer/alphafold/alphafold_3.0.1.sif"
apptainer run \
    -B ${ALPHAFOLD3_PUBLIC_DATABASES}:/app/public_databases \
    --nv \
    ${ALPHAFOLD3_APPTAINER_IMAGE} /alphafold3_venv/bin/python /app/alphafold/run_alphafold.py \
    # here you need to add any other parameters including  --model_dir pointing to the model parameters
    --model_dir=${MODEL_DIR_PATH} \
    --output_dir=<path to output> \
    --json_path=<path to input json> # or --input_path=<path to input>

Loading and using LLMs on the cluster¶

The compute nodes do not in general have access to the Internet. So you may have run into errors when trying to follow an LLM tutorial online because the model files could not be fetched at runtime.

The simple solution to that is to download the model files you need to the cluster beforehand. Then point to these local files when running your code from a compute node.

For example, if you have huggingface-cli installed in your (virtual) environment and want to download the Qwen/Qwen3-4B model, you can run the following from a transfer node:

MODEL=Qwen/Qwen3-4B
OUTDIR=path/to/your/local/Qwen3-4B

mkdir -p ${OUTDIR}

huggingface-cli download ${MODEL} config.json model*safetensors* tokenizer* \
    --local-dir ${OUTDIR} \
    --local-dir-use-symlinks False

This will download the model files to the specified OUTDIR.

Then, when running your code from a compute node, you can point to the local files like this:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "path/to/your/local/Qwen3-4B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

SRA Toolkit¶

The Sequence Read Archive (SRA) Toolkit a collection of tools and libraries for using data in the The International Nucleotide Sequence Database Collaboration (INSDC).

The SRA Toolkit is available via the cluster’s module system. To explore the available versions, run:

ml spider SRA-Toolkit

Once you load a version with ml SRA-Toolkit/<version>, commands like fastq-dump and fasterq-dump will be available to you. Check the official documentation for more information on commands.

Running PostgreSQL on the cluster¶

PostgreSQL is a popular object-relational database system, but running it on the cluster requires some configuration due to the lack of root privileges. Below are two approaches that run PostgreSQL in an isolated environment within your user space.

Via Apptainer¶

The Apptainer container system is available by default to all users:

$ which apptainer
/usr/bin/apptainer

To run PostgreSQL with Apptainer, you must first pull a PostgreSQL image from a container registry. For example, you can use the official PostgreSQL image from Docker Hub:

apptainer pull docker://postgres:latest

This should create a file named postgres_latest.sif in your current directory. This is the image where the PostgreSQL commands will run from.

Now, create folders to store your PostgreSQL databases and temporary files:

mkdir -p $HOME/pg_data
mkdir -p $HOME/pg_tmp

Pointing to the image you pulled, you can start a PostgreSQL server with the in the login node:

$ apptainer instance start \
    -B $PG_DBS_PATH:/var/lib/postgresql/data \
    -B $PG_TMP_PATH:/var/run/postgresql \
    postgres_latest.sif \
    postgres
INFO:    instance started successfully

You can verify that the instance is running with:

apptainer instance list

Now, connect to the PostgreSQL server from inside the container:

$ apptainer shell instance://postgres
Apptainer>

Inside the container, all PostgreSQL commands are available to you. For example, you can initialize a new database with:

initdb "./my_db"

When you’re done working, you should stop the PostgreSQL instance with:

apptainer instance stop postgres

Via pixi¶

PostgreSQL is available on conda-forge. So if you have the pixi package-manager in your user space, you can install PostgreSQL as a global tool with the following command:

pixi global install postgresql

This exposes commands like createdb, initdb, pg_ctl, psql, … to the user’s PATH:

$ which psql
/scicore/home/pi/user/.pixi/bin/psql

You can then use PostgreSQL commands as you would normally do if you had root access:

# Initialize database at the `my_db` directory
initdb "./my_db"