Bridges-2 RM

Jobs on RM nodes are scheduled with Slurm and run in one of three partitions, depending on how much of a node you need.

Usage is charged in core-hours: one core for one hour is 1 SU. So a full 128-core node costs 128 SUs per hour, while a part-node job in RM-shared is charged only for the cores it uses. Specific details are documented on the Accounting for Usage page on the Bridges-2 guide.

• RM — for jobs that use one or more whole 256 GB nodes. You request the number of nodes (-N); each node gives you all 128 cores and is charged as 128 SUs per hour. Up to 64 nodes per job, 72-hour max walltime.
• RM-shared — for jobs that need only part of a single 256 GB node. You request a number of cores (--ntasks-per-node, up to 64) and are charged only for those — so 2 cores for 30 minutes costs just 1 SU. Jobs get 2 GB of memory per core and can't span nodes. Choosing RM-shared when you don't need a full node lowers your cost and usually shortens your wait.
• RM-512 — the same as RM, but for the 16 higher-memory 512 GB nodes; these are always allocated whole (up to 2 nodes per job).

All three default to a 1-hour walltime (72-hour max) and 1 node if you don't specify otherwise.

For more detail, please visit the Bridges-2 Regular Memory page or the Summary table for the RM partition on the Bridges-2 guide.

A sample job script for the RM partition would look like this:

#!/bin/bash
#SBATCH -N 1
#SBATCH -p RM
#SBATCH -t 5:00:00
#SBATCH --ntasks-per-node=128

# type 'man sbatch' for more information and options
# this job will ask for 1 full RM node (128 cores) for 5 hours
# this job would potentially charge 640 RM SUs

#echo commands to stdout
set -x

# move to working directory
# this job assumes:
# - all input data is stored in this directory
# - all output should be stored in this directory
# - please note that groupname should be replaced by your groupname
# - PSC-username should be replaced by your PSC username
# - path-to-directory should be replaced by the path to your directory where the executable is

cd /ocean/projects/groupname/PSC-username/path-to-directory

# run a pre-compiled program which is already in your project space

./a.out

After that, you need a sbatch command to submit a job to the RM partition. An example of a sbatch command to submit a job to the RM partition, requesting one node for 5 hours is

sbatch -p RM -t 5:00:00 -N 1 myscript.job

where:

-p indicates the intended partition
-t is the walltime requested in the format HH:MM:SS
-N is the number of nodes requested
myscript.job is the name of your batch script

Detailed information about batch jobs can be found in the Batch Jobs page of the guide.