Expanse GPU

Expanse GPU has 52 nodes with four NVIDIA V100s (32 GB each) linked by NVLink. Suits multi-GPU AI/ML training and other workloads that scale across GPUs.

Get Support

GET AN ACCOUNT ON EXPANSE GPU

2FA Enroll Link

Jobs

 

GPU nodes are allocated as a separate resource. The GPU nodes can be accessed via either the "gpu" or the "gpu-shared" partitions.

#SBATCH -p gpu

or

#SBATCH -p gpu-shared

 

When users request 1 GPU, in gpu-shared partition, by default they will also receive, 1 CPU, and 1G memory.  

For more information and example job scripts see the Expanse User Guide

https://www.sdsc.edu/systems/expanse/user_guide.html#narrow-wysiwyg-7

The charging equation will be:

GPU SUs = (Number of GPUs) x (wall clock time)

Regular SUs = (Number of Cores) x (wall clock time)

Queue specifications

Metrics updated 2026-06-16

Name Purpose Nodes CPU cores / node GPUs / node Node RAM Jobs
30 days
 Wait Time
30-day trend
 Wall Time
30-day trend
gpu Used for exclusive access to the GPU nodes 56 Xeon Gold 6248 (40 cores) 4 NVIDIA V100 SXM2 (32 GB vRAM) 384 GB 643
gpu wait time: average 6.0 hours, range 0 to 21.6 hours over 30 days
gpu wall time: average 6.6 hours, range 0 to 11.7 hours over 30 days, wall-time limit 48h
gpu-debug Priority access to gpu-shared nodes set aside for testing of jobs with short wall time and limited resources; max two gpus per job 2 Xeon Gold 6248 (40 cores) 4 NVIDIA V100 SXM2 (32 GB vRAM) 384 GB 554
gpu-debug wait time: average 0.1 hours, range 0 to 46.2 hours over 30 days
gpu-debug wall time: average 0.2 hours, range 0 to 0.5 hours over 30 days, wall-time limit 30m
gpu-shared Non-refundable discounted jobs to run on unallocated nodes that can be pre-empted by higher priority queues 56 Xeon Gold 6248 (40 cores) 4 NVIDIA V100 SXM2 (32 GB vRAM) 384 GB 11,196
gpu-shared wait time: average 1.9 hours, range 0 to 18.7 hours over 30 days
gpu-shared wall time: average 6.2 hours, range 0.9 to 10.4 hours over 30 days, wall-time limit 48h
nairr-gpu Expanse AI resource using compute with Nvidia GPUs 34 2 Intel Sapphire Road processors (36 cores) 4 Nvidia H100 (80 GB vRAM) 1 TB — — —

Datasets

Name Description
OceanTopography

OpenTopography provides efficient, user-friendly access to high-resolution topography data, processing tools, and resources to advance understanding of the Earth's surface, vegetation, and built environment.
OpenAltimetry

OpenAltimetry is a web based data visualization and discovery tool for exploring surface elevation profiles over time using satellite altimetry data from NASA's ICESat and ICESat-2 missions.
OpenForest4D

OpenForest4D is a web-based platform that leverages multi-source remote sensing data and artificial intelligence to generate on-demand, research-grade estimates of forest structure and above-ground biomass in four dimensions for global forest monitoring. 

Storage

File System

Directory Path Quota Purge Backup Notes
Scratch Lustre /expanse/lustre/scratch 10 TB 90 days after allocation expiration. No backups stored. This is not an archival file system, it is not backed up, and will be purged according to purge policy.
Scratch GPU Node /scratch/$USER/job_$SLURM_JOB_ID 1600 GB Users only have access to these SSDs during job execution at the local file system path to the gpu node.
Home /home 100 GB 8 week rolling backup The home directory is limited in space and should be used only for source code storage. Jobs should never be run from the home file system, as it is not set up for high performance throughput.

External Storage

In addition to the local scratch storage, users will have access to global parallel filesystems on Expanse. Every Expanse node has access to a 12 PB Lustre parallel file system and a 7 PB Ceph Object Store system, 140 GB/second performance storage. For more information on this see the Expanse User Guide - https://www.sdsc.edu/systems/expanse/user_guide.html#narrow-wysiwyg-10

File Transfer

Data transfer on expanse is done through Globus 

For more information on data transfer see the Expanse User Guide 

https://www.sdsc.edu/systems/expanse/user_guide.html#narrow-wysiwyg-9

Supported Methods Data Transfer Node URL
GLOBUS /expanse/projects