Anvil CPU

Anvil CPU is a compute cluster consisting of 1,000 nodes with two 64-core AMD EPYC 7763 processors and 256 GB of memory each, plus 32 large-memory nodes with 1 TB. It is particularly well suited for CPU-bound work that scales across many cores or many nodes, and is often used for simulation, modelling and large-scale data processing. It includes a great deal of general-purpose scientific software, with compilers, MPI libraries and domain packages loaded through the module system.

Submitting Jobs Documentation

You can run jobs at different sizes and durations on Anvil CPU. The following lists the different queues that you can submit to, describing how many nodes you get, how long you can run, the type of resources you get, and the average wait time.

Jobs are submitted through the Slurm scheduler. Run batch jobs with sbatch, start interactive work with srun or salloc, or launch an application through Open OnDemand.

Always name the partition you want with -p. If you do not, the job goes to shared, where jobs from different users run on the same node and you are charged for the cores you request or the share of memory you request, whichever is larger. The wholenode and wide partitions are node-exclusive: even a one-core request is given an entire node and charged for all 128 cores, and squeue reports it as 128 cores. The highmem partition is charged at a higher rate than the regular partitions.

Jobs can run for up to four days on up to 16 full nodes, and the largest job Anvil accepts is 7,168 cores - anything larger needs a consulting ticket. Per-queue wall-clock limits are in the queue table; if you do not set one, the partition default applies, which scontrol show partition reports.

For more information about running jobs on Anvil, see [Anvil Running Jobs].

Queue specifications Documentation

Metrics updated 2026-09-29

Queue	CPU cores / node	GPUs / node	Num nodes	Node RAM	Max wallclock	Number of jobs run 30 days
highmem Jobs that need more memory on one node than a regular node provides. Charged at a higher rate, so use it only when the memory is the reason.	2x AMD EPYC 7763 (128 cores)	None	32	1 TB	48h	4,653
shared Jobs that need only part of a node, charged for the cores or the memory share you request. This is where your job goes if you do not name a partition.	2x AMD EPYC 7763 (128 cores)	None	250	256 GB	96h	585,869
debug Short test and debugging runs, with priority access so you wait less. Use it before submitting production work.	2x AMD EPYC 7763 (128 cores)	None	17	256 GB	2h	527
wide Large multi-node jobs that need many nodes for a short time. Node-exclusive.	2x AMD EPYC 7763 (128 cores)	None	746	256 GB	12h	450
wholenode Production runs that need whole nodes to themselves. Use this for most multi-node work.	2x AMD EPYC 7763 (128 cores)	None	750	256 GB	96h	23,922

Software Documentation

The following software packages are among the most frequently used on Anvil CPU, based on job data from XDMoD.

Most Frequently Used

Application	Description	Research Discipline	Jobs
python	Python is a high-level, interpreted programming language known for its simplicity and readability. It supports multiple programming paradigms and has a vast ecosystem of libraries and frameworks.	Computer & Information Sciences, Software Engineering, Systems & Development	100,376
lammps	LAMMPS (Large-scale Atomic/Molecular Massively Parallel Simulator) is a classical molecular dynamics code designed for simulating large-scale atomistic systems. It is highly versatile and can be used to model a wide range of materials and complex molecular structures.	Chemical Sciences	30,277
orca	Orca is a flexible, efficient, and powerful quantum chemistry software written by Frank Neese and co-workers. It provides accurate and reliable computational simulations of molecular structures and properties based on quantum mechanical methods.	Chemical Sciences	11,730
q-espresso	Quantum ESPRESSO is an integrated suite of computer codes for electronic-structure calculations and materials modeling at the nanoscale.	Condensed Matter Physics	5,767
gamess	The General Atomic and Molecular Electronic Structure System (GAMESS) is a suite of ab initio quantum chemistry programs designed for electronic structure calculations of molecules and molecular reactions. It provides a wide range of methods for both molecules and periodic solids.	Chemical Sciences	3,819
r	R is a free software environment for statistical computing and graphics. It compiles and runs on a wide variety of UNIX platforms, Windows, and MacOS.	Computer Science	3,444
gromacs	GROMACS (GROningen MAssive Parallel MD for Molecular Dynamics) is a versatile package for molecular dynamics simulations with a strong emphasis on high-performance computing capabilities.	Biological Sciences	3,256
spec			3,125
openfoam	OpenFOAM (Open Source Field Operation and Manipulation) is a free, open-source computational fluid dynamics (CFD) software package developed by the OpenFOAM Foundation. It is widely used for simulating complex fluid flows in various industries and academic research.	Physical Sciences	2,724
cactus			2,279

Storage Documentation

Anvil has three storage areas for your own data: Home for personal files, scripts and small software; Scratch for high-speed job input and output; and Projects for data shared across your allocation.

Scratch is temporary, and deletions there are silent and permanent, with no warning and no way to recover a file. The snapshots on Home and Projects are not backups, so keep a separate copy of anything you cannot lose.

Run myquota to check your usage on each file system, and flost to recover a file from a snapshot. For more information see the [Anvil File Management] guide.

File System

Directory	Path	Quota	Purge	Backup	Notes
Home	$HOME	25 GB	Never purged	Snapshots only, retained 60 days	ZFS. Area for personal software, scripts, compiling and editing. There is no limit on the number of files. Snapshots live on the same hardware as the primary copy and are not a backup - back up anything you cannot lose.
Scratch	$SCRATCH	100 TB, 1,048,576 files	Purged 30 days after last access	Not backed up	GPFS. Working space for job input and output, not long-term storage. Deletions here are silent and permanent - there is no warning email and no way to recover a file.
Projects	$PROJECT	5 TB, 1,048,576 files	Purged 90 days after allocation expires	Snapshots only, retained 60 days	GPFS. Shared across your allocation for project data. Reached via $PROJECT or $WORK. Snapshots live on the same hardware as the primary copy and are not backups; backup anything you cannot lose.
Applications	/apps	See notes	Never purged		ZFS. Read-only area holding the software Anvil installs centrally. Not user storage, so no quota applies.
Datasets	/anvil/datasets	See notes	Never purged		GPFS. Read-only collection of community datasets curated by Anvil. Not allocated to users and not counted against your quota.

External Storage Documentation

Anvil Object Storage is a separate software-defined storage system offering object storage for large or long-lived research data - shared datasets, long-term outputs, and data used by containerized or cloud-integrated workflows. It complements the ZFS and GPFS tiers rather than replacing them.

For more information see [Anvil Object Storage].

File Transfer Documentation

Use Globus for large transfers or transfers with many files - it retries and resumes on its own. Use scp, rsync or sftp for smaller command-line transfers.

Anvil does not publish a separate data transfer node: command-line transfers go to the login host, anvil.rcac.purdue.edu, and need an SSH key set up first.

For more information see [Anvil Transferring Files].

Supported Methods	Data Transfer Node / Globus Collection	Notes
GLOBUS \| RECOMMENDED	ACCESS Anvil	Anvil Transferring Files
SCP	anvil.rcac.purdue.edu	Anvil Transferring Files
RSYNC	anvil.rcac.purdue.edu	Anvil Transferring Files
SFTP	anvil.rcac.purdue.edu	Anvil Transferring Files

Login to Anvil CPU Documentation

2FA/MFA required for login

Full instructions are in [Anvil Logging In].

Before your first login you need an active Anvil allocation and an ACCESS account with Duo two-factor authentication enrolled. Anvil does not issue its own accounts, so there is nothing separate to register for.

Your Anvil username is not your ACCESS ID: it is your ACCESS username with x- in front of it, so ACCESS ID jdoe logs in as x-jdoe.

SSH to Anvil is key-based - password authentication is not supported. If you have not set up a key yet, sign in to Open OnDemand with your ACCESS credentials first, open a terminal there, and add your key.