ACES (Accelerating Computing for Emerging Sciences) is a Texas A&M HPRC testbed built around composable infrastructure: ACES pools GPUs, FPGAs, and other accelerators separately from its CPU nodes and lets you assemble custom node configurations on demand over a high-speed PCIe fabric rather than fixing specific accelerators to specific nodes. This makes it a strong fit for researchers who want to experiment with accelerator combinations rather than commit to a single fixed hardware setup up front.
ACES is also notable for hosting accelerator types that are hard to find elsewhere, including Graphcore IPUs, NextSilicon co-processors, and NEC Vector Engines, alongside more familiar NVIDIA H100 and Intel GPUs/FPGAs. This breadth makes it especially useful for AI/ML researchers and others looking to benchmark or prototype on emerging hardware architectures before committing to a particular accelerator for production-scale work.
Jobs
Jobs on ACES are scheduled by Slurm and charged in Service Units (SUs), where 1 SU equals one core-hour, one core used for one hour of walltime. For example, using a full compute node (96 cores) for one hour costs 96 SUs. Jobs that use GPUs or other accelerators are charged at a higher rate on top of the core charge; current accelerator rates are listed on the HPRC Account Management System (AMS) page. SUs are drawn from the account named in the job, or your default account if none is specified.
Each job selects a partition (with --partition), which determines the hardware it runs on, and must request a core count and a walltime limit; if no walltime is given, the partition default applies. Compute nodes provide 96 cores and 512 GB of memory (488 GB usable for jobs), and walltime limits vary by partition. If you want more info about the compute nodes or the system architecture, you can check out the Hardware page. For more detail, please visit the Batch System page where the building job files basics are explained in depth.
The Batch System page also provides information about required batch job specifications and optional/alternative specifications. Here is an example on how to run a serial job (single core, single node):
#!/bin/bash
##NECESSARY JOB SPECIFICATIONS
#SBATCH --job-name=JobExample1 #Set the job name to "JobExample1"
#SBATCH --time=01:30:00 #Set the wall clock limit to 1hr and 30min
#SBATCH --ntasks=1 #Request 1 task
#SBATCH --mem=5000M #Request 5000MB (5GB) per node
#SBATCH --output=Example1Out.%j #Send stdout/err to "Example1Out.[jobID]"
##OPTIONAL JOB SPECIFICATIONS
##SBATCH --account=123456 #Set billing account to 123456
##SBATCH --mail-type=ALL #Send email on all job events
##SBATCH --mail-user=email_address #Send all emails to email_address
#First Executable LineNote: The `pvc` node has 2–8 GPUs per node, varies by node.
The Grace-Hopper node (gh01) is not a batch partition; it is reached by connecting to it directly over SSH from an ACES login node. Its RAM varies 256–512+ GB.
Note on node counts: ACES uses a composable Liqid PCIe fabric, so accelerators (H100, A30, PVC, FPGA, NextSilicon) are dynamically attached to a shared pool of Sapphire Rapids compute nodes rather than being fixed to a partition. A single node can belong to several partitions at once, live gpuavail output shows nodes assigned to cpu,pvc simultaneously, so the per-queue node counts overlap and are not additive (they sum to more than the 110-node Sapphire Rapids pool by design). Counts are configured totals from live sinfo and include nodes that were offline/drained at capture time. For a real-time view, run sinfo -s -p <partition> or gpuavail on a login node.
Queue specifications
Metrics updated 2026-06-16
| Name | Purpose | Nodes | CPU cores / node | GPUs / node | Node RAM | Jobs
30 days
|
Wait Time
30-day trend
|
Wall Time
30-day trend
|
|---|---|---|---|---|---|---|---|---|
| cpu | General CPU-only jobs | 88 | Intel Xeon 8468 (Sapphire Rapids) (96 cores) | — | 488 GB | 91,196 |
|
|
| gpu | NVIDIA GPU workloads (AI/ML, CUDA, parallel GPU jobs). High-memory GPU nodes | 8 | Intel Xeon 8468 (Sapphire Rapids) (96 cores) | 2 NVIDIA H100 | 488 GB | 2,621 |
|
|
| gpu_debug | Short GPU testing/debugging (1 node max, 2-hour limit). Mixes A30s and H100s across nodes. | 3 | Intel Xeon 8468 (Sapphire Rapids) (96 cores) | 2 NVIDIA A30 | 488 GB | 628 |
|
|
| gpu_debug | Short GPU testing/debugging (1 node max, 2-hour limit). Mixes A30s and H100s across nodes. | 3 | Intel Xeon 8468 (Sapphire Rapids) (96 cores) | 2 NVIDIA H100 | 488 GB | 628 |
|
|
| pvc | Intel GPU Max (PVC) jobs — AI/ML, LLM inference | 30 | Intel Xeon 8468 (Sapphire Rapids) (96 cores) | 8 Intel Data Center GPU Max 1100 (Ponte Vecchio) | 488 GB | 418 |
|
|
| bittware | FPGA-based workloads and hardware acceleration (2 FPGA devices) | 2 | — | — | 488 GB | — | — | — |
| nextsilicon | Experimental NextSilicon accelerator workloads (restricted access. Utilizes a NextSilicon coprocessor. | 2 | Intel Xeon 8468 (Sapphire Rapids) (96 cores) | — | 488 GB | — | — | — |
| nec | Vector-engine architecture for vectorized HPC and MPI-based scientific computing. Utilizes a NEC Vector Engine, Type 20B-P card. | 1 | Intel Xeon 8268 (Cascade Lake) (48 cores) | — | 760 GB | — | — | — |
| gh01 | Grace-Hopper node for High-bandwidth AI/ML and HPC workloads requiring fast CPU-GPU data movement and memory interconnect | 1 | ARM Neoverse V2 (NVIDIA Grace ARM) (72 cores) | 1 NVIDIA H100 | 488 GB | — | — | — |
Datasets
| Name | Description |
|---|---|
| pytorch-computer-vision-datasets | A collection of standard computer vision datasets formatted for PyTorch, supporting tasks like image classification and object detection. On ACES, these are used to benchmark GPU performance and test distributed deep learning workflows across accelerators. |
| pytorch-language-modelling-datasets | Text-based datasets for training NLP and language models in PyTorch. In ACES, they support benchmarking of large-scale, memory-intensive workloads and evaluating performance of transformer-based models across hardware. |
| tensorflow-computer-vision-datasets | Computer vision datasets optimized for TensorFlow, covering tasks such as classification and segmentation. Within ACES, they enable framework comparisons and validation of TensorFlow pipelines on heterogeneous accelerators. |
| tensorflow-language-modelling-datasets | NLP datasets prepared for TensorFlow, used for language modeling, translation, and text analysis. On ACES, they help evaluate distributed training performance and accelerator efficiency for sequential data workloads. |
| videollama_dataset | A multimodal dataset combining video and text for tasks like video understanding and captioning. In ACES, it is used to test high-throughput, multi-accelerator workflows and benchmark complex AI pipelines. |
Storage
File System
| Directory | Path | Quota | Purge | Backup | Notes |
|---|---|---|---|---|---|
| $HOME | /home/username | 6 months after account deactivation | Daily | Small scripts, config files, not for general use | |
| $SCRATCH | /scratch/user/username | 6 months after account deactivation or when quotas are exceeded. | None | Primary working directory for jobs, not for long-term storage. | |
| $PROJECT | /scratch/group/projectid | 90 days after allocation expiration | None | Shared storage for group members |
External Storage
Extra storage is available through Texas A&M HPRC: Google Drive (25GB free, with a paid expansion available), Microsoft OneDrive (25GB free), and HPRC Long Term Storage (paid dedicated storage for longer-term needs). For full details and current rates, see Texas A&M HPRC's Extra Storage Options guide: https://hprc.tamu.edu/kb/Helpful-Pages/Storage/
For data storage policies, please visit https://hprc.tamu.edu/kb/User-Guides/ACES/Policies/#data-storage
File Transfer
ACES supports several file transfer methods depending on your needs — Globus Connect is recommended for most transfers, but alternatives are available for more specific use cases. For details and setup instructions, see Texas A&M HPRC's File Transfer guide: https://hprc.tamu.edu/kb/Helpful-Pages/File-Transfer/
Note: FTP is not recommended, as it does not encrypt usernames, passwords, or data during transfer. Use SFTP whenever possible.
| Supported Methods | Data Transfer Node | URL |
|---|---|---|
| GLOBUS | RECOMMENDED | ACCESS TAMU ACES DTN | https://app.globus.org/dashboard |
| SCP/SFTP | ACCESS TAMU ACES DTN | |
| FTP | ACCESS TAMU ACES DTN | |
| RSYNC | ACCESS TAMU ACES DTN | |
| RCLONE | ACCESS TAMU ACES DTN | |
| GDOWN | ACCESS TAMU ACES DTN | |
| PORTAL | ACCESS TAMU ACES DTN | https://portal.hprc.tamu.edu |
Login to ACES
2FA/MFA required for login
The recommended method is through the ACES OnDemand Portal which provides browser-based access to files, terminals, and interactive applications with no key setup required. You can access it through the yellow button below labeled "ACCESS OnDemand Login"
The other option is through the command-line (SSH). ACES does not accept a direct SSH connection. Traffic is routed through a jump host to the login node (login.aces.hprc.tamu.edu), and authentication uses a certificate-signed key pair generated from the portal, not one you create locally. Note that the SSH key is valid only for 49 hours.
On your local computer, clear any old aces-jump or login.aces entries from your local ~/.ssh/known_hosts file (~\.ssh\known_hosts on Windows) to avoid host-key mismatch errors. Generally, you can find the file here:
- /home/LOCAL_USERNAME/.ssh for Linux
- C:\Users\LOCAL_USERNAME\.ssh for Windows
Note that known_hosts is a file, not a directory. The directory is (...)/.ssh or (...)\.ssh depending on your operating system.
- Generate and download your key pair from the ACES portal under Utilities > sshca (https://portal-aces.hprc.tamu.edu/pun/sys/sshca). This page will have a link to "Open file app to download the pubkey pair". Follow that link and download both id_aces_tamu and id_aces_tamu-cert.pub, and note where they are saved. A shell-based alternative using gen_ssh_cert.sh is documented in the guide.
If you plan to keep the keys somewhere other than your Downloads folder, move them to that final location now, before continuing; copying or moving files can reset file permissions. On your local computer, restrict access to both downloaded files so that only your account can read them. SSH will refuse to use a private key that other accounts can read.
- Linux, macOS, WSL, or Git Bash:
chmod 600 id_aces_tamu chmod 600 id_aces_tamu-cert.pub- Windows (PowerShell or Command Prompt), if you are using the built-in OpenSSH client. Windows uses file ACLs instead of Unix permissions. Navigate to the folder containing the keys and run, replacing YOUR_USERNAME with your Windows account name:
icacls id_aces_tamu /inheritance:r icacls id_aces_tamu /grant:r "YOUR_USERNAME:R" icacls id_aces_tamu-cert.pub /inheritance:r icacls id_aces_tamu-cert.pub /grant:r "YOUR_USERNAME:R"If you use PuTTY or another third-party client instead of OpenSSH, follow that client's own key-handling instructions.
The guide offers two methods to connect.
Method 1 (preferred): On your local computer, create and edit the $HOME/.ssh/config file (~\.ssh\config in Windows).
Note that the config file must not have an extension. In Windows, you can create the file with your preferred method (e.g. the notepad app), but once you do make sure to navigate to the \.ssh folder and rename the config file to make sure you remove its extension (e.g. remove the ".txt" from config.txt).
Add the following to the bottom of the file. Replace ACES_USERNAME with your ACES username and PATH_to_pubkey with the known location mentioned previously (folder in which you saved id_aces_tamu and id_aces_tamu-cert.pub).
Make sure there's a blank line between any previous entries in the file:Host aces-jump.hprc.tamu.edu Hostname aces-jump.hprc.tamu.edu User ACES_USERNAME IdentityFile "/PATH_to_pubkey/id_aces_tamu" Port 8822 MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com Host login.aces.hprc.tamu.edu Hostname login.aces.hprc.tamu.edu ProxyJump aces-jump.hprc.tamu.edu User ACES_USERNAME IdentityFile "/PATH_to_pubkey/id_aces_tamu" Port 22 MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.comAfter that, run the command below in your terminal:
ssh login.aces.hprc.tamu.eduMethod 2: Skip the config file and use a single-line ssh command with a ProxyCommand argument, substituting the same ACES_USERNAME and PATH_to_pubkey values:
ssh -o ProxyCommand="ssh -W %h:%p ACES_USERNAME@aces-jump.hprc.tamu.edu -p 8822 -i /PATH_to_pubkey/id_aces_tamu" ACES_USERNAME@login.aces.hprc.tamu.edu -i /PATH_to_pubkey/id_aces_tamuWindows PowerShell users: if you get a "Corrupted MAC on input" error, the guide documents a MACs line to add at the very top of your .ssh/config file.
Detailed instructions for SSH setup, including key configuration and connection commands, are available at the SSH Login page.