Stampede3

RP account needed

Stampede3 is a compute cluster consisting of Intel Sapphire Rapids high-bandwidth-memory nodes, Ice Lake CPU nodes, large-memory Ice Lake NVDIMM nodes, NVIDIA H100 GPU nodes, Intel Data Center GPU Max nodes, and Skylake CPU nodes. It is particularly well suited for large-scale simulation, high-throughput computing, artificial intelligence, GPU-accelerated workloads, and large-memory applications, and is often used for groundbreaking open science research projects.

Submitting Jobs Documentation

You can run jobs at different sizes and durations on Stampede3. The following lists the different queues that you can submit to, describing how many nodes you get, how long you can run, the type of resources you get, and the average wait time.

Stampede3 uses the Slurm scheduler and supports multi-node and MPI jobs. Nothing is assumed for you: your job script must set --partition, --time, --account and --ntasks explicitly. See [Stampede3 Running Jobs] for submission commands, sample job scripts and job management.

TACC does not share nodes between users on any of its compute resources, so the smallest thing you can request is a whole node and a job that uses one core is charged for all of it. There is no shared or interactive partition; skx-dev is the development queue and is capped at 16 nodes and 2 hours, while every other queue allows up to 48 hours.

Stampede3 is five different node types behind one scheduler, so choosing a queue is choosing the hardware. The spr nodes carry 128 GB of on-package high-bandwidth memory and no DDR, so work that needs more memory per core belongs on icx or skx; nvdimm is single-node only and exists for jobs that fit nowhere else; and the two GPU queues take at most 4 nodes per job, with pvc providing Intel Data Center GPU Max GPUs rather than NVIDIA ones, so CUDA binaries will not run there. Per-queue limits are listed in [Stampede3 Production Queues].

Jobs are charged in node-hours on the nodes reserved, not on the cores used, and the rate differs by queue. Run /usr/local/etc/taccinfo on a login node to see your project balances and disk quotas, and see [Stampede3 Job Accounting] for the billing formula and current rates.

Queue specifications Documentation

Queue CPU cores / node GPUs / node Num nodes Node RAM Max wallclock
nvdimm
Jobs that need more memory than any other Stampede3 node can offer; one node per job
2x Intel Xeon Platinum 8380 (80 cores) None 3 4 TB 48h
pvc
Workloads built for Intel GPUs; up to 4 nodes per job
2x Intel Xeon Platinum 8480 (96 cores) 4 Intel Data Center GPU Max 1550 (124 GB vRAM) 20 1 TB 48h
h100
CUDA and deep learning workloads on NVIDIA GPUs; up to 4 nodes per job
2x Intel Xeon Platinum 8468 (96 cores) 4 NVIDIA H100 SXM5 (96 GB vRAM) 24 4 TB 48h
icx
General-purpose CPU jobs that need more memory per core than the SPR nodes provide; up to 32 nodes per job
2x Intel Xeon Platinum 8380 (80 cores) None 224 256 GB 48h
spr
Memory-bandwidth-bound jobs that benefit from on-package high-bandwidth memory; up to 32 nodes per job
2x Intel Xeon CPU MAX 9480 (112 cores) None 560 128 GB 48h
skx
General-purpose CPU jobs, including the largest multi-node runs; up to 256 nodes per job
2x Intel Xeon Platinum 8160 (48 cores) None 1,060 192 GB 48h
skx-dev
Development and testing on Skylake nodes; short jobs only, up to 16 nodes and 2 hours
2x Intel Xeon Platinum 8160 (48 cores) None 1,060 192 GB 2h

Software Documentation

The following software is reported as available on Stampede3 by the Software Documentation Service.

Available Software

Application Description Research Discipline
adcirc
adios2 ADIOS 2 (Adaptable Input/Output System) is a next-generation high-performance I/O middleware library that provides a simple, high-level interface for efficiently managing data on parallel storage systems. It is designed to address the increasing challenges of achieving high performance and scalability when dealing with I/O operations in large-scale scientific simulations and data processing applications. Computer & Information Sciences
advisor Advisor is a comprehensive software tool designed for optimizing HPC applications and maximizing performance on current and future HPC systems. It offers insights into performance bottlenecks, parallel patterns, and potential improvements. Engineering & Technology
alps ALPS (Algorithms and Libraries for Physics Simulations) is an open-source software framework designed for the simulation of strongly correlated quantum lattice models. It provides a collection of libraries and applications implementing state-of-the-art algorithms such as quantum Monte Carlo, exact diagonalization, and density matrix renormalization group methods, enabling researchers to study quantum many-body systems efficiently. Condensed Matter Physics
amask Amask is a high-level molecular visualization program that is designed to create and manipulate atomic structures for research and educational purposes. It provides a user-friendly interface for visualizing complex molecular structures and interactions. Biological Sciences
ansys Ansys provides engineering simulation software used to predict how product designs will behave and how manufacturing processes will operate in real-world environments. The software enables engineers to simulate interactions of all disciplines like structural, fluids, heat transfer, electromagnetic, and systems engineering. Mechanical Engineering
apptainer Apptainer is a tool for creating and managing containerized applications in a simplified and efficient manner. It provides a user-friendly interface for developers and system administrators to package, distribute, and deploy applications using containerization technology. Engineering & Technology
arpack ARPACK is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems. It is capable of solving complex, real, symmetric, and nonsymmetric eigenvalue problems across different fields of scientific computing. Mathematics
autoconf Autoconf is a tool for producing configure scripts for building, installing, and packaging software on Unix-like systems. It assists in the process of automating the configuration of software packages and generating portable makefiles. Engineering & Technology
automake Automake is a build automation tool that automatically generates makefiles for software compilation. Engineering & Technology
autotools Autotools is a suite of programming tools designed to assist in making source code packages portable across different Unix-like systems. It includes autoconf, automake, libtool, and others and is commonly used in open-source software projects. Computer & Information Sciences
bar
berkeleygw BerkeleyGW is a software package designed for performing many-body perturbation theory calculations, specifically GW and Bethe-Salpeter equation (BSE) methods, to compute quasiparticle and optical properties of materials from first principles. Condensed Matter Physics
biocontainers BioContainers is a project that provides bioinformatics tools and workflows in a containerized format, making it easier to package, distribute, and run bioinformatics software in a reproducible manner. Biological Sciences
bison Bison is a general-purpose parser generator that converts a grammar description for an LALR context-free grammar into a C program to parse that grammar. It is part of the GNU Project. Computer & Information Sciences
blis BLAS (Basic Linear Algebra Subprograms) implemented in native C and suitable for high-performance implementations. BLIS is capable of achieving high levels of performance on various architectures. Computer & Information Sciences
boost Boost is a set of C++ libraries that provides support for tasks and structures such as linear algebra, pseudorandom number generation, multithreading, image processing, regular expressions, and unit testing. It aims to extend the functionality of C++ programming language and make it more efficient and powerful. Computer & Information Sciences
cantera Cantera is an open-source suite of tools for chemical kinetics, thermodynamics, and transport processes. Chemical Engineering
catch2 Catch2 is a C++ testing framework that is easy to use, yet powerful. It supports test-driven development, behavior-driven development, and specification by example. Engineering & Technology
cdo Climate Data Operators (CDO) is a collection of command-line operators for manipulating and analyzing climate and numerical weather prediction (NWP) model data. Earth & Environmental Sciences
cdtools cdtools is a Python library designed for ptychography and related coherent lensless imaging methods. It provides tools for processing, reconstructing, and analyzing coherent diffraction imaging data, facilitating advanced imaging workflows in scientific research. Physical Sciences
cfitsio CFITSIO is a library of C and Fortran subroutines for reading and writing data files in FITS (Flexible Image Transport System) data format. Physical Sciences
charliecloud Charliecloud is a software tool that provides user-defined software stacks in userspace with no privileged operations or daemons. It uses containers for software isolation and runs in user space, making it lightweight and secure. Computer & Information Sciences
checklist Checklist is an HPC user-environment diagnostic utility. It runs a set of independent check modules that test system access and setup, including SSH, file systems, storage quotas, environment modules, compilers, allocations, the batch system, reservations, and blocked-user status, and prints a PASS/FAIL/WARNING status and message for each. It supports terse and verbose output modes and is extensible: users can point it at a directory of their own check scripts (via the CHECKLIST_USER_DIR variable) to run additional custom checks. Artificial Intelligence and Intelligent Systems
cmake CMake is an open-source, cross-platform family of tools designed to build, test, and package software. It is used to control the software compilation process using simple platform and compiler-independent configuration files. Engineering & Technology
codeserver
cp2k CP2K is an open-source quantum chemistry and solid state physics software package designed to perform atomistic simulations of solid-state, liquid, molecular, and biological systems. It can be used to study a wide range of properties, including electronic structures, molecular dynamics, and vibrational spectra. Physical Sciences
cxxopts A lightweight C++ library for parsing command line arguments, inspired by Python's argparse library. Computer & Information Sciences
ddt DDT is a powerful, flexible, scalable, and easy-to-use debugging and performance analysis tool for parallel and serial codes. Computer & Information Sciences
eigen Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms. Computer & Information Sciences
ffmpeg FFmpeg is a free and open-source software project consisting of a large suite of libraries and programs for handling video, audio, and other multimedia files and streams. Computer Science
fftw2 FFTW (Fastest Fourier Transform in the West) is a C library for computing the discrete Fourier transform (DFT) in one or more dimensions, of arbitrary input size, and of both real and complex data. Applied Mathematics
fftw3 The Fastest Fourier Transform in the West (FFTW) is an open-source software library for calculating discrete Fourier transforms (DFTs) efficiently. It provides a wide range of algorithms for executing Fourier transforms of various sizes, both real and complex data. Physical Sciences
fmtlib fmtlib is a C++ formatting library that provides a safe and fast way to format strings, similar to Python's `str.format()` or `printf` in C. Applied Computer Science
gamess The General Atomic and Molecular Electronic Structure System (GAMESS) is a suite of ab initio quantum chemistry programs designed for electronic structure calculations of molecules and molecular reactions. It provides a wide range of methods for both molecules and periodic solids. Chemical Sciences
gaussian Gaussian is a computational chemistry software suite used for electronic structure modeling. It is widely utilized in research and industry for predicting molecular properties and simulating chemical processes. Natural Sciences
gcc compilers The GNU Compiler Collection (GCC) is a set of compilers for various programming languages, including C, C++, and Fortran. It is widely used for compiling applications and is an essential tool in many software development environments. Computer Science
gdal GDAL (Geospatial Data Abstraction Library) is a translator library for raster and vector geospatial data formats that is released under an X/MIT style Open Source license. It is a collection of geospatial data manipulation libraries that support raster and vector data formats, and includes protocols to access geospatial data sources. Earth & Environmental Sciences, Engineering & Technology
gklib gklib is a C++ library for geometric kernels that provides efficient and accurate geometric algorithms for computational geometry and related fields. Mathematics
gmp The GNU Multiple Precision Arithmetic Library (GMP) is a free library for arbitrary-precision arithmetic, operating on signed integers, rational numbers, and floating-point numbers. Mathematics
gmsh GMSH is a three-dimensional finite element mesh generator with a built-in CAD engine and post-processor. Finite Element Analysis
gnuplot Gnuplot is a portable command-line driven graphing utility for Linux, OS/2, MS Windows, OSX, VMS, and many other platforms. It is capable of producing interactive plots and supports various output formats. Physical Sciences
gromacs GROMACS (GROningen MAssive Parallel MD for Molecular Dynamics) is a versatile package for molecular dynamics simulations with a strong emphasis on high-performance computing capabilities. Biological Sciences
gsl The GNU Scientific Library (GSL) is a numerical library for C and C++ programmers. It provides a wide range of mathematical routines such as special functions, linear algebra, interpolation, numerical integration, random numbers, and more. Other Mathematics
hdf5 HDF5 (Hierarchical Data Format version 5) is a file format and a suite of tools developed by the HDF Group for managing and storing large and complex data. It is designed to store and organize large amounts of data efficiently, enabling fast data access and sharing. Computer Science
highfive HighFive is a modern C++14 header-only library for managing HDF5 files. It provides a simple and intuitive interface to read, write, and manipulate HDF5 datasets and attributes, leveraging C++ features such as templates and exceptions to improve usability and safety over the native HDF5 C API. Computer Science
hmmer Hmmer is a software suite used for protein sequence analysis based on profile hidden Markov models (HMMs). It is widely used for searching sequence databases for homologs of protein sequences, identifying conserved protein domains, and annotating protein sequences based on sequence alignment. Biological Sciences
hypre HYPRE (High Performance Preconditioners) is a library of high performance preconditioners and solvers for the solution of large, sparse linear systems of equations on massively parallel computers. It provides scalable algorithms for solving large-scale scientific and engineering applications. Mathematics
idev idev is a compiler wrapper that simplifies the use of Intel's data parallel C++ (DPC++) compiler for offloading computations to GPUs. Computer & Information Sciences
idl IDL (Interactive Data Language) is a programming language used for data analysis, visualization, and application development. It provides a comprehensive set of tools for scientific research and data processing. Physical Sciences
intel oneapi compiler Intel oneAPI Compiler is a suite of compilers and tools designed to optimize performance across various architectures, including CPUs, GPUs, and FPGAs. It supports multiple programming languages, including C, C++, Fortran, and SYCL. Software Engineering
intel(r) mpi library Intel MPI Library is a high-performance implementation of the Message Passing Interface (MPI) standard, designed for parallel programming in distributed computing environments. Computer Science
itac Intel Trace Analyzer and Collector (ITAC) is a performance analysis tool designed for parallel applications. It provides in-depth performance data to help developers optimize the performance of their parallel code. Computer & Information Sciences
jsonc JSON-C implements a reference counting object model that allows you to easily construct JSON objects in C, output them as JSON formatted strings and parse JSON formatted strings back into the C representation of JSON objects. It aims to conform to RFC 8259.
kokkos Kokkos is a programming model for writing performance portable parallel applications in C++. It allows developers to write code once and run efficiently on multiple types of heterogeneous architectures. Computer & Information Sciences
lammps LAMMPS (Large-scale Atomic/Molecular Massively Parallel Simulator) is a classical molecular dynamics code designed for simulating large-scale atomistic systems. It is highly versatile and can be used to model a wide range of materials and complex molecular structures. Chemical Sciences
libaec libaec is a C library for lossless image compression. Computer & Information Sciences
libmesh Libmesh is a C++ finite element library designed to provide a framework for the numerical simulation of partial differential equations in high dimensions. It supports 1D, 2D, and 3D mesh generation, provides a wide range of finite element types, and includes solvers for various types of PDEs. Other Mathematics
libxc LibXC is a library of exchange-correlation functionals for density-functional theory. Chemical Sciences
lmod Lmod is an environment module system that helps users manage their software environment by providing a convenient way to dynamically modify the user's environment through modulefiles. It supports multiple collections of modules that can be easily swapped in and out, allowing users to load the software packages and versions they need for their work. Engineering & Technology
map Map is a software tool used for creating, visualizing, and analyzing geographical maps. It is commonly used in cartography and geographic information systems (GIS). Other Natural Sciences
mathematica Mathematica is a computational software program used in scientific, engineering, mathematical and computing fields. It provides a wide range of tools for symbolic and numerical computations, visualization, data analysis, and programming. Mathematics
matlab MATLAB is a high-performance language for technical computing and visualization. It integrates computation, visualization, and programming in an easy-to-use environment where problems and solutions are expressed in familiar mathematical notation. Mathematics, Applied Mathematics
mdspan mdspan is a C++ library that provides a multidimensional array data structure with a fixed size at compile time. It offers a high-performance alternative to traditional dynamically allocated multidimensional arrays. Computer & Information Sciences
metis Metis is a computational software library for partitioning graphs, meshes, and hypergraphs. It provides algorithms that can efficiently partition large graphs and meshes while minimizing the edge-cuts and balancing the partition sizes. Computer & Information Sciences
mfem MFEM is a lightweight, general, scalable C++ library for finite element methods. Its features include a wide range of finite element spaces, high-order and variable-order discretizations, parallel computing support, and multiple examples and applications. Engineering & Technology
mkmod mkmod is a command-line utility used in Unix-like operating systems to create loadable kernel modules from object files. It automates the process of building a kernel module package that can be inserted into the running kernel, managing dependencies and module metadata. Computer Science
mpfr The GNU MPFR (Multiple Precision Floating-Point Reliable) library is a C library for arbitrary-precision arithmetic on floating-point numbers that guarantees correctly rounded results. Mathematics
mpl MPL is a message passing library written in C++17 based on the Message Passing Interface (MPI) standard. Since the C++ API has been dropped from the MPI standard in version 3.1, it is the aim of MPL to provide a modern C++ message passing library for high performance computing.
mumps MUMPS (MUltifrontal Massively Parallel sparse direct Solver) is a software package for the parallel solution of large sparse systems of linear equations arising from computational science and engineering applications. Mathematics
mvapich MVAPICH (MPI for MVAPICH2) is an open-source MPI implementation that delivers high performance, scalability, and fault tolerance for parallel computing on large HPC systems. It is specifically designed for InfiniBand and Omni-Path architectures. Engineering & Technology
mvapich-plus-cpu MVAPICH (pronounced as “em-vah-pich”) is an open-source MPI software to exploit the novel features and mechanisms of high-performance networking technologies (InfiniBand, Cray Slingshot 11, iWARP, RDMA over Converged Enhanced Ethernet (RoCE v1 and v2), Cornelis OPX, Intel PSM3 and Rockport Networks) and deliver best performance and scalability to MPI applications. The above interconnects are supported through the OFI libfabric library or the UCX communication library.
mvapich-plus-pvc MVAPICH (pronounced as “em-vah-pich”) is an open-source MPI software to exploit the novel features and mechanisms of high-performance networking technologies (InfiniBand, Cray Slingshot 11, iWARP, RDMA over Converged Enhanced Ethernet (RoCE v1 and v2), Cornelis OPX, Intel PSM3 and Rockport Networks) and deliver best performance and scalability to MPI applications. The above interconnects are supported through the OFI libfabric library or the UCX communication library.
namd NAMD (NAnoscale Molecular Dynamics) is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. It is optimized for the simulation of biomolecular systems containing millions of atoms. Biochemistry and Molecular Biology
nclncarg
nco NCO (NetCDF Operators) is a suite of programs designed to help manipulate and analyze netCDF and HDF4 files. It provides a range of powerful tools for processing and analyzing climate and weather data stored in netCDF format. Earth & Environmental Sciences
ncview ncview is a visual browser for netCDF format files used for earth system data. It provides functionalities for viewing, analyzing, and interacting with data stored in netCDF files. Earth and Environmental Sciences
netcdf NetCDF (Network Common Data Form) is a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. Other Natural Sciences
netcdff netcdff is a software library for accessing and creating netCDF format data files in Fortran. Natural Sciences
ninja ninja is a small build system with a focus on speed. It differs from other build systems in that it is designed to have its input files generated by a higher-level build system (like CMake) and will not execute commands in parallel by default. Engineering & Technology
nvidia compilers NVIDIA compilers are a suite of compilers provided by NVIDIA for high-performance computing, specifically optimized for NVIDIA GPUs. They support languages such as C, C++, and Fortran, enabling developers to leverage GPU acceleration in their applications. Computer Science
nwchem NWChem is an open-source computational chemistry package designed for high-performance simulations of large-scale molecular systems. It provides various tools and algorithms for both classical and quantum chemical calculations. Chemical Sciences
octopus Octopus is a software package designed for the simulation of quantum systems, particularly in the field of condensed matter physics. It allows for the study of various physical phenomena using time-dependent density functional theory (TDDFT) and other quantum mechanical methods. Physical Sciences
opencilk OpenCilk is an open-source implementation of the Cilk Plus language extensions for C and C++ that provides a simple and efficient model for parallel programming. It enables developers to write parallel programs with minimal effort by using keywords to express parallelism, while the runtime system manages scheduling and load balancing across multiple processors or cores. Computer Science
openfoam OpenFOAM (Open Source Field Operation and Manipulation) is a free, open-source computational fluid dynamics (CFD) software package developed by the OpenFOAM Foundation. It is widely used for simulating complex fluid flows in various industries and academic research. Physical Sciences
opensees OpenSees is an open-source software framework developed for simulating the seismic response of structural and geotechnical systems. It is widely used in civil engineering for earthquake engineering research, structural analysis, and performance-based design. Engineering & Technology
p4est p4est (parallel framework for dynamic load balancing and adaptive mesh refinement) is a software library that provides support for dynamic load balancing and adaptive mesh refinement in parallel numerical simulations. It is particularly useful for solving partial differential equations on unstructured grids efficiently. Engineering & Technology
papi PAPI (Performance Application Programming Interface) is a portable and efficient API that provides a consistent interface and methodology for collecting performance counter data from the underlying hardware. Computer Science
parallel-netcdf Parallel-NetCDF is a library providing high-performance I/O in parallel environments for big data applications with large scientific datasets. It offers parallel I/O capabilities for NetCDF files, which are commonly used in climate modeling, weather forecasting, and other scientific disciplines. Physical Sciences
parallelnetcdf
parmetis ParMETIS is an MPI-based parallel graph partitioning and sparse matrix ordering library designed for efficiently solving large, sparse graph problems. It provides high-performance graph partitioning, matrix reordering, and fill-reducing matrix ordering capabilities for a wide range of scientific computing applications. Computer & Information Sciences
parpack PARPACK is a software library for solving large scale eigenvalue problems. It is specifically designed to compute a few eigenvalues and corresponding eigenvectors of large sparse matrices. Mathematics
pcre2 PCRE2 is a C library implementing Perl-compatible regular expressions. It is a reimplementation of PCRE but with a different API that is designed to be compatible with Perl 5. It provides a regular expression matching engine that supports Perl-style patterns while also providing some features not found in Perl, such as matching up to 2^16 code units. Computer & Information Sciences
petsc PETSc, the Portable, Extensible Toolkit for Scientific Computation, is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations. Natural Sciences
phdf5 ph5util provides utilities for reading, writing, and manipulating PHDF5 files. PHDF5 is a parallel I/O file format based on the HDF5 file format and is used for storing large datasets efficiently across distributed storage systems. Computer & Information Sciences
pnetcdf PnetCDF is a parallel I/O library for accessing NetCDF files in parallel. It is specifically designed to address the scalability and performance limitations of serial NetCDF I/O by providing parallel I/O support. Physical Sciences
precice preCICE is a coupling library for multi-physics simulations, enabling the interaction between different simulation codes in a flexible and efficient manner. Multi-Physics Simulation
proj PROJ is a standard Unix filter function designed to convert geographic longitude and latitude coordinates into Cartesian coordinates. It is widely used in geospatial applications for coordinate transformation and map projection. Other Natural Sciences
pylauncher pylauncher is a lightweight Python-based job launcher designed to facilitate the execution and management of parallel and distributed computing tasks. It provides a simple interface to launch Python scripts or commands across multiple CPUs or nodes, supporting MPI and other parallel execution frameworks. Computer Science
python3 Python3 is a versatile, high-level programming language that is known for its readability and simplicity. It supports multiple programming paradigms such as object-oriented, imperative, functional, and procedural programming. Python3 has a comprehensive standard library and a large ecosystem of third-party packages, making it suitable for a wide range of applications. Computer & Information Sciences, Computer Science, Other Computer & Information Sciences
python_cacher python_cacher is a Python package for caching function results and improving computational efficiency by storing and retrieving expensive function calls. Computer & Information Sciences
qchem Q-Chem is a comprehensive quantum chemistry software package that provides a wide range of capabilities for electronic structure calculations, including methods for molecular geometry optimization, vibrational analysis, and excited state calculations. Quantum Chemistry
qt5 Qt is a comprehensive cross-platform C++ application framework that is widely used for developing application software with a graphical user interface (GUI), and also used for developing non-GUI programs such as command-line tools and consoles for servers. It allows developers to write code once and deploy it across various desktop, mobile, and embedded operating systems without rewriting the codebase. Qt5 is the latest version of the Qt development framework. Engineering & Technology
qt6 Qt 6 is a cross-platform application framework that is widely used for developing application software with a graphical user interface (GUI) as well as non-GUI programs such as command-line tools and consoles for servers. Software Engineering
quantum espresso Quantum ESPRESSO is an integrated suite of computer codes for electronic-structure calculations and materials modeling at the nanoscale. Condensed Matter Physics
remora Remora is a lightweight and extensible Python library for parallel and distributed computing workflows. It simplifies the process of distributing tasks across multiple workers and managing dependencies in a distributed environment. Computer & Information Sciences
sanitytool Sanitytool is a software tool designed for ensuring the sanity and quality of software code by performing various checks and validations. It helps in identifying potential issues, bugs, and vulnerabilities in the codebase, ensuring a higher level of code quality and reliability. Engineering & Technology
scotch Scotch is a software package and library for graph and mesh partitioning, ordering, and clustering. It is designed to efficiently distribute data and computations across distributed memory parallel computers. Computer & Information Sciences
settarg Settarg is a command-line utility designed for setting target specific optimizations in C/C++ compilers. It helps in specifying optimization flags tailored for a particular CPU architecture to improve performance.
siesta Siesta is a computer program that performs electronic structure calculations based on density-functional theory. It is especially designed for tight-binding and ab-initio calculations in materials science. Physical Sciences
silo Silo is a data format and library for storing and managing scientific data, particularly for high-performance computing applications.
sqlite SQLite is a C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code for SQLite is in the public domain. Computer & Information Sciences
suitesparse SuiteSparse is a suite of sparse matrix algorithms and solvers available in the form of libraries that are designed to efficiently handle sparse matrices commonly encountered in numerical computations. Other Mathematics
sundials SUNDIALS (Suite of Nonlinear and Differential/Algebraic Equation Solvers) is a software package for the solution of large-scale, time-dependent nonlinear differential/algebraic systems. Other Mathematics
superludist Distributed-memory sparse direct solver for large nonsymmetric systems of linear equations.
swig SWIG (Simplified Wrapper and Interface Generator) is a software development tool that connects programs written in C and C++ with scripting languages such as Perl, Python, Ruby, Tcl, and more. It simplifies the process of creating interfaces for these languages, enabling developers to easily incorporate C/C++ code into their scripting language-based projects. Computer & Information Sciences, Software Engineering, Systems & Development
sz The sz modulefile defines the following variables: TACC_SZ_DIR, TACC_SZ_LIB, TACC_SZ_INC
tacc TACC (Texas Advanced Computing Center) provides comprehensive advanced computing resources and support services to researchers, enabling them to address some of society's most challenging problems. Engineering & Technology
tacc tips
texlive TeX Live is a comprehensive distribution of TeX and related programs, providing a complete environment for typesetting documents. Computer Science
tinker Tinker is a software package for molecular modeling and simulations, providing tools for energy minimization, molecular dynamics, and Monte Carlo simulations. Chemical Sciences
trilinos Trilinos is an open-source collection of solvers and libraries for the solution of large-scale, complex multi-physics engineering and scientific problems. It provides a framework for the development of parallel linear and nonlinear algebraic solvers, preconditioners, and other scientific computing capabilities. Engineering & Technology
tuning analysis utilities
ucc UCC (Universal Chemistry Converter) is a versatile software tool designed to convert chemical structures from one format to another. It supports a wide range of chemical structure file formats, making it an essential utility for chemists and researchers working with molecular structures. Natural Sciences
ucx Unified Communication X (UCX) is a flexible and high-performance communication library for applications and libraries. Computer Science
udunits UDUNITS supports conversion of unit specifications between formatted, human-readable strings and machine-friendly structures. In addition, UDUNITS defines functions that can be used to identify whether pairs of units are conformable for mathematical operations, to convert numerical values from one unit to another, and to generate strings representing the units of quantities. Computer and Information Sciences
vasp VASP (Vienna Ab-initio Simulation Package) is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles. Physical Sciences, Engineering & Technology
visit Visit is an open-source interactive parallel visualization and graphical analysis tool. It is designed to handle very large data set visualization and enable users to visualize and analyze data from a variety of sources. Computer & Information Sciences
vtk The Visualization Toolkit (VTK) is an open-source software system for 3D computer graphics, image processing, and visualization used by thousands of researchers and developers around the world. VTK offers a wide range of functionality for tasks such as scientific visualization, medical imaging, computational fluid dynamics, and more. Physical Sciences
vtune VTune Profiler is a performance profiling tool for optimizing application performance on Intel architecture. It provides deep insights to help you understand and improve performance and power characteristics of your application. Computer & Information Sciences
xalt XALT (XALT Accounting for Loadable and Transient Objects) is a tool designed to track the detailed usage of HPC systems by monitoring library dependencies and executables during job executions. It captures information on modules, compilers, libraries, and executables used in each job, providing detailed insights into the software stack and usage patterns on the HPC system. Engineering & Technology
yamlcpp yaml-cpp is a YAML parser and emitter in C++. Other Computer and Information Sciences
zlib zlib is a software library for data compression that provides functions for file compression and decompression using the DEFLATE compression algorithm. It is widely used in various applications for reducing file size and conserving storage space. Computer & Information Sciences

Storage Documentation

Stampede3 has three file systems: a small, backed-up $HOME for source code and configuration, $WORK on TACC's shared Stockyard file system for data you want to keep, and a large $SCRATCH for job output. Run your jobs out of $SCRATCH rather than $HOME.

Two things surprise people. The 1 TB $WORK quota is shared across every TACC system you use, not granted separately on each one, and files on $SCRATCH are purged once they have not been accessed for ten days. Only $HOME is backed up. See [Stampede3 Managing Your Files] for the details.

File System Documentation

Directory Path Quota Purge Backup Notes
Home $HOME 15 GB, 300,000 files After account deactivation Backed up regularly VAST filesystem. Keep source code, scripts and configuration here; run jobs from $SCRATCH.
Work $WORK 1 TB, 3,000,000 files Not on a schedule Not backed up Lustre filesystem on TACC's Stockyard. The 1 TB quota is shared across all TACC systems, not granted per system. Not intended for parallel or high-intensity file operations.
Scratch $SCRATCH See notes Subject to purge if access time is more than 10 days old. Not backed up VAST filesystem. Main working area - write job output here rather than to $HOME

External Storage Documentation

Every Stampede3 allocation comes with 2 TB on Ranch, TACC's long-term archival tape system, at no additional cost, and it does not consume ACCESS credits. Ranch is for final data that is unlikely to change and does not need frequent access. Move data to it with Globus, or with scp myfile ${ARCHIVER}:${ARCHIVE}/myfilepath. See [Ranch User Guide].

Projects that need more space than $HOME and $WORK provide can apply for Corral, TACC's online project storage system, which Stampede3 reaches through the same transfer methods. See [Corral User Guide].


File Transfer Documentation

Use [Globus Data Transfer Guide at TACC] for datasets larger than about 200 GB, and the SSH tools scp, rsync and sftp for anything smaller - see [SSH-based Data Transfer].

TACC does not publish a separate data transfer node for Stampede3: command-line transfers go to stampede3.tacc.utexas.edu, the same host you log in to. Keep those transfers small, because the login nodes are shared, and move large data with Globus instead.

Supported Methods Data Transfer Node / Globus Collection Notes
GLOBUS | RECOMMENDED TACC Stampede3 Globus Data Transfer Guide at TACC
SCP stampede3.tacc.utexas.edu Using scp
RSYNC stampede3.tacc.utexas.edu Using rsync
SFTP stampede3.tacc.utexas.edu Using sftp