Globus Documentation
0
Globus is a data transfer, sharing, automation, and discovery service used by hundreds of thousands of researchers to manage "big data" at universities, research labs, and national systems such as ACCESS. The Globus documentation website provides how-to guides, reference documentation, and examples for Globus's web application, command-line interface, Python software development kit (SDK), and APIs.
Docker - Containerized, reproducible workflows
0
Docker allows for containerization of any task - basically a smaller, scalable version of a virtual machine. This is very useful when transferring work across computing environments, as it ensures reproducibility.
Slurm Scheduling Software Documentation
0
Slurm is an open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters. Slurm requires no kernel modifications for its operation and is relatively self-contained. As a cluster workload manager, Slurm has three key functions. First, it allocates exclusive and/or non-exclusive access to resources (compute nodes) to users for some duration of time so they can perform work. Second, it provides a framework for starting, executing, and monitoring work (normally a parallel job) on the set of allocated nodes. Finally, it arbitrates contention for resources by managing a queue of pending work.
ACCESS Getting Started Quick-Guide
0
A step-by-step guide to getting your first allocation for Access computing and storage resources.
Introductory Tutorial to Numpy and Pandas for Data Analysis
0
In this tutorial, I present an overview with many examples of the use of Numpy and Pandas for data analysis. Beginners in the field of data analysis can find It incredibly helpful, and at the same time, anyone who already has experience in data analysis and needs a refresher can find value in it. I discuss the use of Numpy for analyzing 1D and 2D multidimensional data and an introduction on using Pandas to manipulate CSV files.
Scipy Lecture Notes
0
Comprehensive tutorials and lecture notes covering various aspects of scientific computing using Python and Scipy.
Official Python Documentation
0
The official documentation for Python 3.11.5. Python comes with a lot of features built into the language, so it is worth taking a look as you code.
Understanding LLM Fine-tuning
0
With the recent uprising of LLM's many business are looking at way to adopt these LLMs and fine-tuning these models on specfic data sets to ensure accuracy. These models when fine-tuned can be optimal for fulfilling the specific needs of a company. This site explains explicitly when, how, and why models should be trained. It goes over various strategies for LLM fine -tuning.
Python Tools for Data Science
0
Python has become a very popular programming language and software ecosystem for work in Data Science, integrating support for data access, data processing, modeling, machine learning, and visualization. In this webinar, we will describe some of the key Python packages that have been developed to support that work, and highlight some of their capabilities. This webinar will also serve as an introduction and overview of topics addressed in two Cornell Virtual Workshop tutorials, available at https://cvw.cac.cornell.edu/pydatasci1 and https://cvw.cac.cornell.edu/pydatasci2
Open-Source Server Virtualization Platform
0
Proxmox Virtual Environment is a hyper-converged infrastructure open-source software. It is a hosted hypervisor that can run operating systems including Linux and Windows on x64 hardware.
ACCESS KB Guide - Expanse
0
Expanse at SDSC is a cluster designed by Dell and SDSC delivering 5.16 peak petaflops, and offers Composable Systems and Cloud Bursting.
Warewulf documentation
0
Warewulf is an operating system provisioning platform for Linux that is designed to produce secure, scalable, turnkey cluster deployments that maintain flexibility and simplicity. It can be used to setup a stateless provisioning in HPC environment.
Setting up PyFR flow solver on clusters
0
These instructions were executed on the FASTER and Grace cluster computing facilities at Texas A&M University. However, the process can be applied to other clusters with similar environments. For local installation, please refer to the PyFR documentation.
Please note that these instructions were valid at the time of writing. Depending on the time you're executing these, the versions of the modules may need to be updated.
1. Loading Modules
The first step involves loading pre-installed software libraries required for PyFR. Execute the following commands in your terminal to load these modules:
module load foss/2022b
module load libffi/3.4.4
module load OpenSSL/1.1.1k
module load METIS/5.1.0
module load HDF5/1.13.1
2. Python Installation from Source
Choose a location for Python 3.11.1 installation, preferably in a .local directory. Navigate to the directory containing the Python 3.11.1 source code. Then configure and install Python:
cd $INSTALL/Python-3.11.1/
./configure --prefix=$LOCAL --enable-shared --with-system-ffi --with-openssl=/sw/eb/sw/OpenSSL/1.1.1k-GCCcore-11.2.0/ PKG_CONFIG_PATH=$LOCAL/pkgconfig LDFLAGS=/usr/lib64/libffi.so.6.0.2
make clean; make -j20; make install;
3. Virtual Environment Setup
A virtual environment allows you to isolate Python packages for this project from others on your system. Create and activate a virtual environment using:
pip3.11 install virtualenv
python3.11 -m venv pyfr-venv
. pyfr-venv/bin/activate
4. Install PyFR Dependencies
Several Python packages are required for PyFR. Install these packages using the following commands:
pip3 install --upgrade pip
pip3 install --no-cache-dir wheel
pip3 install --no-cache-dir botorch pandas matplotlib pyfr
pip3 uninstall -y pyfr
5. Install PyFR from Source
Finally, navigate to the directory containing the PyFR source code, and then install PyFR:
cd /scratch/user/sambit98/github/PyFR/
python3 setup.py develop
Congratulations! You've successfully set up PyFR on the FASTER and Grace cluster computing facilities. You should now be able to use PyFR for your computational fluid dynamics simulations.
Machine Learning in Astrophysics
0
Machine learning is becoming increasingly important in field with large data such as astrophysics. AstroML is a Python module for machine learning and data mining built on numpy, scipy, scikit-learn, matplotlib, and astropy allowing for a range of statistical and machine learning routines to analyze astronomical data in Python. In particular, it has loaders for many open astronomical datasets with examples on how to visualize such complicated and large datasets.
AI/ML TechLab - Accelerating AI/ML Workflows on a Composable Cyberinfrastructure
0
This technology lab contains a set of sessions to help a new user start an AI project on the ACES cluster, a composable accelerator testbed at Texas A&M University. You will learn how to create and activate a virtual environment, manipulate and visualize data with Pandas and Matplotlib, use Scikit-learn for linear regression and classification applications, and use Pytorch to create and train a simple image classification model with deep neural networks (DNN).
Geocomputation with R (Free Reference Book)
0
Below is a link for a book that focuses on how to use "sf" and "terra" packages for GIS computations. As of 5/1/2023, this book is up to date and examples are error free. The book has a lot of information but provides a good overview and example workflows on how to use these tools.
How the Little Jupyter Notebook Became a Web App: Managing Increasing Complexity with nbdev
0
A tutorial entitled "How the Little Jupyter Notebook Became a Web App: Managing Increasing Complexity with nbdev" presented at SciPy 2023 in Austin, TX. This tutorial is hosted in a series of Jupyter Notebooks which can be accessed in the click of a button using Binder. See the README for more information.
Performance Engineering Of Software Systems
0
A class from MITOpenCourseware that gives a hands on approach to building scalable and high-performance software systems. Topics include performance analysis, algorithmic techniques for high performance, instruction-level optimizations, caching optimizations, parallel programming, and building scalable systems.
Higher Ed Controlled Unclassified Information Slack (HigherEdCUI)
0
Slack channel for the Higher Ed CUI community
Examples of Thrust code for GPU Parallelization
0
Some examples for writing Thrust code. To compile, download the CUDA compiler from NVIDIA. This code was tested with CUDA 9.2 but is likely compatible with other versions. Before compiling change extension from thrust_ex.txt to thrust_ex.cu. Any code on the device (GPU) that is run through a Thrust transform is automatically parallelized on the GPU. Host (CPU) code will not be. Thrust code can also be compiled to run on a CPU for practice.
Introduction to GPU/Parallel Programming using OpenACC
0
Introduction to the basics of OpenACC.
ACCESS Campus Champion Example Allocation
0
ACCESS requests proposals to be written following NSF proposal guidelines. The link provides an example of an ACCESS proposal using an NSF LaTeX template. The request is at the DISCOVER level appropriate for Campus Champions. The file is 2 pages: the first page details the motivation, approach, and resources requested; and the second page is a 1-page bio.
Tutorial for OpenMP Building up and Utilization
0
The following link elaborates the usage of OpenMP API and its related syntax. There are also several exercises available for learners to help them get familiar with this widely-used tool for multi-threaded realization.
Fundamentals of R Programming
0
This course is an introduction to the R programming language and covers the fundamental concepts needed to operate in the R environment. This course was taught for the ACCESS community on September 26, 2023, but the materials for the course are still available on the ACES cluster and can be completed independently. All materials are presented as learnR notebooks and cover several topics, including data types, variables, built-in functions, data structures, and plotting.
NCSA HPC-Moodle
0
Self-paced tutorials on high-end computing topics such as parallel computing, multi-core performance, and performance tools. Some of the tutorials also offer digital badges.