Knowledge Base Resources

In this tutorial, I present an overview with many examples of the use of Numpy and Pandas for data analysis. Beginners in the field of data analysis can find It incredibly helpful, and at the same time, anyone who already has experience in data analysis and needs a refresher can find value in it. I discuss the use of Numpy for analyzing 1D and 2D multidimensional data and an introduction on using Pandas to manipulate CSV files.

This code showcases how to work with the header-only nlohmann JSON library for C++. In order to compile, change the extensions from json_test.txt to json_test.cpp and test.txt to test.json. You must also download the header files from https://github.com/nlohmann/json. Complilation instructions are at the bottom of json_test. This code is very helpful for creating config files, for example.

Warewulf is an operating system provisioning platform for Linux that is designed to produce secure, scalable, turnkey cluster deployments that maintain flexibility and simplicity. It can be used to setup a stateless provisioning in HPC environment.

EasyBuild is a software installation framework that allows administrators to easily build and install software on high-performance computing (HPC) systems. It supports a wide range of software packages, toolchains, and compilers. Supported software are found in the EasyConfigs repository, one of several resositories in EasyBuild project.

A degree in business analytics looks different in today’s world than it did a decade ago. In its most current application, business analytics uses modern data science and capabilities in machine learning (ML). The magic comes into play when these are leveraged for strategic planning.

Active inference is an emerging study field in machine learning and computational neuroscience. This website in particular introduces "active inference institute", which has established a couple of years ago, and contains a wide variety of resources for understanding the theory of active inference and for participating a worldwide active inference community.

Docker allows for containerization of any task - basically a smaller, scalable version of a virtual machine. This is very useful when transferring work across computing environments, as it ensures reproducibility.

A library of short videos about ACCESS allocations, resources and support.

Comprehensive tutorials and lecture notes covering various aspects of scientific computing using Python and Scipy.

Describes effective mentorship (both ways).

The official documentation for Python 3.11.5. Python comes with a lot of features built into the language, so it is worth taking a look as you code.

Campus Champions foster a dynamic environment for a diverse community of research computing and data professionals sharing knowledge and experience in digital research infrastructure.

These instructions were executed on the FASTER and Grace cluster computing facilities at Texas A&M University. However, the process can be applied to other clusters with similar environments. For local installation, please refer to the PyFR documentation. Please note that these instructions were valid at the time of writing. Depending on the time you're executing these, the versions of the modules may need to be updated. 1. Loading Modules The first step involves loading pre-installed software libraries required for PyFR. Execute the following commands in your terminal to load these modules: module load foss/2022b module load libffi/3.4.4 module load OpenSSL/1.1.1k module load METIS/5.1.0 module load HDF5/1.13.1 2. Python Installation from Source Choose a location for Python 3.11.1 installation, preferably in a .local directory. Navigate to the directory containing the Python 3.11.1 source code. Then configure and install Python: cd $INSTALL/Python-3.11.1/ ./configure --prefix=$LOCAL --enable-shared --with-system-ffi --with-openssl=/sw/eb/sw/OpenSSL/1.1.1k-GCCcore-11.2.0/ PKG_CONFIG_PATH=$LOCAL/pkgconfig LDFLAGS=/usr/lib64/libffi.so.6.0.2 make clean; make -j20; make install; 3. Virtual Environment Setup A virtual environment allows you to isolate Python packages for this project from others on your system. Create and activate a virtual environment using: pip3.11 install virtualenv python3.11 -m venv pyfr-venv . pyfr-venv/bin/activate 4. Install PyFR Dependencies Several Python packages are required for PyFR. Install these packages using the following commands: pip3 install --upgrade pip pip3 install --no-cache-dir wheel pip3 install --no-cache-dir botorch pandas matplotlib pyfr pip3 uninstall -y pyfr 5. Install PyFR from Source Finally, navigate to the directory containing the PyFR source code, and then install PyFR: cd /scratch/user/sambit98/github/PyFR/ python3 setup.py develop Congratulations! You've successfully set up PyFR on the FASTER and Grace cluster computing facilities. You should now be able to use PyFR for your computational fluid dynamics simulations.

With the recent uprising of LLM's many business are looking at way to adopt these LLMs and fine-tuning these models on specfic data sets to ensure accuracy. These models when fine-tuned can be optimal for fulfilling the specific needs of a company. This site explains explicitly when, how, and why models should be trained. It goes over various strategies for LLM fine -tuning.

R GIS packages "rgdal", "rgeos", and "maptools" are package set to be archived and no longer supported by end of 2023. Many other R GIS packages are build on top of these packages, including "sp" and "raster". The packages recommended as replacement for "sp" is "sf" and the replacement for "raster" is "terra". Below are links to published articles regarding this transition. Additionally, I am including links to the documentation for the new packages recommended to be used "sf" and "terra".

Proxmox Virtual Environment is a hyper-converged infrastructure open-source software. It is a hosted hypervisor that can run operating systems including Linux and Windows on x64 hardware.

Spatial Data Science is a growing field across a wide range of industries and disciplines. The open-source programming language Python has many libraries that support spatial analysis, but what do you do when your computer is unable to tackle the massive file sizes of high-resolution data and the computing power required in your analysis? There materials have been prepared to teach you spatial data science and how to execute your analysis using a high-performance computer (HPC).

Slides for a tutorial on Machine Learning applications in Engineering and parameter tuning given at the RMACC conference 2019.

Machine learning is becoming increasingly important in field with large data such as astrophysics. AstroML is a Python module for machine learning and data mining built on numpy, scipy, scikit-learn, matplotlib, and astropy allowing for a range of statistical and machine learning routines to analyze astronomical data in Python. In particular, it has loaders for many open astronomical datasets with examples on how to visualize such complicated and large datasets.

The following link elaborates the usage of OpenMP API and its related syntax. There are also several exercises available for learners to help them get familiar with this widely-used tool for multi-threaded realization.

These slides provide an introduction on how Termius and Cursor, two new and freemium apps that use AI to perform more efficient work, can be used for faster HPC research.

The MOOSE Navier-Stokes Cahn-Hilliard (NSCH) application is a library for implementing simulation tools that solve the Navier-Stokes Cahn-Hilliard equations with non-matching densities using Galerkin finite element methods with a residual-based stabilization scheme.

This is the official University of Oxford FSL group lecture page. This includes information on upcoming and past courses (online and in-person), as well as lecture materials. Available lecture materials includes slides and recordings on using FSL, MR physics, and applications of imaging data.