Knowledge Base Resources

• https://scipy-lectures.org/advanced/mathematical_optimization/

Mathematical optimization deals with the problem of finding numerically minimums or maximums of a functions. This tutorial provides the Python solutions for the optimization problems with examples.

• Powerpoint - Cursor and Termius benefits for HPC

These slides provide an introduction on how Termius and Cursor, two new and freemium apps that use AI to perform more efficient work, can be used for faster HPC research.

• Data Analysis with R for Educators

This webinar series is an orientation to R. We start with an overview of R’s history and place in the larger data science ecosystem. Next, we introduce the R Studio user interface and how to access R’s excellent documentation. Finally, we present the fundamental concepts you need to use the R environment and language for data analysis. Along the way, we compare R script files (.R) to R Notebook (.Rmd) files and show how the features of R Notebook support better communication and encourage more dynamic engagement with statistical analysis and code. It is helpful to be familiar with tabular data analysis using statistical software, database tools, or spreadsheet programs. Workshop materials, including setup directions and slides are available at https://github.com/CornellCAC/r_for_edu/ The Rstudio Cloud project used in the workshop is https://rstudio.cloud/project/4044219.

• Using ACCESS at Duke

This is a short video on how to exchange ACCESS credits and connect to Jetstream 2 (please note this was created for Duke users but applies to all) .

• The GitHub that contains the library itself, and examples
• Library documentation

JSON is a lightweight format for storing and transporting data, for example in a config file. This library is header-only, and has easy-to-read documentation. It is a C++ library.

• CaRCC Data Facing Track Page

The Data-Facing Track of the People Network brings together people from research computing groups, libraries, research institutes, and other organizations who support data-enabled research. Many of us are also Researcher-Facing, but this track is an opportunity to discuss the varied challenges of working with data.

• Tutorial - Using Rclone to transfer data into the OSN

Learn how to use Rclone to transfer data, specifically from your local drive to the Open Storage Network, vice versa.

• Containerization Explained

Containerization is a software development method in which applications are packaged into standard units for development, shipment, and deployment.

• Cybersecurity Guide

Cybersecurity Guide is a comprehensive resource for students and early career professionals that provides users with a wide range of resources and up-to-date information on cybersecurity, including cybersecurity degree programs and bootcamps, career guides, as well as online courses and training opportunities. Additionally, it covers trends, best practices, and much more.

• Fairness and Machine Learning

The "Fairness and Machine Learning" book offers a rigorous exploration of fairness in ML and is suitable for researchers, practitioners, and anyone interested in understanding the complexities and implications of fairness in machine learning.

• PyTorch Tutorial at Stanford University

This is a very barebones introduction to the PyTorch framework used to implement machine learning. This tutorial implements a feed-forward neural network and is taught completely asynchronously through Stanford University. A good start after learning the theory behind feed-forward neural networks.

5 Days of recordings of Python data analysis and visualization training.

• Regression forecasting and predicting - Practical Machine Learning Tutorial with Python p.5

This video series provides a holistic understanding of machine learning, covering theory, application, and inner workings of supervised, unsupervised, and deep learning algorithms. It covers topics such as linear regression, K Nearest Neighbors, Support Vector Machines (SVM), flat clustering, hierarchical clustering, and neural networks. Goes over the high level intuitions of the algorithms and how they are logically meant to work. Apply the algorithms in code using real world data sets along with a module, such as with Scikit-Learn.

• Representation Learning in Deep Learning

Representation learning is a fundamental concept in machine learning and artificial intelligence, particularly in the field of deep learning. At its core, representation learning involves the process of transforming raw data into a form that is more suitable for a specific task or learning objective. This transformation aims to extract meaningful and informative features or representations from the data, which can then be used for various tasks like classification, clustering, regression, and more.

• Awesome Jupyter Widgets List

A curated list of awesome Jupyter widget packages and projects for building interactive visualizations for Python code

• Slurm Community Mailing List

• Using MATLAB with Other Programming Languages

MATLAB is a really useful tool for data analysis among other computational work. This tutorial takes you through using MATLAB with other programming languages including C, C++, Fortran, Java, and Python.

Purdue University is the home of Anvil, a powerful supercomputer that provides advanced computing capabilities to support a wide range of computational and data-intensive research spanning from traditional high-performance computing to modern artificial intelligence applications.

• How do you add your institutional HPC cluster to the Open Science Grid (OSG)?

Discussion about contributing cycles to the Open Science Grid.

• Ask.CI

• unix list of basic commands

Unix is incredibly common and useful. This website provides all the common commands and explanations for one to get started with a unix system.

• Managing and Optimizing Your Jobs on HPC

An overview of tools and methods to manage and optimize jobs and HPC workflows

• University of Arizona Visualization homepage
• Getting Started with Paraview
• Paraview Cameras and Keyframes
• Graphs and Data Exporting
• Visualizing netcdf files

These links take you to visualization resources supported by the University of Arizona's HPC visualization consultant (rtdatavis.github.io). The following links are specific to the Paraview program and the workflows that have been used my researchers at the U of Arizona. Some of the pages linked are very beginner friendly: getting started, working with cameras and keyframes for rendering, visualizing external files (netcdf climate data), graphs and data exporting. Many of the workflows involve using remote desktops via the Open On Demand interface, but if this isn't set up at your university you can use paraview locally on a desktop. Feel free to post on access ci https://ask.cyberinfrastructure.org/ if you need assistance getting a paraview gui open for your work on HPC.

• GDAL Multi-threading

Multi-threading guidance when using GDAL.

• https://www.htslib.org/doc/

Samtools is a suite of programs for interacting with high-throughput sequencing data, especially in the SAM/BAM format. It offers various utilities for processing, analyzing, and managing sequence data generated from next-generation sequencing (NGS) experiments. Samtools is widely used in bioinformatics and genomics research for tasks such as read alignment, variant calling, and data manipulation.