Affinity Groups

Logo Name Description Tags Join
R for HPC People who use r on HPC systems and want to exchange experiences, best practices and/or collaborate. r Login to join

Announcements

There are no announcements with this tag. View All Announcements.

Upcoming Events & Trainings

No events or trainings are currently scheduled.

Topics from Ask.CI

Loading topics from Ask.CI...

Knowledge Base Resources

Title Category Tags Skill Level
Automated Machine Learning Book Learning aidata-analysisdeep-learning +4 more tags Intermediate, Advanced
CI Computing Module For All Learning aicomputer-visionneural-networks +17 more tags Beginner
Cornell Virtual Workshop Learning jetstreamstampede2cloud-computing +12 more tags Beginner, Intermediate, Advanced

Engagements

Bayesian nonparametric ensemble air quality model predictions at high spatio-temporal daily nationwide  1 km grid cell
Columbia University

I aim to run a Bayesian Nonparametric Ensemble (BNE) machine learning model implemented in MATLAB. Previously, I successfully tested the model on Columbia's HPC GPU cluster using SLURM. I have since enabled MATLAB parallel computing and enhanced my script with additional lines of code for optimized execution. 

I want to leverage ACCESS Accelerate allocations to run this model at scale.

The BNE framework is an innovative ensemble modeling approach designed for high-resolution air pollution exposure prediction and spatiotemporal uncertainty characterization. This work requires significant computational resources due to the complexity and scale of the task. Specifically, the model predicts daily air pollutant concentrations (PM2.5​ and NO2 at a 1 km grid resolution across the United States, spanning the years 2010–2018. Each daily prediction dataset is approximately 6 GB in size, resulting in substantial storage and processing demands.

To ensure efficient training, validation, and execution of the ensemble models at a national scale, I need access to GPU clusters with the following resources:

  • Permanent storage: ≥100 TB
  • Temporary storage: ≥50 TB
  • RAM: ≥725 GB

In addition to MATLAB, I also require Python and R installed on the system. I use Python notebooks to analyze output data and run R packages through a conda environment in Jupyter Notebook. These tools are essential for post-processing and visualization of model predictions, as well as for running complementary statistical analyses.

To finalize the GPU system configuration based on my requirements and initial runs, I would appreciate guidance from an expert. Since I already have approval for the ACCESS Accelerate allocation, this support will help ensure a smooth setup and efficient utilization of the allocated resources.

Status: In Progress

People with Expertise

Micholas Smith

University of Tennessee, Knoxville

Programs

ACCESS CSSN

Roles

cssn, CIP

Placeholder headshot

Expertise

Sydney Shearer

Juniata College

Programs

CAREERS

Roles

student-facilitator

Placeholder headshot

Expertise

Patrick Clemins

Vermont EPSCoR, University of Vermont

Programs

Northeast

Picture of Patrick Clemins

Expertise

People with Interest

Wirawan Purwanto

Old Dominion University

Programs

Campus Champions, Northeast, ACCESS CSSN

Roles

mentor, research computing facilitator, cssn

Placeholder headshot

Interests

Tolu Oyeniyi

University of Maine at Fort Kent

Programs

Northeast

Roles

student-facilitator

Interests

Kyle Randall

Programs

ACCESS CSSN

Roles

student-facilitator

NASA Langley Picture

Interests