Submission information
Submission Number: 80
Submission ID: 112
Submission UUID: 7b2294d3-3d13-4ecb-b1d6-9aab9b50f43a
Submission URI: /form/project
Created: Wed, 12/02/2020 - 16:38
Completed: Wed, 12/02/2020 - 16:52
Changed: Mon, 05/23/2022 - 17:02
Remote IP address: 24.34.184.238
Submitted by: Gaurav Khanna
Language: English
Is draft: No
Webform: Project
| Project Title | An optimized search algorithm for gravitational waves from post-merger remnants |
|---|---|
| Program | CAREERS |
| Project Image |
|
| Tags | astrophysics (297), conda (227), cuda (222), gpu (80), parallelization (223), python (69) |
| Status | Complete |
| Project Leader | Robert Coyne |
| robcoyne@uri.edu | |
| Mobile Phone | |
| Work Phone | |
| Mentor(s) | Rob Coyne, Gaurav Khanna |
| Student-facilitator(s) | Christopher Nadeau |
| Mentee(s) | |
| Project Description | Despite the groundbreaking detection of gravitational waves (GWs) by the ground-based LIGO and Virgo interferometers, there remain many open questions in the burgeoning field of gravitational wave astronomy. In particular, the nature of the compact remnant of binary neutron star (NS) mergers that drive gamma-ray bursts (GRBs) is a subject of continued debate, and a key question left open by the multi-messenger observations of GW170817 and GRB170817A. While GRB afterglow observations have suggested that long-lived magnetized NSs (magnetars) may form in at least some binary NS mergers, direct proof is still missing. Direct observation of GWs from a GRB merger remnant could go a long way toward settling this uncertainty, but existing analysis techniques are not suited to detecting such signals. Historically, searches for GWs have fallen into two coarsely defined categories: those that leverage accurate modeling to produce searches that are extremely sensitive, and those that require minimal assumptions about the nature of the signal enabling searches that are highly robust. Most GRB remnant emission mechanisms are too complex to model with sufficient accuracy for our most sensitive searches, yet the GWs they produce are too weak to be detected by the more robust, unmodeled techniques. To bridge the gap, a generalized data analysis pipeline dubbed the Cross-Correlation Algorithm (CoCoA) has been developed. CoCoA is a tunable GW analysis technique that can leverage source modeling to improve the sensitivity of a search, without sacrificing robustness against deviations from the expected waveform. At one extreme, CoCoA can function with minimal signal assumptions and produce results comparable to traditional unmodeled searches. At the other extreme, with a sufficiently accurate signal model, CoCoA approaches our most sensitive modeled searches. But unlike traditional techniques, CoCoA is able to leverage partial modeling to span the gap between these two extremes. The results are impressive, with preliminary estimates resulting in a factor of ~4 improvement over the sensitivity of comparable GRB remnant searches. Due to the vastness of the physical parameter space over which we must search, and the dependence on a discrete bank of template waveforms, CoCoA is computationally expensive. As a result it is well-suited to deployment in a highly-parallelized distributed computing environment. It is already constructed to run on “typical” high-performance computing (HPC) clusters, though it is currently limited to execution on standard CPUs. But there are several aspects of CoCoA’s core functionality that are well-suited for deployment on graphics processing units (GPUs), which are becoming increasingly essential to modern HPC platforms. The goals of this project are three-fold: update CoCoA’s codebase for compatibility with the most recent versions of Python, run CoCoA on a medium-scale HPC resource and then port core components for deployment on GPUs. |
| Project Deliverables | CoCoA with Python version issues addressed; HPC cluster and GPU-accelerated version tested. |
| Project Deliverables | |
| Student Research Computing Facilitator Profile | - Grad or undergrad - Interested in gravitational wave research - Experience with Python - Familiarity with HPC clusters will be helpful - Familiarity with general relativity and gravitational waves will be helpful |
| Mentee Research Computing Profile | |
| Student Facilitator Programming Skill Level | Some hands-on experience |
| Mentee Programming Skill Level | |
| Project Institution | University of Rhode Island |
| Project Address | |
| Anchor Institution | CR-University of Rhode Island |
| Preferred Start Date | 02/01/2021 |
| Start as soon as possible. | No |
| Project Urgency | Already behind3Start date is flexible |
| Expected Project Duration (in months) | 6 |
| Launch Presentation | |
| Launch Presentation Date | 02/10/2021 |
| Wrap Presentation | |
| Wrap Presentation Date | 08/09/2021 |
| Project Milestones |
|
| Github Contributions | |
| Planned Portal Contributions (if any) | |
| Planned Publications (if any) | |
| What will the student learn? | Signal analysis methods Python, libraries compatibility issues and their resolution Running code on HPC resources Profiling code GPU acceleration |
| What will the mentee learn? | |
| What will the Cyberteam program learn from this project? | State-of-the-art computational methods and challenges in a new emerging field (gravitational wave astronomy) |
| HPC resources needed to complete this project? | Medium scale CPU/GPU resources (~50 nodes) |
| Notes | |
| What is the impact on the development of the principal discipline(s) of the project? | Multi-messenger astronomy is one of NSF's thrust areas part of the "Big Ideas" initiative. This research area includes efforts that support gravitational wave, optical, x-ray, gamma-ray and neutrino observations from astrophysical systems. Signal detection and analysis is perhaps the most challenging parts of these efforts. CoCoA is a tunable GW analysis technique that can leverage source modeling to improve the sensitivity of a search, without sacrificing robustness against deviations from the expected waveform. In this CyberTeams project, the CoCoA algorithm was implemented on a parallel-HPC environment and GPU-acceleration was explored. This allows for much higher throughput on important signal searches thus significantly benefiting detection efforts. Despite being heavily dependent on high performance computing, most existing gravitational wave analysis techniques don't leverage the full potential of modern architecture. This project has helped change that, by building modern tools into an analysis technique that hopes to make first-of-its-kind detections starting with the next gravitational wave observing run in Winter 2022-2023. |
| What is the impact on other disciplines? | The approach taken towards parallelism and the lessons learned are applicable to a variety of other areas in science and engineering. Those outcomes may benefit other disciplines in a similar context -- increased throughput on relevant computations. The lack of GPU support and optimization is pervasive across many (if not most) disciplines that have pivoted to high performance computing in recent decades, and the framework developed during this progress is discipline agnostic. Even now, Chris has pivoted directly to working in applied mathematics – wholly unrelated to gravitational wave astronomy. |
| Is there an impact physical resources that form infrastructure? | None. |
| Is there an impact on the development of human resources for research computing? | The supported RCF: Chris Nadeau is very interested now in a career in the area of research computing owing to his experience in CyberTeams. The supported student was effectively retained i.e. will continue on to a STEM career as a direct outcome of this grant. Funded research opportunities appear to have a strong impact on students in STEM disciplines. In addition, these student was mentored and trained on how research in physics (and other sciences) is conducted, the mathematical and technological tools involved, and how to overcome roadblocks and challenges when working with unknowns. |
| Is there an impact on institutional resources that form infrastructure? | Given the outcome of this CyberTeams project, URI better appreciates the role that can played by students to support research computing efforts. URI is now building a student support team within its research computing department. The project objectives related to signal search enhancement will improve the data-analysis capabilities of NSF's largest project LIGO and other associated gravitational wave observatories around the world. Improvements of the type this project will contribute may result in effectively higher- sensitivity for these instruments that would enhance their capabilities. |
| Is there an impact on information resources that form infrastructure? | None. |
| Is there an impact on technology transfer? | None. |
| Is there an impact on society beyond science and technology? | Advancement of scientific efforts and the development of a STEM trained workforce has many established positive impacts on society beyond those particular areas. While difficult to quantify, we envision that in the long run there will be a tangible positive impact of such projects well beyond their domains. |
| Lessons Learned | Beyond the scientific benefits that can be obtained from advanced instrumentation such as HPC systems, it became clear that a student with the right combination of interests and background can positively impact the throughput of a research lab over a short-term engagement as long as there is a supporting team available to the student as a resource. Additionally, such positive short-term engagements seemed to be sufficient to get the student enthusiastic about a career in research computing support. At a technical level -- Bottlenecking occurs during transfer to and from GPUs that requires careful forethought about what processes to offload to the GPU. This was true even for processes that were highly optimized for GPU processing. |
| Overall results | Improved throughput of multi-messenger astronomy signal search algorithms -- the results were quite promising, showing dramatic speedup in certain circumstances on numerous different computing platforms; Positive impact from short-term student engagement on research lab throughput and student retention in a STEM discipline; Workforce development and training in a STEM area (research computing). |