Characterization of 3D organelle motion using curve fitting and clustering by unsupervised machine learning

Project Title: Characterization of 3D organelle motion using curve fitting and clustering by unsupervised machine learning
Program:
CAREERS (323)

Project Image: {Empty}
Tags:
bioinformatics (277)

Status: Complete
Project Leader
--------------
Project Leader:
Chandra Kambhamettu

Email: chandrak@udel.edu
Mobile Phone: {Empty}
Work Phone: {Empty}

Project Personnel
-----------------
Mentor(s):
Jeffrey Caplan (516)

Student-facilitator(s):
Huining Liang (582)

Mentee(s):
{Empty}


Project Information
-------------------
Project Description:
This project will build upon our current work to track a filamentous biological structure called stromules. Previously, our approached used 2D maximum intensity projections of 3D data, which resulted in the loss of any 3D information. In this project, a 3D version of a hybrid CNN-Transformer architecture will be used for 3D segmentation of stromules. For tracking, we are using the tracking-by-attention paradigm which not only applies attention for data association but jointly performs tracking and detection, using TrackFormer, an end-to-end trainable MOT (multi-object tracking) approach based on an encoder-decoder Transformer architecture. That will be combined with a curve fitting based algorithm that was previously developed. The stromules will exhibit different motion behaviors and unsupervised machine learning clustering methods will be explored to find different classes of stromule motion.

Project Information Subsection
------------------------------
Project Deliverables:
{Empty}

Project Deliverables:
{Empty}

Student Research Computing Facilitator Profile:
{Empty}

Mentee Research Computing Profile:
{Empty}

Student Facilitator Programming Skill Level: {Empty}
Mentee Programming Skill Level: {Empty}
Project Institution: University of Delaware
Project Address:
Newark, Delaware. 19716

Anchor Institution: CR-University of Delaware
Preferred Start Date: 05/17/2024
Start as soon as possible.: No
Project Urgency: Already behind3Start date is flexible
Expected Project Duration (in months): 6
Launch Presentation: https://support.access-ci.org/system/files/webform/project/4510/Huining%20-%20launch%20presentation%20for%20Careers%20240621.pptx
Launch Presentation Date: 06/21/2024
Wrap Presentation: https://support.access-ci.org/system/files/webform/project/4510/Huining_CAREERS_Wrap_Presentation_241211.pptx
Wrap Presentation Date: 12/11/2024
Project Milestones:
- Milestone Title: Milestone 1
  Milestone Description: Learn the curve fitting based algorithm for modeling deformable objects and employ the curve fitting algorithm to obtain stromule motion data from microscopy imagery. Give a Launch Presentation.
  Completion Date Goal: 2024-06-05
- Milestone Title: Milestone 2
  Milestone Description: Learn the basics of unsupervised machine learning methods such as K-Means, DBSCAN, and affinity propagation.  Further analyze the stromule motion data using the clustering method and a rule-based classification method. 
  Completion Date Goal: 2024-07-05
- Milestone Title: Milestone 3
  Milestone Description: 3D TransUNet, a 3D version of the advanced hybrid CNN-Transformer architecture, will be trained to perform the segmentation task of 3D microscopy images. This part of the project will add new functions to our deep learning-based microscopy image processing pipeline. Give a Wrap-up presentation.
  Completion Date Goal: 2024-08-16

Github Contributions: {Empty}
Planned Portal Contributions (if any):
{Empty}

Planned Publications (if any):
{Empty}

What will the student learn?:
{Empty}

What will the mentee learn?:
{Empty}

What will the Cyberteam program learn from this project?:
{Empty}

HPC resources needed to complete this project?:
{Empty}

Notes:
{Empty}



Final Report
------------
What is the impact on the development of the principal discipline(s) of the project?:
Huining Liang has worked on a new deep-learning Transformer-based segmentation technique. Since no one has applied it to stromule data before, this helps plant science researchers achieve a higher order, high-throughput multiplexing. Huining has also applied Transformers for tracking. The overall impact includes the improvement over the previous pipeline, which involves UNet and traditional computer vision-based methods. The current pipeline achieved higher accuracy than the earlier work.  Furthermore, Huining classified stromules and their motion using a curve fitting based algorithm for modeling deformable objects. Then she evaluated clustering by unsupervised machine learning and more classic methods, such as DBSCAN and K-means.  The cluster analysis has the potential to reveal novel biological types of stromules based on how they move.

What is the impact on other disciplines?:
Dr. Kambhamettu directs the Video/Image Modeling and Synthesis (VIMS) Lab, which has ten PhD students working on deep learning approaches. The approaches developed in this project open a way to incorporate some of the concepts in other projects.
Dr. Caplan directs a Bio-Imaging Center that is used by 19 different departments at the University of Delaware, spanning a wide array of disciplines. The approaches developed in this project can be translated to other projects in the network. Tracking the motion and deformation of objects in biological system is a common computational problem.  Doing so makes it possible to look for significant differences after a biological system is altered by a treatment or a genetic knockout of a component.  This is a powerful complement to other biological approaches and provides a new method for functional validation of genes or proteins involved in the movement of cellular components.

Is there an impact physical resources that form infrastructure?:
Nothing to report.

Is there an impact on the development of human resources for research computing?:
Yes, Huining Liang will use this training in her Ph.D. work and to assist others in research computing, therefore, adding to our human resource infrastructure.

Is there an impact on institutional resources that form infrastructure?:
Nothing to report.

Is there an impact on information resources that form infrastructure?:
In VIMS Lab, this project helps update the repository of techniques impacted by Huining’s work under CAREERS. It now includes transformer-based techniques as a contribution to this repository and further will be used in other collaborations. The cluster-based approaches will be added to the available approaches in the Bio-Imaging Center to examine biological motion data.

Is there an impact on technology transfer?:
Nothing to report.

Is there an impact on society beyond science and technology?:
The project that Huining Liang worked on developed the use of a latest deep learning technique and compared against the previous pipeline, and achieved improved results. It will assist her future innovation in deep learning and approaches being developed to modify the movement and positioning of chloroplasts to improve photosynthesis. The Caplan lab is currently investigating how beneficial microbes change chloroplast morphology and movement to improve crop yields.  The approaches developed here can determine if a specific class of chloroplast movement or stromules increases or decreases in response to microbes.  Thus, this project may potentially benefit crop production and food security which is a major societal impact.

Lessons Learned:
Learn the knowledge of different clustering methods for data analysis. 
Train a 3D UNet to perform 3D segmentation of microscopy images.
Extend and add new functions to the current image processing pipeline.
Facilitate research with machine learning methods and HPC resources.

Overall results:
Learn the curve fitting based algorithm for modeling deformable objects.
Apply the curve fitting algorithm to obtain stromule motion data from microscopy imagery.

****** From Huining's Exit Interview ******
This series of projects helped her to build a pipeline, modify and add additional functions to the pipeline. Using HPC resources allowed for the pipeline to use 3D features and more complex models reducing compute time from months to weeks to days. She will be graduating in  Dec2026 with her PhD in Computer Vision.