Submission information
Submission Number: 197
Submission ID: 4510
Submission UUID: 0f4ab83e-d97e-49e7-852b-715bcf234a12
Submission URI: /form/project
Created: Fri, 04/26/2024 - 14:38
Completed: Fri, 04/26/2024 - 14:38
Changed: Fri, 07/25/2025 - 07:27
Remote IP address: 76.100.157.126
Submitted by: Anita Schwartz
Language: English
Is draft: No
Webform: Project
Characterization of 3D organelle motion using curve fitting and clustering by unsupervised machine learning
{Empty}
Complete
Project Leader
Project Personnel
Project Information
This project will build upon our current work to track a filamentous biological structure called stromules. Previously, our approached used 2D maximum intensity projections of 3D data, which resulted in the loss of any 3D information. In this project, a 3D version of a hybrid CNN-Transformer architecture will be used for 3D segmentation of stromules. For tracking, we are using the tracking-by-attention paradigm which not only applies attention for data association but jointly performs tracking and detection, using TrackFormer, an end-to-end trainable MOT (multi-object tracking) approach based on an encoder-decoder Transformer architecture. That will be combined with a curve fitting based algorithm that was previously developed. The stromules will exhibit different motion behaviors and unsupervised machine learning clustering methods will be explored to find different classes of stromule motion.
Project Information Subsection
{Empty}
{Empty}
{Empty}
{Empty}
{Empty}
{Empty}
University of Delaware
Newark, Delaware. 19716
CR-University of Delaware
05/17/2024
No
Already behind3Start date is flexible
6
06/21/2024
12/11/2024
- Milestone Title: Milestone 1
Milestone Description: Learn the curve fitting based algorithm for modeling deformable objects and employ the curve fitting algorithm to obtain stromule motion data from microscopy imagery. Give a Launch Presentation.
Completion Date Goal: 2024-06-05 - Milestone Title: Milestone 2
Milestone Description: Learn the basics of unsupervised machine learning methods such as K-Means, DBSCAN, and affinity propagation. Further analyze the stromule motion data using the clustering method and a rule-based classification method.
Completion Date Goal: 2024-07-05 - Milestone Title: Milestone 3
Milestone Description: 3D TransUNet, a 3D version of the advanced hybrid CNN-Transformer architecture, will be trained to perform the segmentation task of 3D microscopy images. This part of the project will add new functions to our deep learning-based microscopy image processing pipeline. Give a Wrap-up presentation.
Completion Date Goal: 2024-08-16
{Empty}
{Empty}
{Empty}
{Empty}
{Empty}
{Empty}
{Empty}
{Empty}
Final Report
Huining Liang has worked on a new deep-learning Transformer-based segmentation technique. Since no one has applied it to stromule data before, this helps plant science researchers achieve a higher order, high-throughput multiplexing. Huining has also applied Transformers for tracking. The overall impact includes the improvement over the previous pipeline, which involves UNet and traditional computer vision-based methods. The current pipeline achieved higher accuracy than the earlier work. Furthermore, Huining classified stromules and their motion using a curve fitting based algorithm for modeling deformable objects. Then she evaluated clustering by unsupervised machine learning and more classic methods, such as DBSCAN and K-means. The cluster analysis has the potential to reveal novel biological types of stromules based on how they move.
Dr. Kambhamettu directs the Video/Image Modeling and Synthesis (VIMS) Lab, which has ten PhD students working on deep learning approaches. The approaches developed in this project open a way to incorporate some of the concepts in other projects.
Dr. Caplan directs a Bio-Imaging Center that is used by 19 different departments at the University of Delaware, spanning a wide array of disciplines. The approaches developed in this project can be translated to other projects in the network. Tracking the motion and deformation of objects in biological system is a common computational problem. Doing so makes it possible to look for significant differences after a biological system is altered by a treatment or a genetic knockout of a component. This is a powerful complement to other biological approaches and provides a new method for functional validation of genes or proteins involved in the movement of cellular components.
Dr. Caplan directs a Bio-Imaging Center that is used by 19 different departments at the University of Delaware, spanning a wide array of disciplines. The approaches developed in this project can be translated to other projects in the network. Tracking the motion and deformation of objects in biological system is a common computational problem. Doing so makes it possible to look for significant differences after a biological system is altered by a treatment or a genetic knockout of a component. This is a powerful complement to other biological approaches and provides a new method for functional validation of genes or proteins involved in the movement of cellular components.
Nothing to report.
Yes, Huining Liang will use this training in her Ph.D. work and to assist others in research computing, therefore, adding to our human resource infrastructure.
Nothing to report.
In VIMS Lab, this project helps update the repository of techniques impacted by Huining’s work under CAREERS. It now includes transformer-based techniques as a contribution to this repository and further will be used in other collaborations. The cluster-based approaches will be added to the available approaches in the Bio-Imaging Center to examine biological motion data.
Nothing to report.
The project that Huining Liang worked on developed the use of a latest deep learning technique and compared against the previous pipeline, and achieved improved results. It will assist her future innovation in deep learning and approaches being developed to modify the movement and positioning of chloroplasts to improve photosynthesis. The Caplan lab is currently investigating how beneficial microbes change chloroplast morphology and movement to improve crop yields. The approaches developed here can determine if a specific class of chloroplast movement or stromules increases or decreases in response to microbes. Thus, this project may potentially benefit crop production and food security which is a major societal impact.
Learn the knowledge of different clustering methods for data analysis.
Train a 3D UNet to perform 3D segmentation of microscopy images.
Extend and add new functions to the current image processing pipeline.
Facilitate research with machine learning methods and HPC resources.
Train a 3D UNet to perform 3D segmentation of microscopy images.
Extend and add new functions to the current image processing pipeline.
Facilitate research with machine learning methods and HPC resources.
Learn the curve fitting based algorithm for modeling deformable objects.
Apply the curve fitting algorithm to obtain stromule motion data from microscopy imagery.
****** From Huining's Exit Interview ******
This series of projects helped her to build a pipeline, modify and add additional functions to the pipeline. Using HPC resources allowed for the pipeline to use 3D features and more complex models reducing compute time from months to weeks to days. She will be graduating in Dec2026 with her PhD in Computer Vision.
Apply the curve fitting algorithm to obtain stromule motion data from microscopy imagery.
****** From Huining's Exit Interview ******
This series of projects helped her to build a pipeline, modify and add additional functions to the pipeline. Using HPC resources allowed for the pipeline to use 3D features and more complex models reducing compute time from months to weeks to days. She will be graduating in Dec2026 with her PhD in Computer Vision.