image-processing

The UVM Art and AI Initiative is exploring approaches to artistic image production, comparing the results of StyleGAN and Genetic Algorithms*. More broadly, the project explores emerging artistic practices with Machine Learning and AI while referencing an artistic lineage to the artists Wassily Kandinsky, Jonn Cage and Yoko Ono; these artists employ(ed) instructions and systems in their non-digital artworks. Kandinsky distinguished systems and developed a science of aesthetics with the basic elements of point, line and plane; Cage used the oracle 'I Ching' like a computer to inform his compositional decisions; Ono writes poetic scores that turn her audience into active participants when they follow a series of imaginative instructions. Through this ongoing research and practice, we intend to join the larger conversation about art and A.I and design new curriculum for UVM undergraduate students.

This work began in February 2020 and is led by Jennifer Karson of UVM’s Department of Art and Art History and the CEMS UVM FabLab. The team has included three UVM students: two graduate students in data science and one undergraduate mechanical engineering student. The team currently uses RunwayML for the StyleGAN experiments and Processing, an open-source language and development environment built on top of the Java programming language, for Genetic Algorithms.

Additional summer funding ($2,000) is sought for one of the UVM Art and A.I. Initiative student coders. The funding will assist the team in reaching a short-term goal to present initial findings this July at Alife 2020 Montreal; a longer-term goal is to create an art installation for the UVM Fleming Museum of Art in the spring of 2021. This is a unique opportunity to exhibit as part of the statewide project 2020 Vision: Seeing the World through Technology and alongside the work of internationally renowned computer artist and co-founder of the Processing programming language Casey Reas.

Milestone 1:

Genetic Algorithms: Develop successful genetic algorithm code that meets compositional standard (color, architecture, appropriate datasets) while creating new compositions from the elements of existing hand-drawn compositions. The program should output image files that can be stored and printed at high resolutions on paper.

StyleGAN: Transition from RunwayML to coding in Python and employing VACC computer cluster. The process should output image files that can be stored and printed at high resolutions and on paper to be exhibited.

Milestone 2:

Genetic Algorithms: Create an interactive version of the program that allows for audience participation; can be exhibited in a museum gallery and online.

StyleGAN: Develop video that can be exhibited in museum gallery and online.

*Our Genetic Algorithm base code was developed by Daniel Shiffman

Our research group is developing a workflow for generating subject-specific finite element head models from medical imaging data. These head models are applied to simulate blunt impact and blast loading scenarios with a goal of predicting the severity of brain injury. The model generation process includes segmenting the functional and structural regions of the brain, generating an appropriate finite element mesh, and incorporating structural details from medical imaging data. We are seeking a student who is interested in learning the steps in the workflow and developing new methods for improving the anatomical accuracy of these models.

Executive Summary

Funding is requested from the Northeast CyberTeam to support an undergraduate intern who will help advance remote sensing deep learning workflows supporting Vermont’s high-resolution land cover initiative. The internship will be based out of the University of Vermont Spatial Analysis Laboratory (SAL) and supervised by SAL Director and faculty member Jarlath O’Neil-Dunne. This internship will make extensive use of the Vermont Advanced Computing Core (VACC), particularly the DeepGreen GPU cluster.

Background

The State of Vermont is under both regulatory and public pressure to improve the water quality of Lake Champlain. State agencies must have access to high-resolution land cover information that is detailed enough to provide parcel-level quantification of land cover features. The University of Vermont, with funding from the State of Vermont, led the development of the 2016 statewide, high-resolution land cover dataset. This 2016 land cover dataset is the most accurate, detailed, and comprehensive land cover map ever made of Vermont. The existing workflows employed to develop this land cover dataset are slow and expensive, running on individual desktop computer workstations. Moreover, the land cover dataset was already out of date the moment it was produced.

In February 2020, a meeting was held consisting of the state agency representatives, the Vermont Advanced Computing Core, and the Spatial Analysis Laboratory. State agencies voiced their desire to have an approach to land cover mapping that would allow for more rapid updates of high-resolution land cover products, and that would capture fine-scale changes that could influence water quality, such as the construction of a new building.

Activity

This project will focus on integrating deep learning approaches into the SAL’s feature extraction workflows. Deep learning has shown tremendous potential for mapping land cover from high-resolution remotely sensed datasets. Deep learning techniques by themselves may not always be optimal for updating existing land cover datasets as false change can result in differences stemming from the source data or errors in the mapping itself. We propose to leverage deep learning to more efficiently update the Sate’s high-resolution land cover maps through a hybrid approach. Our desire is to take advantage of the potential that deep learning offers while still employing the methodologies that ensure quality specifications are met. The goal of this hybrid approach is to have a faster, more efficient, and more accurate approach to updating existing high-resolution land cover products. High-performance computing will be employed to tackle the most computationally intensive aspects of deep learning, the model training process. These models will then be integrated into the existing workflows to produce areas showing areas of change, and the existing high-resolution land cover to enable rapid updating of the statewide landcover data set. This project will leverage the University of Vermont’s recent investments in high-performance computing architecture. Deep Green, an NSF-funded supercomputer, will be employed.

The phases for this project are: 1) deep learning system design, 2) deep learning system development, 3) deep learning system implementation, 4) integration of deep learning into object-based feature extraction workflow, 5) production of an updated statewide land cover map. The software technologies employed will include TensorFlow and eCognition for feature extraction and ArcGIS for visualization.

This project is incredibly valuable to the state of Vermont as the State is struggling to meet regulatory requirements to reduce non-point source pollution to Lake Champlain, the state’s largest lake that extends into New York and Quebec. Access to current, accurate high-resolution land cover is imperative if the State is going to make decisions on how to reduce non-point source pollution best and fund these activities. Furthermore, the State has no dedicated remote sensing scientists on staff and lacks the computing and technical resources to carry out land cover mapping on this scale. The intern funded as part of this project will work with a talented team that consists of individuals who are internationally recognized for their expertise in automated feature extraction.