Shrivastava Receives NSF CAREER Award to Advance Computers’ Understanding of Temporal Phenomena

A University of Maryland expert in computer vision, machine learning and artificial intelligence will use funding from the National Science Foundation (NSF) to develop innovative technologies so that computers can better understand temporal phenomena, the term used to describe observable changes over time.

A stronger computational approach in this area is important for observing and interpreting human actions in videos, with the potential to transform applications in broader areas such as security, health and robotics.

Abhinav Shrivastava, an assistant professor of computer science with an appointment in the University of Maryland Institute for Advanced Computer Studies, is principal investigator of the NSF Faculty Early Career Development Program (CAREER) award, expected to total $598K over the next five years.

“The technologies we hope to develop have the potential to address challenges in several areas, such as analyzing weather patterns and forecasting extreme events, provenance search for malicious content on the internet, and indexing and searching internet-scale video collections using natural activity-level queries,” Shrivastava says.

Temporal phenomena can exhibit structure at various time scales, he explains. Long events, such as cooking breakfast, are composed of multiple long-term activities like cooking an omelet, which in turn are composed of various atomic actions, or actions that effectively happen all at once. For breakfast, that might mean cracking the eggs, cutting vegetables, sprinkling in cheese, etc.

The NSF funding will focus on developing computational representations of temporal phenomena that capture how these concepts evolve over time by accentuating specific temporal cues in video captures, Shrivastava says.

Leveraging these representations can lead to the development of new algorithms that can identify distinctive recurring atomic actions from video collections, and the composition of these atomic actions can help understand and explain long-term, complex temporal phenomena.

Shrivastava says the outcomes of the research could enable computers to be significantly better at recognizing complex activities, detecting anomalies and forecasting future actions.

Integrated with the research is a comprehensive plan for education, mentoring and outreach, including training students in research at multiple levels, contributing to curriculum development for undergraduate and graduate courses, and designing outreach programs to attract diverse students at multiple levels.

—Story by Melissa Brachfeld, UMIACS communications group

"CAREER: Representing, Discovering, and Assembling Motifs for Video Understanding"(link is external) is supported by NSF grant #2238769 from the NSF’s Division of Information & Intelligent Systems.

This article was published by the University of Maryland Institute for Advanced Computer Studies.