Towards Open World Event Understanding with Neuro Symbolic Reasoning
Friday, December 9, 2022 11 AM to 12 PM
About this Event
6760 Forest Park Pkwy, St. Louis, MO 63105, USA
Sathyanarayanan (Sathya) Aakur
Assistant Professor
Department of Computer Science
Oklahoma State University
Deep learning models for multimodal understanding have taken great strides in task such as event recognition, segmentation and localization. However, there appears to be an implicit closed world assumption in these approaches, i.e., they assume that all observed data is composed of a static, known set of objects (nouns), actions (verbs), and activities (noun+verb combination) that are in 1:1 correspondence with the vocabulary from the training data. One must account for every eventuality when training these systems to ensure their performance in real-world environments. In this talk, I will present our recent efforts to build open world understanding models that leverage the general-purpose knowledge embedded in large scale knowledge bases for providing supervision using a neuro symbolic framework based on Grenenader’s Pattern Theory formalism. Finally, I will talk about how this framework can be extended to abductive reasoning for commonsense natural language inference, in addition to commonsense reasoning for visual understanding.
Event Details
See Who Is Interested
Dial-In Information
Join Zoom Meeting:
https://wustl.zoom.us/j/99034843616?pwd=S1lSSHE2MGlFT0RrRnljOXdLdEhsZz09
About this Event
6760 Forest Park Pkwy, St. Louis, MO 63105, USA
Sathyanarayanan (Sathya) Aakur
Assistant Professor
Department of Computer Science
Oklahoma State University
Deep learning models for multimodal understanding have taken great strides in task such as event recognition, segmentation and localization. However, there appears to be an implicit closed world assumption in these approaches, i.e., they assume that all observed data is composed of a static, known set of objects (nouns), actions (verbs), and activities (noun+verb combination) that are in 1:1 correspondence with the vocabulary from the training data. One must account for every eventuality when training these systems to ensure their performance in real-world environments. In this talk, I will present our recent efforts to build open world understanding models that leverage the general-purpose knowledge embedded in large scale knowledge bases for providing supervision using a neuro symbolic framework based on Grenenader’s Pattern Theory formalism. Finally, I will talk about how this framework can be extended to abductive reasoning for commonsense natural language inference, in addition to commonsense reasoning for visual understanding.