CLASP Papers in Computational Linguistics

Abstract

This volume showcases research which aims to bridge the gaps between research on dialogue and research on perception.

Dialogue research investigates how natural language is used in interaction between interlocutors and how coordination and successful communication is achieved. However,this research often takes for granted that we align our perceptual representations - taken to be part of common ground (grounding in dialogue). They have also typically remained silent about how we integrate information from different sources and modalities. This is unsustainable when we consider interactions between agents with obviously different perceptual capabilities, such as dialogues between humans and avatars or robots.

Contrarily, studies of perception have focussed on how an agent interacts with and interprets the information from their perceptual environment. There is significant research on how language is grounded in perception, and connected to perceptual representations and actions and therefore assigned meaning (grounding in action and perception). Recently, there has been progress on integrated computational approaches to language, action, and perception, especially with the introduction of deep learning methods in the field of image descriptions that use end-to-end training from data. However, these have a limited integration to the dynamics of dialogue and often fail to take into account the incremental and context sensitive nature of language and the environment.