SLIM Research

Because it’s hard to tell from the name Speech, Language, & Interactive Machines, you can read about some of our interests and projects here. Our work has a theoretical as well as practical aspect to it. We take inspiration from linguistic and cognitive science theories to guide our research resulting in practical dialogue systems to evaluate the theories.

Language comprehension and understanding such that an abstraction sufficient enough for a later consumer (such as a dialogue manager) is produced. This of necessity includes speech recognition, and may include syntax and semantics where necessary. In this research area, questions like how should meaning be represented? and how are entities related? are explored.

Language acquisition. A dialogue system should be able to learn new concepts (i.e., words and language constructs) online as it interacts. Some concepts of necessity are grounded in the “real world” (e.g., what is the notion of redness / what does red mean?) where other concepts are more abstract or deal with how dialogue should work (e.g., when should a clarification request be made, and what kind of request?). We often use embodied platforms, such as robots (like Ankiz Cozmo and Misty II, shown below), to house our learning systems and evaluate them.

Interaction between humans as well as interaction between humans and machines. It is through this interactive process that common ground is established, which is key to comprehension, and it is through interaction that new concepts are acquired and built upon. The interaction should be tightly-coupled (i.e., incremental) as it is with humans, not only because we want people to interact with and use our systems, but also to understand ourselves.

Non-verbal behavior. Language isn’t the only medium of communication. Gestures too can convey a lot of information (e.g., when someone points and says “that” or when someone shapes their hands in a certain way to convey a specific shape). Also, body language, head nods, facial features, and eye gaze can be very communicative. I am mostly interested in how these mediums interact with language.