One member of the SLIM group is off to ICMI in November to present a short paper:
Title: Multimodal Alignment within the Incremental Unit Framework
Authors: Casey Kennington, Ting Han, David Sclangen
Abstract: Time alignment is one of the many challenges of multimodal systems. In this paper, we propose a method for time alignment using the incremental unit dialogue system framework which gives the system flexibility in how it handles alignment: either by delaying a modality for a specified amount of time, or by revoking (i.e., backtracking) information that had been processed so multiple modalities can process simultaneously. We evaluate our approach in an offline experiment with multimodal data and find that using the incremental framework of spoken dialogue systems is flexible and shows promise as a solution to the problem of time alignment in multimodal systems.