Computing Colloquium Student Presentation featuring Clayton Fields

Modeling Vision and Language at Human Scales

Presenter: Clayton Fields
Computing PhD Student, Data Science Emphasis

Location: In person in CCP 259 or register to attend via Zoom

Abstract: Recent advances in natural language processing have been driven by the introduction of the transformer architecture, large models and large datasets. Large models and datasets create a number of obstacles for doing research. Furthermore, language models are also trained on text alone and can’t relate concrete words to the sensory information that gives them meaning. In this presentation, we present a body of research that aims to fill these gaps by using the transformer architecture to model language at more reasonable scales of model size and data. Additionally, we also model language alongside other sensory information, such as vision.