When humans learn language, there are many extralinguistic cues, such as pointing to objects and displaying emotions, that aid the learning process. Even with no prior knowledge, human infants can use emotion to provide feedback.
Today’s world brings humans and robots closer together than ever before. So what would happen if robots were able to have novel emotional behavior cues that help with language learning?
Josue-Torres Fonesca, a senior in the College of Engineering, received a prestigious National Science Foundation Graduate Research Fellowship to build novel emotional behavior models to address the inability of systems to use emotion in dialogue, especially without prior linguistic knowledge.
“I have always been an academic at heart,” Torres-Fonesca said. “I remember as a 5-year-old attending college classes with my parents when they both decided to get their degrees. Even though the concepts were often beyond my understanding, there was something about advanced knowledge that spoke to my soul.”
Torres-Fonesca’s path hasn’t always been a straight shot to success and academic achievement. Despite the adversity, he has discovered his passion for research while continuing to push the boundaries of what is achievable for spoken dialogue systems and Hispanic representation in STEM.
Uninhibitedly Curious
For Torres-Fonesca, a degree in computer science was always the plan, but after choosing two minors, working part-time and additional extracurriculars, he became overwhelmed. With the support of his family, he refocused all efforts on getting his computer science degree. While the end goal was set, the path to success was unclear. One day a professor discussed her Research Experiences for Undergraduates and his interest was piqued and he ended up applying to five programs.
“I was accepted to a Texas State University Smart and Connected Communities REU where I fell in love with the research process,” Torres-Fonesca said. “I couldn’t believe I had the opportunity to develop my own voice and share my knowledge with others.”
Being able to develop and pursue novel solutions to problems was not only exciting for Torres-Fonesca, but it was also this project that led to his interest in machine learning and an opportunity at Boise State to begin working in the Speech, Language and Interactive Machines group with Assistant Professor of Computer Science Casey Kennington
Due to his newfound interest in research, Torres-Fonesca joined the McNair Scholars Program to prepare him for his pursuit of a doctoral degree. With the McNair support system, he found his passion for spoked dialogue systems and began to make connections between research and his experiences as a minority.
“Josue now has a long list of accomplishments and it has been such a rewarding experience to see him go from someone who wonders if he has what it takes to do great things to someone who thinks ‘why not me?’,” SLIM Director Casey Kennington said. “Josue has an amazing drive which led him to publish two papers, while working on a third, all while being the president of two student clubs, applying for and getting national scholarships and fellowships, successfully applying to graduate programs at top universities, with a full-time class load with challenging courses.”
Torres-Fonesca’s research received two separate publications and he was accepted to spend the summer of 2022 in at the Massachusetts Institute of Technology’s Summer Research Program, where he researched interpretable machine learning while working with MIT’s MILAN –a model that annotates behaviors of individual neurons.
This summer research was well received among his peers and MIT gave him the opportunity to continue his research through the fall of 2022 to work on creating multi-word compositional descriptions.
Moving Representation Forward
Despite publishing multiple papers and the opportunities research has brought him, Torres-Fonesca has often felt frustrated, not seen and not able to find representation in his field. Then, after he returned from his first REU, he met a faculty member who looked like him.
Assistant Professor Sole Pera, an Argentinian computer science faculty member, provided Torres-Fonesca with the representation he had been missing during his time in college and through his research programs. He credits Pera for his perseverance in his research and his inspiration to help others.
“It was the first time I met a faculty member who looked like me,” he said. “I found myself seeking her out as someone I could relate to and talk about research with. She is the person who helped me recognize the importance of diversity and inclusion in higher education.”
Torres-Fonesca is a Goldwater Scholar and Boise State Top Ten Scholar, and despite the hours and days spent preparing application packages, continuing classes and research, he also developed plans to build community in the Department of Computer Science.
He began to tutor the data structures course, a course with complex concepts that challenges undergraduate students. Personal experiences helped attract him to students within the program who felt alone. He helped carry this community through the COVID-19 pandemic by keeping the Artificial Intelligence and Cybersecurity Clubs alive with online activities, meetings and a Discord server with nearly 300 participating members.
“Josue is the kind of student every research faculty dreams of: knowledgeable, hard-working, curious, communicative, and teachable,” Kennington said. “I am very proud of Josue. I am eager to see what amazing things he will do.”
Torres-Fonesca said his time mentoring students and leading clubs increased his desire to become involved in diversity and inclusion programs, which he intends to pursue during graduate school. He will be pursuing a doctoral degree in order to develop spoken dialogue systems that are highly effective at providing intelligent responses and actions.
He hopes to mentor other minorities and help them succeed in STEM by embracing their unique perspectives and characteristics. His time in his research programs showed him there’s still a lot of work to be done in higher education that strengthened his desire on diversity and inclusion.
“As a member of a marginalized community, I understand what it’s like to have your needs ignored,” Torres-Fonesca said. “My experience in research, mentorship, and diversity programs like McNair and MIT’s summer research program showed me that I have the potential to succeed as a mentor that can provide inclusivity and embrace diversity.”
On his way to teaching and mentoring the next generation of computer scientists with his doctoral degree, Torres-Fonesca will be supported with the five-year fellowship of the NSF and an annual stipend for three-years so that he can focus on his innovative research for inclusive spoken dialogue systems.
-by Jamie Fink