Heather Pon-Barry

Associate Professor of Computer Science
Mount Holyoke College

ponbarry at mtholyoke.edu
Office: Clapp 226 

Photo: Heather Pon-Barry Photo: Heather Pon-Barry


Fall 2021
FYSEM 110-BT: Talking Robots


Research interests: spoken language processing, human-robot interaction, and computer science education. I am the director of the Interactive Computing Research Lab at Mount Holyoke College where we are studying spoken language processing in the context of human-robot interaction.

While speech is a natural way to communicate with robots, most robots are not able to recognize or respond to the subtleties of spoken language. Our spoken interactions with robots, dialogue systems (e.g., Siri), and other devices are not as natural as conversing with another human.

Sometimes, it's not what you say, but how you say it: in everyday communication, people use intonation, loudness, and timing to convey emphasis and emotion — a layer of meaning beyond the semantic content of the words that are spoken. For example, questions are frequently signaled by rising intonation. Affective and cognitive states such as annoyance, engagement, confidence and uncertainty might conveyed through a combination of signals.

In the Interactive Computing Research Lab, we address basic scientific questions about how humans use spoken natural language when communicating with robots, computers, and other humans. We study human-human conversation to understand phenomena such as acoustic-prosodic entrainment. We develop algorithms to automatically find patterns in speech data, which enable affect recognition. And we explore how these methods can inform the design of intelligent, adaptive human-robot interactions.

Watch the videos below to see some of the recent projects with Nico, a humanoid robot. We are in the process of creating a physical testbed for conducting human-robot natural language interaction experiments.

Take a look at my publications page to see past projects I have worked on in spoken dialogue systems, intelligent tutoring systems, recognition of uncertainty in speech, and analysis of acoustic-prosodic entrainment.

For Students

Chart of MHC Computer Science Courses

Instructions for requesting letters of recommendation

For Visitors

Map of Clapp Laboratory/where to park


Other Activities and Affiliations

I'm an Adjunct Assistant Professor in the College of Information and Computer Sciences at UMass Amherst.

Contact Information

Heather Pon-Barry
Computer Science Department
Mount Holyoke College
50 College Street
South Hadley, MA 01075

(413) 538-2241