TalkBack interview series: Maggie Perkoff

NLP has the potential to break down global communication barriers

to improve opportunities and increase connection. 

Maggie Perkoff, PhD student, CU Boulder

Natural Language Processing (NLP) allows us to glean new information from large databases of text, and build predictive models of how natural language is used. TalkBack uses NLP to analyse a large database of teacher and student speech transcriptions in order to sort them into talk moves. Getting these transcriptions is difficult and expensive, and so accurate speech recognition that correctly transcribes speech in classroom environments is needed to generate the required text data, and that’s where the collaboration with SoapBox comes in. But once the text is transcribed there is a huge playground for researchers in NLP to explore and refine new techniques and glean new information from natural language usage.

One such researcher is Maggie Perkoff, a PhD student in Computer Science and Cognitive Science at CU Boulder and a member of the TalkBack team. Today I talked to her about her work in NLP and how she is contributing to the TalkBack project.

Tell us a little about your studies and career to date.

I double majored in Computer Science and Linguistics at Haverford college, and my undergraduate thesis was on sign language recognition. Post college, I joined Amazon Alexa as a software engineer in July 2015, where I stayed until July 2020. A few years post-college, I realized that I wanted to challenge myself more and started to consider graduate school. I took a computational syntax class at the University of Washington and decided to apply to graduate programs. In August 2020, I moved to Boulder to start my PhD in Computer Science and Cognitive Science at CU Boulder. My research is focused on NLP applications for Augmentative and Alternative Communication (AAC) devices.

How are you contributing to the TalkBack project?

II work on the Talk Moves modeling team – we try to improve the accuracy of classifying teacher dialogue with the appropriate Talk Move.  Currently, I’m working to incorporate Dialogue Act information into our teacher TalkMoves model. This will provide more discourse context for our system and hopefully improve accuracy.

Assistive communication devices can give voices to those who were previously silent as a result of cognitive and motor impairments. 

Maggie Perkoff, PhD student, CU Boulder

What is a Dialogue Act and how can I find one?

A dialogue act is somebody’s pragmatic intent in an utterance, something that is more than the sum of its constituent words. Dialogue acts occur in natural discourse – questions and statements are dialogue acts. You can evaluate the words in a sentence combined with the conversational context of the earlier sentences to determine its dialogue act. Tagging classroom discourse with dialogue act information might make it easier to spot talk moves.

In your opinion and experience, how can NLP help to make a better and more equal society?

NLP has the potential to break down global communication barriers to improve opportunities and increase connection. Through enhanced computer understanding, it becomes easier for speakers of different languages to connect and grow through learning the perspectives of others across the world.  We can increase access to appropriate medical and legal services through machine translation. Assistive communication devices can give voices to those who were previously silent as a result of cognitive and motor impairments. Voice powered assistants allow those with motor and visual impairments to have more independence and be more active participants in social, academic, and professional circles.