Bob McMurray F. Wendell Miller Professor of Psychological and Brain Sciences University of Iowa
Decoding the neural dynamics of spoken word recognition
Abstract
There is unparalleled consensus on the mechanisms of spoken word recognition. From the earliest moments of the input, listeners activate multiple words that compete for recognition. This has been shown by psycholinguistic measures like the Visual World Paradigm (VWP). However, we have little understanding of the neural basis of lexical competition. No ERP components directly reflect this lexical competition process. Moreover, while fMRI and work with brain damaged populations has revealed a network of structures involved in word recognition, these have not elucidated the fundamental question of where competition takes place. I will first present results from a recent paradigm that combines machine learning with electro-corticography (ECoG) – recordings from the surface of the brain of awake humans undergoing treatment for epilepsy. Stimuli were words that overlapped at onset (dinosaur/dynamite, manatee/manicure), in a passive listening task. We trained a support vector machine to identify which item was heard on each trial over successive 25 msec increments. Results mirrored empirical results (from the VWP) and computational models: Early on, the decoder was equally likely to report the target or the competitor, but by around 500 msec, competitors were suppressed. This was only seen in auditory and phonological areas, and not in higher level language areas, but even in auditory cortex we see evidence for memory-like processes. This supports the view that the fundamental auditory integration processes necessary for word recognition largely play out in lower level processing areas. Next, I describe collaborative work with Haskins that seeks to develop an EEG analogue to this task. This is crucial for children and people with cognitive or communicative impairments for whom standard psycholinguistic measures may be problematic. We used a similar set of stimuli – this time including both words and non-words—while recording 64 channel EEGs from 14 listeners. A similar classification scheme was applied, and results mirrored the ECoG results, showing the smooth timecourse of competition and integration. This demonstrates the value of dynamic machine learning approaches applied to electrophysiology for understanding the dynamics of language processing.
Join the Zoom meeting on Thursday, October 15 at 12:30pm at https://zoom.us/my/haskins