ICASSP 2026 Full-Day Satellite Workshop









Modeling Eye, Brain, Speech, and Behavioral Signals for Cognitive Resource Allocation

๐Ÿ“… May 4, 2026 | Barcelona, Spain
Matthew H. Davis

Matthew H. Davis

MRC Cognition and Brain Sciences Unit, University of Cambridge

Adaptive processing of spoken language

Keynote Title:

"Multimodal measures of speech understanding in typical and impaired individuals"

๐Ÿ‘ค Biography

After an undergraduate degree in Experimental Psychology, Matt Davis completed his PhD at Birkbeck College, London, before taking up a post-doctoral position at the MRC Cognition and Brain Sciences Unit (CBU) at the University of Cambridge, UK.

He has been a Programme Leader in Hearing and Language at the CBU since 2012. His research explores the cognitive and neural foundations of human speech perception, comprehension, and language learning.

Brain imaging studies using fMRI provide neural markers of speech processing in healthy individuals that have been applied to detecting residual comprehension during anaesthetic sedation and for vegetative patients.

MEG and EEG measures of neural speech tracking have similarly been applied to characterising comprehension impairments for individuals with language disorders following stroke or neurodegeneration, and in assessing the complementary role of auditory and visual speech signals for comprehension. These methods converge in providing evidence for predictive computations and oscillatory mechanisms that support speech understanding and learning.

๐Ÿ“„ Keynote Abstract

Successful speech understanding for human listeners is the outcome of multiple processing stages as sounds are perceived, words are recognised and meanings are computed. These processes must operate in near-synchrony with sensory signals if speech is to be heard and understood. Neural and behavioural measures that are time-locked to the stimulus (speech tracking) can provide new insights into the mechanisms that support resilient understanding and the causes of comprehension failures for typical and impaired listeners.

In this presentation, Iโ€™ll review recent studies from my group which use Electro- and Magneto-encephalography (EEG and MEG) to measure neural correlates of acoustic, phonetic, lexical and semantic processing while listeners understand connected speech. A particular interest in this work is to link neural measures with behavioural assessments so as to predict and explain individual differences in comprehension outcomes. Using novel behavioural methods, we can measure inter-individual variation in the abilities of typical listeners to use visual speech (i.e. lip-reading) to support perception of acoustically degraded speech [1], and to use delayed contextual information to resolve the meaning of ambiguous words like bank, or bark in sentences [2]. Stable measures of perceptual and cognitive differences are also reflected in time-locked neural activity to acoustic events (e.g. the amplitude envelope) or linguistic events (e.g. word onsets), during sentence listening that we can measure with MEG and localise to brain regions in the frontal and temporal lobe [3, 4].

In ongoing work, weโ€™re using neural speech tracking as measured with EEG to explore when and how comprehension fails either for: (i) typical individuals listening to speech that is acoustically degraded and/or in a language that they donโ€™t speak, or (ii) individuals with receptive aphasia (comprehension difficulties) following strokes that affect neural tissue in the frontal or temporal lobe. In both cases, we observe dissociations between behavioural and neural outcomes suggesting that no single neural measure or simple linear relationship can be used to predict behavioural outcomes from brain activity. Implications for attempts at decoding listening outcomes directly from neural activity will be discussed.

[1] von Seth, J., Aller, M., Davis, M.H. (2025) Unimodal speech perception predicts stable individual differences in audiovisual benefit for phonemes, words and sentences. Journal of the Acoustical Society of America, 157(3), 1554-1576.

[2] Gilbert, R.A., Davis, M.H, Gaskell, M.G., Rodd, J.M. (2021) The relationship between sentence comprehension and lexical-semantic retuning. Journal of Memory & Language, 116, 104188.

[3] Aller, M., Okland, H.S., MacGregor, L.J., Blank, H., Davis, M.H. (2022) Differential auditory and visual phase-locking are observed during audio-visual benefit and silent lip-reading for speech perception. Journal of Neuroscience, 42(31) 6108-6120.

[4] MacGregor, L., Rodd, J., Gilbert, R., Hauk, O., Sohoglu, E., Davis, M.H. (2020) The neural time course of semantic ambiguity resolution in speech comprehension. Journal of Cognitive Neuroscience, 32(3), 403-425.