Research

The world is filled with predictability, and sensory systems excel at capitalizing on this structure. This allows perceptual systems to process stimuli efficiently when they are predictable, and emphasize how new stimuli are different when stimuli are unpredictable. This has profound effects on how we discriminate, identify, recognize, and learn sounds, particularly speech sounds. Our research explores these ideas through the following projects:

Acoustic context effects in speech sound categorization

All perception takes place in context. How we perceive any given sound is influenced by the acoustic properties of sounds heard around it. If the spectral characteristics of earlier sounds (the context) differ from the spectrum of a later sound (the target), responses to the target sound will become biased. This is known as a spectral contrast effect. We have shown that spectral contrast effects are quite general, occurring for a wide range of stimuli (consonants, vowels, musical instruments), in a wide variety of cases (robust as well as subtle spectral characteristics of earlier sounds), and for different listener populations (normal hearing, listeners with sensorineural hearing loss, simulations of cochlear implant processing). We continue to pursue the mechanisms behind spectral contrast effects in order to better understand how, when, and by how much these effects shape everyday speech perception. We are also examining these questions in temporal contrast effects (also known as speaking rate normalization), when earlier sounds that are spoken quickly can make the target sound seem slower (e.g., /w/) and vice versa, where earlier sounds being spoken slowly make the target sound seem faster (e.g. /b/). These effects are similarly widespread in the speech perception literature, raising additional important questions about how they manifest in everyday listening situations.

Adapting to consistency versus variability in auditory perception

Hearing different talkers makes speech perception slower and/or less accurate than when hearing a single talker. This is known as "talker adaptation" (or sometimes "talker normalization"), and has been observed repeatedly over the last few decades of speech research. We have broadened this research program by revealing the importance of specific talkers' acoustic properties as driving this adaptation, then broadened it further by demonstrating comparable effects in the perception of tones being produced by one versus multiple musical instruments. This type of adaptation is not speech-specific but appears to be general to auditory perception.

Expertise in speech and music perception

Listeners hear their native language(s) more than any other sound. This prodigious experience contributes to remarkable perceptual capabilities, including understanding speech from new talkers and/or in challenging listening conditions. But how did their perception evolve as this expertise accumulated? Relatedly, experienced musicians have heard and practiced music far more than less-experienced or inexperienced musicians (nonmusicians). Many studies report that musical training results in superior pitch perception compared to listeners who have not had musical training, with an active debate about what other perceptual benefits are produced by formal musical training (the "musician advantage"). In this case, expertise varies more widely among adult listeners, so changes in perceptual abilities as a function of accumulating expertise may be more visible. We are interested in how listeners use lower-level (acoustic) and higher-level (linguistic, particularly semantic) context when perceiving lower-level sounds (individual speech sounds, individual musical tones) and higher-level sounds (word pairs and sentences, harmonies and melodies). We are interested in how perceptual expertise compares across linguistic and musical domains, and how the accumulation of expertise in each domain affects the use of different types of context in auditory perception.

Natural signal statistics in speech and music perception

Natural sounds are incredibly acoustically complex, but they are far from random. Speech acoustics exhibit many different types of predictability and structure (or nonrandomness), and the same is true for musical acoustics. This leads to two questions: (1) What are these regularities and/or structure in the acoustic properties of natural sounds? (2) Which regularities matter for perception? We could conduct acoustic analyses of speech and music until we are blue in the face, but these analyses are only worthwhile if they teach us about how listeners perceive these sounds. Recently, we have learned about when and how often speech signal statistics contribute to spectral context effects in speech categorization. The reach of regularities in the speech signal is very wide and will likely make contact with many different areas of speech perception, with the same being true about acoustic regularities in music.

Speech perception for hearing-impaired listeners

We extend the above questions to listeners with impaired or atypical hearing to understand how they utilize spectral changes in the speech signal. One way that we accomplish this is through acoustic simulations of hearing loss or cochlear implant processing. This allows us to efficiently test a number of different parameters that might take on greater importance when perceiving speech with impaired or atypical hearing. This informs testing hearing-impaired listeners directly in similar experiments. Understanding how speech perception is different across healthy and impaired hearing can generate new signal processing techniques in digital hearing aids and cochlear implants in order to improve speech perception for these listeners.

This research is influenced by a wide range of fields: auditory neuroscience and physiology, neural network modeling and computational perception, high-level perception in other modalities (particularly vision), and information theory and the efficient coding hypothesis. See the Publications and Presentations pages for more information.