Search Results

Scene analysis is the name given to the strategy by which a computer attempts to put together all the visible properties – edges, surface textures, colours, distances, and so on – that belong to the same object. Only then can the correct global shape and properties of that object be determined. By analogy, auditory scene analysis is the process whereby all the auditory evidence that comes, over time, from a single environmental source is put together as a perceptual unit. This chapter describes the methods that the auditory system employs and some of the research which has discovered them.

Musical passages exhibit a wide range of textures. These can include monophony, tune-and-accompaniment, homophony, close harmony, polyphony, pseudo-polyphony, heterophony, and a wealth of specially tailored arrangements with various hierarchical structures. Introductory music theory textbooks generally focus on Baroque voice-leading rules to the virtual exclusion of other types of part-writing. Although most music-making bears little resemblance to Baroque-style four-part chorale writing, there are excellent reasons why this particular practice has formed the core theory curriculum for so long. The evidence suggests that late Baroque practice most closely reflects known principles of auditory scene analysis. The perceptual principles underlying voice leading provide an important entry point for understanding any musical texture—no matter what the style, culture, or genre of music-making. Like a theatrical stage, composers set a “musical scene.” Auditory scene analysis is the process by which listeners subjectively apprehend that scene.

An introduction to the perception of sound is given, with special emphasis on topics useful for understanding the organization of music. The chapter covers essential concepts in acoustics and auditory perception, including basic auditory anatomy and physiology. Core concepts are defined such as vibrational mode, pure tone, complex tone, partial, harmonic, cochlea, basilar membrane, resolved partial, auditory image, auditory stream, acoustic scene, auditory scene, and auditory scene analysis.

The disposition to parse auditory scenes is probably an evolved innate behavior. However, the means by which this is achieved likely involves a mix of innate and learned mechanisms. This chapter reviews research showing how the sonic environment plays a formative role in various aspects of auditory processing. What we commonly hear shapes how we hear sounds. For example, research shows that how musicians hear pitch is affected by what instrument they play. Even the language you speak has an impact on how you hear. It is wrong to assume that everyone parses an acoustic scene in the same way. In general, the research suggests that cultural background and individual experience may be directly relevant to our understanding of auditory scene analysis, and hence to voice leading.

Hierarchical streaming is discussed. Auditory scenes can exhibit different levels of organization. At the lowest level, individual partials may amalgamate to form auditory images corresponding to distinct sound sources. At a higher level, these auditory images can combine together to form intermediate musical layers, dubbed textural streams. At the highest level, all of the sound sources may combine to form a unitary percept of the whole experience, dubbed a musical stream. The distinction between auditory streams and textural streams means that there is more to part-writing than simply writing parts; there is also the counterpoint of textural streams. This chapter also introduces scene analysis trees as a graphical analytic tool for better understanding different kinds of musical textures.

Accurate representation of the dynamic aspects of sound in the auditory nervous system are crucial in understanding speech, enjoying music, being able to localize a sound source, and make it possible to communicate in noisy environments. Stationary sounds are represented in the auditory system in a way that onsets and offsets produce more activity than the parts in between. This is called perstimulatory adaptation of firing rate. The mechanisms underlying it also play a role in the masking effects of other sounds on the ones of interest, and determine our sensitivity to periodically time varying sounds. Stimulus-specific adaptation also may be largely determined by this mechanism. The first five chapters in the book describe the role of adaptation mechanisms from auditory nerve to auditory cortex. Chapter 6 describes a simple phenomenological model that links together perstimulatory adaptation and recovery therefrom with forward masking and temporal modulation transfer functions. Neural synchronization and its role in brain rhythms and perception are elucidated. Based on this, the role of temporal processing in periodicity pitch, sound localization, stream segregation and scene analysis are reviewed. Temporal processing ability of the nervous system is affected by maturation as well as aging, and on top of that by hearing loss. Less clear is the role of temporal processing deficits in dyslexia, specific language impairment and auditory processing disorders, potentially because of the confounding role of maturation and aging. Various neurological disorders such as auditory neuropathy, multiple sclerosis, schizophrenia, autism and epilepsy present itself with temporal processing deficits. These deficits are often multimodal in nature and this is reflected on in the final chapter.

It is amazing that in a noisy, multiple-people-talking environment, listeners with normal hearing can still recognize and understand the attended speech and simultaneously ignore background noise and irrelevant speech stimuli. How do we recognize what one person is saying when others are speaking at the same time? There are two challenges for a listener in a “cocktail party” situation. The first is the problem of sound segregation. The auditory system must derive the properties of individual sounds from the mixture entering the ears. The second challenge is that of directing attention to the sound source of interest while ignoring the others. In addition temporal structure has a key role in stream segregation. Timing synchrony of frequency partials allow fusion into a more complex sound, and if the frequency partials are harmonic the fusion is more likely. In contrast, timing asynchrony is a major element to distinguish one stream from two streams. Animal experiments have highlighted the role of temporal aspects by comparing behavioral data and recordings from the forebrain in the same species. Feature dependent forward suppression in auditory cortex may underlie streaming. Modeling studies suggest that stream formation depends primarily on temporal coherence between responses that encode various features of a sound source. Furthermore, it is postulated that only when attention is directed towards a particular feature (e.g. pitch) do all other temporally coherent features of that source (e.g. timbre and location) become bound together as a stream that is segregated from the incoherent features of other sources.

Chapter 3 first explores the principles by which we organize elements of an array into groupings. The Gestalt psychologists proposed a set of grouping principles that have profoundly influenced the study of hearing and vision ever since—these include “proximity,” “similarity,” “good continuation,” “common fate,” and “closure.” Passages of conventional tonal music illustrating these principles are described, along with several illusions and other surprising characteristics of music and speech, all presented as sound examples. They involve the segregation of pitch sequences into separate streams based on proximity in pitch or in time, and also on timbre or sound quality. Figure–ground relationships, analogous to those in vision, are also discussed. Much information arrives at our sense organs in fragmented form, and the perceptual system needs to infer continuities between the fragments, and fill in the gaps appropriately. It is shown that this occurs in both music and speech. We have evolved mechanisms to perform these tasks, but these mechanisms often fool us into “hearing” sounds that are not really there. Another approach to perceptual organization in music exploits the use of orchestral sound textures to create ambiguous images. This approach has been used to excellent effect in 20th-century music such as film scores; for example, it contributes to the mysterious ambience in Stanley Kubrick’s 2001: A Space Odyssey.