NSF Award
1023954
Human languages use pitch to convey meaning in a bewildering variety of ways. In all languages, pitch (as one aspect of speech prosody) can express attitude or emotion. In some languages, like English, pitch patterns, usually called intonation contours, also express distinctions such as that between a question and a statement. In languages like Mandarin Chinese, pitch patterns usually called tones go still further to signal differences between words that are otherwise identical. Despite significant advances in recent decades, a unified theoretical account of such linguistic phenomena remains elusive. What is missing is a common acoustic or articulatory vocabulary for expressing the relevant distinctions---a single measurable dimension within which spoken pitch contours (rises and falls) can be reliably distinguished regardless of the language under investigation. In recent work, the research team developed a new mathematical approach to tone and intonation, based on the notion of Tonal Center of Gravity. TCoG is a gestalt or global approach to tone perception and production that reconciles seemingly contradictory results from different strands of the experimental literature, moving toward a model that incorporates the best aspects of past theories, while avoiding their characteristic weaknesses. That earlier work has established that the TCoG approach accounts well for production and perception data involving two contrasting English intonation contours. This project aims to expand the empirical range of the approach in three crucial ways: First it extends the model to additional English intonation patterns. Second, it moves beyond English to look at other intonation languages (e.g., German), as well as so-called "tone languages" (e.g., Serbian). Lastly, whereas the previous work concentrated primarily on the timing of tonal events in speech, this project goes further, to investigate the interaction of tonal timing patterns with the scaling of tonal events in the pitch domain. The experimental work will be of two primary kinds: automatic classification of pitch contours recorded from native speakers in an experimental setting, and direct manipulation (through speech synthesis) of pitch contours in perception studies designed to determine which aspects of the acoustic signal have the greatest effect on listeners' judgments of utterance meaning.<br/><br/>Given the central role of intonation patterns in speech communication, one major contribution of the Tonal Center of Gravity approach is its potential to transform methods for speech synthesis and speech understanding. Synthetic speech is typically described as repetitive, detached, and often unhelpfully neutral; listeners recognize that they are talking with a machine that ?doesn?t get it?. By providing a more detailed understanding of how intonational patterns help to convey a message, TCoG could be used to devise algorithms for the synthesis of more natural and appropriate-sounding speech. Likewise, for automatic understanding of the aspects of meaning that depend on intonational patterns, TCoG could allow an automatic system to detect levels of nuance beyond simply whether a word is emphasized or not, or whether an utterance is a statement or a question. A final application of this work could be in the development of software tools for second language learning, in which automated instruction and feedback on the subtleties of second language intonation patterns could help learners master important aspects of communication that are typically ignored in current approaches to language pedagogy.
