Patent Grant
8447592
The present invention relates to voice synthesis, and more particularly, to formant-based voice synthesis.
Speech synthesis is a growing technology with applications in areas that include, but are not limited to, automated directory services, automated help desks and technology support infrastructure, human/computer interfaces, etc. Speech synthesis typically involves the production of electronic signals that, when broadcast, mimic human speech and are intelligible to a human listener or recipient. For example, in a typical text-to-speech application, text to be converted to speech is parsed into labeled phonemes which are then described by appropriately composed signals that drive an acoustic output, such as one or more resonators coupled to a speaker or other device capable of broadcasting sound waves.
Speech synthesis can be broadly categorized as using either concatenative or formant-based methods to generate synthesized speech. In concatenative approaches, speech is formed by appropriately concatenating pre-recorded voice fragments together, where each fragment may be a phoneme or other sound component of the target speech. One advantage of concatenative approaches is that, since it uses actual recordings of human speakers, it is relatively simple to synthesize natural sounding speech. However, the library of pre-recorded speech fragments needed to synthesize speech in a general manner requires relatively large amounts of storage, limiting application of concatenative approaches to systems that can tolerate a relatively large footprint, and/or systems that are not otherwise resource limited. In addition, there may be perceptual artifacts at transitions between speech fragments.
Formant-based approaches achieve voice synthesis by generating a model configured to build a speech signal using a relatively compact description or language that employs at least speech formants as a basis for the description. The model may, for example, consider the physical processes that occur in the human vocal tract when an individual speaks. To configure or train the model, recorded speech of known content may be parsed and analyzed to extract the speech formants in the signal. The term formant refers herein to certain resonant frequencies of speech. Speech formants are related to the physical processes of resonance in a substantially tubular vocal tract. The formants in a speech signal, and particularly the first three resonant frequencies, have been identified as being closely linked to, and characteristic of, the phonetic significance of sounds in human speech. As a result, a model may incorporate rules about how one or more formants should transition over time to mimic the desired sounds of the speech being synthesized.
Generally speaking, there are at least two phases to formant-based speech synthesis: 1) generating a speech synthesis model capable of producing a formant tract characteristic of target speech; and 2) speech production. Generating the speech synthesis model may include analyzing recorded speech signals, extracting formants from the speech signals and using knowledge gleaned from this information to train the model. Speech production generally involves using the trained speech synthesis model to generate the phonetic descriptions of the target speech, for example, generating an appropriate formant tract, and converting the description (e.g., via resonators) to an acoustic signal comprehensible to a human listener.
On embodiment according to the present invention includes a method of processing a voice signal to extract information to facilitate training a speech synthesis model, the method comprising acts of detecting a plurality of candidate features in the voice signal, performing at least one comparison between one or more combinations of the plurality of candidate features and the voice signal, and selecting a set of features from the plurality of candidate features based, at least in part, on the at least one comparison.
Another embodiment according to the present invention includes a computer readable medium encoded with a program for execution on at least one processor, the program, when executed on the at least one processor, performing a method of processing a voice signal to extract information from the voice signal to facilitate training a speech synthesis model, the method comprising acts of detecting a plurality of candidate features in the voice signal, performing at least one comparison between one or more combinations of the plurality of candidate features and the voice signal, and selecting a set of features from the plurality of candidate features based, at least in part, on the at least one comparison.
Another embodiment according to the present invention includes computer readable medium encoded with a speech synthesis model adapted to, when operating, generate human recognizable speech, the speech synthesis modeled trained to generate the human recognizable speech, at least in part, by performing acts of detecting a plurality of candidate features in the voice signal, performing a comparison between combinations of the candidate features and the voice signal, and selecting a desired set of features from the candidate features based, at least in part, on the comparison.
The efficacy by which a speech synthesis model can produce speech that sounds natural and/or is sufficiently intelligible to a human listener may depend, at least in part, on how well training data used to train the speech synthesis model describes the phonemes and other sound components of the target language. The quality of the training data, in turn, may depend upon how well characteristics and features of voice signals used to describe speech can be identified and selected from the voice signals. Applicant has appreciated that various methods of analysis by synthesis facilitate the selection of features from a voice signal that, when synthesized, produce a synthesized voice signal that is most similar to the original voice signal, either actually, perceptually, or both. The selected features may be used as training data to train a speech synthesis model to produce relatively natural sounding and/or intelligible speech.
As discussed above, generating a speech synthesis model typically includes an analysis phase wherein pre-recorded voice signals are processed to extract formant characteristics from the voice signals, and a training phase wherein the formant transitions for various language phonemes are used as a training set for a speech synthesis model. By way of highlighting at least some of the distinctions between conventional analysis and aspects of the present invention,
In act 110, the digitized voice signal may be parsed into segments of speech at regular intervals of time. For example, the digitized speech signal may be segmented into 20 ms windows at 10 ms intervals, such that the windows overlap each other in time. Each window may then be analyzed to identify formant candidates in the respective speech fragment. The windowing procedure may also process the voice signal, for example, by the use of a Hanning window. Processed or unprocessed, the discrete intervals of the speech signal are referred to herein as frames. In act 120, formant candidates are identified in each of the frames. Multiple candidates for the actual formants are typically identified in each frame due to the difficulty in accurately identifying the true formants and their associated parameters (e.g., formant location, bandwidth and amplitude), as discussed in further detail below.
In act 130, the candidate formants and associated parameters are further analyzed to identify the most likely formant sequence or formant tract. Conventional methods employ some form or combination of continuity constraints to select a formant tract from the candidates identified in act 120. Such conventional methods are premised on the notion that the true formant tract in the speech signal will have a relatively smooth transition over time. This smoothness constraint may be employed to eliminate candidates and to select formants for each frame that maximize the smoothness or best satisfy one or more continuity constraints between successive frames in the voice signal. The selected formants from each frame together make up the formant tract used as the description of the respective pre-recorded voice signal. In particular, the formant tract operates as a compact description of the phonetic make-up of the voice signal.
The term “tract” refers herein to a sequence of elements, typically ordered according to the respective element's position in time (unless otherwise specified). For example, a formant tract refers to a sequence of formants and conveys information about how the formants transition over time (e.g., about frame to frame transitions). Similarly, a feature tract is a sequence of one or more features. Each element in the tract may be a single value or multiple values. That is, a tract may be a sequence of scalar values, vectors or a combination of both. Each element need not contain the same number of values, and may represent and/or refer to any feature, characteristic or phenomena.
The selected formant tract may then be used to train the speech synthesis model (act 140). Common training schemes include Hidden Markov Models (HMM); however, any training method may be used. It should be appreciated that multiple speech signals may be analyzed and decomposed into formant tracts to provide training data that exemplifies how formants transition over time for a wide range of language phonemes for which the speech synthesis model in being trained. The trained speech synthesis model, therefore, is typically configured to generate a formant tract that describes a given phoneme that the model has been requested to synthesize. The formant tracts corresponding to the phonemes or other components of a target speech may then be generated as a function of time to produce the description of the target speech. This formant description may then be provided to one or more resonators for conversion to an acoustical signal comprehensible to a human listener.
Applicant has appreciated that conventional methods for selecting formants identified in a speech signal may not result in selected formants that provide a faithful description of the voice signal, resulting in a speech synthesis model that may not produce particularly high fidelity speech (e.g., natural sounding and/or intelligible speech). In particular, Applicant has appreciated that conventional constraints (e.g., continuity constraints, derivative constraints, etc.) applied to a formant tract may not be an optimal measure for selecting formants from formant candidates extracted from a speech signal. Applicant has noted that continuity and/or relatively smooth derivative characteristics in the formant tract may not be the best indicator of and/or may not lend itself to the most intelligible and/or natural sounding speech.
In one embodiment according to the present invention, formant tracts employed as training data are selected by selecting formants from available formant candidates based on a comparison with the speech signal. Exploiting the actual voice signal in the selection process may facilitate identifying formants that generate speech that is perceptually more similar to the voice signal then formants selected by forcing constraints on the formant tract that may have little correlation to how intelligible the synthesized speech ultimately sounds. Furthermore, Applicant has identified and appreciated that a speech synthesis model may be improved by incorporating, in addition to formant information, parameters describing other features of the voice signal into one or more feature tracts used to train the speech synthesis model.
Various embodiments of the present invention derive from Applicant's appreciation that analysis by synthesis may facilitate selecting features of a speech signal to train a speech synthesis model capable of producing speech that is relatively natural sounding and/or easily understood by a human listener. The resulting, relatively compact, speech synthesis model may then be employed in applications wherein resources are limited and/or are at a premium, in addition to applications wherein resources may not be scarce.
One embodiment of the present invention includes a method of processing a voice signal to determine characteristics for use in training of a speech synthesis model. The method comprises acts of detecting candidate features in the voice signal, performing a comparison between various combinations of the candidate features and the voice signal, and selecting a desired set of features from the candidate features based, at least in part, on the comparison. For example, in some embodiments, one or more formants are detected in the voice signal, and information about the detected formants are grouped into candidate feature sets.
Combinations of the candidate feature sets (e.g., a candidate feature set from each of a plurality of frames formed by respective intervals of the voice signal) may be grouped into candidate feature tracts presumed to provide a description of the voice signal. The voice signal, the candidate feature tracts or both may be converted into a format that facilitates a comparison between each candidate feature tract and the voice signal. The candidate feature tract that, when synthesized, produces a synthesized voice signal most similar to the original voice signal, may be selected as training data to train the speech synthesis model.
In another embodiment, a speech synthesis model trained via training data selected according to one or more analysis by synthesis methods is stored on a device to synthesize speech. In some embodiments, the device is a generally resource limited device, such as a cellular or mobile telephone. The speech synthesis model may be configured to convert text into speech so that small message service (SMS) messages may be listened to, or a user can interact with a telephone number directory via a voice interface. Other applications for said trained speech synthesis model include, but are not limited to, automated telephone directories, automated telephone services such as help desks, emails services, etc.
Following below are more detailed descriptions of various concepts related to, and embodiments of, methods and apparatus according to the present invention. It should be appreciated that various aspects of the inventions described herein may be implemented in any of numerous ways. Examples of specific implementations are provided herein for illustrative purposes only.
As discussed above, formants have been shown to be significantly correlated with the phonetic composition of speech. However, the true formants in speech are generally not trivially identified and extracted from a speech signal. Formant identification approaches have included various techniques of analyzing the frequency spectrum of speech signals to detect the speech formants. The formants, or resonant frequencies, often appear as peaks or local maxima in the frequency spectrum. However, noise in the voice signal or spectral zeroes in the spectrum often obscure formant peaks and cause “peak picking” algorithms to be generally error prone. To reduce the frequency of error, additional complexity may be added to the criteria used to identify true formants. For example, to be identified as a formant, the frequency peak may be required to meet a certain bandwidth and/or amplitude requirement. For example, peaks having bandwidths that exceed some predetermined threshold may be discarded as non-formant peaks. However, such methods are still vulnerable to mischaracterization.
To combat the general difficulty in identifying formants, a large number of formant candidates may be selected from the speech signals. Using a more inclusive identification scheme reduces the probability that the true formants will go undetected. By the same token, at least some (and likely many) of the identified formants will be spurious. That is, the inclusive identification scheme will generate numerous false positives. For example, one method of identifying candidate formants includes Linear Predictive Coding (LPC), wherein a predictor polynomial describes possible frequency and bandwidths for the formants. However, some of the identified formants are not true speech formants, resulting not from resonant frequencies, but from other voice phenomena, noise, etc. Numerous other methods have been used to identify multiple candidate formants in a speech signal.
The term “candidate” is used to describe an element (e.g., a formant, characteristic, feature, set of features, etc.) that is identified for potential use, for example, as a descriptor in training a speech synthesis model. Candidate elements may then be further analyzed to select desired elements from the identified candidates. For example, a pool of candidate formants (however identified) may be subjected to further processing in an attempt to eliminate spurious formants identified in the signal, i.e., to eliminate false positives. Predetermined criteria may be used to discard formants believed to have been identified erroneously in the initial formant detection stage, and to select what is believed to be the actual formants in the speech signal.
As discussed above, conventional methods of selecting the formant tract from candidate formants typically involve enforcing continuity and/or derivative constraints on the formant tract as it transitions between frames, or other measures that focus on characteristics of the resulting formant tracts. However, as indicated above, such selection methods are prone to selecting sub-optimal formant tracts. In particular, conventional selection methods may often select formant tracts that provide a relatively inaccurate description of the speech such that a speech synthesis model so trained may not produce particularly high quality speech. In one embodiment, various analysis by synthesis methods, in accordance with aspects of the present invention, are employed to improve upon the selection process.
In each frame, formants F are identified by some desired detection method, for example, by performing LPC on the speech signal. In the example of
Multiple candidates for each of the first three formants F1, F2 and F3 may be identified in each frame. For example, c candidates were chosen for each of frames 2120-212n, where c may be the same or different for each frame and/or different for each formant in each frame. The candidate formants are then provided to selection criteria 230, which selects one formant vector f=<F1, F2, F3> for each frame in the speech signal. Accordingly, the result of selection criteria 230 is a formant tract Ψ=<f0, f1, f2 . . . fn> where n is the number of frames in the speech signal. Formant tract Ψ may then be used as training data that characterize formant transitions for one or more phonemes or other sound components in voice signal 205, as described in further detail below.
It should be appreciated that the quality of speech synthesized by a model trained by various selected formant tracts Ψ may depend in significant part on how well Ψ describes the voice signal. Accordingly, Applicant has developed various methods that employ the actual voice signal to facilitate selecting the most appropriate formants to produce formant tract Ψ. For example, selection component 230 may perform various comparisons between the actual voice signal and voice signals synthesized from candidate formant tracts, such that the formant tract Ψ that is ultimately selected produces a voice signal, when synthesized, that most closely resembles the actual voice signal from which the formant tract was extracted. Various analysis by synthesis methods may result in a speech synthesis model that produces higher fidelity speech, as discussed in further detail below.
As discussed above, Applicant has appreciated that formants alone may not capture all the important characteristics of a voice signal that may be significant in producing quality synthesized speech. Various analyses by synthesis techniques may be used to select an optimal feature tract, wherein the features may include one or more formants, alone or in combination with, other features or characteristics of the voice signal. For example, exemplary features include one or any combination pitch, voicing, spectral slope, timing, timbre, stress, etc. Any property or characteristic indicative of a feature may be extracted from the voice signal. It should be appreciated that one or more formant features may be used exclusively or in combination with any one or combination of other features, as the aspects of the invention are not limited in this respect.
In
As discussed above, the detection process may include identifying multiple candidates for any particular feature to reduce the chance of noise or spectral zeroes obscuring the actual features being detected, or to mitigate otherwise failing to identify the true features of interest in the voice signal. Accordingly, in each frame, numerous feature candidates may be identified. For example, LPC may be used to identify formant candidates. Similarly, other feature detection algorithms may be used to identify other features or to identify candidate formants in the voice signal. As a result, each frame may produce multiple potential combinations of features. That is, each frame may have multiple candidate feature vectors Γ, where the feature vector Γ has a component for each feature of interest being identified in the voice signal. Each component may, in turn, be a vector or scalar quality or some other representation. For example, each component associated with a formant may have values corresponding to formant parameters such as peak frequency, bandwidth, amplitude, etc. Similarly, components associated with other features may have one or multiple values with which to characterize or otherwise represent the feature as desired.
Moreover, the process of feature identification will produce multiple candidate feature vectors F for each respective frame. As a result, the feature tract ΨB=(Γ0, Γ1, Γ2 . . . Γn), ultimately selected for use in training the speech synthesis model may be chosen from a relatively large number of possible combinations of candidate features. In the embodiment illustrated in
For example, a feature vector Γmi may be chosen from each frame to form candidate feature tract Ψj, where m is the index identifying the particular feature vector in a frame, and i is the frame from which the feature vector is chosen. Feature tract Ψj may then be provided to voice synthesizer 332 to convert the feature tract into a synthesized voice signal 335. Numerous methods of transforming a description of a voice signal into a relatively human intelligible voice signal are known in the art, and will not be discussed in detail herein. For example, one or a combination of resonators may be employed to convert the feature tract into a voice waveform which may be stored, further processed or otherwise provided for comparison with the actual voice signal or appropriate portion of the voice signal. Alternatively, voice synthesizer may convert the feature tract into an intermediate format, such as any number of digital or analog sound formats for comparison with the actual voice signal.
Voice synthesizer 332 may be any type of component or algorithm capable of reconstituting a voice signal in some suitable format from the selected description of the voice signal (e.g., reconstituting the voice signal from the relatively compact description Ψ). It should be appreciated that voice synthesizer 332 may provide a voice signal from a candidate feature tract in digital or analog form. Any format that facilitates a comparison between the synthesized voice signal and the actual voice signal may be used, as the aspects of the invention are not limited in this respect.
The synthesized voice signal 335 and the actual voice signal 305 may then be provided to comparator 337. In general, comparator 337 analyzes the two voice signals and provides a similarity measure between the two signals. For example, comparator 337 may compute a difference between the two voice signals, wherein the magnitude of the difference provides the similarity measure; the smaller the difference, the more similar the two signals (e.g., a least squares distance measure). However, it should be appreciated that comparator 337 may perform any type of analysis and/or comparison of the voice signals. In particular, comparator 337 may be provided with any level of sophistication to analyze the voice signals according to, for example, an understanding of particular differences that will result in speech that sounds less natural and/or is less intelligible to the human listener.
Applicant has appreciated that certain relatively large differences in the two signals may not result in proportional perceptual differences to a human listener. Likewise, Applicant has identified that certain characteristics of the voice signal have greater impact on how the voice signal is perceived by the human ear. This knowledge and understanding of what differences may be perceptually significant may be incorporated into the analysis performed by comparator 337. It should be appreciated that any comparison and/or analysis may be performed that results in some measure of the similarity of the synthesized and actual voice signals, as the aspects of the invention are not limited for use with any particular comparison, analysis and/or measure.
After each candidate feature tract Ψj has been synthesized and compared with the actual voice signal, the feature tract ΨB resulting in a synthesized voice signal most similar to the actual voice signal or portion of the voice signal may be selected as training data associated with voice signal 305 to be used in training the voice synthesis model on one or more phonemes or sound components present in the voice signal. It should be appreciated that any number of candidate feature tracts may be used in the comparison, as the aspects of the invention are not limited in this respect. As discussed in further detail below, this procedure may be repeated on any number of voice signals of any type and variety to provide a robust set of training data to train the speech synthesis model.
For example, interpreter 433 may perform a function H such that H(Ψj)=Y*, where Y* is the feature tract expressed in an intermediate format. Similarly, interpreter 433 may perform a function G such that G(S)=Y, where S is the appropriate portion of voice signal 405 and Y is the voice signal expressed in the intermediate format. Since both signals are in the same general format, they can be compared by comparator 437 according to any desired comparison scheme that provides an indication of the similarity between Y and Y*. Accordingly, the selection process may include selecting the Ψj that minimizes differences between Y and Y*. As discussed above, the difference may include any measure, for example, a least squares distance, or may be based on a comparison that incorporates information about what differences may have greater or lesser perceptual impact on the resulting synthesized voice signal. It should be appreciated that any comparison may be used, as the aspects of the invention are not limited in this respect.
In some embodiments, the voice signal Y is already in the proper format. For example, the digital format in which the voice signal is stored may operate as the intermediate format. Accordingly, in such embodiments, interpreter 433 may only operate on the feature tract via a function H that converts the feature tract into the same format as the voice signal. It should be appreciated that either the voice signal, the feature tract or both may be converted to a new format to prepare the two signals 435 and 405′ for comparison and interpreter 433 may perform any type of conversion that facilitates a comparison between the two signals, as the aspects of the invention are not limited in this respect.
It should be appreciated that feature tracts may be selected according to the above for any number and type of voice signals. As a general matter, feature tracts are selected from chosen voice signals such that the training mechanism used to train the speech synthesis model has feature tracts corresponding to the significant phonemes in the target language of the speech desired to be synthesized. For example, one or more feature tracts may be selected that describe each of the vowel and consonant sounds used in the target language. By extension, feature tracts may be selected to train a speech synthesis model in any number of languages by performing any of the exemplary embodiments described above on voice signals recorded in other languages. In addition, feature tracts may be selected to train a speech synthesis model to provide speech with a desired prosody or emotion, or to provide speech in a whisper, a yell or to sing the speech, or to provide some other voice effect, as discussed in further detail below.
As discussed above, many forms of training a speech synthesis model M are known in the art, and any training mechanism may be used as training 550, as the aspects of the invention are not limited in this respect. For example, Hidden Markov Models (HMM) are commonly used and well understood techniques for training a speech synthesis model. In the embodiment in
In particular, training 550 may receive training data Ω=(ΨB0, ΨB1, ΨB2, . . . ΨBw), wherein the various selected feature tracts Ψ provide a desired coverage of the phonemes that constitute the desired speech. In some embodiments, training data 545 includes feature tracts that describe phonemes of speech deemed significant in forming natural and/or intelligible speech. For example, the training data may include one or more feature tracts that describe each of the vowel sounds of a target language. In addition, the various feature tracts may describe various consonant sounds, sibilance characteristics, transitions between one more phonemes, etc. The feature tracts provided to training may be chosen at any level of sophistication to train the speech synthesis model, as the aspects of the invention are not limited in this respect. Training 550 then operates on the training data and generates speech synthesis model 555, for example, exemplary speech model M.
As discussed above, by utilizing a formant based description (and perhaps other selected features), a speech synthesis model can be generated that uses a relatively compact language to describe speech. Accordingly, speech synthesis models so derived may be employed in various applications where resources may be generally scarce, such as on a cellular phone. Applicant has appreciated that numerous applications may benefit from such models generated using methods in accordance with the present invention, where compact description and relatively high fidelity (e.g., natural sounding and/or intelligible speech) speech synthesis is desired.
For example, the user may speak into the telephone the name of the person the user would like to contact (act 610). Speech recognition software also stored on the phone (not shown) may convert the voice signal into text or another digital representation (act 620). The digital representation, for example, a text description of the contact person, is used to index into the directory stored on the phone (act 630). When and if a match is found, the directory entry (e.g., a name index that may be in text or other digital form) is provided to the speech synthesis model to confirm that the matched contact is correct (act 640). That is, the name of the matched directory entry may be converted to a voice signal that is broadcast out of the phone's speaker so that the user can confirm that the intended contact and the matched contact are in agreement. Once confirmed, the telephone number associated with the matched contact may be automatically dialed by the telephone. Applicant has appreciated that speech synthesis models derived according to various aspects of the present invention may be compact enough to be stored on generally resource limited cellular phones and can produce relatively natural sounding speech and/or speech that is generally intelligible to the human listener, although such benefits and advantages are not a requirement or limitation on the aspects of the present invention.
Another application wherein a speech synthesis model may be applied on a cell phone is in the context of text messages, for example, short message service (SMS) messages sent from one cellular phone user to another. Such a feature would allow user's to listen to their text messages, and may be desirable to sight impaired users, or as a convenience to other users, or for entertainment purposes, etc. It should be appreciated that speech synthesis models derived from various aspects of the invention may be used in any application where speech synthesis is desirable and is not limited to applications where resources are generally limited, or to any other application specifically described herein. For example, speech synthesis models derived as described herein may be used in a telephone directory service, or a phone service that permits the user to listen to his or her e-mails, or in an automated directory service.
As discussed in the foregoing, features tracts may be identified and selected based on any number and type of voice signals. Accordingly, a model may be trained to generate speech in any of various languages. In addition, feature tracts may be selected that describe voice signals recorded from speakers of different gender, using different emotions such as angry or sad or using other speech dynamics or effects such as yelling, laughing, singing, or a particular dialect or slang. Moreover, prosody effects such as questioning or exclamatory statements, or other intonations may be trained into a speech synthesis model.
Applicant has appreciated that additional components may be added to a speech synthesis model to enhance the speech synthesis model with one or more of the above add-ons. In
The model M may also be trained on voice signals recorded according to any number of effects, to generate multiple components Ci. A library 760 of such components may be generated and stored or archived. For example, library 760 may include a component adapted to generate speech perceived as being spoken with a desired emotion (e.g., angry, happy, laughing, etc.). In addition, library 760 may include a component for any number of desired languages, dialects, accents, gender, etc. Library 760 may include a component for one or any combination of speech attributes or effects, as the aspects of the invention are not limited in this respect.
The library may be made available for download or otherwise distributed for sale. For example, a cellular phone user may access the library over a network via the cellular phone and download additional components in a fashion similar to downloading additional ring tones or games for a cellular phone. The speech synthesis model, stored on the cellular phone with the standard the component, may be enhanced with one or more other components as desired by the owner/user of the cellular phone.
It should be appreciated that enhancement components may be independent of one another or may alternatively be modifications to the existing speech synthesis model. For example, Ci may instruct model M on which particular formant tracts or phonemes generated by component C0 need to be changed in order produce the desired effect. That is, Ci may supplement the existing model M operating on C0, and instruct the model how to modify or adjust the description of the voice signal such that the resulting voice signal has the desired effect. Alternatively, Ci may be a relatively independent component, wherein when the desired effect characterized by C1 is desired, model M generates a description (e.g., one or more feature tracts) according to C, with little or no involvement from C0. Other methods of making a generally scaleable voice synthesis model may be used, as aspects of the invention are not limited in this respect.
The above-described embodiments of the present invention can be implemented in any of numerous ways. For example, the embodiments may be implemented using hardware, software or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers. It should be appreciated that any component or collection of components that perform the functions described above can be generically considered as one or more controllers that control the above-discussed function. The one or more controller can be implemented in numerous ways, such as with dedicated hardware, or with general purpose hardware (e.g., one or more processor) that is programmed using microcode or software to perform the functions recited above.
It should be appreciated that the various methods outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages and/or conventional programming or scripting tools, and also may be compiled as executable machine language code.
In this respect, it should be appreciated that one embodiment of the invention is directed to a computer readable medium (or multiple computer readable media) (e.g., a computer memory, one or more floppy discs, compact discs, optical discs, magnetic tapes, etc.) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the invention discussed above. The computer readable medium or media can be transportable, such that the program or programs stored thereon can be loaded onto one or more different computers or other processors to implement various aspects of the present invention as discussed above.
It should be understood that the term “program” is used herein in a generic sense to refer to any type of computer code or set of instructions that can be employed to program a computer or other processor to implement various aspects of the present invention as discussed above. Additionally, it should be appreciated that according to one aspect of this embodiment, one or more computer programs that when executed perform methods of the present invention need not reside on a single computer or processor, but may be distributed in a modular fashion amongst a number of different computers or processors to implement various aspects of the present invention.
Various aspects of the present invention may be used alone, in combination, or in a variety of arrangements not specifically discussed in the embodiments described in the foregoing and is therefore not limited in its application to the details and arrangement of components set forth in the foregoing description or illustrated in the drawings. The invention is capable of other embodiments and of being practiced or of being carried out in various ways. In particular, various aspects of the invention may be used to train voice synthesis models of any type and trained in any way. In addition, any type and/or number of features may be selected from any number and type of voice signals or recordings. Accordingly, the foregoing description and drawings are by way of example only.
Use of ordinal terms such as “first”, “second”, “third”, etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term) to distinguish the claim elements.
Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having,” “containing”, “involving”, and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4087632 | Hafer | May 1978 | A |
| 5146539 | Doddington et al. | Sep 1992 | A |
| 5644680 | Bielby et al. | Jul 1997 | A |
| 5664054 | Su | Sep 1997 | A |
| 5867814 | Yong | Feb 1999 | A |
| 6047254 | Ireton et al. | Apr 2000 | A |
| 6064960 | Bellegarda et al. | May 2000 | A |
| 6101470 | Eide et al. | Aug 2000 | A |
| 6260009 | Dejaco | Jul 2001 | B1 |
| 6366883 | Campbell et al. | Apr 2002 | B1 |
| 6484139 | Yajima | Nov 2002 | B2 |
| 6505152 | Acero | Jan 2003 | B1 |
| 6708154 | Acero | Mar 2004 | B2 |
| 6801931 | Ramesh et al. | Oct 2004 | B1 |
| 20010007973 | Yajima | Jul 2001 | A1 |
| 20010021904 | Plumpe | Sep 2001 | A1 |
| 20020049594 | Moore et al. | Apr 2002 | A1 |
| 20020135618 | Maes et al. | Sep 2002 | A1 |
| 20050027528 | Yantorno et al. | Feb 2005 | A1 |
| 20050137862 | Monkowski | Jun 2005 | A1 |
| 20050182619 | Azara et al. | Aug 2005 | A1 |
| 20060074676 | Deng et al. | Apr 2006 | A1 |
| Number | Date | Country |
|---|---|---|
| WO 9926234 | May 1999 | WO |
| Entry |
|---|
| A. Acero, “Formant Analysis and Synthesis using Hidden Markov Models,” Proc. of the Eurospeech Conference, Sep. 1999, pp. 1-4, Budapest, Hungary. |
| E. Bryan George and Mark J. T. Smith, “Speech Analysis/Synthesis and Modification Using an Analysis-by-Synthesis/Overlap-Add Sinusoidal Model,” IEEE Trans on Speech and Audio Processing, vol. 5, No. 5, Sep. 1997, pp. 389-406. |
| Katsunobu Fushikida, “A Formant Extraction Method Using Autocorrelation Domain Inverse Filtering and Focusing Method,” ICASSP 88, Apr. 11, 1988, pp. 2260-2263. |
| Number | Date | Country | |
|---|---|---|---|
| 20070061145 A1 | Mar 2007 | US |
