The present invention is directed, in general, to automatic speech recognition (ASR) and, more specifically, to systems and methods employing stochastic bias compensation and Bayesian joint additive/convolutive compensation in ASR.
Over the last few decades, the focus in ASP has gradually shifted from laboratory experiments performed on carefully enunciated speech received by high-fidelity equipment in quiet environments to real applications having to cope with normal speech received by low-cost equipment in noisy environments.
In such situations, an ASR system may often be required to work with mismatches conditions between pre-trained speaker-independent acoustic models and a speaker-dependent voice signal. Mismatches are often caused by environmental distortions. Environmental distortions may be additive in nature—background noise, such as a computer fan, a car engine or road noise (see, e.g., Gong, “A Method of Joint Compensation of Additive and Convolutive Distortions for Speaker-Independent Speech Recognition,” IEEE Trans. on Speech and Audio Processing, vol. 13, no. 5, pp. 975-983, 2005). Environmental distortions may be convolutive in nature—changes in microphone type (e.g., a hand-held microphone or a hands-free microphone) or position relative to the speaker's mouth, which determines the envelope of the speech spectrum. Speaker-dependent characteristics, such as variations in vocal tract geometry, introduce mismatches. These mismatches tend to degrade the performance of an ASR system dramatically. In mobile ASR applications, these distortions occur routinely. Therefore, a practical ASR system needs to be able to operate successfully despite these distortions.
Hidden Markov models (HMMs) are widely used in the current ASR systems. The above distortion may affect HMMs in many aspects. Among them, shift of mean vectors, or additional biases to the pre-trained mean vectors, is a major effect. Many techniques have been developed in an attempt to compensate for these distortions. Generally, the techniques may be classified into two approaches: front-end techniques that recover clean speech from a noisy observation (see, e.g., ETSI, “Evaluation of a Noise-Robust DSR Front-End on Aurora Databases,” in ICSLP, 2002, vol. 1, pp. 17-20, Acero, et al., Environmental Robustness in Automatic Speech Recognition, in ICASSP, 1990, vol.2, pp. 849-852, Deng, et al., “Recursive Estimation of Nonstationary Noise Using Iterative Stochastic Approximation for Robust Speech Recognition,” IEEE Trans. on Speech and Audio Processing, vol. 11, no. 6, pp. 568-580, 2003, Moreno, et al., “A Vector Taylor Series Approach for Environment-Independent Speech Recognition,” in ICASSP, 1996, vol. 2, pp. 733-736, Hermansky, et al., “Rasta-PLP Speech Analysis Technique,” in ICASSP, 1992, pp. 121-124, Rahim, et al., “Signal Bias Removal by Maximum Likelihood Estimation for Robust Telephone Speech Recognition,” IEEE Trans. on Speech and Audio Processing, vol. 4, no. 1, pp. 19-30, January 1996, and Hilger, et al., “Quantile Based Histogram Equalization for Noise Robust Speech Recognition,” in EUROSPEECH, 2001, pp. 1135-1138) and back-end techniques that adjust model parameters to better match the distribution of a noisy speech signal (see, e.g., Gales, et al., “Robust Speech Recognition in Additive and Convolutional Noise Using Parallel Model Combination,” Computer Speech and Language, vol. 9, pp. 289-307, 1995, Sankar, et al., “A Maximum-Likelihood Approach to Stochastic Matching for Robust Speech Recognition,” IEEE Trans, on Speech and Audio Processing, vol. 4, no. 3, pp. 190-201, 1996, Yao, et al., “Noise Adaptive Speech Recognition Based on Sequential Noise Parameter Estimation,” Speech Communication, vol. 42, no. 1, pp. 5-23, 2004, Zhao, “Maximum Likelihood Joint Estimation of Channel and Noise for Robust Speech Recognition,” in ICASSP, 2000, vol. 2, pp. 1109-1113, Woodland, et al., “Improving Environmental Robustness in Large Vocabulary Speech Recognition,” in ICASSP, 1996, pp. 65-68, and Chou, “Maximum a Posterior Linear Regression based Variance Adaptation of Continuous Density HMMs,” Technical Report ALR-2002-045, Avaya Labs Research, 2002).
Usually, back-end techniques adapt original acoustic models with a few samples from a testing speech signal. The adaptation may be done parametrically with a parametric mismatch function that combines clean speech and distortion. For example, parallel model combination, or PMC (see, e.g., Gales, et al., supra) transforms original acoustic model by combining clean speech mean vectors with those from noise samples. Adaptation may also be done without a parametric mismatch function, instead applying linear regression on noisy and original observations with some optimization criteria. For example, maximum-likelihood linear regression, or MLLR (see, e.g., Woodland, et al., supra), estimates cluster-dependent linear transformations by increasing likelihood of noisy signal given the original acoustic models and the transformations. These linear regression methods are more general than the above-described parametric methods such as PMC, as the linear regression methods can deal with distortion other than that is modeled by the parametric mismatch function used, for example, in PMC. However, to achieve reliable regressions, sufficient data may be required in these linear-regression based techniques. In mobile application of ASR, since it is not realistic to obtain enough adaptation data due to frequent changes of testing environment, the parametric methods such as PMC are more often used than the regression methods such as MLLR.
While techniques employing explicit mismatch functions often require relatively few adaptation utterances to transform acoustic models reliably, they have so far proven unable to deal with other types of distortion in speech recognition, such as mismatches caused by accent, etc, which are difficult to be modeled with a precise parametric function describing their effects on speech recognition. Notice that mobile devices are used widely in a variety of environments, which may have distortions caused not only by background noise and convolutive channel distortions, but also by changes of speakers and different accents. Such devices often contain a digital signal processor (DSP).
Accordingly, what is needed in the art are systems and methods based on improved techniques, applicable to ASR, for providing compensation for a wide variety of mismatch. The improved techniques may combine the parametric methods and the linear regression methods and should compensate background noise, channel distortion and other types of distortion jointly. The systems and methods should be adaptable for use in platforms in which computing resources are limited, such as mobile communication devices.
To address the above-discussed deficiencies of the prior art, the present invention provides improved techniques, applicable to ASR, for providing compensation for mismatch.
The foregoing has outlined features of the present invention so that those skilled in the art may better understand the detailed description of the invention that follows. Additional features of the invention will be described hereinafter that form the subject of the claims of the invention. Those skilled in the art should appreciate that they can readily use the disclosed conception and specific embodiment as a basis for designing or modifying other structures for carrying out the same purposes of the present invention. Those skilled in the art should also realize that such equivalent constructions do not depart from the spirit and scope of the invention.
For a more complete understanding of the invention, reference is now made to the following descriptions taken in conjunction with the accompanying drawing, in which:
Two related techniques applicable to ASR for providing back-end compensation for mismatch, caused by, for example, environmental effects, will be described herein. The first is called “stochastic bias compensation,” or SEC, and the second is called “Bayesian joint additive/convolutive compensation,” or B-IJAC. An exemplary environment and system within which the two techniques may be carried out will first be described. Then, various embodiments of each technique will be described. Finally, experiments will be set forth regarding the performance of SEC and B-IJAC.
Accordingly, referring to
One advantageous application for the system or method of the present invention is in conjunction with the mobile telecommunication devices 110a, 110b. Although not shown in
Certain embodiments of the present invention described herein are particularly suitable for operation in the DSP. The DSP may be a commercially available DSP from Texas Instruments of Dallas, Tex. An embodiment of the system in such a context will now be described.
Turning now to
The system includes a background noise estimator 210. The background noise estimator 210 is configured to generate a current background noise estimate from a current utterance. The system further includes an acoustic model compensator 220. The acoustic model compensator 220 is associated with the background noise estimator 210 and is configured to use a previous channel distortion estimate and the current background noise estimate to compensate acoustic models and recognize a current utterance in the speech signal.
The system further includes an utterance aligner 230. The utterance aligner 230 is associated with the acoustic model compensator 220 and is configured to align the current utterance using recognition output. The system further includes a channel distortion estimator 240. The channel distortion estimator 240 is associated with the utterance aligner and is configured to generate a current channel distortion estimate from the current utterance.
The system further includes a bias estimator 250. The bias estimator 250 is associated with the utterance aligner 230, the noise estimator 210 and the channel estimator 240 and is configured to generate estimates of bias terms from the current utterance. Once the bias estimator 250 has generated the bias term estimates, the next utterance is analyzed whereupon the background noise estimator 210 regards the just-generated current channel distortion estimate as the previous channel distortion estimate and the just-generated bias terms estimates as the previous estimates of bias terms and the process continues through a sequence of utterances.
Stochastic Bias Compensation
SBC is a back-end model transformation technique for decreasing mismatch between a testing speech signal and trained acoustic models applied to robust ASR. SBC in uses a parametric function to model environmental distortion, such as background noise and channel distortion, and a cluster-dependent bias to model other types of distortion.
Effects of channel distortion and background noise on mean vectors of clean speech are modeled with a parametric mismatch function, and these distortions are estimated from noisy speech. In addition, biases to the compensated mean are introduced to account for possible other distortions that are not well modeled by the parametric mismatch function. These biases are phonetically clustered. In some embodiments, an E-M-type algorithm may be used to estimate channel distortion, background noise and the biases jointly.
SBC is based on two assumptions. The first assumption is that environmental effects on clean MFCC features can be represented as a non-linear mismatch function (see, e.g., Acero, supra, Gales, et al., supra, and Yao, et al., supra) . The second assumption is that other distortion may be represented as an additional bias. Based upon these two assumptions, the observation in the log-spectral domain is represented as two terms as follows:
Yl(k)=g(Xl(k),Hl(k),Nl(k))+C−1B(k), (1)
where the first term, g(Xl(k),Hl(k),Nl(k)), is:
g(Xl(k),Hl(k),Nl(k))=log(exp(X1(k)+Hl(k))+exp(Nl(k))), (2)
and Xl(k), H1(k) and Nl(k) respectively denote clean speech, channel distortion and noise in the log-spectral domain. The superscript l denotes the log-spectral domain. The second term, B(k), is a bias term that represents effects due to other distortions. C−1 denotes an inverse Cosine transformation. Feature vectors are implicitly assumed in the cepstral domain. Hence the superscript denoting the cepstral domain is ignored herein.
The goal is to derive a segmental algorithm for estimating statistics of Hl(k), Nl(k) and B(k) and compensating for their effects on clean MFCC feature vectors. Acoustic models are continuous-density hidden Markov models (CD-HMMs), represented as ΛX={{πq, aqq, cqp, μqp, Σqp}: q,q′=1 . . . S,p=1 . . . M}, where μqp has elements {μqpd:d=1 . . . D} and Σqp has elements {σ2qpd:d=1 . . . D}. The acoustic model is trained on clean MFCC feature vectors.
Let R be the number of utterances available for estimating distortion factors. Let Kr be the number of frames in utterance r. m denotes a mixture component in state s. Let S={Sk} and L={mk} be the state and mixture sequences corresponding to the observation sequence Yr(1:Kr) for utterance r. The Bayesian estimates or maximum a posteriori probability (MAP) estimates of channel distortion can be written below as
Because of the hidden nature of the state and mixture occupancy in HMMs, the MAP optimization problem described in Equation (3) is difficult to solve directly, particularly in view of the limited resources of a mobile communication device. Fortunately, the problem can be more readily solved indirectly using an iterative algorithm called Expectation-Maximization (E-M) (see, e.g., Dempster, et al., “Maximum Likelihood from Incomplete Data Via the E-M Algorithm,” J. Royal Stat. Soc., vol. 39, no. 1, pp. 1-38, 1977) by maximizing the auxiliary function:
Q(R)(Hl|
where Hl is the channel estimate from the previous E-M iteration.
The first (E) step of the E-M algorithm involves deriving the right-hand side of Equation (4). The second (M) step of the E-M algorithm involves deriving Hl such that Q(R)(Hl|
Although channel distortion may be considered slowly varying, background noise may change dramatically from one utterance to the next. Therefore, the well-known maximum likelihood principle may be used in lieu of the above-mentioned MAP estimates to estimate background noise from the current utterance R. The objective function can be written as:
The E-M algorithm may be similarly applied to obtain NlML. The auxiliary function for noise estimates is:
Q(R)(Nl|
where
Similarly, the bias term B may be estimated by the E-M algorithm with the following auxiliary function:
Q(R)(Bl|
where
To obtain a triplet of (HlMAP, NlML,BML) that increases the auxiliary functions of Equations (4), (6) and (7), the following approach may be taken. First, Nl is fixed equal to
The auxiliary function corresponding to the right-hand side of Equation (4) can be rewritten as:
where the posterior probability γsm r(k)=p(sk=s,mk=m)|Yr(1:Kr),
The variance of a Gaussian density is assumed not to be distorted due to environmental effects. B(k) can therefore be moved to the left-hand side of Equation (1), yielding the following form for p(Yr(k)|sk=s,mk=m,Hl, Nl, B, ΛX):
p(Yr(k)|Sk=s,mk=m,Hl, NlB,ΛX)=bc(sm)(Yr(k))˜N(Yr(k)−Bc(sm); {circumflex over (μ)}smσsm2, (9)
where {circumflex over (μ)}sm=g(μsm, Hl, Nl) is the noisy mean after compensating for environmental distortion, bc(sm) is a cluster-dependent bias term, and c(sm) determines the cluster for state Sk=S and mixture mk=m.
As is usual in MAP estimation, the choice of the prior density p(Hl) may be based on either some physical characteristics of the channel distortion Hl or on some attractive mathematical attribute, such as the existence of conjugate prior densities, which can greatly simplify the maximization of Equation (8) (see, e.g., Gauvain, et al., “Maximum a Posteriori Estimation for Multivariate Gaussian Mixture Observations of Markov Chains,” IEEE Trans. on Speech and Audio Processing, vol. 2, no. 2, pp. 291-298, 1994). Prior densities from a family of elliptically symmetric distributions called “matrix version of multivariate normal prior density,” may be useful (see, e.g., Chou, supra).
One peculiarity of MAP estimation is that the formulation is still valid when the prior density is not a probability density function. The only constraint that the prior density should be a nonnegative function. It is therefore possible to select from many different prior densities as long as good estimates of their location and scale parameters can be derived. Without limiting the scope of the present invention, the following prior density is chosen for use herein:
p(Hl)˜N(Hl; Vl,Wl), (10)
where Vl and Wl are the prior mean and variance of the channel distortion Hl. The motivation to select this density is that its hyper-parameters Vl and Wl can be derived in a straightforward manner. In particular, Vl is selected to be the channel estimates from previous iteration, yielding the following function:
p(Hl˜N(Hl;
where ΣH
An iterative technique may be used to estimate channel distortion and thereby maximize Equation (8) with respect to Hl. A Gauss-Newton technique may be advantageously used to update the channel distortion estimate due to its rapid convergence rate. Using the Gauss-Newton technique, the new estimate of channel distortion is:
where ε is a factor between 0.0 and 1.0.
Using the chain rule of differentiation, the first-order differentials with respect to channel distortion Hl are:
where β is the weight of the prior density, and σqp2
The second order differentials with respect to the channel distortion Hl are:
where, by straightforward algebraic manipulation of Equation (2), the first- and second-order differentials of g(μqpl, Hl, Nl) in Equations (13) and (14) are:
Updating Equations (13) and (14) may be further simplified in consideration of reducing computational costs. Specifically, the variance term in the log-spectral domain is costly to obtain due to heavy transformations between the cepstral and the log-spectral domains. Equations (13) and (14) may be simplified by removing the variance vector in the first terms of Equations (13) and (14); i.e.:
By setting β=0, the above functions correspond to a non-Bayesian joint additive/convolutive compensation technique called “IJAC” (see, U.S. Patent Application Serial No. [Attorney Docket Number TI-39862AA], supra) . A further simplification may arrive at another non-Bayesian joint additive/convolutive compensation technique called “JAC” (Gong, supra) and where Equations (17) and (18) are:
Equations (19) and (20) relate to Equations (17) and (18) with the following four assumptions:
By Equation (15), 1−ΔH
Some modeling error may arise as a result of some of these simplifications. If so, the updating of Equation (12) may result in a biased estimate of channel distortion. To counter effects due to the simplification, a discounting factor ξ is introduced herein. The discounting factor ξ is multiplied with the previous estimate to diminish its influence. With the discounting factor ξ, the updating function becomes:
In the illustrated embodiment, the discounting factor ξ is not used in calculating the sufficient statistic of the E-M algorithm. Therefore, introduction of discounting factor ξ causes a potential mismatch between the Hl used for the sufficient statistic and the Hl used for calculating derivatives in g(μqpl, Hl, Nl). However, both the modeling error and the potential Hl mismatch may be alleviated by choosing ξ carefully. ξ is empirically set to a real number between 0 and 1.
The efficiency of the Bayesian technique used depends upon the quality of the prior density. In the context of SBC, the prior density should reflect the fluctuation of channel distortion Hl occurring when environment compensation is conducted for different filter banks. Accordingly, the following estimates are suitable for P(Hl):
P(Hl)=N(Hl;
ΣH
where, in one embodiment, IJAC was used to produce averaged estimates to obtain E(Hl).
Background noise is often estimated by averaging non-speech frames in the current utterance. However, since the estimates are not directly linked to trained acoustic models ΛX, the estimates may not be optimal. In addition, since averaging is prone to distortion by statistical outliers occurring at high noise levels, the estimates may not be reliable.
Following the objective function in Equation (5), a technique for achieving reliable noise estimates according to SBC will now be presented. The technique assumes that the beginning frames of the current utterance are background noise and therefore uses these frames to train a silence model. One embodiment of the technique for achieving reliable noise estimates will now be described. First, parameters of the silence model are trained and fixed in a clean acoustic model. Then, Nil at iteration i=0 is set to be the average noise vector from the beginning non-speech frames of the current utterance. Then, for each iteration i in the noise segments and for frame k=1 to T, the following steps are executed:
The step size α in Equation (28) controls the updating rate for noise estimation. In various alternative embodiments, the step size α changes depending upon the estimated noise level, the iteration number i or both.
Notice that the illustrated embodiment includes several approximations designed to increase computation speed. These are: (1) the variance of acoustic models is not used (as was the case with channel estimation); (2) the approximation of posterior probabilities is set at either zero or one for each frame k and (3) the estimation of posterior probability of frame k is made without consideration of feature vectors in other frames. Alternative embodiments may omit one or more of these approximations.
Maximizing the auxiliary function of Equation (7) with respect to the bias term B yields the following updating equation:
The bias estimation is the same as that in MLLR (see, e.g., Woodland, et al., supra) and therefore can also make use of a binary regression tree. The tree groups Gaussian components in the acoustic models Ax according to their phonetic classes, so that the set of biases to be estimated can be chosen according to:
One embodiment of the E-M algorithm for estimating the biases is carried out using the following process:
The above process is a reliable and dynamic way of estimating the biases. If a small amount of data is available, a global bias may be used for every HMM. However, as more adaptation data becomes available, the biases become more ascertainable and therefore may be different for each HMM or group of HMMs.
A forgetting factor ρ may be introduced to force parameter updating with more emphasis on recent utterances. Therefore, the sufficient statistics in Equations (17) and (18) may be weighted by a factor ρR−r.
The performance of E-M-type algorithms depends upon the sufficient statistic γsmr(k). A forward-backward algorithm (see, e.g., Rabiner,“A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition”, Prentice Hall P T R, 1993) may be used to obtain the sufficient statistic. State sequences may be obtained from Viterbi alignment during the decoding process. This is usually called “unsupervised estimation” and contrasts with “supervised estimation,” which uses ground-truth state sequence alignments.
The channel and noise distortion factors and cluster-dependent biases are advantageously estimated before recognition of an utterance. The following technique for estimating these factors may be used for the current utterance:
Turning now to
Bayesian Joint Additive/Convolutive Compensation Having described several embodiments of SBC, several embodiments of Bayesian joint additive/convolutive compensation, or B-IJAC, will now be described. By setting B(k) to 0 in Equation (1), the bias terms in the above described SBC may be ignored. Using the same notation, the noise estimate is obtained via Equations (25) to (28) with the bias term B and
Turning now to
a. Accumulate sufficient statistics via Equations (17) and (18), but with Bc(qp) set to zero.
b. Update channel distortion estimate for the next utterance by Equation (22).
Experimental Results
Having described several embodiments of SEC and B-IJAC, several experiments will now be set forth regarding SEC and B-IJAC.
SBC was compared to JAC (Gong, supra), non-Bayesian IJAC and maximum-likelihood bias removal (MLBR) on name recognition under a representative variety of hands-free conditions. ε was fixed at 0.9 for the experiments. A technique called “sequential variance adaptation,” or “SVA” (see, e.g., Cui, et al., “Improvements for Noise Robust and Multi-Language Recognition,” Tech. Rep., Speech Technologies Laboratories, Texas Instruments, 2003), was used together with these techniques to transform the variance of the acoustic models.
A database, called “WAVES,” was used in the experiments. WAVES was recorded in a vehicle using an AKG M2 hands-free distant talking microphone in three recording sessions: parked (engine off), city driving (car driven on a stop-and-go basis), and highway driving (car driven at relatively steady highway speeds). In each session, 20 speakers (ten male, ten female) read 40 sentences each, resulting in 1325 English name utterances.
The baseline acoustic model CD-HMM was a gender-dependent, generative tied-mixture HMM (GTM-HMM) (U.S. Patent Application Serial No. 11/196,601, supra), trained in two stages. The first stage trained the acoustic model from the Wall Street Journal (WSJ) with a manual dictionary. Decision-tree-based state tying was applied to train the acoustic model. As a result, the model had one Gaussian component per state and 9573 mean vectors. In the second stage, a mixture-tying mechanism was applied to tie mixture components from a pool of Gaussian densities. After the mixture tying, the acoustic model was re-trained using the WSJ database.
Table 1, below, shows recognition results by SBC, together with those by MLLR and IJAC. MLLR was implemented without rotation of mean vectors. Nevertheless, the MLLR implementation applied phonetic clustering. Interestingly, the widely used maximum-likelihood signal bias removal technique (see, e.g., Rahim, et al., supra) may be considered as a special case of the MLLR with only one cluster.
From Table 1, it may be observed that:
Next, interference was added to the speech by introducing different levels of background conversation, or “babble” noise, to the WAVES name database under the parked condition. The total number of utterances was 1450. Table 2, below, shows the results of different techniques in babble noise.
From Table 2, it may be observed that:
Next, SEC was implemented in an embedded speech recognition system. The acoustic model used was a single-mixture-per-state, intra-word triphone model trained from the WSJ database. As before, three driving conditions—highway driving, city driving and parked conditions—were used in the experiment. SBC's performance under the three different driving conditions, together with those achieved by other techniques are shown in Table 3, below.
Compared to JAC, SBC's average WER reduction was 39%.
SEC was implemented in fixed-point C for an embedded ASR system. In a live-mode recognition experiment, fixed-point SEC obtained the results given in Table 4, below.
Next, the performance of SEC was evaluated as a function of clusters. A threshold Dmin controls the number of clusters for cluster-dependent biases. Dmin and the number of clusters bear an inverse relationship; the larger the Dmin, the fewer the clusters.
Next, the forgetting factor p and threshold Dmin were dynamically adjusted. The threshold Dmin was set to be smaller with the increase of SNR, i.e.:
where η is the SNR of the current utterance. D1 and D0 are respectively the maximum and the minimum of the threshold Dmin. η1 and η0 each denote empirically set maximum and minimum SNRs. The forgetting factor ρ is similarly adjusted according to the SNR η.
where ρ1 and ρ0 each denote the maximum and the minimum of the forgetting factor ρ.
The parameters varied were D0, D1, ρ0 and ρ1. Table 4 shows WERs that result as these parameters were changed.
From Table 4, it may be observed that WERs by SBC did not vary much as D0, D1, ρ0 and ρ1 were changed. Nevertheless, the lowest WERs were achieved with same setup of ρ0=0.7 and D1=700. When ρ1=1.0, 2.59%, 0.22% and 0.22% WERs resulted under highway driving, city driving and parked conditions, respectively. When ρ1=0.9, 2.57%, 0.18% and 0.22% WERs resulted under highway driving, city driving and parked conditions, respectively.
Although the present invention has been described in detail, those skilled in the art should understand that they can make various changes, substitutions and alterations herein without departing from the spirit and scope of the invention in its broadest form.
The present invention is a continuation-in-part of, and claims priority based on, U.S. patent application Ser. No. 11/195,895 by Yao, entitled “System and Method for Noisy Automatic Speech Recognition Employing Joint Compensation of Additive and Convolutive Distortions,” filed Aug. 3, 2005, and is further related to U.S. patent application Ser. No. 11/196,601 by Yao, entitled “System and Method for Creating Generalized Tied-Mixture Hidden Markov Models for Automatic Speech Recognition,” filed Aug. 3, 2005, commonly assigned with the present invention and incorporated herein by reference.
Number | Date | Country | |
---|---|---|---|
Parent | 11195895 | Aug 2005 | US |
Child | 11278877 | Apr 2006 | US |