Method, medium, and system decoding and encoding a multi-channel signal

Information

  • Patent Grant
  • 9369164
  • Patent Number
    9,369,164
  • Date Filed
    Wednesday, January 10, 2007
    17 years ago
  • Date Issued
    Tuesday, June 14, 2016
    8 years ago
Abstract
A method, medium, and system decoding and/or encoding multiple channels. Accordingly, down-mixed multiple channels can be decoded/up-mixed to a left channel and a right channel during a first stage, thereby enabling a high quality sound output even in scalable channel decoding.
Description
BACKGROUND OF THE INVENTION

1. Field of the Invention


One or more embodiments of the present invention relate to audio coding, and more particularly, to a surround audio coding method, medium, and system for encoding and/or decoding an audio signal.


2. Description of the Related Art


Generally, multi-channel audio coding includes waveform multi-channel audio coding and parametric multi-channel audio coding. The waveform multi-channel audio coding typically further includes moving picture experts group (MPEG)-2 MC audio coding, AAC audio coding, and BSAC/AVS MC audio coding, for example. In the waveform multi-channel audio coding, 5 channel signals may be encoded by an encoder terminal and 5 encoded channel signals may be received and ultimately decoded by a decoding terminal. The parametric multi-channel audio coding typically includes MPEG surround coding, where a multi-channel signal may be encoded and 1 or 2 encoded channel signals may be received and ultimately decoded to output 6 or 8 multi-channels, for example.


Here, such MPEG surround coding uses a first 5-1-5 tree structure, illustrated in FIG. 1A, or a second 5-1-5 tree structure, illustrated in FIG. 1B, in order to decode such a signal to multi-channels up-mix. The tree structures receive a down-mixed mono signal, i.e., a signal that has been encoded from multi-channel signals and output as a mono signal, and up-mixes the mono signal to multi-channel signals of a Front Left (FL) channel, a Front Right (FR) channel, a Center (C) channel, a Low Frequency Enhancement (LFE) channel, a Back Left (BL) channel, and a Back Right (BR) channel, using combinations of 1-to-2 (OTT) modules. Here, the up-mixing of the mono signal through the stages of OTT modules can be accomplished with previously generated spatial information of Channel Level Differences (CLDs) and/or Inter-Channel Correlations (ICCs), with the CLD being information about an energy ratio or difference between predetermined channels in multi-channels, and with the ICC being information about correlation or coherence corresponding to a time/frequency tile of input signals. With respective CLDs and ICCs, each staged OTT can up-mix a single input signal to respective output signals through each staged OTT.


Referring to FIG. 1A, in the first 5-1-5 tree structure, the first module OTT0 outputs a signal including mixed information for FL, FR, C and LFE channels, and a signal including mixed information for BL and BR channels.


Similarly, referring to FIG. 1B, in the second 5-1-5 tree structure, the first module OTT0 outputs a signal including mixed information for FL, BL, RF and BR channels, and a signal including mixed information for C and LFE channels. However, when selective decoding (“pruning”), e.g., if only selective channels are desired, the inventors have found it difficult to selective decode the different channels in either of the first or second 5-1-5 tree structures without performing all stages of OTT module decoding. For example, though an output of the OTT0 modules of either first or second 5-1-5 tree structure could potentially be sampled, they are up-mixed signals representing information for a number of front channels and back channels mixed together and would not be properly representative of any one selective channel, such as if a stereo signal of left and right channels was desired.


Accordingly, embodiments of the present invention overcome the above drawbacks.


SUMMARY OF THE INVENTION

One or more embodiments of the present invention include a method, medium, and system encoding and/or decoding multiple channels such that a left channel and a right channel are capable of being selectively down-mixed at an earliest opportunity during a decoding up-mixing stage.


Additional aspects and/or advantages of the invention will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the invention.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a method of decoding an input encoded multi-channel signal, including generating signals to be input to decorrelators using a down-mixed signal, as the encoded multi-channel signal, based on spatial cues, decorrelating the generated signals, as decorrelated signals, and mixing the down-mixed signal and the decorrelated signals using the spatial cues, wherein spatial cues first used to up-mix the down-mixed signal include respective information to decode a left channel and a right channel.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a method of decoding an input encoded multi-channel signal, consisting of defined channels, including generating signals to be input to decorrelators using a down-mixed signal, as the encoded multi-channel signal, based on spatial cues, decorrelating the generated signals, as decorrelated signals, and mixing the down-mixed signal and the decorrelated signals using the spatial cues, wherein spatial cues first used to up-mix the down-mixed signal include respective information to decode at least one of the defined channels.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a method of decoding an input encoded multi-channel signal, including generating signals to be input to decorrelators using a down-mixed signal, as the encoded multi-channel signal, based on spatial cues, decorrelating the generated signals, as decorrelated signals, and mixing the down-mixed signal and the decorrelated signals using the spatial cues, wherein spatial cues first used to up-mix the down-mixed signal include respective channel level differences (CLDs) or inter channel correlations (ICCs) between a left channel and a right channel to decode the left channel and the right channel.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a method of decoding an input encoded multi-channel signal, including generating signals to be input to decorrelators using a down-mixed signal, as the encoded multi-channel signal, based on spatial cues, decorrelating the generated signals, as decorrelated signals, and mixing the down-mixed signal and the decorrelated signals using the spatial cues, wherein, when the spatial cues are selectively used with a single up-mixing module, a corresponding up-mix of the down-mixed signal generates a left channel and a right channel from the up-mixing of the down-mixed signal.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a method of decoding an input encoded multi-channel signal, including generating signals to be input to decorrelators using a down-mixed signal, as the encoded multi-channel signal, based on spatial cues, decorrelating the generated signals as decorrelated signals, and mixing the down-mixed signal and the decorrelated signals using the spatial cues, wherein the decorrelated signals are selectively based only on spatial cues for up-mixing the down-mixed signal to a left channel and a right channel.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include at least one medium including computer readable code to control at least one processing element to implement an embodiment of the present invention.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a system decoding an input encoded multi-channel signal, including a pre-matrix application unit to generate signals to be input to decorrelators using a down-mixed signal, as the encoded multi-channel signal, based on spatial cues, a decorrelation unit to decorrelate the generated signals through respective decorrelators, as decorrelated signals, and a post-matrix unit to mix the down-mixed signal and the decorrelated signals using the spatial cues, wherein spatial cues first used to up-mix the down-mixed signal include respective information to decode a left channel and a right channel.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a system of decoding an input encoded multi-channel signal, consisting of defined channels, including a pre-matrix application unit to generate signals to be input to decorrelators using a down-mixed signal, as the encoded multi-channel signal, based on spatial cues, a decorrelation unit to decorrelate the generated signals through respective decorrelators, as decorrelated signals, and a post-matrix unit to mix the down-mixed signal and the decorrelated signals using the spatial cues, wherein spatial cues first used to up-mix the down-mixed signal include respective information to decode at least one of the defined channels.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a system of decoding an input encoded multi-channel signal, including a pre-matrix application unit to generate signals to be input to decorrelators using a down-mixed signal, as the encoded multi-channel signal, based on spatial cues, a decorrelation unit to decorrelate the generated signals through respective decorrelators, as decorrelated signals, and a post-matrix unit to mix the down-mixed signal and the decorrelated signals using the spatial cues, wherein spatial cues first used to up-mix the down-mixed signal include respective channel level differences (CLDs) or inter channel correlations (ICCs) between a left channel and a right channel to decode the left channel and the right channel.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a system of decoding an input encoded multi-channel signal, including a pre-matrix application unit to generate signals to be input to decorrelators using a down-mixed signal, as the encoded multi-channel signal, based on spatial cues, a decorrelation unit to decorrelate the generated signals through respective decorrelators, as decorrelated signals, and a post-matrix unit to mix the down-mixed signal and the decorrelated signals using the spatial cues, wherein, when the spatial cues are selectively used with a single up-mixing module, a corresponding up-mix of the down-mixed signal generates a left channel and a right channel from the up-mixing of the down-mixed signal.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a system of decoding an input encoded multi-channel signal, including a pre-matrix application unit to generate signals to be input to decorrelators using a down-mixed signal, as the encoded multi-channel signal, based on spatial cues, a decorrelation unit to decorrelate the generated signals through respective decorrelators, and a post-matrix unit to mix the down-mixed signal and the decorrelated signals using the spatial cues, wherein the decorrelated signals are selectively based only on spatial cues for up-mixing the down-mixed signal to a left channel and a right channel.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a method of encoding multi-channels to a down-mixed signal, including down-mixing multi-channel signals to a left channel and a right channel lastly, as the down-mixed signal, extracting spatial cues of the down-mixed signal, including spatial cues generated during down-mixing of channels of the multi-channels for the channels in addition to the left channel and the right channel, and generating a bitstream including the down-mixed signal and the extracted spatial cues.


To achieve the above and/or other aspects and advantages, embodiments of the present invention include a system for encoding multi-channels to a down-mixed signal, including a down-mixing unit to down-mix multi-channel signals to a left channel and a right channel lastly, as the down-mixed signal, an information extracting unit to extract spatial cues of the down-mixed signal, including spatial cues generated during down-mixing of channels of the multi-channels for the channels in addition to the left channel and the right channel, and a bitstream generation unit to generate a bitstream including the down-mixed signal and the extracted spatial cues.





BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects and advantages of the invention will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:



FIGS. 1A and 1B illustrate conventional first and second respective 5-1-5 tree structures for MPEG surround audio;



FIG. 2 illustrates a method of decoding multiple channels, according to an embodiment of the present invention;



FIG. 3 illustrates a system for decoding multiple channels, according to an embodiment of the present invention;



FIG. 4 illustrating a 5-1-5 tree structure, according to an embodiment of the present invention;



FIG. 5 illustrates a mathematical relationship between an input signal and an output signal in a decoding of multiple channels, according to an embodiment of the present invention;



FIG. 6 illustrates a method of encoding multiple channels, according to an embodiment of the present invention; and



FIG. 7 illustrates a system for encoding multiple channels, according to an embodiment of the present invention.





DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. Embodiments are described below to explain the present invention by referring to the figures.



FIG. 2 illustrates a method of decoding multiple channels, according to an embodiment of the present invention. Below, initially a decoding operation will be discussed for a first level/stage OTT0 (1-to-2) module and a TTT0 (2-to-3) module, such as shown as in FIG. 4, to first up-mix an input down-mixed signal to left (L) and right (R) channel signals and then to L, R, and center (C) channel signals, respectively. With such a decoding operation, selective up-mixing/decoding of an input down-mixed signal is now possible.


Thus, initially, by parsing a surround bitstream, e.g., as transmitted by an encoding terminal, spatial cues and additional information may be extracted, in operation 200.


By using the extracted spatial cues, spatial cues may be selectively smoothed in order to prevent sudden changes of the spatial cues at a low bitrate, in operation 203.


In order to maintain compatibility with a conventional matrix surround techniques, a gain and a pre-vector may be calculated with respect to each additional channel and if an external down-mix is used in a corresponding decoding terminal, a variable for compensating for a gain in each channel may be extracted, thereby generating a matrix R1, in operation 206. The matrix R1 is a matrix used to generate a signal to be input to decorrelators, e.g., disposed in a decorrelation unit 340 shown in FIG. 3. When the matrix R1 is generated in operation 206, spatial cues are used. Here, the spatial cues being used include, for example, information on the difference or correlation between a left (L) channel and a right (R) channel, such as respective channel level difference (CLD) or inter channel correlation (ICC) between the L channel and the R channel. For example, the spatial cues may include a respective spatial cue to be used by the first OTT0 module 400, for example, as illustrated in FIG. 4, which decodes and up-mixes an input down-mixed signal, such as a down-mixed signal generated by an encoding terminal, directly into L and R channels. When only the L channel and the R channel are being decoded the spatial cues may only include the respective CLD or ICC between the L channel and the R channel.


In an embodiment, R1 may be generated differently in operation 206 depending on the mode of the TTT0 module illustrated in FIG. 4. For example, in the MPEG surround coding, the variable bsTttModeLow, having relations illustrated in Table 1 below, is used as a variable indicating the mode of the TTT0 module:










TABLE 1





bsTttModeLow
Meaning







0
prediction mode (2 CPC, ICC)



with decorrelation


1
prediction mode (2 CPC, ICC)



without decorrelation


2
energy-based mode (2 CLD)



with subtraction, matrix compatibility enabled


3
energy-based mode (2 CLD)



with subtraction, matrix compatibility disabled


4
energy-based mode (2 CLD)



without subtraction, matrix compatibility enabled


5
energy-based mode (2 CLD)



without subtraction, matrix compatibility disabled


6
reserved


7
reserved









Here, in such an embodiment, if bsTttModeLow(0) is less than 2, a matrix R1 according to the below Equation 1 may be generated.











Equation





1









R
1

l
,
m


=

[



1


0




1


0






γ

l
,
m




1
3



(



(


α

l
,
m


+
2

)



c

1
,

OTT
0



l
,
m



+


(


β

l
,
m


-
1

)



c

2
,

OTT
0



l
,
m




)




1






γ

l
,
m




1
3



(



(


α

l
,
m


-
1

)



c

1
,

OTT
0



l
,
m



+


(


β

l
,
m


+
2

)



c

2
,

OTT
0



l
,
m




)




1






γ

l
,
m




1
3



(



(

1
-

α

l
,
m



)



2



c

1
,

OTT
0



l
,
m



+


(

1
-

β

l
,
m



)



2



c

2
,

OTT
0



l
,
m




)





-

2





]


,





0

m
<


m
tttLowProc



(
0
)



,

0

l
<
L






where αl,m, βl,m and γl,m are given by:







γ

l
,
m


=

{




1


D
ICC



(

4
,
l
,
m

)







m
>


m
resProc



(
4
)



,


bsResidualPresent


(
4
)


=
1












bsResidualCoding
=
1





1


otherwise









αl,m=DCPC_1(0,l,m),0≦m<mtttLowProc(0),0≦l<L
βl,m=DCPC_2(0,l,m),0≦m<mtttLowProc(0),0≦l<L


If bsTttModeLow(0) is 3, a matrix R1 according the below Equation 2 may be generated.











R
1

l
,
m


=

[



1


0













OTT
0



1




0






1
3



(



w
11

l
,
m




c

1
,

OTT
0



l
,
m



+


w
12

l
,
m




c

2
,

OTT
0



l
,
m




)




1






1
3



(



w
21

l
,
m




c

1
,

OTT
0



l
,
m



+


w
22

l
,
m




c

2
,

OTT
0



l
,
m




)




1






1
3



(



w
31

l
,
m




2



c

1
,

OTT
0



l
,
m



+


w
32

l
,
m




2



c

2
,

OTT
0



l
,
m




)





-

2





]


,




Equation





2







0

m
<


m
tttLowProc



(
0
)



,

0

l
<
L











where











w
11

l
,
m


=





q
1

l
,
m




q
2

l
,
m



+


q
2

l
,
m




(

1
+

q
2

l
,
m



)






q
1

l
,
m




q
2

l
,
m



+


(

1
+

q
2

l
,
m



)

2
















w
12

l
,
m


=





q
2

l
,
m




(

1
+

q
2

l
,
m



)


2



(


q
1

l
,
m


+

q
2

l
,
m


+
1

)



(



q
1

l
,
m




q
2

l
,
m



+


(

1
+

q
2

l
,
m



)

2


)
















w
21

l
,
m


=

-




(

1
+

q
2

l
,
m



)

2



(



q
1

l
,
m




q
2

l
,
m



+

q
2

l
,
m


+
1

)



(



q
1

l
,
m




q
2

l
,
m



+


(

1
+

q
2

l
,
m



)

2


)

















w
22

l
,
m


=





q
1

l
,
m




q
2

l
,
m



+

q
2

l
,
m


+
1




q
1

l
,
m




q
2

l
,
m



+


(

1
+

q
2

l
,
m



)

2
















w
31

l
,
m


=


(



q
1

l
,
m




q
2

l
,
m



+


(

1
+

q
2

l
,
m



)

2


)



-
1

/
2














w
32

l
,
m


=



q
2

l
,
m




(



q
1

l
,
m




q
2

l
,
m



+


(

1
+

q
2

l
,
m



)

2


)




-
1

/
2














If bsTttModeLow(0) is 5, a matrix R1 according to the below Equation 3 may be generated.











R
1

l
,
m


=

[



1


0




1


0






1
3



w
11

l
,
m




c

1
,

OTT
0



l
,
m





1






1
3



w
22

l
,
m




c

2
,

OTT
0



l
,
m





1






1
3



(



w
31

l
,
m




2



c

1
,

OTT
0



l
,
m



+


w
32

l
,
m




2



c

2
,

OTT
0



l
,
m




)





-

2





]


,




Equation





3







0

m
<


m
tttLowProc



(
0
)



,

0

l
<
L











where











w
11

l
,
m


=






q
1

l
,
m




q
2

l
,
m





q
2

l
,
m


+
1
+


q
1

l
,
m




q
2

l
,
m












w
22

l
,
m



=




q
1

l
,
m









q
1

l
,
m


+

q
2

l
,
m


+
1
















w
31

l
,
m


=



1
2






q
2

l
,
m


+
1



q
2

l
,
m


+
1
+


q
1

l
,
m




q
2

l
,
m












w
32

l
,
m



=


1
2






q
1

l
,
m






+
1



q
1

l
,
m


+

q
2

l
,
m


+
1

















By performing interpolation with the matrix R1, e.g., as generated in operation 206, a matrix M1 may be generated, in operation 208.


A matrix R2 to be used in mixing decorrelated signals and a direct signal may further be generated using the below Equation 4, in operation 210. When the matrix R2 is generated in operation 210, spatial cues are used. Here, the spatial cues being used include respective information on the difference or correlation between an L channel and an R channel, for example, such as a respective CLD or ICC between the L channel and the R channel, and such spatial cues may solely be used if only decoding of the L channel and the R channel is to be performed. For example, the spatial cues may include a spatial cue used in the first OTT0 module 400, illustrated in FIG. 4, which decodes and up-mixes the aforementioned input down-mixed signal, to an L channel and an R channel.










R
2

l
,
m


=

[





κ

TTT
0


l
,
m



H






11

OTT
3


l
,
m




1
3




γ

l
,
m


(



(


α

l
,
m


+
2

)


H






11

OTT
0


l
,
m



+







κ

TTT
0


l
,
m



H






11

OTT
3


l
,
m




1
3




γ

l
,
m


(



(


α

l
,
m


+
2

)


H






12

OTT
0


l
,
m



+






H






12

OTT
3


l
,
m





0


0







(


β

l
,
m


-
1

)


H






21

OTT
0


l
,
m



)






(


β

l
,
m


-
1

)


H






22

OTT
0


l
,
m



)





















H






21

OTT
3


l
,
m




1
3




γ

l
,
m


(



(


α

l
,
m


+
2

)


H






11

OTT
0


l
,
m



+






H






21

OTT
3


l
,
m




1
3




γ

l
,
m


(



(


α

l
,
m


+
2

)


H






12

OTT
0


l
,
m



+






H






22

OTT
3


l
,
m





0


0







(


β

l
,
m


-
1

)


H






21

OTT
0


l
,
m



)






(


β

l
,
m


-
1

)


H






22

OTT
0


l
,
m



)






















κ

TTT
0


l
,
m



H






11

OTT
2


l
,
m




1
3




γ

l
,
m


(



(


α

l
,
m


-
1

)


H






11

OTT
0


l
,
m



+







κ

TTT
0


l
,
m



H






11

OTT
2


l
,
m




1
3




γ

l
,
m


(



(


α

l
,
m


-
1

)


H






12

OTT
0


l
,
m



+





0



H






12

OTT
2


l
,
m





0







(


β

l
,
m


+
2

)


H






21

OTT
0


l
,
m



)






(


β

l
,
m


+
2

)


H






22

OTT
0


l
,
m



)





















H






21

OTT
2


l
,
m




1
3




γ

l
,
m


(



(


α

l
,
m


-
1

)


H






11

OTT
0


l
,
m



+






H






21

OTT
2


l
,
m




1
3




γ

l
,
m


(



(


α

l
,
m


-
1

)


H






12

OTT
0


l
,
m



+





0



H






22

OTT
2


l
,
m





0







(


β

l
,
m


+
2

)


H






21

OTT
0


l
,
m



)






(


β

l
,
m


+
2

)


H






22

OTT
0


l
,
m



)






















κ

m
0


l
,
m




c

1
,

OTT





1



l
,
m





2

3




γ

l
,
m


(



(

1
-

α

l
,
m



)


H






11

OTT
0


l
,
m



+







κ

TTT
0


l
,
m




c

1
,

OTT





1



l
,
m





2

3




γ

l
,
m


(



(

1
-

α

l
,
m



)


H






12

TTT
0


l
,
m



+





0


0


0







(

1
-

β

l
,
m



)


H






21

OTT
0


l
,
m



)






(

1
-

β

l
,
m



)


H






22

OTT
0


l
,
m



)






















c

2
,

OTT





1



l
,
m





2

3




γ

l
,
m


(



(

1
-

α

l
,
m



)


H






11

OTT
0


l
,
m



+







c

2
,

OTT





1



l
,
m





2

3




γ

l
,
m


(



(

1
-

α

l
,
m



)


H






11

OTT
0


l
,
m



+





0


0


0







(

1
-

β

l
,
m



)


H






21

OTT
0


l
,
m



)






(

1
-

β

l
,
m



)


H






21

OTT
0


l
,
m



)



















]





Equation





4








where, the elements are defined using the definition of arbitrary matrix elements






H






11

OTT
x


l
,
m









H






22

OTT
x


l
,
m







as outlined in section, for which,

CLDXl,m=DCLD(X,l,m),0≦X≦3,0≦m<Mproc,0≦l≦L
ICCXl,m=DICC(X,l,m),0≦X≦3,0≦m<Mproc,0≦l≦L


By performing interpolation with the generated matrix R2, a matrix M2 may be generated, in operation 213.


A signal obtained by ACC-encoding and then, residual-coding of the difference between a signal down-mixed from multi-channels and an original signal at an encoding terminal may be decoded, in operation 216.


Thereafter a modified discrete cosine transform (MDCT) coefficient, e.g., decoded in operation 216, may be transformed to a quadrature mirror filter (QMF) domain, in operation 218.


Overlap-add between frames may then be performed with respect to the signal output in operation 218, in operation 220.


Since a low frequency band signal has insufficient frequency resolution because of a QMF filterbank, the frequency resolution may be increased through additional filtering, in operation 223.


Thereafter, in one embodiment, a configuration of available channels or speakers in the decoding terminal may be recognized, in operation 230. Here, the configuration of the channels or speakers indicates the number of speakers disposed or available at the decoding end, the positions of operable speakers among the speakers disposed at the decoding end, and information on channels that can be used in the multi-channels arranged at the decoding end among the channels encoded in the encoding terminal, for example, noting that additional embodiments are equally available for determining how selective to make the decoding of the input down-mixed signal.


By using the recognized configuration, for example, the number of up-mixing stages/levels can calculated in operation 233.


By using a QMF hybrid analysis filterbank, the input signal may be divided into frequency bands, in operation 236.


By using the matrix M1, a direct signal and signals to be input to decorrelators may further be generated, in operation 238. Here, a signal to be input to a decorrelator D0OTT, decorrelating an L channel and an R channel, corresponding to the first OTT0 module, a signal to be input to a decorrelator D0TTT decorrelating an L channel, an R channel and a center (C) channel, corresponding to the TTT0, module, a signal to be input to a decorrelator D3OTT decorrelating a front left (FL) channel and a back left (BL) channel, corresponding to an OTT3 module, and a signal to be input to a decorrelator D2OTT decorrelating a front right (FR) channel and a back right (BR) channel, corresponding to an OTT2 module, are generated. Also, in operation 238, the number of levels to be decoded is adjusted according to the number of levels calculated in operation 233, so that decoding can be performed through the aforementioned scalable up-mixing.


With the signals generated in operation 238 to be input to decorrelators, decorrelation is performed in decorrelators and the signals are rearranged such that the signals can provide a spatial effect.


Again, the decorrelator D0OTT decorrelates an L channel and an R channel, corresponding to the first OTT0 module, the decorrelator D0TTT decorrelates an L channel, an R channel, and a C channel, corresponding to the TTT0 module, the decorrelator D2OTT decorrelates an FR channel and a BR channel corresponding to the OTT2 module, and the decorrelator D3OTT decorrelates an FL channel and a BL channel corresponding to the OTT3 module.


The matrix M2 generated in operation 213 may thus be applied to the signals decorrelated in operation 240 and the corresponding direct signal generated in operation 238 individually, in operation 243.


Here, in operation 243, the number of levels to be decoded may be adjusted, for example, according to the number of levels calculated in operation 233, so that decoding can be performed through the aforementioned scalable up-mixing.


Temporal envelope shaping (TES) may also be applied to the signal to which the matrix M2 is applied in operation 243, in operation 246.


The signal to which the TES is applied in operation 246 may further be transformed to the time domain by using a QMF hybrid synthesis filterbank, in operation 248.


Temporal processing (TP) may then be applied to the signal transformed in operation 248, in operation 250.


Here, operations 243 and 250 can be performed in order to improve the sound quality of a signal in which a temporal structure is important as in an applause signal, though operations 243 and 250 can be selectively applied and are not necessarily required.


The direct signal and the decorrelated signals can then be mixed in operation 253.


Though this embodiment was explained through the illustrated example of a 5.1-channel signal, embodiments of the present invention are not limited thereto. The method of decoding multi-channels, according to embodiments of the present invention, can be equally applied to all multi-channels in which an input down-mixed signal is first up-mixed to an L channel and an R channel during the decoding of the down-mixed signal.


Accordingly, FIG. 3 illustrates a system for decoding multiple channels, according to an embodiment of the present invention.


A bitstream decoder 300 may parse a surround bitstream and extract spatial cues and additional information.


A smoothing unit 302 may smooth the spatial cues in order to prevent sudden changes of the spatial cues at a low bitrate.


In an embodiment, in order to maintain compatibility with a conventional matrix surround method, a matrix component calculating unit 304 may calculate a gain with respect to each additional channel.


A pre-vectors calculating unit 308 calculates a pre-vector.


If external down-mix is used in a decoder, an arbitrary down-mix gain extracting unit 308 may extract a variable for compensating for a gain in each channel.


Accordingly, by using the outputs from the matrix component calculating unit 304, the pre-vectors calculating unit 308, and the arbitrary down-mix gain extracting unit 308, the matrix generation unit 312 generates a matrix R1. When the matrix R1 is generated in the matrix generation unit 312, spatial cues are used. Here, the spatial cues being used include information on the difference or correlation between an L channel and an R channel, such as the aforementioned respective CLD or ICC between the L channel and the R channel. For example, the spatial cues may include a spatial cue used in the first OTT0 module 400, illustrated in FIG. 4, which decodes and up-mixes an input down-mixed signal to an L channel and an R channel.


Here, the matrix generation unit 312 generates R1 differently depending on the mode of the TTT0 module illustrated in FIG. 4. For example, in the MPEG surround coding, bsTttModeLow having relations illustrated the above Table 1 can be used as a variable indicating the mode of the TTT0 module.


Here, similar to above, if bsTttModeLow(0) is less than 2, the aforementioned matrix R1 in the above Equation 1 may be generated.


If bsTttModeLow(0) is 3, the aforementioned matrix R1 in the above Equation 2 may be generated.


If bsTttModeLow(0) is 5, the aforementioned matrix R1 in the above Equation 3 may be generated.


Thus, by performing interpolation with the matrix R1 generated in the matrix generation unit 312, an interpolation unit 314 may generate a matrix M1.


A mix-vectors calculating unit 310 may then generate a matrix R2 for mixing the signals decorrelated in the decorrelation unit 340 and the direct signal. When the matrix R2 is generated in the mix-vectors calculating unit, spatial cues are used. Here, the spatial cues being used include the information on the difference or correlation between an L channel and an R channel, such as the aforementioned CLD or ICC between the L channel and the R channel. For example, the spatial cues may include a spatial cue used in the OTT0 module 400, illustrated in FIG. 4, which decodes and up-mixes the input down-mixed signal to an L channel and an R channel.


The mix-vectors calculating unit 310 may, thus, generate a matrix R2 according to the above Equation 4.


Thus, by performing interpolation with the matrix R2 generated in the mix-vectors calculating unit 310, the interpolation unit 316 may generate the matrix M2.


An AAC decoder 320 may then decode a signal generated by ACC-encoding and then, residual-coding the difference between the input down-mixed signal and the original signal at the encoding terminal.


An MDCT transform unit (MDCT2QMF unit) 322 may then transform the MDCT coefficient output from the AAC decoder 320 to the QMF domain, and up-mixes the QMF domain signal, substituting for the decorrelation unit 340.


An overlap-add unit 324 performs overlap-add between frames with respect to the signal output from the MDCT transform unit 322.


Since a low frequency band signal has insufficient frequency resolution because of a QMF filterbank, a hybrid analysis unit 326 may be used to increase the frequency resolution through additional filtering.


Depending on the embodiment, a decoding level calculating unit 327 may then be used to recognize a configuration of channels or speakers, for example, in the decoding terminal and calculate the number of stages/levels of up-mixing/decoding.


Here, as only an example, such a configuration of multi-channels at the decoding end indicates the number of speakers disposed at the decoding end, the positions of operable speakers among the speakers disposed at the decoding end, and information on channels that can be used in the multi-channels arranged at the decoding end among the channels encoded in the encoding end.


A decoding level control unit 329 may then output a control signal so that decoding can be performed according to the number of levels calculated in the decoding level calculating unit 327.


A hybrid analysis unit 330 may be a QMF hybrid analysis filterbank and divide an input signal into frequency bands.


A pre-matrix application unit 335 may further generate a direct signal and signals to be input to decorrelators, by using the matrix M1.


Here, in this embodiment, the pre-matrix application unit 335 generates a signal to be input to a decorrelator D0OTT 342 decorrelating an L channel and an R channel, corresponding to the first OTT0 module, a signal to be input to a decorrelator D0TTT 344 decorrelating an L channel, an R channel and a C channel, corresponding to the TTT0 module, a signal to be input to a decorrelator D2OTT 346 decorrelating an FR channel and a BR channel, corresponding to the OTT2 module, and a signal to be input to a decorrelator D3OTT 348 decorrelating an FL channel and a BL channel, corresponding to the OTT3 module. Also, the pre-matrix application unit 335 may adjust the number of levels to be decoded in response to the control signal output from the decoding level control unit 329, for example, so that decoding can be performed through the aforementioned scalable up-mixing.


The decorrelation unit 340 may perform decorrelation with the signals generated in the pre-matrix application unit 335, thereby rearranging the signals such that the signals can provide a spatial effect. The decorrelator D0OTT 342 decorrelates an L channel and an R channel corresponding to the OTT0 module, the decorrelator D0TTT 344 decorrelates an L channel, an R channel, and a C channel, corresponding to the TTT0 module, the decorrelator D2OTT 346 decorrelates an FR channel and a BR channel corresponding to the OTT2 module, and the decorrelator D3OTT 348 decorrelates an FL channel and a BL channel, corresponding to the OTT3 module.


A mix-matrix application unit 350 may further apply the matrix M2 individually to the signals output from the decorrelation unit 340 and the direct signal output from the pre-matrix application unit 335.


Here, depending on the embodiment, the mix-matrix application unit 350 may adjust the number of levels to be decoded, e.g., in response to the control signal output from the decoding level control unit 329, so that decoding can be performed through the aforementioned scalable up-mixing.


A TES application unit 335 may apply TES to the signal output from the mix-matrix application unit 350.


A QMF hybrid synthesis unit 360 is a QMF hybrid synthesis filterbank and may transform the signal to the time domain.


A TP application unit 365 applies TP to the signal output from the QMF hybrid synthesis unit 360.


Here, the TES application unit 335 and the TP application unit 365 are to improve the sound quality of a signal in which a temporal structure is important, such as in an applause signal. The TES application unit 335 and the TP application unit 365 may be selectively used and are not necessarily required.


A mixing unit 370 mixes the direct signal and the decorrelated signals.


Again, though this embodiment is implemented with a 5.1-channel signal, as illustrated in FIG. 3, embodiments are not limited thereto. Embodiments of to the present invention can be applied to all multi-channels in which an input down-mixed signal, e.g., as down-mixed in an encoding terminal, is initially up-mixed to an L channel and an R channel in a first stage/level of decoding.



FIG. 4 illustrates a 5-1-5 tree structure, according to an embodiment of the present invention, for decoding up-mixing an input down-mixed signal into multiple channels. By using the 1-to-2 (OTT) modules and a 2-to-3 (TTT) module, FIG. 4 conceptually illustrates an order in which a signal is decoded and up-mixed, according to an embodiment of the present invention.


The OTT0 module 400 receives an input of a down-mixed mono signal, e.g., as down-mixed in an encoder terminal, and decodes and up-mixes the signal to an L signal and an R signal. Here, this first up-mixing of the L and R channel signals is substantially different from the aforementioned 5-1-5 tree structures of FIGS. 1A and 1B, where the first stage OTT module output cannot be used to properly reproduce the L and R channel signals, but rather additional stages are necessary to further up-mix the output of such a first stage OTT module.


Thus, after the OTT0 module, the TTT0 module 410 receives inputs of the L signal and the R signal, output from the OTT0 module 400, and decodes and up-mixes the signal to an L signal, an R signal, and a C signal.


Thereafter, a OTT1 module 420 may receive an input of the C signal, output from the TTT0 module 410, and decode and up-mix the C signal to a C signal and an LFE signal.


A OTT3 module 430 may receive an input of the L signal, output from the TTT0 module 410, and decode and up-mix the L signal to an FL signal and a BL signal.


A OTT2 module 440 may receive an input of the R signal output from the TTT0 module 410, and decode and up-mix the R signal to an FR signal and a BR signal.



FIG. 5 illustrating a mathematical relationship between an input signal and an output signal, according to an embodiment of the present invention.


Using a CLD and ICC, a pre-decorrelator matrix M1 receives an input of a down-mixed mono signal (Xm), e.g., down-mixed by an encoder terminal, and output signals to be input to decorrelators D0OTT, D0TTT, D2OTT, and D3OTT.


The decorrelators D0OTT, D0TTT, D2OTT, and D3OTT decorrelate signals calculated in the matrix M1.


Using a CLD and ICC, a mix-matrix M2 mixes the direct signal (m) and the decorrelated signals, d0, d1, d2, and d3, and up-mixes the signal. Here, the mix-matrix M2 receives inputs of the direct signal (m) and the decorrelated d0, d1, d2, and d3, and outputs an FL signal, a BL signal, an FR signal, a BR signal, a C signal, and an LFE signal.



FIG. 6 illustrates a method of encoding multiple channels, according to an embodiment of the present invention.


First, the multiple channels are down-mixed, in operation 600. For example, the multiple channels may be include an FL channel, a surround L channel, an FR channel, a surround R channel, a C channel and a woofer channel.


In the down-mixing, in operation 600, the L channel and the R channel are down-mixed lastly. For example, in a 5.1 channel system, the FL, surround L, FR, surround R, C and woofer channels may initially be down-mixed to L, R, and C channels previous levels, and the down-mixed 3 channels may then be down-mixed to an L channel and an R channel at a last stage/level.


During the down-mixing of the multiple channels, spatial cues of the corresponding down-mixing of the multiple channels may be respectively extracted, in operation 610. For example, spatial cues extracted for a 5.1 channel system may include information to be used in respectively up-mixing each of the L and R channels, then each of the L, R, and C channels, and then each of the FL and BL channels, each of the FR and BR channels, and each of the C and woofer channels.


A bitstream including the down-mixed signal and the spatial cues may then be generated, in operation 620.



FIG. 7 illustrates a system for encoding multiple channels, according to an embodiment of the present invention. The system may include a down-mixing unit 700, an information extracting unit 710, and a bitstream generation unit 720.


The down-mixing unit 700 may, thus, down-mix the multiple channels corresponding to input terminals IN 0 through IN M. For example, in a 5.1 channel system, the multiple channels may include FL, surround L, FR, surround R, C and woofer channels.


Here, the down-mixing unit 700 down-mixes the L channel and the R channel lastly. For example, in such a 5.1 channel system, the FL, surround L, FR, surround R, C and woofer channels are down-mixed to L, R, and C channels, and then the down-mixed 3 channels are further down-mixed to the L channel and the R channel.


The information extracting unit 710 may extract respective spatial cues during the staged down-mixing in the down-mixing unit 700. For example, the spatial cues extracted from such a 5.1 channel system may include information to be used in respective staged up-mixing with respect to each of L and R channels, L, R, and C channels, FL and BL channels, FR and BR channels, and C and woofer channels.


The bitstream generation unit 720 may thereafter generate a bitstream including the signal down-mixed in the down-mixing unit 700 and the spatial cues extracted in the information extracting unit 710, and output the bitstream through an output terminal OUT.


According to an embodiment of the present invention, an L channel and an R channel may be down-mixed and encoded lastly, and the same L channel and the R channel may be decoded and up-mixed with the first stage of up-mixing the down-mixed input signal.


In this way, even in scalable channel decoding, appropriate L and R channels can be selectively initially output so that sound quality is not deteriorated and high quality sound can be output, even in a low scaled level of decoding, e.g., even when only a single up-mixing OTT stage is implemented.


In addition, power consumption can also be reduced for easy implementation in mobile applications requiring high quality for stereo sound.


In addition to the above described embodiments, embodiments of the present invention can also be implemented through computer readable code/instructions in/on a medium, e.g., a computer readable medium, to control at least one processing element to implement any above described embodiment. The medium can correspond to any medium/media permitting the storing and/or transmission of the computer readable code.


The computer readable code can be recorded/transferred on a medium in a variety of ways, with examples of the medium including magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.), optical recording media (e.g., CD-ROMs, or DVDs), and storage/transmission media such as carrier waves, as well as through the Internet, for example. Here, the medium may further be a signal, such as a resultant signal or bitstream, according to embodiments of the present invention. The media may also be a distributed network, so that the computer readable code is stored/transferred and executed in a distributed fashion. Still further, as only an example, the processing element could include a processor or a computer processor, and processing elements may be distributed and/or included in a single device.


Although a few embodiments of the present invention have been shown and described, it would be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the claims and their equivalents.

Claims
  • 1. A method of generating a stereo signal from a mono down-mixed signal by using at least one processing device, the method comprising: calculating a first spatial parameter for up-mixing the mono down-mixed signal to the stereo signal, based on a second spatial parameter for up-mixing the mono down-mixed signal to a multi-channel signal other than the stereo signal;generating a first matrix for generating a direct signal and an input signal of a decorrelator;generating a second matrix, by using the calculated first spatial parameter;generating the direct signal and the input signal of the decorrelator, by using the first matrix and the mono down-mixed signal;decorrelating the input signal of the decorrelator; andmixing the direct signal and the decorrelated signal, by using the second matrix, to generate the stereo signal.
  • 2. The method of claim 1, wherein the stereo signal is determined based on a decoder channel configuration.
  • 3. The method of claim 1, wherein the second matrix is obtained by using a calculated channel level difference (CLD) and a calculated inter-channel correlation (ICC).
  • 4. A method of generating a stereo signal from a mono down-mixed signal by using at least one processing device, the method comprising: calculating a first spatial parameter for up-mixing the mono down-mixed signal to the stereo signal, based on a second spatial parameter for up-mixing the mono down-mixed signal to a multi-channel signal other than the stereo signal;generating, using a first operation set and the mono down-mixed signal, a signal to be decorrelated and at least one direct signal to be input to a second operation set, at least one of the first and second operation sets being determined by the calculated first spatial parameter;decorrelating the signal to be decorrelated; andmixing the direct signal and the decorrelated signal, by using the second operation set, to generate the stereo signal.
  • 5. The method of claim 4, wherein the stereo signal is determined based on a decoder channel configuration.
  • 6. The method of claim 4, wherein the second operation set is obtained by using a calculated channel level difference (CLD) and a calculated inter-channel correlation (ICC).
Priority Claims (1)
Number Date Country Kind
10-2006-0049034 May 2006 KR national
CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Patent Application No. 60/757,857, filed on Jan. 11, 2006, U.S. Provisional Patent Application No. 60/758,985, filed on Jan. 17, 2006, U.S. Provisional Patent Application No. 60/759,543, filed on Jan. 18, 2006, U.S. Provisional Patent Application No. 60/782,616, filed on Mar. 16, 2006, in the U.S. Patent and Trademark Office, and Korean Patent Application No. 10-2006-0049034, filed on May 30, 2006, in the Korean Intellectual Property Office, the disclosures of which are incorporated herein in their entirety by reference.

US Referenced Citations (41)
Number Name Date Kind
5524054 Spille Jun 1996 A
5850456 Ten Kate et al. Dec 1998 A
7356152 Vernon et al. Apr 2008 B2
7391870 Herre et al. Jun 2008 B2
7394903 Herre et al. Jul 2008 B2
7447317 Herre et al. Nov 2008 B2
7447629 Breebaart Nov 2008 B2
7702407 Oh et al. Apr 2010 B2
7809580 Hotho et al. Oct 2010 B2
7860720 Thumpudi et al. Dec 2010 B2
7885819 Koishida et al. Feb 2011 B2
7970144 Avendano et al. Jun 2011 B1
7987097 Pang et al. Jul 2011 B2
20020006081 Fujishita Jan 2002 A1
20020154900 Shimada Oct 2002 A1
20030219130 Baumgarte et al. Nov 2003 A1
20030236583 Baumgarte et al. Dec 2003 A1
20040117193 Kawai Jun 2004 A1
20050157883 Herre et al. Jul 2005 A1
20050195981 Faller et al. Sep 2005 A1
20050226426 Oomen et al. Oct 2005 A1
20050276420 Davis Dec 2005 A1
20050281408 Kim et al. Dec 2005 A1
20060165184 Purnhagen et al. Jul 2006 A1
20060206323 Breebaart Sep 2006 A1
20060239473 Kjorling et al. Oct 2006 A1
20070019813 Hilpert et al. Jan 2007 A1
20070043563 Comerford et al. Feb 2007 A1
20070078550 Pang et al. Apr 2007 A1
20070094036 Pang et al. Apr 2007 A1
20070189426 Kim et al. Aug 2007 A1
20070203697 Pang et al. Aug 2007 A1
20070233293 Villemoes et al. Oct 2007 A1
20070258607 Purnhagen et al. Nov 2007 A1
20070291951 Faller Dec 2007 A1
20080091436 Breebaart et al. Apr 2008 A1
20080161035 Tomioka et al. Jul 2008 A1
20080255856 Schuijers et al. Oct 2008 A1
20080263285 Sharma et al. Oct 2008 A1
20090232317 Emerit et al. Sep 2009 A1
20100087097 Hogue et al. Apr 2010 A1
Foreign Referenced Citations (28)
Number Date Country
1669359 Sep 2005 CN
101647158 Feb 2010 CN
07-234863 Sep 1995 JP
11-225390 Aug 1999 JP
2001-352599 Dec 2001 JP
2002-149483 May 2002 JP
2004-194100 Jul 2004 JP
2004-312484 Nov 2004 JP
2044-312484 Nov 2004 JP
96-13080 Apr 1996 KR
1996-0039668 Nov 1996 KR
1997-7007528 Dec 1997 KR
5781882 Jul 1998 KR
2001-0086976 Sep 2001 KR
10-2002-0018730 Mar 2002 KR
10-2002-0082117 Oct 2002 KR
10-2005-0038096 Apr 2005 KR
10-2006-0047444 May 2006 KR
10-2006-0049941 May 2006 KR
10-2006-0109299 Oct 2006 KR
10-2007-005469 Jan 2007 KR
10-2007-0035411 Mar 2007 KR
10-2007-0078398 Jul 2007 KR
10-2007-0080850 Aug 2007 KR
10-0763919 Sep 2007 KR
03028407 Apr 2003 WO
2004008805 Jan 2004 WO
WO 2006008697 Jan 2006 WO
Non-Patent Literature Citations (37)
Entry
Title: MPEG Spatial Audio Coding/MPEG Surround: Overview and Current Status Author:J Breebaart et al Audio Engineering Society Convention Paper 119th convention Oct. 7-10, 2005.
Title: MPEG Spatial Audio Coding/MPEG sorround: Overview and Current Status Author: J Breebaart et al Journal: Audio Enfineering Society Convnetion Paper 119th convention Oct. 7-10, 2005.
Breebaart, J. et al. “MPEG Spatial Audio Coding / MPEG Surround: Overview and Current Status” In: Proc. 119th AES Convention. New York. Oct. 2005.
Herre, J. et al. “The Reference Model Architecture for MPEG Spatial Audio Coding” In: Proc. 118th AES Convention. Barcelona. May 2005.
ISO/IEC JTC 1/SC 29/WG 11 N7530 “Information technology—Coding of audio-visual objects—Part 3: Audio, Amendment 4: MPEG Surround” Oct. 2005.
PCT International Search Report issued Apr. 12, 2007 in corresponding International Application No. PCT/KR2007/000203.
Junghoe Kim et al., “Comments on signaling in MPEG Surround”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11, and ITU-T SG16 Q6), XX, XX No. M12886, Jan. 12, 2006, XP0300415555.
Extended European Search Report issued Jan. 7, 2010 corresponds to European Patent Application 07708488.7-2225.
Korean Non-Final Rejection mailed Jul. 18, 2011 corresponds to Korean Patent Application No. 10-2011-0056345.
Korean Notice of Allowance mailed Jul. 26, 2011 corresponds to Korean Patent Application No. 10-2007-0067134.
J. Herre et al., The Reference Model Architecture for MPEG Spatial Audio Coding, Audio Engineering Society Convention Paper 6447, USA, Audio Engineering Society, May 28, 2005.
Japanese Office Action issued Jun. 7, 2011 corresponds to Japanese Patent Application No. 2008-550238.
Extended European Search Report for European Application No. 12002670.3, mailed Sep. 10, 2012.
Korean Office Action for related Korean Patent Application No. 10-2006-0049304, mailed on Sep. 28, 2012.
Korean Office Action for related Korean Patent Application No. 10-2012-0083520, mailed on Sep. 28, 2012.
Korean Office Action for related Korean Patent Application No. X10-2011-0056345, mailed on Aug. 14, 2012.
Japanese Final Rejection mailed Jul. 17, 2012 in the corresponding Japanese Patent Application No. 2008-550238.
Korean Office Action for related Korean Patent Application No. 10-2012-0064601, mailed on Jun. 27, 2012.
Korean Office Action for related Korean Patent Application No. 10-2006-0049034, mailed on Apr. 30, 2012.
Korean Notice of Allowance issued Aug. 29, 2007 in corresponding Korean Patent Application 10-2006-0075301.
Japanese Notice of Preliminary Reexamination dated Feb. 19, 2013 in corresponding Japanese Patent Application 2008-550238.
Korean Notice of Last Non-Final Rejection issued Feb. 27, 2013 in corresponding Korean Patent Application 10-2012-0064601.
Extended European Search Report issued Dec. 3, 2012 in European Patent Application No. 12164460.3.
Korean Office Action issued Dec. 18, 2012 in Korean Patent Application No. 10-2012-0108275.
Communication dated Jan. 20, 2015 issued by The State Intellectual Property Office of People's Republic of China in counterpart Chinese Patent Application No. 201210459124.7.
US Office Action issued Jan. 7, 2014 in U.S. Appl. No. 12/071,286.
US Office Action issued Sep. 13, 2013 in U.S. Appl. No. 12/071,286.
US Office Action issued Jul. 17, 2012 in U.S. Appl. No. 12/071,286.
US Office Action issued Apr. 12, 2012 in U.S. Appl. No. 12/071,286.
US Office Action issued Jul. 26, 2011 in U.S. Appl. No. 12/071,286.
U.S. Appl. No. 12/071,286, filed Feb. 19, 2008, Chang-yong Son, et al. Samsung Electronics Co., Ltd.
Communication from the State Intellectual Property Office of P.R. China dated Apr. 23, 2015 in a counterpart Chinese application No. 201210458826.3.
Japanese Office Action issued Jun. 11, 2013 in corresponding Japanese Patent Application 2012-253715.
Korean Office Action dated Jul. 30, 2013 in Korean Patent Application No. 10-2012-0108275.
Korean Office Action dated Jul. 30, 2013 in Korean Patent Application No. 10-2012-0064601.
Korean Office Action issued Oct. 24, 2013 in Korean Patent Application No. 10-2012-0108275.
Korean Office Action issued Oct. 24, 2013 in Korean Patent Application No. 10-2012-0064601.
Related Publications (1)
Number Date Country
20070189426 A1 Aug 2007 US
Provisional Applications (4)
Number Date Country
60757857 Jan 2006 US
60758985 Jan 2006 US
60759543 Jan 2006 US
60782616 Mar 2006 US