Patent Grant
10284989
The present invention relates to a method and a device for reproducing a stereophonic sound and. more particularly, to a method and a device for reproducing a stereophonic sound by using a single speaker and at least one external speaker.
A stereophonic sound is a signal having an excellent directional property in terms of realism and immersion. A basic idea of providing a stereophonic sound is to add an IID (Interaural Intensity Difference), ITD (Interaural Time Delay), and pinna effect to a sound signal to be transferred through signal processing, which is used for a person to identify the directionality of a signal in a 3-dimensional space.
The methods of providing a stereophonic sound may be divided into a method of using a headphone and a method of using a speaker.
The method of using a headphone cannot be applied to a single speaker, and if a listener is moving or the distance between the listener and the speaker changes there are limitations in the method of using a speaker to play a stereophonic sound.
The present invention has been designed to solve the above problems and has an object to provide a method and a device for playing a stereophonic sound.
In order to solve the above problems, a method for playing a stereophonic (3D) sound in a portable device having a first speaker and a microphone according to the present invention comprises receiving an audio signal including one of left direction information and right direction information as a first signal and an audio signal including the other one of the left direction information and the right direction information as a second signal from a media playing device through wired or wireless communication, receiving a third signal output by a second speaker of the media playing device through the microphone, determining a compensation parameter for a fourth signal to be output through the first speaker on the basis of the first signal and the third signal, and generating the fourth signal from the second signal by using the determined compensation parameter.
Further, a method for playing a stereophonic (3D) sound in a media playing device according to the present invention comprises transmitting an audio signal including one of right directionality information and left directionality information as a first signal and an audio signal including the other one of the right directionality information and the left directionality information as a second signal to a portable device through wired or wireless communication and outputting the first signal to an external speaker after processing the signal on the basis of information received from the portable device.
Further, a portable stereophonic (3D) sound playing device having a first speaker according to the present invention comprises a wireless communication unit configured to receive an audio signal including one of right directionality information and left directionality information as a first signal and an audio signal including the other one of the right directionality information and the left directionality information as a second signal from a media playing device through wired or wireless communication, a microphone configured to receive a third signal from a second speaker of the media playing device, and a signal processing unit configured to generate a fourth signal from the second signal by determining a compensation parameter of the fourth signal to be output to the first speaker on the basis of the first signal and the third signal.
Further, a media playing device for playing a stereophonic sound according to the present invention comprises a wireless communication unit configured to transmit an audio signal including one of right directionality information and left directionality information as a first signal and an audio signal including the other one of the right directionality information and the left directionality information as a second signal to a portable device through wired or wireless communication and an audio processing unit configured to output the first signal after processing the signal on the basis of information received from the portable device.
According to the present invention, a stereophonic sound can be played by using a single speaker and one or more external speakers.
Further, according to the present invention, a high quality stereophonic sound can be played by using a feedback of a wireless communication unit even though a location of a listener wearing a speaker changes.
Further, according to the present invention, a stereophonic sound can be played with a bone conduction speaker by eliminating an interference signal.
Hereinafter, embodiments of the present invention are described in detail with reference to the accompanying drawings. The same reference symbols are used throughout the drawings to refer to the same or like parts. Detailed descriptions of well-known functions and structures incorporated herein may be omitted to avoid obscuring the subject matter of the present invention.
For the same reason, some components in the accompanying drawings are emphasized, omitted, or schematically illustrated, and the size of each component does not fully reflect the actual size. Therefore, the present invention is not limited to the relative sizes and distances illustrated in the accompanying drawings.
The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of various embodiments of the present invention as defined by the claims and their equivalents. It includes various specific details to assist in that understanding, but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present invention. In addition, descriptions of well-known functions and constructions may be omitted for clarity and conciseness.
In order to provide a stereophonic sound, a HRTF (Head Related Transfer Function) database modeled by reflecting factors used for identifying directionality such as an IID (Interaural Intensity Difference) between both ears, ITD (Interaural Time Delay) between both ears, and pinna effect can be used. The HRTF database is configured with a plurality of HRTFs measured for the horizontal and vertical directions on the basis of a location of a listener's head. The HRTF measured as a spatial transfer function may utilize a database disclosed by major research centers.
In more detail, the principal is to output a sound to a listener's right and left ear by applying the HRTF for a channel between a sound source and the listener's ears to a sound signal through signal processing. In this case, the methods of playing a stereophonic sound may be divided into a method of using a headphone and a method of using a speaker.
First, a method for playing a stereophonic sound by using a headphone will be briefly described.
In case of using a headphone, 5 channels and HRTF databases (left: 5 and right: 5) corresponding to a transfer function between both ears of a listener are required to transmit a stereophonic sound by using a 5.1 channel input signal. If C, L, R, Ls, and Rs are defined as channel signals respectively of center, left, right, left surround, and right surround, hrCl is defined as a HRTF from a center channel to the left ear, hrCr is defined as a HRTF from the center channel to the right ear, hrLl is defined as a HRTF from a left channel to the left ear, hrLr is defined as a HRTF from the left channel to the right ear, hrRl is defined as a HRTF from the right channel to the left ear, hrRr is defined as a HRTF from the right channel to the right ear, hrLSl is defined as a HRTF from a left surround channel to the left ear, hrLSr is defined as a HRTF from the left surround channel to the right ear, hrRSr is defined as a HRTF from a right surround channel to the right ear, and hrRSl is defined as a HRTF from the right surround channel to the left ear. A left signal sL(t) and a right signal sR(t) of the headphone for a stereophonic sound can be expressed in Formula 1.
sL(t)=C*hrCl+L*hrLl+R*hrRl+LS*hrLSl+RS*hrRSl
sR(t)=C*hrCr+L*hrLr+R*hrRr+LS*hrLSr+RS*hrRSr
The stereophonic sound can be played by transmitting the left signal sL(t) to the left side of the headphone (i.e., listener's left ear) and the right signal sR(t) to the right side of the headphone (i.e., listener's right ear).
Next, a method for playing a stereophonic sound by using left and right speakers will be described.
A signal yR(t) transmitted from a speaker to the right ear and a signal yL(t) transmitted from the other speaker to the left ear can be expressed as shown by Formula 2.
yR(t)=hRRsR(t)+hLRsL(t)
yL(t)=hRLsR(t)+hLLsL(t)
Here, hRR and hLR indicate channel responses from right and left speakers to the right ear and hLL and hRL indicate channel responses from the left and right speaker to the left ear. In case of using a speaker, signals other than desired signals must be removed because signals of both speakers are recognized by the right and left ears through a spatial transfer path (crosstalk cancellation). For playing effectively a stereophonic sound, sections including a crosstalk must be removed, and accordingly additional signal processing is required for the output signals yR(t) and yL(t). Here, if signal processing parameters suitable for removing sections including hLR and hRL are defined as p00, p01, p10, and p11, the output signal can be expressed in the formula y=HPs. Vectors and matrixes being used for this formula are defined as follows.
Because conditions for removing a crosstalk are yR(t)=sR(t) and yL(t)=sL(t), the condition for removing a crosstalk becomes P=H−1.
A method and a device for playing a stereophonic sound according to the present invention may include a media playing device 100 having at least one external speaker and a portable device 200 having a speaker and at least one microphone.
As shown in
The media playing device 100 according to the present invention may include all devices that have a communication function and enable the playing of video data including an audio signal. For example, a TV, desktop computer, mobile phone, tablet PC, and smartphone may be applied to the media playing device; however, the present invention is not limited to these examples.
The wireless communication unit 110 transmits and receives data through wireless communication. The wireless communication unit 110 performs a series of operations for transmitting and receiving a control signal and an audio signal through a wireless interface. The wireless communication unit 110 may use communication methods such as Bluetooth, infrared, and Zigbee; however, the wireless communication is not limited to one of these methods. Further, the wireless communication unit 110 can output data received through the wireless communication to the control unit 160 and transmit data output by the control unit 160 through the wireless communication.
The audio processing unit 120 may include an audio codec for performing signal processing of an audio signal. Further, the audio processing unit 120 may include an external speaker 122 for outputting an analog audio signal and a delay compensating unit 121 for processing an audio signal to be output to the external speaker 122. The processing of an audio signal to be output to the external speaker 122 may include an operation of delaying an output of a signal. Further, the delay compensating unit 121 can receive values required for signal processing through the wireless communication unit 110.
The input unit 130 receives a user operation for controlling the media playing device 100 according to the present invention and generates an input signal to transmit to the control unit 160. The input unit 130 may be located in the media playing device 100 or can transmit an input signal remotely through wired or wireless communication. Further, the input unit 130 can be omitted if the present invention is applied to a media playing device having a touch screen and all the operations can be performed with the touch screen.
The display unit 140 may be configured with an LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diodes), or AMOLED (Active Matrix Organic Light Emitting Diodes). The display unit 140 provides visually menus of the media playing device 100, input data, and various media play information for a user.
The display unit may include a touch screen. The touch screen may include a touch sensor. The touch sensor detects a user's touch input. The touch sensor may be configured with a touch detection sensor such as a capacitive overlay, resistive overlay, and infrared beam, or may be configured with a pressure sensor. The touch sensor detects a user's touch input and generates a detection signal to transmit to the control unit 160. The detection signal may include coordinate data of the user's touch input.
The media playing device 100 of the present invention may be configured with a touch screen as described above; however, embodiments of the present invention described hereafter may not be limited to the media playing device 100 having a touch screen.
The storage unit 150 stores programs and data required for operating the media playing device 100 and may be divided into a program area and a data area.
The program area can store programs for controlling general operations of the media playing device 100, OS (Operating System) for booting the media playing device 100, application programs required for playing multimedia contents, and other optional functions of the media playing device 100.
The control unit 160 controls general operations of each component of the media playing device 100. In particular, the control unit 160 can control operations of the delay compensating unit 121 installed in the audio processing unit 120 and an audio signal being output to the external speaker 122 when the media playing device 100 plays a stereophonic sound.
According to
The wireless communication unit 210 transmits and receives data through wireless communication. The wireless communication unit 210 performs a series of operations for transmitting and receiving a control signal and an audio signal for a portable device through a wireless interface. The wireless communication unit 210 may utilize one of Bluetooth, infrared, and Zigbee communication systems; however, the wireless communication system 210 is not limited to these systems. Further, the wireless communication unit 210 can output data received from the wireless communication to the signal processing unit 240 and transmit data output by the signal processing unit 240 through the wireless communication.
The microphone 220 can transmit an audio signal to the signal processing unit 240 by receiving from the external speaker 122 of the media playing device 100. Further, the portable device of the present invention may include a plurality of microphones, and audio signals received from each microphone are transmitted to the signal processing unit 240.
The speaker 230 outputs an audio signal processed by the signal processing unit 240. The speaker 230 of the portable device according to the present invention may be a single speaker which is worn at one of ears.
The signal processing unit 240 may be called a control unit. The signal processing unit 240 can estimate an imbalance parameter between an audio signal received through the wireless communication unit 210 and an audio signal received through the microphone 220. For example, the signal processing unit 240 can estimate an imbalance parameter between signals of microphone 220 received in an audible frequency band and right and left signals of a stereophonic sound received through the wireless communication unit 210. The imbalance parameter may include parameters such as a volume, echo, background noise difference, and frequency response difference.
Further, the signal processing unit 240 can determine a compensation parameter for compensating a signal to be output to the speaker 230 by using an extracted imbalance parameter. The signal processing unit 240 can generate a signal to be output to the speaker 230 by modifying some parts of an audio signal received through the wireless communication unit 210 according to the determined compensation parameter.
Although the portable device of
In
Providing a stereophonic sound can be achieved by processing an audio signal with a transfer function reaching to the right or left ear of a listener and transmitting the processed audio signal to each ear of the listener. The transfer function reaching to the right or left ear of the listener can be selected from a HRTF (Head Related Transfer Function) database on the basis of a location of a listener who wears the portable device of the present invention.
Hereafter, it is assumed that a signal processed with HRTF reaching to the left ear of a listener is a first signal and a signal processed with HRTF reaching to the right ear of the listener is a second signal. Namely, the first signal is an audio signal including left directionality information and the second signal is an audio signal including right directionality information. In order to provide a high performance stereophonic sound, the first signal must be transmitted to the left ear of the listener and the second signal must be transmitted to the right ear of the listener. The right ear of the listener directly receives a signal output by the speaker 230 of the portable device according to the present invention (for example, an earphone type) and the left ear of the listener receives a signal output by the external speaker 122 of the media playing device 100 according to the present invention.
Accordingly, in order to provide signals transmitted to both ears in a stereophonic sound, the signals to be output by the speaker 230 and the external speaker 122 require proper signal processing.
First, the media playing device 100 transmits a first signal and a second signal to a portable device according to the present invention through the wireless communication unit 110. Further, the control unit 160 transmits the first signal to the audio processing unit 120 in order to output to the external speaker 122.
The audio processing unit 120 can output a third signal to the external speaker 122. The audio signal output to the external speaker can be received by the microphone 220 of the portable device. Differently from the signal transmitted to the portable device through the wireless communication unit 110, the signal transmitted from the external speaker 122 to the portable device is transmitted in an audible frequency band.
The delay compensating unit 121 can perform a proper pre-processing to output a signal to the external speaker 122. The delay compensating unit 121 can perform processing in various ways for a signal to be output, and the most typical processing is a delay compensation.
If the distance between the external speaker 122 of the media playing device 100 and the microphone 220 of the portable device is great, problems may be generated because of an arrival time difference between the first signal output by the external speaker 122 and the second signal received by the microphone 220 of the portable device. This is because the third signal is transmitted in the sound velocity differently from the first signal which is transmitted electronically. In this case, the signal processing unit 240 installed in the portable device according to the present invention can transmit data related to the delay time of the third signal to the media playing device 100 through the wireless communication unit 210.
The control unit 160 receives data related to the delay time from the portable device and transmits the data to the audio processing unit 120. The delay compensating unit 121 installed in the audio processing unit 120 compensates the third signal by using the data related to the delay time and outputs the third signal to the external speaker 122.
In the meantime, the wireless communication unit 210 of the portable device receives a first signal and a second signal from the media playing device 100. The received signals are then transmitted to the signal processing unit 240.
Further, the microphone 220 receives an audio signal (third signal) output by the external speaker 122 of the media playing device 100 and transmits the audio signal to the signal processing unit 240.
Because the distance between the microphone 220 and the left ear of the listener is very close, the signal received by the microphone 220 and the signal received by the left ear of the listener may be considered as identical signals. Hereinafter, the signal output by the external speaker 122 of the media playing device 100 and received by the left ear of the listener may be called a third signal, which is the same as the signal received by the microphone 220.
The signal processing unit 240 can estimate various imbalance parameters from the difference between the first signal received through the wireless communication unit 210 and the third signal received through the microphone 220. A compensation parameter is determined by using the estimated imbalance parameters, and signal processing is performed by applying the compensation parameter to the second signal. A signal (hereinafter, fourth signal) processed by using the compensation parameter is output through the speaker 230.
The left ear of the listener receives the third signal from the external speaker 122 and the right ear of the listener receives the fourth signal from the speaker 230. The fourth signal is a signal processed from the second signal by using the compensation parameter. Accordingly, a stereophonic sound can be played for the listener through the external speaker 122 and the speaker 230.
According to
The first imbalance parameter estimating unit 241 is a device configured to estimate a channel between the external speaker 122 and a listener, and a channel response between the external speaker 122 and the microphone 220 can be obtained by comparing the first signal transmitted through the wireless communication unit 210 and the third signal received through the microphone 220. Delay information according to the distance between the microphone 220 and the external speaker 122, location correlation between the external speaker 122 and the microphone 220, and frequency response characteristics can be estimated from the channel response. The first imbalance parameter estimating unit 241 transmits data related to the channel response (i.e., estimated imbalance parameter) to the compensation parameter determining unit 244.
The second imbalance parameter estimating unit 242 estimates a noise being added to a first signal while the first signal is transmitted from the external speaker 122 through a spatial transfer path. The first signal output through the external speaker 122 is transmitted to an ear of the listener with a background noise; however, the third signal output through the speaker 230 includes only the audio signal without the background noise. Accordingly, the imbalance of the audio signals reaching both ears of the listener can be resolved by adding the background noise to the third signal. The second imbalance parameter estimating unit 242 transmits the data related to the estimated background noise to the compensation parameter determining unit 244.
The third imbalance parameter estimating unit 243 estimates the intensity of a signal received through the microphone 220. A gain of a signal output to the speaker 230 can be adjusted corresponding to the audio signal intensity transmitted to the left ear of the listener by using data related to the signal intensity estimated by the third imbalance parameter estimating unit 243. The third imbalance parameter estimating unit 243 transmits the data related to the estimated signal intensity to the compensation parameter determining unit 244.
The compensation parameter determining unit 244 determines a compensation parameter for compensating the second signal by receiving imbalance parameters estimated by the first imbalance parameter estimating unit 241, second imbalance parameter estimating unit 242, and third imbalance parameter estimating unit 243. The compensation parameter determining unit 244 transmits the determined compensation parameter to the signal compensating unit 245.
The signal compensating unit 245 can compensate the second signal to be output through the speaker worn at the right ear among the first signal and the second signal received through the wireless communication unit 210. In more detail, the signal compensating unit 245 generates a fourth signal by compensating the second signal so that the fourth signal includes a signal level and a background noise similar to those of the third signal reaching the left ear. The fourth signal generated by the signal compensating unit 245 is output through the speaker 230 and transmitted to the right ear of the listener.
Hereinafter, a method for playing a stereophonic sound in a portable device is described in more detail with reference to
The portable device according to the present invention may be configured with a headset including a single speaker and a microphone; however, the portable device is not limited to this configuration. A wearable device can be applied to the portable device. For example, the wearable device may be configured with glasses, watch, bracelet, anklet, band, necklace, shoes, clothing, gloves, socks, contact lens, sports equipment, or medical equipment (medical diagnostic device). The wearable device can be attached to the skin of the human body or transplanted in the human body. The speaker used for the portable device may include a normal speaker unit blocking an external noise or a bone conduction speaker unit (leaky headphone unit) speaker allowing an inflow of an adjacent signal into an ear.
If the normal speaker unit is used for the portable device, a signal transmitted from the external speaker 122 to the speaker 230 is very weak and thereby can be disregarded.
If the first signal transmitted to the left ear of the listener is defined as sL(t), the second signal transmitted to the right ear of the listener is defined as sR(t), and the outputs of the external speaker 122 and the speaker 230 are defined respectively as yL(t) and yR(t), an output value of the speaker 230 before compensation can be expressed as shown by Formula 3.
yR(t)=sR(t)
yL(t)=hsL(t)
In Formula 3, h indicates a channel (channel response) between the external speaker 122 and the left ear.
The wireless communication unit 210 of the portable device receives a first signal and a second signal from the media playing device 100 at step S400.
The left ear and the microphone 220 receives an audio signal output by the external speaker 122 of the media playing device 100. The external speaker 122 outputs a first signal to transmit to the left ear at step S410, and the signal reaching to the left ear or the microphone 220 is a third signal of which a background noise is added to the first signal in a spatial transfer path. If the background noise is defined as i(t), the third signal is expressed as shown by the following Formula 4.
yL(t)=hsL(t)+i(t)
The signal processing unit 240 of the portable device estimates an imbalance parameter from the first signal received through the wireless communication unit 210 and the third signal received through the microphone 220 at step S420.
A method for estimating the imbalance parameter from the first signal and the third signal is described in more detail with reference to
According to
The estimation of the first imbalance parameter is expressed as shown in the following Formula 5.
Namely, the first imbalance parameter estimating unit 241 estimates the first imbalance parameter by sampling a signal having a correlation with the first signal (signal transmitted to the left ear).
The second imbalance parameter estimating unit 242 estimates a background noise to be added to the second signal from the background noise included in the third signal at step S510. The background noise is expressed as shown by the following Formula 6.
{circumflex over (i)}(t)=yL(t)−ĥsL(t)
Further, the third imbalance parameter estimating unit 243 estimates an output level suitable for the second signal from the intensity of the third signal received through the microphone 220 at step S520.
The compensation parameter determining unit 244 determines a compensation parameter for compensating the second signal by receiving each parameter estimated by the first imbalance parameter estimating unit 241, second imbalance parameter estimating unit 242, and the third imbalance parameter estimating unit 243 at step S530. The determined compensation parameter is transmitted to the signal compensating unit 245.
The signal compensating unit 245 generates a fourth signal by compensating the second signal according to the estimated parameters at step S430. The generated fourth signal is expressed as shown by the following Formula 7.
yR(t)=ĥsR(t)+{circumflex over (i)}(t)
The fourth signal generated by the signal compensating unit 245 to have a similar signal level and a similar background noise to those of the signal (third signal) received by the left ear is transmitted to the right ear of the listener through the speaker 230.
Subsequently, a case of using a bone conduction speaker unit for the speaker 230 is described.
In case of a bone conduction speaker unit, an audio signal transmitted to the microphone 220 is transmitted to the right ear. If the first signal transmitted to the left ear of the listener is indicated as sL(t) and the second signal transmitted to the right ear of the listener is indicated as sR(t), an output yR(t) of the speaker 230 before the compensation is expressed as shown by the following Formula 8.
yR(t)=sR(t)+hsL(t)
In order to play a stereophonic sound, the section hsL(t) must be removed from the output of the speaker 230. After compensating a channel response estimated by channel response estimating unit 241 by using a compensation parameter determined both for the first signal and the second signal received through the wireless communication unit 210, a signal processed by removing two signal sections is output to the speaker 230. In this case, the fourth signal reaching the right ear is expressed as shown by the following Formula 9.
yR(t)=ĥsR(t)−ĥsL(t)+hsL(t)
According to Formula 9, a stereophonic sound can be played effectively because the section hsL(t) has been removed.
In the above description, a method for playing a stereophonic sound and a principal of operating a device have been described according to the present invention. The above description assumes that a listener wears the speaker 230 of the portable device at the right ear; however, the speaker 230 can be worn alternatively at the left ear. In this case, an audio signal output by the external speaker 122 is transmitted to the right ear of the listener.
According to the present invention, a stereophonic sound can be played by using a single speaker and one or more external speakers.
While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2014-0021952 | Feb 2014 | KR | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/KR2015/001842 | 2/25/2015 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2015/130086 | 9/3/2015 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8788080 | Kallai | Jul 2014 | B1 |
| 20030031333 | Cohen et al. | Feb 2003 | A1 |
| 20080056503 | McGrath | Mar 2008 | A1 |
| 20080304361 | Peng et al. | Dec 2008 | A1 |
| 20100135118 | Van Leest et al. | Jun 2010 | A1 |
| 20130156198 | Kim | Jun 2013 | A1 |
| 20130157721 | Nystrom | Jun 2013 | A1 |
| 20130216071 | Maher | Aug 2013 | A1 |
| Number | Date | Country |
|---|---|---|
| 2008-546245 | Dec 2008 | JP |
| 2008-546345 | Dec 2008 | JP |
| 10-0927637 | Nov 2009 | KR |
| Number | Date | Country | |
|---|---|---|---|
| 20170064485 A1 | Mar 2017 | US |
