Patent Grant
9432793
1. Field of the Invention
The present invention relates to a convolution method and convolution device for convoluting into an audio signal a head-related transfer function (hereafter abbreviated to “HRTF”) for enabling a listener to hear a sound source situated in front or the like of the listener, during acoustic reproduction with an electric-acoustic unit such as an acoustic reproduction driver of headphones for example, which is disposed near the ears of the listener.
2. Description of the Related Art
In a case of the listener wearing the headphones on the head for example, and listening to acoustically reproduced signals with both ears, if the audio signals reproduced at the headphones are commonly-employed audio signals supplied to speakers disposed to the left and right in front of the listener, the so-called lateralization phenomenon, wherein the reproduced sound image stays within the head of the listener, occurs.
A technique called virtual sound image localization is disclosed in WO95/13690 Publication and Japanese Unexamined Patent Application Publication No. 03-214897, for example, as having solved this problem of the lateralization phenomenon. This virtual sound image localization enables the sound image to be reproduced (virtually localized in the relevant position) such that when reproduced with a headphone or the like, the sound image is reproduced as if there were a sound source, e.g., speakers in a predetermined perceived position, such as the left and right in front of the listener, and is realized as described below.
As shown in
In a state where a dummy head 1 (alternatively, this may be a human, the listener himself/herself) is present, an acoustic reproduction of an impulse for example, is performed at one channel, the left channel speaker SPL for example, and the impulse emitted by that reproduction is picked up with each of the microphones ML and MR and an HRTF for the left channel is measured. In the case of this example, the HRTF is measured as an impulse response.
In this case, the impulse response serving as the left channel HRTF includes, as shown in
Next, an acoustic reproduction of an impulse is performed at the right channel speaker SPR in the same way, and the impulse emitted by that reproduction is picked up with each of the microphones ML and MR and an HRTF for the right channel, i.e., the HRTF of the right channel, is measured as an impulse response.
In this case, the impulse response serving as the right channel HRTF includes an impulse response HRd of the sound waves from the right channel speaker SPR picked up with the microphone MR (hereinafter, referred to as “impulse response of right primary component”), and an impulse response HRc of the sound waves from the right channel speaker SPR picked up with the microphone ML (hereinafter, referred to as “impulse response of right crosstalk component”).
The impulse responses for the HRTF of the left channel and the HRTF of the right channel are convoluted, as they are, with the audio signals supplied to the acoustic reproduction drivers for the left and right channels of the headphones, respectively. That is to say, the impulse response of left primary component and impulse response of left crosstalk component, serving as the left channel HRTF obtained by measurement, are convoluted, as they are, with the left signal audio signals, and the impulse response of right primary component and impulse response of right crosstalk component, serving as the right channel HRTF obtained by measurement, are convoluted, as they are, with the right signal audio signals.
This enables sound image localization (virtual sound image localization) such that sound is perceived to be just as if it were being reproduced from speakers disposed to the left and right in front of the listener in the case or two-channel stereo audio of left and right for example, even though the acoustic reproduction is nearby the ears of the listener.
A case of two channels has been described above, but with a case of three or more channels, this can be performed in the same way by disposing speakers at the virtual sound image localization positions for each of the channels, reproducing impulses for example, measuring the HRTF for each channel, and convolute impulse responses of the HRTFs obtained by measurement as to the audio signals supplied to the drivers for the acoustic reproduction by the two channels, left and right, of the headphones.
Incidentally, when a place where measurement of an HRTF is performed is not an anechoic chamber, not only a direct wave from a perceived sound source (corresponding to a virtual sound image localization position) and but also the components of a reflected wave such as shown in a dotted line in
In order to eliminate properties of the room or place where measurement is performed, measuring in an anechoic chamber, where there are no reflections from the floor, ceiling, walls, and so forth, can be conceived. However, in the event of convoluting HRTFs measured in an anechoic chamber as they are into audio signals, there is a problem that virtual sound image localization and orientation are somewhat fuzzy since there is no reflected waves in the case of attempting to virtually localize a sound image.
Accordingly, with the related art, measurement of HRTF to be used as they are for convolution with audio signals is not performed in an anechoic chamber, but rather, HRTFs are measured in a room with a certain amount of reverberation. Further, there has been proposed an arrangement wherein a menu of rooms or places where the HRTFs were measured, such as a studio, hall, large room, and so forth, being presented to the user, so that the user who wants to enjoy music with virtual sound image localization can select the HRTF of a desired room or place from the menu.
However, as described above, with the related art, measurement of HRTFs is performed with not only impulse responses of direct waves from a perceived sound source position but also accompanying impulse responses from reflected waves without being able to separate the impulse response of direct waves and reflected waves, including both, so only an HRTF according to a measured place or room is obtainable, and accordingly, it has been difficult to obtain an HRTF according to a desired ambient environment or room environment, and convolute this into an audio signal. For example, it has been difficult to convolute an HRTF corresponding to a perceived listening environment into an audio signal such as where speakers are disposed in front on a vast plain which has neither walls nor obstructions thereabout.
Also, in the case of attempting to obtain an HRTF in a room having a perceived predetermined shape and inner volume, and a wall of a predetermined degree of sound absorption (corresponding to the attenuation rate of a sound wave), heretofore, there has been no way other than a method to look for or fabricate such a room, and an HRTF is measured and obtained in this room. However, in reality, it is difficult to look for or fabricate such a desired listening environment or room, and present used techniques are not sufficient to convolute an HRTF corresponding to a desired arbitrary listening environment or room environment into an audio signal.
It has been found desirable to provide a head-related transfer function convolution method and device, which enables convolution of an HRTF corresponding to a desired arbitrary listening environment or room environment to be performed, and a desired virtual sound image localization feeling to be obtained.
A head-related transfer function convolution method according to an embodiment of the present invention arranged, when an audio signal is reproduced acoustically by an electro-acoustic conversion unit disposed in a nearby position of both ears of a listener, to convolute a head-related transfer function into the audio signal, which allows the listener to listen to the audio signal such that a sound image is localized in a perceived virtual sound image localization position, the head-related transfer function convolution method including the steps of: measuring, when a sound source is disposed in the virtual sound image localization position, and a sound-collecting unit is disposed in the position of the electro-acoustic conversion unit, a direct wave direction head-related transfer function regarding the direction of a direct wave from the sound source to the sound-collecting unit, and a reflected wave direction head-related transfer function regarding the direction of selected one reflected wave or reflected wave direction head-related transfer functions regarding the directions of selected multiple reflected waves, from the sound source to the sound-collecting unit, to obtain such head-related transfer functions, separately beforehand; and convoluting the obtained direct wave direction head-related transfer function, and the reflected wave direction head-related transfer function regarding the direction of the selected one reflected wave or the reflected wave direction head-related transfer functions regarding the directions of the selected multiple reflected waves, into the audio signal.
Heretofore, as described above, integral head-related transfer functions including both of a direct wave direction head-related transfer function and reflected wave direction head-related transfer function are measured, and are convoluted into an audio signal without change, on the other hand, with the above configuration, at a head-related transfer function measuring process a direct wave direction head-related transfer function and reflected wave direction head-related transfer function are measured separately beforehand. Subsequently, the obtained direct wave direction head-related transfer function and reflected wave direction head-related transfer function are convoluted into an audio signal.
Here, the direct wave direction head-related transfer function is a head-related transfer function obtained from only a sound wave for measurement directly input to a sound-collecting unit from a sound source disposed in a perceived virtual sound image localization position, and does not include the components of a reflected wave.
Also, the reflected wave direction head-related transfer function is a head-related transfer function obtained from only a sound wave for measurement directly input to a sound-collecting unit from a perceived reflected wave direction, and does not include components reflected at whichever and input to a sound-collecting unit from a sound source in the relevant reflected wave direction.
Subsequently, in the measuring, as described above, a head-related transfer function for a direct wave, and a head-related transfer function for a reflected wave are obtained separately when a virtual sound image localization position is a sound source, but at this time, as a reflected wave direction for obtaining a reflected wave direction head-related transfer function one or multiple reflected wave directions are selected according to a perceived listening environment or room environment.
For example, in the case of assuming that a listening environment is a vast plain, there is neither surrounding walls nor ceiling, and there are only a direct wave from a sound source perceived in a virtual sound image localization position, and a sound wave reflected at the ground surface or floor from the sound source, and accordingly, a direct wave direction head-related transfer function, and a reflected wave direction head-related transfer function in the direction of a reflected wave from the ground surface or floor are obtained, and these head-related transfer functions are convoluted into an audio signal.
Also, in a case wherein a rectangular parallelepiped common room is assumed as a listening environment, as reflected waves, there are sound waves reflected at the surrounding wall, ceiling, and floor of a listener, and accordingly, the reflected wave direction head-related transfer function regarding each of the reflected wave directions is obtained, and the relevant reflected wave direction head-related transfer functions and direct wave direction head-related transfer functions are convoluted into an audio signal.
In the convoluting, corresponding convolution of the direct wave direction head-related transfer function and the reflected wave direction head-related transfer functions may be executed upon a time series signal of the audio signal from each of a start point in time to start convolution processing of the direct wave direction head-related transfer function, and a start point in time to start convolution processing of each of reflected wave direction head-related transfer functions, determined according to the path length of sound waves from the virtual sound image localization position and the position of the electro-acoustic conversion means of each of the direct waves and the reflected waves.
With the above configuration, a start point in time for starting convolution processing of a direct wave direction head-related transfer function, and a start point in time for starting convolution processing of each of a single or multiple reflected wave direction head-related transfer functions are determined according to the path lengths of sound waves from the virtual sound image localization positions of a direct wave and reflected wave to the electro-acoustic conversion unit. In this case, the path length regarding a reflected wave is determined according to a perceived listening environment or room environment.
In other words, the convolution start point in time of each of the head-related transfer functions is set according to the path lengths regarding the direct wave and reflected wave, whereby an appropriate head-related transfer function according to a perceived listening environment or room environment can be convoluted into an audio signal.
With regard to the reflected wave direction head-related transfer functions, gain may be adjusted according to an attenuation rate of sound waves at a perceived reflected portion, and the convolution is executed.
With the above configuration, in a perceived listening environment or room environment, a reflected wave direction head-related transfer function in the direction from a reflection portion which reflects a sound wave is adjusted by gain worth corresponding to an attenuation rate determined with the material or the like of the relevant reflection portion, and is convoluted into an audio signal. Thus, according to the above configuration, a head-related transfer function, wherein an attenuation rate caused by noise absorption or the like at a reflection portion of a sound wave in a perceived listening environment or room environment is taken into consideration, can be convoluted into an audio signal.
According to the above arrangements, a suitable HRTF can be convoluted into an audio signal, which corresponds to a perceived listening environment or room environment.
As described above, with an HRTF convolution method according to the related art, an arrangement has been made wherein a speaker is disposed in a perceived sound source position to localize a virtual sound image, an HRTF is measured assuming that an impulse response caused by a reflected wave is involved instead of an impulse response caused by a direct wave from the relevant perceived sound source position being involved (assuming that impulse responses between a direct wave and reflected wave are both included without being separated), the measured and obtained HRTF is convoluted into an audio signal without change.
That is to say, heretofore, the HRTF for a direct wave and the HRTF for a reflected wave from a sound source position perceived so as to localize a virtual sound image have been measured as an integral HRTF including both without being separated.
On the other hand, with an embodiment of the present invention, the HRTF for a direct wave and the HRTF for a reflected wave from a sound source position perceived so as to localize a virtual sound image are measured separately beforehand.
Therefore, with the present embodiment, an HRTF regarding a direct wave from a perceived sound source perceived in a particular direction as viewed from a measurement point position (i.e., sound wave reaching directly the measurement point position including no reflected wave) is to be obtained. With the direction of a sound wave after being reflected off a wall or the like as a sound source direction, the HRTF for a reflected wave is measured as a direct wave from the sound source direction thereof. That is to say, in the case of considering a reflected wave which is reflected off a predetermined wall, and input to a measurement point position, the reflected sound wave from the wall after being reflected off the wall can be regarded as a direct wave of a sound wave from a sound source perceived in a reflected position direction at the relevant wall.
Accordingly, with the present embodiment, when measuring an HRTF for a direct wave from a sound source position perceived so as to localize a virtual sound image, an electro-acoustic converter serving as a measuring sound wave generating unit, e.g., speaker is disposed in the perceived sound source position so as to localize the relevant virtual sound image, but when measuring an HRTF for a reflected wave from a sound source position perceived so as to localize a virtual sound image, an electro-acoustic converter serving as a measuring sound wave generating unit, e.g., speaker is disposed in the incident direction to the measurement point position of a reflected wave to be measured.
Accordingly, an HRTF regarding reflected waves from various directions is measured by disposing an electro-acoustic converter serving as a measuring sound wave generating unit in the incident direction to the measurement point position of each reflected wave.
Subsequently, with the present embodiment, HRTFs regarding a direct wave and reflected waves thus measured are convoluted into an audio signal, thereby obtaining virtual sound image localization within target reproduction acoustic space, but with regard to HRTFs for reflected waves, only a reflected wave in a direction selected according to the target reproduction acoustic space is convoluted into an audio signal.
Also, with the present embodiment, HRTFs regarding a direct wave and reflected waves are measured by removing propagation delay worth corresponding to the path length of a sound wave from a measuring sound source position to a measurement point position, and at the time of performing processing for convoluting each of the HRTFs into an audio signal, the propagation delay worth corresponding to the path length of a sound wave from a measuring sound source position (virtual sound image localization position) to a measurement point position (acoustic reproduction unit position) is taken into consideration.
Thus, an HRTF regarding a virtual sound image localization position arbitrarily set according to the size of a room or the like can be convoluted into an audio signal.
Subsequently, properties such as the degree of reflection, degree of sound absorption, or the like due to the material of a wall or the like relating to the attenuation rate of a reflected sound wave are perceived as the gain of a direct wave from the relevant wall. That is to say, with the present embodiment, for example, an HRTF according to a direct wave from a perceived sound source position to a measurement point position is convoluted into an audio signal without attenuation, and also with regard to reflected sound wave components from the wall, an HRTF according to a direct wave from a sound source perceived in the reflected position direction of the wall thereof is convoluted with an attenuation rate according to the degree of reflection or degree of sound absorption corresponding to the properties of the wall.
The reproduction sound of an audio signal into which an HRTF is thus convoluted is listened to, whereby verification can be made whether to obtain what type of a virtual sound image localization state according to the degree of reflection or degree of sound absorption corresponding to the properties of the wall.
Also, acoustic reproduction from convolution in audio signals of HRTFs of direct waves and HRTFs of selected reflected waves, taking into consideration the attenuation rate, enables simulation of virtual sound image localization in various room environments and place environments. This can be realized by separating a direct wave and reflected waves from the perceived sound source position, and measuring as HRTFs.
Description of HRTF Measurement Method
As described above, HRTFs regarding a direct wave from which the reflected wave components have been eliminated can be obtained by measuring in an anechoic chamber, for example.
Accordingly, with an anechoic chamber, HRTFs are measured regarding a direct wave from a desired virtual sound image localization position, and perceived multiple reflected waves, and are employed for convolution.
That is to say, with an anechoic chamber, HRTFs are measured by disposing a microphone serving as an acousto-electric conversion unit for collecting a sound wave for measurement in a measurement point position in the vicinity of both ears of a listener, and also disposing a sound source for generating a sound wave for measurement in the positions of the directions of the direct wave and multiple reflected waves.
Incidentally, even if HRTFs are obtained within an anechoic chamber, the properties of speaker and microphone of a measuring system for measuring an HRTF are not eliminated, which causes a problem wherein the HRTFs measured and obtained have been affected by the properties of the speaker and microphone employed for measurement.
In order to eliminate the effects of properties of the microphones and speakers, using expensive microphones and speakers having excellent properties with flat frequency properties as the microphones and speakers used for measuring the HRTFs. However, even such expensive microphones and speakers do not yield ideally flat frequency properties, so there have been cases wherein the effects of the properties of such microphones and speakers could not be completely eliminated, leading to deterioration in the sound quality of the reproduced audio.
Also, eliminating the properties of the microphones and speakers can be conceived by correcting audio signals following convolution of the HRTFs, using inverse properties of the measurement system microphones and speakers, but in this case, there is the problem that a correction circuit has to be provided to the audio signal reproduction circuit, so the configuration becomes complicated, and also correction complete eliminating the effects of the measurement system is difficult.
In order to eliminate the influence of a room or place for measurement in light of the above-mentioned problems, with the present embodiment, HRTFs are measured within an anechoic chamber, and also in order to eliminate the influence of the properties of a microphone and speaker employed for measurement, the HRTFs measured and obtained are subjected to normalization processing such as described below. First, an embodiment of the HRTF measurement method according to the present embodiment will be described with reference to the drawings.
In a case where the electro-acoustic conversion unit for performing acoustic reproduction of audio signals in which the HRTFs have been convoluted are headphones with two channels of left and right for example, a microphone for the left channel is situated at the position of the headphone driver of the left channel, and a microphone for the right channel is situated at the position of the headphone driver of the right channel.
Subsequently, a speaker serving as an example of a measurement sound source is situated at one of the directions regarding which an HRTF is to be measured, with the listener or microphone position serving as a measurement point position as a basing point. In this state, measurement sound waves for the HRTF, impulses in this case, are reproduced from this speaker, and impulse responses are picked up with the two microphones. Note that in the following description, a position in a direction regarding which an HRTF is to be measured, where the speaker for the measurement sound source is placed, will be referred to as a “perceived sound source position”.
With the HRTF measurement unit 10, the impulse responses obtained from the two microphones represent HRTFs. With this embodiment, the measurement at the HRTF measurement unit 10 corresponds to a first measuring.
With a natural-state transfer property measurement unit 20, measurement of natural-state transfer properties is performed under the same environment as with the HRTF measurement unit 10. That is to say, with this example, the transfer properties are measured in a nature state wherein there is neither the human nor the dummy head at the listener's position, i.e., there is no obstacles between a measurement source position and a measurement point position.
Specifically, with the natural-state transfer property measurement unit 20, the dummy head or human situated with the HRTF measurement unit 10 in the anechoic chamber is removed, a natural state with no obstacles between the speakers which are the perceived sound source position and the microphones is created, and with the placement of the speakers which are the perceived sound source position and the microphones being exactly the same state as with the HRTF measurement unit 10, in this state, measurement sound waves, impulses in this example, are reproduced by perceived sound source position speakers, and the impulse responses are picked up with the two microphones.
The impulse responses obtained form the two microphones with the natural-state transfer property measurement unit 20 represent natural-state transfer properties with no obstacles such as the dummy head or human.
Note that with the HRTF measurement unit 10 and the natural-state transfer property measurement unit 20, the above-described HRTFs and natural-state transfer properties for the left and right primary components, and HRTFs and natural-state transfer properties for the left and right crosstalk components, are obtained from each of the two microphones. Later-described normalization processing is performed for each of the primary components and left and right crosstalk components. In the following description, normalization processing will be described regarding only the primary components for example, and description of normalization processing regarding the crosstalk components will be omitted, to facilitate description. Of course, normalization processing is performed in the same way regarding the crosstalk components, as well.
The impulse responses obtained with the HRTF measurement unit 10 and the natural-state transfer property measurement unit 20 are output of digital data of 8,192 samples at a sampling frequency of 96 kHz with this example.
Now, the data of the HRTF obtained from the HRTF measurement unit 10 is presented as X(m), where m=0, 1, 2 . . . , M−1 (M=8192), and data of the natural-state transfer property obtained from the natural state transfer property measurement unit 20 is presented as Xref(m), where m=0, 1, 2 . . . , M−1 (M=8192).
The HRTF data X(m) from the HRTF measurement unit 10 and the natural-state transfer property data Xref(m) from the natural-state transfer property measurement unit 20 are subjected to removal of data of the head portion from the point in time at which reproduction of impulses was started at the speakers, by an amount of delay time equivalent to the arrival time of sound waves from the speaker at the perceived sound source position to the microphones for obtaining pulse responses, by delay removal shift-up units 31 and 32, and also at the delay removal shift-up units 31 and 32 the number of data is reduced to a number of data of a power of two, such that orthogonal transform from time-axial data to frequency-axial data can be performed next downstream.
Next, the HRTF data X(m) and the natural-state transfer property data Xref(m), of which the number of data has been reduced at the delay removal shift-up units 31 and 32, are supplied to FFT (Fast Fourier Transform) units 33 and 34 respectively, and transformed from time-axial data to frequency-axial data. Note that with the present embodiment, the FFT units 33 and 34 perform Complex Fast Fourier Transform (Complex FFT) which takes into consideration the phase.
Due to the complex FFT processing at the FFT unit 33, the HRTF data X(m) is transformed to FFT data made up of a real part R(m) and an imaginary part jI(m), i.e., R(m)+jI(m).
Also, due to the complex FFT processing at the FFT unit 34, the natural-state transfer property data Xref(m) is transformed to FFT data made up of a real part Rref(m) and an imaginary part jIref(m), i.e., Rref(m)+jIref(m).
The FFT data obtained from the FFT units 33 and 34 are X-Y coordinate data, and with this embodiment, further polar coordinates conversion units 35 and 36 are used to convert the FFT data into polar coordinates data. That is to say, the HRTF FFT data R(m)+jI(m) is converted by the polar coordinates conversion unit 35 into a radius γ(m) which is a size component, and an amplitude θ(m) which is an angle component. The radius γ(m) and amplitude θ(m) which are the polar coordinates data are sent to a normalization and X-Y coordinates conversion unit 37.
Also, the natural-state transfer property FFT data Rref(m)+jIref(m) is converted by the polar coordinates conversion unit 35 into a radius γref(m) and an amplitude θref(m). The radius γref(m) and amplitude θref(m) which are the polar coordinates data are sent to the normalization and X-Y coordinates conversion unit 37.
At the normalization and X-Y coordinates conversion unit 37, first, the HRTF measured including the dummy head or human is normalized using the natural-state transmission property where there is no obstacle such as the dummy head. Specific computation of the normalization processing is as follows.
With the radius following normalization as γn(m) and the amplitude following normalization as θn(m),
γn(m)=γ(m)/γref(m)
θn(m)=θ(m)/θref(m) (Expression 1)
holds.
Subsequently, at the normalization and X-Y coordinates conversion unit 37, the polar coordinate system data following normalization processing, the radius γn(m) and the amplitude θn(m), is converted into normalized HRTF data of frequency-axial data of the real part Rn(m) and imaginary part jIn(m) (m=0, 1 . . . M/4-1) of the X-Y coordinate system.
The normalized HRTF data of the frequency-axial data of the X-Y coordinate system is transformed into impulse response Xn(m) which is normalized HRTF data of the time-axis at an inverse FFT unit 38. The inverse FFT unit 38 performs Complex Inverse Fast Fourier Transform (Complex Inverse FFT).
That is to say, computation of
Xn(m)=IFFT(Rn(m)+jIn(m))
where m=0, 1, 2 . . . M/2-1, is performed at the Inverse FFT (IFFT (Inverse Fast Fourier Transform)) unit 38, which obtains the impulse response Xn(m) which is time-axial normalized HRTF data.
The normalized HRTF data Xn(m) from the inverse FFT unit 38 is simplified to impulse property tap length which can be processed (which can be convoluted, described later), at an IR (impulse response) simplification unit 39. With this embodiment, this is simplified to 600 taps (600 pieces of data from the head of the data from the inverse FFT unit 38).
The normalized HRTF data Xn(m) (m=0, 1 . . . 599) simplified at the IR simplification unit 39 is written to the normalized HRTF memory 40 for later-described convolution processing. Note that the normalized HRTF written to this normalized HRTF memory 40 includes a normalized HRTF which is a primary component, and a normalized HRTF which is a crosstalk function, at each of the perceived sound source positions (virtual sound image localization positions), as described earlier.
The description above has been description regarding processing for obtaining normalized HRTFs as to a speaker position in a case where a speaker for reproducing impulses as an example of measurement sound waves is situated at one perceived sound source position separated from a microphone position with a measurement point position by a predetermined distance, in one particular direction as to a listener position.
With this embodiment, the perceived sound source position, which is the position at which the speaker for reproducing the impulses serving as the example of a measuring sound wave is positioned, is changed variously in different directions as to the measurement point position, with a normalized HRTF being obtained for each perceived sound source position.
That is to say, with the present embodiment, HRTFs are obtained regarding not only a direct wave but also reflected waves from a virtual sound image localization position, and accordingly, a virtual sound source position is set to multiple positions in light of the incident direction to measurement point positions for reflected waves, thereby obtaining normalized HRTFs thereof.
Now, the perceived sound source position which is the speaker placement position is changed in increments of 10 degrees at a time for example, which is a resolution for a case of taking into consideration the direction of a reflected wave direction to be obtained, over an angular range of 360 degrees or 180 degrees center on the microphone position or listener which is the measurement position, within a horizontal plane, to obtain normalized HRTFs regarding reflected waves from both side walls of the listener.
Similarly, the perceived sound source position which is the speaker placement position is changed in increments of 10 degrees at a time for example, which is a resolution for a case of taking into consideration the direction of a reflected wave direction to be obtained, over an angular range of 360 degrees or 180 degrees center on the microphone position or listener which is the measurement position, within a vertical plane, to obtain a normalized HRTF regarding a reflected wave from the ceiling or floor.
A case of taking into consideration an angular range of 360 degrees is a case wherein there is a virtual sound image localization position serving as a direct wave behind the listener, for example, a case assuming reproduction of multi-channel surround-sound audio such as 5.1 channels, 6.1 channels, 7.1 channels, and so forth, and also a case of taking into consideration a reflected wave from the wall behind the listener. A case of taking into consideration an angular range of 180 degrees is a case assuming that the virtual sound image localization position is only in front of the listener, or a state where there are no reflected waves from a wall behind the listener.
Also, with this embodiment, the position where the microphones are situated is changed in the measurement method of the HRTF and natural-state transfer property at the HRTF measurement units 10 and 20, in accordance with the position of acoustic reproduction drivers such as the drivers of the headphones actually supplying the reproduced sound to the listener.
Specifically,
Also, with this example of the case of the inner headphones, the two microphones ML and MR are situated at positions within the auditory capsule positions of the ears of the dummy head or human, as shown in
The above-described normalization processing is carried out by normalizing HRTFs measured at each of the perceived sound source positions indicated by dots P1, P2, P3, . . . in
Next,
More specifically,
The measurement state at the natural-state transfer property measurement unit 20 in the case that the acoustic reproduction unit is over-head headphones is a measurement environment wherein the dummy head or human OB in
Next,
More specifically,
Also, as shown in
The measurement state at the natural-state transfer property measurement unit 20 in the case that the acoustic conversion reproduction unit is electro-acoustic conversion drivers attached to the headrest of the chair is a measurement environment wherein the dummy head or human OB in
Next,
Subsequently, speakers for reproducing impulses are disposed in perceived sound source positions in directions desired to measure an HRTF, for example, for each 10 degrees interval with the listener position or the center position of the seven microphones as the center, such as shown in circles P1, P2, P3, and so on, in the same way as with the above-mentioned case.
Subsequently, an impulse serving as a sound wave for measurement reproduced with the speaker in each perceived sound source position is sound-collected at each of the microphones ML1 through ML7 and MR1 through MR7, respectively. Subsequently, in a state in which there is a dummy head or person in the listener position, an HRTF is obtained from each of the output audio signals of the microphones ML1 through ML7, and MR1 through MR7. Also, in a natured state in which there is neither dummy head nor person, natured-state transfer properties are obtained from each of the output audio signals of the microphones ML1 through ML7, and MR1 through MR7. Subsequently, as described above, a normalized HRTF is each obtained from the HRTF and natured-state transfer properties, and is stored in a normalized HRTF memory 40.
In the case of the example in
From the above, impulse responses from a virtual sound source position are measured in an anechoic chamber, for example, at 10 degree intervals, centered on the center position of the head of the listener or the center position of the electro-acoustic conversion unit for supplying audio to the listener at the time of reproduction, as shown in
The obtained normalized HRTFs have properties of speakers generating the impulses and properties of the microphones picking up the impulses eliminated by normalization processing.
Further, the obtained normalized HRTFs have had a delay removed which corresponds to the distance between the position of speaker generating the impulses (perceived sound source position) and position of microphones for picking up the impulses (assumed driver positions), so this is irrelevant to the distance between the position of speaker generating the impulses (perceived sound source position) and position of microphones for picking up the impulses (assumed driver positions). That is to say, the obtained normalized HRTFs are HRTFs corresponding to only the direction of the speaker generating the impulses (perceived sound source position) as viewed from the position of microphones for picking up the impulses (assumed driver positions).
Accordingly, at the time of convolution of the normalized HRTF in the audio signals, providing a delay to the audio signals corresponding to the distance between the virtual sound source position and the assumed driver position enables acoustic reproduction with the distance position corresponding to the delay in the direction of the perceived sound source position as to the assumed driver positions as a virtual sound image localization position. With reflected waves from the direction of the perceived sound source position, this can be achieved by providing the audio signals with a delay corresponding to the path length of sound waves from the position at which virtual sound image localization is desired, reflected off of reflection portions such as walls or the like, and input to the assumed driver position from the perceived sound source position.
That is to say, in the case of convoluting a normalized HRTF into an audio signal regarding a direct wave and reflected waves, the audio signal is subjected to delay corresponding to the path length of a sound wave to be input from a desired virtual sound image localization position to a perceived driver position.
Note that signal processing in the block diagram in
Note that with the example in
Also, the IR simplification unit 39 is for reducing the amount of convolution processing at the time of the later-described convolution processing of the HRTFs, and accordingly this can be omitted.
Further, in the above-described embodiment, the reason that the frequency-axial data of the X-Y coordinate system from the FFT units 33 and 34 is converted into frequency data of a polar coordinate system is taking into consideration cases where normalization processing does not work in the state of frequency data of the X-Y coordinate system, so with an ideal configuration, normalization processing can be performed with frequency data of the X-Y coordinate system as it is.
Note that with the above-described example, normalized HRTFs are obtained regarding a great number of perceived sound source positions, assuming various virtual sound image localization positions and the perceived driver positions of the incident directions of the reflected waves thereof. The reason why normalized HRTFs regarding the multiple perceived sound source positions have been thus obtained is for enabling an HRTF in the direction of an employed perceived sound source position to be selected therefrom later. However, it goes without saying that in a case wherein a virtual sound source localization position is fixed beforehand, and the incident direction of a reflected wave is determined beforehand, normalized HRTFs as to the fixed virtual sound image localization position and the perceived sound source position in the incident direction of a reflected wave may be obtained.
Now, while measurement is performed in an anechoic chamber in the above-described embodiment in order to measure the HRTFs and natural-state transfer properties regarding only the direct waves from multiple perceived sound source positions, but direct wave components can be extracted even in rooms with reflected waves rather than an anechoic chamber, if the reflected waves are greatly delayed as to the direct waves, by applying a time window to the direct wave components.
Also, by using TSP (Time Stretched Pulse) signals instead of impulses for the measurement sound waves for HRTFs emitted by the speaker at the perceived sound source positions, reflected waves can be eliminated and HRTFs and natural-state transfer properties can be measured regarding direct waves alone even if not in an anechoic chamber.
Verification of Advantages of Employing Normalized HRTF
The speaker used here is an industrial-use speaker which is supposed to have quite good properties, but even then properties as shown in
With the related art, the properties of the speaker and microphones are added to the HRTF, and are not removed, so the properties and sound quality of the sound obtained with the HRTFs convoluted are effected of the properties of the speaker of and microphones.
With the embodiment according to the present invention described above, complex FFT processing is performed, and normalized HRTFs are used taking into consideration the phase component, so the normalized HRTFs are higher in fidelity as compared to cases of using HRTFs normalized only with the amplitude component.
An arrangement wherein processing for normalizing the amplitude alone without taking into consideration the phase is performed, and the impulse properties remaining at the end are subjected to FFT again to obtain properties, is shown in
Also, in the processing procedures in
That is to say, in the event of performing simplification for reducing the number of data first for the data obtained with the HRTF measurement unit 10 and natural-state transfer property measurement unit 20 (case of performing normalization with those following the number of impulses used at the end as 0), the properties of the normalized HRTFs are as shown in
Description of HRTF Convolution Method
The reflected waves include a high-order reflected wave, and also include a reflected wave of which the path length from a virtual sound image localization position to a measurement point position is long, and accordingly, a convolution process section according to the related art becomes a relatively long section such as shown in
As compared to the HRTF convolution method according to the related art such as in
Basically, with the present embodiment, when determining a virtual sound image localization position, a normalized HRTF for a direct wave between the virtual sound image localization position and a measurement point position (acoustic reproduction driver installation position) is convoluted into an audio signal. Note however, with regard to normalized HRTFs for reflected waves, only an HRTF selected according to a perceived listening environment, room configuration, or the like is convoluted into an audio signal.
For example, in the case of perceiving a listening environment such as the above-mentioned vast plain, only a reflected wave from a virtual sound image localization position to the ground surface (floor) is selected of reflected waves, a normalized HRTF obtained in a direction where the relevant reflected wave is input to the measurement point position is convoluted into an audio signal. Also, for example, in the case of a common rectangular parallelepiped shaped room, all of the reflected waves from a ceiling, floor, walls on the left and right of the listener, and walls of the forward and backward of the listener are selected, normalized HRTFs obtained in directions where these reflected waves are input to measurement point positions are convoluted.
Also, in the case of the latter room, a secondary reflection, third reflection, and so forth as well as a primary reflection are caused as reflected waves, but for example, a primary reflection alone is selected. According to an experiment, even with an audio signal in which a normalized HRTF regarding a primary reflection is convoluted, the audio signal thereof is reproduced acoustically, thereby obtaining excellent virtual sound image localization feeling. Note that if normalized HRTFs regarding a second reflected wave and thereafter are convoluted into an audio signal, when the audio signal thereof is reproduced acoustically, further excellent virtual sound image localization feeling are obtained in some cases.
A normalized HRTF regarding a direct wave is basically convoluted into an audio signal without changing the gain thereof, but with regard to reflected waves, a normalized HRTF is convoluted into an audio signal with gain corresponding to whether the reflected wave is primary reflection or second reflection or further high-order reflection. This is because normalized HRTFs obtained with the present embodiment are each measured regarding a direct wave from a perceived sound source position set in a predetermined direction, and normalized HRTF regarding reflected waves in the relevant predetermined directions are attenuated as to the direct wave. Note that the higher the order of a reflected wave is, the more the attenuation amount of a normalized HRTF regarding the reflected wave as to a direct wave increases.
Also, as described above, with regard to HRTFs of reflected waves, the present embodiment enables gain to be set further in light of the degree of sound absorption (attenuation rate of a sound wave) corresponding to the surface shape, surface configuration, material, or the like of a perceived reflection portion.
As described above, with the present embodiment, a reflected wave for convoluting an HRTF is selected, and the gain of the HRTF of each reflected wave is adjusted, whereby convolution of an HRTF as to an audio signal can be performed according to an arbitrary perceived room environment and listening environment. That is to say, like the related art, an HRTF with a room or space perceived to provide an excellent acoustic field space can be convoluted into an audio signal without measuring an HRTF with a room or space which provides an excellent acoustic field.
First example of Convolution Method (
With the present embodiment, a normalized HRTF for a direct wave (direct wave direction HRTF), and a normalized HRTF for each of reflected waves (reflected wave direction HRTF) are, as described above, obtained independently, and accordingly, with a first example, HRTFs for a direct wave and each of reflected waves are convoluted into an audio signal independently.
For example, a case will be described wherein three reflected waves (reflected wave directions) as well as a direct wave (direct wave direction) are selected, normalized HRTFs corresponding to both (direct wave direction HRTF and reflected wave direction HRTF) are convoluted.
Delay time corresponding to the path length from a virtual sound image localization position to a measurement point position is obtained as to each of a direct wave and reflected waves beforehand. This delay time is obtained by a calculation if a measurement point position (acoustic reproduction driver position) and virtual sound image localization position are determined, and a reflection portion is determined. Subsequently, with regard to the reflected waves, the attenuation amount (gain) as to a normalized HRTF is also determined beforehand.
Subsequently, the normalized HRTF regarding the direction of the relevant direct wave obtained as described above is convoluted into the audio signal at a convolution process section CP0 of data length worth of the relevant normalized HRTF (600 pieces worth of data in the above example) which is started from the above-mentioned point in time t0.
Next, of the three reflected waves, with regard to the normalized HRTF of a first reflected wave 1 (reflected wave direction HRTF), delay DL1 corresponding to a path length where the first reflected wave reaches a measurement point position from a virtual sound image localization position is taken into consideration as to the audio signal. That is to say, a convolution start point of the normalized HRTF for the first reflected wave 1 becomes a point in time t1 obtained by delaying the audio signal by the delay DL1, which is shown at the bottom of
Subsequently, the normalized HRTF regarding the direction of the first reflected wave 1 obtained as described above (reflected wave direction HRTF) is convoluted into the audio signal at a convolution process section CP1 of data length worth of the relevant normalized HRTF (600 pieces worth of data in the above example) which is started from the above-mentioned point in time t1. At the time of this convolution processing, the above-mentioned normalized HRTF is multiplied by gain G1 (G1<1) in light of what order the first reflected wave 1 is, and the degree of sound absorption (or the degree of reflection) at a reflection portion.
Also, similarly, with regard to the normalized HRTFs of a second reflected wave 2 and third reflected wave 3 (reflected wave direction HRTFs), delay DL2 and DL3 corresponding to a path length where the first reflected wave and third reflected wave reach a measurement point position from a virtual sound image localization position is taken into consideration as to the audio signal. That is to say, as shown at the bottom of
Subsequently, the normalized HRTF regarding the direction of the second reflected wave 2 obtained as described above (reflected wave direction HRTF) is convoluted into the audio signal at a convolution process section CP2 of data length worth of the relevant normalized HRTF (600 pieces worth of data in the above example) which is started from the above-mentioned point in time t2, and the normalized HRTF regarding the direction of the third reflected wave 3 obtained as described above (reflected wave direction HRTF) is convoluted into the audio signal at a convolution process section CP3 of data length worth of the relevant normalized HRTF (600 pieces worth of data in the above example) which is started from the above-mentioned point in time t3.
At the time of this convolution processing, the above-mentioned normalized HRTFs are multiplied by gain G2 and G3 (G2<1 and G3<1) in light of what order each of the second reflected wave 2 and third reflected wave 3 is, and the degree of sound absorption (or the degree of reflection) at a reflection portion.
The example in
With this example, an input audio signal Si into which an HRTF should be convoluted is supplied to each of the delay units 511, 521, 531, and 541. The delay units 511, 521, 531, and 541 delay the input audio signal Si into which an HRTF should be convoluted to conversion start points in time t0, t1, t2, and t3 of the normalized HRTFs for the direct wave and first through third reflected waves, respectively. Accordingly, with this example, as shown in the drawing, the delay amounts of the delay units 511, 521, 531, and 541 are determined as DL0, DL1, DL2, and DL3, respectively.
Each of the HRTF conversion circuits 512, 522, 532, and 542 is a portion to execute processing for convoluting a normalized HRTF into an audio signal, and with this example, configured of an IIR (Infinite Impulse Response) filter or FIR (Finite Impulse Response) filter, of 600 taps.
The normalized HRTF memory 513, 523, 533, and 543 are for storing and holding a normalized HRTF to be convoluted at each of the HRTF convolution circuits 512, 522, 532, and 542. The normalized HRTF memory 513 stores and holds a normalized HRTF regarding the direction of a direct wave, the normalized HRTF memory 523 stores and holds a normalized HRTF regarding the direction of the first reflected wave, the normalized HRTF memory 533 stores and holds a normalized HRTF regarding the direction of the second reflected wave, and the normalized HRTF memory 543 stores and holds a normalized HRTF regarding the direction of the third reflected wave, respectively.
The stored and held normalized HRTF regarding the direction of a direct wave, the stored and held normalized HRTF regarding the direction of the first reflected wave, the stored and held normalized HRTF regarding the direction of the second reflected wave, and the stored and held normalized HRTF regarding the direction of the third reflected wave are, for example, selected and read out from the above-mentioned normalized HRTF memory 41, and are written in the corresponding normalized HRTF memory 513, 523, 533, and 543, respectively.
The gain adjustment units 514, 524, 534, and 544 are for adjusting the gain of a normalized HRTF to be convoluted. The gain adjustment units 514, 524, 534, and 544 multiply the normalized HRTFs from the normalized HRTF memory 513, 523, 533, and 543 by the gain values (<1) stored in the gain memory 515, 525, 535, and 545, and supply the multiplication results to the HRTF convolution circuits 512, 522, 532, and 542, respectively.
With this example, the gain value G0 (≦1) regarding a direct wave is stored in the gain memory 515, the gain value G1 (<1) regarding the first reflected wave is stored in the gain memory 525 the gain value G2 (<1) regarding the second reflected wave is stored in the gain memory 535, and the gain value G3 (<1) regarding the third reflected wave is stored in the gain memory 545.
The adder 55 adds and composites the audio signals into which the normalized HRTFs from the convolution processing unit 51 for a direct wave, and the convolution processing units 52, 53, and 54 for the first through third reflected waves have been convoluted, and outputs an output audio signal So.
With such a configuration, an input audio signal Si into which an HRTF should be convoluted is supplied to each of the delay units 511, 521, 531, and 541, and the respective input audio signals Si are delayed to the convolution start points in time t0, t1, t2, and t3 of the normalized HRTFs for the direct wave and first through third reflected waves. The input audio signals Si delayed to the convolution start points in time t0, t1, t2, and t3 of the HRTFs at the delay units 511, 521, 531, and 541 are supplied to the HRTF convolution circuits 512, 522, 532, and 542.
On the other hand, the stored and held normalized HRTF data is read out sequentially from each of the convolution start points in time t0, t1, t2, and t3 from each of the normalized HRTF memory 513, 523, 533, and 543. The readout timing control of the normalized HRTF data from each of the normalized HRTF memory 513, 523, 533, and 543 will be omitted here.
The readout normalized HRTF data is subjected to gain adjustment by being multiplied by the gain G0, G1, G2, and G3 from the gain memory 515, 525, 535, and 545 at each of the gain adjustment units 514, 524, 534, and 544, following which is supplied to each of the HRTF convolution circuits 512, 522, 532, and 542.
With each of the HRTF convolution circuits 512, 522, 532, and 542, the gain-adjusted normalized HRTF data is subjected to convolution processing at each of the convolution process sections CP0, CP1, CP2, and CP3 shown in
In the case of the first example, each of the normalized HRTFs regarding a direct wave and multiple reflected waves can be convoluted into an audio signal independently, so the delay amounts at the delay units 511, 521, 531, and 541, and gain stored in the gain memory 515, 525, 535, and 545 are adjusted, and further, the normalized HRTFs to be stored in the normalized HRTF memory 513, 523, 533, and 543 and convoluted are changed, whereby convolution of HRTFs can be readily performed according to the difference of an listening environment, such as the difference of listening environment space types such as indoor, outdoor, or the like, the difference of the shape and size of a room, and the material of a reflection portion (the degree of sound absorption and degree of reflection), and so forth.
In a case wherein the delay units 511, 521, 531, and 541 are configured of a variable delay unit capable of varying a delay amount according to external operation input such as an operator or the like, a unit for writing an arbitrary normalized HRTF selected from the normalized HRTF memory 40 by the operator in the normalized HRTF memory 513, 523, 533, and 543, and further, and a unit for allowing the operator to input and store arbitrary gain in the gain memory 515, 525, 535, and 545 are provided, convolution of an HRTF can be performed according to a listening environment such as listening environment space set arbitrarily by the operator, room environment, or the like.
For example, in a listening environment having the completely same room shape, gain can be readily changed according to the material of a wall (the degree of sound absorption and degree of reflection), and a virtual sound image localization state can be simulated according to a situation wherein the material of a wall is changed variously.
Note that, with the arrangement of the example in
Note that the above-mentioned first example is description regarding the case wherein in addition to a direct wave, three reflected waves are selected, and these normalized HRTFs are convoluted into an audio signal, but in a case wherein there are three or more normalized HRTFs regarding reflected waves to be selected, with the configuration in
Note that, with the example in
Also, in a case wherein the convolution process sections CP0, CP1, CP2, and CP3 are not overlapped mutually, the delay circuits and convolution circuits may be connected in serial while taking the time lengths of the convolution process sections CP0, CP1, CP2, and CP3 into consideration. In this case, if we say that the time lengths of the convolution process sections CP0, CP1, CP2, and CP3 are TP0, TP1, TP2, and TP3, the delay amounts at the delay units 521, 532, and 542 can be regarded as DL1-DL0-TP0, DL2-DL1-TP1, and DL3-DL2-TP2, and accordingly, further can be reduced.
Second Example of Convolution Method (Coefficient Composite Processing,
This second example is employed in a case wherein an HRTF regarding a predetermined listening environment is convoluted. That is to say, in a case wherein a listening environment is determined beforehand, such as the type of listening environment space, the shape and size of a room, the material of a reflection portion (the degree of sound absorption and degree of reflection), or the like, the convolution start points in time of the normalized HRTFs regarding a direct wave and selected reflected wave are determined beforehand, and the attenuation amount (gain) at the time of convoluting each of the normalized HRTFs is also determined beforehand.
For example, HRTFs regarding a direct wave and three reflected waves are taken as an example, as shown in
Therefore, with the second example, as shown in
Here, as shown in
In the case of the second example, a hardware configuration example of a normalized HRTF convolution unit is shown in
A composite normalized HRTF from composite normalized HRTF memory 63 is supplied to the HRTF convolution circuit 62, and is convoluted into an audio signal. The composite normalized HRTF stored in the composite normalized HRTF memory 63 is the composite normalized HRTF described with reference to
The second example involves rewriting of all of the composite normalized HRTFs even in the case of changing a delay amount, gain, or the like, but as shown in
Other Examples of Convolution Method
With both of the above-mentioned first and second examples, a normalized HRTF regarding the corresponding direction measured beforehand is convoluted into an audio signal at each of the convolution process sections CP0, CP1, CP2, and CP3, regarding a direct wave and selected reflected waves.
Note however, the convolution start points in time of HRTFs regarding selected reflected waves, and the convolution process sections CP1, CP2, and CP3 have importance, and accordingly, a signal to be convoluted actually may not be the corresponding HRTF.
Specifically, for example, with the above-mentioned first and second examples, at the convolution process section CP0 for a direct wave a normalized HRTF regarding a direct wave (direct wave direction HRTF) is convoluted, but at the convolution process sections CP1, CP2, and CP3 for reflected waves HRTFs attenuated by multiplying the same direct wave direction HRTF as the convolution process section CP0 by employed gain G1, G2, and G3 may be convoluted in a simplified manner, respectively.
Specifically, in the case of the first example, the same normalized HRTF regarding a direct wave as that in the normalized HRTF memory 513 is stored in the normalized HRTF memory 523, 533, and 543 beforehand. Alternatively, an arrangement may be made wherein the normalized HRTF memory 523, 533, and 534 are omitted, and only the normalized HRTF memory 513 is provided, the normalized HRTF for a direct wave is read out from the relevant normalized HRTF memory 513 to supply this to the gain adjustment units 524, 534, and 544 as well as the gain adjustment unit 514 at each of the convolution process sections CP1, CP2, and CP3.
Further, similarly, with the above-mentioned first and second examples, at the convolution process section CP0 for a direct wave a normalized HRTF regarding a direct wave (direct wave direction HRTF) is convoluted, but at the convolution process sections CP1, CP2, and CP3 for reflected waves an audio signal obtained by delaying an audio signal serving as a convolution target by the corresponding delay amounts DL1, DL2, and DL3 may be convoluted in a simplified manner, respectively. Specifically, holding units are provided, which are configured to hold an audio signal serving as a convolution target by the above-mentioned delay amounts DL1, DL2, and DL3 respectively, and the audio signals held at the holding units are convoluted at the convolution process sections CP1, CP2, and CP3 for reflected waves, respectively.
Example of Acoustic Reproduction System Employing HRTF Convolution Method (
Next, an HRTF convolution method according to an embodiment of the present invention will be described with reference to an example of application to a reproduction device capable of reproduction using virtual sound image localization, by applying the present embodiment to a case wherein a multi-surround audio signal is reproduced by employing headphones.
An example described below is a case wherein the placements of 7.1 channel multi-surround speakers conforming to ITU (International Telecommunication Union)-R are assumed, and an HRTF is convoluted such that the audio components of each channel are subjected to virtual sound image localization on the disposed positions of the 7.1 channel multi-surround speakers.
In
Subsequently, in a range of 60 degrees through 150 degrees on the left and right of the front position C of the listener, a pair of speaker positions LS and LB, and a pair of speaker positions RS and RB are set on the left side and right side. These speaker positions LS and LB, and RS and RB are to be set in symmetrical positions as to the listener. The speaker positions LS and RS are speaker positions of a left lateral channel and right lateral channel, and the speaker positions LB and RB are speaker positions of a left rear channel and right rear channel.
With this acoustic reproduction system example, over-head headphones are employed wherein seven headphone drivers each are disposed as to each of both ears described above with reference to
Accordingly, with this example, as shown in the above
Subsequently, when a 7.1 channel multi-surround audio signals are reproduced acoustically with the over-head headphones of the present example, a selected normalized HRTF is convoluted into the audio signal of each channel of the 7.1 channel multi-surround audio signals such that the 7.1 channel multi-surround audio signals are reproduced acoustically with the direction of each of the speaker positions C, LF, RF, LS, RS, LB, and RB in
Note that in
As shown in
As shown in
Also, seven headphone drivers 90R1, 90R2, 90R3, 90R4, 90R5, 90R6, and 90R7 for the right ear are employed as for a crosstalk channel xLF of the left lateral channel, for the right lateral channel RS, for the right front channel RF, for the right rear channel RB, for the center channel C, for the low-pass effect channel LFE, and for a crosstalk channel xLS of the left lateral channel, respectively.
With the present example, an arrangement is made wherein the audio signal for the center channel C, and the audio signal for the low-pass effect channel LFE are generated in common and supplied to the left and right headphone drivers 90L5 and 90R5, and headphone drivers 90L6 and 90R6, respectively. As described above, with the acoustic reproduction system shown in
As shown in
The HRTF convolution processing unit 74xRF is for the crosstalk channel xRF of the right front channel, HRTF convolution processing unit 74LS is for the left lateral channel LS, HRTF convolution processing unit 74LF is for the left front channel LF, HRTF convolution processing unit 74LB is for the left rear channel LB, HRTF convolution processing unit 74xRS is for the crosstalk channel xRS of the right lateral channel, HRTF convolution processing unit 74LFE is for the low-pass effect channel LFE, HRTF convolution processing unit 74C is for the center channel C, HRTF convolution processing unit 74xLS is for the crosstalk channel xLS of the left lateral channel, HRTF convolution processing unit 74RB is for the right rear channel RB, HRTF convolution processing unit 74RF is for the right front channel RF, HRTF convolution processing unit 74RS is for the right lateral channel RS, and HRTF convolution processing unit 74xLF is for the crosstalk channel xLF of the left lateral channel.
With the present example, the HRTF convolution processing units 74xRF, 74LS, 74LF, 74LB, 74xRS, 74LFE, 74C, 74xLS, 74RB, 74RF, 74RS, and 74xLF have the same hardware configuration such as shown in
In the case of the present example, as shown in
Therefore, the HRTF convolution processing units 74xRF, 74LS, 74LF, 74LB, 74xRS, 74LFE, 74C, 74xLS, 74RB, 74RF, 74RS, and 74xLF are, as shown in
Each of the seven normalized HRTF convolution units 101 through 107 executes convolution processing of a normalized HRTF as to an input audio signal thereof. As the hardware configuration of each of the seven normalized HRTF convolution units 101 through 107, the hardware configuration of the first example in
With each of the HRTF convolution processing units 74xRF, 74LS, 74LF, 74LB, 74xRS, 74LFE, 74C, 74xLS, 74RB, 74RF, 74RS, and 74xLF, each of selected normalized HRTFs to be convoluted (normalized HRTFs regarding a direct wave and reflected waves) to localize a virtual sound image as the reproduction sound field of the 7.1 channel multi surround is convoluted.
Note that, with the present example, the HRTF convolution unit 74LFE does not perform convolution processing of an HRTF, inputs the audio signal of the low-pass effect channel, and outputs this without change.
The output audio signals from the HRTF convolution processing units 74xRF, 74LS, 74LF, 74LB, 74xRS, 74LFE, 74C, 74xLS, 74RB, 74RF, 74RS, and 74xLF are, as shown in
The analog audio signals from the D/A converters 76xRF, 76LS, 76LF, 76LB, 76xRS, 76LFE, 76C, 76xLS, 76RB, 76RF, 76RS, and 76xLF are supplied to current-to-voltage converters 77xRF, 77LS, 77LF, 77LB, 77xRS, 77LFE, 77C, 77xLS, 77RB, 77RF, 77RS, and 77xLF, and are converted into voltage signals from the current signals, respectively.
Subsequently, the audio signals converted into voltage signals from the current-to-voltage converters 77xRF, 77LS, 77LF, 77LB, 77xRS, 77LFE, 77C, 77xLS, 77RB, 77RF, 77RS, and 77xLF are subjected to level adjustment as level adjustment units 78xRF, 78LS, 78LF, 78LB, 78xRS, 78LFE, 78C, 78xLS, 78RB, 78RF, 78RS, and 78xLF, following which are supplied to gain adjustment units 79xRF, 79LS, 79LF, 79LB, 79xRS, 79LFE, 79C, 79xLS, 79RB, 79RF, 79RS, and 79xLF, and are subjected to gain adjustment, respectively.
Subsequently, output audio signals from the gain adjustment units 79xRF, 79LS, 79LF, 79LB, and 79xRS are supplied to the headphone drivers 90L1, 90L2, 90L3, 90L4, and 90L7 for the left ear through amplifiers 80L1, 80L2, 80L3, 80L4, and 80L7, respectively.
Also, output audio signals from the gain adjustment units 79LxLS, 79RB, 79RF, 79RS, and 79xLF are supplied to the headphone drivers 90R7, 90R4, 90R3, 90R2, and 90R1 for the right ear through amplifiers 80R7, 80R4, 80R3, 80R2, and 80R1, respectively.
Also, an output audio signal from the gain adjustment unit 79C is supplied to the headphone driver 90L5 through an amplifier 80L5, and is also supplied to the headphone driver 90R5 through an amplifier 80R5. Further, an output audio signal from the gain adjustment unit 79LFE is supplied to the headphone driver 90L6 through an amplifier 80L6, and is also supplied to the headphone driver 90R6 through an amplifier 80R6.
Example of Normalized HRTF Convolution Start Timing with Acoustic Reproduction System (
Next, description will be made regarding normalized HRTFs to be convoluted at the HRTF convolution processing units 74xRF, 74LS, 74LF, 74LB, 74xRS, 74LFE, 74C, 74xLS, 74RB, 74RF, 74RS, and 74xLF in
For example, convolution of HRTFs will be described when assuming a room of a rectangular parallelepiped shape of vertical×horizontal=4550 mm×3620 mm, and the reproduction acoustic space of 7.1 channel multi surround conforming to ITU-R wherein the distance between the left front speaker position LF and right front speaker position RF is 1600 mm. Note that, with regard to reflected waves, ceiling reflection and floor reflection will be omitted, and only wall reflection will be described here to simplify description.
With the present embodiment, a normalized HRTF regarding a direct wave, normalized HRTF regarding the crosstalk components thereof, normalized HRTF regarding a primary reflected wave, and normalized HRTF regarding the crosstalk components thereof will be convoluted.
First, in order to set the right front speaker position RF to a virtual sound image localization position, the directions of sound waves regarding normalized HRTFs may be employed such as shown in
Specifically, in
Also, RFsR denotes a reflected wave primarily reflected at the right side wall from the position RF, and xRFsR denotes crosstalk to the left channel thereof. Also, RFfR denotes a reflected wave primarily reflected at the front wall from the position RF, and xRFfR denotes crosstalk to the left channel thereof. Also, RFsL denotes a reflected wave primarily reflected at the left wall from the position RF, and xRFsL denotes crosstalk to the left channel thereof. Further, RFbR denotes a reflected wave primarily reflected at the rear wall from the position RF, and xRFbR denotes crosstalk to the left channel thereof.
With regard to each of a direct wave and crosstalk thereof, and reflected wave and crosstalk thereof, normalized HRTFs to be convoluted are normalized HRTFs measured regarding directions where those sound waves have been input to the listener position Pn lastly. Specifically, normalized HRTFs to be convoluted are seven normalized HRTFs to be measured corresponding to the seven headphone drivers as to a sound wave in one direction, respectively. Subsequently, each of the seven normalized HRTFs is convoluted into the audio signal of the channel to be supplied to the corresponding headphone driver.
Subsequently, points in time to start convolution of normalized HRTFs of the direct wave RFd and crosstalk xRFd thereof, and reflected waves RFsR, RFfR, RFsL, and RFbR and crosstalk xRFsR, xRFfR, xRFsL, and xRFbR thereof, as to the audio signal of the right front channel RF are calculated from the path lengths of the sound waves thereof, and the calculation results such as shown in
Subsequently, with regard to the gain of a normalized HRTF to be convoluted, the attenuation amount for a direct wave is set to zero. Also, the attenuation amount for reflected waves is set according to a perceived degree of sound absorption.
Note that
Specifically, each of the normalized HRTFs of the direct wave RFd and crosstalk xRFd thereof, and reflected waves RFsR, RFfR, RFsL, and RFbR and crosstalk xRFsR, xRFfR, xRFsL, and xRFbR thereof is convoluted at the HRTF convolution unit for the channel selected from the above-mentioned HRTF convolution processing units 74xRF, 74LS, 74LF, 74LB, 74xRS, 74LFE, 74C, 74xLS, 74RB, 74RF, 74RS, and 74xLF beforehand.
This can be applied to a relation between normalized HRTFs to be convoluted to set the speaker position of another channel to a virtual sound image localization position, and an audio signal serving as a convolution target as well as the normalized HRTFs to be convoluted to set the right front speaker position RF to a virtual sound image localization position.
Next, in order to set the left front speaker position LF to a virtual sound image localization position, the directions of sound waves regarding normalized HRTFs to be convoluted can be taken as those obtained by moving the drawing shown in
Also, similarly, in order to set the center speaker position C to a virtual sound image localization position, the directions of sound waves regarding normalized HRTFs to be convoluted are such as shown in
Specifically, the directions of sound waves regarding normalized HRTFs to be convoluted are a direct wave Cd, a reflected wave CsR from the right side wall and crosstalk xCsR thereof, and a reflected wave CbR from the rear wall. Only the reflected wave on the right side is illustrated in
Subsequently, normalized HRTFs to be convoluted are determined according to the incident directions of the direct wave and reflected wave, and crosstalk thereof as to the listener position Pn, and the convolution start timing points in time thereof are the same as those shown in
Next, in order to set the right lateral speaker position RS to a virtual sound image localization position, the directions of sound waves regarding normalized HRTFs to be convoluted are such as shown in
Specifically, a direct wave RSd and crosstalk xRSd thereof, a reflected wave RSsR from the right side wall and crosstalk xRSsR thereof, a reflected wave RSfR from the front wall and crosstalk xRSfR thereof, a reflected wave RSsL from the left side wall and crosstalk xRSsL thereof, and a reflected wave RSbR from the rear wall and crosstalk xRSbR thereof are obtained. Subsequently, normalized HRTFs to be convoluted are determined according to the incident directions of these as to the listener position Pn, and the convolution start timing points in time thereof are the same as those shown in
In order to set the left lateral speaker position LS to a virtual sound image localization position, the directions of sound waves regarding normalized HRTFs to be convoluted can be taken as those obtained by moving the drawing shown in
Also, in order to set the right rear speaker position RB to a virtual sound image localization position, the directions of sound waves regarding normalized HRTFs to be convoluted are such as shown in
Specifically, a direct wave RBd and crosstalk xRBd thereof, a reflected wave RBsR from the right side wall and crosstalk xRBsR thereof, a reflected wave RBfR from the front wall and crosstalk xRBfR thereof, a reflected wave RBsL from the left side wall and crosstalk xRBsL thereof, and a reflected wave RBbR from the rear wall and crosstalk xRBbR thereof are obtained. Subsequently, normalized HRTFs to be convoluted are determined according to the incident directions of these as to the listener position Pn, and the convolution start timing points in time thereof are the same as those shown in
In order to set the left rear speaker position LB to a virtual sound image localization position, the directions of sound waves regarding normalized HRTFs to be convoluted can be taken as those obtained by moving the drawing shown in
Description has been made so far regarding the directions of a direct wave and reflected waves into which normalized HRTFs should be convoluted, and the convolution start timing thereof, and an example regarding whether to execute the convolution processing of these normalized HRTFs at which channel of the HRTF convolution processing units 74xRF, 74LS, 74LF, 74LB, 74xRS, 74LFE, 74C, 74xLS, 74RB, 74RF, 74RS, and 74xLF is illustrated in
With the present example,
Though normalized HRTFs regarding a direct wave and reflected waves and crosstalk thereof to be convoluted at the HRTF convolution processing unit 74xLF which is for the crosstalk channel xLF of the left front channel are not shown in the drawing, normalized HRTFs obtained by inverting both sides of the direct wave and reflected waves and crosstalk thereof shown in
Though not shown in the drawing, only the normalized HRTF regarding the direct wave RFd of the right front channel is convoluted at the HRTF convolution processing unit 74RF which is for the right front channel RF as well.
Though not shown in the drawing, with the HRTF convolution processing unit 74RB which is for the right rear channel RB, normalized HRTFs obtained by inverting both sides of the direct wave and reflected waves shown in
Though not shown in the drawing, only the normalized HRTF regarding the direct wave RSd of the right lateral channel is convoluted at the HRTF convolution processing unit 74RS which is for the right lateral channel RS as well.
Though normalized HRTFs regarding a direct wave and reflected waves and crosstalk thereof to be convoluted at the HRTF convolution processing unit 74xLS which is for the crosstalk channel xLS of the left lateral channel are not shown in the drawing, normalized HRTFs obtained by inverting both sides of the direct wave and reflected waves and crosstalk thereof shown in
Note that, as described above, the above description regarding convolution of normalized HRTFs for a direct wave and reflected waves has been made regarding only wall reflection, but may be applied to ceiling reflection and floor reflection completely in the same way.
Specifically,
With regard to these reflected waves and crosstalk thereof as well, normalized HRTFs to be convoluted are normalized HRTFs measured regarding directions where these sound waves have been input to the listener position Pn lastly. Subsequently, the path length regarding each of the reflected waves is calculated, and the convolution start timing of each of the normalized HRTFs is determined. Subsequently, the gain of each of the normalized HRTFs is determined to be attenuation amount according to the degree of sound absorption perceived from the material, surface shape, and the like of the ceiling and floor.
Configuration Example of Second Example of Acoustic Reproduction System (
The acoustic reproduction system shown in
On the other hand, another example described below is a case wherein 7.1 channel multi surround audio signals are reproduced acoustically by common over-head headphones including a headphone driver each for both ears.
Let us say that the example described below employs, as shown in
With the acoustic reproduction system according to the present example, as shown in
Also, the audio signals from the level adjustment units 75LFE, 75C, 75xLS, 75RB, 75RF, 75RS, and 75xLF are supplied to an adder 110R for the right channels to add these.
Subsequently, output signals from the adders 110L and 110R are supplied to D/A converters 111L and 111R, and are converted into analog audio signals, respectively. The analog audio signals from the D/A converters 111L and 111R are supplied to current-to-voltage converters 112L and 112R, and are converted into voltage signals from the current signals, respectively.
Subsequently, the audio signals converted into voltage signals from the current-to-voltage converters 112L and 112R are subjected to level adjustment at level adjustment units 113L and 113R, following which are supplied to gain adjustment units 114L and 114R to subject these to gain adjustment, respectively.
Subsequently, output audio signals from the gain adjustment units 114L and 114R are supplied to a headphone driver 120L for the left ear, and headphone driver 120R for the right ear, through amplifiers 115L and 115R, and are reproduced in an acoustic manner, respectively.
According to the second example of the acoustic reproduction system, a 7.1 channel multi surround sound field can be reproduced well with virtual sound image localization by the headphones including a head driver each for both ears.
With the related art, in the case of performing signal processing using HRTFs, properties of the measurement system were not removed, so the sound quality following the final convolution processing deteriorated unless good-sounding expensive speakers and microphones are used for measurement. On the other hand, with the normalized HRTFs according to the present embodiment, properties of the measurement system can be removed, so HRTF convolution processing with no deterioration in sound quality can be performed even if using a measurement system using inexpensive speakers and microphones without flat properties.
Further, while ideal properties (completely flat) are elusive no matter how expensive and having good properties the speakers and microphones may be, with this embodiment HRTFs more ideal that any properties according to the related art can be obtained.
Also, HRTFs regarding only direct waves, with reflected waves eliminated, are obtained with various directions as to the listener for example as the virtual sound source position, so HRTFs regarding sound waves form each direction can be easily convoluted in the audio signals, and the reproduced sound field when convoluting the HRTFs regarding the sound waves for each direction can be readily verified.
That is to say, as described above, an arrangement may be made wherein, with the virtual sound image localization set to a particular position, not only HRTFs regarding direct waves from the virtual sound image localization position but also HRTFs regarding sound waves from a direction which can be assumed to be reflected waves from the virtual sound image localization position are convoluted, and the reproduced sound field can be verified, so as to perform verification such as which reflected waves of which direction are effective for virtual sound image localization, and so forth.
While the above description has been made regarding a case wherein headphones are primarily the electro-optical conversion unit for performing acoustic reproduction of audio signals to be reproduced, application can be made to applications where speakers are the output system, such as front surround and so forth, taking into consideration the measurement method and processing contents.
The acoustic reproduction system employing the multi surround method has been described so far, but it goes without saying that the above embodiment can be applied to common two-channel stereo.
Also, it goes without saying that the above embodiment can be applied to other multi surround cases such as 5.1 channels, 9.1 channels, and so forth other than 7.1 channels.
Also, the placements of 7.1 channel multi-surround speakers have been described with the placements of ITU-R speakers as an example, but it can be readily understood that the above embodiment can be applied to a case of the placements of speakers recommended by THX Ltd.
It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2008-045597 | Feb 2008 | JP | national |
The present application claims the benefit under 35 U.S.C. §120 as a divisional application of U.S. patent application Ser. No. 12/366,095 filed Feb. 5, 2009 and entitled “HEAD-ELATED TRANSFER FUNCTION CONVOLUTION METHOD AND HEAD-RELATED TRANSFER FUNCTION CONVOLUTION DEVICE,” which contains subject matter related to Japanese Patent Application JP 2008-045597 filed in the Japanese Patent Office on Feb. 27, 2008, the entire contents of both of which are incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4731848 | Kendall et al. | Mar 1988 | A |
| 5181248 | Inanaga et al. | Jan 1993 | A |
| 5440639 | Suzuki et al. | Aug 1995 | A |
| 5467401 | Nagamitsu et al. | Nov 1995 | A |
| 5717767 | Inanaga et al. | Feb 1998 | A |
| 5844816 | Inanaga et al. | Dec 1998 | A |
| 6243476 | Gardner | Jun 2001 | B1 |
| 6501843 | Usui et al. | Dec 2002 | B2 |
| 6738479 | Sibbald et al. | May 2004 | B1 |
| 6801627 | Kobayashi | Oct 2004 | B1 |
| 8503682 | Fukui et al. | Aug 2013 | B2 |
| 8520857 | Fukui et al. | Aug 2013 | B2 |
| 8831231 | Fukui et al. | Sep 2014 | B2 |
| 8873761 | Fukui et al. | Oct 2014 | B2 |
| 20040136538 | Cohen et al. | Jul 2004 | A1 |
| 20050047619 | Murata et al. | Mar 2005 | A1 |
| 20050135643 | Lee et al. | Jun 2005 | A1 |
| 20060045294 | Smyth | Mar 2006 | A1 |
| 20060115091 | Kim et al. | Jun 2006 | A1 |
| 20070160217 | Chun | Jul 2007 | A1 |
| 20080273708 | Sandgren et al. | Nov 2008 | A1 |
| 20090010440 | Jung et al. | Jan 2009 | A1 |
| 20090028345 | Jung et al. | Jan 2009 | A1 |
| 20090043591 | Breebaart et al. | Feb 2009 | A1 |
| 20090060205 | Jung et al. | Mar 2009 | A1 |
| 20090208022 | Fukui et al. | Aug 2009 | A1 |
| 20090214045 | Fukui et al. | Aug 2009 | A1 |
| 20100322428 | Fukui et al. | Dec 2010 | A1 |
| 20110128821 | Choi et al. | Jun 2011 | A1 |
| 20110135098 | Kuhr et al. | Jun 2011 | A1 |
| 20110176684 | Katayama | Jul 2011 | A1 |
| 20110286601 | Fukui et al. | Nov 2011 | A1 |
| 20110305358 | Nishio et al. | Dec 2011 | A1 |
| 20130287235 | Fukui et al. | Oct 2013 | A1 |
| Number | Date | Country |
|---|---|---|
| 1 545 154 | Jun 2005 | EP |
| 2096882 | Sep 2009 | EP |
| 61-245698 | Oct 1986 | JP |
| 3-214897 | Sep 1991 | JP |
| 5-260590 | Oct 1993 | JP |
| 6-147968 | May 1994 | JP |
| 06-165299 | Jun 1994 | JP |
| 06-181600 | Jun 1994 | JP |
| 07-288899 | Oct 1995 | JP |
| 07-312800 | Nov 1995 | JP |
| 8-047078 | Feb 1996 | JP |
| 8-182100 | Jul 1996 | JP |
| 09-037397 | Feb 1997 | JP |
| 09-135499 | May 1997 | JP |
| 09-187100 | Jul 1997 | JP |
| 09-200898 | Jul 1997 | JP |
| 09-284899 | Oct 1997 | JP |
| 10-042399 | Feb 1998 | JP |
| 11-313398 | Nov 1999 | JP |
| 2000-036998 | Feb 2000 | JP |
| 2001-285998 | Oct 2001 | JP |
| 2002-191099 | Jul 2002 | JP |
| 2002-209300 | Jul 2002 | JP |
| 2003-061196 | Feb 2003 | JP |
| 2003-061200 | Feb 2003 | JP |
| 2004-080668 | Mar 2004 | JP |
| 2005-157278 | Jun 2005 | JP |
| 2006-352728 | Dec 2006 | JP |
| 2007-202021 | Aug 2007 | JP |
| 2007-240605 | Sep 2007 | JP |
| 2007-329631 | Dec 2007 | JP |
| 2008-311718 | Dec 2008 | JP |
| WO 9513690 | May 1995 | WO |
| WO 9523493 | Aug 1995 | WO |
| WO 0131973 | May 2001 | WO |
| Entry |
|---|
| Speyer et al., A Model Based Approach for Normalizing the Head Related Transfer Function. IEEE. 1996; 125-28. |
| No Author Listed, Free Sun Power, Advanced Tutorials: Ohm's Law: Watts & Power for Solar Energy, “Power and Voltage Relation”, Feb. 2008, 2 pages, Accessed online http://web.archive.org/web/20080321165756/http://www.freesunpower.com/watts—power.php. |
| Kendall et al. “A Spatial Sound Processor for Loudspearker and Headphone Reproduction” Journal of the Audio Engineering Society, May 30, 1990, vol. 8 No. 27, pp. 209-211, New York, NY. |
| Number | Date | Country | |
|---|---|---|---|
| 20130287235 A1 | Oct 2013 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12366095 | Feb 2009 | US |
| Child | 13927983 | US |
