Patent Grant
11265651
The present application claims priority to Japanese Patent Application JP 2019-105257 filed in the Japan Patent Office on Jun. 5, 2019, the entire content of which is hereby incorporated by reference.
The present invention relates to a vibration output apparatus and a computer-readable, non-transitory storage medium storing a vibration output program.
There have been proposed methods of making a predetermined notification or providing a realistic sound environment by causing the user to perceive a vibration generated by a vibration generator. For example, Japanese Unexamined Patent Application Publication Nos. 2007-65038 and 2008-72165 disclose seat audio systems in which a full-range speaker is installed near the headrest of the seat and a subwoofer is installed in the backrest or seating portion of the seat.
A full-range speaker is able to output a sound in a low-to-high wide band on the basis of an input signal, and the sound outputted from the full-range speaker is able to stimulate the auditory sense of the user. A subwoofer is able to output one or both of a low-band sound and a vibration on the basis of an input signal, and the sound and/or vibration outputted from the subwoofer are able to stimulate the auditory sense and/or the tactile sense of the user.
Examples of a speaker installed in the seat include dynamic speakers using cone paper or the like and linear resonant actuators, such as exciters, which vibrate the contact surface. If a linear resonant actuator is used as a speaker, the single output means is able to output both a sound and a vibration.
Humans perceive a sound through the auditory sense and perceive a vibration through the tactile sense. The dynamic range of the tactile sense of humans differs from that of the auditory sense.
As shown in Table 1 of
Also, even if a sound or vibration satisfies the above conditions, whether a human is able to perceive the sound or vibration depends on the frequency range and the dynamic range (the level difference or relative level difference) of the sound or vibration.
The frequency range of sounds that humans are able to perceive through the auditory sense is about 20 to 20,000 Hz, and humans are not able to auditorily perceive sounds having a frequency exceeding this frequency range. Also, the dynamic range (level difference) of the difference between sounds that humans are able to perceive is about 120 dB. On the other hand, the frequency range of vibrations that humans are able to perceive through the tactile sense is about 10 to 150 Hz, and the dynamic range (level difference) of the difference between vibrations that humans are able to perceive is about 40 dB.
There are large differences between the frequency range and dynamic range (level difference) of sounds that humans are able to perceive through the auditory sense and those of vibrations that humans are able to perceive through the tactile sense.
The frequency and level of an acoustic signal outputted from a sound source in order to output a sound from a speaker is set in a frequency range and level difference corresponding to the dynamic range of the auditory sense. The dynamic range (level difference) of vibrations that humans are able to perceive through the tactile sense is narrower than that of sounds that humans are able to perceive through the auditory sense. For this reason, if a full-range speaker and a subwoofer output a sound and a vibration, respectively, on the basis of the same acoustic signal, the perceptibility of the vibration through the tactile sense of the user would be lower than the perceptibility of the sound through the auditory sense of the user. That is, when a vibration is generated on the basis of an acoustic signal, the perception level at which the user perceives the acoustic signal as a vibration may be lower than the perception level at which the user perceives the acoustic signal as a sound.
The frequency range of vibrations that humans are able to perceive through the tactile sense is lower than that of sounds that humans are able to perceive through the auditory sense. For this reason, even when a vibration is generated on the basis of an acoustic signal, if the acoustic signal does not include a large amount of frequency components that the allows the user to perceive the vibration through the tactile sense, the user would be able to perceive a sound through the auditory sense but have difficulty in perceiving a vibration through the tactile sense.
Theater systems or game systems are currently being provided that provide a realistic appreciation environment by outputting the images of movies, games, or the like such that sounds and vibrations are emphasized. In theater systems or game systems, sounds and vibrations are often generated on the basis of the music or sound effects of movies or games. However, whether the user is able to effectively perceive sounds and vibrations such that the sounds and vibrations are combined together depends on the frequency range and dynamic range (level difference) of the acoustic signals, that is, depends on the acoustic characteristics of the acoustic signals.
An object of the present invention is to provide a vibration output apparatus and a computer-readable, non-transitory storage medium storing a vibration output program that are able to cause the user to effectively perceive vibrations.
A vibration output apparatus according to one aspect of the present invention includes a low-band signal generator configured to generate a low-band signal by extracting low-band frequency components from an acoustic signal, a mid-band signal generator configured to generate a mid-band signal by extracting mid-band frequency components from the acoustic signal, the mid-band signal including samples arranged in a time-series manner and each having amplitude information, a frequency compressor configured to generate a compressed signal by converting frequency components of the mid-band signal into the low-band frequency components of the low-band signal by increasing the total number of samples by a factor of n and thus compressing a frequency of the amplitude information included in the mid-band frequency components of the mid-band signal to 1/n by interpolating amplitude information for interpolation between the adjacent samples having the amplitude information, a low-band envelope signal calculator configured to calculate a low-band envelope signal by performing an integration process on the low-band signal, a vibration signal generator configured to, when a level of the low-band envelope signal is lower than a predetermined threshold level, generate a vibration signal by combining the compressed signal with the low-band signal and to, when the level of the low-band envelope signal is higher than the predetermined threshold level, generate the vibration signal by directly using the low-band signal, and a vibration output unit configured to output a vibration on the basis of the vibration signal generated by the vibration signal generator.
A computer-readable, non-transitory storage medium storing a vibration output program according to another aspect of the present invention is a computer-readable, non-transitory storage medium storing a vibration output program executed by a vibration output apparatus configured to output a vibration from a vibration output unit on the basis of a vibration signal. The vibration output program causes a controller to perform a low-band signal generation process of generating a low-band signal by extracting low-band frequency components from an acoustic signal, a mid-band signal generation process of generating a mid-band signal by extracting mid-band frequency components from the acoustic signal, the mid-band signal including samples arranged in a time-series manner and each having amplitude information, a frequency compression process of generating a compressed signal by converting frequency components of the mid-band signal into the low-band frequency components of the low-band signal by increasing the total number of samples by a factor of n and thus compressing a frequency of the amplitude information included in the mid-band frequency components of the mid-band signal to 1/n by interpolating amplitude information for interpolation between the adjacent samples having the amplitude information, a low-band envelope signal calculation process of calculating a low-band envelope signal by performing an integration process on the low-band signal, a vibration signal generation process of, when a level of the low-band envelope signal is lower than a predetermined threshold level, generating the vibration signal by combining the compressed signal with the low-band signal and, when the level of the low-band envelope signal is higher than the predetermined threshold level, generating the vibration signal by directly using the low-band signal, and a vibration output process of outputting the vibration from the vibration output unit on the basis of the vibration signal generated in the vibration signal generation process.
The term “predetermined threshold level” refers to the minimum level of the low-band envelope signal that when the vibration signal is generated by directly using the low-band signal without combining the compressed signal with the low-band signal and a vibration is outputted from the vibration output unit, allows the user to perceive the vibration as a sufficient magnitude of vibration.
The vibration output apparatus and computer-readable, non-transitory storage medium storing a vibration output program according to an embodiment of the present invention generate the compressed signal by changing the level of the signal consisting of the mid-band frequency components of the acoustic signal to the level of the low-band frequency components of the low-band signal by compressing the frequency and combines the compressed signal with the low-band signal. Thus, the vibration output apparatus and vibration output program are able to reinforce changes in the level of the mid-band frequency components of the acoustic signal as changes in the level of the low-band frequency components that the user is able to perceive as a vibration. Consequently, the vibration output apparatus and vibration output program are able to generate a vibration with strength that the user is able to sufficiently perceive and to cause the user to perceive the vibration with higher perceptibility.
The vibration output apparatus and computer-readable, non-transitory storage medium storing a vibration output program according to the embodiment of the present invention reinforce the signal level of the low-band frequency components by combining the compressed signal with the low-band signal when the level of the low-band signal is lower than the level of the acoustic signal. Thus, the vibration output apparatus and computer-readable, non-transitory storage medium storing a vibration output program are able to properly control the balance between the perceptibility of the sound through the auditory sense of the user and the perceptibility of the vibration through the tactile sense of the user. As a result, the vibration output apparatus and computer-readable, non-transitory storage medium storing a vibration output program are able to increase the togetherness of the sound and vibration and to output the sound and vibration without causing a feeling of strangeness.
A more complete appreciation of the invention and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings.
Now, a vibration output apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings.
[Vibration Output Apparatus]
As shown in
Specifically, as shown in
As shown in
[Sound Source Playback Device]
The sound source playback device 10 is a device that outputs acoustic signals to the vibration output apparatus 1. The sound source playback device 10 is, for example, a CD player, DVD player, or the like that outputs acoustic signals (input signals) indicating images or sounds stored in a CD, DVD, or the like to the vibration output apparatus 1.
The sound source playback device 10 outputs acoustic signals to the downsampler 100 and volume controller 400. Note that the sound source playback device 10 outputs two types of acoustic signals: a right-channel acoustic signal and a left-channel acoustic signal. The acoustic signals for respective channels are inputted to the full-range speakers SP1 and SP2 and subwoofer SW, which then separately output sounds or vibration.
[Volume Setting Device]
The volume setting device 20 is a device that controls the volume level of the acoustic signals outputted by the sound source playback device 10. The volume setting unit 20 is, for example, a typical volume setting control mechanism or the like. By setting the volume using the volume setting device 20, the user is able to control the volume of sounds outputted from the full-range speakers SP1 and SP2.
[Full-Range Speakers SP1 and SP2 and Subwoofer]
The full-range speakers SP1 and SP2 and subwoofer SW are installed in the seat. The full-range speakers SP1 and SP2 are speakers that output high-band and mid-band sounds and are installed, for example, adjacent to the headrest of the seat so as to be bilaterally symmetrical. The subwoofer SW is a speaker that outputs low-band sounds and vibrations and is installed, for example, inside the seating portion of the seat. In the present embodiment, a case will be described in which the vibration output apparatus 1 outputs both low-band sounds and vibrations from the subwoofer SW. However, the subwoofer SW only has to be capable of outputting at least vibrations and does not necessarily have to output both vibrations and low-band sounds. As will be described later, the subwoofer SW outputs vibrations using signals (vibration signals) based on acoustic signals (input signals) outputted by the sound source playback device 10. For this reason, the basic configuration of the subwoofer SW is preferably based on a structure, such as a linear resonant actuator. Note that the full-range speakers SP1 and SP2 and/or subwoofer SW may be included in the configuration of the vibration output apparatus according to the embodiment of the present invention.
[First Amplifier and Second Amplifier]
The first amplifier 31 amplifies the acoustic signals volume-controlled by the volume controller 400 and outputs the amplified acoustic signals to the full-range speakers SP1 and SP2. The second amplifier 32 amplifies signals (vibration signals) upsampled by the upsampler 600 (to be discussed later) and outputs the amplified signals to the subwoofer SW. Note that the first amplifier 31 and/or second amplifier 32 may be included in the configuration of the vibration output apparatus according to the embodiment of the present invention.
[Downsampler and Upsampler]
The downsampler 100 acquires the acoustic signals for two channels outputted by the sound source playback device 10 and downsamples the acoustic signals and thus reduces the processing loads on the low-band processing unit 200, mid-band processing unit 300, and weight generator 500. In this downsampling process, the downsampler 100 downsamples the acoustic signals for two channels to a one-channel acoustic signal.
The downsampler 100 applies a low-pass filter to the acoustic signals and then decimates the sampling frequency. The downsampler 100 according to the present embodiment sets the sampling frequency to 48 kHz and sets the downsampling number (decimation number) to 16. The sampling frequency of the downsampled acoustic signal is 3 kHz. The downsampler 100 according to the present embodiment uses a 256-tap finite impulse response (FIR) filter as a low-pass filter and sets the cutoff frequency to 600 Hz in order to pass the low/mid-band components of the acoustic signals through the low-pass filter.
The upsampler 600 upsamples the acoustic signal (vibration signal) acoustically processed by the low-band processing unit 200, mid-band processing unit 300, and weight generator 500, under set conditions corresponding to the downsampling performed by the downsampler 100. Specifically, the upsampler 600 interpolates zeros between the samples decimated by upsampling, then eliminates the folded components using a low-pass filter similar to that of the downsampler 100, and thus upsamples the resulting signal to a sampling frequency similar to that of the sound source.
[Low-Band Processing Unit and Mid-Band Processing Unit]
The low-band processing unit 200 includes a low-band extractor (low-band signal generator) 210, a first envelope detector (low-band envelope signal calculator) 230, a first edge emphasizer (low-band edge processor) 240, and a first level corrector 250. The mid-band processing unit 300 includes a mid-band extractor (mid-band signal generator) 310, a frequency compressor 320, a second envelope detector (compressed envelope signal calculator) 330, a second edge emphasizer (compression edge processor) 340, and a second level corrector 350. Although different signals are inputted thereto, the first envelope detector 230 and second envelope detector 330 perform the same process. Similarly, although different signals are inputted thereto, the first edge emphasizer 240 and second edge emphasizer 340 perform the same process. Similarly, although different signals are inputted thereto, the first level corrector 250 and second level corrector 350 perform the same process. In the present embodiment, only the first envelope detector 230, first edge emphasizer 240, and first level corrector 250 will be described, and the second envelope detector 330, second edge emphasizer 340, and second level corrector 350 will not be described in detail.
The downsampler 100 outputs the downsampled acoustic signal to the low-band extractor 210 of the low-band processing unit 200 and the mid-band extractor 310 of the mid-band processing unit 300.
[Low-Band Extractor and Mid-Band Extractor]
The low-band extractor 210 extracts low-band frequency components from the downsampled acoustic signal by applying a band-pass filter for the low frequency band thereto. The mid-band extractor 310 extracts mid-band frequency components from the downsampled acoustic signal by applying a band-pass filter for the mid frequency band thereto.
The cutoff frequencies set for the band-pass filter for the low frequency band are values set considering the frequency range of vibrations. As described above, the frequency range of a vibration that humans are able to perceive through the tactile sense is about 10 to 150 Hz. For this reason, cutoff frequencies that fall within a frequency range of about 10 Hz to 150 Hz are set for the band-pass filter for the low frequency band. The low band-side cutoff frequency is set to 30 Hz rather than 10 Hz considering the fact that the user is less likely to perceive a low-frequency vibration. Also, the high band-side cutoff frequency is set to 100 Hz rather than 150 Hz. This is because if a vibration of around 150 Hz is outputted, the user may feel tickled, leading to a reduction in the perceptibility, or the user may feel that the vibration is uncomfortable.
The low-band extractor 210 outputs the acoustic signal (low band-extracted signal, low-band signal) consisting of the low-band frequency components extracted by the band-pass filter for the low frequency band to the first envelope detector 230. The mid-band extractor 310 outputs the acoustic signal (mid-band-extracted signal, mid-band signal) consisting of the mid-band frequency components extracted by the band-pass filter for the mid frequency band to the frequency compressor 320.
[First Envelope Detector and Second Envelope Detector]
The first envelope detector 230 first detects the absolute value of the acoustic signal (low band-extracted signal) consisting of the low-band frequency components extracted by the low-band extractor 210. The first envelope detector 230 then performs an integration process on the absolute value-detected, low band-extracted signal by applying a low-pass filter thereto and thus detects the envelope of the low band-extracted signal. The first envelope detector 230 according to the present embodiment uses a secondary Butterworth filter having a cutoff frequency of 10 Hz as the low-pass filter.
As with the first envelope detector 230, the second envelope detector 330 also detects the absolute value of the received signal and then detects the envelope thereof. Note that the second envelope detector 330 performs the above process on a compressed signal obtained by extracting the mid-band frequency components from the acoustic signal using the mid-band extractor 310 and compressing the frequency of the resulting signal using the frequency compressor 320 (to be discussed later). That is, the second envelope detector 330 detects the absolute value of the compressed signal and performs an integration process on the compressed signal by applying a low-pass filter thereto and thus detects the envelope thereof. The second envelope detector 330 then outputs the envelope-detected signal for the compressed signal (compressed envelope signal) to the second level corrector 350 and second edge emphasizer 340.
[First Edge Emphasizer and Second Edge Emphasizer]
The first edge emphasizer 240 generates an edge-emphasized (edge-processed), low band-extracted signal by emphasizing the edge of the low band-extracted signal received from the low-band extractor 210 using the low-band, envelope-detected signal received from the first envelope detector 230.
The decibel converter 241 generates a decibel-converted, envelope-detected signal by converting (decibel conversion) the amplitude value of the envelope-detected signal received from the first envelope detector 230 into a decibel value thereof.
The differentiator 242 differentiates the decibel-converted, envelope-detected signal generated by the decibel converter 241 by applying a high-pass filter thereto. The rising response speed and falling response speed of the differentiated signal and the respective levels can be set by controlling the cutoff frequency and gain of the high-pass filter. The differentiator 242 according to the present embodiment uses a primary Butterworth filter as the high-pass filter.
The rising-edge detector 243 detects the rising edge by extracting only a signal having a level equal to or higher than zero on the basis of the level of the differentiated, envelope-detected signal generated by the differentiator 242 and generates a rising-edge signal. The falling-edge detector 244 detects the falling edge by extracting only a signal having a level equal to or lower than zero on the basis of the level of the differentiated, envelope-detected signal generated by the differentiator 242 and generates a falling-edge signal.
The adder 245 generates an edge-detected signal whose level rises and falls in accordance with the rising and falling of the envelope-detected signal by combining (adding up) the rising-edge signal generated by the rising-edge detector 243 and the falling-edge signal generated by the falling-edge detector 244. The linear converter 246 generates an edge-emphasized signal by linearly converting the edge-detected signal generated by the adder 245. The multiplier 247 generates an edge-emphasized, low band-extracted signal by multiplying the acoustic signal (low band-extracted signal) consisting of the low-band frequency components extracted by the low-band extractor 210, by the edge-emphasized signal.
In the edge-emphasized, low band-extracted signal shown in
By emphasizing the edge in this manner, the low band-extracted signal having relatively uniform amplitude characteristics as shown in
Since the auditory sense and tactile sense have different dynamic ranges, the user perceives changes in the vibration level of a vibration to a degree much different from the degree to which the user perceives changes in the signal level of a sound. For this reason, unless changes in the level (changes in the amplitude) of the acoustic signal-based vibration signal to be perceived as a vibration are increased compared to changes in the level (changes in the amplitude) of the acoustic signal to be perceived as a sound, the user may perceive the vibration weakly. In this respect, emphasizing the edge of the signal for outputting the vibration is very effective in causing the user to perceive the vibration with higher perceptibility.
The first edge emphasizer 240 outputs the generated edge-emphasized, low band-extracted signal to the first level corrector 250.
As with the first edge emphasizer 240, the second edge emphasizer 340 includes a decibel converter, a differentiator, a rising-edge detector, a falling-edge detector, an adder, a linear converter, and a multiplier. The second edge emphasizer 340 receives the compressed signal obtained by compressing the frequency of the mid-band-extracted signal using the frequency compressor 320. The second edge emphasizer 340 generates an edge-emphasized, compressed signal by emphasizing the edge of the compressed signal using the envelope-detected signal for the compressed signal received from the first envelope detector 330. The second edge emphasizer 340 then outputs the generated edge-emphasized, compressed signal to the second level corrector 350.
[First Level Corrector and Second Level Corrector]
The first level corrector 250 corrects the level of the edge-emphasized, low band-extracted signal received from the first edge emphasizer 240.
The decibel converter 251 generates a decibel-converted, envelope-detected signal by converting the amplitude value of the low-band, envelope-detected signal received from the first envelope detector 230 into a decibel value thereof. The maximum value detector 252 detects the maximum value of the decibel-converted, envelope-detected signal generated by the decibel converter 251 by shifting the decibel-converted, envelope-detected signal corresponding to one frame (e.g., 128 samples for the decibel converter 251; see Table 2 of
The hold time controller 253 holds the maximum value detected by the maximum value detector 252 by a predetermined time. For example, the hold time controller 253 holds the maximum value by 0.5 sec (see Table 2 of
The level converter 254 includes an input/output conversion table. The level converter 254 converts the level of the signal (low-band, maximum value-held signal, low-band maximum value signal) received from the hold time controller 253 on the basis of the input/output conversion table and outputs the resulting signal to the attack/release time controller 255.
When the level converter 254 receives an input signal having a level (input level) of −70 to 0 dB, it converts the input signal into an output signal having a level (output level) of 50 to −20 dB on the basis of the input/output conversion table so that the level of the signal to be outputted to the attack/release time controller 255 is inversely proportional to the level of the input signal. That is, as the level of the input signal is increased from −70 to 0 dB, the level of the output signal is reduced from 50 to −20 dB.
When the level converter 254 receives an input signal having a level (input level) of −80 to −70 dB, it converts the input signal into an output signal having a level (output level) of 0 to 50 dB so that the level of the signal to be outputted to the attack/release time controller 255 is proportional to the level of the input signal. That is, as the level of the input signal is increased from −80 to −70 dB, the level of the output signal is increased from 0 to 50 dB.
When the level converter 254 receives an input signal having a level (input level) equal to or lower than −80 dB, it converts the input signal into an output signal having a level (output level) of 0 dB and outputs it to the attack/release time controller 255.
The attack/release time controller 255 performs response control corresponding to a predetermined attack time and a predetermined release time (attack/release time control) on the signal acquired from the level converter 254. As used herein, the term “attack time control” refers to a process of controlling the time required for the signal level to rise, and the term “release time control” refers to a process of controlling the time required for the signal level to fall to the minimum level. For example, the attack/release time controller 255 sets the attack time to, for example, 0.5 sec and sets the release time to, for example, 10 sec (see Table 2 of
The attack/release time controller 255 also uses a primary Butterworth low-pass filter to perform attack time control and release time control. The attack time and release time are set by setting the filter coefficients of the Butterworth low-pass filter. The attack/release time controller 255 according to the present embodiment sets the attack time to 0.5 sec by setting the cutoff frequency to 2 Hz. Also, the attack/release time controller 255 sets the release time to 10 sec by setting the cutoff frequency to 0.1 Hz.
As described above, the attack/release time controller 255 receives the maximum value-held signal from the hold time controller 253. The attack/release time controller 255 determines whether the level of the maximum value-held signal (low-band, maximum value-held signal, low-band maximum value signal) is a preset control minimum value (control determination). For example, the attack/release time controller 255 sets the control minimum value to −40 dB (see Table 2 of
If the level of the low-band, maximum value-held signal is equal to or lower than −40 dB (control minimum value), the attack/release time controller 255 stops the attack time control and release time control. Thus, the attack/release time controller 255 is able to control the extent to which the first level corrector 250 corrects the low band-extracted signal whose amplitude varies greatly and thus to prevent the correction made by the first level corrector 250 from becoming overcontrol. The attack/release time controller 255 then outputs, to the linear converter 256, the signal that has been subjected to attack/release time control or the signal that has yet to be subjected to attack/release time control on the basis of a determination that the level of the maximum value-held signal is equal to or smaller than the control minimum value.
The linear converter 256 converts the signal subjected to attack/release time control by the attack/release time controller 255 into a linear signal and outputs the linear signal to the smoothing filter unit 257. The smoothing filter unit 257 applies a smoothing filter to the signal received from the linear converter 256. Specifically, the smoothing filter unit 257 smooths the signal using the smoothing filter such that the signal (control signal) updated at the maximum value detection interval of the maximum value detector 252 is updated on a sample-by-sample basis. The smoothing filter unit 257 then outputs the smoothed signal to the multiplier 258.
The multiplier 258 corrects the level of the low band-extracted signal by multiplying the edge-emphasized, low band-extracted signal received from the first edge emphasizer 240 by the smoothed signal received from the smoothing filter unit 257. The multiplier 258 then outputs the level-corrected, low band-extracted signal to the weight generator 500 as a low-band, level-corrected signal.
Use of the low-band, level-corrected signal allows for increasing the level of the signal used by the subwoofer SW to output a vibration. Thus, even if the acoustic signal received from the sound source playback device 10 is a signal for causing the user to perceive music through the auditory sense and the frequency range and dynamic ranges (level difference) of a vibration perceivable through the tactile sense are lower than those of a sound perceivable through the auditory sense as shown in Table 1 of
As with the first level corrector 250, the second level corrector 350 also includes a decibel converter, a maximum value detector, a hold time controller, a level converter, an attack/release time controller, a linear converter, a smoothing filter unit, and a multiplier. The second level corrector 350 corrects the level of the edge-emphasized compressed signal received from the second edge emphasizer 340.
The first level corrector 250 outputs the generated low-band, level-corrected signal and low-band, maximum value-held signal (low-band maximum value signal) to the weight generator 500. The second level corrector 350 outputs the generated compressed, level-corrected signal and maximum value-held signal for the compressed signal (compressed maximum value signal) to the weight generator 500.
[Frequency Compressor]
Next, the frequency compressor 320 will be described. The frequency compressor 320 is not included in the low-band processing unit 200 and is included only in the mid-band processing unit 300. As shown in
The sample extractor 321 extracts a predetermined number of samples from the mid-band-extracted signal generated by the mid-band extractor 310 in a predetermined cycle. The sample extractor 321 according to the present embodiment sets the predetermined number to 112 samples as shown in Table 3 of
(2) of
The upsampler 322 upsamples the sample-extracted signal received from the sample extractor 321. The upsampler 322 performs, on the sample-extracted signal, an upsampling process different from a typical sampling rate conversion process in which a signal sampled at one sampling frequency is converted into a signal sampled at another sampling frequency.
In the typical sampling rate conversion process, the amount of data per unit time is changed without changing the temporal length of the signal (the temporal amount of data). In a typical upsampling process, the amount of data per unit time is increased.
On the other hand, in the upsampling process realized by the upsampler 322, the amount of data of the signal is temporally increased by increasing the amount of data while maintaining the amount of data per unit time rather than changing the amount of data per unit time.
In (2) of
The upsampler 322 then upsampler the extracted 112 samples into a signal consisting of a total of 224 samples by interpolating amplitude information of zero (amplitude information for interpolation) between the extraction points of the 112 samples.
When the number of samples forming one waveform is doubled, the wavelength of the waveform is increased, resulting in a reduction in the frequency (a shift to a lower frequency range). That is, the frequency is compressed. As a result, the frequency of the upsampled signal is lower than the frequency of the yet-to-be-upsampled signal. Typically, when the number of samples of a mid-band-extracted signal is multiplied by n, the frequency of the amplitude information included in the mid-band-extracted signal is compressed to 1/n and thus the frequency components of the mid-band-extracted signal are converted into low-band frequency components. The upsampler 322 then outputs the upsampled signal (compressed signal) to the weighting unit 323.
The weighting unit 323 controls changes in the amplitude of a predetermined number of starting samples and a predetermined number of ending samples of the upsampled signal.
As shown by portions corresponding to the weight A in (4) of
As shown in (4) and (5) of
The band limiter 325 limits the band of the overlapped signal. The band limiter 325 sets a low band-side cutoff frequency of 30 Hz and a high band-side cutoff frequency of 120 Hz for a quaternary Butterworth filter and limits the band using this filter. Amplitude information of zero (amplitude information for interpolation) is interpolated between the samples extracted from the mid-band-extracted signal in the sample extraction process by the sample extractor 321 and the upsampling process by the upsampler 322. For this reason, the signal may become discontinuous, and unwanted components, such as harmonics, may be generated, as shown in
Note that the compression ratio used by the frequency compressor 320 in the frequency compression process is not limited to 1/2. The compression ratio may be changed by changing the upsampling number. For example, as shown in Table 4 of
[Volume Controller]
The volume controller 400 control or changes the level of the acoustic signal inputted to the vibration output apparatus 1 from the sound source playback device 10 to the signal level of a sound and the signal level of a vibration in accordance with the volume level set by the volume setting device 20. The volume controller 400 then outputs the acoustic signal having the signal level of the sound to the first amplifier 31 and outputs the signal having the signal level of the vibration (volume-controlled signal) to the weight generator 500.
When the volume level (input level) set by the volume setting device 20 is equal to or smaller than −50 dB, the output level of the vibration is increased or reduced in a range equal to or smaller than −50 dB in proportional to the set volume level. When the volume level (input level) set by the volume setting device 20 is −50 to −40 dB, the output level of the vibration is increased or reduced in a range of −50 to −10 dB in accordance with an increase or reduction in the set volume level. When the volume level (input level) set by the volume setting device 20 is −40 to −0 dB, the output level of the vibration is increased or reduced in a range of −10 to 0 dB in accordance with an increase or reduction in the set volume level.
When the volume level set by the volume setting device 20 is increased or reduced in a range of −40 to 0 dB as shown in
The reason why the output level of the vibration (vibration level) is controlled to the range of −10 to 0 dB is that when the volume level of the sound perceived through the auditory sense is changed in a range of −40 to 0 dB, the vibration output level (vibration level) at which the user is able to favorably perceive the signal level change as a vibration is −10 to 0 dB. When the volume level of the sound is low, for example, equal to or lower than −40 dB, the vibration output level (vibration level) is sharply reduced to less than −10 dB so that the user does not perceive the vibration much.
[Weight Generator]
The weight generator 500 generates a vibration signal to be outputted to the subwoofer SW on the basis of the low-band, level-corrected signal generated by the first level corrector 250, the low-band, maximum value-held signal (low-band maximum value signal), the compressed, level-corrected signal generated by the second level corrector 350, the maximum value-held signal for the compressed signal (compression maximum value signal), and the volume-controlled signal generated by the volume controller 400.
When the value Lm (dB) of the low-band maximum value signal is equal to or smaller than a determination threshold α, the amount of weight W is calculated by the following Formula 1:
W=10(((Mm−Lm)·Wc)/20) Formula 1
where Mm represents the value (dB) of the compressed maximum value signal and We represents a weight coefficient.
When β2<W<β1 where β1 represents the upper limit of the amount of weight and β2 represents the lower limit of the amount of weight, W calculated by Formula 1 is determined as the amount of weight.
When W≥β1, β1 is determined as the amount of weight Was shown in Formula 2:
W=β1 Formula 2
When W≤β2, β2 is determined as the amount of weight W as shown in Formula 3:
W=β2 Formula 3
When Lm>α, zero (0) is determined as the amount of weight W as shown in Formula 4:
W=0 Formula 4
If the value Lm of the low-band maximum value signal is equal to or smaller than the determination threshold α (YES in S.1), the weight amount generator 510 calculates the amount of weight W on the basis of Formula 1 (S.3). The weight amount generator 510 then determines whether the amount of weight W is equal to or greater than the upper limit β1 of the amount of weight (S.4). If the amount of weight W is equal to or greater than the upper limit β1 (YES in S.4), the weight amount generator 510 sets the amount of weight W to β1 (S.5), ending the weight amount W determination process.
If the amount of weight W is not equal to or greater than the upper limit 131 (No in S.4), the weight amount generator 510 determines whether the amount of weight W is equal to or smaller than the lower limit β2 of the amount of weight (S.6). If the amount of weight W is equal to or smaller than the lower limit β2 (Yes in S.6), the weight amount generator 510 sets the amount of weight W to β2 (S.7), ending the weight amount W determination process. If the amount of weight W is not equal to or smaller than the lower limit β2 (No in S.6), the weight amount generator 510 determines the value calculated by Formula 1 as the amount of weight W, ending the weight amount W determination process.
The weight amount generator 510 also smooths changes in the value of the continuously generated amount of weight W (weight amount signal). The weight amount generator 510 then outputs the generated amount of weight W (weight amount signal) to the first multiplier 520.
The first multiplier 520 multiplies the compressed, level-corrected signal by the amount of weight W (weight amount signal) acquired from the weight amount generator 510. The first multiplier 520 then outputs the multiplied signal to the adder 530. The adder 530 combines (adds up) the multiplied signal acquired from the first multiplier 520 and the low-band, level-corrected signal. The adder 530 then outputs the added-up signal to the second multiplier 540. The second multiplier 540 generates a vibration signal by multiplying the signal acquired from the adder 530 (the signal added up by the adder 530) by the volume-controlled signal acquired from the volume controller 400 and thus controlling the vibration level. The second multiplier 540 then outputs the generated vibration signal to the upsampler 600.
For example, the weight generator 500 according to the present embodiment generates a vibration signal by setting the weight coefficient Wc to 0.7, setting the determination threshold α to −24 dB, setting the weight amount upper limit β1 to 16 dB, and setting the weight amount lower limit β2 to 0 dB.
As shown in Formula 1, the amount of weight W is calculated using the value obtained by subtracting the value Lm of the low-band maximum value signal from the value Mm of the compressed maximum value signal. For this reason, if there is a large difference between the value Lm of the low-band maximum value signal and the value Mm of the compressed maximum value signal, the value of the amount of weight W of the weight amount signal is increased, as shown in
On the other hand, if it can be determined that the acoustic signal includes low-band frequency components having a sufficient signal level, there is no need to increase the amount of weight W. The value of the determination threshold α is used as a criterion for determining whether the acoustic signal includes low-band frequency components having a sufficient signal level. The weight amount generator 510 according to the present embodiment sets the determination threshold α to −24 dB. Thus, if the value Lm of the low-band maximum value signal is greater than −24 dB, the weight amount generator 510 determines that the acoustic signal includes low-band frequency components having a sufficient signal level.
The value Lm of the low-band maximum value signal is the value of the signal obtained by holding the maximum value of the low-band, envelope-detected signal and represents the value of the signal level in the low frequency range set by the low-band extractor 210 considering the dynamic range of the vibration. Accordingly, if the value Lm of the low-band maximum value signal is greater than the determination threshold α and it is determined that the acoustic signal includes low-band frequency components having a sufficient signal level, zero (0) is determined as the amount of weight W (see Formula 4). Thus, the compressed, level-corrected signal is multiplied by the value of zero (0), and the compressed, level-corrected signal inputted to the adder 530 is substantially lost. As a result, a vibration signal is generated from only the low-band, level-corrected signal.
Even if the vibration signal is generated from the low-band, level-corrected signal without using the compressed, level-corrected signal, the subwoofer SW is able to output a magnitude of vibration that the user is able to perceive, since the acoustic signal includes the low-band frequency components having a sufficient signal level. The low-band, level-corrected signal is the level-corrected signal generated by the level converter 254 of the first level corrector 250. For this reason, even if the vibration signal is generated from the low-band, level-corrected signal without using the compressed, level-corrected signal, the weight generator 500 is able to ensure a sufficient vibration level.
If the value Lm of the low-band maximum value signal is equal to or smaller than the determination threshold α and the acoustic signal does not include low-band frequency components having a sufficient signal level, the weight generator 500 determines the amount of weight W on the basis of Formulas 1 to 3. By combining (adding up) the compressed, level-corrected signal multiplied by the determined amount of weight W and the low-band, level-corrected signal using the adder 530, the weight generator 500 is able to generate a vibration having a sufficient magnitude including not only the signal level of the low-band frequency components but also the signal level of the mid-band frequency components. Thus, the subwoofer SW is able to output a magnitude of vibration that the user is able to perceive.
The determination threshold α is used as a criterion for determining whether the acoustic signal includes low-band frequency components having a sufficient signal level. Accordingly, the determination threshold α represents the minimum signal level that allows the weight generator 500 to generate a vibration signal having a sufficient magnitude including only the signal level of low-band frequency components without having to add the signal level of mid-band frequency components.
A specific example of a method for determining the determination threshold α involves previously empirically setting the values of multiple determination thresholds a and determining which determination threshold allows the user seated on the seat to perceive a sufficient magnitude of vibration from the subwoofer SW. Such a method is able to determine a determination threshold α most suitable for the environment in which the vibration output apparatus 1 is installed.
If the determination threshold α is set to a value greater than the most suitable value (signal level) (for example, the determination threshold α is set to −12 dB), mid-band frequency components may be added up in the acoustic signal, although the acoustic signal includes a sufficient level of low-band frequency components. Consequently, a vibration signal having an excessively high level (vibration level) may be generated.
On the other hand, if the determination threshold α is set to a value smaller than the most suitable value (signal level) (for example, the determination threshold α is set to −48 dB), mid-band frequency components may not be added up in the acoustic signal, although the acoustic signal does not include low-band frequency components having a sufficient level. Also, the level converter 254 of the first level corrector 250 may not sufficiently correct the level when performing level conversion. In this case, a vibration signal having a low level (vibration level) may be generated and thus the user may not be able to perceive the vibration.
For these reasons, it is important to set the determination threshold α to a value that allows the user to reliably and sufficiently perceive the vibration and that is most suitable for the operating environment of the vibration output apparatus 1.
The weight generator 500 outputs the generated vibration signal to the upsampler 600. As described above, the upsampler 600 upsamples the vibration signal acquired from the weight generator 500 and outputs the upsampled vibration signal to the second amplifier 32. As described above, the second amplifier 32 amplifies the signal acquired from the upsampler 600 and outputs the amplified vibration signal to the sub-woofer SW, which then outputs (generates) a vibration.
As described above, the frequency range or dynamic range (signal level difference) of a vibration signal that the user is able to perceive as a vibration through the tactile sense tends to be narrower than the frequency range or dynamic range (signal level difference) of an acoustic signal that the user is able to perceive as a sound through the auditory sense.
The volume controller 400 of the vibration output apparatus 1 performs level control by narrowing the variation width (−10 to 0 dB) of the vibration level of the vibration signal outputted to the subwoofer SW through the second amplifier 32 compared to the variation width (−40 to 0 dB) of the level of the acoustic signal outputted to the full-range speakers SP1 and SP2 through the first amplifier 31, considering the difference in dynamic range (signal level difference) between the vibration and sound. Thus, the vibration output apparatus 1 allows the user to perceive a vibration in a dynamic range corresponding to the signal level difference of a sound perceived by the user, as well as is able to enhance the togetherness of the sound and vibration and to output the sound and vibration without causing a feeling of strangeness.
The frequency compressor 320 of the vibration output apparatus 1 converts the frequency components of 100 Hz or more, which the user is able to perceive as a sound but has difficulty in perceiving as a vibration, of the frequency components of the acoustic signal inputted to the vibration output apparatus 1 into a signal of 100 Hz or less, which the user easily perceives as a vibration by compressing (shifting) the frequency considering the difference between the frequency range of an acoustic signal that the user is able to perceive as a sound through the auditory sense and the frequency range of a vibration signal that the user is able to perceive as a vibration through the tactile sense. Thus, the frequency compressor 320 converts changes in the signal level in the frequency range of a sound that the user is able to perceive through the auditory sense into changes in the vibration level in the frequency range of a vibration that the user is able to perceive through the tactile sense. As a result, the user is able to perceive a vibration through the tactile sense with effective realism similar to the realism of a sound that the user is able to perceive through the auditory sense.
When combining the compressed, level-corrected signal with the low-band, level-corrected signal, the weight generator 500 of the vibration output apparatus 1 determines the amount of weight by which the compressed, level-corrected signal is multiplied and controls the level of the compressed, level-corrected signal to be added to the low-band, level-corrected signal in accordance with the signal level of the low-band frequency components of the frequency components of the acoustic signal, or the like.
Specifically, the amount of weight W is calculated and determined on the basis of Formulas 1 to 4. For example, if there is a large difference between the value Mm of the compressed maximum value signal and the value Lm of the low-band maximum value signal (Mm−Lm is large) as shown in Formula 1, it can be determined that the signal level of the low-band frequency components that the user is able to perceive as a vibration is lower than the signal level of the mid-band frequency components that the user is less likely to perceive as a vibration. In this case, the perceptibility of a vibration through the tactile sense of the user may be lower than the perceptibility of a sound based on the acoustic signal through the auditory sense of the user. For this reason, the frequency compressor 320 compresses the mid-band frequency components to low-band frequency components and thus shifts the signal level of the mid-band frequency components to the signal level of the low-band frequency components. Thus, the weight generator 500 is able to control changes in the signal level in the wider frequency range so that the user is able to perceive the signal-level changes as a vibration.
If there is a small difference between the value Mm of the compressed maximum value signal and the value Lm of the low-band maximum value signal (Mm−Lm is small), it can be determined that the acoustic signal includes low-band frequency components having a sufficient level. In this case, the weight generator 500 sets the amount of weight W to a lower value and thus is able to prevent the signal level of the mid-band frequency components from being excessively included in the low-band frequency components. Thus, the weight generator 500 is able to prevent the perceptibility and realism of the vibration from becoming excessively greater than those of the sound and thus to realize the togetherness of the sound and vibration.
If the value Lm of the low-band maximum value signal is greater than the predetermined threshold α (Lm≥α), the weight generator 500 sets the amount of weight W to 0 (zero). Thus, the weight generator 500 is able to prevent the frequency-compressed mid-band frequency components from being added to the low-band frequency components and thus to prevent the signal level of the low-band frequency components from being excessively increased.
If the value Lm of the low-band maximum value signal is equal to or smaller than the predetermined threshold α (Lm≤α), the weight generator 500 previously sets the upper limit β1 and lower limit β2 of the amount of weight and controls the amount of weight W so that the amount of weight W falls within a range between the upper limit β1 and lower limit β2. Thus, the weight generator 500 is able to add a proper amount of signal components of the frequency-compressed signal to the low-band frequency components and thus to properly control the vibration level without impairing the togetherness of the sound and vibration.
The edge emphasizers 240 and 340 of the vibration output apparatus 1 are able to accentuate the vibration signal by performing a rising emphasis process of emphasizing the rising of the vibration level when a vibration is outputted and a falling emphasis process of quickly performing falling of the vibration level when the vibration is reduced. Thus, the subwoofer SW is able to output an accentuated vibration.
The perceptibility of a vibration that the user perceives through the tactile sense (the level change perceptibility) tends to be lower than the perceptibility of a sound that the user perceives through the auditory sense (the level change perceptibility). The edge emphasizers 240 and 340 are able to improve the perceptibility of a vibration that the user perceives through the tactile sense, by emphasizing the rising and falling of the vibration level and thus to compensate for the difference with the perceptibility of a sound that the user perceives through the auditory sense and further improve the togetherness of the sound and vibration.
The attack/release time controllers of the first level corrector 250 and second level corrector 350 perform attack time control and release time control on the signals subjected to the vibration level rising/falling emphasis process by the edge emphasizers 240 and 340. However, if the level of the maximum value-held signal is equal to or smaller than −40 dB (control minimum value), the attack/release time controllers stop the attack time control and release time control. Thus, the attack/release time controllers are able to control the extent to which the first level corrector 250 and second level corrector 350 correct the acoustic signals whose amplitude greatly varies (low band-extracted signal, compressed signal) and thus to prevent the correction made by the first level corrector 250 and second level corrector 350 from becoming overcontrol.
The vibration output apparatus and computer-readable, non-transitory storage medium storing a vibration output program according to the embodiment of the present invention have been described in detail using the vibration output apparatus 1. However, the vibration output apparatus and computer-readable, non-transitory storage medium storing a vibration output program according to the embodiment of the present invention is not limited to the configuration or the like of the vibration output apparatus 1 and may be configured otherwise.
For example, the set values specifically described in the above embodiment, for example, the numerical values shown in Tables 2 to 4 are only illustrative and are not limiting. The settings described in Formulas 1 to 4 are also only illustrative and are not limiting.
While, in the above embodiment, both the edge emphasizers 240 and 340 of the vibration output apparatus 1 perform the vibration level rising/falling emphasis processes, the vibration output apparatus and computer-readable, non-transitory storage medium storing a vibration output program according to the embodiment of the present invention need not necessarily include both the edge emphasizers 240 and 340 and one or both of the edge emphasizers may be omitted as necessary.
The edge emphasizers 240 and 340 need not necessarily perform both rising/falling emphasis processes and may perform one of the rising/falling emphasis processes. Performing at least one of the rising/falling emphasis processes allows for accentuating a vibration signal and thus outputting an accentuated vibration from the subwoofer SW.
| Number | Date | Country | Kind |
|---|---|---|---|
| JP2019-105257 | Jun 2019 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 11039234 | Linjama | Jun 2021 | B2 |
| 20070237342 | Agranat | Oct 2007 | A1 |
| 20190342662 | Fukue et al. | Nov 2019 | A1 |
| Number | Date | Country |
|---|---|---|
| 3244628 | Nov 2017 | EP |
| 3310071 | Apr 2018 | EP |
| 2007-065038 | Mar 2007 | JP |
| 2008-072165 | Mar 2008 | JP |
| WO 2017031500 | Feb 2017 | WO |
| Entry |
|---|
| Extended European Search Report for corresponding EP Application No. 20178115.0-1207, dated Sep. 7, 2020. |
| Number | Date | Country | |
|---|---|---|---|
| 20200389730 A1 | Dec 2020 | US |
