Patent Grant
7668319
The present invention relates to signal processing systems, signal processing apparatuses and methods, recording media, and programs, and in particular, to a signal processing system, a signal processing apparatus and method, a recording medium, and a program in which high frequency components of a narrow-range signal whose high frequency components are suppressed are restored with good accuracy.
Regarding technologies for expanding an audio signal having a narrow frequency band into a broadband, various propositions have been made to date.
For example, Japanese Unexamined Patent Application Publication No. 7-36490 (hereinafter referred to as Document 1) discloses a method in which, based on an input audio signal spectrum, lost harmonic components are generated by an arithmetic operation and are added.
Also, in a paper from Saitama University to the Acoustical Society of Japan, a method has been announced in which a phonic sound source waveform is generated by an autocorrelation arithmetic operation on an input signal and is used as high frequency components.
In addition, Japanese Unexamined Patent Application Publication No. 11-126097 and Japanese Unexamined Patent Application Publication No. 11-68515 (hereinafter referred to as Documents 3) discloses a sample-value interpolating method in the case of performing D/A conversion at a higher sampling frequency compared with an input signal.
Moreover, Japanese Unexamined Patent Application Publication No. 6-118995 (hereinafter referred to as Document 4) discloses a method that restores a broad-range audio signal by establishing correlation between a broad-range-audio-signal vector-quantization codebook and a narrow-range-audio-signal vector-quantization codebook.
The technology in Document 1 is such that, in a relatively-broad-range audio signal as in MP3 (Moving Picture Experts Group-1, Audio Layer-3) or the like, a high frequency signal which is lost in the process of compressing the audio signal is predicted and added, and its application is difficult, for example, when the bandwidth of an extremely-narrowed-range signal, such as a telephone audio range, is doubled.
Also, the technology in Document 2 is a process only on voiced sound and has a problem in that unvoiced sound (consonant portion) cannot be restored.
The technology in Documents 3 is such an extension of an interpolating filter that, in the case of sample value interpolating processing for increasing a sampling frequency, the level of the interpolating sample value is set to a value which is considered as appropriate after observing circumstances in which several adjacent samples change, and thus does not restore lost frequency components.
The technology in Document 4 is considered as assuming a vocal tract model as an input signal since an LPC (Liner Prediction Coefficient) analysis and synthesizing processing are used as a processing core. Accordingly, its application to musical sound is difficult.
The present invention is made in view of the above circumstances, and is intended to enable accurate restoration of high frequency components of a narrow-range signal having suppressed high frequency components.
In a signal processing system of the present invention, a first signal processing apparatus comprises: a first-signal input means for inputting a first signal having a broad range of frequency components; a second-signal input means for inputting a second signal in the first signal, the second signal having a narrow range having suppressed high frequency components; an extracting means for extracting high frequency components from the first signal input by the first-signal input means; a first determining means for determining each first class code based on the second signal input by the second-signal input means; and a storage means for storing, for the first class code determined by the first determining means, the high frequency components extracted by the extracting means; and a second signal processing apparatus comprises: a third-signal input means for inputting a third signal having a narrow range having suppressed high frequency components; a second determining means for determining a second class code based on the third signal input by the third-signal input means; a combining means for combining the high frequency components stored in the storage means, which correspond to the second class code determined by the second determining means, and the third signal input by the third-signal input means; and an output means for outputting the combined signal generated by the combining means.
In a signal processing method of the present invention for a signal processing system, a first signal processing apparatus includes: a first-signal input control step for controlling input of a first signal having a broad range of frequency components; a second-signal input control step for controlling input of a second signal in the first signal, the second signal having a narrow range having suppressed high frequency components; an extracting step for extracting high frequency components from the first signal whose input is controlled by processing in the first-signal input control step; a first determining step for determining each first class code based on the second signal whose input is controlled by processing in the second-signal input control step; and a storage step for storing, for the first class code, the high frequency components extracted by processing in the extracting step; and a second signal processing apparatus includes: a third-signal input control step for controlling input of a third signal having a narrow range having suppressed high frequency components; a second determining step for determining a second class code based on the third signal whose input is controlled by processing in the second-signal input control step; a combining step for combining the high frequency components stored by processing in the storage step, which correspond to the second class code determined by processing in the second determining step, and the third signal whose input is controlled by processing in the third-signal input control step; and an output control step for controlling output of the combined signal generated by processing in the combining step.
A first signal processing apparatus of the present invention comprises: first-signal input means for inputting a first signal having a broad range of frequency components; a second-signal input means for inputting a second signal in the first signal, the second signal having a narrow range having suppressed high frequency components; an extracting means for extracting high frequency components from the first signal input by the first-signal input means; a determining means for determining each class code based on the second signal input by the second-signal input means; and a storage means for storing, for the class code determined by the determining means, the high frequency components extracted by the extracting means.
A first signal processing method of the present invention includes: a first-signal input control step for controlling input of a first signal having a broad range of frequency components; a second-signal input control step for controlling input of a second signal in the first signal, the second signal having a narrow range having suppressed high frequency components; an extracting step for extracting high frequency components from the first signal whose input is controlled by processing in the first-signal input control step; a determining step for determining each class code based on the second signal whose input is controlled by processing in the second-signal input control step; and a storage step for storing, for the class code determined by processing in the determining step, the high frequency components extracted by processing in the extracting step.
A program on a first recording medium of the present invention includes: a first-signal input control step for controlling input of a first signal having a broad range of frequency components; a second-signal input control step for controlling input of a second signal in the first signal, the second signal having a narrow range having suppressed high frequency components; an extracting step for extracting high frequency components from the first signal whose input is controlled by processing in the first-signal input control step; a determining step for determining each class code based on the second signal whose input is controlled by processing in the second-signal input control step; and a storage step for storing, for the class code determined by processing in the determining step, the high frequency components extracted by processing in the extracting step.
A first program of the present invention allows a computer to execute: a first-signal input control step for controlling input of a first signal having a broad range of frequency components; a second-signal input control step for controlling input of a second signal in the first signal, the second signal having a narrow range having suppressed high frequency components; an extracting step for extracting high frequency components from the first signal whose input is controlled by processing in the first-signal input control step; a determining step for determining each class code based on the second signal whose input is controlled by processing in the second-signal input control step; and a storage step for storing, for the class code determined by processing in the determining step, the high frequency components extracted by processing in the extracting step.
An audio signal may be input to the input means in the first signal processing apparatus of the present invention.
When the signal input to the first signal processing apparatus of the present invention is an audio signal, based on the second signal, which is processed by weighting based on equal loudness characteristics, the determining means may determine the class code.
When the signal input to the first signal processing apparatus of the present invention is an audio signal, based on the positions of the average power level and maximum power level of an audio signal spectrum, the determining means may determine the class code.
When the signal input to the first signal processing apparatus of the present invention is an audio signal, the determining means may determine the class code of a spectral component equal to or greater than a frequency which is ½ times or ⅓ times the lowest frequency of the high frequency components extracted by the extracting means.
An image signal may be input to the input means in the first signal processing apparatus of the present invention.
When the signal input to the first signal processing apparatus of the present invention is an image signal, based on the alternating-current average-power level, direct-current power level, and alternating-current-component peak-power level of the spectrum of the image signal, the determining means may determine the class code.
In the storage means in the first signal processing apparatus of the present invention, the high frequency components extracted by the extracting means may be stored so as to be correlated to specific information.
The specific information in the first signal processing apparatus of the present invention may be the telephone number of a telephone set.
A generating means for generating the second signal by decimating the first signal in the first signal processing apparatus of the present invention at predetermined time intervals may be further provided.
A second signal processing apparatus of the present invention comprises: a signal input means for inputting a signal having a narrow range having suppressed high frequency components; a determining means for determining a class code based on the signal input by the signal input means; a storage means for storing high frequency components; a combining means for combining high frequency components stored in the storage means, which correspond to the class code determined by the determining means, and the signal input by the signal input means; and an output means for outputting the combined signal generated by the combining means.
A second signal processing method of the present invention includes: a signal-input control step for controlling input of a signal having a narrow range having suppressed high frequency components; a determining step for determining a class code based on the signal whose input is controlled by processing in the signal-input control step; a storage control step for controlling storage of high frequency components; a combining step for combining the high frequency components whose storage is controlled by processing in the storage control step, which correspond to the class code determined by processing in the determining step, and the signal input by processing in the signal-input control step; and an output control step for controlling output of the combined signal generated by processing in the combining step.
A program on a second recording medium of the present invention includes: a signal-input control step for controlling input of a signal having a narrow range having suppressed high frequency components; a determining step for determining a class code based on the signal whose input is controlled by processing in the signal-input control step; a storage control step for controlling storage of high frequency components; a combining step for combining the high frequency components whose storage is controlled by processing in the storage control step, which correspond to the class code determined by processing in the determining step, and the signal input by processing in the signal-input control step; and an output control step for controlling output of the combined signal generated by processing in the combining step.
A second program of the present invention allows a computer to execute: a signal-input control step for controlling input of a signal having a narrow range having suppressed high frequency components; a determining step for determining a class code based on the signal whose input is controlled by processing in the signal-input control step; a storage control step for controlling storage of high frequency components; a combining step for combining the high frequency components whose storage is controlled by processing in the storage control step, which correspond to the class code determined by processing in the determining step, and the signal input by processing in the signal-input control step; and an output control step for controlling output of the combined signal generated by processing in the combining step.
An audio signal may be input to the input means in the second signal processing apparatus of the present invention.
When the signal input to the second signal processing apparatus of the present invention is an audio signal, based on the second signal, which is processed by weighting based on equal loudness characteristics, the determining means may determine the class code.
When the signal input to the second signal processing apparatus of the present invention is an audio signal, based on the positions of the average power level and maximum power level of an audio signal spectrum, the determining means may determine the class code.
When the signal input to the second signal processing apparatus of the present invention is an audio signal, the determining means may determine the class code of a spectral component equal to or greater than a frequency which is ½ times or ⅓ times the lowest frequency of the high frequency components extracted by the extracting means.
An image signal may be input to the input means in the second signal processing apparatus of the present invention.
When the signal input to the second signal processing apparatus of the present invention is an image signal, based on the alternating-current average-power level, direct-current power level, and alternating-current-component peak-power level of the spectrum of the image signal, the determining means may determine the class code.
The high frequency components stored in the storage means in the second signal processing apparatus of the present invention may be updated with predetermined timing.
In the storage means in the second signal processing apparatus of the present invention, the high frequency components extracted by the extracting means may be stored so as to be correlated to specific information.
The specific information in the second signal processing apparatus of the present invention may be the telephone number of a telephone set.
In a signal processing system and signal processing method of the present invention, a first signal processing apparatus extracts high frequency components from a broad range of frequency components, determines a first class code based on a second signal in the first signal which has suppressed high frequency components, and stores the high frequency components for each class code. In the second signal processing apparatus, based on a third signal having a narrow range having suppressed high frequency components, a second class code is determined, and a third signal is generated by combination with the high frequency components which correspond to the second class code.
In a first signal processing apparatus and method, recording medium, and program, high frequency components are extracted from a first signal having a broad range of high frequency components, and based on a second signal in the first signal which has a narrow range having suppressed high frequency components, each class code is determined. The high frequency components are stored for the class code.
In a second signal processing apparatus and method, recording medium, and program, based on a signal having a narrow range having high frequency components, a class code is determined, the high frequency components, which correspond to the class code, and the signal having a narrow range are combined, whereby the combined signal is generated and output.
The signal processing apparatus may be a separate apparatus or may be a block that performs signal processing for an apparatus.
A narrow-range audio signal having low frequency components (not having high frequency components) is input to a frame-unit dividing unit 11. The input audio signal is divided by a predetermined frame (unit time) into blocks, and is output to a DCT (Discrete Cosine Transform) converting unit 12.
The DCT converting unit 12 performs, for each frame, a spectrum transform on the input audio signal, which is in time domain and is input from the frame-unit dividing unit 11, into a signal in frequency domain, and outputs the transformed signal to a classifying unit 13 and to a spectrum combining unit 16. As a method for performing the spectrum transform, an M-DCT (modified DCT) which allows adjacent blocks to half overlap each other is here used.
Spectrum transforms other the above M-DCT transform include DFT (Discrete Fourier Transform), DCT (Discrete Cosine Transform), FFT (Fast Fourier Transform), or methods such as LPC. They may include whatever transforms an audio signal in time domain into a signal in frequency domain.
The classifying unit 13 calculates a class code by using a classifying process (its details are described later with reference to
The learning-produced-data table 14 is a table memory in which class codes are used as addresses, and vector data items of an audio signal spectrum of high frequency components found by a learning apparatus, which is described later using
The gain adjusting unit 15 adjusts the gain of the high-frequency-component audio-signal-spectrum data input from the learning-produced-data table 14 to match the gain of low-frequency-component audio-signal-spectrum data to be combined by a spectrum combining unit 16.
The spectrum combining unit 16 combines the high-frequency-component audio-signal-spectrum vector data input from the gain adjusting unit 15 with the last or subsequent row of the low-frequency-component audio-signal-spectrum vector data input from the DCT transform unit 12, and outputs the combined data to an inverse DCT transform unit 17.
The inverse DCT transform unit 17 transforms spectrum data in frequency domain into a spectrum in time domain by performing the inverse of the M-DCT transform, and outputs the spectrum to a frame combining unit 18.
The frame combining unit 18 performs a process of adding overlapping frame portions for preventing sound from discontinuing in frame boundaries before outputting an audio signal.
The DCT spectrum data M-DCT-transformed, output by the DCT transform unit 12, is input to a power-level converting unit 31. The input DCT spectrum data is converted into a power level, and is converted into a dB (decibel) value before being output to a weighting unit 32.
The weighting unit 32 performs, on the input DCT spectrum data, a weighting process in which an equal loudness characteristic, which is described later with reference to
The spectrum extracting unit 33 extracts, from the input DCT spectrum data of low frequency components, DCT spectrum data equal to or less than a frequency which is ½ or ⅓ times the lowest frequency of a high frequency component to be restored, and outputs the extracted data to a subband-forming dividing unit 34.
The subband-forming dividing unit 34 divides the DCT spectrum data extracted by the spectrum extracting unit 33 into a predetermined number of subbands, and outputs the subbands to a quantizing unit 35.
The quantizing unit 35 arranges the DCT spectrum data divided into subbands, in units of subbands, finds the average of power levels in the input portion (extracted by the spectrum extracting unit 33), and uses quantization on the average to generate a class code. It also finds a subband position corresponding to the maximum power level and generates a class code based on it. Two class codes are output to a class-code determining unit 36.
The class-code determining unit 36 generates a single class code by multiplexing the class code based on the average of power levels which is input from the quantizing unit 35 and the class code based on the maximum power position, and outputs the generated class code to a learning-produced-data table 14 (
Next, an audio-range broadening process of the audio-signal broadening apparatus 1 is described with reference to the flowchart in
In step S1, the frame-unit dividing unit 11 divides, into frames, the audio signal input to the audio-range broadening process of the audio-signal broadening apparatus 1, which has suppressed high frequency components. Specifically, the audio signal is divided into blocks for each predetermined frame (unit time).
In step S2, the DCT transform unit 12 performs an M-DCT transform on the audio signal divided into frames in step S1.
In step S3, the classifying unit 13 calculates a class code by performing a classifying process, which is described later with reference to
In step S4, the learning-produced-data table 14 reads DCT vector data of a high frequency component which is stored at an address corresponding to the class code output in step S3 from the classifying unit 13.
In step S5, the spectrum combining unit 16 combines the low-frequency-component DCT spectrum data (the portion denoted by spectrum A1 in
In step S6, the inverse DCT transform unit 17 performs an inverse M-DCT transform on the DCT spectrum data combined in spectrum.
In step S7, the frame combining unit 18 performs overlapping addition on a frame boundary, and outputs the obtained result before ending the audio-range broadening process. This overlapping prevents the audio from being discontinuous also in the frame boundary.
Next, the classifying process in
At first, the fundamental principle of the classifying process is described with reference to
The classifying process is performed based on the assumption that there is a linking relationship between the amount of high frequency components included in a narrow-range signal and frequency components whose spectrum form is to be restored. In other words, in the case of assuming a model in which an audio signal is composed of a pitch frequency and its harmonic components, it is expected that a signal corresponding to a harmonic component which is two or three times the higher range region P which is approximately a half of the bandwidth B0 in
Next, the classifying process is described with reference to
In step S21, the power-level-forming converting unit 31 converts the spectrum data input from the DCT transform unit 12 into a power level, and further converts the level into a dB (decibel) value. In other words, the input spectrum data x[k](k=0, 1, . . . , N−1) is converted into the average power level PWR by the following expression:
In addition, the power level PWR is converted into the dB value PWRdB by the following expression:
PWRdB=10 log10(PWR)
In step S22, in the weighting unit 32, weighting into which human ear characteristics are considered is performed on the power decibel value PWRdB calculated in step S23.
A relationship between sound magnitude that a human can subjectively feel and a sound pressure level can be indicated by equal loudness curves as shown in
Such a process that frequency correction based on human ear characteristics is performed is the weighting performed in step S22. Specifically, correction which is called A-characteristic curve denoted by symbol A in
After the weighting is performed in step S22, in step S23, the spectrum extracting unit 33 extracts, from the input low-frequency-component spectrum data, spectrum data equal to or less than a frequency which is ½ or ⅓ times the lowest frequency of a high frequency component to be restored, and outputs the extracted data to the subband-forming dividing unit 34. This is performed based on the above-described fundamental principle of the classifying process.
In other words, the lowest frequency in the region Q in
In step S24, the subband-forming dividing unit 34 divides the spectrum data (the spectrum data from the frequency fQ/2 in
In step S25, as shown in
In step S26, the class-code determining unit 36 multiplexes a class code based on the average power level PAV output in step S25 and a class code based on the maximum power position Bn. In the example in
Methods for modeling the spectrum form can include, in addition to the above, a method that detects a plurality of spectrum peaks, a method that detects a spectrum inclination and power distribution, and a method that changes the number of subbands.
Next, a learning apparatus that generates vector data items of high frequency components stored for class codes, which are to be written in the learning-produced-data table 14 in the audio-signal broadening apparatus 1, is described with reference to
Initially, a broad-range audio signal which is used as a supervising signal and which includes high frequency components and a narrow-range audio signal as a learner signal in which the high frequency components of the supervising data are suppressed are prepared. In the example in
Accordingly, let the frame length of the learner data input to a frame-unit dividing unit 66 be N, the frame length of the supervising data input to a frame-unit dividing unit 61 is 2N. Also, data in each frame at this time is synchronized.
The functions of the frame-unit dividing unit 61, the frame-unit dividing unit 66, a DCT transform unit 62, and a DCT transform unit 67 are similar to those of the frame-unit dividing unit 11 and the DCT transform unit 12 in
A high-frequency-component extracting unit 63 extracts the higher half of the supervising data spectrum and outputs the extracted half to an adder unit 64.
A classifying unit 68 generates class codes similar to those described above in the classifying unit 13 in
The integrating table 69 receives, as input addresses, the class codes output by the classifying unit 68, and reads and outputs DCT vector data items-stored at the addresses to the adder unit 64. The adder unit 64 performs vector addition on N high-frequency-component DCT-vector-data items output from the high-frequency-component extracting unit 63 and N DCT-vector-data items output from the integrating table 69, and outputs the results to the positions of the addresses of the class codes identical to those used when the integrating table 69 outputs data. Because N DCT-vector-data items are stored at one class code address, the integrating table 69 is a table containing (N×the number of classes) data items. Also, with predetermined timing, the integrating table 69 divides the vector data items stored at their class code addresses by the class-occurrence frequency output from the encoding parameter encoder classifying unit 68, and outputs the results to a learning-produced-data table 70.
The learning-produced-data table 70 stores the (N×the number of classes) DCT-vector-data items output from the integrating table 69. The data items in the learning-produced-data table 70 are transferred to the audio-signal broadening apparatus 1 by a network and is stored in the learning-produced-data table 14. Alternatively, the data items in the learning-produced-data table 70 are written in a semiconductor memory or the like, and are set up as the learning-produced-data table 14 of the audio-signal broadening apparatus 1.
Since the structure of the classifying unit 68 in the learning apparatus 50 is similar to that of the classifying unit 13 in the audio-signal broadening apparatus 1, the structure of the classifying unit 13 in
Next, the learning process of the learning apparatus 50 is described with reference to the flowchart in
In step S51, the spectrum of the supervising data and the spectrum of the learner data are formed into frames. In other words, the frame-unit dividing unit 61 divides the input supervising data in each unit time into frames, and extracts 2N samples per frame. The decimating circuit 65 generates learner data by extracting only the low frequency components from the supervising data and alternately decimates the low frequency components. The frame-unit dividing unit 66 divides the learner data in each unit time into frames and extracts N samples per frame.
In step S52, the DCT transform unit 62 and the DCT transform unit 67 performs M-DCT transforms on the supervising data and the learner data, respectively.
In step s53, by performing a classifying process, which is described later with reference to
In step S54, the integrating table 69 reads vector data items stored at addresses corresponding to the class codes output by the classifying unit 68.
In step S55, the high-frequency-component extracting unit 63 extracts high frequency components from the supervising data input from the DCT transform unit 62.
In step S56, the adder unit 64 performs vector addition on the vector data (the data shown in
In step S57, the adder unit 64 stores the vector data (the data shown in
In step S58, it is determined whether the learning process has ended. When it is determined that the learning process has not ended yet, the process returns to step S51 and the subsequent steps are repeated. When it is determined that the learning process has ended, the process proceeds to step S59.
When it is determined in step S58 that the learning process has ended, in step S59, the integrating table 69 calculates the average of vector data items by dividing N items of the vector data stored at their class code addresses by the class-occurrence frequency of the corresponding class code, and outputs and stores the average in the learning-produced-data table 70 before ending the learning process.
Next, the classifying process in step S53 in
Although a description of processing in step S71 to step S76 is omitted since they are similar to step S21 to step S26 of the classifying process in the audio-signal broadening apparatus in
In step S77, the class-code determining unit 36 (similar to the audio-signal broadening apparatus 1 in
As described above, the classifying process in the audio-signal broadening apparatus 1 and the classifying process in the learning apparatus 50 are similar to each other until the class codes are calculated. After that, both differ only in whether to calculate the class-occurrence frequency.
DCT data obtained by the M-DCT transform in the DCT transform unit 92 is input to a power-level-forming converting unit 101. The input DCT data is converted into a power level similarly to the above-described audio-signal broadening apparatus 1, and is further converted into a power decibel value. The value is output to a quantizing unit 102.
The quantizing unit 102 calculates, for DCT data in the DCT block, the average power level of alternating-current components and the position of DCT data corresponding to the peak value of the alternating-current components, and generates and outputs, to a class-code determining unit 103, a class code based on both.
The class-code determining unit 103 generates a class code by multiplexing class codes based on the quantized data of the input DCT block, and outputs and stores the class code in the learning-produced-data table 94 (
Next, an image-broadening process in the image-signal broadening apparatus 80 is described. Since its flowchart is similar to that of the audio-signal broadening apparatus 1 in
In step S1, the image signal which is input to the image-signal broadening apparatus 80 and which has suppressed high frequency components is divided in units of blocks (e.g., 2×2 pixels) by the image-signal broadening apparatus 80.
In step s2, the DCT transform unit 92 performs an M-DCT transform on the image signal divided into blocks.
In step S3, the classifying unit 93 calculates class codes by performing a classifying process, which is described later with reference to
In step S4, the learning-produced-data table 94 reads vector data items of DCT coefficients stored at addresses corresponding to the class codes output in step S3 by the classifying unit 93.
In step S5, the combining unit 95 performs vector addition on the low-frequency-component DCT data (
In step S6, the inverse DCT transform unit 96 performs an inverse M-DCT transform on the combined DCT data.
Since the overlapping in step S7 does not always need to be performed in the case of an image signal, as described above, the image-signal broadening process is terminated without performing processing in step S7.
Next, the classifying process which is performed by the classifying unit 93 is described with reference to the flowchart in
In step S91, in a power-level converting unit 101, the DCT data input from the DCT transform unit 92 is converted into a power level and is further converted into a power decibel value. This processing is similar to that in the above case of the audio signal.
In step S92, the quantizing unit 102 calculates the alternating-current average power level and direct-current power level of the DCT block data, and detects an alternating-current-component peak power position. For example, a DCT block BL having 4×4 pixels as shown in FIG. 18 is input. The DCT data item of each pixel is c[n], and its power range is 0 to 127 [dB]. The alternating-current average power level is the average of DCT data items c[1] to c[15] in region AC in
In step S93, the classifying unit 93 generates a class code by multiplexing the class codes found based on three values in step S92, and outputs the class code to the learning-produced-data table 94 before the classifying process ends. In the above case, there are 128 class codes based on the alternating-current average power level, 128 class codes based on the direct-current power level, and 15 class codes based on the alternating-current-component peak power position. Thus, any one of 128×128×15=245760 class codes in total is used.
Next, the learning process of the learning apparatus 120 is described. Since its flowchart is similar to that of the audio-signal broadening apparatus 1 in
In step S51, DCT data of supervising data and DCT data of learner data are divided into blocks. In other words, the block-forming dividing unit 131 forms input supervising data into blocks for each set of 2N×2N pixels. The decimating circuit 135 generates learner data by extracting only low frequency components from the supervising data and alternately decimating the low frequency components in each of the horizontal direction and the vertical direction. The block-forming dividing unit 136 divides the learner data into blocks for each set of N×N pixels.
In step S52, the DCT transform unit 132 and the DCT transform unit 137 perform M-DCT transforms on the supervising data and the learner data.
In step S53, the classifying unit 138 performs, on the DCT data as the learner data, a classifying process, which is described later with reference to the
In step S54, the integrating table 139 reads a vector data item which is stored at an address corresponding to the class code output by the classifying unit 138.
In step s55, the high-frequency-component extracting unit 133 extracts high frequency components of the supervising data.
In step S56, the adder unit 134 performs vector addition of the vector data (the data shown in
In step S57, the adder unit 134 stores the vector data (the data shown in
In step S58, it is determined whether the learning process has ended. When it is determined that the learning process has not ended, the process returns to step S51, and the subsequent processing is repeated. When it is determined that the learning process has ended, the process proceeds to step S59.
When it is determined in step S58 that the learning process has ended, in step S59, the integrating table 139 calculates the average of vector data items by dividing (3N×N) vector data items at the addresses of class codes by a class-occurrence frequency of the corresponding class code which is output from the classifying unit 138, and outputs the average to the learning-produced-data table 140 before the process ends.
Next, the classifying process which is performed by the classifying unit 138 in
Since processing from step S101 to step S103 are similar to step S91 to step S93 in the classifying process in the image-signal broadening apparatus in
In step S104, the class-code determining unit 103 increases the class code count determined in step S103 by one, and outputs, to the integrating table 139, the count, which indicates how often which class code occurs before the classifying process ends.
In this transmission system, cellular phones 3011 and 3012 perform radio transmission and reception with base stations 3021 and 3022, and each of the base stations 3021 and 3022 performs transmission and reception with a switching center 303, whereby audio transmission and reception between the cellular phones 3011 and 3012 can be performed by using the base stations 3021 and 3022 and the switching center 303. The base stations 3021 and 3022 may be the same base station and may be different base stations.
The base stations 3021 and 3022 are hereinafter referred to as the cellular phone 301 unless they particularly need to be distinguished.
An antennal 311 receives radio waves from the base station 3021 or 3022, and supplies the received signal to a modem unit 312, while it transmits a signal from the modem 312 to the base station 3021 or 3022 by radio. The modem unit 312 demodulates the signal from the antenna 311 by using, for example, a CDMA (Code Division Multiple Access) method or the like, and supplies the resultant demodulated signal to a receiving unit 314. Also, the modem unit 312 modulates the transmitting data supplied from the transmitting unit 313 by using, for example, a CDMA method or the like, and supplies the resultant modulated signal to the antenna 311. A transmitting unit 313 obtains transmitting data by performing predetermined processing such as coding of an input user's voice, and supplies the data to the modem unit 312. The receiving unit 314 receives received data as the demodulated signal from the modem unit 312, performs decoding, and outputs high quality audio.
An operation unit 315 is operated by the user when the user inputs the telephone number of a calling destination, a predetermined command, or the like, and an operation signal corresponding to the operation is supplied to the transmitting unit 313 and the receiving unit 314.
Information can be exchanged between the transmitting unit 313 and the receiving unit 314, if needed.
The user's voice is input to a microphone 321, and the microphone 321 outputs the user's voice in the form of an audio signal as an electric signal to an A/D (Analog/Digital) converting unit 322. The A/D converting unit 322 performs A/D conversion on the analog audio signal from the microphone 321 into digital audio data, and outputs the data to a coding unit 323.
The coding unit 323 uses predetermined coding to encode the audio data from the A/D converting unit 322, multiplexes class codes, and outputs the resultant coded audio data to a transmission control unit 324.
A management unit 327 manages the telephone number of the calling destination, the telephone number of the cellular phone as a calling party, and other necessary information, which are input by operating the operation unit 315, if needed, and outputs these to the coding unit 323 and the transmission control unit 324.
The transmission control unit 324 performs transmission control of the coded audio data output by the coding unit 323 and the data output by the management unit 327. In other words, the transmission control unit 324 selects the coded audio data output by the coding unit 323 or the data output by the management unit 327, and outputs the selected data as transmitting data to the modem unit 312 (
The received data as a demodulated signal output by the modem unit 312 in
The decoding unit 332 separates the coded audio data supplied by the reception control unit 331 from class codes, or adds high frequency components, and supplies the resultant decoded audio data to a D/A (Digital/Analog) converting unit 333.
The D/A converting unit 333 performs D/A conversion on the decoded audio data in digital form output by the decoding unit 332 and supplies the resultant analog audio signal to a speaker 334. The speaker 334 outputs audio corresponding to the audio signal from the D/A converting unit 333.
When receiving a call, a management unit 335 receives the telephone number of the transmitting end and supplies the received telephone number to the decoding unit 332.
At first, the structure of the coding unit 323 in
Since the functions of a frame-unit dividing unit 351 and a DCT transform unit 352 are similar to those of the frame-unit dividing unit 11 and DCT transform unit 12 of the audio-signal broadening apparatus 1 in
A high-frequency-component eliminating unit 353 outputs, to a classifying unit 354 and a class-code multiplexing unit 355, low frequency components generated by eliminating high frequency components of the input DCT spectrum data. A compression ratio is dependent on an eliminating ratio of the high frequency components. For example, when a band is compressed to half, the amount of data is reduced to ½. One obtained by adding, to the reduced data amount, a data amount of several words to be multiplexed by the class-code multiplexing unit 355, which is described later, is a final data amount.
The classifying unit 354 performs a classifying process similar to that in the case of the classifying unit 13 in the audio-signal broadening apparatus 1, on the low-frequency-component DCT spectrum data input from the high-frequency-component eliminating unit 353, and outputs class codes to the class-code multiplexing unit 355.
The class-code multiplexing unit 355 multiplexes the class code output from the classifying unit 354 and the low-frequency-component DCT spectrum data output from the high-frequency-component eliminating unit 353, and outputs the multiplexed data to an inverse DCT transform unit 356. By performing multiplexing of the class code with the low-frequency-component DCT spectrum data, resistance to error on a class-code code-transmission path can be enhanced, and by checking a degree of coincidence between a class code obtained by performing the classifying process again in the decoding unit 332 (
The inverse DCT transform unit 356 performs an inverse M-DCT transform on data in which the input class code and the low-frequency-component DCT spectrum data are multiplexed, and outputs the result to a frame combining unit 357.
The frame combining unit 357 performs overlapping similar to that in the case of the frame combining unit 18 in the audio-signal broadening apparatus 1, and outputs the obtained data as coded audio data to the transmission control unit 324 (
Next, the structure of the decoding unit 332 in
Since a frame-unit dividing unit 371 and a DCT transform unit 372 have functions similar to those of the frame-unit dividing unit 11 and the DCT transform unit 12 in the audio-signal broadening apparatus 1 in
A class-code-forming separating unit 373 separates the multiplexed DCT spectrum data into a class code and low-frequency-component DCT spectrum data, and outputs the low-frequency-component DCT spectrum data to a spectrum combining unit 376 and outputs the class code to a learning-produced-data table 374.
The learning-produced-data table 374 reads and outputs, to a gain adjusting unit 375, high-frequency-component DCT spectrum data stored at an address corresponding to the class code output by the class-code-forming separating unit 373.
Similarly to the gain adjusting unit 15 in the audio-signal broadening apparatus 1, the gain adjusting unit 375 performs gain adjustment on the high-frequency-component DCT spectrum data output by the learning-produced-data table 374 so as to match the gain of the high-frequency-component spectrum to be combined in the spectrum combining unit 376, and outputs the result to the spectrum combining unit 376.
The spectrum combining unit 376 combines the low-frequency-component DCT spectrum data output by the class-code-forming separating unit 373 and the high-frequency-component DCT spectrum data output by the gain adjusting unit 375, and outputs the combined data to the inverse DCT transform unit 377.
The inverse DCT transform unit 377 performs an inverse M-DCT transform on the input spectrum data, and outputs the transformed data to a frame combining unit 378.
The frame combining unit 378 performs overlapping of frames, and outputs the result as a broad-range audio signal to the D/A converting unit 333.
In addition,
At first, the structure of the coding unit 323 in
A spectrum dividing unit 381 separates the input DCT spectrum data into a high frequency component and a low frequency component, and outputs the high frequency component to a table memory 382 and outputs the low frequency component to a classifying unit 354 and a class-code multiplexing unit 355.
The classifying unit 354 calculates a class code, and outputs the class code to the table memory 382 and the class-code multiplexing unit 355.
The table memory 382 stores, at an address corresponding to the class code output by the classifying unit 354, the high-frequency-component DCT spectrum data input from the spectrum dividing unit 381. The content of this table memory is transferred with predetermined timing to a table memory 391 (
In the coding unit 323 having the above structure, whenever a call is made, high-frequency-component DCT spectrum data for each calling party is stored in the table memory 382. Because the content of the table memory 382 is a type of statistical information, it is expected that, as more information is input, the information will converges to an appropriate deviation in units of class codes. Based on this effect, an increased number of times the content is used, more accurate decoding comes to be performed. The other components in
Next, the structure of the decoding unit 332 in
A table memory 391 reads and outputs, to the gain adjusting unit 375, the high-frequency-component DCT spectrum data stored at the position designated as the address corresponding to the class code output by the class-code-forming separating unit 373.
The operation of the cellular phone 301, in a case in which a learning-produced-data table is stored in a form correlated with the telephone number of the transmitting end and is updated with predetermined timing, as shown by the structures in
At first, a transmitting process that the transmitting unit 313 in the cellular phone 3011 performs is described with reference to the flowchart in
In step S201, when the user operates the operation unit 315 (
In step S202, the transmitting unit 313 performs a memory-data transmitting process, which is described with reference to
In step S203, the microphone 321 receives an input user's voice.
In step S204, the A/D converting unit 322 performs A/D conversion on the audio signal input from the microphone 321 and outputs the converted signal as a digital audio signal to the coding unit 323.
In step S205, the coding unit 323 performs a coding process. Its details are described later with reference to
In step S206, the transmission control unit 324 transmits, to the modem unit 312 (
In step S207, the management unit 327 determines whether the call has ended. When it is determined that the call has not ended, the management unit 327 returns the process to step S203 and the subsequent steps are repeated. When it is determined that the call has ended, the management unit 327 terminates the transmitting process.
Next, the memory-data transmitting process in step S202 in
In step S221, the transmission control unit 324 outputs, as transmitting data, the telephone number of the cellular phone 3012 input by operating the operation unit 315, whereby calling to the cellular phone 3012 is performed.
After the user of the cellular phone 3012 operates the operation unit 315 to set the cellular phone 3012 to be in an off-hook state in response to calling by the cellular phone 3011, the process proceeds to step S222. The transmission control unit 324 establishes a communication link with the cellular phone 3012 at the call receiving end, and proceeds to step S223.
In step S223, the management unit 327 reads and supplies data stored in the table memory 382 in the coding unit 323 to the transmission control unit 324. In step S223, the transmission control unit 324 selects the memory data from the management unit 327 and transmits the data as transmitting data. The memory data is transmitted with updating information representing a date and time of obtainment of the memory data.
After that, the process proceeds from step S223 to S224, and the management unit 327 determines whether a ready-completion notification has been transmitted from the cellular phone 3012 at the call receiving end.
Specifically, after the cellular phone 3012 at the call receiving end is in a state enabling a normal voice call, it can transmit a ready-completion notification representing completion of readiness of the voice call (in step S277 in
When it is determined in step S224 that the ready-completion notification has not been transmitted, the process returns to step S224 and waits for the ready-completion notification to be transmitted.
When it is determined in step S224 that the ready-completion notification has been transmitted, the process proceeds to step S225, and the transmission control unit 324 terminates the memory-data transmitting process by selecting an output from the coding unit 323 so that a voice call is possible.
Next, the coding process in step S205 in
In step S231, the frame-unit dividing unit 351 divides the audio signal input from the A/D converting unit 322 into frames. The input audio signal has both high frequency components and low frequency components.
In step S232, the DCT transform unit 352 performs a M-DCT transform on the audio signal divided into frames in step S231, and outputs the transformed signal as DCT spectrum data to the spectrum dividing unit 381.
In step S233, the spectrum dividing unit 381 divides the DCT spectrum data into a high frequency component and a low frequency component, and outputs the high frequency component to the table memory 382 and outputs the low frequency component to the classifying unit 354.
In step S234, the classifying unit 354 determines a class code by the cellular phone, and outputs the class code to the class-code multiplexing unit 355 and the table memory 382. Since the classifying process is similar to the classifying process in the audio-signal broadening apparatus 1 described using
In step S235, the table memory 382 stores, at an address corresponding to the class code output by the classifying unit 354, the high-frequency-component DCT spectrum data input from the spectrum dividing unit 381.
In step S236, the class-code multiplexing unit 355 multiplexes the class code input from the classifying unit 354 and the low-frequency-component DCT spectrum data input from the spectrum dividing unit 381, and outputs the multiplexed result to the inverse DCT transform unit 356.
In step S237, the inverse DCT transform unit 356 performs an inverse DCT transform on the multiplexed low-frequency-component DCT spectrum data.
Finally, in step S238, the frame combining unit 357 performs overlapping to generate coded audio data.
Next, a receiving process that is performed by the receiving unit 314 in the cellular phone 3012 at the call receiving end in response to the transmitting process in
In step S251, the management unit 335 detects reception of radio waves by using the reception control unit 331.
In step S252, the receiving unit 314 performs a memory-data updating process, which is described with reference to
After the received data input to the reception control unit 331 is switched to the coded audio data in step S253, the reception control unit 331 outputs the coded audio data to the decoding unit 332.
In step S254, the decoding unit 332 performs decoding. Its details are described with reference to
In step S255, the D/A converting unit 333 performs D/A conversion on the input broad-range audio signal input from the decoding unit 332 and outputs the converted signal to the speaker 334. In step S256, the speaker 334 outputs the analog audio signal input from the D/A converting unit 333.
In step S257, the management unit 335 determines whether the call has ended. When it is determined that the call has not ended, the management unit 335 returns the process to step S253 and the subsequent processing is repeated. When it is determined that the call has ended, the management unit 335 terminates the receiving process.
Next, the memory-data updating process in step S252 in
In step S271, the reception control unit 331 determines whether an off-hook state has been set by operating the operation unit 315. When it is determined that the off-hook state has not been set, the process returns to step S271.
Also, when it is determined in step S271 that the off-hook state has been set, the process proceeds to step S272, and the reception control unit 331 establishes a communication link and proceeds to step S273. In step S273, the reception control unit 331 receives received data including the last memory data transmitted from the cellular phone 3011 at the transmitting end and supplies the received data to the management unit 335.
In other words, in the memory-data transmitting process in
After that, proceeding to step S274, the management unit 335 refers to the updating information received from the cellular phone 3011 at the transmitting end and determines whether the last memory data about the user of the cellular phone 3011 is stored in the decoding unit 332.
When it is determined in step S274 that the last memory data about the user of the cellular phone 3011 is already stored in the decoding unit 332, the process proceeds to step S275, and the management unit 335 abandons the memory data and updating information received in step S273 before proceeding to step S277.
Also, when it is determined in step S274 that the last memory data about the user of the cellular phone 3011 has not been stored yet in the decoding unit 332, the process proceeds to step S276. The management unit 335 updates the content of the table memory 391 in the decoding unit 332 by storing, in the decoding unit 332, the last memory data obtained in step S273, in a form correlated with the telephone number of the cellular phone 3011 at the transmitting end, which is received in the call receiving mode, and the updating information transmitted with the memory data.
Proceeding to step S277, the management unit 335 controls the transmission control unit 324 in the transmitting unit 313 to transmit, as transmitting data, a ready-completion notification representing completion of a voice call, and proceeds to step S278.
In step S278, the reception control unit 331 comes into a vocal-call enabling state in which coded audio data included in the received data which is supplied thereto is output to the decoding unit 332, and the memory-data updating process ends.
Next, decoding in step S254 in
In step S291, the frame-unit dividing unit 371 divides the coded audio data input from the reception control unit 331 into frames.
In step S292, the DCT transform unit 372 performs an M-DCT transform on the coded audio data divided into frames in step S291 and outputs the transformed data as DCT spectrum data to the class-code-forming separating unit 373.
In step S293, the class-code-forming separating unit 373 separates the DCT spectrum data into a class code and a low frequency component, and outputs the class code to the table memory 391 and outputs the low frequency component to the spectrum combining unit 376.
In step S294, the table memory 391 outputs, to the gain adjusting unit 375, high-frequency-component DCT spectrum data at an address corresponding to the class code output by the class-code-forming separating unit 373.
In step S295, the spectrum combining unit 376 combines, in spectrum, the high frequency component input after being adjusted in gain by the gain adjusting unit 375 and the low frequency component input from the class-code-forming separating unit 373, and outputs the combined component to the inverse DCT transform unit 377.
In step S296, the inverse DCT transform unit 377 performs an inverse DCT transform on the combined DCT spectrum data.
Finally, in step S297, the frame combining unit 378 performs overlapping and outputs the result as a broad-range digital audio signal to the D/A converting unit 333 (
According to the memory-data updating process in
However, the updating process is not limited thereto. The updating process may be performed only when the user requests updating of the memory data if needed and the request is enabled.
Also, in the embodiment, in a call receiving mode, a telephone number transmitted from a transmitting end is used as identification information for identifying a transmitting end by a call receiving end. However, it is possible that unique ID (Identifications) be assigned to users, etc., and the IDs be used as identification information.
A case in which learning-produced-data tables are stored corresponding to the telephone numbers of transmitters and are updated with predetermined timing has been described. An operation in a case in which the learning-produced-data table 374 (table memory) is fixed in
The above consecutive processing may be performed by hardware or may be performed by software. When the consecutive processing is performed by software, programs constituting the software are installed in a multi-purpose computer or the like.
Accordingly,
The programs can be pre-recorded in a hard disk 405 or a ROM 403 as a recording medium built into the computer.
Alternatively, the programs can be temporarily or eternally stored (recorded) on a removable recording medium 411 such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. Such a removable recording medium 411 can be provided as so-called package software.
Not only the programs can be installed from the above recording medium 411 into the computer, but also the programs can be installed into the built-in hard disk 405 after being transferred by radio from a download site to the computer via a digital-satellite-broadcasting satellite or transferred to the computer via a network such as a LAN (Local Area Network) or the Internet, and being received by a communication unit 408.
The computer has a built-in CPU (Central Processing Unit) 402. An input/output interface 410 is connected to the CPU 402 by a bus 401. When an instruction is input such that the user performs an operation, or the like, on an input unit 407 including a keyboard, a mouse, and a microphone, the CPU 402 executes a program stored in a ROM (Read Only Memory) 403 in accordance with the instruction. Alternatively, the CPU 402 executes, in a form loaded into a RAM (Random Access Memory) 404, a program stored on the hard disk 405, a program installed into the hard disk 405 after being transferred from a satellite or a network and being received by the communication unit 408, or a program installed into the hard disk 405 after being read from the removable recording medium 411 in a state loaded into a drive 409. This causes the CPU 402 to perform the processes in accordance with the above flowcharts or processes based on the structures in the above block diagrams. The CPU 402 outputs the processing result, for example, from an output unit 406 constituted by an LCD (Liquid Crystal Display), a speaker, etc., through the input/output interface 410, transmits the result from the communication unit 408, or records the result on the hard disk 405.
In this specification, processing steps constituting programs for allowing a computer to perform various types of processing do not always need to be performed in a time-series manner in order in a flowchart form, and include processes (e.g., parallel processes or object-based processes) which are executed in parallel or separately.
In addition, the program may be processed by a single computer or may be processed in a distributed manner by a plurality of computers. The program may be executed after being transferred to a remote computer.
In the embodiment, a case in which the present invention is applied to an audio (image signal) broadening apparatus and a transmission system for performing a voice call between cellular phones has been described. In addition, the present invention can be widely applied to voice communication systems for tone improvement in fixed telephone voice, AM (Amplitude modulation)/FM (Frequency Modulation) radio-broadcast voice, analog TV (Television) broadcast received voice, etc.
In the specification, the term, system, means an entire apparatus constituted by a plurality of devices and means, etc.
As described above, according to a signal processing system and signal processing method for the signal processing system, of the present invention, a system in which high frequency components of a narrow-range signal having suppressed high frequency components can be restored with high accuracy is realized.
As described above, according to a first signal processing apparatus, signal processing method, recording medium, and program of the present invention, high frequency components of a narrow-range signal having suppressed high frequency components can be restored with high accuracy for another signal processing apparatus.
As described above, according to a second signal processing apparatus, signal processing method, recording medium, and program of the present invention, high frequency components of a narrow-range signal having suppressed high frequency components can be restored with high accuracy for another signal processing apparatus.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2002-266327 | Sep 2002 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP03/10824 | 8/27/2003 | WO | 00 | 11/26/2004 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2004/025625 | 3/25/2004 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5579435 | Jansson | Nov 1996 | A |
| 6658155 | Kondo et al. | Dec 2003 | B1 |
| 6772127 | Saunders et al. | Aug 2004 | B2 |
| 6907413 | Kondo et al. | Jun 2005 | B2 |
| 7584008 | Kondo et al. | Sep 2009 | B2 |
| 20050177257 | Kondo et al. | Aug 2005 | A1 |
| Number | Date | Country |
|---|---|---|
| 0 732 687 | Sep 1996 | EP |
| 0 945 852 | Sep 1999 | EP |
| 8 123484 | May 1996 | JP |
| 8 278800 | Oct 1996 | JP |
| 09 101798 | Apr 1997 | JP |
| 2000 305599 | Nov 2000 | JP |
| WO 02 13180 | Feb 2002 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20050073986 A1 | Apr 2005 | US |
