Patent Application
20080120098
The present invention relates to adjusting a computational complexity of a signal encoder based on a resource shortage. The signal encoder may comprise a speech encoder.
Speech processing by a mobile device is often a complex process, thus taxing the available resources of the mobile device. For example, a wideband adaptive multi-rate (AMR-WB). AMR-WB is relatively high complex process and thus can utilize a significant portion of a mobile device's resources, e.g., computational resources and memory resources. Moreover, the mobile device may be simultaneously executing other processes. If any of the needed resources exceed the available resources, a corresponding service may not be completed in a timely basis, causing a perceived problem by the user.
With advanced services that are currently supported and that will be supported in the future, the demands on available resources of a mobile device are continuously increasing. Reducing the demands on the resources of mobile device may enable the mobile device to better execute a plurality of processes. Consequently, the support of advanced services on a mobile device is facilitated.
An aspect of the present invention provides methods, computer-readable media, and apparatuses for tuning and adjusting the computational complexity of algorithm that is executed by a signal encoder. The signal encoder may comprise a speech encoder.
With an aspect of the invention, a resource shortage on a computer platform is detected. A degree of the resource shortage and a corresponding complexity adjustment for a speech encoder are determined. The speech encoder is then tuned to adjust the computational complexity of an executed speech processing algorithm.
With another aspect of the invention, the resource shortage may correspond to a computational capability, audio buffer memory, or battery of a mobile device. A speech process being executed by the mobile device is tuned to adjust the computational demands in accordance with a complexity adjustment.
With another aspect of the invention, a number of iteration rounds is adjusted while the speech encoder is executing a speech processing algorithm. The iterations may correspond to an algebraic codebook search.
With another aspect of the invention, when the resource shortage ceases, a complexity adjustment returns a speech encoder to normal operation.
With another aspect of the invention, resource availability may increase sufficiently so that the computational complexity of a signal encoder may be increased.
A more complete understanding of the present invention and the advantages thereof may be acquired by referring to the following description in consideration of the accompanying drawings, in which like reference numbers indicate like features and wherein:
In the following description of the various embodiments, reference is made to the accompanying drawings which form a part hereof, and in which is shown by way of illustration various embodiments in which the invention may be practiced. It is to be understood that other embodiments may be utilized and structural and functional modifications may be made without departing from the scope of the present invention.
The prior art typically uses a standard Third Generation Project Plan (3GPP) Wideband Adaptive Multi-rate (AMR-WB) speech encoding having a fixed number of iteration rounds for each encoding mode. For example, AMR-WB mode at 23.85 kbps uses three iteration rounds, and AMR-WB mode at 23.05 kbps utilizes four iteration rounds for an algebraic codebook search.
As shown in
With an embodiment of the invention, the number of iteration rounds can be adaptively controlled during the encoding process on a frame-by-frame basis. Even during encoding one frame (typically having a 20 msec duration), the AMR-WB complexity can be decreased. This approach may enable good performance during unexpected peak computational load. After the peak computation load has ended, encoding can return to normal operation, in which the original, fixed number of iteration rounds provides standard performance.
Computer system 100 includes control module (operating system) 101 which administers resources, e.g., battery 161, audio buffer memory 163, and processing (CPU) resources (not shown), to applications (processes) 107-113 and speech encoding 103. For example, application 107 provides playback capability for a MP3 musical recording, application 109 supports “mobile karaoke,” application supports telephone call recording, and application 113 supports video recording. Computer system may support one or more applications at a given time.
Control module 101 administers the resources of computer system 100 and assigns the resources to applications 107-113 and speech encoding process 103 in accordance with the needs and the priorities of the processes. When a resource shortage is detected by control module 101, control module 101 provides a resource indication 153 to complexity determination module 105, where resource indication 153 is indicative of a degree of the resource shortage for the corresponding resource. (With embodiments of the invention, control module 101 and complexity determination module 105 may be combined into one module.) Complexity determination module 105 subsequently determines complexity adjustment 151 that tunes adjustable speech encoder 103 in order to control the computational complexity of the speech processing algorithm being executed by adjustable speech encoder 103. With an embodiment of the invention, the number of iterative rounds (as will be further discussed), is determined by complexity determination module 105 to adjust the computational complexity.
Unexpected computational load may be caused by high priority processes. For example, when a telephone call is incoming, the telephone call is serviced by a high priority process and may consume a substantial portion of a computational resource. Because mobile devices typically have limited computational resources, there is a possibility that the performance of low priority processes (e.g., voice/audio recording applications 111 and 113) may be degraded. In fact, it is possible that low priority processes cannot even be executed.
An embodiment of the invention may be utilized to decrease the computational load, for example, when battery 161 has been sufficiently discharged (corresponding to a low energy level). When a low battery lifetime notification is indicated by operation system 101 of the handset, adjustable speech encoder 103 can be configured to a low complex “mode” according to an embodiment to preserve battery lifetime. As will be further discussed, adjustable speech encoder may utilize an AMR-WB algorithm.
Embodiments of the invention support other types of encoders, e.g., video encoders and image encoders, in which complexity reduction is possible for the associated encoding algorithm during the encoding/compression process without changing the bit rate/bit stream and without losing the compatibility with the decoder.
With an embodiment of the invention, the number of search iterations for an algebraic codebook search algorithm can be tuned based on the computational load of a mobile device. By decreasing the number of iteration rounds, the AMR-WB encoding complexity can be decreased with acceptable audio quality degradation. While AMR-WB encoding is a relatively complex algorithm to execute on mobile devices, it is important to decrease AMR-WB encoding complexity to support simultaneous processes having a higher priority. An exemplary embodiment includes an AMR-WB encoder-enabled audio recorder, where other playback or audio capture functionalities are supported (for example, background music generation).
An embodiment of the invention provides a method to adjust encoding complexity of an adjustable speech encoder that is consistent with a standard 3GPP AMR-WB speech codec executing on a mobile device platform. While mobile terminals typically have limited computational resources, complexity requirements are often stringent for executing processes, e.g., encoding and decoding algorithms. Also, simultaneous processes often must execute. An embodiment facilitates tuning an AMR-WB speech encoder to control the computational requirement during a high computational load for a mobile device.
With an embodiment of the invention, recording is supported with fixed audio input buffer 203 as shown in
With prior art, sufficiently large buffers are typically used to avoid recording interruptions, which may be caused by unexpected computational load or other reasons which has an effect on the audio encoding performance. With prior art, buffer control is often achieved by changing the length of the buffer.
With mobile devices, short buffers are suitable because of the limited memory space in mobile devices. Also, a desired requirement for a recording application is that the recording length not be limited (e.g. 1 minute recording) and that the recording length be based on available storage space (e.g. memory card).
While short audio buffer and unlimited recording are desirable, audio input buffer 200 is controlled to avoid buffer overflow and emptying. Buffer overflow may occur if enough computational resources are not available for buffer emptying (by encoding/compressing the buffer data), while the buffer is filled (corresponding to audio data 205) by audio source 211 (e.g., a microphone). With buffered recording, it may be necessary to catch up with the real-time recording before the recording operation is finished. An embodiment of the invention may be utilized to enable quicker encoding of a recording buffer in order to do so.
Buffer controlling can be achieved, in accordance with an embodiment of the invention, in which the AMR-WB encoder complexity may be adaptively controlled by buffer control 209 during the encoding process to decrease the encoding complexity. If used buffer 205 approaches the size of audio input buffer 203 (i.e., available memory approaches a lower limit), the computational complexity of AMR-WB encoder 201 is reduced by decreasing the number of codebook search loops. Buffered audio data 205 can be compressed more quickly from buffer 203, even during the heavy computational peaks, and thus overflowing can be avoided. When buffer 203 is sufficiently empty and enough computational resources are available, a standard fixed number of codebook search iterations can be configured for normal operation during the encoding process.
Speech encoding apparatus 300 is compatible with an AMR-WB speech codec as developed by 3GPP for GSM/EDGE and WCDMA channels, where the standardized codec is based on conventional ACELP technology. In addition, the standardized AMR-WB speech codec may be utilized in packet switched networks and in different kind of multimedia applications. The standardized AMR WB codec consists of different active speech modes with discontinuous transmission (DTX) functionality. The applied mode selection is based on the network capacity and radio channel conditions. However, the AMR WB codec may also be operated using a variable rate scheme.
As shown in
Speech encoder 103 supports multi-rate configurations with independent coding modes. The applied mode selection is based on the network capacity and radio channel conditions. However, speech encoder 103 may also be operated using a variable rate scheme. While source adaptation (SA) extension is supported by source adaptation algorithm 307, the encoding mode may be selected independently for each analysis (encoding) frame (with 20 ms intervals) depending on the source signal characteristics as determined by rate determination algorithm (RDA) 309. The encoding process is also dependent on desired average bit rate target and supported mode set.
With an embodiment of the invention, the number of search iterations may be tuned based on computational load of mobile devices. By decreasing the number of iteration rounds, the AMR-WB encoding complexity, as performed by speech encoder 103, can be decreased with an acceptable degree of audio quality degradation. While AMR-WB encoding is a relatively complex algorithm that is executed on a mobile device platform, it may be important to decrease AMR-WB encoding complexity to enable simultaneous processes. An illustrative example is an AMR-WB encoder enabled audio recorder, where other playback or audio capture functionalities are supported (for example background music generation). By decreasing the number of iteration rounds, as will be further discussed, the AMR-WB encoding complexity can be decreased with a degree of audio quality degradation as shown in
While embodiments of the invention may reduce computational complexity of speech encoder 103 with a shortage of resources, embodiments of the invention may also increase the computational complexity when additional resources become available.
Embodiments of the invention are applicable to different user scenarios. For example, when a mobile device is recording something from in a user's environment (e.g., audio recording), the mobile device may receive an incoming video/phone call. The incoming call usually has the highest priority for the mobile device. The needed computational resources for the incoming call may have an impact on the computational resources available for recording process. Therefore, in this case, the mobile device can decrease the complexity of the audio encoder (e.g., adjustable speech encoder 103). Consequently, the mobile device can maintain the continuous recording process and also handle the incoming call at the same time. Embodiments of the invention support other scenarios, including recording/compressing something during a call (where the process has a higher priority in regards to computational resources).
The codebook structure is based on interleaved single-pulse permutation (ISPP) design. The 64 positions in the codevector are divided into four tracks of interleaved positions, with 16 positions in each track. The different codebooks at the different rates are constructed by placing a certain number of signed pulses in the tracks (from one to six pulses per track). The codebook index, or codeword, represents the pulse positions and signs in each track. Thus, no codebook storage is needed, since the excitation vector at the decoder can be constructed through the information contained in the index itself (no lookup tables).
An important feature of the used codebook is that it is a dynamic codebook consisting of an algebraic codebook followed by an adaptive prefilter F(z) which enhances special spectral components in order to improve the synthesis speech quality. A prefilter relevant to wideband signals is used whereby F(z) consists of two parts: a periodicity enhancement part 1/(1−0.85z−T) and a tilt part (1−β1z−1), where T is the integer part of the pitch lag and β1 is related to the voicing of the previous subframe and is bounded by [0.0,0.5]. The codebook search is performed in the algebraic domain by combining the filter F(z) with the weighed synthesis filter prior to the codebook search. Thus, the impulse response h(n) must be modified to include the prefilter F(z). That is, h(n)←h(n)*f(n). The codebook structures of different bit rates are given below.
Based on the degree of a resource shortage, an adjustable speech encoder may be instructed to perform from one to four iterations when determining the codebook excitation for the following modes.
23.85 and 23.05 kbit/s Mode
In this codebook, the innovation vector contains 24 non-zero pulses. All pulses can have the amplitudes +1 or −1. The 64 positions in a subframe are divided into four tracks, where each track contains six pulses, as shown in Table 1.
The six pulses in one track are encoded with 22 bits. This gives a total of 88 bits (22+22+22+22) for the algebraic code.
19.85 kbit/s Mode
In this codebook, the innovation vector contains 18 non-zero pulses. All pulses can have the amplitudes +1 or −1. The 64 positions in a subframe are divided into four tracks, where each of the first two tracks contains five pulses and each of the other tracks contains four pulses, as shown in Table 2.
The five pulses in one track are encoded with 20 bits. The four pulses in one track are encoded with 16 bits. This gives a total of 72 bits (20+20+16+16) for the algebraic code.
18.25 kbit/s Mode
In this codebook, the innovation vector contains 16 non-zero pulses. All pulses can have the amplitudes +1 or −1. The 64 positions in a subframe are divided into four tracks, where each track contains four pulses, as shown in Table 3.
The four pulses in one track are encoded with 16 bits. This gives a total of 64 bits (16+16+16+16) for the algebraic code.
15.85 kbit/s Mode
In this codebook, the innovation vector contains 12 non-zero pulses. All pulses can have the amplitudes +1 or −1. The 64 positions in a subframe are divided into 4 tracks, where each track contains three pulses, as shown in Table 4.
The three pulses in one track are encoded with 13 bits. This gives a total of 52 bits (13+13+13+13) for the algebraic code.
14.25 kbit/s Mode
In this codebook, the innovation vector contains 10 non-zero pulses. All pulses can have the amplitudes +1 or −1. The 64 positions in a subframe are divided into four tracks, where each track contains two or three pulses, as shown in Table 5.
Each two pulse positions in one track are encoded with eight bits (four bits for the position of every pulse), and the sign of the first pulse in the track is encoded with one bit. The three pulses in one track are encoded with 13 bits. This gives a total of 44 bits (13+13+9+9) for the algebraic code.
12.65 kbit/s Mode
In this codebook, the innovation vector contains eight non-zero pulses. All pulses can have the amplitudes +1 or −1. The 64 positions in a subframe are divided into four tracks, where each track contains two pulses, as shown in Table 6.
Each two pulse positions in one track are encoded with eight bits (total of 32 bits, 4 bits for the position of every pulse), and the sign of the first pulse in the track is encoded with one bit (total of four bits). This gives a total of 36 bits for the algebraic code.
8.85 kbit/s Mode
In this codebook, the innovation vector contains four non-zero pulses. All pulses can have the amplitudes +1 or −1. The 64 positions in a subframe are divided into four tracks, where each track contains one pulse, as shown in Table 7.
Each pulse position in one track is encoded with four bits and the sign of the pulse in the track is encoded with one bit. This gives a total of 20 bits for the algebraic code.
6.60 kbit/s Mode
In this codebook, the innovation vector contains two non-zero pulses. All pulses can have the amplitudes +1 or −1. The 64 positions in a subframe are divided into two tracks, where each track contains one pulse, as shown in Table 8.
Each pulse position in one track is encoded with 5 bits and the sign of the pulse in the track is encoded with one bit. This gives a total of 12 bits for the algebraic code.
In the above section, the number of bits needed to encode a number of pulses in a track was given. In this section, the procedures used for encoding from one to six pulses per track will be described. The description will be given for the case of four tracks per subframe, with 16 positions per track and pulse spacing of four (which is the case for all modes except the 6.6 kbit/s mode).
Encoding One Signed Pulse Per Track
The pulse position index is encoded with four bits and the sign index with one bit. The position index is given by the pulse position in the subframe divided by the pulse spacing (integer division). The division remainder gives the track index. For example, a pulse at position 31 has a position index of 31/4=7 and it belong to the track with index 3 (4th track). The sign index here is set to 0 for positive signs and 1 for negative signs. The index of the signed pulse is given by
I
ip
=p+s×2M (EQ. 1)
where p is the position index, s is the sign index, and M=4 is the number of bits per track.
Encoding Two Signed Pulses Per Track
In case of two pulses per track of K=2M potential positions (here M=4), each pulse needs one bit for the sign and M bits for the position, which gives a total of 2M+2 bits. However, some redundancy exists due to the unimportance of the pulse ordering. For example, placing the first pulse at position p and the second pulse at position q is equivalent to placing the first pulse at position q and the second pulse at position p. One bit can be saved by encoding only one sign and deducing the second sign from the ordering of the positions in the index. Here the index is given by
I
2p
=p
1
+p
0×2M+s×22M (EQ. 2)
where s is the sign index of the pulse at position index p0. If the two signs are equal then the smaller position is set to p0 and the larger position is set to p1. On the other hand, of the two signs are not equal then the larger position is set to p0 and the smaller position is set to p1. At the decoder, the sign of the pulse at position p0 is readily available. The second sign is deduced from the pulse ordering. If p0 is larger than p1 then the sign of the pulse at position p1 is opposite to that at position p0. If this is not the case then the two signs are set equal.
Encoding Three Signed Pulses Per Track
In case of three pulses per track, similar logic can be used as in the case of two pulses. For a track with 2M positions, 3M+1 bits are needed instead of 3M+3 bits. A simple way of indexing the pulses is to divide the track positions in two sections (or halves) and identify a section that contains at least two pulses. The number of positions in the section is K/2=2M/2=2M−1, which can be represented with M−1 bits. The two pulses in the section containing at least two pulses are encoded with the procedure for encoding two signed pulses which requires 2(M−1)+1 bits and the remaining pulse which can be anywhere in the track (in either section) is encoded with the M+1 bits. Finally, the index of the section that contains the two pulses is encoded with one bit. Thus the total number of required bits is 2(M−1)+1+M+1+1=3M+1. A simple way of checking if two pulses are positioned in the same section is done by checking whether the most significant bits (MSB) of their position indices are equal or not. Note that a MSB of 0 means that the position belongs to the lower half of the track (0-7) and MSB of 1 means it belongs to the upper half (8-15). If the two pulses belong to the upper half, they need to be shifted to the range (0-7) before encoding them using 2×3+1 bits. This can be done by masking the M−1 least significant bits (LSB) with a mask consisting of M−1 ones (which corresponds to the number 7 in this case). The index of the 3 signed pulses is given by
I
3p
=I
2p
+k×22M−1+I1p×22M (EQ. 3)
where I2p is the index of the two pulses in the same section, k is the section index (0 or 1), and I1p is the index of the third pulse in the track.
Encoding Four Signed Pulses Per Track
The four signed pulses in a track of length K=2M can be encoded using 4M bits. Similar to the case of three pulses, the K positions in the track are divided into two sections (two halves) where each section contains K/2=8 positions. Here we denote the sections as Section A with positions 0 to K/2−1 and Section B with positions K/2 to K−1. Each section can contain from zero to four pulses. Table 9, as shown below, shows the five cases representing the possible number of pulses in each section:
In cases 0 or 4, the four pulses in a section of length K/2=2M−1 can be encoded using 4(M−1)+1=4M−3 bits (this will be explained later on). In cases 1 or 3, the one pulse in a section of length K/2=2M−1 can be encoded with M−1+1=M bits and the three pulses in the other section can be encoded with 3(M−1)+1=3M−2 bits. This gives a total of M+3M−2=4M−2 bits. In case 2, the pulses in a section of length K/2=2M−1 can be encoded with 2(M−1)+1=2M−1 bits. Thus for both sections, 2(2M−1)=4M−2 bits are required. The case index can be encoded with two bits (four possible cases) assuming cases 0 and 4 are combined. Then for cases 1, 2, or 3, the number of needed bits is 4M−2. This gives a total of 4M−2+2=4M bits. For cases 0 or 4, one bit is needed for identifying either case, and 4M−3 bits are needed for encoding the 4 pulses in the section. Adding the 2 bits needed for the general case, giving a total of 1+4M−3+2=4M bits. The index of the four signed pulses is given by
I
4p
=I
AB
+k×24M−2 (EQ. 4)
where k is the case index (2 bits), and IAB is the index of the pulses in both sections for each individual case. For cases 0 and 1, IAB is given by
I
AB
0,4
=I
4p
section
+j×24M−4 (EQ. 5)
where j is a 1-bit index identifying the section with 4 pulses and I4p
I
AB
I
=I
3p
B
+I
1p
A×23(M−1)+1 (EQ. 6)
where I3p
IAB
where I2p
I
AB
3
=I
1p
B
+I
3p
A×2M (EQ. 8)
where I1p
Encoding Five Signed Pulses Per Track
The five signed pulses in a track of length K=2M can be encoded using 5M bits. Similar to the case of four pulses, the K positions in the track are divided into 2 sections A and B. Each section can contain from zero to five pulses. A simple approach to encode the five pulses is to identify a section that contains at least three pulses and to encode the three pulses in that section using 3(M−1)+1=3M−2 bits, and to encode the remaining two pulses in the whole track using 2M+1 bits. This gives 5M−1 bits. An extra bit is needed to identify the section that contains at least three pulses. Thus, a total of 5M bits are needed to encode the five signed pulses. The index of the five signed pulses is given by
5p
=I
2p
+I
3p×22M+k×25M−1 (EQ. 9)
where k is the index of the section that contains at least three pulses, I3p is the index of the three pulses in that section (3(M−1)+1 bits), and I2p is the index of the remaining two pulses in the track (2M+1 bits).
Encoding Six Signed Pulses Per Track
The six signed pulses in a track of length K=2M are encoded using 6M−2 bits. Similar to the case of five pulses, the K positions in the track are divided into 2 sections A and B. Each section can contain from zero to six pulses. Table 10, as shown below, shows the 7 cases representing the possible number of pulses in each sections:
Note that cases 0 and 6 are similar except that the six pulses are in different section. Similarly, cases 1 and 5 as well as cases 2 and 4 differ only in the section that contains more pulses. Therefore these cases can be coupled and an extra bit can be assigned to identify the section that contains more pulses. Since these cases initially need 6M−5 bits, the coupled cases need 6M−4 bits taking into account the Section bit. Thus, we have now four states of coupled cases, that is (0,6), (1,5), (2,4), and (3),with 2 extra bits needed for the state. This gives a total of 6M−4+2=6M−2 bits for the six signed pulses. In cases 0 and 6, one bit is needed to identify the section which contains six pulses. five pulses in that section are encoded using 5(M−1) bits (since the pulses are confined to that section), and the remaining pulse is encoded using (M−1)+1 bits. Thus a total of 1+5(M−1)+M=6M−4 bits are needed for this coupled case. An extra two bits are needed to encode the state of the coupled case, giving a total of 6M−2 bits. For this coupled case, the index of the six pulses is given by
I
6p
=I
1p
+I
5p×2M+j×26M−5+k×26M−4 (EQ. 10)
where k is the index of the coupled case (2 bits), j is the index of the section containing six pulses (1 bit), I5p is the index of five pulses in that section (5(M−1) bits), and I1p is the index of the remaining pulse in that section ((M−1)+1 bits). In cases 1 and 5, one bit is needed to identify the section which contains five pulses. The five pulses in that section are encoded using 5(M−1) bits and the pulse in the other section is encoded using (M−1)+1 bits. For this coupled case, the index of the six pulses is given by
I
6p
=I
1p
+I
5p×2M+j×26M−5+k×26M−4 (EQ. 11)
where k is the index of the coupled case (2 bits), j is the index of the section containing five pulses (1 bit), I5p is the index of the five pulses in that section (5(M−1) bits), and I1p is the index of the pulse in the other section ((M−1)+1 bits). In cases 2 or 4, 1 bit is needed to identify the section which contains four pulses. The four pulses in that section are encoded using 4(M−1) bits and the two pulses in the other section are encoded using 2(M−1)+1 bits. For this coupled case, the index of the six pulses is given by
I
6p
=I
2p
+I
4p×22(M−1)+1+j×26M−5+k×26M−4 (EQ. 12)
where k is the index of the coupled case (2 bits), j is the index of the section containing four pulses (1 bit), I4p is the index of four pulses in that section (4(M−1) bits), and I2p is the index of the two pulses in the other section (2(M−1)+1 bits). In case 3, the three pulses in each section are encoded using 3(M−1)+1 bits in each Section. For this case, the index of the six pulses is given by
I
6p
=I
3pB
+I
3pA×23(M−1)+1+k×26M−4 (EQ. 13)
where k is the index of the coupled case (two bits), I3pB is the index of three pulses Section B (3(M−1)+1 bits), and I3pA is the index of the three pulses in Section A (3(M−1)+1 bits).
The algebraic codebook is searched by minimizing the mean square error between the weighted input speech and the weighted synthesis speech. The target signal used in the closed-loop pitch search is updated by subtracting the adaptive codebook contribution. Thus,
x
2(n)=x(n)−gpy(n), n=0, . . . 63 (EQ. 14)
where y(n)=v(n)*h(n) is the filtered adaptive codebook vector and gp is the unquantized adaptive codebook gain. The matrix H is defined as the lower triangular Toeplitz convolution matrix with diagonal h(0) and lower diagonals h(1), . . . ,h(63), and d=Htx2 is the correlation between the target signal x2(n) and the impulse response h(n) (also known as the backward filtered target vector), and Φ=HtH is the matrix of correlations of h(n).
The elements of the vector d are computed by
and the elements of the symmetric matrix Φ are computed by
The vector d and the matrix Φ are usually computed prior to the codebook search. The algebraic structure of the codebooks allows for very fast search procedures since the innovation vector ck contains only a few nonzero pulses. The correlation in the numerator of Equation (43) is given by
where mi is the position of the ith pulse, ai is its amplitude, and Np is the number of pulses. The energy in the denominator of Equation (43) is given by
To simplify the search procedure, the pulse amplitudes are predetermined based on a certain reference signal b(n). In this so-called signal-selected pulse amplitude approach, the sign of a pulse at position i is set equal to the sign of the reference signal at that position. Here, the reference signal b(n) is given by
where Ed=dtd is the energy of the signal d(n) and Er=rLTPtrLPT is the energy of the signal rLTP(n) which is the residual signal after long term prediction. The scaling factor α controls the amount of dependence of the reference signal on d(n), and it is lowered as the bit rate is increased. Here α=2 for 6.6 and 8.85 modes; α=1 for 12.65, 14.25, and 15.85 modes; α=0.8 for 18.25 mode; α=0.75 for 19.85 mode; and α=0.5 for 23.05 and 23.85 modes.
To simplify the search the signal d(n) and matrix Φ are modified to incorporate the pre-selected signs. Let sb(n) denote the vector containing the signs of b(n). The modified signal d′(n) is given by
d′(n)=sb(n)d(n) n=0, . . . ,N-1 (EQ. 21)
and the modified autocorrelation matrix Φ′ is given by
φ′(i,j)=sb(i)sb(j)φ(i,j), i=0, . . . ,N-1; j=i, . . . ,N-1. (EQ. 22)
and the energy at the denominator of the search criterion Qk is given by
The goal of the search now is to determine the codevector with the best set of Np pulse positions assuming amplitudes of the pulses have been selected as described above. The basic selection criterion is the maximization of the above mentioned ratio Qk. In order to reduce the search complexity, a fast search procedure known as depth-first tree search procedure is used, whereby the pulse positions are determined Nm pulses at a time. More precisely, the Np available pulses are partitioned into M non-empty subsets of Nm pulses respectively such that N1+N2 . . . +Nm . . . +NM=Np. A particular choice of positions for the first J=N1+N2 . . . +Nm−1 pulses considered is called a level-m path or a path of length J. The basic criterion for a path of J pulse positions is the ratio Qk(J) when only the J relevant pulses are considered.
The search begins with subset #1 and proceeds with subsequent subsets according to a tree structure whereby subset m is searched at the mth level of the tree. The purpose of the search at level 1 is to consider the N1 pulses of subset #1 and their valid positions in order to determine one, or a number of, candidate path(s) of length N1 which are the tree nodes at level 1. The path at each terminating node of level m−1 is extended to length N1+N2 . . . +Nm at level m by considering Nm new pulses and their valid positions. One, or a number of, candidate extended path(s) are determined to constitute level-m nodes. The best codevector corresponds to that path of length Np which maximizes the criterion Qk(Np) with respect to all level-M nodes.
A special form of the depth-first tree search procedure is used here, in which two pulses are searched at a time, that is, Nm=2, and these 2 pulses belong to two consecutive tracks. Further, instead of assuming that the matrix Φ is precomputed and stored, which requires a memory of N×N words (64×64=4 k words), a memory-efficient approach is used which reduces the memory requirement. In this approach, the search procedure is performed in such a way that only a part of the needed elements of the correlation matrix are precomputed and stored. This part corresponds to the correlations of the impulse response corresponding to potential pulse positions in consecutive tracks, as well as the correlations corresponding to φ(j,j), j=0, . . . ,N-1 (that is the elements of the main diagonal of matrix Φ).
In order to reduce complexity, while testing possible combinations of two pulses, a limited number of potential positions of the first pulse are tested. Further, in case of large number of pulses, some pulses in the higher levels of the search tree are fixed. In order to guess intelligently which potential pulse positions are considered for the first pulse or in order to fix some pulse positions, a “pulse-position likelihood-estimate vector” b is used, which is based on speech-related signals. The pth component b(p) of this estimate vector b characterizes the probability of a pulse occupying position p(p=0, 1, . . . N-1) in the best codevector we are searching for. Here the estimate vector b is the same vector used for preselecting the amplitudes and given in Equation (46).
The search procedures for all bit rate modes are similar. Two pulses are searched at a time, and these two pulses always correspond to consecutive tracks. That is the two searched pulses are in tracks T0-T1, T1-T2, T2-T3, or T3-T0. Before searching the positions, the sign of at pulse a potential position n is set the sign of b(n) at that position.
Then the modified signal d′(n) is computed as described above by including the predetermined signs. For the first two pulses (1st tree level), the correlation at the numerator of the search criterion is given by
R=d′(m0)+d′(m1) (EQ. 25)
and the energy at the denominator of the search criterion Qk is given by
E=φ′(m0,m0)+φ′(m1,m1)+2φ′(m0,m1) (EQ. 26)
where the correlations φ′(mi,mj) has been modified to include the preselected signs at positions mi and mj.
For subsequent levels, the numerator and denominator are updated by adding the contribution of two new pulses. Assuming that two new pulses at a certain tree level with positions mk and mk+1 from two consecutive tracks are searched, then the updated value of R is given by
R=R+d′(mk)+d′(mk+1) (EQ. 27)
and the updated energy is given by
E=E+φ′(mk,mk)+φ′(mk+1,mk+1)+2φ′(mk,mk+1)+2Rhv(mk)+2Rhv(mk+1) (EQ. 28)
where Rhv(m) is the correlation between the impulse response h(n) and a vector vh(n) containing the addition of delayed versions of impulse response at the previously determined positions. That is,
At each tree level, the values of Rhv(m) are computed online for all possible positions in each of the two tracks being tested. It can be seen from Equation (48) that only the correlations φ′(mk,mk+1) corresponding to pulse positions in two consecutive tracks need to be stored (4×16×16 words), along with the correlations φ′(mk,mk) corresponding to the diagonal of the matrix Φ (64 words). Thus the memory requirement in the present algebraic structure is 1088 words instead of 64×64=4096 words. The search procedures at the different bit rates modes are similar. The difference is in the number of pulses, and accordingly, the number of levels in the tree search. In order to keep a comparable search complexity across the different codebooks, the number of tested positions is kept similar.
The search in the 12.65 kbit/s mode will be described as an example. In this mode, 2 pulses are placed in each track giving a total of 8 pulses per subframe of length 64. Two pulses are searched at a time, and these two pulses always correspond to consecutive tracks. That is the two searched pulses are in tracks T0-T1, T1-T2, T2-T3, or T3-T0. The tree has 4 levels in this case. At the first level, pulse P0 is assigned to track T0 and pulse P1 to track T1. In this level, no search is performed and the two pulse positions are set to the maximum of b(n) in each track. In the second level, pulse P2 is assigned to track T2 and pulse P3 to track T3. four positions for pulse P2 are tested against all 16 positions of pulse P3. The four tested positions of P2 are determined based on the maxima of b(n) in the track. In the third level, pulse P4 is assigned to track T1 and pulse P5 to track T2. Eight positions for pulse P4 are tested against all 16 positions of pulse P5. Similar to the previous search level, the 8 tested positions of P4 are determined based on the maxima of b(n) in the track. In the fourth level, pulse P6 is assigned to track T3 and pulse P7 to track T0. Eight positions for pulse P6 are tested against all 16 positions of pulse P7. Thus, the total number of tested combination is 4×16+8×16+8×16=320. The whole process is repeated from one to four times (one to four iterations) by assigning the pulses to different tracks. For example, in the 2nd iteration, pulses P0 to P7 are assigned to tracks T1, T2, T3, T0, T2, T3, T0, and T1, respectively. Thus, the total number of tested position combinations is 4×320=1280.
As another search example, in the 15.85 kbit/s mode, three pulses are placed in each track giving a total of 12 pulses. There are six levels in the tree search whereby two pulses are searched in each level. In the first two levels, four pulses are set to the maxima of b(n). In the subsequent four levels, the number of tested combinations are 4×16, 6×16, 8×16, and 8×16, respectively. Based on the degree of a resource shortage, one to four iterations may be preformed by the adjustable speech encoder. When four iterations are used, there are a total of 4×26×16=1664 combinations.
Performance Evaluation
As discussed above, the computational complexity of an adjustable speech encoder is a function of a resource shortage. Also, embodiments of the invention may be utilized with a low battery life time. In this case, all the recording/compression activities can be processed by using complexity adjustment while the battery is being recharged. As shown in
As can be appreciated by one skilled in the art, a computer system with an associated computer-readable medium containing instructions for controlling the computer system can be utilized to implement the exemplary embodiments that are disclosed herein. The computer system may include at least one computer such as a microprocessor, digital signal processor, and associated peripheral electronic circuitry.
While the invention has been described with respect to specific examples including presently preferred modes of carrying out the invention, those skilled in the art will appreciate that there are numerous variations and permutations of the above described systems and techniques that fall within the spirit and scope of the invention as set forth in the appended claims.
