Efficient and robust adaptive algorithm for silence detection in real-time conferencing

Abstract

HomeMeeting Inc. provides complete Internet service (www.homemeeting.com) for multipoint multimedia IP-communication network. To the best of our knowledge, this is the first attempt of fully Internet-based interactive multipoint multimedia WAN communication service with enhanced quality of service (QoS) and a complete suite of presentation/discussion functionalities over narrowband (as low as 26.4 Kbps) connections. Every registered member of this service can sign into the Member Meeting Center from HomeMeeting's website, schedule meeting, invite meeting participants, and pre-upload documents for online discussion. To avoid the need of multiple microphone requirement which is feasible for most low-end audio/video conferencing terminals, and to avoid the need of using very complex signal processing algorithms which call for higher computational needs and longer voice delay, in this invention, a low complexity and effective silence detection technique based on an intelligent determination of adaptive threshold value is proposed to enable real-time audio/video conferencing.

Description

REFERENCES

[0001] [1] K. Bullington, J. M. Fraser, “Engineering Aspect of Time Assigned Speech Interpolation (TASI),” Bell System Technical Journal (BSTJ), vol. 38, pp. 353-364, 1959.

[0002] [2] M. Rangoussi, A. Delopoulos, M. Tsatsanis, “On the Use of Higher Order Statistics for Robust Endpoint Detection of Speech,” pp. 56-60, IEEE Signal Processing Workshop on Higher-Order Statistics, South Lake Tahoe, Calif., 1993.

[0003] [3] L. Rabiner, M. Sambur, “An Algorithm for Determining the Endpoints of Isolated Utterance,” Bell System Technical Journal (BSTJ), vol. 54, pp. 297-315, 1975.

[0004] [4] ITU-T, G.729 Annex B, “A Silence Compression Scheme for G.729 Optimized for Terminal Conforming to Recommendation V.70,” October 1996. http://www.itu.int/re/recommendation.asp?type=items&lang=e&parent=T-REC-G.729-199610-I!AnnB

[0005] [5] IC-Tech. Inc., “Enhanced Silence Detection in Variable Rate Coding Systems using Voice Extraction,” White paper, April 2000, http://www.ic-tech.com/pdf_docs/bandwidthwhitepaper.pdf

TECHNICAL FIELD

[0006] This invention proposes a low complexity and effective silence detection technique based on an intelligent determination of adaptive threshold value to enable real-time audio/video conferencing.

BACKGROUND OF THE INVENTION

[0007] Thanks to the recent advances in audio/video compression, processor design, and communication network architecture, it is now quite feasible to implement multimedia communication applications (e.g., audio/video conferencing) using standard computing and networking facilities. This shift of multimedia communication equipment and services from dedicated systems to general purpose computers and packet-based communication networks has introduced a quite different operating environment and has prompted the reexamination of several key algorithms. Silence detection and removal is an essential building block of any multimedia video conferencing system. It reduces the bandwidth requirements of the underlying network transport service and helps to maintain an acceptable end-to-end delay for audio.

[0008] HomeMeeting Inc. provides complete Internet service (www.homemeeting.com) for multipoint multimedia IP-communication network. To the best of our knowledge, this is the first attempt of fully Internet-based interactive multipoint multimedia WAN communication service with enhanced quality of service (QoS) and a complete suite of presentation/discussion functionalities over narrowband (as low as 26.4 Kbps) connections. Every registered member of this service can sign into the Member Meeting Center from HomeMeeting's website, schedule meeting, invite meeting participants, and pre-upload documents for online discussion. To avoid the need of multiple microphone requirement which is feasible for most low-end audio/video conferencing terminals, and to avoid the need of using very complex signal processing algorithms which call for higher computational needs and longer voice delay, in this invention, a low complexity and effective silence detection technique based on an intelligent determination of adaptive threshold value is proposed to enable real-time audio/video conferencing.

PRIOR ART

[0009] The issue of silence detection has been explored since digital speech processing research was initiated more than 40 years ago [1]. The use of energy levels and/or zero crossing rates for silence detection can be satisfactory only at high signal-to-noise ratios. A wide variety of approaches have been proposed, from the simplest form based on comparing the signal magnitude with a pre-specified threshold which results in poor performance in the presence of background noise and varying magnitudes, to very sophisticated algorithm, such as the use of third-order statistics to exploit the non-linearity of speech characteristics at the changeovers of speech and silence [2] which is too complex, particularly for real-time software based implementation on general purpose computers.

[0010] Based on the short-term energy and zero-crossing measures of speech signals, a low complexity, while less effective and less flexible, silence detection algorithm was proposed in [3]. More specifically, the pre-specified Ethresh can be determined as follows:

I 1 =0.03(Emax−Emin)+Emin

I 2 =4Emin

E thresh =5×min(I1,I2)

[0011] where Emax and Emin are the maximum and minimum energy values (sum of squared magnitudes over certain interval of time, e.g., 10 msec) estimated over entire speech interval.

[0012] A somewhat more complex algorithm, adopted in ITU G.729 Annex B [4], uses the degree of periodicity in signals to determine the presence of voice. However, it is not very effective in a conference call environment where several people may speak at the same time, and its computational requirement makes it harder to implement for a real-time application using low-end hardware devices (such as handheld PDAs). Another attempt is made by IC Tech. Inc. [5], which specifically combats the silence detection problem in noisy environment, especially when the distance between the microphone and the user's lips is varying, using a proprietary voice extraction (VE) technique which is achieved by exploiting inter-microphone differential information and the statistical properties of independent signal sources. This technique requires the use of multiple (at least two) microphones for recording mixtures of sound sources, which are then processed to separate out a single voice signal of interest from the mixture. For low-end audio/video conferencing terminals, the requirement of multiple microphones is never a feasible alternative.

OBJECTS AND ADVANTAGES

[0013] This invention proposed a low complexity and effective silence detection technique based on an intelligent determination of adaptive threshold value to enable real-time audio/video conferencing. More specifically, by appropriately low passing the speech signal to remove the less influential high-frequency component as well the DC component of speech for an effective calculation of speech magnitude, we can best measure the most important portion of uttered speech. Moreover, through our invented adaptive threshold determination scheme, the silence detection system can adaptively update the silence threshold value by incorporating the new background signal magnitude so as to dynamically detect the silence from the real speech.

SUMMARY OF THE INVENTION

[0014] Thanks to the recent advances in audio/video compression, processor design, and communication network architecture, it is now quite feasible to implement multimedia communication applications (e.g., audio/video conferencing) using standard computing and networking facilities. This shift of multimedia communication equipment and services from dedicated systems to general purpose computers and packet-based communication networks has introduced a quite different operating environment and has prompted the reexamination of several key algorithms. Silence detection and removal is an essential building block of any multimedia video conferencing system. It reduces the bandwidth requirements of the underlying network transport service and helps to maintain an acceptable end-to-end delay for audio.

[0015] To avoid the need of multiple microphone requirement which is feasible for most low-end audio/video conferencing terminals, and to avoid the need of using very complex signal processing algorithms which call for higher computational needs and longer voice delay, in this invention, a low complexity and effective silence detection technique based on an intelligent determination of adaptive threshold value is proposed to enable real-time audio/video conferencing.

DETAILED DESCRIPTION OF THE INVENTION

[0016] I. Measuring the Sound Wave Magnitude

[0017] To determine the magnitude of sound waves, the incoming speech data are first separated into non-overlapping frames for effective processing. Each frame consists of 1200 samples (i.e., 150 msec of speech under 8000 samples/sec input rate). The input sound data s(t) is first low-pass filtered to remove the high frequency components.

f (0)=s(0)×2,

f (t)=s(t−1)+s(t), 1≦t<1200

[0018] The DC component is then removed from f(t), and the absolute value is computed for each sample.

g (t)=|f(t)−{overscore (f)}|, 0≦t<1200,

[0019] where 1$\stackrel{\_}{f}=\frac{\sum _{i=0}^{1199}f\left(i\right)}{1200}$

[0020] The magnitude of speech signal σ in this frame is defined by the equation. 2$\sigma =\sum _{i=0}^{1199}\&LeftBracketingBar;g\left(i\right)-\stackrel{\_}{m}\&RightBracketingBar;,\mathrm{where}\stackrel{\_}{m}=\frac{\sum _{i=0}^{1199}g\left(i\right)}{1200}$

[0021] If σ is smaller than a threshold value λ, this frame is determined to be a silent frame.

[0022] II. Determining the Adaptive Threshold Value

[0023] During the conferencing, the background environment changes along the time, the intensity of participants' speech also varies all the time due to the movement of heads (in case a fixed location microphone is used). The threshold value λ needs to be changed according to the environments. To change λ, a value d is computed for 8 consecutive frames. 3$d=\sum _{i=0}^7\&LeftBracketingBar;{\sigma }_i-\stackrel{\_}{\sigma }\&RightBracketingBar;,$

[0024] where 4$\stackrel{\_}{\sigma }=\frac{\sum _{i=0}^7\&LeftBracketingBar;{\sigma }_i\&RightBracketingBar;}{8}.$

[0025] If d is greater than a pre-specified empirical constant k, then λ is not updated. If d is smaller, the source of the sound is determined from the background and λ is updated as a function of d and σmax accordingly:

λ←λ+φ(d,σmax),

[0026] where the function φ can be any general function. In our current implementation, a relatively simple function was chosen, i.e., 5$\begin{array}{c}\lambda \leftarrow \lambda +\Delta \mathrm{if}m\times {\sigma }_{\max }>\lambda \\ \lambda \leftarrow \lambda -\Delta \mathrm{if}m\times {\sigma }_{\max }\le \lambda -100\\ \lambda \leftarrow \lambda \mathrm{else}\end{array}\}\mathrm{if}d<k$${\sigma }_{\max }=\stackrel{7}{\underset{i=0}{\max }}{\sigma }_i$

[0027] where Δ is an empirical positive constant, m is another empirical constant with value greater than 1.

Claims

1. A low complexity and effective silence detection technique based on an intelligent determination of adaptive threshold value to enable real-time audio/video conferencing comprising: a) means (framing of speech) to best measure the most important portion of uttered speech; b) means (adaptive threshold determination) to adaptively update the silence threshold value by incorporating the new background signal magnitude.

2. The system of claim 1 further comprises techniques to low pass the speech signal so as to remove the less influential high-frequency component of speech for an effective calculation of speech magnitude.

3. The system of claim 1 further comprises techniques to remove the DC component of the speech signal, which is commonly microphone dependent, for an effective calculation of speech magnitude.

4. The system of claim 1 further comprises techniques to effectively measure the potential presence of speech by measuring the temporal variation of calculated speech magnitude.

5. The system of claim 1 further comprises techniques to update the silence threshold value by incorporating the temporal variations of speech magnitude.