Noise suppression for speech processing based on machine-learning mask estimation

Abstract

Described are noise suppression techniques applicable to various systems including automatic speech processing systems in digital audio pre-processing. The noise suppression techniques utilize a machine-learning framework trained on cues pertaining to reference clean and noisy speech signals, and a corresponding synthetic noisy speech signal combining the clean and noisy speech signals. The machine-learning technique is further used to process audio signals in real time by extracting and analyzing cues pertaining to noisy speech to dynamically generate an appropriate gain mask, which may eliminate the noise components from the input audio signal. The audio signal pre-processed in such a manner may be applied to an automatic speech processing engine for corresponding interpretation or processing. The machine-learning technique may enable extraction of cues associated with clean automatic speech processing features, which may be used by the automatic speech processing engine for various automatic speech processing.

Description

TECHNICAL FIELD

The application generally relates to digital audio signal processing and, more specifically, to noise suppression utilizing a machine-learning framework.

BACKGROUND

An automatic speech processing engine, including, but not limited to, an automatic speech recognition (ASR) engine, in an audio device may be used to recognize spoken words or phonemes within the words in order to identify spoken commands by a user is described. Conventional automatic speech processing is sensitive to noise present in audio signals including user speech. Various noise reduction or noise suppression pre-processing techniques may offer significant benefits to operations of an automatic speech processing engine. For example, a modified frequency domain representation of an audio signal may be used to compute speech-recognition features without having to perform any transformation to the time-domain. In other examples, automatic speech processing techniques may be performed in the frequency-domain and may include applying a real, positive gain mask to the frequency domain representation of the audio signal before converting the signal back to a time-domain signal, which may be then fed to the automatic speech processing engine.

The gain mask may be computed to attenuate the audio signal such that background noise is decreased or eliminated to an extent, while the desired speech is preserved to an extent. Conventional noise suppression techniques may include dynamic noise power estimation to derive a local signal-to-noise ratio (SNR), which may then be used to derive the gain mask using either a formula (e.g., spectral subtraction, Wiener filter, and the like) or a data-driven approach (e.g., table lookup). The gain mask obtained in this manner may not be an optimal mask because an estimated SNR is often inaccurate, and the reconstructed time-domain signal may be very different from the clean speech signal.

SUMMARY

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

The aspects of the present disclosure provide for noise suppression techniques applicable in digital audio pre-processing for automatic speech processing systems, including but not limited to automatic speech recognition (ASR) systems. The principles of noise suppression lie in the use of a machine-learning framework trained on cues pertaining to clean and noisy speech signals. According to exemplary embodiments, the present technology may utilize a plurality of predefined clean speech signals and a plurality of predefined noise signals to train at least one machine-learning technique and map synthetically generated noisy speech signals with the cues of clean speech signals and noise signals. The trained machine-learning technique may be further used to process and decompose real audio signals into clean speech and noise signals by extracting and analyzing cues of the real audio signal. The cues may be used to dynamically generate an appropriate gain mask, which may precisely eliminate the noise components from the real audio signal. The audio signal pre-processed in such manner may then be applied to an automatic speech processing engine for corresponding interpretation or processing. In other aspects of the present disclosure, the machine-learning technique may enable extracting cues associated with clean automatic speech processing features, which may be directly used by the automatic speech processing engine.

According to one or more embodiments of the present disclosure, there is provided a computer-implemented method for noise suppression. The method may comprise the operations of receiving, by a first processor communicatively coupled with a first memory, first noisy speech, the first noisy speech obtained using two or more microphones. The method may further include extracting, by the first processor, one or more first cues from the first noisy speech, the first cues including cues associated with noise suppression and automatic speech processing. The automatic speech processing may be one or more of automatic speech recognition, language recognition, keyword recognition, speech confirmation, emotion detection, voice sensing, and speaker recognition. The method may further include creating clean automatic speech processing features using a mapping and the extracted one or more first cues, the clean automatic speech processing features being for use in automatic speech processing. The machine-learning technique may include one or more of a neural network, regression tree, a non-linear transform, a linear transform, and a Gaussian Mixture Model (GMM).

According to one or more embodiments of the present disclosure, there is provided yet another computer-implemented method for noise suppression. The method may include the operations of receiving, by a second processor communicatively coupled with a second memory, clean speech and noise; and producing, by the second processor, second noisy speech using the clean speech and the noise. The method may further include extracting, by the second processor, one or more second cues from the second noisy speech, the one or more second cues including cues associated with noise suppression and noisy automatic speech processing; and extracting clean automatic speech processing cues from the clean speech. The process may include generating, by the second processor, a mapping from the one or more second cues associated with the noise suppression cues and noisy automatic speech processing cues to clean automatic speech processing cues, the generating including at least one second machine-learning technique.

The clean speech and noise may each obtained using at least two microphones, the one or more first and second cues each including at least one inter-microphone level difference (ILD) cues and inter-microphone phase difference (IPD) cues. The automatic speech processing comprises one or more of automatic speech recognition, language recognition, keyword recognition, speech confirmation, emotion detection, voice sensing, and speaker recognition. The cues may include at least one of inter-microphone level difference (ILD) cues and inter-microphone phase difference (IPD) cues. The cues may further include at least one of energy at channel cues, voice activity detection (VAD) cues, spatial cues, frequency cues, Wiener gain mask estimates, pitch-based cues, periodicity-based cues, noise estimates, and context cues. The machine-learning technique may include one or more of a neural network, regression tree, a non-linear transform, a linear transform, and a Gaussian Mixture Model (GMM).

According to one or more embodiments of the present disclosure, there is provided a system for noise suppression. An example system may include a first frequency analysis module configured to receive first noisy speech, the first noisy speech being each obtained using at least two microphones; a first cue extraction module configured to extract one or more first cues from the first noisy speech, the first cues including cues associated with noise suppression and automatic speech processing; and a modification module being configured to create clean automatic speech processing features using a mapping and the extracted one or more first cues. The clean automatic speech processing features being for use in automatic speech processing.

According to some embodiments, the method may include receiving, by a processor communicatively coupled with a memory, clean speech and noise, the clean speech and noise each obtained using at least two microphones; producing, by the processor, noisy speech using the clean speech and the noise; extracting, by the processor, one or more cues from the noisy speech, the cues being associated with at least two microphones; and determining, by the processor, a mapping between the cues and one or more gain coefficients using the clean speech and the noisy speech, the determining including at least one machine-learning technique.

Embodiments described herein may be practiced on any device that is configured to receive and/or provide audio such as, but not limited to, personal computers (PCs), tablet computers, phablet computers; mobile devices, cellular phones, phone handsets, headsets, media devices, and systems for teleconferencing applications.

Other example embodiments of the disclosure and aspects will become apparent from the following description taken in conjunction with the following drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements.

FIG. 1 is an illustration of an environment in which embodiments of the present technology may be used.

FIG. 2 is a block diagram of an exemplary audio device.

FIG. 3 is a block diagram of an exemplary audio processing system.

FIG. 4 is a block diagram of an exemplary training system environment.

FIG. 5 illustrates a flow chart of an example method for training a machine-learning technique used for noise suppression.

FIG. 6 illustrates a flow chart of an example method for noise suppression.

FIG. 7 illustrates a flow chart of yet another example method for training a machine-learning technique used for noise suppression.

FIG. 8 illustrates a flow chart of yet another example method for noise suppression.

FIG. 9 is a diagrammatic representation of an example machine in the form of a computer system, within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed.

DETAILED DESCRIPTION

Various aspects of the subject matter disclosed herein are now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of one or more aspects. It may be evident, however, that such aspects may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form to facilitate describing one or more aspects.

INTRODUCTION

The techniques of the embodiments disclosed herein may be implemented using a variety of technologies. For example, the methods described herein may be implemented in software executing on a computer system or in hardware utilizing either a combination of processors or other specially designed application-specific integrated circuits, programmable logic devices, or various combinations thereof. In particular, the methods described herein may be implemented by a series of processor-executable instructions residing on a non-transitory storage medium such as a disk drive or a processor-readable medium. The methods may be implemented in software that is cloud-based.

In general, the techniques of the embodiments disclosed herein provide for digital methods for audio signal pre-processing involving noise suppression appropriate for further use in various automatic speech processing systems. The disclosed methods for noise suppression employ one or more machine-learning algorithms for mapping cues between predetermined, reference noise signals/clean speech signals and noisy speech signals. The mapping data may be used in dynamic calculation of an appropriate gain mask estimate suitable for noise suppression.

In order to obtain a better estimate of the gain mask, embodiments of the present disclosure may use various cues extracted at various places in a noise suppression (NS) system. In addition to an estimated SNR, additional cues such as an ILD, IPD, coherence, and other intermediate features extracted by blocks upstream of the gain mask generation may be used. Cues extracted from previous or following spectral frames, as well as from adjacent frequency taps, may also be used.

The set of cues may then be used in a machine-learning framework, along with the “oracle” ideal gain mask (e.g., which may be extracted when the clean speech is available), to derive a mapping between the cues and the mask. The mapping may be implemented, for example, as one or more machine-learning algorithms including a non-linear transformation, linear transformation, statistical algorithms, neural networks, regression tree methods, GMMs, heuristic algorithms, support vector machine algorithms, k-nearest neighbor algorithms, and so forth. The mapping may be learned from a training database, and one such mapping may exist per frequency domain tap or per group of frequency domain taps.

During this processing, the extracted cues may be fed to the mapper, and the gain mask may be provided by the output of the mapper and applied to the noisy signal, yielding a “de-noised” spectral representation of the signal. From the spectral representation, the time-domain signal may be reconstructed and provided to the ASR engine. In further embodiments, automatic speech processing specific cues may be derived from the spectral representation of the signal. The automatic speech processing cues may be but are not limited to automatic speech recognition, language recognition, keyword recognition, speech confirmation, emotion detection, voice sensing, and speaker recognition. The cues may be provided to the automatic speech processing engine directly, e.g., bypassing the automatic speech processing engine's front end. Although descriptions may be included by way of example to automatic speech recognition (ASR) and features thereof to help describe certain embodiments, various embodiments are not so limited and may include other automatic speech processing and features thereof.

Other embodiments of the present disclosure may include working directly in the automatic speech processing feature, e.g., ASR feature, domain. During the training phase, available NS cues may be produced (as discussed above), and the ASR cues may be extracted from both the clean and the noisy signals. The training phase may then learn an optimal mapping scheme that transforms the NS cues and noisy ASR cues into clean ASR features. In other words, instead of learning a mapping from the NS cues to a gain mask, the mapping may be learned directly from NS cues and noisy ASR cues to the clean ASR cues. During normal processing of input audio signal, the NS cues and noisy ASR cues provided to the mapper, which produces clean ASR cues, which in turn may be used by the ASR engine.

In various embodiments of the present disclosure, the optimal gain mask may be derived from a series of cues extracted from the input noisy signal in a data-driven or machine-learning approach. The training process for these techniques may select the cues that provide substantial information to produce a more accurate approximation of the ideal gain mask. Furthermore, in the case of the use of regression trees as machine-learning techniques, substantially informative features may be dynamically selected at run time when the tree is traversed.

These and other embodiments will be now described in greater details with respect to various embodiments and with reference to accompanying drawings.

Example System Implementation

FIG. 1 is an illustration of an environment in which embodiments of the present technology may be used. A user may act as an audio source 102 (e.g., speech source 102 or user 102) to an audio device 104. The exemplary audio device 104 may include two microphones: a primary microphone 106 relative to the audio source 102 and a secondary microphone 108 located a distance away from the primary microphone 106. Alternatively, the audio device 104 may include a single microphone. In yet other embodiments, the audio device 104 may include more than two microphones, such as, for example, three, four, five, six, seven, eight, nine, ten or even more microphones. The audio device 104 may constitute or be a part of, for example, a wireless telephone or a computer.

The primary microphone 106 and secondary microphone 108 may be omnidirectional microphones. Alternatively, embodiments may utilize other forms of microphones or acoustic sensors, such as directional microphones.

While the microphones 106 and 108 receive sound (i.e., audio signals) from the audio source 102, the microphones 106 and 108 also pick up noise 110. Although the noise 110 is shown coming from a single location in FIG. 1, the noise 110 may include any sounds from one or more locations that differ from the location of audio source 102, and may include reverberations and echoes. The noise 110 may be stationary, non-stationary, and/or a combination of both stationary and non-stationary noise.

Some embodiments may utilize level differences (e.g., energy differences) between the audio signals received by the two microphones 106 and 108. Because the primary microphone 106 is much closer to the audio source 102 than the secondary microphone 108 in a close-talk use case, the intensity level is higher for the primary microphone 106, resulting in a larger energy level received by the primary microphone 106 during a speech/voice segment, for example.

The level difference may then be used to discriminate speech and noise in the time-frequency domain. Further embodiments may use a combination of energy level differences and time delays to discriminate speech. Based on such inter-microphone differences, speech signal extraction or speech enhancement may be performed.

FIG. 2 is a block diagram of an exemplary audio device 104. In the illustrated embodiment, the audio device 104 includes a receiver 200, a processor 202, the primary microphone 106, an optional secondary microphone 108, an audio processing system 210, and an output device 206. The audio device 104 may include further or other components necessary for audio device 104 operations. Similarly, the audio device 104 may include fewer components that perform similar or equivalent functions to those depicted in FIG. 2.

The processor 202 may execute instructions and modules stored in a memory (not illustrated in FIG. 2) in the audio device 104 to perform functionality described herein, including noise reduction for an audio signal. The processor 202 may include hardware and software implemented as a processing unit, which may process floating point operations and other operations for the processor 202.

The exemplary receiver 200 is an acoustic sensor configured to receive or transmit a signal from a communications network. Hence, receiver 200 may be used as a transmitter in addition to a receiver. In some embodiments, the receiver 200 may include an antenna device. The signal may then be forwarded to the audio processing system 210 to reduce noise using the techniques described herein, and provide an audio signal to the output device 206. The present technology may be used in the transmit path and/or receive path of the audio device 104.

The audio processing system 210 is configured to receive the audio signals from an acoustic source via the primary microphone 106 and secondary microphone 108 and process the audio signals. Processing may include performing noise reduction within an audio signal. The audio processing system 210 is discussed in more detail below. The primary and secondary microphones 106, 108 may be spaced a distance apart in order to allow for detecting an energy level difference, time difference, or phase difference between the audio signals received by the microphones. The audio signals received by primary microphone 106 and secondary microphone 108 may be converted into electrical signals (i.e., a primary electrical signal and a secondary electrical signal). The electrical signals may themselves be converted by an analog-to-digital converter (not shown) into digital signals for processing, in accordance with some embodiments.

In order to differentiate the audio signals for clarity purposes, the audio signal received by the primary microphone 106 is herein referred to as the primary audio signal, while the audio signal received from by the secondary microphone 108 is herein referred to as the secondary audio signal. The primary audio signal and the secondary audio signal may be processed by the audio processing system 210 to produce a signal with an improved signal-to-noise ratio. It should be noted that embodiments of the technology described herein may be practiced utilizing only the primary microphone 106.

The output device 206 is any device that provides an audio output to the user. For example, the output device 206 may include a speaker, an earpiece of a headset or handset, or a speaker on a conference device.

Noise Suppression by Estimating Gain Mask

FIG. 3 is a block diagram of an exemplary audio processing system 210. The audio processing system 210 of this figure may provide for noise suppression of digital audio signals to be used, for example, in the audio processing system of FIG. 2. The audio processing system 210 may include a frequency analysis module 310, a machine-learning (MN) module 320, optional reconstruction (Recon) module 330, and optional ASR engine 340. The MN module 320 in turn may include a feature extraction (FE) module 350, a mask generator (MG) module 360, a memory 370, and a modifier (MOD) module 380.

In operation, the audio processing system 210 may receive input audio signals including one or more time-domain input signals from the primary microphone 106 and the secondary microphone 108. The input audio signals, when combined by the frequency analysis module 310, may represent noisy speech to be pre-processed before applying to the ASR engine 340. The frequency analysis module 310 may be used to combine the signals from the primary microphone 106 and the secondary microphone 108 and optionally transform them into a frequency-domain for further noise suppression pre-processing.

Further, the noisy speech signal may be fed to the FE module 350, which is used for extraction of one or more cues from the noisy speech. As discussed, these cues may refer to at least one of ILD cues, IPD cues, energy at channel cues, VAD cues, spatial cues, frequency cues, Wiener gain mask estimates, pitch-based cues, periodicity-based cues, noise estimates, context cues, and so forth. The cues may further be fed to the MG module 360 for performing a mapping operation and determining an appropriate gain mask or gain mask estimate based thereon. The MG module 360 may include a mapper (not shown), which employs one or more machine-learning techniques. The mapper may use tables or sets of predetermined reference cues of noise and cues of clean speech stored in the memory to map predefined cues with newly extracted ones in a dynamic, regular manner. As a result of mapping, the mapper may associate the extracted cues with predefined cues of clean speech and/or predefined noise so as to calculate gain factors or a gain map for further input signal processing. In particular, the MOD module 380 applies the gain factors or gain mask to the noise signal to perform noise suppression. The resulting signal with noise suppressed characteristics may be then fed to the Recon module 330 and the ASR engine 340 or directly to the ASR engine 340.

Training System

FIG. 4 is a block diagram of an exemplary training system environment 400. The environment 400 of this figure may provide more detail for the audio processing system of FIG. 2 and may be a part of the audio processing system 210. As shown in the figure, the environment 400 may include a training system 410, a clean speech database 420, a noise database 430, and a mapping module 440.

As follows from this figure, a frequency analysis module 450 and/or combination module 460 of the training system 410 may receive predetermined reference clean speech signals and predetermined reference noise signals from the clean speech database 420 and the noise database 430, respectively. These reference clean speech and noise signals may be combined by a combination module 460 of the training system 410 into “synthetic” noisy speech signals. The synthetic noisy speech signals may then be processed, and one or more cues may be extracted therefrom, by a Frequency Extractor (FE) module 470 of the training system 410. As discussed, these cues may refer to at least one of ILD cues, IPD cues, energy at channel cues, VAD cues, spatial cues, frequency cues, Wiener gain mask estimates, pitch-based cues, periodicity-based cues, noise estimates, context cues, and so forth.

With continuing reference to FIG. 4, a learning module 480 of the training system 410 may apply one or more machine-learning algorithms such as regression trees, a non-linear transform algorithms, linear transform algorithms, statistical or heuristic algorithms, neural networks, or a GMM to determine mapping between the cues and gain coefficients using reference clean speech and noise signals. It should be noted that in some embodiments, the one or more machine-learning algorithms of the training system 410 may be the same machine-learning algorithms as used in the MG module 360. In some other alternative embodiments, the one or more machine-learning algorithms of the training system 410 differ from the one or more machine-learning algorithms used in the MG module 360. In either case, the learning module 480 may employ the one or more machine learning algorithms to determine mapping between the extracted cues and one or more gain coefficients or factors utilizing the reference clean speech signals from the clean speech database 420 and using the reference noise signals from the noise database 430. The result of the determination may then be provided to optional mapping module 440 for further use. In other words, the mapping module 440 may store the correlation between synthetic noise speech and reference clean and reference noise signals for appropriate selection or construction of a gain mask in the system. The mapping may be optionally stored in the memory 370.

Example Operation Principles

FIG. 5 illustrates a flow chart of example method 500 for training a machine-learning technique used for noise suppression. The method 500 may be practiced, for example, by the training system 410 and its components as described above with references to FIG. 4.

The method 500 may commence in operation 510 with the frequency analysis module 450 receiving reference clean speech and reference noise from the databases 420, 430, accordingly, or from one or more microphones (e.g., the primary microphone 106 and the secondary microphone 108). At operation 520, the combination module 460 may generate noisy speech using the clean speech and the noise as received by the frequency analysis module 450. At operation 530, the FE module 470 extracts NS cues from noisy speech and oracle gain from clean speech. At operation 540, the learning module 480 may determine/generate a mapping from the NS cues to the oracle gain using one or more machine learning techniques.

FIG. 6 illustrates a flow chart of example method 600 for noise suppression. The method 600 may be practiced, for example, by the audio processing system 210 and its components as described above with references to FIG. 3.

The method 600 may commence in operation 610 with the frequency analysis module 310 receiving noisy speech from the primary microphone 106 and the secondary microphone 108 (e.g., the inputs from both microphones may be combined into a single signal and transformed from time-domain to a frequency domain). At this operation, the memory 370 may also provide or receive an appropriate mapping data generated at a training process of at least one machine-learning technique as discussed above, for example, with reference to FIG. 5.

Further, at operation 620, the FE module 350 extracts one or more cues from the noisy speech as received by the frequency analysis module 310. The cues may refer to at least one of ILD cues, IPD cues, energy at channel cues, VAD cues, spatial cues, frequency cues, Wiener gain mask estimates, pitch-based cues, periodicity-based cues, noise estimates, context cues, and so forth. At operation 630, the MG module 360 determines a gain mask from the cues using the mapping and a selected one or more machine-learning algorithms. At operation 640, the MOD module 380 applies the gain mask (e.g., a set of gain coefficients in a frequency domain) to the noisy speech so as to suppress unwanted noise levels. At operation 650, the Recon module 330 may reconstruct the noise suppressed speech signal and optionally transform it from the frequency domain into a time domain.

FIG. 7 illustrates a flow chart of yet another example method 700 for training a machine-learning technique used for noise suppression. The method 700 may be practiced, for example, by the training system 410 and its components as described above with references to FIG. 4.

The method 700 may commence in operation 710 with the frequency analysis module 450 receiving predetermined reference clean speech from the clean speech database 420 and predetermined reference noise from the noise database 430. At operation 720, the combination module 460 may generate noisy speech using the clean speech and the noise received by the frequency analysis module 450. At operation 730, the FE module 470 may extract noisy automatic speech processing cues and NS cues from the noisy speech and clean ASR cues from clean speech. The automatic speech processing cues may be, but are not limited to, automatic speech recognition, language recognition, keyword recognition, speech confirmation, emotion detection, voice sensing, or speaker recognition cues. At operation 740, the learning module 480 may determine/generate a mapping from noisy automatic speech processing cues and NS cues to clean automatic speech processing cues, the mapping may be optionally stored in the memory 370 of FIG. 3 for future use.

FIG. 8 illustrates a flow chart of yet another example method 800 for noise suppression. The method 800 may be practiced, for example, by the audio processing system 210 and its components as described above with references to FIG. 3.

The method 800 may commence in operation 810 with the frequency analysis module 310 receiving noisy speech from the primary microphone 106 and the secondary microphone 108, and with the memory 370 providing or receiving mapping data generated at a training process of at least one machine-learning technique as discussed above, for example, with reference to FIG. 7.

Further, at operation 820, the FE module 350 extracts NS and automatic speech processing cues from the input noisy speech. At operation 830, the MOD module 380 may apply the mapping to produce clean automatic speech processing features. The automatic speech processing features may be, but are not limited to, automatic speech recognition, language recognition, keyword recognition, speech confirmation, emotion detection, voice sensing, or speaker recognition features. In one example for ASR, at operation 840, the clean automatic speech processing features are fed into the ASR engine 340 for speech recognition. In this method, the ASR engine 340 may generate clean speech signals based on the clean automatic speech processing (e.g., ASR) features without a need to reconstruct the noisy input signal.

In some embodiments, the processing of the noise suppression for speech processing based on machine-learning mask estimation may be cloud-based.

Example Computer System

FIG. 9 is a diagrammatic representation of an example machine in the form of a computer system 900, within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed.

In various example embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a PC, a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a portable music player (e.g., a portable hard drive audio device such as a Moving Picture Experts Group Audio Layer 3 (MP3) player), a web appliance, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

The example computer system 900 includes a processor or multiple processors 910 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), memory 920, static mass storage 930, portable storage device 940, which communicate with each other via a bus 990. The computer system 900 may further include a graphics display unit 970 (e.g., a liquid crystal display (LCD), touchscreen and the like). The computer system 900 may also include input devices 960 (e.g., physical and/or virtual keyboard, keypad, a cursor control device, a mouse, touchpad, touchscreen, and the like), output devices 950 (e.g., speakers), peripherals 980 (e.g., a speaker, one or more microphones, printer, modem, communication device, network adapter, router, radio, modem, and the like). The computer system 900 may further include a data encryption module (not shown) to encrypt data.

The memory 920 and/or mass storage 930 include a computer-readable medium on which is stored one or more sets of instructions and data structures (e.g., instructions) embodying or utilizing any one or more of the methodologies or functions described herein. The instructions may also reside, completely or at least partially, within the main memory 920 and/or within the processors 910 during execution thereof by the computer system 900. The memory 920 and the processors 910 may also constitute machine-readable media. The instructions may further be transmitted or received over a wired and/or wireless network (not shown) via the network interface device (e.g. peripherals 980). While the computer-readable medium discussed herein in an example embodiment is a single medium, the term “computer-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database and/or associated caches and servers) that store the one or more sets of instructions. The term “computer-readable medium” shall also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the machine and that causes the machine to perform any one or more of the methodologies of the present application, or that is capable of storing, encoding, or carrying data structures utilized by or associated with such a set of instructions. The term “computer-readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic media. Such media may also include, without limitation, hard disks, floppy disks, flash memory cards, digital video disks, random access memory (RAM), read only memory (ROM), and the like.

In some embodiments, the computing system 900 may be implemented as a cloud-based computing environment, such as a virtual machine operating within a computing cloud. In other embodiments, the computing system 900 may itself include a cloud-based computing environment, where the functionalities of the computing system 900 are executed in a distributed fashion. Thus, the computing system 900, when configured as a computing cloud, may include pluralities of computing devices in various forms, as will be described in greater detail below.

In general, a cloud-based computing environment is a resource that typically combines the computational power of a large grouping of processors (such as within web servers) and/or that combines the storage capacity of a large grouping of computer memories or storage devices. Systems that provide cloud-based resources may be utilized exclusively by their owners or such systems may be accessible to outside users who deploy applications within the computing infrastructure to obtain the benefit of large computational or storage resources.

The cloud may be formed, for example, by a network of web servers that comprise a plurality of computing devices, such as the computing device 200, with each server (or at least a plurality thereof) providing processor and/or storage resources. These servers may manage workloads provided by multiple users (e.g., cloud resource customers or other users). Typically, each user places workload demands upon the cloud that vary in real-time, sometimes dramatically. The nature and extent of these variations typically depends on the type of business associated with the user.

While the present embodiments have been described in connection with a series of embodiments, these descriptions are not intended to limit the scope of the subject matter to the particular forms set forth herein. It will be further understood that the methods are not necessarily limited to the discrete components described. To the contrary, the present descriptions are intended to cover such alternatives, modifications, and equivalents as may be included within the spirit and scope of the subject matter as disclosed herein and defined by the appended claims and otherwise appreciated by one of ordinary skill in the art.

Claims

1. A method for noise suppression, comprising: receiving, by a first processor communicatively coupled with a first memory, first noisy speech, the first noisy speech obtained using two or more microphones;extracting, by the first processor, one or more first cues from the first noisy speech, the one or more first cues including cues associated with noise suppression and automatic speech processing; andcreating clean automatic speech processing features using a mapping and the extracted one or more first cues, the clean automatic speech processing features being for use in automatic speech processing and the mapping being provided by a process including: receiving, by a second processor communicatively coupled with a second memory, clean speech and noise;producing, by the second processor, second noisy speech using the clean speech and the noise;extracting, by the second processor, one or more second cues from the second noisy speech, the one or more second cues including cues associated with noise suppression and noisy automatic speech processing;extracting clean automatic speech processing cues from the clean speech; andgenerating, by the second processor, the mapping from the one or more second cues to the clean automatic speech processing cues, the generating including at least one machine-learning technique.

2. The method of claim 1, wherein the automatic speech processing comprises automatic speech recognition.

3. The method of claim 1, wherein the automatic speech processing comprises one or more of automatic speech recognition, language recognition, keyword recognition, speech confirmation, emotion detection, voice sensing, and speaker recognition.

4. The method of claim 1, wherein receiving, by the second processor, the clean speech and the noise comprises receiving predetermined reference clean speech and predetermined reference noise from a reference database.

5. The method of claim 1, wherein the clean speech and noise are each obtained using at least two microphones, the one or more first and second cues each including at least one inter-microphone level difference (ILD) cues and inter-microphone phase difference (IPD) cues.

6. The method of claim 4, wherein the automatic speech processing comprises one or more of automatic speech recognition, language recognition, keyword recognition, speech confirmation, emotion detection, voice sensing, and speaker recognition.

7. The method of claim 1, wherein the one or more first cues and the one or more second cues each further include at least one of energy at channel cues, voice activity detection (VAD) cues, spatial cues, frequency cues, Wiener gain mask estimates, pitch-based cues, periodicity-based cues, noise estimates, and context cues.

8. The method of claim 1, wherein the at least one machine-learning technique includes one or more of a neural network, regression tree, a nonlinear transform, a linear transform, and a Gaussian Mixture Model (GMM).

9. The method of claim 1, wherein the generating applies the at least one machine-learning technique to the clean speech and the second noisy speech.

10. A system for noise suppression, comprising: a first frequency analysis module, executed by at least one processor, that is configured to receive first noisy speech, the first noisy speech being each obtained using at least two microphones;a second frequency analysis module, executed by the at least one processor, that is configured to receive clean speech and noise;a combination module, executed by the at least one processor, that is configured to produce second noisy speech using the clean speech and the noise;a first cue extraction module, executed by the at least one processor, that is configured to extract one or more first cues from the first noisy speech, the one or more first cues including cues associated with noise suppression and automatic speech processing;a second cue extraction module, executed by the at least one processor, that is configured to extract one or more second cues from the second noisy speech, the one or more second cues including cues associated with noise suppression and noisy automatic speech processing;a third cue extraction module, executed by the at least one processor, that is configured to extract clean automatic speech processing cues from the clean speech; anda learning module, executed by the at least one processor, that is configured to generate a mapping from the one or more second cues associated with the noise suppression cues and the noisy automatic speech processing cues to the clean automatic speech processing cues, the generating including at least one machine-learning technique; anda modification module, executed by the at least one processor, that is configured to create clean automatic speech processing features using the mapping and the extracted one or more first cues, the clean automatic speech processing features being for use in automatic speech processing.

11. The system of claim 10, wherein the automatic speech processing comprises automatic speech recognition.

12. The system of claim 10, wherein the automatic speech processing comprises one or more of automatic speech recognition, language recognition, keyword recognition, speech confirmation, emotion detection, voice sensing, and speaker recognition.

13. The system of claim 10, wherein the second frequency analysis module is configured to receive the clean speech and the noise from a reference database, the clean speech and noise being predetermined reference clean speech and predetermined reference noise.

14. The system of claim 10, wherein the at least one machine-learning technique includes one or more of a neural network, regression tree, a non-linear transform, a linear transform, and a Gaussian Mixture Model (GMM).

15. The system of claim 10, wherein the one or more first cues and the one or more second cues each include at least one of ILD cues and IPD cues.

16. The system of claim 10, wherein the one or more first cues and the one or more second cues each include at least one of energy at channel cues, VAD cues, spatial cues, frequency cues, Wiener gain mask estimates, pitch-based cues, periodicity-based cues, noise estimates, and context cues.

17. The system of claim 14, wherein the at least one machine-learning techniques each include one or more of a neural network, regression tree, a non-linear transform, a linear transform, and a GMM.

18. The method of claim 1, wherein the first processor communicatively coupled with the first memory are included in a cloud-based computing environment.