This application relates to acoustic devices and, more specifically, to the operation of these devices.
Different types of acoustic devices have been used through the years. One type of device is a microphone. In a microelectromechanical system (MEMS) microphone, a MEMS die includes a diagram and a back plate. The MEMS die is supported by a substrate and enclosed by a housing (e.g., a cup or cover with walls). A port may extend through the substrate (for a bottom port device) or through the top of the housing (for a top port device). In any case, sound energy traverses through the port, moves the diaphragm and creates a changing potential of the back plate, which creates an electrical signal. Microphones are deployed in various types of devices such as personal computers or cellular phones.
Microphones are used in various applications that utilize voice trigger applications. In previous approaches, an acoustic activity detector detects a voice signal and sends out a signal to wake up a digital signal processor (DSP) for the detection of key phrases in the voice. Once the key phrase is found, all input speech data can be processed. Consequently, any time that the acoustic activity detector is triggering, the DSP is constantly searching for key phrases using power. Mobile and wearable devices have small batteries that can be depleted by the repeated triggering described above.
The problems of previous approaches have resulted in some user dissatisfaction with these previous approaches.
For a more complete understanding of the disclosure, reference should be made to the following detailed description and accompanying drawings wherein:
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity. It will further be appreciated that certain actions and/or steps may be described or depicted in a particular order of occurrence while those skilled in the art will understand that such specificity with respect to sequence is not actually required. It will also be understood that the terms and expressions used herein have the ordinary meaning as is accorded to such terms and expressions with respect to their corresponding respective areas of inquiry and study except where specific meanings have otherwise been set forth herein.
The present approaches provide for low power operation of a microphone during voice triggering applications. The output of the acoustic activity detector inside of the microphone is stored in internal memory (e.g., a random access memory (RAM)) via direct memory access (DMA) techniques. When the memory device reaches a predetermined capacity, a digital signal processor (DSP) (or other processing device) is woken up and the stored data is clocked from the internal memory device to the DSP via DMA (e.g., at a high frequency) via some data bus (e.g., an advanced high-speed bus (AHB)).
Power consumption is reduced (especially in noisy environments) because in the approaches presented herein the DSP is periodically activated for processing small fragments of data very quickly to determine if a key phrase was detected. Also, the system is enabled to deactivate the DSP at times when acoustic activity is detected. Additionally, the present approaches allow for the periodic wake up and sleep of the DSP in noisy environments when the acoustic activity detector (AAD) would (in previous systems) be triggering.
In many of these embodiments, microphone output triggered by an acoustic activity detector (AAD) is clocked into memory. When the data input into the memory reaches a predetermined value, the data is clocked out of the memory at a high frequency to a digital signal processor (DSP) via a data bus. If any part of a predetermined phrase is found by the DSP, the DSP processes more data stored in the memory to determine if the key phrase was received. If the entire phrase is recovered, the entire system (e.g., the DSP and associated consumer electronic devices it may be coupled to) is awakened. If the entire phrase is not recovered, the DSP returns to a sleep (low power) mode of operation.
Referring now to
The charge pump 102 is a voltage or current source that is used to charge the MEMS device 104. The MEMS device 104 includes a diaphragm and a back plate, and converts sound energy into electrical signals. The sigma delta converter 106 converts analog electrical signals into pulse density modulation (PDM) data.
The AAD module 108 determines whether voice is detected in the incoming signal from the MEMS device 104. These functions may be accomplished by various techniques known to those skilled in the art. The buffer 110 stores data, and in one example provides 250 ms of delay. The trigger control module 112 is triggered to release data when human voice is detected by the AAD module 108. The decimator 114 converts the PDM data into PCM data. The DMA control module 116 controls the flow of data to and from the memory 120, and to the DSP 122. The memory controller 118 keeps a record of the amount of data that the DMA control module has loaded into the memory 120 and informs the DMA control module 116 when this amount exceeds a predetermined value. The DSP 122 determines whether particular trigger words or phrases are present in the data.
It will be appreciated that these elements may be implemented in any combination of computer hardware and/or software. For instance, many if not all of these elements may be implemented using computer instructions executed on a processor. It will be further appreciated that these components may be disposed within a single assembly or covering structure.
In one example of the operation of the system of
The DSP 122 looks for any part of the key phrase. If any part is detected (even if in the later half of the phrase), the DSP 122 looks further back in the data to see if the beginning of the phrase was recorded to correlate for key word recognition. The above steps may be repeated if the memory 120 reaches the predetermined threshold again. It will be appreciated that various types of digital data (e.g., PDM, PCM and SoundWire).
Referring now to
Beginning with state 202, the system senses sound energy, for example, using a MEMS device (but other transducers such as piezoelectric transducers can also be used). When voice activity is determined by the AAD module, control moves to state 204 where the data is written to memory, for example, a RAM.
When RAM reaches a predetermined capacity, control continues at step 206, where the DSP is woken up and a burst of data is transmitted from the RAM to the DSP using the DMA control module and a data bus. When the DSP receives the data, control continues at step 208, where key phrase recognition is performed. When no part of the predetermined key phrase is determined, control returns to step 202. When part of the phrase is determined, control continues with step 210.
At step 210, the DSP looks back in RAM for the entire phrase (assuming step 208 did not find the whole phrase). If the rest of the phrase is not found, control returns to step 202. If the phrase is found, the system is woken up to perform further processing since the key phrase was found.
Referring now to
It will be appreciated that while the higher frequency processing of greater amounts of data will require more power at some small intervals in time, it will allow the processing of data in significantly less periods of time. And, this mode of operation uses significantly less power than previous approaches.
Put another way, although the peak values of amounts 302 are higher than the peak value of power amounts 304, peak values 302 are consumed over very small periods or intervals of time, while power amounts 304 are consumed over comparatively much greater and longer periods or intervals of time. Thus, the total power consumed by power amounts 302 is significantly less than the power consumed by amounts 304.
Also, this mode of operation requires significantly less power consumption than previous voice triggers in noisy situations or environments when ambient noise levels are constantly triggering the AAD module.
Preferred embodiments of this invention are described herein, including the best mode known to the inventors for carrying out the invention. It should be understood that the illustrated embodiments are exemplary only, and should not be taken as limiting the scope of the invention.
This patent claims benefit under 35 U.S.C. §119(e) to U.S. Provisional Application No. 62/105,900 entitled “Low Power Voice Trigger for Acoustic Apparatus and Method” filed Jan. 21, 2015, the content of which is incorporated herein by reference in its entirety.
Number | Name | Date | Kind |
---|---|---|---|
4052568 | Jankowski | Oct 1977 | A |
5577164 | Kaneko | Nov 1996 | A |
5598447 | Usui | Jan 1997 | A |
5675808 | Gulick | Oct 1997 | A |
5822598 | Lam | Oct 1998 | A |
5983186 | Miyazawa | Nov 1999 | A |
6049565 | Paradine | Apr 2000 | A |
6057791 | Knapp | May 2000 | A |
6070140 | Tran | May 2000 | A |
6154721 | Sonnic | Nov 2000 | A |
6249757 | Cason | Jun 2001 | B1 |
6282268 | Hughes | Aug 2001 | B1 |
6324514 | Matulich | Nov 2001 | B2 |
6397186 | Bush | May 2002 | B1 |
6453020 | Hughes | Sep 2002 | B1 |
6564330 | Martinez | May 2003 | B1 |
6591234 | Chandran | Jul 2003 | B1 |
6640208 | Zhang | Oct 2003 | B1 |
6756700 | Zeng | Jun 2004 | B2 |
7190038 | Dehe | Mar 2007 | B2 |
7415416 | Rees | Aug 2008 | B2 |
7473572 | Dehe | Jan 2009 | B2 |
7619551 | Wu | Nov 2009 | B1 |
7630504 | Poulsen | Dec 2009 | B2 |
7774202 | Spengler | Aug 2010 | B2 |
7774204 | Mozer | Aug 2010 | B2 |
7781249 | Laming | Aug 2010 | B2 |
7795695 | Weigold | Sep 2010 | B2 |
7825484 | Martin | Nov 2010 | B2 |
7829961 | Hsiao | Nov 2010 | B2 |
7856283 | Burk | Dec 2010 | B2 |
7856804 | Laming | Dec 2010 | B2 |
7903831 | Song | Mar 2011 | B2 |
7936293 | Hamashita | May 2011 | B2 |
7941313 | Garudadri | May 2011 | B2 |
7957972 | Huang | Jun 2011 | B2 |
7994947 | Ledzius | Aug 2011 | B1 |
8171322 | Fiennes | May 2012 | B2 |
8208621 | Hsu | Jun 2012 | B1 |
8275148 | Li | Sep 2012 | B2 |
8331581 | Pennock | Dec 2012 | B2 |
8666751 | Murthi | Mar 2014 | B2 |
8687823 | Loeppert | Apr 2014 | B2 |
8731210 | Cheng | May 2014 | B2 |
8798289 | Every | Aug 2014 | B1 |
8804974 | Melanson | Aug 2014 | B1 |
8849231 | Murgia | Sep 2014 | B1 |
8972252 | Hung | Mar 2015 | B2 |
8996381 | Mozer | Mar 2015 | B2 |
9020819 | Saitoh | Apr 2015 | B2 |
9043211 | Haiut | May 2015 | B2 |
9059630 | Gueorguiev | Jun 2015 | B2 |
9073747 | Ye | Jul 2015 | B2 |
9076447 | Nandy | Jul 2015 | B2 |
9111548 | Nandy | Aug 2015 | B2 |
9112984 | Sejnoha | Aug 2015 | B2 |
9113263 | Furst | Aug 2015 | B2 |
9119150 | Murgia | Aug 2015 | B1 |
9142215 | Rosner | Sep 2015 | B2 |
9147397 | Thomsen | Sep 2015 | B2 |
9161112 | Ye | Oct 2015 | B2 |
20020054588 | Mehta | May 2002 | A1 |
20020116186 | Strauss | Aug 2002 | A1 |
20020123893 | Woodward | Sep 2002 | A1 |
20020184015 | Li | Dec 2002 | A1 |
20030004720 | Garudadri | Jan 2003 | A1 |
20030061036 | Garudadri | Mar 2003 | A1 |
20030144844 | Colmenarez | Jul 2003 | A1 |
20040022379 | Klos | Feb 2004 | A1 |
20050207605 | Dehe | Sep 2005 | A1 |
20060074658 | Chadha | Apr 2006 | A1 |
20060233389 | Mao | Oct 2006 | A1 |
20060247923 | Chandran | Nov 2006 | A1 |
20070168908 | Paolucci | Jul 2007 | A1 |
20070278501 | MacPherson | Dec 2007 | A1 |
20080089536 | Josefsson | Apr 2008 | A1 |
20080175425 | Roberts | Jul 2008 | A1 |
20080201138 | Visser | Aug 2008 | A1 |
20080267431 | Leidl | Oct 2008 | A1 |
20080279407 | Pahl | Nov 2008 | A1 |
20080283942 | Huang | Nov 2008 | A1 |
20090001553 | Pahl | Jan 2009 | A1 |
20090180655 | Tien | Jul 2009 | A1 |
20100046780 | Song | Feb 2010 | A1 |
20100052082 | Lee | Mar 2010 | A1 |
20100057474 | Kong | Mar 2010 | A1 |
20100128894 | Petit | May 2010 | A1 |
20100128914 | Khenkin | May 2010 | A1 |
20100131783 | Weng | May 2010 | A1 |
20100183181 | Wang | Jul 2010 | A1 |
20100246877 | Wang | Sep 2010 | A1 |
20100290644 | Wu | Nov 2010 | A1 |
20100292987 | Kawaguchi | Nov 2010 | A1 |
20100322443 | Wu | Dec 2010 | A1 |
20100322451 | Wu | Dec 2010 | A1 |
20110007907 | Park | Jan 2011 | A1 |
20110013787 | Chang | Jan 2011 | A1 |
20110029109 | Thomsen | Feb 2011 | A1 |
20110075875 | Wu | Mar 2011 | A1 |
20110106533 | Yu | May 2011 | A1 |
20110208520 | Lee | Aug 2011 | A1 |
20110280109 | Raymond | Nov 2011 | A1 |
20120010890 | Koverzin | Jan 2012 | A1 |
20120232896 | Taleb | Sep 2012 | A1 |
20120250881 | Mulligan | Oct 2012 | A1 |
20120310641 | Niemisto | Dec 2012 | A1 |
20130044898 | Schultz | Feb 2013 | A1 |
20130058506 | Boor | Mar 2013 | A1 |
20130223635 | Singer | Aug 2013 | A1 |
20130226324 | Hannuksela | Aug 2013 | A1 |
20130246071 | Lee | Sep 2013 | A1 |
20130322461 | Poulsen | Dec 2013 | A1 |
20130343584 | Bennett | Dec 2013 | A1 |
20140064523 | Kropfitsch | Mar 2014 | A1 |
20140122078 | Joshi | May 2014 | A1 |
20140143545 | McKeeman | May 2014 | A1 |
20140163978 | Basye | Jun 2014 | A1 |
20140177113 | Gueorguiev | Jun 2014 | A1 |
20140188467 | Jing | Jul 2014 | A1 |
20140188470 | Chang | Jul 2014 | A1 |
20140197887 | Hovesten | Jul 2014 | A1 |
20140244269 | Tokutake | Aug 2014 | A1 |
20140244273 | Laroche | Aug 2014 | A1 |
20140249820 | Hsu | Sep 2014 | A1 |
20140257813 | Mortensen | Sep 2014 | A1 |
20140257821 | Adams | Sep 2014 | A1 |
20140274203 | Ganong | Sep 2014 | A1 |
20140278435 | Ganong | Sep 2014 | A1 |
20140281628 | Nigam | Sep 2014 | A1 |
20140337036 | Haiut | Nov 2014 | A1 |
20140343949 | Huang | Nov 2014 | A1 |
20140348345 | Furst | Nov 2014 | A1 |
20140358552 | Xu | Dec 2014 | A1 |
20150039303 | Lesso | Feb 2015 | A1 |
20150043755 | Furst | Feb 2015 | A1 |
20150046157 | Wolff | Feb 2015 | A1 |
20150046162 | Aley-Raz | Feb 2015 | A1 |
20150049884 | Ye | Feb 2015 | A1 |
20150055803 | Qutub | Feb 2015 | A1 |
20150058001 | Dai | Feb 2015 | A1 |
20150063594 | Nielsen | Mar 2015 | A1 |
20150073780 | Sharma | Mar 2015 | A1 |
20150073785 | Sharma | Mar 2015 | A1 |
20150088500 | Conliffe | Mar 2015 | A1 |
20150106085 | Lindahl | Apr 2015 | A1 |
20150110290 | Furst | Apr 2015 | A1 |
20150112690 | Guha | Apr 2015 | A1 |
20150134331 | Millet | May 2015 | A1 |
20150154981 | Barreda | Jun 2015 | A1 |
20150161989 | Hsu | Jun 2015 | A1 |
20150195656 | Ye | Jul 2015 | A1 |
20150206527 | Connolly | Jul 2015 | A1 |
20150256660 | Kaller | Sep 2015 | A1 |
20150256916 | Volk | Sep 2015 | A1 |
20150287401 | Lee | Oct 2015 | A1 |
20150302865 | Pilli | Oct 2015 | A1 |
20150304502 | Pilli | Oct 2015 | A1 |
20150350760 | Nandy | Dec 2015 | A1 |
20150350774 | Furst | Dec 2015 | A1 |
20160012007 | Popper | Jan 2016 | A1 |
20160087596 | Yurrtas | Mar 2016 | A1 |
20160133271 | Kuntzman | May 2016 | A1 |
20160134975 | Kuntzman | May 2016 | A1 |
Number | Date | Country |
---|---|---|
2001236095 | Aug 2001 | JP |
2004219728 | Aug 2004 | JP |
2009130591 | Jan 2009 | WO |
2011106065 | Jan 2011 | WO |
2011140096 | Feb 2011 | WO |
2013049358 | Jan 2013 | WO |
2013085499 | Jan 2013 | WO |
Entry |
---|
International Search Report and Written Opinion for PCT/US2016/013859 dated Apr. 29, 2016 (12 pages). |
U.S. Appl. No. 14/285,585, dated May 2014, Santos. |
U.S. Appl. No. 14/495,482, dated Sep. 2014, Murgia. |
U.S. Appl. No. 14/522,264, dated Oct. 2014, Murgia. |
U.S. Appl. No. 14/698,652, dated Apr. 2015, Yapanel. |
U.S. Appl. No. 14/749,425, dated Jun. 2015, Verma. |
U.S. Appl. No. 14/853,947, dated Sep. 2015, Yen. |
U.S. Appl. No. 62/100,758, filed Jan. 7, 2015, Rossum. |
“MEMS technologies: Microphone” EE Herald Jun. 20, 2013. |
Delta-sigma modulation, Wikipedia (Jul. 4, 2013). |
International Search Report and Written Opinion for PCT/EP2014/064324, dated Feb. 12, 2015 (13 pages). |
International Search Report and Written Opinion for PCT/US2014/038790, dated Sep. 24, 2014 (9 pages). |
International Search Report and Written Opinion for PCT/US2014/060567 dated Jan. 16, 2015 (12 pages). |
International Search Report and Written Opinion for PCT/US2014/062861 dated Jan. 23, 2015 (12 pages). |
Kite, Understanding PDM Digital Audio, Audio Precision, Beaverton, OR, 2012. |
Pulse-density modulation, Wikipedia (May 3, 2013). |
Search Report of Taiwan Patent Application No. 103135811, dated Apr. 18, 2016 (1 page). |
Number | Date | Country | |
---|---|---|---|
20160210051 A1 | Jul 2016 | US |
Number | Date | Country | |
---|---|---|---|
62105900 | Jan 2015 | US |