Mobile computing devices such as tablet computers, smart phones and the like may be provided with speakers, receivers and microphones, which may be disposed behind a grill, a port in a housing of the computing device, or the like. The characteristics of grills, ports or other supporting structures for audio components may be altered by debris or damage. Such alterations may degrade audio performance of the computing devices.
The accompanying figures, where like reference numerals refer to identical or functionally similar elements throughout the separate views, together with the detailed description below, are incorporated in and form part of the specification, and serve to further illustrate embodiments of concepts that include the claimed invention, and explain various principles and advantages of those embodiments.
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of embodiments of the present invention.
The apparatus and method components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
Examples disclosed herein are directed to a computing device including: a memory storing an audio sequence and a reference attribute associated with playback of the audio sequence; a speaker; a microphone; an audio enclosure supporting the speaker and the microphone; a processor configured to: simultaneously (i) control the speaker to play the audio sequence, and (ii) control the microphone to capture a test recording corresponding to playback of the audio sequence; based on a comparison of the reference attribute to a test attribute associated with the test recording, detect occlusion of the audio enclosure; and in response to detecting occlusion of the audio enclosure, generate a notification message.
Additional examples disclosed herein are directed to a method, comprising: simultaneously (i) controlling a speaker to play the audio sequence, and (ii) controlling a microphone to capture a test recording corresponding to playback of the audio sequence; based on a comparison of the reference attribute to a test attribute associated with the test recording, detecting occlusion of an audio enclosure containing the speaker and the microphone; and in response to detecting occlusion of the audio enclosure, generating a notification message.
Example ports 120 of the audio assembly 112 are shown in the detail view 116 of
The debris 124 in the ports, as well as damage to the ports 120 (e.g. due to impact suffered by the device 100) may reduce the performance of the speaker(s) and/or microphone(s). The device 100 therefore implements certain functionality, described herein, to periodically assess the performance of the audio assembly 112 and detect changes in audio performance that indicate structural changes to the assembly 112 such as debris or physical damage. The device 100 may therefore enable detection of degraded audio performance and trigger maintenance procedures such as cleaning or replacement of the assembly 112.
Turning to
The device 100 also includes the display 108 as noted above, a speaker 212 and a microphone 216. The speaker 212 and microphone 216 are components of the audio assembly 112 shown in
The memory 204 stores computer readable instructions for execution by the processor 200. In particular, the memory 204 stores an audio performance assessment application 220 which, when executed by the processor 200, configures the processor 200 to control the speaker 212 and microphone 216 to detect occlusions of the ports 120 or other disruptions to the audio assembly 112. Those skilled in the art will appreciate that the functionality implemented by the processor 200 via the execution of the application 220 may also be implemented by one or more specially designed hardware and firmware components, such as FPGAs, ASICs and the like in other embodiments.
Turning now to
As will be discussed below, the assessment of current audio performance to determine whether the ports 120 are occluded or damaged is performed by comparing data reflective of current audio performance to data reflective of reference audio performance. The reference audio performance is the performance of the audio assembly 112 when it is known not to be affected by occlusions, damage or the like. The functionality implemented via execution of the application 220 may therefore begin by obtaining such reference data.
In particular, at block 305 the processor 200 can be configured to play a predetermined audio sequence via the speaker 212, and generate at least one reference attribute associated with playback of the audio sequence. The audio sequence can be stored in the memory 204 in association with the application 220. At block 305, the processor 200 can therefore be configured to retrieve the audio sequence from the memory 204 and control the speaker 212 to play the audio sequence. Simultaneously with such playback, the processor 200 controls the microphone 216 to capture a reference recording associated with playback of the audio sequence. In other words, the reference recording is a recording of the speaker 212 playing the audio sequence.
Block 305 may be performed automatically upon a first startup of the device 100, or in response to an operator input indicating that the device 100 is in a reference state (e.g. the assembly 212 has been replaced, cleaned or the like), or loaded onto all devices of the same type and manufacturing batch in the factory as part of quality control. Thus, the reference recording captured at block 305 is assumed to represent an ideal level of performance of the audio assembly 112.
The reference attribute generated at block 305 can be the reference recording itself in some examples. In other examples, the reference attribute includes one or more attributes derived from the reference recording. Examples of reference attributes include one or more representations of the recorded power of the reference recording, such as root mean square (RMS) measurements, Mel-frequency cepstral coefficients (MFCC) and the like. Another example reference attribute includes a resonant frequency detected from the reference recording. For example, turning to
In some examples, multiple porting frequencies may be present (e.g. as distinct peaks), and more than one such resonant frequency may be employed as a reference attribute. In further examples, a front porting full width half maximum (FWHM) or other quality indicator may be employed as a reference attribute, instead of or in addition to the front porting resonance frequency itself. The FWHM may indicate how sharp the resonance peak is in the reference recording, and certain substances may render port resonances less well-defined (i.e. lower, broader peaks), which will be reflected in test FWHMs. More generally, the reference attribute may therefore be derived from at least one resonant frequency, whether the reference attribute is the resonant frequency itself or a related attribute such as the FWHM.
The audio sequence itself, which is employed both at block 305 and in subsequent portions of the method 300 to assess audio performance of the assembly 112, can take a wide variety of forms.
In the example of
Generally, the audio sequences may employ primarily, or exclusively, frequencies near or above the upper portion of the range of human hearing (e.g. above about 13 kHz). In some examples, the audio sequences may employ frequencies entirely outside the normal range of human hearing (e.g. above about 20 kHz). The use of such relatively high frequencies may reduce disruption to activities of the operator of the device 100 during automated assessment of audio performance. The use of relatively high frequencies may also reduce interference from environmental noise in the recordings captured by the device 100, as such interference is less likely to have the same frequencies as the audio sequence.
More generally, therefore, the audio sequences may have power spectral densities (PSDs) for lower frequencies that are smaller than their power spectral densities for higher frequencies. For example, the audio sequences may have PSDs below 4 kHz that are lower than their PSDs above 4 kHz by at least 40 dB. In other examples, the audio sequences may have PSDs below 8 kHz that are lower than their PSDs above 8 kHz by at least 37 dB. In further examples, the audio sequences may have PSDs below 4 kHz that are lower than their PSDs above 4 kHz by at least 33 dB.
Returning to
When the determination at block 310 is affirmative, the device 100 proceeds to block 315. At block 315, the processor 200 controls the speaker 212 to play the audio sequence mentioned above, and simultaneously controls the microphone 216 to capture a test recording of the playback of the audio sequence. The test recording can be stored in the memory 204 for further processing.
At block 320, the processor 200 can determine whether environmental conditions during the performance of block 315 are acceptable. For example, the test recording (or attributes derived therefrom) may deviate substantially from the reference recording if the conditions under which the recordings were obtained differ. The processor 200 may therefore be configured to evaluate any of a variety of conditions at block 320, including conditions indicated by the test recording itself and conditions indicated by other sensor data.
For example, the processor 200 may determine an average amplitude for the test recording and compare the amplitude to an average amplitude for the reference recording. If the amplitudes differ by an amount greater than a preconfigured threshold, the determination at block 320 is negative. Divergent amplitudes may indicate, for example, that the test recording was captured while the device 100 was in a bag, pocket or the like, while the reference recording was captured while the device 100 was in open air. In other examples, the processor 200 may determine via a light sensor, camera or the like whether an ambient light level during the performance of block 315 matches a stored ambient light level during the performance of block 305.
When the determination at block 320 is negative, the processor 200 can discard the test recording and return to block 310. When the determination at block 320 is affirmative, the processor 200 proceeds to block 325. In other examples, block 320 can simply be omitted. In further examples, the determination at block 320 can be performed before block 315. In particular, implementations in which the determination at block 320 depends on other sensor data than the test recording itself (e.g. camera data), the determination at block 320 may be performed before initiating playback of the audio sequence and capture of the test recording.
At block 325, the processor 200 is configured to compare the test recording to the reference recording. More specifically, the processor 200 is configured to compare a test attribute derived from the test recording to the reference attribute mentioned earlier (which was, in turn, derived from the reference recording from block 305).
The test attribute derived from the test recording is the same as the reference attribute described in connection with block 305. Thus, the test attribute can include the test recording itself, RMS samples from the test recording, or the like. A plurality of test attributes can also be generated, and the comparison at block 325 can therefore involve multiple distinct comparisons of reference and test attributes.
The comparison at block 325 yields an indication of a degree of difference between the reference recording and the test recording. Therefore, the comparison can be implemented according to a suitable function for determining the degree of similarity between data series. Examples of such functions include the Log-Euclidian distance, assessed by the Kolmogorov-Smirnov test (KS test), the Welch t-test, and the like. Combinations of the output of such functions may also be employed (e.g. a different function may be applied for each compared attribute, and the resulting scores or other outputs from the functions may be summed, averaged or the like).
In some examples, rather than the above-mentioned statistical functions, the test attribute and the reference attribute may be suitable for direct comparison, such that the performance of block 325 can include a subtraction of the test attribute from the reference attribute, or vice versa. For example, when the reference attribute is the frequency of the second peak 408 mentioned earlier, the processor 200 determines a frequency of a second peak in the test recording, corresponding to the resonant frequency of the ports 120 at the time of the test recording. The reference peak 408 and the test peak can then be compared. Turning to
Returning to
For example, in the example of
Following a negative determination at block 330, the processor is configured to generate an alert at block 335. The alert can be generated by transmitting a message via the communications interface 208, e.g. to a server or other computing device that stores a maintenance log, executes fleet management functionality or the like. The alert can also be generated by rendering a notification on the display 108, e.g. informing an operator of the device 100 that the audio assembly 112 requires maintenance (e.g. cleaning of the ports 120 or replacement).
Variations to the above functions are contemplated. In some examples, in addition to comparing the test recording to the reference recording (or attributes derived therefrom) the processor 200 can also compare a current test recording to previous test recordings. For example, test recordings from successive performances of block 315 can be stored in the memory, and in addition to evaluating the difference between the reference recording and the current test recording, the processor 200 can be configured to evaluate the difference between the current test recording and one or more preceding test recordings.
More specifically, the determination at block 330 can be expanded to assess both the difference between the current test recording and the reference recording against a first threshold, and to assess the difference between the current test recording and a preceding test recording against a second threshold. When either of the above thresholds is exceed, the determination at block 330 may be negative. That is, even if the test recording does not differ significantly from the reference recording, if the test recording shows sufficient change over time, the alert at block 335 may nevertheless be generated.
In further embodiments, the reference attributes employed at block 330 can include not only attributes associated with the reference recording as described above, but also attributes associated with a stored recording that corresponds to a defective device. The defective recording can be loaded into the memory 220 at the time of manufacture of the device 100, retrieved from a server, or the like. The defective recording is captured from a device known to have a clogged or otherwise defective audio assembly. Thus, at block 330 the device 100 can compare the test recording to at least one “good” recording (e.g. the reference recording discussed earlier), and at least one “bad” recording (e.g. the defective recording mentioned above).
For example, the device 100 can assess the Log-Euclidean distances between (i) a sliding RMS window of the test recording and the good recording, and (ii) the sliding RMS window of the test recording and the bad recording. Such distances may be assessed against a proximity threshold using the logarithm of geometric mean computation. For example, if the test recording is within the proximity threshold of the reference recording the determination at block 330 may be affirmative, whereas if the test recording is within the proximity threshold of the defective recording the determination at block 330 may be negative. In some examples, when the test recording is further from the reference recording than from the defective recording, the determination at block 330 may be negative.
The results of the above assessment may also be presented on the display 108 in some implementation. For example, turning to
In the foregoing specification, specific embodiments have been described. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the invention as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present teachings.
The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential features or elements of any or all the claims. The invention is defined solely by the appended claims including any amendments made during the pendency of this application and all equivalents of those claims as issued.
Moreover in this document, relational terms such as first and second, top and bottom, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” “has”, “having,” “includes”, “including,” “contains”, “containing” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “comprises . . . a”, “has . . . a”, “includes . . . a”, “contains . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises, has, includes, contains the element. The terms “a” and “an” are defined as one or more unless explicitly stated otherwise herein. The terms “substantially”, “essentially”, “approximately”, “about” or any other version thereof, are defined as being close to as understood by one of ordinary skill in the art, and in one non-limiting embodiment the term is defined to be within 10%, in another embodiment within 5%, in another embodiment within 1% and in another embodiment within 0.5%. The term “coupled” as used herein is defined as connected, although not necessarily directly and not necessarily mechanically. A device or structure that is “configured” in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
It will be appreciated that some embodiments may be comprised of one or more specialized processors (or “processing devices”) such as microprocessors, digital signal processors, customized processors and field programmable gate arrays (FPGAs) and unique stored program instructions (including both software and firmware) that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of the method and/or apparatus described herein. Alternatively, some or all functions could be implemented by a state machine that has no stored program instructions, or in one or more application specific integrated circuits (ASICs), in which each function or some combinations of certain of the functions are implemented as custom logic. Of course, a combination of the two approaches could be used.
Moreover, an embodiment can be implemented as a computer-readable storage medium having computer readable code stored thereon for programming a computer (e.g., comprising a processor) to perform a method as described and claimed herein. Examples of such computer-readable storage mediums include, but are not limited to, a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, a ROM (Read Only Memory), a PROM (Programmable Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory) and a Flash memory. Further, it is expected that one of ordinary skill, notwithstanding possibly significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided by the concepts and principles disclosed herein will be readily capable of generating such software instructions and programs and ICs with minimal experimentation.
The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.
Number | Name | Date | Kind |
---|---|---|---|
7593781 | Rhea | Sep 2009 | B1 |
20030065276 | Akita | Apr 2003 | A1 |
20070025557 | Nackvi | Feb 2007 | A1 |
20070100572 | Zhang | May 2007 | A1 |
20090280800 | Liu | Nov 2009 | A1 |
20090304195 | Fillol | Dec 2009 | A1 |
20100074451 | Usher | Mar 2010 | A1 |
20100191143 | Ganter | Jul 2010 | A1 |
20100272270 | Chaikin | Oct 2010 | A1 |
20130083935 | Yang | Apr 2013 | A1 |
20130129097 | Park | May 2013 | A1 |
20140369512 | Slupeiks | Dec 2014 | A1 |
20150304786 | Partio | Oct 2015 | A1 |
20170084294 | Hartung | Mar 2017 | A1 |
20170231535 | Lasry | Aug 2017 | A1 |
20180018984 | Dickins | Jan 2018 | A1 |
20180255412 | Ouyang | Sep 2018 | A1 |
20190075403 | Wurzbacher | Mar 2019 | A1 |
20200322724 | Lim | Oct 2020 | A1 |