Patent Application
20150281862
A portion of the material in this patent document is subject to copyright protection under the copyright laws of the United States and of other countries. The owner of the copyright rights has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the United States Patent and Trademark Office publicly available file or records, but otherwise reserves all copyright rights whatsoever. The copyright owner does not hereby waive any of its rights to have this patent document maintained in secrecy, including without limitation its rights pursuant to 37 C.F.R. §1.14.
Advances in modern digital hearing aid technology focus almost entirely on improving the intelligibility of speech in noisy environments. The effects of hearing aid processing on musical signals and on the perception of music receive very little attention, despite reports that hardness of hearing is the primary impediment to enjoyment of music in older listeners, and that hearing aid processing is frequently so damaging to musical signals that hearing aid wearers often prefer to remove their hearing aids when listening to music.
Though listeners and musicians who suffer hearing impairment are no less interested in music than normal hearing listeners, there is evidence that the perception of fundamental aspects of (Western) musical signals, such as the relative consonance and dissonance of different musical intervals, is significantly altered by hearing impairment (J. B. Tufts, M. R. Molis, M. R. Leek, Perception of dissonance by people with normal hearing and sensorineural hearing loss, Acoustical Society of America Journal 118 (2005) 955-967). Measures such as the Articulation Index and the Speech Intelligibility Index (American National Standards Institute, New York, N.Y., ANSI S3.5-1997, Methods for the calculation of the speech intelligibility index (1997)) can be used to predict intelligibility from the audibility of speech cues across all frequencies, and a variety of objective tests of speech comprehension are used to measure hearing aid efficacy, but there is no standard metric for measuring a patient's perception of music. Moreover, hearing impaired listeners are less consistent in their judgments about what they hear than are normal hearing listeners (J. L. Punch, Quality judgments of hearing aid-processed speech and music by normal and otopathologic listeners, Journal of the American Audiology Society 3 (1978), no. 4 179-188), and individual differences in performance among listeners having similar audiometric thresholds make it difficult to predict the perceptual effects of hearing aid processing (C. C. Crandell, Individual Differences in Speech Recognition Ability: Implications for Hearing Aid Selection, Ear and Hearing 12 (1991), no. 6 Supplement 100S-108S). These factors, combined with the differences in the acoustical environments in which different styles of music are most often presented, underline the importance of individual preferences in any study of the effects of hearing aid processing on the perception of music. There have been studies on the effect of reduced bandwidth on the perceived quality of music (J. R. Franks, Judgments of Hearing Aid Processed Music, Ear and Hearing 3 (1982), no. 1 18-23), but no systematic evaluation of the effects of dynamic range compression, the most ubiquitous form of gain compensation in digital hearing aids.
There is a need in the art for an improved system for programming hearing assistance devices which incorporates the listener's preferences and provides the listener a convenient interface to subjectively tailor sound processing of a hearing assistance device. There is also a need in the art for a system for hearing assistance devices that allows for better appraisal of the processing of music. Such a system will provide benefit for the fitting of other sound processing technology in hearing assistant devices for which the fitting to hearing loss diagnostics is unknown but for which fitting can be made based on assessment of subjective preference.
This application provides a subjective, listener-driven system for programming parameters in a hearing assistance device, such as a hearing aid. In one embodiment, the listener controls a simplified system interface to organize according to perceived sound quality a number of presets based on parameter settings spanning parameter ranges of interest. By such organization, the system can generate a mapping of spatial coordinates of an N-dimensional space to the plurality of parameters using interpolation of the presets organized by the user. In various embodiments, a graphical representation of the N-dimensional space is used.
In one embodiment, a two-dimensional plane is provided to the listener in a graphical user interface to “click and drag” a preset in order to organize the presets by perceived sound quality; for example, presets that are perceived to be similar in quality could be organized to be spatially close together while those that are perceived to be dissimilar are organized to be spatially far apart. The resulting organization of the presets is used by an interpolation mechanism to associate the two-dimensional space with a subspace of parameters associated with the presets. The listener can then move a pointer, such as PC mouse, around the space and alter the parameters in a continuous manner. If the space and associated parameters are connected to a hearing assistance device that has parameters corresponding to the ones defined by the subspace, then the parameters in the hearing device are also adjusted as the listener moves a pointer around the space; if the hearing device is active, then the listener hears the effect of the parameter change caused by the moving pointer. In this way, the listener can move the pointer around the space in an orderly and intuitive way until they determine one or more points or regions in the space where they prefer the sound processing that they hear.
In one embodiment, a radial basis function network is used as a regression method to interpolate a subspace of parameters. The listener navigates this subspace in real time using an N-dimensional graphical interface and is able to quickly converge on his or her personally preferred sound which translates to a personally preferred set of parameters.
One of the advantages of this listener-driven approach is to provide the listener a relatively simple control for several parameters.
This Summary is an overview of some of the teachings of the present application and is not intended to be an exclusive or exhaustive treatment of the present subject matter. Further details about the present subject matter are found in the detailed description and the appended claims. The scope of the present invention is defined by the appended claims and their legal equivalents.
The following detailed description of the present invention refers to subject matter in the accompanying drawings which show, by way of illustration, specific aspects and embodiments in which the present subject matter may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present subject matter. References to “an”, “one”, or “various” embodiments in this disclosure are not necessarily to the same embodiment, and such references contemplate more than one embodiment. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope is defined only by the appended claims, along with the full scope of legal equivalents to which such claims are entitled.
This application provides a subjective, listener-driven system for programming parameters in a hearing assistance device, such as a hearing aid. In one embodiment, the listener controls a simplified system interface to organize according to perceived sound quality a number of presets based on parameter settings spanning parameter ranges of interest. By such organization, the system can generate a mapping of spatial coordinates of an N-dimensional space to the plurality of parameters using interpolation of the presets organized by the user. In various embodiments, a graphical representation of the N-dimensional space is used.
In one embodiment, a two-dimensional plane is provided to the listener in a graphical user interface to “click and drag” a preset in order to organize the presets by perceived sound quality; for example, presets that are perceived to be similar in quality could be organized to be spatially close together while those that are perceived to be dissimilar are organized to be spatially far apart. The resulting organization of the presets is used by an interpolation mechanism to associate the two-dimensional space with a subspace of parameters associated with the presets. The listener can then move a pointer, such as PC mouse, around the space and alter the parameters in a continuous manner. If the space and associated parameters are connected to a hearing assistance device that has parameters corresponding to the ones defined by the subspace, then the parameters in the hearing device are also adjusted as the listener moves a pointer around the space; if the hearing device is active, then the listener hears the effect of the parameter change caused by the moving pointer. In this way, the listener can move the pointer around the space in an orderly and intuitive way until they determine one or more points or regions in the space where they prefer the sound processing that they hear.
In one embodiment, a radial basis function network is used as a regression method to interpolate a subspace of parameters. The listener navigates this subspace in real time using an N-dimensional graphical interface and is able to quickly converge on his or her personally preferred sound which translates to a personally preferred set of parameters.
One of the advantages of this listener-driven approach is to provide the listener a relatively simple control for several parameters.
Characterizing perceptual dissimilarity as distance in a geometric representation has provided auditory researchers with a rich set of robust methods for studying the structure of perceptional attributes (R. N. Shepard, Multidimensional Scaling, Tree-Filling, and Clustering, Science 210 (1980), no. 4468 390-398). Examples include spaces for vowels and consonants (R. N. Shepard, Psychological Representation of Speech Sounds, E. David, P. B. Denes, eds., Human Communication a United View, McGraw-Hill, New York, N.Y. (1972) 67-113), timbres of musical instruments, rhythmic patterns, and musical chords (A. Momeni, D. Wessel, Characterizing and controlling musical material intuitively with geometric models, Proceedings of the 2003 Conference on New Interfaces for Musical Expression, Montreal, Canada (2003) 54-62). The most common method for generating a spatial representation is the multidimensional scaling (MDS) of pairwise dissimilarity judgments (I. Borg, P. J. F. Groenen. Modern Multidimensional Scaling: Theory and Applications. Springer, New York, N.Y. (2005)). In this method, subjects typically rate the dissimilarity for all pairs in a set of stimuli. The stimuli are treated as points in a low dimensional space, often two-dimensional, and the MDS method finds the spatial layout that maximizes the correlation between distances in the representation and subjective dissimilarity ratings among the stimuli. As an alternative to the MDS method we (A. Momeni, D. Wessel, Characterizing and controlling musical material intuitively with geometric models, Proceedings of the 2003 Conference on New Interfaces for Musical Expression, Montreal, Canada (2003) 54-62) and Wessel (1979) “Timbre space as a musical control structure,” Computer Music Journal, 3(2):45-52) and others (R. L. Goldstone, An efficient method for obtaining similarity data, Behavior Research Methods, Instruments, & computers 26 (1994), no. 4 381-386) have found that directly arranging the stimuli in a subjectively meaningful spatial layout provides representations comparable in quality to MDS.
The present subject matter provides a system having a user interface that allows a listener to organize a number of presets that are designed to span a parameter range of interest. The listener is able to subjectively organize the preset settings in an N-dimensional space. The resulting organization provides the system a relation of the preset parameters that is processed to generate a mapping of spatial coordinates of an N-dimensional space to the plurality of parameters using interpolation of the presets. The listener can then “navigate” through the N-dimensional mapping using the interface while listening to sound processed according to the interpolated parameters and find one or more preferred settings. This system allows a user to control a relatively large number of parameters with a single control and to find one or more preferred settings using the interface. Parameters are interpolated in real time, as the listener navigates the space, so that the listener can hear the effects of the continuous variation in the parameters.
The following description will demonstrate a process for an application using hearing aids, however, it is understood that the present teachings may be used for a variety of other applications, including, but not limited to, listening to music with headphones.
Once the parameters to be controlled are selected, the system can optionally provide a choice of a special nonlinear function to be applied to one or more parameters. For example, the nonlinear function can be a logarithmic function. One demonstrative example is that sometimes signal volume is better processed as the log of the signal volume. Other types of nonlinear functions may be optionally applied without departing from the scope of the present subject matter.
Once the parameters are selected a number of presets can be selected 62. The presets can be chosen to span a parameterization range of interest. The preset parameter values could be selected by an audiologist, an engineer, or could be done automatically using software. Such presets could be based on a listener's particular audiogram. For example, a person with high frequency hearing loss could have presets with a variety of audio levels in high frequency bands to assist in a diverse parameterization for that particular listener. In various embodiments, the presets could be selected based on population data. For example, predetermined presets could be used for listeners with a particular type of audiogram feature. Such settings may be developed based on knowledge of the signal processing algorithm. Such settings may also be determined empirically.
In various embodiments, the presets are selected to provide a diverse listening experience for the particular listener. Interpolations of similar parameter settings generally yield narrow interpolated parameter ranges. Thus, the presets need not be ones determined to sound “good,” but rather should be diverse.
The presets are then arranged on the display 63 for the listener. Such arrangements may be random, as demonstrated by
Sound is played to the listener using the signal processor 64. The parameters fed to the signal processing algorithm are those of the preset selected. Sound played to the listener can be via headphones. In hearing aid applications, the sound played to the listener can be made directly by hearing aids in one or both ears of the listener. In various embodiments, the sound is generated by the computer and/or programmer. In various embodiments the sound is natural ambient sound picked up by one or more microphones of the one or more hearing aids. Regardless, the signal processor 52 receives parameters Z from the Controller 51 based on the selected preset and plays processed sounds according to the selected preset parameters. It is understood that in various embodiments, the computer 2 or 3 or handheld device 12 or 13 could be implementing the controller 51. In various embodiments, the handheld device 12, 13 includes the controller 51, the signal processor 52, and the input device 9. In various embodiments, a hearing aid 8 is implementing the signal processor 52. In various embodiments, the hearing aid 8 implements the signal processor 52 and the controller 51. Other embodiments are possible without departing from the scope of the present subject matter.
The listener organizes the presets in the subjective space depending on sound 65. In one embodiment, the listener is listening to sound played using different presets and uses a graphical user interface on screen 4 to drag the preset icons to different places in the subjective space. In various embodiments, the listener is encouraged to organize things that sound similar closely in the subjective space and things that sound different relatively far apart in the subjective space. In various embodiments the listener is encouraged to use as much of the subjective space as possible.
In various embodiments, the organization of presets in the subjective space is performed by an audiologist, an engineer, or other expert. In various embodiments, the organization of presets is performed according to population data, or according to the listener's audiogram or other attributes. In various embodiments, the listener participates in the programming and navigation modes of operation. In various embodiments, the listener participates only in the navigation mode of operation. Other variations of process are possible without departing from the scope of the present subject matter, and those provided herein are not intended to be exclusive or limiting.
Once the organization is complete, the computer constructs an interpolation scheme that maps every coordinate of the subjective space to an interpolated set of parameters according to the organization of the presets 66. In various embodiments, the organization is interpolated using distance-based weighting (e.g., Euclidean distance and weighted average). In various embodiments, the organization of presets is interpolated using a two-dimensional Gaussian kernel. In various embodiments, a radial basis function network is created to interpolate the organization of the presets. Other interpolation schemes are possible without departing from the scope of the present subject matter.
In various embodiments, the presets can be hidden during the navigation phase so as to not distract the listener from navigating the subjective space.
In some embodiments, a radial basis function network, such as the one demonstrated by
In varying embodiments, the process is repeated for different sound environments. In various embodiments, artificial sound environments are generated to provide speech babble and other commonly encountered sounds for the listener. In various embodiments, measurements are performed in quiet for preferred quiet settings. In various embodiments a plurality of settings are stored in memory. Such settings may be employed by the listener at his or her discretion. In various embodiments, the subjective organization of the presets is analyzed for a population of subject listeners to provide a diagnostic tool for diagnosing hearing-related issues for listeners. It is understood that in various embodiments, the navigation mode may or may not be employed.
In applications involving hearing assistance devices, the interface provides a straightforward control of potentially a very large number of signal processing parameters. In cases where the hearing assistance devices are hearing aids, the system provides information that can be used in “fitting” the hearing aid to its wearer. Such applications may use a variety of presets based on information obtained from an audiogram or other diagnostic tool. The presets may be selected to have different parameterizations based on the wearer's particular hearing loss. Thus, the parameter range of interest for the presets may be obtained from an individual's specific hearing or from a group demographic. Such applications may also involve the use of different acoustic environments to perform fitting based on environment. Hearing assistance devices can include memory for storing preferred parameter settings that may be programmed and/or selected for different environments. Yet another application is the use of the present system by a wearer of one or more hearing aids who wants to find an “optimal” or preferred setting for her/his hearing aid for listening to music. Other benefits and uses not expressly mentioned herein are possible from the present teachings.
In various embodiments, interpolation of the parameter presets may be performed using a radial basis function network 81 composed of a radial basis hidden layer 83 and a linear output layer 84 as shown in
The specifics of the system are shown in
The linear layer consists of a mapping from the q-dimensional weight vector to the P-dimensional parameter space. This linear transformation is carried out using a matrix T, that left multiplies the weight vector w, and a constant vector b which is summed with the resulting matrix product Tw. If Z is the P-dimensional output vector of interpolated parameters, we have
Z=Tw+b. (Eq. 1)
The training of the network is simple and does not require complex iterative algorithms. This allows the network to be retrained in real-time, so that the user can instantly experience the effects of moving presets within the space. The network is trained so that each preset location elicits an output equal to the exact parameter set corresponding to that preset.
The values that must be determined by training are the preset location matrix L, the linear transformation matrix T, and the vector b. The matrix L is trivially constructed by placing each two-dimensional preset location in a separate column of the matrix. The matrix T and vector b are chosen so that if the input location lies directly on a preset, then the output will be the parameters corresponding to that preset. To solve for these, we can set up a linear system of equations. We can place T and b together in a matrix
T′=[T|b]. (Eq. 2)
Then we place the weight vectors corresponding to each preset location into a matrix W and append a row vector of ones, 11×q, so that
Let the matrix V be the target matrix composed of columns of the parameters corresponding to each preset. Now our linear system of equations can be represented by the single matrix equation
T′W′=V (Eq. 4)
Because there are more degrees of freedom in the system than constraints, the system is underdetermined and has infinitely many solutions. We choose the solution, T′ with the lowest norm by right multiplying by the pseudo-inverse of W′. The solution with lowest norm was chosen to prevent the system from displaying erratic behavior and to keep any one weight from dominating the output. After we have solved for T and b, the training is complete. Compared to other neural network training procedures, such as back propagation, this method is extremely fast and still produces the desired results.
We have implemented a prototype listener-driven interactive system for adjusting the high dimensional parameter space of hearing aid signal processing algorithms. The system has two components. The first allows listeners to organize a two dimensional space of parameter settings so that the relative distances in the layout correspond to the subjective dissimilarities among the settings. The second performs a nonlinear regression between the coordinates in the subjective space and the underling parameter settings thus reducing the dimensionality of the parameter adjustment problem. This regression may be performed by a radial basis function neural network that trains rapidly with a few matrix operations. The neural network provides for smooth real-time interpolation among the parameter settings. Those knowledgeable in the art will understand that there are many other ways of interpolating between the presets other than using radial basis functions or neural networks.
The two system components may be used individually, or in combination. The system is intuitive for the user. It provides real-time interactivity and affords non-tedious exploration of high dimensional parameter spaces such as those associated with multiband compressors and other hearing aid signal processing algorithms. The system captures rich data structures from its users that can be used for understanding individual differences in hearing impairment as well as the appropriateness of parameter settings to differing musical styles.
It is understood that in various embodiments, the apparatus and processes set forth herein may be embodied in digital hardware, analog hardware, and/or combinations thereof.
The present subject matter includes hearing assistance devices, including, but not limited to, cochlear implant type hearing devices, hearing aids, such as behind-the-ear (BTE), in-the-ear (ITE), in-the-canal (ITC), or completely-in-the-canal (CIC) type hearing aids. It is understood that behind-the-ear type hearing aids may include devices that reside substantially behind the ear or over the ear. Such devices may include hearing aids with receivers associated with the electronics portion of the behind-the-ear device, or hearing aids of the type having receivers in-the-canal. It is understood that other hearing assistance devices not expressly stated herein may fall within the scope of the present subject matter.
This application is intended to cover adaptations and variations of the present subject matter. It is to be understood that the above description is intended to be illustrative, and not restrictive. The scope of the present subject matter should be determined with reference to the appended claim, along with the full scope of legal equivalents to which the claims are entitled.
This application is a continuation of and claims the benefit of priority under 35 U.S.C. §120 of U.S. patent application Ser. No. 13/368,760, filed on 8 Feb. 2012; which application is a continuation of and claims the benefit of priority under 35 U.S.C. §120 of U.S. patent application Ser. No. 12/190,582, filed on 12 Aug. 2008, which application claims the benefit of U.S. Provisional Application Ser. No. 60/968,700, filed Aug. 29, 2007, the benefit of priority of each of which is claimed hereby, and each of which are incorporated by reference herein in its entirety. All cited references in U.S. Provisional Application Ser. No. 60/968,700 and in this nonprovisional patent application are incorporated herein by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 60968700 | Aug 2007 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13368760 | Feb 2012 | US |
| Child | 14611701 | US | |
| Parent | 12190582 | Aug 2008 | US |
| Child | 13368760 | US |
