Patent Application
20100100362
1. Technical Field
The present disclosure relates to ear canal models and, more specifically, to point-based shape matching and distance applied to ear canal models.
2. Discussion of Related Art
Digital shape modeling is the process by which the shape of a device to be manufactured is designed with the assistance of computer aided design (CAD) software. After the desired shape is modeled in CAD software, the model file may be used in fabrication to produce a device with the desired shape. The use of digital shape modeling is widespread, however, when used for the modeling of organic shapes, such as in the field of medical prostheses, the complexity, irregularity and uncertainty of shapes can pose particular problems for digital shape modeling. One important example is the field of custom hearing aid manufacture.
A hearing aid is an electroacoustic device for the amplification of sound. Modern hearing aids may be worn either partially or fully within the ear canal. Examples of such hearing aids include in the canal (ITC), mini canal (MIC), and completely in the canal (CIC) aids. For these types of hearing aids, the entire hearing aid is designed to fit securely in the wearer's ear. To achieve this secure fit, the hearing aid is incorporated into an outer shell that is custom fitted to the shape of the wearer's ear canal.
Custom fitting of the hearing aid shell is accomplished by first taking an impression of the wearer's ear canal. This may be achieved by applying hardening foam or other molding substance into the ear of the wearer. Once removed and allowed to harden, the three-dimensional shape of the mold may be digitally scanned and imported into CAD software. Technicians may then perform various modifications and transformations to the three-dimensional geometry to convert the geometry of the wearer's ear into a model for fabricating the shell of a hearing aid. These geometry processing steps may be labor intensive, tedious and prone to error. These factors may then add to the cost and time required to produce a suitable hearing aid.
A method for determining a degree of similarity between ear canal models includes receiving a first mesh model representing an inner surface of a first ear. A set of points is sampled within the first mesh model. Each of the sampled set of points is matched to a corresponding point of a second mesh model representing an inner surface of a second ear. A shape distance between the first mesh model and the second mesh model is calculated based on the matched sets of points. A determination of the degree of similarity between the inner surface of the first ear and the inner surface of the second ear is provided based on the calculated shape distance.
Sampling the set of points within the first mesh model may include identifying a randomly selected set of points within the first mesh model and/or identifying a set of points within the first mesh model at predetermined regular increments.
Segmentation may be performed within the first mesh model to identify a plurality of regions. Thereafter one or more regions of interest may be selected and data pertaining to regions that are not of interest may be removed from the first mesh model such that the first mesh model is reduced to one or more regions of interest. The regions of interest may be an ear canal.
The shape distance between the first mesh model and the second mesh model may be calculated as the cumulative difference between each set of corresponding points between the first mesh model and the second mesh model.
After the matching of each of the sampled set of points to a corresponding point of the second mesh model, an alignment step may be performed to adjust characteristics of the points of the first mesh model to more closely conform to the second mesh model. Adjusting the characteristics of the first mesh model may include one or more of adjusting rotation, performing translation, or adjusting scale. After the alignment step is performed, the matching step may be repeated to establish an improved point-to-point correspondence. The matching step and alignment step may be repeated to iteratively improve the matching.
The step of matching each of the sampled set of points to a corresponding point of the second mesh model may include, for each point of the first mesh model, calculating a histogram representing the entire first mesh model from the point of view of each point in a radial dimension, an inclination angle and an azimuth angle and comparing that three-dimensional histogram to similar three-dimensional histograms of points of the second mesh model until a match is found.
The provided determination as to the degree of similarity between the inner surface of the first ear and the inner surface of the second ear may be used to transform the first mesh model into a design for a hearing aid to be placed into the first ear.
The first mesh model may be transformed into a design for a hearing aid using a geometry processing routine, the implementation of which may be dependent upon the calculated shape distance between the first mesh model and the second mesh model.
A shape distance between the first mesh model and a third mesh model representing an inner surface of a third ear may be calculated and may then be compared to the calculated shape distance between the first mesh model and the second mesh model to determine whether the inner surface of the first ear is more similar to the inner surface of the second ear or the inner surface of the third ear.
A first geometry processing routine may be performed on the first mesh model if the inner surface of the first ear is more similar to the inner surface of the second ear and a second geometry processing routine is performed on the first mesh model if the inner surface of the first ear is more similar to the inner surface of the third ear.
A system for designing a hearing aid device includes receiving a first mesh model representing an inner surface of a first ear, sampling a set of points within the first mesh model, matching each of the sampled set of points to a corresponding point of a second mesh model representing an inner surface of a second ear, calculating a shape distance between the first mesh model and the second mesh model based on the matched sets of points, and performing a geometry processing routine that is dependent upon the calculated shape distance between the first mesh model and the second mesh model to transform the first mesh model into a design for a hearing aid to be placed into the first ear.
Sampling the set of points within the first mesh model may include identifying a randomly selected set of points within the first mesh model or identifying a set of points within the first mesh model at predetermined regular increments.
Segmentation may be performed within the first mesh model to identify a plurality of regions. Thereafter one or more regions of interest may be selected and data pertaining to regions that are not of interest may be removed from the first mesh model such that the first mesh model is reduced to one or more regions of interest. The regions of interest may be an ear canal.
The shape distance between the first mesh model and the second mesh model may be calculated as the cumulative difference between each set of corresponding points between the first mesh model and the second mesh model.
After the matching of each of the sampled set of points to a corresponding point of the second mesh model, an alignment step may be performed to adjust characteristics of the points of the first mesh model to more closely conform to the second mesh model.
The step of matching each of the sampled set of points to a corresponding point of the second mesh model may include, for each point of the first mesh model, calculating a histogram representing the entire first mesh model from the point of view of each point in a radial dimension, an inclination angle and an azimuth angle and comparing that three-dimensional histogram to similar three-dimensional histograms of points of the second mesh model until a match is found.
A computer system includes a processor and a program storage device readable by the computer system, embodying a program of instructions executable by the processor to perform method steps for determining a degree of similarity between ear canal models. The method includes receiving a first mesh model representing an inner surface of a first ear, sampling a set of points within the first mesh model, matching the sampled set of points to corresponding points of a second mesh model representing an inner surface of a second ear, calculating a shape distance between the first mesh model and the second mesh model based on the matching, and outputting the calculated shape distance.
A more complete appreciation of the present disclosure and many of the attendant aspects thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:
In describing exemplary embodiments of the present disclosure illustrated in the drawings, specific terminology is employed for sake of clarity. However, the present disclosure is not intended to be limited to the specific terminology so selected, and it is to be understood that each specific element includes all technical equivalents which operate in a similar manner.
Exemplary embodiments of the present invention seek to provide for an automated or partially automated approach to hearing aid modeling and design whereby geometry processing steps traditionally performed by a CAD technician may be partially or fully automated. As anatomical shapes such as that of the ear canal may be substantially different from patient to patient, it may be difficult to apply the same automatic processing steps to every ear canal. Accordingly, exemplary embodiments of the present invention first seek to differentiate between various classifications of ear canal shapes so that automated geometry processing may be specific to the particular class of shape that any given ear canal most conforms to. This classification may be performed using the concept of shape distances whereby the differentiation between a given ear canal and a model ear canal may be quantified as a distance. By calculating the shape distance between a given ear canal and a model ear canal, it may be determined whether the automated geometry processing techniques established for the model ear canal may be successfully applied to the given ear canal and/or what shape class-specific geometry processing rules to apply for the given context. Moreover, where there are multiple model ear canals each with corresponding sets of automated geometry processing techniques, shape distances may be calculated between the given ear canal and each of the multiple model ear canals so that a best match may be found and the corresponding set of automated geometry processing techniques may be applied.
Accordingly, exemplary embodiments of the present invention may make use of a meaningful notion of shape distance. This task may be closely coupled to the problem of establishing pointwise correspondences between two subjects, in this case, the given ear canal and the model ear canal. Pointwise correspondences may be particularly difficult for three-dimensional surfaces, especially for organic shapes, where anatomical variations make automatic solutions for landmark detection very challenging. Exemplary embodiments of the present invention may utilize a three-dimensional shape context to define a non-Euclidian shape metric space between canal structures of the ear anatomy models and the given ear canals, where points are chosen uniformly at random from each exemplar. Here, point matching and similarity transformation may be reiterated for mutual improvement. By using this approach, shape matching may be performed to an extent that is commensurate with the human notion of shape categorization and at the same time robust towards reasonable amounts of noise and matching outliers. Thus the accuracy of human shape categorization may be achieved without the need for human input. By the use of this approach, various classifications of digital hearing aid design process may be established and for the manufacturing of a given hearing aid, a classification may easily be determined to provide for a selection of an optimal processing path.
Exemplary embodiments of the present invention compare the shape of canal parts of a human outer ear to one or more three-dimensional models. Three-dimensional models of the outer ear may be represented as bounded triangular meshes. Shape classes may be identified for several anatomical parts of outer ear three-dimensional models including the canal part. To this end, a meaningful notion of shape distance may be provided to differentiate between subjects in a population of shapes. Shape classes of anatomical parts may be used to derive shape class specific rules for the manipulation of outer ear surface geometries in hearing aid design.
The correspondence computation may be based on invariant attributes of point locations to ensure that the resulting shape distance is invariant with respect to a chosen finite dimensional group (e.g., the similarity group) and to additional non-ideal conditions, such as noise and small local deformations. Exemplary embodiments of the present invention utilize shape context in order to establish point correspondence between two ear canal shapes. These correspondences may be determined by solving a linear assignment problem. It may be assumed herein that corresponding points on similar three-dimensional canal shapes have similar shape context distributions, despite the presence of small local deformations and reasonable amounts of noise.
First, the two three-dimensional mesh models (X,X′) are received (Step S11). As discussed above, the first three-dimensional mesh model X may be the given ear shape being categorized and the second three-dimensional mesh model X′ may be the ear shape of a known shape class.
The three-dimensional mesh models may be geometric representations of the inner surface of an ear. These geometries may be restricted to surface imagery although the presence of image data reflecting internal structures need not prevent exemplary embodiments of the present invention from utilizing such image data.
Next, segmentation may be performed on each three-dimensional mesh model to determine each of a set of anatomical regions, which may include, for example, a canal structure (l=1) and an external ear structure (l=2) (Step S12). An example of segmentation may be seen in
Where one of the three-dimensional mesh models is in fact a known ear shape of a particular shape class, the complete set of anatomical regions may be already segmented, in which case segmentation is performed only with respect to the given ear shape being categorized. Where neither three-dimensional mesh models is known, segmentation may be performed for each model.
Segmentation may be performed within the mesh models to identify multiple regions. Thereafter one or more regions of interest may be selected from among the multiple regions. As described above, the mesh model may be divided into two regions including a canal region and an external ear region. In such a case, the canal region may be considered the region of interest. Data pertaining to regions that are not of interest may be removed from the mesh models such that the mesh models is reduced to one or more regions of interest. Here, the external ear region may be removed and the mesh models may be reduced to include only the ear canal.
Decomposition of the triangular meshes into distinct anatomical regions, for example, the external ear and the canal, may be automatically performed as part of the segmentation step. By removing the regions that are not of interest, point matching and subsequent processing steps may be simplified.
Point sampling may then be performed on each mesh model to characterize each mesh model as a collection of points that is a sample of the complete set of vertices of the corresponding mesh model. Sampling may be performed, for example, either randomly or at fixed intervals. The result is a set of points P representing the first mesh model X and a set of points P′ representing the second mesh model. It may also be possible to skip the step of point sampling and instead consider the full set of vertices as the set of points, however, to make efficient use of limited computational resources, sampling may be used.
Point matching may then be performed to determine point-for-point correspondences between the points of the sets of points P and P′ (Step S14). The result of point matching is to determine which point from P corresponds to which point from P′for all points, or at least for all points that have analogs. Correspondences may be established between two point sets P and P′, for example, by employing a shape context, which in three-dimensions, for a given point pεP and p′εP′ is a descriptor that measures the loci of all other points {circumflex over (p)}εP and {circumflex over (p)}′εP′ relative top and p' based on a three-dimensional statistic of spherical coordinates (r,θ,φ), where r denotes the radial dimension, and θ and φ are angles denoting the elevation and azimuth dimensions, respectively. The resulting correspondence is a quantification of how the entire shape appears from the point of view of each particular sample point and by comparing this quantification from a point p of the first mesh model with a point p′ on the second mesh model, the degree of correspondence may be ascertained. Then, a global cost function may be produced by the sum of pairwise correspondences. Optimal correspondence may then be solved for, for example, using bipartite matching. Optimal correspondences, herein, represent the point matching.
After establishing correspondences between the two point sets P and P′ (point matching), alignment may be performed. Alignment represents the steps necessary to align the set of points P and with the set of points P′, based on the previously-determined point matching. Alignment may include, for example, an adjustment of the rotation, translation, and scale in a manner that best aligns P with P′. Alignment may, in the simplest example, be linear, however, non-ridged transformations may also be used. For the purpose of ease of explanation, exemplary embodiments of the present invention are described herein utilizing rigid alignment.
The steps of point matching (S14) and alignment (S15) may be performed iteratively the step of performing the alignment may allow for a more accurate subsequent point matching, and that in turn may allow for a more accurate subsequent alignment. Accordingly, these steps may be repeated either for a predetermined number of iterations, for example, n where n is a positive integer, for example, 2, or repetition may continue until the point at which subsequent iteration ceases to produce additional refinement.
In either event, after iteration has been completed, the shape distance between P′ and P′ may be calculated using the final point-to-point correspondences (Step S16). The calculated shape distance may then be used, for example, to determine how to process the geometry by choosing a geometry processing routine based on a shape category or by modifying geometry processing steps based on the calculated shape distance. According to some exemplary embodiments of the present invention, a patient's ear shape model X may be compared either to a single model X′ or to multiple models X″, X′″, etc. By repeating the above-described process for multiple known shape models, a set of shape distances may be calculated for the model of the patient's ear so that, for example, a closest match may be found and a corresponding geometry processing routine followed.
However, following the geometry processing routine for a closest match does not preclude the possibility that the geometry processing steps could be dependent upon one or more of the shape distances.
Exemplary embodiments of the present invention may accordingly measure the shape distance between canal parts of outer ear three-dimensional models. As the notion of shape distance may be symmetric, the correspondence may be expressed as a bijective mapping π:P→P′ between two sets of points P⊂X, P′⊂X′ sampled from the canal part of a triangular mesh. Here, P denotes the set of points sampled from the canal part of X, and P′ contains the points sampled from the canal part of X′.
A fixed number of points N is sampled from the set of vertices V of a triangular mesh. Hence, the point sets P are elements of the d-dimensional space Rd, with d=3N. The shape of a subsurface may be reasonably approximated with a roughly uniform spacing between points. It is not necessary that the sampled points correspond to anatomical landmarks or other key points such as the curvature extrema. The sampled points may thus be either randomly selected or selected at fixed intervals, for example, as described above.
The optimal correspondence mapping it may be obtained by minimizing an appropriate energy functional, for example:
The shape distance Dl(X,X′), lε{1, . . . , M} between two corresponding parts l of X and X′ may be defined by:
D
l(X,X′)=Q(π*) (2)
The global cost of matching may be defined by assuming that Q(π) is given by the sum of certain local costs. Accordingly, a function q may be defined as 3×
3→
which assigns a pair of matching points pεP, p′εP′ a scalar, that quantifies the local cost of matching. Thus, the global cost of matching may be given by:
Shape contexts may be highly discriminative and thus may be inherently insensitive to small perturbations of parts of the shape. Scale invariance may be obtained by normalizing all radial distances of a shape context histogram. This may be achieved by computing twice the centroid size for a point set P and P′, respectively. Rotation invariance may also be obtained.
A shape context may be formed by dividing each dimension into bins that are, for example, equally spaced in the angular dimensions and logarithmically spaced along the radial dimension. The optimal number of r-bins, θ-bins and φ-bins may be known for example, having been determined through experimentation. The resulting total number of bins may be expressed as K. Each histogram bin k, 0≦k<K may accumulate the number of points whose spherical coordinates relative to p fall within the discrete interval represented by the k-th bin. Since shape contexts may be represented as normalized histograms, their distance can be computed for example by using the chi-squared test statistic:
where Hp(k) and Hp′(k) denote the K-bin normalized histogram at pεP and p′εP′, respectively.
A robust treatment of matching outliers may be achieved by adding rows/columns with a constant (high) matching cost ε, which may be equivalent to adding “dummy” points to the point sets. In this case, a point would be matched to a “dummy” whenever there is no real match available at smaller cost than ε. As discussed above, the steps of point matching (S14) and alignment (S15) can be iterated, for example, to increase robustness to potential matching outliers.
Alignment may be performed, for example, by using a similarity transform for the transformation of a three-dimensional point cloud. As discussed above, there may be two point sets P and P′. Here, the point sets may be expressed as P′={xi|i=1, . . . , N} and P={yi|i=1, . . . , N}. The goal of alignment may be to align P′ to P and using a similarity transformation by minimizing the sum of square errors, i.e.:
where R, T, β denote rotation, translation, and scale, respectively. Although a closed-form solution of Equation (5) can be derived, the variational approach may also be used to approximate the solution. The update equation for minimizing the objective function w.r.t. scale may simply be given by:
where <•,•> denotes the inner product in three-dimensional Euclidian space.
Exemplary embodiments of the present invention provide a framework for computing the shape distance between canal structures of previously segmented outer ear models. To this end, the three-dimensional shape context has been employed to establish one-to-one point correspondences. The proposed shape distance may closely mimic the human notion of shape similarity. This framework may also be used to find clusters in a population of shapes, and to interpret the classes in terms of manufacturing categories. For increasing robustness of matching, an extension of the shape context descriptor, for example, the geodesic shape context, may be used. In so doing, the topology of triangular meshes may be incorporated in terms of geodesic distances between a histogram basis point and points located in the histogram bins. Additionally, a non-linear dimensionality reduction technique may be used to allow for a visual assessment of the sample distribution. Thereafter, clustering methodologies may be applied on the resulting lower dimensional embedding space.
As discussed above, exemplary embodiments of the present invention may utilize multiple known three-dimensional mesh models so that the approach described above with respect to
Shape matching may then be performed against multiple known mesh models, which may be, for example, X′, X″, X′″, etc. (Step S33). Shape matching may be performed, for example, as described above with respect to
After shape matching has been performed for each of the multiple mesh models, a closest shape model may be determined (Step S34). The closest shape model may be defined as the known shape model with the smallest shape distance to the shape model of the patient's ear.
As each of the multiple known mesh models may have a corresponding geometry processing routine for transforming the mesh model of the patient's ear into a form that is usable for constructing a custom-fitted hearing aid, the geometry processing routine corresponding to the closest known mesh model may be performed (Step S35). There may be a unique geometry processing routine for each known mesh model or there may be a single geometry processing routine that is customizable by a dependence upon the actual shape distances between the shape model for the patient's ear and one or more of the known mesh models. Exemplary embodiments of the present invention may also combine these approaches such that there is a unique geometry processing routine for each known mesh model and each unique geometry processing routine is dependent upon the calculated shape distances.
After the geometry processing routine is performed, and/or before the geometry processing routine is performed, additional automatic, manual or semiautomatic geometry processing steps may be performed to generate a design plan for a hearing aid based on the results of the performed geometry processing routine (Step S36).
The computer system referred to generally as system 1000 may include, for example, a central processing unit (CPU) 1001, random access memory (RAM) 1004, a printer interface 1010, a display unit 1011, a local area network (LAN) data transmission controller 1005, a LAN interface 1006, a network controller 1003, an internal bus 1002, and one or more input devices 1009, for example, a keyboard, mouse etc. As shown, the system 1000 may be connected to a data storage device, for example, a hard disk, 1008 via a link 1007.
Exemplary embodiments described herein are illustrative, and many variations can be introduced without departing from the spirit of the disclosure or from the scope of the appended claims. For example, elements and/or features of different exemplary embodiments may be combined with each other and/or substituted for each other within the scope of this disclosure and appended claims.
The present application is based on provisional application Ser. No. 61/104,399, filed Oct. 10, 2008, the entire contents of which are herein incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61104399 | Oct 2008 | US |
