Patent Application
20050256689
not applicable
This invention relates to an improvement for acquiring 3-dimensional geometrical and localized physical measurements from portions of a solid, rigid or semi-rigid object and for relating each physical datum to the geometrical location at which it was acquired. In particular, the solid object could be a portion of an anatomical body, such as a human limb for example.
Various systems, using various technologies, have been documented, developed, and patented for measuring the surface geometry of and spatial locations on physical objects, where the objects may be portions of animal or human anatomy. Such measurement systems also include those which yield full, 3-dimensional internal cross-sectional images, from which the surface shape information may be derived.
Examples of the latter systems are CT (computed tomography imaging), MRI (magnetic resonance imaging), and diagnostic ultrasound scanners. Examples of more limited systems, which determine only the surface shape, include laser scanners, moiré pattern cameras, stereometric cameras, and similar optical metrology equipment—all of which avoid mechanical contact with the object. Besides those are contact-based, mechanical systems like CMMs (coordinate measuring machines) with touch-sensitive probes, which can be automated to sample many points on the surface of an object. All the above are commercially available, are well known, are common in both medical and industrial markets, and are represented by numerous products from many manufacturers.
Besides the above, there are systems which track the 3-dimenstional location of the tip of a manually moved probe. Some systems mechanically track the probe tip using articulated arms with accurate, electronic joint rotation sensors. Other systems track the location of the probe electro-optically. Still others track the probe electro-magnetically. Like the aforementioned systems, each of these is commercially available from more than one manufacturer. All of these have been employed in both industrial and medical applications to measure the XYZ coordinates of individual points on the object of interest. If sufficient points are measured densely enough, a useful surface model of the object can be generated and graphically displayed by an associated electronic computer. Preferably the object of interest is rigid, but, if only semi-rigid (such as an anatomical body), then at least it can be kept essentially fixed in shape while the measurements are taken.
The system and technology most relevant to the present invention is the sonic-based Freepoint 3D™ Sonic Digitizer by GTCO Calcomp (Columbia, Md.). See also U.S. Pat. Nos. 4,956,824 and 5,379,269 by Sindeband, et al. This device tracks the location of a hand-held probe outfitted with two or more sonic impulse emitters. Electronic timing circuits then measure the distance between each emitter and each of at least three microphones (arranged in a triangular array) by measuring the transit time of the sound pulse through air from each emitter to each microphone (which are at known locations in a 3-d coordinate system). From those distances the XYZ coordinates of the probe tip can be computed by finding the intersection of three spheres centered respectively at the microphones and having radii equal to the three respective distances. (Two such points actually exist, but usually only one is the obvious solution. This ambiguity can be eliminated if more than three microphones are used and are not all in the same plane.)
One limitation of the above system in measuring many points on an object is that the object must remain in a fixed location and orientation during the measurement session. If not, the location and orientation of the object itself must also be explicitly tracked in addition to the position of the probe. In this latter case, the probe tip location can be computed as coordinates within a local coordinate system which moves fixedly with the object. That is, the object itself, in addition to the probe, has sonic impulse emitters attached to it—three or more. Both the probe and the object are then tracked relative to a global coordinate system relative to the microphone array. Then by using well know methods of matrix algebra (inverse transformations and composite transformations), a local coordinate reference system is defined in a fixed relationship to the tracked object, and the probe coordinates can be computed relative to it. (See U.S. Pat. No. 5,920,395.) This is required because the microphone frame is not usually fixedly attached to the object of interest, and the microphone array provides a coordinate system for the tracking device to which both the object and the probe are referenced. It would be beneficial to avoid this added complexity (and source of further inaccuracy) by attaching the microphones directly to the object of interest.
Another limitation of a sonic system is that temperature variations (and to a lesser extent humidity and pressure) greatly influence the accuracy of the system. Therefore, sonic systems typically have some provision to compensate for temperature or to measure the speed of sound between a sonic impulse emitter and a microphone separated by a known distance.
A further limitation of the optical systems is that maintaining “line-of-sight” through the air is generally problematic. This may restrict measurement of all sides of the object of interest, including its “far side” and “the bottom”). Mechanical arms and CMMs have a similar problem freely accessing all sides of an object. Magnetic systems overcome the line-of-sight limitation, but only if the object is non-metallic and there are no ferromagnetic objects nearby to distort the measurements.
For some applications, including some medical applications, the non-contact (optical) systems described above are not appropriate, because in addition to the spatial location data other physical measurements are being acquired, which require contact with the object of interest. For example, color or even temperature could be acquired without contact by a probe, but surface elasticity or electrical conductivity could not.
Therefore, in light of the foregoing limitations of existing approaches, the first objective of this invention is to provide a system which can acquire the surface shape of a rigid object or of a semi-rigid object which is temporarily maintained at a nearly fixed shape. A second objective is to acquire the values of at least one physical or physiological attribute at they relate to locations on the surface of the object. A third objective is to compensate for any local distortion (if any) caused by the force of contact of a probe with a soft surface. A fourth objective is to do so without the usual line-of-sight and environmental limitations of conventional 3-d surface point measuring systems (such as temperature, lighting, or magnetic distortion). A fifth objective is to do all this relatively inexpensively.
The principal advantage of a system satisfying these objectives is that it could be usefully applied to medical applications involving portions of anatomy such as arms or legs.
To accomplish the stated objectives, the present invention comprises a geometrical 3-dimensional coordinate system, a moveable contact probe, a plurality of sonic receivers, a timing and control unit, a digital computer comprising appropriate computation hardware and software, and a graphical display. In one enhanced embodiment, the sonic receivers could also transmit sound, so hereafter they will be referred to as sonic transducers. The probe itself comprises a sonic waveform transmitter for localization purposes. It further comprises a collocated sensor for measuring the local value of at least one local physical or physiological attribute of the portion of the object or anatomy of interest. The attribute, for example, might be the surface normal direction, the surface temperature, or the softness of the surface of the object at the point of contact. The probe is intended to be moved manually but could be moved robotically. Consideration is given to correcting the local distortion of soft surfaces due to the force of contact of the probe.
The method of operation of the invention comprises steps of
While one important application of this invention is for use with animal or human anatomy, it may equally well be applied to inanimate objects—especially solid bodies comprised of homogenous material.
The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate a preferred embodiment of the present invention and, together with the description, serve to explain the principle of the invention.
The numeric identifiers in the figures correspond to the elements as follows:
Preferred embodiment of this invention as an apparatus, as shown in
The sonic waveform transmitted by sonic transmitter 30 (shown in
The control circuit 10 would preferably filter out superfluous noise using well-known techniques (such as a band-pass filter) and reject any waveforms which may have taken an indirect path. Such an indirect path 29 might be one in which the waveform internally reflects off the inside surface or surfaces of the portion of the object, rather than taking the direct path 29 from transmitter 30 to a transducer 14. One way to do this for each transducer 14 is to ignore reception of any waveforms after receiving the first one. The control circuit 10 does not initiate another sonic transmission from sonic transmitter 30 until no further sound from indirect paths is likely. For example, no further waveforms would be expected after waiting several milliseconds in an application involving human anatomy. So control circuit 10 could safely initiate a new waveform as frequently as every 1/100 of a second.
The computation unit 16 obtains from the timing and control circuit 10 the “time-of-flight” of the waveform from the transmitter 12 to each transducer 14. Given the speed of sound through the material of the object or anatomy, the distance between the transmitter 12 and each transducer 14 is easily computed by multiplying the speed times the time. The nominal speed of sound through human tissue is approximately the same speed as in water, which is 1540 meters per second. Then given the known 3-d location of each of at least three non-collinear transducers 14 (the XYZ coordinates in some coordinate system 26), the location of the probe (in the coordinate system 26) may be computed by a well-known triangulation computation (essentially the intersection point of three spheres centered on the three transducers 14 respectively, with radii equal to the three “time-of-flight” distances. (With only three transducers, there actually are two possible solutions, one if which sometimes can be obviously eliminated. For unambiguous results, at least four transducers 14, not all in the same plane, should be used.) If more then three transducer distances and locations are known, some suitable weighted best-fit computation should be used. If the peak intensity of the received waveform is available in addition to the time-of-flight, the relative peak intensity corresponding to each distance could be used to weight the best-fit computation. Alternatively, the computation could simply discard all but the three most intense waveforms for the computation. (Although the discarded waveforms would nevertheless be used to remove the two-point ambiguity present with only three distances.)
There is a more direct computation of the location of the sonic transmitter which does not involve dealing directly with spheres. Assume that point P is the location of the probe transmitter, that A and B are two transducers, and that Q is the point on line AB such that line PQ is perpendicular to AB. See
(|AB|2+|AP|2−|BP|2)/(2*|AB|)=|AQ|
where |AB| is the calibrated distance between the transducers and |AP| and |BP| respectively are the sonically measured distances between the transmitter and the two transducers. (Assume that the location of each transducer or transmitter is the center of the area of contact with the object.) Therefore, point QAB is computed as
QAB=(|AQAB|*B+|BQAB|*A)/|AB|
where points QAB, A, and B in three dimensions are each represented as a coordinate triple. Furthermore, let V be the unit vector parallel to line AB. In vector notation,
VAB=(B−A)/|AB|
The equation for the plane containing P and perpendicular to AB is
VABx*X+VABy*Y+VABzZ=VAB*QAB
where VAB*QAB is the scalar product of two coordinate triples. Then given the coordinates of a third transducer at location C, similarly find the equations of the two planes through P which are respectively perpendicular to lines AC and BC. This involves computing the unit vectors parallel to those lines. Then the location coordinates of point P are the solutions for X, Y, and Z in the set of three equations representing the three planes. X, Y, and Z may be computed using the well-known Kramer's rule.
If there are more than three transducers, there would be more than three of the above equations with the three variables X, Y, and Z. Then one could compute a least-squares, best-fit solution using well-known techniques. Alternatively, one could find the solutions for all combinations of three transducers at a time, and then compute the weighted average of all the solutions, where the weights are proportional to the “merit” of each three-transducer combination. The merit might be chosen to be the area of the three-transducer triangle times the product of the peak amplitude of the waveform for each transducer.
Besides the sonic transmitter 30, the probe 12 may house a sensor 32. Sensor 32 may measure some physical attribute of the object or anatomy such as the temperature at the point of contact, the color, the elasticity of the surface under the probe, the electrical conductance, or the ultrasonically measured internal geometry. Such data in conjunction with the probe's 3-d coordinates at the point of measurement would permit the accumulation of a geometrically-related mapping of the acquired attribute data.
Alternatively, the sensor 32 may be a set of three orthogonal accelerometers which detect the orientation of the probe with respect to gravity. Assuming that the contact surface of the probe is flat and that the probe surface must contact the object's surface more or less tangentially in order to operate, then the accelerometers can detect partial information about the vector normal to the physical surface of the object 99 at the point of contact. However, there would be ambiguity regarding the probe orientation about the vertical axis. Adding a magnetic detector, also part of sensor 32, to detect the orientation of the probe with respect to the Earth's magnetic field could yield a complete estimate of the normal vector at the point of contact. This probe orientation information would be particularly helpful in splicing together 2-d ultrasound image slices, if the probe also included a conventional ultrasound imaging transducer.
Note that the sound paths are expected to follow straight lines from the transmitter 30 through the material 99 to each transducer 14. This suggests that the portion of anatomy (or other object) must be more or less convex and contain no significant cavity. This limitation can be relaxed by including a sufficient number of transducers 14 at well distributed locations, so that at least three transducers 14, not along a straight line, do receive the sound along direct paths from any point on the surface. However, transducers not receiving the sound directly will record an arrival time longer than expected (either being reflected off or refracting around an intervening interior surface). The computation unit 16 can estimate a best-fit location of the transmitter 30, then eliminate inconsistent paths (for example, any longer than expected from the best-fit location), and repeat the estimation until all remaining paths entail a single consistent location for the probe.
Furthermore, note that the sound paths are expected to traverse the object through homogeneous material or at least through materials which transmit sound at approximately the same speed. This may not be true for many objects of interest—in particular human limbs. The speed of sound through fat is 1450 m/s (meters per second), while through muscle it is 1585 m/s (from Beverly Stern, The Basic Concepts of Diagnostic Ultrasound, Yale-New Haven Teachers Institute). However, the nominal speed through a mix of soft tissue is often taken to be 1540 m/s. However, bone transmits sound much faster: 4080 m/s. This speed differential at the boundary of bone and soft tissue means most of the sound is reflected back off bone rather than being transmitted into and through it. So a sound along a path traversing a long bone will tend to refract around the bone—following a path slightly longer than the direct path—rather than traverse through it. Such longer-than-expected paths can be eliminated if enough other transducers exist with direct paths from the probe.
For objects with soft surfaces, or with surfaces of varying softness (elasticity), the contact pressure of the probe may locally distort the surface inward by an unknown amount and therefore provide incorrect computed coordinates for the surface point when it is undistorted. This can be mitigated either actively or passively or both. Passive mitigation might simply mean providing a contact base for the probe with enough area to diminish the inward displacement by spreading forces over a broader area. Active mitigation might measure the contact force of the probe, estimate the elasticity (the spring constant in meters per newton) at the contact point, and compute a correction distance to be applied to the model being constructed.
One way to estimate the elasticity is to compute the change in probe location divided by the change in applied force (using a pressure gauge built into the probe, such as sensor 32) as the applied force changes from zero to some maximum and back to zero at a given sample point. Relating the change in distance relative to distortion force would require calibration, because the area of the probe contact area is also a factor. This approach would require the user to use a “hopping” measurement technique in which the probe is placed into contact with the object surface (non-zero force), then removed from contact with the surface (zero force), and then moved to the next contact sample point on the surface in the same manner. That is, a sequence of measurements would be taken at each sample point which characterizes the dynamics of the contacting, distorting action, thereby providing an estimate of the elasticity (force/displacement ratio). A simpler method would merely reject all measurements with more than a light contact pressure. That is, the invention would use only the surface geometry measurements taken when the probe just touches the surface hard enough to allow the sound waveform from the transmitter 30 to enter the object. The system would simply assume that with the light contact pressure little or no distortion has occurred.
There is a more complex embodiment for estimating the elasticity—for the purpose of computing the surface distortion from the contact pressure—an embodiment which does not require the “hopping” technique. That is, this embodiment would allow the probe 12 to be smoothly slid along the surface of the object 99 while maintaining sufficient contact pressure to insure that the sonic transmitter 30 can inject sound into the object 99. This embodiment adds at least one spring-loaded plunger 34 to the probe, where the contact face of the plunger is much smaller than the contact face of the whole probe. In simplified form, this is shown in
A linear displacement sensor 36 adjacent to the plunger would return the amount of linear plunger displacement as the plunger presses further into the soft surface than does the rest of the probe face. Spring 38 has a known spring constant (force/displacement ratio) and it tries to push plunger 34 into the surface of object 99 relative to the contact base of probe 12. The softer the material, the larger the local “dimple” and the larger the displacement of plunger 34. (The plunger 34 and displacement sensor 36 could simply be the sensor 32 of
The method of this invention begins by affixing the transducers 14 in contact with the object of interest 99, establishing a coordinate system, and calibrating the coordinates of the locations of the transducers within that coordinate system. Thereafter, the probe 12 is placed on a first sample location of the object 99. The timing and control circuit 10 causes the sound transmitter 30 in probe 12 to generate a waveform of sound. Then the transducers 14 each report the arrival of the sound waveform and therefrom circuit 10 acquires the transit time of the sound waveform. From those times, computational unit 16 computes probe-to-transducer path distances 28 and from those computes the XYZ coordinates of the sound transmitter 30 and the probe 12 using 3-d triangulation. Essentially simultaneously, the sensor 32 measures a physical attribute at the first sample location. The value of that attribute is recorded and associated with that location. Then the probe 12 is moved to another location, its location coordinates and attribute value are similarly determined and recorded. The process is repeated until sufficient data is acquired from all over the surface of the object 99 in question. More than one measurement might be taken at each location, especially if elasticity and deformation are being measured.
Notice that the object of interest may be relocated or reoriented in space as long as it is maintained in a more or less fixed shape during the whole measurement acquisition. This is because the positions are relative to the attached transducers, not to some external frame of reference, as would be the case with a typical 3-d optical or mechanical measurement system.
Once all the data has been acquired, the computational device 16 constructs a 3-d geometrical and physical model from the recorded data. Procedures for doing this are well-known in computer graphics: such as using the measurement locations to construct a Delauney triangular mesh to represent the surface. To create a smoothly curved surface the planar triangular facets could optionally be converted to smooth curved patches (bi-cubic or NURBS). Once the model is constructed, the model is rendered on display unit 18—such as a Gouraud smoothly shaded perspective view. In a preferred embodiment, the user finally can interactively rotate, scale, and pan the rendered view of the model. The values of the physical attribute are also rendered, perhaps as numeric values distributed on the visible surface at the corresponding locations. A more appealing rendering of the attribute's values might be to display each attribute value as a corresponding hue from the color spectrum.
It is assumed that sufficient data is collected to build a representative model. One measure of sufficiency is to require that there is at least one sample point within some given distance of any point on the portion of the model. Until that is the case, the system could direct the user to sample more points in any portion of the model for which the sampling is insufficient.
Either during or after the acquisition and recording of the locations and attribute values of contact points, but only if necessary, the XYZ coordinates are corrected for the distortion due to the contract pressure of the probe. This mitigation may be either passive (preventative) or active or both. If active, the coordinates of each point are adjusted by the estimated displacement, which comprises a vector normal to the local surface and a magnitude computed from the locally measured probe force and elasticity. This corrective mitigation would be applied individually to each recorded sample point.
One way to calibrate the locations of the transducers 14 affixed to the object 99 is to place the contact probe at a number of locations and record the sonic transit times to each of the transducers. Given the an assumed speed of sound, the distances between probe locations and the transducers can be computed. This leads to a set of equations of the form
(Pi,x−Tj,x)2+(Pi,y—Tj,y)2+(Pi,z−Tj,z)2=Di,j2
where Pi,x is the X coordinate of the ith calibration point, Tj,x is the X coordinate of the jth transducer, Di,j is the measured (constant) distance between them, and so forth. Given sufficient probe calibration points, an arbitrary coordinate system, and using an known iterative technique, a best-fit solution can be found for the P and T variables. Such methods can be found in the classic work Numerical Recipes in C, by William H. Press, et al (Cambridge University Press).
Furthermore, if the actual distance between two widely spaced probe contact points is known, it can be compared to the distance implied by the calibration to check the assumed value for the speed of sound. Then, if the two distances are unequal, all the coordinates should be scaled by their ratio.
Note that the probe points need not be separately acquired beforehand; the system could simply use some or all the data taken for building the eventual model. That is, rather than recording the calculated coordinates of all the sampled locations during the sampling process, simply record the raw sonic transit times; at the end of data collection, post-process them as above to compute the locations of the transducers and the sample points.
In one enhanced embodiment, the transducers can also be used, one at a time, as transmitters for the purposes of automatically calibrating their locations with respect to each other. That is, instead of requiring the probe to be placed at various locations for calibration, each transducer acting as a sonic transmitter fulfills the role played by the probe in the calibration method described in a preceding paragraph. That is, assuming a given speed of sound in the medium of object 99, the transit time of the sound from each transducer to each other transducer fixes their separation distances. Given an arbitrarily created coordinate system, 3-d coordinates can be assigned to each transducer by an automatic computation. For example some particular transducer would be defined as the origin (0,0,0); another transducer would be defined as (X1,0,0) where X1 is the distance from the first transducer; a third (non-collinear) transducer would be at (X2,Y2,0), which implicitly defines the Y axis, where X2 and Y2 are such that the distances to (0,0,0) and (X1,0,0) are match the measured distances to the previous two transducers; the Z axis is defined perpendicular to the X and Y axis; and the pair-wise distances to the remaining transducers, if any, determine their coordinate triples also.
The advantage of doing calibration with the latter, enhanced embodiment is that the 3-d coordinate triples of the probe locations can be computed on the fly. This would allow the computation unit 16 to immediately render the model on display 18 as it is being constructed and the construction can happen in parallel with the data acquisition. This would provide feedback to the user to show where points have been sampled or not and how dense they are.
While this invention is described above with reference to a preferred embodiment and some variations, anyone skilled in the art can readily visualize other embodiments of this invention. For example, the probe many contain a more than one sensor, so that the value of each of plurality of physical or physiological attributes at each contact point can be acquired, recorded, modeled, and displayed. Therefore, the scope and content of this invention are not limited by the foregoing description. Rather, the scope and content are delineated by the following claims.
