Patent Application
20050068544
The invention relates to a method for the three-dimensional detection of an object. Furthermore, the invention relates to an apparatus for performing the method and a use of the apparatus and the method.
Methods for the three-dimensional detection and digitization of objects are used for various application purposes, e.g., in the development, production and quality control of industrial products and components. In medical technology, use is made of, for example, optical measurement methods for producing the housings of hearing aids that can be worn in the ear.
For the purpose of individually adapting a housing to the auditory canal of a person wearing a hearing aid, impressions of the patient's outer auditory canal are created by an audiologist by way of a rubber-like plastics composition. In order to be able to employ stereolithographic or similar methods for producing the housings, it is necessary to create three-dimensional computer models from the ear impressions. This procedure has previously been effected by the hearing aid manufacturer, where the impressions are measured panoramically three-dimensionally by way of a precision scanner, and a 3D computer model of the outer auditory canal is created on the basis of these data. Afterwards, in a laser sintering process, the individually formed housing shell is produced on the basis of the data of the computer model.
The precision scanners used are usually designed as laser scanners in which a laser beam is guided over the surface of the impression in a controlled manner and the backscattered light is observed by a detector (e.g., a CCD camera) from a direction deviating from the laser beam. The surface coordinates of the impression are then calculated by triangulation. In the case of the known laser scanner VIVID 910 from the company Minolta, a line is generated from the laser beam and is moved over the surface of the object to be detected, e.g., an ear impression. The image of the line is in turn observed by a camera, the surface coordinates of the object to be detected being deduced from the deformation of the line image by triangulation. A rotary stage controller on which the object rotates through 360° during the scanning serves as an accessory to the known laser scanner.
What is disadvantageous about the known laser scanners is their high procurement costs, which are occasionally also caused by the high-precision mechanism of the rotary stage controllers.
Frank Forster, Manfred Lang, Bernd Radig in “Real-Time Range Imaging for Dynamic Scenes Using Color-Edge Based Structured Light”, ICPR '02, Vol. 3, pp. 30645-30648, 2002, disclose a method for the 3D detection of an object by way of structured light. In this case, a projector is used to project a color pattern containing a redundant code with known projection data onto the surface of an object, and the object with the color pattern projected thereon is recorded by a camera from a direction deviating from the projection direction. By decoding the color pattern at each pixel of the camera image, it is possible to determine the associated three-dimensional coordinates of the object surface by way of triangulation. This method permits the reconstruction of a partial region of the surface of the object with a video image.
Japanese Patent Document No. JP 2001108421 A discloses a 3D scanner for the three-dimensional detection of an object. During scanning, the object rotates together with a reference object on which markings are provided. Thus, different views of the object and of the reference object are photographed, the photographs being combined to form a three-dimensional computer model on the basis of the markings on the reference object. What is disadvantageous about the known method is (for some applications) the inadequate correspondence between the computer model and the real object.
It is an object of the present invention to provide a method and also a panoramic scanner which make it possible to detect three-dimensionally an object, in particular an ear impression, in a comparatively simple and cost-effective manner with the accuracy required for producing a hearing aid housing shell.
This object is achieved by a method for the three-dimensional detection of an object, comprising: providing and object to be detected, a projector and rotator configured for rotating the projector and the camera relative to the object; providing markings with a position relative to the object that remains the same during the rotation; projecting a pattern onto the object to be detected with the projector; recording an object image with the camera, and detecting the image of at least one marking in the object image; repeatedly adjusting the projector and the camera relative to the object with respective projection of the pattern and recording of an object image until a termination criterion is reached; automatically combining the object images or data obtained from the latter on the basis of the images of the markings that are contained in the object images; and creating a three-dimensional object model from the combined object images or data.
This object is also achieved by a panoramic scanner for a three-dimensional detection of an object, comprising: a projector configured for projecting a pattern onto the object to be detected; a camera configured for detecting object images; a rotator configured for rotating the object relative to the projector and the camera having a position relative to the object that remains the same during the rotation, images of the markings being present in the object images, configured so that it is possible to combine object images generated at different angles of rotation of the object relative to the projector and the camera, or data obtained from these object images, based on the images of the markings that are present in the object images, to form a three-dimensional object model.
Various embodiments of the invention are discussed below. The three-dimensional detection of an object utilizes a projector, a camera and mechanism for rotating the projector and the camera relative to the object. The projector projects a two-dimensional pattern, e.g., a color pattern, containing a redundant code with known projection data onto the surface of the object. The color pattern projected on is subsequently recorded by a camera, e.g., a CCD camera, from a direction deviating from the projection direction. By decoding the color pattern at each pixel of the camera image, the associated three-dimensional coordinates of the object surface are determined by way of triangulation.
In order to enable a three-dimensional panoramic view, the object rotates relative to the projector and the camera. For this purpose, the object is preferably situated on a rotary stage controller. The rotary stage controller rotates through a predeterminable angle between two recordings, so that it is possible to record a plurality of object images, e.g., 60, per periphery.
During a scan, the object generally rotates once through 360° about the rotation axis. If only a partial region of an object is to be digitized, then the object may also be rotated through an angle of less than 360°. Furthermore, it is also possible for more than one complete revolution to be performed during the detection of an object in order to increase the accuracy of the 3D model to be generated. By way of example, five completely executed revolutions of the object then constitute a termination criterion for the scan.
In order that a contiguous panoramic view of the object can be generated from these individual images, it is advantageous if the 3D data of the individual images are related to a common coordinate system. For the requisite calibration, in accordance with an embodiment of the invention, markings are provided on the scanner and do not change their position with respect to the object during scanning. With the use of a rotary stage controller, the markings are preferably situated on the rotary stage controller or at the edge of the rotary stage controller. The markings are configured in such a way that a specific number of these markings are visible in each camera image and the angle of rotation of the object relative to the projector and the camera can be gathered from these markings unambiguously and with the required accuracy. In this case, a higher number of markings increases the accuracy of the 3D reconstruction.
In an advantageous manner, the position of the markings that are moved with the object is precisely determined once with respect to a “world coordinate system” and communicated to the evaluation system. It is then possible to determine the relative position of the object with respect to the projector and the camera or the angle of rotation of the rotary stage controller from the position and the coding of the markings recorded in the object image in the coordinate system. Successively recorded individual images or the 3D data records obtained from the latter can then be combined in a simple manner by way of a corresponding coordinate transformation to form the overall view in the “world coordinate system”.
Advantageously, a synchronization of the individual image recordings with the rotary movement of the object is achieved in a simple and cost-effective manner without this requiring a high-precision and correspondingly expensive mechanism. A user of the panoramic scanner does not have to perform any calibration or adjustment operations, with the exception of fixing the object to be measured on the rotary stage controller.
Consequently, what has been created is a possibility for detecting the 3D panoramic surface of an object, this possibility being simple to control but nevertheless highly precise and cost-effective. The panoramic scanner is therefore e.g., especially suitable for use by an audiologist who creates an ear impression of a patient and digitizes it three-dimensionally by way of the scanner, so that the model data obtained can be communicated directly to the manufacturer of a housing shell by data transmission (E-mail or the like). This saves time and costs in the production of a hearing aid housing.
In one embodiment of the invention, a plurality of overlapping object images are recorded in the course of a revolution of the object relative to the camera and the projector. In this case, a plurality of the same markings are then visible in each case in successive object images. With the aid of the common visible markings, the object images are combined in such a way as to produce an “image composite”. A precise measurement of the markings is not necessary for this purpose, which simplifies the production of the system.
The relative camera coordinates of each recording can be determined by way of a method that is referred to as “cluster compensation” and is known from photogrammetry. A few markings measured in the “world coordinate system” serve for relating the image composite thereto. After this step, the individual object images can then be combined in a simple manner by way of a corresponding coordinate transformation to form the overall view. In order to simplify the calculation, two axes of the “world coordinate system” lie in the plane spanned by the rotary stage controller and the third axis of the “world coordinate system” coincides with the rotation axis of the rotary stage controller.
The markings are preferably configured in such a way that they contain a coding with the extent 1−n, e.g., in the form of a binary code. The markings advantageously contain a few measurement positions (corners, lines, circles or the like). The markings recorded in the object images are automatically detected, decoded and measured in each object image by way of a suitable image processing software. The markings are preferably embodied in such a way that, for each object image, on the basis of the markings contained therein, it is possible to unambiguously assign the spatial position with respect to the camera and the projector.
In one embodiment of the invention, it is provided that the rotation axis about which the object rotates relative to the projector and the camera can be pivoted relative to the projector and the camera. When using a rotary stage controller, the simplest way of achieving this is by tilting the rotary stage controller by a specific angle in at least one direction. This affords advantages in particular in the digitization of ear impressions since the latter may be comparatively fissured. By pivoting the rotation axis, it is possible to prevent shading and thus gaps or inaccuracies in the three-dimensional computer model.
In an advantageous embodiment of the invention, the markings are arranged and configured in such a way that, in addition to the angle of rotation, the angle by which the rotation axis is pivoted with respect to a starting position can also be detected from each object image. In this case, the position of the rotation axis in the preceding object image or an original position may serve as the starting position.
In an alternative embodiment of the invention, at least two cameras arranged offset with respect to one another are present, so that the object can be recorded simultaneously from different viewing angles. The cameras are fitted at a different height with regard to the rotation axis of the object to be detected, so that even undercuts of the object, which would lead to defects in the computer model when using just one camera, can be detected by the further camera. A pivot movement of the rotary stage controller relative to the cameras can thereby be dispensed with. In an advantageous manner, a second projector is also used in addition to a second camera, so that object images are in each case generated by a camera-projector pair.
The self-calibration property of a panoramic scanner has the advantage that all the individual 3D object images can be combined in a simple manner to form a 3D panoramic image. In this case, no stringent requirements are made of the constancy of the rotary movement. A synchronization of the rotary movement with the image recordings is not necessary. It is possible, therefore, to have recourse to a cost-effective mechanism. The accuracy of the 3D detection can easily be increased by increasing the number of images per revolution.
The robustness and accuracy of the measurement rise significantly as a result of a high number of measurement data and in particular as a result of overlapping object images.
The invention is described below on the basis of exemplary embodiments as illustrated in the Figures.
The apparatus 1 has a projector 4, which projects a color pattern 5 onto the surface 2 of the object 3 to be detected. In the case illustrated in
In the case of the exemplary embodiment illustrated in
In order to enable the three-dimensional spatial coordinates of the point P on the surface 2 to be determined from an individual image 7 even when the surface 2 of the object 3 has depth jumps and occlusions, the color pattern 5 is constructed in such a way that the coding of the projection planes g is as robust as possible with respect to errors. Furthermore, errors based on the coloration of the object can be eliminated by way of the coding.
In the case of the exemplary embodiments illustrated in
The color pattern is then intended to satisfy the following conditions:
An example is provided below of the color pattern 5 which satisfies the five conditions mentioned above. This color pattern 5 relates to the RGB model with a red color channel R, a green color channel G and a blue color channel B. Since color values in each color channel are only permitted in each case to assume the minimum value and maximum value, a total of eight mixed colors are available, which are respectively assigned the following numbers:
A length of four color stripes was chosen for the code words of the color values, with overlapping of adjacent code words in each case with three color stripes.
The color changes were also assigned numerical values. Since the color value can remain the same, decrease or increase in each of the three color channels, the result is a total of 27 different color changes of the mixed color, which were respectively assigned a number between 0 and 26. The length of the code words assigned to the color changes was chosen to be equal to three color changes, with overlapping of adjacent code words in each case with two color changes.
A search algorithm found the following series of numbers, which describes an exemplary embodiment of the color pattern 5 which satisfies the five conditions mentioned above:
1243070561217414270342127216534171614361605306 3527170724163052507471470650356036347435061725 24253607
In the exemplary embodiment specified, the first code word comprises the numerals 1243, the second code word comprises the numerals 2430 and the third code word comprises the numerals 4307. The exemplary embodiment shown constitutes a very robust coding.
The ear impression is detected by way of coded illumination and triangulation. For this purpose, the panoramic scanner comprises a projector 12, which projects a color-coded pattern onto the surface of the ear impression 11. The color pattern projected onto the surface of the ear impression 11 is converted into an image of the ear impression 11 by a CCD camera 13. By virtue of the rotary movement of the rotary stage controller 10, it is possible to record a multiplicity of such imagings from different observation angles.
In order that the individual imagings can be assigned the respective observation angle, markings 14 are provided at the outer edge of the rotary stage controller 10. In addition to the ear impression 11, a number of these markings 14 are also detected in each image. The images of the markings 14 are automatically detected, decoded and measured in the object images by way of a computer 15 with suitable image processing software. On the basis of the angular information obtained therefrom, a three-dimensional computer model of the ear impression 11 is calculated from the individual imagings. The computer 15 is preferably not part of the actual panoramic scanner, i.e., not arranged with the rotary stage controller 10, the projector 12 and the camera 13 in a common housing. Rather, an external powerful PC with a suitable software may be used as the computer 15. The panoramic scanner then has an interface for connection to the computer 15.
Furthermore,
Specifically, the rotation of the rotary stage controller 10 drives a gearwheel 21A connected thereto, which engages in a toothed piece 21B fixedly anchored in the housing of the scanner and thereby leads to the pivot movement of the drive unit with the motor 16 and the toothed belt 18. The markings 14 provided at the edge of the rotary stage controller 10 can furthermore be seen, which markings make it possible to determine the precise angle of rotation of the rotary stage controller 10 and thus of an object mounted thereon (cf.
At the beginning of the detection of an object, the rotation axis is advantageously situated in the starting position envisaged therefor. This may be effected e.g., by a housing cover (not illustrated) being fixed in a pivotable manner to the housing of the panoramic scanner. This housing cover must first be opened before an object is positioned on the rotary stage controller 10. In the course of this housing cover being opened, the entire rotation unit with the motor 16 and the rotary stage controller 10 is then transferred into its starting position by way of a corresponding mechanism (not illustrated).
Consequently, at the beginning of a scan, the rotary stage controller 10 is situated in the starting position illustrated in
As an alternative, the rotary stage controller 10, for execution of the pivot movement, may also be connected to a second motor (not illustrated). The pivot movement may then also be controlled by the computer 15, so that the number of revolutions of the rotary stage controller during which the latter pivots from a starting position into an end position is variable.
In the case of the panoramic scanner in accordance with
Furthermore, the projector 63 is not designed as a point radiation source, but rather emits a coded pattern proceeding from a vertically running line. This ensures the projection of the pattern onto all regions of the object that are detected by the cameras. As an alternative, it is also possible to use a plurality of projectors with a point radiation source (not illustrated).
By virtue of the use of a plurality of cameras, a pivot movement of the rotary stage controller 60 becomes invalid and the drive unit can be simplified compared with previous exemplary embodiments. Thus, the rotary stage controller 60 is driven directly (without the interposition of a toothed belt) in the exemplary embodiment in accordance with
In the case of the panoramic scanner in accordance with
For the purposes of promoting an understanding of the principles of the invention, reference has been made to the preferred embodiments illustrated in the drawings, and specific language has been used to describe these embodiments. However, no limitation of the scope of the invention is intended by this specific language, and the invention should be construed to encompass all embodiments that would normally occur to one of ordinary skill in the art.
The present invention may be described in terms of functional block components and various processing steps. Such functional blocks may be realized by any number of hardware and/or software components configured to perform the specified functions. For example, the present invention may employ various integrated circuit components, e.g., memory elements, processing elements, logic elements, look-up tables, and the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices. Similarly, where the elements of the present invention are implemented using software programming or software elements the invention may be implemented with any programming or scripting language such as C, C++, Java, assembler, or the like, with the various algorithms being implemented with any combination of data structures, objects, processes, routines or other programming elements. Furthermore, the present invention could employ any number of conventional techniques for electronics configuration, signal processing and/or control, data processing and the like.
The particular implementations shown and described herein are illustrative examples of the invention and are not intended to otherwise limit the scope of the invention in any way. For the sake of brevity, conventional electronics, control systems, software development and other functional aspects of the systems (and components of the individual operating components of the systems) may not be described in detail. Furthermore, the connecting lines, or connectors shown in the various figures presented are intended to represent exemplary functional relationships and/or physical or logical couplings between the various elements. It should be noted that many alternative or additional functional relationships, physical connections or logical connections may be present in a practical device. Moreover, no item or component is essential to the practice of the invention unless the element is specifically described as “essential” or “critical”. Numerous modifications and adaptations will be readily apparent to those skilled in this art without departing from the spirit and scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 103 44 922.1 | Sep 2003 | DE | national |
The present application claims the benefit of U.S. Provisional Application No. 60/505,911, filed Sep. 25, 2003, herein incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 60505911 | Sep 2003 | US |
