Patent Application
20150138320
The following is a tabulation of some prior art that presently appears relevant to this application:
A 3D scanner is an apparatus that captures the geometry of real physical objects, and converts them into an accurate digital representation.
Although 3D Scanners have been around for several years, most of them have been unable to provide high accuracy models while maintaining low production cost and a fast scanning speed. This can be evidenced by looking at today's 3D Scanner market, where most high-accuracy scanners cost in the thousands of dollars, and most low-cost scanners generate low-accuracy models. In this application, we will use the term high-accuracy to mean any model where the scanned object representation is within a 1 mm error or less of the physical model.
In particular, some 3D scanners, known as 3D laser scanners, use lasers or other high-intensity light emitters to project a light beam onto an object, and then have a camera or other sensor pick up the projected laser profile, using it to mathematically reconstruct the geometry of the object. Those scanners tend to suffer from being slow and expensive, but are usually very accurate. One of the earliest mentions of such a system dates back to 1969, in Bickel's U.S. Pat. No. 3,625,618. It describes the general method of shining a narrow beam of laser and capturing geometry from it—as we will see later, one of the issues with this patent and the next few patents is that they are not very good at dealing with occlusions and at processing data quickly. A few years later (1976), U.S. Pat. No. 4,089,608 took the idea further and made it more explicit. Further refinements have followed in the eighties and nineties, including U.S. Pat. Nos. 5,027,081, 5,477,371 and 5,636,030, which essentially introduce the turntable system and movable carriages. Later patents such as U.S. Pat. No. 6,917,702 introduce methods to use and calibrate multiple fixed cameras with the turntable but adding more cameras adds more cost and doesn't guarantee a much better model. Most of these scanners only capture geometry and fail to capture color and texture. Many 3D laser scanners also suffer from occlusion problems—for example, any time a camera and a laser are in a fixed position several details of the objects will not be visible. For example, when scanning a teapot with a fixed position laser scanner, it is difficult to capture the inside of the handle as it is occluded by the main body of the teapot on one side and the outside of the handle on the other. Practically, 3D scanners that are based on fixed position light emitters don't perform very well when given non-convex objects. They can mitigate this by making the camera/lasers moveable but that often entails significantly longer scan times.
There have also been successful attempts to improve this by increasing the number of lasers, as suggested by Knighton in U.S. Pat. No. 7,995,834. However, the common theme is that there is usually a trade-off between cost, speed and accuracy. In this case the cost is higher since several lasers have to be involved as those tend to be the most expensive part of a scanner in high-accuracy scanners.
Other scanners use a technique known as structured lighting to quickly get a 3D view of the image. Recently those scanners have become less expensive but currently still suffer from relatively low accuracy. Those have become more main stream recently with patents such as U.S. Pat. Nos. 6,549,288 and 8,493,496. The idea is to project patterns and then capture them. The main disadvantage there is that the projection used becomes very expensive if higher accuracy is required and they currently are only able to deterministically capture only the part of the model that is visible. A simple way to get an estimate is to compare the price of a laser pointer to that of a full high resolution digital video projector.
There are many other methods of 3D scanning available out there, ranging from taking pictures at multiple focal lengths to just using a bed of pins and placing the object on it in U.S. Pat. No. 6,633,416, then measuring the pin displacement. The latter is a contact-based 3D scanner so it differs substantially from ours—it also fails to capture any angled holes or features since the object is placed flat against the pins. Some scanners will just look at the shadows cast by the object from various angles and try to reconstruct it as is the case in U.S. Pat. No. 7,106,898.
Some 3D scanners don't use active illumination (i.e. no structured light, no laser) and just passively take pictures of the target object and then attempt to recombine. This is for example the case with U.S. Pat. No. 5,894,529 where several precisely positioned cameras take pictures. The problem with such scanners is that the reconstructed geometry is overall sparse and inaccurate as the device attempts to recreate a full 3D model out of just a handful of pictures, and they are very susceptible to shadows or other lighting variations. Other methods attempt to match the images to one another and generate a model by identifying distinguishable features. The issue here is that objects which are fairly uniform in color sometimes don't have many distinguishable features. For example, scanning a completely blue ball would be unsuccessful as those devices are unable to match the various pictures of the ball since there are really no distinguishing features. Also, the required processing time for generating such images is significant.
In addition to the aforementioned issues with each of these non-contact scanners, all of these embodiments suffer from some common problems: first, many of them react differently based on the target object's material. For example, if a laser is being shined against a white glossy surface, its reflection will be much more pronounced than if it's being shined against a black matte surface, leading to significant errors. Second, most of them try to find a balance between accuracy, speed and cost but none of them manage to get good results on all three at once. Third, a lot of these have a significant amount of wasted scan time, meaning that they spend a lot of time scanning parts that have already been scanned and don't need to be again. This is more common for laser scanners where often a full scan will be made with a laser, then the camera/laser will be repositioned and another full scan will be made to get more details. This often takes the scan time up to dozens of minutes and sometimes even hours. As we will be dealing with this concept again, we can define wasted scan time as time spent scanning an area of the physical model that our system already has confidently and correctly digitized. Fourth, many scanners have problems in dealing with occlusions and complex models are usually only partially scanned—this is because for scanners with movable cameras/light emitters, there can be an infinite number of positions that can be used and it's impractical and sometimes impossible to scan them all.
In this application, we propose a design and methods for an apparatus capable of combining high accuracy and low-cost, while being resilient to the problems described above.
One embodiment for solving this problem follows: we propose a device comprising an automatically movable light emitter, an automatically movable white light, an automatically movable light receptor, a turntable with positional feedback, an enclosure, and a way to efficiently process the information and select where to scan next. In this embodiment, the device starts by rotating the object on the turntable while shining the light emitter onto it and capturing the profile with the receptor in a manner similar to most turntable scanners. The processor uses this to reconstruct a first pass of the geometry. In a second step, the object is illuminated with white light, and the model's colors are captured. In a third step, the colors are used to refine the originally acquired geometry. For example, if a spot on the model is white, or the same color as the light emitter, the threshold for considering the reflection to be a point is increased, whereas if a laser point had been captured on a black colored point then odds are it is a real physical point. In a fourth step, the processor uses the captured geometry to find areas that haven't been captured (for example, places that appear as holes or that have very low point density) and moves the lasers and cameras to a position and angle that is optimal for capturing those missing parts. The processor may also choose to get closer scans from areas that are blurry or noisy as this indicates there is more detail than the scanner captured. This repeats until all the missing parts have been successfully captured or the desired time limit has elapsed.
The end result is that the object captured will have all the required detail and will not be masked by significant occlusions, while minimizing wasted scan time.
We describe several other embodiments in the detailed description and claims.
We propose a design for a 3D laser scanner that is highly accurate, low cost, high speed, and can deal with several kinds of materials that are otherwise troublesome for 3D scanners, as well as models that contain significant occlusions and holes.
The linear motor drive 111 is best described in
The moveable camera carriage 131 is made of a housing 130 that is designed to hold a camera or other optical receptor, a laser or other optical emitter, and optionally, a white light. In this embodiment, a camera 142 is fitted inside the housing, pointing towards the turntable. The camera is connected to the electronics board 120. In additional a white LED light source 132 is placed near the camera and also connected to the electronics board. A servo motor 134 is push fitted onto a hole of the same size on the housing 130. The servo is connected to the electronics board using electric wires. A line laser 138 is placed inside a metallic laser holder 136. The metallic laser holder 136 is screwed onto the servo 134. The line laser 138 is connected to the electronic board 120 using electric wires. A cover 140 push-fits into the movable camera carriage and further ensures the camera and light remain stationary.
Finally, an electronics board 120 and holder control most of the above components. The board may be optionally connected to an external computer or laptop and respond to its commands, or it can independently perform commands.
The scanner operation involves measuring the profile of the projected laser line onto the target object from various calculated orientations and positions.
To do so, a target is placed on the turntable. The electronics board directs the line laser to be turned on and the camera to start capturing images. A first image is captured, and the profile of the projected laser is detected and used to compute the geometry of the object along that plane since the laser and camera positions in the real world are known. The camera underneath the turntable returns angular the position of the turntable to the electronics board and this allows a single slice of the target model to be reconstructed. Once that is done, the electronics board asks the turntable to spin by the smallest amount that it can spin, and the process is repeated. By combining multiple slices a 3D model of the object is created.
The reconstructed model will typically have several missing parts because many objects cannot be captured with just a fixed camera, a laser and a turntable. At this point, the electronics controller or processor calculate where the missing parts are and ranks them by importance. From that information, it re-adjusts the laser direction by moving the servo, moves the turntable or moves the camera assembly up and down in order to capture a second view of the model that would fill the most holes. This is made possible since each of these systems has a known positional state that is tracked by the processor. The scanner is then re-activated as before. The new model is combined with the old model and the process is repeated. It may also choose to position the laser, camera or object in such a way as to confirm that previous points are valid and not due to noise. For example if a point is seen from two different positions it is more likely to be valid. Other factors come into account when the electronics controller is choosing which position to move the scanner to next: vertex density, meaning the number of measured points in a fixed volume, can be considered. This means that if the laser measured 10 points in a one cubic centimeter volume, but measured 10,000 points in the nearby cubic centimeters, that may be worth some further scanning to confirm. Inter-vertex displacement is another factor—namely if the vertex positions within a specific volume are very noisy and tend to jump a lot. An example formal definition could be: average distance from the vertex to its nearest 8 neighbors. If for some vertices the average distance is much more than the average distance across all vertices, it's possible this is an area of noise that needs to be further investigated.
With that method, the model is repeatedly refined.
For example, if scanning a tall object, the first scan would show that there is a hole at the top of the object since the upper points in real object were out of range of the camera and laser and were not captured. The scanner would respond by moving the camera and laser assembly upwards and performing a second scan.
Optionally, in a second step, the white light is activated and the laser turned off, and a second scan is performed in a manner similar to the first one. The white light allows for capture of color and texture information now that the geometry is known. Using the angular position of the turntable, the position of each of the faces and vertices of the model is estimated and the respective textures and faces are extracted from the image captured by the camera. This allows for a fully-colored and fully-textured model to be reconstructed.
When color is captured, it becomes easier for the system to adjust the measured data. To be more specific, if a captured vertex turns out to be of a darker color, then the threshold for considering it a valid vertex is lowered. This is because darker colors emit less light when a laser is shined upon them so that if a laser profile contained points that turned out to be dark then they are more likely to be correct. On the other hand if a point that turned out to be very light in color, or white, or the color of the laser, was caught in the laser profile, the threshold for considering it a valid point would have to be increased. This is because shining a bright laser onto a bright surface will cause a lot of light to be captured by the camera, even in nearby points. The idea behind this system is to use color information to decrease the noise by adjusting the probability of a point being valid based on its color.
The entire functionality is simplified and summarized in
The overall advantages of such an apparatus are that:
We believe there are several ways to implement the overall system described above. The common factors are an automatically movable optical receptor and light emitter, as well as a controller to decide where to move them next for optimal functionality.
One such embodiment involves mounting the camera assembly onto a mechanical arm that has positional feedback. Such an arm is typically made of one or more motors behaving as joints to the camera assembly. An example is shown in
A third embodiment involves not having a turntable at all for the target object, and instead allowing the camera assembly to rotate around the object while being attached to an arm. This has the benefit of keeping the object stationary.
A fourth embodiment includes an enclosure around the device. Such an enclosure is shown in
Other embodiments can be generated. For example:
Thus the reader will see that at least one embodiment of the 3D scanner provides a more reliable, inexpensive and efficient method for scanning, making such a 3D scanner more affordable to the general population without sacrificing quality.
While my above description contains many specificities, these should not be construed as limitations on the scope but rather as an exemplification of one or several embodiments thereof. Many other variations are possible. For example, it is possible to not have an enclosure with the system. It is possible for the electronics board to be outside of the system and implement instead on a computer for example. The linear drive, shown in the first embodiment as being based on a threaded rod can be based on a belt instead, or on a series of servo motors. The line laser can be replaced with multiple line lasers, or even point lasers.
Accordingly, the scope should be determined not by the embodiments illustrated, but by the appended claims and their legal equivalents.
