Imaging devices that incorporate structured or directional light sources often project light onto a target surface from a source angle different from a viewing angle. This can, depending on the target surface topology, create a shadow condition, herein called Source Shadowing, where the viewing device, herein called a Viewer, can see a point on the target surface but the source light is prevented from reaching that point.
Any viewing device, while looking at a three dimensional target surface from a fixed vantage point, may be unable to see all portions of that target surface, depending on the target surface topology. This viewer obscuration condition is herein called Viewer Shadowing.
Attempts to mitigate these shadow conditions include the following.
There are ‘Multi-Eye’ sensors which mitigate the Viewer Shadowing case by observing the target surface from more than one viewing angle. They achieve this by having two or more camera systems, comprised of separate optics, photo-detecting systems and associated electronics, disposed at angles to one another and observing a focal point plane where structured or directional light from the source will strike the target. These systems are bulky and costly due to their replication of hardware.
A phase profilometry system 3D inspection system, able to mitigate Source Shadowing using multiple source angles, achieved by using macroscopically moving parts, has been disclosed. The macroscopically moving part in this system is a mirror. The mirror moves, either by sliding or rotating, and so directs the source light to one of a plurality of optical channels. Each optical channel is disposed to deliver the light to a target surface from a source angle different from any of the other optical channels. This system is slow, costly and is unreliable due to its moving parts.
A single system able to overcome both shadowing conditions without resorting to macroscopically moving parts or multiple projectors and/or multiple cameras is desirable. For these and other reasons, there is a need for the present invention.
In accordance with aspects of the present invention, an illumination and imaging system, comprising a single light projector and a single camera, is able to project light onto a plurality of focal planes from multiple incident directions and is further able to view a plurality of focal planes from multiple observation directions without macroscopically moving parts.
In the following Detailed Description, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustrating specific embodiments in which the invention may be practiced. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.
Many manufacturing processes, such as those for electronic components and assemblies include inspection and test procedures, which can be either manual or automated. Electronic components, for example, are often mounted on a circuit board, and the components are tested before and/or after final assembly. For example, the surface mount assembly process (SMT) consists fundamentally of three value added process steps: Solder paste printing, component mounting and reflow. These are schematically illustrated in
Several types of errors can occur during these manufacturing processes. Electrical test alone is generally understood to be an incomplete quality control approach. To supplement electrical test, SMT operators nearly always implement some sort of visual inspection approach at the end of the assembly line (after reflow). One type of inspection is by human visual. Another, often more suitable approach, is in-line AOI (Automatic Optical Inspection) and sometimes X-Ray (automatic or manual) inspection. Among other things, such AOI process includes illuminating the component under test.
Referring to
Referring to
Referring now to
The remaining mirrors are deselected in the sense that no light is intentionally projected in their directions. One or more patterns can then be directed toward the selected mirrors so as to deliver, to surface 20, structured light as is required by the inspection technique, for example triangulation range finding. Light, structured or not, so delivered has a source incident angle determined by the physical arrangement of projector 38 and illumination mirrors 34a-34d. Thus, in the illustrated embodiment, the source incident angles can be selected from one of four angles, under computer control and without macroscopically moving parts.
International Patent Application Publication No. WO2008/124397, which is incorporated by reference, discloses a system wherein an LCD projector's entire field of projection (FOP) is divided into more than one portion by mirrors where each section is then further directed by those mirrors or additional mirrors to deliver light to a target surface from more than one source angle. The undesired angles are disabled when the portions of the projector's FOP corresponding to those angles are substantially dark. Conversely, desired angles are enabled when the portions corresponding to those angles are at least partially lit. Structured illumination is achieved by projecting a pattern into the enabled optical channel. Thus the Source Shadowing problem is mitigated without macroscopically moving parts.
In another disclosed system, a plurality of light generating subsystems, comprised of a light source and optics, is deployed at varying angles relative to a micro-mirror array such as a DLP. Additionally, a plurality of optical channels is deployed relative to the micro-mirror array such that there is a one-to-one correspondence between a light energy subsystem and an optical channel. Thus, when any one light source is energized, one and only one optical channel is illuminated. Each optical channel is disposed to deliver the light to a target surface from a source angle different from the others. As is standard for these devices, structure or intensity modulation can be imparted to the light by controlling the duty-cycle of the micro-mirror array over an exposure time that substantially exceeds the switching time of the micro-mirrors. Thus the Source Shadowing problem is mitigated without macroscopically moving parts.
Illumination mirrors 34a-34d are arranged on the surface of an oblate spheroid so that the convergence point 30 of chief rays 36 within projector 38 and the convergence point 20a of those rays on surface 20 are coincident with said oblate spheroid's foci. Thus, the optical path length between the two convergence points 30 and 20a is constant, regardless of which mirror is selected. Although the focal plane corresponding to each illumination mirror intersects convergence point 20a, no two such focal planes are parallel. In many inspection applications this lack of parallelism over the field of view will be inconsequential. This arrangement of mirrors 34 is optimal when the surface to be illuminated is substantially flat. For other illumination applications, where the surface topology is not nominally flat, mirrors 34 would be arranged differently as is optimal for that topology.
The arrangement depicted in
Refer now to
The viewing mirrors 54a-54e are arranged on the surface of an oblate spheroid so that the convergence point 50 of chief rays 56 within camera assembly 40 and the convergence point 20a of those rays on plane 20 are coincident with said oblate spheroid's foci. Thus, the optical path length between the two convergence points 50 and 20a is constant, regardless of which mirror is selected for viewing.
Although the focal plane corresponding to each viewing mirror intersects convergence point 20a, no two such focal planes are parallel. In many inspection applications this lack of parallelism over the field of view will be inconsequential. This arrangement of mirrors 54 is optimal when the surface to be viewed is substantially flat. For other viewing applications, where the surface topology is not nominally flat, mirrors 54 would be arranged differently as is optimal for that topology. Thus, all views are obtained simultaneously from five viewing angles without any moving parts.
This simultaneous ability to look from multiple viewing angles is achieved by sacrificing resolution; since the camera's field of view (FOV) must span all five mirrors, each viewing angle uses only a small portion camera's inherent resolution. Furthermore, the mirrors as illustrated are not adjacent to one another so additional resolution of the camera is wasted in the dead space where there are gaps. However, modern cameras have resolutions that exceed the needs of many inspection or measurement tasks and in those cases the loss of resolution is inconsequential.
Refer now to
Although
The illumination and viewing systems have so far been treated as though they are independent, yet in the embodiment of
Provide additional viewing angles
Allow for synthetic extension of the field of view
Allow for synthetic extension of the depth of field.
Also, the projector 38 is able to illuminate the target surface 20 not only through the illumination mirrors 34, but also through the viewing mirrors 54. Because the projector pupil 30 is not located at a focus of the viewing oblate spheroid (that focus is at the camera pupil 50), the light projected onto the target surface through viewing mirrors 54 will be laterally offset, tilted and out-of-plane. These displacements may be small compared to the application's requirements. Furthermore, these projections are potentially beneficial because they:
Provide additional projection angles
Allow for synthetic extension of the FOP
Allow for synthetic extension of the projector's depth of field.
Referring now to
However, mirror 54f is displaced so that the path length from pupil 50 to plane 20 is increased. Thus, camera 40's view through mirror 54f is focused above plane 20. Mirror 56g is displaced towards camera 40 and thus camera 40's view through mirror 56g is focused below plane 20. Camera 40's composite view of a surface near to plane 20 can be extended beyond what would be achievable without the displacements of mirrors 54f and 56g by selecting the data source, e.g. which mirror's images, should be emphasized depending on the target surface elevation.
Similarly, the illumination mirrors can be displaced to achieve the same effect for the projector.
Another embodiment of the illumination system is illustrated in
A corresponding embodiment of the viewing system is illustrated in
The illumination system of
The two mirror optical path has more degrees of freedom that the single mirror optical path illustrated in
In one embodiment the system of
The computer 2 causes projector 38 to select one of a plurality of possible source incident angles by illuminating one of the illumination mirrors 34 (60) as described above.
The computer 2 causes camera 40 to acquire a single image that encompasses all of a plurality of viewing mirrors 54 (70) as described above. The camera's view of the target surface 20 (100) through each of the viewing mirrors is from a distinct viewing angle. The resulting data are transferred from the camera to the computer. Note that computer 2 is omitted from
The processes described above are optionally repeated while varying the nature of the projected light. For example, in the case of phase profilometry, the light is structured and the phase of the structured light would be shifted between image acquisitions.
These processes are optionally further repeated for other source incident angles.
The data received by the computer from the camera are analyzed to produce inspection results. This analysis step need not be deferred entirely until all data are acquired. For example, in phase profilometry the images resulting from each source angle's illumination(s) (as in 1.3 above) can be processed to result in a plurality of height maps, one for each viewing angle. If multiple source angles are used, as in step 1.4 above, even more height maps will result. Once all height maps are available from all viewing and source incident angles, they can be combined.
The ability of the systems hereinabove described to generate a plurality of source incident angles mitigates the likelihood of Source Shadowing. If Source Shadowing is nevertheless present, this plurality of source incident angles mitigates its extent.
The ability of the systems hereinabove described to use a plurality of viewing angles mitigates the likelihood of Viewer Shadowing. If Viewer Shadowing is nevertheless present, this plurality of viewing angles mitigates its extent.
The ability of the system to illuminate a target surface from different source angles and to view that target surface from multiple observation angles improves the accuracy and repeatability of measurements of portions of the surface where the surface is visible from more than one viewing angle and/or from more than one source angle. This improvement in measurement fidelity is available, in its most basic form, by averaging the several available results. Additionally, as is the case with phase profilometry, a quality score of the measurement is often available along with the measurement itself, and this can be used to weight each of the several results accordingly.
Operation as described above is reliable, because it is done without macroscopically moving parts.
Operation as described above is fast because no macroscopically moving parts are needed to vary the source incident angle and because data from multiple viewing angles is acquired simultaneously.
The system as described above is comparatively inexpensive, compact and of low weight because it does not require multiple light projectors, multiple cameras, multiple lenses, duplicate electronics or macroscopically moving parts.
In another embodiment the system of
The computer causes the projector to select more than one illumination mirror 34 (60) at one time. Light from the two or more simultaneously selected paths reaches the target surface 20 (100) and is seen by the viewer as described above. Data from the simultaneously enabled source projection angles may need to be separated before subsequent processing. If required, this separation can be achieved, for example, by:
Color encoding the source light, e.g.: by using a color capable projector 38 and by illuminating one illumination path with blue light and a second illumination path with red light and
Using a color camera 40 where, continuing with the above example, data from the camera's blue and red pixels are separated knowing which color light came from which source illumination path.
Using multiple projection wave numbers where the wave numbers are chosen so that the height reconstruction resulting from one source projection angle does not effect the height reconstruction of another.
Delivering light to more than one illumination mirror at one time offers a speed improvement over using one illumination mirror at a time.
In yet another embodiment, the computer causes the projector to deliver unstructured light to all possible source incident angles.
When all illumination sources are concurrently selected and when no pattern is imposed, the aggregate light becomes less directional and can, if there are enough angled sources, approximate a diffuse light source. Diffuse lighting is advantages for some inspection tasks, for example, fiducial finding.
Commonly available stereo vision range finding techniques can be employed to process data acquired from multiple angles can thus yield 3D data of the target surface.
Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and/or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the present invention. This application is intended to cover any adaptations or variations of the specific embodiments discussed herein.
This Utility Patent Application is a non-provisional application of U.S. Provisional Application Ser. No. 61/157,020, filed Mar. 3, 2009, which is incorporated herein by reference.
Number | Date | Country | |
---|---|---|---|
61157020 | Mar 2009 | US |