Dual-axis scanning system and method

FIELD OF THE INVENTION

[0002] Aspects of the present invention relate generally to image acquisition systems and image processing techniques, and more particularly to an efficient system and method of capturing image data along a non-orthogonal optical axis.

DESCRIPTION OF THE RELATED ART

[0003] Digital imaging techniques are typically employed to facilitate identification and localization of objects in three-dimensional (3D) space; various techniques have utility in numerous applications and are used in conjunction with, for example, machine vision apparatus, manufacturing and fabrication systems, and inspection tools. Traditional image acquisition and processing methods such as deconvolution and tomography, for example, are computationally expensive and generally difficult to implement; conventional optical methods such as confocal microscopy are expensive and slow.

[0004] A conventional 3D localization technique is illustrated in FIGS. 1 and 2. Referring now to FIG. 1A, a traditional system is generally represented by reference numeral 100. In accordance with standard operating principles of system 100, finding the location of an object 199 supported on a structure 111 in 3D space (i.e., represented by x, y, and z coordinates) is generally accomplished by moving a camera or other imaging device 110 to a given x,y coordinate to image a particular section of the object space. Imaging device 110 then acquires a series of images, each at a respective focal plane 113; for each successive image, the z coordinate of focal plane 113 is changed accordingly, either by selectively adjusting (with respect to the z axis) the location of structure 111 supporting object 199, by changing the focal depth of imaging device 110 or other components of the optical system to focus into or away from the previous focal plane 113, or some combination of the foregoing.

[0005] In accordance with such an imaging strategy, both the z axis and the optical axis are orthogonal to the x and y axes. As indicated in FIG. 1B, a resulting image 171 generally captures data in a plane which is parallel to the x,y plane. A series of images 171-175 representing successive focal planes in the z direction is illustrated in FIG. 2A; such a series of images is generally referred to as a “stack” of images, and is designated by reference numeral 201.

[0006] Upon completion of image stack 201 at a given x,y coordinate (stack 201 is indicated at location x1,y1 in FIG. 2B), one or both of the x and y locations are changed or incremented, and a new stack of images is acquired. As illustrated in FIG. 2B, for example, stack 202 may be acquired at location x2,y1; imaging device may then be moved to location x2,y2, where stack 203 is acquired; and so forth. The x and y coordinates are incremented or adjusted successively until the entire object space has been sampled. Though stacks 201-204 are represented as slightly separated in FIG. 2B for clarity, it will be appreciated that system 100 is generally configured to acquire stacks 201-204 that abut each adjacent stack such that the entire object space is imaged without gaps.

[0007] Conventional systems such as those illustrated and described with reference to FIGS. 1 and 2 are difficult to implement, require complicated instrumentation and very precise motion control systems, and generally capture more images than necessary; accordingly, such systems are inefficient and unnecessarily expensive to design and operate.

SUMMARY

[0008] Embodiments of the present invention overcome the above-mentioned and various other shortcomings of conventional technology, providing an image acquisition system and method employing a non-orthogonal optical axis. As set forth in detail below, a system and method of dual-axis scanning allow rapid determination of object locations in three-dimensional space without excessive costs.

[0009] In accordance with one embodiment, for example, a method of imaging an object space comprises: supporting an object to be imaged within the object space; and acquiring image data at a focal plane positioned in a selected orientation relative to the object space along a non-orthogonal optical axis.

[0010] The supporting may comprise utilizing a structure movable along any of three axes defining the object space; additionally or alternatively, the structure may be rotatable about any of the three axes. Accordingly, the acquiring comprises selectively orienting the structure relative to an imaging device to establish the non-orthogonal optical axis. In some embodiments, the acquiring comprises selectively orienting an imagining device relative to the object space to establish the non-orthogonal optical axis. The acquiring may further comprise selectively translating the focal plane through the object space. Such selectively translating may comprise utilizing a raster scan strategy or a serpentine scan strategy, for example.

[0011] As set forth in detail below with reference to specific exemplary embodiments, the acquiring may comprise selectively positioning the focal plane to extend throughout an entire depth dimension of the object space. Efficient imaging of the entire object space may be facilitated by combining two coordinate axes. A method of imaging an object space may further comprise selectively repeating the acquiring operation.

[0012] In accordance with some embodiments of a dual-axis scanning system operative to obtain image data representing an object space, the system comprises: a structure operative to support an object to be imaged within the object space; and an imaging device selectively oriented to position a focal plane in a selected orientation relative to the object space along a non-orthogonal optical axis; the imaging device is generally operative to acquire image data at the focal plane.

[0013] As set forth in detail below, at least one of the structure and the imaging device is translatable along any of three axes defining the object space; additionally or alternatively, at least one of the structure and the imaging device is rotatable about any of the three axes. Accordingly, relative movement of the structure and the imaging device may cooperate to establish the non-orthogonal optical axis.

[0014] In some exemplary implementations, the system further comprises a control element such as a microprocessor or microcontroller, for example, operative to control the relative movement of the structure and the imaging device.

[0015] The system may further comprise an image processor operative to receive acquired image data from the imaging device. Such an image processor may be operative to normalize the image data, for example, or to synthesize image stacks from the image data.

[0016] As with the efficient method embodiments noted above, the system may be configured wherein the focal plane extends throughout an entire depth dimension of the object space.

[0017] The structure operative to support an object to be imaged within the object space may comprise a semiconductor wafer, a microscope slide, or a multi-well plate, for example.

[0018] The foregoing and other aspects of various embodiments of the present invention will be apparent through examination of the following detailed description thereof in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019]
FIG. 1A is a simplified functional block diagram illustrating one embodiment of a conventional image acquisition system.

[0020]
FIG. 1B is a simplified block diagram illustrating an image captured by the system of FIG. 1A.

[0021]
FIGS. 2A and 2B are simplified block diagrams illustrating image stacks acquired by conventional imaging systems.

[0022]
FIG. 3 is a simplified functional block diagram illustrating one embodiment of a dual-axis image acquisition system.

[0023]
FIG. 4 is a simplified diagram illustrating one embodiment of a dual-axis scan.

[0024]
FIG. 5A is a simplified representation of successive image frames captured by the dual-axis scan of FIG. 4.

[0025]
FIG. 5B is a simplified representation of successive layers of an image frame stack synthesized from data captured by the dual-axis scan of FIG. 4.

[0026]
FIG. 6 is a simplified flow diagram illustrating the general operation of one embodiment of a dual-axis scanning method.

DETAILED DESCRIPTION

[0027] As set forth above, FIG. 1A is a simplified functional block diagram illustrating one embodiment of a conventional image acquisition system, and FIG. 1B is a simplified block diagram illustrating an image captured by the system of FIG. 1A. The goal of such systems is to generate stacks of images; each stack is composed of a series of images acquired at successive z locations in the object space. In that regard, FIGS. 2A and 2B are simplified block diagrams illustrating image stacks acquired by conventional imaging systems.

[0028] To acquire the data illustrated in FIG. 2B, a conventional system must collect twenty discrete images (ie., five images 171-175 for each of four stacks 201-204). Assuming video rate acquisition performance and similar component movement rates and settling time, then the time required to acquire these images may be estimated by the following equation:

Time=(Total Frames)×(Exposure+Movement+Settling)

[0029] For twenty frames, this equation yields a time of approximately 2.0 seconds for a conventional system operative in accordance with the parameters noted above. If the object space to be imaged is very large (e.g., requiring hundreds of total frames), then the time required to capture data for the entire space can be substantial.

[0030]
FIG. 3 is a simplified functional block diagram illustrating one embodiment of a dual-axis image acquisition system. In the exemplary embodiment, system 300 generally comprises an imaging device 310 operative to acquire image data of a three-dimensional (3D) object space 390, generally represented by the dashed lines. An object 399 to be imaged may be supported within object space 390, such as on a structure 311; in some embodiments, for example, structure 311 may be embodied as or comprise a microscope stage or microscopic slide, a microarray, a microtiter or other multi-well plate, a semiconductor chip, die, or wafer, or any other similar structure configured and operative to support objects, specimens, sample material, and the like for viewing or imaging.

[0031] Structure 311 may be disposed on a precision motion stage or other movable support element (not shown), and may be moved or translated by precise motion of the stage. As is generally known in the art, structure 311 may be movable in any or all of the x, y, and z directions; this movement may be accomplished through translation of structure 311 itself, through motion of any stage or other apparatus upon which structure 311 is disposed, or both. Accordingly, selective translation of structure 311 along one or more coordinate axes may allow precise positioning of object 399 within space 390. Additionally or alternatively, structure 311 may be rotatable about one or more coordinate axes. Numerous and varied apparatus and methods of providing controlled movement or translation of structure 311 are known and well within the capabilities of an ordinarily skilled artisan. The scope of the present disclosure is not intended to be limited by any structures and techniques employed to manipulate structure 311 and to position and orient object 399 within object space 390.

[0032] In some implementations, imaging device 310 may be embodied in or comprise a camera incorporating charge-coupled device (CCD) technology, for. example, or complementary metal oxide semiconductor (CMOS) image sensors. Additionally or alternatively, imaging device 310 may comprise supplementary optical elements or imaging components such as optical microscopes, scanning electron microscopes (SEM), spectrophotometers, or any other apparatus or instrument configured and operative in conjunction with image sensors or sensor arrays to acquire video or image data.

[0033] Imaging device 310 may additionally comprise or be coupled to one or more image processing components (such as image processor 381) operative to process, store, or otherwise to manipulate captured image data as desired. Image processor 381 may comprise one or more microprocessors or microcontrollers, for example, capable of executing software code or other instruction sets for interpolating, extrapolating, filtering, deconvolving, or otherwise manipulating image data captured by and transmitted from device 310. Image processor 381 may execute or run a real-time operating system, for example, enabling reconfiguration or selective programming of processor 381 functionality.

[0034] As is generally known in the art, some image processing techniques are processor intensive (i.e., computationally expensive) and require significant computing power and other resources for data manipulation and storage. Accordingly, image processor 381 may additionally comprise computer readable storage media such as: read-only memory (ROM); random access memory (RAM); hard or floppy disk drives; digital versatile disk (DVD) drives; or other magnetic, optical, or magneto-optical computer storage media and attendant hardware. Sufficient storage media may be provided to support the computational functionality of image processor 381, as well as to enable reconfiguration or selective programming thereof as noted above.

[0035] In a manner similar to that of structure 311, imaging device 310 may be movable in any or all of the x, y, and z directions; accordingly, selective movement or translation of device 310, or of one or more components thereof, along one or more coordinate axes may enable precise positioning of a focal plane 313 within object space 390. Various apparatus and methods of providing controlled movement of device 310 or providing accurate placement of focal plane 313 are generally known in the art., In that regard, device 310 may be operably coupled to guide rails or tracks, stepper motors, articulated arms, or other automated structures or robotic systems operative selectively to position device 310 for scanning operations.

[0036] Additionally, device 310, or one or more components thereof, may be rotatable about one or more of the x, y, and z coordinate axes. In that regard, device 310 may be operably coupled to or mounted on appropriate hardware such as hinges, gimbals, journal and bearing assemblies, or other pivotable structures capable of selectively orienting, supporting, and maintaining device 310 at a predetermined or dynamically adjustable angle relative to the coordinate axes. In some embodiments, selective or dynamic rotation of device 310 about one or more axes may not be necessary; in such embodiments, device 310 may be fixed at a particular angular orientation to support the functionality set forth below.

[0037] The scope of the present disclosure is not intended to be limited by any structures and techniques employed to manipulate device 310 and to position focal plane 313 within object space 390. Where structure 311 has sufficient range of motion (i.e., to position all portions of object space 390 in a location relative to device 310 suitable for imaging), for example, movement of device 310 may not be required. Alternatively, structure 311 and device 310 may be moved in a cooperating manner to enable imaging of the entirety of object space 390.

[0038] In that regard, system 300 may further comprise one or more microprocessors, microcontrollers, or other electronic devices (control electronics 382) operative to control relative movement, placement, and orientation of device 310 and structure 311. In the exemplary embodiment of FIG. 3, electronics 382 is illustrated as operably coupled to image processor 381. In some such embodiments, image processor 381 may initiate, execute, or terminate scanning or image capture operations, for example, responsive to control signals or other data (e.g., indicative of placement or relative movement of device 310 and structure 311) received from electronics 382. Similarly, control electronics 382 may receive data or instructions sets (e.g., relating to desired movements or the timing thereof) from image processor 381, and may arrange or orient device 310 and structure 311 accordingly. It will be appreciated that the functionality of image processor 381 and control electronics 382 may be combined, incorporated, or integrated into a single device or hardware arrangement.

[0039] In the exemplary embodiment, the z axis is orthogonal to the x and y axes, but the optical axis is non-orthogonal to the x,y plane, i.e., the optical axis may be selectively oriented at a first angle, α, relative to the x axis (FIG. 4) and at a second angle, β, relative to the y axis (FIG. 3). Accordingly, the image planes (acquired at focal planes such as 313) may be selectively positioned at a desired angle relative to the lateral axes (x and y) of structure 311, i.e., at a selected orientation relative to object space 390.

[0040] It is noted that first angle α and second angle β may be independently adjustable, and may be selected in accordance with any of various factors including, but not limited to: movement, placement, and precision limitations with respect to motion control of device 310 and structure 311; surface features or general configuration of structure 311; the dimensions of object space 390; the type and direction of scanning motion requested or required by image processor 381; optical capabilities of imaging device 310; and so forth. In some embodiments of system 300, second angle β may be fixed at 90° in such simplified arrangements, the optical axis may be considered non-orthogonal to the x axis only.

[0041] In this context, the term “non-orthogonal” generally refers to the quality of being selectively oriented at some angle other than 90° relative to at least one of two axes defining a plane or surface representing one boundary of the object space, i.e., a non-orthogonal optical axis is not perpendicular to a plane defining the object space. For example, in the FIG. 3 embodiment, the x and y axes define a plane representing a surface of structure 311, which in turn functions as one boundary of object space 390. The optical axis illustrated in FIG. 3 may be considered “non-orthogonal” to the extent that it is not perpendicular to the x axis, the y axis, or both (i.e., it is not perpendicular to the x,y plane).

[0042] Referring now to FIGS. 3 and 4, it is noted that FIG. 4 is a simplified diagram illustrating one embodiment of a dual-axis scan. As set forth above, first angle α and second angle β may be adjusted, varied, or selected independently, and second angle β may be fixed at 90° in some applications. The exemplary embodiment illustrated in FIG. 4 is operative with second angle β not equal to 90°, though the scope and contemplation of the present disclosure are not so limited.

[0043] As described above, the angle of collection (i.e., the angle of optical axis relative to the x,y plane) may be selectively varied in accordance with numerous factors, one or more of which may be related to the optical capabilities or operational characteristics of imaging device 310. In that regard, first angle α in FIG. 4 may be determined as a function of, inter alia, the ratio of the axial optical resolution (i.e., along the optical axis) to the lateral optical resolution (i.e., in focal plane 313) of the optics employed at device 310 or any additional optical components used in conjunction therewith. For instance, given a 40X/0.90 NA (numerical aperture) lens, the foregoing ratio may suggest a value for first angle α of about 33°; as noted above, other considerations may affect selection of a value for first angle α.

[0044] As indicated in FIG. 4, object space 390 may be scanned (e.g., from right to left in the drawing figure) using a non-orthogonal optical axis; accordingly, focal plane 313 extends in the z direction through a plurality of image layers 471-475 which generally correspond to those described above in detail with reference to FIGS. 1 and 2. Focal plane 313 also extends along the y axis, which is normal to the plane of FIG. 4. In particular, while focal plane 313 may generally be oriented to acquire x,y data, those data are acquired simultaneously across a plurality of layers in the z direction.

[0045] Scanning as indicated in FIG. 4 enables a dual-axis scanning system and method to integrate the x and z dimensions into a single axis (i.e., a combined or “dual” axis). In that regard, appropriate angular orientation of focal plane 313 may enable acquisition of data across the entire z dimension of object space 390 while simultaneously avoiding the need for component movement or translation (and attendant settling time) along the z axis, refocusing of optical elements at successive z locations, and the like. Specifically, focal plane 313 may be positioned to extend throughout an entire depth dimension (in this case, the z direction) of object space 390; in this context, the term “depth” generally refers to a direction which is substantially perpendicular to the scan direction. In accordance with dual-axis scanning methodologies, translating focal plane 313 in a particular scan direction also enables acquisition of image data in the depth direction.

[0046] Specifically, combining the x and z dimensions into a dual-axis may generally limit the motion requirements for various system components (e.g., structure 311 and device 310) to two dimensions; elimination of motion in one out of the three axial directions may reduce movement and positioning overhead by as much as 30% (or more in some applications). Additionally, since scanning of both the x and z axes occurs simultaneously, imaging operations on individual stacks of images (201-204 in FIG. 2B) are not required, enabling substantially more efficient relative motion of structure 311 and device 310 as compared with conventional systems; accordingly, as noted above, movement and positioning savings may be even greater than 30%, though only one axis of motion is eliminated.

[0047] For example, an entire row (representing the entire x range of object space 390, for example, or a portion thereof) of dual-axis images may be collected in a single continuous motion. In the FIG. 4 embodiment, for instance, the row of dual-axis images may provide image data for the portion of object space 390 equivalent to the combination of stacks 201 and 202 in FIG. 2B; image data may be acquired in a single pass or sweep in the x direction across space 390 as opposed to ten discrete motions (five images for two stacks) and attendant setting times, refocusing operations, and the like.

[0048] Upon completion of a dual-axis row such as shown in FIG. 4, the y coordinate may be incremented, decremented, or otherwise adjusted; it will be appreciated that scanning may proceed in accordance with various strategies. In some embodiments, for example, either a raster scan or serpentine scan may be implemented. As each frame of image data is collected, captured image data may be written to computer memory or to computer readable media resident, for example, at imaging device 310, image processor 381, or both. Traditional vertical stacks of images (such as represented by reference numerals 201-204 in FIG. 2B) may be assembled or synthesized from the dual-axis data by software, firmware, or other instruction sets at image processor 381. Lateral resolution may be limited by, among other factors, the motion precision of structure 311, device 310, or the combination thereof. To compensate for resolution limitations, each synthesized stack may be aligned to neighboring stacks using any one of various alignment techniques or algorithms (such as cross-correlation, for instance).

[0049]
FIG. 5A is a simplified representation of successive image frames captured by the dual-axis scan of FIG. 4. In FIG. 5, the coordinates x′ and y′ are used to indicate that focal plane 313 is skewed (by first angle α and second angle β) as it is translated in the x direction; this skew is a function of the non-orthogonal nature of the optical axis. The apparent movement of object 399 from right to left across successive frames may be influenced by the magnitude of second angle β between the optical axis and the y axis (where 0<β≦90°). For small values of second angle β, the relative motion of object 399 from frame to frame will be comparatively greater than for large values of second angle β; in the case where β=90°, for instance, no lateral motion of object 399 will be detected.

[0050]
FIG. 5B is a simplified representation of successive layers of an image frame stack synthesized from data captured by the dual-axis scan of FIG. 4. Blurring may occur when object 399 does not reside in focal plane 313; such blurring may be caused by depth of field (DOF) limitations and other optical characteristics of imaging device 310.

[0051] Employing the formula set forth above to estimate scan time for a dual-axis scanning system and method, it is noted that the motion time (between frames) and the settling time factors are reduced to zero, or are minimized to such an extent that they are negligible. Accordingly, the time required to acquire an equivalent amount of image data may be estimated by the following equation:

Time=(Total Frames)×(Exposure)

[0052] It will be appreciated that the data acquisition rate is affected mostly by the exposure time requirements of the imaging device employed and by the speed with which the object space can be traversed. The net result of employing dual-axis scanning methodologies may be a reduction in the scan time by as much as 66% for equivalent data sets.

[0053] In some cases, particular features of interest (e.g., object 399) to be imaged may be on top of a non-planar support or structure 311, such as, for example, in many applications associated with semiconductor wafer manufacturing, processing, inspection, or testing. In such situations, an alignment algorithm used to synthesize the stacks of images may also construct those images relative to a reference surface on the support. For example, in semiconductor inspection, it is often desirable to measure the heights of features (“bumps”) disposed on the surface of the silicon wafer or die. The features may be measured relative to this surface even though the typical silicon wafer is not necessarily planar. In conjunction with synthesizing stacks from dual-scan image data, an algorithm may normalize the z positions of imaged features relative to height of the wafer in the stack (i.e., at that particular x,y coordinate location).

[0054]
FIG. 6 is a simplified flow diagram illustrating the general operation of one embodiment of a dual-axis scanning method. As indicated in FIG. 6, a object to be imaged may be supported on a structure within the object space (block 601). The structure may be oriented or arranged to facilitate imaging as set forth in detail above. In that regard, a structure supporting the object may be movable or translatable along any of three axes defining the object space; additionally, the structure may be rotatable about any of those axes.

[0055] As indicated at block 602, an imaging device may be oriented relative to the object space in general, or the support structure in particular, to establish a non-orthogonal optical axis. In some embodiments, both the support structure and the imaging device may be moved relative to each other, facilitating establishment of a desired, optimal, or other predetermined angle for the non-orthogonal optical axis.

[0056] Specifically, the optical axis may be established in such a manner as to position a focal plane of the imaging device in a selected orientation relative to the object space (block 603). Such selective positioning and orientation of the focal plane allows the integration of two coordinate axes and enables dual-axis scanning of the object space. In the embodiment described above in detail with reference to FIGS. 3-5, for example, the focal plane may extend throughout the entire depth dimension of the object space.

[0057] Image data may be acquired as indicated at block 604. Data acquisition may be facilitated or enabled by a CCD or CMOS camera, for example, embodied or integrated in the imaging device. As noted above, image data are acquired at the focal plane, the angular orientation of which is determined by the non-orthogonal optical axis.

[0058] In some embodiments, the entirety of the object space may be scanned such that image data are acquired for all portions thereof. Depending upon, for example, the dimensions of the object space, the lateral resolution of the imaging device, the size of each acquired image frame, and other optical characteristics or system parameters, more than one scan or pass may be required to image the entire object space. In that regard, a determination may be made as indicated at decision block 605. Where another scan is not required (e.g., the entire object space has already been imaged, or a particular object of interest has been identified and located) as determined at decision block 605, the imaging process may end (block 699).

[0059] Where another scan is required as determined at decision block 605, however, the imaging operation may loop back to block 603. In that regard, control may pass to block 606, which represents movement of the imaging device, the supporting structure, or both, i.e., these components may be repositioned in preparation for subsequent scanning. Specifically, the focal plane may be repositioned (block 603) at a new scan location, and imaging may continue.

[0060] As set forth in detail above, some or all the foregoing movements, orientations, acquisitions, and determinations may be accomplished under control of, or may be influenced by, software code or computer executable instructions resident at an image processor, a control element or device, or both. Various methods and apparatus for motion control and precise relative positioning of structural components are generally known in the art of image processing.

[0061] The FIG. 6 embodiment is presented for illustrative purposes only, and is not intended to imply an order of operations to the exclusion of other possibilities. By way of specific example, the operations depicted at blocks 602 and 603 may be reversed in order, for instance, or combined to occur substantially simultaneously; in accordance with one such alternative, an optimum or selected angular orientation of the focal plane may be determined prior to orientation of the imaging apparatus. Those of skill in the art will appreciate that the particular sequence in which the operations depicted in FIG. 6 are conducted may be influenced by, among other factors, the functionality and structural configuration of a particular imaging device or image processor, the operational characteristics and limitations of the motion control element or any movable structures associated therewith, or some combination thereof.

[0062] The present invention has been illustrated and described in detail with reference to particular embodiments by way of example only, and not by way of limitation. Those of skill in the art will appreciate that various modifications to the exemplary embodiments are within the scope and contemplation of the present disclosure. Accordingly, it is intended that the present invention be limited only by the scope of the appended claims.

Dual-axis scanning system and method

Information

Publication Number

Date Filed

Date Published

Inventors

CPC

US Classifications

International Classifications

Abstract

Description

Claims

Parent Case Info

Provisional Applications (1)