1. Field
Imaging systems, such as optical microscopes, can benefit from automatic focus enhancements including sample detection. For example, systems that use a charge coupled device (CCD) video camera to capture a field of view for making focus determinations can benefit from automatic focus and sample detection.
2. Description of the Related Art
Some microscopes include a hardware component that analyzes a charge coupled device (CCD) camera image to measure the sample focus quality. In the CCD camera image, only a portion of the field of view (FOV) is used by the hardware component to measure the focus quality. This is called the Active Focus Area (AFA). The hardware component analyzes the sample in the AFA and adjusts the focus for best image. The CCD camera then captures the focused image for this FOV. However, conventionally it sometimes happens that the sample being observed under the microscope does not fall within the AFA, which can lead to the focus not being appropriate for the sample.
A method according to certain embodiments includes obtaining, by a machine, a high level image of a sample. The method also includes determining, by the machine, whether a plurality of auto-focus areas of a plurality of fields of view are aligned with a portion of the sample. The method further includes obtaining, by the machine, a low level image of the sample when the plurality of auto-focus areas are aligned.
In certain embodiments, an apparatus includes a high level image source configured to provide a high level image of a sample. The apparatus also includes a scan position determination section configured to determine whether a plurality of auto-focus areas of a plurality of fields of view are aligned with a portion of the sample. The apparatus further includes an image capture section configured to obtain a low level image of the sample when the plurality of auto-focus areas are aligned.
An apparatus, in certain embodiments, includes high level means for obtaining a high level image of a sample. The apparatus also includes determining means for determining whether a plurality of auto-focus areas of a plurality of fields of view are aligned with a portion of the sample. The apparatus further includes low level means for obtaining a low level image of the sample when the plurality of auto-focus areas are aligned.
For proper understanding of the invention, reference should be made to the accompanying drawings, wherein:
Imaging devices, such as optical microscopes, can use a charge coupled device (CCD) based video camera to acquire an electronic image of, for example, a sample under observation. Although this discussion uses a CCD camera as an example of a type of camera that can be used, other camera types can also be used in connection with certain embodiments.
As shown in
The microscope can include a particular hardware component that analyzes the CCD camera image to measure the sample focus quality. In the CCD camera image, only a portion of the FOV is used by the hardware component to measure the focus quality. This are that is used to measure the focus quality can be referred to as the active focus area (AFA). The hardware component can analyze the sample in the AFA and then adjust the focus to produce an image with an appropriate focus. The CCD camera can then capture the focused image for this FOV. As noted above, however, if the sample being observed under the microscope does not fall within the AFA, the focus measurement can be inappropriate for the sample, resulting in an out of focus image being acquired for that FOV.
In
Samples can be mounted on glass slides in order to be observed and imaged under a microscope. When the sample being observed under the microscope is larger than a single FOV, then multiple images, namely multiple FOVs, can be acquired and combined in order to obtain a single microscopic image of the entire sample.
In some fields of view, the sample may exist in the AFA. However, in the fields of view at (X2, Y2), (X4, Y1), (X3, Y3), (X1, Y4), and (X5, Y4) the sample make not exist within the AFA, even though the sample exists within the field of view.
The shift unit can be defined by an absolute amount or by, for example, the percentage of the shifted site area that is not part of the original site. Thus, in certain embodiments, values can be in a range from about 10% to about 90%.
Likewise, in
As noted above, in
The amount of shift of images can be determined based on a dynamic analysis of the data. The implementation can calculate a best focus position in the site surrounding the original FOV and then can make the decision on the shift.
Due to the complexity of the sample, there may be multiple combinations based on shift sizes.
If horizontal scanning is not the primary scanning direction, then the system can begin by checking vertical shifts at 915. The system can then check whether the shifts are within the maximum permitted shift values at 916, and terminate if they are. Otherwise, the system can check horizontal shifts and then be done.
As shown
As shown
As shown
Thus, in view of the above, certain embodiments can use information from a relatively low resolution image or GSI, which may be referred to as a high level image, to extract focus information for each FOV at a higher magnification, which may be referred to as a low level image. Certain embodiments can also make the scanning sites position based on the focus applicability. Moreover, certain embodiments compose one image from two or more captured images and can improve image quality.
In the above virtual microscopy, one of the initial steps can be to obtain a digital representation of the slide, the low resolution image, which also referred to as the high level image, because it may provide a broad overview. By analyzing the low resolution image, the system can identify regions of interest on the slide. Those regions of interest on the slide can be the area on the slide where a sample is present. In this discussion, the sample can refer to a biological sample, such as a tissue sample, but could also be another kind of sample, such as microprocessor chip. The regions of interest on the slide can enable the tool to scan in high magnification, which can be referred to as a low level image, only areas with sample present, thereby reducing the scanning time and storage resource usage. For example, referring to
In certain embodiments, a dedicated CCD equipped with appropriate optics can acquire the whole slide area in one image, which can be referred to as the high level image or macro image. In the system setting procedure, a blank slide can be loaded and its image, for example, a high level image, can be stored for later use. While or before scanning, the slide can be loaded to the macro image station and its image can be acquired.
The system can use the comparison between the current slide macro image and the blank slide macro image to detect the sample.
As shown in
A region of interest (ROI) for the detection area can be defined in the system setting. For example, in the illustration shown in
In
At 1102, the working detection buffer, DIFF buffer, can be created by performing the following. For each pixel (i,j), the system take the maximum value between the absolute value of: R(i,j) buffer in the sample minus R(i,j) buffer in the blank slide; G(i,j) buffer in the sample minus G(i,j) buffer in the blank slide; and B(i,j) buffer in the sample minus B(i,j) buffer in the blank slide. The system can then write this value into the DIFF buffer.
At 1103, the system can go over the DIFF buffer to find the maximum contrast, which can be referred to as the high threshold.
A high contrast sample can be handled differently from a low contrast sample. At 1104, a DIFF buffer can be sent to a high contrast method, at 1105, or to a low contrast method, at 1106.
At 1111, a first step can be to generate blobs using the high threshold as defined in the tool configuration. Then, at 1112, if the number of blobs is greater than 2, the system can start to go over, for each blob at 1113, all blobs in the sample to verify if they meet the expand criteria, at 1114.
If a blob meets the expand criteria the system can expand it at 1115, otherwise, the system can move on to the next blob at 1113. For an expanded blog, the system can look for new threshold by examine thresholds, at 1116, and checking whether the threshold is good enough at 1117. Once the threshold is found to be good enough, the current threshold is updated at 1118 and the blob with its new threshold is written into the DIFF buffer at 1119.
Then, for each valley, at 1152, the system can check whether the number of blobs is equal to one, at 1153. If equal to one the assumption by the system can be that this threshold will give a good detection for this slide, the system can set this valley position as a threshold, at 1155, and can update the DIFF buffer at 1156. Otherwise, at 1154, if the Maximum Area of the blobs, divided by half the number of pixels above noise is greater than 0.25, a new threshold is set, at 1155 otherwise the system can continue to the next valley.
In the slide image acquisition, the system can make averaging of N images to improve the signal to noise ratio. N can be a tool configuration parameter. The averaging of N images can be used to provide, in effect, a reference slide that has a more typical experience of noise.
In run time before the sample detection the system can take, at 1230, 5×5 pixels around the selected location and, at 1240, can calculate the average in the Blank slide (BSa) and the sample slide (SSa). The number of pixels can be varied as desired. For example, if desired a 10×10 pixel selection can be used instead. The normalization can be done by multiplying, at 1250, each pixel in the Blank Slide by the factor SSa/BSa.
Certain embodiments, therefore, can provide a delta calculation for comparing sample slide image with blank slide image. Moreover, certain embodiments can provide normalization in between a sample image and a blank image. Certain embodiments can provide for acquiring multiple macro images and doing an averaging for reducing noise. The multiple macro images can be obtained at periodic intervals or around the same time.
The system 1300 can also include an image capture section 1350, which can include a low level, high resolution image capture device, such as a CCD camera, or the controls for operating such a device, if the camera is external to the imaging system 1300. The imaging system 1300 can also include a threshold adjustment section 1360. The threshold adjustment section 1360 can be configured to control which in which the imaging system 1300 determines the location of a sample within a slide, and can be used as an input to the scan position determination section 1330.
The various sections of the imaging system 1300 are shown connected by a physical bus. Other kinds of interconnections are also permitted. It is permitted to divide up the imaging system 1300 into multiple physical sections that are separate from one another, although the various components are shown together. The various sections can be implemented in hardware or in software and hardware combined.
One having ordinary skill in the art will readily understand that the invention as discussed above may be practiced with steps in a different order, and/or with hardware elements in configurations which are different than those which are disclosed. Therefore, although the invention has been described based upon these preferred embodiments, it would be apparent to those of skill in the art that certain modifications, variations, and alternative constructions would be apparent, while remaining within the spirit and scope of the invention. In order to determine the metes and bounds of the invention, therefore, reference should be made to the appended claims.
AFA Active Focus Area
Active Focus Area can refer to the area within a camera FOV where the autofocus hardware will evaluate the image information for focus measurement.
Blob can refer to a group/region of pixels in an image.
Cover Glass or Cover Slip can refer to a thin flat piece of transparent material; cover slips can be made of glass and put on a slide with mounting medium.
CCD Charge Coupled Device
FOV Field Of View
GSI Global Slide Image
Global Slide Image can refer to an image that a CCD video camera with appropriate optics is capable of acquiring a full slide in one image.
MB Mega Bytes
Slide can refer to a thin flat piece of glass used to hold objects (samples) for examination under a microscope.
Strip can refer to a slice of the camera image near the image border that is used for stitching to the neighboring image.
This application is related to, claims the priority of, and incorporates by reference herein the entirety of U.S. Provisional Patent No. 61/539,898, filed Sep. 27, 2011.
Number | Date | Country | |
---|---|---|---|
61539898 | Sep 2011 | US |