The present invention relates to an automatic focusing device for controlling a focal point when acquiring an image of a sample and a microscope apparatus using the automatic focusing device.
When an image of a sample is acquired by use of a microscope, out-of-focus owing to a tilt of the optical system or mechanical system in an apparatus or a tilt or an uneven shape of the sample itself becomes a problem. To cope therewith, conventionally, automatic focusing (autofocus) has been carried out in a microscope apparatus for automatically controlling a focal point of imaging by an image pickup device such as a CCD camera. A device that carries out such automatic focusing includes, for example, a device described in Patent Document 1.
Patent Document 1: Japanese Patent Application Laid-Open No. H05-288981
In the above-described Document 1, disclosed is a method for acquiring a sample image at every fixed changing interval while continuously changing a focal position between an objective lens of a microscope and a sample to be an observation target of the microscope and determining an in-focus position based on a plurality of acquired images. However, in such a method, since a device such as a stepping motor or a piezo actuator is used for driving respective portions of the microscope so as to change the focal position, a delay unique to the device occurs when continuously changing the focal position. Therefore, when an in-focus position is determined by a focal point measurement using the above-described method, there is a problem in occurrence of an error caused by a delay in a change of the focal position.
On the other hand, as a method for eliminating influence of such a delay in a change of the focal position, there is a method which can be used of stopping the device at every fixed changing interval when changing the focal position and repeating acquiring a sample image after the focal position has been stabilized. However, by this method, stabilization of the focal position must wait at every time of image acquisition when acquiring a plurality of images necessary for determining the in-focus position. Therefore, there is a problem such that a very long time is necessary for determining the in-focus position.
The present invention has been made to solve the above problems, and an object thereof is to provide an automatic focusing device that is capable of accurately determining an in-focus position in a short time and a microscope apparatus using the same.
In order to achieve this object, an automatic focusing device according to the present invention includes: (1) image pickup means that acquires an image by an optical image of a sample to be an observation target; (2) a light guiding optical system that includes an objective lens into which light from the sample is made incident and guides the optical image of the sample to the image pickup means; (3) focal point changing means that changes a focal position on the sample for the image pickup means and the light guiding optical system in an optical axis direction; and (4) focal point control means that acquires, by use of the image pickup means and the focal point changing means, focus information when acquiring an image of the sample, wherein (5) the focal point control means calculates a first focal point measurement value from a plurality of images of the sample acquired by a first focal point measurement that is executed while continuously changing the focal position in one direction by the focal point changing means, calculates a second focal point measurement value from a plurality of images of the sample acquired by a second focal point measurement that is executed while continuously changing the focal position in a direction opposite that of the first focal point measurement, and determines an in-focus position for the sample based on the first focal point measurement value and the second focal point measurement value.
In the above-described automatic focusing device, an in-focus position is determined by use of a focal point measurement method for acquiring a plurality of images of the sample while continuously changing the focal position, not by stopping a change in the focal position at every fixed changing interval. Thereby, it becomes possible to acquire focus information for an observation position of the sample in a short time. Furthermore, with regard to the focal point measurement for determining an in-focus position, two times of focal point measurement are carried out in different focal position changing directions, and an in-focus position is determined with reference to focal point measurement values calculated in each thereof. At this time, for the two times of focal point measurement, influence of a delay in a change of the focal position is in opposite directions. Consequently, by using the focal point measurement values each calculated from a plurality of images for the two times of focal point measurement, it becomes possible to accurately determine a final in-focus position.
A microscope apparatus according to the present invention includes the above-described automatic focusing device and image acquisition control means that controls acquisition of an image of the sample based on the focus information including the in-focus position determined for an observation position of the sample by the focal point control means.
In the above-described microscope apparatus, a microscope apparatus is formed with the above-described automatic focusing device that carries out two times of focal point measurement in different focal position changing directions and determines an in-focus position for an observation position of the sample with reference to focal point measurement values calculated in each thereof. Thereby, it becomes possible to accurately determine an in-focus position when acquiring an image of a sample in a short time so as to efficiently execute observation of the sample with a high accuracy.
According to the automatic focusing device and the microscope apparatus using the same of the present invention, with regard to the focal point measurement for determining an in-focus position, by acquiring a plurality of images of a sample while continuously changing the focal position, carrying out two times of focal point measurement in different focal position changing directions, and determining an in-focus position with reference to focal point measurement values calculated in each thereof, it becomes possible to accurately determine an in-focus position for an observation position of the sample in a short time.
[
[
[
[
[
[
[
[
1A—microscope apparatus, 10—sample stage, 15—stage driving section, 20—light guiding optical system, 21—objective lens, 25—optical system driving section, 30—CCD camera (image pickup device), 50—control device, 51—focal point control section, 52—image acquisition control section, 55—focus information storing section, 58—input device, 59—display device.
Hereinafter, preferred embodiments of an automatic focusing device and a microscope apparatus using the same according to the present invention will be described in detail with reference to the accompanying drawings. Here, in the description of the drawings, identical elements are designated with identical numerical symbols so as to avoid overlapping descriptions. Dimensional ratios of the drawings are not always coincident with those in the description.
The sample stage 10 is formed of an XY stage movable in the x-direction and y-direction (horizontal direction), and by driving this XY stage 10 in an xy plane, an observation position in the microscope apparatus 1A to the sample S is set or changed. In addition, the sample stage 10 is drive-controlled by an XY stage driving section 15. In this configuration, the XY stage driving section 15 functions as observation position changing means that changes an observation position on the sample S for the microscope apparatus 1A in the direction perpendicular to the optical axis.
For the sample S on the sample stage 10, a light guiding optical system 20 for guiding an optical image of the sample S is provided above the stage 10. This light guiding optical system 20 includes an objective lens 21 into which light from the sample S is made incident and optical components necessary for guiding and converging an optical image of the sample S. Moreover, at a predetermined position on an optical path along which an optical image of the sample S is guided by the light guiding optical system 20, a CCD camera 30 being an imaging device is installed. This CCD camera 30 is image pickup means that acquires an image by an optical image of a sample S. The light guiding optical system 20 and the CCD camera 30 are integrally fixed with their optical axes, a distance between the optical system 20 and the camera 30, and the like having been adjusted.
Moreover, for the light guiding optical system 20 and the CCD camera 30, an optical system driving section 25 is installed. The optical system driving section 25 is constructed with use of, for example, a stepping motor or a piezo actuator, and shifts the optical system 20 and the camera 30 fixed to the optical system 20 in the z-axis direction being the optical axis direction. In this configuration, the optical system driving section 25 functions as focal point changing means that changes the focal position on the sample S for the CCD camera 30 and the light guiding optical system 20 in the optical axis direction. This optical system driving section 45 is used for controlling a focal point in the acquisition of an image of the sample S.
For the sample stage 10, the light guiding optical system 20, and the CCD camera 30, a control device 50 having a focal point control section 51 and an image acquisition control section 52 is installed. The focal point control section 51 acquires focus information including an in-focus position concerning acquisition of an image of the sample S, by use of the CCD camera 30 and the optical system driving section 25.
In the microscope apparatus 1A shown in
The image acquisition control section 52 controls acquisition of an image of the sample S based on focus information including an in-focus position determined for the observation position on the sample S by the focal point control section 51 when actually executing image acquisition of the sample S by use of the CCD camera 30. In the configuration shown in
The control device 50 having the focal point control section 51 and the image acquisition control section 52 is composed of, for example, a computer including a CPU and necessary storage such as a memory and a hard disk. Moreover, to the control device 50, an input device 58 and a display device 59 are connected. The input device 58 is composed of, for example, a keyboard, a mouse, and the like connected to a computer and is used for input and the like of information and instructions necessary for executing a focus information acquiring operation by the focal point control section 51 in the present microscope apparatus 1A or an image acquiring operation by the image acquisition control section 52. Moreover, the display device 59 is composed of, for example, a CRT display, a liquid crystal display, or the like connected to a computer and is used for displaying necessary information with regard to focus information acquisition and image acquisition in the present microscope apparatus 1A.
Next, a focus information acquiring method in the automatic focusing device and the microscope apparatus 1A shown in
In the microscope apparatus 1A of the present embodiment, acquisition of focus information including an in-focus position concerning image acquisition of the sample S is carried out by the focal point control section 51 of the control device 50 as described above. The focal point control section 51 executes a focus information acquisition for image acquisition of the sample S by use of the CCD camera 30 being image pickup means, the optical system driving section 25 being focal point changing means, and, if necessary, the stage driving section 15 being observation position changing means.
The focal point control section 51 drives the CCD camera 30 and the light guiding optical system 20 in the z-axis direction being the optical axis direction by the optical system driving section 25 so as to continuously change the focal position on the sample S in one direction along the z-axis direction. And, during the continuous change of the focal position, it carries out a focal point measurement to acquire an image of the sample S at every predetermined changing interval and carries out an analysis for a plurality of acquired images, so as to calculate a focal point measurement value indicating an in-focus position where a focal point of image acquisition by the CCD camera 30 and the light guiding optical system 20 comes into focus.
A changing range of the focal position to execute a focal point measurement is appropriately set so that the in-focus position is included in the changing range. In addition, for calculation of the focal point measurement value using a plurality of images acquired by the CCD camera 30 throughout the set changing range of the focal position, used is, for example, a method for evaluating a change in image characteristics such as high-low in contrast and high-low in brightness between a plurality of images and calculating, based on the evaluation results, a position where the focal point of image acquisition is most in focus as a focal point measurement value.
As can be understood from this graph, when the focal position is continuously changed by the optical system driving section 25 using a device such as a stepping motor or a piezo actuator, a delay in a change shown by an arrow A6 occurs in an actual change in the focal position shown in graph A2 in relation to an instructed change in the focal position shown in graph A1. Therefore, when a focal point measurement value is calculated from a plurality of images acquired by the above-described focal point measurement, an error caused by a delay in a change of the focal position occurs in the calculated focal point measurement value.
In the automatic focusing device and the microscope apparatus 1A shown in
Specifically, the focal point control section 51 executes a first focal point measurement while continuously changing the focal position on the sample S in one direction along the z-axis direction. And, it calculates a first focal point measurement value from images of the sample S acquired at every predetermined changing interval during that time. At this time, the first focal point measurement value to be obtained has, as shown by an arrow B6 in
The focal point control section 51 further executes a second focal point measurement while continuously changing the focal position on the sample S in a direction opposite that for the first focal point measurement along the z-axis direction. And, it calculates a second focal point measurement value from images of the sample S acquired at every predetermined changing interval during that time. At this time, the second focal point measurement value to be obtained has, as shown by an arrow B7 in
Effects of the automatic focusing device and the microscope apparatus according to the above-described embodiment will be described.
In the automatic focusing device shown in
Furthermore, with regard to the focal point measurement executed while continuously changing the focal position as described above, two times of focal point measurement are carried out in different focal position changing directions in the optical axis direction, and an in-focus position is determined with reference to focal point measurement values calculated in each thereof. At this time, in the two times of focal point measurement, as shown by the arrows B6 and B7 in
Effects of the automatic focusing device and the microscope apparatus 1A will be described in detail. When a piezo element or the like is used for the optical system driving section 25, a delay in a focal position changing operation due to the same is on the order of a few tens to a hundred ms (milliseconds) when it is large. Whereas, when a frame rate to load images by the CCD camera 30 is provided as 30 Hz and acquisition of an image of the sample S is carried out while continuously changing the focal position, with a delay in the changing operation of 100 ms, the focal position instructed by the focal point control section 51 and the actual focal position set by the optical system driving section 25 are shifted from each other by three images in terms of the number of images. Such an image shift further increases when an image pickup device having a high frame rate is used. By the above-described device configuration and focus information acquiring method, by carrying out a two-directional focal point measurement, an influence of such an image shift is eliminated.
In addition, by using such a focus information acquiring method, acquisition of focus information can be speedily carried out. For example, it is assumed to carry out a focal point measurement by stopping a change in the focal position on the sample S at every fixed changing interval. In this case, where a delay in the changing operation is provided as 100 ms, and a time until the focal position that has been changed is stabilized, as 300 ms, which is three times as long as the delay, and a frame rate of image acquisition, as 30 Hz (image acquiring time 1/30 s), and the number of acquired images used for a focal point measurement, as 30, a time required for the focal point measurement is (image acquiring time+focal position stabilizing time)×number of acquired images=(1/30+0.3)×30=10 seconds.
On the other hand, in the above-described method for carrying out a two-directional focal point measurement while continuously changing the focal position, where a frame rate of image acquisition is similarly provided as 30 Hz, and the number of acquired images used for a focal point measurement, as 30 in one direction, a time required for the two-directional focal point measurement is 2 seconds. As such, by the automatic focusing device and microscope apparatus with the above-described configuration, it is possible to considerably reduce the time required for acquiring focus information.
Here, the delay in a change of the focal position in a focal point measurement and the error in the focal point measurement value due to influence thereof change in size depending on a focal position changing speed.
As can be understood from these graphs, a delay in a change of the focal position between graphs D1 and D2 when a slow changing speed is set is smaller than a delay in a change between graphs C1 and C2 when a fast changing speed is set. Namely, the delay in a change of the focal position in a focal point measurement and the error that occurs in the focal point measurement value differ depending on specific conditions of the focal point measurement such as a focal position changing speed. Moreover, such a difference in errors also occurs due to, for example, the type of a device used for the optical system driving section 25, individual characteristics, or a change in device characteristics over time and the like. Whereas, in the above-described focus information acquiring method, since a method for canceling out influences of delays in a change of the focal position by executing two times of focal point measurement is used, irrespective of such uncertainty of the error, it is possible to reliably determine the in-focus position.
Here, with regard to the two times of focal point measurement, it is preferable to carry out the focal point measurements while providing identical conditions other than the focal position changing direction, for example, the changing range, changing interval, and the like of the focal position. Alternatively, the conditions other than the focal position changing direction can also be set to different conditions. Moreover, with regard to the changing interval of the focal position on the sample S in the first focal point measurement and the second focal point measurement, it is preferable to acquire an image of the sample S at every focal position changing interval set equal to or lower than a depth of field so as to acquire a plurality of images used for calculation of a focal point measurement value.
By setting the focal position changing interval as such for a focal point measurement, an in-focus position for the observation position on the sample S can be determined with a sufficiently high accuracy. Although the depth of field of the microscope apparatus 1A differs depending on a numerical aperture NA of the objective lens 21 used in the light guiding optical system 20, this is, for example, on the order of 1.5 μm when NA=0.4, and when NA=0.7, on the order of 0.5 μm. For a specific setting of a focal position changing interval, a method for, for example, setting the changing interval with a resolution half as much as the depth of field can be used. Alternatively, the focal position changing interval may be set greater than the depth of field according to a required focal position accuracy.
For derivation of the in-focus position using the first focal point measurement value and the second focal point measured value, it is preferable, as described above, to determine the in-focus position by a mean value of the first and second focal point measurement values. In the first and second focal point measurements where the focal position changing direction is opposite each other, delays in a change of the focal position are usually opposite in direction and approximately equal in size. Therefore, by providing a mean value of the first and second focal point measurement values as an in-focus position as described above, the in-focus position where influences of delays in a change of the focal position have been canceled out can be easily and reliably determined. Moreover, as such an in-focus position deriving method, a method other than the method for providing a mean value as an in-focus position may be used according to a specific focal point measurement method, characteristics of a device used for changing the focal position, and the like.
Moreover, as the image pickup means that acquires an image of the sample S, the identical CCD camera 30 has been used for acquisition of focus information by the focal point control section 51 and acquisition of an image of the sample S by the image acquisition control section 52 in the configuration shown in
Generally, it is preferable, as an image pickup device for focal point control, to use an image sensor such as a CCD camera that is capable of acquiring a two-dimensional image of the sample S. In addition, as an image pickup device for image acquisition, a linear sensor capable of acquiring a one-dimensional image of the sample S or an image sensor capable of acquiring a two-dimensional image can be used. For example, when acquiring an image while scanning a certain range of an observation target region of the sample S, it is preferable to use, as an image pickup device for image acquisition, a linear sensor (line sensor) or a two-dimensional image sensor capable of a TDI operation. Moreover, in the configuration shown in
The focus information acquiring method in the automatic focusing device and the microscope apparatus 1A shown in
In the microscope apparatus 1A of
When acquisition of an image of the sample S is carried out for a certain observation target region as such, as a method for acquiring focus information for the entire observation target region, it is preferable to use a method of the focal point control section 51 sectioning the observation target region of the sample S into a plurality of focal point measurement regions, determining an in-focus position for each of the plurality of focal point measurement regions, and carrying out mapping of the in-focus position for the observation target region by these in-focus positions.
In contrast thereto, by the focus information acquiring method shown in
Here, with regard to sectioning conditions such as a number of sections of the observation target region R into focal point measurement regions RF, it is sufficient to appropriately set these according to a specific size of the observation target region R, a required focus information accuracy, and the like. Moreover, with regard to a focal point measurement and derivation of an in-focus position in each focal point measurement region RF, it is preferable to, for example, as shown in
When actually obtaining an image of the sample S by use of focus information acquired by such a method, it is sufficient for the image acquisition control section 52 to read out mapping data of focus information stored in the focus information storing section 55 and carry out scanning in the observation target region R while referring to the data so as to acquire an image of the sample S.
Alternatively, as a method for acquiring focus information for the entire observation target region of the sample S, it is preferable to use a method of the focal point control section 51 setting three or more focal point measurement positions in an observation target region of the sample S, determining an in-focus position for each of the three or more focal point measurement positions, and determining an in-focus position for each observation position in the observation target region by use of those in-focus positions.
Such a method is particularly effective when a change in the focal position is planar such as, for example, when the sample stage 10 is sufficiently planar in shape and a change in the focal position in the observation target region R is only due to a tilt of the stage 10. However, when three focal point measurement positions RF are used as shown in
In addition, as in the examples shown in
The automatic focusing device and microscope apparatus according to the present invention are not limited to the above-described embodiment and configuration examples, and can be variously modified. For example, for the focal point changing means that changes a focal position on the sample S for the image pickup means and the light guiding optical system in an optical axis direction, the optical system driving section 25 that drives the CCD camera 30 and the light guiding optical system 20 in the z-axis direction is used in the configuration shown in
Moreover, for the observation position changing means that changes an observation position (imaging position) on the sample S in a direction perpendicular to the optical axis, the XY stage driving section 15 that drives the sample stage 10 in an xy plane is used in the configuration shown in
Here, it is preferable that the automatic focusing device includes: (1) image pickup means that acquires an image by an optical image of a sample to be an observation target; (2) a light guiding optical system that includes an objective lens into which light from the sample is made incident and guides the optical image of the sample to the imaging means; (3) focal point changing means that changes a focal position on the sample for the image pickup means and the light guiding optical system in an optical axis direction; and (4) focal point control means that acquires, by use of the image pickup means and the focal point changing means, focus information when acquiring an image of the sample, wherein (5) the focal point control means calculates a first focal point measurement value from a plurality of images of the sample acquired by a first focal point measurement that is executed while continuously changing the focal position in one direction by the focal point changing means, calculates a second focal point measurement value from a plurality of images of the sample acquired by a second focal point measurement that is executed while continuously changing the focal position in a direction opposite that of the first focal point measurement, and determines an in-focus position for the sample based on the first focal point measurement value and the second focal point measurement value.
In addition, it is preferable that the focal point control means determines the in-focus position by a mean value of the first focal point measurement value and the second focal point measurement value. In this case, an in-focus position where influences of delays in a change of the focal position have been canceled out can be reliably determined. Alternatively, as the method for determining an in-focus position, another method may be used according to a specific focal point measurement method.
In addition, it is preferable that the focal point control means carries out, in the first focal point measurement and the second focal point measurement, acquisition of a plurality of images of the sample at every changing interval of the focal position set equal to or lower than a depth of field. In this case, an in-focus position for an observation position of the sample can be determined with a sufficiently high accuracy.
In addition, it may be possible that the automatic focusing device includes observation position changing means that changes an observation position on the sample for the image pickup means and the light guiding optical system in a direction perpendicular to the optical axis, wherein the focal point control means sections an observation target region of the sample into a plurality of focal point measurement regions, and carries out mapping of the in-focus position for the observation target region by the in-focus positions determined respectively for the plurality of focal point measurement regions.
Alternatively, it may be possible that the automatic focusing device includes observation position changing means that changes an observation position on the sample for the image pickup means and the light guiding optical system in a direction perpendicular to the optical axis, wherein the focal point control means sets three or more focal point measurement positions in an observation target region of the sample, and determines the in-focus position for each observation position in the observation target region by use of the in-focus positions determined respectively for the three or more focal point measurement positions.
According to these configurations, when acquisition of images at a plurality of observation positions is carried out for a sample of an observation target, focus information such as in-focus positions for an observation target region including the plurality of observation positions can be efficiently acquired.
In addition, it is preferable that a microscope apparatus includes the above-described automatic focusing device and image acquisition control means that controls acquisition of an image of the sample based on the focus information including the in-focus position determined for an observation position of the sample by the focal point control means.
An automatic focusing device and a microscope apparatus according to the present invention can be utilized as an automatic focusing device that is capable of accurately determining an in-focus position in a short time and a microscope apparatus using the same.
Number | Date | Country | Kind |
---|---|---|---|
2004-220718 | Jul 2004 | JP | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/JP05/13299 | 7/20/2005 | WO | 00 | 5/9/2008 |