Patent Application
20060066734
Priority is claimed to the provisional application entitled “Method for the Overlay of Optical Information in a Scan Microscope,” filed on Sep. 7, 2005, by applicants, and to German patent application DE 10 2004 047 519.9, the entire subject matters of both of which are hereby incorporated by reference herein.
The invention relates to a method for superimposing optical information in a scanning microscope, at least one CCD image A being superimposed with at least one additional piece of image information B.
Methods for superimposing optical information in the realm of scanning microscopy are known from actual practice. In this process, CCD images are often superimposed with additional image information. In this context, the additional image information is often a scanned image. Through such a superimposition, additional information can be obtained about the specimen to be examined, said information going beyond the information that an individual CCD image and, for example, a scanned image can provide on their own. For example, potentials in individual calls can be depicted here. The normally colored CCD image can be superimposed with an image or with image information that supplies merely light-and-dark information. The superimposition of a CCD image with additional image information yields a higher resolution of the image depiction in comparison to individual images. Ultimately, the information obtained through a CCD image is increased by a superimposition with at least one additional piece of information.
With the prior-art methods for superimposing optical information, it is a common procedure to carry out a superimposition manually or by making complicated adjustments in order to achieve congruence of the superimposed optical information. This is extremely time-consuming, as a result of which the individual scanning microscopes do not achieve a high specimen throughput rate.
Particularly when a CCD image is superimposed with a scanned image, system-related differences and shifts in the images occur in terms of the position, rotation and size of the images that have to be compensated for by manual processing or complicated adjustments in order to achieve a congruence of the images.
Consequently, it is an objective of the present invention to provide a method for superimposing optical information of the type described above, in which a reliable superimposition of the optical information and a high specimen throughput rate are ensured.
The present invention provides a method for superimposing optical information in a scanning microscope, at least one CCD image A being superimposed with at least one additional piece of image information B. A transformation matrix T is determined that allows the superimposition of the optical information of the at least one CCD image A and of the at least one piece of image information B.
It has been recognized according to the invention that a superimposition of optical information in a scanning microscope does not necessarily have to be carried out manually or by making complicated adjustments. In another manner according to the invention, it has been recognized that a transformation matrix T can be determined that allows the superimposition of the optical information of the at least one CCD image A and of the at least one piece of image information B. This translates into a superimposition of image information that has been obtained in different ways. With the method according to the invention, first of all, a method is put forward that allows a reliable superimposition of the optical information by the operator without errors in the case of a manual superimposition. Secondly, due to the superimposition by means of the transformation matrix T, a much faster superimposition of the optical information can be carried out virtually automatically.
Consequently, the method according to the invention for superimposing optical information in a scanning microscope constitutes a method with which reliable superimposition of the optical information and a high specimen throughput rate are ensured.
In an embodiment, the at least one piece of image information B can comprise logical data or information. Such data or information can be superimposed with a CCD image in a simple manner.
Depending on the requirements, the at least one piece of image information B can be acquired via at least one logical channel. The number of channels is to be adapted to the application case in question.
In another application, the at least one piece of image information B could be a scanned image or a confocal scanned image. In the case of superimposition with a CCD image, the information obtained through the CCD image can be linked, so to speak, with the information acquired through the scanned image.
In an embodiment, the image information B could be transferred into the CCD image A by means of one or more matrix multiplications A=T*B. As an alternative to this, the CCD image A could be transferred into the image information B by means of one or more matrix multiplications B=T*A. The selection of the transfer direction between the image information B and the CCD image A should be adapted to the application case in question.
For example, when a CCD image is superimposed with a scanned image, different influencing variables have to be taken into account that initially stand in the way of a correct superimposition. Here, the images might differ in terms of their rotation. Moreover, the individual image mid-points might not match the other mid-point in each case. Furthermore, different imaging techniques might entail different magnifications of the individual images relative to each other. Finally, non-linear deviations due to imperfect galvanometer movements can occur with the scanning microscope.
In order to take into account the above-mentioned influencing variables, the transformation matrix T could be formed by the multiplication of individual matrices so that T=T1*T2 . . . . Here, each of the independent influencing variables can be expressed and compensated for by individual matrices T1, T2, . . . .
In an embodiment, the individual matrices T1, T2, . . . could each take into account differences in the at least one CCD image A and in the at least one piece of image information B in terms of the rotation and/or of the image magnification and/or of the centering.
Moreover, at least one matrix T1, T2, . . . could take into account non-linear deviations due to imperfect galvanometer movements. Finally, any desired influencing variable could be taken into account by one of the matrices T1, T2, . . . .
With an eye towards an especially simple computation of the transformation matrix T, the transformation matrix T could be computed by means of mathematical models based on a reference structure. In other words, the matrix could be computed by solving different mathematical models. A coordinate transfer could be performed during the computation.
As an alternative, the transformation matrix T could be computed by means of a correlation formation. Here, the at least one piece of image information B could be convolved within the scope of the computation that is performed by means of a correlation formation. The correlation formation could serve as a model for the transfer of information or data.
In an embodiment, at least one superimposition parameter could be indicated. Such a superimposition parameter could comprise, for example, rotating the optical information that is to be superimposed with respect to each other or it could comprise magnification factors or shifts.
In order to ensure the highest possible quality of the superimposition of the optical information, the scanning microscope arrangement could be adjusted taking into account the superimposition parameters. In this manner, depending on the desired superimposition of image information, an individual adjustment of the scanning microscope arrangement could be effectuated.
In an embodiment, the adjustment of the scanning microscope arrangement could comprise a rotation and/or a size adaptation and/or a shift and/or a centering. Other types of adjustment are likewise conceivable in this context.
Depending on the images to be superimposed, it might happen that a complete superimposition or overlapping of the images cannot be achieved. Here, an intersection area of the images to be superimposed and usually several areas are present that do not overlap or that cannot be superimposed. The resultant intersection area usually does not have the normally desired rectangular or square shape of an image. In order to prepare such a desired rectangular or square shape of the superimposed image, the optical information in non-overlapping or in non-superimposable areas of the at least one CCD image A and of the at least one additional piece of image information B can be augmented by so-called “zero” information. As a result of this, an image shape can be attained that is square or rectangular or that has any other desired shape.
In an alternative overlapping adaptation, either the at least one CCD image A or the at least one additional piece of image information B can be reduced in size in order to achieve a maximum overlapping. In other words, each CCD image A or image information B is reduced in size until it is completely covered by the image information B or by the CCD image A. In this case, the complete size and shape of the reduced CCD image A or of the reduced image information B could be used as the overlapping image surface.
The above-mentioned adjustment of the scanning microscope arrangement and/or the above-mentioned augmentation by so-called “zero” information and/or the above-mentioned size reduction of the at least one CCD image A or the at least one additional piece of image information B could be done conveniently online. In this manner, an extremely flexible method is provided for superimposing optical information in a scanning microscope.
It is fundamentally possible to work with several imaging channels within the scope of the method according to the invention. This relates to the CCD image A as well as to the imaging of the additional image information B. Furthermore, it would be possible to work with several imaging devices on one scanning microscope. The selection of the number of imaging channels and/or of the CCD imaging devices should be adapted to the application case in question or to the particular specimen to be examined.
With the method according to the invention, the CCD image A could be used as a reference image. The additional image information B, which could be generated by a channel of a confocal scanning device, can then be transferred into the CCD image A in that matrix multiplications T are performed, which take in account differences and influencing variables that can occur due to different imaging techniques. Here, an equation system can be set up that meets the condition A=T*B, wherein T is the product of individual matrices that take into account the independent influencing variables and differences.
There are various possibilities to configure and refine the teaching of the present invention in an advantageous manner. For this purpose, on the one hand, reference is made to the subordinate claims and, on the other hand, to the explanation below of a preferred embodiment of the method according to the invention with reference to the drawing. In conjunction with the explanation of the preferred embodiment of the method according to the invention with reference to the drawing, preferred embodiments and refinements of the teaching will be explained in general. The drawings show the following:
FIGS. 3 to 6 each show the CCD image A and the image information B; in
FIGS. 4 to 6 now successively show the progress of the superimposition by means of the multiplication of the image B by another individual matrix that is indicated below the image B in each case. Consequently, the image B in
In addition, for example, another multiplication by an individual matrix T_nonlin could be performed, with which non-linear deviations due to imperfect galvanometer movements could be taken into account.
Regarding additional advantageous embodiments of the method according to the invention, in order to avoid repetitions, reference is hereby made to the general part of the description as well as to the accompanying patent claims.
Finally, it should be explicitly pointed out that the above-described embodiment of the method according to the invention serves merely to explain the claimed teaching, but this is not limited to the embodiment.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2004 047 519.9 | Sep 2004 | DE | national |
