Patent Grant
7304733
This application is a national stage of PCT/EP03/05094 filed 15 May 2003 and based upon DE 102 22 359.9 filed 21 May 2002 under the International Convention.
The invention refers to a method for spectrally differentiating imaging measurement of fluorescent light
Furthermore, the invention refers to a device for spectrally differentiating imaging measurement of fluorescent light.
Such methods and devices find application especially in modern biology today. Especially for fluorescence microscopy, a number of specific fluorescence probes have been developed. These are suitable, for example, for specific labeling of antibodies, certain DNA sequences or other biological structures. Furthermore, they include fusion constructs of certain proteins with fluorescent proteins, such as GFP (Green Fluorescent Protein) or YFP (Yellow Fluorescent Protein), etc. Furthermore, special indicator dyes are included, the fluorescence of which is correlated with the concentration of certain ions, for example calcium, with respect to their intensity and/or emission spectrum.
Modern biology attempts to adjust the complexity of the measurement methods to the complexity of the investigated samples and thus many times it is interested in localizing as large a number of different markings in a sample as possible and to resolve these spatially from one another.
Another especially current problem is the quantitative determination of fluorophores which enter into interaction with one another through fluorescenceless [sic] energy transfer FRET (Fluorescence Resonance Energy Transfer). Such FRET-pairs consisting of donor and acceptor cannot be resolved from one another in an optical microscope. Rather, the superimposition of the donor and acceptor spectra or their relationship to one another is measured.
Another current problem is the separation of the fluorescence of an indicator dye into the portions of the bound and free form of the fluorophore for the purposes of obtaining the ratio and subsequent calculation of the activity of a ligand.
Another problem which occurs in almost all imaging fluorescence methods in biology is the consideration of the so-called autofluorescence, that is, the nonspecific background fluorescence which many structure exhibit, such as cells and substrate carriers.
In principle, an essential limitation of this method lies in the fact that the organic fluorophores that are usually used have relatively broad excitation and emission spectra which is attributed to the large number of phononic sublevels that participate in these organic molecules. Thus it becomes comparatively difficult to excite species of fluorophores contained in a sample in a specific way or to detect them specifically. Rather, usually one obtains a complex composition of the contributions of different species as signal.
Conventionally, the method employed is to use excitation channels which are as far away from each other as possible, and to employ as narrow detection channels as possible. The concept of excitation channel in this connection is understood to mean the sum of the properties of the light that excites the fluorescence. This includes especially the spectral properties which also includes in the framework of this description the intensity of the particular spectral components. In any case, other properties, such as the time of excitation and/or the duration of excitation combined as excitation time, can be used for the definition of an excitation channel. Analogously, here the concept of detection channel is defined as the sum of the properties of the elements which guide the fluorescent light emitted by the sample and filter and detect them. This includes again the spectral properties including the particular sensitivities toward the individual spectral components, and on the other hand the detection time, the detection time point and detection duration. Special combinations of excitation channels and detection channels are described below in a summarizing way as measurement channels.
In current practice, various methods are known which are aimed at optimum spectral resolution of the different types of fluorophores in the presence of one another and depend on the properties of the fluorophores and on their combination. Thus, for example, it is possible to carry out several recordings in succession with different excitation wavelengths in a given detection channel, where the excitation wavelengths are always chosen so that the absorption maximum of a fluorophore species is included as accurately as possible. In this case, one measurement channel is used per measurement. Another possibility consists in exciting the sample at an excitation wavelength which lies in the region of the excitation spectra of several fluorophore species and to divide the emitted light using filter sets or by cascades of beam splitters into spectral regions and then introduce these parts to separate photosensors. Thus, in this method, several measuring channels are used simultaneously. If the emission or excitation bands of the fluorophores of interest are sufficiently widely separated from one another, the frequency regions of the individual measurement channels can be chosen so that each channel corresponds to a fluorophore.
The disadvantage of these techniques is that mostly a certain cross-talk between the channels is unavoidable. This applies especially when a number of different fluorophores are used in a sample where the spectra overlap due to the limited bandwidth of usable wavelengths. Although this can be counteracted by sharply limiting the spectral limits of the individual detection channels, for example, by using narrow band pass filters, the consequence is that a large number of the fluorescence photons do not contribute to the signal, which has an adverse effect on the quality of the detected signal. This is especially undesirable because, due to bleaching processes of the fluorophores in the sample, the total number of photons that can be emitted by a given preparation is limited, but also because, due to photon noise, the quality and resolution of a measurement becomes better when more photons contribute to the measurement. Although almost all fluorescence photons can be made useful by breaking up the emitted fluorescent light spectrally and then treating the spectrum with the aid of a large number of spectral channels, the relative noise increases considerably in each individual extremely narrow channel, because only comparatively few photons are available for each individual channel, so that this method is only suitable in cases of application where the light intensity is especially high.
The problems addressed above can be reduced greatly when broad measurement channels are used, the cross-talk of which is deliberately taken into account and the received data are subjected to a considerable mathematical processing or evaluation. For this purpose, the received signals are converted in the detectors or in connected conversion units into digital data, and then these are stored in a memory unit of a digital data processing equipment. In many cases, for example in laser scanning microscopy (LSM), digitalization and subsequent processing of the data is an essential component of the technique.
The evaluation of the data mentioned above is usually done with the aid of a computer unit of the digital data processing equipment. Especially good results were obtained with the so-called “linear unmixing” method. This method is based on setting up and solving an inhomogeneous linear system of equations which, using the known properties of the measurement channels, establishes a relationship between the measured signal and the fluorophore composition in the sample. This system of equations can be represented mathematically in a matrix representation as
{right arrow over (y)}=A{right arrow over (B)}+{right arrow over (I)}·{right arrow over (b)} (1)
or in a component representation
These formulas are to be understood as follows: The vector {right arrow over (B)} represents the different species of fluorophores in their relative concentration at a given image point. Let p be the number of different fluorophore species. Thus, the vector {right arrow over (B)} has p components Bμ. The vector {right arrow over (y)} represents the signal detected in each measurement channel. Let q be the number of measurement channels. Thus, the vector {right arrow over (y)} has q yr components. For example, if four different excitation wavelengths and four different spectral detection windows were used, the number of measurement channels is q=16. The vector {right arrow over (I)} represents the excitation intensity used for each measurement channel and thus has q components Ir. The matrix A is the coefficient matrix which links the chemical composition {right arrow over (B)} of the fluorophores through the excitation intensities Ir of the excitation channels and the other properties aμr of the measurement channels to the resulting signal {right arrow over (y)}. Thus, the matrix A has pq elements Iraμr. Finally, the vector {right arrow over (b)} with q components br is a correction quantity which represents the scattered light or another background light in each measurement channel. The quantities Bμare usually to be regarded as location dependent, while the other quantities on the right side of equations (1) and (2) represent parameters which are normally the same for all pixels. Autofluorescence of the measured object can be treated either as fluorescence of an additional fluorophore Bμor as background light br (in case it is location independent). In the case of FRET, an FRET-pair can be considered as an independent chromophore, the concentration of which is given through one of the quantities Bμ.
The goal of“linear unmixing” is to find the solution B of the above linear system of equations, which is possible mathematically by simple inversion of the coefficient matrix A as long as the number of equations q is greater than or equal to the number of different fluorophore species p. For the algorithmic conversion of this mathematical operation, a number of numerical methods are known to the person skilled in the art. An explanation of this technique is given in Farkas et al.: “Non-invasive image acquisition and advanced processing in optical bio-imaging”, Computerized Medical Imaging and Graphics, 22 (1998), p. 89-102 or Dickinson et al.: “Multi-spectral imaging and linear unmixing at whole new dimension to laser scanning fluorescent microscopy”, BioTechnics, 31, No. 6 (2001), p. 1272-1278 as well as Boardman: “Inversion of imaging spectroscopy data using singular value decomposition”, Proc. IGARSS, 89, No. 4 (1989), p. 2069-2072. An implementation of this method of evaluation in LSM device was realized by the company Carl Ziess, Jena, Germany, in their Laser-Scanning-Microscope LSM 510 meta.
As explained, the method of linear unmixing represents a proven means of data evaluation when knowing the properties of the measurement channels used. However, a disadvantage is that the selection of suitable measurement channels, that is, the adjustment of all parameters, such as excitation wavelength, excitation intensity, excitation time and detection wavelength and detection time is left to the intuition of the user, as before. However, since intuition is guided by concrete rules which are obvious to the user, as before, if possible, a fluorophore species should be assigned to each measurement channel so that the possibilities provided by complex data analysis are usually not utilized.
The task of the present invention is to provide a generic process with which the quality of the obtained data is improved in a simple way.
Another task of the present invention is to further develop the generic device in such a way that the results obtained with it can be improved in a simple manner.
These tasks are solved by a method and by a device with the characteristics of the independent claims.
Further embodiments of the present invention are given in the dependent claims.
The invention is based on a generic method by the fact that the properties of several measurement channels are adjusted automatically before carrying out the measurement, or the user is given the corresponding instructions for manual adjustment. The automatic adjustments or the instructions are based according to the results of a mathematical optimization method carried out by a computer unit of the digital data processing equipment. In this, the fluorescence characteristic of at least some of the fluorophores contained in the sample are taken into consideration. Such a mathematical optimization method is strictly distinct from the automatic control of certain standard adjustments which can be programmed from the manufacturing plant or can be oriented by a setting library installed by the user himself. Rather, the user of the data processing equipment enters the fluorophores that he presumes to be present in the sample and their characteristics. These data are then used as the basis of the mathematical optimization method which calculates the optimized settings of the measurement channels for the special needs of the user. Thus, especially contrasting to intuition, but based on a mathematical evaluation of obtained data, a very favorable segmentation of the entire fluorescence spectrum can be performed, instead of cutting out narrow bands with filters and reducing the number of photons contributing to the signal and reducing the signal quality unnecessarily.
Advantageously, the characteristics of a number of fluorophores are stored in one or several libraries in a memory unit of the digital data processing equipment so that when the user identifies the fluorophores presumed to be present in the samples, this is sufficient without having to enter all their complete characteristics.
It is especially favorable when, within the framework of the mathematical optimization method, a linear system of equations is set up which describes the relationship between the chemical composition of the sample presumed by the user and the signal resulting based on the properties of the measurement channels to be optimized. This characteristic is based on the mathematical foundation of “linear unmixing”. In any case, this basic idea operates exactly opposite to the state of the art. Namely, while in the case of “linear unmixing” in the case of known components Yr and known coefficients aμr that describe the measurement channel and the relative concentrations Bμ of the individual fluorophore species are searched for, the aim of the method according to the invention is rather to vary the coefficients aμr describing the measurement channels and thereby to carry out an optimization of the system in the sense that the solution of the system of equations can be done as uniquely as possible. The optimization is directed towards the fact that the system of equations can be solved, especially that they are unique. In this way, it can be ensured that the evaluation method which follows the measurement, which is based on the method of “linear unmixing”, does not fail due to the fact that the coefficients which describe the measurement channel were chosen in the special case so that the number of linear independent equations of the system is lower than the number of existing fluorophore species, and thus the system of equations can no longer be solved uniquely.
However, advantageously, the optimization method is designed to be so flexible that it is not optimized exclusively with regard to the solvability or uniqueness of the solution of the system of equations, but additionally other conditions selected by the user are also taken into consideration. Examples of such other considerations will be explained in more detail below.
Advantageously, the optimization method includes the optimization of a condition number of a matrix expression which contains the matrix formed from the coefficients of the above-mentioned linear system of equations. Within the framework of the present invention, this is to be understood so that the algorithmic implementation of the method according to the invention can be represented mathematically as optimization of a condition number of the said matrix expression. Depending on the concrete conversion, it can be possible to omit explicit definition of a matrix or of an array within the framework of a computer program.
In an especially preferred manner, the matrix expression on which the calculation of the condition number is based is formed as matrix product on the left side of the matrix A with its transpose AT, that is ATA. This is based on the consideration that the expressions of equation (1) and (2) can be converted by multiplication on the left side by AT to obtain
AT({right arrow over (y)}−{right arrow over (I)}·{right arrow over (b)})=ATA{right arrow over (B)} (3)
As it is well-known, multiplication on the left with the transposed matrix makes the obtained matrix expressions symmetric, which corresponds to a calculation of most probable values in the sense of the Gaussian minimization of the residual sum of squares.
In case of explicit consideration of the noise of the measurement channels, a matrix weighted by the measurement errors expected by the user can be employed, which can be regarded as minimization of the quantity χ2 (chi square) known from statistics.
Thus, for example, in a preferred embodiment, the condition number to be optimized corresponds essentially to the determinant of the matrix expression, especially of the expression ATA. Alternatively, condition numbers which also contain the trace N of the matrix expression can also be used as optimization criterion. In a further practical example of the method according to the invention, the quantity
is used as the condition number to be optimized, where det is the determinant, N the trace and n the dimension of the matrix expression. In another advantageous practical example of the method according to the invention, the ratio of the smallest to the largest eigenvalue of the matrix expression is used as condition number. It was shown that the optimization of each of these condition numbers leads to a selection of measurement channel properties by variation of the matrix elements Iraμr, which, although oppose intuition in many cases, provide outstanding results with regard to the data evaluation performed after the measurement, especially using the method of “linear unmixing”.
Here, the optimization of different condition numbers leads to results of different quality in different constellations of cases. It is therefore especially advantageous to further develop the method according to the invention so that the user is given the possibility to give a global characteristics of the expected measurement result, for example, very weak fluorescence, especially large number of different fluorophores which are especially close spectrally, etc., and thus to establish or to directly establish the condition number to be optimized.
The conversion of the calculated optimization result into the realization of physical properties of the measurement channels can be carried out in many ways. Thus, for example, a selection of a frequency or frequency band in the excitation and/or detection beam path can be used through adjustable filters, such as AOTFs (Acousto-Optic Tunable Filters) or LCTFs (Liquid Crystal Tunable Filters). Similarly, fixed cut-off filters, band-pass filters and/or beam splitters, which are arranged, for example, on motor-driven filter sleds or filter wheels can be used. Another possibility of automatically influencing the measurement channel characteristics consists in the variation of the excitation intensities, for example, by the introduction of the so-called wedge filters in the excitation beam path. The time characteristics of the measurement channels can be varied in the conversion of the optimization method according to the invention. For example, the duration of excitation can be varied or time detection windows can be defined for separation of fluorescent components with shorter and longer life. A number of conversion possibilities are known to the person skilled in the art in this regard.
Condition numbers of matrices as mentioned above essentially provide estimates of maximum errors, but in practice these can be much smaller. This applies especially when known structures of a given problem are taken into consideration. In an especially advantageous further development of the method of the invention, therefore the possibility of far-reaching optimization is provided in which the special properties of the experimental sources of perturbations are formulated for fluorescence measurements. Especially advantageously, a second optimization step is provided in which the noise of the expected signal is optimized by variation of the coefficients that describe the properties of the measurement channels.
This inventive idea is based on the following recognition. If the coefficients of matrix A or of the matrix expression ATA are determined, preferably optimized by using the first optimization step described above, the solution of the linear system of equations is as follows:
<{right arrow over (B)}>=(ATA)31 1AT({right arrow over (y)}−{right arrow over (I)}·{right arrow over (b)}) (5)
Here <{right arrow over (B)}> shows the expectation value of the solution {right arrow over (B)} in which it is taken into consideration that the vector {right arrow over (y)}−{right arrow over (I)}·{right arrow over (b)} has an experimental variation {right arrow over (σ)}. The vector {right arrow over (σ)} is to be understood as component-wise square root of the expression σr2, which are always to be understood as the expectation values of the variance of the measured value yr. These were composed of two components, namely the photon noise, the variance of which is proportional to the signal level, and the constant detector noise, which is statistically independent of it, and is composed of dark current and the selection noise [read-out noise] of the particular detector.
σr2=yrs+σ0,r2 (6)
Here s is a proportionality constant (a suitably calculated individual photon contribution) and σ0,r2 is the sum of all constant contributions to the variance of the signal in channel r.
According to the method of the Gaussian error propagation, the variation σμB of component Bμof the vector {right arrow over (B)} can described by
Since equation (5) is a linear system of equations and {right arrow over (I)}·{right arrow over (b)} does not depend on yr, we have
where cμr is the element of the r-th line of the μ-th column of the matrix C=(ATA)−1AT. Therefore, we have
This expression or also the sum of all squared deviations
can be minimized in the space of all measurement channel parameters. In any case, according to equation (6), the quantities σr2 contain the measured values yr, so that for the minimization of SB2, the user has to provide information about the size of the expected signals.
In an especially advantageous embodiment of the method according to the invention, it is essentially this quantity SB2 which is optimized by variation of the coefficients describing the measurement channels.
Hereby, it is preferably provided that the noise of the signal to be expected is optimized with consideration of one or more of additional conditions that can be introduced by the user. One of these additional conditions, which can also be used within the framework of the first optimization step described above, in an advantageous embodiment of the method according to the invention, could be a maximum limit for the bleaching of one or several fluorophores. Since as bleaching progresses, the signal will decrease, while certain components of the noise are independent of time, under certain circumstances the optimization can be performed with reference to the duration of illumination or its intensity. As another possible additional condition, the minimization of the noise of a signal of a certain intensity, preferably predetermined by the user, can be utilized with advantage. This additional condition is especially useful when the expected signal is so low that the total noise of the measurement channel is dominated by the dark current and by the read-out noise of the detector.
As another possible additional condition, in an advantageous embodiment of the method according to the invention, the maximum spectral resolution of the different fluorophores in a given region of a previously recorded test image can be utilized. This becomes especially significant where one or several different fluorophores are to be resolved with the background of a general non-specific autofluorescence of the sample or in case a certain image region is of special interest to the user.
In another favorable embodiment of the method according to the invention, it is provided that the minimization of the relative error of the measurement channels be used as an additional condition. This form of additional condition is preferably introduced when ratio measurement such as in FRET measurements are to be performed.
It can be especially favorable when the user has the possibility to enter, in addition to one or several additional conditions, or, instead of that, to enter information to a presumed model of the noise, for example, based on poison.
In an especially preferred manner, the optimization method according to the invention is carried out within the framework of an iterative, dialog-controlled process for the definition of additional conditions, which permits the user to enter additional information after performing a preliminary optimization step and to add one or several additional optimization steps.
In order to be able to utilize the advantages and special characteristics of the method according to the invention in an especially advantageous manner, according to the invention a device is made available, for example, a laser scanning microscope, the digital data processing equipment of which is programmed in such a way that the previously described optimization process according to the invention can be performed and which has the above-mentioned technical devices for automatic adjustment of the measurement channel properties.
A preferred embodiment of the method according to the invention and of the system according to the invention will be described below with the aid of the attached drawing. The only drawing is
The especially advantageous practical example of the device according to the invention shown in
Furthermore, the data processing installation 10 has a data interface 13 through which the recorded data as well as user inputs are fed through the user interface 20 into the data processing installation 10 and control commands to adjustable components of the optical/electronic structure 30 of the device as well as information to the user interface 20 can be outputted.
The above elements of the data processing installation 10 can be realized in many ways known to the person skilled in the art and the technical details can be adapted to the special particular configuration.
In order to measure a fluorescent sample 40, it is placed under the microscope objective. Through the user interface 20, the user can enter various data into the data processing installation 10, such as the presumed chemical fluorophore composition, expected intensities and/or optimization conditions, for example bleaching limits. Based on these data, the computing unit 11 of the data processing installation 10 calculates according to the optimization method of the invention the values according to which the optical/electronic structure 30 has to be set. As a result of this the measurement channels are defined as specific combinations of special excitation and detection channels. In the practical example shown in
The control of the scanning mirror 35 through control line 135 is done in the conventional manner.
The fluorescent light, which is shown in
The fluorescent light thus preselected falls on various detectors 37a, 37b and 37c, the data of which are fed through the input lines 137a, 137b and 137c into the data processing installation 10. Depending on the special construction, the data are digitized already in detectors 37a, 37b, 37c, or only in the data interface 13 of the data processing installation 10. The data thus recorded and stored in the memory unit 12 of the digital processing installation 10 are evaluated by the computer unit 11 using known data evaluation programs, where preferably the method of “linear unmixing” finds application.
The adjustment or definition of the measurement channels according to the invention refers to the practical example, to the spectral sectioning of the fluorescent light as well as to the number of measurement channels used, that is, the number of combinations of excitation and detection channels used. Here the number and nature of the detectors is as variable as the light sources are. Especially, not shown in
Naturally, the embodiment described and shown in
| Number | Date | Country | Kind |
|---|---|---|---|
| 102 22 359 | May 2002 | DE | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/EP03/05094 | 5/15/2003 | WO | 00 | 11/22/2004 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO03/098174 | 11/27/2003 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5243546 | Maggard | Sep 1993 | A |
| 5324940 | Ekstrom | Jun 1994 | A |
| 5568400 | Stark et al. | Oct 1996 | A |
| 6166853 | Sapia et al. | Dec 2000 | A |
| 6200818 | Eigen | Mar 2001 | B1 |
| 6300639 | Wiederhoeft | Oct 2001 | B1 |
| 6384914 | Drexhage | May 2002 | B1 |
| 6741346 | Gerstner | May 2004 | B1 |
| 6995840 | Hagler | Feb 2006 | B2 |
| 20040022684 | Heinze et al. | Feb 2004 | A1 |
| 20040181375 | Szu et al. | Sep 2004 | A1 |
| 20050017160 | Wolleschensky et al. | Jan 2005 | A1 |
| Number | Date | Country |
|---|---|---|
| 42 10 970 | Oct 1993 | DE |
| 197 18 016 | Nov 1998 | DE |
| 197 57 740 | Jul 1999 | DE |
| 198 29 944 | Jan 2000 | DE |
| 199 15 137 | Oct 2000 | DE |
| 199 30 532 | Jan 2001 | DE |
| 199 56 620 | May 2001 | DE |
| 100 08 594 | Aug 2001 | DE |
| 100 35 190 | Feb 2002 | DE |
| WO 01 09592 | Feb 2001 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20050243313 A1 | Nov 2005 | US |
