Embodiments of the present invention relate to a microscope control arrangement, to a microscope system, to a method of controlling a microscope and to a computer program.
EP 3 721 279 B1 discloses a microscope system comprising a detection unit adapted to detect fluorescence responses of fluorophores in different colour channels using a plurality of detectors. Each of the fluorophores may be excited using light of a light source such as a light emitting diode. In an embodiment discussed in connection with
Embodiments of the present invention provide a microscope control arrangement. The microscope control arrangement includes one or more processors and one or more storage devices. The one or more processors are configured to process fluorophore information indicating one or more fluorophores, derive control parameters based on the fluorophore information, and render a graphical user interface on a display device. The graphical user interface includes one or more fluorophore control widgets corresponding to the one or more fluorophores indicated by the fluorophore information. Each respective fluorophore control widget includes a first widget zone providing a user feedback indicating the corresponding fluorophore and a second widget zone indicating an illumination intensity associated with the corresponding fluorophore.
Subject matter of the present disclosure will be described in even greater detail below based on the exemplary figures. All features described and/or illustrated herein can be used alone or combined in different combinations. The features and advantages of various embodiments will become apparent by reading the following detailed description with reference to the attached drawings, which illustrate the following:
Embodiments of the present invention can improve the operation of microscope systems in which fluorescence responses of one or more fluorophores are detected, in particular in terms of effectiveness and user friendliness.
According to some embodiments, a microscope control arrangement includes one or more processors and one or more storage devices. The microscope control arrangement is configured to provide control parameters. The microscope control arrangement is configured to process fluorophore information indicating one or more fluorophores and to derive the control parameters on the basis of the fluorophore information. The microscope control arrangement is also configured to render a graphical user interface and, as a part of the graphical user interface, one or more fluorophore control widgets, the or each of the fluorophore control widgets corresponding to the or one of the fluorophores indicated by the fluorophore information. The or each of the fluorophore control widgets comprises a first widget zone and a second widget zone, the first widget zone providing a user feedback indicating the corresponding fluorophore and the second widget zone indicating an illumination intensity associated with the fluorophore. In the fluorophore control widget or each of the fluorophore control widgets, the second widget zone may, in an embodiment of the invention, be rendered as a rim surrounding the first widget zone of which a proportion corresponding to the illumination intensity is rendered differently from a remaining proportion.
The proposed microscope control arrangement, including the user interface and the fluorophore control widgets, allows for an essentially fluorophore-based operation of a fluorescence microscope. This is a departure from conventional operating paradigms based on specific instrument settings and in particular enables unexperienced or less experienced users, but also experienced users, to adjust and control settings for examining a sample in a more user-friendly and less error-prone manner.
The term “widget” shall, in the understanding used herein, refer to any element of interaction rendered as a part of a user interface including, but not limited to, elements configured for selection and for the display of elements or collections such as buttons (including radio buttons, check boxes, toggle switches, toggle buttons, split buttons, cycle buttons), sliders, list boxes, spinners, drop-down lists, menus (including context menus and pie menus), menu bars, tool bars (including ribbons), combo boxes, icons, tree views, grid views, elements configured for navigation such as links, tabs and scrollbars, elements for textual input such as text and combo boxes, elements for output of information such as labels, tool tips, help balloons, status bars, progress bars and information bars, and containers such as (modal) windows, windows, dialog boxes, palettes, frames and canvas elements. The term “user interface” shall generally be understood to refer to a graphical user interface.
The microscope control arrangement may, in an embodiment of the invention, be configured to receive a user input in different input modes and to translate the user input in the different input modes to an interaction with the fluorophore control widgets in different interaction modes. This enables a user to interact with the system in a way which is familiar to the user from user input devices in other technical fields. Different interaction modes for the same user input device allows advantageously using one device for triggering different functions.
The microscope control arrangement may, in an embodiment of the invention, be configured to receive the user input in the different input modes from one or more user input devices selected from a mouse, a touchpad, a joystick, a trackball and a touchscreen, wherein the interaction modes include at least one of a click of a first button, a click of a second button, a drag-and-drop operation, a touch operation and a mouse-over or hover operation. In particular, each of said interaction modes may be associated to a specific function. A user may therefore, while e.g. having positioned a cursor or a mouse pointer at a specific position, use the same input device to trigger different functions without distraction.
The microscope control arrangement may, in an embodiment of the invention, be configured to toggle, in response to the interaction in one of the interaction modes, at least one of an illumination for exciting the corresponding fluorophore, use of fluorophore information relating to the corresponding fluorophore in an un-mixing method determining contributions of the fluorophore in a common fluorescence response, and displaying an image obtained on the basis of a fluorescence response of the corresponding fluorophore. That is, using a single user operation, a fluorophore (i.e. parameters relating to its excitation and detection) may advantageously be switched “on” and “off” using a single user operation instead of adjusting a plurality of instrument settings individually.
The microscope control arrangement may also be configured, in an embodiment of the invention, to render a fine-tuning panel in response to the interaction in one of the interaction modes. Therefore, a user may fine-tune parameters relating to one of the fluorophores without specifically selecting a different window for such adjustments but use the very same control widget. This allows for operating a microscope in a less distractive manner.
The microscope control arrangement may, in this connection, be configured to modify the control parameters on the basis of a user input received using the fine-tuning panel which is, as mentioned, advantageously possible in a less distractive manner than in microscope control arrangements according to the prior art.
In particular, the microscope control arrangement may, in an embodiment of the invention, be configured to reposition the fluorophore control widgets in the graphical user interface upon determining an interaction in one of the interaction modes. Such a repositioning and regrouping may particularly serve to assign the control widgets, and the fluorophores they correspond to, to different ways of processing in a particularly intuitive manner.
In this connection, the fluorophore information may indicate several fluorophores and the microscope control arrangement may be configured to provide the control parameters as specifying an excitation of the fluorophores sequentially or in parallel. In this connection, by repositioning the fluorophore control widgets, one or more of the fluorophores may be particularly easily assigned to a specific way of processing.
In an embodiment of the invention, the microscope control arrangement may be configured to provide the control parameters as specifying to request the excitation of the fluorophores sequentially or in parallel on the basis of a state of a grouping of the corresponding fluorophore control widgets in the graphical user interface. An intuitive assignment of fluorophores to different ways of processing (sequentially or in parallel) is therefore possible, e.g. a grouping of fluorescence control widgets in one sub-unit of the user interface such as a canvas region or sub-window, may indicate a parallel processing while a placement of fluorescence control widgets in different sub-units may indicate a sequential processing.
The microscope control arrangement may, in an embodiment of the invention, be configured to provide the control parameters as further specifying at least one of camera settings and scanner settings. In such an embodiment, even more parameters than parameters relating to an illumination may be controlled on the basis of the fluorophore information and therefore the advantageous operation paradigm provided may be further improved.
In an embodiment of the invention, the control arrangement may be configured to control an operation of a fluorescence microscope comprising a plurality of light sources, and the illumination intensity controlled may be an illumination intensity of one or more of the light sources mainly used for excitation of the corresponding fluorophore. That is, the main fluorescence response of a fluorophore (not including a fluorescence response due to cross-excitation) may simply be switched “on” and “off” using a single user interface operation.
In the microscope control arrangement, the control parameters may, in an embodiment of the invention, be configured to be used in a wide-field fluorescence examination technique and a confocal fluorescence examination technique. The microscope control arrangement is therefore particularly useful, using the advantageous operating paradigm mentioned, for controlling fluorescence microscopes operable in both a wide-field and a confocal fluorescence examination technique such as mentioned at the outset.
A microscope system comprising a fluorescence microscope and a microscope control arrangement is also provided. As to features and advantages thereof, reference is made to the explanations of different aspects above which also apply to the microscope system.
Further proposed is a method for providing microscope control parameters, wherein fluorophore information indicating one or more fluorophores are processed and the control parameters are derived on the basis of the fluorophore information. A graphical user interface and, as a part thereof, one or more fluorophore control widgets are rendered, the or each of the fluorophore control widgets corresponding to the or one of the fluorophores indicated by the fluorophore information. The or each of the fluorophore control widgets comprises a first widget zone and a second widget zone, the first widget zone providing a user feedback indicating the corresponding fluorophore and the second widget zone indicating an illumination intensity associated with the fluorophore. Again, as to features and advantages of such a method, reference is made to the explanations of different aspects above which apply to method for controlling a microscope as well.
This also holds for a corresponding method according to an embodiment of the invention, in which a microscope control arrangement or a microscope system as explained before in different embodiments is used.
A computer program with program code for performing a method as explained in different aspects before, when the computer program is run on a processor, is also provided and likewise takes profit of the corresponding advantages.
As already mentioned at the outset, fluorescence microscopes comprising a plurality of detectors which may be used to detect fluorescence responses of different fluorophores are known. In such fluorescence microscopes, but also if fluorescence microscopes comprise a single detector only, adjusting fluorescence illumination settings properly, such as the intensity of a single light source, or of one of a plurality of light sources, and of the corresponding detection settings is no trivial task. Such adjustments become even more complex in microscopes which allow a user to select between different microscopy operation modes including, but not limited to, a wide-field and a confocal operation mode.
This will be discussed hereinbelow with reference to
As will be explained, the technical elements used in operating a fluorescence microscope 300 are numerous, and their influences on each other, on a sample observed, and on the results obtained in terms of fluorescence images are not always predictable, particularly for an unexperienced user. For example, if an illumination intensity of a light source is adjusted to a high value, the fluorescence response of a fluorophore whose excitation wavelength peak corresponds to a centre wavelength of the light source might be increased. However, in such cases, also cross-excitation of other fluorophores whose main excitation wavelength peak does not correspond to the centre wavelength of the light source may become remarkably high and make the results obtained less specific (“cross-talk”). This means that the assignment of excitation and response is becoming increasing difficult and also the output of known un-mixing techniques will become unreliable. Furthermore, too intense an illumination setting can cause light damage to a sample or the fluorophores therein, which is also referred to as “sample bleaching”.
Too low an illumination setting, however, may result in images of too low contrast in one fluorescence channel or to have too high relative contributions of cross-excitation from other light sources, thus making the results obtained less specific.
Certain effects, such as a local environment of the fluorophore, can cause the main excitation and/or emission wavelengths to shift to a certain extent and therefore an adjustment of the wavelength used for excitation and/or detection may be necessary or the excitation or detection bandwidth may have to be increased. In other cases, however, for example to reduce cross-talk and cross-excitation, the width of an excitation and detection band may advantageously be narrowed.
Excitation and detection settings particularly influence the result of spectral un-mixing techniques which may be used in connection with fluorescence microscopy. Spectral un-mixing is a technique which tackles the problems of overlapping emission spectra of fluorophores as a result of cross-excitation or “bleed-through” between different detection channels. These phenomena, when not addressed properly, may lead to false-positive results. Corresponding problems become particularly pronounced when samples are labelled with three or more fluorophores. Spectral un-mixing may include linear un-mixing, non-negative matrix factorization, deconvolution, and principal component analysis. Un-mixing techniques may be based on a priori knowledge of emission spectra or may be used in connection with restricting the number of fluorophores to be the same or lower as the number of detection channels. At its core, and in the understanding as used herein, spectral un-mixing is the task of decomposing mixed multichannel images into spectral signatures and abundances of each signature in each pixel.
The settings of specific components influencing each other and the outcome of microscopic examination mentioned before are not exhaustive. For example, also a detection time or frame rate (i.e. the frequency in which images are taken, and the time used for acquiring one image) in an area detector or corresponding scan times in a scanning system with point or line detectors may have a strong influence on contrast, brightness and potentially detector noise and are closely related with the illumination intensities used.
As mentioned,
The computer system 100 may be configured to execute at least a part of a method described herein. The computer system 100 and the microscope 300, as well as the interface unit 200, which is entirely optional, may be separate entities but can also be integrated together in one common housing. The computer system 100, even if illustrated as a laptop computer, may be part of a central processing system of the microscope 300 and/or the computer system 100 may be part of a subcomponent of the microscope 300, such as a sensor, an actor, a camera or an illumination unit, etc. of the microscope 300. Essentially the same holds true for the interface unit 200.
The computer system 100 may be a local computer device (e.g. personal computer, laptop computer, tablet computer or mobile phone) with one or more processors 140 and one or more storage devices 150 or may be a distributed computer system (e.g. a cloud computing system with one or more processors and one or more storage devices distributed at various locations, for example at a local client and/or one or more remote server farms and/or data centers). The computer system 100 may comprise any circuit or combination of circuits.
In embodiments of the present invention, the computer system 100 may include one or more processors 140 which are illustrated as being integrated into a housing of the computer system 100 in
The one or more storage devices 150 the computer system 100 is shown to comprise may include one or more memory elements suitable to the particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives, and/or one or more drives that handle removable media such as compact disks (CD), flash memory cards, digital video disk (DVD), and the like.
The computer system 100 may also include a display device 110, one or more loudspeakers, a keyboard 120 and/or one or more controllers or user interaction devices. User interaction devices may be or include a mouse or, as illustrated, a trackpad 130 with buttons 132 and 134, a trackball, a touch screen, a joystick, a voice-recognition device, or any other device that permits a system user to input information into and/or receive information from, the computer system 100.
As illustrated in
The microscope 300 is shown to comprise, among others, a microscope housing 310, a stage 320 on which a sample 20 may be placed, a focus adjustment knob 330, a transmitted light illumination unit 340, at least one objective or lens 350, a tube 360 with an eyepiece or eyepiece set 370, a camera or detection unit 380, and an incident illumination 390 unit which is illustrated, without limitation, to comprise different light sources 392 to 396. Light of illumination unit 390 is, as illustrated with a dotted line, coupled into a beam path, which is illustrated with a dashed line, with a beam splitter 398. Further components of an embodiment of a microscope 300 are shown in
The computer system 100 and the interface unit 200 may be referred to as a microscope control arrangement 10, but this term, as used herein, is not limited by it comprising a computer and an interface unit 200. The term “microscope control arrangement”, as used herein, is to be understood functionally and to refer to a unit or group of units comprising one or more processors 140 and one or more storage devices 150 provided in a computer system 100 or otherwise and being configured to render a graphical user interface 1000 as described above and further illustrated below in embodiments.
As mentioned, operation software for corresponding microscope systems 1 and microscopes 300 conventionally requires a user to adjust, using different user interface elements, a large number of illumination and detection settings, each of them typically allowing for a specific and direct adjustment of the elements used, such as a light source and a detector chip, respectively. In conventional user interfaces, a computer control may be used in which certain settings are coupled, but a user still generally has to have detailed knowledge of the technical background of each of the settings and their effect on the result achieved.
Embodiments of the present invention, in contrast, include a change in an operating paradigm of a microscope system 1 and a different approach for operation which, in an embodiment, is a workflow-based approach. At the centre of the new approach there may be fluorophore information, which may be part of a so-called sample definition. This is the first step in the workflow-based approach. The sample definition may be given by a user and, in the sample definition, the user may describe the nature of the sample to be examined. The sample definition may include a definition of a sample carrier (slide, petri dish, well plate) and its dimensions (shape, diameter, etc.) and characteristics (adherent, fixed, live, etc.). Fluorophore information may include specifications of the dyes (stains or fluorophore) the sample is provided with and optionally their names, chemical data, shelf-life, sensitivity to light, etc. Via said fluorophore information, the microscope system 1 is informed about the spectra to be used, i.e. about excitation and emission wavelengths.
Thus, for the first time, according to embodiments of the present invention, the fluorophore is central to the operation of a microscope system 1 instead of conventional (direct) hardware settings. This fluorophore-based approach allows that, as compared to conventional systems, this information can be used to offer pre-sets and automations to the user. This results in a completely new way of operating fluorescence applications. With only a few user interface controls, in embodiments of the present invention, a lean and intuitive user interface can be offered to the user. The system, in embodiments, may also be configured to automatically locate features in the sample, such as cells, and to automatically identify optimum illumination settings. The user can fully concentrate on the questions the microscopic examination is intended to answer and does not need to have expert knowledge about technical parameters and their interaction for a properly illuminated sample.
With the user interface provided according to embodiments of the present invention, new user groups may be addressed, as users are, in embodiments of the present invention, required to have little or no knowledge the underlying technology due to “smart controls”. A new approach to fluorescence operation is suggested for the first time, which also applies for fluorescence microscopes operable in different operation modes, e.g. a wide-field (camera) and confocal (scanner/detector) mode.
The first detection unit 380a is, in the example shown in
The second detection unit 380b is, in the example shown in
As evident from the sheer number of components present in a microscope operable in different microscopy operation modes, such as the microscope 300 as illustrated in
In other words, the difficulties described before for adjusting illumination and detector parameters for a single fluorophore and/or operation mode become even more pronounced in microscopes which allow a user to select between different microscopy operation modes. In such microscopes, as explained, the operation concepts in the operation modes are substantially different from each other. Settings influencing the images obtained in these operation modes include, for example, illumination settings of wide-field light sources (area sensors or “cameras”) versus illumination settings for scanning light sources and components of the respective illumination beam paths, and detection settings for an area detector versus detection settings for components such line or point detectors including components of the respective detection beam paths. Each of these components may generally, if adjustable, may have an influence on the image results and must therefore be individually and carefully adjusted.
Embodiments of the present invention allow for a unified control of different microscopy operation modes in a unified operation concept which has not yet been realized before due to the different devices, components and their parameters involved. Automatisms, mathematical models and combination of functions, in embodiments of the present invention, may help to share common control elements or widgets between the different operation modes. Appropriate settings for each of the operation modes transparently differentiated in the background without further user interaction.
While a user of a microscope 300 in each of the different operation modes conventionally requires a user to “switch” conceptually and mentally between generally different operation and interaction concepts, this is not required according to embodiments of the present invention. Particularly for users unexperienced in one of the operation modes, or in more stressful observation situations, such as when observing moving samples, embodiments of the present invention overcome considerable hurdles particularly for unexperienced users and reduces distraction from the actual task of observation and examination of a sample.
In other words, according to embodiments of the invention, an advantageous operation concept abstracting an operation of a microscope, even in different operation modes, from the underlying technical details is realized. According to embodiments of the present invention, a target-oriented operation may be realized, i.e. the operation concept may start from or may be centred on the fluorophores used and results to be achieved such as an image resolution, an image size, and an exposure time or rather an acquisition speed. Generally, embodiments of the present invention ensure that a user needs to know generally less about the technology underlying the different operation modes but may focus on actual operation or the quality and parameters of microscopic images. As mentioned, embodiments of the present invention are not limited to wide-field and confocal operation even if described hereinbelow with as focus on these specific microscopy operation modes.
In conventional concepts for operating a fluorescence microscope 300, a user still has to have detailed knowledge of the technical background of each of the operation modes and what their effect to the result achieved is. As mentioned, a user may even be required a user to “switch” conceptually and mentally between generally different operation and interaction concepts in conventional arrangements. This problem is overcome according to embodiments of the present invention, in which, as will now be further explained, a fluorophore-based control of a microscope (system), particularly in different microscopy operation modes in a unified operation concept, is realized.
In other words, according to embodiments of the invention, an advantageous operation concept abstracting an operation of a microscope, particularly in different operation modes, from the underlying technical details is realized. In such a concept, a user interface 1000 as generally illustrated in
As illustrated in
Embodiments of the present invention are not in any way limited by the number or the images 1201 to 1204 displayed and their origin, manner of processing, or source. For example, images 1201 to 1204 may also be displayed in pseudo-colour(s) or, when captured as greyscale images, as (pseudo-) coloured representations. For example, images 1201 to 1204 may be displayed in colours corresponding to a peak of a fluorescence emission spectrum in a fluorescence channel in which the images 1201 to 1204 are captured, or a display colour may be freely selected by a user of a microscope system 1 or the microscope control arrangement 10, for example in order to differentiate them in an overlay image.
Images captured in fluorescence microscopes may be monochrome (greyscale) and may be coloured according to the wavelength and displayed in the display panel 1200. Thus an overlay of the single channels with different colours is possible. If the user uses fluorophores that are very close to each other in the sample (e.g. Alexa 568 and Alexa 594 are both orange fluorophores), the structures in the overlay from the two channels are natively detected in the same colour. Here one could like to have a contrast as large as possible, in order to be able to distinguish both fluorophores better. To solve this problem, a user may assign a different display colour to one or both fluorophores.
Images 1201 to 1204 may be displayed as static images 1201 to 1204 or as motion images 1201 to 1204 or a mixture of static and motion images 1201 to 1204, and the user interface may be configured to switch between a static and motion view of images 1201 to 1204, for example to take “snapshots” of a moving sample, in predefined time intervals or in response to a request of a microscope user, e.g. via a widget of the user interface 1000. A mixed view of static images 1201 to 1204 and motion images 1201 to 1204 at the same time may also generally be provided according to embodiments of the present invention, for example to be able to visually track a motion of a sample 20 or a component of a sample 20 and in parallel to inspect the sample 20 or the component of the sample 20 in detail.
Aspects of embodiments of the present invention may include acquiring and/or displaying images 1201 to 1204 such as shown in
This is realized, in embodiments of the present invention, and as illustrated in
To this purpose, each of the fluorophore control widgets 1110, 1120, 1130, 1140 is provided to correspond to the or one of the fluorophores indicated by the fluorophore information. The number of control widgets 1110, 1120, 1130, 1140 is dynamic and depends on the number of fluorophores indicated and specified by a user, for example in sample information in which such fluorophore information are provided in addition to other sample parameters as indicated above. Alternatively, fluorophore information may be provided by a user separately from further sample information or without sample information. Fluorophore information may, whether or not being provided in addition to or together with sample information, and without limitation, include at least one of a name of the fluorophore, an excitation wavelength or wavelength range usable for exciting the fluorophore, and an emission wavelength or wavelength range characterizing a fluorescence response of the fluorophore. Wavelength ranges may be provided in the form of a centre wavelength and a bandwidth of a wavelength band. Fluorophore information may also comprise information on a chemical or physical stability of a fluorophore. For example, and without limitation, one, two, three, four or five fluorophores may be specified in fluorophore information, the number corresponding to, or being smaller than, a number of detection channels (i.e. camera chips, area detectors, scanning detectors, etc.) of the microscope 300.
The control widgets 1110, 1120, 1130, 1140 are provided to provide a user with information on, and an interaction possibility with, the excitation and the detection of the fluorophores they correspond to. The control widgets 1110, 1120, 1130, 1140 may optionally substitute conventional information widgets information means of a graphical or non-graphical user interface conventionally used to provide information on illumination settings, a fluorescence response, and parameters set in a corresponding microscope, and they may optionally substitute adjustment means for such an illumination and detection.
The control widgets 1110, 1120, 1130, 1140 are control elements that allow a user to make all settings relating to different fluorophore with a small number of interaction steps, such as a few mouse clicks. Without expert knowledge, the user can set up an experiment for a specific problem using the control widgets 1110, 1120, 1130, 1140. In doing so, the user may proceed much faster to obtain meaningful results as compared to conventional systems. This time saving applies to both wide-field and confocal microscopy, if a corresponding microscope 300 is configured to be operated in these modes. On the confocal side, the savings are even greater, because mathematical models stored for this mode of operation take intrinsically more parameters into account, which users no longer have to set their self and for each component concerned.
The control widgets 1110, 1120, 1130, 1140 may thus be used for different microscopy operation modes or detection types, and a user interaction with the control widgets 1110, 1120, 1130, 1140 may be translated to either of the operation modes or detection types, realizing a common control concept via the user interface 1000. Aspects of embodiments of the present invention therefore include setting parameters for different microscopy operation modes, such as a wide-field and a confocal mode, using the same user interface widgets and transparently translating corresponding user inputs to specific settings for either of the operation modes, as further explained below.
As illustrated in
The fluorophore control widgets 1110, 1120, 1130, 1140 may be configured to provide the second widget zone 1112, 1122, 1132, 1142 in a manner which informs the user of an illumination intensity, e.g. of a light source 392, 394, 396 such as a light emitting diode or a laser, which is used for illuminating the sample and exciting the corresponding fluorophore. The user may therefore, without distraction and without e.g. selecting a different window or part of the user interface, be informed at any time on the illumination intensity and in this way be warned of a too high and/or extended illumination to avoid damaging the fluorophore. Parameters of displaying the illumination intensity may also be adjusted on the basis of the fluorophore information, e.g. taking into account the stability of the fluorophore.
In an embodiment illustrated in
When the second widget zone 1114, 1124, 1134, 1144 is rendered as a rim surrounding the first widget zone 1112, 1122, 1132, 1142, a brighter part of the rim may correspond to the illumination intensity in the form of a percentage of a maximum illumination intensity and a darker part may be corresponding to a remainder. The brighter part is displayed in white in
In the example illustrated in
In fluorescence microscopy, especially with living specimens, the avoidance or at least reduction of the bleaching effect plays a very important role. With conventional systems, the user has to adjust the light manually. Embodiments of the present invention may relieve the user of this task. Controls for adjusting and balancing or laser intensities may longer exist. An auto-illumination procedure may be used to calculate the optimal light intensities. However, since many different combinations of fluorophores in a sample may be present, very different dynamics in intensity between different wavelengths may be encountered. For this reason, it is of great interest to the user to get a feedback of the light exposure of his sample. The second widget zone 1114, 1124, 1134, 1144, in the way discussed, may be used for this purpose. The second widget zone 1114, 1124, 1134, 1144 shows, per fluorophore, the illumination intensity, e.g. of the main excitation laser or light emitting diode, for example in relation to the used attenuator. Thus, the user has direct feedback for the used intensities and can at each step of an experiment evaluate the light exposure of the sample with only a short look at the fluorophore control widgets 1110, 1120, 1130, 1140 and take corrective action if necessary.
A colour of the second widget zone 1114, 1124, 1134, 1144, or more precisely the part thereof indicating the illumination intensity, may be rendered in a colour relating to, or resembling the, excitation wavelength used for the corresponding fluorophore, according to an embodiment of the present invention.
Further control widgets 1151, 1152, 1161, 1162 may be provided, which may be adapted to start an experiment or an acquisition of data and a fast acquisition mode or any other parameters relating to an experiment.
As illustrated in
In more general terms, the microscope control arrangement 10 the user interface 1000 is rendered with is configured to toggle, in response to the user interaction in one of the interaction modes, at least one of an illumination for exciting the corresponding fluorophore, use of fluorophore information relating to the corresponding fluorophore in an un-mixing method determining contributions of the fluorophore in a common fluorescence response, and displaying an image obtained on the basis of a fluorescence response of the corresponding fluorophore. As mentioned, according to the embodiment illustrated here, a plurality of settings is toggled with a single operation.
When a user is searching for an interesting location in three dimensions in a sample, the system must be set to live mode and the sample must be illuminated. Here it is important to protect the sample from illumination to the maximum extent possible for the duration of the search. It is not always necessary to illuminate all fluorophores for this procedure. This makes it possible to switch off all light sources that are not needed. By left-clicking the fluorophore control widgets 1110, 1120, 1130, 1140 (or any other way of interaction defined), it is possible to deactivate or activate fluorophores which is equivalent to indirectly switching on or off the corresponding light sources, but in a more user-friendly manner. Switching forth and back between activation and deactivation is substantially easier. A switched off fluorophore may be represented by a fluorophore control widget 1110, 1120 in grey or in a faded representation. If the corresponding light source it is activated, the first widget zone 1112, 1122, 1132, 1142 may be rendered in a colour corresponding to, or selected on the basis of, the emission colour or any other colour defined by a user for a display colour in the display panel 1200, for example a (pseudo-) colour as mentioned above.
By interacting with the fluorophore control widgets 1110, 1120, 1130, 1140 in a specific interaction mode, for example by right-clicking, the colour used in the display panel 1200 for an image of the corresponding fluorophore may be changed, as explained for fluorophores in colours which are very close to each other in the sample. If a user selects a new colour, the first widget zone 1112, 1122, 1132, 1142 may be rendered accordingly.
In an embodiment of the present invention, selecting and deselecting fluorophores by acting upon the fluorophore control widgets 1110, 1120, 1130, 1140 may initiate a new auto-illumination procedure because the illumination scenario has changed, e.g. a cross-excitation of other fluorophores is reduced.
In practice, there are combinations of fluorophores whose spectra overlap unfavourably and un-mixing thereof is not possible. Instead of excluding such combinations for a microscope system 1, a sequential acquisition can be used as a fall-back. By a drag-and-drop operation, the user can, in an embodiment of the invention illustrated in
To this purpose, a microscope control arrangement 10 provided according to an embodiment of the invention may be configured to reposition the fluorophore control widgets 1110, 1120, 1130, 1140 in the graphical user interface 1000 upon determining an interaction in one of the interaction modes. In the example illustrated, the excitation and detection of the fluorophores is determined to be performed sequentially or in parallel on the basis of a state of a grouping of the corresponding fluorophore control widgets 1110, 1120, 1130, 1140 in the graphical user interface 1100. As shown in
Obviously, further sequential excitation and detection can be configured by providing further user interface regions such as the interface regions 1101, 1102 and in each of the interface regions 1101, 1102 any number of fluorophore control widgets 1110, 1120, 1130, 1140 may be grouped in order to demand a parallel excitation and detection.
Auto-illumination provides the user with an image with an improved or optimized signal-to-noise ratio in relation to the current lighting environment. However, the image impression can be subjectively evaluated very differently by a user. To enable a user a leeway in this connection, a fine adjustment of the auto-illumination may be provided in an embodiment of the present invention as a response to a user interaction with the fluorophore control widgets 1110, 1120, 1130, 1140 in a user interaction mode such as a mouse-over operation.
In other words, by providing a fine-tuning panel 1126 comprising suitable means according to such an embodiment, the image impression can be adjusted to the user's personal preferences. If the user moves, in other words, the mouse over one of the fluorophore control widgets 1110, 1120, 1130, 1140, a slider or slider set may be displayed after a certain time, e.g. for a few seconds, which can be used for fine-tuning. An “optimize” button may be provided to restart auto-illumination and the new parameters may be incorporated into the image. Slider values may, in an embodiment, also be changed via a mouse wheel.
In more general terms, a microscope control arrangement 10 used herein may be configured to render a fine-tuning panel 1126 in response to the interaction in one of the interaction modes. And the microscope control arrangement 10 may be configured to modify the control parameters on the basis of a user input received using the fine-tuning panel 1126, in particular control parameters relating to an auto-illumination setting.
As used herein the term “and/or” includes any and all combinations of one or more of the associated listed items and may be abbreviated as “/”.
Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.
Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a non-transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
Some embodiments of the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.
Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
A further embodiment of the present invention is, therefore, a storage medium (or a data carrier, or a computer-readable medium) comprising, stored thereon, the computer program for performing one of the methods described herein when it is performed by a processor. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary. A further embodiment of the present invention is an apparatus as described herein comprising a processor and the storage medium.
A further embodiment of the invention is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.
A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.
A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
A further embodiment of the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may e.g. comprise a file server for transferring the computer program to the receiver.
In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.
While subject matter of the present disclosure has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. Any statement made herein characterizing the invention is also to be considered illustrative or exemplary and not restrictive as the invention is defined by the claims. It will be understood that changes and modifications may be made, by those of ordinary skill in the art, within the scope of the following claims, which may include any combination of features from different embodiments described above.
The terms used in the claims should be construed to have the broadest reasonable interpretation consistent with the foregoing description. For example, the use of the article “a” or “the” in introducing an element should not be interpreted as being exclusive of a plurality of elements. Likewise, the recitation of “or” should be interpreted as being inclusive, such that the recitation of “A or B” is not exclusive of “A and B,” unless it is clear from the context or the foregoing description that only one of A and B is intended. Further, the recitation of “at least one of A, B and C” should be interpreted as one or more of a group of elements consisting of A, B and C, and should not be interpreted as requiring at least one of each of the listed elements A, B and C, regardless of whether A, B and C are related as categories or otherwise. Moreover, the recitation of “A, B and/or C” or “at least one of A, B or C” should be interpreted as including any singular entity from the listed elements, e.g., A, any subset from the listed elements, e.g., A and B, or the entire list of elements A, B and C.
This application is a U.S. National Phase application under 35 U.S.C. § 371 of International Application No. PCT/EP2021/084389, filed on Dec. 6, 2021. The International Application was published in English on Jun. 15, 2023 as WO 2023/104279 A1 under PCT Article 21(2).
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/EP2021/084389 | 12/6/2021 | WO |