Patent Application
20230316573
The invention relates to a method for operating a medical microscope, and to a medical microscope arrangement.
One aim of digital visualization in surgery and microsurgery is to display to a surgeon, during an operation, an optimum three-dimensional image of an operation site and, if appropriate, an optimum three-dimensional superimposition of additional information onto a field of view on display devices (e.g., monitors or head-mounted display, etc.) of the medical microscope. The digital calibration data used to optimize the camera images should always be kept current in order to avoid a deterioration in the three-dimensional visualization. An image representation resulting from this, which is perceived as uncomfortable, may lead to the surgeon becoming frustrated and also tired.
To keep the calibration data current, a recalibration of the medical microscope in the field, that is to say at an application or operation location of the medical microscope, for example carried out by a service technician, is mandatory. The reason for this being that the calibration data are valid for a specific state of an imaging optical unit and, optionally, further optical components of the medical microscope. An interchange of a component of the medical microscope or arising vibrations and displacements (misalignment) of the components connected therewith however cause a deterioration in the effect of the calibration data, or even render the latter invalid. Depending on the type of effect, a more or less comprehensive recalibration (with a fewer or greater number of operating points) must be carried out in this case.
Methods for the extrinsic recalibration of cameras of stereoscopic surgical microscopes in the field with the aid of a defined calibration object (which is also referred to as target) and for various operating points are known. Such a calibration method is known from DE 10 2019 131 646 A1, for example. To simplify the optical calibration of optical observation equipment, a stand for an optical observation unit comprising a calibration object arranged directly on the stand in a fixed location is used in that case. Moreover, optical observation equipment, which comprise such a stand and an optical observation unit connected to the stand, a method for calibrating such optical observation equipment, and a computer program are described. A method for calibrating a digital microscope is further known from DE 10 2013 012 987 A1.
The invention is based on the object of developing a method for operating a medical microscope and a medical microscope arrangement, by means of which a recalibration can be carried out more easily, in particular more efficiently, in particular in the field, following an interchange of one or more components of the medical microscope and/or a misalignment (an unwanted change of pose) of one or more components. In this case, a misalignment refers in particular to deviation of a pose (position and/or orientation) of a component from a target pose, for example because the pose of the component has changed as a result of a vibration or other influences (influence of temperature, mechanical tolerances, etc.).
According to the invention, the object is achieved by a method having the features of patent claim 1, a medical microscope arrangement having the features of patent claim 14, and a calibration object having the features of patent claim 15. Advantageous configurations of the invention emerge from the dependent claims.
One of the basic concepts of the invention lies in the fact that an interchange by a service technician and/or a misalignment, in general a change, of at least one component of the medical microscope, for example a varifocal objective or a camera, is followed by the interchanged and/or misaligned at least one component being identified. In particular, this should ensure that all interchanged and/or misaligned components are known to the subsequent measures and are able to be taken into account. In this case, an interchange and/or a misalignment should, in general terms, in particular denote a change in at least one component. Required adjustment measures and/or calibration measures are determined in automated fashion using the identified at least one interchanged and/or misaligned component as a starting point. In particular, this is implemented by means of a data processing device which may be part of the medical microscope or else may be formed separately therefrom as a part of the medical microscope arrangement. The determined required adjustment measures and/or calibration measures are subsequently carried out. The automated determination of the required adjustment measures and/or calibration measures allows a service technician in the field to be provided with valuable information for assisting with the service operation. This can avoid errors in the recalibration and an incomplete recalibration. Further, it is possible in particular to reduce the time required to carry out the recalibration since the recalibration can be restricted to the required adjustment measures and/or calibration measures.
In particular, a method for operating a medical microscope, in particular in the field, is provided, with an interchange and/or a misalignment of at least one component of the medical microscope being followed by an identification of the interchanged and/or misaligned at least one component, with required adjustment measures and/or calibration measures being determined in automated fashion using the identified at least one interchanged and/or misaligned component as a starting point, and with the determined required adjustment measures and/or calibration measures being carried out.
Further, a medical microscope arrangement, in particular, is developed, comprising a medical microscope with storable calibration data and a data processing device, the data processing device being configured, following an interchange and/or a misalignment of at least one component of the medical microscope, to identify the interchanged and/or misaligned at least one component and/or to receive identifying information in relation to the interchanged and/or misaligned at least one component, to determine required adjustment measures and/or calibration measures in automated fashion using the identified at least one interchanged and/or misaligned component as a starting point, and to cause the determined required adjustment measures and/or calibration measures to be carried out.
One of the advantages of the method and the medical microscope arrangement is that these consider the challenges arising during a calibration in the field. Thus, calibration data of medical microscopes are usually very comprehensive and, for example, further calibration steps are required in addition to an extrinsic calibration of cameras of the medical microscope in order to be able to restore a complete calibration data set. While the required equipment and processes for complex calibration are available at the manufacturer, it generally is very difficult for a service technician to carry out an optimal recalibration in the field, that is to say at the application or operation location of the medical microscope. The same applies to required adjustment measures. Reasons for this include, in particular, that the technician firstly lacks knowledge about the required measures dependent on the implemented service operation and secondly lacks tools optimized for the overall process.
A further advantage is that, moreover, a downtime and, therewith, a duration of the service operation on the medical microscope can be minimized.
A medical microscope is a surgical microscope in particular. However, a medical microscope may also be a microscope used for medical examinations and/or for diagnostic purposes, for example in the field of ophthalmology or in other fields. A medical microscope arrangement is a surgical microscope arrangement in particular. The medical microscope is a digital microscope and/or a stereoscopic medical microscope in particular. One component of the medical microscope is an optical component in particular, that is to say in particular a component that influences an optical imaging and/or capture procedure in the medical microscope. In a manner known per se, the medical microscope comprises, in particular, a control device for controlling the components, an input device for operation, and a display device for displaying captured image representations.
The data processing device may also be part of the medical microscope, for example part of a control device of the medical microscope. However, in principle the data processing device may also be formed separately from the medical microscope, for example as a desktop, laptop or tablet computer, or else in a cloud-based solution.
The calibration measures may in particular comprise one or more of the following measures: an extrinsic calibration of cameras, an intrinsic calibration of cameras, a distortion correction, a trimming correction (camera vignetting and/or illumination geometry), a chromatic aberration correction, an offset correction, a rotation correction, etc.
Parts of the medical microscope arrangement, in particular the data processing device, can be designed, either individually or together, as a combination of hardware and software, for example as program code that is executed on a microcontroller or microprocessor. However, provision can also be made for parts to be designed as application-specific integrated circuits (ASICs) and/or field-programmable gate arrays (FPGAs), either on their own or in combination.
In an embodiment, provision is made for a calibration object with two-dimensional features and/or three-dimensional features to be used to carry out the adjustment measures and/or calibration measures and for said calibration object to be arranged in a capture region of the medical microscope to this end. This can simplify the recalibration since, in particular, all required adjustment measures and/or calibration measures can be carried out with the aid of the one calibration object. In particular, this can avoid changing calibration objects for different measures. In particular, the calibration object is a combined calibration object, which combines a two-dimensional calibration object with two-dimensional features and a three-dimensional calibration object with three-dimensional features.
Therefore, a calibration object comprising two-dimensional features and three-dimensional features is also developed, with the two-dimensional features comprising a plane with two-dimensional structures, and the three-dimensional features comprising a plane that is inclined in relation to the plane with the two-dimensional structures. By way of example, the two-dimensional structures may comprise a checkerboard pattern, ChArUco structures, a grid with anchor points, and/or similar, uniquely identifiable geometric structures. The three-dimensional features may also be designed completely differently.
In an embodiment, provision is made for the identification of the interchanged and/or misaligned at least one component to be implemented at least partially manually, with at least one information item identifying the at least one interchanged and/or misaligned component being collected to this end by means of an operating device by way of a manual input. As a result, the interchanged and/or misaligned at least one component can be identified directly by a service technician by way of a manual input. The operating device is, in particular, an operating device of the data processing device and/or medical microscope. By way of example, provision can be made in this context for serial numbers of the interchanged and/or misaligned at least one component to be collected. However, provision can also be made for the interchanged and/or misaligned at least one component to be identified on the operating device with the aid of selectable symbols.
In an embodiment, provision is made for the identification of the interchanged and/or misaligned at least one component to be implemented at least partially in automated fashion, with at least one information item identifying the at least one interchanged and/or misaligned component being read and/or retrieved to this end out of the interchanged and/or misaligned component and/or from at least one control device of the medical microscope and/or from a service database. As a result, the identification can be implemented in automated fashion. As a result, the method can be implemented in accelerated fashion and (manual) incorrect inputs can be reduced or even prevented. By way of example, the identifying information can be a serial number or a unique identification number, which are retrieved and/or read directly from the interchanged and/or misaligned at least one component and/or from a control device which controls the interchanged and/or misaligned at least one component, and/or from a service database in which a service task to be carried out, which describes the components to be interchanged and/or the misaligned components, is stored. By way of example, the identifying information can be retrieved and/or read from an EEPROM or flash memory. In particular, the medical microscope is configured to recognize whether components were interchanged and/or have become misaligned. Following an interchange, this can be implemented, for example, by a comparison between a serial number and/or a unique identification number, which is stored in a memory of the medical microscope for the (old) component, and a read and/or retrieved serial number and/or unique identification number of the interchanged (new) component. If the comparison yields that a component has been interchanged, the identifying information is read and provided.
In an embodiment, provision is made for the required adjustment measures and/or calibration measures to be at least partially determined by means of a lookup table, with adjustment measures and/or calibration measures that are required in each case following the interchange and/or misalignment being stored in the lookup table for interchangeable and/or adjustable components of the medical microscope. As a result, the required adjustment measures and/or calibration measures can be determined in a particularly time-saving and efficient manner. By way of example, the lookup table may be stored in a memory of the data processing device. Entries in the lookup table are or were determined, in particular, by means of expert knowledge and/or empirical trials and/or simulations. In this case, provision can be made for this to be followed by an optimization of the entries by means of statistical evaluations. In particular, every possible service operation is reconstructed and/or simulated in order to determine which adjustment measures and/or calibration measures are required. To this end, effects arising by the interchange of the at least one component, in particular, are determined by expert knowledge and, if required, by empirical trials and/or simulations. The required adjustment measures and/or calibration measures are determined on the basis thereof. In the process, the measures may also be trialed, in particular, and the effectiveness and/or necessity thereof may be evaluated statistically. The determined measures are then stored in the lookup table for the respective interchanged and/or misaligned at least one component that is the focus of the service operation or for combinations of components.
By way of example, if a camera of the medical microscope is interchanged, consideration of the “camera interchange” scenario yields that the interchanged camera initially needs to be adjusted mechanically with the aid of the (combined) calibration object and subsequently still needs to be (digitally) calibrated with the aid of the calibration object. In particular, this only needs to be implemented for one operating point (magnification and focus). Further empirical trials and/or a simulation are not required any more since these measures can be derived from the expert knowledge. Naturally, empirical trials and/or simulations may nevertheless be carried out. By way of example, the following measures are then stored in the lookup table: arrange calibration object in the capture region of the medical microscope, mechanically adjust the interchanged camera (focus), mechanically adjust the camera (x/y-direction and rotation), (digital) calibration (x/y-direction and rotation).
By way of example, if a varifocal objective of the medical microscope is interchanged, a consideration of the “varifocal objective interchange” scenario yields that offset values that serve for an image stabilization for travel of the varifocal objective have to be updated (recalibrated). The offset values are usually available for a given number of support points/varifocal objective positions. Empirical trials are used to determine the number of support points at which the offset values need to be redetermined and at which points an interpolation of the offset value is sufficient for a given accuracy. By way of example, if offset values from an initial assembly of the (old) varifocal objective are available for ten positions and the trials yield that a remeasurement is only required for every second position since the intermediate values can be interpolated, this yields that five positions have to be remeasured by calibration measures. By way of example, the following measures are then stored in the lookup table: arrange calibration object in the capture region of the medical microscope (if this has not yet been implemented), home in on the five positions of the varifocal objective and measure the respective offset values.
Provision can be made for an interchange and/or misalignment of a plurality of components to be followed by the combination of measures for these components. By way of example, an appropriate logic may be stored to this end. If, for example, both a camera and a varifocal objective are interchanged using the above-described examples as a starting point, it is possible to at least partially combine the adjustment measures and/or calibration measures so that a number and/or scope of the measures can be reduced.
In an embodiment, provision is made for the required adjustment measures and/or calibration measures to be at least partially retrieved from a database, with adjustment measures and/or calibration measures that are required in each case following the interchange and/or misalignment being stored in the database for interchangeable and/or adjustable components of the medical microscope. The entries in the database can be generated in a manner analogous to what was described above for the lookup table. By way of example, the database may be stored in a memory of the data processing device. However, in principle the database may also be stored on a central or decentralized (e.g., cloud-based) server, with, by way of a communications link configured to this end, the database being retrieved from the central server when required by means of the data processing device.
In an embodiment, provision is made for the required adjustment measures and/or calibration measures to be at least partially determined by means of a trained artificial intelligence method. As a result, it is possible, in particular, to also determine the measures for combinations of interchanged and/or misaligned components that are not stored in a lookup table or database, or for combinations for which no measures were determined in advance. In particular, the measures are estimated by means of the trained artificial intelligence method. Methods known per se, for example pattern recognition methods and artificial neural networks, can be used to this end. In particular, the machine learning method is trained for a multiplicity of scenarios, that is to say for a multiplicity of interchanged and/or misaligned components and the combinations thereof. In this case, respective pairs of a component or a combination of a plurality of components and adjustment measures and calibration measures assigned thereto, in particular, are generated and used as training data. The training data can be generated analogously, by definition on the basis of expert knowledge, by empirical trials and/or by simulation, as has already been described above for the lookup table and the database. The artificial intelligence method, for example a suitable neural network, is then trained using these training data. A structure and parameters of the trained artificial intelligence method, for example the trained neural network, are then stored in a memory of the data processing device such that the trained artificial intelligence method can subsequently be provided and used by the data processing device.
In an embodiment, provision is made for the determined required adjustment measures and/or calibration measures to be displayed by means of a display device (for example by means of a (computer) monitor or a head-mounted display). As a result, a service technician can be provided with instructions regarding the measures to be carried out following the interchange and/or misalignment of the at least one component. In this context, provision can for example be made for the service technician to be guided step by step through the required adjustment measures and/or calibration measures with the aid of a graphical user interface, and for the service technician to receive detailed instructions as a result. Such instructions can also be enriched with multimedia content, for example explanatory videos and/or voice information. By way of example, such instructions can be prepared for all possible adjustment measures and/or calibration measures and can be stored in the data processing device, and so said instructions can be retrieved and displayed according to requirements.
In an embodiment, provision is made for at least some of the determined required calibration measures to be carried out in automated fashion. This can unburden a service technician. Further, servicing or repair work and the subsequent implementation of the required adjustment measures and/or calibration measures may optionally also be carried out without a service technician by a user of the medical microscope themselves. In that case, the calibration measures are carried out, in particular, by automated control and/or closed-loop control of actuator systems of the medical microscope for the purposes of homing in on required operating points, an implementation of required measurements, and a subsequent determination and adjustment of calibration data. In this case, the calibration measures in particular also comprise an adjustment of calibration data for signal or image processing of a camera system of the medical microscope.
In an embodiment, provision is made for at least one calibration object to be arranged in automated fashion in a capture region by means of an actuator system in order to carry out the determined required adjustment measures and/or calibration measures. As a result, the arrangement of the calibration object in the capture region can also be carried out in automated fashion. In particular, the at least one calibration object is a calibration object having both two-dimensional features and three-dimensional features.
Alternatively or additionally, provision can also be made for the medical microscope to be controlled so that a capture region of the medical microscope captures at least one calibration object in order to carry out the determined required adjustment measures and/or calibration measures. By way of example, a (robotic) actuator system of a stand of the medical microscope can be controlled in such a way that a calibration object arranged at the medical microscope, for example at the stand of the medical microscope, is captured by the capture region of the medical microscope as a result of arranging/aligning the medical microscope.
In an embodiment, provision is made for the required adjustment measures and/or calibration measures to be combined and/or optimized with one another in the case of a plurality of interchanged and/or misaligned components, in order to minimize a number and/or a scope of the measures. This can reduce the time required to carry out the adjustment measures and/or calibration measures. The combination allows a reduction of measures which would otherwise be carried out twice or whose operating points partially overlap. A number of operating points required for the calibration can likewise be reduced within the scope of the optimization, for example because calibration data can be obtained by interpolation and the associated operating points therefore do not have to be measured. Provision can likewise be made for an optimal sequence for homing in on the individual operating points to be determined and provided within the scope of the combination and/or optimization, the optimal sequence for example having a minimum value for an accumulated travel during the adjustment of the operating points or the shortest duration.
In an embodiment, provision is made for data for anticipatory servicing to be collected and/or recorded in the medical microscope, with data for determining and/or optimizing the determination of the adjustment measures and/or calibration measures being evaluated, said data having been collected before, during and/or after the interchange and/or the misalignment of the at least one component. If these data are collected for a multiplicity of medical microscopes in the field, optimal measures can be identified on the basis of these data. As a result, optimized adjustment measures and/or calibration measures can be obtained. In particular, an effect of the interchange and/or the misalignment and the subsequent adjustment measures and/or calibration measures can be assessed and taken into account when defining the respective adjustment measures and/or calibration measures in the lookup table and/or in the database, or when training the artificial intelligence.
In an embodiment, provision is made for a respective state of the medical microscope to be captured before and after the interchange and/or misalignment, with associated image representations of at least one calibration object being captured by means of at least one capture device of the medical microscope and with the required adjustment measures and/or calibration measures being determined by means of a trained artificial intelligence method, with the states captured before and after the interchange and/or the misalignment and the associated captured image representations being taken into account. The artificial intelligence method is learned in advance, with training data being generated by virtue of the calibration data determined by the calibration being systematically modified and respective image pairs (before and after the interchange or misalignment) being captured. This can be implemented either manually on the medical microscope or by simulation. The artificial intelligence method, for example a neural network, is trained with the aid of the image pairs generated in this way and the respective modification of the calibration data accompanying this (basic truth). Subsequently, a structure and parameters of the trained artificial intelligence method can be stored for application purposes in a memory of the data processing device, and so the trained method can then be provided for application purposes. This embodiment can be used, in particular, also in assistive fashion or in combination with the other described embodiments.
Further features relating to the configuration of the medical microscope arrangement arise from the description of configurations of the method. Here, the advantages of the medical microscope arrangement are in each case the same as in the configurations of the method.
The invention is explained in greater detail below on the basis of preferred exemplary embodiments with reference to the figures. In the figures:
The medical microscope 1 comprises a stereo camera system 3, which comprises a left camera 31 and a right camera 3r. The cameras 31, 3r capture a capture region imaged by way of an imaging optical unit 4 (only illustrated schematically here) of the medical microscope 1. To correct errors in an adjustment in particular, calibration data 30, for example for an optical unit actuator system 5 configured to change properties of the imaging optical unit 4, can be stored for the imaging optical unit 4.
The imaging optical unit 4 or parts thereof and the cameras 31, 3r of the stereo camera system 3 form components 8 of the medical microscope 1. In particular, these components 8 are interchangeable, that is to say the components 8 can be swapped by way of a service operation, for example if said components are defective and no longer operate correctly. The term interchangeable should in particular also comprise a component 8 being removed from the medical microscope 1, being cleaned and/or serviced, and subsequently being reinstalled in the medical microscope 1.
Further, the medical microscope 1 comprises a signal processing device 6, which processes a raw signal 101 provided by an image sensor of the left camera 31 and a raw signal 10r provided by an image sensor of the right camera 3r and which generates and provides image representations 20 from the raw signals 101, 10r. By way of example, the image representations 20 can be displayed on at least one display device 7, for example one or more (computer) monitors or a head-mounted display (HMD), which in particular may also be part of the medical microscope 1. The signal processing may comprise, in particular, a picture element-dependent modification of a brightness and/or a generation of an offset and/or a rotation about an image center (or any other point), and further manipulations (filtering, color correction, etc.). Calibration data 30 for correcting the raw signals 101, 10r may be stored in the signal processing device 6. Provision can also be made for the calibration data 30 to be stored only in the image processing device 6.
The data processing device 2 comprises a computing device 2-1, for example a microprocessor or microcontroller, and a memory 2-2. The data processing device 2 may also be a part of the medical microscope 1 and, for example, may be integrated in a control device (not shown here) of the medical microscope 1.
The data processing device 2 is configured, following an interchange and/or a misalignment of at least one component 8 of the medical microscope 1, to identify the interchanged and/or misaligned at least one component 8 and/or to receive identifying information 9 in relation to the interchanged and/or misaligned at least one component 8, to determine required adjustment measures 50 and/or calibration measures 51 in automated fashion using the identified at least one interchanged and/or misaligned component 8 as a starting point, and to cause the determined required adjustment measures 50 and/or calibration measures 51 to be carried out. The adjustment measures 50 and/or calibration measures 51 are then carried out in a manner known per se.
Provision can be made for a calibration object 40 with two-dimensional features 41 and three-dimensional features 42 to be used to carry out the adjustment measures 50 and/or calibration measures 51 and for said calibration object to be arranged in a capture region of the medical microscope 1 to this end. An embodiment of such a calibration object 40 is shown in exemplary fashion in
Provision can be made for the identification of the interchanged and/or misaligned at least one component 8 to be implemented at least partially manually, with at least one information item 9 identifying the at least one interchanged and/or misaligned component being collected to this end by means of an operating device 11 by way of a manual input. By way of example, the operating device 11 may comprise a display and input means (neither of which are shown here). In this case, the operating device 11 may also be part of the medical microscope 1 or data processing device 2. Following the interchange and/or the misalignment of the at least one component 8, a service technician can provide the at least one identifying information item 9 by way of a manual input on the operating device 11. The at least one identifying information item 9 may for example comprise a unique serial number and/or identification number of the at least one interchanged and/or misaligned component 8. The manual input may also comprise scanning a barcode or QR code arranged on the interchanged component 8. In this case, this is already implemented before the component 8 is interchanged, in particular.
Provision can be made for the identification of the interchanged and/or misaligned at least one component 8 to be implemented at least partially in automated fashion, with at least one information item 9 identifying the at least one interchanged and/or misaligned component 8 being read and/or retrieved to this end out of the interchanged and/or misaligned component 8 and/or from at least one control device of the medical microscope 1 and/or from a service database 60. Reading and/or retrieving the at least one interchanged and/or misaligned component 8 is implemented, for example, by way of a communications line or a bus system, by means of which the components 8 of the medical microscope 1 are connected to one another and to the at least one control device.
Provision can be made for the required adjustment measures 50 and/or calibration measures 51 to be at least partially determined by means of a lookup table 52, with adjustment measures 50 and/or calibration measures 51 that are required in each case following the interchange and/or misalignment being stored in the lookup table 52 for interchangeable and/or adjustable components 8 of the medical microscope 1. The lookup table 52 is stored in the memory 2-2 of the data processing device 2. The data processing device 2 then retrieves the required adjustment measures 50 and/or calibration measures 51 from the lookup table 52 on the basis of the identified interchanged and/or misaligned components 8.
Provision can be made for the required adjustment measures 50 and/or calibration measures 51 to be at least partially retrieved from a database 53, with adjustment measures 50 and/or calibration measures 51 that are required in each case following the interchange and/or misalignment being stored in the database 53 for interchangeable and/or adjustable components 8 of the medical microscope 1. The database 53 can be stored in the memory 2-2 of the data processing device 2. Alternatively or additionally, the required adjustment measures 50 and/or calibration measures 51 may also be at least partially retrieved from an (external) service database 60. In principle, the procedure is the same as already described above for the example of the lookup table 52.
Provision can be made for the required adjustment measures 50 and/or calibration measures 51 to be at least partially determined by means of a trained artificial intelligence method 54. The trained artificial intelligence method 54 is provided by means of the data processing device 2 and estimates the required adjustment measures 50 and/or calibration measures 51 using the identified interchanged and/or misaligned components 8 as a starting point. By way of example, a trained neural network can be used as trained artificial intelligence method 54.
In particular, provision is made for the determined required adjustment measures 50 and/or calibration measures 51 to be displayed by means of the display device 7. In particular, this is implemented within the scope of step-by-step instructions, which provide a service technician with detailed instructions for the individual measures 50, 51. By way of example, a graphical user interface can be displayed on the display device 7 to this end, the individual instructions being provided on said display device in graphical and/or text form.
Provision can be made for at least some of the determined required calibration measures 51 to be carried out in automated fashion. This comprises those calibration measures 51, in particular, which relate to the signal processing of the captured raw signals 10l, 10r in the signal processing device 6. By way of example, these calibration measures 51 may relate to an extrinsic and/or intrinsic calibration of the cameras 3l, 3r (camera projection matrices), an offset (x/y-offset), and/or a rotation, a distortion, trimming of the brightness (camera vignetting and/or illumination geometry), and/or a chromatic aberration. The calibration measures 51 to this end are known per se, and are therefore not described in more detail here.
Provision can be made for at least one calibration object 40 to be arranged in automated fashion in a capture region by means of an actuator system 12 in order to carry out the determined required adjustment measures 50 and/or calibration measures 51. To this end, the actuator system 12 is controlled accordingly. To this end, the actuator system 12 for example receives an appropriate control signal, which causes the actuator system 12 to bring the at least one calibration object 40 from a stored position to a measurement position. By way of example, the control signal may be generated by the data processing device 2 or a control device (not shown here) of the medical microscope 1 and be fed to the actuator system 12. In this context, provision may also be made for a distance between calibration object 40 and an objective of the medical microscope 1 (z-direction) to be able to be varied or set by means of the actuator system 12.
Provision can be made for the required adjustment measures 50 and/or calibration measures 51 to be combined and/or optimized with one another in the case of a plurality of interchanged and/or misaligned components 8, in order to minimize a number and/or a scope of the measures 50, 51. This can be implemented when the method is carried out. However, provision can also be made for the combination and/or optimization to be carried out when generating the lookup table 52 and/or database 53, and/or when training the artificial intelligence method 54. This is advantageous in that computational power and memory requirements are not restricted by the data processing device 2.
Provision can be made for data 70 for anticipatory servicing to be collected and/or recorded in the medical microscope 1, with data 70 for determining and/or optimizing the determination of the adjustment measures 50 and/or calibration measures 51 being evaluated, said data having been collected before, during and/or after the interchange and/or the misalignment of the at least one component 8. In particular, the data 70 are collected by means of sensors (not shown here) of the medical microscope 1 configured to this end. The determination and/or optimization is implemented, in particular, by means of a central server (not shown here) on the basis of data 70 which were collected by a multiplicity of medical microscopes 1 in the field.
Provision can be made for a respective state of the medical microscope 1 to be captured before and after the interchange and/or misalignment, with associated image representations 20 of at least one calibration object 40 being captured by means of at least one capture device of the medical microscope 1 and with the required adjustment measures 50 and/or calibration measures 51 being determined by means of a trained artificial intelligence method 54, with the states captured before and after the interchange and/or the misalignment and the associated captured image representations 20 being taken into account. At least one information item 9 identifying the at least one interchanged and/or misaligned component 8 and the image representations 20 captured beforehand and afterwards are fed to the trained artificial intelligence 54, for example a trained neural network, as input data. Using this as a starting point, the trained artificial intelligence method 54 estimates the required adjustment measures 50 and/or calibration measures 51.
In a method step 200, a component, for example a varifocal objective, of the medical microscope 1 is swapped by a service technician. Alternatively or additionally, a misalignment of at least one component may also be present.
The readjustment and/or recalibration is started by the service technician in a method step 201. By way of example, this is implemented by an appropriate input on the data processing device of the medical microscope arrangement.
Instructions for the readjustment and/or recalibration by method steps 203-206 is provided in a method step 202.
In a method step 203, the interchanged and/or misaligned at least one component, the varifocal objective in this example, is identified. In this case, provision can be made for the identification of the interchanged and/or misaligned at least one component to be implemented at least partially manually, as has already been described above. Alternatively or additionally, provision can be made for the identification of the interchanged and/or misaligned at least one component to be implemented at least partially in automated fashion, as has likewise already been described above. In this example, information identifying the varifocal objective is available, in particular, following method step 203.
In a method step 204, required adjustment measures and/or calibration measures are determined in automated fashion using the identified at least one interchanged and/or misaligned component as a starting point, using the varifocal objective as a starting point in this example, as has already been described above with respect to
The required calibration measures are displayed on a display device in method step 205. This can be implemented in a graphically configured form (pictograms, flowcharts, animations, etc.) in particular.
In this way, step-by-step instructions for the service technician for homing in on the individual operating points and for carrying out the required calibration measures are implemented in method step 206. Once the calibration measures have been implemented, calibration data generated in the process, relating to the varifocal objective in this example, are stored in a memory of the medical microscope provided to this end, and subsequently serve for the (digital) correction.
If other components require adjustment measures, instructions for the service technician for carrying out the adjustment measures are displayed on the display device in an analogous fashion.
As a result of the method, the service technician can be efficiently guided through the required adjustment measures and/or calibration measures. This can reduce outlay and duration, and can consequently save costs.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2022 200 820.0 | Jan 2022 | DE | national |
