Patent Application
20150348311
This application claims priority of German patent application no. 10 2014 210 051.8, filed May 27, 2014, the entire content of which is incorporated herein by reference.
The invention relates to a method and an apparatus for determining a surface topography of a body in a coordinate system fixed in space and/or fixed on the body.
In many applications, particularly medical engineering, it is necessary to compare data sets for the same body or body part, which were recorded at different times and possibly with different devices, or to bring these into correspondence in terms of their spatial or planar orientation. This process is also known by the term “registration”. Here, in a first step, data of a body or a body part of a patient are acquired using a first device, for example, a computed tomography or nuclear magnetic resonance imaging scanner or, in general, an imaging device, and stored in a first data set, which is defined in a coordinate system set by the first device. In medical engineering, this first step is often performed on the patient prior to actual operation. In a second step, following in time and often performed during an operation, a second data set of the same body or body part is established with a second device in a coordinate system that is fixed by the second device. Here, the second device can be identical to the first device or differ functionally from the first device and, in particular, be configured as a camera or an OCT (optical coherence tomography) device. Since the position of the body or the body part relative to the first device during the first step generally differs from the position of the body or body part relative to the second device during the second step, it is often necessary to bring the data set obtained in the first step into correspondence with the data set obtained in the second step in respect of its planar or spatial position and orientation.
The prior art has disclosed, for registration purposes, the practice of sticking markers onto the body part to be examined, which markers remain on the patient during the first step and the second step. The position of the markers in space is acquired during the first and the second step by a suitable camera system and stored with the first and the second data set. A transformation prescription, on the basis of which the patient data sets can be converted into the desired coordinate systems, can be established from a comparison between the established marker positions. A disadvantage of this method is that the sticky markers can slip and that only a few discrete points are acquired, and so only a small data pool is available for establishing a transformation prescription. Furthermore, the camera system in practice is often at a distance of one to two meters from the operation site, with care having to be taken that the direct connecting line between the markers and cameras is not interrupted. This leads to restrictions in the operation procedure.
In a known, alternative method, markers are fixed on the patient, for example screwed to a bone. A disadvantage of this method is that it requires an invasive intervention, which is uncomfortable for the patient.
U.S. Pat. Nos. 6,873,867 and 7,577,474 disclose the practice of scanning the surface of a body to be examined via a handheld laser, with the laser points being acquired by a 3D camera system. A topography of the surface is calculated from the established position of the laser points, which topography is compared to a data set obtained pre-surgery. Alternatively, a product distributed by Brainlab under the trade name Softtouch enables a topography to be determined with the aid of a scanning head, which is connected to the surface to be acquired and the position and orientation of which in space is once again established by a camera system when contact is made with the surface. A disadvantage of this method lies in the great time outlay for establishing the topography as a result of the sequential scanning of the surface. Moreover, these methods require a camera system for establishing the position of the laser points with the restrictions in the operation procedure connected therewith.
It is an object of the invention to provide a method and an apparatus for determining a surface topography of a body in a coordinate system fixed in space and/or fixed on the body, which overcome the disadvantages of the aforementioned methods and apparatuses.
This object is achieved by a method which includes the following steps: recording a stereoscopic image of the surface of the body with an image recording device; and generating a topography data set from the stereoscopic image in a coordinate system connected to the image recording device. Here, a stereoscopic image should be understood to mean, for example, recordings which are recorded with the aid of two cameras or with one camera, which is moved in space, or with a 3D camera using the time-of-flight method. By using the stereoscopic image, a multiplicity of data points are available. The data points can be used via suitable methods at the same time for the generation of the topography data set. As a result, the time required for establishing a topography data set is reduced.
In one embodiment of the method, in a further method step, a position of the image recording device is established in a coordinate system that is fixed in space and/or fixed on the body. Here, the phrase “fixed on the body” relates to the body to be examined. This simplifies the comparison between the data established in the first step and in the second step, particularly if the body was moved therebetween or during the second step.
In a further embodiment of the method, at least one further stereoscopic image of the surface of the body is recorded by the image recording device from a perspective, which differs from the perspective when recording the first stereoscopic image, for the purposes of generating the topography data set. Consequently, a larger number of stereoscopic images, and therefore data points, are available, which can be used for generating the topography data set.
In a further embodiment of the method, a transformation prescription is determined between the coordinate system fixed in space, or the coordinate system fixed on the body, and the coordinate system connected to the image recording device and the topography data set is transformed into the coordinate system fixed in space or fixed on the body with the aid of the transformation prescription.
In a further embodiment of the method, a 3D camera is used as image recording device, which 3D camera can be held on a microscope, in particular a surgical microscope, or can be attached to such a microscope.
In a further embodiment of the method, the position of the image recording device in the coordinate system fixed in space is established with the aid of a navigation device arranged fixed in space, which navigation device is configured to determine the position of a marker arranged on the image recording device.
In a further embodiment of the method, a stereoscopic image of a marker arranged fixed in space or fixed on the body is recorded with the image recording device and the position of the image recording device in the coordinate system fixed in space or the coordinate system fixed on the body is established from the stereoscopic image of the marker arranged fixed in space or fixed on the body. This simplifies the establishment of the position of the image recording device relative to the body or the surrounding space, with it being possible to dispense with additional navigation devices in the surroundings of the body.
In a further embodiment of the method, the body is fixed relative to the marker arranged fixed in space.
In a further embodiment of the method, the topography data set from the stereoscopic image of the surface of the body is generated as a dense 3D reconstruction according to the method of optical flow or epipolar geometry or a sparse surface representation on the basis of node points with subsequent optimization of a cost function.
In a further embodiment of the method, the topography data set is generated as a depth map and/or a metrically correct topographic reconstruction, in particular as a mesh, grayscale image or point cloud.
In a further embodiment of the method, the topography data set is generated at least approximately in real-time by processing on one or more computers with a parallel computing structure.
In a further embodiment of the method, the topography data set and/or an established position of a marker arranged fixed in space is provided by way of an interface for use by other internal or external applications.
The object is further achieved by an apparatus for determining a surface topography of a body in a coordinate system fixed in space and/or a coordinate system fixed on the body. The apparatus includes a stereoscopic image recording device, which has a marker and which is configured for recording a stereoscopic image in a coordinate system fixed by the image recording device; a navigation device, which is configured to establish a position of the marker of the image recording device in the coordinate system fixed in space and/or the coordinate system fixed on the body; and a control unit, which is configured to generate a topography data set from the stereoscopic image in a coordinate system connected with the image recording device and determine a transformation prescription between the coordinate system fixed in space, or the coordinate system fixed on the body, and the coordinate system connected with the image recording device and transform the topography data set into the coordinate system fixed in space, or the coordinate system fixed on the body, with the aid of the transformation directive.
In an alternative embodiment of the invention, the apparatus for determining a surface topography of a body in a coordinate system fixed in space or a coordinate system fixed on the body includes a stereoscopic image recording device, which has a marker and which is configured to record a stereoscopic image in a coordinate system connected with the image recording device and establish a position in respect of the marker fixed in the coordinate system fixed in space or the coordinate system fixed on the body; and a control unit, which is configured to generate a topography data set from the stereoscopic image in a coordinate system connected with the image recording device and determine a transformation prescription between the coordinate system fixed in space, or the coordinate system fixed on the body, and the coordinate system connected with the image recording device and transform the topography data set into the coordinate system fixed in space, or the coordinate system fixed on the body, with the aid of the transformation prescription.
In one embodiment of the invention, the image recording device is embodied as a camera, in particular as a 3D camera.
In one embodiment of the invention, the image recording device is integrated in a microscope, in particular a surgical microscope, or connected to a microscope, in particular a surgical microscope.
The invention will now be described with reference to the single FIGURE of the drawing (
In
During the operation, the data stored in the computing unit and established pre-surgery are intended to be brought into correspondence in terms of the spatial position and orientation thereof with data obtained during surgery. In this embodiment, the data obtained during surgery include a live image of the patient, recorded with the aid of a surgical microscope 4, onto which the CT or MRI data obtained pre-surgery are intended to be superposed. The surgical microscope 4 is held on a stand not shown in
For the purposes of registration, a stereoscopic image of the head of the patient is recorded with an image recording device in the form of a stereo camera 6 that is integrated into the surgical microscope, from which stereoscopic image a topography data set is generated in a coordinate system fixed by the stereo camera. Here, the head 7 of the patient is fixed in relation to the operating table 2 via a stereotactic frame or another suitable apparatus. At the same time, the stereo camera 6 acquires a navigation point, for example in the form of a marker 8 at the stereotactic frame or at the operating table 2, which marker is visible in the image and the position of which is fixed relative to the examination location (the head of the patient). The provision of a fixed navigation point at the stereotactic frame or at the operating table provides the advantage of simplified navigation during the subsequent course of the operation when the body of the patient is, for example, covered by sterile towels.
A topographically dense reconstruction of the scene is established from the stereoscopic image. The reconstruction is preferably implemented in real-time by using a parallel architecture in the computing unit 3 or by parallel use of a plurality of computing units (cluster). To this end, the computing unit may, for example, include a multicore CPU or a suitable graphics processing unit (GPU) or may be configured as a field programmable gate array (FPGA).
The result of the reconstruction can be made available as a depth map and/or metrically correct topographic construction (mesh, grayscale image, point cloud).
The topographically dense reconstruction is subsequently compared to a topography of the surface of the head generated from the data established pre-surgery and brought into correspondence therewith. A transformation prescription for converting the data established pre-surgery into a coordinate system fixed on the body or fixed in space can be established from the comparison such that the data established pre-surgery can subsequently be superposed into a live image recorded during the operation and updated in the case of movements of the patient relative to the surgical microscope.
Preferably there is an automatic detection of the 3D position of the fixed navigation point relative to the topography of the body part to be examined.
By way of an interface, topography and 3D position of the fixed navigation point can be made available to further internal and/or external applications and/or stored for the subsequent course of the operation.
In a further embodiment, relatively large surfaces and/or volumes of the body part to be examined are acquired; this may also be implemented in a fully automated manner. To this end, the surgical microscope with the image recording device integrated therein or arranged thereon is preferably held on a robotic stand, that is, a stand with drive-assisted movement options. A plurality of positions are approached with the aid of the drives, at which positions stereoscopic recordings of the surface of the body, which ideally overlap, are recorded. There is a topographic reconstruction of the surface for each recording. By putting together the partial recordings (so-called “stitching”) via a combination of the intrinsic position information from the surgical microscope in a coordinate system fixed in space and/or fixed on the body (approximate initialization) and the image information from the stereoscopic recordings, it is possible to put together the topographic data obtained from the recordings so as to form a relatively large surface or a relatively large volume.
An in turn further embodiment enables a semi-automatic acquisition of relatively large surfaces or volumes. To this end, the surgical microscope with the image recording device integrated therein or arranged thereon is held on a stand with a pivot functionality. Here, a pivot functionality should be understood to mean a suitability of the stand for rotating the surgical microscope about a fixed point in space, wherein the fixed point always lies in an observation beam path of the surgical microscope. After recording a stereoscopic image, the microscope is aligned in such a way that a further recording can be made from a different perspective. This process can be implemented automatically by virtue of the surgical microscope automatically being displaced in an appropriate direction after the brakes of the surgical microscope are released. As soon as a distance to a target position drops below a threshold, the brakes are reactivated and a new stereoscopic recording is generated. This process is repeated until the whole desired area has been acquired.
In a further embodiment of the invention, the image recording device is equipped with a single camera. Without a patient movement and with a known change in position of the image recording device relative to the patient, a so-called structure-from-motion approach can also be selected for the topographic reconstruction.
It is understood that the foregoing description is that of the preferred embodiments of the invention and that various changes and modifications may be made thereto without departing from the spirit and scope of the invention as defined in the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2014 210 051.8 | May 2014 | DE | national |
