Patent Application
20090292175
This application claims the benefit of Japanese Application No. 2008-135635 filed in Japan on May 23, 2008, the contents of which are incorporated herein by this reference.
1. Field of the Invention
The present invention relates to a medical device having an image pickup section that is able to pick up images of a tube cavity in a subject, more particularly, to a medical device that performs examination or treatment with high accuracy using virtual endoscopic images of a tube cavity based on three-dimensional image data of a subject.
2. Description of the Related Art
In recent years, diagnoses have been widely made using three-dimensional images. For example, a diagnosis for a target site can be made using three-dimensional image data of a subject which is obtained by picking up tomographic images of the subject with an X-ray CT (Computed Tomography) apparatus.
In the CT apparatus, a subject is continuously moved while X-ray radiating positions and detection positions are continuously rotated for continuous helical scanning of the subject (helical scan). A number of the resulting continuous two-dimensional tomographic images of the subject are used to create a three-dimensional image.
A three-dimensional image of bronchus of lungs is one type of the three-dimensional images used in diagnoses. Such a three-dimensional image of bronchus is used in a three-dimensional detection of the position of a diseased area with suspected lung cancer for example. In order to check the diseased area by a biopsy, a bronchus endoscope is inserted into the subject and a biopsy needle or biopsy forceps are extended out from a distal end portion of the bronchus endoscope, so as to collect tissue samples of the target site.
In a tract such as bronchus in a body that is branched in multiple steps, in a case where a diseased area is located at the end of a bronchus, it is hard to bring the distal end of an insertion section of an endoscope to a position near the target site in a short period of time with accuracy. Thus, for example, Japanese Patent Application Laid-Open Publication No. 2004-180940 and Japanese Patent Application Laid-Open Publication No. 2005-131042 disclose navigation systems for insertion of endoscope in which a three-dimensional image of a tube cavity in a subject is created based on image data of a three-dimensional area in the subject, and a route along the tract to a target on the three-dimensional image is obtained, so that virtual endoscopic images of the tract along the route can be created based on the image data.
A medical device of the present invention includes: an image pickup section that is able to pick up an image of a tube cavity in a body of a subject; a virtual endoscopic image generation section configured to generate a virtual endoscopic image in the tube cavity from a plurality of different line-of-sight positions based on three-dimensional image data of the subject that is obtained in advance; an image retrieval section configured to retrieve a virtual endoscopic image highly similar to the endoscopic image of the tube cavity picked up by the image pickup section; and a reference-point setting section configured to set a predetermined position near the image pickup section as a reference point based on the line-of-sight positions of the highly similar virtual endoscopic image.
Now, with reference to the drawings, an embodiment according to the present invention will be explained below.
As shown in
The endoscope apparatus 2 includes: an insertion section 2E insertable through the bronchus 9 of the patient 7; an endoscope 2A having an image pickup section 2D such as a CCD arranged at a distal end portion 2C of the elongated insertion section 2E; an endoscope control section 2B configured to control the endoscope 2A; a display section 6; and the like. The display section 6 also displays information of an insertion assist apparatus 3. The insertion section 2E has a channel (not shown) formed therein through which the treatment instrument 4 as a medical instrument can be inserted, and the distal end portion 2C is provided with a treatment instrument port 2F that is an opening of a channel 2F1, thereby as shown in
The insertion section 2E also has a tract (not shown) formed therein for liquid-supply, and the distal end portion 2C is provided with a liquid-supply port 2G as an opening of the tract. Therefore, a surgeon is able to eject or spray a desired liquid from the liquid-supply port 2G to the target site 9G.
As shown in
The insertion assist apparatus 3 includes: an image processing section 10; a CT image data storing section 13; a virtual endoscopic image (Virtual Bronchus Scope Image: hereinafter, also referred to as “VBS image”) generation section 12 configured to generate a virtual endoscopic image; an image retrieval section 11 configured to retrieve a virtual endoscopic image highly similar to an endoscopic image; and a first coordinate calculation section 14. The image processing section 10 processes an endoscopic image (hereinafter, also referred to as “real image”) picked up by the image pickup section 2D. The CT image data storing section 13 stores three-dimensional image data in a format such as DICOM (Digital Imaging and Communication in Medicine) that is generated by a known CT apparatus (not shown) for picking up X-ray tomographic images of the patient 7. The VBS image generation section 12 generates a VBS image from the image data in DICOM format based on line-of-sight parameters which will be explained later.
The insertion assist apparatus 3 may further include a VBS image storing section 12B configured to store the VBS images generated by the VBS image generation section 12, and a reference point setting section 15 configured to calculate a reference point based on a first coordinate. The image retrieval section 11 and the first coordinate calculation section 14 will be explained later in detail.
First, with reference to
As shown in
Upon a start of an insertion operation of the endoscope, next, as shown in
When a surgeon inserts the distal end portion 2C into a route hole having the marker 6G superimposed thereon, the VBS image 6d on the insertion navigation screen is switched to the VBS image at the second branch.
As a result, using the insertion navigation system, the surgeon is able to insert the distal end portion 2C to the branch of the bronchus 9 having the minimal diameter for insertion of the distal end portion 2C to a position near the target site 9G without making a mistake, at every branch, in determining a route hole in which the distal end portion 2C should be inserted.
After the distal end portion 2C is inserted to a position near the target site 9G, in the medical device 1, the first coordinate calculation section 14 calculates a position (a three-dimensional coordinate) and a direction of the distal end portion 2C.
First, an allowable error e0 is set for determination of similarity which is performed by the image retrieval section 11. A smaller allowable error e0 allows the first coordinate calculation section 14 to more accurately calculate the position and direction of the distal end portion 2C, but takes a longer time. Thus, the allowable error e0 is changeable by a surgeon depending on the purpose.
The VBS image generation section 12 can generate a VBS image from a large number of different line-of-sight positions based on image data in DICOM format by changing six line-of-sight parameters. The parameters of line-of-sight positions as used herein are six-dimensional parameters including positions (x, y, z) and angles (θx, θy, θz). At Step S11, the six initial values of the line-of-sight parameters are set.
In a case where the endoscope 2A is inserted using the insertion assist apparatus 3 that is provided with a function of insertion navigation system, the line-of-sight parameters of the VBS image at the last branch J are preferably set as the initial values.
The VBS image generation section 12 generates one VBS image using three-dimensional image data of the bronchus 9 of the patient 7 stored in the CT image data storing section 13, based on the initial values of the line-of-sight parameters.
<Step S13>The image retrieval section 11 compares the real image and the VBS image generated by the VBS image generation section 12 on the similarity to each other. The comparison between the images is performed by a known image process which may be matching process on the level of pixel data or matching process on the level of features extracted from images. Because the matching process between the real image and the VBS image is performed for every frame of the real image, the actual comparison is made based on the similarity between the static endoscopic image and the virtual endoscopic image. The matching process need not be performed for all of the frames of the real image, but is performed at appropriate intervals.
When the error e calculated by the image retrieval section 11 for the similarity between the real image and the VBS image is larger than the allowable error e0 (No), at Step S15, the image retrieval section 11 outputs a line-of-sight parameter of a slightly different line-of-sight position or direction to the VBS image generation section 12. Then, at Step S12, the VBS image generation section 12 generates a next VBS image according to the new line-of-sight parameter set at Step S15.
The insertion assist apparatus 3 repeats the above loop operations, that is, outputs a different line-of-sight parameters, and as a result of that the VBS image generated by the VBS image generation section 12 is gradually changed to an image similar to the real image, and the error e between the images becomes equal to the allowable error e0 or less (Yes) after the loop operations are repeated some times.
When the similarity error e between the VBS image and the real image becomes equal to the allowable error e0 or less, the first coordinate calculation section 14 calculates a position (coordinate) and a direction of the distal end portion 2C using the line-of-sight parameters of the VBS image having higher similarity.
As explained at Step S13, the real image is updated at appropriate intervals, but while the distal end portion 2C remains at a position, because the position and the direction of the distal end portion 2C do not significantly change, the number of times the loop operations are repeated is not increased in spite of the updates of the real image. However, a large movement of the distal end portion 2C causes a large change in the real image, which requires a large number of repetitions of the above described loop operations to make the errors e of the images equal to an allowable error e0 or less. During a biopsy of the target site 9G, because the distal end portion 2C does not usually move a lot, any large movement of the distal end portion 2C implies an abnormal situation. So, when the medical device 1 determines that there is a large change in a real image, that is, the error e of the loop operations is suddenly increased, the medical device 1 preferably generates an alarm, and discontinues the loop operations for a predetermined period of time, for example, several seconds.
In the above description, the endoscope 2A is operated to be inserted by the insertion assist apparatus having a function of insertion navigation system, but even when the endoscope 2A is operated to be inserted without the insertion navigation system, the first coordinate calculation section 14 of the medical device 1 is able to calculate a position and a direction of the distal end portion 2C.
For example, the initial values for a line-of-sight position used at Step S11 may be randomly determined. Alternatively, the VBS image generation section 12 may generate virtual endoscopic images at branches of the bronchus in advance from a plurality of different line-of-sight positions, and store the image in the VBS image storing section 12B, so that the image retrieval section 11 can retrieve the VBS image most highly similar to a real image from the stored VBS images, and set the line-of-sight parameters of the most highly similar VBS image as initial values at the line-of-sight position that are used at Step S11.
Also, in the above description, a position and a direction of the distal end portion 2C is calculated using the six-dimensional line-of-sight parameters by the first coordinate calculation section 14 when the error e of the similarity between the VBS image and the real image becomes equal to the allowable error e0 or less. However, the first coordinate calculation section 14 may calculate a position and a direction of the distal end portion 2C using the line-of-sight parameters of a VBS image which is determined to be most highly similar to the real image by the image retrieval section 11 among the virtual endoscopic images of bronchus branches from a plurality of different line-of-sight positions generated by the VBS image generation section 12 in advance. Needless to say, in the latter case, the number of VBS images generated in advance should be increased to enhance the accuracy of the obtained position and direction of the distal end portion 2C.
As explained above, the VBS image generation section 12 generates a plurality of VBS images based on different line-of-sight parameters using the information from the image retrieval section 11 so as to generate a more highly similar VBS image, that is, a VBS image having a smaller error e. Generally, the VBS images generated by the VBS image generation section 12 at early steps of the above loop operations have larger errors e, but the more VBS images are generated, the smaller error e becomes.
As already explained above, the image retrieval section 11 compares the real image and the VBS image generated based on line-of-sight parameters by the VBS image generation section 12 on the similarity, and when the error e of the similarity between the images becomes equal to the allowable error e0 or less, the first coordinate calculation section 14 sets a coordinate of the line-of-sight positions of the VBS image as a first coordinate.
In the medical device 1, preferably, the allowable error e0 is individually set to be different in a tube cavity longitudinal direction ZA and in a tube cavity orthogonal direction YA. The tube cavity longitudinal direction ZA as used herein is the longitudinal direction of a bronchus tube cavity through which the distal end portion 2C is inserted and along which the distal end portion 2C moves in accordance with an insertion operation. When VBS image data includes information of the centroidal line of a tube cavity, so-called centerline information, the allowable error e0 may be changed by using the centerline direction instead of the longitudinal direction of the tube cavity, and the direction orthogonal to the centerline direction instead of the tube cavity orthogonal direction.
Alternatively, in the medical device 1, the accuracy of the coordinate of the tube cavity longitudinal direction ZA can be higher than the accuracy of the coordinate of the tube cavity orthogonal direction YA in the positional information of a calculated first coordinate point, by setting an increment/decrement of the coordinate of the tube cavity longitudinal direction ZA to be smaller than an increment/decrement of the coordinate of the tube cavity orthogonal direction YA at Step S15 during the loop operations shown in
This is because the distal end portion 2C in a narrow tube cavity having a diameter similar to the diameter of the distal end portion 2C hardly moves in the tube cavity orthogonal direction YA in case of any wrong operation, but easily moves in the tube cavity longitudinal direction ZA. Also, this is because a treatment is performed with the treatment instrument 4 being protruded from the treatment instrument port 2F of the channel 2F1, based on the coordinate of the tube cavity longitudinal direction ZA.
Now, with
As shown in
The point on endoscope that corresponds to a line-of-sight parameter of a VBS image represented by a first coordinate point calculated by the first coordinate calculation section 14 constitutes a pupil position A0 and the direction of the line-of-sight S1 as often called in an optical system.
Here, the coordinate of the first coordinate point A0 is expressed in a coordinate system of the virtual endoscopic image, in other words, a CT coordinate system, which means a lot to the medical device 1. That is, as already explained above, because the target site 9G for a biopsy and the like is located at a bronchus end which the distal end portion 2C cannot reach, a surgeon cannot perform a biopsy and the like using the treatment instrument 4 while checking real images for the target site 9G. Therefore, a surgeon performs a biopsy based on the position of the target site 9G shown in a CT coordinate system using the display section 6 in the three-dimensional image data that is obtained by a CT apparatus in advance. However, the surgeon can check the position of the distal end portion 2C, more specifically, the position of the distal end of the treatment instrument 4 for the biopsy protruded from the distal end portion 2C only in an endoscope coordinate system based on the distal end portion 2C which has no relationship with the CT coordinate system.
To the contrary, in the medical device 1, the coordinate of the first coordinate point A0 on a part of the distal end portion 2C that is close to the coordinate of the target site 9G is expressed in the same CT coordinate system, which allows the surgeon to use the coordinate to bring the treatment instrument 4 to the target site 9G for a biopsy and the like based on the first coordinate point A0 as a reference point.
The examination or treatment performed using the medical device 1 herein may be spray of medication, biopsy, mucus sampling, extraction of foreign object, high-frequency cauterization, or the like. And, the treatment instrument 4 such as forceps or a probe such as an ultrasound probe as a medical instrument depending on the examination or treatment is inserted through the channel 2F1 for use.
The endoscope coordinate system for the medical device 1 shown in
In the above description, the first coordinate point A0 is used as a reference point as it is, but in the medical device 1, based on the first coordinate point A0, another position represented in an endoscope coordinate system which is set by the reference-point setting section 15 using a transformation parameter may be set as a reference point.
As the reference point Al, for example, a position in the liquid-supply port 2G or the treatment instrument port 2F, more specifically, the center of the liquid-supply port 2G or the treatment instrument port 2F is used. That is, the reference point A1 is set at the center of the liquid-supply port 2G or the treatment instrument port 2F. Also, as the reference point A2, an intersection point of the central line of the tube cavity, for example, the centerline C1 and the plane AS1 is selected.
As compared to the case where the first coordinate point A0 is used as a reference point, the above described reference point A1 enables more precise examination or treatment because the liquid-supply port 2G for treatment and the like or the treatment instrument port 2F providing a starting point from which the treatment instrument 4 is protruded is used as a reference point. In addition, the above described reference point A2 facilitates the calculation of a distance to the target site 9G.
As explained above, the medical device 1 for examination or treatment based on a reference point sets a predetermined position at a part near the image pickup section 2D which is located near the target site 9G as a reference point. The position at a part near the image pickup section 2D is in the bronchus that is a tube cavity in a body of the patient 7 who is the subject, and is a predetermined position in the medical device 1 including the inside of the image pickup section 2D. The position near the image pickup section 2D is preferably in the bronchus between a line-of-sight position of the image pickup section 2D and the target site 9G, more preferably a predetermined position on the distal end portion 2C.
In the above description, only one reference point is set, but a plurality of reference points may be set, and alternatively, a reference plane may be set instead of a reference point.
As explained above, a medical device of the present invention includes:
a treatment instrument or probe for examination or treatment based on a reference point;
an insertion section having a channel formed therein through which the treatment instrument or the probe is insertable, and having an opening of the channel and an image pickup section that is able to pick up an image of bronchus of a subject at a distal end portion thereof;
a virtual endoscopic image generation section configured to generate a virtual endoscopic image in the bronchus from a plurality of line-of-sight positions based on three-dimensional image data of the subject that is obtained in advance, and also generate a more highly similar virtual endoscopic image based on information of a most highly similar virtual endoscopic image retrieved by an image retrieval section configured to retrieve the virtual endoscopic image most highly similar to the endoscopic image of the bronchus picked up by the image pickup section among the plurality of the virtual endoscopic images that are generated in advance; and
a reference-point setting section configured to set a position in the opening as a reference point based on the line-of-sight positions of the more highly similar virtual endoscopic image.
In the above description, the example with the endoscope apparatus 2 having the elongated insertion section 2E is used, but a medical device of the present invention may be a medical device such as a capsule endoscope apparatus having an image pickup section 2D that is able to pick up an image of tube cavity in the body of a patient 7, which also provides the same operational effects as those of the endoscope apparatus 2 having the elongated insertion section 2E.
Having described the preferred embodiments of the invention referring to the accompanying drawings, it should be understood that the present invention is not limited to those precise embodiments and various changes and modifications thereof could be made by one skilled in the art without departing from the spirit or scope of the invention as defined in the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2008-135635 | May 2008 | JP | national |
