The present invention relates to labeling an invisible fluorescent area; more particularly, relates to instantly projecting a visible-light spot to the invisible fluorescent area for a surgeon to observe the area the visible-light spot projecting to.
The part of spectra of excited fluorescence of a fluorescent dye (Indocyanine Green, ICG) commonly used in clinical is a near-infrared part, which is invisible and the location of the fluorescent dye can not be observed directly by naked eye. Hence, the problem of the conventional ICG fluorescence imaging technology is that, since the ICG fluorescence is not visible, all images can be seen through screen only and a surgeon must look at the screen to know where the fluorescent area is; yet there is no way to see the surgical field at the same time. Furthermore, since the image is a light-emitting fluorescent image in black and white, the background light is very weak and the surgeon must repeatedly reference the screen to know about the fluorescent area for the surgical site. Hence, this is the biggest inconvenience in the existing fluorescent developing method: the method uses an indirect imaging device. In this regard, it was proposed to use Google glasses of wearable infrared developing device, so that the surgeon can see infrared light (Reference: Liu et al, Surgery, Volume 149, Number 5). It was an attempt to solve the problem of looking up to the screen. However, it did not consider the surgeon's maneuverability and comfortness.
Fluorescent developing devices on the market have also tried to solve the above-mentioned inconvenience. For example, HyperEye Medical Systems (HEMS) have a color camera and an infrared camera installed together in a device. Through image fusion, the surgeon can see the background color and the fluorescent area (Reference: InTech Article by Masaki Yamamoto, et al.) Nevertheless, such mode of operation still could not escape the indirect images.
Another prior art proposed an automatic control method of laser displacement for a laser skin-treatment apparatus. This prior art did not disclose how to deal with the change in distance from the camera with respect to the skin treatment. Another prior art was disclosed, where the device comprised an initial and a final mirror and a few adjusting lens groups are located in between and each of the lens groups includes two lens for angle adjustment, which shows a complex structure.
Traditionally, a surgery is done with eyes, which is the most intuitive operation. But the prior arts do not make the fluorescent area visible.
Hence, the prior arts do not fulfill all users' requests on actual use.
The main purpose of the present invention is to provide direct observation for a surgeon as getting rid of the screen without repeated location confirmations between surgical site and onscreen mark and to instantly observe movement of a fluorescent dye fluid, where, through instantly projecting a visible-light spot to a fluorescent area, the surgeon can observe the fluorescent area by observing the position the visible-light spot projecting to.
Another purpose of the present invention is to provide a method of self-correction concerning projection scale for coordination with a zoom lens.
Another purpose of the present invention is to adjust a projecting angle of the visible-light spot through an initial and a final mirror only without adjusting a lens group.
To achieve the above purposes, the present invention is a method of labeling invisible fluorescence by a visible light with self-correction, comprising steps of: (a) setting a correcting plate at an initial position and setting an excited-fluorescence light source to shine on the correcting plate; (b) capturing an image by an optical imaging unit contained in an image sensor; (c) moving the correcting plate to align a center position of the correcting plate to a center position of the image and inputting the image into an image analysis processor to select reference points; (d) positioning a light-spot position controller at a neutral position and projecting a visible-light spot on the correcting plate by a point-like visible light source through an optical device; (e) capturing an image by the optical imaging unit with the image sensor; inputting the image to the image analysis processor to obtain the visible-light spot; through analyzing an error between one of the reference points and the visible-light spot, adjusting initial settings of the light-spot position controller to minimize the error to process accurate alignment of the visible-light spot to the reference points through the center position of the correcting plate; recording a view field of the optical imaging unit and drive-settings of the light-spot position controller; (f) generating a standard round by the light-spot position controller; (g) capturing the standard round by the image sensor to be inputted into the image analysis processor and recording a location of the standard round in the image; and (h) obtaining a spatial mapping relationship between the visible-light spot and the image. Accordingly, a novel method of labeling invisible fluorescence by a visible light with self-correction is obtained.
The present invention will be better understood from the following detailed description of the preferred embodiment according to the present invention, taken in conjunction with the accompanying drawings, in which
The following description of the preferred embodiment is provided to understand the features and the structures of the present invention.
Please refer to
(a) A correcting plate 1 is placed at an initial position with an excited-fluorescence light source 2 shining on the correcting plate 1.
(b) An image is captured by an optical imaging unit 31 contained in an image sensor 3.
(c) The correcting plate 1 is moved to align its center position to the center position of the image. Then, the image is inputted into an image analysis processor 4 to select reference points 11 from the image for correction.
(d) A light-spot position controller 5 is set at a neutral position (x=0, y=0). The light-spot position controller 5 comprises a spot-like visible-light generator 51 and an optical device 52 (as shown in
(e) The image sensor 3 captures an image by the optical imaging unit 31. The image is inputted into the image analysis processor 4 to obtain the visible-light spot 53. Through analyzing an error between a reference point 11 and the visible-light spot 53, initial settings of the light-spot position controller 5 are adjusted for minimizing the error to process accurate alignment of the visible-light spot 53 to the reference points 11 through the center position of the correcting plate 1. A view field of the optical imaging unit 31 and drive-settings of the light-spot position controller 5 are recorded.
(f) The light-spot position controller 5 projects a standard round 54.
(g) The standard round 54 is captured by the image sensor 3 to be inputted into the image analysis processor 4 for recording a position of the standard round 54 in the image.
(h) A spatial mapping relationship between the visible-light spot 53 and the image is obtained.
Therein, the correcting plate 1 has nine reference points 11 for correction. Through adjusting the drive settings of the light-spot position controller 5, the visible-light spot 53 is accurately aligned to the reference points 11 (as shown in
Thus, a novel method of labeling invisible fluorescence by a visible light with self-correction is obtained.
The optical imaging unit 31 is a zoom lens for a dynamical self-correction process, including space correction and self-correction.
For obtaining information of the space correction, a plurality of view distances are uniformly selected between a farthest view field and a nearest view field of the optical imaging unit 31 to separately adjust the light-spot position controller 5 and record a location of the standard round at each view field.
On processing the self-correction, when a user adjusts a magnifying ratio of a lens as changing a view field of the optical imaging unit with the image shrunk, the light-spot position controller 5 is driven to generate the standard round; the location of the standard round in the image is obtained by the image sensor 3; all information of space correction is obtained; the location of the standard round is compared with locations of pre-corrected standard rounds; a position having a smallest distance error to the standard round is obtained; the spatial mapping relationship between the visible-light spot 53 and the image is obtained through interpolation; and the corresponding drive-settings of the light-spot position controller 5 is obtained.
On using the present invention, the actual scanning are shown in
The fluorescent areas 61 have gray-level values greater than the non-fluorescent background areas. A threshold, which is 255, is set for selecting the fluorescent areas 61 having the gray-level values greater than the threshold. The other areas having the gray-level values smaller than the threshold are labeled as the background areas with their gray-level values set to 0. For avoiding too-small fluorescent areas or noises, a filtering condition is set for selecting fluorescent areas. At last, a smallest rectangle surrounding all fluorescent areas 61 is obtained. As shown in
The threshold mentioned above is determined by a surgeon. Or, all surgeons' experiences are inputted into a database to obtain a recommended value through analysis by the system and the recommended value can be fine-tuned by the surgeon. The filtering condition mentioned above is based on sizes of fluorescent areas, which is determined according to observed-area sizes, organ types, etc. The filtering condition can also be determined according to how many details the surgeon wants to see in the fluorescent areas. Hence, the filtering condition is generally determined by the surgeon.
The present invention allows the surgeon to get rid of the screen for direct observation without repeated location confirmations between surgical site and onscreen mark. The movement of a fluorescent dye fluid can be observed in a real-time mode. Through projecting a visible-light spot to a fluorescent area at real time, the surgeon observes the fluorescent are by observing the visible-light spot projected. Hence, the problem that fluorescence imaging techniques must rely on the screen to see the position of the fluorescent area is solved. Thus, the present invention provides a method for self-correction of projection scale with coordination of a zoom lens. When a patient moves or fluorescent areas changes, an image sensor automatically adjusts an area labeled with a visible-light spot by changing a photographing area with a focus automatically set. In addition, the present invention adjusts an angle of the visible-light spot with an initial and a final mirror only, where adjustment of lens group is not required and cost is thus saved.
To sum up, the present invention is a method of labeling invisible fluorescence by a visible light with self-correction, where a surgeon gets rid of screen for direct observation without repeated location confirmations between surgical site and onscreen mark; the movement of a fluorescent dye fluid can be observed in a real-time mode; through projecting a visible-light spot on a fluorescent area at real time, the surgeon observes the fluorescent area the visible-light spot projecting to; and, hence, the problem that fluorescence imaging techniques must rely on the screen to see the location of the fluorescent area is solved.
The preferred embodiment herein disclosed is not intended to unnecessarily limit the scope of the invention. Therefore, simple modifications or variations belonging to the equivalent of the scope of the claims and the instructions disclosed herein for a patent are all within the scope of the present invention.