Exemplary embodiments of the present invention relate to a tracking system and tracking method using the same. More particularly, exemplary embodiments of the present invention relate to a tracking system and tracking method using the same for surgery capable of detecting a spatial and direction information of a target by tracking coordinates of markers attached on the target, in which the target are markers attached on a patient or surgical instrument.
Recently, a robot surgery have been studied and introduced to reduce the pain of patients and to recover faster in an endoscopic surgery or an otolaryngology surgery (ENT surgery).
In such a robot surgery, in order to minimize a risk of the surgery and to operate the surgery more precisely, a navigation system is used to navigate to an exact lesion of a patient by tracking and detecting a spatial position and direction of a target such as lesion portion or surgical instrument.
The navigation system described above includes a tracking system which is capable of tracking and detecting a spatial position and direction of a target such as lesion or surgical instrument.
The tracking system described above includes a plurality of markers attached on a lesion or a surgical instrument, first and second image forming units to form images of lights emitted from the markers, and a processor calculating a 3-dimensional coordinates of the markers which are coupled to the first and second image forming units and calculating a spatial position and a direction of the target by comparing pre-stored information of straight lines connecting the markers adjacent to each other and angle information formed by a pair of straight lines adjacent to each other with the 3-dimensional coordinates of the markers.
Herein, in order to calculate the 3-dimensional coordinates of the markers, conventionally, two detectors are required to calculate 3-dimensional coordinates of each marker through a processor, a triangulation method is used in an assumption that a coordinate of marker which is emitted from one marker and formed image in a first image forming unit and a coordinate of marker which is emitted from one marker and formed image in a second image forming unit are identical.
Thus, conventional tracking system requires two image forming units to form images of lights which are emitted from each of the markers positioned different to each other, a manufacturing cost increases as well as a whole size also increases, therefore, a restriction of surgical space is generated.
Therefore, the technical problem of the present invention is to provide a tracking system and method using the same capable of reducing a manufacturing cost as well as minimizing a restriction of a surgical space by achieving compact of the system through calculating 3-dimenstional coordinates for each marker using only one image forming unit.
In one embodiment of the present invention, a tracking system includes at least three markers which are attached on a target to emit lights, a pair of light sources which emit lights in different position to each other, a lens portion which passes the lights emitted from the pair of the light sources and reflected by the markers, an image forming unit which forms a pair of maker images for each marker by receiving the lights which have passed by the lens portion, a processor which calculates three-dimensional coordinates of the markers by using the pair of the maker images formed on the image forming unit for each marker and calculates spatial position information and direction information of the target by comparing the three-dimensional coordinates of the markers with pre-stored geometric information between the markers adjacent to each other.
Meanwhile, the tracking system may further include a beam splitter which is arranged between the lens portion and the image forming unit to partially reflect the light, which is emitted from one light source of the pair of the light sources, toward a center of the markers, and to partially pass the light, which is emitted toward the center of the markers, re-reflected by the markers, and flowed through the lens unit, toward the image forming unit.
Also, the tracking system may further include a diffuser arranged between the light source, which emits light toward the beam splitter, and the beam splitter to diffuse the lights emitted from the light sources.
In one embodiment, the image forming unit may be a camera to form a pair of images for each marker by receiving the lights reflected by the markers and having sequentially passed the lens portion and the beam splitter.
In one embodiment, the geometric information between the markers may be length information of straight lines connecting the markers adjacent to each other and angle information formed by a pair of straight lines adjacent to each other.
In one embodiment of the present invention, a tracking method includes emitting lights from a pair of light sources which are positioned at different position to each other toward at least three markers, reflecting the lights emitted from the pair of the light sources toward a lens portion by the markers, forming a pair of maker images on an image forming unit for each marker through the lights emitted from the markers and have passed the lens portion, calculating three-dimensional coordinates for each marker through a processor by using the pair of maker images formed on the image forming unit for each marker, and calculating spatial position information and direction information of the target by comparing the three-dimensional coordinates of each marker with geometric information which is pre-stored in the processor between the markers adjacent to each other.
Herein, one light source of the light sources emits the light toward the beam splitter arranged between the lens portion and the image forming unit, and emits the light towards a center of the markers through the lens portion by partially reflecting the light by the beam splitter, and the other light source directly emits the light toward the markers.
Meanwhile, the light emitted toward the beam splitter may be emitted toward the beam splitter by a diffuser arranged between the beam splitter.
In one embodiment, the geometric information between the markers may be length information of straight lines connecting the markers adjacent to each other and angle information formed by a pair of the straight lines adjacent to each other.
In one embodiment, calculating three-dimensional coordinates of the markers may further include calculating two-dimensional central coordinates for each marker through the processor by using image forming positions of the pair of maker images formed on the image forming unit for each marker, and calculating three-dimensional coordinates of the markers by using the two-dimensional central coordinates of each marker.
Thus, according to an embodiment of the present invention, in a tracking system and tracking method using the same, one image forming unit is used to calculate spatial position information and direction information of a target by calculating three-dimensional coordinates of the markers through a trigonometry since it is possible to form a pair of maker images for each marker in different image forming positions by a pair of light sources positioned different to each other.
Therefore, there is an effect of reducing a manufacturing cost of the tracking system, making small and lightweight, and relatively low restriction of surgical space comparing with conventional tracking system.
The present invention is described more fully hereinafter with reference to the accompanying drawings, in which example embodiments of the present invention are shown. The present invention may, however, be embodied in many different forms and should not be construed as limited to the example embodiments set forth herein. Rather, these example embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present invention to those skilled in the art. In the drawings, the sizes and relative sizes of layers and regions may be exaggerated for clarity.
It will be understood that, although the terms first, second, third etc. may be used herein to describe various elements, components, regions, layers and/or sections, these elements, components, and/or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or section from another region, layer or section. Thus, a first element, component, or section discussed below could be termed a second element, component, or section without departing from the teachings of the present invention.
The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Hereinafter, with reference to the drawings, preferred embodiments of the present invention will be described in detail.
A tracking system and method using the same according to an embodiment of the present invention attaches at least three markers at a target such as a lesion or a surgical instrument and calculates 3-dimenstional coordinates of the markers, compares geometric information of markers adjacent to each other which are pre-stored in a processor with the 3-dimenstional coordinates of the markers through the processor, and therefore, is capable of calculating spatial position information and direction information of a target such as a lesion or surgical instrument. The detailed description is explained referencing figures.
Referring to
The at least three markers 110111 and 112 are attached on a target 200 such as a lesion of a patient or a surgical instrument. Herein, the at least three markers 110111 and 112 are separated to adjacent markers 110111 and 112 to each other at a predetermined interval and the markers are attached on the target 200 such as a lesion or a surgical instrument such that a pair of straight lines L1 L2 and L3 in which each of the markers are virtually connected to adjacent markers forming specific angles A1 A2 and A3.
Herein, geometric information between the markers 110111 and 112 which are adjacent to each other, in other words, length information of straight lines L1 L2 and L3 which connect the markers 112 which are adjacent to each other and angle information A1 A2 and A3 which are formed by a pair of the straight lines connecting the adjacent markers, are stored in a memory 141 of the processor 140.
For example, the markers 110111 and 112 may be attached on the target 200 such as a lesion and a surgical instrument in a triangle shape, straight lines information L1 L2 and L3 forming sides of the triangle in which the markers are used as vertices and angle information A1 A2 and A3 which are formed by a pair of adjacent and virtual straight lines which connect the markers 110111 and 112 may be pre-stored in the memory 141 included in a processor 140.
Meanwhile, the markers 110111 and 112 may be passive markers which reflect the lights emitted from the first and second light sources 150 and 151.
The first and second light sources 150 and 151 emit the lights toward the markers 110111 and 112 at different position to each other. For example, the first light source 150 may directly emit the light toward the markers 110111 and 112, and the second light source 151 may emit the light toward the beam splitter 160, which is arranged between the lens portion 120 and the image forming unit 130. The light emitted from second light source 151 is partially reflected by the beam splitter 160, and is emitted toward a center of the markers 110111 and 112 after passing the lens portion 120.
In other words, a portion of the light emitted from the second light source 151 is reflected by the beam splitter 160 and emitted toward the center of the markers 110111 and 112 through the lens portion 120, and the rest of the light passes the beam splitter 160.
Alternatively, the second light source 151 may directly emit the light toward the makers 110111 and 112, and the first light source 150 may emit the light toward the beam splitter 160 such that the light is emitted toward the center of the markers 110111 and 112 through the beam splitter 160.
For example, it may be preferable to use a spot illumination as the first and second light sources 150 and 151 such that the lights are reflected at one point within the whole surface of the markers 110111 and 112.
The lens portion 120 passes the lights which is directly reflected by the markers 110111 and 112, which is emitted from one of the selected light source among the first and second light sources 150 and 151, and the re-reflected light which is emitted from the other light source, reflected by the beam splitter 160 and emitted toward the center of the markers 110111 and 112 through the lens potion 120
The beam splitter is arranged on a rear portion of the lens portion 120. The beam splitter 160 partially passes the light emitted from one light source which is selected among the first and second light sources 150 and 151, and reflects and emits the remaining light toward the center of the markers 110111 and 112 after passing the lens portion 120.
In other words, the image forming unit 130 is arranged on a rear portion of the beam splitter 160 and forms a pair of maker images for each of the markers 110111 and 112 by receiving the lights which is directly emitted from one light source which is selected among the first and second light sources 150 and 151, reflected by the markers 110111 and 112, and have sequentially passed the lens portion 120 and the beam splitter 160, and the light which is emitted from the other light source toward the beam splitter 160, reflected by the beam splitter 160 and have passed the lens portion 120, emitted and re-reflected toward the center of the markers 110111 and 112, and have sequentially passed the lens portion 120 and the beam splitter 160.
Herein, a portion of the light which is emitted from the first and second light sources 150 and 151, re-reflected by the markers 110111 and 112 and have passed the lens portion 120 and flowed in the beam splitter 160, is reflected by the beam splitter 160 and the rest of the light is flowed into the image forming unit 130 after passing the beam splitter and forms a pair of maker images for each of the markers 110111 and 112.
For example, the image forming unit 130 may be a camera integrating an image sensor 131 which forms a pair of maker images for each of the markers 110111 and 112 by receiving the lights which are emitted from the first and second light sources 150 and 151, reflected by the markers 110111 and 112, sequentially have passed the lens portion 120 and the beam splitter 160.
Meanwhile, a diffuser 170 may be arranged between the light source which emits light toward the beam splitter 160 among the first and second light sources 150 and 151 and the beam splitter 160 to diffuse the light toward the beam splitter 160.
The processor 140 calculates three-dimensional coordinates of the markers 110111 and 112 by using the pair of the maker images for each of the markers 110111 and 112 which are formed on the image forming unit 130, in which the lights are emitted from the first and second light sources 150 and 151, reflected by the markers 110111 and 112 and are formed on the image forming unit 130, and calculates spatial position information and direction information of the markers 110111 and 112 which are attached on the target 200 such as a lesion or a surgical instrument by comparing the three-dimensional coordinates of the markers 110111 and 112 with geometric information between the markers 110111 and 112 adjacent to each other.
Herein, the memory 141 is integrated in the processor 140. Meanwhile, the memory 141 integrated in the processor 140 may pre-store geometric information of the markers 110111 and 112 adjacent to each other, in other words, length information of straight lines L1 L2 and L3 connecting the markers 110111 and 112 adjacent to each other, and angle information A1 A2 and A3 formed by a pair of the straight lines connecting the markers 110111 and 112 adjacent to each other.
In addition, the memory 141 integrated in the processor 140 may pre-store spatial position information and direction information of the first and second light sources 150 and 151.
Referring to
For the convenience of explanation, it will be described as an example in the case that a first light source is arranged to directly emit a light toward to markers, and a second light source is arranged to emit a light toward a markers through a beam splitter.
Referring to
In more detail, a spot illumination emitted from the first light source 150 are directly irradiated toward the first to third markers 110111 and 112, and a spot illumination emitted from the second light source 151 is irradiated toward a beam splitter 160 such that a portion of the light passes the beam splitter and the remaining light is reflected by the beam splitter, passes a lens portion 120, and emits toward a center of the first to third markers 110111 and 112.
The spot illuminations emitted from the first and second light sources 150 and 151, which are emitted toward the markers 110111 and 112, are reflected toward a lens portion 120 by the first to third markers 110111 and 112 (S120).
In more detail, the light emitted from the first light source 150 is directly reflected at one position of a surface of the first to third markers 110111 and 112, reflected toward the lens portion 120 through a first to third optical paths AX1 AX2 and AX3, the light emitted from the second light source 151 is irradiated toward the beam splitter 160, a portion of the light passes the beam splitter 160 and the remaining is reflected by the beam splitter 160, passes the lens portion 120 through a fourth to sixth optical paths AX4 AX5 and AX6, and emits toward the first to third markers 110111 and 112.
The lights which pass the lens portion 120 through the first to sixth optical paths AX1 AX2 AX3 AX4 AX5 and AX6 form a pair of maker images on an image forming unit 130 for each of the markers 110111 and 112 (S130).
In more detail, the light which is emitted from the first light source 150, reflected by the first marker 110 through the first optical path AX1 and have passed the lens portion 120, forms a first image of the first marker 110 on the image forming unit 130, and the light which is emitted from the second light source 151, reflected by the first marker 110 through the fourth optical path AX4 and have passed the lens portion 120, forms a second image of the first marker 110 on the image forming unit 130. Also, the light which is emitted from the first light source 151, reflected by the second marker 111 through the second optical path AX2 and have passed the lens portion 120, forms a first image of the second marker 111 on the image forming unit 130, and the light which is emitted from the second light source 151, reflected by the second marker 111 through the fifth optical path AX5 and have passed the lens portion 120, forms a second image of the second marker 111 on the image forming unit 130. Also, the light which is emitted from the first light source 150, reflected by the third marker 112 through the third optical path AX3 and have passed the lens portion 120, forms a first image of the third marker 112 on the image forming unit 130, and the light which is emitted from the second light source 151, reflected by the third marker 112 through the sixth optical path AX6 and have passed the lens portion 120, forms a second image of the third marker 112 on the image forming unit 130.
As described above, after forming the first and second images of the first to third markers 110111 and 112 on the image forming unit 130, three-dimensional coordinates of the first to third markers 110111 and 112 are calculated by using the first and second images of the first to third markers 110111 and 112 formed on the image forming unit 130 through a processor 140 (S1410).
Detailed explanation of calculating the three-dimensional coordinates of the first to third markers 110111 and 112 is described below.
In order to calculate the three-dimensional coordinates of the first to third markers 110111 and 112, first, two-dimensional central coordinates are calculated through the processor 140 by using image forming positions of the first and second images of the first to third markers 110111 and 112 formed on the image forming unit 130 (S141).
Calculating the two-dimensional central coordinates of each of the markers 110111 and 112 is explained in detail. For the convenience of explanation, in
Also, as shown in
As shown in
Therefore, the two-dimensional central coordinates of each of the markers 110111 and 112 are calculated through the processor 140 by using the image forming positions of the first images of the first to third markers 110111 and 112, the image forming positions of the second maker images, the position information of the first and second light sources 150 and 151 which are pre-stored in the processor 140, and radius information of the first to third markers 110111 and 112.
The process of calculating the two-dimensional central coordinates of the markers by the processor is described below.
As shown in
Meanwhile, a straight line Ũ including U1, V1 and x1,y1, a straight line {tilde over (V)} including U2, V2 and x2,y2, a straight line Ī including I1, J1 and x1,y1, and a straight line {tilde over (J)} including I2, J2 and x2,y2 may be represented as a Formula 2.
Herein, t1, t2 are values which determine the length.
Meanwhile, Ũ,{tilde over (V)},Ĩ, and {tilde over (J)} may be represented as a Formula 3.
And referring to
Meanwhile, and an error E of x, y, t1, and t2 may be represented as a Formula 6 by using the four equations included in the Formula 4 and Formula 5.
Since x, y, t1, and t2 are parameters of the Formula 6, the two-dimensional central coordinate of the first marker 110 is calculated by a processor 140.
Two-dimensional central coordinates of second and third markers 111 and 112 are calculated by the processor 140 by repeating the process described above.
Meanwhile, a process of calculating a coordinate of a reflection position of a light which is emitted from the first light source 150 or the second light source 151 and reflected on a surface of a first marker 110 is described below.
As shown in
[Formula 7]
(x−e)2+(y−f)2=r2 (1)
y=(tan θ−tan θ′)/(1+tan θ tan θ′)×−(tan θ−tan θ′)/(1+tan θ tan θ′)+f (2)
y=tan θx (3)
y=((tan θ−(2 tan θ′/(1−tan θ′2)))/(1+tan θ(2 tan θ′)/(1−tan θ′2))))x+d (4)
Therefore, a square of a radius of a first marker 110 may be represented as a Formula 7, and the Formula 1 is represented as equations (2)-(4) of the Formula 7, a coordinate of y-axis of a reflection position in which a light emitted from a first light source 150 is reflected on a surface of a first marker 110 by solving the equation (4) of the Formula 7.
Meanwhile, a coordinate of x-axis of a reflection position in which a light emitted from a first light source 150 is reflected on a surface of a first marker 110 by solving the equation (1) of the Formula 7.
Therefore, coordinates of reflection positions in which lights emitted from first and second light sources 150 and 151 and reflected on surfaces of a first to third makers 110111 and 112 are calculated by repeating the process described above.
Then, three-dimensional coordinates of the first to third markers 110111 and 112 are calculated through the processor 140 by using the two dimensional coordinates of the first to third markers 110111 and 112 (S142).
As described above, after calculating the three-dimensional coordinates of the makers 110111 and 112, spatial position information and direction information of a target 200 are calculated by comparing the three-dimensional coordinates of each of the markers 110111 and 112 with geometric information, which is pre-stored in the processor 140, between the markers 110111 and 112 adjacent to each other (S150).
Herein, the geometric information between the markers 110111 and 112 adjacent to each other may be length information L1 L2 and L3 virtually connecting the markers 110111 and 112 adjacent to each other and angle information A1 A2 and A3 formed by a pair of the straight lines which connect the markers 110111 and 112 adjacent to each other.
In other words, spatial position information and direction information of a target 200 are calculated by comparing three-dimensional coordinates of each of the markers 110111 and 112 with length information L1 L2 and L3 virtually connecting the markers 110111 and 112 adjacent to each other and angle information A1 A2 and A3 formed by a pair of the straight lines which connect the markers 110111 and 112 adjacent to each other, which are pre-stored in the processor 140.
As described above, in a tracking system and a method using the same according to an embodiment of the present invention, one light source of a pair of light sources 150 and 151 which is positioned different to each other directly emits a light toward a specific point of a surface of each of the markers 110111 and 112 and reflects the light toward a lens portion 120 such that first images of each of markers 110111 and 112 are formed on an image forming unit 130, the other light source emits a light such that the light is emitted toward a center of each of the markers 110111 and 112 through a lens portion 120 and the light is re-reflected toward a lens portion 120, and second images of each of makers 110111 and 112 are formed on an image forming unit 130, and therefore, a pair of images of each of markers 110111 and 112 are formed on an image forming unit 140.
In other words, three-dimensional coordinates of each of markers 110111 and 112 are calculated by using one image forming unit 130 through a trigonometry since a pair of images of markers for each of markers 110111 and 112 are formed on an image forming unit 130 positioned different to each other.
Therefore, in a tracking system and a method using the same according to an embodiment of the present invention, spatial position information and direction information of markers 110111 and 113 which are attached on a target 200 are calculated by using only one image forming unit 130.
Therefore, there is an effect of reducing a manufacturing cost of the tracking system with small and lightweight, and relatively low restriction of surgical space comparing with conventional tracking system.
It will be apparent to those skilled in the art that various modifications and variation can be made in the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
10-2013-0018469 | Feb 2013 | KR | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/KR2014/000979 | 2/5/2014 | WO | 00 |