Robotic laser scalpels are used in different surgical tasks, such as eye surgery, neurosurgery, and dermatology. Precise control of laser energy delivery to tissues ensures optimal treatment of targeted lesions and tissue regions, making robotic control of these systems an essential element for precision laser guided surgery. A single laser pulse can create a 3D volumetric cavity on the tissue surface, and its shape is related to the laser incident angle and laser energy delivery. Incorrect orientation planning and laser energy delivery can result in damage to collateral “non-target” (e.g., healthy) tissue removal. In other words, different laser orientations can create various tissue ablation cavities, and incorrect incident angles can cause over-irradiation of healthy tissue that should not be ablated.
Robotic laser orientation problems have been widely studied in robotic laser cutting, industrial robotic manipulation, and robotic laser surgery. However, prior efforts do not minimize errant tissue overcutting based on laser incident angle. These methods generally leverage the vision or user inputs to develop an orientation planning strategy. The laser scalpel is typically affixed to the 6-DOF (degree-of-freedom) robotic arm end-effector, with the robot controlling the laser in order to move towards a predefined planning trajectory. Such robotic orientation planning can improve the safety and robustness of these surgical systems. For example, as shown in
The development of a laser orientation planner is an important problem in robotic laser surgery. However, two problems associated with the laser orientation planning include:
The Summary is provided to introduce a selection of concepts that are further described below in the Detailed Description. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in limiting the scope of the claimed subject matter.
As disclosed herein, robotically controlled laser orientation is optimized to minimize errant overcutting of healthy tissue during the course of pathological tissue resection. An optimization problem is formulated to find the optimal laser orientation in order to minimize the possibility of excessive laser-induced tissue ablation. First, a 3D data-driven geometric model is developed to predict the shape of the tissue cavity after a single laser ablation. Modelling the “target” and “non-target” tissue region by an obstacle boundary, the determination of an optimal orientation is converted to a collision-minimization problem. The goal of this optimization formulation is maintaining the ablated contour distance from the obstacle boundary, which is solved by the projected gradient descent method. Simulation experiments validate the proposed method with conditions of various obstacle shapes and different initial incident angles.
As disclosed herein, an optimal laser incident orientation is described to minimize the distance between the ablated profile and a pre-defined obstacle boundary. This boundary can be defined as either a preoperative “edge” between healthy and non-healthy tissue, or a pre-designed cavity contour with a fixed geometry.
As disclosed herein, an optimization model is provided to solve the robotic laser orientation planning problem with an application of minimizing the over-cutting of healthy tissue for robotic laser surgery. The laser-tissue cavity can be predicted under various incident angles and the orientation planning can be guided by the reference vector field. With this method, the surgeon can manually define a preoperative boundary or a 3D contour, and the planner can calculate an optimal incident angle to minimize the probability of cutting incorrect tissue regions during surgery.
One aspect of the present disclosure provides a method of controlling a robotic laser orientation angle for surgical planning and simulation, with application for biological laser-tissue resection. The method develops a laser-tissue geometric model by using a 3D data driven method (e.g., Gaussian function fitting for parameter estimation). A laser-tissue kinematic model estimates the depth of cut based on the ablation center on the biological tissue surface. The analytical gradient of the laser-tissue kinematic system is derived and used to formulate an optimization problem, which is solved with a gradient-based method.
In one aspect, the present disclosure provides a method comprising ablating a substrate with a laser at an orientation to create a cavity in the substrate, scanning the cavity, and creating a three-dimensional surface for the cavity. The method further includes storing the three-dimensional surface in a dataset. The dataset includes a laser projected distance as an independent variable and a depth of cut as a dependent variable. The method further includes fitting parameters of a gaussian-based model for the laser and the substrate based on the dataset.
In some embodiments, the orientation is a first orientation, the cavity is a first cavity, the three-dimension surface is a first three-dimensional surface, and wherein the method further includes: ablating the substrate with the laser at a second orientation to create a second cavity in the substrate; scanning the second cavity; creating a second three-dimensional surface of the second cavity; and storing the second three-dimensional surface in the dataset.
In some embodiments, the substrate is a biological tissue.
In some embodiments, scanning the cavity is with optical coherence tomography or micro computed tomography.
In some embodiments, the method further includes creating a sequence of cross-sectional images of the cavity.
In some embodiments, the sequence of cross-sectional images is filtered, segmented, and concatenated to create the three-dimensional surface for the cavity.
In some embodiments, the method further includes predicting a post-ablation surface using the gaussian-based model.
In some embodiments, predicting the post-ablation surface includes providing a pre-ablation profile and a set laser orientation.
In some embodiments, the method further includes projecting a point on the pre-ablation surface to a laser reference plane and calculating a projected distance to a laser center on the gaussian-based model.
In some embodiments, the method further includes determining a predicted depth of cut based on the projected distance to the laser center.
In some embodiments, the method further includes collecting predicted depth of cuts for all points on the pre-ablation surface to generate the predicted post-ablation surface.
In one aspect, the present disclosure provides a method comprising providing a pre-ablation surface; labeling a three-dimensional obstacle boundary that separates material to be remove by a laser and material to remain; and determining an orientation of the laser that results in a predicted post-ablation surface that does not intersect the three-dimension obstacle boundary.
In some embodiments, the method further includes predicting a plurality of post-ablation surfaces for a plurality of orientations of the laser.
In some embodiments, predicting the plurality of post-ablation surface utilizes a gaussian-based model for the laser.
In some embodiments, the method further includes generating a Euclidean distance transform metric for each of the plurality of post-ablation surfaces.
In some embodiments, the Euclidean distance transform metric is a measurement of a distance between a query point on the post-ablation surface to the closest point on the three-dimensional obstacle boundary.
In some embodiments, a raw distance value from the Euclidean distance transform metric is post-processed to an oriented distance value with a negative value if the query point crosses the three-dimensional obstacle boundary.
In some embodiments, the Euclidean distance transform metric is converted to an obstacle cost.
In some embodiments, the obstacle cost is minimized with a gradient-based constrained optimization method.
In some embodiments, determining the orientation of the laser that results in the predicted post-ablation surface that does not intersect the three-dimension obstacle boundary includes maximizing a Euclidean distance transform metric between the predicted post-ablation surface and the three-dimensional obstacle boundary.
In some embodiments, the method further includes moving the laser to the orientation and energizing the laser.
In some embodiments, the pre-ablation surface is tissue, the three-dimensional obstacle boundary separates tissue to be remove by the laser and tissue to remain; and wherein the laser is energized for laser-tissue resection.
The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
The accompanying figures are provided by way of illustration and not by way of limitation.
Section headings as used in this section and the entire disclosure herein are merely for organizational purposes and are not intended to be limiting.
All publications, patent applications, patents and other references mentioned herein are incorporated by reference in their entirety.
The terms “comprise(s),” “include(s),” “having,” “has,” “can,” “contain(s),” and variants thereof, as used herein, are intended to be open-ended transitional phrases, terms, or words that do not preclude the possibility of additional acts or structures. The singular forms “a,” “and” and “the” include plural references unless the context clearly dictates otherwise. The present disclosure also contemplates other embodiments “comprising,” “consisting of” and “consisting essentially of,” the embodiments or elements presented herein, whether explicitly set forth or not.
For the recitation of numeric ranges herein, each intervening number there between with the same degree of precision is explicitly contemplated. For example, for the range of 6-9, the numbers 7 and 8 are contemplated in addition to 6 and 9, and for the range 6.0-7.0, the number 6.0, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8, 6.9, and 7.0 are explicitly contemplated. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise-Indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. For example, if a concentration range is stated as 1% to 50%, it is intended that values such as 2% to 40%, 10% to 30%, or 1% to 3%, etc., are expressly enumerated in this specification. These are only examples of what is specifically intended, and all possible combinations of numerical values between and including the lowest value and the highest value enumerated are to be considered to be expressly stated in this disclosure.
As used herein, the term “subject” and “patient” as used interchangeably herein and refer to both human and nonhuman animals. In some embodiments, the subject comprises a human who is undergoing a procedure using a system or method as prescribed herein. The subject or patient may be undergoing various forms of treatment.
As used herein, the term “pre-ablation surface” refers to the original surface of the exposed tissue, and this is a 3D contour. For example, in neurosurgery, the pre-ablation surface is the exposed portion of the tumor tissue. As another example, in eye surgery, the pre-ablation surface is the exposed surface of the cornea.
As used herein, the term “post-ablation surface” refers to an 3D surface that is updated after a one-shot laser-tissue ablation to the pre-ablation surface.
As used herein, the term “3D query point” refers to one of the 3D points at the pre-ablation tissue surface.
As used herein, the term “depth of cut” refers to the distance between the pre-ablation surface and the post-ablation surface of a 3D query point towards the laser incident orientation.
As used herein, the term “laser reference plane” refers to a synthetic plane defined by a laser incident orientation and a reference center. The reference center is determined by projecting a surface point (e.g., the one that shows the greatest depth of cut) towards the laser incident orientation. The laser incident orientation and the reference center can thus formulate a synthetic reference plane as a representation of the laser energy profile. The symmetric Gaussian-based profile represents that the closer to the laser origin (e.g., the center of the Gaussian function), the greater the energy received and longer depth of cut.
As used herein, the term “laser projected distance” refers to the distance between the projected coordinate and the reference center (both at the laser reference plane). Given a specific laser incident orientation, a 3D query point can be projected to a laser reference plane, thereby determining a 2D projected coordinate. The distance between the projected coordinate and the reference center is calculated as the laser projected distance.
As used herein, the term “Gaussian-based model” refers to a standard 3D bell-shaped Gaussian function parameterized by a standard deviation and an amplitude.
As used herein, the term “safe tissue region” refers to the volume of tissue that can be removed. For example, the tumorous region in brain surgery or the volume of tissue to be removed during a corneal reshaping in ophthalmic surgery.
As used herein, the term “un-safe tissue region” refers to surrounding or underlying healthy tissue that should not be removed during a procedure and is protected by employing the methods detailed herein.
As used herein, the term “obstacle boundary” refers to the intersection of the safe tissue region and the un-safe tissue region. In some embodiments, the obstacle boundary is the boundary of the tumor where it touches healthy tissue. In other embodiments, the obstacle boundary is a synthetic boundary between the area to be removed and the area to be left in place after a comeal reshaping surgery.
As used herein, the term “voxel” refers to the smallest unit of volume in a volumetric 3D image, such as those collected from biomedical imaging systems (e.g., MRI, optical coherence tomography, CT, etc.). A voxel is akin to a pixel in a 2D image, but for a 3D image.
Unless otherwise defined herein, scientific and technical terms used in connection with the present disclosure shall have the meanings that are commonly understood by those of ordinary skill in the art. The meaning and scope of the terms should be clear; in the event, however of any latent ambiguity, definitions provided herein take precedent over any dictionary or extrinsic definition. Further, unless otherwise required by context, singular terms shall include pluralities and plural terms shall include the singular.
The laser-tissue geometric model refers to a system capable of predicting the ablated tissue contour under a given laser incident angle. Given a 3D point on the tissue surface, the model disclosed herein predicts the depth-of-cut, and thus estimates the new position after tissue removal. The main modelling approaches can be categorized as Model-based and Model-free methods.
Model-based methods estimate the ablated contour by calculating the energy delivery for each tissue position. Material removal is then predicted based on the energy incident at each point on the surface and an ablation rate. However, modelling the complex physics for laser-tissue interaction (e.g., thermal effect, optical property, etc.) is difficult and prone to different experimental conditions such as water spray, surface geometry and tissue material.
Model-free methods model the tissue removal by using an entirely data-driven approach. Data driven approaches are advantageously because it is difficult to model the laser-tissue interaction due to the heterogeneity of tissue material and the complex physical mechanism. The laser beam profile can be modelled by a Gaussian function. The tissue of removal should follow the similar pattern since the depth-of-cut is related to the strength of energy delivered to the target. Therefore, the Gaussian-based model is used to describe the laser-tissue relation, and the parameters of the Gaussian function can be learned through the 3D cavity data collected. In some embodiments, the 3D cavity data is collected by high resolution scanners such as confocal microscopy and computed tomography (CT).
The method of learning laser-tissue physics by Gaussian function fitting is applied in various robotic laser applications, such as tissue depth control, surgical simulation, and generating a cutting path for volumetric resection. However, these studies have not discussed the problem of laser orientation planning with various laser angles and controlling the ablated profiles for robotic laser surgery, which is detailed herein.
Gaussian Profile. In some embodiments, the laser is a CO2 laser with a wavelength of approximately 10.6 μm and a l/e2 spot size of approximately 0.80 mm. The CO2 beam profile can be generally described by a Gaussian function.
To model the laser-tissue geometry, a “laser reference plane” is defined with the laser center and the orientation vector, where the laser center aligns with the surface point that shows the greatest depth of cut. The surface points can be projected to this plane to estimate the depth-of-cut. A region of interest (ROI) is defined on the superficial tissue surface. In this configuration, qc ∈3 denotes the laser ablation center on the 3D surface and qi ∈
3 is a query point around this ablation center, with EQN. 1.
p
c
=q
c
−v*L
ref [1]
Where qc ∈3 is defined as the laser incident center and v∈R3 is the incident vector. The operator ∥·∥2 refers to the L2-norm and typically restricted to ∥v∥2=1. Lref is a reference distance that can be set as an arbitrary constant, since the geometric configuration does not depend on this value (set as “1”).
The depth-of-cut di for the surface point qi is defined as the tissue removal at the incident direction. Based on the Gaussian beam profile assumption, di can be calculated by the projected distance si between pi and pc as EQN. 2.
Where exp(·) is an exponential operator. The geometric gaussian profile is assumed symmetric and the parameters of LG and σG can be estimated by the data-driven method for a specified tissue material and laser setting.
Another important task is to determine si, which is the altitude of the triangle with the sides of pc and qc, as EQN. 3.
Where √{square root over (h(h−a) (h−b) (h−c))} is the triangle area denoted by the three sides a(·)=∥qc−qi∥2; b(·)=∥qi−qc+v(θ)*L∥2; c(·)=∥pc−qc∥2=L=1, and have
Data-driven Method for Gaussian Function Parameters. Given a laser-tissue geometric model, the amplitude LG and variance parameter σG are estimated. In practice, the amplitude is a function of laser power, time, and the optical properties of the tissue that drive ablation. Herein, any ablation amplitude is assumed achievable by tuning the laser parameters and thus considered as a controllable variable. The LG and σG can be function-fitted with the 3D data collected by high-resolution scanner. Micro-CT data is used to characterize the laser-tissue cavities under various incident angles for 2D and 3D analysis.
A dataset of 3D measurements from a list of 3-DOF orientation angles {θi}∈3. Each laser pulse at θi can create a tissue cavity described by the Micro-CT point cloud data, which is defined as {qi,j}∈
3. The i refers to the index of incident angle and j as the index of measurement. The point cloud can be converted to the projected coordinates at the laser reference plane and this is referred as {si,j}∈
3.
With each si,j, the depth-of-cut di,j can be calculated by EQN. 2 and obtain {di,j}∈3. Therefore, a dataset {si,j, di,j}∈
3 is obtained, which can be used to estimate the Gaussian function parameters (LG and σG). In some embodiments, the function fitting is achieved by using the log operation and the nonlinear least-squares fitting metric.
At the end of model tuning, a function is provided that is parametrized by laser parameters and laser incident angle, which can accurately predict the geometric shape of a crater created by a one-shot laser ablation in the target tissue. This model is specifically tuned for the tissue the interrogation cut was made it. Thus, if this method is being used in a surgery, the function is specific to that patient's tissue (e.g. tumor).
In one embodiment, the method includes (STEP 1) perform an interrogation cut with a single laser ablation in the tissue at a known angle of incidence. (STEP 2) acquires a profile (e.g., a 3D image”) of the created cavity with a 3D imaging system. In some embodiments, the 3D image is created by concatenating multiple 2D images scanned from the 3D imaging system, such as a biomedical imaging system (MRI, CT, Optical Coherence Tomography). (STEP 3) in some embodiments includes performing image processing steps on the 3D image to prepare it for algorithm use. In some embodiments, the image processing includes smoothing the image, noise filtering, and other suitable image processing operations. (STEP 4) includes extracting geometric parameters from the acquired 3D image of depth of cut and the laser projected distance to prepare for data-driven model fitting. These two parameters formulate a geometric relation about how a laser orientation angle can affect the shape of the surface cavity created. The crater shape (e.g., depth of cut) from the 3D image acquired and the laser projected distance (e.g., distance between the surface-to-plane projected point to the center of the laser origin). The “depth of cut” is the distance between the pre-ablation and post-ablation surface points. The “projected distance” is measured by first projecting a surface to a reference plane defined by the laser origin and the incident orientation to get a projected coordinate, and secondly, calculating the distance between the projected coordinate and the laser origin (at the reference surface). (STEP 5) includes fitting a Gaussian-based model to the geometric parameters of the cavity. The Gaussian beam is assumed to be symmetric and normally distributed function described by two scalar parameters: amplitude and variance. The regression problem (e.g., Gaussian function fitting to find the amplitude and variance) performs over all the points within the cavity. Each point is considered as a unique data point to fit the model.
The result of the method disclosed herein is a tuned function that is parameterized by laser parameters (e.g. wavelength, pulse time or irradiance time, spot size, and power profile which is assumed to be Gaussian in this case) and incident angle for the specific tissue being cut. This method does not require the complex physical modelling associated with the heat transfer, chemical reaction and optical tissue property, and but rather solves the non-linear regression problem. As mentioned, modelling the laser-tissue interaction is a difficult problem due to the heterogeneity of tissue material and the complex physical mechanism. The data-driven method disclosed herein encodes the physical mechanism into the geometric model and can be uniquely adjusted to different tissue material. The learned laser-tissue interaction model plays an important role in the design of laser orientation planner with numerical optimization approaches. Optimizing the laser orientation can show potential applications of minimizing errant overcutting of healthy tissue during the course of pathological tissue resection.
In some embodiments, the method disclosed herein is used with other surgical laser systems with lasers of different wavelengths and power profiles shapes other than gaussian, such as those equipped with Nd:YAG and Er:YAG lasers or lasers with flat-top beam profiles.
Laser-Tissue Kinematic System. The laser-tissue geometric model can be referred as a kinematic system that describes the relation between the laser incident angle and the resulting ablation contour. For a given incident direction, this contour is formulated by the surface sampled points after the tissue removal. For each surface point qik ∈3 at k-th time step, results in EQN. 4.
q
i
k+1(v, qik)=qik+v*di [4]
Where
from EQN. 2. qik+1 ∈3 is the updated position from qik. The LG and σG are the learned Gaussian function parameters. The si(v, qik) depends only on v and qi. This kinematic system can be used to predict new surface profiles given the initial surface point cloud and the incident angle.
The data-driven Laser-tissue interaction model is used to formulate the kinematics to update the shape of the laser-tissue cavities. Each query point, as considered as a small robot system, can formulate a kinematic model where the given laser orientation and ablation center can be mapped to a new positioning output. The summation of new positions of all the 3D query points at the surface equal to a new surface cavity to show the resulting effect of the laser-tissue interaction. Method steps include: (STEP 1) For each 3D query point at the pre-ablation surface, project the point to the laser reference plane. This can be used to calculate the projected distance to the center of the laser origin. (STEP 2) The laser projected distance is served as an input to the data-driven laser-tissue model and mapped to an output of the depth of cut. (STEP 3) A kinematic model is used the predicted depth of cut to describe the positioning output. (STEP 4) the new positions of all the query points at the surface and formulate the post-ablation laser cavity.
In some embodiments, the optimal laser orientation problem is modelled as an “Obstacle avoidance” planning problem by which the ablated contour, as created by the current laser angle, should keep the adequate distance to the pre-defined boundary (between “target” and “non-target” tissue region). For example, in
Given a pre-defined obstacle boundary and a laser-tissue geometric model, the goal is to find an optimal laser orientation that creates a collision-free ablation contour.
Collision Modelling with Obstacle Boundary. Regarding the EDT field, in some embodiments, the goal is to find an optimal laser orientation that can maintain the distance between the ablated contour and the obstacle boundary. The obstacle boundary is modeled by the Euclidean distance transforms (EDT). EDT represents the distance to the nearest obstacle position for each 3D voxel, which can be used to measures the level of collision for an arbitrary 3D position.
Regarding obstacle modelling, the obstacle boundary can be considered as a 3D surface where the EDT value is zero. Specifically, Φ(·) is defined as the EDT levelset function and Φ(·)=0 denotes the boundary positions. Φ(·)>0 denotes the target region and Φ(·)<0 denotes the non-target region, as shown in
Where ϵ is the obstacle distance threshold; Φ is the EDT value and x is the 3D voxel position. The gradient of the cost function
is negative if the position is getting closer to the obstacle boundary, which will guide the point to move to a reverse direction.
Orientation Planning Algorithm. The orientation planning minimizes the total obstacle costs for the ablated contour. The contour can be represented by a point cloud data that can be collected by any intraoperative 3D sensors. In practice, a laser incident orientation is usually perpendicular to the point of interest and is only allowed to adjust to a small angle. A constrained optimization problem is formulated as EQN. 6.
This objective function f(·) denotes the summation of obstacle costs for each point at the ablated contour. The incident orientation v(θ) is controlled by the angle vector θ∈3. The C(·) represents the obstacle cost at the current 3D voxel position. Φ(·) is the EDT distance function. Q(·) is the function operator following the kinematic model in EQN. 4. Θ1 and θ2 describe the 3-DOF angle range. As the ablation center qc is fixed, the laser orientation v(θ) is related to the incident angle and the surface point qi.
Regarding Analytical gradient, the analytical gradient of the objective function is important for the optimization solver and can be derived by chain rule. ∇C(·) and ∇Ψ(·) can be obtained by EQN. 5 and the EDT field. For ∇q(θ), the kinematic system model q( ) is simplified and rewritten as EQN. 7.
q=q
i
+v(θ)*LG*exp(μ(v(θ))) [7]
To take the derivative of q(_) with respect to the incident angle θ, EQN. 8.
The J is the Jacobian matrix of the system model and each item is
(I and j are index of the item in J). Also, EQN. 9.
Where Rx, Ry and Rz are the rotation angles that control the initial orientation vector v0. The
can be obtained by taking the derivative for the rotation matrix. The
is a scalar function of si( ). Finally, the analytical gradient ∇f(θ, qi) is obtained for the objective function as EQN. 10.
is the Jacobian matrix in EQN. 9 and can be calculated from EQN. 9.
Regarding the projected gradient descent, for the incident angle constraint, the Projected gradient descent method, referred to as Proj-GD, is used in some embodiments to solve the optimization problem. For each iteration, Proj-GD first computes the gradient update as EQN. 11.
θk+1=θk−α*∇f(θ) [11]
Where α is the step size for the gradient update. For the angle constraint, Proj-GD finds a new update by solving a quadratic optimization problem of EQN. 12.
Where θk+1 is from EQN. 11 and θ*k+1 denotes the final gradient after Proj-GD. The Proj(_) is the operator for Projected gradient descent and it represents the nearest feasible point to gradient-updated position. An example of Proj-GD is depicted in
gain(θk)=C(θ*k+1)−C(θk) [13]
For each iteration, the gain is ranked with an increasing order and select the Top-N sample data (N can be set as 30% of the sample data). This method ensures the adequate cost reduction and guarantees that effective sample points could be selected to guide the laser orientation planning.
As detailed herein, once a model is tuned via the method described above, it can be used to optimize ablation cuts thereby optimizing surgical planning. This requires defining an “obstacle boundary”, usually based on where the final desired shape of the tissue surface is (in the case of tumor surgery this is the boundary of the tumor, in eye surgery this is the final shape of the cornea). The obstacle boundary is defined by the surgeon, where the final desired shape should be located inside this boundary (otherwise it will cut healthy tissue outside of the “safe region”, referred as “overcutting”). In essence, given a target geometric profile, the surgeon (or surgical system) can now use the tuned model to determine the appropriate laser parameters and incidence angle to perform a cut that produces a feasible profile.
In one embodiment, the method includes: (STEP 1) acquire a profile (e.g., 3D image) of the crater with an imaging system; (STEP 2) label an obstacle boundary profile from the 2D slide images (obtained from the 3D scanner) to be achieved after laser ablation. In some embodiments, this can be a user defined obstacle boundary or automatically determined through segmentation in the biomedical image. (STEP 3) combine multiple 2D labelled boundaries to create a final 3D obstacle boundary profile. (STEP 4) convert the obstacle boundary profile to a Euclidean Distance Transform (EDT). As detailed herein, EDT is a method to quantify the distance between a query 3D point to the closest point at the obstacle boundary. EDT is used to measure the level of the collision for a 3D point (e.g., the one closer to the boundary is assigned a smaller EDT value because it is closer to the obstacle boundary). (STEP 5) determine the laser parameters that will achieve the desired cut within the obstacle boundary. The laser parameters are determined by solving the optimization problem detailed herein. The optimal laser incident angle is determined that can minimize the sum of obstacle costs (e.g., maximize the Euclidean Distance Transform metric between the predicted resulting surface profile and the obstacle boundary profile. This indicates that the resulting surface profile is far away from the obstacle boundary, which is located at a “safe region.” In some embodiments, a mapping function converts the EDT metric to the obstacle cost, thus the optimization becomes a minimization problem instead of maximization. In some embodiments, a piecewise mapping function maps an EDT metric to an obstacle cost. In other words, given a 3D obstacle boundary with the associated Euclidean distance transforms, an arbitrary 3D position with a value referred as “obstacle cost” can be assigned as a way to measure the level of collision. This minimization can be achieved in some embodiments with the projected gradient method, or any suitable gradient-based constrained optimization method. (STEP 6) perform the ablation with the laser parameters determined. The results of the method disclosed herein finds the laser incident angle that results in the desired geometry of the post-ablation surface. In other words, the method determines an optimal laser incident vector to create a post-ablation profile that is inside the safe tissue region.
In some embodiments, the methods disclosed herein are utilized with conducting surgical simulation with different 3D sensor guided robotic laser platforms, such as stereovision, RGB-D camera and optical coherence tomography (OCT).
Embodiments of the present disclosure include a method comprising ablating a substrate with a laser at an orientation to create a cavity in the substrate, scanning the cavity, and creating a three-dimensional surface for the cavity. The method further includes storing the three-dimensional surface in a dataset. The dataset includes a laser projected distance as an independent variable and a depth of cut as a dependent variable. The method further includes fitting parameters of a gaussian-based model for the laser and the substrate based on the dataset.
In some embodiments, the orientation is a first orientation, the cavity is a first cavity, the three-dimension surface is a first three-dimensional surface, and wherein the method further includes: ablating the substrate with the laser at a second orientation to create a second cavity in the substrate; scanning the second cavity; creating a second three-dimensional surface of the second cavity; and storing the second three-dimensional surface in the dataset. In other words, the model fitting can be conducted for a single cavity or a plurality of a cavities.
In some embodiments, the substrate is a biological tissue. In some embodiments, the substrate is a tissue phantom. In some embodiments, the substrate is human tissue.
In some embodiments, scanning the cavity is with optical coherence tomography or micro computed tomography. In some embodiments, the method further includes creating a sequence of cross-sectional images of the cavity. In some embodiments, the sequence of cross-sectional images is filtered, segmented, and concatenated to create the three-dimensional surface for the cavity.
In some embodiments, the method further includes predicting a post-ablation surface using the gaussian-based model. In some embodiments, predicting the post-ablation surface includes providing a pre-ablation profile and a set laser orientation. In some embodiments, the method further includes projecting a point on the pre-ablation surface to a laser reference plane and calculating a projected distance to a laser center on the gaussian-based model.
In some embodiments, the method further includes determining a predicted depth of cut based on the projected distance to the laser center. In some embodiments, the method further includes collecting predicted depth of cuts for all points on the pre-ablation surface to generate the predicted post-ablation surface.
As described further herein, in some embodiments, a method comprising providing a pre-ablation surface; labeling a three-dimensional obstacle boundary that separates material to be remove by a laser and material to remain; and determining an orientation of the laser that results in a predicted post-ablation surface that does not intersect the three-dimension obstacle boundary.
In some embodiments, the method further includes predicting a plurality of post-ablation surfaces for a plurality of orientations of the laser. In some embodiments, predicting the plurality of post-ablation surface utilizes a gaussian-based model for the laser.
In some embodiments, the method further includes generating a Euclidean distance transform metric for each of the plurality of post-ablation surfaces. In some embodiments, the Euclidean distance transform metric is a measurement of a distance between a query point on the post-ablation surface to the closest point on the three-dimensional obstacle boundary. In some embodiments, a raw distance value from the Euclidean distance transform metric is post-processed to an oriented distance value with a negative value if the query point crosses the three-dimensional obstacle boundary. In other words, if a new point is inside the obstacle boundary (e.g., in a safe region), the oriented distance value should be positive, and if a new point is outside the obstacle boundary (e.g., in an overcutting non-safe region), the oriented distance value should be negative.
In some embodiments, the Euclidean distance transform metric is converted to an obstacle cost. In some embodiments, the obstacle cost is minimized with a gradient-based constrained optimization method.
In some embodiments, determining the orientation of the laser that results in the predicted post-ablation surface that does not intersect the three-dimension obstacle boundary includes maximizing a Euclidean distance transform metric between the predicted post-ablation surface and the three-dimensional obstacle boundary.
In some embodiments, the method further includes moving the laser to the orientation and energizing the laser. In some embodiments, the pre-ablation surface is tissue, the three-dimensional obstacle boundary separates tissue to be remove by the laser and tissue to remain; and wherein the laser is energized for laser-tissue resection.
The systems and methods described herein can be implemented in hardware, software, firmware, or combinations of hardware, software and/or firmware. In some examples, the systems and methods described in this specification may be implemented using a non-transitory computer readable medium storing computer executable instructions that when executed by one or more processors of a computer cause the computer to perform operations. Computer readable media suitable for implementing the systems and methods described in this specification include non-transitory computer-readable media, such as disk memory devices, chip memory devices, programmable logic devices, random access memory (RAM), read only memory (ROM), optical read/write memory, cache memory, magnetic read/write memory, flash memory, and application-specific integrated circuits. In addition, a computer readable medium that implements a system or method described in this specification may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms.
One skilled in the art will readily appreciate that the present disclosure is well adapted to carry out the objects and obtain the ends and advantages mentioned, as well as those inherent therein. The present disclosure described herein are presently representative of preferred embodiments, are exemplary, and are not intended as limitations on the scope of the present disclosure. Changes therein and other uses will occur to those skilled in the art which are encompassed within the spirit of the present disclosure as defined by the scope of the claims.
It will be readily apparent to those skilled in the art that other suitable modifications and adaptations of the methods of the present disclosure described herein are readily applicable and appreciable, and may be made using suitable equivalents without departing from the scope of the present disclosure or the aspects and embodiments disclosed herein. Having now described the present disclosure in detail, the same will be more clearly understood by reference to the following examples, which are merely intended only to illustrate some aspects and embodiments of the disclosure, and should not be viewed as limiting to the scope of the disclosure. The disclosures of all journal references, U.S. patents, and publications referred to herein are hereby incorporated by reference in their entireties.
The present disclosure has multiple aspects, illustrated by the following non-limiting examples.
The data-driven method is validated using the Micro-CT data. 3D cavity point cloud data is collected from four incident angles with 10 repeated measurements. This formulates a dataset for the estimation of Gaussian function parameters LG and σG. The Gaussian function fitting is a non-linear curve fitting problem (e.g., least-squares minimization) and in some embodiments MATLAB “lsqcurvefit” optimization toolbox is used to find the fitted parameters. For the specified tissue material and laser parameters, in one embodiment, LG=1.4376 and σG=0.6486 were obtained. The model re-projected Room-mean-square error is 0.1468 and this contributes to 10.2% of the LG. This demonstrates the laser can generate a maximum depth of cut with around 1.44 mm at the ablation center. For generality of the problem, in some embodiments, a look-up table of laser-tissue models and parameters are provided, which can be used directly in robotic surgery.
Validating the Analytical Gradient: Test 1 (Example 1) aims to validate the analytical gradient ∇q(θ) by making a hypothesis that the update of the orientation can follow the EDT gradient vectors ∇Φ(q(θ)). The ∇Φ(q(θ)) refers to a vector direction with higher EDT values with respect to a surface position q(θ), which indicates a lower collision cost. For example, if given a vector ∇Φ(q(θ))>0 and using the gradient ascent θk+1=θk+α*∇θΦ(q(θk)) as the update rule, that results in EQN. 14.
This indicates that if the angle is updated by the gradient ascent direction, the EDT value can increase and the ablated contour can maximize the distance to the obstacle boundary (minimize cost). Thus, the gradient ascent with ∇Φ(θ) is equal to the gradient descent with ∇C(θ), because ∇C(θ) is negative when the EDT value is smaller than the collision threshold. For brevity, the goal of using ∇Φ(·) with the gradient ascent rule is used to validate the feasibility of the analytical gradient.
For the simulated experiments, a “point robot” is defined for a surface position qi which follows the kinematic system in EQN. 4. The “point robot” refers to the change of movement of a surface position after a single laser incidence. The validation of the analytical gradient ∇q(θ) can be achieved by observing the movement of the point robot.
First, a 2D test is conducted to show that the 1-DOF orientation angle update can be guided by a reference direction.
To validate the method, the orientation error is used to measure the difference between the reference vector and the point vector defined as vk=qk+1−qk. As the point trajectory is located at a fixed surface terrain, as shown in
Sample Points Validation: Instead of using a fixed reference vector, Test 2 (Example 2) validates the hypothesis that the point robots can follow a vector field created by an arbitrary obstacle boundary. It is noted that the gradient descent rule with an analytical gradient ∇f(θ, qi) is used in this example. The laser-tissue model parameters are estimated by the Micro-CT data, as discussed herein. Six Gaussian-shape obstacle boundaries are defined by setting various variance parameters and formulate the obstacle vector fields in simulation. For each obstacle terrain, a surface grid is defined with different initial positions by setting nine random incident angles. In this example, each surface grid contains 49 point robots and this ensures that each sample point is located at an arbitrary starting position. The goal is to control each point robot to move to a low-cost position and keep adequate distance to the obstacle boundary.
Surface Validation: Test 3 (Example 3) validates the orientation planning method by using the gradient information from all the point robots at the surface, instead of an individual one. The experimental conditions in Example 2 are applied to this example. The Projected gradient descent and the Data selection methods are tested with the analytical gradient ∇f(θ.qi).
The obstacle costs of all the point robots are collected for evaluation.
It is understood that the foregoing detailed description and accompanying examples are merely illustrative and are not to be taken as limitations upon the scope of the disclosure, which is defined solely by the appended claims and their equivalents.
Various changes and modifications to the disclosed embodiments will be apparent to those skilled in the art.
This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63/291,507 filed Dec. 20, 2021, which is incorporated herein by reference in its entirety for all purposes.
This invention was made with Government support under Federal Grant No. R01EB030982 awarded by the National Institutes of Health. The Federal Government has certain rights to the invention.
Number | Date | Country | |
---|---|---|---|
63291507 | Dec 2021 | US |