Patent Grant
9320593
This application relates generally to treatment planning systems, methods and method of their use. In particular, this application relates to hair transplantation planning systems i.e., harvesting and/or implantation of hair follicular units in a body surface, usually a scalp, and methods of their use.
Hair transplantation procedures are well-known, and typically involve (e.g., in a patient having male pattern baldness) harvesting donor hair grafts from the side and back fringe areas (“donor areas”) of the patient's scalp, and implanting the harvested follicular units in a bald area (“recipient area”). Historically, the harvested grafts were relatively large (3-5 mm), although more recently, the donor grafts may be single follicular units, which are naturally occurring aggregates of 1-3 (and much less commonly, 4-5) closely spaced hair follicles that are distributed randomly over the surface of the scalp.
In one well-known hair transplantation process, a linear portion of the scalp is removed from a donor area using a scalpel cutting down into the fatty subcutaneous tissue. The strip is dissected (under a microscope) into component follicular units, which are then implanted into a recipient area in respective incisions or puncture holes made using a needle. Forceps may be used to grasp and place the individual follicular unit grafts into the needle puncture locations, although other instruments and methods are known for performing this task.
U.S. Pat. No. 6,585,746 discloses a hair transplantation system utilizing a robotic system, including a robotic arm and a hair follicle introducer associated with the robotic arm. An imaging system is used to produce a three-dimensional image of the patient's scalp, which is used to plan the locations to receive hair grafts. The entire disclosure of U.S. Pat. No. 6,585,746 is incorporated herein by reference.
No matter what type of hair transplant procedure is adopted, it is the aim of the physician to provide his patient with a natural looking head of hair. Currently physicians try and do this based on their experience based on procedures performed on prior patients, hoping that when they harvest hair from donor areas of the patient's head and implant it into recipient areas, they manage to do so in such a way that a natural looking head of hair results. However, it is not likely that a realistic image of what the patient might look like may be obtained based on prior experience with other patients. Everyone's hair is different, it is not of the same thickness, texture or density, and hair does not lie or fall in the same way on heads of different people. Moreover, not everyone's head shape or size is the same either, so there are many differences with respect to how the hair falls, and how it is parted too. Consequently, a result of a hair transplantation procedure performed on one patient does not necessarily look the same if the same procedure is performed on a second patient. Currently, it is only possible for one to look at the before and after photos of other people, and to guess what the outcome of a hair transplantation procedure might be.
Commonly assigned U.S. Pat. No. 7,806,121 and U.S. Pat. No. 8,104,480 (hereinafter collectively, “Bodduluri”) illustrate systems and methods for planning transplantation of follicular units into a body surface of the patient. Both patents are incorporated by reference herein.
A variety of systems and methods for planning various cosmetic and dermatological procedures, including hair transplantation, are provided in the present application. These procedures may be performed on the scalp, face and other skin and body surfaces.
According to one aspect of the present application, a method of planning for transplantation of follicular units in a body surface is provided. The method comprising: generating and displaying on a three-dimensional model of a body surface, at least two hair elements, each of the respective hair elements comprising one or more control points, the control points having corresponding parameters associated therewith; associating, with a use of a processor, a first and a second of the at least two hair elements with each other to form a hair group; generating or modifying at least one site within or on the first hair element, based at least in part on the parameters of at least one of the one or more control points of the second hair element. In some embodiments proposed follicular units sites are automatically modified, or generated and displayed based on the parameters of the control points of the hair group.
By the way of non-limiting examples, the hair element may be a hair patch or a boundary curve (e.g., hairline), the control points maybe located on a periphery or inside of the hair patch. The proposed sites may be hair harvesting sites, incision making sites, hair implantation sites, as well as various sites for placing and removing tattoos, or performing various cosmetic and dermatological procedures.
In some embodiments the parameter of the control point may be one or more of a location, an orientation or a density. In the case of density being the parameter, for example, in reference to hair transplantation and related procedures, the generation of the proposed sites may be based on an inverse transform algorithm between the proposed site and one or more control points. The density parameter may be weighed based on the distance of the associated control points from the proposed site.
In some embodiments, the method may further comprise projecting the 3-dimensional coordinates of a hair element into a 2-dimensional coordinate system and subsequently interpolating a procedure site (e.g., implantation site) in the 2-dimensional coordinate system. The interpolated implantation sites are then projected back into the 3-dimensional coordinate system, in which they are depicted on the 3-dimensional model of the patient. In some embodiments, the interpolation method may take into consideration density, and utilize a density parameter of a plurality of control points, wherein the plurality of control may comprise the one or more control points.
According to another aspect of the application, a method of planning for transplantation of follicular units in a body surface is provided, comprising: generating and displaying on a three-dimensional model of a body surface, a hair patch comprising a set of control points on the periphery thereof, the control points having corresponding parameters associated therewith; creating additional control points within the hair patch and/or on the periphery thereof; automatically generating and displaying a proposed follicular unit implantation site within or on the periphery of the hair patch based on the parameters of the control points and the additional control points.
According to another aspect of the current application, a system and a method of generating proposed procedure sites based on density interpolation is provided. The method comprising: projecting a location of a plurality of contour points of a body surface element in a three-dimensional coordinate system onto a two-dimensional plane; generating candidate procedure sites in the two-dimensional plane; determining an interpolated spacing value for each candidate procedure site, based on a density parameter of a plurality of control points associated with the body surface element; generating proposed procedure sites; projecting the two-dimensional location of the proposed procedure sites back to the three-dimensional coordinate system. In some embodiments, the method further comprises identifying candidate procedure sites that are not available as procedure sites due to the required spacing between the proposed procedure sites. In some embodiments, a formulation may be determined which associates density and spacing of the procedure sites. Additionally, the step of determining the interpolated spacing value may comprise utilizing the determined formulation.
According to another aspect of the application, a system for planning a transplantation procedure is provided, the system comprising: a user interface including a user input device, at least one non-transitory storage medium storing instructions, and a processor comprising one or more modules for executing operations on image data, the one or more modules comprising instructions for: generating and displaying on a three-dimensional model of a body surface, at least two hair elements, each of the respective hair elements comprising one or more control points, the one or more control points having corresponding parameters associated therewith; associating a first and a second of the at least two hair elements with each other to form a hair group; generating or modifying at least one site within or on the first hair element, based at least in part on the parameters of at least one of the one or more control points of the second hair element. In one embodiment the system further comprises an image repository. The system and method of the present application is especially useful when implemented on, or integrated with, an automated or computer-controlled system, for example, an image-guided robotic system comprising a robotic arm. The system may further include an imaging device to provide image data containing one or more images of the body surface and a tool for performing a procedure. The system may comprise an interface adapted to receive an image data containing images of a body surface, a processor programmed to perform various procedures and steps described in reference to various embodiments and methods of the present application, and a monitor configured to display a treatment plan as described herein.
In another aspect of the present application, systems and methods of planning for the transplantation of follicular units in a body surface is provided. Such systems and methods comprise generating and displaying a three-dimensional model of a body surface, selecting from an image repository a template, and generating and displaying one or more proposed hair elements based on the template. The step of generating may comprise generating a portion of the template. The portion may comprise a boundary curve or a hair patch.
According to another aspect of the present application, a system and method of planning for modification of a body surface (including facial features) is provided. Such systems and methods provide for associating various body surface elements into the groups and modifying certain features in a first body surface element based at least in part on one or more parameters of the control points from the second body surface element. In some embodiments modifications may be made automatically or with the user input. By way of non-limiting examples, the body surface elements may be a mole, freckle, wrinkle, scars, facial features such as eyes, nose, eyes, eye-lids, lips, ears, chin, birthmarks, or facial defects.
Other and further objects and advantages of the invention will become apparent from the following detailed description when read in view of the accompanying figures.
Features and advantages of the embodiments described herein will become appreciated as the same become better understood with reference to the specification, claims, and appended drawings wherein:
In the following Detailed Description reference is made to the accompanying drawings that show by way of illustration specific embodiments in which the application may be practiced. In this regard, directional terms, such as “inner”, “front”, “away”, “top”, “right”, “left”, etc., are used with reference to the orientation of the Figure(s) being described. Because components or embodiments of the present application can be positioned in a number of different orientations, and the methods can be carried out in a number of different ways, the directional terminology is used for purposes of illustration and is in no way limiting. Also, the terms “coupled,” or “attached,” or “connected,” or “mounted” as used herein, means directly or indirectly coupled, attached, connected, integrated, or mounted, for example, through one or more intervening components. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present application. The following Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of the present application is defined by the appended claims.
When planning a hair transplantation procedure for a particular patient the physician has to take many parameters that vary from one patient to another into consideration. For example, hair density and the variation of density throughout different regions of the patient's scalp, the caliber of hair which may also vary throughout the patient's scalp, the manner in which the patient styles and/or combs his hair. Having taken these parameters into consideration, it is however not possible to know whether the hair that is implanted will match the already existing hair, give a natural-looking appearance, or look as if it has been transplanted. It is also not possible to know how the area from which hair is being harvested will look like after hair has been removed or harvested.
Natural looking randomness is important not only for the critical hairline element but in the balance of the recipient sites, and the donor site too. To this end, each hair element, comprising, for example, hairlines, hair patches and the donor area should not be designed or planned independent from each other. In planning a hair transplant, the physician has to ensure that his plan offers the best outcome based upon the existing hair that exists on the patient's head, for example, the density or the distribution of existing hair, its spacing, angles, how the patient generally parts and/or combs his hair, or whatever else contributes to making the hair transplant look natural. One aspect of the current application enables such a natural looking randomness to be maintained, for example, throughout the patient's scalp. In addition, another feature of the current application enables a physician to obtain an image of what his/her patient may look like at various stages during, as well as, after a hair transplantation procedure is completed, based upon an image of their head, and their actual hair. Utilization of various aspects of the current application provides not only an image of what the newly implanted hair may look like, but of what the areas from which the donated hair will look like once the hair has been harvested from the donor element(s). According to another aspect of the current application a means is also provided by which a physician is able to propose potential visual appearances and aesthetic outcomes to a patient, explaining the various functional and/or aesthetic advantages and disadvantages of the potential treatment plans, along with a discussion of the time and/or cost associated with each proposed treatment plan. In this manner the patient is able to see what he/she may look like in each of the scenarios discussed, which reduces the chances of a patient misunderstanding what the physician may be trying to convey to a patient. In yet a further aspect of the current application a means is also provided by which the physician is able to plan one or more sessions of transplant surgery, optimizing the process to potentially reduce the number of sessions required to complete a hair transplant. This application provides the physician with a tool to enable him or her to ensure that there is, for example, sufficient hair in the donor areas, and hair of the correct type and caliber to provide the look that the patient is hoping for. If there is not, the current application provide the physician with a tool to enable him/her to illustrate to the patient what possible hair transplantation options are available to him or her based on the number, type, and caliber of hair they have available in the donor areas. The system and methods described in the present application also provide the physician with a means for planning the hair transplantation process, the number of sessions etc.
It should be noted that although the application is particularly useful in hair harvesting and implantation, it is not limited to hair transplantation. The application may also be beneficial to other procedures that require a model of the patient's body surface and parts, including, facial and head features, for example, various cosmetic and dermatological procedures involving treatment planning (e.g., plastic surgery, wrinkle removal or reduction, injections of cosmetic substances, skin grafting procedures, correction or removal of birth mark defects, facial reconstruction, rhinoplasty, contouring of the eyes or lips, remodeling of ears, nose, eye-lids or chins, facial rejuvenation, laser skin resurfacing, skin tightening, etc.) may benefit from the system and method of the applications described herein. One example of applicability of the application is in diagnostic skin imaging for cosmetic or other medical purposes, for example skin grafting or tattoo removal. For convenience of description, the following description will be discussed by example in reference to hair transplantation procedures. It should be noted, however, that such description is for the purposes of illustration and example only and is not intended to be exhaustive or limiting.
In accordance with one embodiment of the application, a treatment plan for a hair transplantation procedure may be created at least partially by a computer. The treatment plan automatically modifies, or generates one or more proposed follicular unit sites within a first hair element based on one or more parameters of hair follicles or follicular units in one or more second hair elements. In other embodiment of the application, the treatment plan automatically modifies, or generates proposed follicular unit sites within a first hair element based on one or more parameters of follicular units in the hair group, the hair group comprising a first and one or more second hair elements. In either case, the parameter information between two hair regions is shared, thus providing a tool to facilitate a more natural looking hair transplantation result. The treatment plan may take into consideration one or more parameters, including, but not limited to: a location of one or more hair elements, the geometric profile of hair elements, the number and/or type of follicular units to be harvested/implanted, a degree of randomness associated with particular harvest/implant locations, spacing between adjacent implant locations, depth of follicle, depth of implant, orientation of follicles or follicular units, patient identification (for example hair color, ethnic origin, age etc.), marker location(s), and/or the density of harvest/implant sites. In some embodiments, information about a particular follicular unit or combination of follicular units can be obtained from a database of follicular units. This database may contain information specific to the patient in question or to patients in general. The database may contain information categorized by ethnic origin, gender, and/or age, for example. In yet a further embodiment, the treatment plan may be generated or modified based on control points on or within the first and/or second hair elements. The control points can be individual points, lines, shapes, or other markers that can be used to provide a basis for the generation or modification of follicular unit sites within the first hair element.
After the treatment plan has been created at least partially by the computer, the user can accept or alter the treatment plan. Once the treatment plan meets the expectation of the physician, user and/or the patient, the user may register the treatment plan with a patient. In some embodiments, this may be accomplished by using one or more cameras to identify one or more markers that could be positioned directly on the patient or on a device used on the patient. The marker may be a reflector that is secured to the patient, an ink mark drawn on the patient, or an anatomy of the patient. Alternatively the marker may be, for example, a marking on a skin tensioning device that may be utilized by the physician in the hair transplantation procedure. The identified marker(s) may be used to determine a position and/or orientation of an element on the patient.
Hair transplantation procedures that are carried out using automated (including robotic) systems or computer-controlled systems have been described, for example, in U.S. Pat. No. 7,962,192, commonly owned by the assignee of the present application, which is incorporated herein by reference. After the robotic system has been initiated and calibrated, image data of the body surface is acquired and processed by the system computer to identify objects, in particular follicular units in a donor element, for example, on a human scalp. From images of this element of interest, image segmentation and screening software residing in the computer identifies and selects particular follicular units of interest for harvesting from the scalp.
In accordance with various embodiments of the application, a system for planning a procedure for transplantation of follicular units in a body surface (e.g., a scalp) of a patient may comprise a user interface, processor (e.g., software-controlled), a monitor, and at least one input device. These components are common to virtually all modern computer systems, whether a stand alone (e.g., “personal”) computer system, or in a system employing a centralized server with multiple remote terminal(s). It will be appreciated that embodiments of the planning system are preferably (if not exclusively from a practical point of view) software implemented, and may be run on any computer system having the basic components (processor, monitor, input device), so long as such computer system is equipped with sufficient available memory and an appropriate graphic generation and display capability. The computing system may include one or more processing units, one or more non-transitory storage media (which may take the form of, but is not limited to, a magnetic storage medium; optical storage medium; magneto-optical storage medium; read only memory; random access memory; erasable programmable memory; flash memory; and so on), and/or one or more input and/or output components for transmitting output to and/or receiving input from one or more other components (such as one or more displays, keyboards, mice, track pads, track balls, styluses, printers, speakers, cameras, video cameras, and so on). The processing unit may comprise one or more modules to execute instructions stored in the storage medium in order to perform one or more computing device functions, such as one or more treatment planning methods. The system or the processing unit may additionally include an image repository, the image repository comprising templates, images of one or more patients and/or images of portions of templates or patients. The system can be configured to implement all the methodologies, processes and techniques described herein.
Although it may be suggested that the computing system include particular components arranged in a particular configuration, it is understood that this is for the purposes of example. In various implementations, the computing system may include any number of computing system components (such as one or more busses, displays, networking components, dedicated image processors, co-processors, memories, hard drives, ports, graphics adapters, and so on) arranged in different configurations without departing from the scope of the present disclosure. For example, in one or more implementations the computing system may include multiple cameras and/or video cameras arranged to capture images and/or video of the same scene. By way of another example, in various implementations the computing system may include one or more interfaces for controlling machinery such as automated and/or computer-assisted surgical machinery.
It will also be appreciated that embodiments of the application may be implemented over the internet, e.g., with a user of such system employing his or her home computer as at least a part of the user interface (monitor and input device) that interacts with a remote server or computer. In such an internet-based planning system, the software that implements and controls the user interface may reside in whole or part on the user's computer or on the remote server/computer, preferably transparent to the user. In one such embodiment, the remote server downloads one or more software modules to the user's computer for temporary or permanent use.
Exemplary embodiments of a software implemented and controlled user interface for planning a follicular unit transplantation procedure will now be described in conjunction with the accompanying figures. It will be appreciated that various and multiple variations of the described embodiments may be implemented without departing from the general scope of the application, which is set forth in the appended claims.
In step 120, a processor or an image processor, processes and records information associated with the image data. Such information comprising, for example, the location and orientation of existing follicular units or hair follicles, information pertaining to scars, moles, freckles, wrinkles, facial features, tattoos or any other such body surface features.
In step 130, based on the information determined at step 120, the processor proposes hair elements, or body surface elements, including facial elements (and consisting of the proposed modification of existing body surface features) and one or more presentation components in the treatment planning system, such as a monitor, are used to visually present information indicative of the proposed hair elements/body surface elements. In reference to hair transplantation, the proposed hair elements may comprise, for example, a hairline or a hair patch. The proposed hair elements may comprise elements from which follicular units are to be harvested from or implanted into. In one embodiment, the proposed hair elements comprise a proposed boundary curve and/or proposed harvesting/implantation sites. In some embodiments, the proposed hair elements are proposed based on the image data alone. In other embodiments the hair elements are proposed based additionally on templates residing in the image repositories of the computer system. These templates may be automatically selected by the computer system based on the image data processed, or alternatively they may be selected by the user, or by a combination of both. In some embodiments once a template has been selected, portions of the template may further be selected.
Optionally in step 140, executable instructions may be provided to enable the user to modify the proposed hair elements/body surface elements by modifying the proposed boundary curve and/or the proposed harvesting/implanting sites based upon their analysis of the displayed image. The proposed boundary curve and/or the proposed harvesting/implanting sites may comprise control points to assist in the modification process.
In step 150, two or more hair elements/body surface elements are associated to form a group, such as hair group. Sites are modified or generated in step 160 based on the defined hair groups. Take, for example, a hair group comprising two hair elements. In one example, one or more proposed follicular unit sites in a first hair element may be now modified or generated based on one or more parameters of follicular units in the second hair element that is in the same hair group with the first element. In another example, one or more proposed follicular unit sites in a first hair element are modified or generated based on one or more parameters of follicular units in the hair group as a whole.
In the event that the user wishes to associate a different combination of hair elements to modify or generate more proposed sites, steps 150 and 160 may be repeated until all desired combinations have been associated and the desired proposed sites modified or generated.
In step 170, the treatment plan is accepted and, optionally, in step 180 a grid plan (described in more detail below) may be generated.
Although the method 100 is illustrated and described as including specific operations performed in a specific order, it is understood that this is for purposes of example. In various implementations, some operations may be performed in another order without departing from the scope of the present disclosure. In other implementations, only a subset of the various operations may be required, again without departing from the scope of the present application.
At any stage of the procedure, the proposed plan may be saved for further modification, editing or updating at a later date. This may be beneficial if after execution of a particular session of the treatment plan, the patient's hair does not grow as predicted, requiring modification of the treatment plan. Or, after execution of a particular session of the treatment plan, the patient has a change of mind, or cannot afford to have the entire procedure performed as planned, requiring modification to address his financial concerns, while at the same time providing him with a hair transplant that has a natural looking appearance.
In an alternate embodiment in which acquired images of the patient's head/scalp (or other applicable body surface) are not provided, the modeling software is configured to generate (by selecting menu option “generate”) a three-dimensional model based on inputs relating to selected characteristics, such as ethnic origin or race, and/or other characteristics selected through one or more menus, some objective (e.g., gender, age) and others purely subjective (e.g., attractive, symmetrical features, long forehead).
In some embodiments, further input information is provided to the planning system in addition to generating the body surface model. For example, particular features specific to hair follicles (e.g., color or coarseness) may be derived from the images and/or input through the user interface. Further information may be either user input or determined from the acquired images using image processing, such as geometric shape of the body surface (e.g., patient's head), existing hairlines, the existence of scars or moles, the location of where follicles have previously been harvested from or implanted into, and a number of each type (i.e., single or multiple follicle) and color (e.g., dark, light, gray, etc.) of follicular unit that are available for harvesting. It will be appreciated that the three dimensional model can alternatively be generated by a number of other methods, for example, using a 3D laser scanner and/or by stitching multiple digital images together. The system of embodiments of the present application will use the three dimensional information in the same way, without regard to how it is generated.
Whether from the acquired images, or through other descriptive feature inputs, the modeling software generates and displays on the user interface monitor of the planning system a three-dimensional model of the patient's head/scalp. With reference to hair transplantation, this model will represent the patient's head/scalp, and therefore will typically exhibit characteristic of male/female baldness patterns, comprising of bald regions 210, follicular unit populated regions 215 and, and/or regions which are more or less densely populated than others. The populated regions 215 will comprise follicular units which are grouped to form regions of hair which border less populated regions, or bald regions. These regions of hair may have geometry associated with them. The follicular units have various parameters such as type, caliber, orientation with respect to the scalp, spacing between adjacent follicular units, height of follicles above the scalp, for example.
Based on these characteristics, information, and/or one or more physical features determined from processing the acquired images, the modeling software automatically generates and the planning system displays on the user interface monitor of the planning system, a visual representation of one or more proposed elements 220 for follicular units sites as shown in
According to some embodiments, a treatment plan may be generated based at least in part on imaging data such as images from one or more image repositories. For example, the planning system may map anatomical elements determined from the acquired images to model heads or templates that are within the image repositories to automatically propose a treatment plan. In some embodiments, the user may select an image or template from patient information repositories containing images from one or more patients. For example, the template may identify the final outcome or the subsequent stage that the patient wishes to reach in terms of his/her hair transplantation treatment process. These templates may, for example suggest different hair style outcomes that may be achievable. These templates may comprise templates for men, women, boys, or girls; templates corresponding to various ethnic origins; templates corresponding to various head shapes; and/or templates corresponding to various degrees of hair loss, such as a template based on the Norwood scale of hair loss. For example, a template exhibiting a male Caucasian patient with hair receding mildly in a wedge-shaped pattern may be used for a patient with similar wedge-shaped pattern of baldness. Having selected the most appropriate template or combination of templates, the modeling software generates and the planning system displays on the user interface monitor of the planning system, a three-dimensional model of a patient's head/scalp, adding in or superimposing one or more elements as dictated by the selected template using, for example, a best-fit algorithm. That is, a visual representation of the patients existing hair regions, including hair elements into which it is desired that follicular units be implanted. These representations are generated such that they are scaled to fit on the three dimensional model of the patient's head/scalp. Optionally, the representations may be modified in shape and size to fit the three dimensional model of the patient's head/scalp.
In another implementation, the images from the repositories can be used to identify regions of interest through segmentation, contouring, or the like. For example, images from the image repositories may identify hair elements which are proposed to be added to the three-dimensional model of a patient's head/scalp. For example, the user may select a predetermined hair harvesting/implantation element from a range of possible options. This is illustrated in
According to yet another aspect of the application, having selected from a range of possible templates a template which best identifies his/her patient, the modeling software generates and planning system displays on the user interface monitor of the planning system, a selection of treatment plan options. These treatment plan options may, for example, include a high density follicular unit implantation in the frontal element only, a lower density follicular unit implantation in the frontal element only, follicular unit implantation in the frontal and mid-scalp elements, or perhaps follicular unit implantation throughout the scalp of the patient, for example. It will be apparent to the reader that the number of options available is numerous and as such all examples are not described herein. In another embodiment, the modeling software may additionally provide an indication of which of the many options are actually available to the patient, for example by making it impossible to select certain options. For example, if an option of a high density follicular unit implantation throughout the patient's scalp is not possible due to the lack of donor follicular units that can be harvested, for example, the modeling software will not enable that specific treatment option, and will not make it an option that can be selected by the user. To implement this option, it will be apparent that the computer system will have had to attain information pertaining to the number of, and optionally the type, of follicular units available for implantation. This information may be input by the user into the treatment planning system, or be provided by the treatment planning system itself, in analyzing the image data, and determining how many follicular units are available for harvesting without detriment to the aesthetic appearance of the hair in a donor area on the patient's head. In an alternative configuration, the treatment planning system may be configured to plan the harvesting of follicular units from the existing follicular units, and based on the number and/or type of follicular units proposed to be harvested, can utilize this number and/or type in generating a proposed implantation plan. It will be apparent to the reader that the options that are enabled to be selected by the user will vary based on context.
However generated, the modeling software generates and the planning system may display on the user interface at least one proposed hair element, for example a particular hair implantation site. Having selected the hair elements within which, for example, follicular unit implantation is desired, the treatment plan may be further customized.
According to another aspect, the present application also provides a treatment plan which automatically modifies, or generates proposed follicular unit sites within a first hair element based on one or more parameters of follicular units in one or more second hair elements. This aspect of the application will now be described with reference to
This scenario is depicted in greater detail in
It will be appreciated that there are numerous methods of interpolation that may be utilized for this application. One example of an interpolation technique is one in which the proposed hair site parameter is based on an inverse distance transform algorithm. This method of interpolation calculates the distance between the point to be interpolated (e.g., 510) and each of the respective control points, Q0, Q1, Q2, Q3, P0, P1, P2, P3 and P4, and applies a weighting to each of them based on the distance from the point to be interpolated (510). In this manner, the control P1 will have a greater influence on the interpolated value than the control points P0 and P2. Similarly control points Q2 and Q3 will have the least influence on the value of the interpolated point 510. For this algorithm, each of the control points may be utilized and provide a contribution to the determined value of the point 510 to be interpolated. In an alternative, only those within a certain distance from the point to be interpolated may be considered or, for example, only the two closest points. The interpolated value applied to the point to be interpolated 510 may be based on a linear interpolation of the control points, a polynomial interpolation, spline interpolation, non-linear interpolation (such as a Gaussian process), or other forms of interpolation that will be known to those skilled in the art. The type of interpolation utilized will generate varying results, and some will utilize more time in the generation process.
Another embodiment will now be described with reference to
In the examples indicated above, it can be seen that the referenced control points were all disposed on the periphery of the hair elements, that is, they were contour control points. To further customize the treatment planning process however, the system may also add specific control points 710 other than contour control points 720 to produce specific effects, such as an implant angle or density, or to achieve hair swirl and parting. Individual control points 710 may be placed inside the boundary curve 730 by the user, as shown in
Implementation of the selection of hair elements may be provided by a listing of the elements and an associated check box for each element on the monitor. By checking each of the desired hair elements that the user desires to be associated with one another, these hair elements can be grouped together, in addition to any control points associated therewith. Thus proposed sites may be influenced by all associated elements and their associated control points.
When multiple hair elements are placed on the same body surface, it is possible that they may at least partially overlap each other. This is illustrated in
Let us assume that the user were to indicate that the proposed implantation sites in the hair element 820 were to be based on the parameters of control points on the contour of, and within, the hair elements 820 and 830. Based on the explanation given above, when a proposed implantation site 960 is generated using interpolation techniques, the interpolation technique will take into consideration the parameters of control points on the contours of hair elements 820 and 830, and of control points within those elements, such as the control point 980, as illustrated. However, control point 980 falls within both elements 820 and 830, and as such will be considered twice when the interpolated value of the proposed implantation site 960 is determined. It will be considered once when the values of control points on the contour and in element 820 are considered, and again when the values of control points on the contour and in element 830 are considered. If the parameter value generated were, for example, density, it will be apparent that the interpolated value of the proposed implantation site will therefore be incorrect (potentially with the density being twice as high), because the control point 980 will have been considered twice. For the case of the user requesting that the proposed implantation sites in the hair element 820 were to be based on the parameters of control points on the contour of, and within, the hair elements 820, 830 and 810, it will be apparent that some control points may be considered up to three times, if such control points are located in all three elements. Processing the interpolated value in this manner may lead to a proposed site plan in which the elements of overlap influence the interpolated value of site, giving an incorrect, distorted or undesirable value. This is particularly evident if density is the parameter of consideration. The treatment plan in this situation may be proposing a higher density value for proposed implantation site 960 than would be expected, potentially leading to a not so natural-looking hair transplant.
To address this issue, the present application proposes a solution. For example, each of the hair elements 810, 820 and 830 may be divided into multiple segments based on where they intersect with one another, and use a Boolean operator on those segments to reconstruct the real-estate or floor plan of the combination of the elements 810, 820 and 830 without the overlap. This is illustrated in
In some situations, a region of overlap may be one hundred percent, as illustrated in
It should be noted that although the points which define the boundaries of the hair patch or the contour points, are all 3-dimensional (3-D) points in a 3-D coordinate system, in order to compute some of the formulae indicated above, and to calculate, for example, the polygon/element intersection, in one embodiment, the system may first project those 3-D contour points on a primary 2-dimensional (2-D) plane reconstructed using principle component analysis, a technique well-known to those in the art. Those projected 2-D polygons having coordinates in a 2-D coordinate system, may then be used for intersection computation. Once the computation has been completed, if required, the 2-D coordinates may be projected back into the 3-D coordinate system.
It will be appreciated that although the planning of implantation sites has been described above, the procedures can equally be applied to harvesting sites. The physician may identify harvesting elements from which hair is to be donated, along with associated control points, and request that, for example, the number of hairs in that identified element by reduced, reducing the number of hairs in such a way that the density is interpolated to be, for example, 80% of the pre-donated density value. The other implementations described throughout this application can equally be applied to harvesting.
When front hairline 1110 and a hair element 1120 are placed close to each other (see
One of the factors that contribute to the successful outcome of a hair transplantation procedure is the natural-looking appearance of the front hairline. According to another aspect of the current application, method for generating a natural-looking hairline based on control points, and the techniques of modulation and interpolation are provided. These methods can be executed substantially automatically, or with assistance from the user.
Initially, in step 1210, control points are identified on the contour of a base hairline 1330 (as seen in
Once the base hairline 1330 has been generated to the user's initial satisfaction, in step 1220, the respective location and direction of the follicular units to be implanted (1320a, 1320b, 1320c, 1320d etc.) along this base hairline 1330 are automatically determined by the planning module, along the curve, in order to complete the design of the base hairline 1330. The proposed implantation sites may be generated based on density, spacing and/or randomness parameters provided by the user through the user interface. Once again, if the proposed implantation sites are not to the user's satisfaction, the user may manipulate the control points to facilitate the desired changes in the interpolated implantation sites of the base hairline 1330. In some implementations, the interpolated sites of the base hairline may be created without first generating the hair curve on which the interpolated sites will reside.
When designing the front hairline or hairline, a physician typically uses more than one row of hair to define it. Therefore, designing a final hairline may comprise designing two or more rows of hair. In step 1230 of process 1200 for the generation of a proposed final hairline, the user may provide through the user interface the number of rows to be interpolated, and the modulation (that is, how the number of hairs in each subsequent row if reduced) from the base hairline 1330. As illustrated in
Before the final hairline design is proposed, the planning module may additionally execute further steps to aid in the creation of a natural-looking hairline. In step 1250, the spacing between any two neighboring implantation sites can be calculated, for example, a spacing between any two neighbor hairs in each of the rows 1330, 1340 and 1350, as well as the spacing between hairs located in neighbor rows (e.g. rows 1330 and 1340, or 1340 and 1350). If the proposed implantation sites are closer than the identified spacing provided by the user earlier, the location of the proposed implantation sites may be automatically adjusted to correct this. A randomness factor 1260 may additionally be applied. Having executed these additional steps, the result is a final hairline having a pre-determined number of rows of hair, modulation, and randomness factor.
Another of the factors that contribute to the successful outcome of a hair transplantation procedure is the natural-looking appearance of the density of hair throughout the patient's head. According to another aspect of the current application, methods (and corresponding processor systems) for generating a natural-looking interpolated density of implantation sites based on control points is provided. These methods can be executed substantially automatically, or with assistance from the user.
One of the initial steps 1410 in the process 1400 is to determine spacing between proposed implantation sites for the hair element (e.g., hair patch) for which internal implantation sites are to be generated. There are numerous ways in which this can be determined, one of which, according to the present application, is converting density information (which may be provided via the user interface by the user, or calculated by a processing unit, for example) to spacing information. For example, assuming that the pattern of hair is simulated by triangles, one possible formula for the conversion of density to spacing information could be:
where the spacing of adjacent implantation sites from an implantation site is based on the Density of the implantation site which they are adjacent to. It will be appreciated that other formulations may be utilized and may be tailored to specific situations. For example, one may prefer to simulate a pattern of hair by squares rather than triangles perhaps, thus requiring the formula to be modified to reflect this information.
Having determined the spacing information, a hair element in the form of a hair patch is identified. This hair patch comprises an outer boundary curve, which may be defined by one or more contour points. The hair patch may comprise conventional shapes such as circular, oval, polygonal or other such shapes, or be customized. Whatever the shape, it is intended that this identified hair patch will be on the patient's head, and as such will not comprise a flat surface, or 2-dimensional (2D) surface, but will comprise a 3-dimensional (3D) element. In this particular implementation of this aspect of the current application, step 1420 comprises projecting the 3D contour points which define the outer boundary curve in a 3D coordinate system, onto a 2D image in a 2D coordinate system. Methods for executing this step are well-known to those skilled in the art and will not be described herein. For ease in explanation, the resulting boundary curve 1510 and contour points 1520a, 1520b, 1520c and 1520d in the 2D coordinate system, as depicted in
Step 1430 comprises the generation of candidate implantation sites in the 2D coordinate system. Once again there are numerous implementations to accomplish this task that will be apparent to those skilled in the art. One of these is to identify a primary axis 1530, and to move in a direction away from the primary axis 1530, across the proposed internal implantation area towards the portion of the boundary curve 1510 on the other side of the internal implantation area (in
Having generated candidate implantation sites in the 2D coordinate system, the planning module utilizes the formulation which determines the spacing between proposed implantation sites for the hair patch. However, to do this control points may be required. These control points may be discrete control points, or they may comprise, for example, the contour points as indicated above. In this particular example, let us assume that the contour points 1520a, 1520b, 1520c and 1520d serve additionally as control points, with control points 1520a and 1520b having associated density values of, for example 40 hairs per cm2 and control points 1520c and 1520d having associated density values of, for example 20 hairs per cm2. Along the primary axis 1530, the interpolated density values for each of the proposed implantation sites will therefore range from 40 hairs per cm2 at 1520a to 20 hairs per cm2 at 1520d, resulting in interpolated spacing value (step 1440), that is the spacing value between hairs being smaller at the end closest to 1520a (higher density), and larger at the end closest to 1520d (lower density). Each of the candidate implantation sites has an interpolated density value associated with it, and therefore, has a spacing value also associated with it, the spacing value representing the minimum spacing required to achieve the desired density interpolation value.
Referring to
The process is repeated, interpolating spacing values, determining unavailable site locations and generating proposed implantation site locations until the entire hair patch has been populated in this manner. The results of such a process are illustrated in
Once the process has been completed, in step 1470, the 2D contour points which defined the outer boundary curve in a 2D coordinate system, and all the 2D coordinates of the proposed implantation sites (for example proposed implantation sites 1560a, 1560b, 1560c and 1560d are projected back into the 3D coordinate system. Once again, methods for executing this step are well-known to those skilled in the art and will not be described herein. In this manner, generation of internal implantation sites based on density interpolation is provided.
According to yet another aspect of the present application, additional methods and systems for improving treatment planning are provided. Having created a plan of proposed implantation sites, the treatment planning system may also be able to optimize the treatment process, by enabling the user to plan the number of procedures that may be required to obtain the desired result. As described in U.S. Pat. No. 8,388,631, a skin tensioner that can conform to a body surface and create tension, can be used to facilitate various procedures on the body surface, for example, harvesting of follicular units (FUs) from various locations on a body surface. These tensioning devices may come in various sizes and shapes and may be configured to conform to a particular element of the body surface.
Having generated and displayed on the user interface at least one implantation element (or region), but typically several implantation elements, the modeling software may utilize various algorithms to generate and overlay a representation of one or more region locators or, in some embodiments, skin tensioners to cover the proposed implantation or treatment area. The region locators may be used to hold fiducials or markers for directing image-guided systems that may be used in implementing various methods according to the present disclosure. The optimization algorithm may also operate to minimize the total number of tensioners to be used in the treatment procedure, minimize the overlay area between tensioners, and/or potentially reduce the number of sessions a patient is required to undergo.
To facilitate this, the user may be required to provide some constraints for the region locator or tensioner placement, such as for example, the maximum number of region locators or tensioners they would like to use in the procedure, the type of tensioner, the size of the tensioner, etc. This could be done via the user interface. Optionally, the user can manually adjust a plan that may be generated by the computer, indicating the number and placement of tensioners proposed to be adopted by the user in executing the proposed treatment plan. For example, the user may adjust the proposed locations of the tensioners to perhaps avoid aggravating existing scars, perhaps.
It will be apparent to those skilled in the art, that there are many algorithms that may be utilized to provide such functionality. In one example of this aspect of the current application, the processing unit may determine the outer contour of the combined hair elements, and based on that information, and the size of the tensioner, place one or more representations of the tensioner such that the entire area within the outer contour is covered by tensioners. In one embodiment of this aspect of the current application, the processing unit may remove from consideration any virtual tensioner which encompasses an area with no or a predetermined limited number of proposed implantation sites.
In another example of this aspect of the current application, the area covered by the combined hair elements can be virtually divided into grid sections, and a linear algebra cost function may be utilized to optimize the coverage of the grid points. Utilizing a set of one or more parameters, including but not limited, for example, to the number of tensioners, the location of tensioner and the size of tensioners, a value can be assigned which represents an objective value of any particular combination of values of the parameters in question. The objective value may vary depending upon which of the one or more parameters is weighted most highly, or has more value in any particular situation. For example, even though the use of fewer large tensioners may be possible for a particular treatment plan, the values may be configured such that smaller tensioners have a greater value, and as such the objective value may be found to be higher in that situation. This would, for example, be the case where the treatment plan was to be carried out on a child, or a patient with a small head, and it would be easier to utilize the smaller sized tensioner or region locator rather than the larger ones.
When utilizing this aspect of the current application, the treatment plan therefore comprises the treatment plan in terms of the proposed site locations/orientations, and the location/orientation of tensioners. Once the full plan meets the expectation of the physician, user and/or the patient, the user may then register the treatment plan with an actual patient. In some embodiments, this may be accomplished by using one or more cameras to identify one or more markers on the patient or a device on the patient. The marker may be a reflector that is secured to the patient, an ink mark drawn on the patient, an anatomy of the patient, the tensioner itself (or any portion thereof). Alternatively the marker may be a marking on a body surface tensioning device utilized by the physician in the hair transplantation procedure. The identified marker(s) may be used to determine a position and/or orientation of the implantation region of on the patient. Fiducial detection algorithms may be used to locate and refine the position of these markers. It will be apparent that the closer the fiducials/reference markers are to the proposed implantation area, the more accurate the registration of the treatment plan will be to the patient.
It will be apparent that although the methodology described above as discrete steps, one or more steps may be combined or even deleted, without departing from the intended functionality of the embodiments of the application. It will also be apparent that the methods described above may be performed manually, or they may be partially or substantially automated, including performed using robotic systems. Although described in a manner indicating that hair is harvested from and implanted into the same patient, hair can similarly be harvested from one patient and implanted into another. Alternatively, hair can be received from another source and implanted.
The foregoing illustrated and described embodiments of the application are susceptible to various modifications and alternative forms, and it should be understood that the applications as generally disclosed herein, as well as the specific embodiments described herein, are not limited to the particular forms or methods disclosed, and that many other embodiments are possible within the spirit and the scope of the present applications. Moreover, although individual features of one embodiment may be discussed herein or shown in the drawings of the one embodiment and not in other embodiments, it should be apparent that individual features of one embodiment may be combined with one or more features of another embodiment or features from a plurality of embodiments. By way of non-limiting example, it will be appreciated by those skilled in the art that particular features or characteristics described in reference to one figure or embodiment may be combined as suitable with features or characteristics described in another figure or embodiment. Furthermore, the methodologies described can be applied to any treatment, and is not limited to hair transplantation.
It will be further appreciated by those skilled in the art that the application is not limited to the use of a particular system, and that automated (including robotic), semi-automated, and manual systems and apparatus may be used for positioning and actuating the respective removal tools and other devices and components disclosed herein.
While the application has been described in its preferred embodiments, it is to be understood that the words which have been used are words of description and not of limitation. Therefore, changes may be made within the appended claims without departing from the true scope of the application.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4807163 | Gibbons | Feb 1989 | A |
| 5060171 | Steir et al. | Oct 1991 | A |
| 5687259 | Linford | Nov 1997 | A |
| 6081611 | Linford et al. | Jun 2000 | A |
| 6445943 | Ferre et al. | Sep 2002 | B1 |
| 6572625 | Rassman | Jun 2003 | B1 |
| 6585746 | Gildenberg | Jul 2003 | B2 |
| 7083611 | Lemchen | Aug 2006 | B2 |
| 7206627 | Abovitz et al. | Apr 2007 | B2 |
| 7452357 | Vlegele et al. | Nov 2008 | B2 |
| 8104480 | Bodduluri | Jan 2012 | B2 |
| 20020103500 | Gildenberg | Aug 2002 | A1 |
| 20040029068 | Sachdeva et al. | Feb 2004 | A1 |
| 20040034282 | Quaid, III | Feb 2004 | A1 |
| 20040197728 | Abolfathi et al. | Oct 2004 | A1 |
| 20040204760 | Fitz et al. | Oct 2004 | A1 |
| 20070078466 | Bodduluri et al. | Apr 2007 | A1 |
| 20070106306 | Bodduluri et al. | May 2007 | A1 |
| 20070150247 | Bodduluri | Jun 2007 | A1 |
| 20080004633 | Arata et al. | Jan 2008 | A1 |
| 20080122839 | Berglund et al. | May 2008 | A1 |
| 20110107270 | Wang et al. | May 2011 | A1 |
| 20110115790 | Yoo et al. | May 2011 | A1 |
| 20110251483 | Razzaque et al. | Oct 2011 | A1 |
| 20120158019 | Tenney et al. | Jun 2012 | A1 |
| 20120215231 | Wesley | Aug 2012 | A1 |
| 20120303384 | Stepaniak et al. | Nov 2012 | A1 |
| 20130058543 | Thomas et al. | Mar 2013 | A1 |
| 20130236074 | Hillebrand et al. | Sep 2013 | A1 |
| Number | Date | Country |
|---|---|---|
| 102693352 | Sep 2012 | CN |
| 0714642 | Jun 1996 | EP |
| H08-227454 | Sep 1996 | JP |
| 2002-083318 | Mar 2002 | JP |
| WO 9821695 | May 1998 | WO |
| WO 0064379 | Nov 2000 | WO |
| WO 2006127142 | Nov 2006 | WO |
| WO 2007041014 | Apr 2007 | WO |
| WO 2014150459 | Sep 2014 | WO |
| Entry |
|---|
| “ComputerWorld: Dispatches & Images from the Fringes of the Electronic Frontier”, ComputerWorld. (Nexis abstract), Mar. 30, 1998. |
| “Total Body Photography from Canfield”, Imaging Systems and Software for Tracking Pigmented Lesions. Downloaded Nov. 22, 2013, (4 pages). |
| “Vectra m3 Sculpt the Dream”, Brochure from Canfield Imaging Systems, Downloaded Nov. 22, 2013 (4 pages). |
| Alhaddab, Mohammed et al., “Effect of Graft Size, Angle, and Intergraft Distance on Dense Packing in Hair Transplant”, Dermatol Surg. 31:6, Jun. 2005, 650-654. |
| Bernstein, Robert M. et al., “Follicular Transplantation”, International Journal of Aesthetic and Restorative Surgery. vol. 3, No. 2., 1995, 119-132. |
| Bernstein, Robert M. et al., “The Logic of Follicular Unit Transplantation”, Dermatologic Clinics vol. 17, No. 2, Apr. 1999, 277-296. |
| Bodduluri, R.M.C. et al., “Geometric Design and Fabrication of Developable Bezier and B-spline Surfaces”, Transactions of the ASME vol. 116,, Dec. 1994, pp. 1042-1048. |
| Hoffman, Rolf et al., “Recent Findings with Computerized Methods for Scalp Hair Growth Measurements”, The Journal of investigative dermatology symposium proceedings 2005, vol. 10, No. 3, 2005, pp. 285-288. |
| Jimenez, Francisco et al., “Distribution of Human Hair in Follicular Units—A Mathematical Model for Estimating the Donor Size in Follicular Unit Transplantation”, Dermatol Surg. 25:4, Apr. 1999, 294-298. |
| Rassman, William R. et al., “Micrografting in Extensive Quantities”, http://www.newhair.com/resources/mp-1995-micrografting.asp, (7 pages). |
| Riordan, Teresa , “Implanting Hair is Tedious, Exacting Work—the Perfect Work for a Robot”, The New York Times Sep. 15, 2003, (1 page). |
| Shapiro, Ronald , “Principles and techniques used to create a natural hairline in surgical hair restoration”, Facial Plast Surg Clin N Am 12 (2004), 2004, pp. 201-217. |
| PCT International Search Report and Written Opinion from the International Search Authority, for commonly assigned PCT/US2014/023318, Applicant: Restoration Robotics Forms PCT/ISA/210, 220 and 237 mailed Aug. 14, 2014 (18 pages). |
| English Translation of Office Action mailed Jul. 8, 2015 in connection with commonly assigned Taiwan Patent Application No. 103109329 Restoration Robotics, Inc. (4 pages). |
| Number | Date | Country | |
|---|---|---|---|
| 20140261467 A1 | Sep 2014 | US |
