This application relates generally to medical robots, and particularly to robotic systems that perform non-invasive therapies on the human body.
Current medical treatments rely predominantly on prescription drugs and surgery. These have very high rates of effectiveness but are often coupled with serious risks, such as drug side-effects and surgical complications. In the face of these risks there is a compelling (though less well-known) alternative to treat various kinds of ailments: acupuncture. Acupuncture is a form of therapy in which needles are used to stimulate specific points on the body, to relieve targeted conditions. Despite its use of needles, acupuncture is fundamentally non-invasive. This is because treatment is predominantly restricted to the body surface, and the needles themselves are usually quite thin. Currently, acupuncture is mostly performed by trained chiropractors. It may be desirable to have robotic systems that can deliver accurate and non-invasive treatment, such as acupuncture.
One or more aspects of the disclosed technology relate to robotic systems that perform non-invasive therapies on human body. The system integrates methods from multiple fields to locate anatomy-based points on the human body, and then applies non-invasive therapies at these points using robotic mechanisms. Methods utilized by the system relate to fields, including biomechanics, computer vision, artificial intelligence, and robotics.
In some variations, a robotic system comprises a robotic arm, an end effector coupled to a distal end of a robotic arm, one or more cameras, and a processors configured to construct a three-dimensional (3D) model of a user based on received data from the one or more cameras, identify automatically one or more target therapy points on the user based on the constructed 3D model, and actuate the end effector to apply a therapy to the target therapy points.
In some variations, a computer-implemented method comprises receiving, by a processor, data about a user from one or more sensors; constructing a three-dimensional (3D) model of the user based on the received data from the one or more sensors; identifying, by the processor, a target point on the user based on the constructed 3D model; and actuating an end effector to apply therapy to the target point.
In some variations, an apparatus comprises one or more sensors and a control unit configured to construct a 3D model of a user based on received data from the one or more sensors; identify a therapy point on the user based on the constructed 3D model; and actuate an end effector to treat the target point.
Certain features of the subject technology are set forth in the appended claims. However, for purpose of explanation, several embodiments of the subject technology are set forth in the following figures.
The detailed description set forth below is intended as a description of various configurations of the subject technology and is not intended to represent the only configurations in which the subject technology may be practiced. The appended drawings are incorporated herein and constitute a part of the detailed description. The detailed description includes specific details for the purpose of providing a thorough understanding of the subject technology. However, the subject technology is not limited to the specific details set forth herein and may be practiced using one or more implementations. In one or more instances, structures and components are shown in block diagram form in order to avoid obscuring the concepts of the subject technology.
Sometimes acupuncture is thought of as whimsical Eastern medicine, lacking any scientific footing. However, this is an unfounded misconception. Both the National Institutes of Health (NIH) and the World Health Organization (WHO) have reported scientific consensus that there is positive, incontrovertible evidence for the effectiveness of acupuncture in a number of conditions. Acupuncture therapy is commonly user for 1) pain treatments (e.g., back pain, arthritis pain, and migraines) as an alternative to opioids, corticosteroids or triptans, 2) surgery avoidance (e.g., kidney stones), and 3) mental health (e.g., depression or PTSD).
Acupuncture therapy can be applied with or without any needle penetration at all. Stimulation of therapy points, or acupoints, can be done in the form of acupressure (applying pressure with fingertips or other blunt objects), electric acupuncture, and even laser acupuncture (i.e., photo-acupuncture). In general, non-invasive treatments, such as acupuncture, can be referred to as non-invasive contact therapies, and have been prescribed to people of all ages. But notably, because of their low risk, simplicity of treatment, and ease of delivery, they are particularly attractive given current global demographic and socio-economic trends such as an aging world population.
An increasingly older world population demands ever growing and more affordable healthcare services. According to projections from the United Nations, in China alone the population aged 60 and above will increase by about 125 million (roughly equivalent to the entire population of Japan) over the next decade, while the total population will remain about the same. This trend is also present, although a bit less dramatically, in most industrialized countries. This means that the ratio of healthcare practitioners to healthcare consumers will rapidly decrease, and thus produce an explosive global demand for more accessible healthcare and wellness services.
In non-traditional markets for acupuncture, such as Western countries, its use has been rising steadily over the last few decades. In the United States, acupuncture treatment is covered by several large health insurance plans including Aetna, Blue Cross, and MHS (U.S. military healthcare services). Additionally, acupuncture coverage is mandated by state law in several states, such as California, Texas, Florida, Nevada, Montana, Maine and Virginia. Conspicuously, being of Asian descent is not a significant predictor of acupuncture use in the U.S. market.
Disclosed herein is a robotic system that can perform non-invasive therapies on the human body. The system integrates multidisciplinary methods to identify anatomy-based therapy points on human body and applies non-invasive therapies using robotic mechanisms. Methods utilized by the system relate to fields including biomechanics, computer vision, artificial intelligence and robotics.
The system utilizes a sensor array together with a human biomechanical model to build a personalized model of the underlying anatomy for the user based on observations of the user's skin surface in a few different poses. It constructs this user-specific biomechanical model by transferring elements of an anatomical template to the specific size and kinematics of the user. The system then maps therapy points to the anatomy model of the user and applies prescribed therapies using robotic apparatus. The system can follow user movement while treating the therapy points. The system also includes end effector to apply desired therapies, such as pressure, vibration, heat, electricity, laser, acoustic waves, needling, moxibustion, and vacuum cupping, at the therapy points.
As shown in
Axial motion of robot pedestal 108 along the top of frame 104 facilitates positioning of the system relative to therapy point TP. The design is to maximize unobstructed views of user 101 while minimizing occlusions caused by robot arm 110. Axial motion of the robot pedestal 108 (e.g., moving to either end of the frame 104) also facilitates physical access when the user enters or exits the system.
As shown in
In some aspects, the process of scaling block 212 illustrated in
Referring back to
In contrast to the scaling block 212, which relies on physics-based methods for tasks such as skin deformation, the modeling block 214 may not need to. Instead, modeling block 214 can use geometric skinning techniques, such as linear blend skinning and dual quaternion skinning. These simplifications ensure modeling block 214 performs much faster than scaling block 212, thus more amenable for real-time tracking of the user on the table during therapy. In addition, the modeling block 214 can also be augmented through a mapping between therapy point locations and musculoskeletal geometry of human body. An example of the geometry mapping of therapy point is shown in
Once the real-time biomechanical model of the user is established by the modeling block 214, a therapy session can take place. During the therapy session, a continuous loop of sequenced steps can be performed by imaging block 232, therapy-point (or key-point) localization block 234, 3D motion tracking block 250, therapy point tracking 252, and therapy application block 254, as shown in
The therapy session starts with the imaging block 232, which can produce a synthetic 2D image of the user 101. The synthetic image can then be fed to therapy point localization block 234 to identify the therapy points on the user in 2D image.
Image synthesis process further includes a virtual view synthesis with gap-filling 520. The method may involve reprojection of the user into a virtual view, image blending and gap filling, as shown in
Next, image blending module combines the reprojected views into a synthesized virtual view. The blending process can include brightness adjustments, to reduce inconsistent brightness and discontinuous colors. Finally, necessary gap-filling is performed because the blended virtual view may include parts of the scene not visible in any of the input images, caused by the limited number of discrete viewpoints. Several standard techniques exist for gap-filling, including inpainting methods, which use texture from adjacent regions to fill gaps in the blended synthetic image.
Synthetic image 530 thus generated can then be input to key-point detection block 540, which may use real-time 2D key-point detection methods, such as machine learning algorithms trained on large databases of manually annotated images with information of interest (e.g., locations of key-points). For example, AI models with an artificial neural network (ANN) trained on an extensive number of images to detect key-points in real-time. The 2D key-point detection method can perform real-time inference of therapy points on body key-points (e.g., shoulder, elbows, hips, and knees), hand key-points (e.g., wrist, finger joints, and fingertips), foot key-points, and facial key-points (e.g., eyes, nose, and ears), and integrate the inference into a consistent body arrangement. The method is robust with respect to occluded (blocked from view) body parts and can also determine whether the user is positioned face-up or face-down. In short, the output of block 540 is an annotate 2D image 550 with key-point locations marked.
Referring back to
In some aspects, motion tracking performed by block 250 concerns optimization variables that correspond to degrees of freedom (DOFs) of the skeletal armature in real-time biomechanical model, as illustrated in
Note that since the key-point detection relies on machine learning trained from manually annotated 2D images of anatomical landmarks, joint locations marked by AI inference may not be highly accurate. However, once these coarse locations 720 are known, the tracking algorithm can further optimize the armatures's DOFs for the key-point localization.
Referring back to
Referring back to
The system also constructs 1020 a 3D model of the subject based on received data from the one or more sensors. For instance, the scaling block 212 and modeling block 214 in
Next, the system identifies a target therapy point (or key-point) on the subject based on the constructed 3D model of the user. For example, imaging block 232 and key-point localization block 234 in
Users may obtain customized therapy sessions in various ways, such as: 1) provided by the system (via kiosk, terminal, or mobile device), in which user can select menu options or automated recommendation based on user needs, such as certain common conditions, and generic treatments (e.g., relaxation against stress); 2) prescribed by therapy expert, such as an acupuncturist either remotely via videotelephony or in-person. Prescription instructions may include: 1) a set of therapy points or acupoints, 2) a sequence and timing of acupoint treatment, and 3) treatment modality per acupoint, such as pressure and/or laser.
The system may include one or more robot arms, each of which may be mounted on any support structure such as a frame, table, chair, mobile cart, and so on. The fixture on which the user rests may be a therapy table, chair, frame, or any other similar fixture. Sensors conforming the sensor array may be mounted relative to the resting surface, mounted mobile relative to the resting surface, or any combination thereof. The sensor array may be replaced by a single sensor or a sensor module, and may use alternate sensing methods such as optical, lidar, magnetic, electro-magnetic, and so on.
The end-effector may be interchangeable with end-effectors of various designs. An end-effector may be designed to deliver one therapy modality or a combination of several modalities. A therapy modality may be used individually or combined with other therapy modalities at the same time (e.g., pressure and laser). The end-effector does not have to be used exclusively to deliver therapy. It may also be used to place and/or remove a number of pods that include appropriate apparatus to deliver the therapy. These pods can be wireless or wired. The pods may be temporarily attached to the user's skin via suction cups, weak adhesive, or any other method of temporary attachment.
The real-time biomechanical model may be of a similar kind as the user-scale model, which can be based on physics methods and geometric methods. 3D key-point localization may be done without previous occlusion removal, or by performing key-point detection on the 2D input images separately and then triangulating locations. The end effector may use acoustic waves as a therapy, such as in shockwave therapy (ESWT), or may use them in the form of ultrasound to assist in locating internal anatomy.
The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein but are to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more. Headings and subheadings, if any, are used for convenience only and do not limit the subject disclosure.
This application claims the benefit under 35 U.S.C. § 119(e) of US Provisional Application No. 63/175,681, filed Apr. 16, 2021, which is incorporated by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
63175681 | Apr 2021 | US |