Patent Application
20250239350
This application claims priority of Korean Patent Application No. 10-2024-0008902, filed on Jan. 19, 2024, and priority of Korean Patent Application No. 10-2024-0177019, filed on Dec. 3, 2024, in the KIPO (Korean Intellectual Property Office), the disclosure of which is incorporated herein entirely by reference.
The present invention relates to a personalized artificial intelligence-based insulin dose determination system.
Diabetes can be characterized by hyperglycemia and relative insulin deficiencies. There are two main types of diabetes, type 1 diabetes (insulin dependent diabetes mellitus) and type 2 diabetes (non-insulin dependent diabetes mellitus). In some cases, diabetes is also characterized by insulin resistance.
For the treatment of such diabetes, various non-invasive transdermal (e.g., transdermal) and/or insertable electrochemical sensors have been developed to continuously detect and/or quantify blood glucose values. Continuous glucose monitoring is becoming more popular as a way to easily monitor glucose levels.
Along with the continuous glucose monitoring, a bolus calculator device is used, and in this case, variables such as basal insulin, a Correction Factor (CF), a carbohydrate-to-insulin ratio (CIR), and the like, have to be measured and input.
Accordingly, an object to be achieved by the present disclosure is to provide a system capable of more effectively determining variables used in a precision insulin control system, and a bolus calculator using the same.
In order to achieve the above object, the present invention provides a personalized artificial intelligence-based insulin dose determination system by a computing terminal capable of communicating with an insulin pump and a continuous blood glucose system used by a user, the system comprising: an interface layer capable of communicating with the insulin pump and the continuous blood glucose system, and signal-processing information from the insulin pump and the continuous blood glucose system; a control layer receiving the signal-processed information from the interface layer, and generating output information related to an insulin injection amount; an outer safety layer determining whether the output information generated from the control layer satisfies a preset threshold condition, and transmitting the output information to the interface layer when the output information satisfies the preset threshold condition; and a personalized safety layer receiving prescription information including Total Daily Dose of Insulin (TDD) information of the user from the outside, determining a personalized safety control variable, and transmitting the determined control variable as an input variable of the control layer.
In one embodiment of the present invention, the control layer determines, as output information, the amount of insulin injected into the user using at least one of a continuous blood glucose level (g), an insulin-on-board (iob), a continuous blood glucose level (dg/dt or vg), and a continuous blood glucose acceleration (d2g/dt2 or ag) measured by the continuous blood glucose system as input information from the signal-processed information.
In one embodiment of the present invention, iobt at time t is determined by the following equation (1).
In an embodiment of the present disclosure, the control layer determines the amount of insulin injected into the user as output information using a deep-reinforced learning model.
In an embodiment of the present invention, the deep-reinforced learning model uses a soft actor critical (SAC) algorithm, the control layer determines a blood glucose control policy of the soft actor critical (SAC) algorithm at predetermined time intervals, and the control layer trains the soft actor critical (SAC) algorithm model which is trained for the purpose of maximizing an objective function according to the following equation.
The control layer uses the SAC (Soft Actor Critical) algorithm model for the purpose of maximizing the objective function according to the following equation (4).
In an embodiment of the present invention, the control layer determines the insulin injection amount as an output variable within any one or more of the following three limit value ranges:
In one embodiment of the present invention, the insulin injection amount is determined by the following equation.
In an embodiment of the present disclosure, the insulin injection amount SN during the night is a value obtained by multiplying the insulin injection amount SD during the day by a coefficient between 0 and 1.
In one embodiment of the present invention, the coefficient is determined according to TDD (Total Daily Dose of Insulin).
In an embodiment of the present disclosure, the maximum insulin-on-board value (iobmax) is determined as a constant according to the TDD (Total Daily Dose of Insulin), and the maximum insulin-on-board value (iobmax) is provided as a threshold of the outer safety layer.
It provides an accurate insulin management (AIM) system applicable in large-capacity endocrine clinicians and shows an excellent balance between high accuracy and characteristics that are short, easy to remember and easy to use to reduce administration errors.
The above and other features and advantages will become more apparent to those of ordinary skill in the art by describing in detail exemplary embodiments with reference to the attached drawings, in which:
In the following description, the same or similar elements are labeled with the same or similar reference numbers.
The present invention now will be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “includes”, “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. In addition, a term such as a “unit”, a “module”, a “block” or like, when used in the specification, represents a unit that processes at least one function or operation, and the unit or the like may be implemented by hardware or software or a combination of hardware and software.
Reference herein to a layer formed “on” a substrate or other layer refers to a layer formed directly on top of the substrate or other layer or to an intermediate layer or intermediate layers formed on the substrate or other layer. It will also be understood by those skilled in the art that structures or shapes that are “adjacent” to other structures or shapes may have portions that overlap or are disposed below the adjacent features.
In this specification, the relative terms, such as “below”, “above”, “upper”, “lower”, “horizontal”, and “vertical”, may be used to describe the relationship of one component, layer, or region to another component, layer, or region, as shown in the accompanying drawings. It is to be understood that these terms are intended to encompass not only the directions indicated in the figures, but also the other directions of the elements.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Preferred embodiments will now be described more fully hereinafter with reference to the accompanying drawings. However, they may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The system according to the present invention may be implemented in a terminal (e.g., a mobile phone, a tablet PC, etc.) that can communicate with an insulin pump and a continuous blood glucose system that are attached to a body used by a user and can be computed by itself, and the following layers may be configured to be divided in software in the terminal.
That is, the system according to the present invention is a device that is mounted on a terminal or the like and can be processed by a processor of the terminal, and the system 10 according to an embodiment of the present invention includes an interface layer 100 that can communicate with an insulin pump 20 and a continuous blood glucose meter 30 used by a user, and can receive information from the insulin pump 20 and the continuous blood glucose meter 30 and process a signal. In an embodiment of the present invention, the interface layer 100 may transmit information on the insulin injection amount determined by the system to the insulin pump 20 so that the insulin pump 20 may operate as much as the determined injection amount.
The system 10 includes: a control layer 200 configured to receive signal-processed information from the interface layer 100 and generate output information associated with the insulin injection amount; an outer layer 300 configured to determine whether the output information (i.e., the calculated insulin injection amount) generated from the control layer 200 satisfies a preset threshold condition, and transmit the output information to the interface layer when the output information satisfies the preset threshold condition; and a personalized safety layer 400 configured to receive prescription information including TDD (Total Daily Dose of Insulin) information of a user from the outside, determine a personalized safety control variable, and transmit the determined control variable as an input variable of the control layer.
In particular, the system according to an exemplary embodiment of the present invention includes the external safety unit 200 between the interface unit 100 for receiving a signal by communicating with the CGM 10 attached to the user and the insulin pump 20 or generating a signal for driving the insulin pump and the control unit 300 for determining an injection amount of the insulin pump, and checks once again the insulin injection amount determined by the reinforcement learning based on a threshold value, which is a safety threshold value, before the insulin injection amount determined by the reinforcement learning is changed to the final interface.
That is, the insulin injection amount determination system according to the present invention has four stacked structures based on reinforcement learning, thereby maximizing flexibility and adaptability.
In an exemplary embodiment of the present invention, the smart phone executes the algorithm according to the present invention, and accordingly, the interface layer of
In the system according to one embodiment of the present invention, several thresholds obtained on silica subjects are set, and to this end, the outer safety layer 200 is placed between the interface layer 100 and the control unit 300. This will be described in more detail below.
A control layer 300 based on deep reinforcement learning is provided on the outer safety layer 200.
In an embodiment of the present invention, the amount of insulin is calculated based on at least one of the observed GM (g) value, the insulin dose (insulin-on-board, iob) that means insulin still valid in the previous bolus, the CGM velocity (dg/dt or vg) and the CGM acceleration (d2g/dt2 or ag). That is, by combining variables obtainable from blood glucose data obtained from at least a plurality of continuous blood glucose systems, the rate of increase and decrease and acceleration of blood glucose can be predicted, and when at least one of the variables is used, all of them fall within the scope of the present invention.
At the top of the system structure according to the present invention, there is a personalized safety layer 400 for personal safety. In an embodiment of the present invention, the personalized safety layer 400 includes two sub-modules, namely, a OffSI module 410 and a OnSL module 420, which will be described in more detail below.
In the present invention, three new adjustable, time-dependent control variables are used to determine the safety of the automatic insulin injection prescription between users, which are a coefficient of day (SD), a coefficient of night (SN), and a maximum insulin dose (maximum IOB, IOBmax), which limits the output value of the insulin injection amount based on personal information associated with TDD.
The OffSI module, which is a lower module of the personalized safety layer 400 according to an embodiment of the present disclosure, determines a correlation between the total insulin injection amount (TDD, total daily dose) and the three variables.
The OnSL module, which is another lower module, performs a function of securing adaptability of the PersonAI algorithm according to the present invention by updating and adjusting the control variable. The OnSL module adaptively and finely adjusts the control variables to ensure the performance and stability of a control algorithm for long-term automatic insulin infusion (AID). For example, one of the functions of the OnSL module is to set a priority between variables in order to prevent hypoglycemia and increase a Time-In-Range (TIR) within a range of a blood glucose value every week.
Referring to
Referring to
Hereinafter, the present invention illustrated in
As described above, the system according to an embodiment of the present invention includes the separate outer safety layer 300 between the interface layer 100 and the control layer 200. The outer safety layer measures real-time metabolic state based on a continuous glucose monitor (CGM) and insulin infusion data, wherein g, vg, ag and iob values are used as input values. where g is the blood glucose level measured by the CGM, vg is the rate of change in blood glucose, ag is the rate of change in blood glucose, and iob is insulin-on-board, which is the amount of insulin already injected to lower current blood glucose. That is, g denotes a blood glucose change rate, ag denotes a variable which can be obtained from a blood glucose level, and all values are included in the scope of the present invention as long as at least one of g, vg, ag, and iob values is used as a variable.
As shown in
In order to obtain iob, which is another input value, the present invention calculated the total amount of insulin injected according to the following equation (1).
The present invention assumes that the cumulative iob remaining in the body of the user (e.g., 5 hours) operates using CDF. k is the number of time steps at which insulin dose data is collected, and n is the maximum number of time steps at which insulin dose data is collected during the time assuming that accumulated iob remains in the body of the user. For example, when it is assumed that an insulin operation is performed in the body for 5 hours and iob is collected at an interval of 5 minutes, n is 60.
In order to solve the problem of the prior art that the personalized iob is not suitable for fully automated insulin infusion in particular, the scale factor (SF) was used as described above.
Since the system according to the present invention is based on the Model-Free approach, an output safety layer is needed to improve reliability from problems in a serious external environment, such as CGM errors.
The present invention suppresses a Basal rate (CSII pump insulin dose (i)) by a hard thresholding method in which a coefficient whose absolute value is lower than a threshold value is set to 0, instead of suggesting a CSII (BR) according to the prior art.
In an embodiment of the present invention, three types of limit values are provided based on the time-dependent upper boundary values imax and iobmax of i and iob and the lower boundary value gmin of g.
First, two conditions for insulin limitation are as shown in Equation (2) below.
The allowable insulin injection according to the following one operation instruction can be defined as imax=BR [U/h], where the equation is as follows (3).
Third, the possibility of a problem such as hypoglycemia should also be predicted, wherein the CSII pump insulin dose (i) is i(t)=0.
As described above, the control layer of the system according to the present invention is based on a deep reinforcement learning (DRL) framework, and thus, in an embodiment of the present invention, the fully closed rope-automatic insulin injection problem is schematized with a Markov Decision Process consisting of six states where S is the user's hypoglycemic state (probabilistic observation value=0).
In the present invention, four values were used as input variables of the control layer, which are blood glucose g(t) (mg/dL), insulin dose (insulin-on-board, iob), CGM velocity (dg/dt or vg) and CGM acceleration (d2g/dt2 or ag).
The operation interval A is assumed to be continuous, and the amount of insulin injected by the CSII pump is defined as ī.
For learning, in the present invention, the operation interval A is set to 0<A<basalmax. Here, basalmax is the basal rate (BR) size allowed for the CSII pump to inject insulin (i.e., 30 U).
The state transfer constant P defined in the physiological model depends on each individual metabolic kinetics (i.e., UVA/Padova model and Hovorka model).
The compensation function R is related to the degree of blood glucose control, and this is a factor that reflects an optimization purpose using R: (st, at) and ν⊆[0.1] which is a discount factor that determines a trade-off between immediate compensation and delayed compensation.
In the present invention, two types of compensation functions were used, one being a long-term compensation function and the other being a short-term compensation function. Here, the long-term compensation function is to simulate basal insulin secretion of β cells, and the short-term compensation function is to simulate basal excess insulin secretion of β cells. Therefore, in the present invention, the discount factor is set to γ=0.99.
In one embodiment of the present invention, with respect to insulin prescription strategies, the Soft Actor Critic (SAC) algorithm was adopted to learn blood glucose control strategies (see T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, et al., “Soft actor-critic algorithms and applications,” arXiv preprint arXiv vol. 1812, no. 05905, 2018; T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” International conference on machine learning, vol. 35, no. 80, pp. 1861-1870. PMLR, 2018.)
In an embodiment of the present invention, a policy action, which is an output value of the algorithm, corresponds to a desired amount of insulin injection at each preset time step (5 minutes in an embodiment of the present invention). The policy network in the present invention generates μ, log(σ) as an output value, which parameterizes a normal distribution N(μ,σ), and the operation is distributed according to a sigmoid normal distribution or
According to the SAC algorithm according to an embodiment of the present invention, the maximum entropy learning is corrected in a continuous operation region in which the probabilistic policy π is expressed as a parameter Ø.
According to an embodiment of the present invention, the maximum entropy objective function of the deep learning policy πØ may be represented by the following equation (4), wherein the equation is learned to maximize the objective function J(π).
In an embodiment of the present invention, an optimal temperature parameter was calculated using a conventional automatic temperature adjustment method. Each variable and parameter of the objective function is considered with reference to T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, et al., “Soft actor-critic algorithms and applications,” arXiv preprint arXiv, vol. 1812, no. 05905, 2018.
In order to secure prescription safety between users corresponding to universal safety, not individual safety, the present invention sets a time-dependent insulin delivery limit value with respect to a calculation value of a DRL-based control layer. That is, in each step k, the limit value of the following equation (5) is applied to the calculated value of the control layer.
In an embodiment of the present invention, the day's adjustment coefficient SD>the night's adjustment coefficient SN, SD is determined by a calibration method described below, and SN is defined by the following equation (6).
Finally, in the present invention, iobmax is defined as the following equation (7), which is included in the outer safety layer equation (2).
The present invention provides a new PDSA algorithm for the personal safety layer, which will be described in more detail below.
The OffSI module according to the present invention provides an individual initial value for control variables of the artificial intelligence-based insulin injection system according to the present invention for all treatment patients.
As described above, the most realistic method for personalizing the control algorithm model without retraining is to subdivide the algorithm processing aggressiveness based on the ICR, BW, TDD, age, and gender of type 1 diabetic patients. This personal information is linked to the interindividual variability of insulin sensitivity (IS). Accordingly, the module according to the present invention is a OffSI module based on a regression model, which defines the association between TDD and optimized control variables.
In the present invention, an optimized control variable (i.e., SD) of each TID-VP (type 1 virtual diabetic) was found through a correction repetition procedure. On the other hand, other parameters, i.e., iobmax and SN, were determined from the hard threshold based on TDD, as shown in Equations (6) and (7), respectively.
In-silico testing experiments were performed according to protocols that specify the time and amount of meals.
A high SD value indicates an aggressive control value, which can be seen at higher infusion insulin values and may be suitable for some type 1 diabetics with higher TDD.
On the other hand, a low SD value represents a stable control value around the known value, which may be sufficient for some patients to maintain glucose in the normal range and to inhibit blood glucose elevation.
A calibration iteration procedure according to the present invention is illustrated in Algorithm 1 of
Here, the TIR and the time below the range were calculated from glucose concentration time in the normal range (between 70 and 180 mg/dL per day), and glucose time in the hypoglycemic state (less than 70 mg/dL), respectively. β>1 is an adjustable parameter.
In each experimenter iteration, if certain conditions were met, a quick rest was performed.
For example, a “good” state is defined as a repetit ion TIR≥90% day for five consecutive skday, and a “bad” state is defined as a repetition TIR25% day for five consecutive skd ay.
Finally, the regression equation used in the present invention is shown in
This may be represented by the following formula (9).
S
dInit=a·eb·TDD (9)
Here, constants a and b were 0.013 and 0.086, respectively.
In an embodiment of the present invention, an object of the OnSL module is to update a control variable and improve a performance index in order to prevent a hypoglycemic accident.
Since the intra-day IS in the patient varies in each type 1 diabetic patient, the purpose of the OnSL module in one embodiment of the present invention is divided into daily and weekly, e.g., the daily pattern of insulin sensitivity (IS) is lower in the morning (morning) than at noon and night (lunch and dinner) on average, and other uncontrolled parameters (e.g., CGM) may fail. Specifically, in an embodiment of the present invention, the focus was on preventing hypoglycemia every day and increasing the TIR every week.
At each date k, the update rule has blood glucose of less than 60 mg/dL (TBR60), and follow performance based on performance percentage of TBR60x={TBR60D, TBR60N, TBR60W} (wherein D corresponds day, N corresponds night, W corresponds to TBR60).
TBR 60 was selected as the update algorithm according to an embodiment of the present invention in consideration of an error from a CGM value, and the update rule according to the present invention is as follows.
Here, δ={SD, SN} is a control variable of day and night updated by the algorithm according to the present invention. In an embodiment of the present invention, iobmax is not updated daily in consideration of the stability of control.
The constraints f1 and f2 represent gains of 0.2 and 0.1, respectively.
A similar update rule was applied to weekly updates. In an embodiment of the present invention, a percentage constant was used to update the control variable. Thus, at each date j, the average TIRx={TIRD, TIRN, TIRW} was calculated, and at date k the update rule follows the following equation (11).
Here, δ may be changed based on a specific performance matrix obtained daily, and constants θ1 and θ2 represent gains of 0.2 and 0.1. In one example of the present invention, when TIRW>0.9, θ1>θ2, and the control variable was maintained. However, if any one of the conditions of TBR60W>0 and TIRW<80 at the subsequent date (k+1) is satisfied, the update process is resumed again. From Equation (10), it can be seen that the control parameter is updated if any one of the conditions is satisfied. Accordingly, any one or all of the δ variables may or may not be updated in one iteration process. The present invention further relates to maintaining the current variable value if a condition is not satisfied in a particular iteration (i.e., SDK=SDK+1).
Referring to
In
Here, in scenario 1, the Hovorka model of the UVA/Padova model and the built-in variability of daily IS embedded in DMMS.R were used. Each TID (type 1 diabetes) VP consumed a fixed amount of food at a specific intake time. In scenario 2, we tested an algorithm that consumed a relatively large amount of 90 g of carbohydrates in the morning, lunch, and evening. This scenario is closely mimicked the test protocol of Dassau et al. (2015). Earlier mentioned in Dassau et al. 2015, this scenario may exhibit a troublesome stress test for the AP controller, particularly for the FCL-AID. Finally, scenario 3 was used to determine the importance of the proposed OnSL module as a more challenging and realistic scenario (Gondhalekar et al., 2018). We tested the scenario designed in UVA/Padova TID VP here, but modified the original version and introduced additional daily variability in insulin uptake and IS. The nominal pattern of IS presented by Visentin et al. (2015) was implemented and randomly selected for each in silico subject. In addition, the daily variability in sensitivity was characterized by random ±50% variation in nominal IS from the beginning to the end of the test.
During the last 31 days of the simulated scenario and the proposed algorithm during the first 2 weeks of the open loop according to the prior art. It shows that the algorithm according to the present invention has succeeded in customizing and adapting to each patient state.
While the present disclosure has been described with reference to the embodiments illustrated in the figures, the embodiments are merely examples, and it will be understood by those skilled in the art that various changes in form and other embodiments equivalent thereto can be performed. Therefore, the technical scope of the disclosure is defined by the technical idea of the appended claims. The drawings and the forgoing description gave examples of the present invention. The scope of the present invention, however, is by no means limited by these specific examples. Numerous variations, whether explicitly given in the specification or not, such as differences in structure, dimension, and use of material, are possible. The scope of the invention is at least as broad as given by the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2024-0008902 | Jan 2024 | KR | national |
| 10-2024-0177019 | Dec 2024 | KR | national |
