Patent Application
20240023838
This disclosure generally relates to the field of non-invasive blood glucose monitoring in combination with machine learning.
Diabetes mellitus is a disease characterized by high blood glucose levels resulting from dysregulation of the hormone insulin. Diabetes is managed through physical activity and dietary modification and requires careful monitoring of blood-glucose concentration. Diabetes affects approximately one out of every 10 people in the United States (USAHealth. “About Diabetes Care at USA Health” (2022)). Its prevalence has increased from 23.4 million Americans in 2015 to 30.3 million in 2021 and continues to rise at an alarming rate (CDC, “National Diabetes Statistics Report”, (2022)).
The standard technique for determining blood glucose concentration involves using a glucometer (Salacinski, A. J., et al., J. of Diabetes Sci. and Tech. 95-99 (2014)). In brief, this device determines glucose concentration either in droplets of blood from a finger prick in portable devices or with a laboratory blood draw in a clinical setting. Such techniques may be invasive, painful, uncomfortable, and otherwise disadvantageous for health. Successful management of diabetes involves monitoring blood glucose levels multiple times per day using this glucometer. Taking repeated finger pricks over the course of a day is painful and creates a risk of infection at the collection site (Farage, M. A., et al., American J. of Clinical Derm., 73-86 (2009)). The situation is especially burdensome in the aging population, where skin elasticity is reduced, and the immune response is slow. Glucose monitoring in diabetes may also be performed by pervasive devices with the capacity of in situ computation. J. Hartz, et al., Current cardiology reports, 18(12): 1-11 (2016). Some methods involve implanting a thin lancelet subcutaneously. This provides continuous monitoring. An inherent issue of this minimally invasive method is the risk it poses for tissue damage and infection. C. F. So, et al., Medical Devices (Auckland, NZ), 5: 45 (2012).
According to the current literature (Shang, T., et al., J. of diabetes science and technology, 168-214 (2022)), there are several types of non-invasive glucose monitoring systems including thermal, electrical, nanotechnology, and optical systems. Non-invasive optical glucose monitoring (NIO-GM) systems are based on optical glucose monitoring. Non-invasive fluid sampling (NIFS-GM) systems are based on fluid sample glucose estimation. Minimally invasive devices (MI-GM) involve the insertion of a sensor into the subcutaneous tissue underneath the skin.
These noninvasive approaches to monitoring blood glucose have limitations. Electrical approaches are sensitive to temperature and may lack sources capable of producing meaningful amounts of energy that can penetrate the tissue for noninvasive analysis. W. V. Gonzales, et al., Sensors, 19(4): 800 (2019); A. Tura, et al., Sensors, 10(6): 5346-5358 (2010); M. Gourzi, et al., Journal of Med. Eng. & Tech., 29(1): 22-26 (2005); H. Melikyan, et al., Medical Eng. & Phys., 34(3): 299-304 (2012); P. H. Siegel, IEEE, 1-3 (2015); S. A. Weinzimer, Diabetes Tech. & Thera., 6(4): 442-444 (2004). Thermal approaches are susceptible to interference from environmental conditions and may be sensitive to sweat. C. D. Malchoff, et al., Diabetes Care, 25(12): 2268-2275 (2002); J. M. Buchert, Optical Security and Safety, 5566.100-111 (2004); O. K. Cho, et al., Clinical Chem., 50(10): 1894-1898 (2004); F. Tang, et al., Sensors, 8(5): 3335-3344 (2008); Y. Tanaka, et al., Photons Plus Ultrasound: Imaging and Sensing, 10494. SPIE, 494-498 (2018). Thermal approaches are also expensive and have long integration times. W. V. Gonzales, et al., Sensors, 19(4): 800 (2019). Nanotechnology approaches, potentially in a combination with optical resources, have potential toxicity issues, short lifespans, limitations associated with photostability, and high costs. P. W. Barone, et al., Analytical Chem., 77(23): 7556-7562 (2005); J. C. Pickup, et al., Biosensors and Bioelectronics, 20(12): 2555-2565 (2005); D. C. Klonoff, J. of Diabetes Sci. and Tech., 6(6): 1242-1250 (2012); P. W. Barone et al., J. of Diabetes Sci. and Tech., 3(2): 242-252 (2009); L. Chen, et al., Sensors, 18(5): 1440 (2018).
Optical technologies include Mid-Infrared Spectroscopy, which is a vibrational spectroscopy technique. J. Coates, Applied Spec. Rev., 33(4): 267-425 (1998).; S. Liakat, et al., Biomedical Opt. Exp., 4(7): 1083-1090 (2013); S. Liakat, et al., Biomedical Opt. Exp., 5(7): 2397-2404 (2014). Optical technologies may be expensive and limited due to light penetration depths. Raman Spectroscopy provides a way to measure molecular compositions through inelastic scattering but may be prone to interference by other molecules such as hemoglobin. R. Pandey, et al., Accounts of Chem. Res., 50(2): 264-272 (2017); Y. Xu, et al., Biomed. Sensing, Imaging, And Tracking Tech., 2976. SPIE, 10-19 (1997); S. M. Lundsgaard-Nielsen, PloS one, 13(5): e0197134 (2018). Raman Spectroscopy may also include a long collection time. Far-Infrared Spectroscopy may have less scattering of mid-infrared approaches but may also have strong water absorption that makes the identification of molecules in the sample difficult. F. Tang, et al., Sensors, 8(5): 3335-3344 (2008). Time of Flight and Terahertz Time-Domain Spectroscopies use short and ultrashort laser pulses to measure travel time of reflected signals. S. Gusev, et al., 2017 Progress. In Electromagnetics Research Symposium Spring (PIERS). IEEE, 3229-3232 (2017); O. Cherkasova, et al., Optical and Quantum Electronics, 48(3): 1-12 (2016). They may have long measurement times and low spatial and depth resolutions.
Near-Infrared Spectroscopy (NIR), which relies on absorption and scattering of wavelengths, may be low cost. NIR has a signal intensity that is directly proportional to the concentration of the analyte, requires minimal sample preparation, and works in the presence of interfering substances such as glass and plastic. Monte-Moreno et. al used this method to estimate blood sugar, obtaining a Clarke error grid placed 87.7% of points in zone A, 10.3% in zone B, and 1.9% in zone C. E. Monte-Moreno, Artificial Int. in Med., 53(2): 127-138 (2011). Yamakoshi et al. also used NIR for estimating glucose, obtaining a Clarke error grid placed 90.05% of points in zone A, and 9.95% in zone B. K. Yamakoshi, J. of Biomed. Opt., 11(5): 054028 (2006). Also, Alarcon-Paredes et al. used this technology to estimate blood sugar with a Clarke error placed of points in zone A, and 9.68% in zone B. A. Alarcon-Paredes, Applied Sci., 9(15): 3046 (2019).
Therefore, there exists a need for a monitoring system that allows someone suffering from diabetes to monitor their blood sugar in a better way.
It is to be understood that this summary is not an extensive overview of the disclosure. This summary is exemplary and not restrictive and it is intended to neither identify key or critical elements of the disclosure nor delineate the scope thereof. The sole purpose of this summary is to explain and exemplify certain concepts of the disclosure as an introduction to the following complete and extensive detailed description.
The present disclosure relates to a blood glucose monitoring system. The system can be a non-invasive optical glucose monitoring (NIO-GM) system.
The present disclosure relates to a main body configured to be positioned on a portion of a user. Such a main body can further comprise one or more sensors disposed within the main body and configured to collect information relating to user characteristics from the portion of the user. Sensors may include but are not limited to small, intelligent glucose sensors. As a non-limiting example, a main body includes but is not limited to a clip, as shown in
The present disclosure relates to a computing device. A computing device is functionally disposed to allow operations of a model. A model may include but is not limited to any neural network or other machine learning enabled function. Such a neural network can be selected from a group consisting of but not limited to a convolutional neural network (CNN) and an Artificial Neural Network (ANN). The neural network receives the data collected by the main body as input. The neural network uses data collected by the main body to first be trained. About 80% of the collected data is used for training, and about 20% of the collected data is used for testing. The neural network then analyzes the data to produce a blood glucose level estimation of the user.
The present disclosure relates to a cloud-based database configured to store the glucose level estimation from the neural network. The cloud-based database allows for real-time results for a user.
The present disclosure relates to a mobile application coupled to the cloud-based database to display real-time glucose level estimations to a user. The mobile application provides continuous glucose monitoring and history data for users. The mobile application also allows users to manually enter glucometer readings for comparison purposes and for additional training of the neural network.
The features and components of the following figures are illustrated to emphasize the general principles of the present disclosure. Corresponding features and components throughout the figures can be designated by matching reference characters for the sake of consistency and clarity.
The present disclosure can be understood more readily by reference to the following detailed description, examples, drawings, and claims, and their previous and following description. However, before the present compositions, systems, and/or methods are disclosed and described, it is to be understood that this disclosure is not limited to the specific devices, systems, and/or methods disclosed unless otherwise specified, as such can, of course, vary. It is also to be understood that the terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting.
It should be appreciated that this disclosure is not limited to the compositions and methods described herein. It is also to be understood that the terminology used herein is for the purpose of describing certain embodiments only, and is not intended to be limiting, since the scope of the present disclosure will be limited only by the appended claims.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Although any compositions, methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention. All publications mentioned are incorporated herein by reference in their entirety.
Unless defined otherwise, all composition percentage values used herein are given in terms of weight percentage.
The use of the terms “a,” “an,” “the,” and similar referents in the context of describing the presently claimed invention (especially in the context of the claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context.
Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein.
Use of the term “about” is intended to describe values either above or below the stated value in a range of approx. +/−10%; in other embodiments the values may range in value either above or below the stated value in a range of approx. +/−5%; in other embodiments the values may range in value either above or below the stated value in a range of approx. +/−2%; in other embodiments the values may range in value either above or below the stated value in a range of approx. +/−1%. The preceding ranges are intended to be made clear by context, and no further limitation is implied. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.
As used herein, the Beer-Lambert Law of Absorption is a law associated with light absorption that is governed by the below equation.
I=I
010(−l,∈,c)=I010(−l,μ
where I0 is the initial light intensity (W/cm2), I is the intensity of the ith at any depth within the absorption medium in W/cm2, 1 is the absorption depth within the medium in cm, ε is the molar extinction coefficient in L/(mmol cm), and c is the concentration of absorbing molecules in mmol/L. The product of c and ε is proportional to the absorption coefficient (μa).
II. Non-Invasive Blood Glucose Monitoring System
The present disclosure relates to a non-invasive, reliable, and user friendly glucose monitoring system. The system offers significant improvements in accuracy, accessibility, and usability compared to existing techniques. Accessibility may be enhanced by a reduced number of components required for some embodiments described herein. In a non-limiting example, some devices may require only a laser light, a camera, and a processing system.
Systems disclosed herein are capable of being used with medical diagnostics, spectroscopy-based techniques, machine learning, and/or image processing. In a particular example, spectroscopy, image processing, and machine learning techniques may be integrated into the monitoring system to achieve accurate, non-invasive blood glucose estimation. Advanced algorithms and spectroscopic image analysis allow devices described herein to overcome limitations of existing techniques, such as inaccuracies, inconsistency, and invasiveness. Such systems described herein can estimate glucose levels and increase life quality for people with diabetes. Such a system eliminates needs for frequent invasive procedures, empowers individuals to monitor their glucose levels conveniently, and facilitates timely interventions for better disease management.
The present disclosure relates to a non-invasive optical glucose monitoring (NIO-GM) system. The system utilizes laser technology and spectroscopy images to accurately estimate and monitor blood glucose levels. Such laser technology may include a light beam. The light beam is directed at a human tissue. Once the light beam has contacted the human tissue, the energy absorption, reflection, or scattering is used to estimate glucose concentration (Pitzer, K. R., et al., Clinical Diabetology 307-314 (2001)). The system of the present disclosure is portable and inexpensive. The present disclosure additionally includes novel features such as incorporating a machine learning statistical approach that has not been used before for glucose estimation. The present disclosure additionally relates to a light method that can be run in pervasive devices presents results in real-time. An Example of a non-invasive system is shown in
Optical glucose monitoring systems may utilize various optical methodologies. Such optical methodologies include but are not limited to fluorescence spectroscopy (Pickup, J. C., et al., J. of diabetes science and technology, 62-71 (2013)), Raman spectroscopy (Enejder, A. M. K., et al., J. of biomedical optics 031114 (2005)), photoacoustic spectroscopy (Pai, P. P., et al., IEEE Transactions on Circuits and Systems I: Regular Papers, 663-676(2017)), optical coherence tomography (Haxha, S., et al., IEEE Photonics Journal 1-11 (2016)), occlusion spectroscopy (Amir, O., et al., “Continuous noninvasive glucose monitoring technology based on occlusion spec-troscopy” (2007)), and near-infrared absorption spectroscopy (Robinson, M. R., et al., Clinical Chemistry 1618-1622 (1992); Alarcon-Paredes, A., et al., Applied Sciences 3046 (2019); Kasahara, R., et al., Biomedical optics express, 289-302 (2018); Rachim, V. P., et al., Sensors and Actuators B: Chemical 173-180 (2019); Maruo, K., et al., IEEE Journal of selected topics in quantum electronics, 322-330(2003)).
As a non-limiting example, near-infrared absorption spectroscopy (NIR) provides advantages due to its low cost and practicality. The present disclosure relates to the use of near-infrared absorption spectroscopy, which is based on the Beer-Lambert Law of Absorption as understood in the art. In NIR, a polychromatic light source (Light Emitting Diode (LED)) is radiated through the sample. A diffraction grating then splits the transmitted radiation into its constituent wavelengths to a camera (sensor) and the images are analyzed by a computer board (detector).
The present disclosure relates to computing power of sensors and Internet of Things (IoT) devices. The present disclosure relates to a combination of computing power of sensors and IoT devices with NIO-GM systems to continuously analyze blood glucose from a microcomputer and a sensor embedded within a main body positioned on the finger or ear. Images of the rotational and vibrational transitions of chemical bonds within the glucose molecule are created using infrared spectroscopy, and incident light reflection is used to measure the corresponding fluctuation. The images are converted into an array list, which is used to provide entries to an Artificial Neural Network (ANN) to create an estimate of blood glucose concentration. The system is easy to use and is paired with a mobile application for free-living environments.
The present disclosure relates to IoT (Internet of Things) technologies. IoT technologies leverage power computing and low energy consumption of sensor devices and a Raspberry Pi camera for building the glucose-monitoring prototype (Raspberry Pi, “Raspberry pi 4 model B specifications”, (2021)). A Raspberry Pi camera is capable of capturing a set of images where a visible light laser passes through skin tissue. Glucose concentration can then be estimated by an artificial neural network model using the absorption and scattering of light in the skin tissue. Various programming languages can be used in conjunction with the glucose estimation including but not limited to TensorFlow, Keras, and Python code. While the Raspberry Pi camera captures images, a laser light captures absorption. The specifications of the laser light can be found in Table 1.
The present disclosure relates to a main body of a device that can be positioned on a portion of a user. Such a main body includes a compact and portable spectroscopy module integrated with advanced image processing and machine learning algorithms. A portion of a user may include but is not limited to a finger or earlobe. The optional choice to position a device on an earlobe of a user is unique and may allow for embodiments of the device as an earring or other jewelry. Such an application may allow a user to constantly wear the device for continued glucose monitoring. The main body of a device may capture spectroscopic images of the target area, such as the forearm or fingertip, using the device's sensors. These images contain valuable information about glucose levels within the tissues. This main body includes a laser on its first portion and a camera on its second portion, as shown in
Once the spectroscopy images are obtained, the device applies image processing techniques to extract relevant features, such as image tensors, color intensity, and statistical image information. These features are then fed into a machine learning model that has been trained on a large dataset of spectroscopy images and corresponding glucose measurements.
The present disclosure relates to models. The present disclosure relates to machine learning models and methodologies. The present disclosure further relates to a combination of machine learning models and methodologies with NIO-GM systems to monitor blood glucose levels in real time. The machine learning model utilizes its learned knowledge to accurately estimate the glucose levels from spectroscopy images. The estimation is based on the correlation between the extracted image features and glucose concentrations. The device provides real-time glucose readings, displaying them on a user-friendly interface for easy interpretation.
The present disclosure may relate additionally to a head model, which sits on top of a base model. The present disclosure additionally relates to an activation layer, which uses the Rectified Linear Unit (ReLu) activation function (Ramachandran, P., et al., arXiv preprint, arXiv:1710.05941 (2017)). The ReLU is a piecewise linear function that will output the input directly if it is positive; otherwise, it will output zero. The present disclosure additionally relates to a pooling layer, which incorporates feature-down sampling. It is applied to each layer in the three-dimensional volume. The present disclosure additionally relates to a fully connected layer, which involves flattening. The entire pooling feature map matrix is transformed into a single column, which is then supplied to the neural network for processing. These attributes are put together to make a model using the fully linked layers. Finally, the output is classified using a ‘Softmax’ activation function. The ANN model was trained using the ADAM technique, which included a total of 20 epochs, a batch size of 1, an initial learning rate of 1e-4, and a 0.5 dropout was considered. The next step was to train and test the model. About 80% of the data was used for training the model, and about 20% was used for testing the model.
Due to the large number of images, a convolutional neural network (CNN/ConvNet) approach can properly be used. The large number of images required to continuously monitor blood glucose levels daily provides a large database to train and analyze neural networks. The convolutional layer is the first layer of a CNN network and is the main building block that handles most of the computational work. A number of libraries, such as VGG16, Tensor Flow, Keras, MobileNetV2, Matplotlib, Numpy, and the like can be used. Sandler, M., et al., Proceedings of the IEEE conference on computer vision and pattern recognition, 4510-4520 (2018); Tammina, S., Int'l J. of Sci. and Res. Pub., 9(10): 143-150 (2019). CNN models pass filters through images (represented as tensors) to extract features such as edges, shapes, and colors. These two-dimensional features are then flattened and mapped as scalar data, which is then processed through normal neural network layers. Albawi, S., 2017 Int'l Conf. on Eng. and Tech., 1-6 (2017). CNN models can use different types of filters for images of varying sizes, providing a wide range of applications.
One, non-limiting example of a CNN is VGG16, as mentioned above. VGG-16 is a 16 layered deep CNN. A pre-trained version of the network can be loaded. Some pre-trained versions are already trained on more than a million images from an ImageNet database. Russakovsky, O., et al., Int'l J. of Comp. Vision, 115(3): 211-252 (2015). A pre-trained network can classify images into about 1000 object categories. Such networks learn rich feature representations for a wide range of images. Such models can also be changed to output a single numeric value (blood glucose), instead of the 1000 categories a model was trained on. The network may have an image input size of 224-by-224. This size can be changed to fit 160-by-120 and other sizes as needed. VGG16 has the ability to detect many different features and patterns and may perform better in some aspects compared to other models. Tammina, S., Int'l J. of Sci. and Res. Pub., 9(10): 143-150 (2019). An example of the VGG-16 architecture can be found in
An additional, non-limiting example of a CNN includes MobileNetV2, as discussed herein. MobileNetV2 is a mobile architecture that enhances the performance of mobile models across various model sizes, tasks, and benchmarks. In contrast to conventional residual models, which use expanded representations for the input, the MobileNetV2 architecture uses an inverted residual structure. An inverted residual structure includes inputs and outputs of the residual block that are thin bottleneck layers. Such model performs well considering its computational power. Therefore, MobileNetV2 is capable of being trained normally without using a pre-trained model version. MobileNetV2 includes numerous advantages including but not limited to low computational power usage, fast training times, and high-performance. Sandler, M., et al., Proceedings of the IEEE conference on computer vision and pattern recognition, 4510-4520 (2018). Additionally, MobileNet-v2 is a lightweight, 53-layer deep CNN method used to improve the classification of images with a limited dataset.
Additionally, linear models can be used instead of CNN models. Linear models perform machine learning and statistical modeling. Linear models additionally are capable of using a large number of algorithms for function approximation, decision making, regression, classification, clustering, prediction, and the like. Like CNNs, linear models are also capable of a wide range of applications and display enhanced performance. Linear models may be faster and less computationally intensive compared to neural networks and may be capable of providing similar or better results than neural networks in certain instances. The present disclosure may relate to linear models applying bagging, boosting, or ensemble learning techniques. Such techniques allow for higher performance, lower error, and more optimized training. A mix of models using these techniques may be used to determine the most effective for estimating blood glucose.
Linear models include but are not limited to Random Forest, Support Vector Machine, Bayesian Ridge, XGBoost, AdaBoost Ensemble, Histogram Gradient Boosting, Elastic Net, and KNeighbors. Donges, N., Contributor, E., entrepreneur, N.a.: Random forest algorithm: A complete guide; Raj, A.: Unlocking the true power of support vector regression (2020); Rothman, A.: The bayesian paradigm amp; ridge regression (2020); Sklearn.ensemble.adaboostregressor, https://scikit-learn.org; Brownlee, J.: Histogram-based gradient boosting ensembles in python (2021); Verma, Y.: Hands-on tutorial on elasticnet regression (2021); Brownlee, J.: Xgboost for regression (2021). To apply these methods to the spectroscopic images, data transformation techniques may be required to create new suitable databases for each method, as discussed herein. CNN and linear models may offer different advantages. CNN models can be used on tensor data because the algorithms are based on Linear Algebra suitable for use with multi-dimensional matrices (tensors). Linear models are suitable for all scalar data and use a wide variety of statistical techniques to approximate the function of the data.
As a non-limiting example of a linear model, random forest regressor (RFR) may be used. RFR is a supervised learning algorithm built on Decision Trees and the Ensemble Learning Approach. Dong, X., et al., Frontiers of Comp. Sci., 14(2): 241-258 (2020). Decision Trees are tree-diagrams of statistical decisions that lead users to a specific outcome, result, or prediction. Random Forest uses an optimized approach to ensemble learning called bagging (bootstrap-aggregating). Bagging creates multiple decision trees that train on random segments of the training data. These trees are then used in unison to predict unknown values. RFR's combination of Decision Trees and bagging in addition to its high performance in many domains may prove advantageous. Donges, N., Contributor, E., entrepreneur, N.a.: Random forest algorithm: A complete guide., https://builtin.com/data-science/random-forest-algorithm.
As an additional, non-limiting example of a linear model, support vector regressor (SVR) may be used. SVR works on the principle of the Support Vector Machine (SVM). Raj, A.: Unlocking the true power of support vector regression (2020); Noble, W. S., Nature Biotech., 24(12): 1565-1567 (2006). This model uses simple regression algorithms to fit a line, curve, and/or plane through data to create an approximate function. In simple regression, a goal is to minimize the error rate while in SVR it is to fit the error inside a certain threshold. The flexibility of SVR allows a user to decide how much error is acceptable in the model. SVR will then find an appropriate line (or curve or plane) to fit the data accordingly. This technique may be advantageous due to its ability to reduce overfitting and handle outliers in data. SVR is a well-performing and versatile model.
As an additional, non-limiting example of a linear model, Bayesian ridge regressor (BRR) may be used. Ridge Regression is a classical regularization technique used in Statistics and ML. Rothman, A.: The bayesian paradigm amp; ridge regression (2020). Bayesian regression allows a natural mechanism to survive insufficient or less distributed data by generalizing the data. Generalizing is capable of significantly reducing overfitting and handling outliers. In addition, this model is capable of outputting a probability distribution. A probability distribution allows output of multiple predicted values where the model chooses the most likely value. BRR may be advantageous due to its ability to perform well regardless of data quality.
As an additional, non-limiting example of a linear model, XGBoost Regressor (XGB) may be used. XGB uses gradient boosting, an ensemble learning technique similar in some aspects to Random Forest. XGB trains multiple decision trees to create an ensemble learner and uses intuition that the best possible next model, combined with previous model(s), minimizes the overall prediction error. Through combining multiple models training, XGB is capable of achieving high performance. XGB effectively deals with insufficient data and outliers. Extreme Gradient Boosting (XGBoost) is an efficient, open-source implementation of this gradient boosting algorithm. XGBoost allows for increased training speed and model performance.
As an additional, non-limiting example of a linear model, a Histogram Gradient Boosting Regressor (HGB) may be used. Histogram-based gradient boosting is an algorithm that uses a similar gradient boosting compared to XGBoost. HGB employs binning, instead of outputting a single value for blood glucose as with XGBoost. Binning converts continuous values into categories, similar to those used in classification scenarios. Brownlee, J.: Histogram-based gradient boosting ensembles in python (2021). By converting regression values to classification values, training speed is increased, and the amount of memory used is reduced. Such benefits may make HGB faster and lighter compared to the XGBoost algorithm.
As an additional, non-limiting example of a linear model, an AdaBoost Ensemble Regressor (ABR) may be used. An AdaBoost regressor is a metaestimator that begins by fitting another model on the original dataset. An ABR then fits additional copies of that model on the same dataset. While fitting additional copies, the weights of instances are adjusted according to the error of the current prediction. Sklearn.ensemble.adaboostregressor, https://scikit-learn.org. ABR creates more versions of the same model to tackle different sections of the training data, reducing error overall. Due to the large number of varying estimators that AdaBoost creates, the model is less prone to overfitting than other models. The present disclosure relates to AdaBoost trained with the KNeighbors model, as described herein.
As an additional, non-limiting example of a linear model, a KNeighbors Regressor (KNN) may be used. K-Nearest Neighbors (KNN) classifies a data point based on its nearest neighbors in a graph. Kramer, O.: K-nearest neighbors. In: Dimensionality reduction with unsupervisednearest neighbors, Springer: 13-23 (2013). This algorithm is a nonparametric supervised learning method used for classification and regression. In regression cases, the model takes the output value from a specific number of its nearest neighbors in the data, averages those values, and outputs that average. This algorithm does not make assumptions, so it may handle outliers and minimize error better than decision trees and linear regression. This model represents a novel approach to ensemble learning and is capable of high training speeds and high performance.
As an additional, non-limiting example of a linear model, an Elastic Net Regressor (ENR) may be used. Elastic Net is a regularized regression model that combines 11 and 12 penalties. This may be described as using lasso and ridge regression. Verma, Y.: Hands-on tutorial on elasticnet regression (2021). By combining both penalties, ENR dramatically reduces overfitting. This model also performs feature selection, removing unnecessary features from the data. ENR represents a novel use of penalties and feature selection.
Data captured using main bodies of the present disclosure must be prepared prior to processing via pre-processing. The image data set is converted into arrays with preprocessing and stored in a list format with assigned labels. Images are appended to a single data array with a corresponding label array. Data augmentation techniques including cropping, zooming, height and width shift, and horizontal flipping can be used, as well.
The present disclosure relates to cloud integration for real-time measuring. The glucose level estimation obtained from an ANN model is sent to the Cloud using Hypertext Transfer Protocol Secure (HTTPS). Additionally, a cloud-based database in the cloud is configured to store the glucose concentration data from the ANN model. An exemplary database for use is InfluxDB, which is written in the Go programming language for storing and retrieving time series data in fields such as operations monitoring, application metrics, Internet of Things sensor data, and real-time analytics. A database must be flexible enough to store data from each subject separately using tags. The integration with the Cloud is done using the Raspberry Pi, which is connected in Real-Time. The values that arrives to the database are immediately read by a mobile application for informing a subject. An overall workflow of the system of the present disclosure can be seen in
The present disclosure relates to a mobile application to display real-time results, as described above. The application is capable of displaying at least current values, historical values, and statistics relating to glucose levels. Within the application, users can review their current glucose measure and also view a chart of their previous measures, allowing them to track glucose variation over a specific period of time. The application is connected to the database and provides continuous glucose monitoring and history data for users. The application further is enabled to pair with smart voice assistants. Such assistants are capable of replying to commands related to the current blood glucose concentration and recent history of blood glucose patterns. Assistants may be connected to a mobile application or device using various methods of interfacing including but not limited to by python script.
As a non-limiting example, a smart voice assistant may be connected as follows. First, two primary connections are established including one from a smart voice assistant to a python script in the middleware and a second from this script to a database. To set up the connection between the smart voice assistant and python script, intents and utterances are constructed in a developer console to enable the smart voice assistant to accept specific requests from the user by voice. After the successful setup in the developer console, the developer console is connected to the python script using Ngrok and Flask-ask. Next, in the python script, functions are implemented corresponding to each built intent including but not limited to retrieve current, maximum, or minimum values, and enter data into the database. Then a connection is established between the python script and the database that returns the value to the voice assistant for reproducing the voice with the glucose value.
The present disclosure relates to a non-invasive glucose monitoring system that leverages the computation power of IoT devices and can be used for diabetes management. The system disclosed does not require blood samples and is based on images taken from the finger or ear. The system provides an integrated end-to-end framework that helps with continuous blood glucose monitoring and, at the same time, provides a management system for daily-life decision support. Such support may include but is not limited to providing a user information to decide when to increase or decrease blood sugar (via food consumption, sugar consumption, beverage consumption, exercise, etc.). In addition, the framework is capable of accurately predicting blood glucose behavior regardless of factors like age, race, and physical condition. Compared with other current approaches, the presently disclosed system can be attached to additional extremities (e.g. ear) and still obtain reliable results. An ANN model is used to classify and estimate the blood glucose concentration from the images. Images taken from both the ear and fingers of subjects display acceptable accuracy.
Advantages of devices, systems, and methods described herein are many. As non-limiting examples, the present disclosure eliminates the need for invasive blood sampling, providing a painless and convenient alternative for glucose monitoring. The present disclosure offers devices, systems, and methods for improved accuracy and reliability by leveraging machine learning algorithms and analyzing spectroscopy images. Systems and devices described herein include compact and portable designs. Such designs enable users to monitor glucose levels anytime and anywhere. Furthermore, the user-friendly interfaces described herein make non-invasive glucose monitoring accessible to individuals of varying technical expertise, empowering them to manage their diabetes effectively.
Devices, systems, and methods described herein represent significant advancements in non-invasive blood glucose estimation. The present disclosure offers reliable, convenient, and accurate solutions for routine glucose monitoring. As a non-limiting example, one or more combinations of spectroscopy, image processing, and machine learning techniques establish a new standard in the field and hold great potential to enhance the quality of life for individuals with diabetes.
Methods:
The monitoring system used Raspberry Pi (RPi), a portable camera (RPi camera), and a visible light laser. The RPi camera captured sets of images when visible light laser passed through skin tissue. The RPi camera captures one image every eight seconds over two minutes, for a total of 15 images. Brightness and contrast levels are set to 70 cycles/degree, camera ISO sensitivity is set to 800 and resolution is set to 640*480.
Data were collected from 8 individuals under IRB approval at Kennesaw State University (IRB-FY22-318). Two different datasets were collected using the system. The datasets consisted of 7 subjects that provided images from their ears, and 8 volunteers that provided images from their fingers. About 80% of the data was used for training the model, as described herein, and about 20% of the data was used for testing. The procedure that was followed for data acquisition involved 2 steps. First, a finger prick test was performed to get the blood glucose reading using a commercially available Glucometer (FORA 6 Connect BG50 Blood Glucose Starter Testing (FOR A, “FORA 6 Connect BG50KT50 Blood Glucose”, (2020)). The system then captured 15 images each of finger and ear.
Starting with data preparation, Labelbinarizer module of the Python library sklearn was used to convert the image data to the needed binary image format and store it into an array associated with its corresponding labels/categories (85-95 mg/dL, 96-110 mg/dL, 111-125 mg/dL). The next step was data augmentation, which provided the amount of data needed to train and test the model. Data augmentation is a method known in the art and includes but is not limited to cropping, zooming, height and width shift, and horizontal flipping. Finger and ear datasets were not mixed to check how each model performed individually. Hence, two models were built, one for the finger dataset and the second one for the ear dataset. The models estimated the blood glucose concentration for each place, finger and ear. For evaluation purposes, classification reports, accuracy, and a confusion matrix were used to assess the model. Accuracy of the system was assessed by comparing the glucose readings from the system with readings from a glucometer. A confusion matrix was used to show the classification accuracy. The x-axis and y-axis showed labels of the blood glucose value; in this case, 111-125, 85-95, and 96-110, as shown in
Results:
Initial results using limited data provided an accuracy rate of about 79% for finger readings, as shown in a confusion matrix in
A confusion matrix for ear readings showed an accuracy rate of about 62%, as seen in FIG. The model classified 5 images correctly and 4 images incorrectly in the 111-125 mg/dL category. In addition, 6 images were correctly classified and 0 images were classified incorrectly in the 85-95 mg/dL category. Finally, 2 images were correctly classified and 4 images were incorrectly classified in the 96-110 mg/dL category.
The results showed that optical techniques and machine learning methodologies can effectively measure blood glucose when the light is transmitted through the skin tissue and absorbed post-transmission. Overall, the system described herein exhibited an acceptable accuracy compared to other studies, as shown in Table 2. Of these other studies, the presently disclosed system is the only study to have shown efficacy with the human ear.
79%
90%
Methods:
Example 2 used the results of Example 1 described above. Images were chosen instead of other forms of spectroscopy measurement, such as light intensity and PPG signals, because image capture is more replicable, accessible, and faster than other methods of spectroscopy data collection. Spectroscopy images were collected from the fingers of 43 participants between 18-65 years old. The demographic of the participants were 23 females and 20 males, aged between 18 and 65 years, and with ethnic/racial self-identification as 7 white, 7 black, 13 Latino, and 16 Asian individuals. Two sets of 15 images were collected per participant. The first set was collected in a low-glucose fasting state. The second set was collected one hour following a meal. Blood glucose was determined via finger prick using a commercial glucometer (FORA 6 Connect BG50) per manufacturer instructions. A set of 4 images is presented in
Data Set Creation
Data transformation techniques were applied to the original data to generate three datasets, as described below.
An Image Tensor Dataset was prepared. The “Tensor Dataset” was created in order to train the CNN models (VGG16 and MobileNetV2). Tensors are multi-dimensional matrices of numbers used in linear algebra. Tensor application extends to images since images are multi-dimensional matrices of numbers as well. An image matrix consists of at least three dimensions including but not limited to height, width, and color (including but not limited to red, green, and blue).
To convert an image into a tensor, a three-dimensional matrix (tensor) was created with the resolution of the image and the color format. Since images used herein were 640×480 pixels using the RGB color format, the image tensor was also 640×480 pixels by three colors. Then each color value for each pixel was entered into each value in the tensor, obtaining a tensor of 921,600 values. The resulting image tensor dataset was maintained at 160×120×3 pixels to decrease computational time and necessary resources, when compared to a 640×480 pixels dataset. The final dataset included the tensors with their corresponding blood glucose value(s). A visual demonstration of the image-tensor conversion can be seen in
CNN models can be trained with such tensors because they use filtering techniques to analyze and process them. These filtering techniques may not be available in other machine learning algorithms. This may make the two CNN models MobileNetV2 and VGG16 advantageous.
Color Intensity Datasets were prepared. Four datasets were created based on extracting color intensity from original images. For each possible value of red, green, and blue (0-255), the number of pixels with that same value in an image can be counted and recorded in a histogram. Guzm′an-Guzm′an, I.P., et al., An iot-based noninvasive glucose level monitoring system using raspberry pi (2019). Through this process, a histogram with RGB values on the x-axis (256 possible values for red, green, and blue) and the number of pixels on the y-axis can be created, as shown in
Image Measurement Datasets were created. Five datasets were created by extracting measurement data from the images. To create the dataset, each image in the dataset was split into four channels including red, green, blue, and grayscale (the image with color removed). Then, for each color channel, the channel's pixel center of mass, minimum, maximum, mean, median, standard deviation, and variance were calculated. To calculate these values, the images were first converted into numerical tensors, then their tensors (3-dimensional matrix) were converted into an array for each channel, and then each channel array (1-dimensional list) was used for calculations such as mean, median, minimum, maximum, and the like. A demonstration of this process can be seen in
Values for each channel were compiled into the same dataset with the correct blood glucose value and repeated for every image. The resulting “Measurement Dataset”, consisted of 29 features: seven measurements for each of the four channels and one feature for the blood glucose. After the creation of this dataset, four new datasets were created by merging the measurement features of each image with the intensity values of the same image created in the previously mentioned intensity datasets. This process resulted in four new datasets including “Red-Measurement”, “Green-Measurement”, “Blue-Measurement”, and “RGBMeasurement.” The first three new datasets contained 285 features including 256 for the pixel intensities, 28 for the measurement features, and one for the blood glucose value. The last new dataset contained 797 features including 256 for each color channel, 28 measurement features, and one for the blood glucose value.
Model Training, Tuning, Testing, and the Like
After creating the datasets, each model was trained, tuned, and tested to each dataset to compare results, with only two exceptions. Since they can only be trained on tensor data, VGG16 and MobileNetV2 were only trained on the Tensor Dataset. Furthermore, the other linear models could only be trained on scalar data, so they were trained on every dataset except for the Tensor Dataset. The CNN models were trained using image data generators, which come with the TensorFlow library for Python that was used for training models. Moreover, before the training process, the image data generators were used to scale down the pixel values from 0-255 to 0-1 to reduce error and GPU usage. Besides these changes during training, the testing of CNN models was the same as the other models. Additionally, since the AdaBoost Ensemble Learning algorithm used another algorithm as a base estimator, for each dataset, the AdaBoost model was trained with the model that had the highest accuracy for that dataset. A summary of the models trained with each dataset can be seen in Table 2A.
To train the models, all of the datasets were split into training/testing splits where the training data was used to fit the model, and the testing data was used to measure the model's performance. The training/testing split ratio was 75:25 to ensure sufficient data to train the models and to ensure that no overfitting. After creating training/testing splits, each model in the set was fitted to training data and then tested. To ensure that the models were compared effectively, each model's hyper-parameters were tuned to each specific dataset to minimize error and overfitting. After the models were finished with training and tuning, they were tested, and the results were recorded in a table.
Three distinct metrics were considered for testing/tuning the models including mean absolute error (MAE), root mean squared error (RMSE), and the Clarke Error Grid. The MAE is the mean of all errors between the blood glucose values that a model predicts and the actual blood glucose value tied to an image. The RMSE is the root of the mean of each error squared. MAE may be advantageous as a more direct metric for calculating error, as it is unbiased towards all errors and treated as an average. Because it squares the errors, RMSE may be advantageous as being biased against large prediction errors, making it weighted against outliers. RMSE may be used in scenarios when an increase in error is disproportionate to the effect. As a non-limiting example, if the error increases from 5 to 10 and the effect is four times as bad, RMSE may be used. Since RMSE is always higher or equal to MAE, the difference between the two values is useful for evaluating outliers. If RMSE is significantly higher than MAE, then there are outliers in the predictions. For this reason, RMSE was used to tune the models to reduce overfitting but not recorded in the results or evaluation. Lastly, Clarke Error Grids were used to evaluate models since they have been widely used for several decades to evaluate the performance of blood glucose meters. Clarke Error Grids are scatterplots with predicted blood glucose values on the y-axis and actual blood glucose values on the x-axis. The grid is split into several zones, and each zone signifies a level of risk of a negative outcome due to the measurement error in blood glucose values, as shown in
Standards.
The ISO 15197 standard defines specifications for reliable medical devices, including glucometers. The ISO standard released in 2013 for glucose monitoring devices and systems for selftesting (15197:2013) has tighter accuracy requirements than the previous ISO standard set in 2003. The 2013 standard requires that 95% of the results are within a glucose concentration of pm15 mg/dL, compared with the reference method, for concentrations less than 100 mg/dL, or pm15% of zones A and B of the Parkes (Consensus) Error Grid for diabetes type 1.
Results:
For comparing the performance of the models, MAE and Clarke Error Grid (Zone A Percentage) metrics were used. The percentage of data points that fell into each zone of the clinical outcome were determined by analyzing the grid. Zone A's percentage was calculated by recording the number of predictions in Zone A (Clinically Accurate) as a percentage of the total number of predictions made. After the models were trained and tuned, they were tested with the testing data, and the results were recorded in Table 3 and Table 4 respectively.
AdaBoost with KNeighbors trained on the Red-Measurement dataset provided the most accurate estimates of blood glucose among all of the dataset-models tested. This dataset-model combination had an MAE of 9.4 mg/dl, an RMSE of 16.72 mg/dl, and a Clarke Error Grid Zone A Percentage of 90.78% as illustrated in
Table 5 shows a sample of 15 individual participants and the comparison between the glucometer reading and the proposed approach estimations. The average deviation between the actual value and the estimated value was within ±1.02 mg/dL and ±4.5 mg/dL. Based on these results, the proposed prototype and model met the International Standards Organization (ISO) 15197 standard that requires ±15% of zones A and B.
The Clark Error method was used to compare accuracy with studies of similar noninvasive approaches.
From best to worst, the models ranked AdaBoost, KNeighbors, Random Forest, XGBoost, HGB, Support Vector, Bayesian Ridge, Elastic Net, MobileNetV2, and VGG16 as displayed in
The intensity datasets performed better than the measurement datasets, and the dataset with only measurement values performed significantly worse. The image tensor dataset performed the worst of all datasets, while the CNN models performed the worst among the group of models. AdaBoost and KNeighbors performed the best with every dataset they were trained on, while XGBoost, Random Forest, and HGB generally outperformed the other models. These results suggested that the best data for blood glucose estimation by spectroscopy may be color intensity data focused on either the red channel or all three channels, though other data still provides value. The results further suggest that the KNeighbors algorithm is well-suited for blood glucose estimation with scalar data and using AdaBoost as an ensemble learner can boost performance. Models that used boosting and bagging (XGBoost, AdaBoost, HGB, etc.) outperformed models that did not (Elastic Net, Bayesian Ridge, Support Vector). Furthermore, the penalties and feature selection in Elastic Net and the binning in HistogramBased Gradient Boosting did not seem to increase performance compared to bagging and boosting. Finally, both the dataset and model results suggested that Convolutional Neural Networks and Tensor datasets performed worse than Linear Models, Ensemble Learners, and Scalar Data.
From training, tuning, and testing ten machine learning models on ten different datasets, it was determined that a particularly advantageous model for estimating blood glucose through spectroscopy images was AdaBoost trained with KNeighbors. Color intensity data collected from the red channel represented an advantageous image data for training a model. The highest performing dataset and model recorded a final Mean Absolute Error of 9.4, a Root Mean Squared Error of 16.72, and a Clark Error Grid Zone A Percentage of 90.78%. Intensity data outperformed measurement and tensor data, while the red and RGB channels outperformed all other color channels. Models that utilized bagging and boosting outperformed those which did not, while linear models outperformed CNN models, regardless of their support for bagging or boosting.
Results:
The accuracy of the device was tested based on differences in race and ethnicity, age, and gender per methods defined above. No significant difference in mean glucose estimates were observed between males and females (
No difference was observed in mean glucose concentration when data was stratified by age.
Differences in skin color and pigmentation were tested. In
This application is a U.S. Nonprovisional of and claims the benefit of U.S. Provisional Application No. 63/390,336 filed Jul. 19, 2022, which is hereby incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 63390336 | Jul 2022 | US |
