This application relates to a method and system for use with data processing and imaging systems, according to one embodiment, and more specifically, for a mobile architecture using cloud for data mining application such as Hashimoto Thyroiditis (HT) classification and diagnosis.
Imaging-based technologies have been active for over a century and today the same imaging-based technologies are used electronically for creating pictures of the human body and examining it. Majority of these imaging modalities are non-invasive and painless. Depending upon the symptoms of the patient's disease, a physician will choose a type of the imaging modality, its diagnosis, treatment and monitoring. Some of the most famous medical imaging modalities are Ultrasound, X-ray, MR, CT, PET, SPECT and now more molecular and cellular level. These imaging modalities are conducted by the radiologist or a technologist who are well trained, to operate and know the safety rules.
The importance of imaging-based techniques for diagnosis, treatment, monitoring is increasing day-by-day. Thus more and more body images are generated every day. Hospitals and health care providers are generating image data at an alarming rate. There is no doubt that one has to design complex medical imaging software for diagnosis, treatment and monitoring, but it is becoming challenging to access these data in this age of the world. Storage of the medial images is one issue and how to access this data for decision making such as diagnosis, treatment and monitoring is another issue.
Hashimoto's Thyroiditis (HT) is an autoimmune disease that is characterized by lymphocytic infiltration and disruption of thyroid gland tissue architecture and production of specific autoantibodies against thyroid. Hashimoto's Thyroiditis is the most common type of inflammation of the thyroid gland, and a most frequent cause of hypothyroidism. Early diagnosis of Hashimoto's Thyroiditis would be advantageous in predicting thyroid failure.
The following are the commonly first lowed diagnostic criteria of Hashimoto's Thyroiditis: (i) a positive test for thyroid autoantibodies in serum, (ii) an elevated serum thyrotropin (TSH) concentration, or (ii) the presence of lymphocytic infiltration of the thyroid in histopathologic examination. Other common diagnostic tests are fine-needle aspiration biopsy and an ultrasound (US) scan. Among these techniques, the most preferred choice is thyroid ultrasonography which is a non-invasive diagnostic test that provides an image of the structure and the characteristics of thyroid. It was reported that autoimmune thyroiditis could be successfully excluded on the basis of ultrasound alone in 1962 cases among 2322 cases studied (84%). Moreover, ultrasound is affordable, widely available, does not use harmful ionizing radiation, and has relatively shorter acquisition time compared to other modalities like Computed Tomography (CT) and Magnetic Resonance Imaging (MRI).
A regular thyroid tissue is characterized by homogeneity and high echogenicity in ultrasound. In Hashimoto's Thyroiditis, the architecture destruction of the follicles and lymphocytic infiltrations result in decreased echogenicity. There is evidence that reduced thyroid echogenicity demonstrated by ultrasonography is a strong predictor of chronic autoimmune thyroiditis even when this disorder has not been suspected clinically. Earlier, this change in echogenicity was evaluated based on a rough visual comparison with the surrounding neck muscular tissue. Subsequently, analysis of grayscale histogram was carried out for quantitative measurement of echogenicity decline. Other studies too have proposed that computerized gray-scale ultrasound gives quantitative determination of thyroid echogenicity and mean tissue density in thyroid autoimmune diseases.
These computerized methods have the advantages of being more objective. However, they are limited by the fact that there is lack of procedure standardization because individual investigators use various initial ultrasound settings. Echogenic appearance of the thyroid gland varies with the adjustment of the gain. Thus, ultrasound diagnosis of Hashimoto's Thyroiditis is still operator-dependent and defined conditions are necessary to evaluate exact data. To compensate the attenuation of ultrasound energy as the pulses traverse the different layers of the neck, a corresponding amplification of ultrasound signals by the operator is necessary. Too much amplification may mask a true reduction in thyroid echogenicity, and too little amplification may lead to a false diagnosis of reduced thyroid echogenicity. Furthermore, in the end stage of Hashimoto's Thyroiditis, mean tissue density assessment may be misleading because of the presence of a combination of the hyperechoic and hypoechoic signals in the examined zone. These operator dependent and echogenic limitations is another reason for development of an objective, non-invasive, and accurate Hashimoto's Thyroiditis diagnosis support systems that use medical image mining techniques.
Image mining uses techniques from statistics and artificial intelligence to determine features which quantitatively characterize the patterns in an image. In this context, these features quantify the histopathologic components of the US thyroid images obtained from normal and Hashimoto's Thyroiditis-affected patients. These features can then be used to train supervised learning based classifiers to relate the extracted features from an image to the corresponding class (normal or Hashimoto's Thyroiditis-affected abnormal). The trained classifiers can then be used to predict the class of a new image which was not used for training. The key objective of this work is to develop one such Computer Aided Diagnosis (CAM-based paradigm that uses classification techniques to automatically differentiate ultrasound images from normal and Hashimoto's Thyroiditis affected cases in cloud-based settings. Thus, the proposed technique will have the following characteristics: (a) It will use thyroid images from the most commonly used, affordable and available, non-invasive and safe ultrasound modality; (b) The interpretations will be more objective and reproducible due to the use of standard image analysis algorithms; (c) Use of this technique will, not incur any additional cost because the proposed algorithm can be written into a software application at no extra cost and can be installed in the physician's computer; and (d) It will act as an adjunct tool that provides to second opinion on the initial diagnosis thereby increasing the confidence of the physician in planning, the subsequent treatment evaluation protocol for the patient.
This application is a novel method that presents a three tier architecture for image-based diagnosis and monitoring application using cloud. The presentation layer is run on the tablet (mobile device), while the business and persistence layer runs on the cloud or as set of clouds. The business and presentation layers can be in one cloud or multiple clouds. Further, the system can accommodate multiple users in this architecture set-up with multiple tenancies.
The application is designed to assist the endocrinologist, internal medicine or a physician in examining the Thyroid Disease and in particular diagnosis the Hashimoto Disease.
Data access from remote locations has become important day-by-day in this high information technology world. Due to this, now Cloud-based imaging can provide solution to such challenges. Even though, HIPPA or security or data ownership technologies are evolving, but the pros of Cloud-based technologies have outweighed the cons.
The Cloud-based technology offers, the first one is pricing. Cloud-based processing is less expensive due to low storage cost. Additional benefit is that if one uses Cloud for Software as a Service (SaaS) application, the storage cost can be free.
Another advantage of Cloud-based processing is the capacity to handle. Compared to costs for the local processing when the data storage requirements are changing dynamically, Cloud-based capacity may be advantageous. Expansion possibility is easy to handle. Emergency storage requirements may also less challenging to handle in Cloud-based processing.
Another major advantage is the disaster recovery. One needs regular backups and maintenance; this can be avoided in the Cloud-based processing.
Having discussed the benefits of Cloud-based processing, it is thus important on how to use Cloud-based services for applications which short time to run applications. This innovative application is about the architecture is designed for medical imaging applications, such as cardiovascular, prostate cancer, ovarian cancer, liver cancer, thyroid cancer and in particular diagnosis of Hashimoto Disease. Today's medical based applications do not just require viewing of the images, but also processing business layers for doctors to get the clinical information such as diagnosis, treatment support and monitoring. Thus the main requirement in today's Cloud-based processing is how to build medical imaging architectures which can benefit from Cloud-based processing, particularly for Thyroid Disease Diagnosis and in particular Hashimoto Disease.
Now that hand held devices have come into the world such as iPad, Samsung tablets or iPhones, it is thus important to understand how to build medical imaging architectures which has several tiers or layers in their architectural designs. This innovative application demonstrates an imaging-based architecture utilizing the Cloud-based processing. The application shows coverage for Thyroid Cancer Diagnosis and in particular Hashimoto Disease. Besides this, the application can be extended to vascular imaging or Cardiac imaging, gynecological imaging, prostate cancer imaging and liver cancer imaging, but is extendable to other anatomies as well.
In view of the foregoing, it is a primary object of the present invention to provide a novel method and apparatus for automated mobile data mining from ultrasound images for diagnostic and monitoring application, particular Hashimoto Disease of Thyroid organ, and further providing extensions to MR or CT images and in general to any other imaging-based data mining application.
It is another object of the present invention to develop a mobile-based architecture which can process images by distributing components of the architecture in different Clouds, but same physical location.
It is another object of the present invention to develop a data mining architecture having the business layer in one Cloud while running the Persistence Layer in another Cloud, not necessarily in the same physical location, particularly applied to the Thyroid Disease Management and in particular for the Hashimoto Disease Diagnosis.
It is another object of the present invention to develop an image-based data mining Cloud-based application which can have multiple-tenants and multiple-users. This data mining application can be where the Business layer is for cardiovascular application (such as IMT measurement, IMTV measurement, Plaque Characterization for Symptomatic vs. Asymptomatic classification of plaque, Stroke Risk computation, and monitoring stroke risk), or urology application such as benign vs. malignant tissue prostate tissue classification for prostate cancer, or gynecological application for classification of ovarian cancer or benign vs. malignant thyroid cancer for endocrinology application, particularly Hashimoto Disease Diagnosis and Classification, or for liver application such as a classification of fatty liver disease (FLD) compared to normal liver.
It is another object of the present invention to provide different configuration options in the Business Layer controlled by the Presentation Layer, where the Presentation Layer can control wirelessly different configurations. Each configuration can be another scientific method for generation of clinical information, such as different set of classifiers used for training and testing during the Thyroid Disease Diagnosis and in particular Hashimoto Disease Diagnosis.
It is another object of the present invention to provide multi-tenancy for data mining applications using distributed architectures, where data mining application can be Business layer for (a) cardiovascular application (such as IMT measurement, IMTV measurement, Plaque Characterization for Symptomatic vs. Asymptomatic classification of plaque, Stroke Risk computation, and monitoring stroke risk); (b) prostate cancer application such as benign vs. malignant prostate tissue classification or characterization for prostate cancer); (c) ovarian cancer tissue characterization and classification; or (d) thyroid cancer application (such as benign vs. malignant thyroid tissue classification or characterization for thyroid cancer) and in particular Diagnosis of Thyroid Disease and its management; or (e) classification of liver tissue such as Fatty Liver Disease.
It is another object of the present invention to provide multi-tenancy for data mining applications using, distributed architectures, where multi-tenancy can be using different imaging modality like MRI, CT, Ultrasound or a combination of these for fusion. The multi-tenancy set-up has data mining application where Business layer is: a) cardiovascular application (such as IMT measurement, IMTV measurement, Plaque Characterization for Symptomatic vs. Asymptomatic classification of plaque, Stroke Risk computation, and monitoring stroke risk); (b) prostate cancer application (such as benign vs. malignant prostate tissue classification or characterization for prostate cancer); (c) ovarian cancer tissue characterization and classification; or (d) thyroid cancer application (such as benign vs. malignant thyroid tissue classification or characterization for thyroid cancer) and in particular Hashimoto Disease Management; or (e) classification of liver tissue such as Fatty Liver Disease.
It is another object of the present invention to provide data mining applications using distributed architectures, where the presentation layer can be hand-held device like iPhone, iPad, Samsung Tablet or notebook or laptop or desktop and data mining application can be for (for (a) cardiovascular application (such as IMT measurement, IMTV measurement, Plaque Characterization for Symptomatic vs. Asymptomatic classification of plaque, Stroke Risk computation, and monitoring stroke risk); (b) prostate cancer application (such as benign vs. malignant prostate tissue classification or characterization for prostate cancer); (c) ovarian cancer tissue characterization and classification; or (d) thyroid cancer application (such as benign vs. malignant thyroid tissue classification or characterization for thyroid cancer) and in particular Hashimoto Disease Diagnosis and Management or (e) classification of liver tissue such as Fatty Liver Disease.
It is another object of the present invention to provide data mining applications where Business layer for (a) cardiovascular application (such as NT measurement, IMTV measurement. Plaque Characterization for Symptomatic vs. Asymptomatic classification of plaque. Stroke Risk computation, and monitoring stroke risk); (b) prostate cancer application (such as benign vs. malignant prostate tissue classification or characterization for prostate cancer); (c) ovarian cancer tissue characterization and classification; or (d) thyroid cancer application (such as benign vs. malignant thyroid tissue classification or characterization for thyroid cancer); and in particular Hashimoto Disease Management or (e) classification of liver tissue such as Fatty Liver Disease, such that it can process the B-mode ultrasound or RF-mode ultrasound image
It is another object of the present invention to provide a method to diagnose a Thyroid Disease, in particular Hashimoto Disease using a combination of training-based image classification, system.
It is another object of the present invention to provide a method to diagnose a Thyroid Disease, in particular Hashimoto, Disease using a combination of training-based image classification system, where the training system (off line system) uses a set of grayscale features such as Entropy features, Gabor wavelet features, Inverse Moment Features, Higher Order Spectra Features.
It is another object of the present invention to provide a method to diagnose a Thyroid Disease, in particular Hashimoto, Disease using a combination of training-based image classification system and testing based image classification system (on line process), where the testing system uses a set of grayscale features such as Entropy features, Gabor wavelet features, Inverse Moment Features and Higher Order Spectra Features.
It is another object of the present invention to provide a method to diagnose a Thyroid Disease, in particular Hashimoto, Disease using a combination of training-based image classification system and testing based image classification system, where the testing system uses a set of grayscale features such as Entropy features, Gabor wavelet features, Inverse Moment Features, Higher Order Spectra Features, such that a feature selection system is able to select the beast combination of features for training and testing classifiers in online and offline processing.
It is another object of the present invention to provide mobile data mining application where Business layer can be a 2D processing unit or a 3D processing unit.
It is another object of the present invention to provide mobile data mining application where Business layer can be a 2D processing unit or a 3D processing unit for diagnostic and monitoring application with different configuration options for the Business Layer.
It is another object of the present invention to provide mobile data mining application where Business layer can be a 2D processing unit or a 3D processing unit for diagnostic and monitoring application with different configuration options for the Business Layer, where these applications use training-based systems.
The various embodiments is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which:
Wavelet transform captures both the spatial and frequency information of a signal. Discrete Wavelet Transform (DWT) uses filter banks composed from finite impulse response filters to decompose signals into low and high pass components. The low pass component contains information about slow varying signal characteristics, and the high pass component contains information about sudden changes in the signal. DWT, however, is not a time-invariant transform. The translation invariance of DWT can be restored by using Stationary Wavelet Transform.
A 2D sub-band transform with three levels of decomposition. When low pass filtering, using filter g[n] is applied to both the rows and columns of the image, the LL coefficients are obtained which are called the approximation coefficients ‘A’. These coefficients are representative of the total energy in the images. When low pass filtering is applied to the rows, and high pass filtering using filer h[n] is applied to the column values, the resultant HL coefficients contain the vertical details of the image ‘V’ Row-wise high pass filtering and column-wise low pass filtering result in the LH coefficients, which contain the horizontal details of the image ‘H’. High pass filtering of both row and column values results in the finest-scale HH coefficients, which contain the diagonal details of the image D. Decomposition is further performed on sub-band LL to attain the next coarser scale of wavelet coefficients. The input approximation coefficients cAj and the results for level j+l. In this application, we first converted the image to grayscale range of [0, 255] and then applied SWT using rhio3.1 as the mother wavelet.
After obtaining, the wavelet coefficients at each level of the three-level SWT decomposition, we determined the following features for each of the ten subsets of coefficients: (a) Relative Wavelet Energy (RWEng); (b) Relative Wavelet Entropy (RWEnt); (c) Probability of Energy (PEng), and (d) Probability of Entropy (Pent). Energy probability distribution in scales is the relative wavelet energy. Relative wavelet entropy tells how similar a probability distribution pj is with respect to another probability distribution qj referenced. In the following, sample equations. EngNa indicates the energy of the approximation coefficients cA obtained at level N. EngNh indicates the energy of the horizontal detail coefficients cDh obtained at level N. EngNv indicates the energy of the vertical detail coefficients cDv obtained at level N. EngNd indicates the energy of the diagonal detail coefficients cDd obtained at level N. Similar definitions hold true for the other terms used in the equations.
where EngNa=Σk|cAN(k)|2; EntN1=−ΣkcAN2(k)log(cAN2(k))
where N is the number of levels of decomposition, taken as 3; and k is the number of coefficients at each decomposition level.
The example computer system 2700 includes a processor 2702 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), a main memory 2704 and a static memory 2706, which communicate with each other via a bus 2708. The computer system 2700 may further include a video display unit 2710 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system 2700 also includes an input device 2712 (e.g., a keyboard), a cursor control device 2714 (e.g., a mouse), a disk drive unit 2716, a signal, generation device 2718 (e.g., a speaker) and a network interface device 2720.
The disk drive unit 2716 includes a machine-readable medium 2722 on which is stored one or more sets of instructions (e.g., software 2724) embodying any one or more of the methodologies or functions described herein. The instructions 2724 may also reside, completely or at least partially, within the main memory 2704, the static, memory 2706, and/or within the processor 2702 during execution thereof by the computer system 2700. The main memory 2704 and the processor 2702 also may constitute machine-readable media. The instructions 2724 may further be transmitted or received over a network 2726 via the network interface device 2720. While the machine-readable medium 2722 is shown in an example embodiment to be a single medium, the term “machine-readable medium” should be taken to include a non-transitory single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term “machine-readable medium” can also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the various embodiments, or that is capable of storing, encoding or carrying data structures utilized by or associated with such a set of instructions. The term “machine-readable medium” can accordingly be taken to include, but not be limited to, solid-state memories, optical media, and magnetic media.
The Abstract of the Disclosure is provided to comply with 17 C.F.R. §1.72(b), requiring an abstract that will allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.
This is a continuation-in-part patent application of co-pending patent application Ser. No. 12/799,177; filed Apr. 20, 2010 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application Ser. No. 12/802,431; flied Jun. 7, 2010 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application Ser. No. 12/896,875; filed Oct. 2, 2010 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application Ser. No. 12/960,491; filed Dec. 4, 2010 by the same applicant. This is also to continuation-in-part patent application of co-pending patent application Ser. No. 13/053,971; filed Mar. 22, 2011 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application Ser. No. 13/077,631; filed Mar. 31, 2011 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application Ser. No. 13/107,935; filed May 15, 2011 by the same applicant. This is also as continuation-in-part patent application of co-pending patent application, Ser. No. 13/219,695; filed Aug. 28, 2011 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application, serial no, 13/253,952; filed Oct. 5, 2011 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application Ser. No. 13/407,602; filed Feb. 28, 2012 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application Ser. No. 13/412,118; filed. Mar. 5, 2012 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application Ser. No. 13/449,518; filed Apr. 18, 2012 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application Ser. No. 13/465,091; filed May 7, 2012 by the same applicant. This is also a continuation-in-part patent application of co-pending patent application Ser. No. 13/589,802; filed Aug. 20, 2012 by the same applicant. This present patent application draws priority from the referenced co-pending patent applications. The entire disclosures of the referenced co-pending patent applications are considered part of the disclosure of the present application and are hereby incorporated by reference herein in its entirety.
Number | Date | Country | |
---|---|---|---|
Parent | 12799177 | Apr 2010 | US |
Child | 13626487 | US | |
Parent | 12802431 | Jun 2010 | US |
Child | 12799177 | US | |
Parent | 12896875 | Oct 2010 | US |
Child | 12802431 | US | |
Parent | 12960491 | Dec 2010 | US |
Child | 12896875 | US | |
Parent | 13053971 | Mar 2011 | US |
Child | 12960491 | US | |
Parent | 13077631 | Mar 2011 | US |
Child | 13053971 | US | |
Parent | 13107935 | May 2011 | US |
Child | 13077631 | US | |
Parent | 13219695 | Aug 2011 | US |
Child | 13107935 | US | |
Parent | 13253952 | Oct 2011 | US |
Child | 13219695 | US | |
Parent | 13407602 | Feb 2012 | US |
Child | 13253952 | US | |
Parent | 13412118 | Mar 2012 | US |
Child | 13407602 | US | |
Parent | 13449518 | Apr 2012 | US |
Child | 13412118 | US | |
Parent | 13465091 | May 2012 | US |
Child | 13449518 | US | |
Parent | 13589802 | Aug 2012 | US |
Child | 13465091 | US |