Patent Application
20240021297
Modern breast care involves extensive analysis of radiological images. Given the recent advances in radiological imaging, the amount of data radiologists are required to parse through and evaluate is increasing exponentially. This exponential increase in data often causes a large variation in the time to read radiological images. This variability in time is further exacerbated by individual circumstances of the radiologists (e.g., years of experience, areas of expertise, available image reading tools, etc.) as well as the specific contents of the data to be reviewed. As a result, optimally distributing workload (e.g., radiological images) to available radiologists in a screening center has proven challenging.
It is with respect to these and other general considerations that the aspects disclosed herein have been made. Also, although relatively specific problems may be discussed, it should be understood that the examples should not be limited to solving the specific problems identified in the background or elsewhere in this disclosure.
Examples of the present disclosure describe systems and methods for predicting reading time and/or complexity of a mammographic exam. In aspects, a first set of data relating to the reading time of mammographic exams may be collected from one or more data sources associated with a healthcare professional, such as image acquisition workstations, image review workstations, healthcare professional profile data, and preexisting patient data. The first set of data may be used to train a predictive model to predict/estimate an expected reading time for various breast images. Subsequently, a second set of data comprising at least one breast image may be provided as input to the trained predictive model. The trained predictive model may output an estimated reading time and/or reading complexity for the breast image. The estimated reading time may be used to prioritize mammographic exams or optimize the utilization of available time for clinical professionals.
Aspects of the present disclosure provide a system comprising: at least one processor; and memory coupled to the at least one processor, the memory comprising computer executable instructions that, when executed by the at least one processor, performs a method comprising: collecting a first set of data, wherein the first set of data comprises: breast image data, user profile data for a reader of the breast image data, and evaluation data for the breast image data; using the first set of data to train a predictive model to predict an expected reading time for the breast image data; collecting a second set of data, wherein the second set of data comprises at least the breast image; applying the second set of data to the trained predictive model; and receiving, from the trained predictive model, an estimated reading time for the breast image.
Aspects of the present disclosure further provide a method comprising: collecting a first set of data, wherein the first set of data comprises: breast image data, user profile data for a reader of the breast image data, and evaluation data for the breast image data; using the first set of data to train a predictive model to predict an expected reading time for the breast image data; collecting a second set of data, wherein the second set of data comprises at least the breast image; applying the second set of data to the trained predictive model; and receiving, from the trained predictive model, an estimated reading time for the breast image.
Aspects of the present disclosure further provide a computer-readable media storing computer executable instructions that when executed cause a computing system to perform a method comprising: collecting a first set of data, wherein the first set of data comprises: breast image data, user profile data for a reader of the breast image data, and evaluation data for the breast image data; using the first set of data to train a predictive model to predict an expected reading time for the breast image data; collecting a second set of data, wherein the second set of data comprises at least the breast image; applying the second set of data to the trained predictive model; and receiving, from the trained predictive model, an estimated reading time for the breast image.
Aspects of the present disclosure provide a system comprising: at least one processor; and memory coupled to the at least one processor, the memory comprising computer executable instructions that, when executed by the at least one processor, performs a method comprising: collecting a set of data, wherein the set of data comprises at least mammography exam data; applying the set of data to a predictive model trained to predict an expected reading time for the mammography exam data; and receiving, from the predictive model, an estimated reading time for the mammography exam data.
Aspects of the present disclosure provide a system comprising: at least one processor; and memory coupled to the at least one processor, the memory comprising computer executable instructions that, when executed by the at least one processor, performs a method comprising: collecting a first set of data, wherein the first set of data comprises: first mammographic exam data for one or more patients; user profile data for one or more mammographic exam readers of the first mammographic exam data; and evaluation data for the one or more mammographic exam readers; updating a case complexity index based on the first set of data, wherein the case complexity index comprises mappings between complexity values and factors affecting an amount of time required to interpret second mammographic exam data; collecting a second set of data, wherein the second set of data comprises the second mammographic exam data; providing the case complexity index and the second set of data to a predictive model, wherein the predictive model is configured to determine a complexity for the second mammographic exam data based on the case complexity index; receiving, from the trained predictive model, an estimated complexity for reading the second mammographic exam data; and displaying the estimated complexity.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additional aspects, features, and/or advantages of examples will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the disclosure.
Non-limiting and non-exhaustive examples are described with reference to the following figures.
Medical imaging has become a widely used tool for identifying and diagnosing abnormalities, such as cancers or other conditions, within the human body. Medical imaging processes such as mammography and tomosynthesis are particularly useful tools for imaging breasts to screen for, or diagnose, cancer or other lesions with the breasts. Tomosynthesis systems are mammography systems that allow high resolution breast imaging based on limited angle tomosynthesis. Tomosynthesis, generally, produces a plurality of X-ray images, each of discrete layers or slices of the breast, through the entire thickness thereof. In contrast to conventional two-dimensional (2D) mammography systems, a tomosynthesis system acquires a series of X-ray projection images, each projection image obtained at a different angular displacement as the X-ray source moves along a path, such as a circular arc, over the breast. In contrast to conventional computed tomography (CT), tomosynthesis is typically based on projection images obtained at limited angular displacements of the X-ray source around the breast. Tomosynthesis reduces or eliminates the problems caused by tissue overlap and structure noise present in 2D mammography imaging.
In modern breast care centers, the images produced using medical imaging are evaluated by healthcare professionals to determine the optimal breast care path for patients. Due to advances in medical imaging (especially radiological imaging), the accuracy and granularity of information in produced images continue to increase. For example, a larger number of computer-aided detection (CAD) marks identifying image features may be added to an image. Although useful for analytical accuracy, the increased number of identified image features may increase the difficult of reading the image, which increases the time necessary to read the image. Additionally, the advances in medical imaging have resulted in an exponential increase in the volume of data healthcare professionals must review. The coupling of increased image reading complexity and the exponential increase in available data causes a large variation in medical image reading times, which depend upon breast types, the presence of disease and abnormalities, breast density, the image reader's experience, the type of mammographic exam performed, etc. Due to the large variation in medical image reading times, it is often difficult for screening centers to distribute workloads optimally to available radiologists. As a result, many screening centers experience decreased productivity and increased costs.
To address such issues with suboptimal workload distributions, the present disclosure describes systems and methods for predicting reading time and/or reading complexity of a mammographic exam. In aspects, a first set of mammographic exam data relating to one or more 2D and/or 3D breast images and information relating to the readers (human or electronic) of the breast images may be collected from various data sources. Mammographic exam data, as used herein, may refer to information relating to breast image data (e.g., pixel image data and image header data), evaluation data for the breast image data (e.g., study open and close times, reader workload, and reading tools used), user profile-related data for a reader/evaluator of the breast image data (e.g., reader experience, reader expertise, etc.), preexisting patient data (e.g., patient history records/reports and previously collected patient image data), reader opinion data for the breast image data (e.g., reader estimations of reading times or reading complexity), biopsy data, annotations and/or labeled data, and the like. A reader, as used herein, may refer to medical or clinical professional who is trained to review a mammographic exam. Examples of data sources include, but are not limited to, image acquisition workstations, image review workstations, hospital information systems (HISs), patient record systems, reader profile systems, training data repositories, and test/training systems. The first set of mammographic exam data, which may include labeled and/or unlabeled training data, may be used as input to train one or more artificial intelligence (AI) models. A model, as used herein, may refer to a predictive or statistical utility or program that may be used to determine a probability distribution over one or more character sequences, classes, objects, result sets or events, and/or to predict a response value from one or more predictors. A model may be based on, or incorporate, one or more rule sets, machine learning (ML), a neural network, or the like.
In aspects, a second set of mammographic exam data may be collected from one or more of the various data sources described above and provided to the trained AI model. The second set of mammographic exam data may comprise data that is similar to, or the same as, the first set of mammographic exam data. In some examples, however, the second set of mammographic exam data may not include training data. Based on the second set of mammographic exam data, the trained AI model may produce one or more outputs. Example outputs include, but are not limited to, predicted/estimated reading times for one or more images in the second set of mammographic exam data, a complexity rating for reading an image in the second set of mammographic exam data, and time slot availabilities and/or assignments for one or more radiologists. The complexity rating may indicate the difficulty or complexity of reading a mammographic exam or images thereof. The difficulty or complexity of reading a mammographic exam may be based on factors, such breast type, breast density, number of CAD marks, etc. The complexity rating may infer or be correlated with a time for reading a mammographic exam. For instance, the complexity rating and the reading time for a mammographic exam may be related such that the reading time increases as the complexity rating increases. In some aspects, the one or more outputs of the trained AI model may be provided to one or more healthcare professionals and used to balance or optimize the workloads of available radiologists. The balancing/optimization of the workloads may be performed manually by a healthcare professional, or automatically by the trained AI model.
Accordingly, the present disclosure provides a plurality of technical benefits including, but not limited to: generating automated estimates of time required for reading a mammographic exam, generating automated classifications of mammographic exam complexity, leveraging image reader information (e.g., statistics, experience, and credentials) to estimate medical image reading times, automating optimized workload distribution, automated scheduling of reading of mammographic exams, training predictive models based on subjective reader profile factors and/or reader statistics, and training predictive models based on preexisting patient data.
As one example, the system 100 may comprise computing devices 102A, 102B, and 102C (collectively, “computing device(s) 102”), processing system 108, and network 106. One of skill in the art will appreciate that the scale of systems such as system 100 may vary and may include more or fewer components than those described in
Computing device(s) 102 may be configured to receive mammographic exam data relating to a healthcare patient and/or healthcare professional. The mammographic exam data may be received using one or more user interfaces (e.g., a graphical user interface (“GUI”), command line, menu driven interface, or data feed) or sensors (e.g., microphones, touch-based sensors, keyboards, pointing/selection tools, optical/magnetic scanners, or accelerometers) of computing device(s) 102. Examples of computing device(s) 102 include, but are not limited to, image acquisition systems (e.g., X-ray, ultrasound, and magnetic resonance imaging (MRI) systems), image review workstations, HIS devices, patient record devices, mobile healthcare devices (e.g., wearable devices, mobile phones, and tablets), and devices storing healthcare professional information. Computing device(s) 102 may store the mammographic exam data locally on computing device(s) 102 and/or remotely in one or more data storage locations, such as data stores 104A and 104B (collectively, “data store(s) 104”), via network 106. Computing device(s) 102 and data store(s) 104 may be located in one or more healthcare facilities, in a facility associated with a healthcare facility, or in the possession of a healthcare professional. In examples, the mammographic exam data may be provided to computing device(s) 102 using manual processes, automatic processes, or some combination thereof. For instance, a healthcare professional located at a healthcare facility may manually enter mammographic exam data into one or more computing devices. Alternately, device located remotely from the healthcare facility may automatically upload mammographic exam data to one or more computing devices of the healthcare facility. As a specific example, a computing device located at the residence of a healthcare professional may automatically upload mammographic exam data to a healthcare facility device as part of a daily synchronization process.
Processing system 108 may be configured to train and/or provide a ML model. In aspects, processing system 108 may have access to one or more sources of mammographic exam data, such as computing device(s) 102 and/or data source(s) 104, via network 106. A first set of mammographic exam data may be provided as input to processing system 108. Processing system 108 may use the first set of mammographic exam data to train one or more AI processing components. For example, processing system 108 may train an artificial neural network, a support vector machine (SVM), a linear reinforcement model, a random decision forest, or a similar ML technique. After the AI processing component has been trained, a second set of mammographic exam data may be provided as input to processing system 108. Based on the second set of mammographic exam data, processing system 108 may generate one or more outputs, such as estimated reading times for an image in the second set of mammographic exam data, a complexity rating or label for reading an image in the second set of mammographic exam data, one or more reader-specific estimated reading times, reading complexities, or reading labels, time slot availabilities and/or assignments for one or more radiologists, etc. The outputs may be provided (or made accessible) to other components of system 100, such as computing device(s) 102. In examples, the outputs may be evaluated by one or more healthcare professionals to determine study prioritization and/or optimization of workload distribution.
With respect to
In other examples, the information may be collected from a user via an interface, such as user interface 203. User interface 203 may be configured to collect user input from or relating to one or more healthcare professionals. Examples of the user input include identification information, performance statistics, current and/or historic workload information, user estimations of reading times or reading complexities for images or image types, and other user profile-related information. User interface 203 may comprise various user interface elements for collecting the user input and/or navigating or interacting with results provided by user interface 203. As one example, user interface 203 may comprise at least an image data list component, an image data review component, and a self-evaluation component. The image data list component may be configured to display a list of one or more image data file names or identifiers. The image data review component may be configured to provide a preview or view of the image data files provided by the image data list component. The self-evaluation component may be configured to present a set of options that enable a user to provide an estimated reading time or reading complexity for image data files provided by the image data list component. The estimated reading time may represent a reader's opinion of the amount of time required by the reader to read a selected mammographic exam. The estimated reading complexity may represent a reader's opinion of the reading complexity of a selected mammographic exam. The estimated reading complexity may be provided as a label (e.g., easy, medium, hard) or a numerical value (e.g., on a scale of 0-100). Similarly, the estimated reading time may be provided for the reader to select as a label (e.g., fast, medium, slow). Such subjective information, such as the reader's perception of the reading may be used to train the AI processing algorithms or models as described below. Similarly, the subjective information may be used as scores or weights in determining the reading time and/or complexity. For example, for low density, fatty breasts without concerning findings which are less stressful, the perception of reading time may be ‘fast’, although in reality a reader may have taken additional time to scroll through and assess the mammographic exam.
Processing engine 204 may be configured to process the received information. In aspects, the received information may be provided to processing engine 204. Processing engine 204 may apply one or more AI processing algorithms or models to the received information. For example, processing engine 204 may apply a supervised learning model for classification and regression analysis of the received information. In some aspects, the received information may be used by processing engine 204 (or an alternate component of input processing system 200) to train the AI processing algorithms or models. The trained AI processing algorithms or models may then be used to evaluate received information in order to determine correlations between the received information and the training data used to train the AI processing algorithms or models. The evaluation may include an analysis of various factors that may influence the reading time of an image. Such factors include, but are not limited to, the number of lesions (or abnormalities/concerning findings) detected in the image, the type of anomalies/findings identified by the image, the location of evaluation within the breast (e.g., superior, medial inferior or lateral), the symmetry between a patient's breasts, the number of image slices generated, breast density, breast thickness, breast area, breast tissue composition structure, breast tissue patterns, number of computer-aided detection (CAD) markers, type of image processing used (e.g., artifact reduction, de-noising, etc.), breast positioning parameters of an imaged breast, noise and contrast parameters, type or study performed (e.g., screening, diagnostic, etc.), number of exposures, types of compression paddles used, time of day of the reading, type of workstation and tools used for the reading (e.g., ease of pan/zoom, availability of “smart” hanging protocols, availability of easy annotation tools, availability of tracking, auto-mapping, and “smart” scrolling tools, etc.), reading tool usage (e.g., number of click events, scroll events, and zoom in/out events), user focus data (e.g., dwell time, eye gaze, and hover events), reader experience, reader specialization and training, reader age, reader reading time/complexity opinions or estimates, and reader proficiency.
In some aspects, the various factors analyzed by processing engine 204 may be manually selected or weighted according to one or more criteria, such as user preference. For example, user interface 203 may enable users to select a set of factors to be considered when analyzing received information. The user interface may also enable users to assign importance scores or weights to the set of factors or modify importance scores or weights previously assigned to the set of factors. For example, the self-evaluation component described above may enable a user to assign or modify scores or weights to the set of factors. Further, the self-evaluation component may enable the user to navigate or otherwise interact with set of factors. The scores or weights may indicate the perceived/determined importance of certain factors with respect to other factors. In other aspects, the various factors analyzed by processing engine 204 may be automatically selected or weighted using one or more AI or ML techniques. As one specific example, reader information for one or more readers may be provided to processing engine 204. The reader information may include at least reader experience, previous evaluation data for the reader (e.g., previous reading times), and reader opinions or estimates of reading time and/or reading complexity for one or more images or image types. Based on the provided reader information and/or previous evaluation results, processing engine 204 may determine that the previous reading times and/or reader estimates of reading times for clinical professionals having more experience and more advanced sensitivity and specificity in detecting malignant findings should be given more weight than the same information (e.g., previous reading times and/or reader estimates of reading times) for clinical professionals having less experience. As a result, processing engine 204 may assign higher weights/scores to analyzed factors associated with clinical professionals having more experience. Based on the evaluation of the various factors, processing engine 204 may identify one or more outputs or output categories.
Output engine 206 may be configured to create one or more outputs for the received information. In aspects, output engine 206 may access the outputs or output categories identified by processing engine 204. Based on the identified outputs or output categories, output engine 206 may create one or more outputs. As one example, output engine 206 may generate a predicted reading time for a mammographic exam in the received information based on an analysis by processing engine 204. The predicted reading time may represent an amount of reading time, a range of reading time (e.g., 15-20 minutes), or a reading time category (e.g., Low, Medium, High) that an “average” reader requires to read a mammographic exam. An “average” reader may be classified as a reader having one or more attributes within a specified attribute range. Alternately, the predicted reading time may represent a category or classification (e.g., fast, medium, or slow) indicating the amount of reading time that an average reader requires to read a mammographic exam. For instance, a “fast” category may correspond to reading times under 10 minutes, a “medium” category may correspond to reading times between 10 minutes and 20 minutes, and a “slow” category may correspond to reading times above 20 minutes.
As another example, output engine 206 may generate a predicted reading time for a mammographic exam in which that predicted reading time represents the amount of reading time, a range of reading time, or a reading time category that a specific reader requires to read a mammographic exam. The specific user may be a user currently logged into input processing system 200, a user who has previously logged into input processing system 200, a user selected by a user currently logged into the input processing system 200, a user selected by input processing system 200, or the like. For example, output engine 206 may generate a predicted reading time that is personalized to a user currently logged into input processing system 200. The predicted reading time for the user may be based on user-profile data and/or previous reading time data of the user, which may indicate that the user is a comparatively fast reader. As a result, the predicted reading time for the user may be lower than the predicted reading time for an average reader or a slow reader.
Alternately, output engine 206 may generate multiple predicted reading times. The multiple predicted reading times may correspond to individual readers or categories of readers. For instance, a first reading time may correspond to readers having less than five years of experience, a second reading time may correspond to readers having between five and ten years of experience, and a third reading time may correspond to readers having more than ten years of experience.
As yet another example, output engine 206 may generate a complexity rating for an image in the received information based on a set of factors or a complexity index. The complexity rating may represent the difficulty or complexity of reading a mammographic exam or images thereof for one or more readers. The difficulty or complexity of reading a mammographic exam may be based on objective and/or subjective factors. Examples of objective factors include, but are not limited to, the number and/or type of findings or CAD marks detected in an image, the breast evaluation location, and breast density and tissue composition. Examples of subjective factors include, but are not limited to, reader experience, reader knowledge, reader evaluation technique, and reader competency. In examples, the difficulty or complexity of reading a mammographic exam may increase based on the quantity of factors considered. For instance, an analysis of 3-5 factors may be less complex than an analysis of 10-15 factors. The difficulty or complexity of reading a mammographic exam may also increase based on the values of the factors considered. For instance, the complexity of reading a mammographic exam may increase as the number CAD marks increases, the number CAD marks detected in a specific portion of the breast increases, or the amount of dense breast tissue evaluated increases. The complexity of reading a mammographic exam may also effectively increase for inexperienced readers or less competent readers.
Alternately, output engine 206 may generate multiple complexity ratings. The multiple complexity ratings may correspond to individual readers or categories of readers. For instance, the complexity rating of “difficult” may be assigned to readers having less than five years of experience, and a complexity rating of “moderate” may be assigned to readers having more than five years of experience.
In aspects, the predicted reading time and/or the complexity value may be stored and associated with the mammographic exam associated with the patient. The mammographic exam (or elements thereof) may be provided to one or more destinations, such as a device, an application, or service. The destination(s) may enable a healthcare professional to view and interact with the mammographic exam (including the predicted reading time and/or the complexity value) via, for example, the user interface described above. As a specific example, a reading time and/or the complexity value that has been associated with a specific mammographic exam may be displayed to a reader in a patient worklist for the user. Alternately, the reading time and/or the complexity value may be displayed to a technologist performing the screening or diagnostic scans on the patient(s). In at least one aspect, output engine 206 may also cause one or more additional actions to be performed. For instance, based on the predicted reading time and/or the complexity value, output engine 206 may identify one or more clinical professionals having the expertise and available time to read the mammographic exam. Reading the mammographic exam, as used herein, may refer to various methods and approaches for reviewing and interpreting images and patient information. For example, one reading method may include a reader verifying that images are of satisfactory quality (e.g., no or minimal motion or blurring). If the images (or at least a portion thereof) are of satisfactory quality, the reader may evaluate the size and symmetry of the breasts. The reader may also compare the current images with previously collected images to identify changes between the images. The comparison may include viewing different image views of the breasts, such as mediolateral-oblique, mediolateral, cranial-caudal, etc. The viewings of the image views may be performed in various orders and viewing combinations. The reader may view a 2D image synthesized from two or more tomography images and investigate the CAD marks indicated by the tomography images. The reader may view various features in the images, such as calcifications, areas of skin thickening, features that are associated with types of cancers, speculated masses, etc. The reader may annotate one or more images based on specific areas of interest and determine a finding or a result. The finding or result may then be recorded in a report using standardizes methodology or categories, such as BI-RADS.
In aspects, identify one or more clinical professionals may be accomplished by output engine 206 by querying one or more HIS devices for reader information, such as reading statistics, reader availability, education/expertise, experience, etc. In aspects, a healthcare professional receiving the output of output engine 206 may use the output to balance or optimize the workloads of available clinical professionals. For example, a clinical professional may prefer reading complex mammography exam early in the day, which enables the clinical professional to leave the less complex and/or less time-consuming mammography exam readings for the end of the day. As such, the clinical professional may use the output of output engine 206 to arrange their workload accordingly. As another example, a clinical professional may only have a small time slot available on a particular day. As such, the clinical professional may use the output of output engine 206 to arrange their workload to maximize the number of mammography exam readings that the clinical professional may perform in the time slot.
Alternately, the output may be used to automate the balancing or optimization of the workloads of available clinical professionals. For instance, the output of output engine 206 may be provided to a workload management system/service that is configured to dynamically create/update clinical professional workloads. The workload management system/service may balance the workloads of two clinical professionals such that the first clinical professional is assigned ten mammography exam readings per day, each categorized as having “Fast” reading times, and the second clinical professional is assigned five mammography exam readings per day, each categorized as having “Slow” reading times. Despite the different number of mammography exam readings assigned to the first and second clinical professional, their respective workloads may require approximately the same amount of time to complete. Alternately, the workload management system/service may balance the workloads of two clinical professionals such that each clinical professional is assigned the same number and mix of complex mammography exam readings or such that the clinical professional having the most experience is assigned a proportionately higher number of complex and/or “Slow” reading time mammography exam readings.
Having described various systems that may be employed by the aspects disclosed herein, this disclosure will now describe one or more methods that may be performed by various aspects of the disclosure. In aspects, methods 300 and 400 may be executed by an example system, such as system 100 of
At operation 304, the first set of data is provided to a predictive model. In aspects, one or more portions of the first set of data may be provided to an evaluation component, such as processing engine 204. The evaluation component may be, comprise, or have access to one or more predictive models. The first set of data may be provided as input to a predictive model to train the predictive model to generate one or more outputs. Example outputs include estimated reading times for breast images, complexity ratings for reading breast images, identification of recommended image readers, and time slot availabilities of recommended image readers. In at least one aspect, the first set of data may be used by the evaluation component to generate or update a case complexity index. For example, the evaluation component may use an index creation model or algorithm to generate or update a case complexity index. The case complexity index may comprise a range of complexities associated with reading one or more images or image types. The case complexity index may be configured to provide one or more complexity ratings for an image based on a set of factors that may influence the time required to read the image. In examples, generating or updating a case complexity index may comprise using ML to map a complexity rating to the positive identification of one or more factors, factor values, or a range of factor values. For instance, a rating of “easy” may be mapped to a set of features in which breast density is low and the number of lesions identified in an image is two or fewer. In other examples, the generating or updating a case complexity index may comprise creating complexity rating categories based on scores associated with various factors or factor values. For instance, various factors/attributes associated with a mammographic exam reading may be scored using a scoring model or algorithm. The score for each factor/attribute may be aggregated and used to establish a range of values indicating one or more rating categories. In some aspects, a trained predictive model and/or a case complexity index may be provided to, and/or implemented by, one or more systems or devices.
At operation 306, a second set of data is collected. In aspects, a data collection component, such as data collection engine 202, may collect or receive a second set of data from one or more data sources. The second set of data may comprise at least a portion of data that is similar in type, category, and/or value to the first set of data. For example, the second set of data may comprise or relate to breast image data, image evaluation data, and/or image reader information. In some examples, however, the second set of data may not include labeled or unlabeled training data.
At operation 308, the second set of data is provided to a trained predictive model. In aspects, one or more portions of the second set of data may be provided to an evaluation component, such as processing engine 204. The evaluation component may provide the second set of data to a predictive model, such as the predictive model trained during operation 304. The trained predictive model may evaluate the second set of data to determine correlations between the second set of data and training data used to train the predictive model. For example, based on training data used to train a predictive model, the predictive model may determine that the reading time of breast images having a particular set of breast attributes (e.g., shape, density, lesions, etc.) varies based on the attributes of the image reader and the conditions under which the mammographic exam reading is performed. For instance, the predictive model may identify that images of breasts having an ACR Breast Imaging Reporting and Data System (BI-RADS) mammographic density (MD) of Type 1, and 0-2 lesions generally require image readers having 5 or more years of experience 10 minutes to read in the morning (when the reader is relatively rested), and 15 minutes to read in the evening (when the reader is relatively fatigued). The predictive model may further identify that images of breasts having the above attributes (e.g., MD Type 1, 0-2 lesions detected) generally require: image readers having less than 5 years of experience 25 minutes to read in the morning, and 30 minutes to read in the evening.
In some aspects, the evaluation component may evaluate the second set of data using the case complexity index described above. The evaluation may include identifying, organizing, and/or classifying one or more features or attributes of the second set of data. The identified features/attributes may be compared to, or evaluated against, the case complexity index using decision logic, such as a ML algorithm or a set of evaluation rules. As one example, a predictive model may determine that predicted reading times for breast images of a particular type or having a particular set of attributes are within the 85th percentile for reading time duration (e.g., indicating an increased reading duration as compared to other breast images of image types). Based on the determined percentile rank, the case complexity index may provide an indication that the determined/predicted complexity for such images. For instance, the case complexity index may provide a designation of “difficult” for any breast images determined to be within at least 80th percentile. As another example, at least a portion of the data used to determine the predicted reading time (e.g., breast image data, image evaluation data, and/or image reader information) may be evaluated using the case complexity index. The evaluation may include assigning values or scores to features in the data. A scoring engine or algorithm may be applied to the assigned values/scores to generate an aggregated mammographic exam reading score. For instance, data in the second set of data may be featurized and used to construct one or more feature vectors. A feature vector, as used herein, may refer to an n-dimensional vector of numerical features that represent one or more objects. The feature vector(s) may be applied to, or evaluated against, the case complexity index. Based on the feature vector values/scores, the case complexity index may provide a corresponding score or designation indicating the complexity or reading a particular breast image or type of breast image.
At operation 310, an estimated mammographic exam reading time is received. In aspects, the trained predictive model and/or the case complexity index may provide one or more outputs for the second set of data. As discussed in operation 304, the outputs may include estimated mammographic exam reading times, estimated complexity ratings for images to be read, recommendations or assignments of image readers, job/task scheduling dates/time, etc. The output may be provided to one or more HIS devices and/or healthcare professional devices. Based on the output, a reader and/or a reading session time may be manually or automatically assigned for the image(s) in the second set of data. In some aspects, the predictive model output, the case complexity index output, and/or the statistics and parameters of the resulting mammographic exam reading/study, may be provided as input to a predictive model (such as the predictive model described in operations 304 and 308) and/or a component implementing or maintaining the case complexity index. The input may be used to further train the predictive model and/or the case complexity index. As one example, based on the output of a predictive model, an image requiring an estimate reading time of 15 minutes may be assigned to a radiologist having 10 years of experience. The mammographic exam reading/study may actually take the radiologist 25 minutes. The estimate reading time, the actual reading time, and the parameters/conditions of the mammographic exam reading/study (e.g., reading tools used, time of day, radiologist information, etc.) may be provided to the predictive model. The predictive model may use the information to adjust future reading time estimates for the radiologist, or to reevaluate the reading times for images having similar attributes the image in the mammographic exam reading/study.
At operation 404, a predictive model is trained using the first set of data. In aspects, the first set of data may be provided to an evaluation component, such as processing engine 204. The evaluation component may be, comprise, or have access to one or more predictive models. The first set of data may be provided as input to a predictive model to train the predictive model to generate one or more outputs. Example outputs include estimated complexity ratings or a complexity system/component for reading or interpreting mammographic exam data. As a particular example, the first set of data may comprise a labeled or annotated breast image, a reported amount of time for reading the image, a reported or suggested complexity rating for reading the image, and profile information for the reader of the image. The first set of data may be provided to a predictive model. The predictive model may use one or more data correlation techniques to determine correlations between the reported/suggested complexity rating and the other factors/data points in the first set of data.
At operation 406, a case complexity index may be estimated. In aspects, a predictive model may use the first set of data to estimate or update a case complexity index. A case complexity index, as used herein, may refer to a data structure or component comprising mappings of complexity values or labels to data/factors that may influence the amount of time required to read or interpret a mammographic exam. Estimating or updating the case complexity index may comprise using an index creation algorithm, a data mapping utility, or data correlation algorithm. The case complexity index may be configured to provide one or more complexity ratings or labels for collected mammographic exam data based on a set of factors or data points associated with the collected mammographic exam data. In examples, the set of factors or data points may influence the time required to read the collected breast image. For instance, a rating of “2” on a scale of 1 to 5 (where “1” is very easy, “2” is easy, “3” is moderate, “4” is difficult, and “5” is very difficult) may be mapped to a set of features in which at least two of the following factors is satisfied: breast density is low, the number of lesions identified in an image is one or fewer, and a reader has more than 10 years of experience.
At operation 408, the case complexity index may be output. In aspects, an estimated/updated case complexity index may be output by a predictive model or the evaluation component. The case complexity index that is output may be a standalone executable file or utility. Alternatively, the output case complexity index may be integrated into a service, application, or system. As one example, the case complexity index (or an instance thereof) may be distributed to and integrated into a user interface of one or more workstations (e.g., image acquisition workstations, image review workstations, other HIS computing devices, etc.). The user interface many enable a healthcare professional to evaluate and/or modify the mappings, mapping logic, classifications, and/or category values of the case complexity index. The user interface may additionally enable a healthcare professional to assign a weighted value or importance to various factors evaluated by the case complexity index. For instance, a healthcare professional that is more interested in the number of lesions identified in a breast image than breast density, may assign a higher importance to the number of identified lesions. Assigning a higher importance may include applying a multiplier (such as ×1.25) to factors relating to the number of identified lesions, or setting a designation for a particular value range (e.g., mapping “0” lesions to “very easy,” “1-2” lesions to “easy,” etc.).
At operation 410, a second set of data is collected. In aspects, a data collection component, such as data collection engine 202, may collect or receive a second set of data from one or more data sources. The second set of data may comprise at least a portion of data that is similar in type, category, and/or value to the first set of data. For example, the second set of data may comprise or relate to breast image data, image evaluation data, and/or image reader information. In some examples, however, the second set of data may not include labeled or unlabeled training data.
At operation 412, a predictive model is trained using the second set of data. In aspects, the second set of data and/or data relating to the case complexity index may be provided to an evaluation component, such as processing engine 204. The evaluation component may use the provided information to train a predictive model. In some examples, the predictive model may be the predictive model trained at operation 404. In other examples, a new or an alternate predictive model may be trained. The predictive model may be trained to generate one or more outputs. Example outputs include estimated mammographic exam reading times, estimated complexity ratings for images to be read, recommendations or assignments of image readers, job/task scheduling dates/time, etc. For instance, the predictive model may determine that the reading time of breast images having a particular set of breast attributes (e.g., shape, density, lesions, etc.) varies based on the attributes of the image reader and the conditions under which the mammographic exam reading is performed. Based on the determination, the predictive model may use one or more data correlation techniques to determine correlations between known reading times for images and the factors/data points corresponding to the known reading times.
At operation 414, a reading time may be estimated. In aspects, the trained predictive model may use the second set of data to generate one or more outputs. For example, the predictive model may evaluate the second set of data to determine correlations between the data used to train the predictive model and the second set of data. Based on the determined correlations, one or more reading times for a breast image associated with the second set of data may be estimated. For instance, based on identifying that an imaged breast has a BI-RADS mammographic density of Type 2 and no lesions have been identified in the breast image, the predictive model may estimate an mammographic exam reading time of 20 minutes for image readers having less than five years of experience, and an mammographic exam reading time of 10 minutes for image readers having five or more years of experience. In another example, based on the correlations determined by the predictive model, a complexity rating/system may alternately or additionally be estimated/generated. For instance, based on a BI-RADS mammographic density classification, a lesion count, and/or an estimated reading time, the predictive model may estimate an reading complexity for a mammographic exam. Alternately, the predictive model may use the case complexity index generated or updated at operation 406 to estimate a reading complexity. For instance, the predictive model may provide at least a portion of second set of data to the case complexity index. In response, the case complexity index may provide a complexity rating for reading a mammographic exam to the predictive model.
At operation 416, an estimated reading time may be output. In aspects, one or more estimated mammographic exam reading times may be output by the predictive model. The estimated reading time(s) may correspond to an individual image reader, multiple image readers, or one or more categories of readers. For example, an estimated reading time may represent the amount of time an “average” image reader requires to read a mammographic exam. Alternately, each estimated reading time may represent the amount of time that an “average” image reader in a particular category of image readers requires to read a mammographic exam. In some aspects, the predicted reading time may be provided to a healthcare professional via, for example, the user interface described above. Providing the predicted reading time may cause one or more additional actions to be performed. For instance, based on the predicted reading time, one or more radiologists having the expertise and available time to read a mammographic exam may be identified and/or notified. The identification/notification may be accomplished by querying one or more HIS devices for radiologist information, such as reading statistics, availability, education/expertise, experience, etc.
Operating environment 600 typically includes at least some form of computer readable media. Computer readable media can be any available media that can be accessed by processing unit 602 or other devices comprising the operating environment. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium which can be used to store the desired information. Computer storage media does not include communication media.
Communication media embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, microwave, and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.
The operating environment 600 may be a single computer operating in a networked environment using logical connections to one or more remote computers. The remote computer may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above as well as others not so mentioned. The logical connections may include any method supported by available communications media. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.
The embodiments described herein may be employed using software, hardware, or a combination of software and hardware to implement and perform the systems and methods disclosed herein. Although specific devices have been recited throughout the disclosure as performing specific functions, one of skill in the art will appreciate that these devices are provided for illustrative purposes, and other devices may be employed to perform the functionality disclosed herein without departing from the scope of the disclosure.
This disclosure describes some embodiments of the present technology with reference to the accompanying drawings, in which only some of the possible embodiments were shown. Other aspects may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments were provided so that this disclosure was thorough and complete and fully conveyed the scope of the possible embodiments to those skilled in the art.
Although specific embodiments are described herein, the scope of the technology is not limited to those specific embodiments. One skilled in the art will recognize other embodiments or improvements that are within the scope and spirit of the present technology. Therefore, the specific structure, acts, or media are disclosed only as illustrative embodiments. The scope of the technology is defined by the following claims and any equivalents therein.
This application is a continuation of application Ser. No. 17/033,372, filed Sep. 25, 2020, which claims priority to provisional application Ser. No. 62/907,257, filed Sep. 27, 2019, entitled “A1 SYSTEM FOR PREDICTING READING TIME AND READING COMPLEXITY FOR REVIEWING 2D/3D BREAST IMAGES,” which applications are incorporated herein by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 62907257 | Sep 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17033372 | Sep 2020 | US |
| Child | 18319727 | US |
