Patent Application
20160085943
Clinical trials (also called “clinical studies”) are generally very expensive to undertake and they often take a long time to complete. During the course of a clinical trial, many steps may be taken to ensure optimal execution of the trial and maintain high quality of the data collected. These steps may be applied to various aspects of the trial data, trial participants, and trial execution at the level of trial sites, which constitute the main operating units in a clinical trial. One element of site (and study) monitoring is to monitor the progress of a trial, which, up until now, has not been adequately performed.
Where considered appropriate, reference numerals may be repeated among the drawings to indicate corresponding or analogous elements. Moreover, some of the blocks depicted in the drawings may be combined into a single function.
In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the invention. However, it will be understood by those of ordinary skill in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the present invention.
Embodiments of the present invention may be used to monitor the progress of clinical trials, but the invention is not limited to such embodiments. In monitoring the progress of a clinical trial, one may examine a variety of data elements—a data point, a data record, or a data page. In a clinical trial that records information using, for example, case report forms (“CRFs”), whether manual or electronic (“eCRF”), a data point (or “datapoint”) may be a field on such a form, a data record may be a number of fields on the form, and a data page (or “datapage”) may be the form itself. A datapage may also imply a composite data element in which history and statuses of individual (“lower”) data elements may be combined into a bigger picture. Any of these types of data elements may be monitored; for ease of understanding, this specification will use “datapage” to mean any or all of these data elements, unless a more specific term is called for.
Site and study monitoring have typically been performed with significant time lags and with a fairly coarse view of the data collection and management. Traditional data monitoring tools may utilize only the current status of data elements (sometimes called “object status”) and may not include input from historical changes in the data collection process. That approach may provide information about the present status of a study, but may not allow adequate insight into the events that preceded the current state. In contrast, aspects of the present invention allow the monitoring of a clinical trial by viewing the current and prior statuses of various data elements.
One way of viewing the progress of a clinical trial is in terms of states in the lifetime of a datapage, as shown in the schematic diagram of
The data on the datapage may then be frozen, such as when a patient visit is concluded or when a datapage is completed. After freezing the data, the datapage or a group of datapages may be locked by a study coordinator or other high-level study participant with such rights. Locking typically occurs after data have been reviewed, queries have been resolved, and issues have been addressed, and the database may be ready to undergo statistical analysis for submission to a regulatory agency.
The labels used in
The typical progression in the life of a datapage is to go forward (i.e., to the right in
For datapages that have already been frozen, it may be possible to go backward 43 to the verified state or 42 to the data entry state, possibly if some error is discovered in the taking of or recording of data. Such “unfreezing” may be performed by personnel with the correct privileges, e.g., a principal investigator may not unfreeze a datapage, but a study coordinator may do so. Similarly, for datapages that have already been locked, it may be possible to go backward 54 to the frozen state, 53 to the verified state, or 52 to the data entry state. Such “unlocking” may be performed by personnel with even higher privileges than those who may unfreeze the data, such as a study designer or a study coordinator.
All of the transitions from one state to another, both forward and backward, may be captured in an “audit trail,” which is a trail of state changes.
The state of a datapage may be derived from the states of some or all of the datapoints contained on the page. For example a datapage may be verified at time point t when all datapoints in the datapage that require verification are verified (at time t). (A similar derivation may be made for frozen and locked datapages.) This process is called “status rollup.” Thus, equivalent states for datapages may be defined even though they may or may not follow the same transition patterns as datapoints (e.g., a datapage could be signed before it is verified). Therefore, the verified (or frozen or locked) state of one datapage can change between being verified (or frozen or locked) and unverified (or unfrozen or unlocked) if the status of its datapoints is being changed.
Based on these states, progress curves may be calculated and plotted. For every given state and every given site (or entire trial), a progress curve at any moment t during the trial may be defined as the number of datapages in that state at moment t. For example,
There are several ways to calculate the progress curves. One way is to calculate and store in real-time all datapage status rollups and changes at any datapoint state change. Another way is to compute datapage states periodically at regular time intervals (e.g. daily) and then calculate and store the resulting progress curves. A third way is to generate progress curves over time by reconstructing datapage statistics historically from a historical trail of datapoint state changes (e.g., audit trails). This last method has the benefit of being computationally feasible, since it does not require continuous/real-time capture of datapage states and does not limit the time resolution at which state changes may be captured.
Information and data may be transmitted from the clinical trial databases to block 320 via a network 310, which may be the Internet, an intranet, or another public or private network. Access to network 310 may be wireless or wired, or a combination, and may be short-range or longer range, via satellite, telephone network, cellular network, Wi-Fi, over other public or private networks, via Ethernet, or over local, wide, or metropolitan area networks (i.e., LANs, WANs, or MANs).
Block 320 may be a computer, having a processor and memory, or a series of computers, processors, and memory that may themselves be connected via a local or wide-area network. Block 320 may include data collection module 322, historical statuses reconstruction module 324, and progress curve generator 326, along with associated databases—statuses/audits database 323, tracking data database 325, and progress curve database 327. Data collection module 322 collects data from one or more of the CT databases 301-309. This may include digestion, cleaning, and aggregation of some or all of the data in the databases. This module may extract, determine, or calculate current statuses and historical audits of clinical trials, and store that information in current statuses, historical audits database 323. Historical statuses reconstruction module 324 may then use the current statuses and historical audits to reconstruct the history of the clinical trial at issue and generate aggregated tracking data, which are stored in aggregated tracking data database 325. Progress curve generator 326 may then use the aggregated tracking data to generate progress curves, which may be stored in progress curve database 327.
Once the progress curves are generated, block 320 may transmit the progress curves to progress tracking service 330, which may also comprise a computer, processor, and memory. Progress tracking service 330 may comprise a web server that transmits and receives requests to and from visualization and metrics applications/computers 350, 360. Such transmitting and receiving may be done via network 340, which operates in the same manner and with the same variations as network 310.
Visualization module 350 may present progress curves on a user's computer, allowing a user to see the curves and manipulate how the data may appear. Users may be able to view the curves on their laptop or desktop computers or on their smartphones or tablets. Some examples of visualization are shown in
Metrics module 360 may present data and curves on a user's computer, which may be the same computer as that used to visualize the curves or may be a different computer. Metrics module 360 may allow a user to choose which metrics to calculate and/or display based on the curves stored in progress curve database 327. This module may provide recommendations and/or status of the clinical trial, and may provide alerts if a trial is not proceeding according to plan. Some examples of metrics are shown in FIGS. 8 and 9A-9D, and are discussed further in the flowcharts in
The operation of some of the modules in
Input 362 may include a list of datapoints with their respective attributes or properties as just described. Input 364 may include a sequence of historical events or audits for the datapoints, which may include the historical audits stored in statuses/audits database 323. Input 366 may include a list of datapages accompanied by a list of datapoints associated with each datapage. Input 368 may include the datapage state for which the progress curve is to be generated, which may include the current statuses stored in statuses/audits database 323.
The blocks shown in
In operation 415, pairs of values may be built for each consecutive sequence of events for a datapoint. To do this, call the first audit event “Initial_DP_Action.” Then, for each subsequent event ET, generate or build a pair of values {DP_Action (ET), DP_Action (ET-1)}. In operation 420, DP_Action (ET) values from operation 415 where the DP_Action values differ (i.e., DP_Action (ET) # DP_Action (ET-1)) are taken and then summed to generate a variable called “SUM_DP_Action.”
Table 1 shows an example of a sequence of events for a single datapoint, with an indication of the events that include a transition to or from the evaluated state (which is verification in this example):
SUM_DP_Action sums the DP_Action values from the time points in which direction the change occurred relative to the first DP_Action (or “Initial_DP_Action”) (i.e., the entries with a “Y” in the Change column, indicating a transition from 1 to −1 or from −1 to 1). Combining SUM_DP_Action with the initial action (Initial_DP_Action) determines the current state and processes all time points in which there was a state change (i.e., all time points with “Y” entries in the Change column).
In operation 425, a state is assigned to each datapoint (i.e., a state is determined for each datapoint) based upon Table 2:
For the data in Table 1, SUM_DP_Action=1, Initial_DP_Action=1, so the Assigned State=1. The sequence of events as shown in Table 1 pertains to one datapoint, so the process is applied to each datapoint separately to determine its state and the course of past events. Then, in operation 430, the datapage state can be set to 1 if all datapoints in the datapage are in state 1, otherwise the datapage state remains 0.
By considering events only up to a time point T, the state at time T may effectively be computed for the datapoints and thus the datapages. This can be done at any time T or later. This method allows for arbitrary time points T, so this method allows state activities to be collapsed to the last state activity, e.g., daily, weekly, monthly, etc. This is one of the aspects of aggregating the tracking data depicted in
Using realistic clinical trial data, generating progress curves that track state changes on short time scales may introduce unnecessary detail that may not be usable or actionable. States may frequently fluctuate in the range of a few hours to a day, but these fluctuations typically have no effect on the overall progress of data management. With this in mind, embodiments of the invention include a method to generate progress curves that may replace the “true” curve with two proxy progress curves called “First State” and “Last State.” First State captures the earliest time point tF at which all datapoints in a given datapage existed in the given state (verified, locked, etc.) at or before time tF, and Last State captures the latest time point tL at which the datapage state changes to a given state. Examples of such progress curves are “First Verified,” “Last Verified,” “First Locked,” “Last Frozen,” etc.
The First State and Last State progress curves can be computed as described in the flowchart in
Similarly, the Last State progress curve can be computed as described in operations 515 and 520. In operation 515, for each datapoint that is to be transitioned to the state the last time LT when the datapoint i transitioned to that state is calculated as LTi (so LTi=tL). If datapoint i never transitioned to the state, then LTi=N/A. In operation 520, if all datapoints within the datapage that are to be transitioned to the state have transitioned to that state, the maximum of all times LT for all those datapoints is calculated: if all LTi≠N/A, then LT(datapage)=max(LTi).
In addition to capturing trends in data acquisition and quality assurance, progress curves as described in this invention can be also used to measure trends that inform or alert users on patterns or events of interest.
Individual progress curves can be characterized individually or with respect to averaged progress curves from multiple trials (e.g., industry standard curves based on similar trials in size, therapeutic area, phase, etc.), but also comparatively to other progress curves. Two ways in which progress curves can be compared are (1) “completion” between the curves, i.e., the degree to which one curve has approached another at various time points; and (2) “delay,” i.e., the time point at which one progress curve will reach the same level as another. When measured systematically over time, these metrics may also be indicative of completion and/or delay trends. In addition to computing these metrics for various curve pairs, they may also be computed for subsets of data in a trial. For example, computing such metrics for each clinical site or a group of sites may be used to monitor the progress of a clinical trial in which resources may be optimized or refocused, or, at the very least, the metrics may provide a comparative baseline for the trial or the sites.
Reference is now made to
One way to compute delay or completion trends is to sample the progress curves at regular time intervals or regular number of datapage intervals. For example, for completion trends, the progress curves may be sampled weekly and form a vector of values that captures the difference between the curves. Similarly, for delay, the progress curves may be sampled at different points in accumulation of datapages (e.g., every 100 datapages) or at regular time intervals on one of the progress curves (e.g., weekly on datapage creation).
The progress of a clinical trial may be monitored using the progress curves and the metrics (completion and delay) discussed above.
In operation 1015, the user may select a profile, which is a sequence of target values over time. Examples of profiles may be historical, manual, and a combination of the two. A historical profile uses data taken from completed and ongoing trials and takes into account trial type, number of subjects, therapeutic area, keywords, trial phase, etc. A manual profile may be created by the user, who can provide criteria against which performance is monitored. One example is setting a target value at each time point in the trial, e.g., 75% datapages verified after two months into the trial, 85% verified after four months into the trial, and then 95% through the end of the trial. A combination profile may also be created by the user, but is based on a historical profile whose parameters are then adjusted by the user according to the needs of the trial. Table 3 shows one example of a profile based on historical data:
After selecting the curve pairs, the metric, and a profile, in operation 1020, the system may build or generate a collection of progress curve pairs and metrics to monitor against the provided profiles.
Besides the operations shown in
One benefit of the present invention is that the progress of a trial may be monitored without needing to monitor individual patients and it may be done remotely. Poor performance may be indicated by large delays or completion differences at one site compared to another, or by not meeting expected profile targets. By being able to identify progress trends on a site basis, a trial basis, or other parametrical basis, site monitors may be able to identify poor-performing sites or slow sites and concentrate monitoring on those sites, with the possible goal of finishing the trial more quickly and therefore saving money for the sponsor or contract research organization (CRO). Such monitoring may include calling the site and asking why there are deviations or sending people to the site to verify the data or the collection processes.
The present invention differs from other systems that monitor progress. For example, those systems may have significant time lags and may not provide a detailed view of the data collection and management. Those systems may also look at only the current status of data and ignore historical changes in the data collection process.
Aspects of the present invention may be embodied in the form of a system, a computer program product, or a method. Similarly, aspects of the present invention may be embodied as hardware, software or a combination of both. Aspects of the present invention may be embodied as a computer program product saved on one or more computer-readable media in the form of computer-readable program code embodied thereon.
For example, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, an electronic, optical, magnetic, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.
A computer-readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Computer program code in embodiments of the present invention may be written in any suitable programming language. The program code may execute on a single computer, or on a plurality of computers. The computer may include a processing unit in communication with a computer-usable medium, wherein the computer-usable medium contains a set of instructions, and wherein the processing unit is designed to carry out the set of instructions.
The above discussion is meant to be illustrative of the principles and various embodiments of the present invention. Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.
