Patent Application
20190006026
This application relates to electronic systems and associated methods for assessing information relating to experimental drugs or the like, to provide indications of the suitability of the information to support regulatory approval or other useful information.
The process of developing a new drug, medical treatment or device typically takes many years, and costs many millions of dollars. In many cases, particularly for new drugs, a regulatory body must approve the drug before it is allowed to be sold on the market. The regulatory body typically requires the drug to pass a number of tests, and otherwise be proven to be safe and efficacious for treatment of one or more conditions. Examples of regulatory bodies that oversee the introduction of new drugs include the United States Food and Drug Administration (“FDA”), the European Medicines Agency (“EMEA”), Health Canada, and so on. Furthermore, bodies such as the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (“ICH”) may promote additional guidelines for new drug testing or approval. Each regulatory body may have its own requirements to obtain approval of any given drug.
A large part of the cost of obtaining approval typically lies in the testing that is performed to satisfy the regulatory body's approval requirements. A single study can cost millions of dollars and take years to complete. Such tests may be in vitro and/or in vivo, and can involve a number of different phases of clinical trials. Even where a drug is already known and approved, the regulatory body may require further testing or approval procedures to permit alternative formulations, uses or delivery methods of the drug, or to allow entities other than the original sponsor to market the drug.
As a consequence of the many regulatory requirements and the great complexity of the supporting information (i.e., the testing and other data that are required to support approval), the drug approval process has become notorious for being a very difficult, expensive, and time-consuming process. Furthermore, interpretations of the regulations and supporting information can vary, resulting in frequent disputes between the drug sponsor and the FDA or other regulatory body reviewing the drug or other new product. Thus, drug sponsors typically make some effort to evaluate the completeness of the supporting information before submitting it to the regulatory body, to help ensure that all of the relevant requirements are satisfied. Identifying errors or missing information before submitting the drug application can safe a significant amount of time, and may reduce the cost of any additional tests that may be required.
In the past, electronic systems have been used to attempt to track information provided in support of a drug approval effort. For example simple spreadsheets and databases of checklists have been used to track whether drug study information conforms with various regulatory requirements. For example, checklists in spreadsheet form have been used to track whether the supporting information satisfies or does not satisfy certain regulatory criteria. However, such systems and procedures could be cumbersome to use, and generally required exceptional user skill to ensure that all of the appropriate checklists were considered (and inappropriate checklists were not considered). Such systems also merely indicated whether a particular aspect of the supporting information was defective, and were unable to further analyze this output to provide a realistic assessment as to whether the defect could or would affect the likelihood of an application being accepted. Thus, such systems were unreliable to the extent that persons reviewing the study information could make erroneous subjective determinations of which checklists should be used, and whether any deficiencies were sufficient to jeopardize the filing. Such systems also were unable to provide input about how to remedy any deficiencies that might be found. Such systems also were unable to provide real-time access to instructions relevant to answer the many questions that might be asked of a new drug or product, and were not configured to help train and improve the user, or to help ensure consistent interpretation of the supporting information and assessment of the new drug application. Such systems also had other deficiencies, as will be appreciated by persons of ordinary skill in the art.
The inventions are described herein by reference to exemplary embodiments. These embodiments are intended to address various problems with prior methods and systems for seeking to obtain regulatory approval for drugs, medical devices, and the like. However, the invention is not limited to solving any or all of the deficiencies described herein, and it will be appreciated that other advantages and uses of the inventions will be appreciated through practice of the invention.
In one aspect, there is provided a computer-implemented method for evaluating drug study information to determine suitability of the drug study information for acceptance by a regulatory authority. The method includes executing with a processor the steps of: receiving, from at least one user interface, an identification of a drug study associated with the drug study information; receiving, from the at least one user interface, a selection of a study type for the drug study, the study type being selected from a plurality of predetermined study types maintained in a memory accessible by the processor; receiving, from the at least one user interface, an indication whether the drug study is either (i) a nonclinical study that is intended to be a Good Laboratory Practice study, or (ii) a clinical study that is intended to support the efficacy indication of a drug that is the subject of the drug study information; and associating at least one checklist with the drug study, the at least one checklist being selected from a plurality of predetermined checklists maintained in the memory based on the identity of the selected study type. The at least one checklist includes a plurality of elements, each element having a respective identification of one or more features to be identified in the drug study information, and a respective one or more queries associated with the respective one or more features to be identified in the drug study information. The one or more queries include a query to select a deficiency type from a predetermined list of deficiency types. The method also includes the steps of: presenting the plurality of elements to the at least one user interface; receiving, from the at least one user interface, a respective reply to each of the plurality of elements; and assigning, by the processor, a respective criticality value to each respective reply comprising a selection of a respective deficiency type from the predetermined list of deficiency types. The step of assigning a criticality value includes assigning a high respective criticality value if the drug study is either (i) a nonclinical study that is intended to be a Good Laboratory Practice study, or (ii) a clinical study that is intended to support the efficacy indication of a drug that is the subject of the drug study information, or assigning a moderate respective criticality value if the drug study is not (i) a nonclinical study that is intended to be a Good Laboratory Practice study, and not (ii) a clinical study that is intended to support the efficacy indication of a drug that is the subject of the drug study information.
A second aspect may modify the first aspect by further including the step of adding one or more additional elements to the plurality of elements if the drug study is either (i) a nonclinical study that is intended to be a Good Laboratory Practice study, or (ii) a clinical study that is intended to support the efficacy indication of a drug that is the subject of the drug study information.
A third aspect may modify the second aspect by providing the one or more additional elements as one or more additional features to be identified in the drug study information. The one or more additional features may be aspects of (i) a bioanalytical method validation report, (ii) an analytical method validation report, or (iii) a biological sample analysis report.
A fourth aspect may modify the second or third aspect by having the step of associating at least one additional checklist with drug study be performed automatically by the processor.
A fifth aspect may modify the second or third aspect by having the step of associating at least one additional checklist with drug study be performed by receiving a selection of the at least one additional checklist from the at least one user interface.
A sixth aspect may modify any of the previous aspects by having the predetermined list of deficiency types include a high risk group of deficiency types representing deficiencies that affect the acceptability of the drug study information and a low risk group of deficiency types representing deficiencies that do not affect the acceptability of the drug study information.
A seventh aspect may modify the sixth aspect by having the step of assigning, by the processor, a respective criticality value to each respective reply comprising a selection of a respective deficiency type from the predetermined list of deficiency types further comprise assigning a low respective criticality value if the respective reply comprises a respective deficiency type selected from the low risk group of deficiency types, regardless of whether the drug study is either (i) a nonclinical study that is intended to be a Good Laboratory Practice study, or (ii) a clinical study that is intended to support the efficacy indication of a drug that is the subject of the drug study information.
An eighth aspect may modify any of the preceding aspects by having at least one of the plurality of elements include a trigger element having a question regarding the content of the drug study information, and an associated dynamic checklist having one or more additional elements regarding the drug study information; and having the method further include adding the one or more additional elements to the plurality of elements upon receiving, from the user interface, a predetermined reply to the question regarding the content of the drug study information.
A ninth aspect may modify the eighth aspect by having the question regarding the content of the drug study information be a question whether the drug study information includes kinetic or blood level data.
A tenth aspect may modify the eighth aspect by having the question regarding the content of the drug study information be a question whether the drug study information includes metabolism and transporter drug interaction-related data.
An eleventh aspect may modify the eighth aspect by having the question regarding the content of the drug study information be a question whether the drug study information includes toxicology-related data.
A twelfth aspect may modify any of the eighth through eleventh aspects by having the step of adding the one or more additional elements to the plurality of elements be performed by the processor automatically associating one or more additional checklists to the drug study.
A thirteenth aspect may modify any of the eighth through eleventh aspects by having the step of adding the one or more additional elements to the plurality of elements be performed by: presenting a list of one or more additional checklists to the at least one user terminal; receiving a selection of at least one of the one or more additional checklists from the at least one user terminal; and associating the selected at least one additional checklist with the drug study.
A fourteenth aspect may modify the thirteenth aspect by having the step of presenting a list of one or more additional checklists to the at least one user terminal include presenting a list of additional drug studies to which the one or more additional checklists were previously associated.
A fifteenth aspect may modify the thirteenth or fourteenth aspect by further including the steps of receiving a selection of one or more additional drug studies to which to add at least one of the one or more additional checklists; and associating the selected at least one additional checklist with the one or more additional drug studies.
A sixteenth aspect may modify any of the preceding aspects by having the one or more queries further include: a review status indicator to indicate whether the drug study information has been evaluated to respond to the respective element, an applicability indicator to indicate whether the element is applicable to the drug study information, and an acceptability indicator to indicate whether the drug study element contains the one or more features to the be identified in the drug study information.
A seventeenth aspect may modify any of the preceding aspects by having at least one of the elements include a regulatory source for information regarding the one or more features to be identified in the drug study information.
An eighteenth aspect may modify the seventeenth aspect by further including the step of presenting, to the at least one user interface, a hyperlink to the regulatory source.
A nineteenth aspect may modify any of the preceding aspects by having at least one of the deficiency types from the predetermined list of deficiency types include an associated record indicating a corrective action to take to overcome the deficiency type.
A twentieth aspect may modify any of the preceding aspects by further including the step of compiling a list of the respective criticality values for each of the one or more checklists associated with the drug study, and indicating the likelihood that the drug study information will be accepted by the regulatory authority.
A twenty-first aspect may modify any of the preceding aspects by having at least one of the one or more queries for at least one of the plurality of elements include the question of whether the drug study is either (i) a nonclinical study that is intended to be a Good Laboratory Practice study, or (ii) a clinical study that is intended to support the efficacy indication of a drug that is the subject of the drug study information.
A twenty-second aspect may modify any of the preceding aspects by repeating the steps of presenting the elements to the user interface, receiving replies for each element, and assigning a criticality value to each reply that includes a deficiency type selected from the list of predetermined deficiency types.
A twenty-third aspect may modify the twenty-second aspect by further including the step of identifying any differences in the respective criticality value of the first reply and the respective criticality value of each subsequent reply to each element.
A twenty-fourth aspect may modify the twenty-third aspect by further including the step of resolving the differences and providing a final respective criticality value of each respective reply.
A twenty-fifth aspect may modify any of the preceding aspects by further including the steps of (a) presenting, the plurality of elements to the at least one user interface during multiple successive review stages, each review stage being based on a respective modified version of the drug study information; (b) receiving, from the at least one user interface, a respective reply to each of the plurality of elements during each of the multiple successive review stages; (c) assigning, by the processor, during each of the multiple successive review stages, a respective criticality value to each respective reply comprising a selection of a respective deficiency type from the predetermined list of deficiency types by: assigning a high respective criticality value if the drug study is either (i) a nonclinical study that is intended to be a Good Laboratory Practice study, or (ii) a clinical study that is intended to support the efficacy indication of a drug that is the subject of the drug study information, or assigning a moderate respective criticality value if the drug study is not (i) a nonclinical study that is intended to be a Good Laboratory Practice study, and not (ii) a clinical study that is intended to support the efficacy indication of a drug that is the subject of the drug study information; and repeating steps (a)-(c) until none of the respective replies is assigned a high respective criticality value.
A twenty-sixth aspect may modify any of the preceding aspects by having the step of associating at least one checklist with the drug study be performed automatically by the processor.
A twenty-seventh aspect may modify any of the preceding aspects by having the step of associating at least one checklist with the drug study be performed by receiving an association from the at least one user interface.
Other aspects of the invention will be apparent from the disclosures provided herein.
Exemplary embodiments are described herein by reference to the attached drawings. The attached drawings illustrate examples of embodiments encompassed within the scope of the present disclosure, and, therefore, are not to be considered limiting. To facilitate understanding, like reference numerals have been used, where possible, to designate like elements common to the figures.
The present disclosure provides examples of a number of inventions that, for convenience, are described collectively in the context of a system for evaluating information relating to drug and medical device regulatory approval. It will be appreciated that the various inventions described herein may be used separately, or in various combinations, and the scope of the claimed invention is not intended to be limited to the collective features and functions of the systems described herein.
The described embodiments generally relate to systems and methods for evaluating documentation (including its content and format) to determine whether the documentation is sufficient and appropriate to satisfy relevant regulatory requirements, such as regulatory requirements associated with applications for approval of drugs or other medical devices or products. The system may be configured for use in preparing and reviewing regulatory filings for drugs, products or devices, and may be used at various stages of the approval process. For example, the system may be configured to support Investigational New Drug Applications (“INDs”), Investigational Device Exemptions (“IDEs”), 510(k) filings, New Drug Applications (“NDAs”), Abbreviated New Drug Applications (“ANDAs”), assessments at intermediate stages of drug development, and submissions of completed applications to health authorities for marketing approval.
More specifically, the embodiments described herein relate to systems and methods for evaluating studies and other information (collectively, “supporting information”) that will be provided to a Health Authority to support the marketing approval of new drugs, devices, and/or the like or the request to study the proposed product that is being developed for commercialization. At its most basic level, the system can be used, in one configuration, to analyze the information to determine whether it meets the regulatory requirements, and automatically characterize any deficiencies or potential approval concerns that might be found. The system may be configured to address requirements that are specific to one or more health authorities, such as the FDA, EMEA, Health Canada, etc. In performing this function, the systems and methods may be used to effectively prescreen applications for potential approval by a regulatory authority, and identify any issues that are likely to cause or that may cause a denial of approval. The system also may generate documents that identify which requirements are not met, and provide recommended corrective actions.
The clients and servers generally are provided in the form of computers that include specialized software or hardware configured to perform one or more of the exemplary methods described herein. As used herein, the term “computer” refers to any device that is capable of processing a signal or other information. Examples of computers include, without limitation, a personal computer, a portable computer, a handheld computer, a cellular phone, a smart phone, a digital tablet, a laptop computer, a netbook, a smart TV, a device configured to connect to the internet, an Internet appliance, a Personal Data Assistant (PDA), an application-specific integrated circuit (ASIC), a programmable logic array (PLA), a microcontroller, a digital logic controller, a digital signal processor (DSP), or the like, or may generally include a suitably configured processor and memory that are operatively associated to work together, as known in the art. Various input devices and displays may be used with the computer, also as known in the art. A computer also may include software in the form of programmable code, micro code, or firmware or other hardware embedded logic. A computer may include multiple processors that operate in parallel, and the processing performed by a computer may be distributed among multiple separate devices. The term computer encompasses all such devices and equivalents thereof, when configured to perform in accordance with the disclosed embodiments. Each of the client(s) and server(s) preferably is capable of transmitting data through the network 160 either in real time (i.e., where data is sent continuously back and forth), or in batched mode (i.e., where data is sent in bulk after an amount of work is performed).
The network 160 may comprise a global computer network (e.g., the Internet), a local area network, or any combination of networked computers or systems. The communications described herein can be accomplished using any kind of wired and/or wireless computing network or communications means capable of transmitting data or signals, such as a wireless and/or wired computing network allowing communication via, for example, an 802.11 wireless LAN (“WLAN”) protocols, “Wi-Fi” protocols, Bluetooth protocols, 60 Ghz Protocols (“WirlessHD” and “WiGig”), Wireless Home Automation protocols (“Z-Wave” and “Zigbee”), cellular data protocol (e.g., EDGE, CDMA, CDMA2000, TD-SCDMA, WCDMA, UMTS, HSDPA, EVDO, TDMA, FDMA, GSM, LTE, HSPA+, WiMAX, OFDMA), and/or the like. Suitable examples include a packet-switched network, a local area network (LAN), wide area network (WAN), virtual private network (VPN), or any other means of transferring data. The network 160 may be a partial or full deployment of most any communication/computer network or link, including any of, any multiple of, any combination of or any combination of multiples of a public or private, terrestrial wireless or satellite, and wireline networks or links. A single network 160 or multiple networks (not shown) that are communicatively coupled to one another can be used. It is contemplated that multiple networks of varying types can be connected together and utilized to facilitate the communications contemplated by the systems and elements described in this disclosure.
In one embodiment, the system is generally configured as a central server that is operated by an administrator to allow permissioned access by users that operate the clients. In this embodiment, the server 115 may be located at essentially any location, and may actually be embedded within a client computer (e.g., client 105 may also perform the functions of the server 115). The server 115 is configured to allow access by the clients through an accessible data portal, which communicates with each client through the network 160. The accessible data portal may comprise any number of security measures to provide a reasonably secure system, suitable for embodiments of the present invention. The accessible data portal may further comprise a graphical user interface (GUI) (e.g., a web-based portal) through which any of the first client 105 or secondary clients 107 may access the server 115.
The server 115 may also comprise a processor and operationally associated memory storing one or more databases or other sortable data storage memory to enable the system and methods disclosed herein. The database may be any commercially available storage database suitable for embodiments of the present disclosure, such as those systems commonly available under the names Oracle, DB2, Microsoft Access, Microsoft SQL, Postgres, MySQL, 4th Dimension, FileMaker, Alpha Five Database Management System, or the like. Often contained within the database is a plurality of data sets, each comprising specific data. A first data set may correlate to a first client 105, wherein a plurality of client-specific data is stored. The database may also include any number of subsequent data sets representing N clients, wherein N represents any number of clients practical for operation of embodiments of the present disclosure. The system may be configured to run more than one instance, multiple instances, and/or the like, of the exemplary systems and methods described herein.
The system users may include any person, business or entity, capable of participating in the system and methods disclosed herein. For example, the first client 105 may be a sponsor entity seeking regulatory approval for a drug, device, product, and/or the like (such a sponsor may act on its own behalf, or through an agent). The secondary clients 107 may include entities or persons that are providing services for the first client 105 in successfully navigating the regulatory process and analyzing data to ensure compliance with a regulatory authority. Alternatively, or in addition, the secondary clients 107 may be other sponsors that are unrelated to the first client 105. Furthermore, one or more of the first client 105 and the second clients 107 may be a regulatory authority, such as the FDA, that uses the system 100 during the approval process. In any of the foregoing configurations or other configurations, the administrator preferably controls access to the system by the clients, manages and maintains the operation of the system, and maintains the data in a secure environment.
The system 100 preferably includes secure access through a username and password combination or other security systems. An example of a login prompt 200 is shown in
After logging in, the user may be presented with a module selection interface 300, such as shown in
The system preferably is configured to have an administrative mode, and a user review mode. In general terms, the administrative mode is used to set up projects and perform final reviews, and the user review mode is used to perform the core information processing. An exemplary user interface 400 for the administrative mode is shown in
In this example, selecting the Set Up application opens the Set Up submenu 406, which includes applications titled Sponsors, Checklists, Study Types, Project, and Users. This submenu, which preferably is only displayed to users having administrator level or greater credentials, is used to set up sponsors, checklists, study types, projects and users.
The “sponsor” is the highest organizational level in the system. The sponsor designation can be used to identify an entity seeking approval of a drug or device. The sponsor may be identified in the system as a company, a division of a company, a consulting organization that works for one or more companies, and so on. Once a sponsor is created, one or more projects can be associated with each sponsor.
The projects are the second organizational level. One or more projects may be created for each drug that is being sponsored by the sponsor (multiple projects may be appropriate where the same drug is being evaluated for different indications of formulations). The administrator can create as many projects as desired for each sponsor.
Study types are the third organizational level. After creating a project, the administrator associates one or more study types with the project. Each study type typically designates a study typically conducted during drug development. As will become more apparent below, the purpose of assigning the study type is to ensure that the proper checklists (and their corresponding elements) are considered during review of the study. During the setup process, the administrator preferably adds all of the study types that are expected to be conducted for each project, based on the administrators assessment of the project details. The studies can address virtually any issue associated with the drug development process. For example, they may address quality issues, chemistry and manufacturing requirements, nonclinical study requirements (including but not limited to pharmacology, pharmacokinetics and metabolism studies), clinical study requirements such as efficacy, safety, and statistics, and formatting and electronic data set requirements.
The Set Up submenu 406 also allows the operator to set up users, such as administrators who have access to create projects and add new users, regular users (the lowest level of access) for users that perform project reviews, and final reviewers who have access to review the work of the regular users. The levels of access for each user can be tailored to specifically allow or disallow any number of system functions. The system preferably also includes the ability to block users, and track each user's system usage.
Finally, the Set Up menu includes an option to manually add checklists to studies. As explained in more detail below, checklists are lists that include various elements that are expected to be found in study information. Checklists (and elements of checklists) also may be included to reflect a sponsor's particular standard operating procedures (“SOPs”). For example a checklist may include questions asking whether certain reports include certain identification information that the sponsor requires in all of its documentation, such as tracking information, manufacturing information, and the like.
The system preferably includes one or more user interfaces to display all of the sponsors, projects, study types, checklists, and users that are entered into the system, and to which the particular user has access. Such an interface may include navigation features, such as word searching and sorting functions, to provide the administrator better access to the desired information.
The user interface also includes a criticality question 608 that is presented based on the user's selection of the study type. If the administrator selects a nonclinical study type, the interface appears with the criticality question in
The criticality question 608 also may be used to prompt the administrator to add more checklists to the study, or to automatically add additional checklists to the study. In this example, answering “Yes” to either question assigns a high level of criticality to any deficiencies, and prompts the addition of additional checklists. The prompt to add additional checklists is shown in
The use of the criticality question is expected to provide significant benefits to the system. For example, it provides a simple and intuitive way to categorize the importance of any deficiencies with the supporting information (as explained in more detail below), without relying on particular user's potentially subjective interpretation of any given deficiency. Instead, the administrator only needs to make the relatively objective decision as to whether the study is a GLP study or is intended to support the efficacy indication. Using the criticality question to add checklists also avoids the possibility that checklists with more detailed questions that are appropriate for, say, a GLP study, are not used to analyze a non-GLP study, which may result in a large number of potentially irrelevant deficiencies being identified for the study. Is will also be appreciated that some study types may not require a criticality question. For example, a study type that is not related either to GLP studies or to supporting the efficacy indication may not require a criticality question.
As noted above, the system preferably is also configured to allow an administrator to import a number of studies, as depicted in step 506 of
The system preferably includes a predetermined list of study types. Each study type preferably is unique, and has a unique associated set of checklists (checklists can, however, be common to multiple study types).
The list of study types, checklists, and associations between the study types and the checklists preferably cannot be changed by end users or administrators. Instead, the study types, checklists, and associations preferably are programmed into the system such that they can only be changed by the program developer (e.g., operating under “super user” credentials), or by formal upgrades to the system that are made by the program developer.
The system preferably includes a predetermined list of checklists. For example, the system may include twenty-four unique checklists 900, as shown in
Each checklist includes any number of (i.e., one or more) elements, and it is expected that in many cases each checklist will include a large number (dozens or hundreds) of elements. Some elements may be present in multiple different checklists. Each element comprises a feature that the regular user and final reviewer should identify and evaluate in the supporting study information, to determine whether the feature is present and compliant with any existing requirements. Examples of some elements are shown in
Each element also preferably has one or more associated citations 1106 to identify regulatory or legal basis or guidelines for the element. For example, in the shown interface, the element includes a citation to the FDA's issued guidance on how to review validation of chromatographic methods. The citation 1106 preferably also includes a hyperlink 1108 that the user can select to direct the user to the citation source. Such a hyperlink 1108 may direct the user's computer to a static link (i.e., a file stored in the system or on an external server 1171-117n that is called up when the hyperlink 1108 is selected), to a dynamic link (i.e., a link to another entity's stored data, such as a link to the FDA's webpage hosting the latest versions of 21 C.F.R.), or to any other source of the citation. In a preferred embodiment, every element includes a citation 1106, but this is not strictly required.
The system may be configured to allow special releases of new study types, study type checklist associations, checklists, elements, and citations to guidance. Such changes may be desirable to account for changes in the controlling law or guidance, to correct errors, or to add additional functionality (e.g., new releases for different regulatory bodies). Special releases may be provided in the form of pushed updates (updates that are automatically sent to the system), manually downloaded updates, and so on. The release may be made immediately upon its preparation, or saved for a periodic (e.g., annual) update. Such upgrades may be provided with release notes to explain the new study type. Upon adding a new study type, checklist, element, etc., it also may be necessary to retire a previous version of the same to prevent further use of the outdated version. In order to account for ongoing user activity, the special release may be conducted by allowing users to check in the old version of a checklist that has been updated, but then retiring that version of the checklist and making it unavailable for further access. The system also may be configured to poll all projects in progress, and provide a summary of any previously-completed checklists that are out of date following the special release. The system also may be configured to require previously-entered information to be reentered to ensure that the analysis of the elements is up to date. Alternatively, the previously-entered information may be retained without reentering, and then reviewed to ensure that any changes made by the special release do not affect the outcome of the analysis. The acts of creating new study types, checklists, elements, etc. and retiring old version of the same preferably are performed only by a central programming authority or super user, to make sure that all versions of the system are operating with the same parameters.
Referring back to
The reviewers may access the system using the same login portal as the administrators, but their access credentials may prevent access to certain applications. For example, a reviewer may not have access to the reports module 304, and the main dropdown menu 404 may include access only to the Study List and Program Level Guidance applications.
The Study List application directs the reviewer to a list of projects, studies or checklists to which the particular reviewer has access. For example, in some cases, a reviewer may have access to all of the projects and all of the study types associated with one sponsor, but may not have access to another sponsor's projects. As another example, a reviewer may have access to checklists that are categorized as regulatory, but not to checklists that are categorized as technical. Other distinctions may be implemented in other embodiments, as will be appreciated by persons of ordinary skill in the art.
Upon selecting a study for review, the user may be presented with the exemplary interface shown in
After opening the study review interface 1300, the reviewer may select and check out a particular checklist for review. Upon doing so, the user reviewer may be receive a prompt 1400 to select the task that the reviewer will be performing on the checklist. An example of such a prompt is shown in
Referring to
The interface 1500 preferably also includes a study level query 1522 asking whether kinetic or blood level data is reported in the supporting information. The purpose of the study level query 1522 is to identify when a report on such data exists in the documentation, which can affect the importance or relevance of the study or how the information is interpreted by a regulatory authority. As noted above, in some cases the administrator may not be aware of the full contents of the supporting information, as it can include many hundreds or thousands of pages of documents. Providing the reviewer the opportunity to identify whether there is kinetic or blood level data in the supporting information allows the system to reconfigure to properly evaluate the supporting information. The reviewer preferably can change the answer to the study level query 1522 at any time during review of the checklist.
When a reviewer changes the study level query 1522 to “yes”, the system preferably automatically adds additional checklists to the study, or prompts the user to add checklists to the study. For example, as shown in
Upon answering the study level query 1522 to identify that the supporting information includes kinetic or blood level data, the system also may be configured to prompt the reviewer to add the suggested checklists to other studies. For example, a prompt 1700 such as shown in
Referring back to
The system may be configured in any suitable way to receive the reviewer's evaluation of the supporting information. In the shown example, a speed entry icon 1542 is provided to allow the reviewer to quickly identify an element as either not applicable or acceptable, and an extended entry icon 1544 is provided to allow the reviewer to perform a more detailed data entry process for other elements.
When a reviewer selects “no” in the acceptable dropdown menu 1912, the prompt 1900 allows the reviewer to add one or more deficiencies. For example, the prompt 1900 may provide a deficiency description query field 1916 into which the reviewer can manually type the nature of a deficiency (e.g., “Page 10 is illegible”). For each deficiency, the reviewer preferably also must respond to a query to select a deficiency type (e.g., “Page(s) illegible”) from a predetermined list of deficiency types 1918. After typing a description and selecting a deficiency type, the reviewer may select an add icon 1920 to add the deficiency to the element. If the element has multiple deficiencies, the reviewer can continue entering descriptions and adding deficiency types from the list 1918 until the review of the element is complete. The prompt 1900 may include a summary list 1922 of all of the deficiencies types added from the list 1918. The system preferably is configured to prevent the reviewer from leaving the extended entry prompt 1900 if “no” is selected in the acceptable dropdown menu 1912, until the reviewer changes the selection to “yes” or adds one or more deficiencies.
The use of a predetermined list of deficiency types is expected to provide a significant benefit to some embodiments. For example, requiring reviewers to select deficiency types from a list can help harmonize the manner in which elements are reviewed by different reviewers, and can help each reviewer better understand what constitutes a deficiency for a particular element. The use of a predetermined list of deficiency types also allows the system to easily identify remedial steps that may be taken to address each deficiency type, and generate a report listing all of the deficiencies and suggested remedial actions. This aspect of the system also helps facilitate various useful data analysis processes. For example, if a sponsor provide supporting data that includes a large number of a particular deficiency type, it may indicate a systemic problem with the sponsor's drug development program. As another example, the performance of individual reviewers can be assessed by examining their practices for assigning deficiencies to the elements. Other uses will be readily apparent to persons of ordinary skill in the art in view of the present disclosure.
The extended entry prompt 1900 also may include other features, such as a list of previous findings 1924 that have been made in relation to the particular element. The extended entry prompt 1900 (and the speed entry prompt 1800) also may include a show reference icon 1926 that may be selected to provide guidance to the reviewer regarding the purpose and proper analysis of the element.
The inclusion of direct access to relevant guidance for each element is a significant improvement over earlier simple spreadsheet-type lists of elements that have been used in the past to help evaluate information intended for regulatory review. By associating hyperlinks to the relevant guidance with the element review prompt, the reviewer is no longer required to manually identify relevant guidance to assess the various elements of a study. This greatly reduces the possibility that the reviewer will review and rely on outdated or incorrect guidance, and can save a significant amount of time. Providing the guidance in this manner also allows the reviewer to self-educate on the important details of the relevant law or guidance.
As noted above, the study type defines the checklists that are to be used during the review stage. However, not all information about a study may be known at the time the study is created by the administrator. This is not an uncommon problem, because a study's supporting information often includes many thousands of pages of documentation, and as a practical matter the administrator cannot always accurately prepare a checklist for use by the reviewers until the contents of the study have been fully reviewed. One possible solution to this problem would be to simply present every possible checklist (including every possible element that might be required by the regulatory authority) to the reviewer. This “brute force” method would be likely to ensure that all regulatory requirements are considered, but this process also would be very time-consuming, and the reviewers would be likely to have trouble keeping track of the large number of checklists and elements to make sure each is addressed.
To address this problem, the system may be configured to prompt the user to add additional checklists based on the answers to certain key questions. This foregoes the necessity to address certain checklists unless the supporting information invokes their consideration, which can save time and money, and prevent the appearance that the supporting information is somehow deficient for lacking the information required in the additional (but unnecessary) checklists.
Processes for adding new checklists have been described above in relation to adding checklists to a study based on the reviewer's answers to criticality questions and study level queries. Embodiments also may be configured to manually or automatically add checklists to a study (or elements to a checklist) based on the reviewer's replies to particular checklist elements. For example, certain checklists may include trigger elements that ask whether the supporting information includes particular data, and if the reviewer answers “yes” to any of these trigger elements, then the system can automatically add one or more additional checklists specific to these types of data to the study. As a more specific example, the “Pharmacokinetic/Toxicokinetic Report” and “Scientific Review of PK Study Report” checklists shown in
Any given study may include a number of checklists, and various checklists may include duplicative elements. As such, the system may be configured to allow the reviewer to skip elements that are duplicative to those that were previously answered in other checklists. For example, the system may be configured to show those duplicative elements in read-only mode, or it may be configured to show where each element has been previously answered so the reviewer can manually determine whether it is necessary to re-answer an element for a different checklist. Other alternatives will be readily apparent to persons of ordinary skill in the art in view of the present disclosure.
The reviewer completes the 1st Review task by addressing all of the necessary elements in a checklist. After the checklist is complete, the reviewer checks the checklist back into the system, and the replies to the elements are recorded and stored. The 1st Review results preferably are permanently stored in the database to allow auditing of the answers at a later date, and the checklist may be locked to prevent further reviews using the 1st Review option shown in the prompt 1400 of
As shown in
The system preferably also includes a Corrections task that may be activated after the Quality Control review. The Corrections task is used to identify differences between the 1st Review and the Quality Control review. For example, the system may present an interface 2100 as shown in
The Corrections task may be used for various purposes. For example, this may provide an opportunity for reviewers to consult and determine whether there is a true discrepancy between their reviews, or whether one reviewer or the other made a mistake. After consulting about any discrepancies between the two reviews, if the reviewers agree to a corrective action, corrections may be made to the relevant element(s) to bring the two reviews into agreement. Corrections may take the form of an annotated record that is associated with the particular element in question, by revising the Quality Control record to make the change, or by other means, as will be appreciated by persons of ordinary skill in the art.
Comparative data from the Corrections task also may be used to evaluate reviewer performance, to assist with training, and so on.
The system may be configured to allow various administrative functions throughout the review process. For example, a record may be kept of each recorded entry for each element (including any correction) to assure a complete audit trail for the study. Such records may include a user identity, edit time, substance of the edit, and so on. The records also may include keystroke-level records to track each individual interaction with the system. The system also may be configured to allow oversight functions, such as the ability to examine checklists in read-only mode even while they are checked out, to actively monitor the review process.
As noted above in relation to
After the 1st Review, Quality Control review and Corrections tasks are complete, the checklists are released for Final Review. The Final Review may be used to evaluate the previous review activities, identify any open questions related to the review process, and obtain an overall evaluation of the review status. The Final Review task preferably can only be performed by users with final reviewer-level credentials, administrators, or other higher-level users.
In one embodiment, Final Review may be initiated by using the Study List application in the user interface 400 (see
The criticality assessment indicates the importance of the deficiency in relation to obtaining approval of the subject of the study (e.g., the drug or device). In this example, the assessments are identified as “critical” and “minor.” Different colors may be used to identify each type of deficiency (e.g., red for “critical,” green for “minor,” and blue for “major”). As explained above, the criticality question 608 may be used to determine the level of criticality that should be assigned to any deficiency for a given study. In the example of
If this were not a GLP study, the system could be configured to designate the six “critical” elements as being “major,” but the “minor” element would still be designated as “minor,” because the “minor” element has little or no bearing on acceptance of the study regardless of whether it is a GLP study. This type of system can be readily established by associating a risk value with each type of deficiency, and categorizing some deficiency types as being high risk (which would be identified either as “critical” or “major” when present), and others as being low risk (which would always be identified as “minor”). Of course, other embodiments could use additional levels of risk assessment, and other embodiments also could use more complex categorization schemes, such as by categorizing some deficiencies as being “minor” in the context of a non-GLP study but critical in the context of a GLP study. Other alternatives will be readily apparent to persons of ordinary skill in the art in view of the present disclosure.
From the foregoing, it will be appreciated that the decision to designate a deficiency as critical can be based on several different variables. In one embodiment, there are two variables: first, the study type must be designated as a GLP study (this can happen during initial setup or during review, as explained above); and second, the reviewer must select a particular deficiency type from a predetermined list of deficiency types that is sufficient to affect regulatory approval. Using this logic, if the study that is the subject of the example of
In other embodiments other variables may be used or added. For example, a third variable may be that the deficiency must be with one or more particular elements in the checklists (e.g., a particular deficiency type, such as “page(s) illegible” may be a critical deficiency for some elements of the checklist, but not for other elements). Other alternatives will be readily apparent to persons of ordinary skill in the art in view of the present disclosure. The system is configured to identify the criticality assessment based on a review of the selected variables, which may be readily accomplished using database associations and logic.
As similar regime preferably is used for clinical studies that are intended to support the efficacy indication of the drug. If a clinical study is intended to support the efficacy indication, then certain deficiency types will result in a “critical” criticality assessment in the report. Alternatively, if a clinical study is not intended to support the efficacy indication, then those same deficiency types will result in a “major” criticality assessment in the report. Other deficiency types also may be includes as “minor” deficiencies in either case.
As noted above, the Deficiency Summary report also may include suggested actions. The suggested actions preferably are based on the deficiency type. For example, any time the reviewer selects the deficiency type “Missing information or clarification required,” the system may provide a suggestion action of “Obtain missing information or clarification and amend report.” If desired, the report may also or alternatively include suggested actions that are selected or typed by the reviewers.
The system also may be configured to allow multiple review stages for each project, study or checklist. For example, after the Final Review task has been performed for all of the checklists in a particular study, the administrator may use the Administrative Functions application (
The system may provide an interface to allow users to review of all of the studies to see what stage each study is in. For example, the system may present a list of all studies associated with one or more projects (e.g., all of the projects for a particular sponsor) with an indication of the review stage for each study. Such an interface is expected to help administrators better manage the overall progress of the project, and may be helpful to make decisions about how to allocate funds and personnel resources to best advance the sponsor's overall product development goals. Other uses will be readily apparent to persons of ordinary skill in the art in view of the present disclosure.
In some embodiments, the system may be configured to generate redline reports for checklists. For example, an administrator may open a Checklist Redline Reports application as shown in the interface of
Embodiments also may be configured to generate audit reports to track the use of the system. For example, a user may be able to select a date range and one or more checklists (or projects, studies or elements), and the system will perform an audit report to show a summary of all changes to the checklist during the date range. For example,
Each activity 2606 in the list also may include an icon 2608 that may be selected to open the activity 2606 to view further details of the activity 2606. For example, selecting an activity may open an interface 2700 such as shown in
The system preferably also allows users to generate printed or electronic reports, such as reports of all of the reviewed elements, or Deficiency Summaries. In some embodiments, the system may be formatted to prepare reports that may be submitted to regulatory authorities in paper or electronic form. For example, where a private sponsor and a regulatory authority both operate versions of the system, the sponsor may electronically transmit reports to the regulatory authority as documentation of diligence and results in analyzing the supporting information. The system also may be configured to prepare reports in particular electronic formats such as electronic common technical document (“eCTD”) formats, and to otherwise prepare reports according to conventional and accepted scientific data formats. It also may be desirable to limit which users can prepare such reports. For example, in may be desirable to only allow administrators to prepare such reports by accessing a reports module 304 as shown in
It will be appreciated from the foregoing description that embodiments may be useful to assist a user in a range of activities, such as: navigating the regulatory and drug development process, managing contract research services, performing data analysis, formulating research and regulatory strategies, training and overseeing reviewers, and assessing sponsor drug development programs. The system may be used to analyze supporting information, identify the risks and/or benefits of the data, and translate the assessment into documents that address health authority requirements and potential concerns while supporting a user's or sponsor's objectives. For example, a sponsor can use the system to oversee and manage the drug development lifecycle to ensure that a submission for drug approval will be successful when it is eventually made. While embodiments have particular utility in the context of obtaining approval for new drugs and the like, the system can be configured to help users evaluate other kinds of product that is or may be subject to manufacturing requirements, SOPs, or other reviews or approval processes (e.g., dietary supplements or regulated or unregulated devices used for medical purposes). As noted above, the system also may be configured to address regulatory requirements of any desired authority or guiding organization.
In step 2818, the administrator responds to a criticality question (e.g., is this a non-clinical study that is intended to be a GLP study?, or is this a clinical study that is intended to support the efficacy indication of the drug?). If the answer is “no,” the process moves to step 2820, and the system sets the criticality of any deficiencies that might be created in the checklist record 2816 to “low,” by generating a criticality record 2822 (which may be embedded in the study record 2806, checklist record 2816, provided as a separate file, or otherwise stored) and setting a criticality variable in the criticality record 2822 to “low.” If the answer to the criticality question is “yes,” then the process moves to step 2824 and the system and/or the administrator adds one or more additional checklists (or elements) to the checklist record 2816, and then moves to step 2826 where the system generates the criticality record 2822 with the criticality variable set to “yes” to indicate that deficiencies that might be created in the checklist record 2816 may be critical.
In step 2828, the study (or one or more checklists of the study) is released and the 1st Review begins. During the 1st Review, the reviewer checks out the checklist, evaluates at least some of the study information 2802, and records whether each element of a checklist is reviewed, applicable, provided, or satisfactory. The recorded replies may be added to the checklist record 2816 (or an associated file) immediately, or at the end of the 1st Review (step 2840) when the checklist is checked back into the system. In step 2820, the reviewer is presented with a study level query 2830, such as described above. If the answer to the study level query is “yes,” the system and/or reviewer changes the criticality record 2822 to “high” in step 2832, and in step 2834 the system and/or reviewer may add one or more additional checklists (or elements) to the checklist record 2816. At the conclusion of step 2834, or if the answer to the study level query in step 2830 is “no,” the process continues to step 2836. In step 2836, the system and/or reviewer evaluates the answer(s) to any trigger elements, such as described above, that might be included in the checklist elements. If the answer is “yes,” the process proceeds to step 2838, where the system and/or reviewer adds one or more additional checklists (or elements) to the checklist record 2816. At the end of step 2838, or if the answer to the trigger element query 2836 is “no,” the process proceeds to step 2840, where the 1st Review is finalized. At this point, the checklist is checked back into the system, and any data entries for elements that have not already been saved to the checklist record are saved there.
The process then proceeds to Quality Control in step 2842, Corrections in step 2844, and Final Review in step 2846. Each of these steps may include additional process steps similar to those that are conducted between the start and finalization of the 1st review (e.g., Quality Control may include answering study level queries, and evaluating trigger elements, changing criticality and adding checklists or elements). Other steps, such as preparing reports, performing checklist redlines, and the like, also may be added throughout the process.
After the Final Review step 2846 is complete, a user or the system may generate a report in step 2848 and save the report to a report record 2850 associated with the checklist. The report may be indicate whether the study information is not adequate for any of the repot elements. For example, the report may be a deficiency summary report, such as described above, indicating whether any elements are deficient, and suggesting remedial action. In step 2852, the system or a user evaluates whether any element includes a critical deficiency. If there are any critical deficiencies, the process may start again at the 1st Review step 2828 to commence an additional review stage, in which any deficiencies may be addressed and corrected. If there are no critical deficiencies, then the process may move to finalization for submission to the relevant regulatory authority in step 2854.
The additional checklists (or elements) selected in steps 2824, 2834 and 2838 may be chosen based on additional checklist associations provided in the checklist association record 2812, or selected using other logical processes. Furthermore, some steps of the process may be automated, while others are performed manually. For example, in preferred embodiments, one or both of the steps of changing the criticality record 2822 to “high” (steps 2826 and 2832) preferably are automatically performed upon answering the relevant question without reviewer approval. It is also preferred for the step of adding additional checklists in reply to a trigger element (step 2838) to be automated and performed without reviewer approval. As another example, it is preferred for step 2834 to be at least partly manually performed by the reviewer to provide an opportunity for the reviewer to determine whether the added checklists are redundant or unnecessary. Other alternatives will be readily apparent to persons of ordinary skill in the art in view of the present disclosure.
For organizational purposes, each of the records may be stored in a common system database, and may include mirrored files in separate databases to provided redundant storage. Each record may include a variable to associate the record with other records. For example, each record may include a study name variable that associates each record with a particular study, and may include other variables to make other associations. Other types of storage and association regimes may be used instead, and the manners in which the records are stored and associated are not critical to the embodiments.
It will be appreciated that the process of
The process described in
In one example, the list of study types 2808, list of checklists 2814, and the checklist association record 2812 may be stored as one or more files in the memory of one or more computers, such as a server system comprising one or more servers 115, 1171, 117n, etc. These files also may be stored on client computers 105, 1071, 107n, etc. As used herein, the term “file” is intended to cover any kind of data storage regime, including separate data records (e.g., the list of study types 2808 is a completely different data record as the list of checklists 2814), collective data records (e.g., the list of study types 2808 is stored collectively with the list of checklists 2814 in a database structure), and so on. It will also be appreciated that files can be distributed among memories in different servers and other computers. Other alternatives will be readily apparent to persons of ordinary skill in the art in view of the present disclosure.
The headings used herein are for organizational purposes only and are not meant to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include,” “including,” and “includes” mean including but not limited to. Furthermore, the fact that some embodiments or features are described as exemplary, optional or the like is not intended to suggest that features not so described are mandatory. Also, the terms “or” and “and” are used herein in a nonexclusive manner, and are intended to cover both conjunctive and alternative selections of items in a list (i.e., “and” means “and/or” and “or” means “and/or”).
In the foregoing detailed description, numerous specific details are set forth in order to provide a thorough understanding of exemplary embodiments or other examples described herein. However, it will be understood that these examples may be practiced without the specific details, or with the details in alternative arrangements (e.g., moving the prompt to answer a criticality question into a separate interface, subdividing interfaces, or providing alternative menu architectures). In other instances, well-known methods, procedures, components and circuits have not been described in detail, so as to not obscure the description. Further, the examples disclosed herein are for exemplary purposes only and other examples may be employed in lieu of, or in combination with, the examples disclosed. It should also be noted that the examples presented herein should not be construed as limiting of the scope of embodiments of the present disclosure, as other equally effective examples are possible and likely.
It will also be understood that, while the foregoing is directed to exemplary embodiments of the present disclosure, other and further embodiments of the systems described herein may be devised without departing from the basic scope of the disclosure, and should be considered part of this disclosure, as if described fully herein. For example, whereas the worldwide web and mobile web are growing content and capabilities at ever-increasing rates, the ability to adapt the systems, methods, applications, and interfaces disclosed herein to existing or new mobile- or web-based technology is contemplated by embodiments of the present disclosure and such modifications would not depart the scope of the disclosure disclosed herein.
This application is a continuation of U.S. application Ser. No. 14/706,334 filed on May 7, 2015 which claims the benefit of U.S. Provisional Application No. 61/989,943, filed May 7, 2014, the entire contents of which are incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61989943 | May 2014 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 14706334 | May 2015 | US |
| Child | 16126749 | US |
