Patent Grant
11875896
Some embodiments of the present disclosure are directed to systems and methods for conducting remote health testing and diagnostics.
Use of telehealth to deliver health care services has grown consistently over the last several decades, and has exploded in usage during the Coronavirus disease 2019 (COVID-19) Public Health Emergency (PHE). Telehealth is the distribution of health-related services and information via electronic information and telecommunication technologies. Telehealth allows long-distance patient and health provider contact, care, advice, reminders, education, intervention, monitoring, and remote admissions. In situations such as the COVID-19 PHE, when many medical and hospital resources are devoted to treating the sick, patients are more reluctant to travel to their health provider in-person, and when access to care is restricted, telehealth provides an invaluable resource.
During the COVID-19 pandemic, testing for coronavirus disease (COVID-19) was extremely limited in various places throughout the world, including the United States. Tracing infected individuals was and continues to be an important step in preventing new cases of infectious diseases. In response, the United States Food and Drug Administration (FDA) has authorized various at-home COVID-19 tests.
At-home testing solves some of the problems with in-person testing. For example, health insurance may not be required, travel to a testing site is avoided, and tests can be completed at a patient's convenience. However, at-home testing introduces various additional logistical and technical issues, such as guaranteeing timely test delivery to a patient's home, providing test delivery from a patient to an appropriate lab, ensuring test verification and integrity, providing test result reporting to appropriate authorities and medical providers, and connecting patients with medical providers, who are needed to provide guidance and/or oversight of the testing procedures remotely. These issues are not unique to COVID-19 and will need to be addressed in relation to remote health diagnostic testing generally. Therefore, a novel health and diagnostic platform, which solves the issues associated with at-home health testing, is needed.
For purposes of this summary, certain aspects, advantages, and novel features of the invention are described herein. It is to be understood that not all such advantages necessarily may be achieved in accordance with any particular embodiment of the invention. Thus, for example, those skilled in the art will recognize that the invention may be embodied or carried out in a manner that achieves one advantage or group of advantages as taught herein without necessarily achieving other advantages as may be taught or suggested herein.
Health testing and diagnostics platforms are provided herein that can facilitate proctored at-home testing. Users performing at home testing may be guided or assisted by proctors that are available over a communication network. The platform can be optimized such that proctors can supervise the at-home tests of a plurality of users. In some embodiments, users that have gained experience and proficiency with the platform and the at-home tests may be eligible for expedited or express proctoring sessions. Such platforms can, for example, greatly increase the availability of testing, and reduce the spread of infectious diseases, and increase the efficiency of proctors in terms of increasing the number of patients that a proctor can simultaneously monitor.
In a first aspect, a computer-implemented system for a proctored examination platform for a medical diagnostic test is described. The computer-implemented system comprises an electronic storage comprising computer-executable instructions and one or more processors in electronic communication with the electronic storage medium and configured to execute the computer-executable instructions in order to: receive, by the computing system from a user device of a user in communication with the computing system over an electronic network, a request for a proctored examination session for a medical diagnostic test; based on the request, establish, by the computing system, the proctored examination session between the user device of the user and a proctor device of a proctor over the electronic network; receive, by the computing system, image data from at least one imaging device of the user device, the image data comprising at least a view of the user or at least one testing material of a test kit for the medical diagnostic test; identify, by the computing system, at least one feature within the image data received from the imaging device of the user device, the at least one feature comprising at least one of an anatomical feature of the user or at least one testing feature of the at least one testing material of the test kit; generate, by the computing system, user display data for display to the user on a user graphical user interface on a display of the user device, the user display data comprising at least: the image data received from the at least one imaging device of the user device, and at least one computer-generated graphic associated with the identified at least one feature within the image data, wherein the at least one computer-generated graphic is overlaid onto the image data at a position associated with at least one identified feature, and the at least one computer-generated graphic is configured to facilitate at least one step of the medical diagnostic test; generate, by the computing system, proctor display data for display to the proctor on a proctor graphical user interface on a display of the proctor device, the proctor display data comprising at least the image data received from the imaging device of the user device and information associated with the medical diagnostic test; transmit, from the computing system over the electronic network, the user display data to the user device for display on the user graphical user interface on the display of the user device; and transmit, from the computing system, the proctor display data to the proctor device for display on the proctor graphical user interface on the display of the proctor device; whereby the user performs the at least one step of the medical diagnostic test based on the user display data and the proctor monitors performance of the at least one step of the medical diagnostic test based on the proctor display data.
The system can include one or more of the following features in any combination: (a) wherein the proctor display data further comprises the at least one computer-generated graphic associated with the identified at least one feature within the image data, wherein the at least one computer-generated graphic is overlaid onto the image data at a position associated with at least one identified feature, and the at least one computer-generated graphic configured to facilitate at least one step of the medical diagnostic test; (b) wherein identifying the at least one feature within the image data received from the imaging device of the user device comprises analyzing, by the computing system, the image data using a computer vision algorithm configured to detect the at least one feature within the image data; (c) wherein transmitting the user display data to the user device and transmitting the proctor display data to the proctor device occur substantially simultaneously such that the user and the proctor view the image data in a substantially synchronized state; (d) wherein the at least one feature within the image data comprises a nostril of the user; (e) wherein the at least one feature within the image data comprises a test swab; (f) wherein the at least one feature within the image data comprises a portion of a testing material to which a testing solution is to be introduced by the user; (g) wherein the information associated with the medical diagnostic test of the proctor display data comprises an indication of a current testing step of the medical diagnostic test; (h) wherein the information associated with the medical diagnostic test of the proctor display data comprises a testing instruction to be read to the user by the proctor; (i) wherein the proctored testing session comprises an electronic video conference session between the user device and proctor device; (j) wherein the user comprises a first user, the image data comprises first image data, and the computer-implemented system is further configured to receive, by the computing system from a second user device of a second user in communication with the computing system over the electronic network, a second request for a second proctored examination session for a second medical diagnostic test; based on the second request for the second proctored examination session for the second medical diagnostic test, establish the second proctored examination session between the second user device of the second user and the proctor device of the proctor over the electronic network; receive, at the computing system, second image data from at least one second imaging device of the second user device, the second image data comprising at least a second view of the second user or at least one second testing material of a second test kit of the second medical diagnostic test; identify, by the computing system, at least one second feature within the second image data received from the second imaging device of the second user device, the at least one second feature comprising at least one of an anatomical feature of the second user or at least one testing feature of the at least one second testing material of the second test kit; generate, by the computing system, second user display data for display to the second user on a second user graphical user interface on a second display of the second user device, the second user display data comprising at least: the second image data received from the at least one second imaging device of the second user device, and at least one second computer-generated graphic associated with the identified at least one second feature within the second image data, wherein the at least one second computer-generated graphic is overlaid onto the second image data at a position associated with at least one identified second feature, and the at least one second computer-generated graphic is configured to facilitate at least one step of the second medical diagnostic test; generate, by the computing system, second proctor display data for display to the proctor on the proctor graphical user interface on the display of the proctor device, the second proctor display data comprising at least the second image data received from the imaging device of the second user device and second information associated with the second medical diagnostic test; transmit, from the computing system over the electronic network, the second user display data to the second user device for display on the second user graphical user interface on the display of the second user device; and transmit, from the computing system, the second proctor display data to the proctor device for display on the proctor graphical user interface on the display of the proctor device; whereby the second user performs the at least one step of the medical diagnostic test aided by the second user display data and the proctor monitors performance of the at least one step of the second medical diagnostic test aided by the second proctor display data; (k) wherein the proctor graphical user interface is configured to display the first proctor display data and the second proctor display data at the same time such that the proctor monitors the first user and the second user at the same time; and/or other features as described throughout this application.
In another aspect, a computer-implemented system for a multi-session proctored examination platform for a medical diagnostic test is described. The computer-implemented system comprises: an electronic storage medium of a computing system, the electronic storage medium comprising computer-executable instructions; one or more processors of the computing system, the one or more processors in electronic communication with the electronic storage medium, the one or more processors in electronic communication through an electronic network with a first user computing device and a second user computing device, the one or more processors configured to execute the computer-executable instructions stored in the electronic storage medium for implementing the multi-session proctored examination platform for the medical diagnostic test by: receiving, by the computing system, through the electronic network a first user request from a first user for a proctored examination at a first time, the first user request received from the first user computing device; generating, by the computing system, display data for displaying a graphical user interface (GUI) on a display of a proctor device, the display data configured to display to a proctor a first set of two or more phases of the medical diagnostic test and a first phase indicator showing a current phase of the first user in the two or more phases of the medical diagnostic test; transmitting, by the computing system, through the electronic network the display data to the proctor device; receiving, by the computing system, through the electronic network a first video conference connection request from the proctor device; establishing, by the computing system, a first electronic video conference session between the proctor device and the first user computing device; receiving, by the computing system, through the electronic network a second user request from a second user for a proctored examination at a second time, the second user request received from the second user computing device, wherein the second time is later in time than the first time; generating, by the computing system, supplemental display data for displaying the graphical user interface (GUI) on the display of the proctor device, the supplemental display data configured to concurrently display: the first set of two or more phases of the medical diagnostic test and a second phase indicator showing the current phase of the first user in the first set of two or more phases of the medical diagnostic test, and a second set of two or more phases of the medical diagnostic test and a third phase indicator showing the current phase of the second user in the second set of two or more phases of the medical diagnostic test, wherein the second and third phase indicators are at different phases in the two or more phases of the medical diagnostic test; transmitting, by the computing system, through the electronic network the supplemental display data to the proctor device; receiving, by the computing system, through the electronic network a first video conference connection termination request and a second video conference connection request from the proctor device; terminating, by the computing system, the first electronic video conference session between the proctor device and the first user computing device; transmitting, by the computing system, through the electronic network to the first user computing device content data configured to be displayed on the first user computing device; and establishing, by the computing system, a second electronic video conference session between the proctor device and the second user computing device.
The system may include one or more of the following features in any combination: (a) wherein the content data comprises augmented reality content data for showing the first user how to administer the medical diagnostic test; (b) wherein the content data comprises virtual reality content data for showing the first user how to administer the medical diagnostic test; (c) the proctor can simultaneously conduct the second video conference session with the second user while monitoring the first user that is viewing the content data; (d) wherein the monitoring the first user that is viewing the content data comprises transmitting, by the computer system, through the electronic network to the proctor device video data of the first user to ensure that the first user is watching the content data; (e) analyzing, by the computing system, using a computer vision algorithm the video data to detect a medical diagnostic test device and to determine if the medical diagnostic test device has moved, and transmitting, by the computing system, through the electronic network, based on determining that the medical diagnostic test device has moved, an alert notification to the proctor device to notify the proctor that the medical diagnostic test device has moved; (f) receiving, by the computing system, through the electronic network a mute request from the proctor device to mute the proctor for the second electronic video conference session, and an unmute request from the proctor device to unmute the proctor associated with the video data being transmitted to the first user; (g) wherein the third phase indicator is in an earlier phase relative to the second phase indicator; (h) the computing system comprises one or more computing systems; (i) wherein the supplemental display data configured to concurrently display the first set of two or more phases of the medical diagnostic test and the second set of two or more phases of the medical diagnostic test enables the proctor to monitor multiple medical diagnostic test sessions for multiple users; and/or other features as described throughout this application.
In another aspect, a computer-implemented method for a multi-session proctored examination platform for one or more medical diagnostic tests, the computer-implemented is described. The method comprises: receiving, by the computing system, through an electronic network a first user request from a first user for a proctored examination at a first time, the first user request received from a first user computing device; generating, by the computing system, display data for displaying a graphical user interface (GUI) on a display of a proctor device, the display data configured to display to a proctor a first set of two or more phases of a first medical diagnostic test and a first phase indicator showing a current phase of the first user in the two or more phases of the first medical diagnostic test; transmitting, by the computing system, through the electronic network the display data to the proctor device; receiving, by the computing system, through the electronic network a first video conference connection request from the proctor device; establishing, by the computing system, a first electronic video conference session between the proctor device and the first user computing device; receiving, by the computing system, through the electronic network a second user request from a second user for a proctored examination at a second time, the second user request received from a second user computing device, wherein the second time is later in time than the first time; generating, by the computing system, supplemental display data for displaying the graphical user interface (GUI) on the display of the proctor device, the supplemental display data configured to concurrently display: the first set of two or more phases of the first medical diagnostic test and a second phase indicator showing the current phase of the first user in the first set of two or more phases of the first medical diagnostic test, and a second set of two or more phases of a second medical diagnostic test and a third phase indicator showing the current phase of the second user in the second set of two or more phases of the second medical diagnostic test, wherein the second and third phase indicators are at different phases in the two or more phases of the first and second medical diagnostic tests; transmitting, by the computing system, through the electronic network the supplemental display data to the proctor device; receiving, by the computing system, through the electronic network a second video conference connection request from the proctor device; terminating, by the computing system, a video feed from the proctor device in the first electronic video conference session between the proctor device and the first user computing device; and establishing, by the computing system, a second electronic video conference session between the proctor device and the second user computing device, wherein the computing system comprises one or more computer processors and an electronic memory.
The system may include one or more of the following features in any combination: (a) transmitting, by the computing system, through the electronic network to the first user computing device a content data comprising augmented reality content data for showing the first user how to administer the first medical diagnostic test; (b) transmitting, by the computing system, through the electronic network to the first user computing device a content data comprises virtual reality content data for showing the first user how to administer the first medical diagnostic test; (c) wherein the proctor can simultaneously conduct the second video conference session with the second user while monitoring the first user through the first electronic video conference session; (d) wherein the monitoring the first user comprises transmitting, by the computer system, through the electronic network to the proctor device video data of the first user to ensure that the first user is present during the first medical diagnostic test; (e) analyzing, by the computing system, using a computer vision algorithm the video data to detect a medical diagnostic test device and to determine if the medical diagnostic test device has moved, and transmitting, by the computing system, through the electronic network, based on determining that the medical diagnostic test device has moved, an alert notification to the proctor device to notify the proctor that the medical diagnostic test device has moved; (f) receiving, by the computing system, through the electronic network a mute request from the proctor device to mute the proctor for the second electronic video conference session, and an unmute request from the proctor device to unmute the proctor for the first electronic video conference session; (g) wherein the third phase indicator is in an earlier phase relative to the second phase indicator; (h) wherein the computing system comprises one or more computing systems; (i) wherein the supplemental display data configured to concurrently display the first set of two or more phases of the first medical diagnostic test and the second set of two or more phases of the second medical diagnostic test enables the proctor to monitor multiple medical diagnostic test sessions for multiple users; and/or other features as described throughout this application.
The drawings are provided to illustrate example embodiments and are not intended to limit the scope of the disclosure. A better understanding of the systems and methods described herein will be appreciated upon reference to the following description in conjunction with the accompanying drawings, wherein:
Although certain preferred embodiments and examples are disclosed below, inventive subject matter extends beyond the specifically disclosed embodiments to other alternative embodiments and/or uses and to modifications and equivalents thereof. Thus, the scope of the claims appended hereto is not limited by any of the particular embodiments described below. For example, in any method or process disclosed herein, the acts or operations of the method or process may be performed in any suitable sequence and are not necessarily limited to any particular disclosed sequence. Various operations may be described as multiple discrete operations in turn, in a manner that may be helpful in understanding certain embodiments; however, the order of description should not be construed to imply that these operations are order dependent. Additionally, the structures, systems, and/or devices described herein may be embodied as integrated components or as separate components. For purposes of comparing various embodiments, certain aspects and advantages of these embodiments are described. Not necessarily all such aspects or advantages are achieved by any particular embodiment. Thus, for example, various embodiments may be carried out in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other aspects or advantages as may also be taught or suggested herein.
Certain exemplary embodiments will now be described to provide an overall understanding of the principles of the structure, function, manufacture, and use of the devices and methods disclosed herein. One or more examples of these embodiments are illustrated in the accompanying drawings. Those skilled in the art will understand that the devices and methods specifically described herein and illustrated in the accompanying drawings are non-limiting exemplary embodiments and that the scope of the present invention is defined solely by the claims. The features illustrated or described in connection with one exemplary embodiment may be combined with the features of other embodiments. Such modifications and variations are intended to be included within the scope of the present technology.
Some embodiments herein are directed to a health testing and diagnostics platform for facilitating remote health testing via remote connection of patients and medical providers. At-home medical testing provides various benefits over in-person visits to medical professionals. For example, at-home medical testing provides both safety and convenience to patients and medical providers. In-person visits by individuals with infectious diseases endangers both medical professionals, as well as anyone who encounters the individuals on their way to the in-person visit. At-home testing does not involve personal contact between the patient and any other individuals who may otherwise be at risk. Furthermore, at-home testing is simply more convenient, as neither medical providers nor patients need to leave the safety or comfort of their home in order to administer a test using remote testing platforms.
Additionally, because of advancements in medical and logistics technology, especially as described herein, at-home testing can now be extremely fast. In some cases, diagnostic tests can be administered and read within seconds. Other tests may require a cure time before being read or may require delivery to a laboratory to receive results, but results can still be received within days in most cases.
Applications for at-home medical testing are abundant. For example, at-home testing can be used by travelers in any location to ensure that the traveler is healthy before and/or after arriving at a destination, without having to locate medical care in an unfamiliar locale. Furthermore, at-home testing may prevent the spread of infectious diseases by providing travelers knowledge of when to quarantine or avoid traveling altogether, and to avoid bringing home an infectious disease. At-home testing may also be useful for sensitive individuals such as the elderly and children. At-home testing may provide a better experience for such sensitive individuals, especially in cases in which the testing procedure is uncomfortable or invasive. At-home testing can mean that the test is done in a safe, comfortable, and familiar environment, so sensitive individuals may feel less stressed and worried during their test, allowing testing to proceed more smoothly. In some instances, at-home testing can be performed in a user's home, although this need not be the case in all instances. For example, as used herein, at-home testing can refer to testing performed in other locations outside the home, such as in hotel rooms, airports, or other remote locations where access to an in-person healthcare provider is not available or desirable. Another consideration for at-home testing is privacy. At-home testing can be private and discreet, which is ideal for high-profile individuals or sensitive individuals who want to get tested without leaving their homes. Also, accessibility considerations favor at-home testing. At-home testing is ideal for anyone who has transportation issues or mobility/accessibility considerations.
Considering the advantages of at-home testing, systems and methods for providing a universal platform for at-home health testing and diagnostics are provided herein. In particular, a health testing and diagnostic platform is provided to connect medical providers (such as remotely located medical providers) with patients and to generate a unique, private testing environment. In some embodiments, the testing environment may facilitate administration of a medical test to a patient with the guidance of a proctor. In some embodiments, the proctor may comprise uncertified personnel, certified medical personnel, and/or a proctor for monitoring an algorithm such as computer software, which may administer a medical test. In some embodiments, the computer software is not administering the medical test but rather is monitoring the medical test for abnormalities or deviations or inconsistencies in the administration or performance or procedure of the medical test that is being administered by the uncertified personnel and/or certified medical personnel and/or medical personnel and/or the like. In some embodiments, the patient may be provided with step-by-step instructions for test administration by the proctor within a testing environment. The platform may display unique, dynamic testing interfaces to the patient and proctor to ensure proper testing protocols and/or accurate test result verification.
In some embodiments, the platform may provide mechanisms for user registration and medical test ordering and fulfillment. In some embodiments, the platform may be connected to one or more pharmacies or third-party providers that may approve and fulfill test kit orders. In some embodiments, the platform may provide a testing environment comprising a private communication channel (such as over the internet) between a proctor and a patient. In some embodiments, the testing environment may comprise one or more unique user interfaces that may facilitate seamless testing, submission and verification. In some embodiments, the platform may provide for automatic transmission of verified test results to users, relevant authorities, and third parties. In some embodiments, the platform may generate a unique health card or passport, which may provide an easily accessible and understandable testing summary for a patient and/or third parties.
In some embodiments, the platform may be configured to register, train, and certify medical providers to proctor diagnostic tests using the platform. In some embodiments, proctors may be enabled to initiate testing sessions via a virtual waiting room and interpret test results using tools provided by the one or more unique user interfaces of the platform. In some embodiments, proctors may be enabled to manage more than one test session concurrently via unique multi-session user interfaces generated by the platform, which assist with management and administration of multiple concurrent test sessions.
In some embodiments, the platform may also be configured to provide urgent care to patients in need by collecting symptom and medical data from patients and providing such data to relevant medical professionals and pharmacies. In some embodiments, the platform may facilitate diagnosis of a patient by a third-party medical provider and fulfillment of a drug prescription by a third-party pharmacy, without any of the parties having direct (e.g., physical or in person) contact.
In some embodiments, at 110, the platform may receive confirmation of a valid prescription of the requested medical test for the patient. The test, along with instructions on how to collect a test sample, may be delivered, for example, via mail, to the patient by the platform administrator or by a third-party. In some embodiments, at 112, a user patient may receive the requested test. Within a prescribed amount of time (or in some cases, an unspecified amount of time), the user patient may enter a test environment of the platform via a user device, such as a personal computer, a cellular phone, a smartphone, a laptop, a tablet computer, an e-reader device, an audio player, or another device capable of connecting to and communicating over a network, whether wired or wireless at 114. Further, the user device at 114 may be the same user device as used at 104 or it may be a different device. For example, the user may access the pre-qualification survey at 104 on a smart phone and then, at 114, enter the test environment on the same smart phone or on a different device, such as a table or laptop. Upon entering the testing environment or a virtual waiting room of the test environment, the platform may manually or automatically match the user patient with a medical provider proctor at 116 to administer the test. The system may provide user-proctor matching in a variety of ways to improve the efficiency and accuracy of the platform. As noted below, various factors of both the users and the proctors may be analyzed to facilitate user-proctor matching.
Upon matching of the user and the proctor, a testing session may begin at 120, with a prompt and testing session being provided to the proctor and a test session recording beginning at 122. In some embodiments, during the testing session, the proctor may guide the user patient through the testing process using provided prompts and tools in order to receive a test result at 124. The proctor and/or the platform may perform validation of the test result using a video feed of the test from the user device or through a static image taken by the user device at 126. In some embodiments, the platform may store the test result in connection with a user profile created for the patient at 128. Furthermore, at 128, the test results may be reported to various government agencies, health authorities, and/or third parties as mandated or desired. In some embodiments, the platform may generate and transmit the verified test result to the patient at 130. In some embodiments, the verified test results may be in the form of a digital health card or “passport” with various information including the test result, date of the test, expiration, and any other relevant details. In some embodiments, the verified test result may be received by the patient via the platform or a third-party website or application at 132.
Also, upon creation of a user account, the user may order a test kit at 206, which may involve a pre-qualification process such as that described with respect to
In some embodiments, one or more of the exchanges between the user and the testing platform as described above with reference to one or both of
As an example, the website for “Duncan's Pharmacy” may include an iFrame element that enables users of the website to interface with the entity that maintains the testing platform without having to manually navigate away from and/or back to the Duncan's Pharmacy website. As such, users of the website for Duncan's Pharmacy may be able to easily complete pre-qualification surveys, create accounts with the testing platform, order test kits, and the like. When ordering test kits through the iFrame on the Duncan's Pharmacy website, in some instances, users may be given the option to have the test kits delivered directly to them or pick their test kits up from their nearest location of Duncan's Pharmacy. Other arrangements are possible.
As mentioned above, in some embodiments, one or more of the exchanges between the user and the testing platform as described above with reference to one or both of
In some embodiments, the example testing interface 1100 may comprise a session progress indicator 1102. The session progress indicator 1102 may provide a visual and/or audible indicator of the progress of a testing session. For example, the session progress indicator 1102 may show a step number, a timer, a timeline, or other indicator of progress. The session progress indicator 1102 may assist a patient in determining the current status of the testing session.
In some embodiments, the example testing interface 1100 may comprise testing instructions 1104. In some embodiments, the testing instructions 1104 may comprise step-by-step instructions that the patient must follow in relation to a provided at-home test kit in order to self-administer the test and achieve a valid test result. In some embodiments, the testing instructions 1104 may be read by a patient in addition to audible instructions provided by the platform or manually by a proctor. The proctor may be heard by the patient via a secure communication channel and in some embodiments, may be visible in a live proctor video feed 1108. In other embodiments, a virtual proctor avatar may be shown instead of the proctor video feed. In some embodiments, a prerecorded video may be displayed rather than a live proctor video feed.
In some embodiments, proctor information 1108 may be displayed and provide information to the patient regarding the proctor that is overseeing the testing session, including, for example, the proctor's name, certifications, and a photograph of the proctor, among others.
In some embodiments, the example testing interface 1100 may also comprise one or more options 1110, a support link 1112, and one or more call options 1114. In some embodiments, the options 1110 may comprise general platform options, including accessibility options, video options, and other miscellaneous options. In some embodiments, the support link 1112 may connect the patient to an administrator or other support personnel, in the event that such support is needed. For example, if there is an emergency with the proctor or patient, or there is an issue with testing, the support link 1112 may allow a third-party to join the testing session. The call options 1118 may, for example allow a patient to mute their device microphone or end the testing session.
In some embodiments, the example testing interface 1500 may comprise a session progress indicator 1502. The session progress indicator 1502 may provide a visual and/or audible indicator of the progress of a testing session. For example, the session progress indicator 1502 may show a step number, a timer, a timeline, or other indicator of progress. The session progress indicator 1502 may assist a proctor in determining the current status of the testing session, especially in situations in which the proctor is managing multiple testing sessions simultaneously.
In some embodiments, the example testing interface 1500 may comprise patient information 1504 and a patient video feed 1506. The patient information 1504 may provide information to the proctor regarding the patient that is the subject of a testing session, including, for example, the patient's name, date of birth, age, language, and a photograph of the patient, among other types of information. In some embodiments, the patient video feed 1506 may comprise a live video feed of the patient during the testing session. In some embodiments, the patient video feed may comprise a recording of the patient. In some embodiments, the patient video feed 1506 may allow a proctor to monitor a patient and testing session, and to direct the patient regarding the test protocol.
In some embodiments, the example testing interface 1500 may also comprise a prompt/script 1508, a notes/test result section 1510, and a patient view 1512. In some embodiments, the prompt 1508 may comprise a written script for the proctor to audibly read to the patient during the testing session. In some embodiments, the prompt 1508 may automatically update based on the current step of the testing session. In some embodiments, a navigation menu 1520 may be sued by the proctor to move forward and back through the testing steps. In some embodiments, the notes/test result section 1510 may comprise various notes for the proctor's consideration during the current testing step. During interpretation of a test, the notes/test result section 1510 may comprise a guide for test interpretation and/or a mechanism to submit a test result. In some embodiments, the patient view 1512 may comprise a section showing the content that is currently being displayed on a patient's computer. In some embodiments, the patient view 1512 may comprise a copy of the content displayed on the patient's screen or a live stream of the patient's screen.
In some embodiments, a prompt 1508 may comprise a written script on the interface, which will indicate, for each step of or whatever phase of testing the particular patient is in, the words that the proctor will read for that specific step. In some embodiments, the prompt may be automatically read by the health testing and diagnostic platform. In other embodiments, the proctor may read the prompt audibly to the patient. In some embodiments, a prerecorded video or dialogue may be played at specific times during the testing, which will comprise a prerecorded script comprising the prompt 1508. In some embodiments, the prerecorded script may be played under supervision of a proctor.
In some embodiments, the example testing interface 1500 may also comprise one or more options 1514, a support link 1516, and one or more call options 1518. In some embodiments, the options 1514 may comprise general platform options, including accessibility options, video options, and other miscellaneous options. In some embodiments, the support link 1516 may connect the proctor to an administrator or other support personnel, in the event that such support is needed. For example, if there is an emergency with the proctor or patient, or there is an issue with testing, the support link 1516 may allow a third-party to join the testing session. In some embodiments, a substitute proctor may be able to join the testing session to resume testing using, for example, the session progress indicator 1502 and the prompt 1508.
In some embodiments, the health testing and diagnostic platform may assign one or more designated users (e.g., supervisors or others) to have administrator rights and privileges. In some embodiments, an administrator may view the full scope of proctor sessions occurring, including a list of proctors active on the platform and pending test sessions. In some embodiments, an administrator can join into a testing session anonymously and quietly as a third-party overseer, for purposes such as random audits. In some embodiments, administrators may join testing session for joint proctor training or feedback. In some embodiments, in an emergency situation, the support link 1516 may connect the proctor to a customer service team, wherein the customer service team may be sent part of or the whole dataset of a testing session, including the user information, the user picture, and proctor information.
In some embodiments, the test analysis and interpretation interface may comprise one or more interactive tools, which the proctor may utilize to ensure test integrity. For example, one tool that may be used would be an interactive pen/marker/highlighter that can be used to annotate the patient video feed to guarantee that the test is not moved or altered.
In some embodiments, a patient video feed or test image is not optimal such that the result may be difficult to interpret. For example, the patient's video may be too bright or dark, may be blurry, or otherwise lack sufficient visibility. Thus, in some embodiments, the test analysis and interpretation interface may comprise one or more image adjustment tools to manipulate image properties such as the contrast and brightness. The image adjustment tools may be used to adjust the image to enrich the view of the test. Example image adjustment tools include, for example, brightness, contrast, saturation, and color adjusters, among others.
In some embodiments, instead of utilizing live video of a patient test, the health testing and diagnostic platform may capture one or more static images of a patient test from the live video feed. In some embodiments, the static image may be captured directly from the user device of the patient. In some embodiments, the test image can be sent as a full-size file over the internet to the proctor and viewed on the test analysis and interpretation interface. In some embodiments, the image can be adjusted using the one or more image adjustment tools. In some embodiments, using an image captured using the user device may result in an image that has enhanced resolution and clarity relative to using a live image streamed over a video conferencing network. In some embodiments, the image of the test obtained from the user device of a patient may be captured during a patient video session with the proctor, but is not captured from the live video stream.
In some embodiments, the multi-session testing interfaces described herein may allow for concurrency of about 2 to about 20 patient testing sessions for a single proctor. In some embodiments, the multi-session testing interfaces may comprise a workflow timeline that identifies patient testing progress for pending testing sessions, such that a proctor can immediately understand the progress of each pending session and switch between each isolated session as needed. For example, for some disease testing, there may be a testing cure time during which the patient and proctor are waiting for the test results. During this time, the multi-session testing interface may allow a proctor to switch an active session to another patient at a different point in the testing process such that multiple sessions can be managed simultaneously. The multi-session interfaces may introduce a supply line of isolated patient video sessions such that the patient sessions are stacked efficiently and proctor availability is maximized. In some embodiments, the multi-session interfaces may comprise a management tool that allows proctors to track the concurrent session timeline and to understand the upcoming timing requirements of each isolated session and the sessions collectively.
In some embodiments, each patient testing session enables a separate audio/video channel such that patients are isolated from each other and cannot hear or see other patient testing sessions, while the proctor may activate each individual patent testing session to communicate directly with each patient. Thus, the audio/video channels for each patient are segmented into separate feeds such that individual sessions are distinctly managed separately with separate session IDs and in such a way that none of the patients can talk to each other or are even aware that multiple sessions are occurring concurrently. In some embodiments, a proctor may activate a specific patient testing session through an activation mechanism on the proctor's screen, such as by selecting a patient video feed. The activation mechanism will open the audio and/or video channel with the specific patient for which the patient testing session has been chosen. This configuration minimizes the risk of exposing sensitive medical or personal data between patients; only the proctor may communicate with each patient. Furthermore, the multi-session interfaces described and shown herein facilitate efficient session switching through visual prompts that allow a proctor to understand the point at which each testing session is currently progressed.
In some embodiments, the multi-session interfaces may comprise individual patient mute mechanisms, which may close the communication channel with an individual testing session, as well as a global mute mechanism, which may close all communication channels between the proctor and all pending testing sessions. In some embodiments, only one patient testing communication channel may be opened at a time, such that a proctor may only communicate with a single patient at a single point in time. For example, opening a communication with one patient may automatically close (e.g., mute) a communication channel with all other patients to ensure that privacy between patients is maintained.
The multi-session interface of
Expedited or Express Proctoring Platforms for at-Home Testing
The platforms and systems described throughout this application can provide a number of advantages relating to the administration and proctoring of at-home health testing and diagnostics. For one, such platforms and systems provide users with the option to perform health testing and diagnostics from the comfort of their own home or otherwise without having to travel to visit a healthcare professional in person. This in itself can provide a number of benefits including, saving time, increasing availability of testing, and decreasing the risk of exposure to and/or spread of an infection disease, among others.
An additional benefit that can be achieved using the platforms and systems described herein is that these can be optimized to provide for a high throughput of user tests, while at the same time minimizing the resources required to run the platform. For example, as described above, the platforms and systems described herein can be configured to allow a single proctor to proctor a plurality of user tests at once. This can provide advantages both for the platform and for the users. For example, the platform can facilitate testing using fewer proctors, thus leading to a reduction in operating costs. At the same time, the testing and diagnostics provided through the system and platforms described herein can still be readily available to a large number of users and wait times for entering a proctored test session can be reduced. Additional advantages achieved by some embodiments described herein may include reduction in costs associated with operating a testing platform (e.g., through a reduction in operating and/or oversight costs) and/or reduction in patient costs (e.g., by having them avoid unnecessary travel costs). Additionally, some embodiments may allow users to multi-task while testing, thereby increasing flexibility for the users.
These and other benefits of the platforms and systems described throughout this application can be further achieved or even improved by, in some instances, offering certain users a more streamlined or express test session. As will be described in more detail below, such express test sessions can, for example, require less proctor involvement and/or decrease the duration of the test session for the user when compared with standard (e.g., non-express) test sessions. This can, in some instances, further increase the benefits and advantages associated with the systems and platforms for proctoring at-home health testing and diagnostics by further improving the user experience (e.g., by decreasing test session duration) and/or maximizing the availability and reach of the platform (e.g., by increasing the platform's ability to provide a high number of test sessions using fewer proctors).
Accordingly, in some instances, the platform can be configured to tailor a proctoring session to a user's level of test-taking experience. For example, users with less experience with the test and/or platform can be directed into a standard proctoring session, while users with more experience with the test and/or platform can be directed into an express proctoring session. As an even more specific example, users who are using the platform to take a test for the very first time (“first-time users”) may be directed to the standard proctoring session (where, for example, a proctor provides full-fledged step-by-step instructions for taking the test), while users who have previously used the platform to take tests (“repeat users”) may be given the option to either participate in (i) the standard proctoring experience, or (ii) an express or expedited proctoring session where the users take the test with reduced proctor participation/intervention.
As will be described in more detail below, various factors can be considered by the system in determining whether to offer or provide a user an express testing session. In some embodiments, the factors used by the systems disclosed herein to determine eligibility for express testing session can include one or more of the following: a test taker's past experience in taking such a test, a test taker's age, a test taker's language skills, a test taker's ability to follow instructions (for example, whether a system detects that the patient has completed certain initial tasks of the test within a certain period of time and/or completed certain initial tasks competently and/or with minimal supervision and/or within a minimum number of attempts, and in some embodiments, one or more of the foregoing factors can be used by the system to enable the system to provide real-time or substantially real-time grading of a user to determine eligibility of the user to take and/or be offered an express testing session), a test taker's computer literacy skills, a test taker's internet connection speed and/or connection quality/stability, a test taker's ability to have a physically present third-party to observe the test taker, an artificial intelligence algorithm used to observe the test taker's skills and/or ability to follow instructions and/or administering the test (for example, the use of artificial intelligence algorithms to determine whether the test taker has inserted a swab to a minimum instructed depth into a nasal cavity (or other orifice), and/or whether a test taker has completed a minimum number of rotations of the swab in the nasal cavity (or other orifice), or the like). In some embodiments, a factor in determining a test taker's eligibility for taking an express testing session can include but is not limited to whether the test taker has paid a necessary fee, and/or the number of other patients in a queue waiting to take a test, and/or the number of available proctors administering tests, and/or whether the test taker has a high priority flag setting in the users profile, and/or whether the user has a time commitment factor (for example, the user is about to board an airplane flight), and/or the location of the user, and/or the local time of a user, and/or whether a doctor has provided expedited instructions, and/or a test takers determined anxiety level, and/or the like. These factors can be selected so that a high degree of testing fidelity and accuracy is maintained. That is, the system may offer express testing sessions only to users for whom, upon consideration of one or more factors associated with the user, the system determines that there is a high probability of the user performing the test accurately with a reduced amount of supervision. At the same time, the system can be optimized such that express testing sessions are still reviewed by proctors and/or other components of the system to ensure sufficient testing accuracy.
Before considering eligibility factors and system requirements for ensuring testing accuracy, several aspects of an express testing session will now be described with the understanding that not each and every aspect need be present in all embodiments and that various modifications are also possible. In general, express testing session can allow a user to perform a test with reduced proctor involvement when compared to standard testing sessions. For example, during a standard testing session, a proctor may walk a user through each step of the process in great detail, including even mundane or simple steps such as opening and arranging the testing materials. By contrast, in some embodiments, during an express testing session, the user may be only engaged by the proctor, if at all, during more complex or critical steps of the test. During an express testing session, for example, a user_may be allowed to perform various steps of the test and proctoring session with little-to-no proctor intervention. By decreasing proctor involvement, in some instances, testing duration may be decreased and/or testing capacity of the system can be increased.
In some embodiments, a script or prompts provided to a proctor (to be read to a test-taking user) may be different or customized based on the type of session being proctored. For example, a longer script or set if prompts may be provided for standard testing sessions, while a shorter script of set of prompts may be provided for express testing sessions. For example, in some embodiments, in an express testing session, the proctor may initially instruct the user to (i) perform the test and (ii) let the proctor know when the user is ready to submit their testing results (which may still ultimately be interpreted by the proctor). In this case, proctor engagement with the user is limiting to an initial instruction at the start of the session and verification of the testing result at the end of the testing session. Other levels of proctor involvement can also be provided in an express testing session. For example, in some embodiments, proctor engagement only begins when the user has completed the test and needs for the result to be verified. In these embodiments, proctor involvement may only occur at the end of the testing session. In some embodiments, even though proctor engagement may be limited, one or more proctors may still observe the user throughout the test (or during certain portions of the test) so as to be able to ensure that the test is performed properly and step in with corrective action or additional instructions if necessary.
As another example, proctor involvement can be required only for certain or critical testing steps. In the case of a nose swab, for example, during an express testing session, a proctor may be required to watch the swab occur so as to ensure that a sufficient swab depth is reached. In some embodiments, the user, during an express testing session, may be provided with written and/or prerecorded video or audio instructions and the proctor may only be involved with verifying that the steps are accurately followed. By contrast, in some embodiments of a standard testing session, a live proctor may read or walk a user through the instructions before watching them perform the steps. Decreasing proctor involvement with express testing sessions can increase proctor capacity and improve overall throughput and efficiency of the system, allowing proctors to focus more of their time on new or inexperienced users that are unfamiliar with the process, while allowing repeat or experienced users to perform certain steps on their own.
In some embodiments, during express testing sessions, a user may not be required to remain on camera for the entirety of the session. For example, the user may only be required to appear before the camera immediately before and after taking the test and/or during certain or critical testing steps. In some embodiments, the user may need to appear on camera prior to taking the test in order to verify their identity. In some embodiments, the user may need to appear on camera after taking the test to have the result verified by a proctor. In some embodiments, the user may need to appear on camera during certain or critical testing steps so that the system or a proctor can ensure that these steps are performed with sufficient accuracy. In some embodiments, during an express testing session, the entire session (or certain critical steps) can be recorded such that the testing session can be audited to ensure accuracy. Such auditing can be performed by a person (e.g., a proctor reviewing the video) or by the system (e.g., by using artificial intelligence or machine learning). In some embodiments, during an express testing session that is recorded, certain steps may be required to be viewed live by a proctor. These and other techniques which do not require a live or concurrent connection to the platform and/or a proctor at all steps of the testing session can be beneficial for several reasons. For example, this can permit reduction in power and/or bandwidth consumption. Additionally, these techniques can permit testing where network access is limited and/or unreliable.
In some embodiments, no live proctor interaction with the user may be required at all. This may not mean that no proctor is involved in the test. For example, a proctor may supervise (e.g., watch video of) express testing sessions without interacting with (e.g., speaking to) the user. In such cases, the proctor may only engage with the user when or if proctor engagement is requested by the user and/or if the proctor needs to intervene to ensure that the testing is performed accurately. By not requiring the proctor to interact with users unless interaction is determined to be necessary, a proctor may be able to supervise a higher number of testing procedures at once. This may also provide additional benefits such as widening the range of environments where testing could occur. For instance, if proctors no longer need to be spoken to or heard due to the express testing format, users can take tests more easily in noisy airports, or even at home, work, or other locations without interrupting or disturbing those around them. Additionally, as mentioned above, this can reduce a bandwidth requirement associated with testing, making testing more accessible by allowing more users to take tests on slower and/or less reliable networks, such as 4G or other slower connection speeds.
During express testing sessions, one or more proctors may be on standby to answer any questions that a user might have and provide general testing oversight. Proctors may step-in or otherwise intervene during an express testing session if they spot the user performing a step incorrectly or otherwise require additional information and/or participation from the user. As one example, a proctor may view a plurality of express testing sessions at once and only intervene with any particular user when necessary. In some embodiments, the proctor viewing the plurality of express testing sessions may not be the same proctor as the proctor that intervenes. For example, a proctoring monitoring a plurality of express testing sessions may realize that a certain user needs assistance and flag that user for intervention by another proctor. In some embodiments, an express session proctor may move a user from an express session to a standard session if it is determined that the user needs additional assistance. In some embodiments, an express session proctor may intervene and engage with an express session user directly, as needed, and maintain the user in the express session.
Determination of eligibility for participation in an express testing session can be determined, in some embodiments, using a score-based approach. A wide number of factors can be considered in determining a patient's express testing session eligibility score. For example, during a standard (or even an express) testing session, proctors may “rate” or otherwise provide an indication of their evaluation of a user's test-taking proficiency at the end of the proctoring session. Such ratings can be considered in determining whether future testing sessions of the user are eligible for participation in an express testing session. In some embodiments, proctor scores of user proficiency may be provided on a scale (e.g., a scale of 1-10, where one indicates that a user requires significant assistance and 10 indicates that a user requires little to no assistance, a star-rating system (e.g., 1-5 stars), etc.). In other embodiments, proctor scores of user proficiency may comprises a binary indicator of whether the user is proficient/competent or not. Each system (scale-based or binary) can provide advantages in certain scenarios. For example, a scale-based system may provide data granularity that may allow the system to make more nuanced determinations of eligibility, while the binary system may be simpler to implement for proctors.
In some embodiments, users may also provide an indication of their own proficiency and/or testing comfort upon completion of testing session (express or standard), using either a scale-based or binary system. Of course, care should be taken to ensure that users ratings of themselves are accurate. This can be achieved by, for example, comparing the user-provided score with the proctor-provided score. Alternatively or additionally, the platform may weigh the user's self-ratings against self-ratings of other users in order to calibrate and standardize self-ratings across the user-base. For example, if the user consistently rates themselves highly compared to other similar users (as determined by the system or other proctor-defined skill ratings), the system may adjust the user's self-ratings with a scaling factor to standardize self-ratings across the platform. In some embodiments, a user's self-ratings can also be compared against his or her own past self-ratings. For example, if a user that has an average self-rating of 5 after a number of past tests gives him or herself a self-rating of a 4 in a current test, the system may determine lower the user's express testing eligibility since the change represents a large deviation from past self-scores. This may occur even though a self-rating of 4 for another user, who consistently self-rates at 4, may remain eligible for express testing. Alternatively or additionally, this can be provided using machine learning and/or artificial intelligence by comparing the user-provided score against one or more additional factors determined by the system. Such factors could include for example, machine learning- or artificial intelligence-based analysis of a recorded video of patient's testing session, analysis of durations taken by the user in performing various steps, etc. In some embodiments, the user may be given the opportunity to formally (or informally) attest to the truthfulness and accuracy of their self-evaluation and/or agree to make a good faith effort to complete every test administered on the platform in accordance with the corresponding procedures, requirements, and guidelines.
Alternatively or additionally, a number of other factors could be used to determine the user's score for eligibility to participate in an express testing session. For example, the score could be determined based, at least in part, on a number of times the user has taken the test, how recently the user last took a test, etc. From this, it is apparent that express testing sessions would be beneficial for frequent testers. For example, a frequent traveler may take tests often, becoming very familiar with the process. On the other hand, users who seldom take tests or have not taken a test in a relatively long time may be less familiar with the process, and thus more likely to benefit from being reacquainted with the process by way of the standard testing session. Upon consideration of how many times, how frequently, and/or how recently (possibly in conjunction with proctor-provided and/or user-provided ratings and/or other factors), the system may determine that the user is eligible for participation in an express proctoring session.
In some embodiments, the score may also be determined based, at least in part, on an assessment of how similar a user's current testing environment is to the user's previous testing environment. In general, a user repeating a test in an environment in which they have previously taken the test or an environment similar to an environment in which they have previously taken a test, may be eligible for or receive in increased eligibility score for using an express testing session. To this end, in some embodiments, the user's eligibility score considers the user's IP address and/or GPS data to determine the user's location, which is compared against previous testing locations. Similarly, the type of user device the user is using may be identified and compared to the type of user device that the user has used previously. For example, a user who previously performed the test using a laptop, but is now attempting a test using a mobile phone, may not be eligible for express testing until they have qualified using a mobile phone.
In some embodiments, responses provided during a prequalification survey (such as the surveys described above) can also be considered in determining whether a user is eligible for an express testing session. For example, during a prequalification survey a user may verify that he or she is a healthcare professional that is competent to administer their own test. This may qualify the user for an express testing session.
A score for each user can be determined and associated with that user's account. If the score generated for the user exceeds a predetermined threshold at the time the user signs in and attempts to initiate a proctoring session, then the platform can give the user the option to enter an express testing session or to enter a standard testing session. The user's score can be continually updated based on subsequent testing experiences.
A user may, in some cases, be able to test in to establish eligibility for express testing sessions. For example, a user desirous of eligibility for express testing can elect to take a test to show that they are eligible for express testing. Such testing can involve a standard testing session or a modified testing session.
The systems and platforms for at-home health testing and diagnostics described herein can be configured to allow at-home testing for a wide variety of health and diagnostic tests. In some instances, a user score that establishes eligibility for express testing sessions can be specific to a certain test. In other instances, a user score that establishes eligibility for express testing can carry over to different types of tests available on the platform. For example, a user who has taken a nasal swab-based test several times and is eligible to use express testing session for that test may also be eligible to use an express testing session when taking a mouth swab-based test, even, in some cases, for the first time the user takes the mouth swab-based test. In other cases, the user may become eligible for express testing for new tests only after taking the new test using a standard testing session. Still, the user's score could decrease the requirement for the user to become express eligible for the new test. For example, the user (who has experience on the platform with a nose swab-based test) could become eligible to use an express testing session to take the mouth swab-based test after only taking said test once (whereas a user who has never used the platform may be required to successfully take the mouth swab-based test three times before becoming eligible for express testing sessions).
In some instances, however, in some examples, a same user may not enjoy the same eligibility reciprocity/recognition when trying a test that is radically different from the nasal swab-based test, such as a test where blood sample collection is required. For example, a user's express testing session eligibility score may be associated with a specific test (e.g., a nose-swab), a specific type of test (e.g., swab-based, whether nose or mouth, or minimally invasive tests), or with all tests available on the platform. In some embodiments, the user has the option whether to enter an express testing session or a standard testing session when taking a test, such that a qualified user has the opportunity to choose the most convenient and comfortable session for them.
In some cases, a user's express testing session eligibility can be revoked for testing infractions (e.g., performing one or more steps incorrectly, trying to rig the test, submitting fraudulent test results, etc.). This can be done to maintain the integrity of the platform as a whole. Accordingly, even in express testing sessions, it can be important to have a level of oversight and/or verification, which can be provided live (concurrent with the testing session) or after the fact (by, for example, reviewing recording, photos, or other data collected during the testing session). Moreover, the oversight and/or verification can be provided by live (e.g., proctor-based) or artificial intelligence- or machine learning-based review.
For implementations in which a score is generated for each user, a given user's score may be negatively impacted for testing infractions, which could lead to the user's express testing eligibility being revoked (e.g., as the score drops below a predetermined threshold value). More severe infractions can more negatively impact a user's score. Additionally or alternatively, a duration of the revocation could, in some examples, vary based on the severity or nature of the infraction, the user's history of infractions (e.g., a three strikes and you're out type policy), etc. As an example, a user who attempts to submit fraudulent test results may have their express testing eligibility (or even their standard testing eligibility) permanently revoked, while a user who forgets to wash their hands before taking a test may only have their express testing eligibility temporarily suspended (e.g., the user may be required to participate in the standard proctoring session the next time they take a test using the platform, but may regain eligibility immediately thereafter). In some instances, the user may be notified of the reasons associated with loss of express eligibility (or a decrease in their user score) and given opportunities to remediate the cause of the infractions. For implementations in which a score is generated for each user, the extent to which a user's score is negatively impacted by testing infractions may vary based on such factors as type and severity of the infraction.
In some embodiments, users who are already established as eligible for express session testing may be required to participate in the standard proctoring session periodically, even in cases where no testing infractions have incurred. For example, users that would otherwise be eligible for express testing session may be required to participate in the standard proctoring session the next time they take a test using the platform if, for example, the test kit has changed since the last time they took the test (e.g., the packaging has changed, the shape/design of items in the test kit have changed, etc.). As another example, users that would otherwise be eligible for express testing session may be required to participate in the standard proctoring session the next time they take a test using the platform if the testing procedure and/or FDA requirements have changed since the last time they took the test (e.g., users were previously required to add six drops of reagent solution to a test strip, but are now required to add eight drops of reagent to the test strip for enhanced accuracy). As another example, users that would otherwise be eligible for express testing session may be required to participate in the standard proctoring session the next time they take a test using the platform if the proctoring session protocols have changed since the last time they took the test (e.g., users were previously asked to hold the testing materials up to the camera on their laptop to show the proctor their results, but are now required to instead leave the testing materials on the table, snap a photo of the testing materials on their smartphone, and submit the photo to the proctor through a downloaded or web-based version of an application on their smartphone).
Additionally or alternatively, users that would otherwise be eligible for express testing session may be required to participate in the standard proctoring session periodically or randomly to ensure eligibility for express testing.
Although generally having been described as a binary system of express testing sessions and standardized testing sessions, this need not be the case in all embodiments. For example, express testing may involve modification or expedition or one or more of the steps of a standardized testing session. That is, in some embodiments, hybrid testing sessions may be provided that are tailored to the user's level of test-taking experience on a step-by-step basis. Referring once again to the example described above in which users were previously required to add six drops of reagent solution to a test strip, but are now required to add eight drops of reagent to the test strip for enhanced accuracy, in some embodiments, the proctor may only provide the “standard” proctoring experience for the step of the test in which the user is required to add the reagent solution to the test strip, and may otherwise provide an express proctoring experience for the other steps of the test (which have not changed). As another example, an express testing experience can be provided for simpler steps, while a standard, more involved testing experience can be provided for more complex or critical steps. In some embodiments in which an express testing session includes both steps that are not and steps that are directly monitored by a live proctor, the system may be configured to provide a cue (e.g., a visual or audio cue) to identify or distinguish between these steps. For example, when a user arrives at a step that requires proctor involvement, the user may be alerted (via the cue) to wait for proctor engagement.
In some embodiments, a user that has qualified for express testing may be given a “badge” or other indicator that appears next to their username when they become eligible for express testing sessions. In some cases, one set of proctors give standard testing sessions and another set of proctors give express testing sessions. In other cases, proctors may give both standard and express testing sessions simultaneously. In these cases, a badge or visual indicator may help visually distinguish express users from standard users. Additionally, in some embodiments, upon qualification for express testing (or upon disqualification), the user is notified of the change in status. Such notification can occur, for example, during a testing session, after a testing session, or at the start of a subsequent testing session.
User-Proctor Matching
As described previously, the testing platform described herein can be configured to allow for a plurality of users to perform remote health and diagnostic testing under the supervision of one or more proctors. As users access the platform, they can be matched with certain proctors based on various criteria selected to improve the users' testing experience and/or to improve the efficacy of the proctors. In some embodiments, users accessing the platform are placed in a user pool and proctors accessing the platform are placed in a proctor pool. Various criteria can be determined for each of the users and for each of the proctors. These criteria can then be analyzed to match or assign users to certain proctors. For example, such criteria can be evaluated at 116 of
Users may be matched with or assigned to proctors based at least in part on one or more of a variety of different factors. Such factors may include, for example, the languages spoken by the users and the proctors, user disabilities and/or special needs, user and proctor location, types of tests to be taken or administered, a user's level of technical proficiency, proctor availability, strength of network connection, and/or a level of testing, verification, or scrutiny desired or required.
For example, users may be matched or assigned to a proctor based on a language spoken by the user. The testing platform may be provided in multiple languages and available all over the world. Accordingly, users accessing the platform may speak a variety of different languages. To facilitate testing, proctors may be provided in languages that correspond to the languages of the user. When an English-speaking user accesses the platform, they may be assigned to an English-speaking proctor. When a Spanish speaking user accesses the platform, they may be assigned to a Spanish speaking proctor. This can ensure that the proctored testing session can run as smoothly as possible as communication barriers between the users and the proctors can be reduced.
In some embodiments, the language desired by the user for the proctored testing session may be indicated by the user. For example, when accessing the platform, the user may create a user profile in which the user's language preference is indicated. In some embodiments, multi-lingual users may indicate that they can speak more than one language and may even indicate a proficiency for each language. When matching such users to proctors, each language and proficiency may be evaluated in view of the available proctors in order to find a matching proctor as quickly as possible. For example, if a user indicates that he or she is a native Spanish speaker but is also fluent in English, the platform may first check to see whether a Spanish speaking proctor is available. If a Spanish speaking proctor is available, the user may be matched to the proctor. If a Spanish speaking proctor is not available, the platform may then check whether an English-speaking proctor is available, and if so, match the user with the English-speaking proctor.
In some embodiments, the platform may attempt to automatically determine the user's language. For example, the platform may analyze the user's network connection (e.g., the user's IP address) in an effort to determine where the user is geographically located. If, for example, the user's IP address is associated with a typically English-speaking geographical location, the user may be initially presented with an English-speaking proctor. If the user's IP address is associated with a typically Spanish speaking geographical location, the user may be initially presented with a Spanish speaking proctor. Other factors may also be analyzed in an effort to automatically determine the user's language. For example, an estimate of the user's language may be made based on a top-level domain associated with the testing platform's website accessed by the user. For example, if a user accesses the testing platform at www.[testingplatform].mx (.mx being the top-level domain associated with Mexico), the testing platform may make an initial determination that the user likely speaks Spanish. In such cases where the platform attempts to automatically determine the user's language, the user may be given the option to change to a more preferred language in the event that the automatically determined language is incorrect.
Proctors may also be associated with profiles that provide information about the proctors that is used to match the proctors to the users. A proctor profile can include information about which languages are spoken by each proctor as well as a proficiency score for each language. In some embodiments, proctors can specify which languages they speak as well as their proficiency in each language. In some embodiments, proctors may be required to take language tests to ensure that they are sufficiently proficient to provide a proctored test in a given language. In some embodiments, the platform may analyze proctor performance and adjust a proctors proficiency score for each spoken language. For example, a proctor may indicate that he or she is fluent in English and Spanish. The platform may thus assign both English speaking users and Spanish speaking users to the proctor. The platform may analyze the proctor's proficiency in providing proctored testing sessions in English and Spanish. For example, the platform may track an average testing session length for both the English and Spanish sessions. Upon determination that the proctor takes sufficiently longer to provide a proctored testing session in one language when compared to the other, the platform may lower the user's proficiency score for the slower session language. Additionally or alternatively, the platform may evaluate testing accuracy for each of the proctor's spoken languages. For example, if it is determined that tests administered by the proctor in Spanish generally are more accurate than tests administered by the proctor in English, the platform may assign more Spanish speaking users to the proctor. As a result, the platform may assign more users who speak the proctor's faster language to the proctor in the future. In some embodiments, in addition to or in place of comparing the proctor's proficiency against itself (e.g., comparing the proctor's English proficiency against his or her Spanish proficiency), the platform may also compare the proctor's proficiency against that of other proctors. In this way, the platform can dynamically balance matching users to proctors in the most efficient manner.
Another factor that may be considered by the platform in matching or assigning users to proctors can be disabilities or special needs of the users. For various reasons, some users may require special accommodations in order to take a test. The platform can consider the users' needs and provide the user with a proctor equipped to provide the necessary accommodations. As one example, a user that is hard of hearing or deaf can be assigned to a proctor that is proficient in sign language or to a proctor that is proficient in communicated with the user through text. Users accessing the platform may provide information about any special needs or disabilities as well as desired or necessary accommodations that must be made to facilitate the testing session. For example, such information can be entered during creation of the user's profile or account or gathered during a pre-qualification survey, for example as described at 104 and 106 of
In some embodiments, the platform may match users to proctors based on the location and/or time zone of the users and proctors. This may improve the testing experience by matching users to proctors that are close to them geographically or in the same time zone.
In some embodiments, proctors and users can be matched based on the type of test to be taken. As noted above, the platform may be configured to allow for a wide variety of remote health tests or other diagnostics. While users accessing the platform may generally do so in order to take a specific kind of test, proctors may be able to administer more than one type of test. For example, a proctor may be able to provide a proctored testing session for a first type of test and also to provide a proctored testing session for a second type of test. Proctor profiles may include a list of each type of proctored testing session that can be provided by a given proctor. In some embodiments, proctors may need to take specific training and/or receive certain verifications to provide different types of proctored testing sessions.
In some embodiments, proctors that are certified or otherwise able to provide different types of proctored testing sessions can be evaluated by the platform such that users are assigned to the proctor in an efficient manner. For example, if both a proctor A and a proctor B are capable of administering a first type of proctored testing session and a second type of proctored testing session, the platform may evaluate proctor A and B's proficiency with each type of testing session and match users accordingly. For example, the platform may determine that proctor A provides the first type of proctored testing session more quickly than proctor B does. The platform may then assign users taking a test associated with the first type of proctored testing session to proctor A and assign users taking a test associated with the second type of proctored testing session to proctor B.
Another factor that can be considered by the platform in assigning users to proctors may be the users' technical ability or proficiency. As an example, an older user may not be as technically proficient as a younger user. Certain proctors may have more proficiency in providing proctored testing sessions to such older users. Accordingly, the platform may assign such older users to such proctors. In some embodiments, the users themselves may provide an indication of their technical proficiency. For example, during user account creation or a pre-qualification survey, the user may input a technical proficiency score or a comfort score associated with using the remote testing platform. In some embodiments, the platform may make a determination about a user's technical proficiency based factors observable by the system. For example, the system may make a determination about a user's technical proficiency based on an amount of time take by the user to set up his or her account or to complete the pre-qualification survey. As another example, the platform may make a determination about the user's technical proficiency based on the user's age or other factors. In some embodiments, the system may automatically administer a short test to the user to determine technical proficiency, for example, by checking whether the user is able to operate his or her webcam, microphone, and speakers.
As before, the platform may continually or periodically evaluate proctor proficiency in providing proctored testing sessions to less technically proficient users. For example, the platform may analyze elapsed session times or test result accuracy for various proctors and assign less technically proficient users to proctors that can most efficiently and accurately administer tests to such users.
The platform may also consider the strength of users' and proctors' network connections in determining how to assign and match the users and proctors. For example, the platform may evaluate or determine an internet speed, latency, and/or reliability of the network connection for each user and proctor. This information can be used to match proctors and users in a manner that ensures that a stable connection between a user and proctor can be maintained during a testing session.
In some embodiments, the platform may also consider a level of testing verification or scrutiny that is required or desired in determining how to assign and match users to proctors. For example, in some instances, a certain level of proctor scrutiny or involvement may be required to complete and validate a test result. For example, government or other regulatory bodies may require certain steps to be completed in order to validate a test. In such cases, users can be matched with proctors that are capable of providing the needed level or scrutiny. In other example, users may request a certain level or proctor involvement or scrutiny. For example, a user that is generally unsure or uncomfortable with remote testing procedures may request a more involved proctoring session.
Accordingly, users who require or desire a proctoring session with additional verification measures or a higher level of testing scrutiny may be matched with proctors who conduct one-on-one proctoring sessions and/or can provide additional verification services. As one example, users who need verified passes in order to comply with CDC rules or laws may be matched with proctors capable of providing such services. As another example, in some embodiments, users may pay for a premium membership/service for the platform that automatically communicates the users' test results to user-specified entities. For example, the platform can provide users with a with a TSA Pre-Check-like pass service where the user provides additional verification information and performs additional verification processes with the proctor in exchange for a pass that allows users to “skip the line” or otherwise more seamlessly travel or access various venues and other locations.
The system may also consider proctor availability in matching users and proctors. As noted above, the platform may consider various factors in matching users and proctors to improve the efficiency and accuracy of the platform. In some embodiments, however, the most efficient proctors may not be available. Accordingly, the platform can further consider proctor availability. For example, in some embodiments, it may be faster to assign a user to a less proficient proctor with a shorter wait time, rather than assigning the user to more proficient proctor with a longer wait time.
The platform may dynamically match proctors and users upon consideration of one or more of the factors described above as well as others. Further, the platform may continually or periodically evaluate the efficiency and accuracy of proctors based on each of these factors and use this information to continually rebalance and recalibrate the platform to maximize efficiency. For example, the platform may adjust how users are assigned to proctors to reduce testing session times and/or improve testing session accuracy by assigning proctors to users in the most efficient manner.
Additionally, the order in which users are placed in queues for meeting with proctors may be determined based at least in part on one or more of a variety of factors. As noted above, in some instances, a user may be placed into a waiting room while waiting for a proctor to be assigned or while waiting for an assigned proctor to become available (see, for example,
For example, a user that is about to board a flight or travel across state lines or other boarders may be given an advantageous place in the queue in an effort to assure that the travel can be completed. As another example, a more at-risk user may be given an advantageous place in the queue to minimize his or wait time. An at-risk user may be one that has a pre-existing condition or is elderly. For example, the elderly and people with certain health conditions may be placed ahead of others in the queue in order to ensure that they can be tested as soon as possible. As another example, a more symptomatic user may be given an advantageous place in the queue. People who are exhibiting worrisome symptoms may be placed ahead of others in the queue so that they can receive the medical attention they might need sooner. This can improve health outcomes for the particular patient as well as minimize the spread of a contagious disease to others.
A user's place in queue may be assigned in part based upon whether or not the user has a test pass that is currently valid and/or the amount of time remaining until expiration of a user's current test pass. For example, people who do not have test passes that are currently valid and people who have test passes that will be expiring shortly may be placed ahead of others in the queue. The user's occupation or other status may also be considered in determining queue placement. For example, doctors or medical personnel may be placed ahead of others in the queue. Additionally, time spent waiting in queue may also be considered. For example, users who have not been waiting for very long may be placed behind users in the queue who have been waiting for a while. In some embodiments, the platform may provide premium memberships or services that can affect a user's placement in the queue. For example, users that pay for a premium membership or service may be placed ahead of others in the queue.
The user-proctor matching and user queuing techniques and features described above have generally been presented in the context of assigning users to proctors for proctored based testing sessions. However, in some embodiments, different proctors may be provided for different portions of a proctored testing session. Accordingly, the user-proctor matching and user queuing techniques and features described above can be implemented at different stages in proctored testing session along the process. For example, a user may be shifted to different proctors throughout a test, with different proctors providing different stages or steps in the process. Proctors can be evaluated for proficiency and accuracy in providing the different stages of the tests (in some embodiments, for different types of users), and users and proctors can be matched accordingly.
In some embodiments, some phases or steps in a testing process may require more direct supervision, while other stages may require less supervision. In some embodiments, during stages that require more direct supervision a lower ratio of proctors to users may be used, while in stages that require less supervision a higher ratio or proctors to users may be used. Proctors can be evaluated to determine whether they are more proficient in proctoring the steps that involve supervision of a higher number of users or in proctoring the steps that involve supervision of a lower number of users, and the platform may match users and proctors accordingly.
At 2706, proctors access the platform. The proctors may access the platform remotely over a network using a proctor device as described above. At 2708, information regarding the proctor may be determined and stored in a profile associated with the proctor. In some embodiments, proctor profile information is input by the proctor. In some embodiments, proctor profile information is determined by the platform, for example, based on previous proctored testing sessions provided by the proctor. In some embodiments, the information about the proctor may include, for example, information about languages spoken, disability accommodations the proctor can provide, location and/or time zone, types of test to be administered, and/or information about the proctor's network connection (e.g., speed, latency, reliability, etc.), among other types of information.
At 2710, users and proctors are matched for proctored testing sessions based on an analysis of one or more matching criteria by the platform. For example, the platform may consider the information stored in the user and proctor profiles and match users to proctors in a manner that maximizes the efficiency of the system as described above. At 2712, a proctored testing session is provided to the user based on the matching established at 2710. At 2714, the efficiency and/or accuracy of the proctored testing session can be analyzed by the platform and used to update the user profile, at 2716, and/or the proctor profile at 2718. For example, if the testing session takes longer than expected or desired, the proctor profile may be updated such that the proctor will not be matched with that type of user in the future. In this way, the platform can continually evaluate proctors and users and match them in a manner that maximizes the efficiency of the system.
Test Integrity and Verification
In some embodiments, during a remote testing procedure, various steps can be implemented before, during, and after the test in order to ensure test integrity and verify the result of the test. For example, before the test is taken, it can be desirable to verify the user's identify and to verify that the test kit is valid (e.g., to verify that the test kit is one that has been purchased or otherwise obtained through an official channel, has not expired, and has not been previously used). During the test, it can be desirable to verify that any sample obtained is actually obtained from the previously verified user and that all test instructions and procedures are followed correctly. Finally, after the test, it can be desirable to ensure that the test result is obtained from the previously verified test kit and to ensure that the test result is interpreted correctly.
As noted above, it can be important to verify the user's identity before the test is taken. This can be because it ensures that the results will be attributed to the correct person. Verifying the user's identity is particularly important in cases of remote testing as the person being tested is not physically located with the person administering the test. Thus, in many situations, extra precaution may be advantageously taken to correctly identify the user's identity. Verification of the user's identity can be accomplished in several ways. For example, the user can be asked to upload a copy of an official identification (e.g., a driver's license or passport) as part of an account creation or pre-qualification step. In some embodiments, the user may be asked to show the official identification to the proctor during the testing session. During the testing session, the proctor can then compare the uploaded or shown identification to the person's appearance in the video feed. In this way the proctor can verify the user's identity by comparing a live (or other) video feed of the user to an identification card associated with the user. In some embodiments, once the user's identity is verified, the user's identity can be associated with the user's account for future tests. For example, after verification, future verifications may be automated with face matching technology.
In another example, user identification can be achieved using biometrics. For example, in some embodiments, a user accessing the platform may be given the option to perform a biometric initialization, in which the user goes to a physical location where their fingerprint or a unique segment of DNA can be sampled and stored. Thereafter, every test taken can sample their fingerprint or DNA to verify identity. This may also be automated. In other embodiments, biometrics may be performed using biometric features of the user's device. For example, many smartphones today are capable of taking a user's fingerprint or recognizing a user's face. These features may be used to verify the user's identity in some embodiments.
In addition to verifying the user's identity, the test or test kit that will be used during the test may be verified as well. This can be important because it ensures that the test results are scientifically valid and can be trusted. Again, this can be particularly important in the case of remote testing where the user is not physically located with the person administering the test. In one embodiment, the test or test kit can be provided with a unique ID (e.g., a UID or serial number) assigned during manufacture, which can be queried when the test is taken. This can take the form of a printed string of characters, barcode/QR code, NFC/RFID tag, or other. This code may either explicitly encode information associated with the test (such as a test identifier, test expiration date, batch/lot codes, indication of whether this number has been used for a test or not) or it may encode a link to a database entry that includes such information. Prior to beginning the test, the code may be scanned to verify the test kit. If anything is amiss, the test does not proceed and the user may be instructed to obtain a new test kit. In some embodiments, it may be preferable to provide the unique ID in a non-human readable manner. This may provide an advantage in that they are harder to misrepresent. A visible code could be duplicated and used on an expired test, for example.
During a test, a sample may be collected from the user, for example, using the test kit. To ensure the integrity of the test, steps may be taken to ensure that the sample is actually collected from the same user whose identity was verified before beginning the test. Again, this can be especially important in the case of remote tests since the user is not physically located with the person administering the test. It can be important to ensure that a user does not swap in a sample obtained from another user when performing the test. Various mechanisms for verifying that the sample is collected from the previously verified user are possible. For example, during a proctored testing session, the proctor (or an automated system) can observe the sample collection process. For example, in the case of a nasal swab test, the proctor or the automated system can observe the user performing the swab procedure. Such observation can be performed live (e.g., over a live video connection) or through a pre-recorded video. In either event, it may be important that all sample collection materials remain in view of the camera at all times. This would prevent a user from swabbing his or her nose and then switching out the swab with another that has been used by a different person. Additionally, it may be beneficial to positively identify the user during the collection process. This can be accomplished by, for example, checking the user's identity immediately before the sample collection process.
During the test, it is also important to ensure that all test instructions and procedures are followed correctly. This can ensure the accuracy and validity of the test results. Similar to verifying that the sample is obtained from the correct user, ensuring that all test instructions and procedures are followed correctly can be accomplished by directly viewing the testing procedure, either over a live video feed or be watching a recording of the testing process. In some embodiments, such observation is provided by a live proctor. In some embodiments, such observation is provided through an automated system (e.g., a computer system that is configured analyze live or pre-recorded video). In the case of a swab, for example, as shown in
Additionally, it is important to ensure that submitted test results actually come from the originally verified test or test kit. This can ensure test continuity, making sure that the same test is used throughout the test (e.g., that the test that was verified is the one for which results are obtained). Otherwise, tests could be exchanged during the process, leading to improper results. In some embodiments, this can be accomplished by reverifying the test or test kit throughout the testing procedure. For example, the method that was used previously to determine the test kit was valid (e.g., scanning the unique ID of the test kit) can be repeated to ensure the UID/serial number are the same. In some embodiments, the test kits can be designed such that the test such that the results are reported in a manner that includes the UID/serial number, such as in a custom visible code (barcode, QR code, etc.) or NFC/RFID, so that when the results are read, the UID can be verified. For example, in some embodiments, test results are verified by viewing strips that appear on a test card. The test card can include the unique ID of the test kit near the area at which the strips will appear such that a view of the strips also includes a view of the unique ID.
Finally, it can also be important to ensure that the test results are interpreted correctly. As described previously, in some embodiments, a proctor interprets the test results by viewing strips that appear on a test card, for example, as described above with reference to
Test Pass
A test pass can be provided and associated with the platform that is used to identify the user and communicate the user's test results. In some embodiments, the test pass can be provided at and/or used to verify a user's identity at a point of care, such as the location where the test is taken (e.g., a medical facility, a home, a hotel, an airport, a etc.), and also used to verify the user's test results at a point of verification (e.g., a location where the user's results are checked (such as the airport, a hotel, a place of employment, etc.). In some embodiments, the point of care and the point of verification may be different. For example, a user may take the test at his or her home and receive the test pass and then can use the test pass to verify his or her results at the airport. In some embodiments, the point of care and the point of verification may be the same. For example, a user desiring to board a flight at an airport can take a test and receive a test pass at the airport and then use that pass to verify the results and board a flight.
In some embodiments, the test pass can be configured to perform one or more of several functions. For example, the test pass can include proof of the user's identity. For example, in
A test pass can be provided in either a physical or virtual manner. For example, a physical test pass may comprise forgery-resistant card provided by the test pass issuer that includes user's name and photo for identification. The physical test pass may also include a barcode, QR code, NFC chip, contact chip, alphanumeric code, or other unique identifier that will access test results from a secure database when scanned. An example is shown in
In some embodiments, each user is provided with a new physical or virtual test pass each time they complete a test. For example, a user may be provided with a new physical test pass indicating the most recent test result, or the user may be provided with a virtual test pass indicating the most recent test result. In other embodiments, in which the user already has a test pass, upon completion of a new test, the existing test pass can be updated to include the most recent result. For example, a machine-readable code on a physical or virtual test pass may be linked to the newest test result.
In some embodiments of a virtual test pass, the user's ID may be verified each time that the virtual test pass is displayed. For example, the user may be required to be biometrically authenticated each time the virtual test pass is to be displayed. As one example, the user may use face recognition on his or her phone in order to verify his or her identity before the virtual test pass can be displayed.
Test Continuity
As described above, the remote testing platform described herein can connect remotely located users with proctors over a network such as the internet. Problems may arise if either the users or the proctors experience poor network connectivity. The platform, however, may implement certain mechanisms and procedures to facilitate remote testing even in circumstances of poor network connectivity. For example, if either the proctor or the user experiences connectivity issues during the proctoring session, the proctoring session may be prematurely terminated, which can be inconvenient for users. However, the platform can be adapted to detect a loss of connection and/or poor connection, and take steps to record images or video of the user and test kit so that testing and proctoring can be resumed if and when connectivity is restored.
The platform may evaluate the strength of the user's network connection. In response to determining that the user's connection has dropped out or slowed down dramatically, the platform may perform one or more operations to begin recording and storing image or video data to a local location (e.g., on the user's hard drive). For example, the user may be participating in a live proctored testing session in which they are actively connected to a live proctor over video. During the session, the platform can monitor the user's network connection. If certain threshold criteria are met, indicating that the user's network connection is unstable or insufficient, the platform may initiate a recording of the user's session. In some embodiments, detection of a network connectivity issue may trigger high full frame rate resolution recording. In some embodiments, the recording is initiated and stored on the user's local machine, such that a network connection is not required for the recording. Depending on the type of test being performed and/or the level of proctor involvement necessary, the testing session may be able to continue, even though the user has lost his or her network connection to the proctor. For example, the user may be provided with prerecorded or written instructions for completing the testing session. Upon regaining connectivity, the recorded data may be uploaded to the platform and provided for proctor (or automated review). For example, once network connectivity is restored, the user's video may be uploaded to the platform where it can be reviewed by a proctor (or an automated system) to determine whether all testing steps were accurately followed. If all testing steps can be verified, the user's test result can also be verified. If one or more of the testing steps cannot be verified, the user may need to redo the test.
In addition to monitoring the user's network connection, the platform may also monitor the proctors' network connection. If a proctor's network connection is determined to be unstable or slow, the platform may perform one or more operations to ensure or facilitate test continuity for users connected to that proctor. For example, the platform may begin recording and storing user image or video data to cloud storage such that the proctor may review the image or video data when his or her network connection is restored. In some embodiments, detection of a network connectivity issue may trigger high full frame rate resolution recording. In some embodiments, the platform may send the recorded image or video data to another proctor or an automated system for review such that the user can continue the proctored testing session without delay. For example, the image or video data may be reviewed (in real-time) using an automated system or by another available proctor. When the original proctor regains network connectivity, in some embodiments, the proctor may be given access to the recorded data for review. In some embodiments, upon determining that a proctor's network connection is unstable, slow, or otherwise not working, one or more users connected to that proctor may be redirected to other proctors. In some embodiments, a backup proctor may be connected to the testing session such that the backup proctor can seamlessly take over in the event that the initial proctor becomes unavailable.
In the event of network connectivity issues (either on the user end or the proctor end), the platform may display messages to the user or the proctor indicating the potential issue and the steps being taken to ensure or facilitate testing continuity. For example, the user may be provided with a notification regarding the connectivity issues, instructions on how to proceed, and a phone number that can be used to contact customer support, etc.
At 2914, reconnect procedures may attempt to reestablish connectivity between the user and the platform. For example, the platform may attempt to reconnect the user for a predetermined amount of time (also referred to herein as the “reconnect window”). At 2916, if the user cannot be reconnected, the platform moves to 2918 at which the platform evaluates whether the reconnect window has expired. If it has not, the platform moves back to 2914. If the reconnect window has expired, the platform moves to 2920, at which it determines whether proctorless completion of the test is possible. Proctorless completion may be possible in some circumstances depending on the type of test and/or the stage of the test at which connectivity was lost. If proctorless completion is not possible, the user may be informed at 2922 that the test cannot be completed. The user will need to retest or resume the test once network connectivity is reestablished. If proctorless testing is possible, the user may be informed of the switch to proctorless mode at 2924. At 2934, the saved test state and/or recorded video can be loaded from the server and analyzed. At 2936, the test can be resumed from the saved state and completed in proctorless mode.
Returning to 2916, if the platform determines that the user is connected, the platform may, at 2926, determine whether the original proctor is connected. If the original proctor is connected, the user and proctor can be reconnected and testing can resume as normal. If the original proctor is not connected, at 2930, the user may enter a waiting room where a new proctor can be assigned. At 2932, the new proctor can accept the request and the test can be resumed with the new proctor.
Augmented Reality (AR) Based Testing Guidance
In some embodiments, the platform may provide augmented reality (AR) based testing guidance to users and/or proctors. Because users and proctors generally interact with the platform using devices that include displays, AR-based testing guidance can be overlaid onto the displays to facilitate testing and/or proctoring. For example, graphics can be overlaid onto a live video feed of the test kit (as provided by a camera of the user's device) to help the user properly open, set up, utilize, and/or dispose of the test kit. As a more specific example, in some tests, the user may be required to deposit drops of a solution onto a certain portion of a test card (see, for example,
AR-based guidance can be implemented in a variety of different ways.
For example, during one portion of the testing procedure, the output of the rearward facing camera (e.g., FOV 3010 in which is positioned the test kit 3012) can be displayed to the user such that the user can view the test kit 3012 on the display of the smartphone 3004. The display can be updated with AR-based guidance to highlight certain areas of the test kit 3012 or items in the test kit 3012 or overlaid with other types of instructions to aid the user in performing the testing procedure. During another portion of the testing procedure, the output of the frontward facing camera (e.g., FOV 3008 in which the user 3002 is positioned) can be displayed to the user such that the user can view his or herself on the display of the smartphone 3004. The display can be updated with AR-based guidance to highlight certain areas of the user (e.g., a nostril) or overlaid with other types of instructions to aid the user in performing the testing procedure.
In some embodiments, the set up illustrated in
In some embodiments, AR-based guidance may facilitate sample collection. For example, graphics can be overlaid onto a live video feed of the user (as provided by a camera of the user's device) to help the user properly collect the required biological sample from their person.
In the case of sample collection via a nasal swab, for example, as illustrated in
In some examples, the user may place the smartphone in a smartphone stand, and may be instructed to position the test kit in front of the smartphone such that both the user and the test kit are visible in the inward-facing camera on the front of the smartphone above the screen, for example, as described above with respect to
Artificial Intelligence (AI) Techniques for Proctor Assistance
In some embodiments, the platform may be configured to use proctor-generated data to train an artificial intelligence (AI) system (e.g., a neural network) to facilitate review of test results. For example, for each test performed using a live proctor, an image of the physical test (e.g., as captured by the camera on the user's device upon completion of the test and presented to the proctor for interpretation) can be stored in association with the test result as interpreted by the proctor. For example, proctor-classified positive and negative test result images can be stored. These stored images and the corresponding proctor-determined test results can be used as data for training one or more neural networks to provide accurate test result interpretations. For example, a single neural network could be trained using data captured and generated during proctoring sessions overseen by many different proctors. Alternatively or additionally, a neural network could be trained using data captured and generated during proctoring sessions overseen by one specific proctor. Training of the neural network could be conducted in a training phase, on an ongoing basis, or a combination thereof. In this way, as the platform generates data using live proctors, the data set for training the neural network can grow, increasing the accuracy with which the neural network can analyze the test results.
Once the neural network is sufficiently trained, the neural network can, in some implementations, be used to analyze the test results. In some embodiments, the neural network can be used as a supplement to the proctors' interpretations of the test results. For example, an AI-assisted test result interpretation can be provided to help the proctors analyze the test results. A machine learning model (e.g., the neural network trained in the aforementioned manner) can be used to provide proctors with recommended suggested test result interpretations in real-time based on images of physical tests (as captured upon test completion and presented to proctors for interpretation). The proctors may then be given the option to agree or disagree with each recommendation. This may facilitate faster test interpretation by the proctor and/or generate additional data that could be used to continually train the neural network.
Once the AI model has been trained, the model may, at 3110, present AI-determined recommendations to the proctors during proctored test sessions. For example, during a proctored test session an image of the test results can be captured and displayed to the proctor. The image can also be transmitted to the AI model, which can make a determination as to the result. The AI-determined result can be presented to the proctor as a recommendation. The proctor may make the final determination of the result based on the image of the test results as well as the AI-determined result. When the proctor determines the result, the result-image pair may be stored and used to further refine the training of the AI model.
Computer Systems
In some embodiments, the systems, processes, and methods described herein are implemented using a computing system, such as the one illustrated in
The computer system 3202 can comprise a health testing and diagnostic module 3214 that carries out the functions, methods, acts, and/or processes described herein. The health testing and diagnostic module 3214 is executed on the computer system 3202 by a central processing unit 3206 discussed further below.
In general, the word “module,” as used herein, refers to logic embodied in hardware or firmware or to a collection of software instructions, having entry and exit points. Modules are written in a program language, such as JAVA, C or C++, PYPHON or the like. Software modules may be compiled or linked into an executable program, installed in a dynamic link library, or may be written in an interpreted language such as BASIC, PERL, LUA, or Python. Software modules may be called from other modules or from themselves, and/or may be invoked in response to detected events or interruptions. Modules implemented in hardware include connected logic units such as gates and flip-flops, and/or may include programmable units, such as programmable gate arrays or processors.
Generally, the modules described herein refer to logical modules that may be combined with other modules or divided into sub-modules despite their physical organization or storage. The modules are executed by one or more computing systems and may be stored on or within any suitable computer readable medium or implemented in-whole or in-part within special designed hardware or firmware. Not all calculations, analysis, and/or optimization require the use of computer systems, though any of the above-described methods, calculations, processes, or analyses may be facilitated through the use of computers. Further, in some embodiments, process blocks described herein may be altered, rearranged, combined, and/or omitted.
The computer system 3202 includes one or more processing units (CPU) 3206, which may comprise a microprocessor. The computer system 3202 further includes a physical memory 3210, such as random access memory (RAM) for temporary storage of information, a read only memory (ROM) for permanent storage of information, and a mass storage device 3204, such as a backing store, hard drive, rotating magnetic disks, solid state disks (SSD), flash memory, phase-change memory (PCM), 3D XPoint memory, diskette, or optical media storage device. Alternatively, the mass storage device may be implemented in an array of servers. Typically, the components of the computer system 3202 are connected to the computer using a standards-based bus system. The bus system can be implemented using various protocols, such as Peripheral Component Interconnect (PCI), Micro Channel, SCSI, Industrial Standard Architecture (ISA) and Extended ISA (EISA) architectures.
The computer system 3202 includes one or more input/output (I/O) devices and interfaces 3212, such as a keyboard, mouse, touch pad, and printer. The I/O devices and interfaces 3212 can include one or more display devices, such as a monitor, that allows the visual presentation of data to a user. More particularly, a display device provides for the presentation of GUIs as application software data, and multi-media presentations, for example. The I/O devices and interfaces 3212 can also provide a communications interface to various external devices. The computer system 3202 may comprise one or more multi-media devices 3208, such as speakers, video cards, graphics accelerators, and microphones, for example.
The computer system 3202 may run on a variety of computing devices, such as a server, a Windows server, a Structure Query Language server, a Unix Server, a personal computer, a laptop computer, and so forth. In other embodiments, the computer system 3202 may run on a cluster computer system, a mainframe computer system and/or other computing system suitable for controlling and/or communicating with large databases, performing high volume transaction processing, and generating reports from large databases. The computing system 3202 is generally controlled and coordinated by an operating system software, such as z/OS, Windows, Linux, UNIX, BSD, SunOS, Solaris, MacOS, or other compatible operating systems, including proprietary operating systems. Operating systems control and schedule computer processes for execution, perform memory management, provide file system, networking, and I/O services, and provide a user interface, such as a graphical user interface (GUI), among other things.
The computer system 3202 illustrated in
Access to the health testing and diagnostic module 3214 of the computer system 3202 by computing systems 3220 and/or by data sources 3222 may be through a web-enabled user access point such as the computing systems' 3220 or data source's 3222 personal computer, cellular phone, smartphone, laptop, tablet computer, e-reader device, audio player, or another device capable of connecting to the network 3218. Such a device may have a browser module that is implemented as a module that uses text, graphics, audio, video, and other media to present data and to allow interaction with data via the network 3218.
The output module may be implemented as a combination of an all-points addressable display such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, or other types and/or combinations of displays. The output module may be implemented to communicate with input devices 3212 and they also include software with the appropriate interfaces which allow a user to access data through the use of stylized screen elements, such as menus, windows, dialogue boxes, tool bars, and controls (for example, radio buttons, check boxes, sliding scales, and so forth). Furthermore, the output module may communicate with a set of input and output devices to receive signals from the user.
The input device(s) may comprise a keyboard, roller ball, pen and stylus, mouse, trackball, voice recognition system, or pre-designated switches or buttons. The output device(s) may comprise a speaker, a display screen, a printer, or a voice synthesizer. In addition, a touch screen may act as a hybrid input/output device. In another embodiment, a user may interact with the system more directly such as through a system terminal connected to the score generator without communications over the Internet, a WAN, or LAN, or similar network.
In some embodiments, the system 3202 may comprise a physical or logical connection established between a remote microprocessor and a mainframe host computer for the express purpose of uploading, downloading, or viewing interactive data and databases on-line in real time. The remote microprocessor may be operated by an entity operating the computer system 3202, including the client server systems or the main server system, an/or may be operated by one or more of the data sources 3222 and/or one or more of the computing systems 3220. In some embodiments, terminal emulation software may be used on the microprocessor for participating in the micro-mainframe link.
In some embodiments, computing systems 3220 who are internal to an entity operating the computer system 3202 may access the health testing and diagnostic module 3214 internally as an application or process run by the CPU 3206.
In some embodiments, one or more features of the systems, methods, and devices described herein can utilize a URL and/or cookies, for example for storing and/or transmitting data or user information. A Uniform Resource Locator (URL) can include a web address and/or a reference to a web resource that is stored on a database and/or a server. The URL can specify the location of the resource on a computer and/or a computer network. The URL can include a mechanism to retrieve the network resource. The source of the network resource can receive a URL, identify the location of the web resource, and transmit the web resource back to the requestor. A URL can be converted to an IP address, and a Domain Name System (DNS) can look up the URL and its corresponding IP address. URLs can be references to web pages, file transfers, emails, database accesses, and other applications. The URLs can include a sequence of characters that identify a path, domain name, a file extension, a host name, a query, a fragment, scheme, a protocol identifier, a port number, a username, a password, a flag, an object, a resource name and/or the like. The systems disclosed herein can generate, receive, transmit, apply, parse, serialize, render, and/or perform an action on a URL.
A cookie, also referred to as an HTTP cookie, a web cookie, an internet cookie, and a browser cookie, can include data sent from a website and/or stored on a user's computer. This data can be stored by a user's web browser while the user is browsing. The cookies can include useful information for websites to remember prior browsing information, such as a shopping cart on an online store, clicking of buttons, login information, and/or records of web pages or network resources visited in the past. Cookies can also include information that the user enters, such as names, addresses, passwords, credit card information, etc. Cookies can also perform computer functions. For example, authentication cookies can be used by applications (for example, a web browser) to identify whether the user is already logged in (for example, to a web site). The cookie data can be encrypted to provide security for the consumer. Tracking cookies can be used to compile historical browsing histories of individuals. Systems disclosed herein can generate and use cookies to access data of an individual. Systems can also generate and use JSON web tokens to store authenticity information, HTTP authentication as authentication protocols, IP addresses to track session or identity information, URLs, and the like.
The computing system 3202 may include one or more internal and/or external data sources (for example, data sources 3222). In some embodiments, one or more of the data repositories and the data sources described above may be implemented using a relational database, such as DB2, Sybase, Oracle, CodeBase, and Microsoft® SQL Server as well as other types of databases such as a flat-file database, an entity relationship database, and object-oriented database, and/or a record-based database.
The computer system 3202 may also access one or more databases 3222. The databases 3222 may be stored in a database or data repository. The computer system 3202 may access the one or more databases 3222 through a network 3218 or may directly access the database or data repository through I/O devices and interfaces 3212. The data repository storing the one or more databases 3222 may reside within the computer system 3202.
In some embodiments, a health testing and diagnostic platform 3302 may be comprised of a registration and purchase module 3304, a testing module 3306, an analytics module 3308, and a reporting module 3310. The health testing and diagnostic platform 3302 may also comprise a user profile database 3312, a proctor database 3314, a test database 3316, and/or a site database 3318. The health testing and diagnostic platform 2002 can be connected to a network 3320. The network 3320 can be configured to connect the health testing and diagnostic platform 3302 to one or more proctor devices 3322, one or more user devices 3324, one or more pharmacy systems 3326, one or more third-party provider systems 3328, and/or one or more government systems 3330.
The registration and purchase 3304 may function by facilitating patient registration through one or more registration interfaces and in conjunction with the user database 3312, store user registration data. The testing module 3306 may be configured to allow a user to initiate and complete a medical test or visit with a proctor through a series of pre-testing and testing interfaces, as described herein. The analytics module 3308 may be configured to dynamically analyze patient tests across a given population stored in the test database 3316 and provide structured data of the test results. The reporting module 3310 may function by dynamically and automatically reporting test results to government entities, patients, and third parties using one or more interfaces, such as one or more application programming interfaces. Each of the modules can be configured to interact with each other and the databases discussed herein.
In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than restrictive sense.
Indeed, although this invention has been disclosed in the context of certain embodiments and examples, it will be understood by those skilled in the art that the invention extends beyond the specifically disclosed embodiments to other alternative embodiments and/or uses of the invention and obvious modifications and equivalents thereof. In addition, while several variations of the embodiments of the invention have been shown and described in detail, other modifications, which are within the scope of this invention, will be readily apparent to those of skill in the art based upon this disclosure. It is also contemplated that various combinations or sub-combinations of the specific features and aspects of the embodiments may be made and still fall within the scope of the invention. It should be understood that various features and aspects of the disclosed embodiments can be combined with, or substituted for, one another in order to form varying modes of the embodiments of the disclosed invention. Any methods disclosed herein need not be performed in the order recited. Thus, it is intended that the scope of the invention herein disclosed should not be limited by the particular embodiments described above.
It will be appreciated that the systems and methods of the disclosure each have several innovative aspects, no single one of which is solely responsible or required for the desirable attributes disclosed herein. The various features and processes described above may be used independently of one another or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of this disclosure.
Certain features that are described in this specification in the context of separate embodiments also may be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment also may be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination. No single feature or group of features is necessary or indispensable to each and every embodiment.
It will also be appreciated that conditional language used herein, such as, among others, “can,” “could,” “might,” “may,” “e.g.,” and the like, unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements and/or steps. Thus, such conditional language is not generally intended to imply that features, elements and/or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without author input or prompting, whether these features, elements and/or steps are included or are to be performed in any particular embodiment. The terms “comprising,” “including,” “having,” and the like are synonymous and are used inclusively, in an open-ended fashion, and do not exclude additional elements, features, acts, operations, and so forth. In addition, the term “or” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some, or all of the elements in the list. In addition, the articles “a,” “an,” and “the” as used in this application and the appended claims are to be construed to mean “one or more” or “at least one” unless specified otherwise. Similarly, while operations may be depicted in the drawings in a particular order, it is to be recognized that such operations need not be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Further, the drawings may schematically depict one more example processes in the form of a flowchart. However, other operations that are not depicted may be incorporated in the example methods and processes that are schematically illustrated. For example, one or more additional operations may be performed before, after, simultaneously, or between any of the illustrated operations. Additionally, the operations may be rearranged or reordered in other embodiments. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged into multiple software products. Additionally, other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve desirable results.
Further, while the methods and devices described herein may be susceptible to various modifications and alternative forms, specific examples thereof have been shown in the drawings and are herein described in detail. It should be understood, however, that the invention is not to be limited to the particular forms or methods disclosed, but, to the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the various implementations described and the appended claims. Further, the disclosure herein of any particular feature, aspect, method, property, characteristic, quality, attribute, element, or the like in connection with an implementation or embodiment can be used in all other implementations or embodiments set forth herein. Any methods disclosed herein need not be performed in the order recited. The methods disclosed herein may include certain actions taken by a practitioner; however, the methods can also include any third-party instruction of those actions, either expressly or by implication. The ranges disclosed herein also encompass any and all overlap, sub-ranges, and combinations thereof. Language such as “up to,” “at least,” “greater than,” “less than,” “between,” and the like includes the number recited. Numbers preceded by a term such as “about” or “approximately” include the recited numbers and should be interpreted based on the circumstances (e.g., as accurate as reasonably possible under the circumstances, for example ±5%, ±10%, ±15%, etc.). For example, “about 3.5 mm” includes “3.5 mm.” Phrases preceded by a term such as “substantially” include the recited phrase and should be interpreted based on the circumstances (e.g., as much as reasonably possible under the circumstances). For example, “substantially constant” includes “constant.” Unless stated otherwise, all measurements are at standard conditions including temperature and pressure.
As used herein, a phrase referring to “at least one of” a list of items refers to any combination of those items, including single members. As an example, “at least one of: A, B, or C” is intended to cover: A, B, C, A and B, A and C, B and C, and A, B, and C. Conjunctive language such as the phrase “at least one of X, Y and Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to convey that an item, term, etc. may be at least one of X, Y or Z. Thus, such conjunctive language is not generally intended to imply that certain embodiments require at least one of X, at least one of Y, and at least one of Z to each be present. The headings provided herein, if any, are for convenience only and do not necessarily affect the scope or meaning of the devices and methods disclosed herein.
Accordingly, the claims are not intended to be limited to the embodiments shown herein, but are to be accorded the widest scope consistent with this disclosure, the principles and the novel features disclosed herein.
Any and all applications for which a foreign or domestic priority claim is identified in the Application Data Sheet as filed with the present application are hereby incorporated by reference under 37 CFR 1.57. This application is a continuation of U.S. application Ser. No. 17/681,098, filed Feb. 25, 2022, which is a continuation of U.S. application Ser. No. 17/452,746, filed Oct. 28, 2021, now U.S. Pat. No. 11,289,196 issued Mar. 29, 2022, which is a continuation of U.S. application Ser. No. 17/452,726, filed Oct. 28, 2021, which claims the benefit under 35 U.S.C. 119(e) to U.S. Provisional Application No. 63/136,575, filed Jan. 12, 2021, to U.S. Provisional Application No. 63/156,012, filed Mar. 3, 2021, and to U.S. Provisional Application 63/179,864, filed Apr. 26, 2021, each of which is hereby incorporated by reference herein.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7110525 | Heller et al. | Sep 2006 | B1 |
| 7176936 | Sauer et al. | Feb 2007 | B2 |
| 7491198 | Kockro | Feb 2009 | B2 |
| 7662113 | Pearl et al. | Feb 2010 | B2 |
| 8108190 | Riener et al. | Aug 2012 | B2 |
| 8253778 | Atsushi | Nov 2012 | B2 |
| 8314815 | Navab et al. | Nov 2012 | B2 |
| 8520080 | Havens | Aug 2013 | B2 |
| 8485038 | Sengupta et al. | Dec 2013 | B2 |
| 8611988 | Miyamoto | Dec 2013 | B2 |
| 8713130 | Logan et al. | Jul 2014 | B2 |
| 8768022 | Miga et al. | Aug 2014 | B2 |
| 8814691 | Haddick et al. | Dec 2014 | B2 |
| 8911358 | Koninckx | Dec 2014 | B2 |
| 8948935 | Peeters et al. | Mar 2015 | B1 |
| 8982156 | Maggiore | May 2015 | B2 |
| 9030446 | Mistry et al. | Jul 2015 | B2 |
| 9082319 | Shimada et al. | Aug 2015 | B2 |
| 9111383 | Fein et al. | Aug 2015 | B2 |
| 9256711 | Horseman | Feb 2016 | B2 |
| 9262743 | Heins et al. | Mar 2016 | B2 |
| 9285871 | Geisner et al. | May 2016 | B2 |
| 9338622 | Bjontegard | May 2016 | B2 |
| 9345957 | Geisner et al. | Jun 2016 | B2 |
| 9380177 | Rao et al. | Aug 2016 | B1 |
| 9424761 | Tuchschmid et al. | Nov 2016 | B2 |
| 9498132 | Maier-Hein et al. | Nov 2016 | B2 |
| 9547917 | Zamer | Feb 2017 | B2 |
| 9563266 | Banerjee et al. | Mar 2017 | B2 |
| 9600934 | Odessky et al. | Mar 2017 | B2 |
| 9606992 | Geisner et al. | May 2017 | B2 |
| 9648436 | Kraft | Oct 2017 | B2 |
| 9788714 | Krueger | Dec 2017 | B2 |
| 9836888 | Skidmore | Dec 2017 | B2 |
| 9877642 | Duret | Feb 2018 | B2 |
| 9886458 | Jung et al. | Feb 2018 | B2 |
| 9892561 | Choukroun et al. | Feb 2018 | B2 |
| 9898662 | Tsuda et al. | Mar 2018 | B2 |
| 9916002 | Petrovskaya et al. | May 2018 | B2 |
| 9972137 | Petrovskaya et al. | Jul 2018 | B2 |
| 10013896 | Feins et al. | Aug 2018 | B2 |
| 10052026 | Tran | Oct 2018 | B1 |
| 10106172 | Wingfield et al. | Oct 2018 | B2 |
| 10108266 | Banerjee et al. | Nov 2018 | B2 |
| 10127734 | Stroila | Nov 2018 | B2 |
| 10156900 | Publicover et al. | Dec 2018 | B2 |
| 10197803 | Badiali et al. | Feb 2019 | B2 |
| 10216957 | Jung et al. | Mar 2019 | B2 |
| 10231614 | Krueger | May 2019 | B2 |
| 10295815 | Romanowski et al. | Jun 2019 | B2 |
| 10322313 | McKirdy | Jul 2019 | B2 |
| 10346889 | Reiss et al. | Aug 2019 | B1 |
| 10386918 | Shin | Oct 2019 | B2 |
| 10430985 | Harrises et al. | Nov 2019 | B2 |
| 10474233 | Swaminathan et al. | Nov 2019 | B2 |
| 10524715 | Sahin | Jan 2020 | B2 |
| 10535202 | Schmirler et al. | Jan 2020 | B2 |
| 10540776 | Tran et al. | Feb 2020 | B2 |
| 10559117 | Kaeser et al. | Mar 2020 | B2 |
| 10593092 | Solomon | May 2020 | B2 |
| 10643210 | Smith et al. | May 2020 | B2 |
| 10660522 | Redei | May 2020 | B2 |
| 10664572 | Bitran et al. | Sep 2020 | B2 |
| 10758209 | Boctor et al. | Sep 2020 | B2 |
| 10788791 | Gelman et al. | Oct 2020 | B2 |
| 10802695 | Daniels et al. | Nov 2020 | B2 |
| 10824310 | Acharya et al. | Nov 2020 | B2 |
| 10832488 | Petrovskaya et al. | Dec 2020 | B2 |
| 10849688 | Rios et al. | Dec 2020 | B2 |
| 10885530 | Mercury et al. | Jan 2021 | B2 |
| 10888389 | Draelos et al. | Jan 2021 | B2 |
| 10892052 | Jordan et al. | Jan 2021 | B2 |
| 10910016 | Rothschild et al. | Mar 2021 | B2 |
| 10932705 | Muhsin et al. | Mar 2021 | B2 |
| 10943407 | Morgan et al. | Mar 2021 | B1 |
| 10945807 | Gibby et al. | Mar 2021 | B2 |
| 10957111 | Weisman et al. | Apr 2021 | B2 |
| 10984910 | Burkholz et al. | Apr 2021 | B2 |
| 10991190 | Luthra et al. | Apr 2021 | B1 |
| 10991461 | Divine et al. | May 2021 | B2 |
| 11004271 | Cvetko et al. | May 2021 | B2 |
| 11017694 | Buras et al. | Oct 2021 | B2 |
| 11152093 | Hopen, Sr. et al. | Oct 2021 | B1 |
| 11194995 | Profida Ferreira et al. | Dec 2021 | B1 |
| 11257572 | Narke et al. | Feb 2022 | B1 |
| 11219428 | Burkholz | Mar 2022 | B2 |
| 11270235 | Daianu et al. | Mar 2022 | B1 |
| 11289196 | Ferro, Jr. et al. | Mar 2022 | B1 |
| 11315053 | Powell et al. | Apr 2022 | B1 |
| 11367530 | Ferro, Jr. et al. | Jun 2022 | B1 |
| 11393586 | Ferro, Jr. et al. | Jul 2022 | B1 |
| 11521736 | Gastfriend et al. | Dec 2022 | B1 |
| 20070048723 | Brewer et al. | Mar 2007 | A1 |
| 20090004055 | Darrigrand et al. | Jan 2009 | A1 |
| 20090263775 | Ullrich | Oct 2009 | A1 |
| 20100121156 | Yoo | May 2010 | A1 |
| 20100159434 | Lampotang et al. | Jun 2010 | A1 |
| 20110164105 | Lee | Jul 2011 | A1 |
| 20110207108 | Dorman | Aug 2011 | A1 |
| 20120053955 | Martin et al. | Mar 2012 | A1 |
| 20120221960 | Robinson | Aug 2012 | A1 |
| 20130096937 | Campbell et al. | Apr 2013 | A1 |
| 20130253339 | Reyes | Sep 2013 | A1 |
| 20130328997 | Desai | Dec 2013 | A1 |
| 20130344470 | Morgan et al. | Dec 2013 | A1 |
| 20140160264 | Taylor et al. | Jun 2014 | A1 |
| 20140253590 | Needham et al. | Sep 2014 | A1 |
| 20140304335 | Fung et al. | Oct 2014 | A1 |
| 20150187048 | Johnson | Jul 2015 | A1 |
| 20160125765 | Meretei et al. | May 2016 | A1 |
| 20160189573 | Bush | Jun 2016 | A1 |
| 20160292378 | Saric | Oct 2016 | A1 |
| 20160371884 | Benko et al. | Dec 2016 | A1 |
| 20170103440 | Xing et al. | Apr 2017 | A1 |
| 20170115742 | Xing et al. | Apr 2017 | A1 |
| 20170146801 | Stempora | May 2017 | A1 |
| 20170173262 | Veltz | Jun 2017 | A1 |
| 20170323062 | Djajadiningrat et al. | Nov 2017 | A1 |
| 20180039737 | Dempers et al. | Feb 2018 | A1 |
| 20180092595 | Chen et al. | Apr 2018 | A1 |
| 20180140362 | Cali et al. | May 2018 | A1 |
| 20180173913 | Pulitzer et al. | Jun 2018 | A1 |
| 20180197624 | Robaina et al. | Jul 2018 | A1 |
| 20180225982 | Jaeh et al. | Aug 2018 | A1 |
| 20180253840 | Tran | Sep 2018 | A1 |
| 20180341919 | Luhman | Nov 2018 | A1 |
| 20180353073 | Boucher et al. | Dec 2018 | A1 |
| 20180357915 | Harlow | Dec 2018 | A1 |
| 20190000564 | Navab et al. | Jan 2019 | A1 |
| 20190005195 | Peterson | Jan 2019 | A1 |
| 20190012176 | McHale | Jan 2019 | A1 |
| 20190020651 | Soon-Shiong et al. | Jan 2019 | A1 |
| 20190025919 | Tadi et al. | Jan 2019 | A1 |
| 20190064520 | Christensen | Feb 2019 | A1 |
| 20190073110 | Bradley et al. | Mar 2019 | A1 |
| 20190139318 | Tierney et al. | May 2019 | A1 |
| 20190179584 | Masters | Jun 2019 | A1 |
| 20190206134 | Devam et al. | Jul 2019 | A1 |
| 20190216452 | Nawana et al. | Aug 2019 | A1 |
| 20190259483 | Potts | Aug 2019 | A1 |
| 20190261914 | Davis et al. | Aug 2019 | A1 |
| 20190346429 | Harris | Nov 2019 | A1 |
| 20190266663 | Keeler et al. | Dec 2019 | A1 |
| 20190378080 | Srinivasan et al. | Dec 2019 | A1 |
| 20190380790 | Fuchs et al. | Dec 2019 | A1 |
| 20190391638 | Khaderi et al. | Dec 2019 | A1 |
| 20200000401 | Dullen | Jan 2020 | A1 |
| 20200020454 | McGarvey | Jan 2020 | A1 |
| 20200020171 | Hendricks et al. | Mar 2020 | A1 |
| 20200073143 | Macnamara et al. | Apr 2020 | A1 |
| 20200101367 | Tran et al. | May 2020 | A1 |
| 20200152339 | Pulitzer et al. | Jun 2020 | A1 |
| 20200174756 | Cerar et al. | Jun 2020 | A1 |
| 20200175769 | Mandala | Jul 2020 | A1 |
| 20200205913 | Carnes et al. | Jul 2020 | A1 |
| 20200207501 | Urquhart et al. | Jul 2020 | A1 |
| 20200211291 | Miller et al. | Jul 2020 | A1 |
| 20200226758 | Carnes et al. | Aug 2020 | A1 |
| 20200250389 | Pulitzer et al. | Sep 2020 | A1 |
| 20200303044 | Stephen | Oct 2020 | A1 |
| 20200312038 | Samec et al. | Oct 2020 | A1 |
| 20200312437 | Wendland | Oct 2020 | A1 |
| 20200337631 | Sahin | Oct 2020 | A1 |
| 20200342679 | Soon-Shiong | Dec 2020 | A1 |
| 20200405257 | Samec et al. | Dec 2020 | A1 |
| 20200409159 | Samec et al. | Dec 2020 | A1 |
| 20210015583 | Avisar | Jan 2021 | A1 |
| 20210022810 | Mahfouz | Feb 2021 | A1 |
| 20210058485 | Devam et al. | Mar 2021 | A1 |
| 20210086989 | Luxford | Apr 2021 | A1 |
| 20210118029 | Koritala | Jun 2021 | A1 |
| 20210183507 | Shaya | Oct 2021 | A1 |
| 20210326474 | Sparks et al. | Oct 2021 | A1 |
| 20210327304 | Buras | Nov 2021 | A1 |
| 20210350883 | Li et al. | Nov 2021 | A1 |
| 20210358068 | Boszczyk et al. | Nov 2021 | A1 |
| 20220020236 | Luthra et al. | Mar 2022 | A1 |
| 20220074000 | Wong | Mar 2022 | A1 |
| 20220101608 | Hu | Mar 2022 | A1 |
| Number | Date | Country |
|---|---|---|
| 103153248 | Jun 2013 | CN |
| 105266897 | Jan 2016 | CN |
| 3 872 813 | Sep 2021 | EP |
| 10-2008-0012745 | Feb 2008 | KR |
| 10-2018-0068703 | Jun 2018 | KR |
| WO 21044064 | Mar 2021 | WO |
| WO 21171226 | Sep 2021 | WO |
| Entry |
|---|
| Baruah, Basant, “Augmented reality and QR codes—What you need to know”, accessed via wayback machine for Apr. 14, 2021, beaconstac blog. |
| Uber expands options for prescription delivery with ScriptDrop, Mar. 24, 2021, Uber.com Blog (Year: 2021). |
| Zheng, Lixia, “Using Augmented Reality with Interactive Narrative to Help Paediatric Patients Take Prescription Medication”, 2020, University College London. |
| Office Action for U.S. Appl. No. 17/452,746 dated Jan. 5, 2022 in 15 pages. |
| International search report and Written opinion for PCT Application No. PCT/US2021/057116 dated Feb. 21, 2022 in 9 pages. |
| Office Action for U.S. Appl. No. 17/452,966 dated Feb. 18, 2022 in 42 pages. |
| Abbott, Dec. 2020, Binaxnow™ covid-19 Ag card home test and Navica™ app, https://web.archive.org/web/20201224151604/https://wwww.globalpointofcare/abbott, 3 pp. |
| U.S. Appl. No. 17/682,128 Final Office Action dated Apr. 6, 2023, pp. 1-10. |
| U.S. Appl. No. 17/682,128, Notice of Allowance dated Jun. 27, 2023, pp. 1-5. |
| U.S. Appl. No. 17/804,227 Non-Final Office Action dated Dec. 22, 2022, pp. 1-15. |
| U.S. Appl. No. 17/804,227, Final Office Action dated Apr. 12, 2023, pp. 1-18. |
| U.S. Appl. No. 17/682,128 Non-Final Office Action dated Feb. 21, 2023, pp. 1-8. |
| Number | Date | Country | |
|---|---|---|---|
| 20230049589 A1 | Feb 2023 | US |
| Number | Date | Country | |
|---|---|---|---|
| 63179864 | Apr 2021 | US | |
| 63156012 | Mar 2021 | US | |
| 63136575 | Jan 2021 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17681098 | Feb 2022 | US |
| Child | 17808365 | US | |
| Parent | 17452746 | Oct 2021 | US |
| Child | 17681098 | US | |
| Parent | 17452726 | Oct 2021 | US |
| Child | 17452746 | US |
