This disclosure relates generally to the field of data processing systems and more particularly to robotic process automation systems.
Robotic process automation (RPA) is the application of technology that allows workers in an organization to configure computer software, known as a “robot” to capture and interpret existing applications for processing a transaction, manipulating data, triggering responses and communicating with other digital systems. The software robots in conventional RPA systems employ the software robots to interpret the user interface of third-party applications and to execute steps identically to a human user. For example, many tasks within organizations require individuals to perform the same repetitive tasks, such as entering data from invoices into an enterprise accounts payable application or entering data from a loan application into a loan processing system. RPA permits the automation of such application level repetitive tasks via software robots that are coded to repeatedly and accurately perform the repetitive task.
When automating human tasks to use applications secured by multi-factor authentication (MFA), automation tools require additional factors other than a login identification and a password for login. There are different methods of MFA. While the form of the additional required factor varies, all MFA techniques require that the additional factor involve a credential or configuration to be first exchanged between the user and the application, and then be used based on the protocol defined by the selected MFA method. This requirement has typically restricted automation from being employed with applications employing MFA.
A robotic process automation (RPA) system provides bots that interact with and provide user credentials to applications, such as for multi-factor authentication (MFA). A first user credential associated with MFA is retrieved from credential storage. A second user credential that corresponds to questions posed to a user of an application is retrieved from credential storage. A second user credential that corresponds to a one-time password is generated by the RPA system from a security key stored in the credential storage. The second user credential may also be generated by a third party authentication service and is retrieved by the bot via a secondary channel such as email where the bot employs the user credential to access an email account associated with the user to retrieve the second user credential. The secondary channel may be an SMS message where the user has configured their mobile device with an application that recognizes the SMS message and provides the code in the SMS message to the RPA system which then provides the code to the bot.
These and additional aspects related to the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the description or may be learned by practice of the invention. Aspects of the invention may be realized and attained by means of the elements and combinations of various elements and aspects particularly pointed out in the following detailed description and the appended claims.
It is to be understood that both the foregoing and the following descriptions are exemplary and explanatory only and are not intended to limit the claimed invention or application thereof in any manner whatsoever.
The accompanying drawings, which are incorporated in and constitute a part of this specification exemplify the embodiments of the present invention and, together with the description, serve to explain and illustrate principles of the inventive techniques disclosed herein. Specifically:
In the following detailed description, reference will be made to the accompanying drawings, in which identical functional elements are designated with like numerals. Elements designated with reference numbers ending in a suffix such as .1, .2, .3 are referred to collectively by employing the main reference number without the suffix. For example, 100 refers to topics 100.1, 100.2, 100.3 generally and collectively. The aforementioned accompanying drawings show by way of illustration, and not by way of limitation, specific embodiments and implementations consistent with principles of the present invention. These implementations are described in sufficient detail to enable those skilled in the art to practice the invention and it is to be understood that other implementations may be utilized and that structural changes and/or substitutions of various elements may be made without departing from the scope and spirit of present invention. The following detailed description is, therefore, not to be construed in a limited sense.
In
MFA is an authentication method in which a computer user is granted access only after successfully presenting two or more pieces of evidence (or factors) to an authentication mechanism. The secondary (or tertiary) factors may fall into any one of three categories. First there is knowledge (something the user and only the user knows). An example of this is posing a question to the user, the answer to which only the user is likely to know. Second, is possession—something the user and only the user has. An example of this is a Time Based One-Time Password (TOTP) which is provided only to the user and which is valid for a brief period of time. The difference from knowledge is that the value sent to application is not the credential itself, but something generated from the credential. Third, is interaction where the secondary credential to be sent to an application is provided from the application via a communication method which only the user can access. An example of this is a One-Time Password (OTP) sent to a user via email.
In
Each client device 102 may take a variety of forms such as a physical machine, for example, a desktop computer, laptop computer or tablet. Each client device 102 may also be a virtual device such as provided by conventional virtualization software which permits multiple operating system sessions to be run on a single computer. Each client device 102 may have applications loaded thereon on local storage.
Some or all of the bots 134 may in certain embodiments be located remotely from the Control Room 130. Moreover, any of the devices 102 may also be located remotely from the Control Room 130. The bots 134 and the tasks 132 are shown in separate containers for purposes of illustration but they may be stored in separate or the same device(s), or across multiple devices. The Control Room 130 performs user management functions, source control of the bots 134, along with providing a dashboard that provides analytics and results of the bots 134, performs license management of software required by the bots 134 and manages overall execution and management of scripts, clients, roles, credentials, and security etc. The major functions performed by the Control Room 130 include: (i) a dashboard that provides a summary of registered/active users, tasks status, repository details, number of clients connected, number of scripts passed or failed recently, tasks that are scheduled to be executed and those that are in progress; (ii) user/role management—permits creation of different roles, such as bot creator, bot runner, admin, and custom roles, and activation, deactivation and modification of roles; (iii) repository management—to manage all scripts, tasks, workflows and reports etc.; (iv) operations management—permits checking status of tasks in progress and history of all tasks, and permits the administrator to stop/start execution of bots currently executing; (v) audit trail—logs creation of all actions performed in the Control Room; (vi) task scheduler—permits scheduling tasks which need to be executed on different clients at any particular time; (vii) credential management—permits password management; and (viii) security: management—permits rights management for all user roles. The Control Room 130 is shown generally for simplicity of explanation. Multiple instances of the Control Room 130 may be employed where large numbers of bots are deployed to provide for scalability of the RPA system 10.
The centralized credential vault (CV) 144 operates to securely store all credentials and provision them to bots on an on-demand basis. In one embodiment, the CV 144 implements National Institute of Standards and Technology (NIST) security and privacy controls IA-2 to uniquely identify and authenticate organizational users (or processes acting on behalf of organizational users). Since sensitive credentials need not be stored in bots or on bot runner systems, the CV 144 facilitates a logical separation of credentials from the bots. CV 144 variables are created from Control Room 130 and are instantly available to all the bot creators and bot runners registered with the respective Control Room 130. The CV 144 adds flexibility and dynamic character to bots since only the credential references are present in the bots and not the credentials. When bots are moved from one environment to another environment, absolutely no change is needed in bots. Bots can seamlessly pick up the credential values applicable for the new environment from the Control Room 130 of that environment. Additionally, the Control Room 130 automatically stores configuration related sensitive data into the CV 144 by default. Additional details of the credential vault 144 are described in U.S. patent application Ser. No. 15/957,917 entitled “ROBOTIC PROCESS AUTOMATION SYSTEM AND METHOD WITH SECURE CREDENTIAL VAULT” which is assigned to the assignee of the present application and which is hereby incorporated by reference in its entirety.
The disclosed embodiments permit the system 10 to deploy bots 134 that interact with applications, such as application 103, that employ MFA, to permit tasks employing MFA enabled applications to be automated with bots 134. Without the RPA system 10 disclosed herein, use of applications that employ MFA cannot be automated, or a user 101 needs to disable MFA on the target application, which leads to lower security.
Operation of the MFA feature of the system 10 is described with reference to
Subsequently, when the bot 134 that has been coded to respond to a request for MFA, such as Bot 1 executing on Device 1, the bot 134 operates with the credentials of a user, such as user 101, to perform operations with the user 101's credentials as if the user were performing the operations which the bot 134 has been coded to perform. The bot 134 responds to a request by the application 103 for a first set of credentials corresponding to those of user 101 (210), such as a username and password. The bot 134, which has been provided with RPA system credentials of user 101 employs the RPA system credentials of user 101 to access credential vault 144 (212) for user 101's first set of credentials corresponding to application 103 to provide such credentials to application 103 (216) which are then validated by validate module 141 of application 103. In some embodiments the foregoing described request for the user's first set of credentials may be generated by the MFA Factor Provider 142 which the application may then provide to the bot 134. Once the application has received a first set of credentials corresponding to the expected credentials for user 101, the MFA Factor Provider 142 will generate a request for second credentials for user 101 in connection with application 103 to application 103. This request will be provided by application 103 to bot 134 (218) which will then provide the second credentials for user 101 in accordance with the second credentials specified for application 103 by user 101.
The second credentials for user 101 for application 103 may be one of several types, four of which are shown in
The second credentials may be a code generated by the MFA Factor Provider 142 and provided to an account or device associated with the user 101 such as by way of an email message to an email account of the user 101 (310) or by way of an SMS message provided to a mobile device 150 of the user 101 (314). If the second factor authentication is a code sent via email by the MFA Factor Provider 142 to an email account 151 provided by user 101, the bot 134, which receives the request via application 103, connects to the user 101's email mailbox and retrieves the one-time password sent from the MFA Factor Provider 142 (311) and provides the retrieved password to the application 103 (312). If the code is provided by the MFA Factor Provider 142 to the user's mobile device 150 via SMS, then an application 152 installed under user 101's permission to the mobile device 150 operates to recognize the received SMS by monitoring received SMS messages on the device 150 (315). The application 152 is one embodiment is provided by the RPA system 10, instead of by a third-party, to ensure security of the user's environment on the device 150 and other devices and applications with which the device 150 interacts. The application 152 provides the received OTP or code to the control room 130 (316) which provides it to the bot 134 (317) which then proceeds to provide the received OTP or code to the application 103 (318) to satisfy the second factor generated by the MFA Factor Provider 142.
Computing system 400 may have additional features such as for example, storage 410, one or more input devices 414, one or more output devices 412, and one or more communication connections 416. An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computing system 400. Typically, operating system software (not shown) provides an operating system for other software executing in the computing system 400, and coordinates activities of the components of the computing system 400.
The tangible storage 410 may be removable or non-removable, and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, DVDs, or any other medium which can be used to store information in a non-transitory way, and which can be accessed within the computing system 400. The storage 410 stores instructions for the software implementing one or more innovations described herein.
The input device(s) 414 may be a touch input device such as a keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or another device that provides input to the computing system 400. For video encoding, the input device(s) 414 may be a camera, video card, TV tuner card, or similar device that accepts video input in analog or digital form, or a CD-ROM or CD-RW that reads video samples into the computing system 400. The output device(s) 412 may be a display, printer, speaker, CD-writer, or another device that provides output from the computing system 400.
The communication connection(s) 416 enable communication over a communication medium to another computing entity. The communication medium conveys information such as computer-executable instructions, audio or video input or output, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can use an electrical, optical, RF, or other carrier.
The terms “system” and “computing device” are used interchangeably herein. Unless the context clearly indicates otherwise, neither term implies any limitation on a type of computing system or computing device. In general, a computing system or computing device can be local or distributed and can include any combination of special-purpose hardware and/or general-purpose hardware with software implementing the functionality described herein.
While the invention has been described in connection with a preferred embodiment, it is not intended to limit the scope of the invention to the particular form set forth, but on the contrary, it is intended to cover such alternatives, modifications, and equivalents as may be within the spirit and scope of the invention as defined by the appended claims.
This application is a continuation of U.S. patent application Ser. No. 16/779,117, filed Jan. 31, 2020, and entitled “AUTOMATION OF WORKLOADS INVOLVING APPLICATIONS EMPLOYING MULTI-FACTOR AUTHENTICATION,” the content of which is hereby incorporated by reference.
Number | Name | Date | Kind |
---|---|---|---|
5949999 | Song et al. | Sep 1999 | A |
5983001 | Boughner et al. | Nov 1999 | A |
6118939 | Nack | Sep 2000 | A |
6133917 | Feigner et al. | Oct 2000 | A |
6226407 | Zabih et al. | May 2001 | B1 |
6389592 | Ayres et al. | May 2002 | B1 |
6427234 | Chambers et al. | May 2002 | B1 |
6473794 | Guheen et al. | Oct 2002 | B1 |
6496979 | Chen et al. | Dec 2002 | B1 |
6640244 | Bowman-Amuah | Oct 2003 | B1 |
6704873 | Underwood | Mar 2004 | B1 |
6898764 | Kemp | May 2005 | B2 |
6954747 | Wang et al. | Oct 2005 | B1 |
6957186 | Guheen et al. | Oct 2005 | B1 |
7091898 | Arling et al. | Aug 2006 | B2 |
7246128 | Jordahl | Jul 2007 | B2 |
7398469 | Kisamore et al. | Jul 2008 | B2 |
7441007 | Kirkpatrick et al. | Oct 2008 | B1 |
7533096 | Rice et al. | May 2009 | B2 |
7568109 | Powell et al. | Jul 2009 | B2 |
7571427 | Wang et al. | Aug 2009 | B2 |
7765525 | Davidson et al. | Jul 2010 | B1 |
7783135 | Gokturk | Aug 2010 | B2 |
7805317 | Khan et al. | Sep 2010 | B2 |
7805710 | North | Sep 2010 | B2 |
7810070 | Nasuti et al. | Oct 2010 | B2 |
7846023 | Evans et al. | Dec 2010 | B2 |
8028269 | Bhatia et al. | Sep 2011 | B2 |
8056092 | Allen et al. | Nov 2011 | B2 |
8095910 | Nathan et al. | Jan 2012 | B2 |
8132156 | Malcolm | Mar 2012 | B2 |
8209738 | Nicol et al. | Jun 2012 | B2 |
8234622 | Meijer et al. | Jul 2012 | B2 |
8245215 | Extra | Aug 2012 | B2 |
8352464 | Fotev | Jan 2013 | B2 |
8365147 | Grechanik | Jan 2013 | B2 |
8396890 | Lim | Mar 2013 | B2 |
8438558 | Adams | May 2013 | B1 |
8443291 | Ku et al. | May 2013 | B2 |
8464240 | Fritsch et al. | Jun 2013 | B2 |
8498473 | Chong et al. | Jul 2013 | B2 |
8504803 | Shukla | Aug 2013 | B2 |
8631458 | Banerjee | Jan 2014 | B1 |
8682083 | Kumar et al. | Mar 2014 | B2 |
8713003 | Fotev | Apr 2014 | B2 |
8724907 | Sampson et al. | May 2014 | B1 |
8769482 | Batey et al. | Jul 2014 | B2 |
8819241 | Washbum | Aug 2014 | B1 |
8819253 | Simeloff | Aug 2014 | B2 |
8832048 | Lim | Sep 2014 | B2 |
8874685 | Hollis et al. | Oct 2014 | B1 |
8943493 | Schneider | Jan 2015 | B2 |
8965905 | Ashmore et al. | Feb 2015 | B2 |
8966458 | Asai | Feb 2015 | B2 |
9032314 | Mital et al. | May 2015 | B2 |
9104294 | Forstall et al. | Aug 2015 | B2 |
9171359 | Lund | Oct 2015 | B1 |
9213625 | Schrage | Dec 2015 | B1 |
9251413 | Meler | Feb 2016 | B2 |
9278284 | Ruppert et al. | Mar 2016 | B2 |
9444844 | Edery et al. | Sep 2016 | B2 |
9462042 | Shukla et al. | Oct 2016 | B2 |
9571332 | Subramaniam et al. | Feb 2017 | B2 |
9600519 | Schoning et al. | Mar 2017 | B2 |
9621584 | Schmidt et al. | Apr 2017 | B1 |
9934063 | Kaina | Apr 2018 | B2 |
9934129 | Budurean | Apr 2018 | B1 |
9946233 | Brun et al. | Apr 2018 | B2 |
9965139 | Nychis | May 2018 | B2 |
9990347 | Raskovic et al. | Jun 2018 | B2 |
10015503 | Ahammad | Jul 2018 | B1 |
10043255 | Pathapati et al. | Aug 2018 | B1 |
10282280 | Gouskova | May 2019 | B1 |
10489682 | Kumar et al. | Nov 2019 | B1 |
10542030 | Oberheide | Jan 2020 | B2 |
10552540 | Marascu et al. | Feb 2020 | B2 |
10592738 | Northrup | Mar 2020 | B2 |
10643246 | Suprasadachandran | May 2020 | B1 |
10654166 | Hall | May 2020 | B1 |
10706218 | Milward et al. | Jul 2020 | B2 |
10706228 | Buisson | Jul 2020 | B2 |
10713068 | Zohar | Jul 2020 | B1 |
10857786 | Korthuis | Dec 2020 | B2 |
10936807 | Walters | Mar 2021 | B1 |
10963731 | Sarkar | Mar 2021 | B1 |
10970109 | Berenstein | Apr 2021 | B1 |
11099972 | Puszkiewicz | Aug 2021 | B2 |
11176443 | Selva | Nov 2021 | B1 |
11182178 | Singh et al. | Nov 2021 | B1 |
11182604 | Methaniya | Nov 2021 | B1 |
11243803 | Anand et al. | Feb 2022 | B2 |
11263391 | Potts | Mar 2022 | B2 |
11348353 | Sundell et al. | May 2022 | B2 |
11494227 | George | Nov 2022 | B2 |
11500952 | Gupta | Nov 2022 | B1 |
11513670 | Tripathi | Nov 2022 | B2 |
11614731 | Anand et al. | Mar 2023 | B2 |
11642783 | Han et al. | May 2023 | B2 |
11775321 | Singh et al. | Oct 2023 | B2 |
11775339 | Anand et al. | Oct 2023 | B2 |
11775814 | Anand et al. | Oct 2023 | B1 |
11782734 | Ginoya et al. | Oct 2023 | B2 |
11820020 | Goyal et al. | Nov 2023 | B2 |
12002245 | Srivastava | Jun 2024 | B2 |
20020029232 | Bobrow et al. | Mar 2002 | A1 |
20020129039 | Majewsku | Sep 2002 | A1 |
20030033590 | Leherbauer | Feb 2003 | A1 |
20030101245 | Srinivasan et al. | May 2003 | A1 |
20030114959 | Sakamoto | Jun 2003 | A1 |
20030159089 | DiJoseph | Aug 2003 | A1 |
20040083472 | Rao et al. | Apr 2004 | A1 |
20040122906 | Goodman | Jun 2004 | A1 |
20040153649 | Rhoads | Aug 2004 | A1 |
20040172526 | Tann et al. | Sep 2004 | A1 |
20040210885 | Wang et al. | Oct 2004 | A1 |
20040243994 | Nasu | Dec 2004 | A1 |
20050033777 | Moraes | Feb 2005 | A1 |
20050188357 | Derks et al. | Aug 2005 | A1 |
20050204343 | Kisamore et al. | Sep 2005 | A1 |
20050257214 | Moshir et al. | Nov 2005 | A1 |
20060095276 | Axelrod et al. | May 2006 | A1 |
20060150188 | Roman et al. | Jul 2006 | A1 |
20060218110 | Simske et al. | Sep 2006 | A1 |
20070030528 | Quaeler et al. | Feb 2007 | A1 |
20070089101 | Romanovskiy | Apr 2007 | A1 |
20070101291 | Forstall et al. | May 2007 | A1 |
20070112574 | Greene | May 2007 | A1 |
20070156677 | Szabo | Jul 2007 | A1 |
20070172063 | Biggs | Jul 2007 | A1 |
20070233741 | Shen | Oct 2007 | A1 |
20080005086 | Moore | Jan 2008 | A1 |
20080027769 | Eder | Jan 2008 | A1 |
20080028392 | Chen et al. | Jan 2008 | A1 |
20080133052 | Jones | Jun 2008 | A1 |
20080209392 | Able et al. | Aug 2008 | A1 |
20080222454 | Kelso | Sep 2008 | A1 |
20080263024 | Landschaft et al. | Oct 2008 | A1 |
20080310625 | Vanstone | Dec 2008 | A1 |
20090037509 | Parekh et al. | Feb 2009 | A1 |
20090103769 | Milov et al. | Apr 2009 | A1 |
20090116071 | Mantell | May 2009 | A1 |
20090172814 | Khosravi et al. | Jul 2009 | A1 |
20090199160 | Vaitheeswaran et al. | Aug 2009 | A1 |
20090217309 | Grechanik et al. | Aug 2009 | A1 |
20090249297 | Doshi et al. | Oct 2009 | A1 |
20090313229 | Fellenstein et al. | Dec 2009 | A1 |
20090320002 | Peri-Glass et al. | Dec 2009 | A1 |
20100023602 | Martone | Jan 2010 | A1 |
20100023933 | Bryant et al. | Jan 2010 | A1 |
20100077475 | Deschenes | Mar 2010 | A1 |
20100100605 | Allen et al. | Apr 2010 | A1 |
20100106671 | Li et al. | Apr 2010 | A1 |
20100138015 | Colombo et al. | Jun 2010 | A1 |
20100235433 | Ansari et al. | Sep 2010 | A1 |
20100251163 | Keable | Sep 2010 | A1 |
20110022578 | Fotev | Jan 2011 | A1 |
20110055484 | Eichenberger | Mar 2011 | A1 |
20110106284 | Catoen | May 2011 | A1 |
20110145807 | Molinie et al. | Jun 2011 | A1 |
20110197121 | Kletter | Aug 2011 | A1 |
20110258195 | Welling | Oct 2011 | A1 |
20110267490 | Goktekin | Nov 2011 | A1 |
20110276568 | Fotev | Nov 2011 | A1 |
20110276946 | Pletter | Nov 2011 | A1 |
20110302570 | Kurimilla et al. | Dec 2011 | A1 |
20120011458 | Xia et al. | Jan 2012 | A1 |
20120042281 | Green | Feb 2012 | A1 |
20120124062 | Macbeth et al. | May 2012 | A1 |
20120131456 | Lin et al. | May 2012 | A1 |
20120143941 | Kim | Jun 2012 | A1 |
20120266149 | Lebert | Oct 2012 | A1 |
20120324333 | Lehavi | Dec 2012 | A1 |
20120330940 | Caire et al. | Dec 2012 | A1 |
20130145006 | Tammam | Jun 2013 | A1 |
20130173648 | Tan et al. | Jul 2013 | A1 |
20130227535 | Kannan | Aug 2013 | A1 |
20130236111 | Pintsov | Sep 2013 | A1 |
20130290318 | Shapira et al. | Oct 2013 | A1 |
20130301935 | Xue | Nov 2013 | A1 |
20140036290 | Miyagawa | Feb 2014 | A1 |
20140045484 | Kim et al. | Feb 2014 | A1 |
20140075371 | Carmi | Mar 2014 | A1 |
20140181705 | Hey et al. | Jun 2014 | A1 |
20140189576 | Carmi | Jul 2014 | A1 |
20140282125 | Duneau | Sep 2014 | A1 |
20140379666 | Bryon | Dec 2014 | A1 |
20150082280 | Betak et al. | Mar 2015 | A1 |
20150206139 | Lea | Jul 2015 | A1 |
20150235193 | Cummings | Aug 2015 | A1 |
20150290795 | Oleynik | Oct 2015 | A1 |
20150301926 | Giannelos | Oct 2015 | A1 |
20150310268 | He | Oct 2015 | A1 |
20150347284 | Hey et al. | Dec 2015 | A1 |
20160019049 | Kakhandiki et al. | Jan 2016 | A1 |
20160034441 | Nguyen et al. | Feb 2016 | A1 |
20160055376 | Koduru | Feb 2016 | A1 |
20160063269 | Liden | Mar 2016 | A1 |
20160078366 | Kakhandiki | Mar 2016 | A1 |
20160078368 | Kakhandiki et al. | Mar 2016 | A1 |
20170091615 | Liu | Mar 2017 | A1 |
20170221032 | Mazed | Aug 2017 | A1 |
20170270431 | Hosabettu | Sep 2017 | A1 |
20170352041 | Ramamurthy | Dec 2017 | A1 |
20180113781 | Kim | Apr 2018 | A1 |
20180144042 | Sheng | May 2018 | A1 |
20180176416 | Ewans | Jun 2018 | A1 |
20180210824 | Kochura | Jul 2018 | A1 |
20180218429 | Guo et al. | Aug 2018 | A1 |
20180267885 | Budurean | Sep 2018 | A1 |
20180275835 | Prag | Sep 2018 | A1 |
20180276462 | Davis | Sep 2018 | A1 |
20180321955 | Liu | Nov 2018 | A1 |
20180322403 | Ron | Nov 2018 | A1 |
20180349730 | Dixon | Dec 2018 | A1 |
20180370029 | Hall | Dec 2018 | A1 |
20190005050 | Proux | Jan 2019 | A1 |
20190026215 | Agarwal | Jan 2019 | A1 |
20190028587 | Unitt | Jan 2019 | A1 |
20190034041 | Nychis | Jan 2019 | A1 |
20190066012 | Abell | Feb 2019 | A1 |
20190075169 | Behunin | Mar 2019 | A1 |
20190095440 | Chakra | Mar 2019 | A1 |
20190114370 | Cerino | Apr 2019 | A1 |
20190126463 | Purushothaman | May 2019 | A1 |
20190130094 | Votaw | May 2019 | A1 |
20190141596 | Gay | May 2019 | A1 |
20190188462 | Nishida | Jun 2019 | A1 |
20190205636 | Saraswat | Jul 2019 | A1 |
20190213822 | Jain | Jul 2019 | A1 |
20190250891 | Kumar | Aug 2019 | A1 |
20190266692 | Stach et al. | Aug 2019 | A1 |
20190317803 | Maheshwari | Oct 2019 | A1 |
20190324781 | Ramamurthy | Oct 2019 | A1 |
20190340240 | Duta | Nov 2019 | A1 |
20190377987 | Price et al. | Dec 2019 | A1 |
20200019767 | Porter et al. | Jan 2020 | A1 |
20200034976 | Stone et al. | Jan 2020 | A1 |
20200059441 | Viet | Feb 2020 | A1 |
20200065384 | Costello | Feb 2020 | A1 |
20200074070 | Boodaei | Mar 2020 | A1 |
20200097742 | Kumar et al. | Mar 2020 | A1 |
20200104350 | Allen | Apr 2020 | A1 |
20200125635 | Nuolf | Apr 2020 | A1 |
20200147791 | Safary | May 2020 | A1 |
20200151444 | Price et al. | May 2020 | A1 |
20200151591 | Li | May 2020 | A1 |
20200159647 | Puszkiewicz | May 2020 | A1 |
20200159648 | Ghare | May 2020 | A1 |
20200206920 | Ma | Jul 2020 | A1 |
20200249964 | Fernandes | Aug 2020 | A1 |
20200257764 | Reisswig | Aug 2020 | A1 |
20200273078 | Xu | Aug 2020 | A1 |
20200285353 | Rezazadeh Sereshkeh | Sep 2020 | A1 |
20200311210 | Nama | Oct 2020 | A1 |
20200334249 | Canim | Oct 2020 | A1 |
20200364291 | Bentabet | Nov 2020 | A1 |
20200410082 | Sharieh | Dec 2020 | A1 |
20210042516 | Panakkal | Feb 2021 | A1 |
20210049128 | Kernick | Feb 2021 | A1 |
20210067492 | Mastracci | Mar 2021 | A1 |
20210107140 | Singh | Apr 2021 | A1 |
20210141497 | Magureanu | May 2021 | A1 |
20210216334 | Barrett | Jul 2021 | A1 |
20210248153 | Sirangimoorthy | Aug 2021 | A1 |
20210279166 | Peng | Sep 2021 | A1 |
20210333983 | Singh | Oct 2021 | A1 |
20210391004 | Yudanov | Dec 2021 | A1 |
20220038445 | LeClair | Feb 2022 | A1 |
20220245936 | Valk | Aug 2022 | A1 |
20220405094 | Farquhar | Dec 2022 | A1 |
20220413997 | Shi | Dec 2022 | A1 |
20230052190 | Goyal et al. | Feb 2023 | A1 |
20230053260 | Goyal et al. | Feb 2023 | A1 |
20230214239 | Singh | Jul 2023 | A1 |
20230419711 | Ramnath | Dec 2023 | A1 |
20240056479 | Senecal | Feb 2024 | A1 |
Number | Date | Country |
---|---|---|
3133539 | Feb 2017 | EP |
3675008 | Jul 2020 | EP |
2019092672 | May 2019 | WO |
2022076488 | Apr 2022 | WO |
Entry |
---|
NPL Search Terms (Year: 2024). |
Al Sallami, Load Balancing in Green Cloud Computation, Proceedings of the World Congress on Engineering 2013 vol. II, WCE 2013, 2013, pp. 1-5 (Year: 2013). |
B.P. Kasper “Remote: A Means of Remotely Controlling and Storing Data from a HAL Quadrupole Gass Analyzer Using an IBM-PC Compatible Computer”, Nov. 15, 1995, Space and Environment Technology Center. |
Bergen et al., RPC automation: making legacy code releant, May 2013, 6 pages. |
Hu et al., Automating GUI testing for Android applications, May 2011, 7 pages. |
Konstantinou et al., An architecture for virtual solution composition and deployment in infrastructure clouds, 9 pages (Year: 2009). |
Nyulas et al., An Ontology-Driven Framework for Deploying JADE Agent Systems, 5 pages (Year: 2006). |
Tom Yeh, Tsung-Hsiang Chang, and Robert C. Miller, Sikuli: Using GUI Screenshots for Search and Automation, Oct. 4-7, 2009, 10 pages. |
Yu et al., Deplying and managing Web services: issues, solutions, and directions, 36 pages (Year: 2008). |
Zhifang et al., Test automation on moble device, May 2010, 7 pages. |
Non-Final Office Action for U.S. Appl. No. 17/230,492, mailed Oct. 14, 2022. |
Notice of Allowance for U.S. Appl. No. 16/398,532, mailed Oct. 23, 2022. |
Non-Final Office Action for U.S. Appl. No. 16/876,530, mailed Sep. 29, 2020. |
Final Office Action for U.S. Appl. No. 16/876,530, mailed Apr. 13, 2021. |
Notice of Allowance for U.S. Appl. No. 16/876,530, mailed Jul. 22, 2021. |
Dai, Jifeng et al., “R-fcn: Object detectiom via region-based fully convolutional networks”, Advances in neural information processing systems 29 (2016). (Year: 2016). |
“Ren, Shaoqing et al., Faster r-cnn: Towards real0time object detection with region proposal network.” Advances in neutral information processing systems 28 (2015). (Year: 2015). |
Zhifang et al., Test automation on mobile device, May 2010, 7 pages. |
International Search Report for PCT/US2021/053669, dated May 11, 2022. |
Embley et al., “Table-processing paradigms: a research survey, International Journal on Document Analysis and Recognition”, vol. 8, No. 2-3, May 9, 2006, pp. 66-86. |
Non-Final Office Action for U.S. Appl. No. 16/925,956, mailed Sep. 16, 2021. |
Notice of Allowance for U.S. Appl. No. 16/925,956, mailed Jan. 7, 2022. |
Pre-Interview Office Action for U.S. Appl. No. 16/398,532, mailed Jul. 8, 2022. |
Notice of Allowance for U.S. Appl. No. 16/398,532, mailed Oct. 13, 2022. |
Non-Final Office Action for U.S. Appl. No. 17/139,838, mailed Feb. 22, 2022. |
Final Office Action for U.S. Appl. No. 17/139,838, mailed Nov. 15, 2022. |
Notice of Allowance for U.S. Appl. No. 17/139,838, mailed Apr. 5, 2023. |
International Search Report and Written Opinion for PCT/US2021/015691, mailed May 11, 2021. |
A density-based algorithm for discovering clusters in large spatial databases with noise, Ester, Martin; Kriegel, Hans-Peter; Sander, Jorg; Xu, Xiaowei, Simoudis, Evangelos; Han, Jiawei; Fayyad, Usama M., eds., Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96). AMI Press. pp. 226-231 (1996). |
Deep Residual Learning for Image Recognition, by K. He, X. Zhang, S. Ren, and J. Sun, arXiv: 1512.03385 (2015). |
FaceNet: A Unified Embedding for Face Recognition and Clustering, by F. Schroff, D. Kalenichenko, J. Philbin, arXiv:1503.03832 (2015). |
Luqman, et al. “Fuzzy multilevel graph embedding”, copyright 2012 Elsevier Ltd. |
Sharma et al. Determining similarity in histological images using graph-theoretic description and matching methods for content-based image retrieval in medical diagnostics, Biomed Center, copyright 2012. |
First Action Interview Pilot Program Pre-Interview communication for U.S. Appl. No. 16/779,462, mailed Dec. 3, 2021. |
Reply under 37 CDT 1.111 to Pre-Interview Communication for U.S. Appl. No. 16/779,462, filed Jan. 25, 2022. |
Notice of Allowance for U.S. Appl. No. 16/779,462 mailed Feb. 9, 2022. |
Notice of Allowance for U.S. Appl. No. 17/131,674, mailed Jun. 22, 2023. |
Non-Final Office Action for U.S. Appl. No. 16/731,044, mailed Jan. 25, 2021. |
Notice of Allowance for U.S. Appl. No. 16/731,044, mailed May 5, 2021. |
Non-Final Office Action for U.S. Appl. No. 18/126,935, mailed Jul. 13, 2023. |
Non-Final Office Action for U.S. Appl. No. 17/139,842, mailed Jul. 18, 2023. |
Notice of Allowance for U.S. Appl. No. 17/588,588, mailed Aug. 2, 2023. |
Pre-Interview Office Action for U.S. Appl. No. 16/859,488, mailed Jan. 25, 2021. |
First Action Interview for U.S. Appl. No. 16/859,488, mailed Mar. 22, 2021. |
Final Office Action for U.S. Appl. No. 16/859,488, mailed Jul. 8, 2021. |
Notice of Allowance for U.S. Appl. No. 16/859,488, mailed Mar. 30, 2022. |
Final Office Action for U.S. Appl. No. 17/463,494, mailed Sep. 6, 2023. |
Final Office Action for U.S. Appl. No. 17/160,080, mailed Sep. 11, 2023. |
Final Office Action for U.S. Appl. No. 17/534,443, mailed Sep. 11, 2023. |
Final Office Action for U.S. Appl. No. 16/930,247 mailed Oct. 12, 2023. |
Notice of Allowance for U.S. Appl. No. 17/534,443 mailed Oct. 24, 2023. |
International Search Report and Written Opinion for PCT/US2022/013026, mailed Sep. 21, 2022. |
Notice of Allowance for U.S. Appl. No. 18/126,935, mailed Jan. 10, 2024. |
Notice of Allowance for U.S. Appl. No. 18/215,126, mailed Jan. 25, 2024. |
Advisory Action and After Final Pilot Decision for U.S. Appl. No. 17/160,080, mailed Jan. 29, 2024. |
Non-Final Office Action for U.S. Appl. No. 17/160,082, mailed Feb. 15, 2024. |
Final Office Action for U.S. Appl. No. 17/206,029, mailed Feb. 28, 2024. |
Non-Final Office Action for U.S. Appl. No. 17/160,080, mailed Mar. 13, 2024. |
Non-Final Office Action for U.S. Appl. No. 17/160,084, mailed Mar. 15, 2024. |
Non-Final Office Action for U.S. Appl. No. 16/779,117, mailed Jan. 12, 2022. |
Notice of Allowance for U.S. Appl. No. 16/779,117, mailed Aug. 10, 2022. |
Non-Final Office Action for U.S. Appl. No. 18/094,305, mailed May 9, 2024. |
Final Office Action for U.S. Appl. No. 17/160,084, mailed May 16, 2024. |
Non-Final Office Action for U.S. Appl. No. 18/218,009, mailed May 20, 2024. |
Final Office Action for U.S. Appl. No. 17/160,080, mailed Jun. 12, 2024. |
Notice of Allowance for U.S. Appl. No. 17/389,300, mailed Jun. 27, 2024. |
Non-Final Office Action for U.S. Appl. No. 17/160,082, filed Jul. 24, 2024. |
Final Office Action for U.S. Appl. No. 18/094,305, mailed Jul. 29, 2024. |
Non-Final Office Action for U.S. Appl. No. 18/103,331, mailed Aug. 15, 2024. |
Notice of Allowance for 16/458,139, mailed Sep. 11, 2024. |
Notice of Allowance for 16/458,138, mailed Sep. 16, 2024. |
Notice of Allowance for U.S. Appl. No. 17/160,082, mailed Sep. 18, 2024. |
Notice of Allowance for 16/398,532, mailed Oct. 23, 2022. |
Non-Final Office Action for U.S. Appl. No. 17/988,660, mailed Sep. 25, 2022. |
Non-Final Office Action for U.S. Appl. No. 17/160,080, mailed Oct. 16, 2024. |
Number | Date | Country | |
---|---|---|---|
20230082185 A1 | Mar 2023 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16779117 | Jan 2020 | US |
Child | 17988660 | US |