This disclosure relates generally to the field of data processing systems and more particularly to robotic process automation systems.
Robotic process automation (RPA) is the application of technology that allows workers in an organization to configure computer software, known as a “robot” to capture and interpret existing applications for processing a transaction, manipulating data, triggering responses and communicating with other digital systems. The software robots in conventional RPA systems employ the software robots to interpret the user interface of third-party applications and to execute steps identically to a human user. For example, many tasks within organizations require individuals to perform the same repetitive tasks, such as entering data from invoices into an enterprise accounts payable application or entering data from a loan application into a loan processing system. RPA permits the automation of such application level repetitive tasks via software robots that are coded to repeatedly and accurately perform the repetitive task.
Many tasks in a business environment are performed within productivity programs, such as a spreadsheet—Microsoft Excel® for example. Automating tasks performed within a spreadsheet by way of RPA exists, but can be cumbersome. There is accordingly a need to simplify the automation of tasks performed in a productivity program, such as a spreadsheet program in order to increase the efficiency and speed with which computerized systems may process operations within a file associated with a productivity program.
Robotic process automation (RPA) tasks for operating on data within a productivity program may be initiated by single user action from within the productivity program. A user device is provided with a plugin program that provides an automation user interface within the productivity program. A request to perform an automation task on data in a productivity file is received along with identification of a software robot to perform the automation task. The request also specifies data from the productivity file. The request is provided to a control room server that controls execution of a plurality of different automation tasks by a plurality of different software robots. Results of the automation task are received from the control room server and are provided to the plugin program, which provides the results of the automation task to the user within a productivity file accessible by the user.
These and additional aspects related to the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the description or may be learned by practice of the invention. Aspects of the invention may be realized and attained by means of the elements and combinations of various elements and aspects particularly pointed out in the following detailed description and the appended claims.
It is to be understood that both the foregoing and the following descriptions are exemplary and explanatory only and are not intended to limit the claimed invention or application thereof in any manner whatsoever.
The accompanying drawings, which are incorporated in and constitute a part of this specification exemplify the embodiments of the present invention and, together with the description, serve to explain and illustrate principles of the inventive techniques disclosed herein. Specifically:
In the following detailed description, reference will be made to the accompanying drawings, in which identical functional elements are designated with like numerals. Elements designated with reference numbers ending in a suffix such as .1, .2, .3 are referred to collectively by employing the main reference number without the suffix. For example, 100 refers to topics 100.1, 100.2, 100.3 generally and collectively. The aforementioned accompanying drawings show by way of illustration, and not by way of limitation, specific embodiments and implementations consistent with principles of the present invention. These implementations are described in sufficient detail to enable those skilled in the art to practice the invention and it is to be understood that other implementations may be utilized and that structural changes and/or substitutions of various elements may be made without departing from the scope and spirit of present invention. The following detailed description is, therefore, not to be construed in a limited sense.
In
A user interface (U/I) plugin 108 operates to permit user 101 to simply, with one-click, run directly from the spreadsheet, a software robot, such as a bot 134 (described below) to perform a task on data in a spreadsheet and to receive the results, either in the spreadsheet file from which the task was performed, or in another spreadsheet, or elsewhere. The spreadsheet file is referred to herein in the context of an embodiment where the productivity program from which the bot is invoked is a spreadsheet program. The spreadsheet file is an embodiment of a productivity file which in other embodiments may take the form of a word processing file or a presentation file (e.g. such as produced by Powerpoint® from Microsoft Corp. or Keynote® from Apple Inc. or similar programs). Data in the spreadsheet is passed by the plugin 108 to the selected bot 134 and results are received by the plugin 108 from the bot 134. The plugin 108 advantageously operates with either a desktop-based spreadsheet, or a web-based spreadsheet such as provided in the Office 365™ application from Microsoft Corporation. In certain embodiments, the plugin 108 is not limited to operation with a single platform or product such as available from Microsoft Corporation but rather is agnostic to the operating system on which it runs and may work on other platforms such as the MacOS® available from Apple Inc., the Android® operating system available from Google, Inc. or the Linux® operating system available from a variety of sources, and with spreadsheet applications other than Excel®.
The plugin 108 communicates with modules 112, 113, 114, and 115 that are part of an Application Program Interface (API) 110 that permits applications to communicate with object models that support the application and that execute on a server-based host. An example of such an API is the Office JavaScript API available from Microsoft Corp., which permits desktop and web-based applications to interact with object models in Office host applications. The application references an office.js library in module 118 described below, which is a script loader. The office.js library loads the object models that are applicable to the Office application that is running an add-in, such as plugin 108. Any number of supported JavaScript object models may be employed. Further information on the Office JavaScript API is available from Microsoft Corp. and may be found at docs.microsoft.com.
Specifically, the API 110 includes a state module 112 that tracks user state, a page renderer 113 that renders the user-interface as needed by the application 104/106 based on the application and physical aspects of the screen on which the application is displaying output. A Write to and Interact with Spreadsheet module 114 operates to write data to a specified spreadsheet and otherwise interact with the specified spreadsheet. A Gather Spreadsheet Detail module 115 operates to transfer data and meta-data from a spreadsheet being used by a user 101 for transmittal to relay server 120. The state module 112 and page renderer 113 interact with a conventional web server 116 which includes conventional web serving capabilities such as providing files of data and code encoded in a variety of conventional encodings 118 such as HyperText Markup Language (HTML), Cascading Style Sheets (CSS), and JavaScript (JS). The Write to and Interact with Spreadsheet module 114 and the Gather Spreadsheet Detail module 115 interact with Relay Server 120, described in further detail below. In one embodiment, the foregoing modules 112, 113, 114, 115 may be implemented by the following API's available in the Microsoft Office JavaScript API, available from Microsoft Corporation: Manage State (module 112), Page Render (module 113), Gather Spread Sheet Data (module 115), Write to and interact with Excel (module 114).
Relay Server 120 operates to facilitate communication between a spreadsheet program being used by user 101 and the RPA system 10, and specifically Control Room 130 which controls the operations of the system 10. The Web Server 116 and the Relay Server 120 are both shown executing on a server 121. This is shown for purposes of illustration and the servers 116 and 121 may execute on separate servers or may each be distributed across more than one server, including physical and virtual machines.
The robotic process automation system 10 includes data storage, seen generally at 136 which stores a plurality of sets of task processing instructions 134. Each set of task processing instructions 134 implements a software robot, also referred to as a bot (seen as Bot 1, Bot 2, . . . , Bot n) which is operable to interact at a user level with one or more designated user level application programs, such as generally shown at 146. As used herein, the term “bot” is generally synonymous with the term software robot. In certain contexts, as will be apparent to those skilled in the art in view of the present disclosure, the term “bot runner” refers to a device (virtual or physical), having the necessary software capability, on which a bot will execute or is executing, such as for example a device 140 loaded with and executing a bot 134. The data storage 136 also stores a plurality of work items 132, where each work item 132 is stored for subsequent processing by executing a corresponding set of task processing instructions 134 (also referred to as a bot which implements each set of task processing instructions). A Control Room, seen generally at 130, is operatively coupled to the data storage 136 and is configured to execute instructions that when executed cause the RPA system 10 to respond to a request from a client device 140 that is issued by a user 101 to act as a server to provide to the client device 140 the capability to perform an automation task to process a work item from the plurality of work items 132. The user 101 interacts with the Control Room 130 to schedule automation tasks to be performed on one or more devices 140 as if the user 101 were manually interacting with the necessary application program(s) and the operating system of the devices 140 to perform the tasks directly. The Control Room 130 in conjunction with credential vault 144 holds all software application license and user information. The Control Room 130 also tracks all bots that have been deployed and knows the status of all deployed bots. Each bot 134 in the course of execution may interact with one or more backend business processes 146 which may be implemented in whole or in part by conventional enterprise applications such as Enterprise Resource Planning (ERP) applications or Customer Relationship Management (CRM) applications.
Each client device 140 may take a variety of forms such as a physical machine, for example, a desktop computer, laptop computer or tablet. Each client device 140 may also be a virtual device such as provided by conventional virtualization software which permits multiple operating system sessions to be run on a single computer. Each client device 140 may have applications loaded thereon on local storage. Typically the client device 140 will be separate from the device 102 used by user 101, but in some instances, particularly if the device 102 is virtual or if the device is served by terminal server 142, the device 102 and device 140 that executes a bot 134 may be the same device.
In some cases, a client device 140 may employ applications hosted by a server and accessed by the client device by way of a terminal server 142. A terminal server enables an organization to centrally host applications and resources and publish them to remote client devices, regardless of the location and platform of the end user device. This permits remote access of an application that may not otherwise be designed to be remotely accessed. Moreover, the central deployment of an application simplifies the installation and update of the application. This also provides cost benefits by reducing in some cases licensing costs of the application. An example of terminal server software is the Parallels Remote Application Server available from Parallels Inc. or Remote Desktop Services available from Microsoft Corporation. With such systems, the user interface is displayed from the server onto the client system and input from the client system is transmitted to the server, where execution of the application takes place.
Some or all of the bots 134 may in certain embodiments be located remotely from the Control Room 130. Moreover, any of the devices 140 may also be located remotely from the Control Room 130. The bots 134 and the tasks 132 are shown in separate containers for purposes of illustration but they may be stored in separate or the same device(s), or across multiple devices. The Control Room 130 performs user management functions, source control of the bots 134, along with providing a dashboard that provides analytics and results of the bots 134, performs license management of software required by the bots 134 and manages overall execution and management of scripts, clients, roles, credentials, and security etc. The major functions performed by the Control Room 130 include: (i) a dashboard that provides a summary of registered/active users, tasks status, repository details, number of clients connected, number of scripts passed or failed recently, tasks that are scheduled to be executed and those that are in progress; (ii) user/role management—permits creation of different roles, such as bot creator, bot runner, admin, and custom roles, and activation, deactivation and modification of roles; (iii) repository management—to manage all scripts, tasks, workflows and reports etc.; (iv) operations management—permits checking status of tasks in progress and history of all tasks, and permits the administrator to stop/start execution of bots currently executing; (v) audit trail—logs creation of all actions performed in the Control Room; (vi) task scheduler—permits scheduling tasks which need to be executed on different clients at any particular time; (vii) credential management—permits password management; and (viii) security: management—permits rights management for all user roles. The Control Room 130 is shown generally for simplicity of explanation. Multiple instances of the Control Room 130 may be employed where large numbers of bots are deployed to provide for scalability of the RPA system 10.
The centralized credential vault (CV) 144 operates to securely store all credentials and provision them to bots on an on-demand basis. The CV 144 preferably implements NIST controls IA-2 to uniquely identify and authenticate organizational users (or processes acting on behalf of organizational users). Since sensitive credentials need not be stored in bots or on bot runner systems, the CV 144 facilitates a logical separation of credentials from the bots. CV 144 variables are created from Control Room 130 and are instantly available to all the bot creators and bot runners registered with the respective Control Room 130. The CV 144 adds flexibility and dynamic character to bots since only the credential references are present in the bots and not the credentials. When bots are moved from one environment to another environment, absolutely no change is needed in bots. Bots can seamlessly pick up the credential values applicable for the new environment from the Control Room 130 of that environment. Additionally, the Control Room 130 automatically stores configuration related sensitive data into the CV 144 by default. Additional details of the credential vault 144 are described in pending U.S. patent application Ser. No. 15/957,917 entitled “ROBOTIC PROCESS AUTOMATION SYSTEM AND METHOD WITH SECURE CREDENTIAL VAULT” which is assigned to the assignee of the present application and which is hereby incorporated by reference in its entirety.
The disclosed embodiments operate to extend the functionality of the Control Room 130 as it operates with spreadsheet programs, such as Microsoft Excel®, and specifically to be able to call a bot 134 from a spreadsheet program and pass data into the spreadsheet program, under the attended running environment. In one embodiment the foregoing functionality is embodied in plugin 108. An embodiment of the user interface is seen in
The Settings section 206 has three buttons: (i) Control Room: this is a web link to the user's 101 Control Room 130, (ii) Config: the user 101 can set the Control Room 130 URL here, (iii) Login: the user 101 can enter their username and password here. In
The Bots section 208 has three buttons: (i) Run: run a bot 134 immediately, (ii) Results: view bot 134 run status and also see the returned data, also run handler for each result, (iii) Handlers: define the result handler and test it. In
The Results panel seen in
A Handler, seen selected in
In one embodiment, a default handler is selected by default to save results to a new spreadsheet. This handler will dump the returned data from the user's bot into a new spreadsheet. In such an embodiment, there are two parameters that the user can customize: spreadsheet name and starting cell location (A1 by default), with each variable occupying its own row. Alternatively, as seen at 220 in
The Favorites section 210, seen selected in
Further details of the plugin 108 are shown in
As noted above, the plugin 108 operates programmatically with the system 10 to invoke functions supported by the Control Room 130 to automate tasks performed on data in a spreadsheet designated by user 101. In one embodiment, the plugin 108 by way of Relay Server 120 operates in conjunction with a Control Room 130 such as provided by Automation Anywhere, Inc. (automationanywhere.com). In such an embodiment, the Relay Server 120 may employ the following API's provided by Control Room 130:
The relay server 120 interacts with the Control Room 130 in a manner as described above in connection with
Results produced by the hots 134 are provided to the cache 308 via the Control Room 130 and are then provided to the plugin 108. Data returned by the relay server 120 in one embodiment is passed to the plugin in JavaScript Object Notation (JSON) format, which is an open-standard file format that uses human-readable text to transmit data objects consisting of attribute-value pairs and array data types (or any other serializable value). Upon receipt of the results the plugin 108 may automatically invoke an assigned handler at 304 to process the received results. Alternatively, the user 101 may be notified of the results and may then select a particular handler to process the received results. The received data by default is received into the same spreadsheet (in the same tab or different tab) from which the data for the processing request was passed. This is for security reasons to simplify security management which requires the spreadsheet to pass a key to the plugin 108, which is verified by the plugin 108 before providing results to the spreadsheet. This prevents the results from being received by another unauthorized file, which may be accessible by an unauthorized user. In another embodiment, the data may be passed to a different file, where the different file is identified by the plugin as an authorized file to receive the results.
Computing system 500 may have additional features such as for example, storage 510, one or more input devices 514, one or more output devices 512, and one or more communication connections 516. An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computing system 500. Typically, operating system software (not shown) provides an operating system for other software executing in the computing system 500, and coordinates activities of the components of the computing system 500.
The tangible storage 510 may be removable or non-removable, and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, DVDs, or any other medium which can be used to store information in a non-transitory way, and which can be accessed within the computing system 500. The storage 510 stores instructions for the software implementing one or more innovations described herein.
The input device(s) 514 may be a touch input device such as a keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or another device that provides input to the computing system 500. For video encoding, the input device(s) 514 may be a camera, video card, TV tuner card, or similar device that accepts video input in analog or digital form, or a CD-ROM or CD-RW that reads video samples into the computing system 500. The output device(s) 512 may be a display, printer, speaker, CD-writer, or another device that provides output from the computing system 500.
The communication connection(s) 516 enable communication over a communication medium to another computing entity. The communication medium conveys information such as computer-executable instructions, audio or video input or output, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can use an electrical, optical, RF, or other carrier.
The terms “system” and “computing device” are used interchangeably herein. Unless the context clearly indicates otherwise, neither term implies any limitation on a type of computing system or computing device. In general, a computing system or computing device can be local or distributed and can include any combination of special-purpose hardware and/or general-purpose hardware with software implementing the functionality described herein.
While the invention has been described in connection with a preferred embodiment, it is not intended to limit the scope of the invention to the particular form set forth, but on the contrary, it is intended to cover such alternatives, modifications, and equivalents as may be within the spirit and scope of the invention as defined by the appended claims.
Number | Name | Date | Kind |
---|---|---|---|
5949999 | Song et al. | Sep 1999 | A |
5983001 | Boughner et al. | Nov 1999 | A |
6133917 | Feigner et al. | Oct 2000 | A |
6389592 | Ayres et al. | May 2002 | B1 |
6427234 | Chambers et al. | Jul 2002 | B1 |
6473794 | Guheen et al. | Oct 2002 | B1 |
6496979 | Chen et al. | Dec 2002 | B1 |
6640244 | Bowman-Amuah | Oct 2003 | B1 |
6704873 | Underwood | Mar 2004 | B1 |
6898764 | Kemp | May 2005 | B2 |
6954747 | Wang et al. | Oct 2005 | B1 |
7091898 | Arling et al. | Aug 2006 | B2 |
7246128 | Jordahl | Jul 2007 | B2 |
7398469 | Kisamore et al. | Jul 2008 | B2 |
7441007 | Kirkpatrick et al. | Oct 2008 | B1 |
7533096 | Rice et al. | May 2009 | B2 |
7568109 | Powell et al. | Jul 2009 | B2 |
7571427 | Wang et al. | Aug 2009 | B2 |
7765525 | Davidson et al. | Jul 2010 | B1 |
7805317 | Khan et al. | Sep 2010 | B2 |
7805710 | North | Sep 2010 | B2 |
7810070 | Nasuti et al. | Oct 2010 | B2 |
7846023 | Evans et al. | Dec 2010 | B2 |
8028269 | Bhatia et al. | Sep 2011 | B2 |
8056092 | Allen et al. | Nov 2011 | B2 |
8095910 | Nathan et al. | Jan 2012 | B2 |
8132156 | Malcolm | Mar 2012 | B2 |
8209738 | Nicol et al. | Jun 2012 | B2 |
8234622 | Meijer et al. | Jul 2012 | B2 |
8245215 | Extra | Aug 2012 | B2 |
8352464 | Fotev | Jan 2013 | B2 |
8396890 | Lim | Mar 2013 | B2 |
8438558 | Adams | May 2013 | B1 |
8443291 | Ku et al. | May 2013 | B2 |
8464240 | Fritsch et al. | Jun 2013 | B2 |
8498473 | Chong et al. | Jul 2013 | B2 |
8504803 | Shukla | Aug 2013 | B2 |
8631458 | Banerjee | Jan 2014 | B1 |
8682083 | Kumar et al. | Mar 2014 | B2 |
8713003 | Fotev | Apr 2014 | B2 |
8769482 | Batey et al. | Jul 2014 | B2 |
8819241 | Washburn | Aug 2014 | B1 |
8832048 | Lim | Sep 2014 | B2 |
8874685 | Hollis et al. | Oct 2014 | B1 |
8943493 | Schneider | Jan 2015 | B2 |
8965905 | Ashmore et al. | Feb 2015 | B2 |
9104294 | Forstall et al. | Aug 2015 | B2 |
9213625 | Schrage | Dec 2015 | B1 |
9278284 | Ruppert et al. | Mar 2016 | B2 |
9444844 | Edery et al. | Sep 2016 | B2 |
9462042 | Shukla et al. | Oct 2016 | B2 |
9571332 | Subramaniam et al. | Feb 2017 | B2 |
9621584 | Schmidt et al. | Apr 2017 | B1 |
9946233 | Brun et al. | Apr 2018 | B2 |
20030033590 | Leherbauer | Feb 2003 | A1 |
20030101245 | Srinivasan et al. | May 2003 | A1 |
20030159089 | DiJoseph | Aug 2003 | A1 |
20040083472 | Rao et al. | Apr 2004 | A1 |
20040172526 | Tann et al. | Sep 2004 | A1 |
20040210885 | Wang et al. | Oct 2004 | A1 |
20040243994 | Nasu | Dec 2004 | A1 |
20050188357 | Derks et al. | Aug 2005 | A1 |
20050204343 | Kisamore et al. | Sep 2005 | A1 |
20050257214 | Moshir et al. | Nov 2005 | A1 |
20060095276 | Axelrod et al. | May 2006 | A1 |
20060150188 | Roman et al. | Jul 2006 | A1 |
20070101291 | Forstall et al. | May 2007 | A1 |
20070112574 | Greene | May 2007 | A1 |
20080005086 | Moore | Jan 2008 | A1 |
20080028392 | Chen et al. | Jan 2008 | A1 |
20080209392 | Able et al. | Aug 2008 | A1 |
20080222454 | Kelso | Sep 2008 | A1 |
20080263024 | Landschaft et al. | Oct 2008 | A1 |
20090037509 | Parekh et al. | Feb 2009 | A1 |
20090103769 | Milov et al. | Apr 2009 | A1 |
20090172814 | Khosravi et al. | Jul 2009 | A1 |
20090199160 | Vaitheeswaran et al. | Aug 2009 | A1 |
20090217309 | Grechanik et al. | Aug 2009 | A1 |
20090249297 | Doshi et al. | Oct 2009 | A1 |
20090313229 | Fellenstein et al. | Dec 2009 | A1 |
20090320002 | Peri-Glass et al. | Dec 2009 | A1 |
20100023602 | Martone | Jan 2010 | A1 |
20100023933 | Bryant et al. | Jan 2010 | A1 |
20100100605 | Allen et al. | Apr 2010 | A1 |
20100138015 | Colombo et al. | Jun 2010 | A1 |
20100235433 | Ansari et al. | Sep 2010 | A1 |
20110022578 | Fotev | Jan 2011 | A1 |
20110145807 | Molinie et al. | Jun 2011 | A1 |
20110197121 | Kletter | Aug 2011 | A1 |
20110276568 | Fotev | Nov 2011 | A1 |
20110276946 | Pletter | Nov 2011 | A1 |
20110302570 | Kurimilla et al. | Dec 2011 | A1 |
20120042281 | Green | Feb 2012 | A1 |
20120124062 | Macbeth et al. | May 2012 | A1 |
20120330940 | Caire et al. | Dec 2012 | A1 |
20130173648 | Tan et al. | Jul 2013 | A1 |
20130290318 | Shapira et al. | Oct 2013 | A1 |
20140181705 | Hey et al. | Jun 2014 | A1 |
20150082280 | Betak et al. | Mar 2015 | A1 |
20150347284 | Hey et al. | Dec 2015 | A1 |
20160019049 | Kakhandiki et al. | Jan 2016 | A1 |
20160078368 | Kakhandiki et al. | Mar 2016 | A1 |
20180329399 | Neelakandan | Nov 2018 | A1 |
20200004798 | Weinert, Jr. | Jan 2020 | A1 |
20200327116 | Perlick | Oct 2020 | A1 |
20210097274 | Gligan | Apr 2021 | A1 |
Entry |
---|
Al Sallami, Load Balancing in Green Cloud Computation, Proceedings of the World Congress on Engineering 2013 vol. II, WCE 2013, 2013, pp. 1-5 (Year: 2013). |
B. P. Kasper “Remote: A Means of Remotely Controlling and Storing Data from a HAL Quadrupole Gas Analyzer Using an IBM-PC Compatible Computer”, Nov. 15, 1995, Space and Environment Technology Center. |
Bergen et al., RPC automation: making legacy code relevant, May 2013, 6 pages. |
Business Process Modeling Notation (BPMN), BPMI.org, v. 1.0, May 3, 2004. |
Hu et al., Automating GUI testing for Android applications, May 2011, 7 pages. |
Konstantinou et al., An architecture for virtual solution composition and deployment in infrastructure clouds, 9 pages (Year: 2009). |
Nyulas et al., An Ontology-Driven Framework for Deploying JADE Agent Systems, 5 pages (Year: 2008). |
Process Modeling Notations and Workflow Patterns, S. White, IBM Corp. |
Tom Yeh, Tsung-Hsiang Chang, and Robert C. Miller, Sikuli: Using GUI Screenshots for Search and Automation, Oct. 4-7, 2009, 10 pages. |
Yu et al., Deploying and managing Web services: issues, solutions, and directions, 36 pages (Year: 2008). |
Zhifang et al., Test automation on mobile device, May 2010, 7 pages. |
Number | Date | Country | |
---|---|---|---|
20210129325 A1 | May 2021 | US |