Patent Application
20250110810
Robotic Process Automation (RPA) systems enable automation of repetitive and manually intensive computer-based tasks. In an RPA system, computer software, namely a software robot (often referred to as a “bot”), may mimic the actions of a human to perform various computer-based tasks. For instance, an RPA system can be used to interact with one or more software applications through user interfaces, as a human would do. Therefore, RPA systems typically do not need to be integrated with existing software applications at a programming level, thereby eliminating the difficulties inherent to integration. Advantageously, RPA systems permit the automation of application level repetitive tasks via software robots that are coded to repeatedly and accurately perform the repetitive task.
RPA systems have generally assisted users in creating software robots that mimic user interactions with software applications to perform various tasks. However, the creation of software robots is not straight forward because sometimes the user interactions for a given task may cause use of an Application Programming Interface (API) to access remote resources (e.g., server-based resources). In doing so, a given API has certain parameters and/or authentication data that is needed in order to successfully interact with a particular server-based resource. While RPA systems may seek to automate such interactions by mimicking user interactions, conventionally when the user interactions cause interactions with APIs, RPA systems conventionally are unable to provide automation. The API has its own requirements, including for example authentications, that if not satisfied, can cause the API request to fail. As such, RPA systems have been unable to or hindered in creation of software robots that include interactions with APIs for access to server-based resources.
Therefore, there is a need for improved approaches to create software robots for RPA systems when the software robots interact with APIs.
Improved techniques and systems for creating software automation process (e.g., software robots, bots) from recordings of interactions with application programs and network-accessible resources are disclosed. The recordings can be captured using a recorder, and the recorder can record not only user interactions with application programs but also API interactions with network-accessible resources. From one or more of the recordings, a software automation process (e.g., software robot, bot) can be created, and the created software automation process can, when carried out, programmatically initiate (i) user interactions that mimic the user interactions that were recorded, and (ii) API interactions that mimic the API interactions that were recorded.
The invention can be implemented in numerous ways, including as a method, system, device, or apparatus (including computer readable medium and graphical user interface). Several embodiments of the invention are discussed below.
As a method for creating a bot, one embodiment can, for example, include at least: initiating a recorder configured to capture a sequence of interactions with one or more application programs operating on at least one computing device and/or with one or more network-accessible resources operating on at least one remote computing device; recording, via the recorder, (i) user interactions with the one or more application programs operating on the at least one user computing device, and (ii) Application Programming Interface (API) interactions with the one or more network-accessible resources operating on at least one remote computing device; and creating the bot in accordance with the recording, the created bot including at least programmatic initiation of the user interactions and programmatic initiation of the API interactions.
As a computer-implemented method for creating a bot, one embodiment can, for example, include at least: initiating capture of user actions with one or more application programs operating on at least one user computing device; initiating capture of Application Programming Interface (API) actions with one or more network-accessible resources operating on at least one remote computing device; determining whether a user action with the one or more application programs has been detected; recording user information descriptive of the detected user action when the determining determines that the user action with the one or more application programs has been detected; determining whether an API call action with the one or more network-accessible resources has been detected; recording API call information descriptive of the detected API call action when the determining determines that the API call action with the one or more application programs has been detected; determining whether to stop the capture of the user actions and the API actions; ending the capture of user actions with the one or more application programs operating on the at least one user computing device after the determining determines that the capture of the user actions should stop; ending the capture of API actions with the one or more network-accessible resources operating on the at least one remote computing device after the determining determines that the capture of the API actions should stop; and creating the bot in accordance with the user information that has been recorded and the API call information that has been recorded, the created bot being configured to provide programmatic initiation of user actions to mimic the user actions that have been captured and to provide programmatic initiation of API actions to mimic the API call actions that have been captured.
As a non-transitory computer readable medium including at least computer program code tangible stored therein for creating a bot, one embodiment can, for example, include at least: computer program code for initiating a recorder configured to capture a sequence of interactions with one or more application programs operating on at least one computing device and/or with one or more network-accessible resources operating on at least one remote computing device; computer program code for recording, via the recorder, (i) user interactions with the one or more application programs operating on the at least one user computing device, and (ii) Application Programming Interface (API) interactions with the one or more network-accessible resources operating on at least one remote computing device; and computer program code for creating the bot in accordance with the recording, the created bot including at least programmatic initiation of the user interactions and programmatic initiation of the API interactions.
As a non-transitory computer readable medium including at least computer program code tangible stored therein for facilitating creation of a bot, one embodiment can, for example, include at least: computer program code for initiating capture of user actions with one or more application programs operating on at least one user computing device; computer program code for initiating capture of Application Programming Interface (API) actions with one or more network-accessible resources operating on at least one remote computing device; computer program code for determining whether a user action with the one or more application programs has been detected; computer program code for recording user information descriptive of the detected user action when the determining determines that the user action with the one or more application programs has been detected; computer program code for determining whether an API call action with the one or more network-accessible resources has been detected; computer program code for recording API call information descriptive of the detected API call action when the determining determines that the API call action with the one or more application programs has been detected; computer program code for determining whether to stop the capture of the user actions and the API actions; computer program code for ending the capture of user actions with the one or more application programs operating on the at least one user computing device after the determining determines that the capture of the user actions should stop; and computer program code for ending the capture of API actions with the one or more network-accessible resources operating on the at least one remote computing device after the determining determines that the capture of the API actions should stop.
Other aspects and advantages of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings which illustrate, by way of example, the principles of the invention.
The invention will be readily understood by the following detailed description in conjunction with the accompanying drawings, wherein like reference numerals designate like elements, and in which:
Improved techniques and systems for creating software automation processes (e.g., software robots, bots) from recordings of interactions with application programs and network-accessible resources are disclosed. The recordings can be captured using a recorder, and the recorder can record not only user interactions with application programs but also API interactions with network-accessible resources. From one or more of the recordings, a software automation process (e.g., software robot, bot) can be created, and the created software robot can, when carried out, programmatically initiate (i) user interactions that mimic the user interactions that were recorded, and (ii) API interactions that mimic the API interactions that were recorded.
Generally speaking, RPA systems use computer software to emulate and integrate the actions of a human interacting within digital systems. In an enterprise environment, the RPA systems are often designed to execute a business process. In some cases, the RPA systems use artificial intelligence (AI) and/or other machine learning capabilities to handle high-volume, repeatable tasks that previously required humans to perform. The RPA systems also provide for creation, configuration, management, execution, and/or monitoring of software automation processes.
A software automation process can also be referred to as a software robot, software agent, automation program or bot. A software automation process can interpret and execute tasks on one's behalf. In doing so, sometimes the software automation process operates to access one or more remote resources via a network. In one embodiment, the remote resource are network accessible resources. The network accessible resources are, for example, an authentication server, a web server, a remote database, or a cloud-based storage.
Software automation processes are particularly well suited for handling a lot of the repetitive tasks that humans perform every day. Software automation processes can accurately perform a task or workflow they are tasked with over and over. As one example, a software automation process can locate and read data in a document, email, file, or window. As another example, a software automation process can connect with one or more Enterprise Resource Planning (ERP), Customer Relations Management (CRM), core banking, and other business systems to distribute data where it needs to be in whatever format is necessary. As another example, a software automation process can perform data tasks, such as reformatting, extracting, balancing, error checking, moving, copying, or any other desired tasks. As another example, a software automation process can grab data desired from a webpage, application, screen, file, or other data source. As still another example, a software automation process can be triggered based on time or an event, and can serve to take files or data sets and move them to another location, whether it is to a customer, vendor, application, department or storage. These various capabilities can also be used in any combination. As an example of an integrated software automation process making use of various capabilities, the software automation process could start a task or workflow based on a trigger, such as a file being uploaded to an FTP system. The integrated software automation process could then download that file, scrape relevant data from it, upload the relevant data to a database, and then send an email to a recipient to inform the recipient that the data has been successfully processed.
Embodiments of various aspects of the invention are discussed below with reference to
The bot creation system 100 interacts with a browser 102 that couples to a network 104 to gain access to one or more network-accessible resources. The bot creation system 100 can include a browser extension 106. The browser extension 106 is provided within or in communication with the browser 102. The browser extension 106 monitors the browser 102 and captures user actions and/or API actions that occurred during operation of the browser 102. For example, the browser extension 106 can be registered with the browser 102 to intercept (i) user interactions with the browser 102 and/or (ii) web requests (e.g., API requests).
The bot creation system 100 can also include browser agent 108. The browser agent 108 receives the user actions and/or API actions from the browser extension 106. For example, the user extensions and/or the web requests that have been intercepted by the browser extension 106 can be provided to the browser agent 108.
The bot creation system 100 can also include a recorder 110. The recorder 110 interacts with the browser agent 108 so as to receive the various user actions and/or API actions that were captured by the browser extension 106. In other words, the browser agent 108 communicates between the browser extension 106 and the recorder 110, such as forwarding the user actions (e.g., user inputs) and/or the web requests (e.g., API actions) that have intercepted by the browser extension 106 to the recorder 110. In doing so, the browser agent provides descriptive information of the user actions and/or the web requests to the recorder 110. Besides forwarding the user actions and/or the web requests, in another embodiment, the browser agent 108 can also provide processing of the user actions and/or the web requests for operations such as formatting, filtering, storing, etc.
The recorder 110 can record the series of incoming user actions and/or web requests as recorded data. The recorder 110 can also include a command builder 112 that can convert the recorded data to a particular RPA system format. Different RPA systems use different command formats, such as when creating a bot to carry out the recording. Hence, the command builder 112 can convert the recorded data to a command format associated with the particular RPA system format. In one embodiment, the converted recorded data has a JSON format. The converted recorded data can then be provided to the associated RPA system via a connection 114. The connection 114 can be a wired or wireless connection, as examples.
In one embodiment, the command builder 112 can also modify a recording gathered by the recorder 110 such that the recording includes variables, which can be identified and accessed by a bot that is created to carry out the interactions associated with the recording. The command builder 112 can produce variablized, recorded data, such as by replacing certain fixed data values in the converted recorded data with variables. The variablized, recorded data can then be provided to the associated RPA system via the connection 114. Here, the variablization operates to make a fixed value into a variable. This can allow a bot to be created that can dynamically assign values to the variables at run time, which can make the bot more sophisticated in its operation and better able to initiate API requests dynamically.
The bot creation system 100 provides the converted recording data (which may also be variablized) that is suitably formatted for usage by the RPA system to an RPA system for creation of a bot. The creation of the bot is typically performed by the RPA system.
In general, the computing environment 200 can support various different types of computing devices that can interact with the RPA system 202. These computing devices can serve as a recording computing device, a playback computing device, or both.
As shown in
The recording computing device 208 can include at least a portion of a bot creation sub-system, such as the bot creation system 100 illustrated in
The recording of API actions can operate to record interactions with various web-based resources that are accessible via the network 206, such as web-resources 214 and 216 illustrated in
The computing environment 200 shown in
The different playback computing devices 218, 224 and 234 can be configured to execute software automation processes that were previously created. When one of the playback computing devices 218, 224 and 234 executes a software automation process (e.g., bot) that has been created from one or more recordings, the playback computing device may not only mimic user interactions with application programs on or accessible to the playback computing device, but may also send one or more API requests to the web-based resources 214, 216 and receive one or more responses to the API requests.
The bot creation process 300 can initiate 302 a recorder to capture a sequence of interactions. After the recorder has been initiated 302, the recorder can record 304 user interactions with one or more application programs. For example, if a user interacts with a graphical user interface of a particular one of the one or more application programs, such as by a control section (e.g., mouse clicks) and/or by entering data with respect to a data entry object (e.g., text entry box), the user interactions can be recorded 304. In addition, after the recorder has been initiated 302, the recorder can also record 306 API interactions with one or more network-accessible resources. One example of a network-accessible resource is a remote server, such as a web server. Other examples of network-accessible resources can include databases, storage device, or web services, which can be referred to as remote or web-based resources that are accessible via one or more networks.
After the user interactions have been recorded 304 and the API interactions have been recorded 306, a bot can be created 308. The created bot, when executed (e.g., run), operates to programmatically perform the recorded user interactions and the recorded API interactions. That is, the created bot can provide automated processing for any interactions that would have otherwise needed to be provided by a user in order to carry out the task(s), including interaction with network-accessible resources. After the bot has been created 308, the bot creation process 300 can end.
The enhanced recording process 400 can initiate 402 capture of user actions. In addition, the enhanced recording process 400 can also initiate 404 capture of API actions. After the capture of user actions and/or API actions have been initiated 402, 404, a decision 406 can determine whether a user action has been detected. When the decision 406 determines that a user action has been detected, user action information pertaining to the detected user action can be sent 408 to a recorder. The recorder is typically utilized by an RPA system to record the actions and events previously performed or initiated by a user so that a bot can be created for future automation of the actions previously performed or initiated by the user. As an example, the recorder can be the recorder 110 illustrated in
Following block 408, or following the decision 406 when the user action has not been detected, a decision 410 can determine whether an API action (e.g., API call) has been detected. When the decision 410 determines that an API action has been detected, API action information (e.g., API call information) can be sent 412 to the recorder.
Following block 412, or following the decision 410 when an API action has not been detected, a decision 414 can determine whether the recording of user actions and/or API actions should stop. When the decision 414 determines that the recording should not stop, then the processing performed by the enhanced recording process 400 can return to repeat the decision 406 and subsequent blocks so that other user actions and/or API actions can be detected and recorded. Alternatively, when the decision 414 determines that the recording of the user actions and/or API actions should stop, then capture of user actions can end 416 and capture of API actions can end 418. Following the block 418, the enhanced recording process 400 can end.
The recording formation process 500 can begin with a decision 502 that determines whether a recording should be started. When the decision 502 determines that a recording has not yet been started, the recording formation process 500 awaits an instruction to begin recording. On the other hand, when the decision 502 determines that a recording should be started, the recording formation process 500 can start a recording and continue with the recording formation process 500.
Once a recording has been started, a decision 504 can determine whether user action information has been received. For example, the recorder can receive user action information from a browser agent, such as the browser agent 108 illustrated in
Following the storage 506 of the user action entry, or following the decision 504 when user action information has not been received, a decision 508 can determine whether API action information has been received. For example, the recorder can receive API action information from a browser agent, such as the browser agent 108 illustrated in
Following the storage 510 of the API action entry, or following the decision 508 when API action information has not been received, a decision 512 can determine whether the recording should end. When the decision 512 determines that the recording should not end, then the recording formation process 500 can return to repeat the decision 504 and subsequent blocks so that subsequently received user action information and/or API action information can be similarly processed and appropriate entries can be added to the recording. Alternatively, when the decision 512 determines that the recording should end, then the recording formation process 500 can end. When the recording formation process 500 ends, the recording that has been started, and for which various user action entries and/or API action entries have been entered, can be closed, thereby ending the recording.
As shown in
The data flow diagram 700 illustrated in
After receiving the Step Response 710 at the browser agent 704, the browser agent 704 forwards a forwarded Step Response 712 to the recorder 706. The forwarded step response 712 may be the same as the Step Response 710 or it may be a modified version of the Step Response 710. In other words, optionally, the browser agent 704 can modify the Step Response 710, such as its format, arrangement or its content, prior to forwarding the forwarded Step Response 712 to the recorder 706.
After the forwarded Step Response 712, whether modified or not, has been received at the recorder 706, the recorder 706 can operate to convert the forwarded Step Response 712 into a converted Step Response 714 and then stored. The recorder 706 can also provide the converted Step Response 714 to a RPS system. In one implementation, the recorder 706 can convert the forwarded Step Response 712 to a format and/or arrangement that is understood by the RPA system that utilizes the converted Step Response 714 when creating a bot that is configured to carry out the user action in an automated fashion by “running” the created bot. Although the user action data flow 708 can pertain to automation of a single user action, the same or similar data flow can be provided for one or more other user actions, and thus can be automated in a similar manner, often in a bot that automates numerous user actions.
An exemplary Step Response (e.g., Step Response 710 or forwarded Step Response 712) following a user action involving a user selecting a button labeled “New” on a user interface (e.g., page) rendered by a browser, is a data file as follows.
An exemplary converted Step Response (e.g., converted Step Response 714) following conversion of the exemplary Step Response, such as for use by a particular RPA system, is a data file as follows.
The data flow diagram 750 illustrated in
After receiving the API Response 760 at the browser agent 754, the browser agent 754 forwards a forwarded API Response 762 to the recorder 756. The forwarded API response 762 may be the same as the API Response 760 or it may be a modified version of the API Response 760. In other words, optionally, the browser agent 754 can modify the API Response 760, such as its format, arrangement or its content, prior to forwarding the forwarded API Response 762 to the recorder 756.
After the forwarded API Response 762, whether modified or not, has been received at the recorder 756, the recorder 756 can operate to convert the forwarded API Response 762 into a converted API Response 764 and then stored. The recorder 756 can also provide the converted API Response 764 to a RPS system. In one implementation, the recorder 756 can convert the forwarded API Response 762 to a format and/or arrangement that is understood by the RPA system that utilizes the converted API Response 764 when creating a bot that is configured to initiate the API action in an automated fashion by “running” the created bot. Although the API action data flow 758 can pertain to automation of a single API action, the same or similar data flow can be provided for one or more other API actions, and thus can be automated in a similar manner, often in a bot that automates numerous API actions.
Additionally, the converted API Response 764 can be further modified by the recorder 756. The modification being perform can be to variablize the converted API Response 764 to provide a variablized API response 766. For example, the recorder 756 can include a command builder to perform processing to variablize the converted API Response 764. Here, fixed input values for input parameters of an API can be replaced with variables. By doing so, a resulting bot can be created such that needed parameters for an associated API action can be provided dynamically via the variables in the variablized API response 766.
An exemplary API Response (e.g., API Response 760 or forwarded API Response 762) that is to be initiated following a user action, such as a user selecting a button labeled “New” on a user interface (e.g., page) rendered by a browser, is a data file as follows.
An exemplary converted API Response (e.g., converted API Response 764) following conversion of the exemplary API Response, such as for use by a particular RPA system, is a data file as follows.
An exemplary variablized API Response (e.g., variablized API Response 766) following variablization of the exemplary converted API Response, such as for use by a particular RPA system, is a data file as follows.
As an example, in the exemplary variablized API response, variables were substituted for various fixed values (e.g., API parameters) in the exemplary converted API response. As examples, various string values for key-value pairs in the exemplary converted API response were replaced by the following variables: $Referer$, $User-Agent$, $User-Agent$, $FDC-Page-Cache$, $FDC-Request-Id$, $sec-ch-ua$, $sec-ch-ua-mobile$, and $sec-ch-ua-platform$. In one implementation, in creating one or more corresponding variables, naming of the corresponding variable can be based on a predetermined naming convention, such as the naming being programmatically determined at least in part by a portion of the data associated to the corresponding variable. Additionally or alternatively, the initial values assigned to the various variables can be the corresponding captured value or can be default values that can be based on the corresponding captured value, which can be done, for example, by inserting appropriate assign actions for each of such variable.
In one embodiment, the various API responses can be provided in a JSON format. For example, the exemplary Step Response, exemplary converted Step Response, exemplary API Response, exemplary converted API response, and exemplary variablized API response noted have a JSON format. Those skilled in the art will recognize that these various files need not have a JSON format, and need not all have the same format.
The various aspects disclosed herein can be utilized with or by robotic process automation systems. Exemplary robotic process automation systems and operations thereof are detailed below.
The RPA system 800 can also include a control room 808. The control room 808 is operatively coupled to the data storage 802 and is configured to execute instructions that, when executed, cause the RPA system 800 to respond to a request from a client device 810 that is issued by a user 812.1. The control room 808 can act as a server to provide to the client device 810 the capability to perform an automation task to process a work item from the plurality of work items 806. The RPA system 800 is able to support multiple client devices 810 concurrently, each of which will have one or more corresponding user session(s) 818, which provides a context. The context can, for example, include security, permissions, audit trails, etc. to define the permissions and roles for bots operating under the user session 818. For example, a bot executing under a user session, cannot access any files or use any applications that the user, under whose credentials the bot is operating, does not have permission to do so. This prevents any inadvertent or malicious acts from a bot under which bot 804 executes.
The control room 808 can provide, to the client device 810, software code to implement a node manager 814. The node manager 814 executes on the client device 810 and provides a user 812 a visual interface via browser 813 to view progress of and to control execution of automation tasks. It should be noted that the node manager 814 can be provided to the client device 810 on demand, when required by the client device 810, to execute a desired automation task. In one embodiment, the node manager 814 may remain on the client device 810 after completion of the requested automation task to avoid the need to download it again. In another embodiment, the node manager 814 may be deleted from the client device 810 after completion of the requested automation task. The node manager 814 can also maintain a connection to the control room 808 to inform the control room 808 that device 810 is available for service by the control room 808, irrespective of whether a live user session 818 exists. When executing a bot 804, the node manager 814 can impersonate the user 812 by employing credentials associated with the user 812.
The control room 808 initiates, on the client device 810, a user session 818 (seen as a specific instantiation 818.1) to perform the automation task. The control room 808 retrieves the set of task processing instructions 804 that correspond to the work item 806. The task processing instructions 804 that correspond to the work item 806 can execute under control of the user session 818.1, on the client device 810. The node manager 814 can provide update data indicative of status of processing of the work item to the control room 808. The control room 808 can terminate the user session 818.1 upon completion of processing of the work item 806. The user session 818.1 is shown in further detail at 819, where an instance 824.1 of user session manager 824 is seen along with a bot player 826, proxy service 828, and one or more virtual machine(s) 830, such as a virtual machine that runs Java® or Python®. The user session manager 824 provides a generic user session context within which a bot 804 executes.
The bots 804 execute on a player, via a computing device, to perform the functions encoded by the bot. Some or all of the bots 804 may in certain embodiments be located remotely from the control room 808. Moreover, the devices 810 and 811, which may be conventional computing devices, such as for example, personal computers, server computers, laptops, tablets and other portable computing devices, may also be located remotely from the control room 808. The devices 810 and 811 may also take the form of virtual computing devices. The bots 804 and the work items 806 are shown in separate containers for purposes of illustration but they may be stored in separate or the same device(s), or across multiple devices. The control room 808 can perform user management functions, source control of the bots 804, along with providing a dashboard that provides analytics and results of the bots 804, performs license management of software required by the bots 804 and manages overall execution and management of scripts, clients, roles, credentials, security, etc. The major functions performed by the control room 808 can include: (i) a dashboard that provides a summary of registered/active users, tasks status, repository details, number of clients connected, number of scripts passed or failed recently, tasks that are scheduled to be executed and those that are in progress; (ii) user/role management-permits creation of different roles, such as bot creator, bot runner, admin, and custom roles, and activation, deactivation and modification of roles; (iii) repository management—to manage all scripts, tasks, workflows and reports etc.; (iv) operations management—permits checking status of tasks in progress and history of all tasks, and permits the administrator to stop/start execution of bots currently executing; (v) audit trail—logs creation of all actions performed in the control room; (vi) task scheduler—permits scheduling tasks which need to be executed on different clients at any particular time; (vii) credential management—permits password management; and (viii) security: management—permits rights management for all user roles. The control room 808 is shown generally for simplicity of explanation. Multiple instances of the control room 808 may be employed where large numbers of bots are deployed to provide for scalability of the RPA system 800.
In the event that a device, such as device 811 (e.g., operated by user 812.2) does not satisfy the minimum processing capability to run a node manager 814, the control room 808 can make use of another device, such as device 815, that has the requisite capability. In such case, a node manager 814 within a Virtual Machine (VM), seen as VM 816, can be resident on the device 815. The node manager 814 operating on the device 815 can communicate with browser 813 on device 811. This approach permits RPA system 800 to operate with devices that may have lower processing capability, such as older laptops, desktops, and portable/mobile devices such as tablets and mobile phones. In certain embodiments the browser 813 may take the form of a mobile application stored on the device 811. The control room 808 can establish a user session 818.2 for the user 812.2 while interacting with the control room 808 and the corresponding user session 818.2 operates as described above for user session 818.1 with user session manager 824 operating on device 810 as discussed above.
In certain embodiments, the user session manager 824 can provide one or more of five functions. First is a health service 838 that maintains and provides a detailed logging of bot execution including monitoring memory and CPU usage by the bot and other parameters such as number of file handles employed. The bots 804 can employ the health service 838 as a resource to pass logging information to the control room 808. Execution of the bot is separately monitored by the user session manager 824 to track memory, CPU, and other system information. The second function provided by the user session manager 824 is a message queue 840 for exchange of data between bots executed within the same user session 818. The third function is a deployment service (also referred to as a deployment module) 842 that connects to the control room 808 to request execution of a requested bot 804. The deployment service 842 can also ensure that the environment is ready for bot execution, such as by making available dependent libraries. The fourth function is a bot launcher 844 which can read metadata associated with a requested bot 804 and launch an appropriate container and begin execution of the requested bot. The fifth function is a debugger service 846 that can be used to debug bot code.
The bot player 826 can execute, or play back, a sequence of instructions encoded in a bot. The sequence of instructions can, for example, be captured by way of a recorder when a human performs those actions, or alternatively the instructions are explicitly coded into the bot. These instructions enable the bot player 826, to perform the same actions as a human would do in their absence. In one implementation, the instructions can compose of a command (action) followed by set of parameters, for example: Open Browser is a command, and a URL would be the parameter for it to launch a web resource. Proxy service 828 can enable integration of external software or applications with the bot to provide specialized services. For example, an externally hosted artificial intelligence system could enable the bot to understand the meaning of a “sentence.”
The user 812.1 can interact with node manager 814 via a conventional browser 813 which employs the node manager 814 to communicate with the control room 808. When the user 812.1 logs in from the client device 810 to the control room 808 for the first time, the user 812.1 can be prompted to download and install the node manager 814 on the device 810, if one is not already present. The node manager 814 can establish a web socket connection to the user session manager 824, deployed by the control room 808 that lets the user 812.1 subsequently create, edit, and deploy the bots 804.
In the embodiment shown in
Turning to the bots Bot 1 and Bot 2, each bot may contain instructions encoded in one or more programming languages. In the example shown in
The control room 808 operates to compile, via compiler 1008, the sets of commands generated by the editor 1002 or the recorder 1004 into platform independent executables, each of which is also referred to herein as a bot JAR (Java ARchive) that perform application-level operations captured by the bot editor 1002 and the bot recorder 1004. In the embodiment illustrated in
As noted in connection with
An entry class generator 1108 can create a Java class with an entry method, to permit bot execution to be started from that point. For example, the entry class generator 1108 takes, as an input, a parent bot name, such “Invoice-processing.bot” and generates a Java class having a contract method with a predefined signature. A bot class generator 1110 can generate a bot class and orders command code in sequence of execution. The bot class generator 1110 can take, as input, an in-memory bot structure and generates, as output, a Java class in a predefined structure. A Command/Iterator/Conditional Code Generator 1112 wires up a command class with singleton object creation, manages nested command linking, iterator (loop) generation, and conditional (If/Else If/Else) construct generation. The Command/Iterator/Conditional Code Generator 1112 can take, as input, an in-memory bot structure in JSON format and generates Java code within the bot class. A variable code generator 1114 generates code for user defined variables in the bot, maps bot level data types to Java language compatible types, and assigns initial values provided by user. The variable code generator 1114 takes, as input, an in-memory bot structure and generates Java code within the bot class. A schema validator 1116 can validate user inputs based on command schema and includes syntax and semantic checks on user provided values. The schema validator 1116 can take, as input, an in-memory bot structure and generates validation errors that it detects. The attribute code generator 1118 can generate attribute code, handles the nested nature of attributes, and transforms bot value types to Java language compatible types. The attribute code generator 1118 takes, as input, an in-memory bot structure and generates Java code within the bot class. A utility classes generator 1120 can generate utility classes which are used by an entry class or bot class methods. The utility classes generator 1120 can generate, as output, Java classes. A data type generator 1122 can generate value types useful at runtime. The data type generator 1122 can generate, as output, Java classes. An expression generator 1124 can evaluate user inputs and generates compatible Java code, identifies complex variable mixed user inputs, inject variable values, and transform mathematical expressions. The expression generator 1124 can take, as input, user defined values and generates, as output, Java compatible expressions.
The JAR generator 1128 can compile Java source files, produces byte code and packs everything in a single JAR, including other child bots and file dependencies. The JAR generator 1128 can take, as input, generated Java files, resource files used during the bot creation, bot compiler dependencies, and command packages, and then can generate a JAR artifact as an output. The JAR cache manager 1130 can put a bot JAR in cache repository so that recompilation can be avoided if the bot has not been modified since the last cache entry. The JAR cache manager 1130 can take, as input, a bot JAR.
In one or more embodiment described herein command action logic can be implemented by commands 1001 available at the control room 808. This permits the execution environment on a device 810 and/or 815, such as exists in a user session 818, to be agnostic to changes in the command action logic implemented by a bot 804. In other words, the manner in which a command implemented by a bot 804 operates need not be visible to the execution environment in which a bot 804 operates. The execution environment is able to be independent of the command action logic of any commands implemented by bots 804. The result is that changes in any commands 1001 supported by the RPA system 800, or addition of new commands 1001 to the RPA system 800, do not require an update of the execution environment on devices 810, 815. This avoids what can be a time and resource intensive process in which addition of a new command 1001 or change to any command 1001 requires an update to the execution environment to each device 810, 815 employed in an RPA system. Take, for example, a bot that employs a command 1001 that logs into an on-online service. The command 1001 upon execution takes a Uniform Resource Locator (URL), opens (or selects) a browser, retrieves credentials corresponding to a user on behalf of whom the bot is logging in as, and enters the user credentials (e.g., username and password) as specified. If the command 1001 is changed, for example, to perform two-factor authentication, then it will require an additional resource (the second factor for authentication) and will perform additional actions beyond those performed by the original command (for example, logging into an email account to retrieve the second factor and entering the second factor). The command action logic will have changed as the bot is required to perform the additional changes. Any bot(s) that employ the changed command will need to be recompiled to generate a new bot JAR for each changed bot and the new bot JAR will need to be provided to a bot runner upon request by the bot runner. The execution environment on the device that is requesting the updated bot will not need to be updated as the command action logic of the changed command is reflected in the new bot JAR containing the byte code to be executed by the execution environment.
The embodiments herein can be implemented in the general context of computer-executable instructions, such as those included in program modules, being executed in a computing system on a target, real or virtual, processor. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The program modules may be obtained from another computer system, such as via the Internet, by downloading the program modules from the other computer system for execution on one or more different computer systems. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Computer-executable instructions for program modules may be executed within a local or distributed computing system. The computer-executable instructions, which may include data, instructions, and configuration parameters, may be provided via an article of manufacture including a computer readable medium, which provides content that represents instructions that can be executed. A computer readable medium may also include a storage or database from which content can be downloaded. A computer readable medium may further include a device or product having content stored thereon at a time of sale or delivery. Thus, delivering a device with stored content, or offering content for download over a communication medium, may be understood as providing an article of manufacture with such content described herein.
The exemplary computing environment 1200 may have additional features such as, for example, tangible storage 1210, one or more input devices 1214, one or more output devices 1212, and one or more communication connections 1216. An interconnection mechanism (not shown) such as a bus, controller, or network can interconnect the various components of the exemplary computing environment 1200. Typically, operating system software (not shown) provides an operating system for other software executing in the exemplary computing environment 1200, and coordinates activities of the various components of the exemplary computing environment 1200.
The tangible storage 1210 may be removable or non-removable, and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, DVDs, or any other medium which can be used to store information in a non-transitory way, and which can be accessed within the computing system 1200. The tangible storage 1210 can store instructions for the software implementing one or more features of a PRA system as described herein.
The input device(s) or image capture device(s) 1214 may include, for example, one or more of a touch input device (such as a keyboard, mouse, pen, or trackball), a voice input device, a scanning device, an imaging sensor, touch surface, or any other device capable of providing input to the exemplary computing environment 1200. For multimedia embodiment, the input device(s) 1214 can, for example, include a camera, a video card, a TV tuner card, or similar device that accepts video input in analog or digital form, a microphone, an audio card, or a CD-ROM or CD-RW that reads audio/video samples into the exemplary computing environment 1200. The output device(s) 1212 can, for example, include a display, a printer, a speaker, a CD-writer, or any another device that provides output from the exemplary computing environment 1200.
The one or more communication connections 1216 can enable communication over a communication medium to another computing entity. The communication medium conveys information such as computer-executable instructions, audio or video input or output, or other data. The communication medium can include a wireless medium, a wired medium, or a combination thereof.
The various aspects, features, embodiments or implementations of the invention described above can be used alone or in various combinations.
Embodiments of the invention can, for example, be implemented by software, hardware, or a combination of hardware and software. Embodiments of the invention can also be embodied as computer readable code on a computer readable medium. In one embodiment, the computer readable medium is non-transitory. The computer readable medium is any data storage device that can store data which can thereafter be read by a computer system. Examples of the computer readable medium generally include read-only memory and random-access memory. More specific examples of computer readable medium are tangible and include Flash memory, EEPROM memory, memory card, CD-ROM, DVD, hard drive, magnetic tape, and optical data storage device. The computer readable medium can also be distributed over network-coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.
Numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will become obvious to those skilled in the art that the invention may be practiced without these specific details. The description and representation herein are the common meanings used by those experienced or skilled in the art to most effectively convey the substance of their work to others skilled in the art. In other instances, well-known methods, procedures, components, and circuitry have not been described in detail to avoid unnecessarily obscuring aspects of the present invention.
In the foregoing description, reference to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Further, the order of blocks in process flowcharts or diagrams representing one or more embodiments of the invention do not inherently indicate any particular order nor imply any limitations in the invention.
The many features and advantages of the present invention are apparent from the written description. Further, since numerous modifications and changes will readily occur to those skilled in the art, the invention should not be limited to the exact construction and operation as illustrated and described. Hence, all suitable modifications and equivalents may be resorted to as falling within the scope of the invention.
This application claims priority to U.S. Provisional Patent Application No. 63/541,870, filed Oct. 1, 2023, and entitled “USER EVENT AND API INTERACTION RECORDING FOR ROBOTIC PROCESS AUTOMATION,” which is hereby incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| 63541870 | Oct 2023 | US |
