Patent Application
20200159821
This disclosure generally relates to systems and methods for automating completion of computer based forms.
People engage in countless activities online many of which require users to enter and submit information electronically, for example, through an electronic form on a website or using an application on their mobile device. A person applying for a job, for instance, may be required to fill out a job application or questionnaire provided on a company's website or via a job portal. But people seeking jobs often apply to multiple positions at one or more companies, requiring the repeated entry of the same or similar information for each job application. The tedious, time-consuming nature of the application process operates as a barrier to those seeking jobs.
To aid in this process, tools have been developed to automatically populate (or auto-fill) electronic forms based on inputs provided by users when completing other forms. Existing tools, however, have rather limited capabilities and are prone to inaccurate results. Existing tools, for instance, are static in nature and are only capable of handling a limited number of pre-defined fields, like, ‘name’ and ‘address’, and even then, they yield less than satisfactory results because of the manner in which they are implemented. These tools, for example, typically rely on the label associated with a particular field to intuit the information being requested (e.g., a determination may be made that a field is requesting the user's name based on a label of “Please enter your name”). But the association between a label and a field may not be readily apparent (e.g., due to the often time complex structure of the electronic form in underlying code) and there may be countless ways in which certain information may be requested (e.g., there are over 25 different ways in which a user can be prompted to enter their e-mail address), making it difficult to include logic that accounts for all possible situations. As a consequence, existing auto-fill tools usually are only able to partially auto-fill an electronic form and even then are prone to inaccurate results, and thus are of limited utility to users.
Traditional auto-fill solutions are also limited in the input types that they are able to handle, typically only auto-filling text fields within a form. But non-text fields (e.g., radio buttons, dropdown lists, etc.) constitute roughly half of the input fields encountered in electronic forms. The inability to handle non-text field types further limits the effectiveness of such tools. Another significant limitation of traditional auto-fill tools stems from the fact that they operate in a localized manner on each device of a user. It is not uncommon for users to make use of several different devices (e.g., a computer, mobile device), but given the localized nature of the auto-fill processing, users are unable to leverage the data entered on one device to facilitate auto-fill on another.
The presently disclosed FillBot system seeks to address the above-mentioned limitations of existing auto-fill techniques by employing a data-centric approach, which leverages user entered data to determine what information is being requested in an electronic form. More particularly, the FillBot system looks at the actual data entered by a user along with information about the input field (e.g., its associated label, expected data type, etc.) to identify a correspondence, across different electronic forms, between input fields requesting the same information. By moving away from a solely field-label approach and focusing on the data actually entered by a user, the FillBot system is able to more completely and accurately determine what information is being requested in a particular input field. Furthermore, in contrast to traditional auto-fill systems, the FillBot system is implemented in a distributed manner where a backend server processes and stores user data and form information, making a user's data portable (i.e., across any and all user devices) and providing improved accuracy and efficiency by allowing users to take advantage of relationships identified by the FillBot system based on the activity of other users.
In a first embodiment, FillBot is enabled by the capabilities of, and executed in the context of, a browser. The FillBot browser extension makes use of the browser's document object model (DOM) when parsing electronic forms, as well as a backend server, which stores user data and implements the relationship mapping algorithms, through an application programming interface (API) service running on the backend server.
In a second embodiment, FillBot is enabled by the capabilities of, and executed in the context of, a native mobile application. The FillBot application makes use of the operating system DOM when parsing electronic forms, as well as a backend server, which stores user data and implements the relationship mapping algorithms, through an application programming interface (API) service running on the backend server.
The accompanying drawings, which are incorporated herein and form part of the specification, illustrate one or more aspects of the present invention and, together with the description, further serve to explain the principles of the invention and to enable a person skilled in the relevant art(s) to make and use the invention.
The features and advantages of the disclosed embodiments will become more apparent from the detailed description set forth below when taken in conjunction with the drawings, in which like reference characters identify corresponding elements throughout. In the drawings, like reference numbers generally indicate identical, functionally similar, and/or structurally similar elements. Unless otherwise indicated, the drawings provided throughout the disclosure should not necessarily be interpreted as to-scale drawings.
This specification discloses one or more embodiments that incorporate the features of this invention. The disclosed embodiment(s) merely exemplify the invention. The scope of the invention is not limited to the disclosed embodiment(s). The invention is defined by the claims appended hereto.
The embodiment(s) described, and references in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” “some embodiments,” etc., indicate that the embodiment(s) described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is understood that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
As mentioned above, the FillBot application 130 may be provided as a browser extension (e.g., on the personal computer 120 of a user 101) or as a native mobile application (e.g., on the mobile phone 110 of a user 101). The FillBot application 130 may operate similarly in both contexts, and may include a form parsing engine 131, a backend communication component 132, a populate field component 133, and a data capture component 134, as well as a blacklist tool.
The form parsing engine 131 may be responsible for scanning an electronic form 150 and generating a document mapping 160 containing a listing of all the information being requested in the electronic form 150, and may utilize the DOM generated by the layout engine 137 of the mobile operating system 135 or web browser 136 in order to do so. As noted above, the electronic form 150 may be presented to users 101 as an online form (e.g., a dynamic HTML web page or the like). In other instances, the electronic form 150 may be a discrete document (e.g., a fillable Adobe PDF document, Microsoft Word document, or the like). In some embodiments, the FillBot application 130 may be able to ingest existing electronic forms 150 that contain user entered data (e.g., a structured document containing user entered data corresponding to particular fields), which may be used to initialize the user data database 177 of the backend server 105 for use in auto-filling subsequent electronic forms 150, as described in further detail below.
The document mapping 160 that is generated may include a form path identifier 161, which identifies the website and form to which the document mapping 160 corresponds. The form path identifier 161 may be generated by the form parsing engine 131 by parsing the website URL on which the electronic form 150 was detected, and may be represented using an X-path like addressing scheme (e.g., ‘jobs.jobvite.com’). The document mapping 160 may also include one or more data pairs that describe the different input fields 151 in the electronic form 150. The document mapping 160, for example, may include a unique field identifier, or SlugID, and a corresponding label, which may be generated by the form parsing engine 131.
The form processing engine 131 may generate the SlugID for each input field 151 based on different attributes of the input field 151, for example, its input ID, label, and input type. The input ID and label, for example, may be the value of the id and label attributes for an HTML input element. And in instances where the id attribute is undefined (i.e., having a null value, or comprising an empty string), the input ID may be treated as an aggregation of (e.g., the sum of the values of) some or all of the HTML attributes of the input element. Input type, likewise, may reflect the field type of the HTML input element (e.g., radio, checkbox, etc.). More generally, the SlugID may be generated such that it identifies a particular input field 151 on a specific electronic form 150 as well as preserves necessary information regarding the particular input field 151. In this way, the SlugID can be used to associate a specific input value provided by a user to the input field 151 in which the value was entered, and the FillBot application 130 may be able to accurately identify and fill each input field 151.
In cases where the electronic form 150 includes multiple input fields having the same label and type, the form processing engine 131 may also consider the sequential number of the input field 151, allowing the FillBot application 130 to accurately fill seemingly duplicate input fields 151. An electronic form 150, for example, may include multiple “Street Address” input fields (e.g., referring to a work and home address), and the form processing engine 131 may generate unique SlugIDs for each of them (e.g., “street-address-1” and “street-address-2”). The FillBot application 130, thus, may be able to disambiguate each of the input fields 151 as well as the values entered by the user into each.
The backend communication component 132 allows the FillBot application 130 to interface with the backend server 105 through one or more endpoints provided by the FillBot API service 170. The backend communication component 132, for example, may transmit the document mapping 160 generated by the form parsing engine 131 to the backend server 105 for processing (e.g., through an HTTP PUT or POST method call). The FillBot application 130 may also include a data capture component 134 that may capture data entered by the user into the electronic form 150, which the backend communication component 132 may transmit to the backend server 105 for processing (e.g., through an HTTP PUT or POST method call). The backend communication component 132 may also receive user data from the backend server 105 in response. The populate field component 133 of the FillBot application 130 may use the provided user data to automatically populate the input fields 151 of the electronic form 150, and may utilize the DOM generated by the layout engine 137 in order to do so (i.e., to actually manipulate the value of particular input fields 151).
The blacklist tool may selectively enable or disable processing of certain electronic forms or certain websites. Users may be able to add or remove forms and websites from the blacklist. In some embodiments, the user's blacklist may be synchronized with the backend server 105, and if enough users blacklist a particular form or website, the backend server 105 may automatically push these sites to other users' blacklists.
The Backend Server
As mentioned above, the backend server 105 may assist the FillBot application 130 in providing auto-fill capabilities to users 101. The backend server 105 and its components may be implemented as a dedicated server or using a cloud computing platform (e.g., Amazon Web Services (AWS), Microsoft Azure, or the like). The backend server 105 may also make use of one or more databases, for example, a document mapping database 175, a synonym mapping database 176, and a user data database 177. These databases may be implemented using any number of commercially available or open-source database technologies. In some embodiments, the databases may be implemented as document-oriented databases (e.g., using MongoDB, or the like) while in other embodiments they may be implemented as relational databases (e.g., using an Azure table, or the like). The particular manner of implementation may be selected based on desired performance characteristics of the FillBot system.
As illustrated in
As described above, the functionality provided by the endpoints of the FillBot API service component 170 may be enabled by the document mapping component 171 and user data management component 172. More particularly, the document mapping component 171 may store received document mappings 160 in document mapping database 175. The document mapping component 171 may also compare received document mappings 160 with those already stored in document mapping database 175 (e.g., based on a common form path 163) in order to determine whether changes have been made to a particular form. The document mapping component 171 may calculate a difference between the received and stored document mapping, and may update the stored mapping in the document mapping database 175 with that difference. The document mapping component 171, for example, may be able to detect when a company has added a new question (i.e., a new input field 151) to a job application hosted through its ATS, or changed the layout of a particular electronic form 150, and may update the stored document mapping to reflect these changes.
The user data management component 172, likewise, may store and process user data entered into electronic forms 150 that is received from user devices via the FillBot API service component 170. The user data management component 172, for instance, may store received user data in user data database 177 as SlugID and value pairs in association with a unique user ID that identifies the user 101 from whom the data was received. The user data management component 172 may implement logic to determine when and how data values are added or updated in the user data database 177. For example, in some embodiments, the user data management component 172 may store all received user data, which may allow for system fallback through data redundancy. Other embodiments may eliminate such redundancy when storing user data, but with the benefit of improved system responsiveness. In some embodiments, the user data management component 172 may also be able to detect when a user 101 has changed, or ‘corrected,’ the value of an input field 151 that was automatically populated. Adaptive techniques may be used to modify the relationship that led to the ‘incorrect’ auto-fill. The adaptive techniques may result in a global modification of the relationship or modifications on a per user basis. In the latter case, a separate cache of mappings specific to the user may be used to capture this.
The user data management component 172 may also process the user data to identify relationships between distinct input fields (i.e., between distinct SlugIDs). The resulting relationships may then be used by subsequent users in automatically populating an electronic form 150. For example, input fields 151 on different electronic forms 150 labeled “Mobile,” “Cell,” or “Personal Number” may all be requesting the same information (i.e., the user's telephone number). By examining the data entered by a user 101 into these input fields 151, the user data management component 172 may be able to identify a relationship (or create a mapping) there between. A user, for example, may enter “415-555-1212” in the “Mobile” input field 151 on a first electronic form 150 as well as in the “Cell” input field 151 on a second electronic form 150. Upon entry of the information in the second electronic form 150 (and transmission of the user data to backend server 105 for processing), the user data management component 172 may establish a relationship, or equivalence, between the “Mobile” and “Cell” input fields 151. As a result, subsequent users 101 who encounter either of these fields will have the same phone number auto-filled in each.
The user data management component 172 may also be responsible for returning stored user data to the FillBot application 130, for example, in a response provided by the FillBot API service 170 (e.g., in response to the invocation of an HTTP POST method providing a document mapping 160). In some embodiments, the user data management component 172 may return all user data stored in user data database 177. However, in other embodiments, it may be desirable to return only a portion of the stored user data (e.g., where the user's data set is quite large and/or the available communication bandwidth is limited). In such cases, the user data management component 172 may only return the relevant user information needed to auto-fill a particular electronic form 150, which may be selected based on the document mapping 160 first provided by the user 101 (i.e., based on the SlugIDs in the document mapping 160).
In some embodiments, the user data management component 172 may be capable of interfacing with external sources to help initialize the user data database 177, potentially eliminating the need for users 101 to manually enter any information at all. The user data management component 172, for example, may be able to process a resume uploaded by the user 101, interface with a human resource information system, or access a credit report database to obtain information to be entered in the user data database 177. In some embodiments, the user data database 177 may be initialized by ingesting and processing existing electronic forms 150. For example, corporate web servers 103 may provide a resume builder service, which may generate a structured document containing user entered data for particular fields. As discussed above, the FillBot application 130 may permit a user to ingest this existing electronic form 150, and may generate a document mapping 160 and capture user entered data, which may then be transmitted to the backend server 105 via the backend communication component 132. In other embodiments, the backend server 105 may include a form parsing engine similar to that of the FillBot application described above, which likewise may be capable of scanning an electronic form 150 and generating the document mapping 160. In such cases, the electronic form 150 may be ingested from an existing storage location or database of the backend server 105, or otherwise accessible by the backend server 105 (e.g., through a shared database). It may also be the case that the backend communication component 132 of the FillBot application 130 may transmit the existing electronic form 150 to the backend server 105, which may process the electronic form 150 and generate a document mapping 160 and capture corresponding user entered data.
Generating the Document Mapping
As discussed above, the form parsing engine 131 may be responsible for generating a document mapping 160 containing a listing of all the information being requested in the electronic form 150. In order to accurately perform auto-fill it is important to correctly identify what information is called for in each field 151 of an electronic form 150 and how that information is to be entered. But there are countless numbers of electronic form structures (e.g., a date can be entered into a text field or selected from a series of drop-down lists) and equally varied manners of implementing a particular structure (i.e., in HTML code or the like), making it difficult to automate this identification process. The electronic forms 150 illustrated in
In contrast to existing auto-fill solutions which can only handle a limited number of simplistic scenarios (e.g., where a label and form control are explicitly linked in the underlying HTML), the FillBot application 130 utilizes a flexible code design to accommodate the various form structures that can be expected in the underlying code and\or DOM. For example, in some instances, an input element may be placed within a label element, but not explicitly linked to the label element. In such cases, the form parsing engine 131 may look at the parent element of the input element in the DOM to see if it is a label type element. In other instances, the input element may be a select element (i.e., a dropdown list). In such cases, the form parsing engine 131 may treat the default option as the label since this is what will be visible to the user when the form is rendered.
In yet other instances, the label element, due to the layout structure of the form, may be several levels away from the input element with which it is associated (i.e., not logically ‘near’ the input element in the DOM) but under the same ‘root’ node. In such cases, the form parsing engine 131 may traverse up the DOM level by level and check at each level whether a particular branch contains any text that looks like valid input label text. However, some electronic forms may employ a hierarchical structure in which branches containing label elements and input elements share the same ‘root’ node, and consequently are all siblings of one another. Traversing up the DOM, as previously suggested, would not be effective in such cases. To account for this scenario, the form parsing engine 131 may first make a determination of whether the node is a ‘shared’ node. If so, it would move from the branch containing the input element in question, to the prior (or successive) sibling branch and check to see whether the sibling branch contains any text that looks like valid input label text. The process would continue by moving to another sibling branch until a valid input label is found.
The form processing engine 131 can also deal with electronic forms that are dynamic in nature, for example where input elements are dynamically added or removed (e.g., using JavaScript) depending on certain user actions. In such cases, the form processing engine 131 examines all possible DOM mutations and processes them accordingly.
Managing User Data & Identifying Input Relationships
As discussed above, the user data management component 172 may store user data received from user devices via the FillBot API service component 170, and may process the user data to identify relationships between different input fields 151.
The identification of matching SlugIDs in the synonym mapping database 176 may also be aided through the use of an in memory cache, which may map all possible slugs for a given SlugID based on the identified relationships stored in the synonym mapping database 176. The user data management component 172 may be configured to update the slug cache at set intervals (e.g., every ‘n’ minutes), and by utilizing the slug cache, the number of database calls that is required to provide responsive user data is reduced (i.e., the only database call that is required is to the user data database 177), improving system responsiveness and reducing computational demand.
Overall System Operation
An Illustrative Example
The same user (i.e., User X) may then access an electronic form on a second website (e.g., a job application hosted at careers.successfactor.com). The FillBot application 130 may again be initiated, and the form parsing engine 131 may generate a document mapping (careers.successfactor.com′, ‘first-name, First Name’, ‘street-address, Street Address’) based on the electronic form. The backend communication component 132 may then transmit the document mapping to backend server 105 for storage and processing. The user data management component 172 may then retrieve user data for the user (i.e., ‘first-name, Joe’ and ‘address, 1 Blow St.’) from the user data database to return in response. Since no relationships are established in the synonym mapping database, no additional SlugIDWalue pairs are generated, and the retrieved user data is returned to the FillBot application 130. The populate field component 133 uses the returned user data to populate the input fields of the electronic form (i.e., entering ‘Joe’ into the field labeled ‘First-Name’). Because the populate field component 133 is unable to populate all fields of the electronic form, the user may proceed to complete the electronic form, for example, entering values (i.e., ‘1 Blow St.’) into the unfilled input fields (i.e., the field labeled ‘Street Address’). The form data is then captured using data capture component 134 and sent to backend server 105, which adds an entry to the synonym mapping database (i.e., adding ‘address, street-address, 1’).
Another user (i.e., User Y) may then access the electronic form on the first website and the the FillBot application 130 may be initiated. The form parsing engine 131 may generate a document mapping (′jobs.jobvite.com′, ‘first-name, First Name’, ‘address, Address’) based on the electronic form, which may be sent to backend server 105 for storage and processing. The user data management component 172 may then attempt to retrieve user data from the user data database to return in response, but since the user data database does not yet contain any entries for the user (i.e., for User Y), a null set will be returned. Since no data was returned, none of the fields of the electronic form will be populated, and the user may proceed to complete the electronic form, for example, entering values (i.e., ‘Jane’ and ‘13 Elm St.’) into the electronic form's input fields (i.e., the fields labeled ‘First Name’ and ‘Address’). This form data is then captured using data capture component 134 and sent to backend server 105, which adds corresponding entries to the user data database (i.e., adding ‘Y, first-name, Jane’ and ‘Y, address, 13 Elm St.’).
The same user (i.e., User Y) may then access the electronic form on the second website, and the FillBot application 130 may again be initiated. The form parsing engine 131 may generate a document mapping (‘careers.successfactor.com’, ‘first-name, First Name’, ‘street-address, Street Address’) based on the electronic form, which may be transmitted to backend server 105 for storage and processing. The user data management component 172 may then retrieve user data for the user (i.e., ‘first-name, Jane’ and ‘address, 13 Elm St.’) from the user data database to return in response. Because there is a match between the SlugID in the retrieved user data and the synonym mapping database, user data management component 172 may generate an additional SlugID and value pair (i.e., ‘street-address, 13 Elm St.’), and the retrieved user data and generated user data (i.e., ‘first-name, Jane’, ‘address, 13 Elm St.’, and ‘street-address, 13 Elm St.’) may be returned to the FillBot application 130. The populate field component 133 uses the returned user data to populate all of the input fields of the electronic form (i.e., entering ‘Jane’ and ‘13 Elm St.’ into the fields labeled ‘First-Name’ and ‘Street Address’ respectively). The user may validate that the information is correct and then submit the electronic form. The form data is then captured using data capture component 134 and sent to backend server 105, which increments the count of the relied upon relationship in the synonym mapping database 176 (i.e., changing the entry to ‘address, street-address, 2’).
The processor 706 may take the form of one or more CPUs, controllers, digital signal processors (DSP), processing circuitry with or without an accompanying DSP, or various other processing devices including integrated circuits such as, for example, an ASIC (application specific integrated circuit), an FPGA (field programmable gate array), a microcontroller unit (MCU), a hardware accelerator, a special-purpose computer chip, or the like. The processor 706 may be configured to execute instructions stored on memory 707 or otherwise available to the processor 706. Meanwhile, the memory 707 may take the form of an electronic storage device (e.g., a computer readable storage medium) comprising gates configured to store data, which may be retrievable by a processor 706. The memory device 707 may comprise a non-transitory tangible memory device, and may be configured to store information, data, files, applications, instructions or the like for enabling the apparatus to carry out various functions in accordance with an example embodiment of the invention. For example, the memory device 707 could be configured to buffer input data for processing by the processor 706. Additionally or alternatively, the memory device 707 could be configured to store instructions for execution by the processor 706. For example, the memory 707 may comprise web server logic, FillBot API service logic 708, document mapping logic 709, and database management logic 710 which may be used or executed by the processor 706 in providing the functionality provided in this disclosure. Additionally, or in the alternative, the memory device 706 may be one of a plurality of databases that store information. For example, the memory 707 may comprise document mapping database 175, synonym mapping database 176, and/or user data database 177, which may be accessed by the processor 706 using database management logic 710.
While specific embodiments of the invention have been described above, it will be appreciated that the invention may be practiced other than as described. The description provided above is intended to be illustrative and non-limiting. Thus, it will be apparent to one skilled in the art that modifications may be made to the present invention as described without departing from the scope of the claims set out below. It is also worth noting that the Summary and Abstract sections may set forth one or more but not all exemplary embodiments of the present invention as contemplated by the inventor(s), and thus, are not intended to limit the present invention and the appended claims in any way.
The foregoing description of the specific embodiments will so fully reveal the general nature of the invention that others can, by applying knowledge within the skill of the art, readily modify and/or adapt for various applications such specific embodiments, without undue experimentation, without departing from the general concept of the present invention. Therefore, such adaptations and modifications are intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein. It is to be understood that the phraseology or terminology herein is for the purpose of description and not of limitation, such that the terminology or phraseology of the present specification is to be interpreted by the skilled artisan in light of the teachings and guidance.
The breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.
| Number | Date | Country | |
|---|---|---|---|
| 62768360 | Nov 2018 | US |
