Patent Application
20200301804
The inventions described herein are in the field of system performance monitors with graphical user interfaces.
There is a long felt need for a system performance monitor that graphically illustrates how quickly a given system, or subsystems within said given system, can perform certain tasks. The system can be a fully automated system, such as a computer-based system for routing messages on the Internet. The system may also comprise persons performing tasks, such as employees of a given company. Subsystems may comprise groups of automata, such as servers in a server farm. Subsystems may also comprise groups of persons, such as departments in an organization. Subsystems may include both automata and persons, such as groups of persons each working at a networked work station.
The disclosure of invention is provided as a guide to understanding the invention. It does not necessarily describe the most generic embodiment of the invention or the broadest range of alternative embodiments.
The system 150 that carries out said one or more tasks is organized into one or more of said subsystem schemas 151 (e.g. A, B, and C). Each of the subsystem schemas comprises one or more members with a member ID (e.g. A1, A2, A3, B1, B2, B3, C1, C2, C3). Each of the members carries out at least a portion of one or more of said tasks (e.g. R1, Rz, R3).
Each of the tasks has an associated task record 131 stored in a task database 130. Each task record comprises:
Each of the subsystem schema databases (e.g. 141) associated with a subsystem schema comprises:
Each of the member records comprises:
The front end comprises computer readable instructions stored on said permanent memory of said front end, said computer readable instructions of said front end being operable to cause said microprocessor of said front end to physically carry out the steps:
The detailed description describes non-limiting exemplary embodiments. Any individual features may be combined with other features as required by different applications for at least the benefits described herein. As used herein, the term “about” means plus or minus 10% of a given value unless specifically indicated otherwise.
A portion of the disclosure of this patent document contains material to which a claim for copyright is made. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but reserves all other copyright rights whatsoever.
As used herein, a “computer-based system” comprises an input device (e.g. keyboard, touch screen, or electronic digital input from another device) for receiving data, an output device (e.g. printer, computer screen or digital connection to another device) for outputting data, a permanent digital memory for storing data, computer code or other digital instructions, and a digital processor for executing digital instructions wherein said digital instructions resident in said permanent memory will physically cause said digital processor to read-in data via said input device, process said data within said microprocessor and output said processed data via said output device. The digital processor may be a microprocessor.
As used herein, the term “shaped” means that an item has the overall appearance of a given shape even if there are minor variations from the pure form of said given shape.
As used herein, the term “generally” when referring to a shape means that an ordinary observer will perceive that an object has said shape even if there are minor variations from said shape.
As used herein, relative orientation terms, such as “up”, “down”, “top”, “bottom”, “left”, “right”, “vertical”, “horizontal”, “distal” and “proximal” are defined with respect to an initial presentation of an object and will continue to refer to the same portion of an object even if the object is subsequently presented with an alternative orientation, unless otherwise noted.
As used herein, disclosure of a singular implies disclosure of a plural and vice versa unless specifically indicated otherwise.
Referring again to
The selected task records in said formatted member record may each comprise a start event with a time stamp and an end event with a time stamp. Each of the displayed data point indicia has an X-value based on said time stamp of said end event. Each of said displayed data point indicia has a Y-value based on said time stamp of said start event.
The X-axis may be horizontal. The Y-axis may be vertical. A surprising advantage of having the X-axis be horizontal is that a user can read the time-cloud graphic from left to right to see how the performance of the selected member has changed over time. Alternatively, the Y-axis may be horizontal and the X-axis may be vertical. An advantage of this orientation is that the horizontal axis now corresponds to when events started.
The displayed data point indicia may be semitransparent. Thus, when two datapoint indicia overlap (e.g. 166), the extent of the overlap will be visible.
The time-cloud scatter plot may have a first indicium line 165 that is located where Y equals X. The line represents the present. Any data point indicium on said line corresponds to a task that was completed immediately after it was initiated.
The time-cloud scatter plot may have a second indicium line 171 located where Y equals X plus an expected amount of time 173 between a start event and an end event for a given task. Thus, a user can view a time-cloud graphic and immediately perceive if tasks are being completed in the expected amount of time. If a data point indicium (e.g. 172) falls below the second indicium line, a user may want to investigate further to find out what the cause of the delay was.
The data point indicia may be interactive with a user. For example, each of said data point indicia may display metadata 167 associated with said associated task of a data point indicium upon activation of said associated data point indicium by said user.
The time-cloud scatter plot may further comprise a statistic 169 based on said selected task records of said selected member and said non-selected task records of said selected member. The statistic will help give context for the displayed data point indicia. For example, the displayed statistic may be the ratio of selected task records to total task records (i.e. selected plus non-selected). Thus, the user will be able to perceive if the number of displayed data point indicia is a lot or a little.
An indication 168 of the selected member may be presented in the time-cloud scatter plot so that the user will be able to show the time-cloud scatter plot to another viewer and the other viewer will know what member has been selected.
The member indication and statistics may also or alternatively be displayed outside of the time-cloud scatter plot, such as in a margin.
The system performance monitor may further comprise a computer implemented data munger 124 comprising:
The back end may be further configured to authenticate and route 121 the call 114 from the front end. The back end may be further configured to call 122 the appropriate subsystem schema database based on the routing request according to the selected subsystem schema.
The front end may be further configured to determine which subsystem schema (e.g. 115, 116, 117).
The United States Patent and Trademark Office (USPTO) is a system that carries out one or more tasks. One of said tasks is the examination of a patent application. The examination is carried out at least in part by patent examiners. One of the subsystem schemas of the patent office is the technology class that is assigned to a patent application. The members of the patent class schema include the different classes (e.g. class 706—Artificial Intelligence). Another subsystem schema is the art unit that a patent application is assigned to. The members of the art unit schema include the individual art units (e.g. AU 2121). Another subsystem schema is the patent examiners. The members of the patent examiner schema include the individual examiners. Other schemas may be contemplated such as a law firm schema, applicant schema or inventor schema.
Each patent application is assigned a serial number (e.g. Ser. No. 12/345,678). The USPTO keeps track of all events (e.g. the transaction history or file wrapper) that occur in the examination of a patent application. These events have an event type (e.g. “Non-Final Rejection”) and a time stamp (e.g. 05-05-2011 or mo-da-year).
The USPTO stores records for metadata and events associated with a patent application serial number in a publicly available database called the “Patent Examination Data System” (PEDS). The records in PEDS are indexed according to the serial number of the application. The metadata for each record includes subsystem labels such as a three-digit art class number for the class schema, a four-digit art unit number for the art unit schema and an alphabetic name for the examiner schema. The USPTO provides a search engine for selecting applications assigned to a particular class, art unit or examiner, but the elapsed time to retrieve a set of records for a given search might be quite long (e.g. 10 minutes or more for a large class). Furthermore, the time required to process said records after they are returned by the search engine to produce a time-cloud graphic can be very long. The data file for a single class might be 1 GB. The time required to process the data of a single class might be several hours on a single work station. There are about 700 members of the class schema. Processing the data for all 700 members has taken a matter of days on a large cloud-based data processing server.
To overcome this problem, a system performance monitor was developed according to the above description. The system performance monitor was generally cloud-based apart from a user's client device. The front end was hosted on a web server, Netlify.com. The user's client device connected to Netlify was considered part of the front end. The back end was hosted on an application server, amazon web services (aws.amazon.com). The subsystem schema databases were hosted on a cloud-based service, MongoDB Atlas (www.mongodb.com/cloud/atlas). A person of ordinary skill will recognize that alternative cloud-based services or dedicated resources such as mainframes or server farms could be used to host the front end, back end and subsystem schema databases. Thus, the inventions described herein are not limited to any particular computing platform.
A single input field was provided in the front end to receive the selection of a member of a subsystem schema from a user. A drop-down menu would display available members from all three schemas depending upon what the user typed in. The user could then select the desired member from the drop-down menu. Since the member names were unique to each subsystem schema, the selection of a member was enough for the back end to determine the selected subsystem schema and hence which subsystem schema database should be queried. In this example, if a selected member ID had an alphabetic character, then the selected subsystem schema was examiner. If the selected member ID was numeric with three digits, then the selected subsystem schema was class. If the selected member ID as numeric with four digits, then the selected subsystem schema was art unit. In this example, the front end merely forwarded the selected member ID to the back end and the back end determined the selected subsystem schema.
A data munger was written for the back end using Python programming language. The data munger read in the entire PEDS database (more than 1 TB). It then selected the metadata and events of interest for each application record, reformatted the data and stored the data in the subsystem schema databases. Three subsystem schema databases were set up. One for class, one for art unit, and one for examiner. A user, such as a registered US patent agent or attorney, would have interest in the performance of members of any of these schemas. Thus, a user could determine, for example, if a particular examiner's performance was in conformance to all of the examiners in a particular art unit. Similarly, the performance of different art units examining patent applications in the same class could be compared. A manager at the USPTO might be similarly interested, particularly if there were significant discrepancies between examiners, art units or classes that should otherwise have similar performance.
After filtering the PEDS data to include only the metadata (e.g. application serial number, title, class, art unit, examiner) and event data (e.g. non final rejection, final rejection, notice of allowance) of interest, the amount of data was reduced by 35×. However, since there were three subsystem schema databases, and each database had replicated data indexed according to the members of each subsystem schema, the net reduction of data was about 12×. Nonetheless, the delay between selecting a subsystem schema and selecting a member of said subsystem schema to when the time-cloud graphic was presented on the output device was only about 10 seconds or less, even for time cloud graphics that presented 100,000 data point indicia.
The Y-axis 231 was the filing date of a patent application. The X-axis 232 was the date of notice of allowance. Both axes start at Jan. 1, 2000 and end at the date the PEDS data was downloaded (i.e. Mar. 15, 2019). It is contemplated herein that after a first download of the entire PEDS database, subsequent downloads may only include records for applications that had new events since the prior download. The data from the subsequent downloads can be used to update the subsystem schema databases.
A first indicium line 223 was located at Y equals X. A second indicium line 224 was located at Y equals X plus three years. Three years is the maximum amount of time that it should take to get a notice of allowance according to US patent law assuming that the applicant has not otherwise delayed prosecution of the patent application.
A data point indicium (e.g. 211) is displayed on the time-cloud scatter plot for each of the selected records (i.e. all applications that had a notice of allowance). A total of about 11,000 data point indicia are presented. The data point indicia have a size of 3 points with a solid black fill and no border line. The transparency of the data points is 80%. Thus, single data point indicium (e.g. 212) are clearly visible while at the same time, high density areas (e.g. 213) can be discerned.
Upon selection of any data point indicium (e.g. 212) by the user, a popup 214 is displayed showing relevant metadata (e.g. serial number and title) of the application associated with the data point indicium.
The selected class (item 204) is displayed on the time-cloud scatter plot. In an alternative embodiment is was alternatively displayed in a margin of the time-cloud scatter plot.
The value (item 206) of a calculated statistic, APOA12, was also presented on the time-cloud scatter plot. APOA stands for “allowances per office action”. The “12” subscript indicates that the APOA was for office actions in the 12 months prior to the date the data was downloaded or last updated. The office actions included nonfinal rejections and final rejections from non-selected records, such as applications that did not have a notice of allowance. Thus APOA12 gives a user an indication of how many office actions overall a user would have to respond to for all applications in a portfolio in order to get a single notice of allowance. An APOA12 indicates that for the past 12 months, applicants had to reply to about 4 office actions in order to get a single notice of allowance.
Vertical grid lines 221 and horizontal grid lines 222 are presented on the time-cloud scatter plot so that a user can readily perceive when particular applicants where allowed and filed respectively.
A user viewed this time-cloud and noticed that the datapoint indicia thinned out 242 starting in the beginning of 2017. The user then created an alternative time-cloud graphic based on where applications with nonfinal or final office actions were represented by data point indicia. The user used the pop ups of several applications receiving rejections after 2017 to determine that many applications were receiving rejections based on 35 U.S.C. 101 when said applications had not received said rejections in the past. The user, therefore was discouraged from filing applications with an anticipated class 706 assigned to said applications. Upon more careful observation, however, the user observed that there was a sudden thickening 243 in the data point indicia of allowances in the first few months of 2019. In January 2019, the USPTO issued new guidance on the proper examination of patent applications under 35 U.S.C. 101. When the user examined the file wrappers of applications allowed in early 2019, said user observed that examiners in this class where allowing cases under the new guidance that they had previously rejected prior to January of 2019. Thus, user was encouraged to continue to file applications with this class. Users can observe both subtle and sudden changes on performance of different members of different subsystem schemas to more effectively draft and prosecute patent applications.
While the disclosure has been described with reference to one or more different exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the disclosure. In addition, many modifications may be made to adapt to a particular situation without departing from the essential scope or teachings thereof. Therefore, it is intended that the disclosure not be limited to the particular embodiment disclosed as the best mode contemplated for carrying out this invention.
| Number | Date | Country | |
|---|---|---|---|
| 62661117 | Apr 2018 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16495851 | Sep 2019 | US |
| Child | 16946226 | US | |
| Parent | 16495851 | Sep 2019 | US |
| Child | 16495851 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 29661850 | Aug 2018 | US |
| Child | 16495851 | US |
