The present invention generally relates to creation and optimization of human-centric processes.
The following detailed description of embodiments of the invention will be better understood when read in conjunction with the appended drawings of an exemplary embodiment. It should be understood, however, that the invention is not limited to the precise arrangements and instrumentalities shown.
In the drawings:
Aspects of the present invention are useful in the field of human-centric operations in which new work procedures are authored to assist workers in performing a hands-on job such as equipment service, manufacturing assembly, or machine calibration, by way of example. The work procedure may include any combination of text, pictures, movies, three dimensional computer aided design (3D CAD), remote expert sessions, and mixed reality sessions, by way of example, to aid the worker in completing a task and tracking task completion. Such work procedures may be executed on any combination of computing devices including, smart phones, tablets, smart glasses, wearable computers, and personal computers, by way of example. Such computing devices may be connected to any number of input devices, such as sensors, smart tools, or smart equipment, by way of example.
In accordance with one aspect of an embodiment of the present invention, artificial intelligence (AI) is used to recommend the optimum the work procedure based on data collected from the execution and modification of prior procedures, or even the current procedure in some embodiments, taking into account high resolution data collected from the worker, the input device(s), and the computing platform(s). As used herein, “high resolution data” may refer to data that provides quality, dependable and/or objective information about the tasks being performed by the workers. More particularly, in this aspect of the present invention, an author of the work procedure is provided with information to assist the author to design the optimal work procedure for performing a particular task (e.g., instructions for hanging a painting or replacing a gasket seal on a generator that requires a specific torque reading for each fastener). This is accomplished using an AI engine which relies on historical data collected from other similar procedures that include similar steps or use similar tools and/or platforms, or the same procedure, to recommend an optimum work procedure design. For example, based on data generated through other workers performing the same/a similar task, the AI engine may determine that a series of time-lapse videos with lengths less than 18% of the expected step duration is ideal, instead of using alternative methods of instruction such as standard video, mixed-reality, or pictures. This results in the lowest total procedure execution time while still meeting a 100% quality target.
Existing software used for authoring (augmented) work procedures does not provide recommendations to the author to optimize the work procedures. Also, such software is not configured to accept as an input high resolution data from the worker, tools and sensors, or the computing platforms used by the worker, to allow such software to make optimization recommendations.
The following provides some examples of the recommendations that may be made by the system to an author of a work procedure, in accordance with exemplary embodiments of the invention. In one embodiment, a recommendation is made to reduce the length of a video used to illustrate a step in the process to make it more likely that the worker can complete the step in less than the target time. In another embodiment, a recommendation is made to replace first media with second media to reduce the time variability in the step, resulting in a lower average step completion time. In another embodiment, a recommendation is made to add visual media because 70% of the times a step is executed, a remote expert call was required, indicating that the instruction may not clear. In one embodiment, a recommendation is made to finalize one set of procedure steps over others based on the results returned by a built-in AB testing capability, which can automatically determine which of two or more steps is optimal for a procedure based on time, quality, or other desired metric. In another embodiment, a recommendation is made to group a series of steps together because each step requires the operator to use the same tool. In still a further embodiment, a recommendation is made to require the worker to watch a short video clip until the worker has executed the procedure at least 5 times.
In accordance with another aspect of an embodiment of the present invention, an AI-based recommendation engine is used for optimizing procedures that are carried out by human workers. This embodiment may be used to help the work procedure author optimize the procedures for the desired outcome metric by analyzing the data collected through instrumenting the human-centric procedures. By way of example, outcomes may be optimized for quality, cycle time, cost, and traceability among other metrics. Using the data collected during the execution of all instances of all procedures (including previous procedures), the AI-based models direct the procedure author towards the optimum series of steps and step instructions/content that lead to the most favorable outcome. In addition, other AI and machine learning techniques can be used to identify root causes of problems that may have occurred during the execution of a procedure. For example, in a manufacturing operation, the data collected by the system might identify which particular step in a procedure seemed anomalous in a production run with a high rate of scrapped work.
This embodiment of the present is superior to existing methods. More particularly, existing methods for human-centric work treat the worker essentially as a “black box”. The worker is given a set of tasks to do, often with paper-based instructions, and very little data is collected during the execution of task until the end result is known. For automated manufacturing systems, on the other hand, computers and sensors track events down to the millisecond level to help optimize such automated systems. The present invention provides a way of instrumenting the worker, like existing automated systems, as well as the sensors and systems used by the worker in the execution of each procedure, without creating an undue burden on the work procedure author or the worker executing the procedure.
The following provides some examples illustrating this aspect of the invention. In one embodiment, a short burst of quality or scrap problems can be analyzed to understand what step in the procedure appears anomalous to previous iterations of the procedures that resulted in satisfactory quality outcomes. The quality assurance team and instruction author can be directed precisely to the area that caused the problem with recommendations on how to fix the problem. In another embodiment, the data collected during the execution of a series of problem troubleshooting procedures may be collected and fed into a machine learning system which generates and continuously improves an automated response system (e.g., a bot), freeing up the human experts to answer more difficult or unusual problems. In a further embodiment, a recommendation based on AB testing of different variations of a procedure is made that identifies the procedure variation that provides the best results when considering time, cost, and quality as factors. In still a further embodiment, a recommendation is made to group a series of steps based on a common tool being used by the worker in each step to increase the efficiency of the operation. In another exemplary embodiment, a recommendation is made to group a series of steps based on a common target area being worked on in order to increase the efficiency of the operation. In another exemplary embodiment, a recommendation is made to combine one or more steps. In another exemplary embodiment, a recommendation is made to add one or more steps. In another exemplary embodiment, a recommendation is made to delete one or more steps.
The exemplary methods of embodiments of the present invention may be carried out by a system 100 that employs a client/server architecture such as, for example, the collection of components illustrated and described with reference to
With reference to
Thus, for example, an author using authoring device 110 to create a work procedure for a task may be guided by software that relies on input from AI engine 160 to make recommendations about the particular steps to be taken, the order in which the steps should be performed, the tools to be used in performing the steps, and/or details about the manner in which the instruction is to be presented to the worker (e.g., the media to be used, the length of the instruction). Further, for example, data associated with a worker using worker device 130 (e.g., a computing device such as a smartphone) and tools/input devices 140 to execute a work procedure is collected. AI Engine 160 can assess this data (e.g., using one or more machine learning algorithms) to identify, with precision, any problems associated with the work procedure and execution of the same and suggest alterations to the work procedure accordingly.
While
To the extent data and information is communicated over network 120 (e.g., the Internet or an Intranet), one or more network servers may be employed. The network server may also comprise one or more processors, computer readable storage media that store programs (computer readable instructions) for execution by the processor(s), and an interface between the processor(s) and computer readable storage media. When data is requested through an application, such as an Internet browser, the network server may receive and process the request. The network server may send the data or application requested along with user interface instructions for displaying a user interface on device (e.g., authoring device 110, worker device 130).
The computers/devices referenced herein are specially programmed to perform the functionality described herein.
The non-transitory computer readable storage media (e.g., implemented on any of the computers or devices provided herein) that store the programs (e.g., software modules comprising computer readable instructions) may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer readable storage media may include, but is not limited to, RAM, ROM, Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), flash memory or other solid state memory technology, CD-ROM, digital versatile disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the computer system and processed.
With reference to
In one embodiment, the HCP task characteristics includes at least one of: a task description characteristic that specifies a description of a task, a worker verification characteristic that specifies how a worker will verify that a step has been completed, a demonstration characteristic that specifies a demonstration about how a step should be completed, and a task expected duration characteristic that specifies an amount of time the worker needs to complete a task.
In some embodiments, the task authoring criteria includes a criterion that is met when an estimated amount of time the worker needs to complete a task exceeds the specified amount of time the worker needs to complete the task. In this embodiment, in step 203, a determination is made (e.g., it is determined whether the estimated amount of time the worker needs to complete a task exceeds the specified amount of time the worker needs to complete the task). If so, in step 205, a recommendation is provided to an author (e.g., to replace the task step with a second task step having a shorter expected execution time and including an alternative task characteristic (e.g., if a demonstration video was provided in the original task, the second task may include a sequence of 3 images).
In some embodiments, the demonstration characteristic includes a first media and the task authoring criteria includes a criterion that is met when an estimated duration of the first media and/or an estimated amount of time the worker needs to complete a task exceeds the specified amount of time the worker needs to complete a task in the task expected duration characteristic. In this embodiment, in step 203 a determination is made (e.g., it is determined whether the estimated duration of the first media and/or an estimated amount of time the worker needs to complete a task exceeds the specified amount of time the worker needs to complete a task in the task expected duration characteristic). If so, in step 205, a recommendation provided to an author (e.g., to replace the task step with a second task step having a second media). In some of these embodiments, the first media and the second media may each be at least one of: mixed reality media, three dimensional computer aided design assembly/disassembly video, standard video, static image, and audio.
In some embodiments, the task authoring criteria includes a criterion that is met when historical data regarding a specified task description indicates that that a worker may need additional assistance to complete the task and the HCP task characteristics do not include a demonstration characteristic. Examples of historical data may include prior worker task status data and/or prior author process modification data. The prior worker task status data may indicate an undesirable outcome such as the worker took too long to complete the task or had to call an expert for assistance. The prior author process modification data may indicate that the prior author for a prior process modified the process step as a result of the prior worker task status data indicating an undesirable outcome. In this embodiment, in step 203, a determination is made (e.g., it is determined whether the historical data regarding a specified task description indicates that a worker may need additional assistance to complete a task and the HCP task characteristics do not include a demonstration characteristic). If so, in step 205, a recommendation is made to an author (e.g., to add a new demonstration).
In some embodiments, the task authoring criteria includes a criterion that is met when the task cards include a first task card and a second task card that are not sequentially ordered, and when the first task card and the second task card both include an indication that a common tool (e.g., a Bluetooth torque wrench) is needed to complete the tasks specified in the first task card and the second task card. In this embodiment, in step 203, a determination is made (e.g., it is determined whether the first task card and the second task card both include an indication that a common tool is needed to complete the tasks specified in the first task card and the second task card). If so, in step 205, a recommendation is provided to the author (e.g., to sequentially order the first task card and second task card together).
In some embodiments, the task authoring criteria includes a criterion that is met when historical data regarding a specified task description indicates that that a novice worker may need additional assistance to complete the task. In this embodiment, in step 203, a determination is made (e.g., it is determined whether the historical data regarding a specified task description indicates that that a novice worker may need additional assistance to complete the task). If so, in step 205, a recommendation is provided to the author (e.g., to add a new demonstration displayed to the novice worker until the novice work has performed the task greater than a predetermined number of times).
In some embodiments, the task authoring criteria includes a criterion that is met when the task cards include a first task card and a second task card that are not sequentially ordered, and when the first task card and the second task card both include an indication that a common area of a work site is needed to complete the tasks specified in the first task card and the second task card. In this embodiment, in step 203, a determination is made (e.g., it is determined whether the first task card and the second task card both include an indication that a common area of a work site is needed to complete the tasks specified in the first task card and the second task card). If so, in step 205, a recommendation is provided to the author (e.g., to sequentially order the first task card and second task card together).
With reference to
In step 221, a human-centric process (HCP) authoring application interface, configured to allow an author to edit one or more tasks associated with implementing the human-centric process, is displayed. The HCP authoring application interface displays one or more task cards. At least one of the task cards includes HCP task characteristic data input fields configured to receive HCP task characteristics of the task card by the author.
While each of a plurality of workers is performing the one or more tasks associated with the implemented human-centric process, worker task status data representative of the respective worker completing one of the tasks associated with the implemented human-centric process is received, in step 222. The current worker task status data comprises objective data, such as data from a tool (e.g., torque wrench) used by the worker, an image taken by the worker using camera-enabled device (e.g., smartphone), information processed from an image taken by the worker using image recognition technology, worker location data, and/or data entered by the worker (e.g., notes, text and/or voice recordings). Prior worker task status data and prior author process modification data for prior human-centric processes may also be received. The prior worker task status data may indicate an undesirable outcome such as the worker took too long to complete the task or had to consult with an expert for assistance. The prior author process modification data may indicate that the prior author for a prior process modified the process step as a result of the prior worker task status data indicating an undesirable outcome.
In step 223, it is determined whether the current worker task status data meets task completion criteria. In response to a determination that the worker task status data does not meet task completion criteria, in step 224, a recommendation is provided to alter one or more characteristics of the task card in a recommendation area of the HCP authoring application window. In response to a determination that the current worker task completion data meets task completion criteria, in step 225, the task card for the HCP is displayed in the task card area of the HCP authoring application window.
In some embodiments, the current worker task completion data comprises at least one of: an image, a task completion time, and tool usage data generated by a tool used by the respective worker.
In some embodiments, the current worker task completion data includes task troubleshooting data.
In some embodiments, the task completion criteria includes a criterion that is met when, for a given task having two versions, factors representative of a first portion of the worker task completion data representative of a first version of the given task and the second portion of the worker task completion data representative of the second version of the given task indicate that the first portion of the work task completion data has a more favorable outcome as compared to a second portion of the worker task completion data. In such embodiments, if it is determined in step 223 that that the first portion of the worker task completion data has a more favorable outcome for the HCP, in step 224, a recommendation is provided to an author to use the first portion of the worker task completion data for the HCP. In response to a determination that the first portion of the worker task completion data has a less favorable outcome for the HCP, in step 224, a recommendation is provided to an author to use the second portion of the worker task completion data for the HCP.
In some embodiments, the task completion criteria includes a criterion that is met when, for a given task, task factors representative of the worker task completion data indicate that the task has an undesirable outcome. In such embodiments, in response to a determination that the factors representative of the worker task completion data indicate that the task has an undesirable outcome in step 223, a recommendation is provided to an author to alter the HCP task characteristics for the HCP, in step 224. In response to a determination that the factors representative of the worker task completion data indicate that the task has a desirable outcome in 223, the task is maintained for the HCP in step 225.
In some embodiments, the task factors representative of the worker task completion data include at least one of: a task complete time, a task completion cost, and a task completion quality value.
In some embodiments, the task authoring criteria includes a criterion that is met when the task cards include a first task card and a second task card that are not sequentially ordered, and the first task card and the second task card both include an indication that a common tool is needed to complete the tasks specified in the first task card and the second task card. In such embodiments, in response to a determination that the first task card and the second task card both include an indication that a common tool is needed to complete the tasks specified in the first task card and the second task card, a recommendation is made to the author, in step 224, to sequentially order the first task card and second task card together.
In some embodiments, the task authoring criteria includes a criterion that is met when the task cards include a first task card and a second task card that are not sequentially ordered, and when the first task card and the second task card both include an indication that a common area of a work site is needed to complete the tasks specified in the first task card and the second task card. In such embodiments, in response to a determination that the first task card and the second task card both include an indication that a common area of a work site is needed to complete the tasks specified in the first task card and the second task card, in step 224, a recommendation is provided to the author to sequentially order the first task card and second task card together.
It will be appreciated by those skilled in the art that changes could be made to the exemplary embodiments shown and described above without departing from the broad inventive concept thereof. It is understood, therefore, that this invention is not limited to the exemplary embodiments shown and described, but it is intended to cover modifications within the spirit and scope of the present invention as defined by the claims. For example, specific features of the exemplary embodiments may or may not be part of the claimed invention and features of the disclosed embodiments may be combined. Unless specifically set forth herein, the terms “a”, “an” and “the” are not limited to one element but instead should be read as meaning “at least one”.
It is to be understood that at least some of the figures and descriptions of the invention have been simplified to focus on elements that are relevant for a clear understanding of the invention, while eliminating, for purposes of clarity, other elements that those of ordinary skill in the art will appreciate may also comprise a portion of the invention. However, because such elements are well known in the art, and because they do not necessarily facilitate a better understanding of the invention, a description of such elements is not provided herein.
Further, to the extent that the method does not rely on the particular order of steps set forth herein, the particular order of the steps should not be construed as limitation on the claims. The claims directed to the method of the present invention should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the steps may be varied and still remain within the spirit and scope of the present invention.
The present application claims priority to U.S. Provisional Patent Application No. 62/684,466, filed Jun. 13, 2018, which is hereby incorporated by reference in its entirety.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US19/36894 | 6/13/2019 | WO | 00 |
Number | Date | Country | |
---|---|---|---|
62684466 | Jun 2018 | US |