Patent Application
20090217183
Embodiments of the present invention relate to a user interface, and more specifically to a graphical user interface that shows a visualization of real and virtual data.
In most manufacturing environments, reactive maintenance and quality control strategies are used to fix problems as they occur. Examples of such reactive strategies include statistical process control (SPC) and advanced process control (APC). A disadvantage of reactive strategies is that they respond to problems after such problems have already occurred. Thus, product may be scrapped, machines may be taken out of commission, employee man hours may be occupied, etc. as a result of the problem.
To prevent problems before they occur, some manufacturing environments implement predictive strategies. One such predictive strategy is to simulate future values of parameters. However, the visualizations, software and/or hardware implementations, etc. of the simulation of such future values are separate and distinct from those that show past and present values of the manufacturing environment. Such simulations are therefore not integrated with information available at the factory floor, and use different data than what is available at the factory floor. If an engineer desires to see predicted future values, he must execute a first application that provides such predictions. However, if an engineer desires to see past or present values, he must execute a separate second application that provides such information. Usually, a simulation application presents data in a different manner and with different controls than those used by an application that presents past and present values.
Simulation solutions also suffer from the use of models that are static approximations of the systems that they simulate at a specified point in time. Therefore, the results that such simulations produce in problem investigation are often based on obsolete or old data. While such simulations may still be valuable, they are often not trusted, and therefore are used only as supplementary information rather than true predictors.
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:
Described herein is a method and apparatus for providing a user interface through which a user can review and interpret both acquired data (e.g., from sensors, test machines, entered data, etc.) and virtual data. In one embodiment, first acquired data that represents past values of a parameter or parameters is displayed in a user interface through which a user can monitor, control and predict system operations. Second acquired data that represents present values of the parameters is displayed in the user interface. Virtual data that represents predicted future values of the one or more parameters is displayed in the same user interface. In one embodiment, a quality indicator that indicates a degree of accuracy of the virtual data is also displayed in the user interface. The virtual data and the quality indicator may be updated as time advances. In one embodiment, the first acquired data, the second acquired data and the virtual data are presented with a unified visual appearance such that a relationship between the past values, present values and predicted future values is visually indicated. The first acquired data, second acquired data and virtual data may be simultaneously represented in a graph, overlay, animation and/or report.
In the following description, numerous details are set forth. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the present invention.
Some portions of the detailed descriptions which follow are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. Unless specifically stated otherwise, as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as “displaying”, “receiving”, “consolidating”, “generating”, “updating”, or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
The present invention also relates to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.
The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will appear as set forth in the description below. In addition, the present invention is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the invention as described herein.
The present invention may be provided as a computer program product, or software, that may include a machine-readable medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to the present invention. A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., a computer) readable storage medium (e.g., read only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices, etc.), a machine (e.g., computer) readable transmission medium (electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.)), etc.
The network 125 may be a public network (e.g., Internet), a private network (e.g., Ethernet or a local area Network (LAN)), or a combination thereof. The network 125 may include multiple private networks, which may be directly connected or connected via a public network. For example, the supply chain database 120 may be connected to a first private network controlled by suppliers, the customer database 115 may be connected to a second private network controlled by a customer, and the MICS 105 and MES 110 may be connected to a third private network. Each of these private networks may be connected via a public network.
A supply chain database 120 includes information available to and/or provided by a supplier or distributor. Such information may include, for example, a supplier's orders (e.g., of parts and goods), supplier inventory (e.g., current inventory, projected inventory, etc.), projected delivery dates, etc. Where materials are acquired from multiple distributors or suppliers, architecture 100 may include multiple supply chain databases 120. For example, a first supply-chain database may include information on raw goods, and a second supply-chain database may include information on manufacturing equipment.
A customer database 115 includes information available to and/or provided by a customer. Such information may include, for example, customer demand for specified articles of manufacture, customer inventory, etc. Architecture 100 may include a single customer database 115 for multiple customers, or multiple customer databases 115, each of which provides information about a distinct customer.
The Manufacturing execution system (MES) 110 is a system that can be used to measure and control production activities in a manufacturing environment. The MES 110 may control some production activities (e.g., critical production activities) or all production activities of a set of manufacturing equipment (e.g., all photolithography equipment in a semiconductor fabrication facility), of a manufacturing facility (e.g., an automobile production plant), of an entire company, etc. The MES 110 may include manual and computerized off-line and/or on-line transaction processing systems. Such systems may include manufacturing machines, metrology devices, client computing devices, server computing devices, databases, etc. that may perform functions such as processing, equipment tracking, dispatching (e.g., determining what material goes to what processes), product genealogy, labor tracking (e.g., personnel scheduling), inventory management, costing, electronic signature capture, defect and resolution monitoring, key performance indicator monitoring and alarming, maintenance scheduling and so on.
In one embodiment the MES 110 is connected with one or more MES data stores 130. The MES data stores 130 may be databases, file systems, or other arrangements of data on nonvolatile memory (e.g., hard disk drives, tape drives, optical drives, etc.), volatile memory (e.g., random access memory (RAM)), or combination thereof. Each MES data store 130 may store, for example, historical process information of manufacturing recipes (e.g., temperatures, pressures, chemicals used, process times, etc.), equipment maintenance histories, inventories, etc.
The manufacturing information and control system (MICS) 105 combines disparate information from multiple different sources (e.g., data stores), and presents this information in a single interface. The MICS 105 can be used to gain an understanding of the manufacturing environment, and can enable a user to determine an efficiency of the manufacturing environment and/or how to improve all or components of the manufacturing environment. The MICS 105 may also draw inferences from, report out, and/or act upon the combined information. For example, MICS 105 can act as an early warning system (e.g., predict scrap, initiate product rework, etc.), provide bottleneck analysis, provide asset management (e.g., reduce unscheduled equipment downtime), improve lean practices, etc. In one embodiment, MICS 105 includes a data consolidator 140, a user interface 145, a decision support module 155, an execution module 160, a predictor 150, and a real-time monitor 165.
The data consolidator 140 consolidates acquired data of past parameter values and/or present parameter values from multiple different sources (e.g., data stores), and presents the acquired data as if it originated from a single data source. The manner in which the acquired data from the data sources is consolidated may be dependant upon relationships between the acquired data. Such relationships may be user defined. Moreover, sources from which data is consolidated, and the manner in which data is consolidated, can be user configurable. Therefore, as new data stores are added and/or old data stores are removed, data consolidator 140 may be adapted to accommodate the change.
In one embodiment, data consolidator 140 consolidates data from multiple MES data stores 130 (e.g., an inventory data store, a maintenance data store, a metrology data store, process data stores, etc.). In a further embodiment, the data consolidator 140 consolidates data from the supply chain databases 120 and/or customer databases 115. In yet a further embodiment, data consolidator 140 consolidates real-time data as the data is collected by real-time monitor 165 (e.g., from manufacturing machines and metrology machines). In still a further embodiment, data consolidator 140 consolidates virtual data that has been generated by predictor 150. Data consolidator 140 can also consolidate manually entered data (e.g., data entered by a device operator, maintenance personnel, etc.).
In one embodiment, data consolidator 140 stores consolidated acquired data in MICS data store 135. Alternatively, data consolidator 140 may store a subset of all consolidated acquired data in MICS data store 135. For example, data consolidator 140 may store consolidated data necessary to generate virtual data in MICS data store 135.
Real-time monitor 165 collects real-time data of present values of one or more parameters. Such real-time data may be collected from sensors and systems to which MICS 105 is connected via network 125. Real-time monitor 165 may, for example, collect data from manufacturing equipment and metrology equipment as the data is generated. In one embodiment, real time monitor 165 provides the real-time data to data consolidator 140.
Predictor 150 uses the consolidated acquired data of past parameter values and/or present parameter values (e.g., of one or more elements of the manufacturing execution system 110) to generate virtual data that predicts future values of the parameters. Predictor 115 may then store the generated virtual data in MICS data store 135. Virtual data may be generated and stored without impacting the historical data stores of the acquired data. Virtual data may be generated based on trend extrapolation, interpolation, simulation (e.g., model-based simulation), etc.
The methods, models and/or algorithms used to generate virtual data may depend on the parameter being predicted. For example, a first simulation model may be used to predict future values of a first parameter, and a second simulation model may be used to predict future values of a second parameter.
In one embodiment, predicting future values of a particular parameter requires access to past values of the parameter. Access to past values of associated parameters that influence the parameter that is to be predicted may also be necessary. For example, predicting a buffer length parameter value for a specific manufacturing device may include collecting parameter values of lots received, lots dispatched, scheduled maintenance for similar manufacturing devices, health statistics for similar manufacturing devices, etc.
Predicting future values of a particular parameter may also include predicting the future values of associated parameters. For example, to accurately predict a pressure of a device, a model may first need to predict a future power and gas flow rate of the device. Associated future predictions are referred to herein as virtual extensions. The virtual extension can have a one to many relationship with nonvirtual components (acquired data), and these non-virtual components can each be associated with one or more virtual component, which means that virtual extensions can be applied to more than one data parameter. Therefore, a single virtual extension (or set of virtual extensions) may be used for predictions of multiple different parameters. A single virtual extension may also be associated with multiple predicted values of a single virtual data parameter. Applying a virtual extension to multiple predicted values can minimize consumption of data storage.
In one embodiment, predictor 150 generates data quality indicators, and associates the data quality indicators with the virtual data. A data quality indicator can be implemented as a specialized type of virtual extension that has no present or past counterpart. Data quality indicators indicate a degree of accuracy of the virtual data. Data quality indicators may include, for example, error bars, standard deviation, variance, mean, etc. These data quality indicators allow the user to understand the confidence of the prediction and make use of the prediction in a quantitative manner.
The further into the future a prediction is extended, the lower the degree of accuracy of the virtual data. Therefore, the data quality generally degrades with time, and this is conveyed with the data quality indicators. As with other virtual extensions, data quality indicators may be associated with multiple parameters. As an example, if a data qualifier includes upper and lower bounds, and these bounds are valid for a number of data points over a period of time, the scheme allows all of these data points to be tied to a single upper and lower bounds set.
The first relationship set 176 shows that multiple virtual data parameters of the virtual data set 172 can share a relationship with a single virtual extension of the virtual extension set 174. The relationships characterized by first relationship set 176 are such that the virtual data set 172 can be dependent on and/or modified by virtual extension set 174. The database schema 170 also includes a second relationship set 178 showing that multiple virtual extensions of the virtual extension set 174 may share a relationship with another virtual extension of the virtual extension set 174. The relationships characterized by second relationship set 178 are such that the virtual extension set 174 can be dependent on and/or modified by virtual extension set 174. This allows a recursive relationship between virtual extensions.
In an example, if a parameter such as temperature is being predicted with a prediction granularity of one second with a time horizon of one minute, then 60 predicted virtual data parameter values would be stored in a virtual database. If a confidence indicator is calculated with each prediction, then 60 confidence indicators would also be stored. If 3-sigma upper and lower bounds are only calculated every tenth data point, then 6 each of upper and lower bounds (rather than 60 each) would be stored, with each of these extensions being linked to 10 predicted values via foreign keys. If a single absolute upper bound is calculated every minute based on the 3-sigma upper bounds, then a single value of absolute upper bound would be linked to the 6 3-sigma upper bounds (and thus are linked to all 60 predicted values). This extension approach also allows for easier reconfigurability of prediction capabilities and the subsequent update of prediction extension information.
Returning to
In one embodiment, data consolidator 140 continuously consolidates new acquired data with existing consolidated real and virtual data. The new acquired data may be real-time data received from real-time monitor 165 and/or additional data added to MES data stores 130, supply chain databases 120, customer databases 115, etc. The new acquired data may replace portions of virtual data (e.g., where a timestamp for the new acquired data matches a timestamp of the virtual data). For example, once a predicted event occurs, a predicted value for the event may be replaced by a real value.
In one embodiment, predictor 150 continuously updates the virtual data using a dynamic prediction model. As new data is gathered and consolidated by data consolidator 140, the new data can be used to recalculate the virtual data using the designated prediction techniques (e.g., models). Therefore, the virtual data may always be based on the most current data, increasing prediction accuracy. Each recalculation can provide a more accurate prediction of future values. In one embodiment, recalculating the virtual data includes recalculating data quality indicators associated with the virtual data. In one embodiment, recalculating virtual data includes replacing the original virtual data. Alternatively, the original virtual data may not be overwritten.
User interface 145 provides interactive displays of the consolidated acquired data and virtual data. Through these displays, a user can view past, present, and future values of the manufacturing environment. In one embodiment, user interface 145 provides flexible visualization capabilities such as customizable plots, overlays, summary data, key performance indicators, and drill downs to individual data point values. In another embodiment, user interface 145 provides animated visualizations of information so that dynamic properties of the information can be viewed in a condensed time format. User interface 145 may access and present virtual data and acquired data in the same manner, thus providing an intuitive access to predicted data alongside historical data. The data quality indicator may provide additional information on quality of predicted virtual data, superimposed on a visualization of the virtual data. Embodiments of the user interface 145 are described in greater detail below in reference to
Referring to
Execution logic component 160 is responsible for taking action on the business systems based on the output of the decision support logic component 155. These actions are in the form of intelligent business rules that can be launched either through real-time system events, predicted events, or scheduled activities. For example, execution logic 160 may automatically schedule maintenance for a machine when certain values are detected, automatically shut down the machine, etc.
Though the exemplary architecture 100 described above is of a manufacturing environment, embodiments of the present invention may also operate in other environments such as an investment environment (e.g., for trading stocks, bonds, currencies, etc.), a research environment, etc. In such alternative environments, no manufacturing execution system may be present, and the manufacturing information and control system may instead be a research information and control system, investment information and control system, etc. However, the functionality of the data consolidator 140, user interface 145, decision support logic component 155, execution logic component 160, predictor 150 and real-time monitor 165 may remain unchanged regardless of the environment.
Referring to
In another embodiment, the key performance indicator visualization may be animated to dynamically display the evolution of indicator values from the past through the present into the future in a compressed time format. For example, the animation may equate one second of display time to one hour of factory time. The indicators may be animated to illustrate data from one week in the past, through the present, to one week in the future (in an approximately 6 minute animation). As the animation crosses the present time boundary (referred to as NOWTIME here) into the prediction space, the visualization may choose to also display data quality indicators along with the predicted values e.g., by superimposing these values on the animation for all time values greater than NOWTIME.
The key performance indicators may be user defined, and may depend on a system with which they are associated. For example, an alternative dashboard view of key performance indicators of a particular department within a manufacturing facility may be shown. Key performance indicators included in such an alternative dashboard view may include statuses of particular categories of machines, throughput, downtime, etc.
Third view 300 includes a timeline perspective of a collection of specified parameters. The exemplary specified parameters shown include outage analytics for manufacturing equipment. However, a timeline perspective of any specified parameter or parameters may be shown. The timeline perspective can show past values, present values, and future predicted values in a visually unified manner. For example, data points shown at the dates Jan. 1, 2005 and Jan. 2, 2005 may represent past values of acquired data, data points shown at the date Jan. 3, 2005 may represent present values of acquired data, and data points shown at the dates Jan. 4, 2005-Jan. 10, 2005 may represent future values of virtual data.
Third view 300 may include multiple playback controls 305 that advance the timeline perspective forward in time and backward in time. Playback controls 305 may, for example, include stop, play forward, play backward, skip forward, skip backward, etc. controls. Playback controls 305 may be used to animate data shown in the third view 300. Alternatively, the data may be shown in a static format.
A data quality indicator is associated with the future values of the parameter. The data quality indicator represents a confidence level for the predicted future values of the parameter. In the exemplary fourth view 400, the data quality indicator includes an upper control limit and a lower control limit. The upper control limit may represent a maximum possible predicted value for the parameter, and the lower control limit may represent a minimum possible predicted value for the parameter at a given time. The further into the future a value for the parameter is predicted, the lower the confidence level of the prediction. For example, the upper control limit for the parameter at time 70 is shown to be approximately 120, while the upper control limit at time 26 is shown to be approximately 80. Likewise, the lower control limit for the parameter at time 70 is shown to be approximately 40, while the lower control limit at time 26 is shown to be approximately 50.
As shown, the data quality indicator is recalculated at NOWTIME_2. Therefore, different upper control limits and lower control limits are shown for the parameter at NOWTIME_2 than were shown at NOWTIME_1. The tighter control limits shown at NOWTIME_2 reflect the fact that the data quality indicators have been recalculated using additional data that was gathered after NOWTIME_1. In one embodiment, the data quality indicators are continuously recalculated as data is gathered. Alternatively, data quality indicators may be recalculated periodically at specified time intervals (e.g., every 5 seconds, every 10 minutes, every day, etc.).
Referring to
At block 510, virtual data is generated by applying acquired data to a predictive model. The predictive model may be a simulation, extrapolation, interpolation, or other type of predictive model. The virtual data may represent predicted future values of the same parameters represented by the acquired data. In one embodiment, generating the virtual data includes generating virtual extensions that represent additional parameters that are associated with the predicted parameter or parameters. In one embodiment, the virtual data is generated by predictor 150 of
A block 515, data quality indicators are generated for the virtual data. The data quality indicators may include, for example, error bars (e.g., upper control limits and lower control limits), standard deviations, variance, etc. In one embodiment, the data quality indicators are generated by predictor 150 of
At block 520, a database is populated with virtual data and/or the data quality indicators. The database may be, for example, a manufacturing information and control system data store. Alternatively, the database may be a distinct database designated for virtual data, data quality indicators, and/or virtual extensions. Once the database has been populated, the virtual data can be consolidated with the acquired data. In one embodiment, this occurs automatically as virtual data is generated.
At block 525, processing logic determines whether additional acquired data has been received. Additional acquired data may be received, for example, from a real-time monitor (e.g., real time monitor 165 of
At block 535, a portion of the virtual data is replaced with the additional acquired data. In one embodiment, all data is associated with a specific time period. The time period may be a historic point in time at which the data was collected, or a projected future point in time. The point in time may be represented by a timestamp. In one embodiment, the portion of the virtual data that is replaced is replaced by acquired data having a same timestamp as the virtual data.
At block 540, the virtual data is dynamically updated by applying the additional data to the predictive model. In one embodiment, the virtual data is continuously updated as additional data is received. Alternatively, the virtual data may be updated on a periodic basis. At block 545, the data quality indicators are dynamically updated.
At block 530, the acquired data and virtual data are presented to a user via a user interface. In one embodiment, the user interface corresponds to the user interface shown in
Referring to
At block 615, virtual data of future values of the one or more parameters are displayed in the user interface. The virtual data can be displayed in a continuous manner with the first rule data and the second acquired data. In one embodiment, the first acquired data, the second acquired data, and the virtual data are presented with a unified visual appearance such that a relationship between the past values, present values, and future values is visually indicated. In a further embodiment, the user interface includes at least one of a graph, overlay, animation, or report that provides simultaneous representation and visualization of the first acquired data, the second acquired data and the virtual data. In another embodiment, the first acquired data, second acquired data, and virtual data are presented in a timeline perspective.
At block 620, a data quality indicator of the virtual data is displayed in the user interface. The data quality indicator may represent a degree of accuracy of the virtual data. In one embodiment, the data quality indicator varies as a function of time.
The exemplary computer system 700 includes a processor 702, a main memory 704 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory 706 (e.g., flash memory, static random access memory (SRAM), etc.), and a secondary memory 718 (e.g., a data storage device), which communicate with each other via a bus 730.
Processor 702 represents one or more general-purpose processing devices such as a microprocessor, central processing unit, or the like. More particularly, the processor 702 may be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, processor implementing other instruction sets, or processors implementing a combination of instruction sets. Processor 702 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. Processor 702 is configured to execute the processing logic 726 for performing the operations and steps discussed herein.
The computer system 700 may further include a network interface device 708. The computer system 700 also may include a video display unit 710 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 712 (e.g., a keyboard), a cursor control device 714 (e.g., a mouse), and a signal generation device 716 (e.g., a speaker).
The secondary memory 718 may include a machine-readable storage medium (or more specifically a computer-readable storage medium) 731 on which is stored one or more sets of instructions (e.g., software 722) embodying any one or more of the methodologies or functions described herein. The software 722 may also reside, completely or at least partially, within the main memory 704 and/or within the processing device 702 during execution thereof by the computer system 700, the main memory 704 and the processing device 702 also constituting machine-readable storage media. The software 722 may further be transmitted or received over a network 720 via the network interface device 708.
The machine-readable storage medium 731 may also be used to store a data consolidator, predictor and/or user interface 145 (e.g., the data consolidator 140, predictor 150 and user interface 145 of
It is to be understood that the above description is intended to be illustrative, and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reading and understanding the above description. Although the present invention has been described with reference to specific exemplary embodiments, it will be recognized that the invention is not limited to the embodiments described, but can be practiced with modification and alteration within the spirit and scope of the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than a restrictive sense. The scope of the invention should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
