Patent Application
20070078966
The subject invention relates generally to industrial automation systems and more particularly toward human machine interfaces (HMIs).
Industrial control systems have enabled modern factories to become partially or completely automated in many circumstances. At the core of the industrial control system, is a logic processor such as a programmable logic controller (PLC). Programmable logic controllers are programmed to operate manufacturing processes via logic programs or routines. These programs are stored in memory and generally executed by the PLC in a sequential manner although instruction jumping, looping and interrupt routines, for example, are also common. Control systems also typically include a plurality of input and output (I/O) modules communicatively coupled to a PLC via a backplane that interface at a device level to switches, contactors, relays, solenoids and sensors, among other devices. Accordingly, such control systems are optimized to control and monitor industrial processes, machines, manufacturing equipment, plants, and the like.
Human machine interfaces (HMIs) or simply user interfaces are important to the successful operation and maintenance of industrial automation devices including both control systems and associated equipment or machinery. User interfaces provide the essential communication link between operators and automation devices. This link allows operators to, among other things, setup and control devices and receive feedback by monitoring device status and health during operation. Without such user interfaces, high-level industrial automation would be difficult if not impossible to achieve.
Over the years, user interfaces have gone through several changes. At first, user interfaces were simply dumb terminals, which merely displayed text messages to end-users indicative of some process performed by a server or processor associated with an automated device. For instance, a failed device would generate an internal error code representing a determined error, which could then be matched to a particular error message and displayed to a user or operator on a display device. Over time, client side processing developed so as to enable a move from a text-based interface to a graphical user interface (GUI). This transition shifted some of the processing burden away from the automated device or associated processor toward the client side GUI. These new GUIs vastly improved the ability of users to access information quickly and easily. Unfortunately, these GUIs were not portable in part because of there size and machine dependencies and therefore not a viable option for managing and controlling a plurality of devices connected together in a network. Shortly thereafter, the processing burden shifted back toward devices and away from interfaces with the advent the Internet and web browsers. As a result, developers began to employ web browsers as interface mechanisms.
The following presents a simplified summary of the invention in order to provide a basic understanding of some aspects of the invention. This summary is not an extensive overview of the invention. It is not intended to identify key/critical elements of the invention or to delineate the scope of the invention. Its sole purpose is to present some concepts of the invention in a simplified form as a prelude to the more detailed description that is presented later.
Briefly described, systems and methods to facilitate presentation and interaction with automation data are provided. More specifically, mechanisms and methods are presented that supply additional information when desired without requiring navigation to a different presentation display.
In accordance with an aspect of the invention, items or objects in a base presentation are identified. Identification of an item causes a graphical element to be generated and superimposed over the base presentation layer. The graphical element can provide, among other things, additional information regarding the identified item. Information can be provided in a myriad of forms including but not limited to alphanumeric characters, graphics, animations, audio and video. The graphical element can be dismissed, thus returning the display to the original base presentation, upon navigation away from or de-identifying a base presentation item.
In accordance with another aspect of the invention, the graphical elements can be interactive. Graphical elements can provide, request and receive data. For example, graphical elements can enable, among other things, user authentication, change or setting of control values and/or display formats, operation execution, navigation to other presentation displays and/or graphical elements.
In accordance with yet another aspect of the invention, the graphical elements can be dynamically updated. Although the graphical elements can provide static information, they can also provide real-time or dynamically updated information. Accordingly, if information provided by the graphical element changes in a source it will also be changed in the graphical element.
According to still another aspect of the invention, presented data and interaction can be provided based on explicit settings and/or learned based on training or previous interaction. Furthermore, presented data and interaction can be limited or restricted based on user security credentials, among other things.
To the accomplishment of the foregoing and related ends, certain illustrative aspects of the invention are described herein in connection with the following description and the annexed drawings. These aspects are indicative of various ways in which the invention may be practiced, all of which are intended to be covered by the present invention. Other advantages and novel features of the invention may become apparent from the following detailed description of the invention when considered in conjunction with the drawings.
a-9c are exemplary graphical interfaces that depict interaction with a base presentation in accordance with an aspect of the subject invention.
The subject invention is now described with reference to the annexed drawings, wherein like numerals refer to like or corresponding elements throughout. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed. Rather, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the subject invention.
As used in this application, the term “component,” “system” and the like are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an instance, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computer and the computer can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
The word “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs.
As used herein, the terms “infer” or “inference” refer generally to the process of reasoning about or inferring states of the system, environment, and/or user from a set of observations as captured via events and/or data. Inference can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The inference can be probabilistic—that is, the computation of a probability distribution over states of interest based on a consideration of data and events. Inference can also refer to techniques employed for composing higher-level events from a set of events and/or data. Such inference results in the construction of new events or actions from a set of observed events and/or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources. Various classification schemes and/or systems (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines . . . ) can be employed in connection with performing automatic and/or inferred action in connection with the disclosed subject matter.
Furthermore, the present invention may be implemented as a method, system, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer or automation device (e.g., controller) to implement the disclosed invention. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media. For example, computer readable media can include but is not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips . . . ), optical disks (e.g., compact disk (CD), digital versatile disk (DVD) . . . ), smart cards, and flash memory devices (e.g., card, stick, jump drive . . . ). Additionally it should be appreciated that a carrier wave can be employed to carry computer-readable electronic data such as those used in transmitting and receiving electronic mail or in accessing a network such as the Internet or a local area network (LAN). Of course, those skilled in the art will recognize many modifications may be made to this configuration without departing from the scope or spirit of the subject invention.
Referring initially to
Automation devices can include any one of a plurality of industrial processes and machines including but not limited to programmable logic controllers (PLCs), pumps providing fluid transport and other processes, fans, conveyor systems, compressors, gearboxes, motion control and detection devices, sensors, screw pumps, and mixers, as well as hydraulic and pneumatic machines driven by motors. Such motors can be combined with other components, such as valves, pumps, furnaces, heaters, chillers, conveyor rollers, fans, compressors, gearboxes, and the like, as well as with appropriate motor drives to form industrial machines and actuators. For example, an electric motor could be combined with a motor drive providing variable electrical power to the motor, as well as with a pump, whereby the motor rotates the pump shaft to create a controllable pumping system. Accordingly, the term automation device refers to control systems (e.g., PLCs) and components thereof (e.g., modules) as well as the equipment, machines and/or systems with which they interact.
Determination component 120 can determine whether an item is identified. Determination component 120 receives, retrieves or otherwise obtains the base presentation or a representation thereof, for example from an interface system proximate or remote from automation devices. Additionally, determination component 120 can optionally receive an input. The input can correspond to navigational input, among other things, such as the movement and location of a mouse, stylus, or other pointing device. It should also be appreciated that the input could be an event (e.g., an error, warning, alarm . . . ) originating with a control system condition related to an item represented on the base presentation. In any case, determination component 120 can determine whether an item is identified based on the base presentation and the input. Upon determination that an item has been identified, determination component 120 can output the identity of the item. Determination component 120 can also determine if and when an identified item is no longer identified and generate and indication thereof.
Element component 130 is communicatively coupled to determination component 120. Upon receipt of the identity of an item, amongst other information (e.g., location of interface relative to devices), element component 130 can modify the base presentation. For example, element component could overlay or superimpose a graphical element on the base presentation. The graphical element can take many forms including but not limited to a bubble or a box (e.g., text box, dialog box . . . ). In accordance with an aspect of the invention, this graphical element can be situated in close proximity to an identified item and provide additional information about the identified item. Thus, the graphical element can correspond to a tool tip, but is not limited thereto. The information provided by the graphical element can be in the form of one or more of text, numbers, graphics, animations, audio, and video, inter alia. Furthermore, it should be appreciated that the modification to the base presentation need not be graphical. It could simply be the addition of audio, for instance. The modification of the base presentation can remain in effect while an item is identified. The addition(s) can be removed and the base presentation restored by element component 130 upon receipt of a signal from determination component that the previous item is no longer identified.
It should also be noted and appreciated that determination component 120 can determine identification of modification to the base presentation such as a graphical element, or portions thereof, generated by element component 130. In one exemplary scenario, a first element in the base presentation can be identified and a graphical element produced in response to the identification. This graphical element could provide additional information concerning the identified item. Subsequently, the graphical element or part of the element such as a graphic or text can be identified and a second graphical element produced as a consequence. This second graphical element can provide further information about the identified graphic. The graphical elements could be dismissed, among other things, upon navigation away from identified items.
The subject invention as claimed has numerous advantages. One advantage that can be gleaned from the description thus far is that the subject invention provides a mechanism for providing additional or less important information if and when needed without requiring navigation to a different presentation display. Further, such mechanism reduces information clutter that can obscure or completely replace the base presentation by providing additional information upon identification and dismissal of such information upon de-identification.
Turning briefly to
Settings component 410 can retrieve user settings regarding identified items. For example, a user can specify, for a base presentation, that they would like to view more detailed information regarding item A, followed by item B and terminating with item C. The settings may also specify a time period associated with each. Accordingly, settings component 410 can review the settings and indicate that item A is selected for five seconds, followed by item B for ten seconds, and then item C for five seconds.
Intelligence component 420 can infer (as that term is defined herein) and automatically select items of interest for a period of time. Intelligence component 420 can infer that a user would desire to view additional information about an item in the base presentation based on past interaction as well as context. Accordingly, intelligence component 420 can employ artificial intelligence (e.g., support vector machines, neural networks, Bayesian belief networks, fuzzy logic . . . ) and/or rule based (e.g., expert systems, knowledge based systems . . . ) methods or mechanisms that resolve whether an item(s) should be designated as identified and facilitate machine learning. By way of example, based on previous interactions (from detection component) intelligence component 420 can determine or learn that a user typically identifies item A for five seconds and item B for ten seconds. Accordingly, on display of the base presentation item A can be identified as being selected followed by item B. Intelligence component 410 may also, sua sponte, set item C as being selected based on context, such as a malfunction or potentially dangerous operation, or upcoming event (e.g., scheduled or periodic maintenance, among other things.
Referring to
Settings 520 are explicitly defined instructions or rules, for example determined by a system designer pertaining to what data is provided, how it is provided, to whom such data is supplied, and whether or the type of interaction allowed. In essence, rules can be defined based upon, inter alia, role and context or situational awareness. As per role, particular data can be designated for presentation based on a user role and/or position. For example, if it is known, that a user is a machine operator, it can be defined that current operating parameters of an automation device such as a pump (e.g., on/off, flow rate, operational alarms . . . ) be presented or displayed first. Alternatively, if the role corresponds to maintenance engineer, information regarding when the pump was last serviced and characteristics such as vibration, can be provided that indicate whether a device needs to be serviced or replaced. Rules or instructions can also be specified with respect to context or situational awareness. Context information can be supplied from outside the element component 130 or determined and provided by context component 512 within the element component 103 which communicates with presentation component such information to presentation component 510. By way of example and not limitation, rules may be defined relative to the physical location to an automation device. Location of an interface relative to an identified device is context information that can be determined and supplied by context component 512. Interaction at a location proximate to a device may present different information and allow initiation of disparate actions than if the user was at a remote location. For instance, for safety purposes a user could be prevented from operating a five ton punch press remotely via a laptop and web browser even though that same user would be able to do so if they were next to the machine. Therefore, location of a user may change the data presented and how a user can interact therewith.
In addition to data provided by the element, the presentation component 510 can control the format thereof, for example, based on context and/or settings. For instance, the settings and/or context could influence graphical element color and measure system such as whether temperature is provided in degrees Celsius, Fahrenheit, or both.
The settings 520 may first be set by designers, but users may modify some settings. For example, users can indicate through interaction that certain data be shown to them again, that particular data not be provided again, or a particular order of presentation. Users may also modify settings regarding format of data presentation. However, there may be some settings that are not modifiable for safety or security reasons.
The element component 130 can also include a learning component 530 communicatively coupled to the presentation component 510. Similar to settings 520, the learning component 530 can provide information (e.g., based on context . . . ) to the presentation component to facilitated appropriate format and presentation. By contrast, learning component 520 can learn or infer (as that term is defined herein) on a user-by-user basis what information or interactive capabilities are desired as well as the order and format of the presentation, among other things. Based on training or previous interaction, learning component 530 can learn what a particular person wants to know about an automation device, for example, rather than presenting information based on settings 520.
Connection component 540 is communicatively coupled to presentation component 510. Connection component 540 facilitates interaction with one or more sources 550 or data sources. In one instance, presentation component 510 can determine that it will need to provide particular information located in a source. Presentation component 510 can identify the source and provide the identity to connection component 540. Upon receipt of a request, connection component 550 can set up the data source connections necessary to supply data to the presentation component 510. The data sources can provide static data (e.g., configuration data), historical data, reporting service data, and/or live or real time data, among other types and forms of data.
It should also be appreciated that the element provided by presentation component 510 can be interactive. Accordingly, it may receive data that is to be stored. Received data can be written to a source via connection component 540. The received data can also be utilized to alter settings or presentation formats, inter alia. Still further yet it should be noted that element component 130, and more specifically presentation component 510, can facilitate navigation to other presentation displays and/or elements, among other things.
Turning to
a-9c illustrate exemplary graphical interfaces 900a, 900b, and 900c. Together the interfaces depict interaction with a base presentation to facilitate clarity and understanding with respect to aspects of the subject invention. It should be noted that these illustrations are provided by way of example and not limitation. As one of skill in the art can appreciate, there is a myriad of ways to arrange and present items on graphical interfaces. The depicted interfaces illustrate only one such arrangement and are not meant to limit the scope of the subject invention to that which is disclosed.
a is an exemplary interface 900a with base presentation 910. The base presentation includes graphical representations of two tanks connected to a valve connected to a pump, a mixer to mix the material provided by the tanks, a motor to power the mixer, another valve and another pump. The tanks have bars on them to indicate pressure within the tanks. Accordingly, the base presentation provides a degree of information concerning a system. The arrow or cursor 920 can be positioned within the base presentation in response to a gesture, voice command, or the like.
b illustrates an exemplary interface 900b with base presentation 910. The base presentation is the same as that of
The aforementioned systems have been described with respect to the interaction between several components and/or systems. It should be appreciated that such systems can include those components and/or systems specified therein, some of the specified components, and/or additional components specified in other systems. By way of example and not limitation, element component 130 can include settings component 520, learning component 530, context component 512, security component 610, and update component 810 or any combination thereof. In particular, context and security information can be provided from outside element component 130 thereby eliminating the need for context component 512 and security component 610. Additionally, it should be noted that one or more components may be combined into a single component to provide aggregate functionality or divided into several subcomponents. For instance, detection component 210 can include gesture sub-component 320 and audio sub-component 330 or alternatively be communicatively coupled to such components outside the detection component 210. The components may also interact or be integrated with one or more other components or systems not specifically described herein for purposes of brevity but known by those of skill in the art.
Furthermore, as will be appreciated various portions of the disclosed systems above and methods below may include or consist of artificial intelligence or knowledge or rule based components, sub-components, processes, means, methodologies, or mechanisms (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines, classifiers . . . ). This includes but is not limited to components previously described with such functionality, for example intelligence component 420 and learning component 530. Such components, inter alia, can automate certain mechanisms or processes performed thereby to make portions of the systems and methods more adaptive as well as efficient and intelligent.
In view of the exemplary systems described supra, methodologies that may be implemented in accordance with the present invention will be better appreciated with reference to the flow charts of
Additionally, it should be further appreciated that the methodologies disclosed hereinafter and throughout this specification are capable of being stored on an article of manufacture to facilitate transporting and transferring such methodologies to computers. The term article of manufacture, as defined supra, is intended to encompass a computer program accessible from any computer-readable device, carrier, or media.
In order to provide a context for the various aspects of the invention,
With reference to
The system bus 1318 can be any of several types of bus structure(s) including the memory bus or memory controller, a peripheral bus or external bus, and/or a local bus using any variety of available bus architectures including, but not limited to, 11-bit bus, Industrial Standard Architecture (ISA), Micro-Channel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), Universal Serial Bus (USB), Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), and Small Computer Systems Interface (SCSI).
The system memory 1316 includes volatile memory 1320 and nonvolatile memory 1322. The basic input/output system (BIOS), containing the basic routines to transfer information between elements within the computer 1312, such as during start-up, is stored in nonvolatile memory 1322. By way of illustration, and not limitation, nonvolatile memory 1322 can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory 1320 includes random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM).
Computer 1312 also includes removable/non-removable, volatile/non-volatile computer storage media.
It is to be appreciated that
A user enters commands or information into the computer 1312 through input device(s) 1336. Input devices 1336 include, but are not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, and the like. These and other input devices connect to the processing unit 1314 through the system bus 1318 via interface port(s) 1338. Interface port(s) 1338 include, for example, a serial port, a parallel port, a game port, and a universal serial bus (USB). Output device(s) 1340 use some of the same type of ports as input device(s) 1336. Thus, for example, a USB port may be used to provide input to computer 1312 and to output information from computer 1312 to an output device 1340. Output adapter 1342 is provided to illustrate that there are some output devices 1340 like displays (e.g., flat panel and CRT), speakers, and printers, among other output devices 1340, that require special adapters. The output adapters 1342 include, by way of illustration and not limitation, video and sound cards that provide a means of connection between the output device 1340 and the system bus 1318. It should be noted that other devices and/or systems of devices provide both input and output capabilities such as remote computer(s) 1344.
Computer 1312 can operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1344. The remote computer(s) 1344 can be a personal computer, a server, a router, a network PC, a workstation, a microprocessor based appliance, a peer device or other common network node and the like, and typically includes many or all of the elements described relative to computer 1312. For purposes of brevity, only a memory storage device 1346 is illustrated with remote computer(s) 1344. Remote computer(s) 1344 is logically connected to computer 1312 through a network interface 1348 and then physically connected via communication connection 1350. Network interface 1348 encompasses communication networks such as local-area networks (LAN) and wide-area networks (WAN). LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet/IEEE 802.3, Token Ring/IEEE 802.5 and the like. WAN technologies include, but are not limited to, point-to-point links, circuit-switching networks like Integrated Services Digital Networks (ISDN) and variations thereon, packet switching networks, and Digital Subscriber Lines (DSL).
Communication connection(s) 1350 refers to the hardware/software employed to connect the network interface 1348 to the bus 1318. While communication connection 1350 is shown for illustrative clarity inside computer 1312, it can also be external to computer 1312. The hardware/software necessary for connection to the network interface 1348 includes, for exemplary purposes only, internal and external technologies such as, modems including regular telephone grade modems, cable modems, power modems and DSL modems, ISDN adapters, and Ethernet cards.
The system 1400 includes a communication framework 1450 that can be employed to facilitate communications between the client(s) 1410 and the server(s) 1430. The client(s) 1410 are operatively connected to one or more client data store(s) 1460 that can be employed to store information local to the client(s) 1410. Similarly, the server(s) 1430 are operatively connected to one or more server data store(s) 1440 that can be employed to store information local to the servers 1430.
What has been described above includes examples of the present invention. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the present invention, but one of ordinary skill in the art may recognize that many further combinations and permutations of the present invention are possible. Accordingly, the present invention is intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims. Furthermore, to the extent that the terms “includes,” “has,” and “having” are variations in form thereof are used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.
