Patent Grant
7822806
The present invention relates generally to user interfaces and more particularly to an extensible interface that draws in resources, as needed, for real-time presentation of data.
Several technologies are available to assist in gathering and presenting data for display through a user interface. One technology is ActiveX of the Microsoft Corporation which provides a robust set of tools to assist in issuing data requests and in processing the requests for presentation on a display. A limitation of ActiveX technology is that it has a large footprint, that is, its memory requirement is not practical for all applications (e.g., for some wireless platforms such as cellular telephones and personal digital assistants (“PDA”)). Another technology is Sun's Java engine but again, an interface that requires that engine has a large footprint and has a predetermined set of abilities as of the release date of the engine. There remains a need in the art for an open, extensible interface engine that has a small native footprint yet is capable of pulling in components, as needed, to suit a given application or platform. The present invention addresses these and other needs.
In one aspect, the present invention includes a method for providing a real-time data display and interface for a network device. In that method, a page is loaded in a browser configured for operation on the network device. The page includes an engine which is executed in the browser. The engine is used to automatically load a definition file from a remote source in response to content within the loaded page. One or more widgets (that is to say, displayed elements such as graphical objects) are instantiated for display in the interface, with each widget making data requests at intervals prescribed by parameters for each respective widget. These data requests are forwarded to one or more servers that are in communication with the network device, and the requested data is thereafter received and presented in the interface thereby providing the user with dynamically updated data.
In another aspect, the present invention includes an engine for providing a real-time data interface to a network device. The engine comprises executable code configured to instantiate one or more widgets for rendering within the display of the real-time data interface. The engine further includes a reference to a definition file which defines prescribed parameters for one or more of the widgets.
These and other aspects, features and advantages of the present invention can be further understood from the accompanying Drawing figures and description of a preferred embodiment.
By way of overview and introduction, the present invention concerns improvements in user interfaces, and more particularly, a lightweight and flexible software engine that permits visualization of real-time data feeds and other constantly changing information. The interface of the present invention is unique in combining portability, low use of resources, and flexibility to accommodate changing operational platforms on which the interface can be presented as well as the changing needs of developers and users. The particular platform that executes the user interface is sometimes referred to herein as the “network device.” The interface of this invention allows the display of visual objects (called “widgets” herein), wherein each object can contain codes governing its own continuously changing behavior and the ability to interface with a user. In part, the interface is suitable for use with a component model as described in U.S. Patent Application Ser. No. 60/357,329, filed on even date herewith and entitled “Component Model for Real Time System Control,” which is hereby incorporated by reference as if set forth in its entirety herein. Such a component model may be part of a building automation and control system having multiple servers operable to monitor systems and devices.
It is generally desirable to allow non-programmers to create their own thin-client user interfaces. By “thin client” we mean a user interface program, which can be displayed in an off-the-shelf web browser such as Internet Explorer or Netscape Navigator/Communicator. Since non-programmers build systems, it is important that these screens be easily created using high-level graphical programming tools without knowledge of programming, HTML, JavaScript, etc.
Among others, the types of screens that would be desirable to display in a browser include graphics, alarms, charts, reports, and properties. The graphical display of real-time status of system operation typically includes the display of current device status using pictorial representations. For example if a fan is on, then the graphic would display the fan blades rotating or air blowing. The display and management of system alarms includes the ability to view, query, sort, and filter alarms from multiple systems and perform various levels of acknowledgement. It is beneficial to provide charts such as pie charts, line graphics, and bar charts to visually illustrate any data that has been mined from the system being monitored or controlled. For example if the system is collecting historical data for outside air temperature and runtimes for a piece of equipment, it is valuable to see that historical data presented in one chart. Users should be able to generically query any data found within the system to build reports using text, tabular information, or charts. It is also desirable to be able to view and edit properties of any component in the system generically without the component developer having to manually create a special web user interface.
The solution of the present invention can be used in the Niagara component model as described in U.S. Patent Application Ser. No. 60/357,329, filed on even date herewith and entitled “Component Model for Real Time System Control,” or in any other application. In the preferred embodiment, a Java applet called a Wisp applet is used to create the user interface; however, other compiled applications can be used. A Java applet is a small Java program, which is downloaded to a web browser to fill a piece of an HTML page. In our case the Wisp applet typically fills the entire HTML page.
Referring now to
At step 110, the user operating a given platform references content that is available on a network from a server to which the browser is in communication. Communications between the platform and other devices on the network are not part of the present invention; however, they maybe in any one of a variety of wired or wireless arrangements as is conventional in the art. Content is referenced, for example, by entering a web address in a location field of the browser, by selecting a destination from a pull down list, book mark, etc. The reference comprises the address of a particular file that is available on the network to which the platform is in communication (e.g. “www.tridium.com/main.htm”). Upon referencing a particular web page or other contents, communications are initiated across the network and the requested content is loaded into the browser, as indicated at step 115. Again, the browser can be a conventional HTML browser, or another browser that is optimized to operate on a given platform.
At step 120, an engine included within the loaded content, is executed. In one implementation, the engine comprises a java applet; however, the engine alternatively could be a plug-in, or other code located within the content. More specifically, the engine is preferably a small footprint segment of executable code (e.g., java code) that is configured to automatically reference various definitions that are necessary to implement a given user interface. In this way, the web page itself can be trivial in design and rely upon the engine to fill the screen, and engines provide different user interfaces by referencing different definition files.
As a result of executing the engine, a remote definition file is referenced at step 125, which defines a portion of the presentation of the user interface. The definition file includes the instructions necessary for the engine to build the desired display and is preferably an XML file and governs the lay-out, appearance, border, font color, and other parameters that influence the visual presentation of widgets within the interface, as well as the interaction of the widgets with sources of data that populate each widget's respective data fields with the dynamic real-time data. As well, the definition file can influence the placement and sizing of each widget relative to one another within the user interface.
Widgets are visual objects. They can represent software components or physical devices. They may comprise graphs, pie-charts or tables and they can be animated and provide information in a number of different ways. Widgets also can have the ability to interact with the user.
As an example, the definition file that is referenced at step 125 can be configured to draw into the browser one or more gif images of a fan as well as the code that causes rotation of the gifs to emulate the movement of fan blades on the screen.
Referencing of definition files is performed on the fly, that is, as the engine is executed. In this way, developers are not constrained to interface resources that are resident on the local browser, and users are not required to update their browsers in order to execute a given interface. Rather, the engine draws in passively and automatically the components and code necessary to implement the widgets that comprise a given user interface. These definition files can be sourced from a remote machine connected to the network, or can be resident locally on the platform. As a result of execution of the engine and references to the definition file, at step 130 the definition file is processed which causes any further extensions that may be necessary to implement a given widget to be fetched from a specified location, together with any code that is needed to process real-time data feeds for that widget. The code to implement the widgets is contained in the extensions, whereas the configuration for the widgets is preferably contained in the definition file. For instance, the code does not need to contain instructions as to whether a specific label of a widget is red or black—the definition file specified that.
As can be appreciated, the engine can be relatively small in size and therefore suitable for use on cellular phones and with network configurations that have security settings that prevent downloading large objects. Rather than providing a large native engine within the browser, the engine preferred embodiment is lightweight and able to render a table, chart, etc. in accordance with a given developer's requirements. The individual widgets are not predefined, but can be added by way of extensions that are fetched at step 130 on an as-needed basis. At step 135, the one or more widgets that comprise the interface are displayed on the platform's browser.
Each widget referenced in the definition file can be supplied with a data feed in accordance with its own specific configuration parameters. The configuration parameters define, among other things, the source for the data field, various protocol and handshaking parameters, and the ranges of the data to be supplied and displayed within a given widget. The data feed requirements are periodically sent out to one or more remote sources available on the network and data that is obtained is returned back to the user interface for display in the browser.
Optionally, the data feed requirements of multiple widgets are bundled together in an intelligent manner in order to minimize the number of requests issued out through the network. The bundled request is done in accordance with the configuration parameters for the given data feed (for example, in order to request data from multiple devices at the same interval). While there are advantages to bundling data feed requirements such as reducing network bandwidth usage, the method in accordance with the present invention can be operated without, that is, free of, the bundling step.
Regardless of whether data feed requirements are bundled at step 140, at step 145 a request for data from one or more remote sources is issued from the platform to a remote source. The data request can be marshaled through the engine, or each individual widget can issue its own data request. At step 150, data is obtained in response to the data feed requirements and, as noted above, can optionally be bundled together from a remote source for transmission back to the browser. Thus, as shown at step 155, a response is provided to the data feed request from the engine or a given widget, optionally as a single response. Of course, if the data comes from sources connected to different machines on the network, there would be no need to bundle all requests to a single response; the point here is that a minimized number of network data-feed requests can be issued through computer networks by aggregating those requests that are destined for the same server.
At step 160, responses to various data feed requests are unbundled at the software engine, assuming they were bundled in the first place. The engine notifies each widget on the page that new data is available, and widgets may change their presentation accordingly. Thereafter at step 165, the user interface display is updated. After a prescribed time interval or other interval, as indicated at step 170, a test is made at step 175 to see whether there was any input from the platform to the browser.
The type of inputs that can be provided via the user interface can vary greatly. The user interface can be a passive display of updated information from a remote source free of any input from the user. This may be appropriate in certain applications in which data is pushed to the user, such as applications in which stock prices and other financial data are being provided to a user. However, there are applications in which the user may wish to input data or an action through his browser to cause a change at a remote device. Such input can cause an action to be invoked on a remote server, for example, the purchase or sale of stock. As another example, the user might wish to change configuration parameters for a given widget (say, the range of information being displayed within a graph) or may wish to change the state of an object being monitored (such as the on/off state of a fan). Other inputs can include requests for new data feeds or requests for adding new gadgets to the user interface. The variety of input that can be made at the client's side is not limited to the foregoing examples, which are meant to merely illustrate the types of input that could be provided. If there has been no input at the client's side, then the process loops back to step 145 to request more data from remote sources in order to update the user interface display.
On the other hand, if there was an input at the client's side, the input is obtained at step 180 and is processed at step 185 either by forwarding the input to a remote server for handling, or handling locally if appropriate. Inputs can be from a human user or provided automatically in response to rules or other program that is executing or providing control-information to the particular platform on which the user interface is running.
The process continues as described above to request additional data from one or more remote sources in order to continuously update the display the user interface with real time data feeds.
The visualization system is designed to be compatible with popular software used for displaying third party content over standardized file formats, such as an HTML browser.
Referring now to
Referring now to
There is some executable code associated with each widget, referenced in the definition file. This code is referred to as an extension. Extensions may be located on the user's computer or on another computer connected to the user's computer by a computer network. Extensions can be located by references included in the definition file. The engine parses the definition file and locates the extensions. The engine then loads the extensions, downloading them over a computer network, if necessary (step 104). The engine preferably executes in parallel all the extensions. Once the extensions are executing they display the user interface with the help of the engine. Each extension displays the widget it is associated with. The engine displays the general layout, described in the definition file, and it manipulates the displayed widgets in order to size and place them according to the definition file. The extensions may communicate with other software modules, and may change/update the widgets they are associated with according to these communications. The extensions may further interact with the user, by accepting and responding to commands by the user. The extensions may attempt to send or receive information from other software components based on interaction from the user. The widgets thus continuously update, based on the software code of the extensions, communications with other software modules, and/or interaction with the user. The extension may furthermore send control signals received from the user to other software modules.
An example of a Wisp HTML page:
The entire page can be a single applet deployed by a Java JAR file called wisp.jar. The wisp jar file doesn't actually contain any predefined screens but rather contains a software engine used to construct any screen on the fly. A parameter passed to the Wisp applet specifies a wisp file which contains the description of how to build the screen. This wisp file can be a simple XML file which contains the instructions for how and what to build. A simple example of what a wisp file looks like is as follows:
The major advantage of this approach is that wisp files can be served off the file system or automatically generated using dynamic web technologies such as CGI, ASP, or Java Servlets. Only one small engine is sent to the client, which knows how to generically build any user interface from a wisp XML definition. The engine can be on the order of around 400 kilobytes in size, more or less within a few hundred thousand kilobytes, and in one implementation the engine is 150,000 bytes in size.
Thus, new user interface widgets can be plugged in by writing wisp extensions such as charts, alarm consoles, schedulers, etc.; a page may be partially re-loaded without requiring an entire refresh of the HTML page; and widgets may display a real-time data feed without requiring the page to be refreshed or reloaded. What makes these features possible is the fact that the wisp engine can maintain a persistent or polled connection back to the server. This connection may be used to refresh individual widgets in the page or to asynchronously receive events from the server. Standard HTML technology requires a refresh of the entire page, but since the Wisp engine is an applet it can selectively do these networking calls under the covers.
To take an example, imagine a wisp page, which displays two widgets. One is a graphical representation of a fan and another is an alarm console. The steps to load this page include:
Since the Wisp architecture only requires that a simple XML file be created in order to build new screens it is extremely easy for developers to write CGI, ASP, or servlets to generate the Wisp XML on the fly. Likewise it is also very easy to create graphical programming tools to construct Wisp files without knowledge of the XML syntax.
Referring again to
Alternatively, the visual interface can better ensure efficient network communications by minimizing the number of open communication channels between the interface and one or more remote servers. Because multiple network communication channels operating in parallel can cause performance problems, the requests from individual widgets out to the network can be channeled through and managed by the engine 210. The engine can aggregate network communication requests over predefined periods of time and combine them as appropriate into bundles of larger size for transmission over the network, thus improving network efficiency. As noted in connection with the process of
The visual interface system has been described using languages, formats and protocols, such as HTML, Java, XML, for the purpose of concreteness and clarity. The visual interface system is not limited to these languages, formats and protocols. The visual interface system may be configured to operate with any language that allows embedded applications. Any data file format, including proprietary data formats may be used instead of XML for the definition file.
The present invention has been described in connection with an exemplary embodiment thereof, but is limited only with respect to the recitations in any claims that follow.
This patent application claims the benefit of priority under 35 U.S.C. 119(e) from U.S. Provisional Patent Application Ser. No. 60/357,329, filed Feb. 15, 2002, entitled “COMPONENT MODEL FOR REAL-TIME SYSTEM CONTROL”, which is hereby incorporated by reference as if set forth in its entirety herein.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/US03/04804 | 2/18/2003 | WO | 00 | 2/2/2004 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO03/071442 | 8/28/2003 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4435764 | El-Gohary | Mar 1984 | A |
| 4635195 | Jeppesen, III et al. | Jan 1987 | A |
| 4677566 | Whittaker et al. | Jun 1987 | A |
| 4747041 | Engel et al. | May 1988 | A |
| 4935863 | Calvas et al. | Jun 1990 | A |
| 4941084 | Terada et al. | Jul 1990 | A |
| 5175852 | Johnson et al. | Dec 1992 | A |
| 5511188 | Pascucci et al. | Apr 1996 | A |
| 5522044 | Pascucci et al. | May 1996 | A |
| 5550980 | Pascucci et al. | Aug 1996 | A |
| 5598566 | Pascucci et al. | Jan 1997 | A |
| 5608870 | Valiant | Mar 1997 | A |
| 5786993 | Frutiger et al. | Jul 1998 | A |
| 5862052 | Nixon et al. | Jan 1999 | A |
| 5916306 | Ruiz | Jun 1999 | A |
| 5918233 | La Chance et al. | Jun 1999 | A |
| 5982362 | Crater et al. | Nov 1999 | A |
| 6012071 | Krishna et al. | Jan 2000 | A |
| 6028988 | Asaka et al. | Feb 2000 | A |
| 6028998 | Gloudeman et al. | Feb 2000 | A |
| 6047222 | Burns et al. | Apr 2000 | A |
| 6055562 | Devarakonda et al. | Apr 2000 | A |
| 6067477 | Wewalaarachchi et al. | May 2000 | A |
| 6119125 | Gloudeman et al. | Sep 2000 | A |
| 6157943 | Meyer | Dec 2000 | A |
| 6185566 | Adams et al. | Feb 2001 | B1 |
| 6189109 | Sheikh et al. | Feb 2001 | B1 |
| 6353853 | Gravin | Mar 2002 | B1 |
| 6356920 | Vandersluis | Mar 2002 | B1 |
| 6370448 | Eryurek | Apr 2002 | B1 |
| 6429880 | Marcos et al. | Aug 2002 | B2 |
| 6591305 | Densmore | Jul 2003 | B2 |
| 6636885 | Martin | Oct 2003 | B1 |
| 6681243 | Putzolu et al. | Jan 2004 | B1 |
| 6728960 | Loomans | Apr 2004 | B1 |
| 6754691 | Gomi et al. | Jun 2004 | B1 |
| 6842906 | Bowman-Amuah | Jan 2005 | B1 |
| 6868441 | Greene et al. | Mar 2005 | B2 |
| 6886158 | Kitayama et al. | Apr 2005 | B2 |
| 7010503 | Oliver et al. | Mar 2006 | B1 |
| 7039624 | Merk et al. | May 2006 | B1 |
| 7080159 | Chu et al. | Jul 2006 | B2 |
| 20020059583 | Kim | May 2002 | A1 |
| 20020080945 | Rojas | Jun 2002 | A1 |
| 20020114276 | Basturk | Aug 2002 | A1 |
| 20030221162 | Sridhar | Nov 2003 | A1 |
| Number | Date | Country |
|---|---|---|
| 2812501 | Feb 2001 | FR |
| WO-0190884 | Nov 2001 | WO |
| WO-03071444 | Aug 2003 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20040215740 A1 | Oct 2004 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60357329 | Feb 2002 | US |
