The present invention relates to user interfaces for networked devise, and particular to user interfaces for navigation of network devices using World Wide Web protocols.
A network generally includes a communication link and various devices with communication capability connected to the communication link. The devices include computers, peripheral devices, routers, storage devices, consumer electronics and appliances with processors and communication interfaces. An example of a network is a home network for a household in which various devices are interconnected. A usual household can contain several devices including personal computers and home devices such as consumer electronics and appliances that are typically found in the home. As such the term “device” generally includes logical devices or other units having functionality and an ability to exchange data, and can include not only all home devices but also general purpose computers. Home devices include such electronic devices as security systems, theater equipment, consumer electronics (e.g., TVS, VCRs, DVD players, stereo equipment, direct broadcast satellite services (DBSS), digital satellite services (DSS), etc.), sprinkler systems, lighting systems, appliances (e.g., microwave, dish washer, ovens/stoves, washers/dryers), a processing system in an automobile, etc.
Consumer electronics, such as home theater equipment, are often controlled using a controller device (e.g., remote control device). However, a drawback associated with using such a controller device to control home devices is that each particular controller device must be specifically programmed to control and command those home devices for which it is intended. This requires the user to program, or to load software into, the controller device to control various devices.
There is, therefore, a need for a method and system that provides a user interface in controller devices for controlling other devices, which reduces the need for a user to program the controller devices.
The present invention addresses the above needs. In one embodiment the present invention provides a method for providing a user interface in a network including interconnected client and service devices, the user interface for controlling the service devices that are currently connected to the network, comprising the steps of: obtaining information from one or more of said service devices currently connected to the network, said information including device information; generating a user interface including status information of said service devices based on said device information; displaying the user interface on a client device capable of displaying a user interface, for user navigation and control of said service devices; tracking user navigation and control of said service devices; and based on the tracking information, providing the user with default service device selection on the user interface.
The step of providing the user with default service device selection can further be based on the service device function and based on one of prior service device selections by the user. The service devices can include source devices and sink devices, such that the step of providing the user with a default sink device selection for a source device can further be based on one of prior sink device selections by the user. Further, the step of providing the user with a default source device selection for a sink device can further be based on one of prior source device selections by the user.
Displaying the user interface on said client device can further include the steps of displaying the user interface on a browser in the client device. The user interface can further display device control information as a service device is selected by the user. The device information in each service device includes a user interface description for user interaction with that device.
In another embodiment the present invention provides a controller in the client device in a network that provides the user interface according to the above steps. Yet, in another embodiment the present provides a network of interconnected client and service devices that implement the above steps.
As such, in one example implementation the present invention provides a method and system of browser-based command and control for devices in a network comprising client devices (e.g., controller devices) and service devices (e.g., controlled devices). The client devices allow the user to control the server devices. To reduce user programming of client device to control the server devices, a web browser is included in each client device and a web server is included in each service device that provides a service to be controlled by a client device. The user's navigational context is tracked as the user navigates around from device to device in the network. This provides information for the context of the web server for each service device, to reduce the need for the user to program that information into the client device.
Other embodiments, features and advantages of the present invention will be apparent from the following specification taken in conjunction with the following drawings.
In one embodiment, the present invention provides a method and system of browser-based command and control for devices in a network comprising client devices (e.g., controller devices) and service devices (e.g., controlled devices). The client devices allow the user to control the server devices. To reduce user programming of client device to control the server devices, a web browser is included in each client device and a web server is included in each service device that provides a service to be controlled by a client device. The user's navigational context is tracked as the user navigates around from device to device in the network. This provides information for the context of the web server for each service device, to reduce the need for the user to program that information into the client device.
Conventionally, connecting across a network with web browsers and web servers without programmatic control therebetween, when a service device is visited by the user via the browser of a client device, the context of the user's visit is not utilized for predicting the user's future preferences. For example, if the network includes a tuner (source device) and two televisions (sink devices), when the user selects a station to play from the tuner, though the tuner becomes a source for video and audio, a sink device is not automatically selected for the user. This requires the user to select one of the sink devices as the sink device for the selected source device. Even if the user has previously visited/selected one of the sink devices, conventional system require the user to select a sink device using the browser of the client device, each time the user selects a source device.
In one embodiment, the present invention keeps track of the user's navigational context and selections, as the user navigates from service device to service device in the network. The information about the context of the web server for each service device visited by the user is provided to the browser of the client device that the user is utilizing to control the service devices. This virtually eliminating the need for the user to provide that information to the client device for controlling the service devices.
In one example, when the network is first powered on, a discovery process is performed to determine the connected physical devices and logical units/devices. Then, the representations for the discovered devices appear on the web browser of a client device to receive user control commands, an initial default context for the networked devices is shown to the user. As the user navigates to, and selects, a sink device using the web browser of the client device, the client device keeps track of that last sink device that the user selected (the controller records the name of the last user selected sink device). Then, when the user selects a source device, the client device provides the user's prior selection of the sink device to that source device, without asking the user to specify a sink device for that source device. As such, the source device connects, and sends information, to the sink device specified by the last user-visited link in the browser of the client device. The last visited sink device becomes the default sink device for the source devices selected thereafter, until the user changes that by navigating to another sink device. The user navigation context is tracked, maintained, in one example, by using global variables in a Javascript to retain the name and associated identifiers and properties of the last device that the user selected.
An example implementation of a method and system according to the present invention is now described in the context of a home network implementing the 1394 protocol. However, as those skilled in the art will recognize, the present invention is useful with other types of networks using different network protocols. Further, thought the example herein utilizes web browser client server methodology (extended to TCP/IP networks in general), those skilled in the art will recognize that other client server protocols may also be used. A Web browser (browser) is used in this description as an example graphical presentation engine network resource, and other graphical presentation engines can also be used.
Further, the following abbreviations are used in this description: ATSC—Advanced Television Systems Committee; AV—Audio/video; AV/C—Audio/video control; DHCP—Dynamic host configuration protocol; DNS—Domain name server; DTV—Digital Television; HDTV—High Definition Television; GIF—Graphics interchange format; GUI—Graphical user interface; HNCP—Home network configuration protocol; XML—Extensible Markup Language; XHTML—XML compliant Hypertext markup language; HTTP—Hypertext transfer protocol; OSD—Onscreen display; and STB—Set top box.
A user interface system according to the example embodiment of the present invention allows a service device to utilize the presentation capabilities in a network-attached client device that includes a renderer, to present status and control interface of the service device to a user. The service device can comprise, e.g., a source of home-network content services such as a cable or terrestrial set-top box, digital VCR, DTV, etc. Further, the renderer can comprise a device that has access to a display for presenting the control interface of the source to the user in a graphical user interface (GUI) such as the aforementioned Web browser. As such, a Web browser is an instance of a GUI, and can include capability for World Wide Web navigation, E-Commerce or Enhanced TV applications, etc. For simplicity of explanation, in this description the terms GUI, Web browser and browser are used interchangeably.
An encapsulation mechanism using Web and Internet protocols is utilized to enable user control of a device via the GUI in another device. As such, in one case, a service device passes its control interface data to the renderer for presentation to the user on the GUI of the renderer, whereby the user can control that service device. A top level controller server that orchestrates a top level GUI is defined, while lower levels are solely presented by the renderer.
The HDTV 12 includes a Web browser 26 for presentation and three logical units (L-Unit) as follows: (1) a controller (L-Unit Controller) 28 that includes a GUI controller to dispatch GUI control, (2) a TV tuner and EPG/PSIP server (L-Unit TV(tuner)LR) 30, and (3) a TV decoder (L-Unit TV(picture)LR) 32 for picture and audio. The AVHDD 14 included one logical unit (L-Unit AVHDD-LR) 33 with a recorder and a content player function. The HDTV 12 further includes three Web servers 34, 36 and 38, corresponding to the service functions of said three logical units therein, wherein the Web servers can be incarnations of a single Web server in the HDTV). Further, the AVHDD 14 includes a Web server 40 for its service function.
A logical unit may only be a sub-unit of a physical device, and presents a 5 separately controllable set of functions (e.g., a logical unit refers to a virtual subunit). In this example, there are five types of logical units: (1) media sink, (2) media source, (3) media sink and source, and (4) neither media sink nor media source, and (5) not visible. For example, a website is a type 4 device. A controller device (i.e., client device) is generally paired with a Web browser and ignores all other controller devices. A controller logical unit is not visible and is a type 5 device. The tuner in the HDTV is a type 2 device. A display device is a type 1 device. The AVHDD logical unit is a type 3 device.
Based on the five device types above, the following apply to presenting a GUI for home network (HN) control for the example embodiment described herein.
When the network 10 is first powered on, a discovery process is performed to determine the connected physical devices 12, 14 and logical devices (logical units) 28, 30, 32 and 33. Then, the representations for the discovered devices appear on the web browser 26 of the HDTV 12 to receive user control commands, where an initial default context for the networked devices is shown to the user. As the user navigates to, and selects, a sink device such at the AVHDD 14 using the web browser 26, a GUI controller 42 in the HDTV 12 keeps track of that last sink device (e.g., AVHDD 14) that the user selected. Then when the user selects a source device (e.g., L-Unit TV(tuner) LR 30), the GUI controller 42 provides the user's prior selection of the sink device to that source device, without asking the user to specify a sink device for that source device. As such, the source device connects, and sends information, to the sink device specified by the last user-visited link in the browser of the client device. The last visited sink device (e.g., AVHDD 14) becomes the default sink device for other source devices selected thereafter, until the user changes that by navigating to another sink device (e.g., L-Unit TV (picture)LR 32).
Another example network 50 implementing a user interface system according to another embodiment of the present invention is shown as a functional block diagram in
When a device is connected to the network 50, its status frame 62 is added to the control page 60. When a device is disconnected, its status frame 62 is removed from the control page 60. During the discovery process, the control page 60 is displayed automatically for a specified duration until exited by the user. A device status presentation can change using e.g. a color highlight scheme to indicate that the device is selectable. In one example, when the device icon is color highlighted (e.g., green), and its status frame 62 on the control page 60 is selected by the user, that device obtain system focus. In the drawings, a highlighted status frame is shown with a dark inner border and pointed by an arrow with the reference 62.
A control frame area (Control Area) 64 in the control page 60 is associated with the device that has the system focus, and the control frame for that device is displayed in the control frame area 64 on the control page 60. In the example of
The following are example commands issued by the GUI controller 42 to a connected device:
An example network according to an embodiment of the present invention includes the following example configuration:
In one example, all controllers 28 have a built-in list of networked devices base URL prefix, wherein the URL prefix identifies the HTTP server within the device. An HDTV may have a tuner server and a decoder server, but has only one IP address. In the example embodiment describe herein, the networked device performs discovery of all other connected devices and the system resolves any IP address conflicts.
An example operation scenario when the network is initially turned on is follows:
Many other example operations of the network are possible.
An example navigation and control process from the HDTV 12 in ), RIGHT (
), and SELECT (←), as shown in
In one case, when the HDTV 12 is powered up the control page 12 is as shown in
Further, from the screen 60 in
If when the TV-pic-LR 32 is selected, the device is in the power off state, then a power on command is sent to the device 32. As the tuner 30 is built into the HDTV 12, the tuner 30 also changes state to powered on. The tuner 30 then tunes the last tuned channel that it tuned just before it was powered off, and the TV-pic-LR device 32 decodes and displays that transport stream from the tuner 30. As such, the TV-pic-LR device 32 goes from a black screen to a picture as the icon in
From the network state shown in
If the status of the network is as shown in
According to another aspect of a user interface system according to an embodiment of the present invention, a “target” symbol 72 indicates corresponds to a device that is in focus and will be the target of any subsequent e.g. source device selection until another target device is selected. In the example in
Whenever the target symbol 72 is used, it is placed by the GUI controller 42 on the control page 60 proximate the status frame of the device that the GUI controller 42 is indicating as the target. In the case of a video presentation device which has the target symbol 72, and has only one main video display screen (as in the case of most all HDTVs today), the display will show each subsequent source selection as the sources are selected.
In general, no change occurs to the service connection of any sink device that is not the target. There is only one target symbol on the control page 60. When a new device frame obtains the target symbol 72, then the target symbol 72 disappears from its previous position on the screen 60. As with the highlights on the control page 60 that are determined by the GUI controller 42, the target symbol 72 is a GUI controller relative indicator. Each GUI controller 42 operates independently with respect to highlighting and target symbol 72 placement on its corresponding control page 60.
In the following, an example of target symbol placement by the controller 42 as a function of user navigation from one sink device to another sink device is explained. Generally, when a user selects an already highlighted status frame 62 of a sink device, the GUI controller 42 moves the target symbol 72 to the status frame 62 of that sink device. This is not always the case for devices that are ambiguous as to being a sink or a source device (e.g., the AVHDD and the DVHS devices are such devices). When the record function (REC) is pending for one of such ambiguous devices, their status frames receive the target symbol 72 until the source for the recording has been selected and then the target symbol 72 is moved to its previous position. To achieve such a function, the GUI controller 42 provides the top level frameset to its web browser 26. The CGI task in the web server 26 is aware that each device is either: (1) Source-only, (2) Sink-only, (3) Neither Sink nor Source, or (4) Either Sink or Source (“Either-type” ambiguous devices). This information is received at the controller server CGI. When the control frame of the ambiguous device is displayed in the control area 64 and the highlight on its status frame 62 changes back from in-focus highlight, the CGI task queries the device to determine whether an open record command is pending.
When a user moves the highlight around the status frames and presses Select on a device that is a Sink-only device (e.g., the TV-pic-BR), then the device receives the target symbol 72 on its status frame 62 and subsequent source device selections are connected to that device.
The cases for the AVHDD and DVHS may be different because these devices can be a source and/or a sink at any one time. The AVHDD and DVHS are “Either-type” devices and have a special behavior. An HDTV would be such a device if the Screen and Audio Output controls were not separated from the tuner as separate logical status and control units with two logical HTTP servers. In order for such a device to operate, it must clearly identify its status for all Exits in its control interface. For example, if there is a Record function that has been requested by the user, then the GUI controller 42 that controls that device must distinguish the situation that requires an immediate source selection. The GUI controller 42 can then assign the recording device as the immediate target and direct the user to navigate to only source capable devices. This would not be necessary if the AVHDD had two separate servers one for recording and one for playing.
An example of controlling the “Either-type” (ambiguous) devices is now explained. For example, if the network status is as in
When the status frame of a device is highlighted or the device is in focus with one of its control frames displayed on the control page 60, then some of the keys on the remote control 24 may be activated. Examples of such keys (darkened in
As noted above, for effective event notification to the user, in another aspect the present provides a Push method that allows a connected device to send status and notification messages directly to the user, independent of the version of HTTP supported, and for all MIME types. An example Push method for HTTP uses a subframe, which is a one pixel iframe without borders (not visible) that is embedded within the device status frame 62, and initially posted by the web server of the device (e.g., web server 40,
While this invention is susceptible of embodiments in many different forms, there are shown in the drawings and will herein be described in detail, preferred embodiments of the invention with the understanding that the present disclosure is to be considered as an exemplification of the principles of the invention and is not intended to limit the broad aspects of the invention to the embodiments illustrated. The aforementioned example architectures, according to the present invention, can be implemented in many ways, such as program instructions for execution by a processor, as logic circuits, as ASIC, as firmware, etc., as is known to those skilled in the art. Therefore, the present invention is not limited to the example embodiments described herein.
The present invention has been described in considerable detail with reference to certain preferred versions thereof; however, other versions are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred versions contained herein.
Priority is claimed from U.S. Provisional Application No. 60/530,501, filed on Dec. 18, 2003, which is incorporated herein by reference.
Number | Date | Country | |
---|---|---|---|
60530501 | Dec 2003 | US |