Patent Application
20080288618
The present invention relates to control systems in general, and more particularly to control systems for networked devices.
Systems that control multiple devices and appliances, such as home automation systems, typically provide limited interoperability between devices and appliances from different manufacturers and require custom programming for their integration into the control system. Such system often employ proprietary protocols rather than industry standard protocols and require physical in-wall wiring to connect devices and appliances to a central processor. This makes an installation in existing environments costly and difficult.
The present invention provides an architecture for controlling multiple devices and appliances in an environment, such as the home, that allows for rapid installation, easy setup, and maintenance. The present invention employs industry-standard protocols, such as TCP/IP and UPnP™, offers wireless connectivity, and a distributed system for scalability, employing multiple controllers that may each function independently. The present invention also provides for integration and bridging between network-ready devices and non-network ready devices, such as where non-UPnP™ appliances are integrated into a UPnP™-based home network.
In one aspect of the present invention a networked device control system is provided including a plurality of networked device controllers operative to implement a protocol of automatic device discovery and control, at least one non-protocol-compliant device connected to any of the controllers and not configured for use with the protocol prior to being connected to the controller, and a management unit operative to generate any of an interface and a control element associated with any of the devices, establish a proxy configured for use with the protocol and operative to control the non-protocol-compliant device, and configure any of the controllers with the interface and control element generated for the device connected to the controller.
In another aspect of the present invention the management unit is operative to configure any of the controllers to which the non-protocol-compliant device is attached to act as the proxy.
In another aspect of the present invention the proxy is operative to translate a protocol-compliant command into a command for controlling the non-protocol-compliant device, and send the translated command to the non-protocol-compliant device.
In another aspect of the present invention the protocol is the UPnP™ protocol.
In another aspect of the present invention a networked device control system is provided including a plurality of networked device controllers operative to implement a protocol of automatic device discovery and control, at least one protocol-compliant device connected to any of the controllers and configured for use with the protocol prior to being connected to the controller, at least one non-protocol-compliant device connected to any of the controllers and not configured for use with the protocol prior to being connected to the controller, and a management unit operative to generate any of an interface and a control element associated with any of the devices, establish a proxy configured for use with the protocol and operative to control the non-protocol-compliant device, and configure any of the controllers with the interface and control element generated for the device connected to the controller.
In another aspect of the present invention the management unit is operative to configure any of the controllers to which the non-protocol-compliant device is attached to act as the proxy.
In another aspect of the present invention the proxy is operative to translate a protocol-compliant command into a command for controlling the non-protocol-compliant device, and send the translated command to the non-protocol-compliant device.
In another aspect of the present invention the protocol is the UPnP™ protocol.
In another aspect of the present invention a method is provided for networked device control, the method including deploying a plurality of networked device controllers operative to implement a protocol of automatic device discovery and control, connecting at least one non-protocol-compliant device to any of the controllers and not configured for use with the protocol prior to being connected to the controller, generating any of an interface and a control element associated with any of the devices, establishing a proxy configured for use with the protocol and operative to control the non-protocol-compliant device, and configuring any of the controllers with the interface and control element generated for the device connected to the controller.
In another aspect of the present invention the method further includes configuring any of the controllers to which the non-protocol-compliant device is attached to act as the proxy.
In another aspect of the present invention the generating step includes defining a non-protocol-compliant device type, including a command set, a communication protocol, and an interface, and generating the proxy from the definition.
In another aspect of the present invention the method further includes translating a protocol-compliant command into a command for controlling the non-protocol-compliant device, and sending the translated command to the non-protocol-compliant device.
In another aspect of the present invention a method is provided for networked device control, the method including deploying a plurality of networked device controllers operative to implement a protocol of automatic device discovery and control, connecting at least one protocol-compliant device to any of the controllers and configured for use with the protocol prior to being connected to the controller, connecting at least one non-protocol-compliant device to any of the controllers and not configured for use with the protocol prior to being connected to the controller, generating any of an interface and a control element associated with any of the devices, establishing a proxy configured for use with the protocol and operative to control the non-protocol-compliant device, and configuring any of the controllers with the interface and control element generated for the device connected to the controller.
In another aspect of the present invention the method further includes configuring any of the controllers to which the non-protocol-compliant device is attached to act as the proxy.
In another aspect of the present invention the generating step includes defining a non-protocol-compliant device type, including a command set, a communication protocol, and an interface, and generating the proxy from the definition.
In another aspect of the present invention the method further includes translating a protocol-compliant command into a command for controlling the non-protocol-compliant device, and sending the translated command to the non-protocol-compliant device.
In another aspect of the present invention a method is provided for communicating UPnP™ commands to a non-UPnP™-compliant device, the method including converting a control specification of a non-UPnP™-compliant device into a mapping between at least one non-UPnP™ command and at least one UPnP™ command, creating an instance of a UPnP™ device to receive UPnP™ commands and output a corresponding command to the non-UPnP™-compliant device, looking up a UPnP™ command in the mapping, and sending a corresponding command to the non-UPnP™-compliant device.
In another aspect of the present invention the control specification is that of any of a serial, IR, relay, I/O, or USB device.
In another aspect of the present invention the converting step includes converting into an xml-based format.
In another aspect of the present invention the method further includes receiving a command from the non-UPnP™-compliant device, looking up a UPnP™ command in the mapping corresponding to the received command, and sending the UPnP™ command to a UPnP™ controller.
In another aspect of the present invention the creating step includes creating the UPnP™ device for a subsystem to which multiple sub-appliances are connected, the UPnP™ device having one UPnP™ service for each sub-appliance type, and each of the UPnP™ services having separate references for each of the sub-appliances.
In another aspect of the present invention the method further includes translating a UPnP™ command into a command, sending the command to the subsystem together with an identifier of any of the subappliances to which the command is directed.
In another aspect of the present invention the method further includes automatically assigning an interface element with any of the commands, and upon the interface element being activated, performing the lookup and sending steps with respect to the command associated with the interface element.
It is appreciated throughout the specification and claims that references to UPnP™ may refer to the UPnP™ protocol or any other protocol that provides for discovering, communicating with, and controlling devices and appliances within a computer network environment.
It is appreciated throughout the specification and claims that references to “devices” and “appliances” without further qualification are interchangeable and refer to electrical devices, whereas references to “UPnP™ devices” refer to a UPnP™ protocol term.
The present invention will be understood and appreciated more fully from the following detailed description, taken in conjunction with the drawings in which:
Reference is now made to
UPnP™ or similar protocols such as Rendezvous™ are preferably used to provide interoperability between electronic devices, including home appliances, and computing devices in a network environment. There are five basic steps involved in UPnP™ Networking: addressing, discovery, description, control and eventing. These steps allow a device to dynamically connect to a local IP-based network, advertise its functionality or be discovered by other elements (i.e., control points) on the network. After a control point has become aware of a device's capabilities and obtained the relevant access permission, it may control the device by sending an action request to the device, which may respond with an appropriate event message containing the names of one or more state variables and the current value of those variables. Implementation and other aspects of the architecture of
A system based on the architecture of
Reference is now made to
Reference is now made to
Management devices 218 and 242 may be implemented as UPnP™ devices for the device controllers (i.e., control box, pc). Its main purpose is to function as the gateway to the device controller, providing information services about the device controller capabilities (e.g., number of ports, available disk space, etc.) and providing management capabilities (e.g., uploading configuration files, interface screens, etc.) Additionally, the management devices may incorporate utility services, providing access to the specific device controller's hardware and software environment (e.g., adjust brightness on touch panel, switch alpha-blending on control box, etc.)
In the example shown, control boxes 200 and 220 provide physical connections to home appliances and host software which manage the use of the appliances. Control boxes 200 and 220 have physical interfaces to various types of connectors found in target devices, an function as a bridge between a UPnP™ compliant network and non-UPnP™ compliant, legacy systems and appliances. Control boxes 200 and 220 may be implemented using the following commercially-available hardware components:
Control boxes 200 and 220 may include any of the following connectors:
Reference is now made to
It will be appreciated that the steps of the method of
Reference is now made to
Reference is now made to
Reference is now made to
Referring now to
Each control box is preferably controlled by a control operating system (Control OS), being cross-platform, portable system software that, analogous to an operating system, acts as a broker between the hardware and the software applications and services. The Control OS preferably includes the following features:
Referring again to
Media server 244 is preferably implemented as a UPnP™ device that provides audio/video content (i.e., media) to other UPnP™ devices on the network. It is based on the UPnP™ AV Architecture Framework and exposes its content via the Content Directory service. As such, the media server 244 can preferably handle any specific type of media, data format, and transfer protocol, and expose a wide-variety of content such as MPEG-1, -2 and -4 video, CD audio, MP3 and/or WMA audio, JPEG images, and other media to the network in a uniform and consistent manner.
Users interact with the system via an interface, such as is provided by TV interface 210, touch panel interface 232, PDA interface 260, an PC interface 240. The user interface allows users to interact with the system, preferably via a web browser which run on control boxes 200 and 220 and/or on the interface devices themselves. The interface resources may be stored on any of the hardware elements and are served by web server 214.
Automation manager 252 creates and manages scheduled automation project tasks. Automation server 212 may be hosted by a control box, connected PC, or touch panel, and may be implemented as a service/daemon. Scheduled automation tasks may include commands, such as Microsoft Windows™ commands, scripts, UPnP™ commands, as well as other tasks. A scheduled task may also be associated with a date/time event.
Automation manager 252 is a visual tool designed to create and manage all aspects of the automation project. Automation manager 252 is fully integrated with management software 248 and preferably functions as the main automation management interface. Alternatively, a limited automation manager may be provided via the user interfaces (i.e., TV, touch panel, PDA, etc.).
UPnP™ network 228 typically includes one or more UPnP™ devices, being a container of services and nested devices. For example, a VCR device may include a tape transport service, a tuner service, and a clock service. A TV/VCR combo device would include not just services, but nested devices as well. Different categories of UPnP™ devices may be associated with different sets of services and embedded devices. For example, services within a VCR will be different than those within a printer. The set of services that a particular device type may provide is typically captured in an XML-based device description document that the device UPnP™ hosts. In addition to the set of services, the device description also lists properties associated with the device, such as the device name and associated icons.
Being the smallest unit of control in a UPnP™ network, a service exposes actions and models its state with state variables. For example, a clock service may be modeled as having a state variable, current_time, which defines the state of the clock, and two actions, set_time and get_time, through which the service may be controlled. Similarly to the device description, this information is typically part of an XML-based service description. A pointer to these service descriptions, such as a URL, is contained within the device description document.
A service in a UPnP™ device is typically associated with a state table, a control server and an event server. The state table models the state of the service through state variables and updates them when the state changes. The control server receives action requests, such as set_time, executes them, updates the state table, and returns responses. The event server publishes events to interested subscribers anytime the state of the service changes. For example, a fire alarm service may send an event to interested subscribers when its state changes to “ringing.”
A control point in a UPnP™ network, such as control point 246, is a controller capable of discovering and controlling other devices. After discovery, a control point may:
Exemplary implementations of elements of the present invention are now described. While specific hardware components may be referred to by manufacturer and model, such references are provided for illustration purposes only, and it will be appreciated that the present invention is not limited to the specific hardware components mentioned.
Reference is now made to
Reference is now made to
In
The control box of
An RS-232 Console driver is preferably provided to provide recognition indications and interrupts are supported by the interface.
The RS-232 port is preferably supported by the serial port driver and provides standard data transfer rates. There are two RS232 serial ports that are fully compatible with the RS232 protocol.
The RS-485 port is preferably supported by the serial port driver and is fully compatible with the RS485 protocol. Its functionality as RS-232 can be set at boot time.
The RS422 port is preferably supported by the serial port driver and is fully compatible with the RS422 protocol. Its functionality as RS-232 can be set at boot time.
The operating system preferably interfaces with the device via the ISA bus. The generated interrupt signal may be shared by all serial ports and preferably causes a voltage transition from low to high which pulls down the INT_UART signal generating the interrupt. The UART interrupt may be shared with the PCI slot interrupt and the operating system determines which device generated the interrupt. The operating system may access the entire device internal register.
There are preferably four additional controlling signals, nCSA-nCSD, where each signal acts as the chip select for the serial interfaces. The operating system preferably generates the CS signal and a Reset signal for the serial interface device.
Additionally, two selectors generated from GPIO are the serial PHY selector that set the two line standard RS-232/RS422 and RS-232/RS-485. The settings are preferably determined via software.
The circuit attached to the physical serial connector is one of the PHY's with the selection line. A TL16C554A, commercially-available from Texas Instruments Inc. may be connected from top to bottom as shown in
The USB device of the control box of
The cardbus interface of the control box of
An optional PCI slot may be provided with the control box of
An X10 device may be provided with the control box of
The control box of
The control box of
The control box of
The control box of
The EM8621L can decode multiple simultaneous MPEG programs, based on source format and resolution, including:
When decoding multiple MPEG programs, each program may be treated differently. One may be played normally, a second program might be used for picture-in-picture and a third program might be output to a second TV or VCR.
The EM8621L hardware and accompanying software support many popular MPEG-based video and audio media formats. The device supports DVD-Video, Superbit DVD, VCD 1.x and 2.0, SVCD, DVD-Audio, CD/CD-R/CD-RW (audio, JPEG, MP3 and MPEG-4 AVI files). The EM8606L includes hardware CSS decryption and supports DVD-Video CSS procedural specifications. It also fully supports DVD-Video control features such as 16:9 and 4:3 aspect ratios, letterboxing, pan and scan, multiple angles, 3:2 pulldown, up to 8 language sound tracks, and 32 subtitle settings.
The EM8621L includes a DSP-based audio decoder. The decoder can support the following audio formats:
Audio services required for digital TV applications are also supported. The audio decoder uses three I2S digital audio output interfaces for 5.1 channel support, and a S/PDIF serial output.
The EM8621L device is connected to the PCI Bus as the fourth device and all communication is done thru this channel. An additional device for the video input signal interfaces the EM8621L via the digital video port and is controlled by I2C bus provided by the EM8621L processor. The interrupt generated by the EM8621L is shared with the X10 and Input IR.
The driver supports all features provided by the EM8621L processor and provides an X-server on the graphic device via the I2C bus to the video-in device. The driver is written to work with the Sigma Mono Media Player which handles all audio, video, and image media streaming.
The control box of
The control box of
The control box of
The control box of
The control box of
The control box of
The control box of
The control box of
The control box of
The control box of
The EM8621L includes hardware CSS decryption and supports DVD-Video CSS procedural specifications. It also fully supports DVD-Video control features such as 16:9 and 4:3 aspect ratios, up to 8 language sound tracks, 32 subtitle settings, letterboxing, pan and scan, multiple angles and 3:2 pulldown.
The control box of
Reference is now made to
A Fast Ethernet 10/100 Mb is preferably supported with a standard RJ-45 connector type. The transceiver is preferably provided on the internal PCI bus provided by the CPU.
The mini control box is preferably able to interact via wireless LAN (802.11b, 802.11b and 802.11g). The design supports Mini PCI standard in order to support third party W-LAN on Mini PCI. The antenna preferably supports 2.4 G and 5.8 G bands.
The mini control box bus preferably controls standalone applications and allows the connection of expansion devices. The mini control box preferably includes a transceiver on the internal PCI bus provided by the CPU, and drives 15 W of power toward external devices. The connector may be a six wire type needed to supply the power driving the driver for the Firewire itself.
The mini control box preferably supports 4 Tx IR transmitters and each driver is able to control one device. The signaling is derived from the RS-232 DTR signal. Only one IR device may be active at any given time. The connectors are of the microphone type.
The mini control box preferably supports one serial port based on the RS-232 protocol (UART) with an RJ-45 standard connector. The serial port functions as a terminal or user UART interface according to an on-board jumper selection.
Two dry contact relays capable of driving 1 Ampere may be provided.
The mini control box is preferably capable of receiving signaling from external devices with single ended wires. The ground connection is shared for all four inputs.
The mini control box may have a rated voltage of 0-24 VDC, an input impedance of no less then 20 KOhm, and a logic threshold of 1.25V.
The mini control box preferably supports Audio In/Out signaling. In addition to the microphone connector, an optional header may be provided to enable the connection of a board microphone device. The audio/microphone connection may be selective, resulting in the disconnection of the microphone where an external microphone is connected.
The mini control box front panel preferably includes the following items:
In addition to the IR receiver, an optional header may be provided to allow the connection of an off-board IR device.
The mini control box back panel preferably includes the following items:
In addition to the microphone connector, an optional header may be provided to enable the connection of a board microphone device. The audio/microphone connection is selective resulting in the disconnection of the microphone in case an external microphone is connected.
Reference is now made to
The touch controller includes the general mini control box design, with an additional LCD controller and LCD display. Preferably, the touch controller uses CSTC LCD or Active TFT LCD displays with a resolution of up to 800×600 (SVGA).
The LCD display preferably includes a touch panel cover which is connected to the LCD board. The touch panel controller communicates with the LCD controller device using any suitable means.
The touch controller preferably includes an AC/DC module built into the wall plug chassis enabling it to connect directly to a 85-264V AC power line. The power module is preferably a switching power supply for a standard power line. The output is DC 5V up to 3 A with standard DC plug. Special power wiring is thus not required to power the touch controller, although the power module may be connected to a power source, such as DC 5V. This might be preferable in environments where a 2-wire DC line is already available and/or where a 100-220V AC power line is not available nearby.
Exemplary methods of operation are now described of aspects of the architecture and systems described hereinabove. For illustration purposes only, such methods are described for a home automation project, although it is appreciated that the present invention is equally applicable in other environments as well.
Reference is now made to
Creating an automation project typically involves with taking the basic layout of an environment, mapping the environment into one or more zones, adding control boxes, and attaching devices to the control boxes.
In
One or more devices, such as a non-UPnP™ appliance, can be connected to the project by clicking on the project explorer area on a specific control box or from the menu or wizard and selecting “Add Device”. The term “appliance” is now used to describe any kind of electrical device or appliance. This process will create a GUI which will provide access to the functionality of the added device (e.g. Play a CD). Preferably, every time a device is added to a control box, the zone containing the control box will be automatically regenerated to provide access to the added device.
Selecting the “Add Device” menu item as shown in
Once the type of device is established a database window preferably opens as shown in
After providing all requested data, the wizard will have all the required information. The added device will then appear in the project tree, the proper template files will be located, and the application will retrieve the required device command file and generate the Interface screens.
An “Automatic GUI Generation for Multiple Target Devices” feature is preferably provided to provide a GUI for appliances, which are part of the home automation project as well as project-wide GUI screens that provide access to control boxes and zones throughout the home. The GUI is updated and/or created automatically for each display type available to the home automation project when a zone, control box, or device is added or modified. The user may choose to regenerate any of the existing interfaces as desired to base the interface on a modified template or even a different skin.
The standard display types supported by default are TV, PC, Touch Panel, and PDA, but integration with third-party products such as Windows™ Media Center™ Edition 2005 is supported as well. The system preferably detects any additional target displays as they are added to the system at run-time. The user may add as many additional display devices as needed and have their GUI generated automatically as well. After the various GUIs have been created, the user may choose to modify the visual and functional elements to his/her liking.
At run-time, a user may request to view a specific GUI from a specific display type. The web server receives an “http get” request with a parameter identifying the display type on which the requested file is to be rendered and returns the requested GUI display.
In order for the Automatic GUI process to work properly, the following building blocks are required:
The GUI screen preferably provides a simple and consistent layout independent of the GUI object as defined in the template file. The objects on the template (and ultimately the GUI screen) may make use of the intrinsic functionality provided by the GUI objects supported as defined by the UI language used (i.e., XUL & HTML). The user may modify any of the properties and/or events to change the visual appearance and/or functionality in the Editor after automatic GUI generation, respectively. Alternatively, the user may make those changes to the template files before automatic GUI generation.
The following tables list typical object attributes/properties.
An event generation mechanism is preferably provided which describes the “action” a user wishes to perform when interacting with a GUI (e.g., pressing a button). The template on which the automatic GUI is generated may include actions corresponding to the standard device commands described above, and may invoke additional actions such as executing a script, switching to different screens, or manipulating the visible GUI. For project-wide GUI screens, the template may include links to zones and control boxes. After automatic GUI generation, the user may modify the existing actions assigned to the objects, rearrange the order of the selected actions, or remove an action item. For example, as shown in
A callback handling mechanism is also preferably provided, allowing for evented-variables to be displayed and/or processed by the Interface screen at run-time. For example, if a project includes a thermostat, the temperature will be the evented variable. The user will be able to see the changing temperature via the user interface in real-time.
An evented-variable may be of type string, range (integer), or allowed-value. If the selected evented variable is of type string or range, the actual value is returned. If the selected evented variable is of type Allowed-value, the user settings will determine alternative data for each possible allowed-value. The mapped replacement data will be associated with the target selection determined above but can be data other than text. For example, if the template defines a change to the icon of a button, the replacement value can be a path to an image file.
The notification processing is preferably handled via a screen builder module. Several objects can receive and manage notifications on evented-variables received from the control point, including Window, Button, Image, and Label objects. All objects preferably employ the same mechanism for processing notifications, but differ in the actions that can be executed when specific conditions are met.
Referring again to
A project-level GUI, referred to as “net main,” is preferably automatically generated. The template of the “net main” interface preferably includes a main file that has two types of buttons: a “home” button that leads to a screen or page which lists all the zones in the home automation project, and “zone” buttons, the pressing of any of which leads to a file that shows a button for each device as well as all the shortcuts to UPnP™ Subdevices belonging to Subsystems. Subdevices such as dimmers and switches are elements of a Subsystem such as a Lighting Control Systems or HVAC systems. Subdevices depend on the Subsystem and may be physically located throughout the installation environment. The Subsystem manages and controls the subdevices and is typically installed and wired by custom installer and electricians. The subsystem typically interfaces with a hardware controller such as a control box via a serial interface allowing a system based on the present invention to control the subsystem which in turn controls the subdevices.
GUIs are also preferably automatically generated for all defined zones by looping through all defined devices and creating for each device a button based on the properties and defined in the template for the control box GUI screens. For example, the “onCommand” attribute of the button which holds the action command for the GUI object will, when clicked by the user, open the GUI of the requested device. This is achieved by preprocessing the address list of the existing devices, creating a button, and assigning the UPnP™ device address which manages the device to that button's “onCommand” event.
GUIs are also preferably automatically generated for all defined devices by searching for an XML definition having the same name of the device and extracting the device type name from it (e.g., a Sony wide screen TV model “xyz” is of type “TV”). Then, the templates folder, such as is shown in
Using the device name, the corresponding Device Command File is retrieved and parsed to create a device command list. For IR devices, this file preferably corresponds to the LIRC format (i.e., open-source project). For all other devices using other connection and communication methods (e.g., serial protocol), a device command file using a proprietary format may be used. Device command files preferably include predefined standardized device commands so that commands extracted from the template will have corresponding commands in the template file.
Once the files have been parsed, the buttons in the template are compared to the commands list extracted from the device command file. Any buttons that exist in the template but do not have a counterpart in the device command list may be left without an “onCommand” action assignment.
In the GUI file, each standardized button preferably has a unique ID identical to the standardized name. The generic function executed when the user presses the button sends the standardized command name as a parameter to the system, thus closing the loop between the GUI and the execution request on the device.
The screen builder preferably does not limit the user to the automatically generated GUI. The user may, at any time, modify each screen, both visually and functionally, or create a new screen. A new screen can be designed to control any number of devices in the project. When modifying or creating a new screen, the user can decide on the overall design of the screen, such as by changing the background, adding buttons, inserting images and assigning events to new items on the screen.
Notification actions may be assigned to any of the screen builder objects via a “Notification” tab associated with the respective object, such as is shown in
Each task may contain one or more conditions tested against an evented-variable belonging to a specific device on the network. If the conditions are met, selected actions are executed.
The user may do the following:
A task condition enables the user to link a device event to the task to be executed on a specific screen builder object. The user may:
The task condition is preferably linked to a specific device on the home network. The condition and possible values are preferably determined (i.e., retrieved from the device XML specification) as soon as the device from which notifications are to be received has been selected. The available options may depend on the selected device type and its connection type. Thus, some devices may offer a specific list of possible events, while others may provide a possible range of numerical values, such as is shown by way of example in
When all specified conditions have been met, one or more actions may be executed. The user may:
The available actions may vary with the GUI object (i.e., Label, Button, Image, or Window) as described below. Depending on the action selected, an appropriate dialog will open to allow the required parameters to be set.
The following table summarizes actions that may be executed on a GUI object when a notification task occurs (e.g., when a button is to be processed when a notification occurs, the user may change the button image, change the border color, the background color, etc.).
The automatic GUI generation preferably merges templates with the relevant device functionality. The templates used may be created and/or modified using the same mechanism employed for device interfaces. Once a template has been created according to the user's requirements, it may be saved to the “templates folder” according to its application.
An automation manager, such as is shown in
A task in the automation manager may be edited in a task editor showing the contents, time conditions, and actions of the task for modification. The task editor may be used to view/edit the association of actions with one or more event or date/time triggers. The task editor GUI also preferably provides date/time and re-occurrences management. Task conditions may be related to evented variables. The task editor preferably provides the following:
When modifications are made to automation tasks, the modifications are preferably synchronized with all automation lists maintained anywhere within the system.
An interface screen is provided for users to interact and control with the system. The interfaces are preferably web-based and are accessible via a browser which is run on any control box, such as using connected television screens as displays, touch panels, PCs, and handheld devices such as PDAs. The interface resources are preferably stored on control boxes or other suitable system elements and are served on request by an embedded web server.
The interface screens may be customized regarding their visual appearance and functionality of GUI elements. Additionally, an automatic GUI generation mechanism allows end-users to quickly create fully functional GUIs.
As shown in
The user interface screen preferably employs XUL (XML-based User-interface Language) or HTML, as with PDA's and the Microsoft Windows™ Media Center™, and JavaScript™ to communicate with a control point. XUL is an application of XML used to describe window layout in the Netscape Mozilla™ browser. Where the embedded OS is Linux based, a version of the Netscape/Mozilla™ web browser (i.e., Firefox™) is preferably employed, being designed to support open Internet standards such as HTML 4.0, CSS, XML, RDF, XUL, and JavaScript™.
The user interface is typically defined as three discrete sets of components:
An infrared remote control may be used to control the web-based UI on the television connected to the control box. The remote control preferably includes functionality that allows the user to execute specific operations on the hardware, such as zooming in or out on a picture or movie, streaming media across the home network, or operating devices connected to the network.
In addition, several buttons on the remote control may be programmed to run UPnP™ actions according to the user's requirements. For example, the Power button may be programmed to execute a series of commands that will turn on a television, switch to A/V mode, turn on the receiver, and switch the cable box to a specific channel.
To configure a remote control for use with a particular control box, the control box object is preferably accessed as described hereinabove, and a remote control properties window is accessed. Remote control buttons may then be selected and configured. Similar to the event assignment of a screen builder object, UPnP™ events may be assigned to a specific command on the remote control.
Using mobile or landline phones that are not web enabled, users can call their home that has been configured with the present invention and access the automation project by interacting with an Interactive Voice Response (IVR) menu. The menu for the IVR mechanism may be created and customized via a user interface using conventional methods.
As shown in
Once connected, the remote access control box preferably authenticates the caller and presents him/her with the IVR menu. The user may listen to the menu options and execute various home automation functions by pressing the buttons on the keypad. Additional services such as user authentication, voicemail retrieval, SMS or voicemail to email may also be provided.
The present invention provides an IP-based platform that facilitates its integration with other IP-based services such as web page browsing, using the integrated web browser, and Voice over IP (VoIP). Some ways in which the present invention may be integrated with VoIP include:
A media manager application is provided to function as front-end to the media server described hereinabove with reference to
The media manager application is preferably designed to allow users to view their media folders and locations and decide which of them to make accessible to the home network. The media files available to the media manager include folders and system-supported media files. The user may navigate through any of the folders and subfolders, if any, available to the home network. Selecting a file in the folder view pane will preferably cause the file's properties to be displayed in a file information pane.
One or more virtual directories may be provided as follows. From the end-user's perspective, the virtual directory may appear as a folder containing media which will be accessible to a media controller or other media processing devices on the network. The media controller is provided to function as front-end to the various media servers available on the network. Contrary to the media manager, the media controller is intended to allow users to access specific media files on the network and stream them to a desired media renderer available on the network. The media controller is preferably implemented using XUL, HTML, and JavaScript™ and is available on all standard display types. The virtual directory may be created without changing the original location of the media file on the user's hard drive. The media server is responsible for maintaining the available virtual directories, preferably including a history of previously used virtual directories. A user may, at any time after the installation, add directories browsing to a folder and adding it to the Virtual Directory list, or remove directories therefrom. By default all directories below this “root” directory may be accessible unless this feature has been disabled through preference settings. Default folders may be established that cannot be removed from the virtual directory list.
The user may scan the system for existing supported media types at any time. This process collects information on all supported media files stored on connected devices and makes them available on the network. Once media files appear in a virtual directory tree view, their availability on the network can be managed using the add/remove feature described above. Scanning may be performed when the media server is initially installed. At the beginning of the scanning process, a tree of the file system about to be scanned is generated. Virtual directories may be added to the media server based on user selections
The user may filter media files based on file size as well (e.g., excluding files smaller than 1 MB). The applied selection and filtering may be saved for use during future scans. New files and folders which are children of active virtual folder may be added and made available to the network automatically as soon as they become available.
At run-time, automatic scanning is performed only on existing virtual directories. Manual scanning may be performed on any hard drive/folder on an attached computer. Virtual directories may be added as desired.
As was described hereinabove, a UPnP™ device is a container of services and nested devices defining the different categories of UPnP™ devices. An XML device description document hosted by the device maintains all device metadata including information about the embedded services and device properties (e.g., device name). The UPnP™ device implementation of the present invention is preferably cross-platform code running on top of the Intel™ UPnP™ SDK on Linux and Windows™. It is preferably used for all UPnP™ device implementations of the present invention and provides the basic UPnP™ device functionalities. In order to provide connection to non-UPnP™ enabled devices, UPnP™ bridge devices may be implemented for several interface types. UPnP™ software include the media server (
The UPnP™ IR device of the present invention acts a bridge between the UPnP™ network and the IR transmission interface may be implemented via the LIRC Open-Source project and embedded in the ControlOS within the hardware. Alternatively, the LIRC Open-Source project may be hosted on a PC attached to the device control network. An important part of LIRC is the lircd daemon that decodes IR signals received by the lirc device drivers and provides the information to a socket. The UPnP™ IR device may be configured to run under Linux, and can preferably be configured to run on different ports depending on the hardware. The UPnP™ IR device preferably contains a single service defined as:
Among the various elements and attributes that may be defined in a description document for the UPnP™ IR device, <lircRemoteName/> is preferably used to communicate with the LIRC daemon with the appropriate remote name. Therefore, any UPnP™ command exposed by this service will automatically be sent to the LIRC daemon identified by the configured remote name.
The UPnP™ serial device of the present invention acts as a bridge between the UPnP™ network and the serial transmission interface implemented and embedded in the ControlOS within the hardware. The serial device description XML document template preferably contains a single service defined as:
The Serial Device Template functions as an adapter between a UPnP™ based TCP/IP network and the serial protocol. It contains a translation of all the device-specific communication settings and parameters in a well-formed XML based interface file. This solution requires no coding or compilation, and the driver may be modified and maintained within any text editor. All information about a specific serial device may be extracted from the manufacturer supplied manual or serial device description document.
As shown in
The Serial Device Application within the Serial Device Adapter receives the UPnP™ Action Request, and translates it to a serial command by retrieving the appropriate data from an in-memory representation of the user created Serial Protocol Definition file (SerialDevicePD.xml). The serial command is then directed to the serial port to which the device has been connected and configured. The serial device receives the serial command and performs the requested action. If the serial device returns a response to the serial device application, an appropriate UPnP™ Action Response is generated and is sent back to the control point to be redirected to the User Interface screen.
A serial device may send a response independent of a UPnP™ Action Request. For example, the user may operate the device directly, thus triggering a response to be sent to the serial device application. The Notification Listener will catch the serial response command and send a UPnP™ Notification to the Control Point if the Serial Protocol Definition contains a relevant evented Variable entry.
The Serial Protocol Definition may include:
The user can override the global definition by placing a delay tag. (default: 0).
Some devices use values or characters that cannot be written to the output buffer. An optional TransTable may be provided that manages the conversion between forbidden values and their corresponding conversion values. This tag can be left empty if no conversion is required. Otherwise, the user must specify the following five tags for each value: Value, valueType, In, Out, Place.
A—the entire buffer except the start and end string.
ES—includes the end string.
SS—includes the start string.
A protocol defining the serial communication type/method may include:
A UPnP™ device supports one or more actions which may be invoked. An action list is preferably provided that contains all actions supported by the UPnP™ serial device, their arguments and the format of the command. Any input parameters are preferably associated with relatedStateVariable. If there are no parameters in the command (which implies that the command is a simple string), no related state variable needs to be specified.
A service state table is preferably provided that is used to send notification messages about changes that occur within the serial device. Thus, the sendEvents attribute must be set and the following fields must be provided for each related state variable: name, dataType, valueType, and allowedValueRange/allowedValueList.
In
Reference is now made to
UPnP™ Media Server devices are used in conjunction with one or more Media Renderer device(s) to allow a Control Point to discover entertainment (AV) content (e.g. video, music, images, etc) on the Media Server and to render that content on any appropriate Media Renderer within the home network. In general terms, the process begins with the Control Points discovering Media Server and Media Renderer devices within the home network. The Control Point interacts with a Media Server(s) to locate a desired piece of content (e.g., a movie, a song, a play list, a photo album, etc). After the content has been identified, the control point needs to identify a common transfer protocol and data format that can be used to transfer the content from the Media Server to the desired Media Renderer. After these transfer parameters have been established, the Control Point controls the flow of the content (e.g. Play, Pause, Stop, Seek, etc.). Depending on the selected transfer protocol, these flow control operations are preferably sent either to the Media Server or the Media Renderer, but not both. The actual transfer of the content is performed directly by the Media Server and Media Renderer using a transfer protocol other than UPnP.
The Media Server is preferably implemented as a Windows™ Service and contains the following components and functionality:
The UPnP™ Media Renderer of the present invention is preferably based on the UPnP™ Media Renderer Device Template running on the Linux platform.
A UPnP™ control point is provided that is a controller capable of discovering and controlling other devices on a network. After discovery, a control point can retrieve the device description and its services, invoke actions to control a service, and subscribe to the service's event source. Anytime the state of the service changes, the event server will send an event to the control point. The specifications are defined and managed by the UPnP™ forum and are described in the UPnP™ Device Architecture document.
The control point is preferably cross platform code that runs on top of the Intel™ UPnP™ SDK, running under Linux, Windows™ and WinCE™, as a Windows™ service and/or a Linux daemon.
The Control Point configuration file is preferably loaded from a ControlPointConfig.xml Configuration file located on Windows™ at /INSTALL_DIR/CP/ServerRoot/, and in Linux at /etc/upnp/CP/ServerRoot/ControlPointConfig.xml. Using this configuration file, the Control Point's Networking parameters, device search types, and logging level may be configured and UPnP™ devices and services may be filtered.
There are two interfaces for applications to interact with the control point.
A command interface is provided that is accessible through a post command to the control point http server, being a localhost with a port number specified in the configuration file <uiHttpPort>.
The available commands may include:
Notification is preferably provided via a TCP port on the local host, with the port number being specified in the configuration file <cpNotificationsPort>.
The available notifications may include:
An IR Learning application is preferably provided which allows new remote control configurations to be learned and existing remote control configurations to be edited. The application preferably includes a wizard to quickly capture and verify IR codes in order to expand the system's database of remote controls. The learning/editing process of a remote control is handled by the IR receiver/transmitter embedded in the control box.
The XML-based serial adapter described hereinabove may be created by using a wizard-based Serial Adapter Creator, where all required information about a specific device can be extracted from the manufacturer supplied user manual or serial protocol definition document. The generated driver may then be added to the system device driver database. The generic design of the serial device template allows an end-user to easily create a device specific interface to enable a non-UPnP™ enabled device to be integrated into the home automation system.
It is appreciated that various features of the invention which are, for clarity, described in the contexts of separate embodiments may also be provided in combination in a single embodiment. Conversely, various features of the invention which are, for brevity, described in the context of a single embodiment may also be provided separately or in any suitable subcombination.
It is appreciated that one or more of the steps of any of the methods described herein may be omitted or carried out in a different order than that shown, without departing from the true spirit and scope of the invention.
While the methods and apparatus disclosed herein may or may not have been described with reference to specific computer hardware or software, it is appreciated that the methods and apparatus described herein may be readily implemented in computer hardware or software using conventional techniques.
While the present invention has been described with reference to one or more specific embodiments, the description is intended to be illustrative of the invention as a whole and is not to be construed as limiting the invention to the embodiments shown. It is appreciated that various modifications may occur to those skilled in the art that, while not specifically shown herein, are nevertheless within the true spirit and scope of the invention.
This application is related to and claims priority from U.S. Provisional Patent Application No. 60/622,008 to Vardi et al., entitled “Networking Architecture,” filed Oct. 27, 2004, and incorporated herein by reference in its entirety.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IL05/01124 | 10/27/2005 | WO | 00 | 6/25/2008 |
| Number | Date | Country | |
|---|---|---|---|
| 60622008 | Oct 2004 | US |
