The present invention generally relates to web-based home surveillance technology and home security systems, and relates in particular to a home networked home surveillance service system and architecture for a portable or remote monitoring device.
Home surveillance is an emerging market for home security. The existing market for home surveillance is limited to burglar alarm and central station notification. This system requires monthly service fees and the system is proprietary; therefore, it involves huge cost associated with upgrading the systems.
In a different market, many networked home surveillance cameras are being sold on the market. These networked cameras are Internet capable and offer continuous monitoring of the home through web technologies. However, the cost of retrofitting these state of the art devices to the existing centralized home security systems is quite high.
The need exists for a way to integrate home surveillance cameras and web technology with home security systems. The present invention fulfills this need.
In accordance with the present invention, a networked mobile home surveillance system includes a home surveillance application running on a user device to access services on a home network and receive data from the home network. A home network gateway uses SIP messaging to allow communication of commands and data on the user device to and from a home networked device through bridging between SIP and a non-SIP protocol of the home network. A home surveillance service on the home network controls home networked remote devices used to monitor the home, and is subject to control by the application.
Further areas of applicability of the present invention will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description and specific examples, while indicating the preferred embodiment of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
The present invention will become more fully understood from the detailed description and the accompanying drawings, wherein:
a and 6b are views of user interface components by which a user can configure camera settings and/or home surveillance settings in accordance with the present invention;
The following description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses.
The system and architecture according to the present invention enables a home networked home surveillance service that is extendable and upgradeable through a home gateway. The service is based on a home network, which significantly reduces the cost for central security service providers. The service is further capable of notifying a remote user directly through SIP and providing a remote real-time monitoring service upon request.
Various embodiments of the present invention can enable and/or provide one or more of the following applications and features: (1) a home surveillance service can be upgraded, installed and run on a home network that is connected to a home gateway and other networked devices that can be accessed and controlled by any service on the network including home surveillance service; (2) a home surveillance service on the home network monitors home and detect multiple configurable events such as burglar, fire, intrusion, false alarm etc.; (3) a home surveillance service can either send a notification to the central station or directly to the user at a remote location upon the detection of events; (4) a user (or central station) can request additional services from home surveillance such as control the camera, triggering sprinkler system, making 911 calls to police station upon fire; etc.; and (5) a home surveillance system can send data and media (e.g. streaming from a camera) to the user at remote location or central station.
Referring to
The remote device 100 can include an operating system 104, a java virtual machine (JVM) 106, applications 112 with GUI, SIP middleware 110, network stack (SIP) 108, and wireless interface 102. It can communicate with the home network via SIP protocol. Therefore, the remote device can also act as a SIP UA (user agent) from SIP network perspective.
Home surveillance applications can be mobile, and can realize the scenarios described previously. The home monitoring application can be able to communicate with a home surveillance service installed on a home network (an OSGi bundle) and be able to receive alarms upon intruders and monitor the home through one or more networked cameras.
SIP stack can enable the mobile device with SIP. Depending on SIP implementation, a JVM may or may not be required.
SIP was originally designed as the protocol for multimedia session creation and termination with its intended use in Voice over IP. To use SIP in home networking applications, a middleware is needed to convert SIP signaling protocols to home networking specific functions. This middleware according to the present invention can be an internal library that receives messages from mobile applications in relation to home entertainment, and converts them to SIP commands. It can also receive SIP messages and events and send relevant messages to applications.
The purpose of building SIP middleware is two fold. One purpose is to simplify SIP signaling protocols for applications, and this middleware can be re-used in future applications. The second purpose is to make the mobile application independent of SIP, therefore providing flexibility to use protocols other than SIP protocols in the future, if needed. For example, if the remote device is equipped with OSGi framework in the future, SIP middleware can be replaced, but the applications do not need to be rewritten.
SIP server/proxy 134 according to the present invention can be an intermediary device that is located within the SIP-enabled network 136 and assists user agents in session establishment and other functions. It is a general term for SIP Proxy, redirect server, or registrar server.
OSGi gateway 114 according to the present invention can provide a basic framework for networked devices to be able to communicate and control each other. OSGi supports a variety of networks, such as UPnP, Jini, Http, etc.
The SIP-OSGi bridging 116 according to the present invention can be an adaptor to allow devices and services (applications) on the OSGi gateway to possess SIP capability that allows them to be able to communicate with other SIP devices in a remote location via SIP server/proxy. Home surveillance-SIP bridging 118 can interface a home surveillance service bundle 120 and other bundles 122 with SIP-OSGI bridging. SIP-OSGi bridging can be constructed in two ways.
One way in which SIP-OSGI bridging can be constructed is referred to herein as SIP Stack Bundle on OSGi. Referring to
Another way in which the SIP-OSGI bridging can be constructed is referred to herein as SIP Service for OSGi. Turning now to
The SIP Service is an extension of OSGi framework, and further details relating to the SIP service can be found in the present application's parent U.S. patent application Ser. No. 10/894,469 filed on Jul. 19, 2004, herein incorporated by reference in its entirety for any purpose. The SIP-OSGi bridging allows sharing and control of devices from outside a home via SIP service. Other services each implementing specific functions can also be installed and executed on the framework. For example, home surveillance service can be installed on the OSGi framework and utilized by devices or other bundle services on the network.
Home surveillance service provides home monitoring and surveillance. It can periodically access home security camera (e.g. a 1394 or UPnP network camera) and detect if there are any intruders. If any intruders are detected or extreme situations occur, an alarm can be sent off to the gateway middleware layer, which in turn can send an alarm to the registered SIP devices. Upon request from the registered remote device through mobile-home middleware, a streaming video session can be established for monitoring surveillance video on the remote device.
The SIP Bridging for home surveillance service 208 is provided within mobile-home middleware to allow other SIP devices to access or control home surveillance service or camera.
SIP Device bridging provides capabilities for home surveillance service to be able to control other OSGi or non-OSGi devices. For example, a SIP-UPnP bridging can convert SIP commands/messages/events to those of UPnP and therefore allow home surveillance service to control UPnP device (e.g. UPnP camera) on OSGi framework.
The home surveillance application registers the mobile device with the home surveillance service and is able to receive any alarms in case of burglars. In case of burglar, the application is also able to request establishment of video stream session with home surveillance service so video captured from the home security camera can be transmitted and displayed on the mobile device. Specifically, the function is composed of the following components: (1) a status window to display the status/response corresponding to each button item; (2) a video window having a quick button to display video images captured from the home surveillance networked camera as follows: (a) when an alarm is received from Home Surveillance Service, a message “Alarm!! Intruder detected!” can be displayed; (b) a user can request a real-time video session by clicking the “video” button; and (c) upon establishment of the video session, the status window displays “Real-time monitoring . . . ”; (3) a connect component that can be selected by a user to connect to SIP server (managed by SIP Service on the home network) and register with the home network, causing a list of services connected to be displayed, such as “Home Surveillance registered”; (4) a configure component that can be selected by a user to configure home surveillance service IP address (SIP Server address) and set services available for subscription, such as home surveillance service; (5) a control component that can be selected by a user to access/control home surveillance service, such as request video session, control camera action or parameters, and set system parameters such as sensitivity for detecting motions, etc.; (6) a disconnect component that can be selected by a user to cancel a video stream session; (7) an exit component that can be selected by a user to cancel a video stream session (if applicable), unregister with the SIP server, and quit the application.
Continuing with
As previously stated, SIP middleware is needed in order to convert SIP signaling protocols to home network flavor functions, and vice versa. Particularly, for home networking applications, several basic functions are needed: (a) register a mobile device with home network; (b) list the devices, services and events available on the home network; (c) subscribe to events on the home network, such system events (such as if new devices are added to the home network), and special events that are provided by devices or services on the network (such as an alarm sent from the home surveillance service); (d) send control commands to a device on the home network, such as “record” on a personal video recorder (PVR); (e) request and receive data from home network devices or services, such as data indicating the home monitoring status and status of other security devices; (f) start and terminate audio/video streaming session with an A/V device on the home network, such as connect to a SIP phone at home, or view video stream that is being captured from a security camera.
The SIP protocol provides the capability of device/service discovery, control, registration and events through methods like REGISTER, MESSAGE, and/or SUBSCRIBE. Both data and media stream need to be “carried” via SIP service. The challenge is that SIP can only carry a small message body, which is not suitable for a large chunk of data or streams of video. Therefore, the transport of data can be implemented in the following ways: (a) short messages such as request and control commands can be carried in SIP MESSAGE body as plain text, and these messages can be transparent to proxies and need to be interpreted at the SIP end point; (b) add additional information such as a chunk of data can be carried in a separate message body attachment as payloads, and it can be either text based or MIME type; and (c) RTP can then be used for media transport. The multimedia streaming session can be negotiated by applications, in SDP (session description protocol)—a simple text based protocol. This is supported by SIP and completely transparent to SIP.
Home surveillance service bundle is an application specific service that is packaged into OSGi bundles to be executed within OSGi framework from any requested devices/services. The home surveillance service constantly monitors video captured from a security camera, and sends an alarm to a registered listing device upon detection of intruders. It can include the following functions: (a) register device for capturing device; (b) capture and store video frames for processing; (c) detect intrusion based on video analysis (e.g., simple motion analysis can be used); (d) send an alarm to a registered device upon detection of intrusion; (e) receive commands to configure the service such as the sensitivity of alarm triggering, configure motion pattern for intrusion detection (in order to prevent certain false alarm from home pet); or establish real-time streaming between the mobile user device and a camera connected to the home network, etc.
Returning now to
The functional flow and message flow of remote applications according to a presently preferred embodiment can be described in terms of the application and SIP middleware, SIP messaging, and Java objects for remote applications. The functional flow and message flow of an OSGI gateway according to a presently preferred embodiment can be described in terms of home surveillance bundle and SIP bridging, and in terms of Java objects for home surveillance and SIP bridging.
Referring now to
All commands 716 from user interface input components 706 of the application, data, and media between application and middleware are processed through Message module 718 in mobile-home middleware. The Message module 718 passes the commands from the application, and converts them to an intermediate data structure to be sent to the Control module 720, which then converts the received information to a SIP compliant message. The message module also passes any messages data 714 and/or media data 712 received from Control module, interprets it, and sends it to the applications. These data include any short messages data 714 or media data 712 to be rendered in the application at windows 702 and/or 704. Control module 720 receives the commands from Message module 718 and packs them into SIP message body, or strips any data from SIP message body, and parses and converts to application data (intermediate data structure). Some user configuration data (such as server IP address, camera settings, alarm sensitivity etc.) are stored locally in Profile manager 722. SIP module 710 acts as SIP UA and directly communicates with other SIP UA or SIP server through SIP protocol. SIP module 710 can register the mobile-home middleware to the SIP server, and receives any SIP messages from other SIP devices through SIP server.
Turning now to
Notice that the 200 OK response to a REGISTER echoes the contact URL that have been successfully registered.
In another example, the mobile-home SIP middleware subscribes to the Home surveillance SIP UA and specifies the “Burglar” event at 810 and 812. Consider the following example message:
SUBSCRIBE sip:hs@myhome.com SIP/2.0
In a further example, upon burglar event, Home surveillance SIP UA notifies Mobile-Home SIP middleware at 814. The sample messages can look like this:
In a still further example, Mobile-Home SIP middleware invites Home surveillance SIP UA for a real-time media session to monitor video at 816. The sample message can look like this:
Optionally, the INVITE message can carry a SDP body to specify media negotiation such as RTP.
Finally, upon acknowledgment of both parties, a video session is established between Mobile-Home SIP middleware and Home surveillance SIP UA at 818, and the user can monitor the home on a mobile device.
Examples of Java Objects for remote applications are supplied below in pseudocode.
Turning now to
Optionally, the home surveillance bundle can implement a service interface module 926 that accepts control commands from the mobile user application 900 to take any actions to respond to the burglar alarm. For example, the user can tell the service interface module to contact other members in the family, notify security service providers or directly dial to police etc.
While the control module is a Java API that allows other OSGi bundles to share its functionality, the burglar detection module can be written in native language for performance. OSGi frameworks support the native environment. The home surveillance bundle can be used by other bundles on the OSGi framework, or the mobile device through home surveillance SIP bridging middleware 902.
The gateway home surveillance SIP bridging middleware acts as an activator of the home surveillance bundle for SIP. This SIP bridging itself can be an OSGi bundle. Upon start, SIP bridging registers to the SIP service (proxy) on OSGi gateway and become a SIP UA. By way of SIP service registration, it is automatically registered to the OSGi framework, which allows it to have access to other OSGi bundles including Home Surveillance Bundle. Since it becomes a SIP UA, it is accessible by other SIP UAs including mobile device on the SIP network. Subsequent SIP communication between mobile device and home surveillance SIP bridging middleware can be used to send/receive data and commands from the mobile device to home surveillance service. SIP commands from a mobile device are translated so appropriate home surveillance bundle control functions can be called. Similarly, any status/data and notification (alarm) from the home surveillance bundle control module are packed into appropriate SIP messages and sent to the mobile device.
SIP-UPnP and SIP-1394 Bridging can alternatively or additionally be implemented. Accordingly, home surveillance and device control SIP bridging middleware can directly query device capabilities (state variables) or control such device accordingly. This direct query and control can be done through SIP-UPnP or SIP-1394 bridging. SIP messages are translated to appropriate function calls to work with OSGi UPnP service or OSGi 1394 control bundle, and vice versa.
The discovery and registration of UPnP devices on the OSGi framework is automatically implemented by the UPnP Service that is included in OSGi release 3. Similar functionalities are also expected in OSGi 1394 control bundle.
Examples of Java Objects for Home Surveillance and SIP Bridging are provided below in pseudocode.
The description of the invention is merely exemplary in nature and, thus, variations that do not depart from the gist of the invention are intended to be within the scope of the invention. Such variations are not to be regarded as a departure from the spirit and scope of the invention.
This application is a continuation-in-part of U.S. patent application Ser. No. 10/894,469 filed on Jul. 19, 2004 which claims the benefit of U.S. Provisional Application No. 60/524,599, filed on Nov. 25, 2003. The disclosures of the above applications are incorporated herein by reference in their entirety for any purpose.
Number | Date | Country | |
---|---|---|---|
60524599 | Nov 2003 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10894469 | Jul 2004 | US |
Child | 11285625 | Nov 2005 | US |