Annotation of metadata through capture infrastructure

Description

FIELD OF TECHNOLOGY

This disclosure relates generally to the technical field of networking, and in one example embodiment, this disclosure relates to a method, an apparatus, and a system of annotation of meta-data through a capture infrastructure.

BACKGROUND

A user may access a mobile communication device while watching television. An advertisement presented to the user on the mobile communication device may be independent of the content being viewed on the television. For example, the user may be watching a television show about car washes. However, advertisements displayed on the mobile device may have nothing to do with water, cars, and/or cleanliness. Similarly, advertisements displayed on the television may not appreciate a present activity of the user when accessing the mobile device. As such, an advertiser may miss key marketing opportunities to the user to deliver highly targeted and relevant advertising to the user.

SUMMARY

An apparatus and a system of annotation of meta-data through a capture infrastructure are disclosed. In one aspect, a method of a client device includes applying an automatic content recognition algorithm to determine a content identifier of an audio-visual data. The client device then associates the content identifier with an advertisement data based on a semantic correlation between a meta-data of the advertisement provided by a content provider and/or the content identifier. A capture infrastructure annotates the audio-visual data with a brand name and/or a product name by comparing entries in the master database with a closed captioning data of the audio-visual data and/or through an application of an optical character recognition algorithm in the audio-visual data. The content identifier may involve a music identification, an object identification, a facial identification, and/or a voice identification. A minimal functionality including accessing a tuner and/or a stream decoder that identifies a channel and/or a content may be found in the networked media device. The networked media device may produce an audio fingerprint and/or a video fingerprint that is communicated with the capture infrastructure.

The capture infrastructure may compare the audio fingerprint and/or the video fingerprint with a master database. The capture infrastructure may further annotate the audio-visual data with a logo name by comparing entries in the master database with a logo data of the audio-visual data identified using a logo detection algorithm. The capture infrastructure may automatically divide the audio-visual data into a series of scenes based on a sematic grouping of actions in the audio-visual data. The audio-visual data may be analyzed in advance of a broadcast to determine content identifiers associated with each commercial in the audio-visual data such that advertisements are pre-inserted into the audio-visual data prior to broadcast.

The capture infrastructure may apply a time-order algorithm to automatically match advertisements to the audio-visual data when a correlation pattern is identified by the capture infrastructure with other audio-visual content previously analyzed. The capture infrastructure may include a buffer that is saved to a persistent storage and/or for which a label is generated to facilitate identification of reoccurring sequences. A post processing operation may be automated through a post-processing algorithm and/or a crowd-sourced operation using a plurality of users in which a turing test is applied to determine a veracity of an input.

A device pairing algorithm may be used in which a cookie data associated with a web page visited by the user stored on a browser on the client device is paired with the networked media device when the client device is communicatively coupled with the networked media device. A transitive public IP matching algorithm may be utilized in which the client device and/or the networked media device communicates each public IP address with any paired entity to the capture infrastructure. A tag that is unconstrained from a same-origin policy may be used to automatically load the advertisement in the browser, the tag is an image tag, a frame, a iframe, and/or a script tag.

An additional metadata including the content identifier and/or the advertisement based on a video processing algorithm may be referenced. The additional meta data may be a title, a description, a thumbnail, a name of an individual, and/or a historical data. The additional metadata may be determined from a browser history captured from the client device based on a capture policy, and/or correlating a relevance of the browser history with the content identifier and/or the advertisement.

In another aspect, a method of a networked device includes applying an automatic content recognition algorithm to determine a content identifier of an audio-visual data, and associating the content identifier with an advertisement data based on a semantic correlation between a meta-data of the advertisement provided by a content provider and/or the content identifier. In this other aspect, a capture infrastructure annotates the audio-visual data with a brand name and/or a product name by comparing entries in the master database with a closed captioning data of the audio-visual data and/or through an application of an optical character recognition algorithm in the audio-visual data.

In yet another aspect, a system includes a networked device and/or a client device to apply an automatic content recognition algorithm to determine a content identifier of an audio-visual data and/or to associate the content identifier with an advertisement data based on a semantic correlation between a meta-data of the advertisement provided by a content provider and/or the content identifier. The system also includes a capture infrastructure to annotate the audio-visual data with a brand name and/or a product name by comparing entries in the master database with a closed captioning data of the audio-visual data and/or through an application of an optical character recognition algorithm in the audio-visual data.

The methods, system, and/or apparatuses disclosed herein may be implemented in any means for achieving various aspects, and may be executed in a form of machine readable medium embodying a set of instruction that, when executed by a machine, causes the machine to perform any of the operations disclosed herein. Other features will be apparent from the accompanying drawing and from the detailed description that follows.

BRIEF DESCRIPTION OF DRAWINGS

Example embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawing, in which like references indicate similar elements and in which:

FIG. 1 is a block diagram of a system of automatic bidirectional communication between multiple devices sharing a common network, according to one embodiment.

FIG. 2 is a block diagram of a system of automatic bidirectional communication between a client device 100 and a networked device 102 using a server, according to one embodiment.

FIG. 3 is an exploded view of the security sandbox 104, according to one embodiment.

FIG. 4 is an exploded view of the pairing server 200, according to one embodiment.

FIG. 5 is an exploded view of the client device 100, according to one embodiment.

FIG. 6 is a table of example network information stored in a database 422 of a pairing server 200, according to one embodiment.

Other features of the present embodiments will be apparent from the accompanying drawings and from the detailed description that follows.

DETAILED DESCRIPTION

An apparatus and a system of annotation of meta-data through a capture infrastructure are disclosed.

In one embodiment, a method of a client device 100 includes applying an automatic content recognition algorithm (e.g., in the algorithm library 107) to determine a content identifier 111 of an audio-visual data (e.g., a movie, a television show, an advertisement, etc.). The client device 100 then associates the content identifier 111 with an advertisement data 113 based on a semantic correlation between a meta-data of the advertisement (a particular advertisement of the advertisement data 113) provided by a content provider (e.g., an organization providing advertisements) and/or the content identifier 111. A capture infrastructure 105 annotates the audio-visual data with a brand name and/or a product name by comparing entries in the master database 109 with a closed captioning data of the audio-visual data and/or through an application of an optical character recognition algorithm (e.g., in the algorithm library 107) in the audio-visual data. The content identifier 111 may involve a music identification, an object identification, a facial identification, and/or a voice identification. A minimal functionality including accessing a tuner and/or a stream decoder that identifies a channel and/or a content may be found in the networked media device (e.g., the networked device 102). The networked media device (e.g., the networked device 102) may produce an audio fingerprint and/or a video fingerprint that is communicated with the capture infrastructure 105.

The capture infrastructure 105 may compare the audio fingerprint and/or the video fingerprint with a master database 109. The capture infrastructure 105 may further annotate the audio-visual data with a logo name by comparing entries in the master database 109 with a logo data of the audio-visual data identified using a logo detection algorithm (e.g., in the algorithm library 107). The capture infrastructure 105 may automatically divide the audio-visual data into a series of scenes based on a sematic grouping of actions in the audio-visual data. The audio-visual data may be analyzed in advance of a broadcast to determine content identifier 111s associated with each commercial in the audio-visual data such that advertisements are pre-inserted into the audio-visual data prior to broadcast.

The capture infrastructure 105 may apply a time-order algorithm (e.g., in the algorithm library 107) to automatically match advertisements to the audio-visual data when a correlation pattern is identified by the capture infrastructure 105 with other audio-visual content previously analyzed. The capture infrastructure 105 may include a buffer that is saved to a persistent storage and/or for which a label is generated to facilitate identification of reoccurring sequences. A post processing operation may be automated through a post-processing algorithm (e.g., in the algorithm library 107) and/or a crowd-sourced operation using a plurality of users in which a turing test is applied to determine a veracity of an input.

A device pairing algorithm (e.g., in the algorithm library 107) may be used in which a cookie data associated with a web page visited by the user stored on a browser on the client device 100 is paired with the networked media device (e.g., the networked device 102) when the client device 100 is communicatively coupled with the networked media device (e.g., the networked device 102). A transitive public IP matching algorithm (e.g., in the algorithm library 107) may be utilized in which the client device 100 and/or the networked media device (e.g., the networked device 102) communicates each public IP address with any paired entity to the capture infrastructure 105. A tag that is unconstrained from a same-origin policy may be used to automatically load the advertisement in the browser, the tag is an image tag, a frame, a iframe, and/or a script tag.

An additional metadata including the content identifier 111 and/or the advertisement based on a video processing algorithm (e.g., in the algorithm library 107) may be referenced. The additional meta data may be a title, a description, a thumbnail, a name of an individual, and/or a historical data. The additional metadata may be determined from a browser history captured from the client device 100 based on a capture policy, and/or correlating a relevance of the browser history with the content identifier 111 and/or the advertisement.

In another embodiment, a method of a networked device includes applying an automatic content recognition algorithm (e.g., in the algorithm library 107) to determine a content identifier 111 of an audio-visual data, and associating the content identifier 111 with an advertisement data 113 based on a semantic correlation between a meta-data of the advertisement provided by a content provider and/or the content identifier 111. In this other aspect, a capture infrastructure 105 annotates the audio-visual data with a brand name and/or a product name by comparing entries in the master database 109 with a closed captioning data of the audio-visual data and/or through an application of an optical character recognition algorithm (e.g., in the algorithm library 107) in the audio-visual data.

In yet another embodiment, a system includes a networked device and/or a client device 100 to apply an automatic content recognition algorithm (e.g., in the algorithm library 107) to determine a content identifier 111 of an audio-visual data and/or to associate the content identifier 111 with an advertisement data 113 based on a semantic correlation between a meta-data of the advertisement provided by a content provider and/or the content identifier 111. The system also includes a capture infrastructure 105 to annotate the audio-visual data with a brand name and/or a product name by comparing entries in the master database 109 with a closed captioning data of the audio-visual data and/or through an application of an optical character recognition algorithm (e.g., in the algorithm library 107) in the audio-visual data.

FIG. 1 is a block diagram of a system of automatic bidirectional communication (e.g., sending and receiving information in both directions without prior configuration by a human) between multiple devices sharing a common network, according to one embodiment. FIG. 1 shows a client device 100, a networked device 102, a security sandbox 104, an executable environment 106, a processor 108, a storage 109, a memory 110, a sandboxed application 112, and a sandbox reachable service 114. The client device 100 communicates bidirectionally with the networked device 102 of FIG. 1.

According to one embodiment, a client device 100 may be a computer, a smartphone, and/or any other hardware with a program that initiates contact with a server to make use of a resource. A client device 100 may constrain an executable environment 106 in a security sandbox 104, execute a sandboxed application 112 in a security sandbox 104 using a processor 108 and a memory 110, and automatically instantiate (e.g., manifest) a connection (e.g., a complete path between two terminals over which two-way communications may be provided) between a sandboxed application 112 and a sandbox reachable service 114 of the networked device 102.

According to one embodiment, a networked device 102 may be a television, stereo, game console, another computer, and/or any other hardware connected by communications channels that allow sharing of resources and information. A networked device 102 may comprise a number of sandbox reachable applications. A networked device 102 may announce a sandbox reachable service 114 using a processor 108 and a memory 110. According to one embodiment, a processor 108 may be a central processing unit (CPU), a microprocessor, and/or any other hardware within a computer system which carries out the instructions of a program by performing the basic arithmetical, logical, and input/output operations of the system. According to one embodiment, a memory 110 may be a random access memory (RAM), a read only memory (ROM), a flash memory, and/or any other physical devices used to store programs or data for use in a digital electronic device.

The security sandbox 104, the processor 108, the storage 109, and the memory 110 each exist within the client device 100 of FIG. 1, and they communicate bidirectionally with each other. According to one embodiment, a security sandbox 104 may be an operating system on which the sandboxed application 112 is hosted, a browser application of the operating system, and/or any other mechanism for separating running programs to execute untested code and/or untrusted programs from unverified third-parties, suppliers, untrusted users, and untrusted websites. According to one embodiment, a storage 109 may be a technology consisting of computer components and recording media used to retain digital data.

The executable environment 106 exists within the security sandbox 104 of FIG. 1. According to one embodiment, an executable environment 106 may be a virtual machine, a jail, a scripting language interpreter, a scratch space on disk and memory, and/or any other tightly controlled set of resources in which to run guest programs.

The sandboxed application 112 exists within the executable environment 106 of FIG. 1. According to one embodiment, a sandboxed application 112 may be an untested code, an untrusted program (e.g., from an untrusted web page), and/or any other software that can be executed with the appropriate runtime environment of the security sandbox 104.

The sandbox reachable service 114 exists within the networked device 102 of FIG. 1. According to one embodiment, a sandbox reachable service 114 may be a smart television application, a set-top box application, an audio device application, a game console application, a computer application, and/or any other service that can be discovered and communicated with from within the sandboxed application 112. FIG. 1 may encompass constraining a sandbox reachable service 114 in a security sandbox 104 where it is described sandbox reachable service 114, according to one embodiment. A security sandbox 104 may not allow a sandbox reachable service 114 that is constrained in the security sandbox 104 to open a server socket and receive inbound connections. However, a sandbox reachable service 114 that is constrained in the security sandbox 104 may still announce and be discovered, but all communications between a client device 100 and a networked device 102 may need to traverse through a relay in a pairing server 200.

FIG. 2 is a block diagram of a system of automatic bidirectional communication between a client device 100 and a networked device 102 using a server, according to one embodiment. FIG. 2 shows a client device 100, a networked device 102, a security sandbox 104, an executable environment 106, a processor 108, a memory 110, a sandboxed application 112, a pairing server 200, a shared network 202, a Wide Area Network (WAN) 204, a devices 206, a global unique identifier (GUID) 208, an alphanumeric name 210, a private address pair 212, a sandbox reachable service 114, an identification data 216, a switch 218, a public address pair 220, and a hardware address 222.

The client device 100, the networked device 102, and the devices 206 communicate bidirectionally with each other through the switch 218 in the shared network 202. According to one embodiment, a devices 206 may be a television, a projection screen, a multimedia display, a touchscreen display, an audio device, a weather measurement device, a traffic monitoring device, a status update device, a global positioning device, a geospatial estimation device, a tracking device, a bidirectional communication device, a unicast device, a broadcast device, a multidimensional visual presentation device, and/or any other devices with a network interface. According to one embodiment, a switch 218 may be a telecommunication device (e.g., a broadcast, multicast, and/or anycast forwarding hardware) that receives a message from any device connected to it and then transmits the message only to the device for which the message was meant.

According to one embodiment, a shared network 202 may be a local area network, a multicast network, an anycast network, a multilan network, a private network (e.g., any network with a private IP space), and/or any other collection of hardware interconnected by communication channels that allow sharing of resources and information. When a sandboxed application 112 and a sandbox reachable service 114 communicate in a shared network 202 common to the client device 100 and a networked device 102 when a connection is established, a client device 100 may eliminate a communication through a centralized infrastructure (e.g., a pairing server 200 which may be used only for discovery), minimize latency in the communication session (e.g., by establishing a connection between a client device 100 and a networked device 102 rather than by relaying via a pairing server 200), and improve privacy in the communication session.

FIG. 2 may encompass establishing a shared network 202 based on a bidirectional communication that does not use a relay service where it is described a shared network 202, according to one embodiment. Multiple local area networks (LANs) may share a public IP address. A client device 100 may reside on one LAN, and a sandbox reachable service 114 may reside on another LAN. A client device 100 may discover a sandbox reachable service by matching public Internet Protocol (IP) addresses. However, a sandbox reachable service 114 that is not constrained to a security sandbox 104 may have an unconstrained view (e.g., it may have access to Media Access Control addresses, Address Resolution Protocol, and/or routing tables) of a shared network 202.

A client device 100 may attempt to communicate with a sandbox reachable service 114 (e.g., by opening a Transmission Control Protocol connection and/or by sending a User Datagram Protocol datagram) without using a relay service. A shared network 202 may be established if a connection successfully handshakes, a datagram arrives, and/or the client device 100 and the sandbox reachable service 114 otherwise communicate bidirectionally without using a relay service.

FIG. 2 may also encompass establishing a shared network 202 based on a determination that a client device 100 and a sandbox reachable service 114 reside on a same LAN where it is described a shared network 202, according to one embodiment. For example, a networked device 102 may broadcast ping (e.g., using Internet Control Message Protocol) and listen for a response from a client device 100.

FIG. 2 may further encompass establishing a shared network 202 by using an address resolution protocol (e.g., ARP) where it is described a shared network 202, according to one embodiment. A sandbox reachable service 114 may determine that a client device 100 resides on a same LAN if the IP address of the client device 100 can be resolved to a LAN address using an IP-to-LAN address resolution protocol (e.g., ARP).

The shared network 202 communicates with the pairing server 200 through the WAN 204. According to one embodiment, a pairing server 200 may be a computer hardware system dedicated to enabling communication between a sandboxed application 112 and a sandbox reachable service 114. According to one embodiment, a WAN 204 may be the Internet and/or any other telecommunications network that links across metropolitan, regional, and/or national boundaries using private and/or public transports. A networked device 102 may announce an availability of a sandbox reachable service 114 across a range of public addresses such that a sandboxed application 112 communicates with the sandbox reachable service 114 in any one of the range of the public addresses. However, a range of public addresses may be known by a pairing server 200 so that the announcement of the availability of a sandbox reachable service 114 across a range of public addresses is unnecessary.

The identification data 216 exists within the sandbox reachable service 114 of FIG. 2. According to one embodiment, an identification data 216 may be a reference information associated with an application sharing a public address with a client device 100, a networked device 102, and/or a devices 206 (e.g., to define a network in which the client device 100, the networked device 102, and/or the devices 206 reside). A client device 100 may access a pairing server 200 when processing an identification data 216 associated with a sandbox reachable service 114 sharing a public address with the client device 100. A pairing server 200 may perform a discovery lookup of any device that has announced that it shares a public address associated with the client device 100. Further, a sandbox reachable service 114 may announce itself to a pairing server 200 prior to the establishment of a communication session between a sandboxed application 112 and the sandbox reachable service 114.

The GUID 208, the alphanumeric name 210, the private address pair 212, the public address pair 220, and the hardware address 222 each exist within the identification data 216 of FIG. 2. According to one embodiment, a GUID 208 may be a 128-bit reference number used by software programs to uniquely identify the location of a data object. For example, FIG. 2 may be applicable to a GUID 208 of a sandbox reachable service 114 and/or a networked device 102 where it is described a global unique ID 208. It may be preferable to have a one-to-one mapping between a GUID 208 and a networked device 102. However, in the case when a sandbox reachable service 114 may be constrained to a security sandbox 104, the sandbox reachable service 114 may have no way of determining its own IP address and/or whether it resides on a same device with other services. In this case, every sandbox reachable service 114 on the same device may have its own GUID 208.

According to one embodiment, an alphanumeric name 210 may be a “Vizio® 36″ TV,” a “living room TV,” a “bedroom printer,” and/or any other human-friendly reference name of a networked device 102. According to one embodiment, a private address pair 212 may be a private Internet Protocol (IP) address and a port number associated with an application that sends and/or receives packets. According to one embodiment, a public address pair 220 may be a public IP address and a port number 604 associated with an application that sends and/or receives packets. According to one embodiment, a hardware address 222 may be a Media Access Control (MAC) address, a physical address, Ethernet hardware address (EHA), and/or any other unique identifier assigned to network interfaces for communications on the physical network segment.

A client device 100 may process an identification data 216 associated with a sandbox reachable service 114 sharing a public address with the client device 100 and determine a private address pair 212 of the sandbox reachable service 114 based on the identification data 216. A networked device 102 may also communicate a global unique identifier 208 and/or an alphanumeric name 210 to a pairing server 200 along with a hardware address 222 associated with the networked device 102, a public address pair 220 associated with a sandbox reachable service 114 of the networked device 102, and/or a private address pair 212 associated with the sandbox reachable service 114 of the networked device 102.

FIG. 3 is an exploded view of the security sandbox 104, according to one embodiment. FIG. 3 shows a security sandbox 104, a sandboxed application 112, a same origin policy exception 300, a web page 302, a script 304, a binary executable 306, an intermediate bytecode 308, an abstract syntax tree 310, an executable application 312, a HyperText Markup Language 5 (HTML5) application 314, a Javascript® application 316, an Adobe® Flash® application 318, an Asynchronous Javascript® and XML (AJAX) application 320, a JQuery® application 324, a Microsoft® Silverlight® application 326, a hyperlink 328, a frame 330, a script 332, an image 334, a header 336, and a form 338.

The sandboxed application 112 exists within the security sandbox 104 of FIG. 3. The web page 302, the script 304, the binary executable 306, the intermediate bytecode 308, the abstract syntax tree 310, and the executable application 312 are listed as general examples of the sandboxed application 112 of FIG. 3. According to one embodiment, a web page 302 may be a document and/or an information resource that is suitable for the World Wide Web and can be accessed through a web browser and displayed on a monitor and/or a mobile device. According to one embodiment, a script 304 may be a program written for a software environment that automates the execution of tasks which could alternatively be executed one-by-one by a human operator.

According to one embodiment, a binary executable 306 may be a binary file that may include a program in machine language which is ready to be run. According to one embodiment, an intermediate bytecode 308 may be a programming language implementation of instruction set designed for efficient execution by a software interpreter. According to one embodiment, an abstract syntax tree 310 may be a tree representation of the abstract syntactic structure of source code written in a programming language. According to one embodiment, an executable application 312 may be a file that causes a computer to perform indicated tasks according to encoded instructions.

The HTML5 application 314, the Javascript® application 316, the Adobe® Flash® application 318, the Microsoft® Silverlight® application 326, the JQuery® application 324, and the AJAX application 320 are listed as specific examples of the general examples of FIG. 3. According to one embodiment, a HTML5 application 314 may be a program written in the fifth revision of the hypertext markup language standard for structuring and presenting content for the World Wide Web. According to one embodiment, a Javascript® application 316 may be a program written in a scripting language commonly implemented as part of a web browser in order to create enhanced user interfaces and dynamic websites. According to one embodiment, an Adobe® Flash® application 318 may be a program written for a multimedia and software platform used for authoring of vector graphics, animation, games and Rich Internet Applications (RIAs) which can be viewed, played, and executed in Adobe® Flash® Player.

According to one embodiment, an AJAX application 320 may be a program using a XMLHttpRequest method, a program using a Msxml2.XMLHTTP method, a program using a Microsoft.XMLHTTP method, and/or any other web program that can send data to and retrieve data from a server in the background without interfering with the display and behavior of the existing page. According to one embodiment, a JQuery® application 324 may be a program written using a multi-browser collection of pre-written Javascript® designed to simply the client-side scripting of HTML. According to one embodiment, a Microsoft® Silverlight® application 326 may be a program written in a framework for writing and running RIAs with features and purposes similar to those of Adobe® Flash®.

The same origin policy exception 300 extends horizontally below the security sandbox 104 of FIG. 3. According to one embodiment, a same origin policy exception 300 may be a cross-domain scripting technique, a cross-site scripting technique, a document.domain property, a Cross-Origin Resource Sharing (CORS), a cross-document messaging, a technique for relaxing a policy preventing access to methods and properties across pages on different sites, and/or an access control algorithm governing a policy through which a secondary authentication is required when establishing a communication between the sandboxed application 112 and the networked device 102.

A client device 100 may establish a communication session between a sandboxed application 112 and a sandbox reachable service 114 using a cross-site scripting technique of a security sandbox 104. A client device 100 may also append a header 336 of a hypertext transfer protocol to permit a networked device 102 to communicate with a sandboxed application 112 as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm. Further, a client device 100 may utilize a same origin policy exception 300 through a use of a hyperlink 328, a form 338, a script 332, a frame 330, a header 336, and/or an image 334 when establishing the connection between a sandboxed application 112 and a sandbox reachable service 114.

For example, FIG. 3 may encompass a HTML5 cross-domain scripting using postMessage where it is described HTML5 application 314. With postMessage, a calling window may call any other window in a hierarchy including those in other domains. A receiving window may set up a message listener to receive said message and can return results by posting a result message back to a calling frame. Assuming a web page residing at http://example.com/index.html:

An iframe may load the following HTML from bar.com:

<b>This iframe is located on bar.com</b>

<div id=“test”>Send me a message!</div>

<script>

document.addEventListener(“message”, function(e){

document.getElementById(“test”).textContent =

e.domain + “ said: ” + e.data;

}, false);

</script>

When a user 820 (e.g., a human agent who uses a service) clicks on the submit button, a message may be posted to the frame read from bar.com which changes “Send me a message!” to http://bar.com said: Message to send.

The hyperlink 328, the frame 330, the script 332, the image 334, the header 336, and the form 338 comprise aspects of the same origin policy exception 300 of FIG. 3. According to one embodiment, a hyperlink 328 may be a reference to data that a reader can directly follow and/or that is followed automatically. FIG. 3 may also be applicable to a hyperlink send message interface (e.g., a mechanism by which a sandboxed application 112 sends a message to a pairing server 200) where it is described a hyperlink 328 using an <A> tag to send a message to a pairing server 200 comprised of a discovery service and a relay service. The <A> tag may link to pages that are not in a same domain as a web page being viewed in a browser. As such a link may point to the pairing server 200 and arguments to be passed in a message may be encoded as key-value pairs in a uniform resource identifier (URI) query string. For example,

A sandboxed application 112 may announce to the pairing server 200. At a later time, a user 820 may visit example.com and view index.html. When the user 820 clicks on a “call f” hyperlink, a HTTP request may be sent to the pairing server 200. “f” may refer to a path to some arbitrary function and key-value pairs a=10 and/or b=bar may be arguments to that function. The pairing server 200 may receive an HTTP GET like this request generated using Google Chrome™:

GET /f?a=10&b=bar HTTP/1.1

Host: pairing_server.com

Connection: keep-alive

Referer: http://example.dom/index.html

Accept:

application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/pn

g,*/*;q=0.5

User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US)

AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3

Accept-Encoding: gzip,deflate,sdch

Accept-Language: en-US,en;q=0.8

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

The URI may not indicate to which service a message is intended. This may be interpreted by the pairing server 200 as a private broadcast meaning that a message passed via a message query interface (e.g., a mechanism to communicate a message from a pairing server 200 to a sandbox reachable service 114) is passed to all sandbox reachable services in a shared network 202. In this case, a response HTML may simply be a new web page that may include a confirmation dialog and/or a notification that a message has been sent.

According to one embodiment, a frame 330 may be a frameset, an inline frame, and/or any display of web pages and/or media elements within the same browser window. According to one embodiment, a script 332 may be a HTML tag used to define a program that may accompany an HTML document and/or be directly embedded in it. FIG. 3 may encompass a SCRIPT tag where it is described a script 332 used to contact the pairing server 200. For example, a server may deliver an http://example.com/index.html that may include a cross-site <script> tag as follows:

<html>...<head>

<script type=″text/Javascript″>

function lookup_cb(d) {

var services = d[″services″];

var slen = services.length;

var s, len;

s= ″<ul>″;

for ( var i = 0; i < slen; ++i )

s = s + ″<li>″ + services[i].name + ″</li>″;

s = s + ″</ul>″;

document.getElementById(″services″).innerHTML=s;

}

</script></head><body>

...

<div id=”services”></div>

...

<script id=″external_script″ type=″text/Javascript″></script>

<script>

document.getElementById(″external_script″).src =

″http://pairing_server.com/fling/lookup?callback=lookup_cb″;

</script></body></html>

In the example above, Javascript® may replace a source of a <script> with id “external_script” with a script downloaded from the pairing server 200. A call being made to a sandbox reachable service 114 may be embedded in a call to “lookup” with a single argument “callback=lookup_cb.” The pairing server 200 may return a script that may include a result, e.g.,

lookup_cb({

“services”: [...],

“yourip”: “69.106.59.218”,

“version”: “1.0”,

“interval”: 900

})

The result above may include a list of “services” discovered in a user's (e.g., the user of the client device 100) shared network 202. The result may be encapsulated inside a call to lookup_cb which was a callback passed in a SRC URI to an external_script <script> tag. A returned script may be automatically executed, causing lookup_cb to be called. lookup_cb may iterate over services in a result and may output them into the HTML of the web page http://example.com/index.html.

According to one embodiment, an image 334 may be a HTML tag that incorporates in-line graphics into an HTML document. FIG. 3 may also encompass an <A> tag encapsulating an <IMG> tag where it is described an image 334, thereby allowing a link to take on the appearance of a button, according to one embodiment. With Javascript® a behavior of the image may be scripted to make the button change appearance when a mouse passes over the button or when a user clicks on the button, thereby making the image behave more like a button. For example,

FIG. 3 may also be applicable to an IMG tag where it is described an image 334 used to communicate a call, according to one embodiment. For example,

<IMG SRC=“http://pairing_server.com/f?a=10&b=bar”>calling f . . . </IMG>

This example may correspond to a call ƒ with arguments a=10 and/or b=bar. The pairing server 200 sees

GET /f?a=10&b=bar HTTP/1.1

Host: ec2-204-236-247-87.compute-1.amazonaws.com:7878

Connection: keep-alive

Referer: http://dave.flingo.org/browser_behavior_tests/img_link.html

Cache-Control: max-age=0

Accept: */*

User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US)

AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63

Safari/534.3

Accept-Encoding: gzip,deflate

Accept-Language: en-US,en;q=0.8

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

A browser may expect an image to be returned by this request. As a result, an IMG send message interface may not threaten a calling web page with script injection attacks. However, it may limit what can be returned with an IMG tag. The pairing server 200 may return a valid transparent IMG with width and height set to communicate a pair. Since an IMG body has been loaded into the calling web page, the height and width of the image are immediately available to the calling page using Javascript®, e.g.,

<HTML>

<HEAD> ...

<script type=″text/Javascript″>

function loaded( ) {

var im = document.getElementById(″image″)

alert( ″image height=″ + im.height + ″ width=″ + im.width );

}

</script>

</HEAD><BODY>...

<IMG ID=″image″ SRC=”http://pairing_server.com/f?a=10&b=bar”

onload=″loaded( );″></IMG>

</BODY>

</HTML>

According to one embodiment, a header 336 may be an origin header, a referrer header, and/or any other supplemental data placed at the beginning of a block of data being stored and/or transmitted. FIG. 3 may be applicable to a passing of a URI of a web page that may include a hyperlink along with a GET request in a “referer [sic]” URI header where it is described a header 336 when a user 820 clicks on a hyperlink rendered from an <A> tag. A pairing server 200 can interpret a referer URI as an URI of a web page to be relayed to a sandbox reachable service 114 that can render web pages. For example, the following hyperlink appears in the web page http://example.com/foo.html

<A HREF=http://pairing_server.com/fling>fling this web page</A>

When a user 820 clicks on “fling this page,” the pairing server 200 may read the referer URI (e.g., associated with a client device 100) to determine that the page http://example.com/foo.html should be relayed to the receiving sandbox-reachable services.

FIG. 3 may also encompass interpreting a referer URI dependent on page content where it is described a header 336, according to one embodiment. For example, a web page 302 that may include a video may cause a reference to the video to be passed to a networked device 102. Similarly, a web page 302 that may include an audio may cause a reference to the audio to be passed to a networked device 102.

According to one embodiment, a form 338 may be a HTML tag that allows a web user to enter data that is sent to a server for processing. For example, FIG. 3 may encompass a sandboxed application 112 sending messages to a sandbox reachable service 114 via HTML FORMs where it is described a form 338. The action of a form may direct the messages via the pairing server 200. Assume a web page may reside at http://example.com/index.html and assume a relay infrastructure may run on a server with example domain “pairing_server.com.” The video to be relayed may be titled “Waxing Love.”

A hidden type may populate an HTTP POST. In this example, an URI of a resource may be passed to a pairing server 200. The pairing server 200 may treat the POST as a message to be forwarded to services. In this example, the server may see something like:

POST /fling HTTP/1.1

Host: pairing_server.com

Origin: http://example.com/index.html

User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us)

AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16

Content-Type: application/x-www-form-urlencoded

Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;

q=0.8,image/png,*/*;q=0.5

Referer: http://example.com/index.html

Accept-Language: en-us

Accept-Encoding: gzip, deflate

Content-Length: 95

Connection: keep-alive

title=Waxing+Love&description=An+example+video.&uri=http%3A%2F%2Fexampl

e.com%2Fwax.mp4

&submit=fling

The intended message may be encoded in key-value pairs of a message body. In this case a title, description, and URI and an operation “fling.”

FIG. 4 is an exploded view of the pairing server 200, according to one embodiment. FIG. 4 shows a pairing server 200, a discovery module 400, a discovery algorithm 402, a relay module 404, a relay algorithm 406, a protocols 408, and a database 422.

The discovery module 400 and the relay module 404 communicate with the database 422, and they all exist within the pairing server 200 of FIG. 4. According to one embodiment, a discovery module 400 may be a self-contained component of a pairing server 200 that detects devices and services on a network. According to one embodiment, a relay module 404 may be a self-contained component of a pairing server 200 that transmits data to an intermediate node located between a source and destination that are separated by a distance that prevents direct communications. According to one embodiment, a database 422 may be a structured collection of information.

A networked device 102 may announce a sandbox reachable service 114 to a discovery module 400. When a shared network 202 is determined to be commonly associated with a client device 100 and a networked device 102, a pairing server 200 may receive, store using a processor 108 and a memory 110, and communicate to a client device 100 a global unique identifier 208 and/or an alphanumeric name 210 in an announcement from a networked device 102 along with a hardware address 222 associated with the networked device 102, a public address pair 220 associated with a sandbox reachable service 114 of the networked device 102, and/or a private address pair 212 associated with the sandbox reachable service 114 of the networked device 102. A shared network 202 is determined to be commonly associated with a client device 100 and a networked device 102 when it is presently shared and/or was previously shared by the networked device 102 and the client device 100.

The discovery algorithm 402 exists within the discovery module 400 of FIG. 4. According to one embodiment, a discovery algorithm 402 may be a procedure for detecting devices and services on a network. A service agent module of a networked device 102 may coordinate communications with a discovery module 400 of a security sandbox 104 and/or a pairing server 200. For example, the service agent sits outside a browser or browser-like security sandbox thereby allowing it to listen on a socket. Thus, it can act as a means for services on the same device to discover one another. The service agent may also announce on behalf of service(s) local to that device.

The relay algorithm 406 exists within the relay module 404 of FIG. 4. According to one embodiment, a relay algorithm 406 may be a procedure for transmitting data to an intermediate node located between a source and destination that are separated by a distance that prevents direct communications. A service agent module of a networked device 102 may coordinate communications with a discovery module 400 of a security sandbox 104 and/or a pairing server 200. For example, the service agent sits outside a browser or browser-like security sandbox thereby allowing it to listen on a socket. Thus, it can act as a relay for messages arriving from a shared network 202.

When a client device 100 and a networked device 102 reside on networks that are incommunicable with each other comprising a firewall separation, a different network separation, a physical separation, and/or an unreachable connection separation, a sandboxed application 112 of a security sandbox 104 of the client device 100 and a sandbox reachable service 114 of the networked device 102 may communicate with each other through a relay service employed by a pairing server 200 having a discovery module 400 and a relay module 404 to facilitate a trusted communication (e.g., by guarding a GUID 208, a private IP address 808, and/or a hardware address 222 of a networked device 102 and/or a sandbox reachable service 114 from a sandboxed application 112) between the sandboxed application 112 and the sandbox reachable service 114.

The discovery module 400 and the relay module 404 can also communicate using the protocols 408 of FIG. 4. According to one embodiment, a protocols 408 may be a system of digital message formats and rules for exchanging those messages in and/or between devices sharing a network.

FIG. 5 is an exploded view of the client device 100, according to one embodiment. FIG. 5 shows a client device 100, a discovery module 500, a relay module 504, a discovery algorithm 502, a relay algorithm 506, an extension 518, a sandboxed application 112, a protocols 508, a Bonjour® protocol 510, a Simple Service Discovery Protocol (SSDP) protocol 512, a local service discovery (LSD) uTorrent® protocol 514, a local area network (LAN) based protocol 516, a multicast protocol 519, and an anycast protocol 520.

The extension 518 exists within the client device 100 of FIG. 5. According to one embodiment, an extension 518 may be a program adding the capabilities of a discovery module 500 and/or a relay module 504 to a browser. A client device 100 may extend a security sandbox 104 with a discovery algorithm 502 and a relay algorithm 506 through a discovery module 500 and a relay module 504 added to the security sandbox 104. A client device 100 may also bypass a pairing server 200 having a discovery algorithm 402 and a relay algorithm 406 when establishing a connection between a sandboxed application 112 and a sandbox reachable service 114 when the security is extended with the discovery algorithm 502 and the relay algorithm 506 through the discovery module 500 and the relay module 504 added to a security sandbox 104.

The discovery module 500, the relay module 504, and the sandboxed application 112 exist within the extension 518 of FIG. 5. The discovery module 500 communicates with the relay module 504 of FIG. 5. According to one embodiment, a discovery module 500 may be a self-contained component of a client device 100 that detects devices and services on a network. According to one embodiment, a relay module 504 may be a self-contained component of a client device 100 that transmits data to an intermediate node located between a source and destination that are separated by a distance that prevents direct communications. A networked device 102 may announce a sandbox reachable service 114 to a discovery module 500. A networked device 102 may also automatically instantiate a communication between a sandbox reachable service 114 of the networked device 102 and a client device 100 when a relay module 504 sends a request from a sandboxed application 112 of the client device 100 to the sandbox reachable service 114.

The discovery algorithm 502 exists within the discovery module 500 of FIG. 5. A client device 100 may apply a discovery algorithm 502 of a security sandbox 104 to determine that a networked device 102 having a sandbox reachable service 114 communicates in a shared network 202 common to the client device 100 and the networked device 102.

The relay algorithm 506 exists within the relay module 504 of FIG. 5. A client device 100 may apply a relay algorithm 506 of a security sandbox 104 to establish a connection between a sandboxed application 112 and a sandbox reachable service 114 of a networked device 102. A client device 100 may utilize a WebSocket (e.g., a web technology providing full-duplex communications channels over a single Transmission Control Protocol connection) and/or a long polling service message query interface to reduce a latency of message delivery during a trusted communication between a sandboxed application 112 and a sandbox reachable service 114. A client device 100 may also optimize a polling period between polling such that it is less than a timeout period of a session through the relay service. A client device 100 may initiate a relay service through a series of web pages where information is communicated using a hyperlink 328 that points at a pairing server 200, and/or a form 338 having a confirmation dialog that is submitted back to the pairing server 200. A global unique identifier 208 (e.g., of a sandbox reachable service 114) may be masked through a pairing server 200 when a confirmation dialog is served from the pairing server 200.

The discovery algorithm 502 and the relay algorithm 506 can communicate using the protocols 508 of FIG. 5. The Bonjour® protocol 510, the SSDP protocol 512, the LSD uTorrent® protocol 514, the LAN-based protocol 516, the multicast protocol 519, and the anycast protocol 520 exist within the protocols 508 of FIG. 5. According to one embodiment, a Bonjour® protocol 510 may be a system of technologies including service discovery, address assignment, and hostname resolution developed by Apple®. According to one embodiment, a SSDP protocol 512 may be a network protocol based on the Internet Protocol Suite for advertisement and discovery of network services and presence information that is accomplished without assistance of server-based configuration mechanisms and without special static configuration of a network host. According to one embodiment, a LSD uTorrent® protocol 514 may be an extension to the BitTorrent® file distribution system that is designed to support the discovery of local BitTorrent® peers, aiming to minimize traffic through an Internet service provider's (ISP) channel and minimize use of higher-bandwidth LAN while implemented in a client with a small memory footprint. According to one embodiment, a LAN-based protocol 516 may be a system of broadcast-based local area network discovery. According to one embodiment, a multicast protocol 519 may be a system of delivering information simultaneously to a group of destination devices in a single transmission from a source. According to one embodiment, an anycast protocol 520 may be a system of routing datagrams from a single sender to the topologically nearest node in a group of potential receivers, though it may be sent to several nodes, all identified by the same destination address.

A discovery algorithm 502 may utilize a protocols 508 comprising a Bonjour® protocol 510, a SSDP protocol 512, a LSD uTorrent® protocol 514, a multicast protocol 519, an anycast protocol 520, and/or another LAN-based protocol 516 that discovers services in a LAN based on a broadcast from any one of an operating system service, a security sandbox 104, a client device 100, a sandbox reachable service 114, and a networked device 102.

FIG. 6 is a table of example network information stored in a database 422 of a pairing server 200, according to one embodiment. FIG. 6 shows a GUID 208, an alphanumeric name 210, a network 600, a service 601, a Network Address Translator (NAT) 602, a port number 604, an IP address 606, and a table 650. The GUID 208, the alphanumeric name 210, the network 600, the service 601, the NAT 602, the port number 604, and the IP address 606 are headings for each column of a table 650 of FIG. 6.

According to one embodiment, a network 600 may be a collection of hardware interconnected by communication channels that allow sharing of resources and information. According to one embodiment, a service 601 may be a description and/or a name of a service provided by a device. According to one embodiment, a NAT 602 may be an indication of whether or not a NAT device is present on a network 600. According to one embodiment, a port number 604 may be a 16-bit reference number for a process-specific software construct serving as a communications endpoint in a computer's host operating system. According to one embodiment, an IP address 606 may be a numerical label assigned to each device participating in a computer network that uses the Internet Protocol for communication. According to one embodiment, a table 650 may be a set of data elements that is organized using a model of vertical columns which are identified by names and horizontal rows. A sandbox reachable service 114 may communicate a GUID 208 and/or an alphanumeric name 210 to a pairing server 200 along with an IP address 606 and/or a port number 604 of the sandbox reachable service 114.

FIG. 7 is a block diagram of a method by which a security sandbox 104 can communicate with a sandbox reachable service 114 that previously operated on a shared network 202, according to one embodiment. FIG. 7 shows a client device 100, a storage 109, a remote access token 702, a private IP address 704, and a hardware address 222. The storage 109 exists within the client device 100 of FIG. 7. The remote access token 702 exists within the storage 109 of FIG. 7. According to one embodiment, a remote access token 702 may be an object encapsulating a security descriptor of a process so that a client device 100 and a networked device 102 that previously established a communication session automatically recognize each other. A cookie associated with a security sandbox 104 may be used to store a remote access token 702 on a storage 109 (e.g., Web storage, HTML5 storage) of a client device 100. A client device 100 can communicate with a sandbox reachable service 114 that previously operated on a common shared network 202 through a remote access token 702.

The private IP address 704 and the hardware address 222 comprise aspects of the remote access token 702 of FIG. 7. According to one embodiment, a private IP address 704 may be an IP address of a node on a private network that may not be used to route packets on the public Internet. A remote access token 702 may identify a set of communicable private Internet Protocol (IP) address (e.g., the private ip address 704) and/or hardware addresses (e.g., the hardware address 222) associated with a sandbox reachable service 114 that previously operated on a common shared network 202 with a client device 100. For example, FIG. 7 may encompass a preference for associating a device with a hardware address 222 where it is described a hardware address 222. A private IP address 704 may change as devices move between networks. However, a hardware address 222 may be a stable, long-term pseudonym for a device and thus may serve a good value from which to derive a remote access token 702.

FIG. 8 is a schematic diagram of a private network 800 and a private network 802 communicating over the public Internet via a NAT device 804 and a NAT device 806, according to one embodiment. FIG. 8 shows a client device 100, a networked device 102, a pairing server 200, a private network 800, a private network 802, a NAT device 804, a NAT device 806, a private IP address 808, a private IP address 810, a public IP address 812, a public IP address 814, a tablet device 816, a printer 818, and a user 820.

The private network 800 and the private network 802 communicate bidirectionally through the pairing server 200 of FIG. 8. According to one embodiment, a private network 800 may be a home network and/or any other network with private IP space that may be behind a NAT device 804. According to one embodiment, a private network 802 may be an office network and/or any other network with private IP space that may be behind a NAT device 806. A client device 100 (e.g., laptop) and a networked device 102 (e.g., television) may reside on networks that are incommunicable with each other comprising a firewall separation, a different network separation, a physical separation, and/or an unreachable connection separation. A sandboxed application 112 of a security sandbox 104 of the client device 100 and a sandbox reachable service 114 of the networked device 102 may communicate with each other through a relay service employed by a pairing server 200 having the discovery module and the relay module to facilitate a trusted communication between the sandboxed application 112 and the sandbox reachable service 114.

The NAT device 804, the networked device 102, and the tablet device 816 are all interconnected and exist within the private network 800 of FIG. 8. According to one embodiment, a NAT device 804 may be a device for modifying IP address information in IP packet headers while in transit across a traffic routing device. According to one embodiment, a tablet device 816 may be a one-piece mobile computer, primarily operated by touchscreen and/or an onscreen virtual keyboard. A NAT device 804 may be coupled with a network on which a networked device 102 operates.

The NAT device 806, the client device 100, and the printer 818 are all interconnected and exist within the private network 802 of FIG. 8. According to one embodiment, a NAT device 806 may be a device for modifying IP address information in IP packet headers while in transit across a traffic routing device. According to one embodiment, a printer 818 may be a peripheral device which produces a representation of an electronic document on physical media. A NAT device 806 may be coupled with a network on which a client device 100 operates.

The NAT device 804 connects to the pairing server 200 through the public IP address 812 of FIG. 8. The NAT device 804 connects to the networked device 102 through the private IP address 808 of the networked device 102 of FIG. 8. According to one embodiment, a public IP address 812 may be an IP address of a private network 800 that may be used to route packets on the public Internet. According to one embodiment, a private IP address 808 may be an IP address of a networked device 102 on a private network 800. A trusted communication may be facilitated in a manner such that a sandboxed application 112 never learns a private IP address 808 and/or a hardware address 222 of a networked device 102 when a NAT device 804 may translate a private IP address 808 of a networked device 102 to a public IP address 812 visible to a sandboxed application 112.

The NAT device 806 connects to the pairing server 200 through the public IP address 814 of FIG. 8. The NAT device 806 connects to the client device 100 through the private IP address 810 of the client device 100 of FIG. 8. According to one embodiment, a public IP address 814 may be an IP address of a private network 802 that may be used to route packets on the public Internet. According to one embodiment, a private IP address 810 may be an IP address of a networked device 102 on a private network 802. A trusted communication may be facilitated in a manner such that a sandboxed application 112 never learns a private IP address 808 and/or a hardware address 222 of a networked device 102 when a NAT device 806 may receive communications from a public IP address 812 of a private network 800 on which a sandbox reachable service 114 operates.

For example, FIG. 8 may encompass a sandboxed application 112 being constrained to know nothing but a description and/or name of a service (e.g., no private IP address 808, no hardware address 222, no GUID 208) where it is described a private IP address 808.

FIG. 8 may also be applicable to a sandboxed application 112 being constrained to know nothing at all about who receives a communication (e.g., no private IP address 808, no hardware address 222, no GUID 208, no description and/or name of a service) where it is described a private IP address 808, according to one embodiment. For example, a sandboxed application 112 may include a hyperlink 328 to a pairing server 200 in which the hyperlink 328 may specify a message but no recipient http://flingo.tv/fling/a?url=url of media to be played. A pairing server 200 may disambiguate an intended recipient (e.g., by returning a form 338 to a user 820 in which the user 820 may select a sandbox reachable service 114). A returned form 338 may execute in a security sandbox 104 associated with a domain of a pairing server 200 which may be different from a security sandbox 104 of a sandboxed application 112.

The user 820 exists within the private network 802 of FIG. 8. According to one embodiment, a user 820 may be a human and/or software agent who uses a computer and/or network service.

In another aspect, a method of a client device includes constraining an executable environment in a security sandbox. The method also includes executing a sandboxed application in the executable environment using a processor and a memory. Further, the method includes automatically instantiating a connection between the sandboxed application and a sandbox reachable service of a networked media device.

The method may include processing an identification data associated with the sandbox reachable service sharing a public address with the client device. The method may also include determining a private address pair of the sandbox reachable service based on the identification data. Additionally, the method may include establishing a communication session between the sandboxed application and the sandbox reachable service using a cross-site scripting technique of the security sandbox. Further, the method may include appending a header of a hypertext transfer protocol to permit the networked media device to communicate with the sandboxed application as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm. The header may be either one of a origin header when the CORS algorithm is applied and a referrer header in an alternate algorithm.

The method may further include accessing a pairing server when processing the identification data associated with the sandbox reachable service sharing the public address with the client device. The pairing server may perform a discovery lookup of any device that has announced that it shares the public address associated with the client device. The sandbox reachable service may announce itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service. The sandbox reachable service may also announce its availability across a range of public addresses such that the sandboxed application communicates with the sandbox reachable service in any one of the range of the public addresses. However, the range of public addresses may be known by the pairing server so that the announcement of the availability of the sandbox reachable service across the range of public addresses is unnecessary. The sandbox reachable service may communicate a global unique identifier and/or an alphanumeric name to the pairing server along with the private address pair of the sandbox reachable service. The private address pair may include a private IP address and a port number associated with the sandbox reachable service.

The method may further include eliminating a communication through a centralized infrastructure when the sandboxed application and the sandbox reachable service communicate in a shared network common to the client device and the networked media device when the connection is established. The shared network may be a local area network, a multicast network, an anycast network, and/or a multilan network. The method may also include minimizing a latency in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device when the connection is established. Further, the method may include improving privacy in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device when the connection is established.

The sandboxed application may be a web page, a script, a binary executable, an intermediate bytecode, an abstract syntax tree, and/or an executable application in the security sandbox. The sandboxed application may comprise a markup language application such as a HyperText Markup Language 5 (HTML5) application, a Javascript® application, an Adobe® Flash® application, a Microsoft® Silverlight® application, a JQuery® application, and/or an Asynchronous Javascript® and a XML (AJAX) application. An access control algorithm may govern a policy through which a secondary authentication is required when establishing a communication between the sandboxed application and the networked media device. The method may include utilizing an exception to a same origin policy through a use of a hyperlink, a form, the script, a frame, a header, and an image when establishing the connection between the sandboxed application and the sandbox reachable service.

The method may include extending the security sandbox with a discovery algorithm and a relay algorithm through a discovery module and a relay module added to the security sandbox. The method may also include bypassing a pairing server having the discovery algorithm and the relay algorithm when establishing the connection between the sandboxed application and the sandbox reachable service when the security sandbox is extended with the discovery algorithm and the relay algorithm through the discovery module and the relay module added to the security sandbox.

The method may further include applying the discovery algorithm of the security sandbox to determine that the networked media device having the sandbox reachable service communicates in a shared network common to the client device and the networked media device. The method may also include applying the relay algorithm of the security sandbox to establish the connection between the sandboxed application and the sandbox reachable service of the networked media device. The discovery algorithm may utilize a protocol comprising a Bonjour® protocol, a SSDP protocol, a LSD uTorrent® protocol, a multicast protocol, an anycast protocol, and/or another Local Area Network (LAN) based protocol that discovers services in a LAN based on a broadcast from any one of an operating system service, the security sandbox, the client device, the sandbox reachable service, and the networked media device.

A cookie associated with the security sandbox may be used to store a remote access token on a storage of the client device. The remote access token may identify a set of communicable private Internet Protocol (IP) addresses and/or hardware addresses associated with sandbox reachable services that previously operated on a common shared network with the client device. The client device may communicate with the sandbox reachable services that previously operated on the common shared network through the remote access token.

The client device and the networked media device may reside on networks that are incommunicable with each other comprising a firewall separation, a different network separation, a physical separation, and/or an unreachable connection separation. The sandboxed application of the security sandbox of the client device and the sandbox reachable service of the networked media device may communicate with each other through a relay service employed by a pairing server having a discovery module and a relay module to facilitate a trusted communication between the sandboxed application and the sandbox reachable service.

The trusted communication may be facilitated in a manner such that the sandboxed application never learns a private IP address and/or a hardware address of the networked media device. This may occur when a first Network Address Translator (NAT) device receives communications from a public IP address of a different network on which the sandbox reachable service operates, and a second NAT device translates the private IP address of the networked media device to the public IP address visible to the sandboxed application. The first NAT device may be coupled with a network on which the client device operates. The second NAT device may be coupled with the different network on which the networked media device operates.

The networked media device may comprise a number of sandbox reachable applications including the sandbox reachable application. A service agent module of the networked media device may coordinate communications with the discovery module of the security sandbox and/or the pairing server. The security sandbox may be an operating system on which the sandboxed application is hosted and/or a browser application of the operating system. The networked media device may be a television, a projection screen, a multimedia display, a touchscreen display, an audio device, and/or a multidimensional visual presentation device.

The method may include utilizing a WebSocket and/or a long polling service message query interface to reduce a latency of message delivery during the trusted communication between the sandboxed application and the sandbox reachable service. The method may also include optimizing a polling period between polling such that it is less than a timeout period of a session through the relay service. The method may further include initiating the relay service through a series of web pages where information is communicated using hyperlinks that point at the pairing server, and/or a form having a confirmation dialog that is submitted back to the pairing server. A global unique identifier may be masked through the pairing server when the confirmation dialog is served from the pairing server.

In one embodiment, a method of a networked device includes announcing a sandbox reachable service of the networked device to a discovery module using a processor and memory. The method also includes automatically instantiating a communication between the sandbox reachable service of the networked device and a client device when a relay module sends a request from a sandboxed application of the client device to the sandbox reachable service.

In yet another embodiment, a system includes a networked device to announce a sandbox reachable service of the networked device to a discovery module using a processor and memory. The system also includes a client device to constrain an executable environment in a security sandbox, to execute a sandboxed application in the security sandbox, and to automatically instantiate a connection between the sandboxed application and the sandbox reachable service of the networked device.

In still another embodiment, a method of a pairing server includes receiving, storing using a processor and a memory, and communicating to a client device a global unique identifier and/or an alphanumeric name in an announcement from a networked device along with a hardware address associated with the networked device, a public address pair associated with a sandbox reachable service of the networked device, and/or a private address pair associated with the sandbox reachable service of the networked device when a shared network is determined to be commonly associated with the client device and the networked device. The shared network is a local area network, a multicast network, an anycast network, and/or a multilan network.

For example, Jane may watch a movie and/or access an application through her mobile device while sitting on a couch in her living room. Jane may wish to automatically display the movie and/or application of a big screen television in front of her couch. Jane may use a gesture to transport the movie and/or application to the big screen television. For example, Jane may ‘fling’ (or flick) the screen on her mobile device in which the movie and/or application is running in an upward motion, and instantly transport the movie and/or application onto her big screen television. In an alternate embodiment, the big screen television may automatically detect that Jane is playing the movie and/or running the application on her mobile device and automatically launch the movie (in its current play state) and/or run the application on the big screen television after detection (without requiring a fling or flick haptic gesture by Jane).

While Jane is playing the movie and/or running the application on the big screen television, Jane may see advertisements that relate to exactly what she is currently watching on the big screen television on her mobile device through the capture infrastructure described herein. Conversely, if Jane were playing the movie and/or running the application on her mobile device, she could see advertisements directly related to the activities she is currently doing on her big screen television (and or simultaneously on the mobile device). This is made possible through the various methods and techniques described herein, particularly with respect to a centralized capture infrastructure that annotates an audio-visual stream with meta-data.

Although the present embodiments have been described with reference to specific example embodiments, it will be evident that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the various embodiments. For example, the various devices and modules described herein may be enabled and operated using hardware circuitry (e.g., CMOS based logic circuitry), firmware, software or any combination of hardware, firmware, and/or software (e.g., embodied in a machine readable medium). For example, the various electrical structure and methods may be embodied using transistors, logic gates, and/or electrical circuits (e.g., application specific integrated (ASIC) circuitry and/or Digital Signal Processor (DSP) circuitry).

In addition, it will be appreciated that the various operations, processes, and/or methods disclosed herein may be embodied in a machine-readable medium and/or a machine accessible medium compatible with a data processing system (e.g., a computer device). Accordingly, the specification and drawings are to be regarded in an illustrative in rather than a restrictive sense.

Claims

1. A method of a client device comprising: applying an automatic content recognition algorithm to determine a content identifier of an audio-visual data; andassociating the content identifier with an advertisement data based on a semantic correlation between a meta-data of the advertisement provided by a content provider and the content identifier,wherein a capture infrastructure annotates the audio-visual data with at least one of a brand name and a product name by comparing entries in the master database with at least one of a closed captioning data of the audio-visual data and through an application of an optical character recognition algorithm in the audio-visual data;wherein the content identifier is at least one of a music identification, an object identification, a facial identification, and a voice identification,wherein a minimal functionality comprising accessing at least one of a tuner and a stream decoder that identifies at least one of a channel and a content is found in the networked media device,wherein the networked media device produces at least one of an audio fingerprint and a video fingerprint that are communicated with the capture infrastructure,wherein the capture infrastructure compares at least one of the audio fingerprint and the video fingerprint with a master database,wherein the capture infrastructure annotates the audio-visual data with a logo name by comparing entries in the master database with a logo data of the audio-visual data identified using a logo detection algorithm,wherein the capture infrastructure automatically divides the audio-visual data into a series of scenes based on a sematic grouping of actions in the audio-visual data,wherein the audio-visual data is analyzed in advance of a broadcast to determine content identifiers associated with each commercial in the audio-visual data such that advertisements are pre-inserted into the audio-visual data prior to broadcast,wherein the capture infrastructure applies a time-order algorithm to automatically match advertisements to the audio-visual data when a correlation pattern is identified by the capture infrastructure with other audio-visual content previously analyzed,wherein the capture infrastructure includes a buffer that is saved to a persistent storage and for which a label is generated to facilitate identification of reoccurring sequences,wherein a post processing operation is at least one of automated through a post-processing algorithm and a crowd-sourced operation using a plurality of users in which a turing test is applied to determine a veracity of an input,wherein a device pairing algorithm is used in which a cookie data associated with a web page visited by the user stored on a browser on the client device is paired with the networked media device when the client device is communicatively coupled with the networked media device,wherein a transitive public IP matching algorithm is utilized in which at least one of the client device and the networked media device communicates each public IP address with any paired entity to the capture infrastructure, andwherein a tag that is unconstrained from a same-origin policy is used to automatically load the advertisement in the browser, wherein the tag is at least one of an image tag, a frame, a iframe, and a script tag.
2. The method of claim 1 further comprising: referencing an additional metadata comprising at least one of the content identifier and the advertisement based on a video processing algorithm,wherein the additional meta data is at least one of a title, a description, a thumbnail, a name of an individual, and a historical data, andwherein the additional metadata is determined from a browser history captured from the client device based on a capture policy, and correlating a relevance of the browser history with at least one of the content identifier and the advertisement;constraining an executable environment in a security sandbox;executing a sandboxed application in the executable environment using a processor and a memory;automatically instantiating a connection between the sandboxed application and the unannounced device associated with the networked media device based on the determination that the internet protocol address of the port from the unannounced device is associated with the networked media device;processing an identification data associated with the sandbox reachable service sharing a public address with the client device;determining a private address pair of the sandbox reachable service based on the identification data;establishing a communication session between the sandboxed application and the sandbox reachable service using a cross-site scripting technique of the security sandbox; andappending a header of a hypertext transfer protocol to permit the networked media device to communicate with the sandboxed application as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm, wherein the header is either one of a origin header when the CORS algorithm is applied and a referrer header in an alternate algorithm.
3. The method of claim 2: wherein the client device and the networked media device reside on networks that are incommunicable with each other comprising at least one of a firewall separation, a different network separation, a physical separation, an unreachable connection separation, andwherein the sandboxed application of the security sandbox of the client device and the sandbox reachable service of the networked media device communicate with each other through a relay service employed by a pairing server having a discovery module and a relay module to facilitate a trusted communication between the sandboxed application and the sandbox reachable service.
4. The method of claim 3: wherein the trusted communication is facilitated in a manner such that the sandboxed application never learns at least one of a private IP address and a hardware address of the networked media device when:a first Network Address Translator (NAT) device coupled with a network on which the client device operates to receive communications from a public IP address of a different network on which the sandbox reachable service operates, andwherein a second NAT device coupled with the different network on which the networked media device operates to translate the private IP address of the networked media device to the public IP address visible to the sandboxed application.
5. The method of claim 4: wherein the networked media device comprises a plurality of sandbox reachable applications including the sandbox reachable application, and wherein a service agent module of the networked media device coordinates communications with the discovery module of at least one of the security sandbox and the pairing server,wherein the security sandbox is at least one of an operating system on which the sandboxed application is hosted and a browser application of the operating system, andwherein the networked media device is at least one of a television, a projection screen, a multimedia display, a touchscreen display, an audio device, and a multidimensional visual presentation device.
6. The method of claim 5 further comprising: utilizing at least one of a WebSocket and a long polling service message query interface to reduce a latency of message delivery during the trusted communication between the sandboxed application and the sandbox reachable service; andoptimizing a polling period between polling such that it is less than a timeout period of a session through the relay service.
7. The method of claim 6 further comprising: initiating the relay service through at least one of a series of web pages where information is communicated using hyperlinks that point at the pairing server,and a form having a confirmation dialog that is submitted back to the pairing server, andwherein a global unique identifier is masked through the pairing server when the confirmation dialog is served from the pairing server.
8. The method of claim 2 further comprising: extending the security sandbox with a discovery algorithm and a relay algorithm through a discovery module and a relay module added to the security sandbox;and bypassing a pairing server having the discovery algorithm and the relay algorithm when establishing the connection between the sandboxed application and the sandbox reachable service when the security sandbox is extended with the discovery algorithm and the relay algorithm through the discovery module andthe relay module added to the security sandbox.
9. The method of claim 8 further comprising: applying the discovery algorithm of the security sandbox to determine that the networked media device having the sandbox reachable service communicates in a shared network common to the client device and the networked media device;and applying the relay algorithm of the security sandbox to establish the connection between the sandboxed application and the sandbox reachable service of the networked media device.
10. The method of claim 9: wherein the discovery algorithm utilizes a protocol comprising at least one of a Bonjour* protocol, a SSDP protocol, a LSD uTorrent® protocol, a multicast protocol, an anycast protocol, and another Local Area Network (LAN) based protocol that discovers services in a LAN based on a broadcast from any one of an operating system service, the security sandbox, the client device, the sandbox reachable service, and the networked media device.
11. The method of claim 2 further comprising: wherein the sandboxed application is at least one of a web page, a script, a binary executable, an intermediate bytecode, an abstract syntax tree, and an executable application in the security sandbox,wherein the sandboxed application comprises at least one of a markup language application such as a HyperText Markup Language 5 (HTML5) application, a Javascript® application, an Adobe® Flash® application, a Microsoft® Silverlight® application, a JQuery® application, and an Asynchronous Javascript® and a XML (AJAX) application, andwherein an access control algorithm governs a policy through which a secondary authentication is required when establishing a communication between the sandboxed application and the networked media device.
12. The method of claim 11 further comprising: utilizing an exception to a same origin policy through a use of at least one of a hyperlink, a form, the script, a frame, a header, and an image when establishing the connection between the sandboxed application and the sandbox reachable service.
13. The method of claim 2 further comprising: accessing a paring server when processing the identification data associated with the sandbox reachable service sharing the public address with the client device,wherein the pairing server performs a discovery lookup of any device that has announced that it shares the public address associated with the client device,and wherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service.
14. The method of claim 13 further comprising at least one of: wherein the sandbox reachable service announces an availability of the sandbox reachable service across a range of public addresses such that the sandboxed application communicates with the sandbox reachable service in any one of the range of the public addresses,wherein the range of public addresses is known by the pairing server so that the announcement of the availability of the sandbox reachable service across the range of public addresses is unnecessary,wherein the sandbox reachable service communicates at least one of a global unique identifier and an alphanumeric name to the pairing server along with the private address pair of the sandbox reachable service, andwherein the private address pair includes a private IP address and a port number associated with the sandbox reachable service.
15. The method of claim 2 further comprising: eliminating a communication through a centralized infrastructure when the sandboxed application and the sandbox reachable service communicate in a shared network common to the client device and the networked media device when the connection is established, wherein the shared network is at least one of the local area network, a multicast network, an anycast network, and a multilan network;minimizing a latency in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device when the connection is established; andimproving privacy in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device when the connection is established.
16. The method of claim 2: wherein a cookie associated with the security sandbox is used to store a remote access token on a storage of the client device,wherein the remote access token identifies at least one of a set of communicable private Internet Protocol (IP) addresses and hardware addresses associated with sandbox reachable services that previously operated on a common shared network with the client device, andwherein the client device can communicate with the sandbox reachable services that previously operated on the common shared network through the remote access token.
17. A method of a networked device comprising: applying an automatic content recognition algorithm to determine a content identifier of an audio-visual data; andassociating the content identifier with an advertisement data based on a semantic correlation between a meta-data of the advertisement provided by a content provider and the content identifier,wherein a capture infrastructure annotates the audio-visual data with at least one of a brand name and a product name by comparing entries in the master database with at least one of a closed captioning data of the audio-visual data and through an application of an optical character recognition algorithm in the audio-visual data;wherein the content identifier is at least one of a music identification, an object identification, a facial identification, and a voice identification,wherein a minimal functionality comprising accessing at least one of a tuner and a stream decoder that identifies at least one of a channel and a content is found in the networked device,wherein the networked device produces at least one of an audio fingerprint and a video fingerprint that are communicated with the capture infrastructure,wherein the capture infrastructure compares at least one of the audio fingerprint and the video fingerprint with a master database,wherein the capture infrastructure annotates the audio-visual data with a logo name by comparing entries in the master database with a logo data of the audio-visual data identified using a logo detection algorithm,wherein the capture infrastructure automatically divides the audio-visual data into a series of scenes based on a sematic grouping of actions m the audio-visual data,wherein the audio-visual data is analyzed in advance of a broadcast to determine content identifiers associated with each commercial in the audio-visual data such that advertisements are pre-inserted into the audio-visual data prior to broadcast,wherein the capture infrastructure applies a time-order algorithm to automatically match advertisements to the audio-visual data when a correlation pattern is identified by the capture infrastructure with other audio-visual content previously analyzed,wherein the capture infrastructure includes a buffer that is saved to a persistent storage and for which a label is generated to facilitate identification of reoccurring sequences,wherein a post processing operation is at least one of automated through a post-processing algorithm and a crowd-sourced operation using a plurality of users in which a turing test is applied to determine a veracity of an input,wherein a device pairing algorithm is used in which a cookie data associated with a web page visited by the user stored on a browser on a client device is paired with the networked device when the client device is communicatively coupled with the networked device,wherein a transitive public IP matching algorithm is utilized in which at least one of the client device and the networked device communicates each public IP address with any paired entity to the capture infrastructure, andwherein a tag that is unconstrained from a same-origin policy is used to automatically load the advertisement in the browser, wherein the tag is at least one of an image tag, a frame, a iframe, and a script tag.
18. The method of claim 17 further comprising: accessing a pairing server when processing an identification data associated with a sandbox reachable service of the networked device that shares a public address with a client device,wherein the pairing server performs a discovery lookup of any device that has announced that it shares the public address associated with the client device,and wherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service,appending a header of a hypertext transfer protocol to permit the networked device to communicate with the sandboxed application as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm, wherein the header is either one of a origin header when the CORS algorithm is applied and a referrer header in an alternate algorithm, andwherein the client device to operate in at least one manner such that the client device:to process an identification data associated with the sandbox reachable service sharing a public address with the client device;to determine a private address pair of the sandbox reachable service based on the identification data; andto establish a communication session between the sandboxed application and the sandbox reachable service using a cross-site scripting technique of a security sandbox.
19. The method of claim 18 wherein the client device: to extend a security sandbox with a discovery algorithm and a relay algorithm through the discovery module and the relay module added to the security sandbox; andto bypass a paring server having the discovery algorithm and the relay algorithm when establishing the communication between the sandboxed application and the sandbox reachable service when the security is extended with the discovery algorithm and the relay algorithm through the discovery module and the relay module added to the security sandbox.
20. The method of claim 19 wherein the client device: to apply the discovery algorithm of the security sandbox to determine that the networked device having the sandbox reachable service communicates in a shared network common to the client device and the networked device; andto apply the relay algorithm of the security sandbox to establish the communication between the sandboxed application and the sandbox reachable service of the networked device.
21. The method of claim 20: wherein the discovery algorithm utilizes a protocol comprising at least one of a Bonjour* protocol, a SSDP protocol, a LSD uTorrent® protocol, a multicast protocol, an anycast protocol, and another Local Area Network (LAN) based protocol that discovers services in a LAN based on a broadcast from any one of an operating system service, the security sandbox, the client device, the sandbox reachable service, and the networked device.
22. The method of claim 21: wherein a cookie associated with the security sandbox is used to store a remote access token on a storage of the client device,wherein the remote access token identifies at least one of a set of communicable private Internet Protocol (IP) addresses and hardware addresses associated with sandbox reachable services that previously operated on a common shared network with the client device, andwherein the client device can communicate with the sandbox reachable services that previously operated on the common shared network through the remote access token.
23. The method of claim 22: wherein the client device and the networked device reside on networks that are incommunicable with each other comprising at least one of a firewall separation, a different network separation, a physical separation, an unreachable connection separation, andwherein the sandboxed application of the security sandbox of the client device and the sandbox reachable service of the networked device communicate with each other through a relay service employed by the pairing server having the discovery module and the relay module to facilitate a trusted communication between the sandboxed application and the sandbox reachable service.
24. The method of claim 23: wherein the trusted communication is facilitated in a manner such that the sandboxed application never learns at least one of a private IP address and a hardware address of the networked device when:a first Network Address Translator (NAT) device coupled with a network on which the client device operates to receives communications from a public IP address of a different network on which the sandbox reachable service operates, andwherein a second NAT device coupled with the different network on which the networked device operates to translates the private IP address of the networked device to the public IP address visible to the sandboxed application.
25. The method of claim 24: wherein the networked device comprises a plurality of sandbox reachable applications including the sandbox reachable application, and wherein a service agent module of the networked device coordinates communications with the discovery module of at least one of the security sandbox and the pairing server,wherein the security sandbox is at least one of an operating system on which the sandboxed application is hosted and a browser application of the operating system, andwherein the networked device is at least one of a television, a projection screen, a multimedia display, a touchscreen display, an audio device, a weather measurement device, a traffic monitoring device, a status update device, a global positioning device, a geospatial estimation device, a tracking device, a bidirectional communication device, a unicast device, a broadcast device, anda multidimensional visual presentation device.
26. The method of claim 25 wherein the client device: to utilize at least one of a WebSocket and a long polling service message query interface to reduce a latency of message delivery during the trusted communication between the sandboxed application and the sandbox reachable service; andto optimize a polling period between polling such that it is less than a timeout period of a session through the relay service.
27. The method of claim 26 wherein the client device: to initiate the relay service through at least one of a series of web pages where information is communicated using hyperlinks that point at the pairing server,and a form having a confirmation dialog that is submitted back to the pairing server, andwherein a global unique identifier is masked through the pairing server when the confirmation dialog is served from the pairing server.
28. The method of claim 18 wherein the client device: to access a pairing server when processing the identification data associated with the sandbox reachable service sharing the public address with the client device,wherein the pairing server performs a discovery lookup of any devices that have announced that they share the public address associated with the client device, andwherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service.
29. The method of claim 28 further comprising at least one of: announcing an availability of the sandbox reachable service across a range of public addresses such that the sandboxed application communicates with the sandbox reachable service in any one of the range of the public addresses; andcommunicating at least one of a global unique identifier, a hardware address, and an alphanumeric name to the pairing server along with the private address pair of the sandbox reachable service, andwherein the private address pair includes a private IP address and a port number associated with the sandbox reachable service.
30. The method of claim 28 further comprising: eliminating a communication through a centralized infrastructure when the sandboxed application and the sandbox reachable service communicate in a shared network common to the client device and the networked device when the communication is established, wherein the shared network is at least one of a local area network, a multicast network, an anycast network, and a multilan network;minimizing a latency in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked device when the communication is established; andimproving privacy in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked device when the communication is established.
31. The method of claim 18: wherein the sandboxed application is at least one of a web page, a script, a binary executable, an intermediate bytecode, an abstract syntax tree, and an executable application in a security sandbox,wherein the sandboxed application comprises at least one of a markup language application such as a HyperText Markup Language 5 (HTML5) application, a Javascript® application, an Adobe® Flash® application, a Microsoft® Silverlight® application, a JQuery® application, and an Asynchronous Javascript® and a XML (AJAX) application, andwherein an access control algorithm governs a policy through which a secondary authentication is required when establishing a communication between the sandboxed application and the networked device.
32. The method of claim 31 wherein the client device: to utilize an exception to a same origin policy through a use of at least one of a hyperlink, a form, the script, a frame, a header, and an image when establishing the communication between the sandboxed application and the sandbox reachable service.
33. A system comprising: a networked device and a client device to apply an automatic content recognition algorithm to determine a content identifier of an audio-visual data and to associate the content identifier with an advertisement data based on a semantic correlation between a meta-data of the advertisement provided by a content provider and the content identifier; anda capture infrastructure to annotate the audio-visual data with at least one of a brand name and a product name by comparing entries in the master database with at least one of a closed captioning data of the audio-visual data and through an application of an optical character recognition algorithm in the audio-visual data;wherein the content identifier is at least one of a music identification, an object identification, a facial identification, and a voice identification,wherein a minimal functionality comprising accessing at least one of a tuner and a stream decoder that identifies at least one of a channel and a content is found in the networked media device,wherein the networked media device produces at least one of an audio fingerprint and a video fingerprint that are communicated with a capture infrastructure,wherein the capture infrastructure compares at least one of the audio fingerprint and the video fingerprint with a master database,wherein the capture infrastructure annotates the audio-visual data with a logo name by comparing entries in the master database with a logo data of the audio-visual data identified using a logo detection algorithm,wherein the capture infrastructure automatically divides the audio-visual data into a series of scenes based on a sematic grouping of actions in the audio-visual data,wherein the audio-visual data is analyzed in advance of a broadcast to determine content identifiers associated with each commercial in the audio-visual data such that advertisements are pre-inserted into the audio-visual data prior to broadcast,wherein the capture infrastructure applies a time-order algorithm to automatically match advertisements to the audio-visual data when a correlation pattern is identified by the capture infrastructure with other audio-visual content previously analyzed,wherein the capture infrastructure includes a buffer that is saved to a persistent storage and for which a label is generated to facilitate identification of reoccurring sequences,wherein a post processing operation is at least one of automated through a post-processing algorithm and a crowd-sourced operation using a plurality of users in which a turing test is applied to determine a veracity of an input,wherein a device pairing algorithm is used in which a cookie data associated with a web page visited by the user stored on a browser on the client device is paired with the networked media device when the client device is communicatively coupled with the networked media device,wherein a transitive public IP matching algorithm is utilized in which at least one of the client device and the networked media device communicates each public IP address with any paired entity to the capture infrastructure, andwherein a tag that is unconstrained from a same-origin policy is used to automatically load the advertisement m the browser, wherein the tag is at least one of an image tag, a frame, a iframe, and a script tag.
34. The system of claim 33 wherein the client device: to extend the security sandbox with a discovery algorithm and a relay algorithm through the discovery module and a relay module added to the security sandbox,and to bypass a paring server having the discovery algorithm and the relay algorithm when establishing the connection between the sandboxed application and the sandbox reachable service when the security is extended with the discovery algorithm and the relay algorithm through the discovery module and the relay module added to the security sandbox.
35. The system of claim 34 wherein the client device: to apply the discovery algorithm of the security sandbox to determine that the networked device having the sandbox reachable service communicates in a shared network common to the client device and the networked device, andto apply the relay algorithm of the security sandbox to establish the connection between the sandboxed application and the sandbox reachable service of the networked device.
36. The system of claim 35: wherein the discovery algorithm utilizes a protocol comprising at least one of a Bonjour* protocol, a SSDP protocol, a LSD uTorrent® protocol, a multicast protocol, an anycast protocol, and another Local Area Network (LAN) based protocol that discovers services in a LAN based on a broadcast from any one of an operating system service, the security sandbox, the client device, the sandbox reachable service, and the networked device.
37. The system of claim 36: wherein a cookie associated with the security sandbox is used to store a remote access token on a storage of the client device,wherein the remote access token identifies at least one of a set of communicable private Internet Protocol (IP) addresses and hardware addresses associated with sandbox reachable services that previously operated on a common shared network with the client device, andwherein the client device can communicate with the sandbox reachable services that previously operated on the common shared network through the remote access token.
38. The system of claim 37: wherein the client device and the networked device reside on networks that are incommunicable with each other comprising at least one of a firewall separation, a different network separation, a physical separation, an unreachable connection separation, andwherein the sandboxed application of the security sandbox of the client device and the sandbox reachable service of the networked device communicate with each other through a relay service employed by the pairing server having the discovery module and the relay module to facilitate a trusted communication between the sandboxed application and the sandbox reachable service.
39. The system of claim 38: wherein the trusted communication is facilitated in a manner such that the sandboxed application never learns at least one of a private IP address and a hardware address of the networked device when:a first Network Address Translator (NAT) device coupled with a network on which the client device operates to receives communications from a public IP address of a different network on which the sandbox reachable service operates, andwherein a second NAT device coupled with the different network on which the networked device operates to translates the private IP address of the networked device to the public IP address visible to the sandboxed application.
40. The system of claim 39: wherein the networked device comprises a plurality of sandbox reachable applications including the sandbox reachable application, and wherein a service agent module of the networked device coordinates communications with the discovery module of at least one of the security sandbox and the pairing server,wherein the security sandbox is at least one of an operating system on which the sandboxed application is hosted and a browser application of the operating system, andwherein the networked device is at least one of a television, a projection screen, a multimedia display, a touchscreen display, an audio device, a weather measurement device, a traffic monitoring device, a status update device, a global positioning device, a geospatial estimation device, a tracking device, a bidirectional communication device, a unicast device, a broadcast device, anda multidimensional visual presentation device.
41. The system of claim 40 wherein the client device: to utilize at least one of a WebSocket and a long polling service message query interface to reduce a latency of message delivery during the trusted communication between the sandboxed application and the sandbox reachable service, andto optimize a polling period between polling such that it is less than a timeout period of a session through the relay service.
42. The system of claim 41 wherein the client device: to initiate the relay service through at least one of a series of web pages where information is communicated using hyperlinks that point at the pairing server, and a form having a confirmation dialog that is submitted back to the pairing server, andwherein a global unique identifier is masked through the pairing server when the confirmation dialog is served from the pairing server.
43. The system of claim 33 wherein the client device: to access a pairing server when processing the identification data associated with the sandbox reachable service sharing the public address with the client device,wherein the pairing server performs a discovery lookup of any device that have announced that they share the public address associated with the client device, andwherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service.
44. The system of claim 43 wherein the networked device to at least one of: announce a sandbox reachable service of the networked device to a discovery module using a processor and memory,announce an availability of the sandbox reachable service across a range of public addresses such that the sandboxed application communicates with the sandbox reachable service in any one of the range of the public addresses,communicate at least one of a global unique identifier and an alphanumeric name to the pairing server along with at least one of a hardware address associated with the networked device, a public address pair associated with a sandbox reachable service of the networked device, and a private address pair associated with the sandbox reachable service of the networked device, andwherein the private address pair includes a private IP address and a port number associated with the sandbox reachable service.
45. The system of claim 44 wherein the client device: to eliminate a communication through a centralized infrastructure when the sandboxed application and the sandbox reachable service communicate in a shared network common to the client device and the networked device when the connection is established, wherein the shared network is at least one of a local area network, a multicast network, an anycast network, and a multilan network;to minimize a latency in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked device when the connection is established;and to improve privacy in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked device when the connection is established.
46. The system of claim 33: wherein the sandboxed application is at least one of a web page, a script, a binary executable, an intermediate bytecode, an abstract syntax tree, and an executable application in the security sandbox,wherein the sandboxed application comprises at least one of a markup language application such as a HyperText Markup Language 5 (HTML5) application, a Javascript® application, an Adobe® Flash® application, a Microsoft® Silverlight® application, a JQuery® application, and an Asynchronous Javascript® and a XML (AJAX) application, andwherein an access control algorithm governs a policy through which a secondary authentication is required when establishing a communication between the sandboxed application and the networked device.
47. The system of claim 46 wherein the client device: to utilize an exception to a same origin policy through a use of at least one of a hyperlink, a form, the script, a frame, a header, and an image when establishing the connection between the sandboxed application and the sandbox reachable service.
48. The system of claim 33: wherein the communication session is established by appending a header of a hypertext transfer protocol to permit the networked device to communicate with the sandboxed application as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm, wherein the header is either one of a origin header when the CORS algorithm is applied and a referrer header in an alternate algorithm.

CLAIM OF PRIORITY

This disclosure claims priority to, and incorporates herein by reference the entire specification of U.S. Provisional Patent application No. 61/118,286 filed Nov. 26, 2008 and titled DISCOVERY, ACCESS CONTROL, AND COMMUNICATION WITH NETWORKED SERVICES FROM WITHIN A SECURITY SANDBOX. This disclosure claims priority to, and incorporates herein by reference the entire specification of U.S. Continuation application Ser. No. 13/470,814 filed May 14, 2012 and titled DISCOVERY, ACCESS CONTROL, AND COMMUNICATION WITH NETWORKED SERVICES FROM WITHIN A SECURITY SANDBOX. This disclosure further claims priority to, and incorporates herein by reference the entire specification of U.S. Continuation application No. 12/592,377 filed Nov. 23, 2009 and titled DISCOVERY, ACCESS CONTROL, AND COMMUNICATION WITH NETWORKED SERVICES FROM WITHIN A SECURITY SANDBOX. This disclosure claims priority to, and incorporates herein by reference the entire specification of U.S. Provisional Patent application No. 61/584,168 filed Jan. 6, 2012 and titled CAPTURING CONTENT FOR DISPLAY ON A TELEVISION. This disclosure claims priority to, and incorporates herein by reference the entire specification of U.S. Provisional Patent application No. 61/696,711 filed Sep. 4, 2012 and titled SYSTEMS AND METHODS FOR RECOGNIZING CONTENT. This disclosure claims priority to, and incorporates herein by reference the entire specification of U.S. Utility patent application Ser. No. 13/736,031 filed Jan. 7, 2013 and titled ZERO CONFIGURATION COMMUNICATION BETWEEN A BROWSER AND A NETWORKED MEDIA DEVICE.

US Referenced Citations (898)

Number	Name	Date	Kind
3849760	Endou et al.	Nov 1974	A
3919479	Moon et al.	Nov 1975	A
4025851	Haselwood et al.	May 1977	A
4230990	Lert, Jr. et al.	Oct 1980	A
4258386	Cheung	Mar 1981	A
4420769	Novak	Dec 1983	A
4450531	Kenyon et al.	May 1984	A
4574304	Watanabe et al.	Mar 1986	A
4677466	Lert, Jr. et al.	Jun 1987	A
4697209	Kiewit et al.	Sep 1987	A
4739398	Thomas et al.	Apr 1988	A
4833449	Gaffigan	May 1989	A
4843562	Kenyon et al.	Jun 1989	A
4888638	Bohn	Dec 1989	A
4918730	Schulze	Apr 1990	A
4955070	Welsh et al.	Sep 1990	A
4967273	Greenberg	Oct 1990	A
4993059	Smith et al.	Feb 1991	A
5014125	Pocock et al.	May 1991	A
5019899	Boles et al.	May 1991	A
5105184	Pirani	Apr 1992	A
5155591	Wachob	Oct 1992	A
5223924	Strubbe	Jun 1993	A
5319453	Copriviza et al.	Jun 1994	A
5321750	Nadan	Jun 1994	A
5436653	Ellis et al.	Jul 1995	A
5481294	Thomas et al.	Jan 1996	A
5522077	Cuthbert et al.	May 1996	A
5539658	McCullough	Jul 1996	A
5557334	Legate	Sep 1996	A
5572246	Ellis et al.	Nov 1996	A
5612729	Ellis et al.	Mar 1997	A
5636346	Saxe	Jun 1997	A
5724521	Dedrick	Mar 1998	A
5732219	Blumer et al.	Mar 1998	A
5742768	Gennaro et al.	Apr 1998	A
5745884	Carnegie et al.	Apr 1998	A
5761601	Nemirofsky et al.	Jun 1998	A
5761648	Golden et al.	Jun 1998	A
5761655	Hoffman	Jun 1998	A
5774170	Hite et al.	Jun 1998	A
5774673	Beuk et al.	Jun 1998	A
5805974	Hite et al.	Sep 1998	A
5815665	Teper et al.	Sep 1998	A
5822525	Tafoya et al.	Oct 1998	A
5838301	Okamoto et al.	Nov 1998	A
5838317	Bolnick et al.	Nov 1998	A
5848396	Gerace	Dec 1998	A
5850517	Verkler et al.	Dec 1998	A
5903729	Reber et al.	May 1999	A
5905942	Stoel et al.	May 1999	A
5907279	Bruins et al.	May 1999	A
5940073	Klosterman et al.	Aug 1999	A
5948061	Merriman et al.	Sep 1999	A
5966705	Koneru et al.	Oct 1999	A
5977962	Chapman et al.	Nov 1999	A
5978835	Ludwig et al.	Nov 1999	A
6002393	Hite et al.	Dec 1999	A
6002443	Iggulden	Dec 1999	A
6009409	Adler et al.	Dec 1999	A
6009410	LeMole et al.	Dec 1999	A
6026368	Brown et al.	Feb 2000	A
6026369	Capek	Feb 2000	A
6032181	Bedgedjian et al.	Feb 2000	A
6043817	Bolnick et al.	Mar 2000	A
6055510	Henrick et al.	Apr 2000	A
6064980	Jacobi et al.	May 2000	A
6084628	Sawyer	Jul 2000	A
6105122	Muller et al.	Aug 2000	A
6118864	Chang et al.	Sep 2000	A
6119098	Guyot et al.	Sep 2000	A
6137892	Powell et al.	Oct 2000	A
6141010	Hoyle	Oct 2000	A
6157941	Verkler et al.	Dec 2000	A
6167427	Rabinovich et al.	Dec 2000	A
6169542	Hooks et al.	Jan 2001	B1
6188398	Collins-Rector et al.	Feb 2001	B1
6192476	Gong	Feb 2001	B1
6195696	Baber et al.	Feb 2001	B1
6216141	Straub et al.	Apr 2001	B1
6219696	Wynblatt et al.	Apr 2001	B1
6247077	Muller et al.	Jun 2001	B1
6286104	Buhle et al.	Sep 2001	B1
6304852	Loncteaux	Oct 2001	B1
6308327	Liu et al.	Oct 2001	B1
6310889	Parsons et al.	Oct 2001	B1
6332127	Bandera et al.	Dec 2001	B1
6349289	Peterson et al.	Feb 2002	B1
6351467	Dillon	Feb 2002	B1
6360254	Linden et al.	Mar 2002	B1
6381362	Deshpande et al.	Apr 2002	B1
6400265	Saylor et al.	Jun 2002	B1
6400996	Hoffberg et al.	Jun 2002	B1
6438594	Bowman-Amuah	Aug 2002	B1
6463585	Hendricks et al.	Oct 2002	B1
6469749	Dimitrova et al.	Oct 2002	B1
6481010	Nishikawa et al.	Nov 2002	B2
6484148	Boyd	Nov 2002	B1
6505169	Bhagavath et al.	Jan 2003	B1
6526491	Suzuoki et al.	Feb 2003	B2
6530082	Del Sesto et al.	Mar 2003	B1
6532218	Shaffer et al.	Mar 2003	B1
6536041	Knudson et al.	Mar 2003	B1
6546554	Schmidt et al.	Apr 2003	B1
6564260	Baber et al.	May 2003	B1
6564263	Bergman et al.	May 2003	B1
6574793	Ngo et al.	Jun 2003	B1
6577346	Perlman	Jun 2003	B1
6597405	Iggulden	Jul 2003	B1
6622171	Gupta et al.	Sep 2003	B2
6628801	Powell et al.	Sep 2003	B2
6631523	Matthews, III et al.	Oct 2003	B1
6651251	Shoff et al.	Nov 2003	B1
6718551	Swix et al.	Apr 2004	B1
6728784	Mattaway	Apr 2004	B1
6738978	Hendricks et al.	May 2004	B1
6757685	Raffaele et al.	Jun 2004	B2
6769009	Reisman	Jul 2004	B1
6771316	Iggulden	Aug 2004	B1
6799196	Smith	Sep 2004	B1
6804659	Graham et al.	Oct 2004	B1
6832239	Kraft et al.	Dec 2004	B1
6836888	Basu et al.	Dec 2004	B1
6845452	Roddy et al.	Jan 2005	B1
6907458	Tomassetti et al.	Jun 2005	B2
6946715	Hong	Sep 2005	B2
6959288	Medina et al.	Oct 2005	B1
6959320	Shah et al.	Oct 2005	B2
6978470	Swix et al.	Dec 2005	B2
6981022	Boundy	Dec 2005	B2
6983478	Grauch et al.	Jan 2006	B1
6983481	Fellenstein et al.	Jan 2006	B2
6993326	Link, II et al.	Jan 2006	B2
7028033	Bright et al.	Apr 2006	B2
7028327	Dougherty et al.	Apr 2006	B1
7043524	Shah et al.	May 2006	B2
7051351	Goldman et al.	May 2006	B2
7064796	Roy et al.	Jun 2006	B2
7080400	Navar	Jul 2006	B1
7089575	Agnihotri et al.	Aug 2006	B2
7089585	Dharmarajan	Aug 2006	B1
7100183	Kunkel et al.	Aug 2006	B2
7111230	Euchner et al.	Sep 2006	B2
7113090	Saylor et al.	Sep 2006	B1
7116661	Patton	Oct 2006	B2
7117439	Barrett et al.	Oct 2006	B2
7136875	Anderson et al.	Nov 2006	B2
7139882	Suzuoki et al.	Nov 2006	B2
7146627	Ismail et al.	Dec 2006	B1
7158666	Deshpande et al.	Jan 2007	B2
7162539	Garcie-Luna-Aceves	Jan 2007	B2
7181415	Blaser et al.	Feb 2007	B2
7185353	Schlack	Feb 2007	B2
7194421	Conkwright et al.	Mar 2007	B2
7210157	Devara	Apr 2007	B2
7243362	Swix et al.	Jul 2007	B2
7243364	Dunn et al.	Jul 2007	B2
7296091	Dutta et al.	Nov 2007	B1
7299195	Tawakol et al.	Nov 2007	B1
7308489	Weast	Dec 2007	B2
7328448	Eldering et al.	Feb 2008	B2
7330875	Parasnis et al.	Feb 2008	B1
7346606	Bharat	Mar 2008	B2
7346649	Wong	Mar 2008	B1
7349967	Wang	Mar 2008	B2
7349980	Darugar et al.	Mar 2008	B1
7360173	Tuli	Apr 2008	B2
7366975	Lipton	Apr 2008	B1
7373381	Rust	May 2008	B2
7380258	Durden et al.	May 2008	B2
7383243	Conkwright et al.	Jun 2008	B2
7421723	Harkness et al.	Sep 2008	B2
7437301	Kageyama et al.	Oct 2008	B2
7444658	Matz et al.	Oct 2008	B1
7444660	Dudkiewicz	Oct 2008	B2
7444666	Edwards et al.	Oct 2008	B2
7454515	Lamkin et al.	Nov 2008	B2
7472398	Corell et al.	Dec 2008	B2
7509402	Moorer et al.	Mar 2009	B2
7516213	Cunningham et al.	Apr 2009	B2
7525955	Velez-Rivera et al.	Apr 2009	B2
7545940	Alessi et al.	Jun 2009	B2
7546619	Anderson et al.	Jun 2009	B2
7552228	Parasnis et al.	Jun 2009	B2
7555165	Luo et al.	Jun 2009	B2
7559017	Datar et al.	Jul 2009	B2
7574723	Putterman et al.	Aug 2009	B2
7584491	Bruckner et al.	Sep 2009	B2
7590998	Hanley	Sep 2009	B2
7593988	Oreizy et al.	Sep 2009	B2
7596620	Colton et al.	Sep 2009	B1
7602748	Sinnreich et al.	Oct 2009	B2
7623823	Zito et al.	Nov 2009	B2
7624142	Jungck	Nov 2009	B2
7631325	Rys et al.	Dec 2009	B2
7634533	Rudolph et al.	Dec 2009	B2
7639387	Hull et al.	Dec 2009	B2
7650616	Lee	Jan 2010	B2
7653008	Patrick et al.	Jan 2010	B2
7664081	Luoma et al.	Feb 2010	B2
7665082	Wyatt et al.	Feb 2010	B2
7672003	Dowling et al.	Mar 2010	B2
7689920	Robbin et al.	Mar 2010	B2
7690006	Birnbaum et al.	Mar 2010	B2
7694319	Hassell et al.	Apr 2010	B1
7698165	Tawakol et al.	Apr 2010	B1
7701882	Jones et al.	Apr 2010	B2
7711748	Bright et al.	May 2010	B2
7716161	Dean et al.	May 2010	B2
7720914	Goodman et al.	May 2010	B2
7729366	Mok et al.	Jun 2010	B2
7734624	Anderson et al.	Jun 2010	B2
7739140	Vinson et al.	Jun 2010	B2
7769756	Krikorian et al.	Aug 2010	B2
7774348	Delli Santi et al.	Aug 2010	B2
7774715	Evans	Aug 2010	B1
7789757	Gemelos et al.	Sep 2010	B2
7793318	Deng	Sep 2010	B2
7797433	Kennedy et al.	Sep 2010	B2
7805740	Gilboa et al.	Sep 2010	B2
7822809	Dhupelia et al.	Oct 2010	B2
7831426	Bennett	Nov 2010	B2
7856644	Nicholson et al.	Dec 2010	B2
7861260	Shkedi	Dec 2010	B2
7870592	Hudson et al.	Jan 2011	B2
7870596	Schackow et al.	Jan 2011	B2
7873716	Maes	Jan 2011	B2
7877461	Rimmer	Jan 2011	B1
7877774	Basso et al.	Jan 2011	B1
7890957	Campbell	Feb 2011	B2
7904925	Jiang	Mar 2011	B2
7908618	Bruckner et al.	Mar 2011	B2
7912822	Bethlehem et al.	Mar 2011	B2
7921037	Hertling et al.	Apr 2011	B2
7929551	Dietrich et al.	Apr 2011	B2
7930207	Merriman et al.	Apr 2011	B2
7933451	Kloer	Apr 2011	B2
7937405	Anderson et al.	May 2011	B2
7941197	Jain et al.	May 2011	B2
7941816	Harkness et al.	May 2011	B2
7950055	Blinn et al.	May 2011	B2
7962007	Abe et al.	Jun 2011	B2
7966309	Shacham et al.	Jun 2011	B2
7978876	Powell et al.	Jul 2011	B2
7979570	Chapweske et al.	Jul 2011	B2
7995503	Yu	Aug 2011	B2
8001124	Svendsen	Aug 2011	B2
8035656	Blanchard et al.	Oct 2011	B2
8041643	Mukerji et al.	Oct 2011	B2
8046839	Lo	Oct 2011	B2
8055784	Kalama et al.	Nov 2011	B2
8060399	Ullah	Nov 2011	B2
8060912	Sato	Nov 2011	B2
8065700	Lee	Nov 2011	B2
8069348	Bacon	Nov 2011	B2
8079045	Krapf et al.	Dec 2011	B2
8087047	Olague et al.	Dec 2011	B2
8090706	Bharat	Jan 2012	B2
8091031	Evans	Jan 2012	B2
8122484	Karjoth et al.	Feb 2012	B2
8126963	Rimmer	Feb 2012	B1
8131585	Nicholas et al.	Mar 2012	B2
8131705	Chevalier et al.	Mar 2012	B2
8131734	Austin et al.	Mar 2012	B2
8140965	Dean et al.	Mar 2012	B2
8141111	Gilley et al.	Mar 2012	B2
8145645	Delli Santi et al.	Mar 2012	B2
8145705	Rust	Mar 2012	B1
8150729	Wilhelm	Apr 2012	B2
8150985	Nakamura	Apr 2012	B2
8155696	Swanburg et al.	Apr 2012	B2
8161511	Kwak et al.	Apr 2012	B2
8171030	Pereira et al.	May 2012	B2
8171510	Kamen et al.	May 2012	B2
8175413	Ioffe et al.	May 2012	B1
8180708	Hurtado et al.	May 2012	B2
8180891	Harrison	May 2012	B1
8189945	Stojancic et al.	May 2012	B2
8191091	Harvey et al.	May 2012	B1
8195689	Ramanathan et al.	Jun 2012	B2
8195692	Baek et al.	Jun 2012	B2
8201080	Basson et al.	Jun 2012	B2
8209397	Ahn et al.	Jun 2012	B2
8209404	Wu	Jun 2012	B2
8214256	Riedl et al.	Jul 2012	B2
8219411	Matz et al.	Jul 2012	B2
8225347	Flickinger et al.	Jul 2012	B1
8229227	Stojancic et al.	Jul 2012	B2
8229751	Cheung	Jul 2012	B2
8239340	Hanson	Aug 2012	B2
8244707	Lin et al.	Aug 2012	B2
8245270	Cooperstein et al.	Aug 2012	B2
8255949	Bayer et al.	Aug 2012	B1
8260665	Foladare et al.	Sep 2012	B2
8261341	Stirbu	Sep 2012	B2
8271649	Kalofonos et al.	Sep 2012	B2
8275791	Raffaele et al.	Sep 2012	B2
8281288	Spencer	Oct 2012	B1
8285880	Ye et al.	Oct 2012	B2
8290351	Plotnick et al.	Oct 2012	B2
8296763	Peercy et al.	Oct 2012	B1
8301596	Lin et al.	Oct 2012	B2
8301732	Chapweske et al.	Oct 2012	B2
8302170	Kramer et al.	Oct 2012	B2
8316450	Robinson et al.	Nov 2012	B2
8326872	Zwilling et al.	Dec 2012	B2
8332885	Williamson et al.	Dec 2012	B2
8335786	Pereira et al.	Dec 2012	B2
8339991	Biswas et al.	Dec 2012	B2
8341242	Dillon et al.	Dec 2012	B2
8352980	Howcroft	Jan 2013	B2
8355711	Heins et al.	Jan 2013	B2
8358966	Zito et al.	Jan 2013	B2
8364541	Roth	Jan 2013	B2
8364703	Ramanathan et al.	Jan 2013	B2
8364959	Bhanoo et al.	Jan 2013	B2
8365217	Legrand	Jan 2013	B2
8375131	Rogers et al.	Feb 2013	B2
8381026	Talla et al.	Feb 2013	B2
8385644	Stojancic	Feb 2013	B2
8418191	Honishi et al.	Apr 2013	B2
8418198	Cansler	Apr 2013	B2
8433306	Rodriguez	Apr 2013	B2
8433574	Jablokov et al.	Apr 2013	B2
8443420	Brown et al.	May 2013	B2
8451762	Liu et al.	May 2013	B2
8452864	Vendrow	May 2013	B1
8479246	Hudson et al.	Jul 2013	B2
8489701	Manion et al.	Jul 2013	B2
8495611	McCarthy et al.	Jul 2013	B2
8495675	Philpott et al.	Jul 2013	B1
8495746	Fissel et al.	Jul 2013	B2
8504551	Anderson et al.	Aug 2013	B2
8510317	Boetje et al.	Aug 2013	B2
8510661	Dharmaji et al.	Aug 2013	B2
8510779	Slothouber et al.	Aug 2013	B2
8520909	Leung et al.	Aug 2013	B2
8527594	Lahaix	Sep 2013	B2
8533192	Moganti et al.	Sep 2013	B2
8537157	Adimatyam et al.	Sep 2013	B2
8539025	Husain et al.	Sep 2013	B2
8539072	Harrison	Sep 2013	B1
8539523	Philpott et al.	Sep 2013	B2
8548820	Matz et al.	Oct 2013	B2
8549052	Miles	Oct 2013	B2
8549066	Donahue et al.	Oct 2013	B1
8549110	Jerbi et al.	Oct 2013	B2
8549550	Lopatecki et al.	Oct 2013	B2
8566154	Merriman et al.	Oct 2013	B2
8566867	Yang et al.	Oct 2013	B1
8577996	Hughes et al.	Nov 2013	B2
8595781	Neumeier et al.	Nov 2013	B2
8607267	Shkedi	Dec 2013	B2
8613045	Shigapov	Dec 2013	B1
8621585	Danieli et al.	Dec 2013	B2
8635316	Barnhill, Jr.	Jan 2014	B2
8645209	Mandyam et al.	Feb 2014	B2
8645994	Vemparala et al.	Feb 2014	B2
8646063	Dowlatkhah	Feb 2014	B2
8655716	Barnes et al.	Feb 2014	B1
8656422	Kumar et al.	Feb 2014	B2
8661010	Lin et al.	Feb 2014	B2
8666168	Stojancic et al.	Mar 2014	B2
8667142	Takei et al.	Mar 2014	B2
8677253	Duquene et al.	Mar 2014	B2
8694656	Douillet et al.	Apr 2014	B2
8695032	Shkedi	Apr 2014	B2
8700699	Shen et al.	Apr 2014	B2
8701134	Whinmill et al.	Apr 2014	B2
8707351	Dharmaji	Apr 2014	B2
8712833	Quach et al.	Apr 2014	B2
8719396	Brindley et al.	May 2014	B2
8719870	Davies et al.	May 2014	B1
8732182	Bethlehem et al.	May 2014	B2
8736764	Amundsen	May 2014	B2
8738779	Binding et al.	May 2014	B2
8745272	Casalaina et al.	Jun 2014	B2
8756686	Plattner et al.	Jun 2014	B2
8763033	Dittus	Jun 2014	B2
8763097	Bhatnagar et al.	Jun 2014	B2
8769584	Neumeier et al.	Jul 2014	B2
8775391	Kalavade	Jul 2014	B2
8776112	Roberts et al.	Jul 2014	B2
8793730	Mowrey et al.	Jul 2014	B2
8799357	Clift et al.	Aug 2014	B2
8804039	Kim et al.	Aug 2014	B2
8804721	He et al.	Aug 2014	B2
8812451	Shukla et al.	Aug 2014	B2
8813232	Sreedharan et al.	Aug 2014	B2
8817757	Luo	Aug 2014	B2
8819249	Harrison	Aug 2014	B2
8819255	Harrison	Aug 2014	B1
8825526	Peters et al.	Sep 2014	B2
8826327	Adimatyam et al.	Sep 2014	B2
8832729	Nussel et al.	Sep 2014	B2
8838149	Hasek	Sep 2014	B2
8838556	Reiner et al.	Sep 2014	B1
8838808	Addala et al.	Sep 2014	B2
8849821	Schloter	Sep 2014	B2
8855796	Otsuka et al.	Oct 2014	B2
8856028	Yang et al.	Oct 2014	B2
8856087	Greene et al.	Oct 2014	B2
8856874	Pieczul et al.	Oct 2014	B2
8863174	Neil et al.	Oct 2014	B2
8898714	Neumeier et al.	Nov 2014	B2
8904021	Harrison	Dec 2014	B2
8910199	Slaney et al.	Dec 2014	B2
8930980	Neumeier et al.	Jan 2015	B2
8966525	Mehta et al.	Feb 2015	B2
9009066	Long et al.	Apr 2015	B2
9026668	Harrison	May 2015	B2
9032451	Cansino et al.	May 2015	B2
9043712	Santoro et al.	May 2015	B2
9055309	Neumeier et al.	Jun 2015	B2
9113107	Jolna et al.	Aug 2015	B2
9131279	Raveendran et al.	Sep 2015	B2
9152727	Balducci et al.	Oct 2015	B1
9154942	Harrison et al.	Oct 2015	B2
9167419	Harrison	Oct 2015	B2
9183560	Abelow	Nov 2015	B2
9204275	Johnson et al.	Dec 2015	B2
9215217	Abu-Hakima et al.	Dec 2015	B2
9232279	Beeson et al.	Jan 2016	B2
9258383	Harrison	Feb 2016	B2
9271052	Holden	Feb 2016	B2
20010011226	Greer et al.	Aug 2001	A1
20010016501	King	Aug 2001	A1
20010016947	Nishikawa et al.	Aug 2001	A1
20010029583	Palatov et al.	Oct 2001	A1
20010036224	Demelloet et al.	Nov 2001	A1
20010039658	Walton	Nov 2001	A1
20010054155	Hagan et al.	Dec 2001	A1
20020012347	Fitzpatrick	Jan 2002	A1
20020015105	Abe et al.	Feb 2002	A1
20020019769	Barritz et al.	Feb 2002	A1
20020026635	Wheeler et al.	Feb 2002	A1
20020032906	Grossman	Mar 2002	A1
20020042914	Walker et al.	Apr 2002	A1
20020044683	Deshpande et al.	Apr 2002	A1
20020052965	Dowling	May 2002	A1
20020059633	Harkness et al.	May 2002	A1
20020066100	Hoang	May 2002	A1
20020069100	Arberman	Jun 2002	A1
20020072966	Eldering et al.	Jun 2002	A1
20020078456	Hudson et al.	Jun 2002	A1
20020083435	Blasko et al.	Jun 2002	A1
20020083441	Flickinger et al.	Jun 2002	A1
20020083443	Eldering et al.	Jun 2002	A1
20020087401	Leapman et al.	Jul 2002	A1
20020087545	Bright et al.	Jul 2002	A1
20020087975	Schlack	Jul 2002	A1
20020087976	Kaplan et al.	Jul 2002	A1
20020087978	Nicholson et al.	Jul 2002	A1
20020091763	Shah et al.	Jul 2002	A1
20020104083	Hendricks et al.	Aug 2002	A1
20020116549	Raffaele et al.	Aug 2002	A1
20020120498	Gordon et al.	Aug 2002	A1
20020120925	Logan	Aug 2002	A1
20020123928	Eldering et al.	Sep 2002	A1
20020133490	Conkwright et al.	Sep 2002	A1
20020133534	Forslow	Sep 2002	A1
20020138842	Chong et al.	Sep 2002	A1
20020143782	Headings et al.	Oct 2002	A1
20020144262	Plotnick et al.	Oct 2002	A1
20020147611	Greene et al.	Oct 2002	A1
20020151992	Hoffberg et al.	Oct 2002	A1
20020152474	Dudkiewicz	Oct 2002	A1
20020162117	Pearson et al.	Oct 2002	A1
20020162118	Levy et al.	Oct 2002	A1
20020174197	Schimke et al.	Nov 2002	A1
20020178447	Plotnick et al.	Nov 2002	A1
20020196789	Patton	Dec 2002	A1
20030001883	Wang	Jan 2003	A1
20030009538	Shah et al.	Jan 2003	A1
20030023489	McGuire et al.	Jan 2003	A1
20030028433	Merriman et al.	Feb 2003	A1
20030030752	Begeja et al.	Feb 2003	A1
20030031176	Sim	Feb 2003	A1
20030036949	Kaddeche et al.	Feb 2003	A1
20030070167	Holtz et al.	Apr 2003	A1
20030079226	Barrett	Apr 2003	A1
20030097426	Parry	May 2003	A1
20030097657	Zhou et al.	May 2003	A1
20030101451	Bentolila et al.	May 2003	A1
20030101454	Ozer et al.	May 2003	A1
20030121037	Swix et al.	Jun 2003	A1
20030121046	Roy et al.	Jun 2003	A1
20030135853	Goldman et al.	Jul 2003	A1
20030145323	Hendricks et al.	Jul 2003	A1
20030149975	Eldering et al.	Aug 2003	A1
20030154475	Rodriguez et al.	Aug 2003	A1
20030172374	Vinson et al.	Sep 2003	A1
20030188318	Liew et al.	Oct 2003	A1
20030188321	Shoff et al.	Oct 2003	A1
20030226141	Krasnow et al.	Dec 2003	A1
20030229765	Suzuoki et al.	Dec 2003	A1
20040006693	Vasnani et al.	Jan 2004	A1
20040006706	Erlingsson	Jan 2004	A1
20040025034	Alessi et al.	Feb 2004	A1
20040025174	Cerrato	Feb 2004	A1
20040045020	Witt et al.	Mar 2004	A1
20040059708	Dean et al.	Mar 2004	A1
20040078809	Drazin	Apr 2004	A1
20040088348	Yeager et al.	May 2004	A1
20040148625	Eldering et al.	Jul 2004	A1
20040158858	Paxton et al.	Aug 2004	A1
20040163101	Swix et al.	Aug 2004	A1
20040210630	Simonnet et al.	Oct 2004	A1
20040215509	Perry	Oct 2004	A1
20040215515	Perry	Oct 2004	A1
20040216171	Barone et al.	Oct 2004	A1
20040225686	Li et al.	Nov 2004	A1
20040226035	Hauser	Nov 2004	A1
20040237102	Konig et al.	Nov 2004	A1
20040240562	Bargeron et al.	Dec 2004	A1
20040260791	Jerbi et al.	Dec 2004	A1
20040267723	Bharat	Dec 2004	A1
20050002640	Putterman et al.	Jan 2005	A1
20050015795	Iggulden	Jan 2005	A1
20050015796	Bruckner et al.	Jan 2005	A1
20050021670	Maes	Jan 2005	A1
20050028200	Sardera	Feb 2005	A1
20050028201	Klosterman et al.	Feb 2005	A1
20050028206	Cameron et al.	Feb 2005	A1
20050071224	Fikes et al.	Mar 2005	A1
20050080876	Peiffer et al.	Apr 2005	A1
20050080878	Cunningham et al.	Apr 2005	A1
20050091301	Oreizy et al.	Apr 2005	A1
20050108213	Riise et al.	May 2005	A1
20050108745	Linzer	May 2005	A1
20050165696	Jakobsson et al.	Jul 2005	A1
20050183143	Anderholm et al.	Aug 2005	A1
20050204381	Ludvig et al.	Sep 2005	A1
20050210502	Flickinger et al.	Sep 2005	A1
20050235318	Grauch et al.	Oct 2005	A1
20050251491	Medina et al.	Nov 2005	A1
20050251577	Guo et al.	Nov 2005	A1
20050251820	Stefanik et al.	Nov 2005	A1
20050267896	Goodman et al.	Dec 2005	A1
20050283796	Flickinger	Dec 2005	A1
20060029368	Harville	Feb 2006	A1
20060064583	Birnbaum et al.	Mar 2006	A1
20060072144	Dowling et al.	Apr 2006	A1
20060072542	Sinnreich et al.	Apr 2006	A1
20060085642	Multerer et al.	Apr 2006	A1
20060092834	Honishi et al.	May 2006	A1
20060133414	Luoma et al.	Jun 2006	A1
20060136964	Diez et al.	Jun 2006	A1
20060143188	Bright et al.	Jun 2006	A1
20060149624	Baluja et al.	Jul 2006	A1
20060153296	Deng	Jul 2006	A1
20060156362	Perrot	Jul 2006	A1
20060168291	van Zoest et al.	Jul 2006	A1
20060168616	Candelore	Jul 2006	A1
20060195860	Eldering et al.	Aug 2006	A1
20060212908	Hunter et al.	Sep 2006	A1
20060230130	Cho et al.	Oct 2006	A1
20060245724	Hwang et al.	Nov 2006	A1
20060247011	Gagner	Nov 2006	A1
20060247937	Binding et al.	Nov 2006	A1
20060248558	Barton et al.	Nov 2006	A1
20060253330	Maggio et al.	Nov 2006	A1
20060265493	Brindley et al.	Nov 2006	A1
20060287912	Raghuvamshi	Dec 2006	A1
20070047781	Hull et al.	Mar 2007	A1
20070050832	Wright et al.	Mar 2007	A1
20070050854	Cooperstein et al.	Mar 2007	A1
20070061724	Slothouber et al.	Mar 2007	A1
20070061831	Savoor et al.	Mar 2007	A1
20070072676	Baluja	Mar 2007	A1
20070078706	Datta et al.	Apr 2007	A1
20070083908	McCarthy et al.	Apr 2007	A1
20070088801	Levkovitz et al.	Apr 2007	A1
20070088852	Levkovitz	Apr 2007	A1
20070089158	Clark et al.	Apr 2007	A1
20070100690	Hopkins	May 2007	A1
20070106721	Schloter	May 2007	A1
20070108721	Bayne et al.	May 2007	A1
20070109449	Cheung	May 2007	A1
20070113243	Brey	May 2007	A1
20070113263	Chatani	May 2007	A1
20070116365	Kloer	May 2007	A1
20070129108	Swanburg et al.	Jun 2007	A1
20070143796	Malik	Jun 2007	A1
20070157231	Eldering et al.	Jul 2007	A1
20070168389	Lipscomb	Jul 2007	A1
20070180459	Smithpeters et al.	Aug 2007	A1
20070186240	Ward et al.	Aug 2007	A1
20070192450	Lewis	Aug 2007	A1
20070198339	Shen et al.	Aug 2007	A1
20070204310	Hua	Aug 2007	A1
20070208619	Branam et al.	Sep 2007	A1
20070220024	Putterman et al.	Sep 2007	A1
20070220575	Cooper et al.	Sep 2007	A1
20070234382	Swix et al.	Oct 2007	A1
20070260520	Jha et al.	Nov 2007	A1
20070266403	Ou et al.	Nov 2007	A1
20070271300	Ramaswamy	Nov 2007	A1
20070283384	Haeuser et al.	Dec 2007	A1
20070283402	Yu	Dec 2007	A1
20070288985	Candelore et al.	Dec 2007	A1
20070291747	Stern et al.	Dec 2007	A1
20070291761	Kauniskangas et al.	Dec 2007	A1
20070300264	Turner	Dec 2007	A1
20070300273	Turner	Dec 2007	A1
20070300280	Turner et al.	Dec 2007	A1
20080004957	Hildreth et al.	Jan 2008	A1
20080005166	Cragun	Jan 2008	A1
20080040666	Wang et al.	Feb 2008	A1
20080040767	McCarthy et al.	Feb 2008	A1
20080046945	Hanley	Feb 2008	A1
20080052195	Roth et al.	Feb 2008	A1
20080059285	Hamoui	Mar 2008	A1
20080060002	Noll et al.	Mar 2008	A1
20080066080	Campbell	Mar 2008	A1
20080089551	Heather et al.	Apr 2008	A1
20080109376	Walsh et al.	May 2008	A1
20080109844	Baldeschwieler et al.	May 2008	A1
20080109888	Ullah	May 2008	A1
20080127263	Klosterman et al.	May 2008	A1
20080140476	Anand et al.	Jun 2008	A1
20080154678	Botelho	Jun 2008	A1
20080155588	Roberts et al.	Jun 2008	A1
20080155591	Mahajan et al.	Jun 2008	A1
20080155627	O'Connor et al.	Jun 2008	A1
20080172243	Kelly	Jul 2008	A1
20080172747	Hurtado et al.	Jul 2008	A1
20080181225	Zampiello	Jul 2008	A1
20080186933	Willman et al.	Aug 2008	A1
20080189757	Schackow et al.	Aug 2008	A1
20080195457	Sherman et al.	Aug 2008	A1
20080201222	Lahaix	Aug 2008	A1
20080201734	Lyon et al.	Aug 2008	A1
20080222045	Mukerji et al.	Sep 2008	A1
20080222711	Michaelis	Sep 2008	A1
20080228581	Yonezaki et al.	Sep 2008	A1
20080229335	Robbin et al.	Sep 2008	A1
20080243535	Binding et al.	Oct 2008	A1
20080244418	Manolescu et al.	Oct 2008	A1
20080263600	Olague et al.	Oct 2008	A1
20080276266	Huchital et al.	Nov 2008	A1
20080276270	Kotaru et al.	Nov 2008	A1
20080288631	Faisal et al.	Nov 2008	A1
20080306820	Passmore	Dec 2008	A1
20080307460	Knudson et al.	Dec 2008	A1
20080310731	Stojancic et al.	Dec 2008	A1
20080313140	Pereira et al.	Dec 2008	A1
20080313648	Wang et al.	Dec 2008	A1
20090006207	Datar et al.	Jan 2009	A1
20090011744	Daley et al.	Jan 2009	A1
20090044223	Jiang et al.	Feb 2009	A1
20090049384	Yau	Feb 2009	A1
20090052784	Covell et al.	Feb 2009	A1
20090055537	Takei et al.	Feb 2009	A1
20090063691	Kalofonos et al.	Mar 2009	A1
20090070473	Baum et al.	Mar 2009	A1
20090077580	Konig et al.	Mar 2009	A1
20090083417	Hughes et al.	Mar 2009	A1
20090088878	Otsuka et al.	Apr 2009	A1
20090089251	Johnston et al.	Apr 2009	A1
20090094093	Phan	Apr 2009	A1
20090100361	Abello et al.	Apr 2009	A1
20090119576	Pepper et al.	May 2009	A1
20090147718	Liu et al.	Jun 2009	A1
20090153289	Hope et al.	Jun 2009	A1
20090163227	Collins	Jun 2009	A1
20090164483	Miles	Jun 2009	A1
20090164641	Rogers et al.	Jun 2009	A1
20090164904	Horowitz et al.	Jun 2009	A1
20090165140	Robinson et al.	Jun 2009	A1
20090172728	Shkedi et al.	Jul 2009	A1
20090172746	Aldrey et al.	Jul 2009	A1
20090185723	Kurtz et al.	Jul 2009	A1
20090199236	Barrett et al.	Aug 2009	A1
20090199283	Jain	Aug 2009	A1
20090210899	Lawrence-Apfelbaum et al.	Aug 2009	A1
20090210902	Slaney et al.	Aug 2009	A1
20090231485	Steinke	Sep 2009	A1
20090232305	Alessi et al.	Sep 2009	A1
20090234738	Britton et al.	Sep 2009	A1
20090235312	Morad et al.	Sep 2009	A1
20090240821	Juncker et al.	Sep 2009	A1
20090248736	Adelman et al.	Oct 2009	A1
20090259612	Hanson	Oct 2009	A1
20090276313	Wilhelm	Nov 2009	A1
20090292610	Quach et al.	Nov 2009	A1
20090299817	Fok et al.	Dec 2009	A1
20090300109	Porter	Dec 2009	A1
20090307048	Grossman	Dec 2009	A1
20100007797	Stojancic	Jan 2010	A1
20100022231	Heins et al.	Jan 2010	A1
20100023392	Merriman et al.	Jan 2010	A1
20100023582	Pedersen et al.	Jan 2010	A1
20100050220	Rys et al.	Feb 2010	A1
20100058380	Yu et al.	Mar 2010	A1
20100063970	Kim	Mar 2010	A1
20100071070	Jawa et al.	Mar 2010	A1
20100083303	Redei et al.	Apr 2010	A1
20100099359	Lee et al.	Apr 2010	A1
20100107189	Steelberg et al.	Apr 2010	A1
20100121891	Zampiello	May 2010	A1
20100131973	Dillon et al.	May 2010	A1
20100145938	Boetje et al.	Jun 2010	A1
20100146552	Hassell et al.	Jun 2010	A1
20100158391	Cunningham et al.	Jun 2010	A1
20100161424	Sylvain	Jun 2010	A1
20100174605	Dean et al.	Jul 2010	A1
20100175078	Knudson et al.	Jul 2010	A1
20100180216	Bates et al.	Jul 2010	A1
20100185513	Anderson et al.	Jul 2010	A1
20100199188	Abu-Hakima et al.	Aug 2010	A1
20100205562	de Heer	Aug 2010	A1
20100226582	Luo et al.	Sep 2010	A1
20100228611	Shenfield	Sep 2010	A1
20100251278	Agarwal et al.	Sep 2010	A1
20100251289	Agarwal et al.	Sep 2010	A1
20100257052	Zito et al.	Oct 2010	A1
20100269138	Krikorian et al.	Oct 2010	A1
20100287026	Smith	Nov 2010	A1
20100287049	Rousso et al.	Nov 2010	A1
20100306193	Pereira et al.	Dec 2010	A1
20100306773	Lee et al.	Dec 2010	A1
20100306805	Neumeier et al.	Dec 2010	A1
20100306808	Neumeier et al.	Dec 2010	A1
20100318628	Pacella et al.	Dec 2010	A1
20100318917	Holladay et al.	Dec 2010	A1
20100319062	Danieli et al.	Dec 2010	A1
20100325495	Talla et al.	Dec 2010	A1
20100325552	Sloo et al.	Dec 2010	A1
20110010737	Bouazizi et al.	Jan 2011	A1
20110029555	Gao et al.	Feb 2011	A1
20110029666	Lopatecki et al.	Feb 2011	A1
20110032334	Raveendran et al.	Feb 2011	A1
20110043652	King et al.	Feb 2011	A1
20110061073	Nicholson et al.	Mar 2011	A1
20110078753	Christianson et al.	Mar 2011	A1
20110082939	Montemurro et al.	Apr 2011	A1
20110082940	Montemurro et al.	Apr 2011	A1
20110088075	Eyer	Apr 2011	A1
20110099065	Georgis et al.	Apr 2011	A1
20110099609	Malhotra et al.	Apr 2011	A1
20110107385	Hudson et al.	May 2011	A1
20110119139	Dean et al.	May 2011	A1
20110122836	Kim	May 2011	A1
20110125586	Evans	May 2011	A1
20110131597	Cera et al.	Jun 2011	A1
20110136539	Jain et al.	Jun 2011	A1
20110138059	Schleifer et al.	Jun 2011	A1
20110145926	Dalcher et al.	Jun 2011	A1
20110154498	Fissel et al.	Jun 2011	A1
20110179010	Lin et al.	Jul 2011	A1
20110179447	Harkness et al.	Jul 2011	A1
20110191178	Newberg et al.	Aug 2011	A1
20110191352	Jones et al.	Aug 2011	A1
20110213881	Stavenow et al.	Sep 2011	A1
20110219322	Ramamurthy et al.	Sep 2011	A1
20110238379	Misra et al.	Sep 2011	A1
20110247044	Jacoby	Oct 2011	A1
20110251987	Buchheit	Oct 2011	A1
20110251992	Bethlehem et al.	Oct 2011	A1
20110265114	Legrand	Oct 2011	A1
20110265116	Stern et al.	Oct 2011	A1
20110270672	Hillard et al.	Nov 2011	A1
20110274179	Holden	Nov 2011	A1
20110279445	Murphy et al.	Nov 2011	A1
20110283322	Hamano	Nov 2011	A1
20110289114	Yu et al.	Nov 2011	A1
20110289524	Toner et al.	Nov 2011	A1
20110289544	Goosen et al.	Nov 2011	A1
20110296303	Duquene et al.	Dec 2011	A1
20110304771	Blanchard et al.	Dec 2011	A1
20110310100	Adimatyam et al.	Dec 2011	A1
20110317885	Leung et al.	Dec 2011	A1
20110321003	Doig et al.	Dec 2011	A1
20110321109	Hudson et al.	Dec 2011	A1
20120011541	McCarthy	Jan 2012	A1
20120017240	Shkedi	Jan 2012	A1
20120023522	Anderson et al.	Jan 2012	A1
20120047277	Keidar et al.	Feb 2012	A1
20120069131	Abelow	Mar 2012	A1
20120072291	Bharat	Mar 2012	A1
20120072420	Moganti et al.	Mar 2012	A1
20120076049	Rudolf et al.	Mar 2012	A1
20120084814	Olague et al.	Apr 2012	A1
20120086857	Kim et al.	Apr 2012	A1
20120089700	Safruti et al.	Apr 2012	A1
20120101907	Dodda	Apr 2012	A1
20120102515	Ramaswamy	Apr 2012	A1
20120124498	Santoro et al.	May 2012	A1
20120130822	Patwa et al.	May 2012	A1
20120130825	Evans	May 2012	A1
20120131095	Luna et al.	May 2012	A1
20120144416	Wetzer et al.	Jun 2012	A1
20120150944	Steelberg et al.	Jun 2012	A1
20120151015	Plastina et al.	Jun 2012	A1
20120151521	Gilley et al.	Jun 2012	A1
20120159542	Minwalla	Jun 2012	A1
20120163770	Kaiser et al.	Jun 2012	A1
20120163776	Hassell et al.	Jun 2012	A1
20120167132	Mathews et al.	Jun 2012	A1
20120174155	Mowrey et al.	Jul 2012	A1
20120191716	Omoigui	Jul 2012	A1
20120207402	Stojancic et al.	Aug 2012	A1
20120209706	Ramer et al.	Aug 2012	A1
20120209726	Dean et al.	Aug 2012	A1
20120210224	Wong et al.	Aug 2012	A1
20120215622	Ramer et al.	Aug 2012	A1
20120233163	Kirkpatrick	Sep 2012	A1
20120240151	Tapper	Sep 2012	A1
20120257110	Amundsen	Oct 2012	A1
20120260184	Dawes et al.	Oct 2012	A1
20120265616	Cao et al.	Oct 2012	A1
20120272134	Steelberg et al.	Oct 2012	A1
20120284746	Evans et al.	Nov 2012	A1
20120284757	Rajapakse	Nov 2012	A1
20120297406	Bartholomay et al.	Nov 2012	A1
20120311629	Zaslavsky et al.	Dec 2012	A1
20120311702	Krstic et al.	Dec 2012	A1
20120317175	Husain et al.	Dec 2012	A1
20120317178	Husain et al.	Dec 2012	A1
20120317181	Husain et al.	Dec 2012	A1
20120324566	Baum et al.	Dec 2012	A1
20130013665	Sng et al.	Jan 2013	A1
20130019262	Bhatia et al.	Jan 2013	A1
20130019268	Fitzsimmons et al.	Jan 2013	A1
20130041664	McKoen et al.	Feb 2013	A1
20130045681	Dua	Feb 2013	A1
20130051300	He et al.	Feb 2013	A1
20130055309	Dittus	Feb 2013	A1
20130060905	Mickens et al.	Mar 2013	A1
20130061259	Raman et al.	Mar 2013	A1
20130061267	Cansino et al.	Mar 2013	A1
20130078946	Pecen et al.	Mar 2013	A1
20130080242	Alhadeff et al.	Mar 2013	A1
20130085865	Zhou et al.	Apr 2013	A1
20130094423	Wengrovitz et al.	Apr 2013	A1
20130104160	Beeson et al.	Apr 2013	A1
20130104232	Johnson et al.	Apr 2013	A1
20130117782	Mehta et al.	May 2013	A1
20130139209	Urrabazo et al.	May 2013	A1
20130139210	Huang et al.	May 2013	A1
20130185153	Howcroft	Jul 2013	A1
20130185422	Rogers et al.	Jul 2013	A1
20130202150	Sinha et al.	Aug 2013	A1
20130205317	Sinha et al.	Aug 2013	A1
20130205319	Sinha et al.	Aug 2013	A1
20130205348	Hudson et al.	Aug 2013	A1
20130238702	Sheth et al.	Sep 2013	A1
20130254803	Huffman	Sep 2013	A1
20130254884	Dalcher et al.	Sep 2013	A1
20130263166	Fleischman	Oct 2013	A1
20130297727	Levy	Nov 2013	A1
20130318157	Harrison	Nov 2013	A1
20130326554	Shkedi	Dec 2013	A1
20130340011	Rodriguez	Dec 2013	A1
20130340050	Harrison	Dec 2013	A1
20140002247	Harrison et al.	Jan 2014	A1
20140007155	Vemparala et al.	Jan 2014	A1
20140007157	Harrison et al.	Jan 2014	A1
20140007162	Harrison	Jan 2014	A1
20140007187	Harrison	Jan 2014	A1
20140007262	Metsäpelto et al.	Jan 2014	A1
20140029847	Frye et al.	Jan 2014	A1
20140032286	Lansford et al.	Jan 2014	A1
20140040027	Anderson et al.	Feb 2014	A1
20140040443	Sheng-Jie Syu et al.	Feb 2014	A1
20140047480	Knudson et al.	Feb 2014	A1
20140074621	Chai et al.	Mar 2014	A1
20140082663	Neumeier et al.	Mar 2014	A1
20140090008	Li et al.	Mar 2014	A1
20140130076	Moore et al.	May 2014	A1
20140150006	Vemparala et al.	May 2014	A1
20140181856	Lewis	Jun 2014	A1
20140184827	Chartrand	Jul 2014	A1
20140195584	Harrison	Jul 2014	A1
20140195620	Srinivasan et al.	Jul 2014	A1
20140195649	Harrison	Jul 2014	A1
20140195690	Harrison et al.	Jul 2014	A1
20140195934	Harrison	Jul 2014	A1
20140201645	Mo et al.	Jul 2014	A1
20140201769	Neumeier et al.	Jul 2014	A1
20140201772	Neumeier et al.	Jul 2014	A1
20140201773	Neumeier et al.	Jul 2014	A1
20140201774	Neumeier et al.	Jul 2014	A1
20140201787	Neumeier et al.	Jul 2014	A1
20140229271	Clapp et al.	Aug 2014	A1
20140237496	Julian	Aug 2014	A1
20140244351	Symons	Aug 2014	A1
20140244863	Bradley et al.	Aug 2014	A1
20140282673	Neumeier et al.	Sep 2014	A1
20140289315	Harrison	Sep 2014	A1
20150082331	Neumeier et al.	Mar 2015	A1
20150095972	Sharma et al.	Apr 2015	A1
20150181268	Navin et al.	Jun 2015	A1
20150181311	Navin et al.	Jun 2015	A1
20150331660	Kalampoukas et al.	Nov 2015	A1
20150365456	Harrison	Dec 2015	A1
20160019598	Harrison	Jan 2016	A1

Foreign Referenced Citations (57)

Number	Date	Country
2413944	Jun 2003	CA
101909201	Jun 2013	CN
1347661	Sep 2003	EP
1592198	Nov 2005	EP
1605416	Dec 2005	EP
2188794	May 2010	EP
2226757	Sep 2010	EP
1887754	Dec 2012	EP
2541961	Jan 2013	EP
1969810	Aug 2013	EP
2520084	Nov 2013	EP
2285066	May 2014	EP
2747370	Jun 2014	EP
2200258	Nov 2014	EP
2457694	Aug 2009	GB
9512278	May 1995	WO
0052929	Sep 2000	WO
0054504	Sep 2000	WO
0144992	Jun 2001	WO
0182625	Nov 2001	WO
0189213	Nov 2001	WO
0189217	Nov 2001	WO
0231742	Apr 2002	WO
03025762	Mar 2003	WO
2005125198	Dec 2005	WO
2007114796	Oct 2007	WO
2008029188	Mar 2008	WO
2008052205	May 2008	WO
2008112858	Sep 2008	WO
2008131247	Oct 2008	WO
2009023647	Feb 2009	WO
2009091338	Jul 2009	WO
2009114622	Sep 2009	WO
2009131861	Oct 2009	WO
2009150425	Dec 2009	WO
2010046123	Apr 2010	WO
2010072986	Jul 2010	WO
2011011002	Jan 2011	WO
2011030231	Mar 2011	WO
2011090540	Jul 2011	WO
2011090541	Jul 2011	WO
2012005994	Jan 2012	WO
2012013893	Feb 2012	WO
2012021538	Feb 2012	WO
2012028976	Mar 2012	WO
2012051115	Apr 2012	WO
2012109666	Aug 2012	WO
2012120253	Sep 2012	WO
2013028899	Feb 2013	WO
2013068619	May 2013	WO
2013089674	Jun 2013	WO
2013147587	Oct 2013	WO
2014052015	Apr 2014	WO
2014142758	Sep 2014	WO
2014145929	Sep 2014	WO
2014145938	Sep 2014	WO
2014145947	Sep 2014	WO

Non-Patent Literature Citations (29)

Entry
“Sandboxes and Silver Bullets: Vendors Promote New/Old Detection Techniques to Stop Zero-Day Threats”, IT Current Analysis Connection Blogs, Mar. 29, 2013 by Paula Musich (p. 1). http://itcblogs.currentanalysis.com/2013/03/29/sandboxes-and-silver-bullets-vendors-promote-newold-detection-techniques-to-stop-zero-day-threats/.
“Screenshot of Wikipedia page of Samba TV”, Jan. 5, 2015 (pp. 2) http://en.wikipedia.org/wiki/Samba—TV.
“Screenshot of Wikipedia page of Smart TV”, Jan. 5, 2015 (pp. 4) http://en.wikipedia.org/wiki/Smart—TV.
“Screenshot of Wikipedia page of Interactive television”, From Wikipedia, Jan. 5, 2015 (pp. 8) http://en.wikipedia.org/wiki/Interactive—television.
“Screenshot of Wikipedia page of Social television”, From Wikipedia, Jan. 5, 2015 (pp. 3) http://en.wikipedia.org/wiki/Social—television.
“Screenshot of Wikipedia page of Enhanced TV”, From Wikipedia, Jan. 5, 2015 (p. 1) http://en.wikipedia.org/wiki/Enhanced—TV.
“Screenshot of Wikipedia page of Digital video fingerprinting”, From Wikipedia, Jan. 5, 2015 (pp. 4) http://en.wikipedia.org/wiki/Digital—video—fingerprinting.
“Screenshot of Wikipedia page of Second screen”, From Wikipedia, Jan. 5, 2015 (pp. 3) http://en.wikipedia.org/wiki/Second—screen.
“Architecture for Interoperability of Services between an ACAP Receiver and Home Networked Devices”, Jan. 15, 2006, by Yu-Seok Bae et al. (pp. 6).
“Design of a multi-sender 3D videoconferencing application over an end system multicast protocol”, '03 Proceedings of the eleventh ACM international conference on Multimedia, New York, NY, USA, 2003 by Mojtaba Hosseini et al., (p. 1) http://dl.acm.org/citation.cfm?id=957119.
“Delivery of Personalized and Adaptive Content to Mobile Devices: A Framework and Enabling Technology”, Communications of the Association for Information Systems (vol. 12, 2003)183-202, by D. Zhang, (pp. 22) http://aisel.aisnet.org/cgi/viewcontent.cgi?article=3178&context=cais.
“MonALISA : A Distributed Monitoring Service Architecture”, CHEP03, La Jolla, California, Mar. 24-28, 2003 by H.B. Newman et al. (pp. 8) http://monalisa.caltech.edu/documentation/MOET001.pdf.
“Exploratory geospatial analysis using GeoVISTA Studio: from a desktop to the Web”, IEEE Xplore, Dec. 3-6, 2001, by M. Takatsuka et al. (p. 1). http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=996715&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs—all.jsp%3Farnumber%3D996715.
“A Framework for Classifying Peer-to-Peer Technologies”, IEEE Xplore, May 21-24, 2002, by K. Kant et al. (p. 1) http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=1540491&url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel5%2F10335%2F32898%2F01540491.pdf%3Farnumber%3D1540491.
“HTML & CSS: The Complete Reference”, The McGraw-Hill Companies, 2010 by Thomas A. Powell (pp. 857) http://www.pdfiles.com/pdf/files/English/Web—Apps—Programming—&—Internet/HTML—&—CSS—The—Complete—Reference.pdf.
“Web Services Essentials”, O'Reilly, Feb. 2002 by Ethan Cerami (pp. 286) http://spurrier.gatorglory.com/PDFs/O'Reilly%20-%20Web%20Services%20Essentials.pdf.
“UPnP Device Architecture 1.0”, UPnP Forum, Oct. 15, 2008 (pp. 81). http://upnp.org/specs/arch/UPnP-arch-DeviceArchitecture-v1.0.pdf.
“Mac OS X Snow Leopard: The Missing Manual”, O'Reilly Media, Inc., 2009 by David Pogue (pp. 903) http://crypto.cs.mcgill.ca˜simonpie/webdav/ipad/EBook/MacOSX/Mac%20OS%20X%20Snow%20Leopard%20The%20Missing%20Manual.pdf.
“The Common Object Request Broker: Architecture and Specification Revision 2.0”, Feb. 1997 (pp. 634) http://www.omg.org/spec/CORBA/2.0/PDF.
“Internet: The Complete Reference”, Tata McGraw-Hill Education Pvt. Ltd., 2002 by Margaret Levine Young http://www.abebooks.com/Internet-Complete-Reference-Second-Edition-Margaret/5122896620/bd.
“HTML 4.0 Sourcebook”, John Wiley & Sons, Apr. 1998 by Ian S. Graham (pp. 656) http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471257249.html.
“Zero Configuration Networking: The Definitive Guide”, O'Reilly Media, Dec. 2005 by Daniel H Steinberg et al. (pp. 254) http://shop.oreilly.com/product/9780596101008.do#tab—04—2.
“Cognitive Radio Technology”, from the Guest Editor in IEEE Signal Processing Magazine on Nov. 2008 by Maria Gabriella di Benedetto et al. (p. 1) http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=4644050.
“Automated Content Recognition creating content aware ecosystems”, in CSI Magazine in Sep. 2012 (pp. 16) http://www.csimagazine.com/csi/whitepapers/ACR%20Creating%20%20content-aware%20ecosystems%20-Civolution%20White%20Paper%20-%20Sept%202012.pdf.
“A Confidence Based Recognition System for TV Commercial Extraction”, in 2008 by Yijun Li et al. (pp. 8) http://crpit.com/confpapers/CRPITV75Li.pdf.
“TV Retargeting”, Market View, wywy—Maximizing TV Advertising ROI, Dec. 31, 2015 (pp. 2) http://wywy.com/market-view/tv-retargeting/.
“The New Age of Second Screen: Enabling Interaction”, adrnonsters webpage, Jun. 5, 2013, Joshua R. Weaver (pp. 3) https://www.admonsters.com/blog/second-screen-enabling-interaction.
“Complaint for Patent Infringement”, Case 2:15-cv-01725-RWS Document 1, Nov. 6, 2015 (pp. 7).
“Merriam-Webster, Launch”, 2015.

Related Publications (1)

	Number	Date	Country
	20140007156 A1	Jan 2014	US

Provisional Applications (3)

Number	Date	Country
61696711	Sep 2012	US
61118286	Nov 2008	US
61584168	Jan 2012	US

Continuations (4)

	Number	Date	Country
Parent	13470814	May 2012	US
Child	14018396		US
Parent	12592377	Nov 2009	US
Child	13470814		US
Parent	14018396		US
Child	13470814		US
Parent	13736031	Jan 2013	US
Child	14018396		US

Annotation of metadata through capture infrastructure

Information

Patent Number

Date Filed

Date Issued

Inventors

Original Assignees

Examiners

Agents

CPC

Field of Search

US

International Classifications

Disclaimer

Term Extension

Abstract