Patent Grant
11336551
A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever.
The present invention relates generally to the field of content and/or data delivery over a network. More particularly, the present invention is related in one exemplary aspect to apparatus and methods for identifying and characterizing latency in the delivery of content and/or data to devices in the network.
Several factors such as e.g., satellite latency, number of physical devices in the headend, the number of amplifiers in the RF plant, the distance of a user device such as a settop box (whether stand alone or within another user device) from the headend, the amount of signal conductor medium (e.g., fiber/coaxial cable) utilized, set top box type, etc. contribute to latency experienced when content “should” be displayed to a user, and when it is actually displayed. Generally, this latency is small (for example it may be only a few seconds). When “rating” broadcast viewing for programming content (such as via the well-known “Nielsen Ratings”), the aforementioned latency (of e.g., a few seconds) is insignificant because the programming content is generally at least 30 minutes in length. However, when rating a commercial (e.g., determining the effectiveness of a commercial, the number of quantifiable views of the commercial, etc.), which may be as short as a few seconds, these latencies become increasingly more significant and potentially problematic.
The aforementioned Nielsen Ratings are a well known system of evaluating the viewing habits of cross sections of the population. Statistical techniques are used to develop a sample population which is a cross section of a larger national population. Theoretically, the viewing habits of the sample population will mirror the larger population. Historically, the Nielsen system has been the primary source of audience measurement information in the television industry. The Nielsen system, therefore, affects various aspects of television including inter alia, advertising rates, schedules, viability of particular shows, etc. However, there are several disadvantages to the Nielsen approach, including e.g., the inability of the system to account for small measures of time (e.g., generate second-by-second viewing statistics).
A variety of entities in the content delivery system may be responsible for introducing latency, including e.g., the content source or store, the user device architecture, and the network topology.
Latency may be introduced at the user device (e.g., settop box (STB), digital settop box (DSTB), consumer premises equipment (CPE), etc.). Each user device manufacturer and software provider develops an architecture that leads necessarily to a discrete average amount of time between an acquired MPEG signal being received “on the wire” (whether wired or wireless) and decoded and sent to a television or other device for viewing. This latency is predictable and can be characterized in a lab environment. Thus, an operator may include in its internal description of a user device (e.g., STB, DSTB, CPE, etc.) a “mean signal processing time” for each model/software combination. For example, user device induced latency may be illustrated by looking at two different televisions in a subscriber's home. Suppose that one of the televisions is connected to an STB which does not have DVR capability, and the other is connected to an STB that does have DVR capability. It will be generally observed that the content being viewed on the television attached to the STB with DVR capabilities is “delayed” by some small amount of time as compared to the content being viewed on the other television. This is due to, inter alia, the DVR storage media (e.g., HDD) access time.
Plant topology is not as easily characterized as that of the user devices. Though it is possible to determine through experimentation the incremental latency incurred “per mile of coaxial cable” or “per mile of fiber” or “per amplifier”, there is often no reliable way to know how many miles of fiber or number of amplifiers lie between a deployed device and the headend. Therefore, a mechanism must be devised to determine real-time the plant topology induced latency for each user device deployed in the network, which may vary from user device to user device. The latency may also be introduced through digital signal processing that occurs in the headend.
Generally, latency does not affect the quality or level of service an MSO provides from the perspective of signal quality or resolution. However, the aforementioned latency does effect interactive abilities and census level advertising consumption and interaction calculations, and may adversely impact user experience. Specifically, when an MSO wishes to offer or enable programmers to offer real-time interactive components (such as voting, t-commerce, etc.), depending on the nature of the application, the aforementioned latency may affect the quality of service a subscriber receives.
Likewise, when an MSO wishes to provide advertisers or networks a “second-by-second” view of subscriber consumption of and response to advertising content, normalizing for latency is critical for accurate representations. For example, if a user tuned away from an advertisement 5 seconds into the advertisement (in actual time, and/or from the user's perspective on the settop box), but the latency made it appear that he/she tuned away after 15 seconds (or the user's settop box may have introduced a latency of 10 seconds) then the advertiser might draw an erroneous conclusion from this data (i.e., if the advertisement duration was 15 seconds total, the advertiser might conclude that the user watched the entire advertisement, when in fact they tuned away almost immediately).
Under current (e.g., Nielsen) rating schemes, programs are credited with “views” if there is any viewing within a quarter hour. That is to say, every 15 min, under the Nielsen system, an impression is logged. However, advertisements may be as short as 15 seconds. Thus, in order to accurately account for advertising views and/or to generate meaningful second-by-second ratings, the offset in the network latency can be significant and, when significant, should be accounted for.
Therefore, what is needed are methods and apparatus able to mark impressions on a smaller scale (e.g., on the order of seconds versus minutes). In order to accredit these impressions, so that for example they may be used as a “currency”, the Media Ratings Council (MRC) requires evidence that the latency is accounted for. Moreover, such methods and apparatus would also ideally be further adapted to compensate for such latency when gathering audience information in real-time in order to identify viewership actions of actual viewers.
These features would also be provided using substantially extant network infrastructure and components (thereby obviating any significant retrofit or re-engineering), and would be compatible with a number of different client device and delivery systems including both wired and wireless technologies.
The present invention addresses the foregoing needs by disclosing, inter alia, apparatus and methods for identification and characterization of, and/or compensation for, latency in a content delivery network.
In a first aspect of the invention, a method for providing data is disclosed. In one embodiment, the data relates to interaction of a client device of a content distribution network with one or more portions of content provided over the network, and the method comprises: determining a latency attributable to the device; determining a latency attributable to a path within the network which the content must take to be provided to the device; utilizing the latency attributable to the device and the latency attributable to the network path to determine a aggregate latency specific to the device; collecting data relating to interaction of the device with the one or more portions of the content; and accounting for the aggregate latency with respect to the data.
In a second aspect of the invention, a method for accounting for an asynchronization between an entity for providing content in a content delivery network and a plurality of devices for receiving the content therefrom is disclosed. In one embodiment, the asynchronization is unique for each of the plurality of devices, and the method comprises: determining values for a latency attributable to respective ones of the plurality of devices, respectively; determining values for a latency attributable to respective network paths taken for the delivery of the content to each of the plurality of devices; and for each of the plurality of devices, utilizing the latency attributable thereto, and the latency attributable to the respective one of the network paths, to determine composite latencies unique to each of the plurality of devices.
In a third aspect of the invention, a method for determining a system latency specific to an individual one of a plurality of devices in a content delivery network is disclosed. In one embodiment, the method comprises: collecting a value for latency attributable to the individual one of the plurality of devices; sending a message requiring a response to the individual one of the plurality of devices at a first time; receiving, in response to the message the response from the individual one of the plurality of devices, the response indicating a second time; and applying at least one algorithm to: determine a portion of the difference between the second and the first time attributable to a one-way traversal of the network to the individual one device; and combine the portion of the difference with the value for latency attributable to the individual one of the plurality of devices to obtain a system latency for that individual one device.
In a fourth aspect of the invention, consumer premises equipment (CPE) for use in a content delivery network is disclosed. In one embodiment, the CPE comprises: at least one first interface configured to communicate with the content delivery network; a storage apparatus; and a digital processor, a caching mechanism, the digital processor configured to run at least one computer program thereon. The program is configured to, when executed: determine a measure of time which the CPE is out of synchronization to at least one entity of the network; collect a plurality of records, each of the plurality of records indicating: a time of collection of the record; and a descriptor of an interaction of a user of the CPE with one or more content elements; and re-synchronize the collected plurality of records by adjusting the time of the collection for each of the records by the measure of time.
In a fifth aspect of the invention, a network apparatus is disclosed. In one embodiment, the apparatus is configured for determining amounts of time by which records collected by a user device must be adjusted to account for latency in a content delivery network, and comprises: at least one first interface configured to communicate at least the user device; a storage apparatus; and a digital processor, the digital processor configured to run at least one computer program thereon. The program is configured to, when executed: obtain a first value representative of processing delay at the user device; derive a second value associated with a transmission delay between the network apparatus and the user device; and utilize the first and the second values to calculate a third value, the third value representing a unique amount of time individual ones of the records collected at the user device must be adjusted.
In a sixth aspect of the invention, a method of obtaining data relating to content is disclosed. In one embodiment, the method comprises: delivering content to a plurality of users of a network; obtaining data relating to interactions of the users with the delivered content; and adjusting the data so that it accurately reflects the timing of the interactions relative to delivery of the content.
In a seventh aspect of the invention, a computer readable apparatus is disclosed. In one embodiment, the apparatus comprises a medium having at least one program stored thereon, the at least one program being configured to at least collect data relating to user interactions with content, and correct the data for any asynchronizations.
In an eighth aspect of the invention, a system for collecting and distributing data relating to user interactions with content is disclosed.
In a ninth aspect of the invention, methods of doing business relating to data describing user interactions are disclosed.
In another aspect of the disclosure, a computerized method of adjusting for an asynchronization between an entity for providing digital content in a content delivery network and a plurality of computerized client devices for receiving the digital content therefrom, is disclosed. In one embodiment, the computerized method includes: determining a plurality of first values of latency attributable to respective ones of the plurality of computerized client devices; determining a plurality of second values of latency attributable to respective ones of a plurality of network paths which may be taken for delivery of the digital content to each one of the plurality of computerized client devices; for the each one of the plurality of computerized client devices, utilizing (i) a first value of the latency attributable to a respective one of the plurality of computerized client devices, and (ii) a second value of the latency attributable to a respective one of the network paths, to determine a respective plurality of third values of composite latency; obtaining a plurality of data records relating to an asynchronization between the entity for providing the digital content and the plurality of computerized client devices; and modifying at least one of the plurality of records based at least on one or more of the plurality of third values of the composite latency.
In one variant, each of the plurality of third values of the composite latency is unique to a respective one of the plurality of computerized client devices.
In another aspect, consumer premises equipment (CPE) for use in a content delivery network is disclosed. In one embodiment, the CPE includes: at least one data interface configured for data communication with the content delivery network; a storage apparatus; and a digital processor, the digital processor in data communication with at least the at least one data interface and the storage apparatus.
In one variant, the storage apparatus includes at least one computer program configured to, when executed, cause the CPE to: determine a measure of time by which the CPE is out of synchronization with respect to at least one entity of the content delivery network; collect a plurality of records, each one of the plurality of records indicating: a time of the collection of the record; and a descriptor of an interaction of a user of the CPE with one or more digital content elements; and re-synchronize the collected plurality of records by adjusting the time of the collection for the each one of the records by the measure of time.
In a further aspect, a computerized network apparatus for determining temporal adjustments for a plurality of data records collected by a computerized user device, is disclosed. In one embodiment, the temporal adjustments to account for asynchronization relating to the data records based at least on latency in a content delivery network, and the computerized network apparatus includes: at least one network interface apparatus configured for data communication with at least the computerized user device; digital processor apparatus in data communication with at least the at least one network interface apparatus; and storage apparatus in data communication with the digital processor apparatus and including a non-transitory storage medium having at least one computer program thereon.
In one variant, the at least one computer program is configured to, when executed by the digital processor apparatus, cause the computerized network apparatus to: obtain a first value representative of processing delay at the computerized user device; derive a second value associated with a transmission delay between the computerized network apparatus and the computerized user device; and utilize at least the first and the second values to calculate a third value, the third value representing a unique amount of time by which individual ones of the data records collected at the computerized user device must be adjusted.
These and other aspects of the invention shall become apparent when considered in light of the disclosure provided herein.
All Figures © Copyright 2010 Time Warner Cable, Inc. All rights reserved.
Reference is now made to the drawings wherein like numerals refer to like parts throughout.
As used herein, the tetra “application” refers generally to a unit of executable software that implements a certain functionality or theme. The themes of applications vary broadly across any number of disciplines and functions (such as on-demand content management, e-commerce transactions, brokerage transactions, home entertainment, calculator etc.), and one application may have more than one theme. The unit of executable software generally runs in a predetermined environment; for example, the unit could comprise a downloadable Java Xlet™ that runs within the JavaTV™ environment.
As used herein, the terms “client device” and “end user device” include, but are not limited to, set top boxes (e.g., DSTBs), personal computers (PCs), and minicomputers, whether desktop, laptop, or otherwise, and mobile devices such as handheld computers, PDAs, personal media devices (PMDs), and smartphones.
As used herein, the term “computer program” or “software” is meant to include any sequence or human or machine cognizable steps which perform a function. Such program may be rendered in virtually any programming language or environment including, for example, C/C++, Fortran, COBOL, PASCAL, assembly language, markup languages (e.g., HTML, SGML, XML, VoXML), and the like, as well as object-oriented environments such as the Common Object Request Broker Architecture (CORBA), Java™ (including J2ME, Java Beans, etc.) and the like.
The terms “Customer Premises Equipment (CPE)” and “host device” refer without limitation to any type of electronic equipment located within a customer's or user's premises and connected to a network.
As used herein, the term “display” means any type of device adapted to display information, including without limitation CRTs, LCDs, TFTs, plasma displays, LEDs, incandescent and fluorescent devices, or combinations/integrations thereof. Display devices may also include less dynamic devices such as, for example, printers, e-ink devices, and the like.
As used herein, the term “DOCSIS” refers to any of the existing or planned variants of the Data Over Cable Services Interface Specification, including for example DOCSIS versions 1.0, 1.1, 2.0 and 3.0. DOCSIS (version 1.0) is a standard and protocol for internet access using a “digital” cable network.
As used herein, the term “headend” refers generally to a networked system controlled by an operator (e.g., an MSO) that distributes programming to MSO clientele using client devices. Such programming may include literally any information source/receiver including, inter alia, free-to-air TV channels, pay TV channels, interactive TV, and the Internet.
As used herein, the terms “Internet” and “internet” are used interchangeably to refer to inter-networks including, without limitation, the Internet.
As used herein, the terms “microprocessor” and “digital processor” are meant generally to include all types of digital processing devices including, without limitation, digital signal processors (DSPs), reduced instruction set computers (RISC), general-purpose (CISC) processors, microprocessors, gate arrays (e.g., FPGAs), PLDs, reconfigurable compute fabrics (RCFs), array processors, and application-specific integrated circuits (ASICs). Such digital processors may be contained on a single unitary IC die, or distributed across multiple components.
As used herein, the terms “MSO” or “multiple systems operator” refer to a cable, satellite, or terrestrial network provider having infrastructure required to deliver services including programming and data over those mediums.
As used herein, the terms “network” and “bearer network” refer generally to any type of telecommunications or data network including, without limitation, hybrid fiber coax (HFC) networks, satellite networks, telco networks, and data networks (including MANs, WANs, LANs, WLANs, internets, and intranets). Such networks or portions thereof may utilize any one or more different topologies (e.g., ring, bus, star, loop, etc.), transmission media (e.g., wired/RF cable, RF wireless, millimeter wave, optical, etc.) and/or communications or networking protocols (e.g., SONET, DOCSIS, IEEE Std. 802.3, ATM, X.25, Frame Relay, 3GPP, 3GPP2, WAP, SIP, UDP, FTP, RTP/RTCP, H.323, etc.).
As used herein, the term “network interface” refers to any signal or data interface with a component or network including, without limitation, those of the FireWire (e.g., FW400, FW800, etc.), USB (e.g., USB2), Ethernet (e.g., 10/100, 10/100/1000 (Gigabit Ethernet), 10-Gig-E, etc.), MoCA, Coaxsys (e.g., TVnet™), radio frequency tuner (e.g., in-band or OOB, cable modem, etc.), Wi-Fi (802.11a,b,g,n), WiMAX (802.16), PAN (e.g., 802.15), or IrDA families.
As used herein, the term “QAM” refers to modulation schemes used for sending signals over cable networks. Such modulation scheme might use any constellation level (e.g. QPSK, 16-QAM, 64-QAM, 256-QAM, etc.) depending on details of a cable network. A QAM may also refer to a physical channel modulated according to the schemes.
As used herein, the term “server” refers to any computerized component, system or entity regardless of form which is adapted to provide data, files, applications, content, or other services to one or more other devices or entities on a computer network.
As used herein, the term “storage device” refers to without limitation computer hard drives, DVR device, memory, RAID devices or arrays, optical media (e.g., CD-ROMs, Laserdiscs, Blu-Ray, etc.), or any other devices or media capable of storing content or other information.
As used herein, the term “Wi-Fi” refers to, without limitation, any of the variants of IEEE-Std. 802.11 or related standards including 802.11 a/b/g/n/v.
As used herein, the term “wireless” means any wireless signal, data, communication, or other interface including without limitation Wi-Fi, Bluetooth, 3G, HSDPA/HSUPA, TDMA, CDMA (e.g., IS-95A, WCDMA, etc.), FHSS, DSSS, GSM, PAN/802.15, WiMAX (802.16), 802.20, narrowband/FDMA, OFDM, PCS/DCS, analog cellular, CDPD, satellite systems, millimeter wave or microwave systems, acoustic, and infrared (i.e., IrDA).
The present invention discloses, inter alia, methods and apparatus for identification and characterization of, and compensation for, latency that may be present in a content delivery network. Interaction of users with content is recorded in one embodiment via the collection of a plurality of tuning records. However, as noted above, the network entities providing content and the client devices may be out of synchronization due to latency. Hence, the architecture and methods described herein may be used to adjust the timing on the tuning records collected from the devices (e.g., “normalize” the timestamps thereof). Once normalized, these tuning records may be relied upon as being accurate representations of subscriber interaction with content on a second-by-second basis. Hence, records may be obtained for the interaction of users with content which lasts less than a minute (e.g. advertisements), and even as short as a few seconds.
In one embodiment, the amount of time which must be accounted for (i.e., the amount by which the timing of the tuning records must be adjusted) is determined by first determining a device-specific latency. The device-specific latency may be derived empirically (e.g., by experimentation) and/or calculations prior to implementing the device in the network. The device-specific latency depends on the hardware and software features of the device; hence, devices having similar hardware and software and being manufactured by the same manufacturer will have very similar if not identical device-specific latencies. The device-specific latency is added to the latency inflicted by the network during transmission of data therein from the source to the premises. In one variant, the network latency is determined by sending an example or test message to the device from the network requesting a current system time (or other response). The time expended for a single (one-way) trip between these entities is in one variant derived by taking half of the difference in time between when the message was sent and the time the response is received by the issuing entity (i.e., the round-trip time). In another variant, the latency is determined by examining a time-stamped response message, the time stamp indicating when the test message was received at the destination (e.g., CPE). Under the various models, the determination of the system latency may be made at the device itself, and/or at the network entity.
Exemplary implementations of the invention also obtain audience information records (e.g., tuning records) directly from customer's premises equipment (i.e., set top boxes, cable modems, PCs, PMDs, IP devices, etc.), for each individual device, or even on a per-user basis where possible, thereby allowing a content provider or other analytical entity to gather specific information in large quantities across a broad geographical area, or demographic/psychographic slice. The information collected may include information relating to e.g., the user's interaction with content, such as tuning in, tuning out, tuning away, trick mode operations, etc. Advantageously, multiple sources of content to which viewership behavior relates can be simultaneously monitored, and subscriber anonymity or privacy maintained (i.e., no use is made of personally identifiable information). The aforementioned latency value is then reflected in the tuning records, so that each accurately represents the time at which the information is obtained.
Business models and rules for the implementation of the aforementioned methods and for the identification, characterization and application of latency to data relating to a user's interaction with content are also described.
Exemplary embodiments of the apparatus and methods of the present invention are now described in detail. While these exemplary embodiments are described in the context of use with the aforementioned hybrid fiber (e.g., HFC) terrestrial delivery system or satellite network architecture having an multiple systems operator (MSO), digital networking capability, IP delivery capability, and plurality of client devices/CPE, the general principles and advantages of the invention may be extended to other types of networks and architectures, whether broadband, narrowband, wired or wireless, or otherwise, the following therefore being merely exemplary in nature. For instance, the invention may be adapted for use on so-called hybrid fiber copper (HFCu) networks, or WiMAX (IEEE Std. 802.16) wireless networks.
It will also be appreciated that while described generally in the context of a consumer (i.e., home) end user domain, the present invention may be readily adapted to other types of environments (e.g., commercial/enterprise, government/military, etc.) as well. Myriad other applications are possible.
Other features and advantages of the present invention will immediately be recognized by persons of ordinary skill in the art with reference to the attached drawings and detailed description of exemplary embodiments as given below.
The data/application origination point 102 comprises any medium that allows data and/or applications (such as a VOD-based or “Watch TV” application) to be transferred to a distribution server 104. This can include for example a third party data source, application vendor website, CD-ROM, external network interface, mass storage device (e.g., RAID system), etc. Such transference may be automatic, initiated upon the occurrence of one or more specified events (such as the receipt of a request packet or ACK), performed manually, or accomplished in any number of other modes readily recognized by those of ordinary skill.
The application distribution server 104 comprises a computer system where such applications can enter the network system. Distribution servers are well known in the networking arts, and accordingly not described further herein.
The VOD server 105 comprises a computer system where on-demand content can be received from one or more of the aforementioned data sources 102 and enter the network system. These servers may generate the content locally, or alternatively act as a gateway or intermediary from a distant source.
The CPE 106 includes any equipment in the “customers' premises” (or other locations, whether local or remote to the distribution server 104) that can be accessed by a distribution server 104.
Although not illustrated, a typical network headend 150 may further include e.g., various billing entities, subscriber management systems, cable modem termination system (CMTS)
It will also be appreciated that the network configuration depicted in
The exemplary headend 150 may further include a multiplexer-encrypter-modulator (MEM) adapted to process or condition content for transmission over the network. As previously described, information is carried across multiple channels. Thus, the headend 150 must be adapted to acquire the information for the carried channels from various sources. Typically, the channels being delivered from the headend 150 to the CPE 106 (“downstream”) are multiplexed together in the headend, as previously described and sent to neighborhood hubs (
It will also be recognized, however, that the multiplexing operation(s) need not necessarily occur at the headend 150 (e.g., in the aforementioned MEM). As one alternative, a multi-location or multi-stage approach can be used, such as that described in U.S. Pat. No. 7,602,820, entitled “APPARATUS AND METHODS FOR MULTI-STAGE MULTIPLEXING IN A NETWORK” incorporated herein by reference in its entirety, which discloses inter alia improved multiplexing apparatus and methods that allow such systems to dynamically compensate for content (e.g., advertisements, promotions, or other programs) that is inserted at a downstream network node such as a local hub, as well as “feed-back” and “feed forward” mechanisms for transferring information between multiplexing stages.
Content (e.g., audio, video, data, files, etc.) is provided in each downstream (in-band) channel associated with the relevant service group. To communicate with the headend or intermediary node (e.g., hub server), the CPE 106 may use the out-of-band (OOB) or DOCSIS channels and associated protocols. The OCAP 1.0 (and subsequent) specification provides for exemplary networking protocols both downstream and upstream, although the invention is in no way limited to these approaches.
It will also be recognized that the multiple servers (broadcast, VOD, or otherwise) can be used, and disposed at two or more different locations if desired, such as being part of different server “farms”. These multiple servers can be used to feed one service group, or alternatively different service groups. In a simple architecture, a single server is used to feed one or more service groups. In another variant, multiple servers located at the same location are used to feed one or more service groups. In yet another variant, multiple servers disposed at different location are used to feed one or more service groups.
In addition to on-demand and broadcast content (e.g., video programming), the system of
The CPE 106 are each configured to monitor the particular assigned RF channel (such as via a port or socket ID/address, or other such mechanism) for IP packets intended for the subscriber premises/address that they serve.
Switching architectures allow improved efficiency of bandwidth use for ordinary digital broadcast programs. Ideally, the subscriber is unaware of any difference between programs delivered using a switched network and ordinary streaming broadcast delivery.
Co-owned and co-pending U.S. patent application Ser. No. 09/956,688 filed Sep. 20, 2001, entitled “Technique For Effectively Providing Program Material In A Cable Television System”, and issued as U.S. Pat. No. 8,713,623 on Apr. 29, 2014, incorporated herein by reference in its entirety, describes one exemplary broadcast switched digital architecture useful with the present invention, although it will be recognized by those of ordinary skill that other approaches and architectures may be substituted.
Referring again to
The edge switch 194 forwards the packets receive from the CMTS 199 to the QAM modulator 189, which transmits the packets on one or more physical (QAM-modulated RF) channels to the CPE. The IP packets are typically transmitted on RF channels that are different than the RF channels used for the broadcast video and audio programming, although this is not a requirement. The CPE 106 are each configured to monitor the particular assigned RF channel (such as via a port or socket ID/address, or other such mechanism) for IP packets intended for the subscriber premises/address that they serve.
While the foregoing network architectures described herein can (and in fact do) carry packetized content (e.g., IP over MPEG for high-speed data or Internet TV, MPEG2 packet content over QAM for MPTS, etc.), they are often not optimized for such delivery. Hence, in accordance with another embodiment of the present invention, a “packet optimized” delivery network is used for carriage of the packet content (e.g., IPTV content) when the request issues from an MSO network (see discussion of
Referring now to
The architecture of
The tuning record processing entity 200 is in communication with a plurality of user devices or customer premises equipment (CPE) 106, which may include, inter alia, personal media devices (PMDs), laptop and personal computers (PCs), set top boxes (STBs), digital video recorders (DVRs), etc., via the network 101. As illustrated, each of the CPE 106 comprises in one embodiment a client latency characterization application 208 running thereon, which will be described in greater detail below. Alternatively, the CPE 106 may comprise a legacy or OEM device which merely can time-stamp received messages or packets as to when they are received.
The CPE 106 in one embodiment comprises a gateway device such as that discussed in co-owned U.S. patent application Ser. No. 11/818,236 filed Jun. 13, 2007, entitled “PREMISES GATEWAY APPARATUS AND METHODS FOR USE IN A CONTENT-BASED NETWORK”, and issued as U.S. Pat. No. 7,954,131 on May 31, 2011, which is incorporated herein by reference in its entirety. As discussed therein, the gateway acts as a unified proxy for all inbound (downstream) and outbound (upstream) communications with a network. In this way, various user devices within a premises may receive data and content via the gateway apparatus.
The network tuning record processing entity 200 is configured to run at least a network latency characterization application 202 process and a tuning record collection, validation and reporting 204 process thereon. Although illustrated and described as comprising software running on a processor of the tuning record processing entity 200 (e.g., a server), it is appreciated that these processes 202, 204 may alternatively take the form of a hardware device or logic, combination of hardware and software, or any other form suitable to achieve the desired degree of automation and processing. Likewise, one or more of the aforementioned processes 202, 204 may be located at a different entity, whether at the headend 150 or elsewhere. It is further appreciated that the tuning record processing entity 200 may be physically located at any other location whether within the network 101 or in a separate network (not shown) in communication therewith.
Content and data are provided to the CPE 106 from the network 101. In one embodiment, content and/or data are provided to the CPE 106 using e.g., a Moving Picture Experts Group (MPEG) transport stream. The MPEG protocol uses Program and System Information. Protocol (PSIP) to deliver content, as discussed in “ADVANCED TELEVISION SYSTEMS COMMITTEE (ATSC) STANDARD: PROGRAM AND SYSTEM INFORMATION PROTOCOL FOR TERRESTRIAL BROADCAST AND CABLE (PSIP)”, Document A/65; published Apr. 14, 2009 and incorporated herein by reference in its entirety. Since PSIP has to propagate through all the wiring, equipment, interfaces, etc. in order to deliver content and/or data to the individual devices, latency is introduced. Hence, a method is utilized in the present invention to identify and characterize this latency (see e.g.,
As discussed in greater detail below with respect to
Latency for the system is determined in one embodiment via the network latency characterization application 202 and the client latency characterization application 208. This latency is applied to the data regarding activity taken with respect to content on the user devices (i.e., tuning records). In other words, the tuning records are normalized (to account for the latency). The normalized records are then validated, and in some cases analyzed at the network tuning record processing entity 200, before being reported out or packaged for subsequent consumption. The processed tuning records (having been normalized, validated and optionally analyzed) are reported to an analyzation entity 206. In the illustrated embodiment, the analyzation entity 206 is located outside of the MSO network 101 of
The tuning records utilized in the exemplary embodiment of the invention may comprise a plurality of types of data. For example, records may be collected relating to: (i) requests to receive specific content elements (e.g., movie, game, etc.) at particular devices such as CPE 106, PMD, etc. (e.g., “tune in” events), (ii) the number of times the content element is requested, (iii) other events or functions such as “trick mode” operations employed with respect to content including e.g., fast forward, rewind, pause, play, etc., (iv) requests to terminate viewing of specific content elements (e.g., “tune away” events), (v) requests to terminate viewing altogether (e.g., “tune out” events), etc. This data may be analyzed with respect to the requesting devices, including e.g., the frequency of fast forward requests during certain types of programming, the subscriber associated to the device, group of subscribers, devices, households, geographic or demographic areas, etc.
Analyzation of the data, such as at the tuning record processing entity 200 and/or at the analyzation entity 206, may be utilized to generate reports. For example, these reports may relate to the number of requests by one or more subscribers, devices, households, geographic zones, demographics, etc. or over a particular time period. In this way, second-by-second data regarding a plurality of users' interaction with content, which would generally tend to be too voluminous and detailed to be useful, may be accredited and summarized to produce useful data. In one embodiment, the methods and apparatus of the previously incorporated U.S. application Ser. No. 12/829,104 entitled “APPARATUS AND METHODS FOR DATA COLLECTION AND VALIDATION” are utilized for analyzing the collected and normalized tuning records, although it will be appreciated that other approaches may be used with equal success consistent with the present invention.
The analyzed data, may also be utilized to support any number of business models including e.g., to make business or operational decisions, make programming or advertising selection decisions “on the fly”, etc., as discussed below with respect to the Business/Operational Rules Engine section hereof.
The tuning records (i.e., data regarding the user's interaction with content) may include interaction with various different types or delivery modes of content. For example, data may be collected regarding the users interaction with linear and/or switched digital broadcast content, VOD/MVOD/FVOD (or other type of on-demand content), content from a personal video recorder (PVR) or digital video recorder (DVR), whether local to the premises or network-based, IPTV content, etc. Further, the requested/provided content may comprise, for example, so called “quick clips” content (described in co-owned U.S. Pat. No. 7,174,126 issued Feb. 6, 2007 and entitled “TECHNIQUE FOR EFFECTIVELY ACCESSING PROGRAMMING LISTING INFORMATION IN AN ENTERTAINMENT DELIVERY SYSTEM” incorporated herein by reference in its entirety), so-called “start-over” content (described in co-owned, co-pending U.S. Patent Publication No. 2005/0034171 entitled “TECHNIQUE FOR DELIVERING PROGRAMMING CONTENT BASED ON A MODIFIED NETWORK PERSONAL VIDEO RECORDER SERVICE” incorporated herein by reference in its entirety), so-called “lookback” content (as described in co-owned, co-pending U.S. patent application Ser. No. 10/913,064 filed Aug. 6, 2004 and entitled “TECHNIQUE FOR DELIVERING PROGRAMMING CONTENT BASED ON A MODIFIED NETWORK PERSONAL VIDEO RECORDER SERVICE” incorporated herein by reference in its entirety), and/or so-called “remote DVR” content (as discussed in co-owned U.S. Pat. No. 7,457,520 issued Nov. 25, 2008 and entitled “TECHNIQUE FOR PROVIDING A VIRTUAL DIGITAL VIDEO RECORDER SERVICE THROUGH A COMMUNICATIONS NETWORK” incorporated herein by reference in its entirety).
Still further, enhanced access to premium content which is not available to non-subscribers or which cannot be delivered across traditional transport may also be provided, such as e.g., behind the scenes outtakes, alternate endings, actor interviews, etc. and data collected relating thereto as well. In yet a further embodiment, the content may comprise interactive content such as that described in co-owned U.S. patent application Ser. No. 12/582,619 filed Oct. 20, 2009, entitled “GATEWAY APPARATUS AND METHODS FOR DIGITAL CONTENT DELIVERY IN A NETWORK”, and issued as U.S. Pat. No. 9,027,062 on May 5, 2015, and in co-owned U.S. patent application Ser. No. 12/582,653 filed Oct. 20, 2009, entitled “METHODS AND APPARATUS FOR ENABLING MEDIA FUNCTIONALITY IN A CONTENT-BASED NETWORK”, and issued as U.S. Pat. No. 8,396,055 on Mar. 12, 2013, each of which is incorporated herein by reference in its entirety.
Referring now to
In the illustrated embodiment, the remaining functions (e.g., collection validation and reporting) are performed at the network tuning record processing entity 200, as in the architecture of
In another embodiment, rather than comprising separate entities, the functionality or processes of the network tuning record processing entity 200 are implemented as a series of distributed application portions located across various entities in the network.
The communication between the CPE 106 and the analyzation entity 206 occurs via any number of different modes such as e.g., via IP packets carried over the Internet 111 (as illustrated). In one implementation, this communication “piggybacks” or extends an existing protocol for the transmission of data such as FTP or UDP, although a dedicated protocol specifically for this purpose may be used as well. The user devices communicate data to and from the Internet via literally any wired (e.g., Ethernet, DSL, cable, fiber, and optical wireline connections) or wireless (e.g., Wi-Fi, WiMAX, WAN, PAN, MAN) networking standard.
The embodiment of
It will be recognized that while the architectures of
The foregoing aggregation approaches simplify processing and obviate having to flood the network with messages relating to events; rather, a limited number of CPE within that group can be sampled and the results extrapolated to all, or relevant statistics applied (e.g., calculation of a mean, median, variance, etc.). Hence, while the present invention can literally provide second-by-second information across the entire subscriber pool, it can also advantageously be scaled and/or statistically implemented where the aforementioned level of precision and/or subscriber coverage is not required.
As shown, per step 302, the latency of the device is first determined. In one embodiment, the device latency is experimentally or computationally determined, such as by a device manufacturer or by one or more MSO entities prior to the use of the device by the subscriber. As noted above, devices may also be aggregated into classes that fall within a certain range of latencies if such approximation is desired/tolerable.
Inherent latency in a user device (such as e.g., CPE 106) is based on the software, hardware, etc. components of the device. This may also include latency introduced by remote control units and the like used to control the CPE 106. However, the contribution of the remote control unit (or other controlling device) may be minimal. For example, if latency for a given event is about 2.8 seconds, 0.1-0.2 seconds of that latency may be attributable to the remote control. For example, DVR capabilities of a user device will affect the latency of the device. Hence, latency of a device is particular to the device type, manufacturer, and hardware/software options.
Next, at step 304, the network latency is determined. The network latency refers to the time necessary for the transmission of content and/or data from the headend to a particular device. The network latency is different for each device in the network, because a particular path which the content and/or data must take to arrive at each particular device is unique. As discussed in detail below with respect to
Lastly, at step 306, an overall system latency is derived from at least the previously determined device and network latencies. The overall system latency takes into account the particular or unique path which data and/or content may take to reach a certain device, as well as the characteristics of the particular device (e.g., hardware and software options). In one embodiment, the overall system latency represents the combination (e.g., addition) of the determined device latency and the determined network latency.
Referring now to
Next, per step 314, a message is sent to the device from the network. The message sent at e.g., step 314 comprises in one embodiment a message requiring an answer from the device. At step 316, a response to the message is received. In one exemplary embodiment, the message comprises a Simple Network Management Protocol (SNMP) “System Time” message as discussed in the previously incorporated “ATSC STANDARD: PROGRAM AND SYSTEM INFORMATION PROTOCOL FOR TERRESTRIAL BROADCAST AND CABLE (PSIP)”, Document A/65; published Apr. 14, 2009. The System Time message requests a response from the device indicating the precise time at which the message is received at that device. Hence, in response to receiving the System Time message, the device reports back (in a response message) the precise time at which it received the System Time request. Per International Organization for Standardization (ISO)/International Electrotechnical Commission (TEC) standard entitled “INFORMATION TECHNOLOGY—GENERIC CODING OF MOVING PICTURES AND ASSOCIATED AUDIO INFORMATION: SYSTEMS”, document number ISO/IEC 13818-1, second edition published Dec. 1, 2010 the System Time is broadcast from the headend 150 once every second. According to the previously referenced standard, the System Time is provided in the transport stream, hence the System Time message accurately reflects the latency induced through plant topology as discussed above, and advantageously requires no new protocols or infrastructure to support its transmission.
It is appreciated however that alternative types of messages may be utilized, the foregoing SNMP System Time message being merely illustrative. For example, literally any message requiring a response from the device may be sent from the headend at step 314, provided the response is structured to include information regarding the system time when the message was received (at the device).
Other embodiments may use a “round trip” timing model to determine latency, without the need for a message format that embeds a system time of receipt therein. For instance, if the time of transmission is known by the network (transmitting entity), and the time of receipt at that same entity of an ACK or other responsive transmission is also known, the round trip delay can be readily determined, with the one-way latency being given by Eqn. (1):
Latency (one way) (Round trip delay/2)−processing delay Eqn. (1)
Processing delay can be determined similar to that described above by, e.g., knowledge of the type of CPE receiving the test message, and its characteristic response. Moreover, a statistical or averaging approach can be utilized, such as by issuing n test messages or “pings”, and calculating the response for each according to Eqn. (1), and then averaging the results, or alternatively taking the lowest value (i.e., which is ostensibly indicative of the best performance of the receiver/CPE).
In the event that the CPE is only one-way capable (i.e., no upstream communication capability), and the reply message cannot be sent, other mechanisms can be employed to determine network delay. For instance, in one such alternative, the CPE (via e.g., an application running thereon) can time-stamp receipt of the downstream test message (the transmission time of which is known) and either (i) save this information to a local storage device for later transmission to a third party entity via e.g., another non-MSO based communication channel, which when received by the third party entity, can be forwarded back to the MSO for processing; or (ii) immediately generate and transmit the timing information to the third party entity via the alternate communication channel.
Next, at step 318 of the method, the device response is utilized by e.g., the network latency characterization application 202 running on the network tuning record processing entity 200, to determine the network latency. For example, suppose it is known that the System Time message was transmitted to the device at 3:42:52 pm GMT, and further that the response message received from the device indicates that the current system time was 3:43:02 pm GMT when the message was received. The network latency application 202 is then able to deduce that the network latency for that particular device is 10 seconds (less the inherent device latency in processing the received message, if any, as previously determined).
The previously collected device latency information (step 312) and the network latency (determined at step 318) are then utilized in a system latency algorithm at step 320. The system latency algorithm is utilized to provide an overall latency for the system which takes into account both the network-related and the device-related latency.
In one embodiment, if one assigns the variable “A” to represent the device related latency, and the variable t1 to represent the network latency, then the overall system latency T is equal to the sum of these values, or
T=t1+A Eqn. (2)
To determine the overall system latency, T, one must first determine the unknown value of the network latency, t1. To do this, a System Time message is utilized. For example, a System Time request may be sent to the device at time St0. In response to receiving the request, the device transmits a response which arrives back at the network at tf0. The time at which the device reply to the request is received (tf0) will necessarily equal the time at which the message was sent (St0) plus double the network latency (because the message is being transmitted and another message is being received, i.e., a “round trip” is made), see Eqn. (3).
tf0=St0+(2t1+B) Eqn. (3)
Where B represents latency on the set top box in processing the message—as opposed to A (Eqn. 2) which represents latency on the video path in the set top box (after validation through characterization in the lab, this term can, in most cases be ignored). However it is noted that, where any significant processing is required or if for any other reason the device-related latency, A, affects the value of T, it must be added thereto.
Accordingly, in the case of Eqn. (3) (i.e., insignificant device processing), the unknown network latency (t1) may be derived by the following:
t1=(tf0−St0−b)/2 Eqn. (4)
referring back to Eqn. (2), the overall system latency (T) becomes:
T=t1+A=(tf0−St0−B)/2+A Eqn. (5)
The overall system latency (T) may then be optionally provided to the device for use thereon, as will be discussed below (step 322). However, it is appreciated that in an alternative embodiment, the system latency is maintained at the network entity 200, and applied to subsequently received tuning records, as also discussed below.
In yet another embodiment, the network tuning record processing entity 200 may be configured to iteratively cycle through the population of device in communication therewith to “ping” each one with a System Time or other such request. Such a system may be “tuned” to ensure that it takes up minimal bandwidth and overhead, while still sending a request to every device in the network at a frequency which will yield useful results (i.e., detect significant changes in network-related latency, such as due to operational changes, equipment failure, etc.).
Referring now to
As shown, per step 352, device-related latency information is collected. According to the embodiment of
Next, per step 354, a message is sent to the network from the device (e.g., to a known network address, such as for a server). The transmitted message requires a response from the network, and at step 356, a response therefrom is received. The message sent upstream and/or the response message may comprise any number of message types. For example, the messages that are used in the regular dialog between a settop box and the SDV Server within the headend could be leveraged. However, it will be appreciated that any message which is timestamped by the network as to when it is received may be utilized according to the present embodiment.
At step 358, the received network response is used by e.g., the client latency characterization application 208 running on the CPE 106 to determine the network latency. For example, suppose it is known that the message was transmitted to the network at a first time (the client knows its local SI or other time reference when it transmits the test message), and that the response message was received at the network at a second time; as discussed above, these times may simply be subtracted (less any processing delays at the network-side responding entity) to determine the network-related latency.
The device latency information (step 352) and the network latency determined at step 358 are then utilized in a system latency algorithm at step 360. The system latency algorithm is, utilized to provide an overall latency for the system which takes into account both the network-related and the device-related latency which is specific to the device type, as well as the particular path content and/or data must travel to reach that particular device. In one embodiment, the overall latency is determined utilizing an equation similar to that discussed above (i.e., Eqn. (5)).
In the event that the network-side processing delay (i.e., the time in responding to the client-generated message once received) is significant, the foregoing equation can be modified to account for this value, which can be e.g., empirically determined for the network-side responding entity by the MSO.
Next, at step 404, tuning records are generated. Tuning records, as noted above, in the exemplary embodiment comprise information which is used to describe each action taken at a user device. For example, when a user requests to access programming, a tune-in record is created. Similarly, when a user changes a channel (i.e., requests different programming), a tune-away record is created (with respect to the first requested channel), and when the user turns off his device, a tune-out record is created. Additionally, records may be created which indicate activity with respect to trick modes functions such as fast forward, rewind, pause, etc. (e.g., time of invocation of the command, duration of the command, number of repeat or similar commands in a given time period, etc.). The tuning records may include for example the precise time at which the event of interest (e.g., user action) was performed or occurred, one or more channel or other program identifiers (e.g., tuned from Channel X, or tuned from Channel X to Channel Y), a device identifier, and an identifier of the action taken (e.g., tune-in, tune-out, tune-away, etc.). Various other alternative tuning record data constructs useful as part of the invention will also be recognized by those of ordinary skill given the present disclosure.
As noted above, tuning records may be generated by each of the devices across an entire MSO footprint, or within selected portions or subsets thereof. In one embodiment, the methods and apparatus discussed in previously referenced and incorporated by reference, co-owned, co-pending U.S. patent application Ser. No. 12/877,062 may be utilized for generating the aforementioned tuning records.
The tuning records are in one embodiment collected across all available platforms (including for example VOD consumption, interactive consumption, linear/broadcast consumption, DVR usage, EPG interaction, etc.) in order to gather user/audience information in real-time or near-real time, with associated actions of actual users or viewers.
The previously obtained latency information may be attached or appended to each tuning record (step 406), and these are then transmitted (step 408). In other words, the tuning records are enriched with the network or device determined overall system latency. The latency information may merely be transmitted alongside the tuning records with an indicator telling the receiving entity that the tuning record is to be associated with the latency information. Alternatively the device may attach the latency information by adjusting the timestamps of each record to account for the latency. For example, suppose a tuning record is collected at 11:42:36, and the overall system latency is established as 2 seconds, then the CPE 106 may adjust the collected time to 11:42:34 to reflect the network latency prior to transmitting the tuning record.
It is appreciated that while the exemplary embodiments described above attach the latency data to each tuning record created, the latency information can be aggregated or distributed across multiple records. For instance, the total latency (i.e., network-related plus device-related) should be largely constant for a given device for at least a period of time (assuming no major operational changes, etc.). Hence, records generated during that time can utilize a common single latency value if desired. In one such implementation, multiple records are aggregated at the CPE 106 over a prescribed period of time, and then “packaged” in a common data structure with a single appended latency value (or similarly, the records can be adjusted using the common latency value before transmission). This approach simplifies CPE-side processing and overhead.
It is further appreciated that the tuning records (having the overall system latency information attached, or being adjusted for the aforementioned latency) may be transmitted at step 408 to (1) a network 101 entity (such as the tuning record collection, validation and reporting entity 204 of the network tuning record processing entity, (ii) a proxy, such as one located at a local service node 182, and/or (iii) a network or third party analyzation entity 206 directly via the internet and/or an intermediary.
Furthermore, the records may be anonymized prior to transmission according to any number of anonymization techniques. For example, a cryptographic hash (e.g., MD5, SHA1, SHA2, HMAC) may be utilized to disguise the subscriber identity and/or CPE 106 identity. In one embodiment, the techniques for providing anonymity utilizing a cryptographic hash described in U.S. patent application Ser. No. 11/186,452 filed Jul. 20, 2005 and entitled “METHOD AND APPARATUS FOR BOUNDARY-BASED NETWORK OPERATION”, which is incorporated herein by reference in its entirety, may be utilized in conjunction with the present invention. As disclosed therein, the identity of a subscriber device or subscriber is anonymized by using a one-way cryptographic hash coupled with an optional “opaque” variable which carries information relating to the subscriber device of the hash with which it is associated. The hash and opaque variable frustrate de-encryption or reverse-engineering of the individual subscriber's identity or specific location. Alternative methods of providing anonymization may also be utilized consistent with the present invention, including those discussed in previously referenced and incorporated, co-owned and co-pending U.S. patent application Ser. No. 12/877,062.
In an alternative embodiment, the system latency information is neither received nor attached (or applied) at the device, e.g., CPE 106. Rather, according to this embodiment, the network entity which determines the system latency (such as e.g., the network latency characterization application 202 running on the network tuning record processing entity 200 as discussed above with respect to
At step 502, the tuning records are collected. As noted previously, these may be collected at e.g., an MSO network 101 entity, a proxy (which then forwards them on to a network entity), or a third party entity (e.g., web server). The records have attached latency information (as discussed above with respect to
At step 504, the receiving entity uses the latency information to normalize the tuning record data as required. That is to say, the receiving entity (e.g., network tuning record processing entity 200 of
The tuning records are then validated at step 506. As noted previously, the aforementioned systems and methods enable collection of a very large amount of data representing each user action across an entire MSO footprint regardless of device, type of content (e.g., VOD content, DVR content, linear broadcast, etc.). Since the amount of data is so large, it generally must be validated to ensure there are no errors in the data in order to be relied upon by consumers of the data. In one embodiment, the methods of co-owned U.S. patent application Ser. No. 12/829,104, filed Jul. 1, 2010, entitled “APPARATUS AND METHOD FOR DATA COLLECTION AND VALIDATION”, and issued as U.S. Pat. No. 8,484,511 on Jul. 9, 2013, which is incorporated herein by reference in its entirety, may be utilized for validating the tuning records. As discussed therein, statistical methods (e.g., linear regression, log linear regression) are used to arrive at an expected value for one or more of the various fields and records of the collected data (i.e., tuning records). Collected data is compared to the derived (e.g., average) or expected value, and if the data is outside of one or more prescribed criteria (e.g., is a given number of standard deviations away from the expected value or more) indicating that the data quality may be unacceptable, an error message is generated and the data optionally excluded. The criteria used to judge the data (e.g., number of standard deviations which the data is permitted to vary from the expected value) may be determined by the network operator, or an algorithm/computer program. In a further variant, the system may be monitored proactively and alert the network operator in the instance erroneous data meeting the test(s) of significance is received. Still further, in one embodiment, the system may be configured to “learn” how data is corrected, eventually enabling data to be corrected automatically without requiring manual error correction.
Lastly, per step 508, the validated and normalized tuning records are reported to an analyzation entity 206. As previously noted, the analyzation entity 206 may be physically located at the MSO network 101, or remote thereto. Reporting the tuning records may include reporting only those records which are requested by the particular entity. Alternatively, the records may be provided on an “as needed” basis. For example, the analyzation entity 206 may be associated with a single advertising entity or content provider. The tuning records provided thereto may comprise only those records which relate to the advertisements or content associated with that particular advertiser or content provider, or alternatively those associated with a particular demographic or psychographic or geographical region which the advertiser is attempting to access with its advertising.
In yet another embodiment, reports may be generated at the MSO network 101 at step 508. Specifically, the tuning record collection, validation, and reporting entity 204 disposed at the network tuning record processing entity 200 may be configured to sort through the collected, normalized and validated data, and parse this data into usable segments. For example, the data may be segmented or filtered to reflect all usage for a particular household or device, usage across all subscribers for a particular program or advertisement, etc. Alternatively, reports may be generated on a per-user, per-household, and/or per-device basis. Literally any type of report may be generated given the collected data and ability to normalize the collected data to accurately and precisely identify the collection time.
In one exemplary embodiment, a single report may be generated illustrating (across a certain demographic of viewers of an advertisement) the number of viewers which tuned away, and precise times during the advertisement when the viewers did so taking into account the latency associated with each viewer. For example, a report may indicate that (i) Users A, B, C and D were viewing an advertisement, (ii) User A tuned away at precisely 12:32:02 (this time is an adjusted time which reflects the herein-described latency), (iii) Users B and C tuned away at precisely 12:32:57 (this time is also an adjusted time reflecting the herein-described latency), and (iv) User D did not tune away during the advertisement. Using this information, an advertiser is able to determine overall penetration of an advertisement by determining the difference between the number of viewers tuned in at the beginning of the advertisement and the number which tuned away. In the example above, the overall penetration is determined by subtracting 3 (the number of users which tuned away, Users A, B, and C) from 4 (the number of users which were tuned in at the beginning of the advertisement (Users A, B, C, and D).
Since the times indicated in the report have been normalized (adjusted to account for individual/specific latencies), the advertiser may also use the aforementioned report to determine which portions of an advertisement are not well received by the audience. Suppose for example that it appears that the majority of viewers who tuned away from an advertisement did so within the first five seconds thereof, the advertiser may determine that a new introductory section for the advertisement is needed. Alternatively, suppose the majority of viewers who tuned away did so at a time which correlates to the appearance or disappearance of a character, the end of a song, etc., the advertiser may associate this tune-away event with that character, song, etc.
As noted above, the aforementioned methods enable tuning data records to be collected with to-the-second accuracy if desired, so that viewership and user activity can be determined with respect to increasingly smaller intervals of time. Accordingly, in one embodiment, the reports generated at step 508 may include a per-household view of the precise moment within a given advertisement or program in which a viewer tuned to, tuned away, and/or tuned-out, utilized a trick mode operation, invoked another function (e.g., VoIP call), etc. As will be discussed below, various business rules may be applied in generating reports, including generating certain reports as a purchasable commodity.
As noted previously, the tuning records may be analyzed at e.g., the MSO network tuning record processing entity 200, and/or at a third party or non-MSO network entity (such as e.g., the analyzation entity 206).
In one embodiment, the normalized and validated tuning records may be analyzed in a manner similar to Nielson analysis, so as to determine viewership for a particular program, network, time of day, day of the week, etc. However, as noted above, this analysis may be further extended to a second-by-second look at programming and advertising content (as well as other content and/or data). The tuning records may also be analyzed for the purpose of providing targeted secondary content insertion opportunities. For example, if the collected data indicates that user or subscriber devices registered to the 92123 ZIP code often view particular programming, then the network tuning record processing entity 200 (or other entity receiving the normalized and validated tuning records) may indicate to an entity providing advertising or other secondary content to the users in that ZIP code (such as e.g., an analyzation entity 206 thereof) that targeted advertising may be provided to these devices during the particular programming Such targeted advertising might address their common geography (e.g., a local upcoming event or service), a common demographic within that geography, and so forth.
In a further embodiment, the normalized and validated tuning records may be utilized by content and advertising providers to identify specific portions of provided content and/or advertising which are/are not being viewed (or are experiencing events such as tune-aways). The content provider (or advertiser) may then use this information to make subsequent content delivery decisions. For example, if a significant number of tuning records indicate that at particular time during an advertisement most viewers tune-away, the advertiser may review the advertisement for ineffective, unappealing content being displayed at that particular time (literally to the second) and make business decisions based thereon (such as removing the particular scene or image, replacing the entire advertisement with a substitute, etc.).
As yet another potential use, MSOs or advertisers may correlate multiple events within single premises or account together, so as to divine yet further insight into the effectiveness of the content. For instance, an advertisement asking viewers to call or text a certain number (say, to obtain information, enter a contest, etc.) might be followed by a VoIP or text message to that number by a viewing subscriber, thereby indicating at least ostensibly the level of penetration or effectiveness of the advertisement. If 40% of viewers who viewed the advertisement (i.e., did not tune away or tune out as indicated by the tuning records) and who have MSO VoIP service call the advertised number via their MSO VoIP service with a prescribed period of time (say, five minutes) after completion of the advertisement, then the advertisement might be deemed to have a very strong appeal or market penetration for the viewing audience.
Similarly, a user tuning to a linear broadcast channel and subsequently invoking a DVR function to record the program within a prescribed period after the tune-in might be indicative of a strong liking by that user.
Repetitive records for the same device/premises for the same content (e.g., advertisement) might also have utility for the MSO and/or advertiser. For example, if a given “saturation” advertisement campaign runs the same or complementary advertisements at a very high frequency (or “back to back”), an increase in tune-away or tune-out events for that premises/device for that same advertisement over time would indicate that the viewer has reached saturation, and they are affirmatively avoiding the advertisement. Note that many contemporary advertising schemes or plans utilize short (e.g., 15 second) “burst” advertisements that run in rapid succession. Under prior art approaches which are not capable of the granularity of the present invention, a viewer's split-second response to these very brief advertisements could not be accurately judged, since one could not be sure whether a user tuned away or tuned out after the completion of the advertisement, or sometime during. In contrast, the high degree of precision and reliability of the present invention advantageously affords an accurate picture of user behavior, even during such very brief segments.
As shown in
The CPE 106 of
In another embodiment, the CPE 106 comprises a converged premises device, such as for example that described in co-owned and co-pending U.S. patent application Ser. No. 11/378,129 filed Mar. 16, 2006 and entitled “METHODS AND APPARATUS FOR CENTRALIZED CONTENT AND DATA DELIVERY”, incorporated herein by reference in its entirety. In yet another embodiment, the CPE 106 may comprise a gateway device such as that discussed in previously referenced, co-owned, co-pending U.S. patent application Ser. No. 11/818,236 entitled “PREMISES GATEWAY APPARATUS AND METHODS FOR USE IN A CONTENT-BASED NETWORK”.
In yet another embodiment, the CPE 106 comprises a media bridge apparatus such as that discussed in co-owned, co-pending U.S. patent application Ser. No. 12/480,597 filed Jun. 8, 2009 and entitled “MEDIA BRIDGE APPARATUS AND METHODS”, incorporated herein by reference in its entirety. As discussed therein, the CPE 106 may act as a connection between a portable media device (PMD) and a user's home network. This bridging apparatus may be used, for example, to convert content stored on the PMD (e.g., an MP3 player such as an iPod®) to a format capable of being presented on a user's set-top box or other client device. Control of the presentation is also provided by the bridging apparatus. In one embodiment, the apparatus enables a user to access and control playback of media from a PMD via a user interface associated with a television, personal computer or other user device. The apparatus may also enable content stored on the PMD to be copied and stored on a user's digital video recorder (DVR) or other storage apparatus, optionally while maintaining appropriate copyright and digital rights management (DRM) requirements associated with the content being manipulated. The media bridging apparatus (e.g., CPE 106) can also work within a premises network or trusted domain for media content, thereby allowing a subscriber total mobility in the premises network. For example, media content from the PMD may be accessed via extant networks for distribution to any STB, PC, mobile device, or other PMD. The media bridging device may also utilize the existing premises network (including a network defined by coaxial cable in the premises, such as a MoCA-enabled network) to allow devices and DVRs to share media content with the PMD. According to this embodiment, device-related latency may further take into account latency associated with connected (or bridged) devices.
As previously noted, the CPE 106 of
In further embodiments, the CPE 106 may be further configured to run at least one application for: (i) data validation, (ii) identification and communication of data validity errors as determined by the aforementioned validation, and/or (iii) error logging and correction functions, as discussed previously herein.
The foregoing diversity of possible configurations of the CPE 106 illustrates the general network-agnosticism of the present invention; i.e., user events of interest may span literally from cable and satellite content delivery networks to unmanaged IP networks to local area networks to ad hoc wireless networks, and beyond. A “tune” event may comprise e.g., a linear broadcast channel change via a front-panel function or remote on a DSTB, or selection of an IPTV stream on a PC or other IP-enabled device, or selection of a VOD or PPV program, or invocation of a VoIP call, or selection of a hyperlink, or yet other types of activities.
Referring now to
The tuning record processing entity 200 generally comprises a network interface 702 for communication with the network 101, a processor subsystem 704 and associated storage 706, as well as additional interfaces 708 for communication with other entities.
The processor subsystem 704 is configured to run a tuning record collection, validation and reporting application 204 and a network latency characterization application 202 thereon. As noted previously, the foregoing may comprise e.g., distributed applications at a headend or third party entity, or yet other configurations of software known in the arts.
The tuning record collection, validation and reporting application 204, when executed, enables inter alia the request and/or receipt of tuning records from the user devices. The application 204 validates the data, implements one or more automated error correction mechanisms, analyzes data to generate reports therefrom, ensures subscriber anonymity, and/or transmits at least portions of the data to an analyzation entity 206 (such as those being associated with a particular content provider or advertiser). The network latency characterization application 202, when executed, is used to determine an overall system latency per device or per grouping of devices (such as via the methods discussed in
It is appreciated that the network tuning record processing entity 200 may comprise additional components (not shown) and functionality well known to those of ordinary skill in the network and embedded system fields, and accordingly not described further herein. For example, management or supervisory processes, and/or business rules software (described in greater detail below) may be run on the network tuning record processing entity 200. Fail-over protection, additional physical or network security mechanisms, etc. may also be implemented.
In another aspect of the invention, the aforementioned network tuning record processing entity 200 and/or the CPE 106 (e.g., including one or more computer programs for providing the above-mentioned functionalities thereof) optionally include an entity having an operations and/or business rules “engine”. This engine comprises, in an exemplary embodiment, a series of software routines that are adapted to control the determination of the overall system latency, the application of the latency to collected tuning records, as well as the actual collection, validation, analysis and transmission of data. These rules may also be fully integrated within the aforementioned one or more computer programs, and controlled via the entity on which the program is run, or remotely if desired (such as having the individual CPE “reconfigurable” as to the type, frequency, scope, etc. of data they collect and/or process). In effect, the rules engine comprises a supervisory entity which monitors and selectively controls the aforementioned functions at a higher level, so as to implement desired operational or business rules of the MSO or other parties of interest (e.g., content sources or advertisers).
The rules engine can be considered an overlay of sorts to the algorithms of the previously described computer applications. For example, the exemplary computer application may invoke certain operational protocols or decision processes based on data received (e.g., historical activity or user data, subscriber preferences, etc.), as well as network operational or historical data, demographic data, geographic data, etc. However, these processes may not always be compatible with higher-level business or operational goals, such as maximizing profit on a network-wide basis, or system reliability and/or flexibility. Moreover, the computer application being “supervised” may be operating on a per-CPE, per-household, or per-request basis (i.e., the collected data may be collected for individual CPE effectively in isolation, and analysis may be performed without considering larger patterns or decisions being made in the same service group, or network as a whole).
Hence, when imposed, the business/operational rules of the engine can be used to dynamically (or manually) control the operation of the aforementioned processes in conjunction with the latency determination, latency application, data collection, validation, analysis and transmission functions previously described.
For example, one rule implemented by the rules engine may comprise selectively performing the above functions resulting in collected viewership data only for certain users; e.g., those who have agreed to have their viewership data collected (whether for consideration or otherwise). Accordingly, only those users who affirmatively “opt in” will have data collected about their household or devices. In another variant, certain content access, delivery or utilization features (e.g., enhanced functionality such as interactive programming, special features, advanced trailers, etc.) may only be provided to users who agree to have data collected.
Another business rule relates to distribution of the normalized and validated tuning records; i.e., they may only be provided to particular parties (e.g., third parties such as third party analyzation entities 206) who meet certain criteria. For instance, these criteria might relate to (i) reliability and/or quality standards; (ii) profitability or revenue potential; (iii) pre-qualification or certification by the MSO (irrespective of whether they would add revenue or profit for the MSO), such as for sufficient security for the data, sufficient processing capability; or (iv) the ability to provide certain time or quality or service (QoS) guarantees, etc., so that the MSO may ensure that the data will be protected and used efficiently and properly.
As previously noted, the “event” data obtained using the present invention (in various stages of processing, ranging from being merely latency-corrected to being validated, latency corrected, filtered, and enriched with ancillary data) may form a commodity of sorts which can be bought, sold, or bartered under any number of different business models. For example, validated and latency-corrected records might be standardized according to a prescribed or de facto format or protocol, such that their use by various different entities (whether MSO, ISP, cellular service provider, advertiser, studio, etc.) is enabled. As can be appreciated, the ability to accurately characterize the behavior of millions of different users (many of whose demographics are known a priori) with respect to explicit events or portions of perceived content is a highly valuable commodity, especially for purposes of increasing advertising penetration and effectiveness (e.g., by proper placement and timing of advertisements, removal of objectionable or ineffective advertisements, etc.), and increasing MSO subscriber satisfaction (e.g., by giving subscribers a user experience which most closely tailors to their individual desires and needs). Hence, the present invention contemplates that the aforementioned records may be sold as commodities, and even develop a secondary market of sorts for buying, selling and trading based on factors such as the demographics the data represents (i.e., very wealthy individuals or those who make frequent purchases historically), the type of events captured (e.g., certain events may be difficult to obtain data on, and hence more highly valued), the “half life” of the data and/or how long ago it was collected (i.e., certain types of data may only be relevant or useful for a certain limited period of time), and so forth.
Many other approaches and combinations of various operational and business paradigms are envisaged consistent with the invention, as will be recognized by those of ordinary skill when provided this disclosure.
It will be recognized that while certain aspects of the invention are described in terms of a specific sequence of steps of a method, these descriptions are only illustrative of the broader methods of the invention, and may be modified as required by the particular application. Certain steps may be rendered unnecessary or optional under certain circumstances. Additionally, certain steps or functionality may be added to the disclosed embodiments, or the order of performance of two or more steps permuted. All such variations are considered to be encompassed within the invention disclosed and claimed herein.
While the above detailed description has shown, described, and pointed out novel features of the invention as applied to various embodiments, it will be understood that various omissions, substitutions, and changes in the form and details of the device or process illustrated may be made by those skilled in the art without departing from the invention. The foregoing description is of the best mode presently contemplated of carrying out the invention. This description is in no way meant to be limiting, but rather should be taken as illustrative of the general principles of the invention. The scope of the invention should be determined with reference to the claims.
This application is a divisional of and claims priority to co-owned, co-pending U.S. patent application Ser. No. 14/589,947 filed on Jan. 5, 2015, entitled “APPARATUS AND METHODS FOR IDENTIFYING AND CHARACTERIZING LATENCY IN A CONTENT DELIVERY NETWORK”, issuing as U.S. Pat. No. 10,728,129 on Jul. 28, 2020, which is a divisional of and claims priority to co-owned U.S. patent application Ser. No. 12/944,648 filed on Nov. 11, 2010 of the same title, issued as U.S. Pat. No. 8,930,979 on Jan. 6, 2015, each of which are incorporated herein by reference in its entirety. Additionally, this application is related to co-owned U.S. application Ser. No. 12/829,104 filed on Jul. 1, 2010 and entitled “APPARATUS AND METHODS FOR DATA COLLECTION AND VALIDATION” which issued as U.S. Pat. No. 8,484,511 on Jul. 9, 2013, and co-owned, co-pending U.S. patent application Ser. No. 12/877,062 filed on Sep. 7, 2010 and entitled “METHODS AND APPARATUS FOR AUDIENCE DATA COLLECTION AND ANALYSIS IN A CONTENT DELIVERY NETWORK”, which issued as U.S. Pat. No. 9,635,421 on Apr. 25, 2017, each of which are incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5528284 | Iwami et al. | Jun 1996 | A |
| 5708961 | Hylton et al. | Jan 1998 | A |
| 5719938 | Haas et al. | Feb 1998 | A |
| 5758257 | Herz et al. | May 1998 | A |
| 5796423 | Robbins et al. | Aug 1998 | A |
| 5818438 | Howe et al. | Oct 1998 | A |
| 5862140 | Shen et al. | Jan 1999 | A |
| 5897635 | Torres et al. | Apr 1999 | A |
| 5988078 | Levine | Nov 1999 | A |
| 6029045 | Picco et al. | Feb 2000 | A |
| 6052145 | Macrae et al. | Apr 2000 | A |
| 6088722 | Herz et al. | Jul 2000 | A |
| 6133910 | Stinebruner | Oct 2000 | A |
| 6167432 | Jiang | Dec 2000 | A |
| 6181697 | Nurenberg et al. | Jan 2001 | B1 |
| 6198744 | Huggins et al. | Mar 2001 | B1 |
| 6219710 | Gray et al. | Apr 2001 | B1 |
| 6252634 | Yuen et al. | Jun 2001 | B1 |
| 6259701 | Shur et al. | Jul 2001 | B1 |
| 6389538 | Gruse et al. | May 2002 | B1 |
| 6446261 | Rosser | Sep 2002 | B1 |
| 6460182 | Buabbud | Oct 2002 | B1 |
| 6463585 | Hendricks et al. | Oct 2002 | B1 |
| 6486907 | Farber et al. | Nov 2002 | B1 |
| 6487721 | Safadi | Nov 2002 | B1 |
| 6519062 | Yoo | Feb 2003 | B1 |
| 6523696 | Saito et al. | Feb 2003 | B1 |
| 6539548 | Hendricks et al. | Mar 2003 | B1 |
| 6608837 | Brodigan | Aug 2003 | B1 |
| 6640145 | Hoffberg et al. | Oct 2003 | B2 |
| 6694145 | Riikonen et al. | Feb 2004 | B2 |
| 6697376 | Son et al. | Feb 2004 | B1 |
| 6698020 | Zigmond et al. | Feb 2004 | B1 |
| 6728269 | Godwin et al. | Apr 2004 | B1 |
| 6760537 | Mankovitz | Jul 2004 | B2 |
| 6785904 | Franken et al. | Aug 2004 | B1 |
| RE38600 | Mankovitz et al. | Sep 2004 | E |
| 6788676 | Partanen et al. | Sep 2004 | B2 |
| 6788775 | Simpson | Sep 2004 | B1 |
| 6799326 | Boylan et al. | Sep 2004 | B2 |
| 6807676 | Robbins et al. | Oct 2004 | B1 |
| 6873622 | Dodson et al. | Mar 2005 | B1 |
| 6925257 | Yoo | Aug 2005 | B2 |
| 6944150 | McConnell et al. | Sep 2005 | B1 |
| 7003670 | Heaven et al. | Feb 2006 | B2 |
| 7006881 | Hoffberg et al. | Feb 2006 | B1 |
| 7009972 | Maher et al. | Mar 2006 | B2 |
| 7027460 | Iyer et al. | Apr 2006 | B2 |
| 7039048 | Monta et al. | May 2006 | B1 |
| 7039932 | Eldering | May 2006 | B2 |
| 7054902 | Toporek et al. | May 2006 | B2 |
| 7068639 | Varma et al. | Jun 2006 | B1 |
| 7085287 | Chapman | Aug 2006 | B1 |
| 7099308 | Merrill et al. | Aug 2006 | B2 |
| 7099348 | Warwick | Aug 2006 | B1 |
| 7106382 | Shiotsu | Sep 2006 | B2 |
| 7149772 | Kalavade | Dec 2006 | B1 |
| 7174126 | McElhatten et al. | Feb 2007 | B2 |
| 7174127 | Otten et al. | Feb 2007 | B2 |
| 7174385 | Li | Feb 2007 | B2 |
| 7185353 | Schlack | Feb 2007 | B2 |
| 7188085 | Pelletier | Mar 2007 | B2 |
| 7209458 | Ahvonen et al. | Apr 2007 | B2 |
| 7213036 | Apparao et al. | May 2007 | B2 |
| 7242960 | Van Rooyen et al. | Jul 2007 | B2 |
| 7242988 | Hoffberg et al. | Jul 2007 | B1 |
| 7254608 | Yeager et al. | Aug 2007 | B2 |
| 7325073 | Shao et al. | Jan 2008 | B2 |
| 7330483 | Peters, Jr. et al. | Feb 2008 | B1 |
| 7359375 | Lipsanen et al. | Apr 2008 | B2 |
| 7363643 | Drake et al. | Apr 2008 | B2 |
| 7376386 | Phillips et al. | May 2008 | B2 |
| 7383243 | Conkwright et al. | Jun 2008 | B2 |
| 7444655 | Sardera | Oct 2008 | B2 |
| 7457520 | Rosetti et al. | Nov 2008 | B2 |
| 7486869 | Alexander et al. | Feb 2009 | B2 |
| 7567983 | Pickelsimer et al. | Jul 2009 | B2 |
| 7592912 | Hasek et al. | Sep 2009 | B2 |
| 7602820 | Helms et al. | Oct 2009 | B2 |
| 7650319 | Hoffberg et al. | Jan 2010 | B2 |
| 7690020 | Lebar | Mar 2010 | B2 |
| 7693171 | Gould | Apr 2010 | B2 |
| 7716668 | Moore et al. | May 2010 | B2 |
| 7721314 | Sincaglia et al. | May 2010 | B2 |
| 7742074 | Minatogawa | Jun 2010 | B2 |
| 7768546 | Boehringer, Jr. | Aug 2010 | B1 |
| 7770200 | Brooks et al. | Aug 2010 | B2 |
| 7809942 | Baran et al. | Oct 2010 | B2 |
| 7873660 | Wong et al. | Jan 2011 | B1 |
| 7878660 | Yang et al. | Feb 2011 | B2 |
| 7889765 | Brooks et al. | Feb 2011 | B2 |
| 7900052 | Jonas et al. | Mar 2011 | B2 |
| 7954131 | Cholas et al. | May 2011 | B2 |
| 8078669 | Ladd et al. | Dec 2011 | B2 |
| 8099757 | Riedl et al. | Jan 2012 | B2 |
| 8244909 | Hanson et al. | Aug 2012 | B1 |
| 8347341 | Markley et al. | Jan 2013 | B2 |
| 8484511 | Engel et al. | Jul 2013 | B2 |
| 8855469 | Maharajh et al. | Oct 2014 | B2 |
| 9213538 | Ladd et al. | Dec 2015 | B1 |
| 9215423 | Kimble et al. | Dec 2015 | B2 |
| 9906838 | Cronk et al. | Feb 2018 | B2 |
| 20010004768 | Hodge et al. | Jun 2001 | A1 |
| 20010033583 | Rabenko et al. | Oct 2001 | A1 |
| 20020007491 | Schiller et al. | Jan 2002 | A1 |
| 20020024943 | Karaul et al. | Feb 2002 | A1 |
| 20020027883 | Belaiche | Mar 2002 | A1 |
| 20020032754 | Logston et al. | Mar 2002 | A1 |
| 20020056125 | Hodge et al. | May 2002 | A1 |
| 20020059619 | Lebar | May 2002 | A1 |
| 20020066033 | Dobbins et al. | May 2002 | A1 |
| 20020078444 | Krewin et al. | Jun 2002 | A1 |
| 20020087976 | Kaplan et al. | Jul 2002 | A1 |
| 20020104002 | Nishizawa et al. | Aug 2002 | A1 |
| 20020123928 | Eldering et al. | Sep 2002 | A1 |
| 20020133513 | Townsend et al. | Sep 2002 | A1 |
| 20020144260 | Devara | Oct 2002 | A1 |
| 20020147771 | Traversat et al. | Oct 2002 | A1 |
| 20020152299 | Traversat et al. | Oct 2002 | A1 |
| 20020166120 | Boylan et al. | Nov 2002 | A1 |
| 20020178446 | Sie et al. | Nov 2002 | A1 |
| 20020188744 | Mani | Dec 2002 | A1 |
| 20020188869 | Patrick | Dec 2002 | A1 |
| 20030002577 | Pinder | Jan 2003 | A1 |
| 20030016627 | MeLampy | Jan 2003 | A1 |
| 20030016701 | Hinson | Jan 2003 | A1 |
| 20030023983 | Pidgeon et al. | Jan 2003 | A1 |
| 20030028888 | Hunter et al. | Feb 2003 | A1 |
| 20030056217 | Brooks | Mar 2003 | A1 |
| 20030115267 | Hinton et al. | Jun 2003 | A1 |
| 20030126611 | Chernock et al. | Jul 2003 | A1 |
| 20030126618 | Wright | Jul 2003 | A1 |
| 20030149975 | Eldering et al. | Aug 2003 | A1 |
| 20030165241 | Fransdonk | Sep 2003 | A1 |
| 20030166401 | Combes et al. | Sep 2003 | A1 |
| 20030208767 | Williamson et al. | Nov 2003 | A1 |
| 20030213001 | Yuen et al. | Nov 2003 | A1 |
| 20030217137 | Roese et al. | Nov 2003 | A1 |
| 20030217365 | Caputo | Nov 2003 | A1 |
| 20030220100 | McElhatten et al. | Nov 2003 | A1 |
| 20030221191 | Khusheim | Nov 2003 | A1 |
| 20030225777 | Marsh | Dec 2003 | A1 |
| 20040015398 | Hayward | Jan 2004 | A1 |
| 20040024880 | Elving et al. | Feb 2004 | A1 |
| 20040034877 | Nogues | Feb 2004 | A1 |
| 20040040035 | Carlucci et al. | Feb 2004 | A1 |
| 20040045032 | Cummings et al. | Mar 2004 | A1 |
| 20040045035 | Cummings et al. | Mar 2004 | A1 |
| 20040045037 | Cummings et al. | Mar 2004 | A1 |
| 20040133923 | Watson et al. | Jul 2004 | A1 |
| 20040137918 | Varonen et al. | Jul 2004 | A1 |
| 20040148625 | Eldering et al. | Jul 2004 | A1 |
| 20040158858 | Paxton et al. | Aug 2004 | A1 |
| 20040181800 | Rakib et al. | Sep 2004 | A1 |
| 20040213389 | Ljubicich et al. | Oct 2004 | A1 |
| 20040226043 | Mettu et al. | Nov 2004 | A1 |
| 20040230994 | Urdang et al. | Nov 2004 | A1 |
| 20040250273 | Swix et al. | Dec 2004 | A1 |
| 20050022247 | Bitran et al. | Jan 2005 | A1 |
| 20050034171 | Benya | Feb 2005 | A1 |
| 20050049886 | Grannan et al. | Mar 2005 | A1 |
| 20050055220 | Lee et al. | Mar 2005 | A1 |
| 20050055685 | Maynard et al. | Mar 2005 | A1 |
| 20050060742 | Riedl et al. | Mar 2005 | A1 |
| 20050086334 | Aaltonen et al. | Apr 2005 | A1 |
| 20050086683 | Meyerson | Apr 2005 | A1 |
| 20050108763 | Baran et al. | May 2005 | A1 |
| 20050114900 | Ladd et al. | May 2005 | A1 |
| 20050125824 | Bienstock | Jun 2005 | A1 |
| 20050177855 | Maynard et al. | Aug 2005 | A1 |
| 20050187823 | Howes | Aug 2005 | A1 |
| 20050190794 | Krause et al. | Sep 2005 | A1 |
| 20050198686 | Krause et al. | Sep 2005 | A1 |
| 20050216933 | Black | Sep 2005 | A1 |
| 20050228725 | Rao et al. | Oct 2005 | A1 |
| 20050235307 | Relan et al. | Oct 2005 | A1 |
| 20050289618 | Hardin | Dec 2005 | A1 |
| 20060017846 | Kim et al. | Jan 2006 | A1 |
| 20060020786 | Helms et al. | Jan 2006 | A1 |
| 20060021004 | Moran et al. | Jan 2006 | A1 |
| 20060041905 | Wasilewski | Feb 2006 | A1 |
| 20060047957 | Helms et al. | Mar 2006 | A1 |
| 20060074875 | Faunce | Apr 2006 | A1 |
| 20060095940 | Yearwood | May 2006 | A1 |
| 20060117354 | Schutte et al. | Jun 2006 | A1 |
| 20060130099 | Rooyen | Jun 2006 | A1 |
| 20060130101 | Wessel Van Rooyen | Jun 2006 | A1 |
| 20060130107 | Gonder et al. | Jun 2006 | A1 |
| 20060130113 | Carlucci et al. | Jun 2006 | A1 |
| 20060136964 | Diez et al. | Jun 2006 | A1 |
| 20060149624 | Baluja et al. | Jul 2006 | A1 |
| 20060149850 | Bowman | Jul 2006 | A1 |
| 20060156332 | Kendall | Jul 2006 | A1 |
| 20060161635 | Lamkin et al. | Jul 2006 | A1 |
| 20060168609 | Chen | Jul 2006 | A1 |
| 20060171390 | La | Aug 2006 | A1 |
| 20060171423 | Helms et al. | Aug 2006 | A1 |
| 20060187900 | Akbar | Aug 2006 | A1 |
| 20060206712 | Dillaway et al. | Sep 2006 | A1 |
| 20060209799 | Gallagher et al. | Sep 2006 | A1 |
| 20060218604 | Riedl et al. | Sep 2006 | A1 |
| 20060225118 | Rolls et al. | Oct 2006 | A1 |
| 20060233372 | Shaheen et al. | Oct 2006 | A1 |
| 20060248553 | Mikkelson et al. | Nov 2006 | A1 |
| 20060248556 | Yuen et al. | Nov 2006 | A1 |
| 20060251097 | Chapman et al. | Nov 2006 | A1 |
| 20060259927 | Acharya et al. | Nov 2006 | A1 |
| 20060287915 | Boulet et al. | Dec 2006 | A1 |
| 20060291506 | Cain | Dec 2006 | A1 |
| 20070019645 | Menon | Jan 2007 | A1 |
| 20070022459 | Gaebel et al. | Jan 2007 | A1 |
| 20070022469 | Cooper et al. | Jan 2007 | A1 |
| 20070033531 | Marsh | Feb 2007 | A1 |
| 20070049245 | Lipman | Mar 2007 | A1 |
| 20070055984 | Schiller et al. | Mar 2007 | A1 |
| 20070061023 | Hoffberg et al. | Mar 2007 | A1 |
| 20070067851 | Fernando et al. | Mar 2007 | A1 |
| 20070076728 | Rieger et al. | Apr 2007 | A1 |
| 20070094691 | Gazdzinski | Apr 2007 | A1 |
| 20070121678 | Brooks et al. | May 2007 | A1 |
| 20070124488 | Baum et al. | May 2007 | A1 |
| 20070136777 | Hasek et al. | Jun 2007 | A1 |
| 20070157262 | Ramaswamy et al. | Jul 2007 | A1 |
| 20070180230 | Cortez | Aug 2007 | A1 |
| 20070186228 | Ramaswamy et al. | Aug 2007 | A1 |
| 20070198839 | Carle et al. | Aug 2007 | A1 |
| 20070204314 | Hasek et al. | Aug 2007 | A1 |
| 20070209059 | Moore et al. | Sep 2007 | A1 |
| 20070217436 | Markley et al. | Sep 2007 | A1 |
| 20070219910 | Martinez | Sep 2007 | A1 |
| 20070245376 | Svendsen | Oct 2007 | A1 |
| 20070250880 | Hainline | Oct 2007 | A1 |
| 20070261116 | Prafullchandra et al. | Nov 2007 | A1 |
| 20070276926 | Lajoie et al. | Nov 2007 | A1 |
| 20070280298 | Hearn et al. | Dec 2007 | A1 |
| 20070288715 | Boswell et al. | Dec 2007 | A1 |
| 20080022012 | Wang | Jan 2008 | A1 |
| 20080059804 | Shah et al. | Mar 2008 | A1 |
| 20080066112 | Bailey et al. | Mar 2008 | A1 |
| 20080091805 | Malaby et al. | Apr 2008 | A1 |
| 20080091807 | Strub et al. | Apr 2008 | A1 |
| 20080098212 | Helms et al. | Apr 2008 | A1 |
| 20080112405 | Cholas et al. | May 2008 | A1 |
| 20080120667 | Zaltsman | May 2008 | A1 |
| 20080133551 | Wensley et al. | Jun 2008 | A1 |
| 20080155059 | Hardin et al. | Jun 2008 | A1 |
| 20080162353 | Tom et al. | Jul 2008 | A1 |
| 20080170551 | Zaks | Jul 2008 | A1 |
| 20080172287 | Tien et al. | Jul 2008 | A1 |
| 20080177998 | Apsangi et al. | Jul 2008 | A1 |
| 20080201386 | Maharajh et al. | Aug 2008 | A1 |
| 20080201731 | Howcroft | Aug 2008 | A1 |
| 20080209489 | Joyce et al. | Aug 2008 | A1 |
| 20080222684 | Mukraj et al. | Sep 2008 | A1 |
| 20080229379 | Akhter | Sep 2008 | A1 |
| 20080235746 | Peters et al. | Sep 2008 | A1 |
| 20080273591 | Brooks et al. | Nov 2008 | A1 |
| 20080279534 | Buttars | Nov 2008 | A1 |
| 20080282299 | Koat et al. | Nov 2008 | A1 |
| 20080306903 | Larson et al. | Dec 2008 | A1 |
| 20090083813 | Dolce et al. | Mar 2009 | A1 |
| 20090086643 | Kotrla et al. | Apr 2009 | A1 |
| 20090098861 | Kalliola et al. | Apr 2009 | A1 |
| 20090100459 | Riedl et al. | Apr 2009 | A1 |
| 20090100493 | Jones et al. | Apr 2009 | A1 |
| 20090133048 | Gibbs et al. | May 2009 | A1 |
| 20090150917 | Huffman et al. | Jun 2009 | A1 |
| 20090150926 | Schlack | Jun 2009 | A1 |
| 20090172776 | Makagon et al. | Jul 2009 | A1 |
| 20090185576 | Kisel et al. | Jul 2009 | A1 |
| 20090201917 | Maes et al. | Aug 2009 | A1 |
| 20090210912 | Cholas et al. | Aug 2009 | A1 |
| 20090217318 | Versteeg et al. | Aug 2009 | A1 |
| 20090217326 | Hasek | Aug 2009 | A1 |
| 20090225760 | Foti | Sep 2009 | A1 |
| 20090228569 | Kalmanje | Sep 2009 | A1 |
| 20090249421 | Liu et al. | Oct 2009 | A1 |
| 20090282241 | Prafullchandra et al. | Nov 2009 | A1 |
| 20090282444 | Laksono | Nov 2009 | A1 |
| 20090293101 | Carter et al. | Nov 2009 | A1 |
| 20100005527 | Jeon | Jan 2010 | A1 |
| 20100027560 | Yang et al. | Feb 2010 | A1 |
| 20100030578 | Siddique et al. | Feb 2010 | A1 |
| 20100042478 | Reisman | Feb 2010 | A1 |
| 20100083329 | Joyce et al. | Apr 2010 | A1 |
| 20100106846 | Noldus et al. | Apr 2010 | A1 |
| 20100115091 | Park et al. | May 2010 | A1 |
| 20100115113 | Short et al. | May 2010 | A1 |
| 20100122276 | Chen | May 2010 | A1 |
| 20100125658 | Strasters | May 2010 | A1 |
| 20100131973 | Dillon et al. | May 2010 | A1 |
| 20100138900 | Peterka et al. | Jun 2010 | A1 |
| 20100169977 | Dasher et al. | Jul 2010 | A1 |
| 20100199299 | Chang | Aug 2010 | A1 |
| 20100211967 | Ramaswamy et al. | Aug 2010 | A1 |
| 20100211982 | Lee et al. | Aug 2010 | A1 |
| 20100218231 | Frink et al. | Aug 2010 | A1 |
| 20100251305 | Kimble et al. | Sep 2010 | A1 |
| 20100280641 | Harkness et al. | Nov 2010 | A1 |
| 20100287209 | Hauser | Nov 2010 | A1 |
| 20100287609 | Gonzalez et al. | Nov 2010 | A1 |
| 20100313225 | Cholas et al. | Dec 2010 | A1 |
| 20110015989 | Tidwell et al. | Jan 2011 | A1 |
| 20110016479 | Tidwell et al. | Jan 2011 | A1 |
| 20110016482 | Tidwell et al. | Jan 2011 | A1 |
| 20110029507 | Au | Feb 2011 | A1 |
| 20110071841 | Fomenko et al. | Mar 2011 | A1 |
| 20110090898 | Patel et al. | Apr 2011 | A1 |
| 20110093900 | Patel et al. | Apr 2011 | A1 |
| 20110099017 | Ure | Apr 2011 | A1 |
| 20110103374 | Lajoie et al. | May 2011 | A1 |
| 20110110515 | Tidwell et al. | May 2011 | A1 |
| 20110126018 | Narsinh et al. | May 2011 | A1 |
| 20110138064 | Rieger et al. | Jun 2011 | A1 |
| 20110173045 | Jaine | Jul 2011 | A1 |
| 20110178943 | Motahari et al. | Jul 2011 | A1 |
| 20110213688 | Santos et al. | Sep 2011 | A1 |
| 20110219229 | Cholas et al. | Sep 2011 | A1 |
| 20110265116 | Stern et al. | Oct 2011 | A1 |
| 20110267952 | Ko et al. | Nov 2011 | A1 |
| 20110283311 | Luong | Nov 2011 | A1 |
| 20120005527 | Engel et al. | Jan 2012 | A1 |
| 20120008786 | Cronk et al. | Jan 2012 | A1 |
| 20120011567 | Cronk et al. | Jan 2012 | A1 |
| 20120030363 | Conrad | Feb 2012 | A1 |
| 20120079518 | Wan et al. | Mar 2012 | A1 |
| 20120124606 | Tidwell et al. | May 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 2005519365 | Jun 2005 | JP |
| 2005519501 | Jun 2005 | JP |
| 2005339093 | Dec 2005 | JP |
| 2008015936 | Jan 2008 | JP |
| 2009211632 | Sep 2009 | JP |
| 2010502109 | Jan 2010 | JP |
| 2010079902 | Apr 2010 | JP |
| 2012505436 | Mar 2012 | JP |
| 2012523614 | Oct 2012 | JP |
| WO-2005015422 | Feb 2005 | WO |
| WO-2012021245 | Feb 2012 | WO |
| Entry |
|---|
| Cantor, et al., Bindings for the OASIS Security Assertion Markup Language (SAML) V2.0, OASIS Standard, Mar. 2005, Document ID saml-bindings-2.0-os ,(http://docs.oasis-open.org/security/saml/v2.0/saml-bindings-2.0-os. pdf). |
| Denning, et al., Cryptography and Data Security, Addison-Wesley Publishing Company, ISBN 0-201-10150-5, 1982, pp. 62. |
| DLNA (Digital Living Network Alliance) protocols described in DLNA Networked Device Interoperability Guidelines Expanded, Mar. 2006 and subsequent expanded version dated Oct. 2006. |
| UTF-32, IBM, retrieved from http://publib.boulder.ibm.com/infocenter/iseries/v5r3/index.jsp?topic=%2Fnls%2Frbagsutf32.htm on Aug. 28, 2013. |
| Number | Date | Country | |
|---|---|---|---|
| 20210014143 A1 | Jan 2021 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 14589947 | Jan 2015 | US |
| Child | 16939989 | US | |
| Parent | 12944648 | Nov 2010 | US |
| Child | 14589947 | US |
