Patent Application
20140337958
A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by any-one of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
The present disclosure generally relates to delivery online of video and other content, and to securing such delivery to prevent unauthorized access thereto.
Distributed computer systems are known in the art. One such distributed computer system is a “content delivery network” or “CDN” that is operated and managed by a service provider. The service provider typically provides the service on behalf of third parties. A “distributed system” of this type typically refers to a collection of autonomous computers linked by a network or networks, together with the software, systems, protocols and techniques designed to facilitate various services, such as content delivery or the support of outsourced site infrastructure. Typically, “content delivery” means the storage, caching, or transmission of content, streaming media and applications on behalf of content providers, including ancillary technologies used therewith including, without limitation, DNS request handling, provisioning, data monitoring and reporting, content targeting, personalization, and business intelligence.
While content delivery networks provide significant advantages, typically they include dedicated platforms to support delivery of content for multiple third party runtime environments that are, in turn, based on their own proprietary technologies, media servers, and protocols. These distinct platforms are costly to implement and to maintain, especially globally and at scale as the number of end users increases. Moreover, at the same time, content providers (such as large-scale broadcasters, film distributors, and the like) desire their content to be delivered online in a manner that complements traditional mediums such as broadcast TV (including high definition or “HD” television) and DVD. This content may also be provided at different bit rates. End users also desire to interact with the content as they can do now with traditional DVR-based content delivered over satellite or cable. A further complication is that Internet-based content delivery is no longer limited to fixed line environments such as the desktop, as more and more end users now use mobile devices such as the Apple® iPhone® to receive and view content over mobile environments.
Thus, there is a need to provide an integrated, secure content delivery network platform with the ability to deliver online content (such as HD-quality video) at broadcast audience scale to the most popular runtime environments (such as Adobe® Flash®, Microsoft® Silverlight®, etc.) as well as to mobile devices such as the iPhone to match what viewers expect from traditional broadcast TV. The techniques disclosed herein address this need.
Disclosed herein are methods and systems for delivering video and other forms of content over the Internet, and other networks, while preventing unauthorized access thereto. Architectures for live streaming and video-on-demand functionality, among others, are discussed.
In one embodiment, for example, a method for authenticating client devices (e.g., a client computer running a media player application) that request streaming content includes receiving a request for streaming content (e.g., video) from a client (e.g., a media player running on a user computer) at a content delivery system. Such a content delivery system may be implemented with a content delivery network having one or more content or “edge” servers therein, a given content server receiving HTTP or other requests for content from clients. The method includes, at the time of delivery of the streaming content, sending a program to the requesting client. Typically, the program is a set of one or more instructions for the client to execute. The program may be a file, such as a SWF file or other file, with one or more functions defined therein. The program is executed by the client and inspects and/or determines information about the client, such as an identifier associated with the client. The identifier may represent an identifier of player (e.g., a byte array thereof), a hardware identifier for the device itself, and/or combination of player software and hardware represented by the device). The information may represent a class number, unique serial number, an authentication code, or other kind of information. The information is sent to the content delivery system. It may be encoded into a token (e.g., using a hash algorithm) before being sent.
The content delivery system receives the information identifying the requesting client. Typically, the information will be returned to the given content server from which the client originally requested the content. That content server, or another machine in the system, determines whether the requesting client is recognized for purposes of receiving the streaming content. Such determination can be made based, at least in part, on the information sent from the client in accordance with the program instructions. (For example, the content server can compute its own token based on the byte array of the player, or other information associated therewith, and compare it to the received token.) Recognized clients may or may not be then subject to a further authorization procedure. In other words, in some implementations, a client player may be recognized but not authorized to play the streaming content.
In some embodiments, if it is determined that requesting client is recognized, the content delivery system can continue with the streaming of the content. If not, then the content delivery system can abort delivery of the content stream, if it has begun, or deny access in the first place.
In related embodiments, the sending of the program to the requesting client at the time of delivery of the streaming content can be accomplished by sending the program at the initiation of content delivery. In other cases, it may be sent during content delivery (e.g., within some given time from the initiation of the streaming content). The program may be embedded within the streaming content or sent separately, such as a through an out of band HTTP connection to the content server serving the stream or another machine, or otherwise outside of the content stream. Such alternative functionality may be configurable in the content delivery system using metadata configuration of the content servers.
In related embodiments, the program sent to the client specifies the encoding to be used to create the aforementioned token from the client identifying information (e.g., by specifying a particular hashing algorithm to use). In some embodiments, the token may be created by taking a hash of the client identifying information combined with a challenge, session id, or other information sent to the client by the content delivery system along with the program.
The determination of whether the requesting client is recognized can be accomplished by the content delivery system, or more particularly the given content server within it, comparing the token received from the client with a token it computes using the same encoding mechanism (e.g., the same hashing algorithm) on a known client identifier (e.g., a known client byte array). The determination also can be made without using tokens and by comparing the client identifying information with a list of known clients accessible to the content delivery system.
Further embodiments include such servers, client devices, content delivery network devices, and other digital data processing components with processors and stored instructions sufficient, when executing those stored instructions, to carry out the above-described functionality.
The foregoing summary is not limiting but merely describes illustrative features of the methods and devices disclosed herein, which will be discussed in more detail in the sections that follow.
The invention will be more fully understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
The following detailed description sets forth embodiments to provide an overall understanding of the principles of the structure, function, and use of the methods and systems disclosed herein. The methods and systems described herein and illustrated in the accompanying drawings are non-limiting examples; the scope of the present invention is defined solely by the claims. The features described or illustrated in connection with one embodiment may be combined with the features of other embodiments. Such modifications and variations are intended to be included within the scope of the present invention. All patents, publications and references cited herein are expressly incorporated herein by reference in their entireties.
In a known system, such as shown in
The distributed computer system may also include other infrastructure, such as a distributed data collection system 108 that collects usage and other data from the content servers, aggregates that data across a region or set of regions, and passes that data to other back-end systems 110, 112, 114 and 116 to facilitate monitoring, logging, alerts, billing, management and other operational and administrative functions. Distributed network agents 118 monitor the network as well as the server loads and provide network, traffic and load data to a DNS query handling mechanism 115, which is authoritative for content domains being managed by the CDN. A distributed data transport mechanism 120 may be used to distribute control information (e.g., metadata to manage content, to facilitate load balancing, and the like) to the content servers. More about the distribution of control information in a CDN can be found in U.S. Pat. No. 7,240,100, the disclosure of which is hereby incorporated by reference in its entirety.
As illustrated in
Client machines 122 include conventional personal computers, laptops, other digital data processing devices. Client machines also include mobile clients, which may include any a variety of mobile devices often referred to as smart-phones, cell phones, and/or personal digital assistants (PDAs).
A CDN content server may be configured to provide one or more extended content delivery features, preferably on a domain-specific, customer-specific basis, preferably using configuration files that are distributed to the content servers using a configuration system. A given configuration file preferably is XML-based and includes a set of content handling rules and directives that facilitate one or more advanced content handling features. The configuration file may be delivered to the CDN content server via the data transport mechanism. U.S. Pat. No. 7,111,057, the disclosure of which is incorporated herein by reference, illustrates a useful infrastructure for delivering and managing content server content control information, and this and other server control information can be provisioned by the CDN service provider itself, or (via an extranet or the like) the content provider customer who operates the origin server.
The CDN may include a storage subsystem (NetStorage), such as described in U.S. Pat. No. 7,472,178, the disclosure of which is incorporated herein by reference.
The CDN may operate a server cache hierarchy (Cache-H) to provide intermediate caching of customer content; one such cache hierarchy subsystem is described in U.S. Pat. No. 7,376,716, the disclosure of which is incorporated herein by reference.
For live streaming delivery, the CDN may include a delivery subsystem, such as described in U.S. Pat. No. 7,296,082, the disclosure of which is incorporated herein by reference.
The CDN may provide secure content delivery among a client browser, content server and customer origin server in the manner described in U.S. Publication No. 2004/0093419 and/or U.S. Pat. No. 7,363,361, the disclosures of which are both incorporated herein by reference. Secure content delivery as described therein enforces SSL-based links between the client and the content server process, on the one hand, and between the content server process and an origin server process, on the other hand. This enables an SSL-protected web page and/or components thereof to be delivered via the content server.
For live streaming delivery, the CDN may include a live delivery subsystem, such as described in U.S. Pat. No. 7,296,082, the disclosure of which is incorporated herein by reference.
The above-identified technologies can be extended to create an integrated, secure, HTTP-based delivery platform that provides for the online delivery of HD-video quality content (and other media) to various runtime environments. The platform supports delivery of both “live” and “on-demand” content.
As used herein, the following terms shall have the following representative meanings. For convenience of illustration only, the description that follows (with respect to live streaming delivery) is presented in the context of the Adobe® Flash® runtime environment, but this is not a limitation, as a similar type of solution may also be implemented for other runtime environments both fixed line and mobile (including, without limitation, Microsoft® Silverlight®, Apple® iPhone®, and others).
An Encoder is a customer-owned or managed machine which takes some raw video feed (e.g., live video) in some format (streaming, satellite, etc.) and delivers the data to an Entry Point encoded for streaming delivery. An Entry Point (EP) typically is a process running on a CDN streaming machine which receives video data from the customer's Encoder and makes this data available to consumers of the live stream. For Adobe Flash, this is a Flash Media Server (FMS) configured to accept connections from Encoders. A Flash Media Server is a server process for Flash media available from Adobe Corporation. In this embodiment, an Intermediate Region (IR) typically is a Flash Media Server which the CDN has configured to act analogously to a streaming set reflector, such as described in U.S. Pat. No. 7,296,082 and U.S. Pat. No. 6,751,673, the disclosures of which are hereby incorporated by reference. These machines relay streams from FMS EPs to FMS Edge regions, providing fan out and path diversity. A “Region” typically implies a set of machines (and their associated server processes) that are co-located and are interconnected to one another for load sharing, typically over a back-end local area network. A Flash Edge machine is a Flash Media Server which has been configured to accept client requests. This is the software running on the Flash EP, IR, and Edge machines in a representative embodiment. Intermediate Format (IF) is an internal (to the CDN) format for sending streaming data from EP to an edge server HTTP proxy. As will be described in more detail below, in certain implementations IF preferably comprises several different pieces, including “Stream Manifest,” “Fragment Indexes,” and “IF Fragments.” Live, DVR and VOD are defined as follows: “Live” refers to media served in real time as an event occurs; “DVR” refers to serving content acquired from a “live” feed but served at a later time; “VOD” refers to media served from a single, complete (i.e., not incrementally changing) file or set of files. Real Time Messaging Protocol (RTMP) is the streaming and RPC protocol used by Flash. Real Time Messaging Protocol Encrypted (RTMPE) is the encrypted version of RTMP using secrets built into the server and client. “SWF” or “Small Web Format” is the format for Flash client applications. SWF verification refers to a technique by which the Flash Player can authenticate to FMS that it is playing an unmodified SWF by sending hashes of the SWF itself along with secrets embedded in the client and server.
Referring to
As also seen in
In this embodiment, a content request (from an end user Client 314) is directed to the CDN edge machine HTTP proxy 312, preferably using techniques such as described in U.S. Pat. Nos. 6,108,703, 7,240,100, 7,293,093 and others, the disclosures of which are hereby incorporated by reference. (Note that in other embodiments an HTTP proxy associated with other CDN machines, such as a CDN machine located in a centralized location, may be employed.) When an HTTP proxy 312 receives an HTTP request for a given stream, the HTTP proxy 312 makes various requests, preferably driven by HTTP proxy metadata (as described in U.S. Pat. Nos. 7,240,100, 7,111,057 the disclosures of which are hereby incorporated by reference, and others), possibly via a cache hierarchy 318 (see, e.g., U.S. Pat. No. 7,376,716, the disclosure of which is hereby incorporated by reference, and others) to learn about and download a stream to serve to the Client 314. Preferably, the streaming-specific knowledge is handled by the edge machine HTTP proxy 312 directly connected to a Client 314. Any go-forward (cache miss) requests (issued from the HTTP proxy) preferably are standard HTTP requests. In one embodiment, the content is delivered to the Client 314 from the HTTP proxy 312 as a progressive-download FLV file. As noted above, the references herein to Adobe FLV are used herein by way of example, as the disclosed architecture is not limited for use with Adobe FLV. For secure streams, preferably the Client 314 first authenticates to the HTTP proxy 312 using an edge server authentication technique and/or a SWF-verification back-channel.
When a Client 314 requests a particular stream, the HTTP proxy 312 (to which the client has been directed, typically via DNS) starts the streaming process by retrieving a “Stream Manifest” that contains preferably only slowly changing attributes of the stream and information needed by the HTTP proxy to track down the actual stream content. The URL to download this manifest preferably is constructed deterministically from metadata delivered (e.g., via the distributed data transport mechanism of
For “live” requests, the HTTP proxy starts requesting content relative to “now,” which, in general, is approximately equal to the time on the edge machine HTTP proxy process. Given a seek time, the HTTP proxy downloads a “Fragment Index” whose name preferably is computed based on information in the indexInfo range and an epoch seek time. Preferably, a Fragment Index covers a given time period (e.g., every few minutes). By consulting the Fragment Index, an “Intermediate Format (IF) Fragment” number and an offset into that fragment are obtained. The HTTP proxy can then begin downloading the file (e.g., via the cache hierarchy 318, or from elsewhere within the CDN infrastructure), skipping data before the specified offset, and then begin serving (to the requesting Client) from there. Preferably, the IF fragments are sized for optimal caching by the HTTP proxy. In general, and unless the Stream Manifest indicates otherwise with a new indexInfo range, for live streaming the HTTP proxy then continues serving data from consecutively-numbered IF Fragments.
As used herein, and in the context of live HTTP-based delivery, the Intermediate Format (IF) describes an internal representation of a stream used to get data from the RTMP Puller through to the edge machine HTTP proxy. A “source” format (SF) is a format in which the Entry Point 306 provides content and a “target” format (TF) is a format in which edge machine HTTP proxy 312 delivers data to the Client 314. According to this disclosure, these formats need not be the same. Thus, SF may differ from TF such that a stream may be acquired in FLV format and served in a dynamic or adaptive (variable bit rate) format. The format is the container used to convey the stream; typically, the actual raw audio and video chunks are considered opaque data, although transcoding between different codecs may be implemented as well. By passing the formats through the HTTP proxy (and delivering to the Client via conventional HTTP), the container used to deliver the content can be changed as long as the underlying codecs are managed appropriately.
Referring now to
At step 3, the Archiver 410 retrieves from the Puller 408 the IF fragments along with their corresponding index information. The Archiver 410 appends the index information for each IF fragment to the current Fragment Index (FI) file. Preferably, the Archiver 410 caches a predetermined number of IF fragments for live play-back.
As the fragments age out, preferably they are deleted from the Archiver 410 and, at step 4, they are archived, e.g., to the Storage subsystem 416. Thus, at set intervals (e.g., every few minutes), the Archiver 410 closes the current FI file, archives it to the Storage subsystem 416, and begins creating a new FI file.
At step 5, and after an end user Client 414 has been associated with a particular edge machine, the HTTP proxy 412 in that machine gets the fragments for live play-back and limited DVR time periods from the Archiver 410 (possibly via the cache-hierarchy 418). Fragments no longer available on the Archiver 410 are retrieved from the Storage subsystem 416. A Muxer process 415 that operates in association with the HTTP proxy 412 facilitates this operation. Preferably, each IF fragment is a separate object for the HTTP proxy 412 that can be and in this embodiment is accessed through HTTP. In other words, according to this disclosure, the live stream is broken up into many small objects/fragments. The HTTP proxy 412 receives DVR commands from the Client player, typically on a separate HTTP connection. When the player requests to begin playing from a new stream position, the HTTP proxy uses metadata from the Stream Manifest file to calculate which FI file contains the target time offset. The FI file is retrieved from the Archiver 410 or the storage sub-system 416 (or, alternatively, from a peer machine co-located with the edge machine) and contains the IF fragment and byte offset to begin streaming to the Client player.
The Fragment Indexes enable the HTTP proxy process (to which a particular Client has been associated) to find a frame around a desired “seek time.” Preferably, each Fragment Index file contains index information covering a fixed amount of time. The exact interval is stored in the Stream Manifest for each indexInfo range. The desired seek time (epoch time) can be rounded down to the nearest interval boundary to find the Fragment Index to request.
Preferably, each stream is represented completely by the Stream Manifest, the Fragment Index and the IF Fragments. In an illustrative embodiment, the Stream Manifest is an XML file that contains the following information: stream epoch time (this time may be the time when the stream started or may be the oldest archived portion of the stream still available); stream Properties (like bit rate, video size, codec information, etc.); information about fragment indexes and which URL pattern to use to request FI file; and URL pattern for the fragments. The Fragment Index (FI) typically comprises the following: information about which key frame to start streaming from for a given time slice; key frame-to-fragment number mapping, key frame-to-time mapping, key frame to byte-offset in that fragment mapping, and so forth. Each IF Fragment contains approximately N seconds of stream, preferably optimized for HTTP proxy caching and not necessarily fragmented on time boundaries. Each fragment is composed of a fragment header, fragment stream header and a payload, and each fragment is uniquely identified by the fragment number. Fragment numbers incrementally increase.
Typically, and with reference back to
The Muxer subsystem 415 associated with (or within) the HTTP proxy determines how to request IF, converts IF to the output stream, and passes this data to the HTTP proxy for serving to the requesting client. In addition, preferably the HTTP proxy process supports a control channel by which the client can make any combination of various requests against an active stream including, without limitation, sessionToken, seek, and switch. The control channel facilitates flow control when working in some runtime environments, such as where the client lacks its own flow control facilities. In this situation, the control channel passes throttle commands that may be based on a percentage of an average bit rate (over the server-to-client connection) to help maintain full a target buffer on the client side of the connection. A sessionToken request is a request to provide additional authentication information, e.g., via SWF Authentication. A “seek” is a request to start sending data as of a different time in the stream (including “jump to live”). A “switch” is a request to start sending data from a different track from the same Stream Manifest. This might be a bit rate switch and/or an angle change.
Thus, the HTTP proxy receives DVR commands from the client player, preferably on a separate HTTP connection. When the client player requests that playback begin from a new stream position, the HTTP proxy uses metadata from the Stream Manifest file to calculate which FI file contains the target time offset. The FI file is retrieved (e.g., from the Archiver or the Storage subsystem, or from a peer machine) and contains the IF fragment and byte offset to begin streaming to the client player.
As described, the Stream Manifest preferably is an XML file and contains information about fragment indexes and how to construct the URL for an FI file, how to construct the URL for the “now” request, and how to construct the URL for the fragments. The HTTP proxy caches the manifest, which can be retrieved to the proxy either from an Archiver (which may be tried first), or the Storage subsystem. Client players connect to the HTTP proxy to play the live stream (i.e., connect to the stream's “now” time). In response, the HTTP proxy makes a forward request to the Archiver to fetch the “now” time on a live stream. Metadata in the Stream Manifest is used by the HTTP proxy to create the “now” URL.
As also described, a stream has a number of FI files. Each contains stream keyframe information for a given time slice. The Fragment Index allows time offsets to be mapped to fragment numbers and byte offsets. The Stream Manifest file defines the time slice for each FI file.
Each IF Fragment contains approximately N seconds of a stream. Each fragment is composed of a header and a payload. The HTTP proxy understands the data in the header, but the payload is opaque. The HTTP proxy links together with a Muxer component to convert the IF-formatted payload to the target format that is streamed to the client player. The fragments are cached in the HTTP proxy for re-use, and each fragment is identified with its stream name and an integer suffix that increases incrementally. As described above, Archiver has the fragments for the most recent N minutes of the stream, and the rest of the fragments are on the Storage subsystem.
For non-authenticated content, preferably the client player connects to an http: //URL to play a stream. Query string parameters can be used to request a particular seek time if the default (live if the stream is live, or the beginning of the stream if it is not live) is not appropriate. For authenticated content, preferably the original http: //URL additionally contains a shared authentication token query string parameter generated by the customer origin. This enables the HTTP proxy process to serve the stream for some configured amount of time (e.g. a given number of seconds). After that time, the HTTP proxy process terminates the connection unless, for example, an out-of-band control POST is received with a signed “session token.” Although not meant to be limiting, in one approach this token preferably is generated by the client by connecting to an FMS (or equivalent) edge machine that can perform SWF Verification (as shown in
A Demuxer process may be integral to the Puller; likewise, a Muxer process may be integral to the HTTP proxy process. There may be one Demuxer process for multiple Pullers; there may be one Muxer process for multiple HTTP proxies (within a particular Region).
As noted above, in terms of functionality, Demuxer converts regular stream packets into IF fragments and Muxer does the opposite. By definition, Demuxer and Muxer should complement each other. As noted, Demuxer can be part of an RTMP Puller process or can be a separate process running on the RTMP Puller machine. Demuxer receives input via the RTMP Puller. It is responsible to do the following: generate IF Fragment Header, take the source format and package the same into IF body, add Fragment Stream Header, Push IF fragment to Archiver, analyze the fragment and generate index information pertinent to key frame location within a given FLV packet, Push Key frame information to the Archiver. This can be done synchronously/asynchronously with respect to the IF fragment transmission. Preferably, Demuxer also is responsible for determining an optimal size of the fragment, which fragment size should be optimal for HTTP proxy caching. Demuxer can base its decision (regarding the optimal size of the fragment) by examining the following stream properties: incoming live stream byte rate/bit rate; Key Frame Interval, or a combination of both. Apart from constructing IF Fragments, Demuxer is also responsible to push Stream Properties and key frame information to the Archiver. Archiver can then create the Stream Manifest file that will be used by the HTTP proxy/Muxer to make fragment index and individual fragment requests. As described above, Muxer complements Demuxer. As Demuxer is responsible for constructing IF Fragments, Muxer is responsible for deconstructing the IF Fragments and converting the IF Payload format to the target format (TF) that the Client requested. The Muxer may also provide the following information to the HTTP proxy: statistics information about HTTP delivered Streams; and client session playback Information, such as playback duration, etc., and Muxer health data.
The Demuxer and Muxer enable dynamic transmux output to other file formats. This enables the system to leverage a single set of content sources for different device capabilities, e.g., iPhone 3.0 streaming using MPEG-2 TS Segments, Microsoft Silverlight 3 (with H.264 playback), Shoutcast®, and so forth.
As a variant to the above-described “pull” model that operates between an Encoder and an Archiver, it is also possible to use a “push-based” approach.
The above-described architecture is useful for live streaming, particularly over formats such as Flash. The following section describes adding video on demand (VOD) support to the platform. In particular, the solution described below provides VOD streaming from customer and Storage subsystem-based origins, provides single and multiple bitrate (SBR and MBR) streaming, provides support for origin content stored in flv and mp4/flv containers (supported mp4/flv codes include, among others, AAC, MP3, PCM for audio, and H.264 for video), and minimizes download of content beyond what is directly requested by the end user.
For VOD delivery, the stream recorder tier 300 (of
Before describing a VOD implementation in detail, the following section describes several ways in which VOD content is off-loaded for HTTP delivery to the CDN. In a first embodiment, a conversion tool (a script) is used to convert source content flv to IF, with the resulting IF files then uploaded to the Storage subsystem. In this approach, metadata is used to have an HTTP proxy go forward to the Storage subsystem to retrieve the stream manifest, which then references the Storage subsystem for the remaining content. In this approach, files in mp4/flv are first converted to flv (e.g., using ffmpeg copy mode) to change the container to flv. Another approach is to have a CDN customer upload raw media files to the Storage subsystem and to run a conversion tool there. Yet another alternative is to have the customer (or encoder) produce content in IF directly.
The translation tier approach is now described. In this approach, an on-demand dynamic IF generator machine takes requests for IF (manifests, indexes, and fragments) and satisfies these requests by dynamically retrieving flv or mp4/f4v input file ranges (either from the Storage subsystem or customer origin). From there, HTTP proxy treatment is essentially the same as the “conversion tool” options described above. The generator machine preferably runs its own HTTP proxy (the “translator HTTP proxy”) to cache various inputs and outputs, together with a translator process (described below) that accepts requests (e.g., from a localhost connection to the translator HTTP proxy) and generates IF based on data retrieved from the HTTP proxy via an associated cache process. In an alternative, the translator process may comprise part of the translator HTTP proxy, in which case IF generation takes place within the proxy. Fragment generation may also be carried out in an edge machine HTTP proxy or even further downstream (into the Client itself), such as where a Client maintains a session connection with one or more peer clients.
An architecture and request flow of a preferred approach is shown in
As used below, “C” refers to the Client, “EG” refers to an edge machine HTTP proxy, “TG” refers to a translator machine HTTP proxy (such as proxy 802 in
C→EG: The process begins when a Client makes a request to the edge machine HTTP proxy for the desired content. A representative URL might be as follows:
http://efvod.customer.com.akamaihd.net/foo/bar/baz.mp4
EG→itself: The EG HTTP proxy then matches on the request pattern, and sets a number of parameters, such as “event,” “angle,” “bitrate,” and “streamed.” The actual pathname is the “event,” and the “streamid” identifies a customer configuration associated with the request.
EG→TG: The EG HTTP proxy then goes forward to retrieve the Stream Manifest from a translator machine; preferably, the HTTP proxy goes forward by providing a metadata-configured hostname (to the CDN DNS) that resolves to addresses associated with the translator machines. Preferably, the request path is prefixed with a customer identifier and protected by a per-customer secret, e.g., based on an authentication mechanism that operates between HTTP proxies. A representative URL might be as follows:
http://translator.customer.com.akamaihd.net/custid1/translate/fo o/bar/baz.mp4?object=manifest
TG→T: If the translator HTTP proxy has already produced the Stream Manifest (or can find it, e.g., via ICP), the Manifest is returned to the edge HTTP proxy that requested it. Otherwise, the translator HTTP proxy itself goes forward to localhost to retrieve the manifest. Preferably, the customer identifier prefix is preserved. The translator HTTP proxy may also apply one or more host headers on the manifest request (to the translator process) that are then echoed back to the translator HTTP proxy in any requests for origin content, metaindexes, etc. Preferably, all TGT interactions carry these same headers. These headers ensure that the translator HTTP proxy is only fulfilling requests for the proper customer, and also to facilitate load balancing. The translator requests may also be authenticated using a cookie and a per-customer secret. A representative URL might be as follows:
http://translator.customer.com.akamaihd.net/custid1/translate/fo o/bar/baz.mp4?object=manifest
Forward IP/port: 127.0.0.1:yyyy
T→TG: The translator in turn retrieves the “metaindex” for the stream from the translator HTTP proxy using the stream cache process and the cookie. (To produce a manifest, typically only the beginning of the metaindex is retrieved.) The translator applies the customer-specific secret (from the original request) to the metaindex request back to the HTTP proxy. With the information in the “metaindex,” the translator produces the Stream Manifest. A representative URL is as follows:
http://translator.customer.com.akamaihd.net/custid1/metaindex/fo o/bar/baz.mp4&format=mp4
TG→T: If the translator HTTP proxy does not have the metaindex already cached, it again goes forward to the translator (same procedure as for the Manifest). A representative URL is as follows:
http://translator.customer.com.akamaihd.net/custid1/metaindex/fo o/bar/baz.mp4&format=mp4
Forward IP/port: 127.0.0.1:yyyy
T→TG: When the translator receives the request for the metaindex, it retrieves (via the stream cache process using same procedure as above for the Manifest) a small subset of the original content, which the translator HTTP proxy is able to retrieve from origin, preferably using partial object caching (POC). For flv files, preferably only the very beginning and the last block will be read. For mp4/f4v files, preferably the “moov” atom at the beginning of the file is processed in its entirety. A customer-specific prefix and secret preferably are passed along this entire path. A representative URL is as follows:
http://translator.customer.com.akamaihd.net/custid1/origin/foo/b ar/baz.mp4
TG→O: The translator HTTP proxy ultimately goes forward to the appropriate origin on POC misses. A representative URL is as follows:
http://some.origin.com/some/customer/prefix/foo/bar/baz.mp4
EG→TG→ . . . →O: At this point, the edge machine HTTP proxy has a manifest and needs an index. The edge machine HTTP proxy again goes forward to the translator for the index. The process proceeds in essentially the same way as for the Manifest. For “mp4/f4v” files, preferably all indexes are pre-computed and stored verbatim in the metaindex so that no mp4/f4v content access is required. This is not a requirement, however. For “flv” files, producing full index information typically requires reading the entire file; thus, for this type of content, preferably small indexes (e.g., each a few seconds) are generated, e.g., using a binary search over the flv content and then downloading only that window of content. The forward request from the HTTP proxy (as specified in the Stream Manifest) may be as follows:
http://translator.customer.com.akamaihd.net/custid1/translate/fo o/bar/baz.mp4?object=index&number=1234&etag=3-1234567890ab
Additional query-string parameters may be passed from the manifest back to the index or fragment requests to ensure consistency between any parameters that might have changed between manifest generation and index/fragment retrieval.
EG→TG→ . . . →O: The final type of request is for the actual fragment. For mp4/f4v files, the metaindex includes instructions sufficient to produce any requested IF fragment without reparsing or otherwise consulting the original “moov” atom. Preferably, the actual raw data is retrieved via POC and/or the stream cache process and assembled into valid IF fragments. For flv files, the fragment is generated by seeking directly to “desired_fragment_size * (fragment_number-1)” and then scanning forward for the first valid flv tag. The end of the fragment is just the first tag boundary at or after file offset “desired_fragment_size * (fragment_number)”. A representative URL is as follows:
http://translator.customer.com.akamaihd.net/custid1/translate/foo/bar/baz.mp4?object=fragment&number=1234&etag=3-1234567890ab
This completes the description of the client to origin request flow for an SBR stream request. The following provides additional details regarding the VOD implementation.
The metaindex is a binary format preferably comprising several sections, preferably in the following order: (i) fixed-size global header with per-stream information; (ii) additional “content-verifier” information, e.g., ETags, used to identify which file has been indexed; (iii) an “on MetaData” tag for the stream; (iv) per-track information, including flv codec identifier and optional codec configuration parameters required by AAC and H.264; (v) (mp4/f4v only) Lookup table to map fragment numbers to “sample_info” below; (vi) (mp4/f4v only) IF Indexes; (vii) (mp4/f4v only) “sample_info” which describes each individual sample to be interleaved into the output. The “sample_info” essentially is an optimized version of the “moov” atom that abstracts out many of the complexities of the mpeg-4 container. In general, this should be a fairly compact representation, often smaller than the corresponding “moov” atom.
A Stream Manifest may include all bitrates of a multi-bit rate (MBR) stream. To produce an MBR Stream Manifest, the manifest request may reference an SMIL file, which in turn contains the individual bitrates and flv or mp4/f4v source files. For MBR delivery, to produce the actual manifest the translator downloads metaindexes for each bitrate. The index and fragment requests only require the metaindex file for the particular bitrate(s) that the edge machine HTTP proxy wants to play. MBR fragment and index requests are similar to their SBR counterparts, except they also include an “smil= . . . ” query-string argument added to each path in the C→EG→TG→T chain via patterns in a <locationInfo> tag in the manifest.
Preferably, the actual format (“mp4” vs. “flv”) is included in metaindex, fragment, and index URLs as a “format” query string argument.
As noted above, IF generation can take place within or in association with the HTTP proxy, including even an edge machine. This enables the creation of content at the edge itself, close to the requesting end user. Indeed, this approach (of IF generation in or in association with the HTTP proxy) can move even further downstream, namely, into the client machine itself. This is possible if the client includes software that facilitates peer-based network session connectivity to a peer-to-peer (P2P) network, at least some portion of which is based on CDN resources including edge machines running HTTP proxies. As noted above, it is possible the IF is generated in an HTTP proxy (or in a Muxer process associated therewith). When such P2P network session software executes within a Client, it is possible to extend the HTTP proxy interface all the way downstream to the Client machine itself. In this approach, muxing takes place in the Client itself, in which case the edge machine becomes just a “parent” cache in a cache-hierarchy that includes the client-side delivery mechanism. This approach could also be used with mobile devices with limited bandwidth.
The techniques described herein facilitate the delivery of high definition video and audio (including advanced video features, such as DVR) over an HTTP-edge network which, in a typical CDN, is the network that has the largest footprint. By implementing such techniques, a provider can leverage its existing HTTP-based servers instead of having to implement and maintain dedicated server networks to support multiple third party runtime environments. Moreover, because the delivery is HTTP-based, the content can be seamlessly delivered to clients operating across fixed line and mobile environments. No special client software is required, as the HTTP proxy (that responds to the client request) dynamically re-assembles fragments that it obtains and serves the requested content via HTTP. Further, because delivery within the set of interconnected machines of the CDN preferably takes advantage of an intermediate format, the network can ingest content in one format yet serve it in another, all while preserving single or multi-bitrates and DVR-like functionality. Thus, for example, the network may take in live RTMP packets and serve the content as an FLV progressive download. Preferably, each IF fragment of the stream is a separate object for the HTTP proxy that can be accessed, cached, and served via HTTP. According to the scheme, the stream is broken up into many small objects (fragments), with each fragment managed separately.
The network is not limited for use with any particular runtime environment such as Flash. By leveraging the approach as described, a single set of content sources can be leveraged for different device capabilities. Thus, the techniques as described herein include dynamically transmuxing content to other file formats in a manner that is transparent to the content provider and the end user.
The intermediate format may be based on or adapted from any convenient multimedia file format that can be used delivery and playback of multimedia content. These include, without limitation, fragmented mp4, protected interoperable file format (piff), and others. More generally, any linked list-based file format may be used.
Preferably, the CDN service provider provides an extranet (a web-based portal) through which the stream delivery is provisioned.
The following section provides an alternative to the RTMPE and SWF file authentication technique referenced above. Three security areas are considered, as relevant to an HTTP proxy (or otherwise): user authorization, player verification, and stream encryption.
User authorization involves, for example, ensuring that a user has permission to play a stream of content. In one implementation, an HTTP proxy, such as HTTP proxy 312 described above in connection with
The use of such URL-based authorization can be signaled by setting the stream-ID in the URL to be played (e.g., by setting a value or a prefix thereof). The user authorization is preferably done before any streaming code/functionality is executed, for example before the stream manifest is fetched.
Representative metadata to implement such authorization at the edge is provided below. In this case the match function looks for a prefix of “URLauth” to determine that URL-based authorization is used:
Player verification typically involves ensuring that a stream is played from player that is approved by the content provider and unaltered. For the Adobe® Flash® runtime environment, the player would be a SWF player, while for the Silverlight® environment, the player would be a .XAP application, and so on. Though the code is running on an untrusted client device, preferably player verification prevents or hinders the use of modified players, allows for efficient revocation of compromised players, and offers configuration/update of security implementation (e.g., the hash function employed and/or keys) without necessarily updating the player. User authorization and player verification can be used independently. For example, it may be determined that content consisting of ad banners or commercial breaks does not require user authorization but player verification is. In the opposite case, some customers might require user authorization to be able to track registered users for marketing purposes, but not care if a player has been tampered with (e.g., because their stream has a low value).
In one implementation, an HTTP proxy (such as previously described) inserts a challenge and a file inside a content stream. In the Flash environment, the file may be a SWF file.
The stream may carry the file with a filename or without a filename, in the latter case the stream delivering the content of the file. For convenience of description herein the file is referred to as “hash.foo” regardless of whether it is delivered with a filename. As noted, in a Flash environment, the filename extension would be swf.
The file provides a set of instructions that represent a program to be executed by the player. For example, it can define a function that obtains identifying information associated with the player and hashes it (along with the challenge, in some implementations) into a token. It then passes the token to the HTTP proxy. The identifier may represent, for example, the byte array of the player, a unique serial number for the player, an authorization code, a category/class number for the player, a hardware identifier for the client device, and/or other kinds of information. In alternate embodiments, the identifier may correspond to a user of the client player stored on the client device, and thereby be used to indicate that the user is authorized. In some embodiments, for the case of identifiers like a hardware ID or serial number, the user may be required to pre-register with their credentials with the HTTP proxy or another component of the content delivery system, such registration causing the content delivery system to capture the hardware identification for later verification of the user's client device.
The use of a file, such as a SWF file, as described above is not limiting, as the HTTP proxy may deliver programmatic instructions to the client in a variety of formats, such as by delivering byte code, source code, compiled instructions suitable for direct execution by the client machine, or other instruction sets, with or without a filename.
The HTTP proxy typically initiates player verification after the request for content is received. For example, the file may be inserted at the beginning of the stream or during delivery of the stream (e.g. typically within some short time period from the beginning). In alternate embodiments, the file may be delivered out of band, e.g., via another HTTP connection with the HTTP proxy serving the content, an in-region player verification server, or on another machine.
After receiving the token, the HTTP proxy determines whether the player is valid. For example, the HTTP proxy can compute its own token using the same hash function expressed as a string in the dynamic hash language defined below, and compares it with the token sent by the Player. If they match, the stream can continue playing (although it may be subject to a further authorization procedure). Otherwise, the HTTP proxy aborts it. The computation of a corresponding token to verify the player is not a limitation, however, as a wide variety of techniques may be used to determine whether the token and/or the information encoded in it indicates a verified player (e.g., comparing the player identifier to a known list of players provided by a CDN customer, or otherwise).
Typically, the player does not know about the need for verification until it receives the first section of the stream, as the HTTP proxy determines whether to use player verification in most cases. As noted previously, in one embodiment, the HTTP proxy embeds information in the stream for the Player. This information can include a Session-ID, Total Archive Time, and other attributes. If player verification is required, the HTTP proxy inserts the file with program (either the content alone or content with a filename) in as a binary blob under the data parameter.
The file defines a function called which the Player can execute. The implementation of this function preferably is restricted. For example, it could use one or more of a closed set of mathematical encoding algorithms, such as the MD5 hashing algorithm.
Preferably the byte array or other identifying information for the player is obtained programmatically inside the function rather than passed as a function parameter. If the function takes byte arrays as input(s) and output, the byte array returned by the function can be delivered as a base64-encoded string.
If the program/function is present under the data parameter in the stream, the player executes it. The returned string is called the session token. Typically, the player has a short time (e.g., a few seconds) to make an authentication request to the HTTP proxy and pass this token inside the X-cdnsp-Streaming-SessionToken or other HTTP request header. Once the token is sent to the HTTP proxy and the authentication request is made, the player can discard the token if it is not needed.
The HTTP proxy decides whether or not the player verification is required based on the secure flag inside the stream manifest. Preferably, metadata can be used to overwrite this flag at the HTTP proxy, rather than re-publishing the same stream with a different secure flag. Note that the HTTP proxy preferably does not use the stream-ID's prefix to decide if player verification is required. As far as the HTTP proxy is concerned, in this implementation, the stream ID prefix is only used to decide if URL-based authorization is required.
The HTTP proxy can utilize a dynamic hash file to perform the verification process described above. A dynamic hash file is a simple XML file that defines a custom hash function expressed under two equivalent forms: (1) The binary-foo string that holds a base64-encoded file “hash.foo.” This data can be passed by the HTTP proxy to the player in base64-decoded form through the data parameter, as described above. As noted, the player executes the function to yield the session token. (2) the “expression” string in the dynamic hash language, e.g., as set forth below. This string is parsed by the HTTP proxy into a small program that verifies that the session token sent by the player is valid.
Here is a sample dynamic hash file:
The URL for the hash.foo file (and/or dynamic hash file with hash.foo file contained in it) is set in metadata for the HTTP proxy. Such files can be hosted on storage provided by a CDN, provided by the customer, or otherwise. Metadata can be used to control their time-to-live (TTL) and setup authentication.
In the foregoing implementation, the HTTP proxy uses a database of authorized players. The HTTP proxy gets a list of authorized hash.foo files and/or dynamic hash files by fetching a URL specified in metadata. This URL may be served by an origin server or CDN storage subsystem. The response can contain a directory listing that shows all the SWF or other files allowed (e.g., as designated by a customer of the CDN), as well as a hash of each file, allowed byte arrays or other identifications of the players, as well as a hash of each byte array or identification, and/or other information.
The HTTP proxy may log errors (such as a failure of player verification, or otherwise) and specify the reason for the error or failure. For example, the player verification process may fail because of a verification timeout, the receipt of a bad session token from the player, or a failure on the part of the HTTP proxy to be able to verify the token (e.g., because there is a problem with its list of authorized players).
The HTTP proxy can be configured using metadata to manage player verification process in a variety of ways. For example, the HTTP proxy may be configured to receive certain information from the client player (e.g., the hash.foo file name, or the first few characters of the hash it calculated for the identifier and the challenge) in order to improve the speed of the HTTP proxy in checking the session token from the player against the database of authorized players. Other metadata may configure the timing of the player verification. For example, the HTTP proxy may or may not be configured to wait for player verification to complete before streaming audio/video or other content to the player.
In alternate embodiments, the client identifying information (e.g., player byte array), encoded into a token, can be delivered to the HTTP proxy with the initial request for content. For example, the client player can be configured with library/code support sufficient to enable it to determine its own client identifying information and to encode that information into a token. The token can then be sent to the HTTP proxy for verification along with the initial request for content, as described above (although the request for content and the token may be transmitted over distinct channels).
Further, the ability of the client player to perform some of the client-side verification functionality itself allows the program delivered via the hash.foo file to be eliminated (if client player is self-sufficient) or, preferably, to contain a reduced or incomplete instruction set for verification. The hash.foo file may then supply a necessary piece of the instruction set in order for the client to fulfill the verification process that was described above. For example, the client player may be able to obtain its client identifying information and how to encode that information into a token, but the hash.foo file delivered from the HTTP proxy may specify a secret to be encoded into the token that is returned. As another example, the hash.foo file may specify how to encode the token, by specifying one of set of mathematical algorithms to use, by specifying the calculation itself, or otherwise. Such an approach advantageously enhances security of the system by ensuring that the complete set of functionality necessary for verification on the client-side is present only at content delivery time.
The communications between the player and the HTTP proxy may be encrypted, e.g., using HTTPS, so as to advantageously obscure the verification scheme and to protect the streamed content.
The clients, servers, and other devices described herein may be implemented on conventional computer systems, as modified by the teachings hereof, with the functional characteristics described above realized in software, hardware, or a combination thereof.
Software may include one or several discrete programs. Any given function may comprise part of any given module, process, execution thread, or other such programming construct. Generalizing, each function described above may be implemented as computer code, namely, as a set of computer instructions, for performing the functionality described via execution of that code using conventional means, e.g., a processor, a computer, a machine, a system, digital data processing device, or other apparatus. In one embodiment, such software may be implemented in a programming language that runs in conjunction with a DNS-compliant name server (e.g., BIND).
Computer system 1000 includes a processor 1004 coupled to bus 1001. In some systems, multiple processor and/or processor cores may be employed. Computer system 1000 further includes a main memory 1010, such as a random access memory (RAM) or other storage device, coupled to the bus 1001 for storing information and instructions to be executed by processor 1004. A read only memory (ROM) 1008 is coupled to the bus 1001 for storing information and instructions for processor 1004. A non-volatile storage device 1006, such as a magnetic disk, solid state memory (e.g., flash memory), or optical disk, is provided and coupled to bus 1001 for storing information and instructions. Other application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or circuitry may be included in the computer system 1000 to perform functions described herein.
A peripheral interface 1012 communicatively couples computer system 1000 to a user display 1014 that displays the output of software executing on the computer system, and an input device 1015 (e.g., a keyboard, mouse, trackpad, touchscreen) that communicates user input and instructions to the computer system 1000. The peripheral interface 1012 may include interface circuitry, control and/or level-shifting logic for local buses such as RS-485, Universal Serial Bus (USB), IEEE 1394, or other communication links.
Computer system 1000 is coupled to a communication interface 1016 that provides a link (e.g., at a physical layer, data link layer, or otherwise) between the system bus 1001 and an external communication link. The communication interface 1016 provides a network link 1018. The communication interface 1016 may represent a Ethernet or other network interface card (NIC), a wireless interface, modem, an optical interface, or other kind of input/output interface.
Network link 1018 provides data communication through one or more networks to other devices. Such devices include other computer systems that are part of a local area network (LAN) 1026. Furthermore, the network link 1018 provides a link, via an internet service provider (ISP) 1020, to the Internet 1022. In turn, the Internet 1022 may provide a link to other computing systems such as a remote server 1030 and/or a remote client 1031. Network link 1018 and such networks may transmit data using packet-switched, circuit-switched, or other data-transmission approaches.
In operation, the computer system 1000 may implement the functionality described herein as a result of the processor executing code. Such code is typically read from or provided by a non-transitory computer-readable medium, such as memory 1010, ROM 1008, or storage device 1006. Other forms of non-transitory computer-readable media include disks, tapes, magnetic media, CD-ROMs, optical media, RAM, PROM, EPROM, and EEPROM. Any other non-transitory computer-readable medium may also be employed. Executing code may also be read from network link 1018 (e.g., following temporary storage in an interface buffer, local memory, or other circuitry).
This application is a continuation of U.S. patent application Ser. No. 12/980,996, filed Dec. 29, 2010, which claims the benefit of priority of U.S. Provisional Application No. 61/290,732, filed Dec. 29, 2009, the disclosure of both of which are hereby incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61290732 | Dec 2009 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12980996 | Dec 2010 | US |
| Child | 14296322 | US |
