Video and digital multimedia aggregator content coding and formatting

[0002] The following U.S. Patents also are incorporated by reference:

[0003] U.S. Pat. No. 5,798,785, entitled “Terminal for Suggesting Programs Offered on a Television Program Delivery System;”

[0004] U.S. Pat. No. 5,986,690, entitled “Electronic Book Selection and Delivery System;”

[0005] The following co-pending U.S. patent applications also are incorporated by reference:

[0006] patent application Ser. No. 07/991,074, filed Dec. 9, 1992, entitled “Remote Control for Menu Driven Subscriber Access to Television Programming;”

[0007] patent application Ser. No. 08/906,469, filed Aug. 5, 1997, entitled “Reprogrammable Terminal for Suggesting Programs Offered on a Television Program Delivery System;”

[0008] patent application Ser. No. 09/191,520, filed Nov. 13, 1998, entitled “Digital Broadcast Program Ordering;”

[0009] patent application Ser. No. 09/289,957, filed Apr. 13, 1999, entitled “Electronic Book Alternative Delivery Systems;”

[0010] patent application Ser. No. 09/289,956, filed Apr. 13, 1999, entitled “Electronic Book Alternative Delivery Methods;” and

[0011] patent application entitled “Video and Digital Multimedia Aggregator;” and patent application entitled “Video and Digital Multimedia Aggregator Remote Content Crawler,” filed on date herein.

TECHNICAL FIELD

[0012] The technical field is television program and digital multimedia delivery systems that incorporate intelligent and flexible program search and delivery mechanisms.

BACKGROUND

[0013] Cable television delivery systems greatly expanded the array of programs available to television viewers over that provided by over-the-air broadcast systems. Subsequently, satellite television delivery systems, and in particular, digital satellite broadcast systems further expanded the viewing choices for consumers. In the near future, digital broadcast television systems will provide many more programming choices for consumers.

[0014] In addition to television programming delivered through television program delivery systems, other programs and events may be sent to consumers. These other programs and events include streaming video sent over wired and unwired, narrowband to broadband services, digital audio programs, and other multimedia data.

[0015] Unfortunately, customers are still limited in their television viewing choices by the local and regional nature of television delivery systems. For example, a broadcaster in Boston may provide programming of local interests to people in Massachusetts while a broadcaster in Seattle may provide different programming to people in the Seattle area. A person in Boston generally cannot access Seattle programming, other than programming that is provided at a national level.

[0016] In addition to this local/regional market segregation, many other sources of programming and events may not be available in a specific viewing area. These other sources may include audio programming, streaming video, local or closed circuit television programming (e.g., education television programming provided by a state education department) and other programming.

[0017] Even if certain programming is available in a local viewing area, a viewer may not be aware of its existence. This situation may be the result of a large array of available programming coupled with a limited program menu or guide. The program guide may be limited in that not all available programming can be listed, some programming changes occur that are not reflected in the program guide, and errors may exist in the program guide. In addition, the more comprehensive the program guide, the harder it is for the viewer to search and navigate the program guide to find a desired program.

SUMMARY

[0018] The problems noted above are solved by the video and digital multimedia aggregator system and method described herein. Program content can be packaged and delivered by the system, including video, television, radio, audio, multimedia, computer software, and electronic books, or any content that can be delivered in digital format.

[0019] The aggregator comprises a request and results processing server, a search engine server coupled to the request and results processing server and a content acquisition server coupled to the request and results processing server. The request and results processing server receives a request for a program, the search engine server searches for the program and the content acquisition server acquires a program for delivery to the user. The request and results processing server includes a search request processor that receives information related to the user's search request and provides the information to a search results form builder that creates an electronic search request. The search request may be augmented by using a content search suggestion engine to add additional search terms and descriptions to the search request. The aggregator may also include a decoder that decodes program content and program metadata from remote sources for storage at the aggregator, and an encoder that encodes content metadata and programs for delivery to the user. The aggregator may also comprise one or more crawlers, such as a content crawler, to look for program content in the digital communications network.

[0020] The search engine server searches at least a local content database. The local database comprises at least two file types. A content file includes a complete program content file. For example, the 1997 movie Titanic may exist in the local content database as a complete program content file. The complete program content file may also include a reference content or metadata that contains additional information related to the content. Such additional information in the reference content may include: a program description, including program rating, program description, video clips, program length, format (e.g., 4×3 television or 16×9 movies), and other information; billing information and digital rights management information; viewing statistics, including number of times viewed, dates/times viewed, identity of users viewing the program; advertisement information to allow ads to be inserted during viewing of the program; and other information.

[0021] The additional information in the reference file may be provided in whole or in part to the system users. For example, the aggregator may provide a program description and accompanying video clips to selected systems users. The reference file may also be used by the aggregator for system administration purposes. For example, billing and digital rights management information may be used to collect appropriate fees from system users and to provide such collected fees to the entities owning rights in the content.

[0022] A remote content crawler continually crawls the digital communication network looking for content to provide to the aggregator. The content provided to the aggregator may be stored in a form of an entire content file. For example, the content may include an entire movie, television program or electronic book. Alternatively, the content provided to the aggregator may be a reference to a content file that is stored at, or that will be available at one of the remote locations. For example, the content may be a reference to a future, scheduled live sports event that will be made available to system users. The sports event may be provided for a one time fee, as part of a sports package, for which a fee is collected, or as a free event. In the examples discussed above, the content may be stored at the aggregator, and may subsequently be provided to system users. For the example of the live sports event, the aggregator may store the live sports event and may then provide the sports event as a replay, in addition to facilitating live viewing of the sports event.

[0023] To ensure the content is delivered to the correct addressee, in a format that is compatible with the addressee's equipment (i.e., hardware and software), the aggregator may include one or more content coders, decoders and formatters. In an embodiment, the coding, decoding and formatting functions may be executed by separate components, which may be embodied as hardware or software, or a combination of hardware and software. In another embodiment, two or more of the functions may be executed by a single device, comprising hardware and/or software.

[0024] In an embodiment, the formatting, decoding and coding functions are executed by a coder-decoder (codec) formatter. The codec formatter may comprise means for receiving a coding and formatting request, means for analyzing parameters contained in the coding and formatting request, means for decoding, formatting and coding target content, means for configuring the decoding, formatting and coding means, and means for routing coded target output content to one or more target addresses.

[0025] In another embodiment, the codec formatter may include means for processing auxiliary service requests. Such auxiliary service requests may be included in the coding and formatting request, or may be separately supplied. The auxiliary services may include closed captioning, descriptive video narration, alternative language audio, content rating, critical review information, device control and commands, future content schedules, advertising, targeted advertising, text and data services, interactive services, and content metadata. The means for processing auxiliary services includes means for analyzing the auxiliary service requests, means for configuring one or more auxiliary services processing means to supply the requested auxiliary services, and means for outputting the requested auxiliary services. The outputted auxiliary services may be combined with the coded target output content.

DESCRIPTION OF THE DRAWINGS

[0026] The detailed description will refer to the following drawings in which like numerals refer to like elements, and in which:

[0027]
FIG. 1 is a block diagram of primary components of a content search, packaging, and delivery system;

[0028]
FIG. 2 is a block diagram of the components of the content search, packaging, and delivery system configured to deliver content to a set top terminal;

[0029]
FIG. 3 is a block diagram of the components of the content search, packaging, and delivery system configured to deliver content to a personal computer terminal;

[0030]
FIG. 4 is a schematic of the components of the content search, packaging, and delivery system showing subsystems of an aggregator, a user terminal, and a remote content server;

[0031]
FIG. 5 is a schematic of subsystems of a content acquisition server and components with which the content acquisition server interacts as part of a content search, packaging, and delivery system;

[0032]
FIG. 6 is a schematic of subsystems of a content delivery server and components with which the content delivery server interacts as part of the content search, packaging, and delivery system;

[0033]
FIG. 7 is a schematic of subsystems of a codec and content formatter and components with which the codec and content formatter interacts as part of the content search, packaging, and delivery system;

[0034]
FIG. 8 is a diagram of components of a formatting codec used with the codec and content formatter of FIG. 7;

[0035]
FIG. 9 shows a flowchart describing a process of decoding, formatting, and coding source content and outputting target content; and

[0036]
FIG. 10 shows a flowchart describing a method of processing auxiliary services that may accompany source and/or target content.

DETAILED DESCRIPTION

[0037]
FIG. 1 is a block diagram of a content search, packaging, and delivery system 200. The content to be packaged and delivered by the system 200 includes video, television, radio, audio, multimedia, computer software and electronic books. Components of the system 200 include an aggregator 201 and a user terminal 202, which are connected using a wide area distribution system 203. Other components are remote content servers 204 that exchange data with the aggregator 201 using a wide area network/Internet 205 connection. The user terminal 202 may incorporate a video display system 207, an audio playout system 208, an audio/video recording system 209, user input devices 214, an electronic book reader 230, and a connection to a home network 229 to interact with other devices in the user's home environment. Alternatively, one or more or all of the video display system 207, the audio playout system 208, the audio/video recording system 209, and the electronic book reader 230 may be separate components that are coupled to the user terminal 202.

[0038] The system 200 allows a user to enter search parameters, such as keywords and category headings, and have the aggregator 201 use these parameters to locate, package, and deliver content to the user terminal 202 from numerous sources. The requests and content deliveries can be sent over communications links including, but not limited to, telephone lines, coaxial cable, fiber-optic cable, wireless connections, wide area networks, the Internet, and other communication media collectively represented by the wide area distribution system 203. The numerous sources of content are shown in FIG. 4 and include, but are not limited to, an aggregator local storage 254, local streaming sources 262, remote content storage 258, and remote streaming sources 259. In an embodiment, the local streaming sources 262 are comprised of the audio and video channels being delivered using a cable headend system that may house the aggregator 201.

[0039] The system 200 will take a user's search request and may perform a search of virtually every active and scheduled radio and television channel in the world, as well as archived sources of video and audio programming such as movies and recorded audio sources and sources of other multimedia, software, and electronic book content. In an embodiment, the system 200 will also search Internet Web sites and other online databases. The user will then be able to select programming or other content for download based on the search results. In an embodiment, the download, or delivery, process can be fulfilled by making the content available on a specific channel of a cable television system, or by transmitting the content using a digital communications protocol, such as the Internet standard TCP/IP, for example. In addition, the content search, packaging, and delivery system 200 is capable of formulating and presenting a list of suggested content based on an analysis of the user's current search parameters, stored information about previous searches and previously selected content downloads and other user-specific or related information. The system 200 is also capable of notifying a user prior to the start time of selected programming and availability of other content using such notification as an electronic mail message and/or an on-screen message indicating that the scheduled program will be broadcast at a specified time. The system 200 may support one or more digital rights management (DRM) systems to track the usage and copyrights rights associated with downloaded content and bill the user's account as appropriate and provide any license and usage fees to the content provider. The system 200 may implement a users' privacy protection scheme allowing users to control what information is gathered, limit what is done with that information, and review and delete the user's profile if desired.

[0040] An overview of an embodiment of the system 200 is shown in FIG. 2 where the user terminal 202 is a set top terminal 206 that communicates with the aggregator 201 through a cable television system headend 210, thereby making use of the cable television system headend 210 high bandwidth concatenated cable system 216. Further, the set top terminal 206 system may include a tuner 228, a demultiplexer 231, the video display system 207, the audio playout system 208, the audio/video recording system 209, user input devices 214, an electronic book reader 230, and a connection to the user's home network 229. These components are used to tune, select, view, listen to, and store audio and video programming and other content delivered by the search, packaging, and delivery system 200. FIG. 2 also shows a communications path from one or more remote content servers 204 through the wide area network/Internet 205 directly to the set top terminal 206, which bypasses the aggregator 201 and cable television system headend 210. This path may be used in the case where the requested content is available in the required format from the remote content server 204 and is authorized for direct delivery to the user. In an alternative embodiment, the aggregator 201 is collocated with the cable television system headend 210 that is acting as the wide area distribution system 203 as is the case when a cable television system is also serving as the user's Internet service provider.

[0041] An overview of another embodiment of the system 200 is shown in FIG. 3, where the user terminal 202 is a personal computer terminal 211 that communicates with the aggregator 201 through an Internet service provider/cable television system headend 215. In this case, the content may be delivered by a cable headend that is operating as an Internet service provider (ISP). The personal computer terminal 211 also may include the video display system 207, the audio playout system 208, the audio/video recording system 209, user input devices 214, the electronic book reader 230, and the connection to the home network 229. A communications path also exists from one or more remote content servers 204 through the wide area network/Internet 205 directly to the personal computer terminal 211, which bypasses the aggregator 201 and the Internet service provider/cable television system headend 215 in the case where requested content is in the required format and is authorized for direct delivery to the user.

[0042] The user can receive video and audio programs (i.e., the content) in a number of different ways. First, the tuner 228 in the user terminal 202, shown in FIG. 2, can simply tune to the desired program, or the demultiplexer 231 can select a program from a digital multiplex, thereby displaying the program at the appropriate time. However, the desired program may not be broadcast by the user's cable television system headend 210, or the user would like to watch a program supplied by the local headend, but not subscribed to by the user, e.g., a boxing match broadcast by a premium channel that the user does not ordinarily receive. In these examples, the program can be delivered to the user by the aggregator 201 using the telephone lines, fiber-optic transmission lines, or other communication media, or using the cable television system headend 210. In an embodiment, the aggregator 201 can supply the desired program to the user by pulling programs from program delivery systems in the United States and abroad and delivering the program to system users. Therefore, a user can have access to programs outside the user's immediate viewing area.

[0043]
FIG. 4 shows the system 200 and system components in more detail. At a user's location, the user terminal 202 includes the tuner 228, the demultiplexer 231, a user terminal processor 227, user local storage 212 and user local database 213. Coupled to the user terminal 202 may be user input devices 214, the video display system 207, the audio playout system 208, the audio/video recording system 209, an electronic book reader 230, and a connection to the home network 229.

[0044] The user terminal 202 is coupled through the wide area distribution system 203 to the aggregator 201 and further through the wide area network/Internet 205 to remote program sources. The remote program sources include remote streaming sources 259 and remote central storage 258. The remote program sources also include remote databases 261 and, through the remote content server 204, a remote server database 260.

[0045] The aggregator 201 may include a communications server 250 that communicates with the user terminal 202 through the wide area distribution system 203. The communications server 250 receives inputs from a request and results processing server 300, a content delivery server 450 and a system administrator 500. The content delivery server 450 receives inputs from a coder and content formatter 253 and a content acquisition server 400. The content delivery server 450 also accesses an aggregator local storage 254 and local streaming sources 262. Finally, the content delivery server 450 provides an output to the system administrator 500.

[0046] The coder and coder formatter 253 receives inputs from the content acquisition server 400, the aggregator local storage 254 and local streaming sources 262. The system administrator 500 receives inputs from the content and delivery server 450, and communicates with the content acquisition server 400, the request and results processing server 300, a search engine server 350 and aggregator archives 255. A decoder and content formatter 252 is coupled to the content acquisition server 400. Finally, a network gateway 251 couples components of the aggregator 201 with the remote content server 204 through the wide area network/Internet 205.

[0047] Programs received at the aggregator 201 may be input to the formatter 253. The formatter 253 reformats all input content into a format that is readily received by all user terminals 202 operably connected to the delivery system 200 (not shown in FIG. 4). In addition, the formatter 253 can store full or partial copies of content in compressed form in the aggregator local storage 254. Furthermore, the aggregator 201 can provide real-time delivery of certain content, such as a boxing match. In an embodiment, if a large number of users want a particular live program, then the cable television system headend 210 can broadcast the program on a particular channel available to all the requestors instead of broadcasting the program to each individual user over the wide area distribution system 203. See U.S. patent application Ser. No. 09/191,520, entitled Digital Broadcast Program Ordering, hereby incorporated by reference, for additional details of broadcast program delivery.

[0048] The aggregator 201 can also implement a screening process for limiting the number of programs captured to those programs with a viewing audience above a predetermined threshold. In effect, the aggregator 201 contains a filter that will pass only programs meeting the predetermined selection criteria. The filter may include programming that screens the content to reject specific items, such as adult content, for example.

[0049] The system administrator 500 records what fees should be paid and to whom. For example, the aggregator 201 will determine to whom any copyright or other fees should be paid when a program is broadcast.

[0050] The user terminal 202 may be a television, a set top terminal 206, a personal computer terminal 211 (not shown), or any device capable of receiving digital or analog data, or some combination thereof. The user terminal 202 is equipped with the user input devices 214 that communicate search criteria to the system 200 as well as navigate through the user terminal menu system and control the user terminal's other various functions. The user local storage 212 is used to store and archive content onto one or more removable and/or non-removable storage devices or media for later access. Removable storage media may include, but is not limited to, magnetic tape, magnetic disks, optical disks and modules, and electronic memory cartridges. The user local database 213 is the repository of all relevant information about a user's profile and account. This information includes, but is not limited to, user name, password, personal information that the user has authorized for storage, billing information, other users allowed access to the account, past search criteria, past content download information, and library information about stored content. As a consumer protection, the user terminal 202 may enable the account user to view the information stored in the user local database 213 and modify certain data fields and select which data fields may be reported to a main system database within the aggregator 201. Certain fields including, but not limited to, account numbers and billing information may not be allowed this level of user access.

[0051] The user terminal processor 227 may include a central processing unit and all associated hardware, software, and firmware modules to perform all operations within the user terminal 202. These operations include, but are not limited to, managing communications with the aggregator 201 and other networked devices, processing search and download requests, displaying search and download results, managing and controlling communications with the user local storage 212 and the user local database 213, responding to user interaction with presentation of graphical user interface (GUI) menus, playing out selected programming content using various audio and video output devices, implementing the user's part of the digital rights management schema, and administering the user's account and billing. The tuner 228 and the demultiplexer 231 are used to select an audio/video channel for playout from the channels available on the cable television system 216.

[0052] In an embodiment, the user terminal 202 may incorporate selected features of the aggregator 201. For example, the user terminal 202 may include a small metadata crawler, an aggregator, and program content and program metadata storage.

[0053] The user terminal 202 communicates with the aggregator 201 using the wide area distribution system 203. Within the aggregator 201, the communications server 250 acts as the interface point to the wide area distribution system 203 for the purpose of managing primary communications to system users. The communications server 250 routes incoming user requests and associated user information to the request and results processing server 300, routes search results and content downloads through the wide area distribution system 203 to end users, and may route billing information from a customer billing server to the end users. The request and results processing server 300 performs the basic processing and routing related to user search requests, content download requests, administrative information requests, search results, related content suggestions, and programming notification.

[0054] The content acquisition server 400, as shown in detail in FIG. 5 receives content download requests from the request and results processing server 300. The content acquisition server 400 includes a content request processor and router 401 and a remote content download processor 402. A download request is made by the user for content selections from sources including, but not limited to, earlier programming searches, recurring scheduled events, an electronic program guide, lists of electronic books and computer software, advertisements, promotions, and affiliated Internet websites. In general, a download request will be accompanied by data indicating the source of the content and whether the content is local or remote. Requests for content that is available only from a remote site may be routed to the network gateway 251 and then to the appropriate remote source. Remote sources include, but are not limited to, the remote content storage 258 and the remote streaming sources 259 shown in FIG. 4. The remote content server 204 verifies the request and returns the content through the wide area network/Internet 205. If the programming content is in the appropriate format and is authorized for direct delivery to the user, the content may be routed through the wide area network/Internet 205 and the wide area distribution system 203 directly to the user terminal 202. During this transfer, administrative data pertaining to billing and verification of delivery may be sent back to the aggregator 201.

[0055] Programming that is not formatted and authorized for direct delivery from a remote source may be transmitted through the network gateway 251 to the remote content download processor 402 contained within the content acquisition server 400. The remote content download processor 402 buffers or caches the programming content while managing the download connection to the remote source. The programming content is then routed to the decoder and content formatter 252, which may be capable of decoding various industry formats and compression schemes and may reformat and encode the decoded data and associated metadata into one or more preferred content formats for delivery and for local storage. The processed programming content, along with user identification and routing data, is then routed to the content delivery server 450, which then manages the delivery of the content through the communications server 250 and wide area distribution system 203 to the user terminal 202. The content delivery server 450 is shown in detail in FIG. 6. The content acquisition server 400 is described in more detail in co-pending patent application entitled “Video and Digital Multimedia Aggregator,” filed on Aug. 3, 2001, which is incorporated by reference.

[0056] Requests for programming available from a local source are routed directly to a local content request processor 451 within the content delivery server 450. The local content request processor 451 initiates delivery of content from local sources including, but not limited to, the aggregator local storage 254 and aggregator local streaming sources 262, and routes the content to the coder and file formatter 253. In an embodiment, the local streaming sources 262 can include any of the cable television channels available at the cable television system headend 210 housing the aggregator 201. If the programming content is not already in the user-requested format and coding scheme, the coder and content formatter 253 (see FIG. 4) performs content formatting and coding of the programming for delivery to the user. The content is then forwarded to a content delivery processor 453, which manages the delivery of the content through the communications server 250 and wide area distribution system 203 to the user terminal 202.

[0057] If the programming being delivered from a remote content server 204 will be stored to the aggregator local storage 254, the programming metadata is analyzed to determine if the content is in an acceptable format for local storage. The aggregator 201 may be configured to store content in one or more specific formats that will balance the highest quality of programming content to be delivered to the users versus available storage space. In an embodiment, one or more of the selected storage formats will make the most efficient use of the aggregator local storage 254 resources as well as support high-speed delivery to system users. If the content acquisition server 400 determines that the content does not meet the format requirements for local storage, the content stream and/or contents are routed to the decoder and content formatter 252 (see FIG. 4). The decoder and content formatter 252 decodes the incoming data and may extract digital and/or analog data representing the content and any metadata associated with the content. The decoder and content formatter 252 then reformats the content into the required formats and coding schemes for local storage. The properly formatted programming content may then be stored to the aggregator local storage 254.

[0058] Returning to FIG. 5, the remote content download processor 402 analyzes the programming content's metadata and determines if the content is in the appropriate format and coding scheme for delivery to the user. The format and coding scheme for delivery may be different from that used for local storage in order to accommodate particular parameters of a user's download request. The user may request specific formats and coding schemes due to considerations including, but not limited to, the method of playback, the type of programming, whether the user intends to store the content, or the type and bandwidth of the connection the user terminal 202 has to the aggregator 201 or the remote content server 204. If the content does not meet the format criteria, the content stream and/or contents are routed to the decoder and content formatter 252 along with the required formatting and coding parameters. The decoder and content formatter 252 decodes the incoming data and may extract digital and/or analog data representing the content and its metadata. The decoder and content formatter 252 then reformats the content into the required formats and coding schemes for delivery to the user.

[0059] If the programming download request is to be fulfilled by the local aggregator 201, the local content request processor 451 of the content delivery server 450 (see FIG. 6) analyzes the programming content's metadata and determines if the content is in the appropriate format and coding scheme for delivery to the user. If the format and coding scheme is not correct, the local content request processor 451 retrieves the programming content from the aggregator local storage 254 or the appropriate local streaming sources 262 and routes the programming to the coder and content formatter 253 along with the required formatting and coding parameters. The coder and content formatter 253 then decodes the incoming data and may extract digital and/or analog data representing the content and its metadata. The coder and content formatter 253 then reformats the content into the required formats and coding schemes for delivery to the user. The correctly coded and formatted programming content file(s) and/or streams, whether directly from local sources or from the coder and content formatter 253, and routes this content to the content delivery server 450.

[0060] The coder and content formatter 253 may employ digital compression techniques to increase existing transmission capacity. A number of digital compression algorithms currently exist or may be developed in the future that can achieve the resultant increase in capacity and improved signal quality desired for the system 200. For television and video content, algorithms generally use one or more of three basic digital compression techniques: (1) within-frame (intraframe) compression, (2) frame-to-frame (interframe) compression, and (3) within carrier compression or a combination of two or more techniques. The coder and content formatter 253 of the content search, packaging, and delivery system 200 will be able to use any one or combination of two or more of these techniques in addition to being able to have its capabilities modularly expanded to include any emerging and future techniques that are determined to be desirable. In an embodiment, although a single digital compression standard may be used for the delivery system 200, different levels of compression and/or digital coding can be utilized when delivering programs to a user requesting a particular program. For example, if the program is sent out using an HDTV format, then less compression is used as opposed to sending out the program using a standard video format. The reason is that the HDTV format requires more digital data per portion of content. Since the HDTV format requires more digital data, the HDTV format will also require more bandwidth to transmit, possibly increasing the cost of delivery. Therefore, the quality of the video delivered may be a variable in the fees charged to users of the system 200.

[0061] The decoder and content formatter 252 and the coder and content formatter 253 may be similar or identical in functionality, software configuration, and/or hardware configuration. In an embodiment, the decoder and content formatter 252 and the coder and content formatter 253 may be embodied by a single subsystem represented by a codec and content formatter 270, which is shown in detail in FIG. 7. In an alternate embodiment, the decoder and content formatter 252 and the coder and content formatter 253 may each comprise one or more separate codec and content formatters 270. The codec and content formatter 270 includes a codec and content formatter processor 271 that may communicate with and control other components of the codec and content formatter 270. In particular, the formatter processor 271 communicates with one or more content routers 272, one or more formatting codecs 274, and one or more auxiliary services processors 276. The formatter processor 271 communicates with one or more content routers 272 in order to transmit routing connection parameters and paths for the transfer of content data between various source and target subsystems. The formatter processor 271 communicates with one or more formatting codecs 274 to identify and manage available codec processing resources, configure codec hardware and/or software parameters for decoding, formatting, and coding operations, and to transmit routing information and coding and formatting parameters necessary for the formatting codecs 274 to decode, format, and code content from various sources. The formatter processor 271 also communicates with one or more auxiliary services processors 276 to allocate resources and transmit parameters associated with requests for auxiliary services processing. Auxiliary services may include closed captioning, descriptive video services, advertising, interactive services, and other data services. The formatter processor 271 may also communicate with other components of the content search, packaging, and delivery system 200 including the content delivery server 450, the content acquisition server 400, the aggregator local storage 254, and the local streaming sources 262 (see FIG. 4). Communications with these other systems includes receiving coding and formatting requests, transmitting content routing data, and managing content flow between various content sources and one or more components of the codec and content formatter 270.

[0062] When the remote content download processor 402 within the content acquisition server 400 and/or the content delivery processor 453 within the content delivery server 450 determine that a particular content entity needs to be decoded, formatted, reformatted, and/or coded, the respective processors (402, 453) transmit a coding and formatting request to the codec and content formatter 270. Coding and formatting requests convey data to the codec and content formatter 270 characterizing the input source content and the desired output target content. Data conveyed about the source and target content may include parameters such as physical and/or logical addresses, coding and compression parameters, format descriptions, content sizes and/or lengths, description or location of auxiliary services combined with the source content, description or location of auxiliary services to be combined with the target content, and other metadata elements that may be required for coding, formatting, and routing. The codec and content formatter 270 may then use these parameters to identify available resources, establish content routing paths, configure formatting codecs 274, configure auxiliary services processors 276, and perform the required coding and formatting operations to fulfill the coding and formatting request. In an embodiment, a coding and formatting request may identify one or more target content coding and formatting schemes, as in the case where one type of formatting may be applied to the target content for storage and another type of formatting may be applied to the target content for delivery to system users, which may require the codec and content formatter 270 to configure and control one or more of the formatting codecs 274 and auxiliary services processors 276.

[0063] The codec and content formatter 270 may use one or more of the content routers 272 to dynamically establish logical and/or physical circuit connections for transmission of content data between the various subsystems shown in FIG. 7. A content router 272 may comprise one or more hardware systems and one or more software modules that may operate under the direction of the formatter processor 271 to establish and manage the interconnections between subsystems required to transfer content in support of one or more coding and formatting operations. In FIG. 7, several of the interconnections between subsystems are shown as two connection lines and a series of dots between the two lines. This convention is used to indicate that at any time there may exist one or more connections or circuits between one or more of the connected subsystems and that between two connected subsystems the number of connections may vary. Each collection of connections or circuits may comprise one or more logical and/or one or more physical connections between software and/or hardware modules. Physical routing configurations, such as may be required for the transfer of analog content, may be achieved using one or more industry standard and/or custom components such as circuit switching devices, matrix switches, distribution amplifiers, signal splitters, input and output ports, and patch bays. Logical routing, such as may be required for the transfer of digital content, may be achieved using one or more digital packet transport protocols that may operate on one or more local area networks supported by the system administrator 500 as part of the content search, packaging, and delivery system 200. In an embodiment, the functionality of the content routers 272 may be implemented by one or more subsystems and may be functionally and/or physically external to the codec and content formatter 270 and may be a subsystem (not shown in FIG. 4) of the aggregator 201. In another embodiment, the functionality of the content routers 272 may be implemented by one or more subsystems and may be functionally and/or physically external to the aggregator 201 so that the aggregator 201 and its subsystems may connect to the content routers 272, or equivalent systems, through external interfaces (not shown in FIG. 4). The number of connections and the logical and physical routing paths of such connections both within the codec and file formatter 270 and external to the codec and file formatter 270 may be changed as required to accommodate a smaller or larger number of content pathways and to accommodate various source and destination combinations required at any given time. These pathways may be created or modified on a demand basis by the formatter processor 271 and may be modified prior to a coding and formatting operation and/or during a coding and formatting operation as may be required.

[0064] Content that is stored locally by the system 200 and/or acquired remotely may exist in one or more of numerous standardized and/or proprietary formats with various coding and compression schemes applied and may need to be decoded, reformatted, and/or recoded to satisfy local storage requirements and requirements of delivery to one or more system users. The content decoding, formatting, and/or coding functions may be performed by one or more of the formatting codecs 274 as shown in FIG. 7. Decoding of a content file or stream may comprise several processing operations that may include performing a packet or frame alignment, reading and caching content metadata, reading embedded time code information, reading and caching auxiliary services data, and recovering and caching content payload data in digital form and/or analog baseband form. Formatting or reformatting of content payload data may comprise several processing operations that may include analog-to-digital conversion, digital-to-analog conversion, packetizing or framing digital data representing content into one or more standard and/or proprietary formats, translating digital data representing content from one or more standard and/or proprietary formats into one or more other standard and/or proprietary formats, modifications to existing auxiliary services that may be packaged with the content, addition of auxiliary services such as closed captioning and/or advertising, frame rate translation of video content, video aspect ratio format conversion (e.g., 4×3 television or 16×9 movies), and any required audio and/or video processing. Coding of a content file or stream may comprise several processing operations that may include applying varying levels of compression and various schemes of error correction coding to one or more content files and/or signals.

[0065] The codec and content formatter 270 may contain one or more banks of one or more formatting codecs 274 that may comprise one or more separate software and/or hardware modules. The hardware and/or software modules may exist as a flexible group of resources that may be accessed on a demand basis and may be dynamically assigned and configured to accommodate virtually any content decoding, formatting, and coding operation. When the formatter processor 271 receives a coding and formatting task request from the content acquisition server 400 or the content delivery server 450, the formatter processor 271 may poll the group of formatting codec 274 resources, identify available resources required for the requested task, configure those resources to perform the requested task thereby configuring a formatting codec 274, and transmit the required decoding, formatting, and coding parameters as well as routing information to the newly configured formatting codec 274. In an embodiment, the formatter processor 271 may configure and activate one or more available hardware and/or software modules by downloading appropriate software modules to perform particular decoding, formatting, and coding tasks as required by a user content download request or other content acquisition task. The ability of the formatting codecs 274 to be software configurable may allow a high degree of configuration flexibility as well as the ability to accommodate coding and formatting schemes yet to be developed.

[0066] An example of a formatting codec 274 configuration is shown in detail in FIG. 8. The formatting codec 274 includes a master codec processing module 315 that is coupled to one or more other processing modules and that may manage the configuration and operation of one or more other processing modules within the formatting codec 274. In an embodiment, the processing modules within the formatting codec 274 may be interconnected using a bus topology that may allow dynamic interconnectivity between up to all of the processing modules and may support logical addressing of modules for communications routing. The master codec processing module 315 may be coupled to a memory management module 319 and a codec memory module 320. The memory management module 319 may dynamically assign and manage resources within the codec memory 320 for use by one or more other processing modules within the formatting codec 274.

[0067] The master codec processing module 315 is coupled to a content decoding module 316 that may manage the configuration and operation of one or more processing modules in order to perform content decoding operations. Coupled to the content decoding module 316 is a frame alignment module 321 that may use decoding parameters received from the formatter processor 271 to synchronize and align the formatting codec 274 with the incoming content data packets and/or frames. Also coupled to the content decoding module 316 is a metadata collection module 322 that may read any metadata information contained in one or more content files and content transport streams and may temporarily store this data in the codec memory 320. A time code processing module 323 is also coupled to the content decoding module 316 and may extract any time code data that may be embedded in one or more content files and/or content transport streams and may store and/or process the time code as required to support coding and formatting operations. The content decoding module 316 is further coupled to a format translation module 324, which may perform the actual translation of the incoming source content data to the desired output target content format. An example of format translation may involve reading a source video file in a proprietary digital format and translating the content data into a standardized format. The content decoding module 316 is also coupled to an auxiliary services divider module 328 and an auxiliary services routing module 329. The auxiliary services divider module 328 may read any auxiliary services that may be a part of the source content files and/or transport stream and may separate, or parse, these data services from the content data and metadata. The auxiliary services routing module 329 may then route the auxiliary services data to one or more of the codec memory 320 and/or auxiliary services processors 276.

[0068] The master codec processing module 315 is coupled to a content formatting module 317 that may perform content formatting operations and/or manage the configuration and operation of one or more processing modules in order to perform content formatting operations. Coupled to the content formatting module 317 is a format translation module 324 that may perform translation of incoming source content data into desired output target content format. Processing steps to accomplish content formatting and format translation may be optimized to minimize degradation of the content data and/or signal. The content formatting module 317 is also coupled to a digital-to-analog and analog-to-digital conversion module 325 that may perform any required conversion of audio and video content between analog and digital formats to support source content that may exist in various digital and/or analog formats and may be delivered to system users in various digital and/or analog formats. The content formatting module 317 is also coupled to an auxiliary services combiner module 330, which may combine and/or recombine any previously existing and/or newly established auxiliary services with the target content files and/or transport stream. The auxiliary services combiner 330 is in turn coupled to the auxiliary services routing module 329 in order to receive appropriate auxiliary services data. The content formatting module 317 is further coupled to an audio/video processing module 331 that may perform various processing functions on the audio and video signals of the source content to meet target content requirements. The audio/video processing functions may include adjusting audio levels, combining and/or splitting audio signals, and adjusting video luminance and chrominance.

[0069] The master codec processing module 315 may also be coupled to a content coding module 318 that may perform content coding operations and/or manage the configuration and operation of one or more processing modules in order to perform content coding operations. Coupled to the content coding module 318 is a digital compression module 326 that, as required, may apply various digital compression schemes to the target content that was not previously digitally compressed during content formatting and or translation. Different levels of digital compression may be applied to one or more target content files and/or transport streams. The different compression levels may be based on requirements such as those regarding content delivery quality and available content delivery bandwidth and/or storage capacities. An error correction coding module 327 is also coupled to the content coding module 318 and may apply one or more of various digital forward error correction coding schemes to one or more of the target content files and/or transport streams to reduce digital errors during content delivery and allow correction of data containing a certain level of errors.

[0070] The codec and content formatter 270 may contain one or more banks of auxiliary services processors 276. The auxiliary services processors may comprise one or more separate software and/or hardware modules. The auxiliary services processors 276 may process and/or generate auxiliary services to be included with target content. Such auxiliary services may include closed captioning services, descriptive video services, advertising, interactive services, and other data services. The hardware and/or software modules may exist as a flexible group of resources that may be accessed on a demand basis and that may be dynamically assigned and configured to accommodate various auxiliary services processing operations. When the codec and content formatter processor 271 receives a coding and formatting request that includes auxiliary services processing requests from the content acquisition server 400 or the content delivery server 450, the formatter processor 271 may analyze the auxiliary processing parameters contained in the coding and formatting request. The analyzed parameters are then used to determine the required configuration of one or more auxiliary services processors 276 necessary to perform the requested operations. The formatter processor 271 may then poll the group of auxiliary services processing resources, identify the available resources required for the requested task, configure those resources to perform the requested task, and transmit the required auxiliary services parameters as well as routing information to the auxiliary services processor 276. In an embodiment, the formatter processor 271 may configure and activate one or more available hardware and/or software modules by downloading appropriate software modules to perform particular auxiliary services processing tasks as required by a user content download request or other content acquisition task. The ability of the formatting codecs 274 to be software configurable allows a high degree of configuration flexibility as well as the ability to accommodate auxiliary services yet to be developed.

[0071] Auxiliary services may accompany both analog and digital content deliveries and may be embedded in one or more content files and/or transport streams. Auxiliary services accompanying video programming content may include closed captioning services for users with hearing impairments, descriptive video services for system users with visual impairments, alternative language audio service, v-chip program rating information, critical review information, device control parameters and commands, future programming schedules, advertising, advertising insert tags, other text and data services, and interactive services data such as browser pages, menus, icons, and links. Auxiliary services accompanying audio programming content may include extended metadata such as graphics files, in-depth data about the producers of the content, critical reviews, future programming schedules, device control parameters and commands, advertising, advertising insert tags, other text and data services, and interactive services data such as browser pages, menus, icons, and links. Auxiliary services processed by the formatter 270 may be pre-existing services recovered from source content and routed from one or more formatting codecs 274 to one or more auxiliary services processors 276 and/or newly established services transferred from one or more other content search, packaging, and delivery system 200 resources through the codec and content formatter processor 271 to one or more auxiliary services processors 276. In an embodiment, one or more auxiliary services processors 276 may comprise an audio closed captioning processor that may receive audio and time code data for a particular video content entity from a formatting codec 274, perform speech-recognition and other processing on the audio data, and output one or more files and/or transport streams that contain time-code-synchronized closed captioning text and data to be combined with the target content as an auxiliary service.

[0072]
FIG. 9 shows an overview of the content formatting and coding process 360 performed by the codec and content formatter 270. The codec and content formatter processor 271 (see FIG. 8) first receives a coding and formatting request from the content acquisition server 400 and/or the content delivery server 450, routine 361. The formatter processor 271 then uses routine 363 to analyze the coding and formatting request parameters contained in the coding and formatting request in order to determine the required configuration of one or more formatting codecs 274 necessary to perform the requested operations. The formatter processor 271 may then poll formatting and coding resources, routine 365, and identify available formatting codec hardware and/or software modules necessary to realize the required configuration. Routine 367 may then configure one or more formatting codecs 274 using the coding and formatting request parameters and required configuration data to access and configure one or more formatting codec 274 hardware and/or software modules. The formatter processor 271 may then use source content and target content routing address information from a decoding and formatting request to configure one or more content routers 272, routine 369. Routine 371 may then initiate and manage the transfer of source content through one or more content routers 272 from one or more source locations to one or more formatting codecs 274. Source locations may include the content acquisition server 400, the content delivery server 450, aggregator local storage 254, and local streaming sources 262. One or more formatting codecs 274 may then execute routine 373, which may make use of the hardware and/or software modules shown in detail in FIG. 8 to perform the requested decoding, formatting, and coding operations to produce the target output content. Routine 374 may then perform processing of auxiliary services as outlined in the content coding and formatting request. Auxiliary services processing starts at the formatting codec 274, which parses existing auxiliary services from the source content and routes those services to one or more auxiliary services processors 276, which perform appropriate processing on existing auxiliary services as well as establish new auxiliary services as required. The auxiliary services processors 276 may then route the appropriate auxiliary services to one or more formatting codecs 274 for combination with one or more target output content files and/or data transport streams.

[0073] Once the one or more target content files and/or transports streams have been assembled, routine 375 within the formatter processor 271 transfers the formatted target content through one or more content routers 272 to one or more target addresses residing in one or more components external to the codec and content formatter 270 which may include the content acquisition server 400, the content delivery server 450, aggregator local storage 254, and/or local streaming sources 262. The formatter processor 271 may then clear content routing connections using routine 377. Content routing connections to be cleared may include one or more physical routing connections and/or one or more logical routing connections. Physical routing connections may be cleared by returning routing circuitry to an idle or standby state and logical routing connections may be cleared by removing logical address information from appropriate memory registers within one or more content routers 272. The formatter processor 271 may then return one or more configured formatting codecs 274 and/or auxiliary services processors 276 to a standby and/or default state, routine 379, and return the one or more formatting codec 274 and or auxiliary services processor 276 hardware and/or software modules to the available pools of formatting resources and/or auxiliary services processing resources, respectively. Routine 381 within the formatter processor 271 may then log appropriate administrative data with the content acquisition server 400 and/or the content delivery server 450. Routine 383 may then stop processing.

[0074]
FIG. 10 shows the auxiliary services processing of routine 374 in more detail. If existing auxiliary services and/or newly established auxiliary services are to be combined with target output content, then routine 385 within one or more of the formatter processor 271 and/or formatting codecs 274 analyzes any auxiliary services processing request parameters that may be part of the coding and formatting request and determines the necessary configuration of one or more auxiliary services processors 276. Routine 387 within the formatter processor 271 may then poll available auxiliary processing resources and configure one or more auxiliary services processors 276 required to perform the requested processing operations. One or more formatting codecs 274 may parse, or separate, existing auxiliary services and time code data that may be combined with one or more source content files and/or transport streams, routine 389. Routine 391 may then transfer these auxiliary services as well as the source content time code data to one or more auxiliary services processors 276. Routine 391 may also transfer data from sources external to the codec and content formatter 270 that may be required to establish new auxiliary services to be combined with the target output content. These external sources may include the content acquisition server 400, the content delivery server 450, and the aggregator local storage 254. One or more auxiliary services processors 276 may then perform one or more processing and formatting operations on one or more sources of auxiliary services data in accordance with processing request parameters in order to output one or more target auxiliary services to be combined with target output content, routine 393. Routine 395, which may reside within the formatter processor 271 and/or one or more auxiliary services processors 276, may then transmit these output auxiliary services to one or more formatting codecs 274. Routine 397 may then be executed by one or more formatting codecs 274 to combine the auxiliary services with one or more target output content files and/or data streams. Processing is then continued with routine 375 as shown in FIG. 9 and as described above.

	Number	Date	Country
Parent	09920723	Aug 2001	US
Child	09973067	Oct 2001	US
Parent	09920615	Aug 2001	US
Child	09973067	Oct 2001	US

Video and digital multimedia aggregator content coding and formatting

Information

Publication Number

Date Filed

Date Published

Inventors

CPC

US Classifications

International Classifications

Abstract

Description

Claims

RELATED APPLICATIONS

Continuation in Parts (2)