Verification of semantic constraints in multimedia data and in its announcement, signaling and interchange

Abstract

A collection of well-formed, but possibly semantically invalid, binary encoded multimedia data components or packages as well as the binary encoded announcement, signaling, and interchange protocols used in their transmission are converted into one or more well-formed extensible markup language (XML) files. Such XML files may then be validated according to one or more pre-defined XML schemas, or similar schema languages, in order to verify that the data and protocol structures and substructures adhere to prior defined semantic constraints.

Description

FIELD OF THE INVENTION

The present invention relates to systems and methods for delivery of enhanced television content to subscribers, and in particular relates to the conversion of binary encoded multimedia data components into extensible markup language (XML) files.

BACKGROUND OF THE INVENTION

Interactive television (iTV) is currently available in varying forms and there is perhaps no single, universally accepted definition for this term. Most, however, would agree that iTV systems combine, in some form or other, the passive viewing experience of traditional TV viewing with the interactive nature of external data and content sources such as the World Wide Web. Thus, in different implementations, iTV might allow viewers (perhaps more appropriately termed subscribers) to review local news headlines or weather forecasts while watching their favorite television shows, to chat with other viewers about the latest plot twists, and/or to search for an obtain enhanced content regarding a currently televised program. Many cable and satellite TV providers have deployed iTV systems that allow subscribers to order products seen in commercials or video on demand presentations with the click of a remote control button. And certainly all iTV systems have a version of an interactive program guide that allows subscribers to learn about and select desired programming.

A specific service offered within the iTV landscape is enhanced TV (ETV) broadcasting that provides interactive content linked to video programming. A user can request information on products within ads, access additional program information such as sports statistics, or otherwise interact with a television broadcast such as a game or reality based show. This functionality is achieved through the execution and transmission of an ETV application and corresponding data sent in the broadcast channel or other mechanism along with the video broadcast.

These and other iTV services are deployed as a set of applications over the service provider's network. Such applications are received at the subscriber's premises via a set-top box or similar device, where they are interpreted (compiled), executed and the corresponding data is displayed on a television.

Many applications may be presented to end users via a common user interface and order/transaction entry and tracking system. However, each application has specific integration, management and distribution issues that arise depending on the environment that the network operators choose to deploy and significant problems are experienced by application providers and distributors in deploying and managing their ETV applications as a result. Stated differently, application providers and distributors face problems in developing and managing their ETV applications because there exists a proliferation of technologies and standards for the delivery of ETV applications and different network operators have chosen to deploy different combinations of these technologies. For example, ETV application providers must cope with networks that have been cobbled together with different technologies such as:

• • 1. Head-end technology from different providers.
  • 2. Set-top boxes from different manufacturers.
  • 3. Various combinations of network topologies.
  • 4. Middleware from different providers.
  • 5. VOD servers from a variety of providers and vendors.
  • 6. Billing systems from multiple different companies.
  • 7. Conditional access systems from different vendors.
  • 8. Differing implementation standards such as HTML, XHTML, XML, Ecmascript, OCAP, MHP, ATVEF, DASE, etc.
  • 9. Various programming languages, such as Java, C, C++, etc.

Content providers and application developers must navigate through this maze of often incompatible and unique combinations of equipment, technologies and standards, and often must develop a unique and different application for each such combination that they wish to target. For their part, network operators must deploy, configure, manage and operate ETV applications on each different network configuration individually, increasing cost, complexity, and staffing needs, while reducing the number of service that can be deployed, and the quality of those services.

Another significant problem for the deployment of ETV applications in the United States is that the majority of in-service set-top boxes have relatively limited capabilities with which to store and process ETV applications. For example, many distributed applications may not be executable on various set-top boxes due to the memory and processing requirements to compile or otherwise interpret the application and data for proper display. These set-top limitations make it very difficult to support the features, functions and viewer response times for ETV applications that are required for a compelling and rich user experience.

SUMMARY OF THE INVENTION

In one embodiment, the present invention provides for converting a binary encoded multimedia data into an XML file, and validating the XML file according to a predetermined XML or other schemas. Following validation, the binary information may be subject to further processing. The binary encoded multimedia data may exist in a streaming format, such as an MPEG-2 transport stream, or an AVC broadcast stream. Validating the XML file may involve validating broadcast video elementary stream formats, validating PMT information; or validating the XML file according to one or more MPEG protocols, MHP protocols, or CableLabs OpenCable™ protocols (or other communication protocols) and/or one or more predefined standards such as those promulgated by or for DVB (including but not limited to DVB-T or DVB-H), ATSC, SCTE, ANSI, IETF, ISO, or SMPTE. Such validation may occur at a computer system remote from where the binary information is stored and/or converted to XML.

In a further embodiment, the present invention permits the conversion of a collection of well-formed, but possibly semantically invalid, binary encoded multimedia data components or packages as well as the binary encoded announcement, signaling, and interchange protocols used in their transmission into one or more well-formed XML files. Such XML files may then be validated according to one or more pre-defined XML schemas, or similar schema languages such as RELAX NG schemas, in order to verify that the data and protocol structures and substructures adhere to prior defined semantic constraints.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which:

FIG. 1 illustrates a method of converting a binary file to an XML file for validation or other purposes according to an embodiment of the present invention.

FIGS. 2A-2C, collectively, present an example of an XML schema that defines and processes widgets according to an embodiment of the present invention.

FIGS. 3A-3D, collectively, present an example of an XML schema that defines and processes tables according to an embodiment of the present invention.

FIGS. 4A-4D, collectively, present an example of an XML schema that defines and processes structures according to an embodiment of the present invention.

FIG. 5 presents an example of an XML schema that defines and processes refs according to an embodiment of the present invention.

FIGS. 6A-6E, collectively, present an example of an XML schema that defines and processes primitives according to an embodiment of the present invention.

FIGS. 7A-7B, collectively, present an example of an XML schema that defines and processes EBIFX according to an embodiment of the present invention.

FIG. 7C presents an example of an XML schema that defines and processes another EBIFX according to a further embodiment of the present invention.

FIGS. 8A-8H, collectively, present an example of an XML schema that defines and processes Actions according to yet another embodiment of the present invention.

DETAILED DESCRIPTION

The present invention permits the conversion of a collection of well-formed, but possibly semantically invalid, binary encoded multimedia data components or packages as well as the binary encoded announcement, signaling, and interchange protocols used in their transmission into one or more well-formed XML files. Such XML files may then be validated according to one or more pre-defined XML schemas, or similar schema languages such as RELAX NG schemas, in order to verify that the data and protocol structures and substructures adhere to each prior defined semantic constraint. For example, such validation could determine whether a field's value is an acceptable value according to its defined type, whether a co-occurrence constraint is satisfied, if field x is in range [x0, x1] then field y must be in range [y0, y1], etc.

The above-cited patent applications describe a system wherein iTV content is created and deployed using a particular authoring specification that provides for the presentation of content through one or more templates. This allows application definition and behavior to remain common across multiple client device types, middleware platforms, and/or iTV operating environments. Of course, many other forms of content creation and distribution exist and the present invention is not limited to the schemes discussed in the above-cited patent applications. Subsequent to distributing content (or as part of the distribution chain), and prior to execution or consumption of the content, it is useful to validate it to ensure compliance with various protocols, standards formats or other pre-defined semantic constraints.

Any standard or specification for formatting content can be expressed as an XML schema; hence, in accordance with the present invention the binary encoded multimedia data is converted to XML content governed by a pre-defined schema and then validated according to rules defining that schema. As illustrated in FIG. 1, the present invention provides a method 10 for the conversion of binary information (data) 12 into an XML format for purposes of validation before being subject to further processing. The binary encoded data may be multimedia (e.g., audio/video) data that exists in a streaming format, such as an MPEG-2 transport stream, or an AVC broadcast stream. Such data binary data may be generated or harvested from any of a number of data feeds from various content providers (e.g., via the Internet or audio/video distribution systems). In some cases, following validation the binary information may be delivered a set top box associated with a cable or satellite audio/video distribution network.

As indicated above, following the conversion 14 of the binary information the resulting XML file may be validated 16 according to one or more pre-defined XML schemas and/or XML processing tools (e.g., such as Xpath), or similar XML schema languages (e.g., such as RELAX NG). Such validation may occur at a computer system remote from where the binary information is located and/or converted to the XML file. In such cases, the XML file would be transmitted to the remote computer system for prior to the validation operations.

Validating the XML file may involve validating broadcast video elementary stream formats, validating PMT information; or validating the XML file according to one or more MPEG protocols, MHP protocols, or CableLabs OpenCable™ protocols and/or one or more standards such as those promulgated by or for DVB (including but not limited to DVB-T or DVB-H), ATSC, SCTE, ANSI, IETF, ISO, or SMPTE. Assuming the validation indicates that the binary information is correct, the binary information may be subject to further processing 18 (e.g., processing by a set top box). The validation of the XML files may be performed in order to ensure that the binary data and protocol structures adhere to pre-defined semantic constraints.

The XML schemas presented herein are examples of an expression of the ETV binary exchange format (EBIF) described in the above-cited patent applications. The following capabilities provide examples of uses to which the converted XML content from the binary encoded content can be applied:

• • 1. Allows for validation of binary encoded content subsequent to distribution (or in the distribution chain) and prior to execution.
  • 2. Provides an interchange format between BIF generators, validation tools, editing tools, etc.
  • 3. Aids with source level debugging for execution, display and debugging of BIF applications. For example, validating the XML file produces validated XML content and the validated XML content may be used as a debugging aid.
  • 4. Provides an extensibility aid for migration to later versions of BIF by migrating BIF applications in XML representations. For example, validating the XML file produces validated XML content and the validated XML content may be converted to another version of the XML schema, and the converted XML content used to create binary multimedia data compliant with another protocol version of binary encoded content. EBIF is one example of binary encoded content and the present invention is not limited thereto. The remainder of this description sets forth some specific examples of the present XML schemas for use in interchanging, decoding, and rendering ETV applications.

Widgets:

FIGS. 2A-2C, collectively, present an example of an XML schema that defines and processes widgets according to an embodiment of the present invention. A widget is a user interface construct that is typically, but not necessarily associated with some direct presentation semantic; e.g., a button widget and an input text field widget have presentation semantics, whereas a hidden field widget and a timer widget do not. A widget may be specified by a set of static properties necessary to construct an instance of the widget in a user agent (or authoring station). A widget may also be associated with a set of dynamic properties that may change value as a consequence of user events, system events, or programmatic actions applied to the widget (e.g., mutating the value of a widget under control of a logic programming construct).

Tables:

FIGS. 3A-3D, collectively, present an example of an XML schema that defines and processes tables according to an embodiment of the present invention.

Structure:

FIGS. 4A-4D, collectively, present an example of an XML schema that defines and processes structures according to an embodiment of the present invention.

Refs:

FIG. 5 presents an example of an XML schema that defines and processes refs according to an embodiment of the present invention.

Primitives:

FIGS. 6A-6E, collectively, present an example of an XML schema that defines and processes primitives according to an embodiment of the present invention.

EBIFX:

FIGS. 7A-7B, collectively, present an example of an XML schema that defines and processes EBIFX according to an embodiment of the present invention. FIG. 7C presents an example of an XML schema that defines and processes another EBIFX according to a further embodiment of the present invention.

Actions:

FIGS. 8A-8H, collectively, present an example of an XML schema that defines and processes Actions according to yet another embodiment of the present invention.

Thus, systems and method that provide for conversion of binary encoded multimedia data components into extensible markup language (XML) files have been described. In one embodiment, the present invention provides for converting a binary encoded multimedia data into an extensible markup language (XML) file, and validating the XML file according to a predetermined XML or other schema. Assuming the validation passes, the binary information may then be subject to further processing. The binary encoded multimedia data may exist in a streaming format, such as an MPEG-2 transport stream, or an AVC broadcast stream. Validating the XML file may involve validating broadcast video elementary stream formats, validating PMT information; or validating the XML file according to one or more protocols and/or standards-defined formats.

Although the present invention has been discussed with respect to several illustrated embodiments, these examples should not be read as limited the scope of the invention, which is best understood by reference to the claims, which follow.

Claims

1. A method, comprising: generating, by a computing device, first extensible markup language (XML) content based on first encoded video, the first encoded video comprising video data formatted according to a first version of a protocol or a multimedia format;validating the first XML content according to a first schema, resulting in validated first content that adheres to one or more constraints of the first version of the protocol or the multimedia format;converting the validated first content to a second schema;generating second XML content based on second encoded video; andvalidating the second XML content according to the second schema.

2. The method of claim 1, further comprising: generating multimedia data using the second schema.

3. The method of claim 1, wherein the second schema comprises an XML schema.

4. The method of claim 1, further comprising: generating, using the second schema, third encoded video compliant with a second version of the protocol or the multimedia format.

5. The method of claim 1, wherein the one or more constraints comprise a semantic constraint on a data value range for the first version of the protocol or the multimedia format.

6. The method of claim 1, wherein the one or more constraints comprise a co-occurrence constraint of the first version of the protocol or the multimedia format.

7. The method of claim 5, wherein validating the first XML content according to the first schema comprises validating that a data value of the video data is within the data value range.

8. The method of claim 6, wherein validating the first XML content according to the first schema comprises validating according to the co-occurrence constraint by at least: determining a first data value of the first encoded video, andvalidating that a second data value of the first encoded video is an acceptable value according to the first data value.

9. The method of claim 1, wherein the protocol or the multimedia format comprises an enhanced television binary exchange format (EBIF).

10. A method, comprising: generating, by a computing device, first extensible markup language (XML) content based on first encoded video, the first encoded video comprising video data that is expected to be formatted according to a first version of a protocol or a multimedia format;validating the first XML content according to one or more constraints of the first version of the protocol or the multimedia format on the video data, resulting in validated content that adheres to the one or more constraints;generating second XML content based on second encoded video; andvalidating, based on the validated content, the second XML content.

11. The method of claim 10, wherein the one or more constraints comprise at least one of a data value constraint and a co-occurrence constraint.

12. The method of claim 11, wherein validating the first XML content comprises validating that a data value of the first encoded video is an acceptable value.

13. The method of claim 11, wherein validating the first XML content comprises: determining a first data value of the first encoded video, andvalidating that a second data value of the first encoded video is an acceptable value according to the first data value.

14. The method of claim 10, wherein validating the first XML content comprises validating the first XML content according to a schema.

15. The method of claim 10, further comprising: converting the validated content to converted content; andgenerating, using the converted content, third encoded video compliant with a second version of the protocol or the multimedia format.

16. The method of claim 10, wherein the protocol or the multimedia format comprises an enhanced television binary exchange format (EBIF).

17. A method, comprising: generating, by a computing device, first extensible markup language (XML) content based on first binary encoded multimedia data, the first binary encoded multimedia data comprising multimedia data that is to be validated according to a first version of a protocol or a multimedia format;validating the first XML content according to one or more constraints of the first version of the protocol or the multimedia format on the multimedia data, resulting in validated content that adheres to the one or more constraints;generating second XML content based on second binary encoded multimedia data; andvalidating, based on the validated content, the second XML content.

18. The method of claim 17, further comprising: converting the validated content to converted content; andgenerating, using the converted content, third binary encoded multimedia data compliant with a second version of the protocol or the multimedia format.

19. The method of claim 17, wherein the one or more constraints comprise a data value constraint of the first version of the protocol or the multimedia format and a co-occurrence constraint of the first version of the protocol or the multimedia format.

20. The method of claim 17, wherein the protocol or the multimedia format comprises an enhanced television binary exchange format (EBIF).