A portion of the disclosure of this document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever. The following notice applies to the software, data, and/or screenshots that may be illustrated below and in the drawings that form a part of this document: Copyright© 2007, Adobe Systems Incorporated. All Rights Reserved.
The present application relates generally to the technical field of algorithms and programming and, in one specific example, to the display of information relating to digital content.
Digital content may have information associated with it in the form of, fir example, close-captioned text, subtitles, or other suitable text appearing with the digital content on a display. This information may be a textual representation of audio portion of the digital content. This display may be a television, computer monitor, or other suitable display.
Some embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings in which:
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of an example embodiment of the present invention. It may be evident, however, to one skilled in the art that the present invention will be practiced without these specific details.
In some example embodiments, a system and method is illustrated for using an input device to retrieve a description (e.g., textual, image, video or audio) relating to digital content. A text-based description may for example include not only text in the form of American Standard Code for Information Interchange (ASC-II) or Unicode based character(s), but may also include images, audio/video digital content, web links and other multi-media supported data. Further, this text-based description may be displayed on a screen (e.g., as part of a popup or persistent screen element) in conjunction with the playing of a portion of digital content. This portion of digital content may be part of a larger portion of digital content. For example, the portion of digital content may be a scene in a movie, where the movie is the larger portion of digital content.
Some example embodiments may include the display of this text-based description within a sub frame that is part of a larger frame. A frame or sub frame may be bounded viewing area in a display, in which the text-based description may be shown. In one example embodiment, a popup screen may be a sub frame that appears within another, larger frame. Larger may mean taking up more of a viewing area than a sub frame. The technologies used to create the popup screen are described below. Further, this popup screen may be part of a stand alone application media player application or a web based application such as a web-browser.
In other example embodiments, the text-based description alone may be displayed on the screen. An input device may be a mouse, light pen, keyboard, or other suitable input device. A text-based description may be text initially formatted as an XML file, a character delimited flat file, or some other suitably formatted file. Digital content may include video content or audio/video content formatted using, for example, the Moving Picture Experts Group (MPEG) codec, the H.261 codec, the H.263 codec, or some other suitable codec. Additionally, video content or audio/video content may include a Joint Photographic Experts Group (JPEG) formatted image, a Portable Network Graphics (PNG) formatted image, Graphics Interchange Format (GIF) formatted image, or some other suitably formatted image. Additionally, a FLASH™ formatted file may be utilized to format the audio or audio/video digital content.
Some example embodiments may include the use of a media-player application UI to display the text-based description as part of a display or UI element (e.g., a popup screen element or a persistent screen element). This media-player application may be, for example, a FLASH™ media-player, a WINAMP™ media-player, a WINDOWS™ media-player, or some other suitable media-player application. A popup screen (e.g., a sub frame within a larger frame in a display) may be generated to display the text-based description. This popup screen may be generated using, for example, Asynchronous JavaScript and XML (AJAX) technology, Dynamic Hyper Text Markup Language (DHTML) technology, Hyper Text Markup Language (HTML), XML, or some other suitable technology to display text on a screen. This popup screen may be a screen object or widget. This popup screen may appear in close proximity to, or be attached to, a timeline toolbar that exists as part of a media-player application. In some example embodiments, a timeline toolbar of a video is a linear representation of the progress of a video from the first to the last video frame. A video frame may be a portion of video based digital content understood in terms Frames Per Second (FPS). Further, each frame may be referenced via temporal reference data (e.g., a temporal value or collection of temporal reference values). In some example cases, a temporal reference value is a field within an MPEG formatted data packet. In a UI for the media-player application, the timeline toolbar may be represented as a scrollbar that can receive mouse inputs. In some example embodiments, timeline tool bar may be some other type of screen object or widget used to display the time and/or sequence information for digital content. A user can move a slider (e.g., a time position) to jump to any frame in the video, using an input device.
In some example embodiments, when a mouse pointer is positioned over any point on the timeline toolbar of a streaming video, the video player may fetch a subtitle (e.g., a text-based description) from a subtitle file description of various time points in the video corresponding to that point, and display the subtitle in a tooltip (e.g., a popup screen). In some example embodiments, subtitles are textual versions of the dialogue in films, television programmes and other suitable digital content. Subtitles may, for example, be either a form of written translation of a spoken dialogue in a foreign language or a written rendering in the same language of dialogue taking place in audio/video content. Further, in some example embodiments, a media-player application playing the digital content (e.g., video) is able to receive events and handle them. These events may include a mouse-over event, a mouse down event (e.g., employed during the execution of a PLAY button), or other types of suitable events.
Some example embodiments may include a text-based description as a subtitle file. A user, in some example embodiments, may position UI pointer or selector using an input device, such as a mouse, and associated pointing device, on a point in the timeline toolbar of a video. A signal may be generated in the form of a mouse-over command to select the point in the timeline toolbar. This signal from the input device may then be processed such that a method may be executed to determine if there is a text-based description in the subtitle file corresponding to the selected point. This selected point may, in some example embodiments, may be a temporal reference value, or some other suitable value.
This media streaming session may be set up between the one or more devices 102, and the web server 110, a media server 111, or a streaming server (not pictured). In other example embodiments, this session will be set up between the one or more devices 102, and, for example, a media server 111 or, for example, a streaming server (not pictured), or even the web server 110. Shown is the media server 111 operatively connected to the web server 110. Further, connected to the media server 111, is a text-based content description database 112. A database server (not pictured) may serve or act to mediate the relationship between the media server 111, and the text-based content description database 112. The media server 111 may query the text-based content description database 112 so as to retrieve a text-based description 113. This text-based description 113 may be transmitted back across the network 108 to the one or more devices 102. Once received by the one or more devices 102, the text-based description 113 may be displayed in, for example, the media-player application UI 107. This text-based description 113, and the data contained therein, may be displayed within a frame for the media-player application UI 107. Further, a new text-based description 113 may be requested from the media server 111 every time the user 101 executes, for example, a mouse-over operation. As is more fully described below, this text-based description 113 may be an XML file, a flat file, or some other suitably formatted file.
In some example embodiments, upon successfully completing a session setup, the web server 203 may transmit a text request to the database server 204, wherein the database server 204 will then retrieve the text referenced in the text request from the text-based content description database 205. A text request may be, for example, a data packet, or series of data packets, formatted using HTTP, and various function associated therewith. HTTP may, in some cases, be used with other languages and protocols such as SQL, or MDX so as to retrieve the data outlined in the text request. In response to this request, a text-based description 206 may be provided to the database server 204. The database server 204 may then provide this text-based description 206 to the web server 203, wherein the web server 203 may format this text-based description 206 so as to generate the text-based description 207. This formatting may take a form of, for example, utilizing HTTP to transmit the text-based description 207 back across the network 108 to the media-player application UI 107. A new text-based description 207 may be requested from the web server 203 every time the user 201 performs, for example, a mouse-over operation. Once received by the media-player application UI 107, the text-based description 207 may be displayed to the user 201. As is more fully described below, the text-based description 207 may be an XML file, a flat file, or some other suitably formatted file.
In some cases, a file containing the complete text-based description (e.g., a complete file) for a portion of digital content may be retrieved via a simple mouse-over operation from the media screen 111 or web server 203. Once retrieved, this complete file may be stored into a native data store residing on the one or more devices 102. Portions of this complete file may then be retrieved via a mouse over operation on an as needed basis, and displayed in the popup screen 504. As shown in
In some example embodiments, the popup screen 304, 404, 504, and 604 may be generated using technologies that include AJAX, DHTML, HTML, XML, or some other suitable technology. In some example cases, the popup screen 304 and 404 are generated as part of a stand-alone application written using Java, C#, C++, or some other suitable programming language.
In some example embodiments, a computer system is illustrated as having a UI 107, shown as part of a display, and a selection device. Further, a method of providing and selecting from a first and second frames (e.g., a primary frame and sub frame) in the display is also illustrated, the method comprising executing an operation to retrieve at least one time point related to a portion of digital content. Additionally, an operation is executed so as to display the at least one time point within the frame. An additional operation is executed to select the at least one time point from the frame using the selection device. An operation is executed to display a text-based description as part of the sub frame within the frame, the text-based description associated with an identifier value related to the at least one time point related to the portion of digital content. This method may also include the identifier value being a temporal reference value. Moreover, the method may include the sub frame created as a popup screen generated by a mouse-over operation. Also, the method may include the text-based description including at least one of text (see e.g.,
As previously referenced, in addition to the field 703 containing a text-based description, a field 704 is shown that contains director's comments. Here, comments are provided by the director of the movie, “Jaws,” namely Steven Spielberg. The type of fields utilized within the text-based description 113 may be limited only by the needs of a particular user such as the user 101.
In some example embodiments, the text-based description 113 may be an XML file, character delimited flat file, or some other suitably formatted file that contains all of the text associated with a particular portion of digital content. This text, as alluded to above, may be the dialog for a portion of digital content, subtitles, or other associated text. Where all the text associated with a particular portion of digital content is down loaded for playing, the media-player application may receive the associated text corresponding to the portion of digital content on an as needed basis. For example, if a user 101 requests information for a particular scene in a portion of digital content via a mouse-over operation, the media-player application may retrieve the requested text from a downloaded text-based description 113 residing on one or more of the devices 102. This in contrast to the one or more devices 102 having to request the text-base description 113 from another computer system every time a mouse-over operation is performed.
In some example embodiments, a temporal reference value 1005 is provided to a media-player application 1001. Upon receiving the temporal reference value 1005, the media-player application 1001 may generate a text request 1006. This text request 1006 may be sent as a part of a session setup such as session setup 109, or may be sent as a simple request for a text-based description, and the data associated therewith. In some example embodiments, a session setup may be a TCP/IP based session, wherein an IP connection results as part of the session setup. Some example cases may utilize UDP/IP in lieu of TCP/IP, such that no session setup occurs. A simple request may be formatted using, for example, the previously referenced HTTP or RTSP. This text request 1006 may, in some example embodiments, be received by a parser 1007 where this parser 1007 may act to parse the text request. Once parsed, a text description 1008 may be generated and sent to a query engine 1009. Once this text description 1008 is received by the query engine 1009, some type of query language such as SQL, or MDX language may be utilized to query the text-based content description database 112. Further, the text description 1008 itself may be formatted in SQL or MDX.
Once a query is provided to the text-based content description database 112, a result may be selected from the text-based content description database 112 and provided to the query engine 1009. The result may then be, for example, transmitted as a result 1010 to a transmit engine 1011, where this transmit engine 1011 which may generate a text-based description 1012. This text-based description 1012 may be considered as to be similar to the text-based description 113, or the text-based description 207. This text-based description 1012 is then provided to the one or more devices 102 for display within the media-player application UI 107 as part of a popup screen. The text-based description 1012 may encompass the entire text file for a portion of digital content that may be downloaded prior to the initiation of the streaming of digital content. The text-based description 1012 may encompass only a portion of the text corresponding to a portion of digital content. Further, images (e.g., JPEG, GIF etc.) or video (e.g., MPEG etc.) may be transmitted as part of the text-based description 1012. This text-based description may be formatted using HTTP/RTSP in combination with other protocols such as TCP/IP, or UDP/IP.
In some example embodiments, the plug-in 1002 may be a computer program that interacts with a host application (e.g., a media-player application, or a web browser), and extends the functionality of this host application. As used herein, the plug-in 1002 may also be a stand alone application that does not extend existing host application functionality, and that does not run within the content of a host program. A stand alone application may be written using C++, Java, C#, or some other suitable programming language.
Some example embodiments may include the execution of an operation 1202 to generate a popup screen. This popup screen may be associated with a time point where a mouse-over occurs. Through executing operation 1003, the text-based description 1212, and the text contained therein, may be displayed within the popup screen. As illustrated elsewhere, the text contained within this text-based description 1012 may include text relating to a particular portion of digital content.
In some example embodiments, operations 1304 through 1306 may be executed on the one or more devices 102. An operation 1304 may be executed so as to receive a signal from an input device, the signal identifying a time point associated with a portion of digital content. Additionally, an operation 1305 may be executed to retrieve an identifier value based upon the signal, the identifier value associated with the portion of digital content. Operation 1306 may be executed to generate a text request based upon the identifier value, the text request containing the identifier value. In some example embodiments, the signal results from an execution of a mouse-over operation. Some example embodiments may include, the identifier value being generated by a plug-in associated with a media-player application. Further, the retrieving of the identifier value may be performed by looking up the identifier value in a data store. Additionally, the generating of the text request may include a media-player application transmitting the text request as part of a setup request.
With regard to the first stream of the dual-stream flow chart, in some example embodiments, an operation 1401 is executed that receives a signal from an input device wherein this signal may be, for example, the previously referenced signal from input device 1004. This signal may be generated by virtue of a mouse-over operation being executed, or may be generated by some other action undertaken through the use of an input device. An operation 1402 may be executed, in some example embodiments, which may retrieve and transmit a current temporal reference value(s) based upon the receiving of the signal from the input device 1004. Once this temporal reference value(s) is retrieved, a player engine 1403 may be executed. Functionalities associated with this player engine 1403 will be more fully described below. An operation 1404 may also be executed, in some example embodiments, such that a formatted text request may be transmitted as a text request 1006. This text request 1006 may be received through the execution of an operation 1405. Further, an operation 1406 may be executed that parses the text request 1006 to retrieve the temporal reference value(s) contained therein. An operation 1407 may then be executed that then generates a database query (e.g., using SQL, or MDX) to retrieve data from the text-based content description database 112. An operation 1408 may be executed that formats the retrieved data and transmits it as a text-based description such as text-based description 1012, 113, or 107. This text-based description 1012 may then be received through the execution of operation 1409. An operation 1410 may then be executed that parses the text-based description 1012 and then displays the portions of the parsed text-based description 1012 within one or more of the previously described popup screens 304 or 404. The various operations, 1401 through 1404, and 1409 through 1410 may reside as a part of the one or more devices 102. Additionally, in some example embodiments, the various operations 1405 through 1408 and the associated text-based content description database 112 may reside as a part of, for example, the media server 111.
Some embodiments may include the various databases (e.g., 112, 1103) being relational databases, or, in some cases, OLAP-based databases. In the case of relational databases, various tables of data are created and data is inserted into and/or selected from these tables using SQL or some other database-query language known in the art. In the case of OLAP databases, one or more multidimensional cubes or hyper cubes, containing multidimensional data from which data is selected from or inserted into using MDX, may be implemented. In the case of a database using tables and SQL, a database application such as, for example, MYSQL™, MICROSOFT SQL SERVER™, ORACLE8I™, 10G™, or some other suitable database application may be used to manage the data. In this, the case of a database using cubes and MDX, a database using Multidimensional On Line Analytic Processing (MOLAP), Relational On Line Analytic Processing (ROLAP), Hybrid Online Analytic Processing (HOLAP), or some other suitable database application may be used to manage the data. The tables or cubes made up of tables, in the case of, for example, ROLAP, are organized into an RDS or Object Relational Data Schema (ORDS), as is known in the art. These schemas may be normalized using certain normalization algorithms so as to avoid abnormalities such as non-additive joins and other problems. Additionally, these normalization algorithms may include Boyce-Codd Normal Form or some other normalization, or optimization algorithm known in the art.
Some example embodiments may include the above-illustrated operations being written as one or more software components. These components, and the functionality associated with each, may be used by client, server, or peer computer systems. These various components can be implemented into the system on an as-needed basis. These components may be written in an object-oriented computer language such that a component oriented or object-oriented programming technique can be implemented using a Visual Component Library (VCL), Component Library for Cross Platform (CLX), Java Beans (JB), Enterprise Java Beans (EJB), Component Object Model (COM), or Distributed Component Object Model (DCOM)), or other suitable technique. These components are linked to other components via various Application Programming Interfaces (APIs) and then compiled into one complete server and/or client application. The method for using components in the building of client and server applications is well known in the art. Further, these components may be linked together via various distributed programming protocols as distributed computing components.
Some example embodiments may include remote procedure calls being used to implement one or more of the above-illustrated components across a distributed programming environment. For example, a logic level may reside on a first computer system that is remotely located remotely from a second computer system containing an interface level (e.g., a GUI). These first and second computer systems can be configured in a server-client, peer-to-peer, or some other configuration. The various levels can be written using the above-illustrated component design principles and can be written in the same programming language or in different programming languages. Various protocols may be implemented to enable these various levels and the components contained therein to communicate regardless of the programming language used to write these components. For example, an operation written in C++ using Common Object Request Broker Architecture (CORBA) or Simple Object Access Protocol (SOAP) can communicate with another remote module written in Java™. Suitable protocols include SOAP, CORBA, and other protocols well-known in the art.
Some embodiments may utilize the OSI model or TCP/IP protocol stack model for defining the protocols used by a network to transmit data. In applying these models, a system of data transmission between a server and client, or between peer computer systems, is illustrated as a series of roughly five layers comprising: an application layer, a transport layer, a network layer, a data link layer, and a physical layer. In the case of software having a three tier architecture, the various tiers (e.g., the interface, logic, and storage tiers) reside on the application layer of the TCP/IP protocol stack. In an example implementation using the TCP/IP protocol stack model, data from an application residing at the application layer is loaded into the data load field of a TCP segment residing at the transport layer. This TCP segment also contains port information for a recipient software application residing remotely. This TCP segment is loaded into the data load field of an IP datagram residing at the network layer. Next, this IP datagram is loaded into a frame residing at the data link layer. This frame is then encoded at the physical layer, and the data transmitted over a network such as an internet, Local Area Network (LAN), Wide Area Network (WAN), or some other suitable network. In some cases, internet refers to a network of networks. These networks may use a variety of protocols for the exchange of data, including the aforementioned TCP/IP, and additionally ATM, SNA, SDI, or some other suitable protocol. These networks may be organized within a variety of topologies (e.g., a star topology) or structures.
The example computer system 1900 includes a processor 1902 (e.g., a CPU, a Graphics Processing Unit (GPU) or both), a main memory 1901, and a static memory 1906, which communicate with each other via a bus 1908. The computer system 1900 may further include a video display unit 1910 (e.g., a Liquid Crystal Display (LCD) or a Cathode Ray Tube (CRT)). The computer system 1900 also includes an alphanumeric input device 1917 (e.g., a keyboard), a UI cursor controller 1911 (e.g., a mouse), a drive unit 1916, a signal generation device 1915 (e.g., a speaker) and a network interface device (e.g., a transmitter) 1920.
The disk drive unit 1916 includes a machine-readable medium 1922 on which is stored one or more sets of instructions and data structures (e.g., software) 1921 embodying or used by any one or more of the methodologies or functions illustrated herein. The software instructions 1921 may also reside, completely or at least partially, within the main memory 1901 and/or within the processor 1902 during execution thereof by the computer system 1900, the main memory 1901 and the processor 1902 also constituting machine-readable media.
The instructions 1921 may further be transmitted or received over a network 1926 via the network interface device 1920 using any one of a number of well-known transfer protocols (e.g., HTTP, Session Initiation Protocol (SIP)).
The term. “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term “machine-readable medium” shall also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies illustrated herein. The term “machine-readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic media, and carrier wave signals.
In some example embodiments, a system and method is illustrated that allows for a text-based description to be provided in a sub frame in a display. This display may be part of a media-player application. The text-based description may include text relating to digital content and may include text relating to subtitles, director's comments, actor's comments, or a scene description associated with the portion of digital content. In one example embodiment, the sub frame may be a popup screen, or popup, wherein the text is displayed in the popup that is associated with a time point in the in the digital content. This text-based description may be streamed to the user, without the user having to download the entire portion of digital content, or even all of the text-based description as it relates to other time points fir the digital content.
The Abstract of the Disclosure is provided to comply with 37 C.F.R. §1.72(b), requiring an abstract that will allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.
Number | Date | Country | |
---|---|---|---|
Parent | 11951493 | Dec 2007 | US |
Child | 14166756 | US |