Patent Application
20050244146
This application is based upon and claims the benefit of priority from prior Japanese Patent Application No. 2004-136688, filed Apr. 30, 2004, the entire contents of which are incorporated herein by reference. BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a method of implementing moving picture hypermedia by combining moving picture data in a client and meta data on a network (or a disc), and displaying a telop, balloon, or the like on a moving picture.
2. Description of the Related Art
Hypermedia define associations called hyperlinks among media such as a moving picture, still picture, audio, text, and the like so as to allow these media to refer to each other or from one to another. For example, text data and still picture data are allocated on a home page which can be browsed using the Internet and is described in HTML, and links are defined all over these text data and still picture data. By designating such link, associated information as a link destination can be immediately displayed. Since the user can access associated information by directly designating a phrase that appeals to him or her, an easy and intuitive operation is allowed.
On the other hand, in hypermedia that mainly include moving picture data in place of text and still picture data, links from objects such as persons, articles, and the like that appear in the moving picture to associated contents such as their text data, still picture data that explain them are defined. When a viewer designates an object, the associated contents are displayed. At this time, in order to define a link between the spatio-temporal region of an object that appears in the moving picture and associated contents, data (object region data) indicating the spatio-temporal region of the object in the moving picture is required.
As the object region data, a mask image sequence having two or more values, arbitrary shape encoding of MPEG-4, a method of describing the loci of feature points of a figure, as described in Jpn. Pat. Appln. KOKAI Publication No. 2000-285253, a method described in Jpn. Pat. Appln. KOKAI Publication No. 2001-111996, and the like may be used. In order to implement hypermedia that mainly include moving picture data, data (action information) that describes an action for displaying other associated contents upon designation of an object is required in addition to the above data. These data other than the moving picture data will be referred to as meta data hereinafter.
As a method of providing moving picture data and meta data to a viewer, a method of preparing a recording medium (video CD, DVD, or the like) that records both moving picture data and meta data is available. In order to provide meta data of moving picture data that has already been owned as a video CD or DVD, only meta data can be downloaded or distributed by streaming from the network. Both moving picture data and meta data may be distributed via the network. At this time, meta data preferably has a format that can efficiently use a buffer, is suited to random access, and is robust against any data loss in the network.
When moving picture data are switched frequently (e.g., when moving picture data captured at a plurality of camera angles are prepared, and a viewer can freely select an arbitrary camera angle; like multi-angle video of DVD-Video), meta data must be quickly switched in correspondence with switching of moving picture data.
Upon distributing meta data on a network to a viewer by streaming wherein the meta data relates to moving picture data at the viewer, or playing back meta data at the viewer, it is preferable
a) to improve the efficiency of use of a buffer;
b) to facilitate random access;
c) to reduce influence of a data loss; and
d) to allow high-speed switching of meta data.
Under the circumstances, it is desired to provide a technique that can improve the efficiency of the processing associated with meta data.
According to one aspect of the present invention, there is provided an information recording medium for use in playback information recorded thereon, the information comprising a stream which includes meta data of a moving picture capable of being played back upon playback of a video content, and which is formed of access units as data units capable of being processed independently; and specific information used to select a stream corresponding to the meta data of the moving picture in accordance with a moving picture playback period of the video content.
According to another aspect of the present invention, there is provided an information playback method, comprising reading out information from a recording medium storing a stream which includes meta data of a moving picture capable of being played back upon playback of a video content, and which is formed of access units as data units capable of being processed independently, and storing specific information used to select a stream corresponding to the meta data in accordance with a moving picture playback period of the video content; and playing back the video content, and playing back the meta data with selecting a stream corresponding to the meta data on the basis of the specific information.
According to still another aspect of the present invention, there is provided an information recording method, comprising recording on a first area of an information recording medium, a stream which includes meta data of a moving picture capable of being played back upon playback of a video content, and which is formed of access units as data units capable of being processed independently; and recording on a second area of the information recording medium, specific information used to select a stream corresponding to the meta data of the moving picture in accordance with a moving picture playback period of the video content.
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention, and together with the general description given above and the detailed description of the embodiments given below, serve to explain the principles of the invention.
Prior to a detailed description of an embodiment of the present invention, items as points will be briefly described below.
Moving picture meta data (its data structure) according to an embodiment of the present invention includes one or a plurality of access units as data units which can be independently processed by a system. Note that the access unit (Vclick_AU in
In the embodiment of the present invention, a data structure (VCD in
Since meta data is formed as a set of access units that can be processed independently, the buffer can be used efficiently, random access is facilitated, the influence of any data loss is small, and meta data can be switched at high speed.
Since the specific information used to select a stream of meta data is available, if meta data includes a plurality of streams, the stream can be selected and loaded without any user's stream selection instruction.
Embodiments of the present invention will be described in detail hereinafter with reference to the accompanying drawings.
(Overview of Application)
Data of a region 102 of the object, action data of a client upon designation of this region by, e.g., clicking or the like, and the like will be referred to as object meta data or Vclick data together. The object meta data may be recorded on a local moving picture data recording medium (optical disc, hard disc, semiconductor memory, or the like) together with moving picture data, or may be stored in a server on the network and may be sent to the client via the network. How to implement this application will be described in detail hereinafter.
(System Model)
Reference numeral 200 denotes a client; 201, a server; and 221, a network that connects the server 201 and client 200. The client 200 comprises a moving picture playback engine 203, Vclick engine 202, disc device 230, user interface 240, network manager 208, and disc device manager 213. Reference numerals 204 to 206 denote devices included in the moving picture playback engine; 207, 209 to 212, and 214 to 218, devices included in the Vclick engine; and 219 and 220, devices included in the server 201. The client 200 can play back moving picture data, and can display a document described in a markup language (e.g., HTML or the like), which are stored in the disc device 230. Also, the client 200 can display a document (e.g., HTML) on the network.
When meta data associated with moving picture data stored in the client 200 is stored in the server 201, the client 200 can execute a playback process using this meta data and the moving picture data in the disc device 230. The server 201 sends media data M to the client 200 via the network 221 in response to a request from the client 200. The client 200 processes the received media data in synchronism with playback of a moving picture to implement additional functions of hypermedia and the like (note that “synchronization” is not limited to a physically perfect match of timings but some timing error is allowed).
The moving picture playback engine 203 is used to play back moving picture data stored in the disc device 230, and has the devices 204, 205, and 206. Reference numeral 231 denotes a moving picture data recording medium (more specifically, a DVD, video CD, video tape, hard disc, semiconductor memory, or the like). The moving picture data recording medium 231 records digital and/or analog moving picture data. Meta data associated with moving picture data may be recorded on the moving picture data recording medium 231 together with the moving picture data. Reference numeral 205 denotes a moving picture playback controller, which can control playback of video/audio/sub-picture data D1 from the moving picture data recording medium 231 in accordance with a “control” signal output from the interface handler 207 of the Vclick engine 202.
More specifically, the moving picture playback controller 205 can output a “trigger” signal indicating the playback status of the video/audio/sub-picture data D1 to the interface handler 207 in accordance with a “control” signal which is transmitted upon generation of an arbitrary event (e.g., a menu call or title jump based on a user instruction) from the interface handler 207 in a moving picture playback mode. In this case (at a timing simultaneously with output of the trigger signal or an appropriate timing before or after that timing), the moving picture playback controller 205 can output a “status” signal indicating property information (e.g., an audio language, sub-picture caption language, playback operation, playback position, various kinds of time information, disc contents, and the like set in the player) to the interface handler 207. By exchanging these signals, a moving picture data read process can be started or stopped, and access to a desired location in moving picture data can be made.
The AV decoder 206 has a function of decoding video data, audio data, and sub-picture data recorded on the moving picture data recording medium 231, and outputting decoded video data (mixed data of the aforementioned video and sub-picture data) and audio data. The moving picture playback engine 203 can have the same functions as those of a playback engine of a normal DVD-Video player which is manufactured on the basis of the existing DVD-Video standard. That is, the client 200 in
The interface handler 207 makes interface control among modules such as the moving picture playback engine 203, the disc device manager 213, the network manager 208, a meta data manager 210, a buffer manager 211, a script interpreter 212, a media decoder 216 (including a meta data decoder 217), a layout manager 215, an AV renderer 218, and the like. Also, the interface handler 207 receives an input event by a user operation (operation to an input device such as a mouse, touch panel, keyboard, or the like) and transmits an event to an appropriate module.
The interface handler 207 has an access table parser that parses a Vclick access table (corresponding to VCA which will be described later with reference to
The network manager 208 has a function of acquiring a document (e.g., HTML), still picture data, audio data, and the like onto a buffer 209 via the network, and controls the operation of an Internet connection unit 222. When the network manager 208 receives a connection/disconnection instruction to/from the network from the interface handler 207 that has received a user operation or a request from the meta data manager 210, it switches connection/disconnection of the Internet connection unit 222. Upon establishing connection between the server 201 and Internet connection unit 222 via the network, the network manager 208 exchanges control data and media data (object meta data). Note that the buffer 209 can be formed using a ring buffer to which a predetermined size is assigned, and its details will be explained later with reference to
Data to be transmitted from the client 200 to the server 201 include a session open request, session close request, media data (object meta data) transmission request, status information (OK, error, etc.), and the like. Also, status information of the client may be exchanged. On the other hand, data to be transmitted from the server 201 to the client 200 include media data (object meta data) and status information (OK, error, etc.)
The disc device manager 213 has a function of acquiring a document (e.g., HTML), still picture data, audio data, and the like onto the buffer 209, and a function of transmitting the video/audio/sub-picture data D1 to the moving picture playback engine 203. The disc device manager 213 executes a data transmission process in accordance with an instruction from the meta data manager 210.
The buffer 209 temporarily stores the media data M1 which is sent from the server 201 via the network (via the network manager). The moving picture data recording medium 231 records media data M2 in some cases. In such case, the media data M2 is stored in the buffer 209 via the disc device manager. Note that media data includes Vclick data (object meta data), a document (e.g., HTML), and still picture data, moving picture data, and the like attached to the document.
When the media data M2 is recorded on the moving picture data recording medium 231, it may be read out from the moving picture data recording medium 231 and stored in the buffer 209 in advance prior to the start of playback of the video/audio/sub-picture data D1. This is for the following reason: since the media data M2 and video/audio/sub-picture data D1 have different data recording locations on the moving picture data recording medium 231, if normal playback is made, a disc seek or the like occurs and seamless playback cannot be guaranteed. The above process can avoid such problem.
As described above, when the media data M1 downloaded from the server 201 is stored in the buffer 209 as in the media data M2 recorded on the moving picture data recording medium 231, the video/audio/sub-picture data D1 and media data can be simultaneously read out and played back.
Note that the storage capacity of the buffer 209 is limited. That is, the data size of the media data M1 or M2 that can be stored in the buffer 209 is limited. For this reason, unnecessary data may be erased under the control (buffer control) of the meta data manager 210 and/or the buffer manager 211.
The meta data manager 210 manages meta data stored in the buffer 209, and transfers meta data having a corresponding time stamp to the media decoder 216 from the buffer 209 upon reception of an appropriate timing (“moving picture clock” signal) synchronized with playback of a moving picture from the interface handler 207.
When meta data having a corresponding time stamp is not present in the buffer 209, it need not be transferred to the media decoder 216. The meta data manager 210 controls to load data for a size of the meta data output from the buffer 209 or for an arbitrary size from the server 201 or disc device 230 onto the buffer 209. As a practical process, the meta data manager 210 issues a meta data acquisition request for a designated size to the network manager 208 or disc device manager 213 via the interface handler 207. The network manager 208 or disc device manager 213 loads meta data for the designated size onto the buffer 209, and sends a meta data acquisition completion response to the meta data manager 210 via the interface handler 207.
The buffer manager 211 manages data (a document (e.g., HTML), still picture data and moving picture data appended to the document, and the like) other than meta data stored in the buffer 209, and sends data other than meta data stored in the buffer 209 to a parser 214 and the media decoder 216 upon reception of an appropriate timing (“moving picture clock” signal) synchronized with playback of a moving picture from the interface handler 207. The buffer manager 211 may delete data that becomes unnecessary from the buffer 209.
The parser 214 parses a document written in a markup language (e.g., HTML), and sends a script to the script interpreter 212 and information associated with a layout to the layout manager 215.
The script interpreter 212 interprets and executes a script input from the parser 214. Upon executing the script, information of an event and property input from the interface handler 207 can be used. When an object in a moving picture is designated by the user, a script is input from the meta data decoder 217 to the script interpreter 212.
The AV renderer 218 has a function of controlling video/audio/text outputs. More specifically, the AV renderer 218 controls, e.g., the video/text display positions and display sizes (often also including the display timing and display time together with them) and the level of audio (often also including the output timing and output time together with it) in accordance with a “layout control” signal output from the layout manager 215, and executes pixel conversion of a video in accordance with the type of a designated monitor and/or the type of a video to be displayed. The video/audio/text outputs to be controlled are those from the moving picture playback engine 203 and media decoder 216. Furthermore, the AV renderer 218 has a function of controlling mixing or switching of video/audio data input from the moving picture playback engine 203 and video/audio/text data input from the media decoder in accordance with an “AV output control” signal output from the interface handler 207.
The layout manager 215 outputs a “layout control” signal to the AV renderer 218. The “layout control” signal includes information associated with the sizes and positions of moving picture/still picture/text data to be output (often also including information associated with the display times such as display start/end timings and duration), and is used to designate the AV renderer 218 about a layout used to display data. The layout manager 215 checks input information such as user's clicking or the like input from the interface handler 207 to determine a designated object, and instructs the meta data decoder 217 to extract an action command such as display of associated information which is defined for the designated object. The extracted action command is sent to and executed by the script interpreter 212.
The media decoder 216 (including the meta data decoder) decodes moving picture/still picture/text data. These decoded video data and text image data are transmitted from the media decoder 216 to the AV renderer 218. These data to be decoded are decoded in accordance with an instruction of a “media control” signal from the interface handler 207 and in synchronism with a “timing” signal from the interface handler 207.
Reference numeral 219 denotes a meta data recording medium of the server 201 such as a hard disc, optical disc, semiconductor memory, magnetic tape, or the like, which records meta data to be transmitted to the client 200. This meta data is associated with moving picture data recorded on the moving picture data recording medium 231. This meta data includes object meta data to be described later. Reference numeral 220 denotes a network manager of the server 201, which exchanges data with the client 200 via the network 221.
(EDVD Data Structure and IFO File)
A basic data structure of the DVD-Video disc will be described below. The recording area of the DVD-Video disc includes a lead-in area, volume space, and lead-out area in turn from its inner periphery. The volume space includes a volume/file structure information area and DVD-Video area (DVD-Video zone), and can also have another recording area (DVD other zone) as an option.
The volume/file structure information area is assigned for the UDF (Universal Disk Format) bridge structure. The volume of the UDF bridge format is recognized according to ISO/IEC13346 Part 2. A space that recognizes this volume includes successive sectors, and starts from the first logical sector of the volume space in
The DVD-Video area records management information called a video manager VMG and one or more video contents called video title sets VTS (VTS#1 to VTS#n). The VMG is management information for all VTSs present in the DVD-Video area, and includes control data VMGI, VMG menu data VMGM_VOBS (option), and VMG backup data. Each VTS includes control data VTSI of that VTS, VTS menu data VTSM_VOBS (option), data VTSTT_VOBS of the contents (movie or the like) of that VTS (title), and VTSI backup data. To assure compatibility to the conventional DVD-Video standard, the DVD-Video area with such contents is also required.
A playback select menu or the like of respective titles (VTS#l to VTS#n) is given in advance by a provider (the producer of a DVD-Video disc) using the VMG, and a playback chapter select menu, the playback order of recorded contents (cells), and the like in a specific title (e.g., VTS#1) are given in advance by the provider using the VTSI. Therefore, the viewer of the disc (the user of the DVD-Video player) can enjoy the recorded contents of that disc in accordance with menus of the VMG/VTSI prepared in advance by the provider and playback control information (program chain information PGCI) in the VTSI. However, with the DVD-Video standard, the viewer (user) cannot play back the contents (movie or music) of each VTS by a method different from the VMG/VTSI prepared by the provider.
The enhanced DVD-Video disc shown in
The ENAV contents include data such as audio data, still picture data, font/text data, moving picture data, animation data, Vclick data, and the like, and also an ENAV document (described in a Markup/Script language) as information for controlling playback of these data. This playback control information describes, using a Markup language or Script language, playback methods (display method, playback order, playback switch sequence, selection of data to be played back, and the like) of the ENAV contents (including audio, still picture, font/text, moving picture, animation, Vclick, and the like) and/or the DVD-Video contents. For example, Markup languages such as HTML (Hyper Text Markup Language)/XHTML (eXtensible Hyper Text Markup Language), SMIL (Synchronized Multimedia Integration Language), and the like, Script languages such as an ECMA (European Computer Manufacturers Association) script, JavaScript(R), and the like, and so forth, may be used in combination.
Since the contents of the enhanced DVD-Video disc in
Especially, as shown in
The Vclick information file VCI is data indicating a portion of DVD-Video contents where the Vclick stream VCS (to be described below) is appended (e.g., to the entire title, the entire chapter, a program chain, program, or cell as a part thereof, or the like of the DVD-Video contents). The Vclick access table VCA is assured for each Vclick stream VCS (to be described below), and is used to access the Vclick stream VCS. The Vclick stream VCS includes data such as location information of an object in a moving picture, an action description to be made upon clicking the object, and the like. The Vclick information file backup VCIB is a backup of the aforementioned Vclick information file VCI, and always has the same contents as the Vclick information file VCI. The Vclick access table backup VCAB is a backup of the Vclick access table VCA, and always has the same contents as Vclick access table VCA.
In the example of
Furthermore, when the user creates an original disc using a video recordable medium (e.g., a DVD-R disc, DVD-RW disc, DVD-RAM disc, hard disc or the like) and a video recorder (e.g., a DVD-VR recorder, DVD-SR recorder, HD-DVD recorder, HDD recorder, or the like), if he or she records ENAV contents including the Vclick data VCD on this disc or prepares the Vclick data VCD on a data storage of a personal computer other than this disc and connects this personal computer and recorder, he or she can enjoy meta data playback in the same manner as in the DVD-ROM video+the ENAV player in
A Vclick stream file describes the relationship between location information (a relative byte size from the head of the file) of each Vclick stream VCS and time information (a time stamp of a corresponding moving picture or relative time information from the head of the file; see
The Vclick stream VCS includes one or more files (VCKSTR01.VCK to VCKSTR99.VCK or arbitrary file names), and can be played back together with the appended DVD-Video contents with reference to the description of the aforementioned Vclick information file VCI. If there are a plurality of attributes (e.g., Japanese Vclick data VCD, English Vclick data VCD, and the like), different Vclick streams VCS (i.e., different files) may be formed in correspondence with different attributes. Alternatively, respective attributes may be multiplexed to form one Vclick stream VCS (i.e., one file) (for example, see
In case of the former configuration (a plurality of Vclick streams VCS are formed in correspondence with different attributes; e.g., an example shown in
Note that each Vclick stream VCS and Vclick access table VCA can be associated using, e.g., their file names. In the aforementioned example, one Vclick access table VCA (VCKSTRXX.IFO; XX=01 to 99) is assigned to one Vclick stream VCS (VCKSTRXX.VCK; XX=01 to 99). Hence, by adopting the same file name except for extensions, association between each Vclick stream VCS and Vclick access table VCA can be identified.
In addition, the Vclick information file VCI describes association between each Vclick stream VCS and Vclick access table VCA (more specifically, the VCI parallelly describes descriptions of VCS and those of VCA), thereby identifying association between each Vclick stream VCS and Vclick access table VCA.
The Vclick information file backup VCIB is formed of a VCKINDEX.BUP file, and has the same contents as the aforementioned Vclick information file VCI (VCKINDEX.IFO). If VCKINDEX.IFO cannot be loaded for some reason (due to scratches, stains, and the like on the disc), desired procedures can be made by loading this VCKINDEX.BUP instead. The Vclick access table backup VCAB is formed of VCKSTR01.BUP to VCKSTR99.BUP files, which have the same contents as the aforementioned Vclick access tables VCA (VCKSTR01.IFO to VCKSTR99.IFO). One Vclick access table backup VCAB (VCKSTRXX.BUP; XX=01 to 99) is assigned to one Vclick access table VCA (VCKSTRXX.IFO; XX=01 to 99), and the same file name is adopted except for extensions, thus identifying association between each Vclick access table VCA and Vclick access table backup VCAB. If VCKSTRXX.IFO cannot be loaded for some reason (due to scratches, stains, and the like on the disc), desired procedures can be made by loading this VCKSTRXX.BUP instead.
FIGS. 55 to 57 show an example of the configuration of the Vclick information file VCI. The Vclick information file VCI is made up of XML, use of XML is declared first, and a Vclick information file VCI made up of XML is declared next. Furthermore, the contents of the Vclick information file VCI are described using a <vclickinfo> tag.
The <vclickinfo> field includes zero or one <vmg> tag and zero or one or more <vts> tags. The <vmg> field represents a VMG space in DVD-Video, and indicates that a Vclick stream described in the <vmg> field is appended to DVD-Video data in the VMG space. Also, the <vts> field represents a VTS space in DVD-Video, and designates the number of a VTS space by appending a num attribute in the <vts> tag. For example, <vts num=“n”> represents the n-th VTS space. It indicates that a Vclick stream described in the <vts num=“n”> field is appended to DVD-Video data which forms the n-th VTS space.
The <vmg> field includes zero or one or more <vmgm> tags. The <vmgm> field represents a VMG menu domain in the VMG space, and designates the number of a VMG menu domain by appending a num attribute in the <vmgm> tag. For example, <vmgm num=“n”> indicates the n-th VMG menu domain. It indicates that a Vclick stream described in the <vmgm num=“n”> field is appended to DVD-Video data which forms the n-th VMG menu domain.
Furthermore, the <vmgm> field includes zero or one or more <pgc> tags. The <pgc> field represents a PGC (Program Chain) in the VMG menu domain, and designates the number of a PGC by appending a num attribute in the <pgc> tag. For example, <pgc num=“n”> indicates the n-th PGC. It indicates that a Vclick stream described in the <pgc num=“n”> field is appended to DVD-Video data which forms the n-th PGC.
Next, the <vts> field includes zero or one or more <vts_tt> tags and zero or one or more <vtsm> tags. The <vts_tt> field represents a title domain in the VTS space, and designates the number of a title domain by appending a num attribute in the <vts_tt> tag. For example, <vts_tt num=“n”> indicates the n-th title domain. It indicates that a Vclick stream described in the <vts_tt num=“n”> field is appended to DVD-Video data which forms the n-th title domain.
The <vtsm> field represents a VTS menu domain in the VTS space, and designates the number of a VTS menu domain by appending a num attribute in the <vtsm> tag. For example, <vtsm num=“n”> indicates the n-th VTS menu domain. It indicates that a Vclick stream described in the <vtsm num=“n”> field is appended to DVD-Video data which forms the n-th VTS menu domain.
Moreover, the <vts_tt> or <vtsm> field includes zero or one or more <pgc> tags. The <pgc> field represents a PGC (Program Chain) in the title or VTS menu domain, and designates the number of a PGC by appending a num attribute in the <pgc> tag. For example, <pgc num=“n”> indicates the n-th PGC. It indicates that a Vclick stream described in the <pgc num=“n”> field is appended to DVD-Video data which forms the n-th PGC.
In the example shown in FIGS. 55 to 57, six Vclick streams are appended to the DVD-Video contents. For example, the initial (first) Vclick stream is designated using an <object> tag in <pgc num=“1”> in <vmgm num=“1”> in <vmg>. This indicates that the Vclick stream designated by the <object> tag is appended to the first PGC in the first VMG menu domain in the VMG space.
The <object> tag indicates the location of the Vclick stream using a “data” attribute. For example, in the embodiment of the present invention, the location of the Vclick stream is designated by “file://dvdrom:/dvd_enav/vclick1.vck”. Note that “file://dvdrom:/” indicates that the Vclick stream is present in the enhanced DVD disc, “dvd_enav/” indicates that the stream is present under a “DVD_ENAV” directory in the disc, and “vclick1.vck” indicates the file name of the Vclick stream. By including the <object> tag which describes the Vclick stream and that which describes a Vclick access table VCA, information of the Vclick access table VCA corresponding to the Vclick stream can be described. In the <object> tag, the location of the Vclick access table VCA is indicated using a “data” attribute. For example, in the embodiment of the present invention, the location of the Vclick access table VCA is designated by “file://dvdrom:/dvd_enav/vclick1.ifo”. Note that “file://dvdrom:/” indicates that the Vclick access table VCA is present in the enhanced DVD disc, “dvd_enav/” indicates that the table is present under a “DVD_ENAV” directory in the disc, and “vclick1.ifo” indicates the file name of the Vclick access table VCA.
The next (second) Vclick stream is designated using an <object> tag in <vmgm num=“n”> in <vmg>. This indicates that a Vclick stream designated by the <object> tag is appended to the whole first VMG menu domain in the VMG space. The <object> tag indicates the location of the Vclick stream using a “data” attribute. For example, in the embodiment of the present invention, the location of the Vclick stream is designated by “http://www.vclick.com/dvd_enav/vclick2.vck”. Note that “http://www.vclick.com/dvd_enav/” indicates that the Vclick stream is present in an external server (201 in
As for a Vclick access table VCA, the location of the Vclick access table VCA is similarly indicated using a “data” attribute in an <object> tag. For example, in the embodiment of the present invention, the location of the Vclick access table VCA is designated by “http://www.vclick.com/dvd_enav/vclick2.ifo”. Note that “http://www.vclick.com/dvd_enav/” indicates that the Vclick access table VCA is present in an external server (201), and “vclick2.ifo” indicates the file name of the Vclick access table VCA.
The third Vclick stream is designated using an <object> tag in <pgc num=“1”> in <vts_tt num=“1”> in <vts num=“1”>. This indicates that the Vclick stream designated by the <object> tag is appended to the first PGC in the first title domain in the first VTS space. In the <object> tag, the location of the Vclick stream is indicated using a “data” attribute. For example, in the embodiment of the present invention, the location of the Vclick stream is designated by “file://dvdrom:/dvd_enav/vclick3.vck”. Note that “file://dvdrom:/dvd_enav/” indicates that the Vclick stream is present under a “DVD_ENAV” directory in the disc, and “vclick3.vck” indicates the file name of the Vclick stream.
The fourth Vclick stream is designated using an <object> tag in <vts_tt num=“n”> in <vts num=“1”>. This indicates that the Vclick stream designated by the <object> tag is appended to the n-th title domain in the first VTS space. In the <object> tag, the location of the Vclick stream is indicated using a “data” attribute. For example, in the embodiment of the present invention, the location of the Vclick stream is designated by “file://dvdrom:/dvd_enav/vclick4.vck”. Note that “file://dvdrom:/dvd_enav/” indicates that the Vclick stream is present under a “DVD_ENAV” directory in the disc, and “vclick4.vck” indicates the file name of the Vclick stream.
The fifth Vclick stream is designated using an <object> tag in <vtsm num=“1”> in <vts num=“1”>. This indicates that the Vclick stream designated by the <object> tag is appended to the first VTS menu domain in the first VTS space. In the <object> tag, the location of the Vclick stream is indicated using a “data” attribute. For example, in the embodiment of the present invention, the location of the Vclick stream is designated by “file://dvdrom:/dvd_enav/vclick5.vck”. Note that “file://dvdrom:/dvd_enav/” indicates that the Vclick stream is present under a “DVD_ENAV” directory in the disc, and “vclick5.vck” indicates the file name of the Vclick stream.
The sixth Vclick stream is designated using an <object> tag in <pgc num=“1”> in <vtsm num=“1”> in <vts num=“1”>. This indicates that the Vclick stream designated by the <object> tag is appended to the first PGC in the first VTS menu domain in the first VTS space. In the <object> tag, the location of the Vclick stream is indicated using a “data” attribute. For example, in the embodiment of the present invention, the location of the Vclick stream is designated by “file://dvdrom:/dvd_enav/vclick6.vck”. Note that “file://dvdrom:/dvd_enav/” indicates that the Vclick stream is present under a “DVD_ENAV” directory in the disc, and “vclick6.vck” indicates the file name of the Vclick stream.
When the user switches these streams, a “Vclick switch button” used to switch the Vclick streams VCS is provided to a remote controller attached to the apparatus shown in
On the other hand, when the contents provider changes Vclick streams VCS, a Vclick switching command (a description format is, e.g., “changeVclick( )”) is described in a Markup language, and this switch command is issued at a timing designated by the contents provider in the Markup language, thus freely changing two or more Vclick streams VCS.
FIGS. 59 to 65 show other description examples (seven examples) of the Vclick information file VCI. In the first example (
Upon switching Vclick streams by the contents provider in the above example, for example, when the playback apparatus (200 in
The contents provider can also select an arbitrary Vclick stream by issuing a command at a timing designated in the Markup language by the contents provider using a Vclick switching command (a description format is, e.g., “changeVclick(priority)”). For example, when a “changeVclick(2)” command is issued, Vclick stream #2 with a “priority” attribute=“2”is played back.
In the next example (
For example, when audio stream #1 of the video contents includes Japanese audio and audio stream #2 includes English audio, Vclick stream #1 is formed in Japanese, as shown in
The above description is merely an example. For example, when the DVD-Video contents include English audio, a Japanese Markup page may be synchronously played back (a combination of the left side of
In the third example (
For example, when sub-picture stream #1 of the video contents includes a Japanese caption and sub-picture stream #3 includes an English caption, Vclick stream #1 is formed in Japanese, as shown in
In the fourth example (
In the fifth example (
This example indicates that the DVD-Video contents themselves have a “16:9” aspect ratio, and are allowed to make a “wide” output to a TV monitor having a “16:9” aspect ratio, and a “letter box (lb)” or “pan scan (ps)” output to a TV monitor having a “4:3” aspect ratio. By contrast, when the (default) display aspect ratio is “16:9” and the (current) display mode is “wide”, Vclick stream #1 is played back synchronously (
For example, a balloon as a Vclick object, which is displayed just beside a person, when the video contents are displayed at a “16:9” aspect ratio, can be displayed on the upper or lower portion (the upper or lower hatching portions in
Also, in accordance with the screen configuration, the balloon size may be decreased to prevent a balloon text display area from protruding the screen. Or the balloon text display area may be increased to obtain an appropriate display balance between the balloon text display area size and object size, or a text display balance of the balloon area may be optimized by decreasing or increasing the text size in the balloon text display area. When the text size in the balloon is decreased, text color is devised to set high contrast between text and its surrounding portion, so that text stands out with respect to the text surrounding portion (red text on a white background or yellow text on a black background), and/or a font is changed to bold, thus preventing the visibility drop of text.
In this manner, Vclick objects can be displayed in correspondence with the display state of the DVD-Video contents. In practice, the playback apparatus refers to “default display aspect ratio” and “current display mode” in SPRM(14) (player configuration for video) as a system parameter in the playback apparatus and searches the Vclick information file VCI for a corresponding Vclick stream VCS and plays it back.
In the sixth example (
Finally, the aforementioned functions can be used in combination as shown in an example (
More specifically, streams Vclick#1 to Vclick#3 are assigned to whole PGC#1 (corresponding to
In PGC#2 of
In PGC#7 in
When an object to be synchronously played back with Vclick streams is DVD-Video contents, Vclick streams can be switched for respective titles (VTS) of DVD-Video as a largest unit, and can be switched for respective parts-of-title (chapters) as a smaller unit. Also, Vclick streams can be switched for respective program chains (PGC) as a still smaller unit, for respective programs (PG) as a yet smaller unit, or for respective cells as a smallest unit.
When Vclick streams according to the embodiment of the present invention are applied to a recording/playback system such as a DVD-VR recorder, DVD-SR recorder, HD-DVD recorder, or the like, Vclick streams may be switched for respective user-defined PGC data (playlists) or respective entry points marked locally in programs that form PGC data.
The playback apparatus (enhanced DVD player) according to the embodiment of the present invention can sequentially change Vclick stream files to be appended in correspondence with the playback state of the DVD-Video contents by loading the Vclick information file VCI in advance or referring to that file as needed, prior to playback of the DVD-Video contents. In this manner, a high degree of freedom can be assured upon forming Vclick streams, and the load on authoring can be reduced.
By increasing the number of files (the number of streams) of unitary Vclick contents, and decreasing each file size, an area (the buffer 209 in the apparatus of
By decreasing the number of files (i.e., forming one stream to include a plurality of Vclick data) although the file size increases, Vclick data can be switched smoothly when the playback state of the DVD-Video contents has changed (since the information size of buffered Vclick data is large).
(Overview of Data Structure and Access Table)
A Vclick stream VCS includes data associated with regions of objects (e.g., persons, articles, and the like) that appear in the moving picture recorded on the moving picture data recording medium 231, display methods of the objects in the client 200, and data of actions to be taken by these objects when the user designates them. An overview of the structure of Vclick data and its elements will be explained below.
Object region data as data associated with a region of an object (e.g., a person, article, or the like) that appears in the moving picture will be explained first.
Reference numeral 401 denotes a header of the Vclick_AU. The header 401 includes an ID used to identify the Vclick_AU, and data used to specify the data size of that AU. Reference numeral 402 denotes a time stamp which indicates that of the start of the lifetime of this Vclick_AU. Since the active time and lifetime of Vclick_AU are normally equal to each other, the time stamp also indicates a time of the moving picture corresponding to the object region described in the object region data 400. As shown in
Referring to
Temporal divisions of respective Vclick_AUs may be arbitrarily determined. However, when the divisions of Vclick_AUs are aligned to all objects, as shown in
Since the selected camera angle is more likely to be switched by the user during viewing, the Vclick stream VCS is preferably prepared by multiplexing Vclick_AUs of different camera angles in this way. This is because quick display switching is allowed at the client 200 side. For example, when Vclick data is stored in the server 201, a Vclick stream VCS including Vclick_AUs of a plurality of camera angles is transmitted intact to the client 200. In this way, since a Vclick_AU corresponding to a currently viewed camera angle always arrives the client, a camera angle can be switched instantaneously. Of course, setup information of the client 200 may be sent to the server 201, and only a required Vclick_AU may be selectively transmitted from a Vclick stream VCS. In this case, since the client must communicate with the server (201), the process delays slightly (although this process delay problem can be solved if high-speed means such as an optical fiber or the like is used in a communication).
On the other hand, since attributes such as a moving picture title, PGC of DVD-Video, the aspect ratio of the moving picture, viewing region, and the like are not so frequently changed, they are preferably prepared as independent Vclick streams VCS so as to lighten the process of the client 200 and to reduce the load on the network. A Vclick stream VCS to be selected of a plurality of Vclick streams VCS can be determined with reference to the Vclick information file VCI, as has already been described above.
Another Vclick_AU selection method will be described below. A case will be examined below wherein the client 200 downloads the Vclick stream (VCS) 506 from the server 201, and uses only required access units (AUs) on the client 200 side. In this case, Ids used to identify required Vclick_AUs may be assigned to respective AUs. Such ID is called a filter ID.
The conditions of the required access units (AUs) are described in, e.g., the Vclick information file VCI as follows:
In this case, two different filtering conditions are described for one Vclick stream VCS. This indicates that two different Vclick_AUs having different attributes can be selected from a single Vclick stream VCS in accordance with the setups of system parameters at the client.
Note that the Vclick information file VCI may be present on the moving picture data recording medium (e.g., the enhanced DVD-Video disc in
If access units (AUs) have no filter IDs, meta the data manager 210 checks the time stamps, attributes, and the like of AUs to select AUs that match the given conditions, thereby identifying the required Vclick_AUs.
An example using the filter IDs will be explained according to the above description. In the above conditions, “audio” represents an audio stream number, which is expressed by a 4-bit numerical value. Likewise, 4-bit numerical values are assigned to subpicture number subpic and angle number angle. In this way, the states of three parameters can be expressed by a 12-bit numerical value. That is, three parameters audio=“3”, subpic=“2”, and angle=“1” can be expressed by 0x321 (hex). This value is used as a filter ID. That is, each Vclick_AU has a 12-bit filter ID in a Vclick_AU header (see filtering_id in
Vclick_AUs which are sent from the buffer 209 to the meta data decoder 217 with the aforementioned procedures have the following properties:
i) All these AUs have the same lifetime, which includes the moving picture clock T.
ii) All these AUs have the same filter ID x.
AUs in the object meta data stream which satisfy the above conditions i) and ii) are not present except for these AUs. Note that identifying and selecting a specific AU by a given filter ID is to also select a Vclick stream including the selected AU. On the other hand, the Vclick stream to be played back can also be selected with reference to the Vclick Info VCI file (see the process in step S8207 in
In the above description, the filter ID is defined by a combination of values assigned to parameters. Alternatively, the filter ID may be directly designated in the Vclick information file VCI. For example, the filter ID is defined in an IFO file as follows:
The above description indicates that Vclick streams VCS and filter ID values are determined based on designated parameters. Selection of Vclick_AUs by the filter ID and transfer of AUs from the buffer 209 to the media decoder 217 are done in the same procedures as in
When Vclick data is stored in the server 201, and a moving picture is to be played back from its head, the server 201 need only distribute a Vclick stream VCS in turn from the head to the client. However, if a random access has been made, data must be distributed from the middle of the Vclick stream VCS. At this time, in order to quickly access a desired position in the Vclick stream VCS, a Vclick access table VCA is required.
The server 201 stores the Vclick access table VCA and uses it for convenience to search for Vclick data to be transmitted in response to random access from the client. However, the Vclick access table VCA stored in the server 201 may be downloaded to the client 200, which may search for a Vclick stream VCS. Especially, when Vclick streams VCS are simultaneously downloaded from the server 201 to the client 200, Vclick access tables VCA are also simultaneously downloaded from the server 201 to the client 200.
On the other hand, a moving picture recording medium such as a DVD or the like which records Vclick streams VCS may be provided. In this case as well, it is effective for the client 200 to use the Vclick access table VCA so as to search for data to be used in response to random access of playback contents. In such case, the Vclick access tables VCA are recorded on the moving picture recording medium as in Vclick streams VCS, and the client 200 reads out and uses the Vclick access table VCA of interest from the moving picture recording medium onto its internal main memory or the like.
Random playback of Vclick streams VCS, which is produced upon random playback of a moving picture or the like, is processed by the meta data decoder 217. In the Vclick access table VCA shown in
Assume that some natural totally ordered relationship is defined for a set of time stamp values. For example, as for PTS, a natural ordered relationship as a time can be introduced. As for time stamps including DVD parameters, the ordered relationship can be introduced according to a natural playback order of the DVD. Each Vclick stream VCS satisfies the following conditions:
i) Vclick_AUs in the Vclick stream VCS are arranged in ascending order of time stamp. At this time, the lifetime of each Vclick_AU is determined as follows: Let t be the time stamp value of a given AU. A time stamp values u of AUs after the given AU satisfy u>=t under the above condition. Let t′ be a minimum one of such “u”s, which satisfies u≠t. A period which has the time t as the start time and t′ as the end time is defined as the lifetime of the given AU. If there is no AU which has the time stamp value u that satisfies u>t after the given AU, the end time of the lifetime of the given AU matches the end time of the moving picture.
ii) The active time of each Vclick_AU corresponds to the time range of the object region described in the object region data included in that Vclick_AU.
Note that the following constraint associated with the active time for a Vclick stream VCS:
The active time of a Vclick_AU is included in the lifetime of that AU.
A Vclick stream VCS which satisfies the above constraints i) and ii) has the following good properties:
First, high-speed random access of the Vclick stream VCS can be made, as will be described later. Second, a buffer process upon playing back the Vclick stream VCS can be simplified. The buffer (209 in
In the Vclick access table VCA shown in
i) A position indicated by offset is the head position of a given Vclick_AU.
ii) A time stamp value of that AU is equal to or smaller than the value of time.
iii) A time stamp value of an AU immediately before that AU is truly smaller than time.
In the Vclick access table VCA, “time”s may be arranged at arbitrary intervals but need not be arranged at equal intervals. However, they may be arranged at equal intervals in consideration of convenience for a search process and the like.
Upon reception of the moving picture clock T from the interface handler 207 (step S4501), the meta data manager 210 searches time of the Vclick access table VCA stored in the buffer 209 for maximum time t′ which satisfies t′<=T (step S4502). A high-speed search can be conducted using, e.g., binary search as a search algorithm. The offset value which forms a pair with the obtained time t′ in the Vclick access table VCA is substituted in a variable h (step S4503). The meta data manager 210 finds AUx which is located at the h-th byte position from the head of the Vclick stream VCS stored in the buffer 209 (step S4504), and substitutes the time stamp value of x in a variable t (step S4505). According to the aforementioned conditions, since t is equal to or smaller than t′, t<=T.
The meta data manager 210 checks Vclick_AUs in the Vclick stream VCS in turn from x and sets the next AU as new x (step S4506). The offset value of x is substituted in a variable h′ (step S4507), and the time stamp value of x is substituted in the variable u (step S4508). If u>T (YES in step S4509), the meta data manager 210 instructs the buffer 209 to send data from the offsets h to h′ of the Vclick stream VCS to the media decoder 216 (steps S4510 and S4511). On the other hand, if u<=T (NO in step S4509) and u>T (YES in step S4601), the value of t is updated by u (i.e., t=u) (step S4602). Then, the value of the variable h is updated by h′ (i.e., h=h′) (step S4603).
If the next AU is present on the Vclick stream VCS (i.e., if x is not the last AU) (YES in step S4604), the next AU is set as new x to repeat the aforementioned procedures (the flow returns to step S4506 in
With the aforementioned procedures, Vclick_AUs sent from the buffer 209 to the media decoder 216 apparently have the following properties:
i) All Vclick_AUs have the same lifetime. In addition, the moving picture clock T is included in this lifetime.
ii) Vclick_AUs in the Vclick stream VCS which satisfy the above condition i) are not present except for these AUs.
The lifetime of each Vclick_AU in the Vclick is stream VCS includes the active time of that AU, but they do not always match. In practice, a case shown in
A Vclick stream VCS in which AUs are arranged in the order of #1, #2, and #3 will be examined. Assume that the moving picture clock T is designated in the example of
When the meta data manager 210 detects based on the flag (not shown) included in the header (“Vclick AU Header” in
That is, the meta data manager 210 receives the moving picture clock T from the interface manager 207 (step S5001), obtains maximum t′ which satisfies t′<=T (step S5002), and substitutes the offset value which forms a pair with t′ in a variable h (step S5003). An access unit AU which is located at the position of the offset value h in the object meta data stream is set as x (step S5004), and the time stamp value of x is stored in a variable t (step S5005). If x is a NULL_AU (YES in step S5006), an AU next to x is set as new x (step S5007), and the flow returns to step S5006. If x is not a NULL_AU (NO in step S5006), the offset value of x is stored in a variable h′ (step S5101). The subsequent processes (steps S5102 to S5105 in
The protocol between the server and client will be explained below. As the protocol used upon transmitting Vclick data from the server 201 to the client 200, for example, RTP (Real-time Transport Protocol) is known. Since RTP has good chemistry with UDP/IP and attaches importance to realtimeness, packets are likely to be omitted. If RTP is used, a Vclick stream VCS is divided into transmission packets (RTP packets) when it is transmitted. An example of a method of storing a Vclick stream VCS in transmission packets will be explained below.
On the other hand,
As a protocol other than RTP, HTTP (Hypertext Transport Protocol) or HTTPS may be used. HTTP has good chemistry with TCP/IP and omitted data is re-sent, thus allowing highly reliable data communications. However, when the network throughput is low, a data delay may occur. Since HTTP is free from any data omission, a method of dividing a Vclick stream VCS into packets upon storage need not be particularly taken into consideration.
(Playback Procedure (Network))
The procedures of a playback process when a Vclick stream VCS is present on the server 201 will be described below.
Assume that the client (200) is notified in advance of the address of the server (201) that distributes data corresponding to a moving picture to be played back by a method of, e.g., recording address information on a moving picture data recording medium. The server 201 sends information of Vclick data to the client 200 as a response to this request. More specifically, the client receives information such as the protocol version of the session, session owner, session name, connection information, session time information, meta data name, meta data attributes, and the like. As a method of describing these pieces of information, for example, SDP (Session Description Protocol) is used. The client 200 then requests the server 201 to open a session (RTSP SETUP method). The server 201 prepares for streaming, and returns a session ID to the client 200. The processes described so far correspond to those in step S3702 when RTP is used.
When HTTP is used in place of RTP, the communication procedures are made, as shown in, e.g.,
In step S3703, a process for requesting the server (201) to transmit Vclick data is executed while the session between the server 201 and client 200 is open. This process is implemented by sending an instruction from the interface handler 207 to the network manager 208, and then sending a request from the network manager 208 to the server (201). In case of RTP, the network manager 208 sends an RTSP PLAY method to the server to issue a Vclick data transmission request. The server specifies a Vclick stream VCS to be transmitted with reference to information received from the client so far and Vclick Info VCI in the server. Furthermore, the server specifies a transmission start position in the Vclick stream VCS using time stamp information of the playback start position included in the Vclick data transmission request and the Vclick access table VCA stored in the server. The server then packetizes the Vclick stream VCS and sends packets to the client by RTP.
On the other hand, in case of HTTP, the network manager 208 transmits an HTTP GET method to issue a Vclick data transmission request. This request may include time stamp information of the playback start position of a moving picture. The server specifies a Vclick stream VCS to be transmitted and the transmission start position in this stream by the same method as in RTP, and sends the Vclick stream VCS to the client by HTTP.
In step S3704, a process for buffering the Vclick stream VCS sent from the server on the buffer 209 is executed. This process is done to prevent the buffer 209 from being emptied when Vclick stream transmission from the server is too late during playback of the Vclick stream VCS. If-the meta data manager 210 notifies the interface handler that the buffer has stored the sufficient Vclick stream VCS, the flow advances to step S3705. In step S3705, the interface handler issues a moving picture playback start command to the controller 205 and also issues a command to the meta data manager 210 to start output of the Vclick stream VCS to the meta data decoder 217.
In step S3806, a process for decoding the Vclick stream VCS in synchronism with the moving picture whose playback is in progress is executed. More specifically, upon reception of a message indicating that a given size of the Vclick stream VCS is stored in the buffer 209 from the meta data manager 210, the interface handler 207 outputs, to the meta data manager 210, an output start command of the Vclick stream VCS to the meta data decoder 217. The meta data manager 210 receives the time stamp of the moving picture whose playback is in progress from the interface handler, specifies a Vclick_AU corresponding to this time stamp from data stored in the buffer, and outputs it to the meta data decoder 217.
In the process procedures shown in
The aforementioned problem is solved after decoding of the Vclick stream VCS starts since the beginning of moving picture playback. If a period until a predetermined size of VCS (Vclick_AU) is decoded after the beginning of playback is shortened inasmuch as the user does not get irritated, the above problem can be solved in practice. Hence, the client 200 and server 201 may be always-on connected via a high-speed line, and the processes in steps S3802 and S3803 may be executed as background processes in advance when a DVD disc that uses Vclick is loaded into the disc device 230 (or after a title to be played back is selected from the loaded disc). In this case, if a user instruction is input in step S3800, DVD playback in step S3801 immediately starts. At the same time, the processes in steps S3802 and S3803 are skipped, and downloading of the Vclick stream VCS onto the buffer via the high-speed line immediately starts (steps S3804 and S3805). If the downloaded size has reached a predetermined size (12 kbytes in an example of
During playback of the moving picture, the network manager 208 of the client 200 receives Vclick streams VCS which are sent in turn from the server 201, and stores them in the buffer 209. The stored object meta data are sent to the meta data decoder 217 at appropriate timings. That is, the meta data manager 210 refers to the time stamp of the moving picture whose playback is in progress, which is sent from the interface handler 207 to specify a Vclick_AU corresponding to that time stamp from data stored in the buffer 209, and sends the specified object meta data to the meta data decoder 217 for respective AUs. The meta data decoder 217 decodes the received data. Note that the decoder 217 may skip decoding of data for a camera angle different from that currently selected by the client 200. When it is known that the Vclick_AU corresponding to the time stamp of the moving picture whose playback is in progress has already been loaded to the meta data decoder 217, the transmission process of object meta data to the meta data decoder 217 may be skipped.
The time stamp of the moving picture whose playback is in progress is sequentially sent from the interface handler 207 to the meta data decoder 217. The meta data decoder 217 decodes the Vclick_AU in synchronism with this time stamp, and sends required data to the AV renderer 218. For example, when attribute information described in the Vclick_AU instructs to display an object region, the meta data decoder generates a mask image, contour, and the like of the object region, and sends them to the AV renderer 218 in synchronism with the time stamp of the moving picture whose playback is in progress. The meta data decoder 217 compares the time stamp of the moving picture whose playback is in progress with the lifetime of the Vclick_AU to-determine old object meta data which is not required and to delete that data.
In step S3902, a process for closing the session with the server (201) is executed. When RTP is used, an RTSP TEARDOWN method is sent to the server, as shown in
(Random Access Procedure (Network))
The random access playback procedures when a Vclick stream VCS is present on the server 201 will be described below.
In step S4004, a process for requesting the server (201) to transmit Vclick data by designating the time stamp of the playback start position is executed while the session between the server 201 and client 200 is open. This process is implemented by sending an instruction from the interface handler 207 to the network manager 208, and then sending a request from the network manager 208 to the server (201). In case of RTP, the network manager 208 sends an RTSP PLAY method to the server to issue a Vclick data transmission request. At this time, the manager 208 also sends the time stamp that specifies the playback start position to the server (201) by a method using, e.g., a Range description. The server 201 specifies an object meta data stream to be transmitted with reference to information received from the client (200) so far and the Vclick Info VCI in the server 201. Furthermore, the server 201 specifies a transmission start position in the Vclick stream VCS using time stamp information of the playback start position included in the Vclick data transmission request and the Vclick access table VCA stored in the server 201. The server 201 then packetizes the Vclick stream VCS and sends packets to the client 200 by RTP.
On the other hand, in case of HTTP, the network manager 208 transmits an HTTP GET method to issue a Vclick data transmission request. This request includes time stamp information of the playback start position of the moving picture. The server 201 specifies a Vclick stream VCS to be transmitted with reference to the Vclick information file VCI, and also specifies the transmission start position in the Vclick stream VCS using the Vclick access table VCA in the server 201 by the same method as in RTP. The server 201 then sends the Vclick stream VCS to the client by HTTP.
In step S4005, a process for buffering the Vclick stream VCS sent from the server (201) on the buffer 209 is executed. This process is done to prevent the buffer 209 from being emptied when Vclick stream transmission from the server (201) is too late during playback of the Vclick stream VCS. If the meta data manager 210 notifies the interface handler that the buffer 209 has stored the sufficient Vclick stream VCS (12 kbytes described in an initialization document in, e.g.,
By contrast, in the process procedures shown in
In step S4107, a process for decoding the Vclick stream VCS in synchronism with the moving picture whose playback is in progress is executed. More specifically, upon reception of a message indicating that a given size of the Vclick stream VCS is stored in the buffer 209 from the meta data manager 210, the interface handler 207 outputs, to the meta data manager 210, an output start command of the Vclick stream VCS to the meta data decoder 217. The meta data manager 210 receives the time stamp of the moving picture whose playback is in progress from the interface handler 207, specifies a Vclick_AU corresponding to this time stamp from data stored in the buffer 209, and outputs it to the meta data decoder 217.
In the process procedures shown in
The aforementioned problem is solved after decoding of the Vclick stream VCS starts since the beginning of moving picture playback. If a period until decoding of VCS starts after the beginning of playback is shortened inasmuch as the user does not get irritated, the above problem can be solved in practice. Hence, the client 200 and server 201 may be always-on connected via a high-speed line, and the processes in steps S4102 to S4104 may be executed as background processes in advance when a DVD disc that uses Vclick is loaded into the disc device 230 (or after a title to be played back is selected from the loaded disc). In this case, if a user instruction is input in step S4100, DVD playback in step S4101 immediately starts. At the same time, the processes in steps S4102 to S4104 are skipped, and downloading of the Vclick stream VCS onto the buffer via the high-speed line immediately starts (step S4106). If the downloaded size has reached a predetermined size (e.g., 12 kbytes), decoding of the Vclick stream VCS (the first Vclick_AU in that stream) starts (step S4107). Since the processes during playback of the moving picture and moving picture playback stop process are the same as those in the normal DVD playback process, a description thereof will be omitted.
(Playback Procedure (Local))
The procedures of a playback process when a Vclick stream VCS is present on the moving picture data recording medium 231 will be described below.
In step S4202, a process for storing the Vclick stream VCS on the buffer is executed. To implement this process, the interface handler 207 issues, to the meta data manager 210, a command for assuring a buffer. The buffer size to be assured is determined as a size large enough to store the specified Vclick stream VCS. Normally, a buffer initialization document that describes this size is recorded on the moving picture data recording medium 231. If no buffer initialization document is recorded, a predetermined size is applied. Upon completion of assuring of the buffer, the interface handler 207 issues, to the controller 205, a command for reading out the specified Vclick stream VCS and storing it in the buffer.
After the Vclick stream VCS is stored in the buffer 209, a playback start process is executed in step S4203. In this process, the interface handler 207 issues a moving picture playback command to the moving picture playback controller 205, and simultaneously issues, to the meta data manager 210, an output start command of the Vclick stream VCS to the meta data decoder 217.
During playback of the moving picture, Vclick_AUs read out from the moving picture data recording medium 231 are stored in the buffer 209. The stored Vclick stream VCS is sent to the meta data decoder 217 at an appropriate timing. That is, the meta data manager 210 refers to the time stamp of the moving picture whose playback is in progress, which is sent from the interface handler 207 to specify a Vclick_AU corresponding to that time stamp from data stored in the buffer 209, and sends the specified Vclick_AU to the meta data decoder 217. The meta data decoder 217 decodes the received data. Note that the decoder 217 may skip decoding of data for a camera angle different from that currently selected by the client. When it is known that the Vclick_AU corresponding to the time stamp of the moving picture whose playback is in progress has already been loaded to the meta data decoder 217, the transmission process of the Vclick stream VCS to the meta data decoder 217 may be skipped.
The time stamp of the moving picture whose playback is in progress is sequentially sent from the interface handler to the meta data decoder 217. The meta data decoder 217 decodes the Vclick_AU in synchronism with this time stamp, and sends required data to the AV renderer 218. For example, when attribute information described in the AU of the object meta data instructs to display an object region, the meta data decoder generates a mask image, contour, and the like of the object region, and sends them to the AV renderer 218 in synchronism with the time stamp of the moving picture whose playback is in progress. The meta data decoder 217 compares the time stamp of the moving picture whose playback is in progress with the lifetime of the Vclick_AU to determine old Vclick_AU which is not required and to delete that data.
If the user inputs a playback stop instruction during playback of the moving picture, the interface handler 207 outputs a moving picture playback stop command and a read stop command of the Vclick stream VCS to the controller 205. With these commands, the moving picture playback process ends.
(Random Access Procedure (Local))
The random access playback procedures when a Vclick stream VCS is present on the moving picture data recording medium 231 will be described below.
In step S4301, a process for specifying a Vclick stream VCS to be used is executed. In this process, the interface handler refers to the Vclick information file VCI on the moving picture data recording medium 231 and specifies a Vclick stream VCS corresponding to the moving picture to be played back designated by the user. Furthermore, the interface handler refers to the Vclick access table VCA on the moving picture data recording medium 231 or that loaded on a memory (the buffer 209 or another work memory area), and specifies an access point in the Vclick stream VCS corresponding to the random access destination of the moving picture.
Step S4302 is a branch process that checks if the specified Vclick stream VCS is currently loaded onto the buffer 209. If the specified Vclick stream is not loaded onto the buffer, the flow advances to step S4304 after a process in step S4303. If the specified Vclick stream is currently loaded onto the buffer, the flow advances to step S4304 while skipping the process in step S4303. In step S4304, random access playback of the moving picture and decoding of the Vclick stream VCS start. In this process, the interface handler 207 issues a moving picture random access playback command to the moving picture playback controller 205, and simultaneously outputs, to the meta data manager 210, a command to start output of the Vclick stream VCS to the meta data decoder 217. After that, the decoding process of the Vclick stream VCS is executed in synchronism with playback of the moving picture. Since the processes during playback of the moving picture and moving picture playback stop process are the same as those in the normal playback process, a description thereof will be omitted.
(Procedure from Clicking Until Related Information Display)
The operation of the client executed when the user has clicked a position within an object region using a pointing device such as a mouse or the like will be described below. When the user has clicked a given position, the clicked coordinate position on the moving picture is input to the interface handler 207. The interface handler 207 sends the time stamp and coordinate position of the moving picture upon clicking to the meta data decoder 217. The meta data decoder 217 executes a process for specifying an object designated by the user on the basis of the time stamp and coordinate position. Since the meta data decoder 217 decodes a Vclick stream VCS in synchronism with playback of the moving picture, and has already generated the region of the object at the time stamp upon clicking, it can easily implement this process. When a plurality of object regions are present at the clicked coordinate position, the frontmost object is specified with reference to layer information included in a Vclick_AU.
After the object designated by the user is specified, the meta data decoder 217 sends an action description (a script that designates an action) described in the object attribute information 403 to the script interpreter 212. Upon reception of the action description, the script interpreter 212 interprets the action contents and executes an action. For example, the script interpreter displays a designated HTML file or begins to play back a designated moving picture. These HTML file and moving picture data may be recorded on the client 200, may be sent from the server 201 via the network, or may be present on another server on the network.
(Detailed Data Structure)
Configuration examples of practical data structures will be explained below.
vcs_start_code indicates the start of a Vclick stream VCS;
data_length designates the data length of a field after data_length in this Vclick stream VCS using bytes as a unit; and
data_bytes corresponds to a data field of a Vclick_AU. This field includes the header 507 of the Vclick stream 506 at the head position, and one or a plurality of Vclick_AUs (
vcs_header_code indicates the start of the header (507) of the Vclick stream VCS (506);
data_length designates the data length of a field after data_length in the header of the Vclick stream VCS using bytes as a unit;
vclick version designates the version of the format. This value assumes 01h in this specification; and
bit_rate designates a maximum bit rate of this Vclick stream VCS.
vclick_start_code indicates the start of each Vclick_AU;
data_length designates the data length of a field after data_length in this Vclick AU using bytes as a unit; and
data_bytes corresponds a data field of the Vclick_AU. This field includes the header 401, time stamp 402, object attribute information 403, and object region information 400.
vclick_header_code indicates the start of the header of each Vclick_AU;
data_length designates the data length of a field after data_length in the header of this Vclick_AU using bytes as a unit;
filtering_id is an ID used to identify the Vclick_AU. This data is used to determine the Vclick_AU to be decoded on the basis of the attributes of the client and this ID;
object_id is an identification number of an object described in Vclick data. When the same object_id value is used in two Vclick_AUs, they are data for a semantically identical object;
object_subid represents semantic continuity of objects. When two Vclick_AUs include the same object_id and object_subid values, they mean continuous objects;
continue_flag is a flag. If this flag is “1”, an object region described in this Vclick_AU is continuous to that described in the next Vclick_AU having the same object_id. Otherwise, this flag is “0”; and
layer represents a layer value of an object. As the layer value is larger, this means that an object is located on the front side on the screen. As described above, since “the Vclick_AU to be decoded” can be determined based on filtering_id, “the Vclick stream VCS including the Vclick_AU to be decoded” can also be identified based on filtering_id. That is, “stream selection of moving picture meta data” can be made using filtering_id.
time_type indicates the start of a DVD time stamp;
data_length designates the data length of a field after data_length in this time stamp using bytes as a unit;
VTSN indicates a VTS (video title set) number of DVD-Video;
TTN indicates a title number in the title domain of DVD-Video. This number corresponds to a value stored in the system parameter SPRM(4) of a DVD player;
VTS_TTN indicates a VTS title number in the title domain of DVD-Video. This number corresponds to a value stored in the system parameter SPRM(5) of the DVD player;
TT_PGCN indicates a title PGC (program chain) number in the title domain of DVD-Video. This number corresponds to a value stored in the system parameter SPRM(6) of the DVD player;
PTTN indicates a part-of-title (Part_of_Title) number of DVD-Video. This number corresponds to a value stored in the system parameter SPRM(7) of the DVD player;
CN indicates a cell number of DVD-Video;
AGLN indicates an angle number of DVD-Video; and
PTS[s . . . e] indicates data of s-th to e-th bits of the display time stamp of DVD-Video.
time_type indicates the start of the time stamp skip; and
data_length designates the data length of a field after data_length of this time stamp skip using bytes as a unit. However, this value always assumes “0” since the time stamp skip includes only time_type and data_length.
vca_start_code indicates the start of the object attribute information of each Vclick_AU;
data_length designates the data length of a field after data_length in this object attribute information using bytes as a unit; and
data_bytes corresponds to a data field of the object attribute information. This field describes one or a plurality of attributes.
Details of attribute information described in the object attribute information 403 will be described below.
attribute_id is an ID included in each attribute data, and is data used to identify the type of attribute. A name attribute is information used to specify the object name. An action attribute describes an action to be taken upon clicking an object region in a moving picture. A contour attribute indicates a display method of an object contour. A blinking region attribute specifies a blinking color upon blinking an object region. A mosaic region attribute describes a mosaic conversion method upon applying mosaic conversion to an object region, and displaying the converted region. A paint region attribute specifies a color upon painting and displaying an object region.
Attributes which belong to a text category define attributes associated with characters to be displayed when characters are to be displayed on a moving picture. Text information describes text to be displayed. A text attribute specifies attributes such as a color, font, and the like of text to be displayed. A highlight effect attribute specifies a highlight display method of characters upon highlighting partial or whole text. A blinking effect attribute specifies a blinking display method of characters upon blinking partial or whole text. A scroll effect attribute describes a scroll direction and speed upon scrolling text to be displayed. A karaoke effect attribute specifies the change timing and position of characters upon changing a text color sequentially.
Finally, a layer extension attribute is used to define the change timing and value of a change in layer value when the layer value of an object changes in the Vclick_AU. The data structures of the aforementioned attributes will be individually explained below.
attribute_id designates a type of attribute data. The name attribute has attribute_id=00h;
data_length indicates the data length after data_length of the name attribute data using bytes as a unit;
language specifies a language used to describe the following elements (name and annotation). A language is designated using ISO-639 “code for the representation of names of languages”;
name_length designates the data length of a name element using bytes as a unit;
name is a character string, which represents the name of an object described in this Vclick_AU;
annotation_length represents the data length of an annotation element using bytes as a unit; and
annotation is a character string, which represents an annotation associated with an object described in this Vclick_AU.
attribute_id designates a type of attribute data. The action attribute has attribute_id=01h;
data_length indicates the data length of a field after data_length of the action attribute data using bytes as a unit;
script_language specifies a type of script language described in a script element;
script_length represents the data length of the script element using bytes as a unit; and
script is a character string which describes an action to be executed using the script language designated by script_language when the user designates an object described in this Vclick_AU.
attribute_id designates a type of attribute data. The contour attribute has attribute_id=02h;
data_length indicates the data length of a field after data_length of the contour attribute data;
color_r, color_g, color_b, and color_a designate a display color of the contour of an object described in this object meta data AU;
color_r, color_g, and color_b respectively designate red, green, and blue values in RGB expression of the color. color_a indicates transparency;
line_type designates the type of contour (solid line, broken line, or the like) of an object described in this Vclick_AU; and
thickness designates the thickness of the contour of an object described in this Vclick_AU using points as a unit.
attribute_id designates a type of attribute data. The blinking region attribute data has attribute_id=03h;
data_length indicates the data length of a field after data_length of the blinking region attribute data using bytes as a unit;
color_r, color_g, color_b, and color_a designate a display color of a region of an object described in this Vclick_AU. color_r, color_g, and color_b respectively designate red, green, and blue values in RGB expression of the color. color_a indicates transparency. Blinking of an object region is realized by alternately displaying the color designated in the paint region attribute and that designated in this attribute; and
interval designates the blinking time interval.
attribute_id designates a type of attribute data. The mosaic region attribute data has attribute_id=04h;
data_length indicates the data length of a field after data_length of the mosaic region attribute data using bytes as a unit;
mosaic_size designates the size of a mosaic block using pixels as a unit; and
randomness represents a degree of randomness upon replacing mosaic-converted block positions.
attribute_id designates a type of attribute data. The paint region attribute data has attribute_id=05h;
data_length indicates the data length of a field after data_length of the paint region attribute data using bytes as a unit; and
color_r, color_g, color_b, and color_a designate a display color of a region of an object described in this Vclick_AU. color_r, color_g, and color_b respectively designate red, green, and blue values in RGB expression of the color. color_a indicates transparency.
attribute_id designates a type of attribute data. The text information of an object has attribute_id=06h;
data_length indicates the data length of a field after data_length of the text information of an object using bytes as a unit;
language indicates a language of described text. A method of designating a language can use ISO-639 “code for the representation of names of languages”;
char_code specifies a code type of text. For example, UTF-8, UTF-16, ASCII, Shift JIS, and the like are used to designate the code type;
direction specifies a left, right, up, or down direction as a direction upon arranging characters. For example, in case of English or French, characters are normally arranged in the left direction. On the other hand, in case of Arabic, characters are arranged in the right direction. In case of Japanese, characters are arranged in either the left or down direction. However, an arrangement direction other than that determined for each language may be designated. Also, an oblique direction may be designated;
text_length designates the length of timed text using bytes as a unit; and
text is a character string, which is text described using the character code designated by char_code.
attribute_id designates a type of attribute data. The text attribute of an object has attribute_id=07h;
data_length indicates the data length of a field after data_length of the text attribute of an object using bytes as a unit;
font_length designates the description length of font using bytes as a unit;
font is a character string, which designates font used upon displaying text; and
color_r, color_g, color_b, and color_a designate a display color of text. A color is designated by RGB. color_r, color_g, and color_b respectively designate red, green, and blue values. color_a indicates transparency.
attribute_id designates a type of attribute data. The text highlight effect attribute data of an object has attribute_id=08h;
data_length indicates the data length of a field after data_length of the text highlight effect attribute data of an object using bytes as a unit;
entry indicates the number of “highlight_effect_entry”s in this text highlight effect attribute data; and
data_bytes includes “highlight_effect_entry”s as many as entry.
The specification of highlight_effect_entry is as follows.
start_position designates the start position of a character to be highlighted using the number of characters from the head to that character;
end_position designates the end position of a character to be highlighted using the number of characters from the head to that character; and
color_r, color_g, color_b, and color_a designate a display color of the highlighted characters. A color is expressed by RGB. color_r, color_g, and color_b respectively designate red, green, and blue values. color_a indicates transparency.
attribute_id designates a type of attribute data. The text blinking effect attribute data of an object has attribute_id=09h;
data_length indicates the data length of a field after data_length of the text blinking effect attribute data using bytes as a unit;
entry indicates the number of “blink_effect_entry”s in this text blinking effect attribute data; and
data_bytes includes “blink_effect_entry”s as many as entry.
The specification of blink_effect_entry is as follows.
start_position designates the start position of a character to be blinked using the number of characters from the head to that character;
end_position designates the end position of a character to be blinked using the number of characters from the head to that character;
color_r, color_g, color_b, and color_a designate a display color of the blinking characters. A color is expressed by RGB. color_r, color_g, and color_b respectively designate red, green, and blue values. color_a indicates transparency. Note that characters are blinked by alternately displaying the color designated by this entry and the color designated by the text attribute; and
interval designates the blinking time interval.
attribute_id designates a type of attribute data. The text scroll effect attribute data of an object has attribute_id=0ah;
data_length indicates the data length of a field after data_length of the text scroll effect attribute data using bytes as a unit;
direction designates a direction to scroll characters. For example, 0 indicates a direction from right to left, 1 indicates a direction from left to right, 2 indicates a direction from up to down, and 3 indicates a direction from down to up; and
delay designates a scroll speed by a time difference from when the first character to be displayed appears until the last character appears.
attribute_id designates a type of attribute data. The text karaoke effect attribute data of an object has attribute_id=0bh;
data_length indicates the data length of a field after data_length of the text karaoke effect attribute data using bytes as a unit;
start_time designates a change start time of a text color of a character string designated by first karaoke_effect_entry included in data_bytes of this attribute data;
entry indicates the number of “karaoke_effect_entry”s in this text karaoke effect attribute data; and
data_bytes includes “karaoke_effect_entry”s as many as entry.
The specification of karaoke_effect_entry is as follows.
end_time indicates a change end time of the text color of a character string designated by this entry. If another entry follows this entry, end_time also indicates a change start time of the text color of a character string designated by the next entry;
start_position designates the start position of a first character whose text color is to be changed using the number of characters from the head to that character; and
end_position designates the end position of a last character whose text color is to be changed using the number of characters from the head to that character.
attribute_id designates a type of attribute data. The layer extension attribute data of an object has attribute_id=0ch;
data_length indicates the data length of a field after data_length of the layer extension attribute data using bytes as a unit;
start_time designates a start time at which the layer value designated by the first layer_extension_entry included in data_bytes of this attribute data is enabled;
entry designates the number of “layer_extension_entry”s included in this layer extension attribute data; and
data_bytes includes “layer_extension_entry”s as many as entry.
The specification of layer_extension_entry will be described below.
end_time designates a time at which the layer value designated by this layer_extension_entry is disabled. If another entry follows this entry, end_time also indicates a start time at which the layer value designated by the next entry is enabled; and
layer designates the layer value of an object.
vcr_start_code means the start of object region data;
data_length designates the data length of a field after data_length of the object region data using bytes as a unit; and
data_bytes is a data field that describes an object region. The object region can be described using, e.g., the binary format of MPEG-7 SpatioTemporalLocator.
(Application Image)
(Lifetime Designation Method of Vclick_AU using Duration Data)
An advantage offered when the duration is also described as data used to specify the Vclick_AU lies in that the duration of the Vclick_AU can be detected by checking only Vclick_AU to be processed. When valid Vclick_AUs with a given time stamp are to be found, it is checked without checking other Vclick_AU data if the Vclick_AU of interest is to be found. However, the data size increases by the duration B02 compared to
If the Vclick stream VCS is played back in synchronism with playback of the DVD-Video contents (a loop of NO in step S8201), the user makes some input (YES in step S8201). Note that the user's input includes a change in playback attribute such as switching of audio, switching of a caption, switching of an angle, and the like, or the start of special playback such as fast forward, fast reverse, skip, and the like.
If the user's input indicates a change in playback attribute (branch to the “change in playback attribute” side in step S8202), the playback apparatus (client 200 in
If the Vclick_AU to be played back cannot be found in the Vclick stream VCS whose playback is in progress (NO in step S8205), a Vclick stream VCS including the corresponding Vclick_AU must be played back. In order to search for this Vclick stream VCS, the Vclick Info VCI file (VCKINDEX.IFO in
If the Vclick Info VCI file is not mapped on the memory (the buffer 209 or the like) of the playback apparatus, the Vclick Info VCI file is loaded from the predetermined location (in the VCD in
If no Vclick stream VCS corresponding to the changed playback state is present on the Vclick Info VCI file (NO in step S8208), playback of the Vclick stream VCS is stopped, and that of the DVD-Video contents is continued.
If the Vclick stream VCS corresponding to the changed playback state is found using the Vclick Info VCI file (YES in step S8208), and that Vclick stream VCS is located on the external server (201) (branch to the “server” side in step S8209), the Vclick stream VCS stored in the buffer 209 before the playback state has changed is erased (step S8220), and the new Vclick stream VCS on the external server (201) is loaded onto that buffer 209 (step S8221). After that, playback of the Vclick stream VCS starts (“start of playback of new Vclick stream”). At this time, since playback of the DVD-Video contents is continued, the Vclick stream VCS is played back in synchronism with the DVD-Video contents whose playback is now in progress with reference to the time information of that DVD-Video contents.
If the Vclick stream VCS corresponding to the changed playback state is found (YES in step S8208), and that Vclick stream VCS is located on the disc (branch to the “disc” side in step S8209), the Vclick stream VCS stored in the buffer 209 before the playback state has changed is erased (step S8210), and the new Vclick stream VCS on the disc is loaded onto that buffer 209 (step S8211). In this case, if the loading speed of the DVD drive is sufficiently high, the Vclick stream VCS can be loaded while continuing playback of the DVD-Video contents whose playback is now in progress. However, if the loading speed of the DVD drive is insufficient, playback of the DVD-Video contents is paused, and the Vclick stream VCS is loaded from the disc. After that, synchronous playback of the DVD-Video contents and Vclick stream VCS starts with reference to the time information of the DVD-Video contents whose playback is to start (step S8212 and “start of playback of new Vclick stream” after step S8212).
If the user input indicates the start of special playback (branch to the “special playback” side in step S8202), it is checked if a Vclick stream VCS as the jump destination of the special playback is the same as that which is currently decoded. If the Vclick stream VCS as the jump destination is the same as that which is currently decoded (YES in step S8203), the special playback is continued while holding the same Vclick stream VCS on the buffer 209. If the Vclick stream VCS as the jump destination is not the same as that which is currently decoded (NO in step S8203), the flow jumps to the process in step S8207.
For example, when the user inputs the above special playback request, the playback apparatus begins to synchronously play back the DVD-Video contents and Vclick stream VCS at the designated playback position when the next designated playback position (a position ahead of the current position in case of fast forward playback, a position behind the current position in case of fast reverse playback, the designated position in case of skip) is located in the Vclick stream VCS whose playback is now in progress.
If the designated playback position is not present in the Vclick stream VCS whose playback is now in progress, a Vclick stream VCS including the designated playback position must be played back. In order to search for this Vclick stream VCS, the Vclick Info VCI file is used (step S8207). This Vclick Info VCI file is mapped on a memory (buffer 209) of the playback apparatus in advance, and the location and file name information of the corresponding Vclick stream VCS can be acquired by giving the playback position information. If the Vclick Info VCI file is not mapped on the memory of the playback apparatus, it is loaded from the predetermined location on the disc or the predetermined location on the external server (201), and is mapped on the memory of the playback apparatus (steps S8210 and S8211 or steps S8220 and S8821). When the Vclick Info VCI file is loaded from the predetermined location on the disc, playback of the DVD-Video contents which are currently played back is stopped, as needed, and the desired Vclick Info VCI file is loaded.
More specifically, if the Vclick stream VCS corresponding to the designated playback position is found using the Vclick Info VCI file (YES in step S8208), and that Vclick stream VCS is located on the external server (201), the Vclick stream VCS stored in the buffer 209 before the special playback is erased (step S8220), and the new Vclick stream VCS on the external server (201) is loaded onto that buffer 209 (step S8221). After that, playback of the Vclick stream VCS starts. At this time, since playback of the DVD-Video contents is continued, the Vclick stream VCS is played back in synchronism with the DVD-Video contents whose playback is now in progress with reference to the time information of that DVD-Video contents.
On the other hand, if the changed Vclick stream VCS is located on the disc, the Vclick stream VCS stored in the buffer 209 before the special playback is erased (step S8210), and the new Vclick stream VCS on the disc is loaded onto that buffer 209 (step S8211). In this case, if the loading speed of the DVD drive is sufficiently high, the Vclick stream VCS can be loaded while continuing playback of the DVD-Video contents whose playback is now in progress. However, otherwise, playback of the DVD-Video contents is paused, and the Vclick stream VCS is loaded from the disc. After that, synchronous playback of the DVD-Video contents and Vclick stream VCS starts with reference to the time information of the DVD-Video contents whose playback is to start (after step S8212).
If no Vclick stream VCS corresponding to the designated playback position is present on the Vclick Info VCI file (NO in step S8208), playback of the Vclick stream VCS is stopped, and that of the DVD-Video contents is continued.
The Vclick Info VCI file (VCKINDEX.IFO in
Upon buffer storage, some methods of determining the order of buffering of a plurality of streams (836 to 838 in
In the second example, the priority of loading is determined in accordance with a language pre-set in a player (the client 200). For example, assume that Japanese is set in the player as the first language (or default language), and English is set as the second language. In this case, when an English attribute (e.g., English is designated by “language” in
In the third example, streams to be loaded, which are recorded on the disk, are given priority over those recorded on the external server. For example, when Vclick stream #2 is recorded on the external server, and Vclick streams #1 and #3 are recorded on the disk, Vclick streams #1 and #3 are loaded onto the buffer 209 in the order of their numbers, and Vclick stream #2 is then loaded onto the buffer 209 upon loading Vclick streams #1 to #3.
After the Vclick stream is stored up to a size assigned to the buffer 209 (for example, 2 Mbytes in the example of
After playback of the DVD-Video contents has started, its playback information (title, PGC number, audio stream number, sub-picture stream number, angle number, aspect ratio information, playback time information, and the like) is acquired (step S8406), and a corresponding Vclick stream is retrieved from the Vclick Info file on the basis of at least some pieces (e.g., PGC number) of acquired information (step S8407). For example, if the playback information acquired in step S8406 is PGC#1 in
If data of the Vclick streams #1 to #3 are not currently stored in the buffer 209 (NO in step S8408), data of the retrieved Vclick streams #1 to #3 are loaded onto the buffer 209 (step S8410) by deleting unnecessary data from the buffer 209 (step S8409), or by overwriting the buffer area of unnecessary data. If the data size to be loaded onto the buffer 209 has reached a minimum playback size (e.g., 12 kbytes), playback of the buffered Vclick streams starts in synchronism with the current DVD playback (step S8405).
Upon loading onto the buffer 209 in steps S8403 and S8410, a “NULL_AU” field shown in
The method of eliminating unwanted data transfer from the buffer 209 to the media decoder 216 and the overhead on the process of the media decoder 216, which is produced accordingly, by arranging the “NULL_AU” on the Vclick stream VCS, has been described. However, when an AU on the Vclick stream VCS is accessed using the Vclick access table VCA, unwanted overhead on the process can be avoided using “Null Pointer” on the Vclick access table VCA in place of the “NULL_AU” on the Vclick stream VCS. This method will be described below.
“NULL” in the access point 851 in
time*<=T<time#4
At this time, the meta data manager 210 searches the Vclick access table VCA in
If the found h is not “NULL” (if t obtained in step S8602 corresponds to “time#4” in the example of
Next, the file pointer fp is changed to a value as the sum of the current fp and the current AU size (step S8607). It is then checked if the changed fp points to the end of the Vclick Info file. If fp does not point to the end of the file (NO in step S8608), the interface handler 207 reads a time stamp value u of the current AU (step S8609). If the read time stamp value u is not larger than the time stamp value t′ (NO in step S8610), the flow returns to step S8607 to repeat the processes in steps S8607 to S8610. If u>t′ (YES in step S8610) or if fp points to the end of the file (YES in step S8608), the interface handler 207 issues a command to the buffer 209 to “transfer data from the offset h to the file pointer fp of the object meta data stream (the Vclick stream VCS including the current AU) to the meta decoder 217” (step S8611). In this manner, the operation example (
In order to define the buffer size of the ring buffer 322 (209 in
The initialization document 320 can be stored in Vclick data VCD (more specifically, in the Vclick Info VCI in, e.g.,
<1> The assign size 321 and playback start size 323 are read out from the initialization document 320 in
<2> It is checked if a stream is to be downloaded from the network (step S8802).
<3> If a stream is to be downloaded from the network (YES in step S8802), the current line state of the network is detected (step S8803).
<4> The buffer assign size is adjusted depending on the detected line state (step S8804). For example, the Internet line is busy and not easy to be connected depending on the use time band. At this time, all the contents of the buffer may be read out before the contents of the buffer are updated by those which are downloaded from the network. In this case, playback of the Vclick stream may be interrupted. In such situation, some buffer sizes according to the line states may be determined on the basis of previous empirical data. For example, if the line is always connected, the buffer size is set to be 2 Mbytes; if the line is not easy to be connected, it is set to be 8 Mbytes; and in the intermediate state, it is set to be 4 Mbytes. When 8 Mbytes are assigned to the buffer size, if the line is connected, Vclick data are fully stored in the buffer using 8 Mbytes. As a result, since the size of Vclick data to be buffered is large after the line is disconnected once until the line is connected again, playback of the Vclick stream can be prevented from being interrupted.
<5> If a stream is not to be downloaded (NO in step S8802), the assign size described in the initialization document 320 is used as the assign size (step S8805) upon assigning the buffer 322.
<6> The buffer 322 is assigned by the assign size (step S8806).
As described above, the process for checking if the assign size described in the initialization document 320 is used or the assign size is adjusted depending on the line state can be executed. When the assign size has been adjusted in accordance with on the line state (change the size of 321 in
In this way, the “changed assign size” need not be increased in consideration of “storage contents size at the playback start timing” (e.g., when the buffer assign size is changed from 4 Mbytes to 2 Mbytes in accordance with the line state, if 3 Mbytes are designated as the playback start size at that time, the actual buffer assign size is set to be 3 Mbytes or more by increasing 1 Mbyte). That is, in this case, the playback start size 323 in
Also, only the assign size at the playback start timing can be similarly adjusted depending on the network state. More specifically, the description of the playback start size (playback start timing determination definition) 323 in
Immediately after data write onto the buffer 322 (or 209) starts, playback does not start until data is stored in the buffer 322 by a size (323) described in the playback start timing determination definition 323 in the initialization document 320 shown in
When the time after the server-client connection is established is used, that time is determined based on the download speed and the size of the buffer 322 (or 209). This time allows to download contents (Vclick data) up to a size that does not interrupt playback even when re-connection is required due to an error such as line disconnection or the like, and when the contents stored in the buffer 322 are used and downloading starts by establishing re-connection. This time is defined and determined by the player side. In this case, since the player side defines the time, an optimal time to each throughput according to an environment can be defined.
When the time is determined based on the contents storage size in the buffer 322 (or 209), the contents storage size can be defined as follows. That is, the aforementioned time is defined by the storage size that stores contents as large as playback is not interrupted in the buffer 322 until the contents stored in the buffer 322 are used and downloading starts by establishing re-connection, even when re-connection is required due to an error such as line disconnection or the like.
When the standard (Enhanced DVD standard or EDVD standard) applied to this embodiment specifies a lowest throughput (e.g., 128 kbps), “defining the contents as large as playback is not interrupted by the storage size of the buffer 322” can be accomplished in the contents playback start timing in addition to the definition on the apparatus (player) side.
After that, “data write” and “read for playing back data” are parallelly executed (a playback initial state 325 in
In
In such use method, in, e.g., the playback start waiting state 324 and playback initial state 325 in
The aforementioned ring buffer 322 is applied to the buffer 209 in
A special buffer model will be explained below with reference to FIGS. 94 to 96. In this model, a gap is formed between the “current reception complete position 331” and “current playback position 332” to “hold past data before the current playback position” on purpose. In this way, the availability or response to the user operation upon special playback is improved. Assume that there are Vclick_AUs at 0.5-ms intervals for two hours. At this time, when two target objects are always displayed on the screen, the size of the Vclick stream VCS is about 7 Mbytes. Also, if the assign size of the buffer 322 (209 in
If the same condition as in the above example is used,
In the example of
(a) at the timing of buffer generation based on the initialization document;
(b) at the timing of a jump operation of a video picture whose playback is in progress to the future or past (if no video data is stored on the buffer);
(c) at the timing of line disconnection (intentional line disconnection such as a change in stream acquisition destination, or the like) . . . clear the buffer after connection is closed.
In other words, when a data access other than that to the buffer (209 in
The HTTP protocol has the property that data are sent in the order of issuance of HTTP:GET methods, and the data output timings are not re-ordered based on the method issuance timings if the connection is kept (an explanation of Keep alive). This property is called the Keep Alive property. Using this Keep Alive property, it becomes possible to execute the following operations. That is, according to the Keep Alive property, information is neither interrupted nor re-ordered unless the buffer is flashed. If an HTTP:GET method is issued every time an event occurs, Vclick streams VCS are downloaded in turn (
This problem can be solved by programming an application (control firmware) to make skip reading of meta data (Vclick data) in buffering upon fast forward or fast reverse playback. For example, the data size upon buffering data for 10 sec at 0.1-sec intervals is equal to that upon buffering data for 100 sec at 1-sec intervals. In this case, even when meta data is read out from the buffer 209 in synchronism with 10×-speed FF/FR playback, the data consumption rate of the buffer 209 is apparently equal to that upon 1×-speed normal 15. playback without skip reading in terms of playback time base. In this manner, the aforementioned problem can be solved if it is devised to make buffering of meta data to the buffer 209 catch up with the fast DVD playback speed.
In
Assume that Vclick streams VCS are stored on the moving picture data recording medium 231. Since seamless playback is required, seek to the moving picture data recording medium 231 must be prevented from being generated. For this purpose, the meta data manager 210 must load all Vclick streams VCS before moving picture playback in accordance with an instruction from the interface handler 207 (
More specifically, the interface handler 207 instructs the disc device manager 213 to acquire Vclick streams VCS (357). The disc device manager 213 acquires Vclick streams VCS for the designated byte size from the moving picture data recording medium 231 and records them on the buffer 209 (358). After the streams are recorded on the buffer 209, the disc device manager 213 transmits a Vclick stream acquisition completion response 359 to the interface handler 207. The interface handler 207 notifies the meta data manager 210 of a Vclick stream acquisition completion response 360. The interface handler 207 plays back a moving picture after transmission of the Vclick stream acquisition completion response 360.
Contrary to the case of
In the aforementioned embodiment, the use method of the buffer 209 can be optimized by the initialization document 320 in
Furthermore, when the user makes special playback such as fast forward, fast reverse, jump playback, shuffle playback, random playback, and the like, meta data for, e.g., a total of 34 minutes can be freely assigned to the buffer by adopting the buffer models shown in FIGS. 94 to 96, and seamless playback of Vclick data together with a moving picture can be guaranteed.
Note that the present invention is not limited to the aforementioned embodiments intact, and various modifications of constituent elements may be made without departing from the scope of the invention when it is practiced. For example, the present invention can be applied not only to widespread DVD-ROM video, but also to DVD-VR (video recorder) whose demand is increasing rapidly in recent years and which allows recording/playback. Furthermore, the present invention can be applied to a playback or recording/playback system of next-generation HD-DVD, which will be prevalent soon.
The buffer 209 shown in
Additional advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described herein. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2004-136688 | Apr 2004 | JP | national |
