Patent Application
20070263982
The present invention relates to a system of resuming a reproduction from a reproduction interruption position, and resuming a reproduction from an arbitrary position specified by an operator (i.e. resume reproduction) which is performed independently by a device, or between devices, for reproducing a moving image file in an arbitrary format, and particularly, to a system of resuming a reproduction when using a terminal.
Conventionally, proposals have been made as to a method of reproducing a moving image file from an arbitrary intermediate position. For example, a method is disclosed which performs a reproduction from an arbitrary intermediate frame by presetting frame information for dynamic image encoded data and several frames of the dynamic image encoded data (for example, see Patent Document 1). A description will be given of the reproduction from the intermediate frame, assuming that the dynamic image encoded data is data A, and the frame information is data B. In the frame information, there are described reference image data required when reproducing the intermediate frame in the dynamic image encoded data, and an address indicating at what position in the data A encoded data of the intermediate frame is stored.
The frame information is set for several frames among all frames composing the dynamic image encoded data. The frames for which the frame information is set can be selected arbitrarily, and a random access becomes possible for the frame selected arbitrarily. Accordingly, as a result, arbitrary frames can be randomly accessed.
The frame information may have a different composition.
Patent Document 1: Japanese Patent Application Laid-Open No. 7-203358
According to the method disclosed in the above-mentioned Patent Document 1, it is necessary to preset the frame information which includes all the frames of the dynamic image encoded data. The frame information contains a plurality of reference image data, and addresses of encoded data following the reference image data, or the encoded data per se. In other words, the frame information in Patent Document 1 is information obtained by extracting arbitrary portions of the dynamic image encoded data into a form of a file. Such a method necessitates man-hour for presetting the frame information, and a storage area for storing the frame information.
By the way, as the number of frames in the dynamic image encoded data increases, i.e., as a reproduction time becomes longer, the frame information increases, necessitating a large amount of storage area. Besides, according to the above-mentioned method, an operator needs to specify an intermediate frame for each reproduction from an arbitrary intermediate position of the dynamic image encoded data.
It is an object of the present invention to provide technique for performing a smooth reproduction from a previous stop position at which a reproduction of dynamic image encoded data is stopped last time during the reproduction, when reproducing the same dynamic image encoded data next time.
Resume reproduction technique according to the present invention includes a reproduction resuming function of storing a position at which a moving image file is stopped during a reproduction, or a position specified arbitrarily, and thereby performing a reproduction automatically from the previous stop position when reproducing the same moving image file next time. The reproduction resuming function is hereinafter referred to as “resume reproduction function.”
That is, the present invention includes in a terminal for reproducing a moving image file, a GUI part for an operator to perform a reproduction operation, a storage part for storing a moving image file composed of different types of data, such as image, audio and telop, a reproduction information file in which control information concerning a reproduction operation is described, and a reproduction processing reference file of an arbitrary number of frames upon a reproduction stop, a moving image file processing part for performing the reproduction process for the moving image file, a reproduction information file processing part for performing a reading/writing and an analysis of the reproduction information file, and an operation managing part for controlling the moving image file processing part and the reproduction information file processing part.
Accordingly, a resume reproduction is made possible which resumes a reproduction from a position at which an operator has interrupted the reproduction, or from a position which the operator has specified arbitrarily. Besides, a creation of a collateral setting file is made unnecessary for reproducing a moving image file from an arbitrary intermediate position.
According to a resume reproduction system of the present invention, reproduction of a moving image file can be resumed from an arbitrary reproduction interruption position and an arbitrary position. Besides, since reproduction control information and reproduction process reference data necessary for the resume reproduction are in the form of files, the reproduction resuming function can be performed independently by a terminal, and between different terminals, including the resume reproduction system. Accordingly, there are advantages that the operation becomes easier, and that a degree of freedom for the operation improves.
Hereinafter, a description will be given, with reference to the drawings, of a resume reproduction system according to an embodiment of the present invention.
The reproduction information file processing part 15 includes an analyzing part 16 for analyzing reproduction information. The storage part 17A is located within the terminal having the resume reproduction system, and includes storage areas for storing a moving image file 18A, a reproduction information file 19A, and a reproduction processing reference file 20A, respectively. The storage part 17B is located on an external network, and includes storage areas for storing a moving image file 18B, a reproduction information file 19B, and a reproduction processing reference file 20B, respectively, as does the storage part 17A.
Here, a description will be given, with reference to flowcharts shown in
In step S202, the operation managing part 12 informs the moving image file processing part 13 of the name of the moving image file acquired from the GUI part 11. The moving image file processing part 13 reads the corresponding moving image file (the moving image file) from the storage part 17A or the storage part 17B in step S203. Either the storage part 17A within the terminal or the storage part 17B on the external network is selectively used, normally by discriminating a pathname in specifying the file name.
For example, in an example of the present embodiment, the pathname is composed of a combination of “drive name”+“moving image file name” for the storage part 17A within the terminal, and a combination of “URL”+“moving image file name” is presented for the storage part 17B on the external network. For example, “c:¥resume¥douga_file.(extension)” implies the storage part 17A, and “http://www.resume.com/douga_file.(extension)” implies the storage part 17B. In the present embodiment, the description is given, assuming that a file within the storage part 17A is specified.
Next, a description will be given of the moving image file. The moving image file is normally composed of different types of data, such as image data, audio data, and telop data. To reduce a size of the file, an encoding process is often applied to the image data and the audio data so that the data is held in a compressed form. In this case, in order to reproduce the data, a decoding process needs to be performed. Encoding methods include a method of generating one basic frame data and difference frame data thereof, and a method of completing data for each frame.
The former is a method publicly known as an encoding method and a decoding method for image which use an interframe prediction, and includes a method, such as MPEG4,H.263, for an image, and a well-known method called AMR for audio. The method in which each frame is completed includes a method called Motion-JPEG and an encoding method called JEPG is used for each frame. Each frame can be considered as a basic frame.
The moving image file also includes an animation. According to a method called vector animation, there is a relation between one basic frame data and difference frame data, and a difference value for each coordinate value set in the basic frame data is recorded in the difference frame data. In the present embodiment, the description is given, using a method which is widespread as a format for a moving image file. That is, the description is premised on a case where image data and audio data are encoded using an interframe prediction, and the file format includes different types of data, such as the image data, the audio data, and text data (i.e. telop), being multiplexed. Hereinafter, the description is continued on the assumption that the moving image file is the above-mentioned multiplex file.
For processing a moving image file, a detailed configuration of the data processing part 14 is shown in
Here, a description will be given, with reference to
A separation process in separating part 141 means a process of searching the addresses of the minimum management units in the header division 41, retrieving the respective minimum management units of the data for image, audio and telop, and restoring a string of the image and audio stream data, and all characters of the telop, on a work memory.
Next, a description will be given of contents of the image stream data. The stream data is a string of data in frame units, and is composed of basic frame data, i.e., an I frame, and difference frame data, i.e., P frames.
The I frame is a frame including all data of information necessary for decoding one frame, and the P frames are frames including only difference data from an immediately preceding frame. That is, upon decoding image data, for example, a decoding process for one frame is first performed using the I frame, and subsequently, a decoding process for a following one frame is performed according to the decoded data of one frame, and a P frame including only the difference.
Next, in step S205, the moving image file processing part 13 transfers the corresponding data to each of the image processing part 142, the audio processing part 143, and the telop processing part 144, and performs a decoding process of the moving image file 18A. In step S206, the moving image file processing part 13 determines whether or not a reproduction time has expired by determining whether or not a time to reproduce predetermined in the moving image file 18A and an elapsed time for the decoding process coincide. In case of “YES,” the process flow proceeds to step S217 to stop the decoding process, and ends the reproduction of the moving image file 18A. In case of “NO,” the process flow proceeds to step S207.
In step S207, the operation managing part 12 determines whether or not a reproduction stop event or a reproduction information file generation instruction event has occurred according to an event from the operator. When the reproduction stop event or the reproduction information file generation instruction event has occurred (in case of “YES”), the process flow proceeds to step S208 to shift to a creation of a related file for performing a resume reproduction. When the reproduction stop event has not occurred (in case of “NO”), the process flow returns to step S204 to continue the separation process of the moving image file 18A.
In step S208, the operation managing part 12 discriminates whether or not a resume reproduction mode is specified according to the event from the operator. In case of “YES,” the process flow proceeds to step S209, and continues the creation of the related file for performing the resume reproduction. In case of “NO,” the process flow proceeds to step S218 to stop the decoding process, and ends the reproduction of the moving image file 18A (END).
In step S209, the image processing part 142 retrieves an arbitrary number of frame data including data of at least one frame from the moving image file, after the occurrence of the reproduction stop event or the reproduction information file generation instruction event. Then, a decoding process is performed for data of at least one frame among a plurality of the frame data, and a recoding process is performed subsequently. Next, a basic frame generated by the recoding process and the remaining frame data retrieved previously from the moving image file are coupled, and stored temporarily in the work memory within the terminal as reproduction process reference data. Of course, all of the arbitrary number of the frame data retrieved from the moving image file may be recoded.
In step S210, the moving image file processing part 13 stops all the decoding processes. Next, in step S211, the moving image file processing part 13 informs the operation managing part 12 of an elapsed time at which the decoding process is executed, i.e., a reproduction elapsed time, and a decoding process time for the last I frame executed during the decoding process, i.e., an I-frame processing time.
In step S212, the operation managing part 12 detects a difference between the reproduction elapsed time and the I-frame processing time, and, in step S213, determines whether or not a value of the difference is equal to or smaller than a predetermined threshold value. Here, the threshold value is a value with which it can be determined whether or not the resume reproduction is effective. When a determination result in step S213 indicates that the value of the difference is equal to or smaller than the threshold value, the process flow proceeds to step S215, and when the determination result in step S213 indicates that the value of the difference is equal to or larger than the threshold value, the process flow proceeds to step S214. In step S214, the moving image processing part 142 stores the reproduction process reference data stored in the above-mentioned work memory, as a reproduction processing reference file in the storage part 17A.
A description will be given, with reference to
In the hereinafter-described resume reproduction operation, it is determined whether or not to set a reproduction start position of an image to an I frame in the image stream data. However, when a reproduction is performed while setting the reproduction start position to the I frame, in a moving image file including the I frame only at a top of the file, or in a moving image file including the I frames at large intervals, an error becomes large between a position at which the operator stops the reproduction and a position at which the resume reproduction is actually started. Specifically, the difference between the reproduction elapsed time and the I-frame processing time becomes the error.
The above-mentioned threshold value indicates an allowable error between the position at which the operator stops the reproduction and the start position at which the resume reproduction is actually to be performed. When the error is equal to or smaller than the threshold value, the resume reproduction is performed from the I frame in the image stream data. When the error is larger than the threshold value, the resume reproduction is performed from the data of one frame created by the recoding process, i.e., the newly created I frame.
Accordingly, the error between the stop position intended by the operator and the actual reproduction start position can be minimized. Besides, unnecessary encoding processes and the creation of the file used only for the resume reproduction, i.e., the reproduction processing reference file, do not need to be performed. When the difference value in step S213 is larger than the threshold value, the process flow proceeds to step S214, and the moving image file processing part 13 records the reproduction processing reference file in the storage part 17A. When the difference value is equal to or smaller than the threshold value, the process flow proceeds to step S215.
In the present resume reproduction system, of course, the reproduction processing reference file may be recorded with no exception without performing the informing and determining process from step S211 to step S213. In this case, although the reproduction processing reference file necessarily uses an area in the storage part for its size, accuracy of the position to start the resume reproduction improves.
In step S215, the operation managing part 12 informs the reproduction information file processing part 15 of the name of the reproduced moving image file and the reproduction elapsed time. Next, in step S216, the reproduction information processing part 15 creates a reproduction information file according to the name of the moving image file and the reproduction elapsed time, and records the reproduction information file in the storage part 17A. Here, a description will be given, with reference to
Next, a description will be given, with reference to the flowcharts shown in
In step S301, the GUI part acquires a name of the moving image file to be reproduced. Then, in step S302, the operation managing part 12 discriminates whether or not the resume reproduction mode has been specified according to an event from the operator. When the resume reproduction mode has been specified, the procedure proceeds to step S304. When the resume reproduction mode is not specified, the procedure proceeds to step S303, and the operation managing part 12 informs the moving image file processing part 13 of the name of the moving image file to be reproduced. The moving image file processing part 13 reads the corresponding moving image file, and starts a decoding process from a top of the file. In step S304, the operation managing part 12 informs the reproduction information file processing part 15 of the name of the moving image file. In step S305, the reproduction information file processing part 15 reads the reproduction information file corresponding to the name of the moving image file.
Here, an example of a name of the reproduction information file corresponding to the name of the moving image file is shown. When the name of the moving image file is “SAMPLE.(extension A),” the reproduction information file is given a name “SAMPLE.(extension B)” to establish a correspondence between the two files. The extension A indicates a moving image file, and the extension B indicates a reproduction information file.
In step S306, the analyzing part 16 determines whether or not the process determination code described in the reproduction information file suits for the resume reproduction, and the name of the moving image file described in the reproduction information file coincides with the name of the moving image file informed of by the operation managing part 12. When the moving image file names coincide, the procedure proceeds to step S307. When the moving image file names do not coincide, the procedure proceeds to step S303, and the operation managing part 12 informs the moving image file processing part 13 of the name of the moving image file to be reproduced. The moving image file processing part 13 reads the corresponding moving image file, and starts a decoding process from the top of the file.
In step S307, the reproduction information file processing part 15 informs the operation managing part 12 of the reproduction start time described in the reproduction information file. In step S308, the operation managing part 12 informs the moving image file processing part 13 of the name of the moving image file to be reproduced and the reproduction start time. In step S309, the moving image file processing part 13 performs search to determine whether or not the reproduction processing reference file 20A corresponding to the name of the moving image file exists in the storage part 17A. When the reproduction processing reference file 20A exists in the storage part 17A, the procedure proceeds to step S311. When the reproduction processing reference file 20A does not exist in the storage part 17A, the procedure proceeds to step S310, and after reading the moving image file, a decoding process is started from an I frame located immediately before the reproduction start time.
In step S311, the moving image file processing part 13 reads the reproduction processing reference file.
In step S312, the image processing part 142 determines whether or not the process determination code in the reproduction processing reference file suits for the resume reproduction, and the name of the moving image file described in the reproduction processing reference file coincides with the name of the moving image file informed of by the operation managing part 12.
When the moving image file names coincide, the procedure proceeds to step S313. When the moving image file names do not coincide, the procedure proceeds to step S310, and after reading the moving image file, a decoding process is started from an I frame located immediately before the reproduction start time.
In step S313, the moving image file processing part 13 reads the moving image file, and further performs a decoding process for frame data in the reproduction processing reference file. In step S314, a decoding process is started from a P frame located at a position ahead of the position of the reproduction start time by a number of frames decoded in step S313.
In step S315, the moving image file processing part 13 decodes the moving image file while controlling the separating part 141, the image processing part 142, the audio processing part 143 and the telop processing part 144. In step S316, the moving image file processing part 13 determines whether or not a reproduction time has expired according to whether or not the time to reproduce predetermined in the moving image file and the elapsed time for the decoding process coincide. When the reproduction time has expired (in case of “YES”), the procedure proceeds to step S317, in which the moving image file processing part 13 stops all the decoding processes, and when the reproduction information file and the reproduction processing reference file corresponding to the moving image file exist, the moving image file processing part 13 deletes the reproduction information file and the reproduction processing reference file. After deleting the reproduction information file and the reproduction processing reference file, the present resume reproduction system ends all of the reproduction operation.
When the reproduction time has not expired (in case of “NO”), the procedure proceeds to step S318. In step S318, the operation managing part 12 determines whether or not a reproduction stop event or a reproduction information file generation instruction event has occurred according to the event from the operator. When the reproduction stop event or the reproduction information file generation instruction event has occurred (in case of “YES”), the procedure proceeds to step S319 to shift to a creation of a related file for performing the resume reproduction. When the reproduction stop event has not occurred (in case of “NO”), the procedure returns to step S315 to continue the separation process of the moving image file.
In step S319, the image processing part 142 retrieves an arbitrary number of frame data including data of at least one frame from the moving image file, after the occurrence of the reproduction interruption event or the reproduction information file generation instruction event. Then, a decoding process is performed for data of at least one frame among a plurality of the frame data, and a recoding process is performed subsequently.
Subsequently, a basic frame generated by the recoding process and the remaining frame data retrieved previously from the moving image file are coupled, and stored temporarily in the work memory within the terminal as reproduction process reference data. Of course, all of the arbitrary number of the frame data retrieved from the moving image file may be recoded.
In step S320, the moving image file processing part 13 stops all the decoding processes. Next, in step S321, the moving image file processing part 13 informs the operation managing part 12 of the elapsed time at which the decoding process is executed, i.e., the reproduction elapsed time, and the decoding process time for the last I frame executed during the decoding process, i.e., the I-frame processing time.
In step S322, the operation managing part 12 detects a difference between the reproduction elapsed time and the I-frame processing time, and, in step S323, determines whether or not a value of the difference is equal to or smaller than the predetermined threshold value. Here, the threshold value is a value with which it can be determined whether or not the resume reproduction is effective. When a determination result in step S323 indicates that the value of the difference is equal to or smaller than the threshold value, the procedure proceeds to step S325, and when the determination result in step S323 indicates that the value of the difference is equal to or larger than the threshold value, the procedure proceeds to step S324. In step S324, the moving image processing part 142 stores the reproduction process reference data stored in the above-mentioned work memory, as a reproduction processing reference file in the storage part 17A.
In step S325, the operation managing part 12 informs the reproduction information file processing part 15 of the name of the reproduced moving image file and the reproduction elapsed time. Next, in step S326, the reproduction information file processing part 15 overwrites the existing reproduction information file with a reproduction information file according to the name of the moving image file and the reproduction elapsed time, and records the reproduction information file in the storage part 17A.
The resume reproduction system ends all of the reproduction operation after recording the reproduction information file.
With reference to the above-described flowcharts, the description was given of the moving image file having the format in which different types of data, such as image data and audio data encoded using an interframe prediction, and text data (i.e. telop), are multiplexed. Here, descriptions will be given of cases with other file formats. In a moving image file having a format in which each image frame is completed, each frame is a basic frame, and therefore the generating process for the reproduction processing reference file is unnecessary. In the resume reproduction system, only a generation of a reproduction information file for specifying a reproduction resuming position of an arbitrary frame may be performed.
In a moving image file in a vector animation format, there is a relation between basic frame data and difference frame data as in the moving image file in the interframe prediction format, and therefore a reproduction resuming position depends on a position of a basic frame or a position of a reproduction process reference frame.
The reproduction process reference frame is generated by accumulating a difference value of each coordinate upon reproducing the difference frame data, and when the reproduction information file generation instruction event has occurred, reflecting the accumulated coordinate difference value on each coordinate value of the basic frame processed immediately before. Then, the generated reproduction process reference frame is stored as the reproduction processing reference file.
The above-described embodiment sets forth the method of performing a recoding process to generate a basic frame, in order to generate reproduction process reference data from image data using an interframe prediction. Of course, the recoding process is not the only method for generating the reproduction process reference data.
A decoded frame as is may also be used as reproduction process reference data as image data in YUV format or RGB format. In this case, since an encoding process is not performed, the reproduction process reference data has a large file size, but an image quality improves upon resuming a reproduction.
With reference to the above-described flowcharts, the description was given of the operation performed independently by the terminal including the resume reproduction system. However, the resume reproduction can be performed also by transferring the moving image file, the reproduction information file and the reproduction processing reference file in the storage part 17A to another terminal including a similar resume reproduction system.
As described above, by using the resume reproduction system according to the present embodiment, reproduction of a moving image file can be resumed from an arbitrary reproduction interruption position. Besides, since reproduction control information and reproduction process reference data necessary for the resume reproduction are in the form of files, the reproduction resuming function can be performed independently by the terminal including the resume reproduction system, or with another different terminal.
Further, upon the resume reproduction, an allowable error between a reproduction resuming position intended by the operator and an actual reproduction resuming position, which is predetermined as a threshold value, is used for determining a creation of the reproduction processing reference file. Accordingly, a moving image file can be reproduced from an arbitrary intermediate position without creating an unnecessary file.
The present invention is applicable to resume reproduction of various multiplexed data.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2003-374877 | Nov 2003 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP04/15237 | 10/15/2004 | WO | 3/9/2007 |
