Patent Application
20060168298
This application claims priority under 35 USC §119 to Japanese Patent Application No. 2004-366862 filed on Dec. 17, 2004, the entire contents of which are hereby incorporating by reference.
A portion of the disclosure of this patent document contains material, which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
1. Field of the Invention
The present invention relates to an animation reproduction apparatus that reproduces animation data, and in particular to a random video data reproduction technology used in a streaming environment.
2. Discussion of the Background Art
Multimedia data, such as an animation, etc., is becoming widely handled in a calculator due to considerably advantageous random accessing. For example, a user interface named a “seek bar” is employed in a conventional video reproduction software, such as Media Player manufactured by Microsoft Corp, etc., and a playing time is optionally changed by clicking and drugging a seek bar, i.e., a random access is accomplished as shown in
The streaming generally allows a calculator to start reproducing data recorded on the other one or more calculators connected over the network upon reception of a portion of a data file necessary for the moment without entirely receiving the data file. Such a streaming, however, needs a long time to correspond to a seeking activity as described below. When a user drugs the marker during video playing, the video playing continues as is during the drugging. When the user releases the mouse button at an appropriate position, a reproduction program requests a stream server for transmission of data of a video starting from a time corresponding to the marker position. The stream server stops currently executing transmission of data, and retrieves and transmits the data included in one or more recorded files corresponding to the time of the request. In order to assure a change of a network band, the reproduction program starts reproduction from the requested time after a prescribed amount of data is received in a buffering process. Since both network communications and buffering of a seek instruction are time consuming, a video scene cannot be changed in real time even when the mark is drugged during the streaming reproduction. Specifically, the reproduction needs long time after the mouse button is released.
As one attempt, the Japanese patent application laid open No. (JOP) 2-113790 discusses a technology of starting reproduction from a prescribed time point among recorded video data. Specifically, a plurality of thumbnail images line up as a menu to be selected, and data is reproduced while seeking a time corresponding to the selected thumbnail image. The Japanese patent application laid open No. 2-113790 discusses that a desirous scene is selected by reproducing a thinned image, such as an animation formed by collecting only images of changed scenes, etc., recorded in a hard disk either slowly or in a division multiple manner. The thinned image can be displayed by executing fast-forwarding and rewinding a tape. A video delivery and retrieval system called Media DEPO manufactured by Fuji-Xerox Co., Ltd, includes a slider retrieval function. That is, when a user moves a seek bar, contents of a thumbnail display, named a scene viewer, employed in addition to a video playing screen changes in turn. When a clicking operation is executed thereafter, video playing starts from the time.
According to JOP02-113790, a user interface is formed by displaying a plurality of thumbnail images side by side as a menu to allow random accessing as widely utilized in a DVD or the like. Such a menu displaying manner necessitates a prescribed area for the menu. In particular, the display area becomes larger or other operations, such as menu scrolling, page switching, etc., are necessitated, when a lot of seeking points (i.e., reproduction start times) are employed. JOP6-105280 simply discusses that animation data is recorded in a tape and does not discuss a streaming environment. As mentioned above, JOP2-113790 discusses the scene viewer beside the video screen, and thus, a displaying manner is different from when a un-compression data is reproduced using the above-mentioned media player. Further, it takes long time in transferring data before starting the slider retrieval.
Accordingly, an object of the present invention is to improve such background arts and provides a new and novel video data reproduction apparatus that displays an animation by receiving and reproducing video data through streaming communication. Such a new and novel video data reproduction apparatus includes a memory that stores index image generated from the video data corresponding to a video display time. The index image represents apart of an image created by the video data. A video playing time changing device is provided to change a current video display time to another. A control device is provided to choose an index image from the memory corresponding to the other video play time. The control device then displays the index image on the display. In another embodiment, the index image is created by spreading compressed video data into a frame image. In yet another embodiment, a video data transmitting apparatus is connected to the data reproduction apparatus. The video data transmitting apparatus includes an inputting device that inputs video data including a plurality of frames. A recording device is provided to record the video data. An extracting device is provided to extract prescribed frames. A compression device is provided to compress the extracted frames to create an animation.
In yet another embodiment, a reproduction start control device is provided to control video data reproduction to start after receiving the index image.
In yet another embodiment, a determination device is provided to determine a level of significance of the index image. A reception device is provided to receive the index image information in order of the significance.
In yet another embodiment, a transmission device is provided to transmit the index image in order of the significance.
A more complete appreciation of the present invention and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:
Referring now to the drawing, wherein like reference numerals designate identical or corresponding parts throughout several views, in particular in
Re; Recordation Delivery PC Software
Now, configuration and operation of software implemented in a recordation delivery PC 11 are described according to one embodiment of the present invention. The recordation delivery PC 11 executes two programs to generate and deliver data.
Re; Data Generation Program
The data generation program compresses and records video data inputted from the video camera 17 in the hard disk 13. The program simultaneously generates and records a corresponding index image queue. Referring now to
Re; Video Input Block 20
The video camera 17 outputs an analog signal in a NTSC format. A video capture board (not shown) included in the PC 11 then converts the analog signal into a digital signal. The program then reads data of the digital signal and generates an image data queue of 30 sheets, each including a pixel size of 320×240, per second.
Re; Video Compression Block 21
The data generation program compresses the digital video data using a known MPEG1 compression algorithm. The video data thus compressed is then recorded in a hard disk included in the PC 11.
Re; Image Selection 22
To form index images, a plurality of frames are extracted from the inputted image data queue at a constant interval every video input per minute (i.e., 1800 frames=30 frames/second×60 seconds).
Re; Still Image Compression 23
The extracted images undergo a compression processing of a known JPEG compression algorithm, and are stored in a hard disk as a file or files. The files are assigned names and serial numbers in order of time.
Re; Index Time Data Generation 25
Simultaneously, information associating the video file with the index image is generated and recorded as index time data as shown in
Re; Data Transmission Program
The data transmission program transmits data generated and recorded by the date generation program through the network in accordance with a request from there production PC 12. The data transfer program handles three categories of data, e.g. video data, index time data, and index image data. Specifically, the video data are transmitted with a streaming protocol. A Media Server manufactured by Microsoft Corp, is utilized as a streaming server program, for example, so that video data recorded in the hard disk can be transmitted in a streaming environment. Both the index time data and index image data are transmitted using a known HTTP protocol. Internet Information Server manufactured by Microsoft Corp, is utilized as a HTTP server program, for example.
Re; Reproduction Program
This software runs in a reproduction PC 12 so as to receive data from the recordation delivery PC 12 and display a video in a random access manner in accordance with an instruction from a user. Now, configuration and operation of the software are described with reference to
Re; Initialization
When the program starts in the reproduction PC 12, an initialization processing is executed as mentioned below with reference to
Re; Index Time Data Reception (S1)
A transmission request for index time data is transmitted to the recordation delivery PC 11, which is previously registered. The index time data is then received and recorded in a memory.
Re; Index Image Data Group Reception (S2)
Transmission of each of still images creating index images is requested with reference to descriptions of the video file names included in the index time data. These are then received and recorded in a memory.
Re; Animation Stream Reception Start (S3)
Transmission of video stream data described in the index time data is requested. Although the recordation delivery PC 11 is previously registered as locations of data to be displayed, data can be chosen from a display list when there are a great number of the data. The video data are received after reception of the still image data in order to utilize an index image when a video is reproduced. However, since reception of the animation practically takes a certain time period in the data streaming environment, it is preferable that a time of requesting animation transmission can be adjusted to be right after requesting the final still image transmission (i.e., before completion of the reception).
Re; Animation Display
When an ordinary streaming video reproduction is executed while receiving video data through the network, contents thereof are displayed on the screen. Further, the marker moves at a prescribed interval (e.g. 1 sec) to a position on the seek bar corresponding to a video time period of current reproduction while the time is displayed on the screen.
When instructions of stopping, starting, and seeking or the like are received through either a button operation or a seek bar operation handler, these operational instructions are then transmitted to the delivery program of the recordation delivery PC 11 through the network, thereby the data transmission operation is requested to change. Such a function can be achieved by a Media Player component manufactured by Microsoft, Corp.
Re; Button Operation Handler
This is called when a mouse click event occurs on the button. Similar to an ordinary video playing software, instructions are issued to an animation display block in order to accomplish functions of starting, stopping, and pausing.
Re; Seek Bar Operation Handler
The below-described operations are performed when mouse button down and up, and drug events occur on the marker.
Re; Button Down
A time designation operation is started as mentioned below. A pausing instruction is issued to an animation display module. A video playing region is prepared by reducing the video playing size into 0×0 pixel (i.e., disappearing). A time corresponding to a current marker position is calculated. A sheet is chosen from the recorded index image group corresponding to the marker time. For example, an index image having the maximum time less than the marker time is preferably chosen, so that an image right before the marker time can be displayed. The thus chosen index image is displayed in the animation display region in the same size as the video.
Re; Drugging
A seek bar marker is moved to a mouse pointer position. A time corresponding to a marker movement destination position is calculated. As in the above-mentioned button down operation, a sheet is chosen from the recorded index image group corresponding to the marker time and is displayed. Since the index image has been separately stored in the reproduction PC 12, and a number thereof is apparently smaller than that of frames of the video data, accessing speed can considerably be higher in comparison with a case when a request is made to a streaming server or compression data using a correlation of frames is spread.
Re; Button Up
An animation display module is instructed to seek a time corresponding to a current marker position. After the seeking and buffering complete, an index image is erased and a video playing size is returned to the original size. Then, video reproduction restarts.
Re; Exemplary Operations
Herein below, an exemplary operation of a system in connection with a user operation is described. It is supposed that recordation program is completed in the recordation delivery PC 11, i.e., data to deliver has already been recorded in the hard disk.
Re; Starting Up
When started by a user, a display program attempts to execute a network communication with a recordation delivery PC 11 and receives an index time data. After that, the display program receives and records an index image group, and starts executing streaming video communications.
Re; Reproduction Starts
After buffering of streaming data, a video starts being played on a screen.
Re; Seek Operation
A user operates a seek bar with a mouse during the video playing.
Re; Pushing a Button on a Marker
The video playing is terminated at a time, while a still image in the vicinity of the time is displayed. As the drugging mouse cursor moves left and right as is, the marker similarly moves. Substantially simultaneously, a still image in the vicinity of a time corresponding to the marker is displayed in the video playing region. Even though, the still image has less number of frames in comparison with that of the video (1 sheet/minute) contents can be roughly guessed.
Re; Take Off Button
The last index image is continuously displayed for a while at this time. However, video reproduction is restarted from a time corresponding to the marker position thereafter.
A great number of image data is utilized as index images in the first embodiment. Ina second embodiment, however, a plurality of index image data are arranged in a form of a sheet to be recorded and communicated. Now, only differences from the first embodiment are described.
Re; Data Generation Program
The recordation delivery PC 11 vertically arranges and combines index images generated by the video recordation in a large sheet image. Further, a video data file name, a combined index image filename, a height of a sheet of an original index image, positions and corresponding times of index images on the combined index image (e.g. a distance from the upper most end) are generated and recorded as index time data as shown in
Re; Reproduction Program
The reproduction program reads a name of a combined index image file from the index time data file, and transmits a transmission request for the combined index image file to the delivery PC 11. Then, the reproduction program receives and stores contents thereof. As mentioned earlier, a corresponding index image is selectively displayed in response to the drugging of the seek bar in the first embodiment. However, a corresponding portion of the combined index image is extracted and displayed in the second embodiment. Specifically, a corresponding index image number is calculated based on a moved position of the seek bar. Then, a corresponding region on the combined index image is calculated based on the index time data. Then, the region is extracted and displayed on the video playing region as shown in
In a third embodiment, index image data is compressed in a prescribed manner and delivered as an animation. Specifically, the reproduction PC 12 receives and requests the index data and handles respective frames as index images. Since there is highly provably a strong correlation between frames, especially when taken by a fixed camera for monitoring, data capacity can be reduced if a prescribed animation compression algorithm is utilized. Thus, a transfer time required in transferring the index information, accordingly, a time period from when a user provides an instruction to when animation is practically reproduced can be reduced.
Herein below, differences from the first embodiment are described.
Re; Recordation Program
The recordation delivery PC 11 combines index images generated by video recordation side by side, and processes and handles those as an animation. The recordation delivery PC 11 then compresses these index images with MPEG1 compression algorithm and records those as a file.
The recordation delivery PC 11 generates video data of a main context and an index, a frame number of each of the index video data, and a corresponding time as index time data as shown in
Re; Reproduction Program
The reproduction program receives and spreads the compressed index image video file in an initialization processing, and stores each frame as an individual image in a memory in steps S11 to S14 in
Then, the index image is handled in a similar manner as in the first embodiment.
The frame image becomes the index image at a prescribed time interval in the first embodiment. However, only a frame-having changed is used as an index image to help retrieve in a fourth embodiment. Further, an animation is enabled to be roughly displayed even right after being started. Specifically, the index image is received in order from that having changed at most together with a streaming video.
Re; Recordation Program
An index still image is generated considering a difference between the frame images. In the first embodiment, frame images are extracted every one minute. However, they are not always made into index images in this embodiment. Specifically, a difference between each of the entire pixels in a screen of a frame image and that of a right before frame image (i.e., one minute before) is calculated, and such absolutes are averaged. Only frames having averages larger than a prescribed threshold are utilized as index images. For example, when 256 gradation data is used, only a frame having an average of a differential absolute larger than 5 is used as an index image. Further, the differential average is recorded in the index time data per index image.
Re; Reproduction Program
After index time data is received, index images are sorted in descending order of the differential average thereof after the sorting in the initialization processing. A transmission request is then made in order of largeness of the differential average and applicable index images are received. A streaming animation reproduction is started right before the transmission request. The animation reproduction module executes a video reproduction processing in parallel with index image data reception thereafter. When a seek bar operation starts before reception of the entire index images, a scene on a screen is updated using the index images having been received until the time. Specifically, an index image having the maximum time less than a time corresponding to the marker moving position is selected among the index images already received and is displayed Thus, video reproduction can be started without waiting reception of the entire index images after the program starts. Further, since the index images are received and used from that having the largest change in steps S21 to S24 of
In a fourth embodiment, the transmission order of index images is determined based on the request from the reproduction PC 12. However, the recordation delivery PC 11 determines the transmission order in a fifth embodiment.
Re; Recordation Program
As in the fourth embodiment, an index image is selected and recorded based on an average of differential absolutes. After video recordation is completed, index images are sorted by the differential average similar to the reproduction program of the fourth embodiment. The index images are handled as frames in order from that having the largest differential average. Then, the entire frames undergo MPEG compression to be an animation. Similar index time data are employed to that in the third embodiment. However, corresponding times are not limited to be in ascending order.
Re; Reproduction Program
As in the fourth embodiment, streaming reproduction starts before completion of reception of index (video) data. The MPEG data is processed by compression algorithm using a difference between frames. However, these data are restorable in a unit called GOP (Group of picture). Then, a part of the data capable of spreading is processed during the reception of the index (video) data and is utilized as an index image display in steps S31 to S35 of
Numerous additional modifications and variations of the present invention are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the present invention may be practiced otherwise that as specifically described herein.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2004-366862 | Dec 2004 | JP | national |
