Patent Application
20060123456
The present invention relates to a method, system, and program distributing encoded moving picture data in at least one session of distribution and controlling the quality of the video received by a receiver by notifying the session information to the receiver.
In recent years, as a method for efficiently transmitting moving picture data, the methods transmitting data encoded by high efficiency compression based on interframe prediction are widely used. In these methods, the amount of moving picture data information with high correlations over time is reduced by encoding prediction parameters obtained by predicting an encoded picture from prior and posterior frames in point of time, and predicted residual differential picture data. Furthermore, moving picture data transmission through a narrow transmission band is made possible by high efficiently compression-encoding the predicted residual differential picture data by conversion encoding and quantization.
This method is represented by compression coding systems such as MPEG (Moving Picture Experts Group)-1, MPEG-2, and MPEG-4. In these compression coding systems, interframe prediction for an input image frame is performed with motion compensation per rectangular area unit with a certain size called macroblock, the motion vectors obtained and predicted residual differential picture data are compressed by two-dimensional discrete cosine transformation and quantization, and the resultant signal data is variable length encoded.
As a technology for distributing such moving picture compression encoding information over an IP (Internet Protocol) network utilizing the packet switching system, multicast or broadcast distribution has become common. In this system, since multiple users receive packets multicast or broadcast, the network traffic does not increase proportionally to the increase in the number of users, therefore distribution more efficient than the unicast system can be achieved.
Further, a picture distribution system structured such that it has a first computer capable of streaming-distributing picture data and at least one second computer receiving the picture data, and based on a connection status notification message from the second computer, the first computer sends the picture data (picture data extracted from picture data with different compression ratios stored in advance or picture data compressed with an optimum ratio each time) compressed most suitably for the communication speed of a line to which the second computer is connected is known. (Refer to Patent Document 1 for instance.)
Also, a service quality dynamic control method and device providing different product qualities such as grade 1 and grade 2 between which the customer can choose in a service such as a moving picture multicast, and further allowing the customer to select product quality for each content is known. (Refer to Patent Document 2 for instance.)
[Patent Document 1]
Japanese Patent Kokai Publication No. JP-P2001-92752A (pp. 3-4, FIGS. 1 and 6)
[Patent Document 2]
Japanese Patent Kokai Publication No. JP-P2002-261800A (p. 12, FIG. 13)
[Non-Patent Document 1]
Schulzrinne, H., Casner, S., Frederick, R., and Jacobson, V., “RTP: A Transport Protocol for Real-Time Applications, RFC1889, January 1996, 5.1 RTP Fixed Header Fields. Obtainable from <URL: http://www.ietf.org/tfc.html><ftp://ftp.rfc-editor.org/in-notes/rfc1889.txt>
However, in the conventional multicast or broadcast distribution system, in order for a sender to control the image quality of moving picture data received by a receiver, moving picture data with various qualities (image quality) have to be distributed, spoiling the distribution efficiency, one of the merits of the multicast or broadcast system.
Further, the multicast or broadcast distribution system does not generally allow the receiver to request retransmission of the missing data when a packet is missing, therefore the receiver (the receiving device) cannot decode the video data accurately. As a countermeasure on the part of the receiver (the receiving device), there is an error concealment technique that generates video data from accurately decoded images of prior and posterior frames in point of time and surrounding image data within the same frame to conceal errors, however, it is impossible to remove the disturbance in the decoded video. Furthermore, once a disturbance occurs in the video, it propagates over the subsequent frames since interframe prediction is used.
Therefore, the present invention was invented considering the circumstances described above, and its first object is to provide a system, device, method, and computer program wherein the quality of moving picture received by a receiver can be variably set by a sender while suppressing and reducing the increase in traffic in multicast or broadcast distribution of moving picture.
Further, a second object of the present invention is to provide a distribution system, device, method, and computer program wherein the disturbance in image caused by packet loss is suppressed by having the receiver receive video data in at least one session of distribution, and select and decode the data according to the video quality in order to increase the reliability of the multicast or broadcast distribution system.
Further, another object of the present invention is to provide a distribution system, device, method, and computer program wherein the processing burden for the sender and receiver caused by achieving the first and second objects described above is kept to a minimum.
A further object of the present invention is to provide a distribution system, device, method, and computer program wherein the first and second objects described above are achieved without sending feedback information from the receiver to the sender.
A still further object of the present invention is to provide a method, device, system, and computer program wherein the increase in power consumption on the part of the receiver caused by achieving the first and second objects described above is kept to a minimum.
A video data distribution device relating to an aspect of the present invention that solves at least one of the above mentioned problems comprises: means for multicast or broadcast distributing video encoded data of the same video but having different compression ratios; and means for selecting a session of multicast or broadcast distribution according to the compression ratio. In the present invention, the video encoded data of the same video but having different compression ratios have the same encoding method and the same frame configuration. Its transmission unit (for instance video packet of MPEG-4 encoding method) is data obtained by encoding the same part of the same frame. The present invention comprises means for distributing video encoded data of the same video but having different compression ratios; and means for selecting a session of distribution according to the compression ratio, and at least one session of distribution is transmitted in multicast or broadcast.
A video data distribution device relating to another aspect of the present invention comprises means for multicast or broadcast distributing video encoded data of the same video but having different compression ratios; and means for controlling the quality of a video received by a receiver by changing session information notified according to the receiver. In the present invention, the video encoded data of the same video but having different compression ratios have the same encoding method and the same frame configuration, and its transmission unit (for instance video packet of MPEG-4 encoding method) is data obtained by encoding the same part of the same frame.
A video data distribution device relating to another aspect of the present invention comprises: means for multicast or broadcast distributing video encoded data having different compression ratios; and means for setting information including whether or not the confidentiality (secrecy) of video encoded data is required, confidentiality method when confidentiality is required, and/or the degree of confidentiality for each session of distribution selected according to the compression ratio. In the present invention, the video encoded data of the same video but having different compression ratios have the same encoding method and the same frame configuration. Its transmission unit (for instance video packet of MPEG-4 encoding method) is data obtained by encoding the same part of the same frame.
A video data distribution device relating to another aspect of the present invention comprises: means for multicast or broadcast distributing video encoded data; and means for selecting a session of multicast or broadcast distributed according to the kind (type) of video frame and/or video block.
A video data distribution device relating to another aspect of the present invention comprises: means for multicast or broadcast distributing video encoded data having different kind (type) of video frame and video block; and means for controlling on the part of the receiver, the quality of a video received by a receiver by changing session information notified according to the receiver.
A video data distribution device relating to another aspect of the present invention comprises: means for multicast or broadcast distributing video encoded data; and means for setting information including whether or not confidentiality is required for distributed data, confidentiality method when confidentiality is required, and/or the degree of confidentiality for each session of distribution selected according to the kind (type) of video frame and/or video block.
A video data distribution device relating to another aspect of the present invention comprises: means for multicast or broadcast distributing video encoded data having different information in terms of whether or not confidentiality is required for distributed data, confidentiality method when confidentiality is required, and/or the degree of confidentiality in at least one session of distribution; and means for controlling, on the part of the receiver, the quality of a video received by a receiver by changing information regarding confidentiality notified according to the receiver.
A video data reception device relating to another aspect of the present invention comprises means: for receiving session information notified by a video data distribution device; means for receiving video encoded data distributed by a video data distribution device based on the session information and selecting video encoded data from encoded data received normally based on the video quality (the compression ratio); and means for decoding video encoded data selected. In the present invention, it can further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
A video data reception device relating to another aspect of the present invention comprises: means for receiving information regarding data confidentiality notified by a video data distribution device; means for restoring confidential video encoded data received from a video data distribution device based on the information regarding confidentiality and selecting video encoded data from the video encoded data restored normally based on the video quality (the compression ratio); and means for decoding the selected video encoded data. In the present invention, it may further comprise means for selecting whether or not at least one piece (string) of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
A video data distribution system relating to a first aspect of the present invention that achieves at least one of the above mentioned objects comprises: a video data distribution device and a video data reception device, and a transmission path transmitting information from the video data distribution device to the video data reception device;
the video data distribution device comprises:
means for multicast or broadcast distributing a plurality of (multiple) video encoded data of the same video, the same encoding method, and the same frame configuration but having different compression ratios in different sessions; and means for notifying the session information on at least one session (multicast address, port number, video quality, etc.), including session information on a session permitted to be distributed to, and/or a quality that a receiver is permitted to receive, to the receiver; and
the video data reception device comprises:
means for receiving video encoded data distributed in at least one session based on session information notified by the video data distribution device;
means for selecting data from received video encoded data based on the video quality (the compression ratio) and reconstructing it into one piece of video encoded data; and
means for decoding the video encoded data reconstructed. In the present invention, it can further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
A video data distribution system relating to another aspect (a second aspect) of the present invention comprises a video data distribution device, a video data reception device, and a transmission path for transmitting information from the video data distribution device to the video data reception device. The video data distribution device comprises: means for multicast or broadcast distributing I picture and P picture encoded data of moving picture data or I picture, P picture and B picture encoded data of moving picture data in a plurality of (multiple) different sessions; and
means for notifying session information on at least one session including session information on a session permitted to be distributed to and/or a video quality permitted to be received by the video data reception device.
The video data reception device comprises:
means for receiving moving picture data distributed in at least one session based on the session information notified by the video data distribution device; means for reconstructing the received moving picture data into one piece of moving picture data based on the session information; and means for decoding the reconstructed moving picture data. In the present invention, it can further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
A video data distribution system relating to another aspect (a third aspect) of the present invention comprises: a video data distribution device, a video data reception device, and a transmission path for transmitting information from the video data distribution device to the video data reception device. The video data distribution device comprises:
means for multicast or broadcast distributing moving picture data and intra macroblock (Intra-MB) encoded data of at least a part of frames of the moving picture data in different sessions: and
means for notifying information on at least one session including information on a session permitted to be distributed to and/or a video quality permitted to be received by the video data reception device.
The video data reception device comprises:
means for receiving data distributed in at least one session based on the session information notified by the video data distribution device;
means for selecting data from the received moving picture data based on the video quality and reconstructing it into one piece of moving picture data; and
means for decoding the reconstructed moving picture data. In the present invention, it may further comprise means for selecting whether or not at least one piece (string) of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
A video data distribution system relating to another aspect (a fourth aspect) of the present invention comprises: a video data distribution device, a video data reception device, and a transmission path for transmitting information from the video data distribution device to the video data reception device.
The video data distribution device comprises:
means for multicast or broadcast distributing I picture and P picture encoded data or I picture, P picture and B picture encoded data of moving picture data and intra macroblock (Intra-MB) encoded data of at least a part of frames of the moving picture in a plurality of (multiple) different sessions; and
means for notifying session information on at least one session including information on a session permitted to be distributed to and/or a video quality permitted to be received by the video data reception device.
The video data reception device comprises:
means for receiving data distributed in at least one session out of the session information notified;
means for selecting data from the received moving picture data based on the video quality and reconstructing it into one piece of moving picture data; and
means for decoding the reconstructed moving picture data.
In the present invention, it can further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
A video data distribution system relating to another aspect (a fifth aspect) of the present invention comprises a video data distribution device, a video data reception device, and a transmission path for transmitting information from the video data distribution device to the video data reception device.
The video data distribution device comprises:
means for multicast or broadcast distributing I picture and P picture encoded data, or I picture, P picture and B picture encoded data of multiple moving picture data of the same video but having different compression ratios in multiple different sessions:
means for notifying information on at least one session including information on a session permitted to be distributed to, and/or a video quality permitted to be received by, the video data reception device.
The video data reception device comprises:
means for receiving data distributed in at least one session out of the session information notified by the video data distribution device;
means for selecting data from the received moving picture data based on the video quality and reconstructing it into one piece of moving picture data; and
means for decoding the reconstructed moving picture data. In the present invention, it can further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
A video data distribution system relating to another aspect (a sixth aspect) of the present invention comprises: a video data distribution device, a video data reception device, and a transmission path for transmitting information from the video data distribution device to the video data reception device. The video data distribution device comprises:
means for multicast or broadcast distributing I picture and P picture encoded data or I picture, P picture and B picture encoded data of multiple moving picture data of the same video, the same encoding method, and the same frame configuration but having different compression ratios and intra macroblock (Intra-MB) encoded data of at least a part of frames of the moving picture in multiple different sessions; and
means for notifying information on at least one session including information on a session permitted to be distributed to, and/or a video quality permitted to be received by, the video data reception device.
The video data reception device comprises:
means for receiving data distributed in at least one session out of the session information notified by the video data distribution device;
means for selecting data from the received moving picture data based on the video quality and reconstructing it into one piece of moving picture data; and
means for decoding the reconstructed moving picture data. In the present invention, it can further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
In a video data distribution system relating to another aspect (a seventh aspect) of the present invention, the video data distribution device comprises: means for setting information including whether or not confidentiality is required for distributed data, confidentiality method when confidentiality is required, and/or the degree of confidentiality for each session of distribution and means for notifying the setting information regarding confidentiality to a receiver. The video data reception device comprises: means for receiving the confidentiality information notified by the video data distribution device and restoring the confidential moving picture data received based on the confidentiality information. In the present invention, it may further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
According to the present invention, a sender is able to effectively control the quality of a video that a receiver is permitted to receive without changing video data multicast or broadcast distributed by setting session information notified to the receiver or confidentiality information, on the part of the receiver.
Further, according to the present invention, a sender is able to effectively set the quality of a video that a receiver receives in multiple stages of grades, keeping the advantages of multicast or broadcast distribution that effectively utilizes network resources and suppressing the increase in the amount of distributed data.
Further, according to the present invention, a distribution unaffected by (or resistance to) a data error or loss in a transmission path becomes possible, and the reliability of the multicast or broadcast system, which is said to be less reliable than the unicast system, is increased.
Further, according to the present invention, an increase in processing load required for both the sender and receiver to achieve the two effects mentioned above can be kept to a minimum.
Further, according to the present invention, an increase in power consumption required for both the sender and receiver to achieve the two effects mentioned above can be kept to a minimum.
According to the present invention, these effects can be achieved without sending feedback information from the receiver to the sender.
Embodiments of the present invention will be described hereinafter with reference to the attached drawings in order to further explain the present invention in detail.
In a first embodiment of the present invention, a video data distribution device comprises means for multicast or broadcast distributing multiple moving picture data of the same video, the same encoding method, the same frame configuration but having different compression ratios in different sessions and means for notifying the session information on at least one session (multicast address, port number, video quality, etc.), including a quality that a receiver is permitted to receive, to the receiver. The first embodiment of the present invention can further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
A video data reception device comprises means for receiving data distributed in at least one session out of the session information notified by the video data distribution device, and means for selecting data from the moving picture data received based on the video quality, and reconstructing same into a piece of moving picture data, followed by decoding.
Further, the video data distribution device comprises means for setting whether or not the confidentiality of the distributed data is required, means for securing confidentiality, and the degree of confidentiality for each session of distribution, and means for notifying the setting information regarding confidentiality to the receiver (the video data reception device).
The video data reception device comprises means for receiving the confidential information notified by the video data distribution device, restoring the data received, and decoding the moving picture data.
Next, the first example of the present invention will be described with reference to the drawings.
The video data distribution device 101 comprises a distribution information transmitting unit 102, a session information managing unit 103, a video data storing/generating unit 104, a video encoded data 1 transmitting unit 105, and a video encoded data 2 transmitting unit 106.
The video data reception device 108 comprises a session information receiving unit 109, a video encoded data 1 receiving unit 110 and a video encoded data 2 receiving unit 111, an encoded data reconstructing unit 112, and a decoder 113. Hereinafter, the operations of the video data distribution device 101 and the video data reception device 108 will be described in order.
In the video data distribution device 101, the video data storing/generating unit 104 transmits:
based on the information set by the session information managing unit 103 from a plurality of (multiple) video encoded data transmitting units, the video encoded data 1 transmitting unit 105 and the video encoded data 2 transmitting unit 106 in the example shown in
The session information managing unit 103 manages the information on a session allowed to be distributed to the receiver (the video data reception device), and according to the receiver, supplies the session information that should be notified and the information regarding video encoding such as video quality to the distribution information transmitting unit 102, and the distribution information transmitting unit 102 notifies these pieces of information to the video data reception device 108.
The video encoded data 1 transmitting unit 105 and the video encoded data 2 transmitting unit 105 have the function of adding information (for instance RTP header information) that corresponds to RTP (Real-Time Transport Protocol) so that, in the video data reception device 108, data can be selected when redundant multiple video data are received, the order of the data can be correctly rearranged when the order of the data received is wrong, and so that data loss can be recognized (for instance, RFC1889, refer to the above mentioned Non-Patent Document 1). Further, the transmission units of the video encoded data 1 and the video encoded data 2 are data obtained by encoding the same part of the same frame of the same video, the same RTP sequence number is assigned, and further a payload type and/or SSRC (Synchronization Source Identifier), and/or CSRC (Contributing Source Identifier) identifying any one of the video encoded data can be assigned as well.
At this time, by transmitting the video encoded data 1 and the video encoded data 2 with a time lag longer than the burst length of data error or loss in the transmission path 107, the possibility of any error or loss influencing both the video encoded data in the transmission path is reduced, and a more stable distribution of video data can be achieved when the video data reception device 108 has a receive buffer longer than this time lag.
As for the time lag between transmissions, the distribution information transmitting unit 102 can notify the time lag, managed by the session information managing unit 103, to the video data reception device 108, or the amount of the receive buffer can be set by the video data reception device 108 based on the time lag that is set in advance between the video data distribution device 101 and the video data reception device 108.
When the session information managing unit 103 is set to make the distributed video data confidential, the video encoded data 1 transmitting unit 105 and the video encoded data 2 transmitting unit 106 apply encryption processing according to the setting. And the information regarding encryption such as whether or not any encryption is applied and encryption method is notified by the distribution information transmitting unit 102 to the video data reception device 108. As for the encryption method, various known methods are used. For instance, for an encryption of payload portion of an RTP packet, the common key system (e.g. DES (Data Encryption Standard)), or the public key system (e.g. RAS code), and scrambling of the video (image) itself (e.g. line rotation system) are used.
When the session information, the information regarding video encoding, and the distribution data from the video data distribution device 101 are encrypted, the information regarding the encryption is received by the session information receiving unit 109, and based on this information, the video encoded data 1 receiving unit 110 and the video encoded data 2 receiving unit 111 open session receiving ports and receive the video data from the transmission path 107.
When the video data received is encrypted, the original video (image) data is restored according to the information obtained from the session information receiving unit 109.
Further, the session information managing unit 103 sets the information corresponding to the encoding settings of the video encoded data 1 and the video encoded data 2, encoding options (tools used), the payload options of the encoded data, the RTP header payload type that discriminates the encoded data, and/or an SSRC and CSRC, and the settings are notified to the video data reception device 108 by the distribution information transmitting unit 102, or these pieces of information can be set in advance between the video data distribution device 101 and the video data reception device 108.
When the video encoded data 1 receiving unit 110 and the video encoded data 2 receiving unit 111 are able to receive both the video encoded data 1 and 2, the encoded data reconstructing (reformulating) unit 112 selects data according to the video quality, reconstructs duplicate video data into one piece of video data based on the function corresponding to RTP, the sequence number of the RTP header, the payload type, the information from an SSRC and CSRC, and/or the information received by the session information receiving unit 109, and outputs the result to the decoder 113.
The decoder 113 only has to decode one piece of the video data selected by the encoded data reconstructing unit 112. Therefore, the decode processing load of the decoder 113 does not increase.
In case where the sender (the video data distribution device) is to control the quality of the video data received by (a video data reception device of) a receiver A and (a video data reception device of) a receiver B respectively, the following technique can be employed:
The video encoded data 1 and the video encoded data 2 have different qualities (compression ratios) and for instance, the video encoded data 1 has a higher quality (lower compression ratio).
When the receiver A is permitted to receive the higher quality video and the receiver B is only permitted to receive the lower quality video, the session information on both the video encoded data 1 and video encoded data 2 is notified to the video data reception device 108 of the receiver A, whereas only the session information on the video encoded data 2 is notified to the video data reception device 108 of the receiver B.
The video data reception device 108 of the receiver A can receive both the high quality video data of the video encoded data 1 and low quality video data of the video encoded data 2 based on the session information on both the video encoded data 1 and video encoded data 2 notified, and even if any error or loss occurs in the video data of the video encoded data 1, it will be able to be supplemented with the video data of the video encoded data 2.
Meanwhile the video data reception device 108 of the receiver B can only receive the low quality video data of the video encoded data 2 according to the notified session information. Therefore, the video qualities of the received video are different between the receivers A and B.
Further, when distributing the same video encoded data 1 and video encoded data 2, if the video encoded data 1 is transmitted only to, for instance, the receiver A in the unicast system, and the video encoded data 2 is transmitted to both the receivers A and B in multicast or broadcast, the difference in the video quality will occur in this case as well.
Further, in case where the session information on both the video encoded data 1 and the video encoded data 2 is notified to the video data reception device 108 of the receiver A and the session information on only the video encoded data 1 is notified to the video data reception device 108 of the receiver B, the receivers A and B will receive the data of the same video quality if the video data of the video encoded data 1 is not influenced by any error or loss in the transmission path. On the other hand, when an error or loss occurs in the video data, the receiver A can supplement the problematic area with the video data of the video encoded data 2, which, however, the receiver B cannot do. Therefore, a difference in the stability of the video quality will occur between the receivers A and B as described above.
Further, if the video encoded data 1 and the video encoded data 2 have the same quality, or if the video encoded data 1 is transmitted from both the video encoded data 1 transmitting unit 105 and the video encoded data 2 transmitting unit 106, it will be possible to keep the degradation in video playback quality, influenced by an error or loss in the transmission path, to a minimum.
Additionally, in case where the video encoded data 2 is transmitted only to the receiver A in the unicast system, and the video encoded data 1 is transmitted to both the receivers A and B in multicast or broadcast, a similar difference in the stability of the video quality still occurs. In this case, since the bandwidth of the video encoded data transmitted in the unicast system is smaller, it is more preferable for suppressing the increase in traffic.
Further, the similar effects can be achieved by setting the quality (compression ratio) of the video encoded data 1 and the video encoded data 2 identically, encrypting only the video encoded data 1 for instance, and notifying how to decrypt and restore the encrypted video encoded data 1 to the video data reception device 108 of the receiver A, but not to the video data reception device 108 of the receiver B, or encrypting both the video encoded data 1 and the video encoded data 2, notifying how to decrypt and restore both the video encoded data to the video data reception device 108 of the receiver A, and notifying how to decrypt and restore only the video encoded data 2 to the video data reception device 108 of the receiver B.
In the present embodiment, when the video data reception device 108 is being used in an environment where the available power is limited such as in battery operation, and/or when it is used in a good reception environment where it can receive the video encoded data 1 with almost no error and loss, the reception of the video encoded data 2 can be stopped in order to maximize the operation time of the device. On the other hand, when the video data reception device 108 is connected to a battery with a large capacity or AC power, it can be set to receive both the video encoded data 1 and video encoded data 2. As described, according to the available power and reception environment, it can control which data to receive. Further, in the present embodiment, the receiver can be allowed to set the number of video data that the video data reception device 108 receives.
It should be noted that it is also possible, using a plurality of (multiple) transmission paths, to transmit the video encoded data 1, the video encoded data 2, and the session information through different transmission paths, respectively.
Further, for encrypting and distributing specific video encoded data, as shown in
Further, not only the structure where all the sessions are multiplexed into one, but also a structure is possible where a plurality of (multiple) packet multiplex transmitter units multiplexing several sessions are provided, and the similar discrimination information is added.
It should be noted that each function and processing of the distribution information transmitting unit 102, the session information managing unit 103, the video data storing/generating unit 104, the video encoded data 1 transmitting unit 105, the video encoded data 2 transmitting unit 106, and the packet multiplex transmitter unit 701 in the video data distribution device 101 can be realized by program control executed on a computer constituting the video data distribution device 101. It should also be noted that each function and processing of the session information receiving unit 109, the video encoded data 1 receiving unit 110 and the video encoded data 2 receiving unit 111, the encoded data reconstructing unit 112, the decoder 113, the packet receiver-separator unit 702, and the encoded data reconstructing unit 703 in the video data reception device 108 can be realized by the program control executed on a computer constituting the video data distribution device 101.
In the second embodiment of the present invention, a video data distribution device comprises means for multicast or broadcast distributing an I picture (Intra-Picture: intraframe encoded picture), a P picture (Predictive-Picture: interframe forward predicted picture), and a B picture (Bi-directionally predictive-Picture: bi-directionally predicted encoded picture) of moving picture, if this exists in the encoding methods used for encoding the moving picture, in different sessions; and means for notifying the session information on at least one session (multicast address, port number, picture kind, etc.), including a quality that a receiver is permitted to receive, to the receiver. The second embodiment of the present invention can further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
A video data reception device comprises means for receiving data multicast or broadcast distributed in at least one session out of the session information (multicast address, port number, picture kind, etc.) notified, and means for reconstructing the moving picture data received into one piece of moving picture data and decoding it.
Further, the video data distribution device comprises means for setting whether or not the confidentiality of the distributed data is required, means for securing confidentiality, and the degree of confidentiality for each session of distribution, and means for notifying the setting information regarding confidentiality to the receiver.
The video data reception device comprises means for receiving the confidential information notified, restoring the data received, and decoding the moving picture data.
The second example of the present invention will be described with reference to the drawings.
The video data distribution device 201 comprises a distribution information transmitting unit 202, a session information managing unit 203, a video data storing/generating unit 204, an I picture encoded data transmitting unit 205, a P picture encoded data transmitting unit 206, and a B picture encoded data transmitting unit 207.
The video data reception device 209 comprises a session information receiving unit 210, an I picture encoded data receiving unit 211, a P picture encoded data receiving unit 212, a B picture encoded data receiving unit 213, an encoded data reconstructing unit 214, and a decoder 215. Hereinafter, the operations of the video data distribution device 201 and the video data reception device 209 will be described in order.
In the video data distribution device 201, the video data storing/generating unit 204 transmits:
based on the information set by the session information managing unit 203 from the I picture encoded data transmitting unit 205, the P picture encoded data transmitting unit 206, and the B picture encoded data transmitting unit 207 via the transmission path 208.
Further, the session information managing unit 203 manages the session information on a session permitted to be distributed to the receiver, and according to the receiver, the session information that should be notified and the information regarding video encoding such as picture kinds (types) is notified by the distribution information transmitting unit 202 to the video data reception device 209.
The I picture encoded data transmitting unit 205, the P picture encoded data transmitting unit 206, and the B picture encoded data transmitting unit 207 have the function of adding information that corresponds to RTP (Real-Time Transport Protocol) so that the order of the data can be correctly rearranged when the order of the data received by the video data reception device 209 is not in order.
When the session information managing unit 203 is set to make the distributed video data confidential, the I picture encoded data transmitting unit 205, the P picture encoded data transmitting unit 206, and the B picture encoded data transmitting unit 207 apply encryption processing according to the settings, and the information regarding encryption such as whether or not any encryption is applied and encryption method is notified by the distribution information transmitting unit 202 to the video data reception device 209.
When the session information, the information regarding video encoding, and the distribution data from the video data distribution device 201 are encrypted, the information regarding the encryption is received by the session information receiving unit 210, and based on this information, the I picture encoded data receiving unit 211, the P picture encoded data receiving unit 212, and the B picture encoded data receiving unit 213 open session receive ports and receive the video data from the transmission path 208. When the video data received is encrypted, the original video data is restored in the I picture encoded data receiving unit 211, the P picture encoded data receiving unit 212, and the B picture encoded data receiving unit 213 according to the information obtained from the session information receiving unit 210.
The session information managing unit 203 sets the information corresponding to the encoding settings of the video encoded data, encoding options (tools used), the payload options of the encoded data, the RTP header payload type that discriminates the encoded data, and/or an SSRC and CSRC, and the settings can be notified to the video data reception device 209 by the distribution information transmitting unit 202, or these pieces of information can be set in advance between the video data distribution device 201 and the video data reception device 209.
The encoded data reconstructing unit 214 reconstructs the I picture, P picture and B picture data received into a stream of moving picture based on the function corresponding to RTP, the sequence number of the RTP header, the payload type, the information from an SSRC and CSRC, and/or the information received by the session information receiving unit 210, and outputs the result to the decoder 215.
In order for the sender (i.e. on the part of the sender) to control the quality of the video data received by each video data reception device of a receiver A, receiver B, and receiver C, the encoded data of I picture, P picture and B picture are transmitted to the video data reception device of the receiver A, to which the information on all the sessions is notified.
To the video data reception device of the receiver B, the information on the session transmitting the encoded data of I picture and P picture is notified.
To the video data reception device of the receiver C, only the information on the session transmitting the encoded data of I picture is notified.
The video data reception device of the receiver A receives the highest quality video (with the smoothest motion) since it receives the encoded data of all the pictures, the video data reception device of the receiver B receives the second highest quality video, and the video data reception device of the receiver C receives the worst quality video with only the I picture data.
As described above, each receiver (the video data reception device) receives the video with different quality.
Further, in the present example, the difference in the video quality among the receivers can be controlled (ensured) by transmitting the encoded data of the specific picture (type) to only a specific receiver in the unicast system. For instance, the similar effects can be achieved by transmitting the I picture encoded data in multicast or broadcast, transmitting the P picture encoded data to the receivers A and B in the unicast system, and transmitting the B picture encoded data only to the receiver A in the unicast system.
Further, the similar effects can be achieved by encrypting only the P picture and B picture encoded data, notifying the information for restoring the encrypted P picture and B picture encoded data (for instance a public key) to the video data reception device of the receiver A, notifying the information for restoring only the P picture encoded data to the video data reception device of the receiver B, and not notifying any information regarding the encryption to the video data reception device of the receiver C.
Further, the I picture and P picture encode data are distributed in one session, and the B picture encoded data is distributed in another.
The I picture encode data is distributed in one session, and the P picture and B picture encoded data are distributed in another.
The same effects can be achieved by similarly notifying the session information or encrypt information on the distributed data.
In the present example, when the video data reception device 209 is being used in an environment where the available power is limited such as in battery operation, the reception of the B picture encoded data and the P/B picture encoded data can be stopped in order to maximize the operation time of the device. On the other hand, when the video data reception device 209 is connected to a battery with a large capacity or AC power, it can be set to receive the encoded data of all the picture types. As described, according to the available power and reception environment, it can control which data to receive. In the present example, the receiver can be allowed to set the picture encoded data that the video data reception device 209 receives.
A structure having multiple transmission paths, and transmitting the I picture encoded data, the P picture encoded data, the B picture encoded data and the session information through different transmission paths, respectively, is also possible.
Further, when encrypting and distributing the encoded data of a specific picture type, as shown in
Alternatively, when the sequence number of RTP header or information corresponding thereto is added to each output of the video data distribution device 201 for all the picture types in the present embodiment, the data are reconstructed into one piece (string) of encoded data by rearranging according to the sequence number even without applying the discrimination information regarding the picture types, therefore the similar effects can be achieved.
As for the packet receiver-separator (demultiplexer) unit 902, it does the same processing as that of the packet receiver-separator unit 702 in the first embodiment.
Further, not only the structure where all the sessions are multiplexed into one, but also a structure is possible where multiple packet multiplex transmitter units multiplexing several sessions are provided, and the similar discrimination information is added.
Further, as in the previous embodiment, it should be noted that each function and processing of each unit in the video data distribution device 201 and the video data reception device 209 can be realized by program control executed on a computer constituting the video data distribution device and the video data reception device in the present example as well.
In the third embodiment of the present invention, a video data distribution device comprises means for multicast or broadcast distributing moving picture data and intra macroblock (Intra-MB) encoded data of at least a part of frames of the moving picture in respective different sessions, and means for notifying the session information on at least one session (multicast address, port number, video type, etc.), including a quality that a receiver is permitted to receive, to the receiver.
A video data reception device comprises means for receiving data multicast or broadcast distributed in at least one session out of the session information notified, and means for selecting data from the moving picture data received based on the video quality, and reconstructing it into a piece of moving picture data and decoding it. It may further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
Further, the video data distribution device comprises means for setting whether or not the confidentiality of the distributed data is required, means for securing confidentiality, and the degree of confidentiality for each session of distribution, and means for notifying the setting information regarding confidentiality to the receiver, and the video data reception device comprises means for receiving confidential information notified, restoring the data received, and decoding the moving picture data.
The third example of the present invention will be described with reference to the drawings.
The video data distribution device 301 comprises a distribution information transmitting unit 302, a session information managing unit 303, a video data storing/generating unit 304, a video encoded data transmitting unit 305, and an intra macroblock (Intra-MB) encoded data transmitting unit 306.
The video data reception device 308 comprises a session information receiving unit 309, a video encoded data receiving unit 310, an intra-MB encoded data receiving unit 311, an encoded data reconstructing unit 312 and a decoder (unit) 313. Hereinafter, the operations of the video data distribution device 301 and the video data reception device 308 will be described in order.
In the video data distribution device 301, the video data storing/generating unit 304 transmits:
based on the information set by the session information managing unit 303 from the video encoded data transmitting unit 305, and the intra macroblock (Intra-MB) encoded data transmitting unit 306 via the transmission path 307. Further, the session information managing unit 303 manages the information on a session permitted to be distributed to the receiver, and according to the receiver, the session information that should be notified and the information regarding video encoding such as video type is notified by the distribution information transmitting unit 302 to the video data reception device 308. The video encoded data transmitting unit 305 and the intra-MB encoded data transmitting unit 306 have the function of adding information that corresponds to RTP (Real-Time Transport Protocol) so that, in the video data reception device 308, data can be selected when duplicate (redundant) plural (multiple) video data are received, and the order of the data can be correctly rearranged when the order of the data received is out of order. At this time, the same RTP sequence number can be added to the video encoded data in the same region as the video that the intra-macroblock encoded data encodes, and a payload type and/or SSRC and/or CSRC indicating which data it is, the video encoded data or the intra-macroblock, can be added.
When the session information managing unit 303 is set to make the distributed video data confidential (secret), the video encoded data transmitting unit 305 and the intra-MB encoded data transmitting unit 306 apply encryption processing according to the settings, and the information regarding encryption such as whether or not any encryption is applied and encryption method is notified by the distribution information transmitting unit 302 to the video data reception device 308.
When the session information, the information regarding video encoding, and the distribution data from the video data distribution device 301 are encrypted, the video data reception device 308 receives the information regarding the encryption with the session information receiving unit 309, and based on this information, the video encoded data receiving unit 310 and the intra-MB encoded data receiving unit 311 open session receive ports and receive the video data from the transmission path 307. When the video data received is encrypted, the original video data is restored by the video encoded data receiving unit 310 and the intra-MB encoded data receiving unit 311 according to the information obtained from the session information receiving unit 309.
Further, the session information managing unit 303 sets the information corresponding to the encoding settings of the video encoded data, encoding options (tools used), the payload options of the encoded data, the RTP header payload type that discriminates the encoded data, and/or an SSRC and CSRC, and the settings can be notified to the video data reception device 308 by the distribution information transmitting unit 302, or these pieces of information can be set in advance between the video data distribution device 301 and the video data reception device 308.
When both the video encoded data and intra macroblock data are received, according to the video quality, the encoded data reconstructing unit 312 reconstructs duplicate video data into one piece of video data based on the function corresponding to RTP, the sequence number of the RTP header, the payload type, the information from an SSRC and CSRC, and/or the information received by the session information receiving unit 309, and outputs the result to the decoder 313. Only one piece of the video data has to be decoded, therefore, the decode processing load of the decoder 113 does not increase.
In order for the sender (the video data distribution device) to control the qualities of the video data received by each video data reception device of receivers A and B respectively, when the receiver A is permitted to receive the video data of both the video encoded data and intra macroblock encoded data, and the receiver B is permitted to receive the video encoded data only, the session information on both the video encoded data and intra macroblock encoded data is notified to the video data reception device of the receiver A, and the session information on only the video encoded data is notified to the video data reception device of the receiver B.
The video data reception device of the receiver A can receive both the video encoded data and intra macroblock encoded data, and even if an error or loss occurs in the video encoded data, the problematic area will be able to be supplemented if the intra macroblock encoded data is received. The video data reception device of the receiver B can only receive the video encoded data, and when it has any error or loss, the video quality deteriorates.
As described above, the video qualities of the received video are different between the video data reception devices of receivers A and B.
Further, when distributing the same video encoded data and intra macroblock encoded data, if the video encoded data 1 is transmitted in multicast or broadcast and the intra macroblock encoded data is transmitted only to the receiver A in the unicast system, the difference in the stability of the video quality of the received video will occur in this case as well.
Further, the similar effects (difference in the quality of received video image) can be achieved by encrypting only the intra macroblock encoded data, and notifying how to decrypt and restore the encrypted intra macroblock encoded data to the video data reception device of the receiver A, but not to the video data reception device of the receiver B, or encrypting both the video encoded data and the intra macroblock encoded data, notifying how to decrypt and restore both the video encoded data to the video data reception device of the receiver A, and notifying how to decrypt and restore only the video encoded data to the video data reception device of the receiver B.
In the present example, when the video data reception device 308 is being used in an environment where the available power is limited such as in battery operation, and/or when it is used in a good reception environment where it can receive the video encoded data with almost no error and loss, the reception of the intra macroblock encoded data can be stopped in order to maximize the operation time of the device. On the other hand, when the video data reception device 308 is connected to a battery with a large capacity or AC power, it can be set to receive both the video encoded data and intra macroblock encoded data. As described, according to the available power and reception environment, it can control which data to receive. Further, in the present example, the receiver can be allowed to set the kind (type) of video encoded data that the video data reception device 308 receives.
Further, for instance, when the video encoded data is being received accurately and it is interframe (reference frame) encoded data, even if an error and loss occurs in the transmission path, the similar effects as those of CIR (Cyclic Intra Refresh) in the MPEG-4 encoding method will be able to be achieved by having the encoded data reconstructing unit 312 select the intra macroblock encoded data periodically, and the degradation in video playback quality caused by the error and loss in the transmission path can be quickly recovered, to suppress the propagation of the quality degradation over the following frames.
It should be noted that a structure having a plurality of (multiple) transmission paths, and transmitting the video encoded data, the intra macroblock encoded data, and the session information through different transmission paths, respectively, is also possible.
Further, when encrypting and distributing a specific type of the video encoded data, as shown in
As for the packet receiver-separator unit 1002, it does the same processing as that of the packet receiver-separator unit 702 in the first example.
Further, not only the structure where all the sessions are multiplexed into one, but also a structure where multiple packet multiplex transmitter units multiplexing several sessions are provided, and the similar discrimination information is added is possible.
As in the previous example, it should be noted that each function and processing of each unit in the video data distribution device 301 and the video data reception device 308 can be realized by program control executed on a computer constituting the video data distribution device and the video data reception device in the present example as well.
In the fourth embodiment of the present invention, a video data distribution device comprises means for multicast or broadcast distributing an I picture, a P picture, a B picture of moving picture, if this exists in the encoding methods used for encoding the moving picture, and intra macroblock (Intra-MB) encoded data of at least a part of frames of the moving picture in different sessions, and means for notifying the session information on at least one session (multicast address, port number, video type, etc.), including a quality that a receiver is permitted to receive, to the receiver.
A video data reception device comprises means for receiving data multicast or broadcast distributed in at least one session out of the session information notified, and means for selecting data from the moving picture data received based on the video quality, and reconstructing it into a piece (string) of moving picture data followed by decoding. The fourth embodiment of the present invention may further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
Further, the video data distribution device may comprise means for setting whether or not the confidentiality of the distributed data is required, means for securing confidentiality, and the degree of confidentiality for each session of distribution, and means for notifying the setting information regarding confidentiality to the receiver, and the video data reception device may comprise means for receiving the confidential information notified, restoring the data received, and decoding the moving picture data.
The fourth example of the present invention will be described with reference to the drawings.
The video data distribution device 401 comprises a distribution information transmitting unit 402, a session information managing unit 403, a video data storing/generating unit 404, an I picture encoded data transmitting unit 405, a P picture encoded data transmitting unit 406, a B picture encoded data transmitting unit 407, and an intra macroblock (intra-MB) encoded data transmitting unit 408.
The video data reception device 410 comprises a session information receiving unit 411, an I picture encoded data receiving unit 412, a P picture encoded data receiving unit 413, a B picture encoded data receiving unit 414, an intra-MB encoded data receiving unit 415, an encoded data reconstructing unit 416, and a decoder 417. Hereinafter, the operations of the video data distribution device 401 and the video data reception device 410 will be described in order.
In the video data distribution device 401, the video data storing/generating unit 404 transmits:
based on the information set by the session information managing unit 403 from the I picture encoded data transmitting unit 405, the P picture encoded data transmitting unit 406, the B picture encoded data transmitting unit 407, and the intra-MB encoded data transmitting unit 408 via the transmission path 409.
Further, the session information managing unit 403 manages the session information on a session permitted to be distributed to the receiver, and according to the receiver, the session information that should be notified and the information regarding video encoding such as video type is notified by the distribution information transmitting unit 402 to the video data reception device 410.
The I picture encoded data transmitting unit 405, the P picture encoded data transmitting unit 406, the B picture encoded data transmitting unit 407, and the intra-MB encoded data transmitting unit 408 have the function of adding information that corresponds to RTP (Real-Time Transport Protocol) so that, in the video data reception device 410, data can be selected when duplicate multiple video data are received, and the order of the data can be correctly rearranged when the order of the data received is out of order.
At this time, the same RTP sequence number can be added to the video encoded data in the same region as the video that the intra macroblock encoded data encodes, and a payload type and/or SSRC and/or CSRC indicating which data it is, the video encoded data or the intra macroblock, can be added.
When the session information managing unit 403 is set to make the distributed video data confidential, the I picture encoded data transmitting unit 405, the P picture encoded data transmitting unit 406, the B picture encoded data transmitting unit 407, and the intra-MB encoded data transmitting unit 408 apply encryption processing according to the settings, and the information regarding encryption such as whether or not any encryption is applied and encryption method is notified by the distribution information transmitting unit 402 to the video data reception device 410.
The session information, the information regarding video encoding, and when the distribution data from the video data distribution device 401 are encrypted, the information regarding the encryption are received by the session information receiving unit 411, and based on this information, the I picture encoded data receiving unit 412, the P picture encoded data receiving unit 413, the B picture encoded data receiving unit 414, and the intra-MB encoded data receiving unit 415 open session receive ports and receive the video data from the transmission path 409. When the video data received is encrypted, the original video data is restored in the I picture encoded data receiving unit 412, the P picture encoded data receiving unit 413, the B picture encoded data receiving unit 414, and the intra-MB encoded data receiving unit 415 according to the information obtained from the session information receiving unit 411.
The session information managing unit 403 sets the information corresponding to the encoding settings of the video encoded data, encoding options (tools used), the payload options of the encoded data, the RTP header payload type that discriminates the encoded data, and/or an SSRC and CSRC, and the settings can be notified to the video data reception device 410 by the distribution information transmitting unit 402, or these pieces of information can be set in advance between the video data distribution device 401 and the video data reception device 410.
When any one of the picture encoded data and intra macroblock data are both received, according to the video quality, the encoded data reconstructing unit 416 selects the duplicate data, reconstructs the I picture, P picture and B picture, and intra macroblock data received into a stream of moving picture based on the function corresponding to RTP, the sequence number of the RTP header, the payload type, the information from an SSRC and CSRC, and/or the information received by the session information receiving unit 411, and outputs the result to the decoder 417. Only one piece of the video data has to be decoded, therefore, the decode processing load of the decoder 417 does not increase.
In order for the sender (on the part of the sender) to control the qualities of the video data received by each video data reception device of receivers A, B, C, and D respectively, the encoded data of I picture, P picture, B picture, and intra macroblock are transmitted to the video data reception device of the receiver A, to which the information on all the sessions is notified.
To the video data reception device of the receiver B, the information on the session transmitting the encoded data of I picture, P picture and B picture is notified.
To the video data reception device of the receiver C, the information on the session transmitting the encoded data of I picture and P picture is notified.
To the video data reception device of the receiver D, only the information on the session transmitting the encoded data of I picture is notified.
When there is no error or loss in the video encoded data received by the video data reception device of each receiver, the video data reception devices of the receivers A and B receive the highest quality video (with the smoothest motion) since they receive the encoded data of all the pictures.
The video data reception device of the receiver C receives the second highest quality video, and the video data reception device of the receiver D receives the worst quality video with only the I picture data. As described above, each receiver receives the video with different qualities.
If an error or loss occurs in the video encoded data, the video data reception device of the receiver A will be able to supplement the problematic area if it receives the intra macroblock encoded data whereas the video qualities of the receivers B, C, and D will degrade since their video data reception devices don't have the supplementary information. Therefore, the difference in the video quality will occur among the receivers.
Further, in the present example, the difference in the video quality among the receivers can be controlled by transmitting specific encoded data to only a specific receiver in the unicast system. For instance, the similar effects can be achieved by transmitting the I picture encoded data and P picture encoded data in multicast or broadcast as mentioned above, transmitting the B picture encoded data to the receivers A and B in the unicast system, and transmitting the intra macroblock encoded data only to the receiver A in the unicast system.
Further, the similar effects can be achieved by encrypting the P picture, B picture, and intra macroblock encoded data for instance, notifying the information for restoring the encrypted P picture, B picture, and intra macroblock encoded data to the video data reception device of the receiver A, whereas notifying the information for restoring the P picture and B picture encoded data to the video data reception device of the receiver B, notifying the information for restoring the P picture encoded data to the video data reception device of the receiver C, and not notifying any information regarding the encryption to the video data reception device of the receiver D.
In the present example, when the video data reception device 410 is being used in an environment where the available power is limited such as in battery operation, the reception of the intra macroblock encoded data, B picture encoded data, and P picture encoded data can be stopped in order to maximize the operation time of the device. On the other hand, when the video data reception device 410 is connected to a battery with a large capacity or AC power, it can be set to receive the encoded data of all the picture types. As described, according to the available power and reception environment, it can control which data to receive. In the present example, the receiver can be allowed to set the picture encoded data that the video data reception device 410 receives.
Further, the similar effects can be achieved by distributing the I picture and P picture encoded data in one session, the B picture encoded data in one session, the intra macroblock encoded data in one session, the I picture encoded data in one session, the P picture and B picture encoded data in one session, the intra macroblock encoded data in one session, and notifying the session information or the encryption information on the distributed data similarly.
Further, for instance, when the video encoded data is being received accurately and it is interframe (reference frame) encoded data, even if an error or loss occurs in the transmission path, the similar effects as those of CIR (Cyclic Intra Refresh) in the MPEG-4 encoding method will be able to be obtained by having the encoded data reconstructing unit 416 select the intra macroblock encoded data periodically, and the degradation in video playback quality caused by the error or loss in the transmission path can be quickly recovered, suppressing the propagation of the quality degradation over the following frames.
Further, a structure having multiple transmission paths, and transmitting the I picture encoded data, the P picture encoded data, the B picture encoded data, the intra macroblock encoded data, and the session information through different transmission paths is also possible.
Further, when encrypting and distributing the encoded data of a specific picture, and type, as shown in
Alternatively, when the sequence number of RTP header or information corresponding to it is added to each output of the video data distribution device 401 for all the picture types in the present example, the data are reconstructed into one piece (string) of encoded data by rearranging according to the sequence number without adding the discrimination information regarding the picture type, therefore the similar effects can be achieved.
As for the packet receiver-separator unit 1102, it does the same processing as that of the packet receiver-separator unit 702 in the first embodiment.
Further, not only the structure is possible where all the sessions are multiplexed into one, but also a structure is possible where multiple packet multiplex transmitter units multiplexing several sessions are provided, and the similar discrimination information is added.
Further, as in the previous example, it should be noted that each function and processing of each unit in the video data distribution device 401 and the video data reception device 410 can be realized by program control executed on a computer constituting the video data distribution device and the video data reception device in the present embodiment as well.
In the fifth embodiment of the present invention, a video data distribution device comprises means for multicast or broadcast distributing a respective I picture, P picture, and if it exists in the encoding methods used for encoding the moving picture, B picture of multiple moving picture data of the same video, the same encoding method, the same frame configuration but having different compression ratios in different sessions, and means for notifying the information on at least one session (multicast address, port number, picture type, etc.), including a quality that a receiver is permitted to receive, to the receiver. A video data reception device comprises means for receiving data multicast or broadcast distributed in at least one session out of the session information notified, and means for selecting data from the moving picture data received based on the video quality, and reconstructing it into a piece (string) of moving picture data, followed by decoding. The fifth embodiment of the present invention can further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
Further, the video data distribution device may comprise means for setting whether or not the confidentiality of the distributed data is required, means for securing confidentiality, and the degree of confidentiality for each session of distribution, and means for notifying the setting information regarding confidentiality to the receiver, and the video data reception device may comprise means for receiving the confidential information notified, restoring the data received, and decoding the moving picture data.
Next, the fifth example of the present invention will be described with reference to the drawings.
The video data distribution device 501 comprises a distribution information transmitting unit 502, a session information managing unit 503, a video data storing/generating unit 504, an I picture encoded data transmitting unit 505 for video data 1, a P picture encoded data transmitting unit 506 for the video data 1, an I picture encoded data transmitting unit 507 for video data 2, and a P picture encoded data transmitting unit 508 for the video data 2.
The video data reception device 510 comprises a session information receiving unit 511, an I picture encoded data receiving unit 512 for the video data 1, a P picture encoded data receiving unit 513 for the video data 1, an I picture encoded data receiving unit 514 for the video data 2, a P picture encoded data receiving unit 515 for the video data 2, an encoded data reconstructing unit 516, and a decoder 517.
Hereinafter, the operations of the video data distribution device 501 and the video data reception device 510 will be described in order.
The video data distribution device 501 has the video data storing/generating unit 504 transmit:
based on the information set by the session information managing unit 503 from the I picture encoded data transmitting unit 505 for the video data 1, the P picture encoded data transmitting unit 506 for the video data 1, the I picture encoded data transmitting unit 507 for the video data 2, and the P picture encoded data transmitting unit 508 for the video data 2 via the transmission path 509.
Further, the session information managing unit 503 manages the information on a session permitted to be distributed to the receiver, and according to the receiver, the session information that should be notified and the information regarding video encoding such as video quality and picture type is notified by the distribution information transmitting unit 502 to the video data reception device 510.
The I picture encoded data transmitting unit 505 for the video data 1, the P picture encoded data transmitting unit 506 for the video data 1, the I picture encoded data transmitting unit 507 for the video data 2, and the P picture encoded data transmitting unit 508 for the video data 2 have the function of adding information that corresponds to RTP (Real-Time Transport Protocol) so that, in the video data reception device 510, data can be selected when duplicate (redundant) multiple video data are received, and the order of the data can be correctly rearranged when the order of the data received is out of order.
At this time, by transmitting the I picture encoded data of the video data 1 and the I picture encoded data of the video data 2, and the P picture encoded data of the video data 1 and the P picture encoded data of the video data 2 with a time lag longer than the burst length of data error or loss in the transmission path 509 respectively, the possibility of any error or loss influencing both the video encoded data in the transmission path is reduced, and a more stable distribution of video data can be achieved.
As for the time lag between transmissions, the distribution information transmitting unit 502 can notify the time lag, managed by the session information managing unit 503, to the video data reception device 510, or the receive buffer size can be set by the video data reception device 510 based on the time lag that the video data distribution device 501 and the video data reception device 510 decide in advance.
When the session information managing unit 503 is set to make the distributed video data confidential (secret), the I picture encoded data transmitting unit 505 for the video data 1, the P picture encoded data transmitting unit 506 for the video data 1, the I picture encoded data transmitting unit 507 for the video data 2, and the P picture encoded data transmitting unit 508 for the video data 2 apply encryption processing according to the settings, and the information regarding encryption such as whether or not any encryption is applied and encryption method is notified by the distribution information transmitting unit 502 to the video data reception device 510.
When the session information, the information regarding video encoding, and the distribution data from the video data distribution device 501 are encrypted, the video data reception device 510 has the session information receiving unit 511 receive the information regarding the encryption, and based on this information, the I picture encoded data receiving unit 512 for the video data 1, the P picture encoded data receiving unit 513 for the video data 1, the I picture encoded data receiving unit 514 for the video data 2, and the P picture encoded data receiving unit 515 for the video data 2 open session receive ports and receive the video data from the transmission path 509. When the video data received is encrypted, the original video data is restored according to the information obtained from the session information receiving unit 511.
Further, the session information managing unit 503 sets the information corresponding to the encoding settings of the video encoded data 1 and the video encoded data 2, encoding options (tools used), the payload options of the encoded data, the RTP header payload type that discriminates the encoded data, and/or an SSRC and CSRC, and the settings are notified to the video data reception device 510 by the distribution information transmitting unit 502, or these pieces of information can be set in advance between the video data distribution device 501 and the video data reception device 510.
When the encoded data of the I picture of the video data 1, the P picture of the video data 1, the I picture of the video data 2, and the P picture of the video data 2 are received, the encoded data reconstructing unit 516 selects data according to the video quality, reconstructs it into a stream of moving picture based on the function corresponding to RTP, the sequence number of the RTP header, the payload type, the information from an SSRC and CSRC, and/or the information received by the session information receiving unit 511, and outputs the result to the decoder 517. Only one piece of the video data has to be decoded, therefore, the decode processing load of the decoder 517 does not increase.
In order for the sender (on the part of the sender) to control the quality of the video data received by a receiver A, receiver B, and receiver C, the video data 1 and the video data 2 are given different qualities (compression ratios) and for instance, the video data 1 is given a higher quality.
The information on all the sessions, in which the encoded data of the I picture of the video data 1, the P picture of the video data 1, the I picture of the video data 2, and the P picture of the video data 2 are transmitted to the video data reception device of the receiver A is notified.
To the video data reception device of the receiver B, the information on a session transmitting all the encoded data of the I picture of the video data 1, the I picture of the video data 2, and the P picture of the video data 2 is notified.
To the video data reception device of the receiver C, only the information on a session transmitting the encoded data of the I picture of the video data 2 and the P picture of the video data 2 is notified.
The video data reception device of the receiver A receives the highest quality video (with the smoothest motion) since it receives all the encoded data, and even when there is an error or loss in the video data of the video encoded data 1, the problematic area can be supplemented with the video data of the video encoded data 2.
The video data reception device of the receiver B receives the second highest quality video, and even when there is an error or loss in the I picture encoded data of the video encoded data 1, the problematic area can be supplemented with the I picture encoded data of the video encoded data 2.
The video data reception device of the receiver C can only receive encoded data with a high compression ratio.
As described above, each receiver receives the video with a different quality from one another.
Further, the difference in the video quality among the receivers can be controlled by transmitting specific encoded data to only a specific receiver in the unicast system. For instance, the similar effects can be achieved by transmitting the I picture and P picture encoded data of the video data 2 in multicast or broadcast, transmitting the I picture encoded data of the video data 1 to the receivers A and B in the unicast system, and transmitting the P picture encoded data of the encoded data 1 only to the video data reception device of the receiver A in the unicast system.
Further, when the session information on a session transmitting the I picture and B picture encoded data of the video data 1 is notified to the video data reception devices of all the receivers, the session information on a session transmitting the I picture encoded data of the video data 2 is notified to the video data reception devices of the receivers A and B, and the information on a session transmitting the P picture encoded data of the video data 2 is notified to the video data reception device of the receiver A, the receiver C cannot supplement it when an error or loss occurs in the data received, however, with the encoded data of the video data 2, the receiver B can supplement the error or loss if it occurs in the I picture encoded data and the receiver A can supplement it in the encoded data of both the pictures, and thus resulting in differences in the stability of the video quality among the receivers.
Further, if the video encoded data 1 and the video encoded data 2 have the same quality, or if the video encoded data 1 is transmitted from the video encoded data 1 transmitting units 505 and 506, and the video encoded data 2 transmitting units 507 and 508, it will be possible to keep the degradation in video playback quality, influenced by an error or loss in the transmission path, to a minimum.
Additionally, when the I picture encoded data of the video data 1 and the P picture encoded data of the video data 1 are transmitted in multicast or broadcast, the I picture encoded data of the video data 2 is transmitted to both the receivers A and B in the unicast system, and the P picture encoded data of the video data 2 is transmitted to the receiver A in the unicast system, a similar difference in the stability of the video quality still occurs. In this case, since the bandwidth of the video encoded data transmitted in the unicast system is smaller, it is more preferable for suppressing the increase in traffic.
Further, the similar effects (the differences in the quality of the video received among the receivers) can be achieved by encrypting the I picture and P picture encoded data of the video data 1 for instance, notifying information for restoring the encrypted video data 1 to the video data reception device of the receiver A, notifying only information for restoring the I picture encoded data of the video data 1 to the video data reception device of the receiver B, and not notifying any information regarding the encryption to the video data reception device of the receiver C.
In the present embodiment, when the video data reception device 510 is being used in an environment where the available power is limited such as in battery operation, and/or when it is used in a good reception environment where it can receive the video encoded data with almost no error and loss, receptions of the P picture encoded data of the video data 2 and the I picture encoded data of the video data 2 can be stopped in order to maximize the operable time of the device. On the other hand, when the video data reception device 510 is connected to a battery with a large capacity or AC power, it can be set to receive the encoded data of all the pictures. As described, according to the available power and reception environment, it can control which data to receive. Further, in the present example, the receiver can be allowed to set the picture encoded data that the receiver receives on the video data reception device 510.
It should be noted that the following structure is also possible, i.e., a structure having multiple transmission paths, and transmitting the I picture encoded data of the video data 1, the P picture encoded data of the video data 1, the I picture encoded data of the video data 2, the P picture encoded data of the video data 2, and the session information through different transmission paths.
Further, when encrypting and distributing specific video encoded data, as shown in
When the sequence number of RTP header or information corresponding to it is added to each output of the video data distribution device 501 for all the picture types, the data are reconstructed into one piece (string) of encoded data by rearranging according to the sequence number without adding the discrimination information regarding the picture type, therefore the similar effects can be achieved.
As for the packet receiver-separator unit 1202, it does the same processing as that of the packet receiver-separator unit 702 in the first example.
Further, not only the structure where all the sessions are multiplexed into one, but also a structure where multiple packet multiplex transmitter units multiplexing several sessions are provided, and the similar discrimination information is added is possible.
Further, as in the previous example, it should be noted that each function and processing of each unit in the video data distribution device 501 and the video data reception device 510 can be realized by program control executed on a computer constituting the video data distribution device and the video data reception device in the present embodiment as well.
In the sixth embodiment of the present invention, a video data distribution device comprises means for multicast or broadcast distributing, in different sessions, an I picture, a P picture, and a B picture, if this exists in the encoding methods used for encoding the moving picture, of multiple moving picture data of the same video, the same encoding method, the same frame configuration but having different compression ratios, and intra macroblock (Intra-MB) encoded data of at least a part of frames of the moving picture; and means for notifying the session information on at least one session (multicast address, port number, video quality, video type, etc.), including a quality that a receiver is permitted to receive, to the receiver.
A video data reception device comprises means for receiving data multicast or broadcast distributed in at least one session out of the session information notified; and means for selecting data from the moving picture data received based on the video quality, and reconstructing it into a piece of moving picture data followed by decoding. The sixth embodiment of the present invention may further comprise means for selecting whether or not at least one piece of video data should be received based on at least one of the following: the error/loss rate of received data, available power, and predetermined settings.
Further, the video data distribution device may comprise means for setting whether or not the confidentiality (secrecy) of the distributed data is required; means for securing confidentiality, and the degree of confidentiality for each session of distribution; and means for notifying the setting information regarding confidentiality to the receiver, and the video data reception device may comprise means for receiving the confidential information notified, restoring the data received, and decoding the moving picture data.
The sixth example of the present invention will be described with reference to the drawings.
The video data distribution device 601 comprises a distribution information transmitting unit 602, a session information managing unit 603, a video data storing/generating unit 604, an I picture encoded data transmitting unit 605 for video data 1, a P picture encoded data transmitting unit 606 for video data 1, an intra macroblock (intra-MB) encoded data transmitting unit 607, an I picture encoded data transmitting unit 608 for video data 2, and a P picture encoded data transmitting unit 609 for the video data 2.
The video data reception device 611 comprises a session information receiving unit 612, an I picture encoded data receiving unit 613 for the video data 1, a P picture encoded data receiving unit 614 for the video data 1, an intra macroblock (intra-MB) encoded data receiving unit 615, an I picture encoded data receiving unit 616 for the video data 2, a P picture encoded data receiving unit 617 for the video data 2, an encoded data reconstructing unit 618, and a decoder 619. Hereinafter, the operations of the video data distribution device 601 and the video data reception device 610 will be described in order.
The video data distribution device 601 has the video data storing/generating unit 604 transmit via the transmission path 610:
based on the information set by the session information managing unit 603 from the I picture encoded data transmitting unit 605 for the video data 1, the P picture encoded data transmitting unit 606 for the video data 1, the intra-MB encoded data transmitting unit 607, the I picture encoded data transmitting unit 608 for the video data 2, and the P picture encoded data transmitting unit 609 for the video data 2.
Further, the session information managing unit 603 manages the session information on a session allowed to be distributed to the receiver, and according to the receiver, the session information that should be notified and the information regarding video encoding such as video quality and picture type is notified by the distribution information transmitting unit 602 to the video data reception device 611.
The I picture encoded data transmitting unit 605 for the video data 1, the P picture encoded data transmitting unit 606 for the video data 1, the intra-MB encoded data transmitting unit 607, the I picture encoded data transmitting unit 608 for the video data 2, and the P picture encoded data transmitting unit 609 for the video data 2 have the function of adding information that corresponds to RTP (Real-Time Transport Protocol) so that data can be selected when duplicate (redundant) plural (multiple) video data are received by the video data reception device 611, and the order of the data can be correctly rearranged when the order of the data received is out of order. Further, the respective transmission unit of the I picture encoded data of the video data 1 and the I picture encoded data of the video data 2, and the P picture encoded data of the video data 1 and the P picture encoded data of the video data 2 is data obtained by encoding the same part of the same frame of the same video (image), the same RTP sequence number is added to these data and the intra macroblock encoded data obtained by encoding the same part as these data, and further a payload type and/or SSRC and/or CSRC identifying video encoded data may be added as well.
At this time, by transmitting the I picture encoded data of the video data 1 and the I picture encoded data of the video data 2, and the P picture encoded data of the video data 1 and the P picture encoded data of the video data 2 with a time lag longer than the burst length of data error or loss in the transmission path 610 respectively, the possibility of any error or loss influencing both the video encoded data in the transmission path is reduced, and a more stable distribution of video data can be achieved.
As for the time lag between transmissions, the distribution information transmitting unit 602 can notify the time lag, managed by the session information managing unit 603, to the video data reception device 611, or the receive buffer size can be set by the video data reception device 611 based on the time lag that can be determined in advance between the video data distribution device 601 and the video data reception device 611 decide in advance.
When the session information managing unit 603 is set to make the distributed video data confidential, the I picture encoded data transmitting unit 605 for the video data 1, the P picture encoded data transmitting unit 606 for the video data 1, the intra-MB encoded data transmitting unit 607, the I picture encoded data transmitting unit 608 for the video data 2, and the P picture encoded data transmitting unit 609 for the video data 2 apply encryption processing according to the settings, and the information regarding encryption such as whether or not any encryption is applied and encryption method is notified by the distribution information transmitting unit 602 to the video data reception device 611.
As for the video data reception device 611, the session information, the information regarding video encoding, and the information on encryption if the distribution data from the video data distribution device 601 are encrypted, are received the session information receiving unit 612, and based on this information, the I picture encoded data receiving unit 613 for the video data 1, the P picture encoded data receiving unit 614 for the video data 1, the intra-MB encoded data receiving unit 615, the I picture encoded data receiving unit 616 for the video data 2, and the P picture encoded data receiving unit 617 for the video data 2 open session receive ports, and receive the video data from the transmission path 610. When the video data received is encrypted, the original video data is restored according to the information obtained from the session information receiving unit 612.
Further, the session information managing unit 603 may set the information corresponding to the encoding settings of the video encoded data 1 and the video encoded data 2, encoding options (tools used), the payload options of the encoded data, the RTP header payload type that discriminates the encoded data, and/or an SSRC and CSRC, and the settings may be notified to the video data reception device 611 by the distribution information transmitting unit 602, or these pieces of information may be set in advance between the video data distribution device 601 and the video data reception device 611.
When the encoded data of the I picture of the video data 1, the P picture of the video data 1, the intra macroblock, the I picture of the video data 2, and the P picture of the video data 2 are received, the encoded data reconstructing unit 618 selects data according to the video quality, reconstructs it into a stream of moving picture based on the function corresponding to RTP, the sequence number of the RTP header, the payload type, the information from an SSRC and CSRC, and/or the information received by the session information receiving unit 612, and outputs the result to the decoder 619. Only one piece (string) of the video data has to be decoded, therefore, the decode processing load of the decoder 619 does not increase.
In order for the sender (on the part of the sender) to control the quality of the video data received by the respective video data reception device of a receiver A, receiver B, receiver C, and receiver D, the video data 1 and the video data 2 are given different qualities (compression ratios) and for instance, the video data 1 is given a higher quality.
To the video data reception device of the receiver A the information on a session transmitting the encoded data of the I picture of the video data 1, the P picture of the video data 1, the I picture of the video data 2, and the P picture of the video data 2 is notified.
To the video data reception device of the receiver B, the information on a session transmitting the encoded data of the I picture of the video data 1, the I picture of the video data 2, and the P picture of the video data 2 is notified.
To the video data reception device of the receiver C, only the information on a session transmitting the encoded data of the intra macroblock, the I picture of the video data 2, and the P picture of the video data 2 is notified.
To the video data reception device of the receiver D, only the information on a session transmitting the encoded data of the I picture of the video data 2 and the P picture of the video data 2 is notified.
The video data reception device of the receiver A can receive the highest quality video (with the smoothest motion) since it receives all the encoded data, and even when there is an error or loss in the video data of the video encoded data 1, the problematic area can be supplemented with the video data of the video encoded data 2.
The video data reception device of the receiver B can receive the second highest quality video, and even when there is an error or loss in the I picture encoded data of the video encoded data 1, the problematic area can be supplemented with the I picture encoded data of the video encoded data 2.
The video data reception device of the receiver C only receives the video data 2 and the intra macroblock encoded data with a high compression ratio, however, even when there is an error or loss in the received encoded data, the problematic area will be supplemented if the intra macroblock encoded data of the same area are received.
The video data reception device of the receiver D only receives the video data 2 with a high compression ratio, and when there is an error or loss in the data, the video quality will degrade.
As described above, each receiver receives the video with a different quality from one another.
Further, according to the present example, the difference in the video quality among the receivers can be controlled by transmitting specific encoded data to only a specific receiver in the unicast system. For instance, the similar effects as above can be achieved by transmitting the I picture and P picture encoded data of the video data 2 in multicast or broadcast, transmitting the intra macroblock encoded data to the video data reception device of the receiver C in the unicast system, transmitting the I picture encoded data of the video data 1 to the reception devices of the receivers A and B in the unicast system, and transmitting the P picture encoded data of the encoded data 1 only to the video data reception device of the receiver A in the unicast system.
Further, when the session information on a session transmitting the I picture and B picture encoded data of the video data 1 is notified to the video data reception devices of all the receivers; the session information on a session transmitting the intra macroblock encoded data is notified to the video data reception device of the receiver C; the session information on a session transmitting the I picture encoded data of the video data 2 is notified to the video data reception devices of the receivers A and B; and the session information on a session transmitting the P picture encoded data of the video data 2 is notified to the video data reception device of the receiver A, the receiver D cannot supplement it when an error or loss occurs in the data received, however, the receiver C can supplement the error or loss if the intra macroblock encoded data of the area concerned is received, the receiver B can supplement an error or loss in the I picture encoded data with the encoded data of the video data 2, and the receiver A can supplement it in the encoded data of either one of both the pictures with the encoded data of the video data 2, resulting in differences in the stability of the video quality among the receivers.
Further, if the video encoded data 1 and the video encoded data 2 have the same quality, or the video encoded data 1 is transmitted from the video encoded data 1 transmitting units 605 and 606, and the video encoded data 2 transmitting units 608 and 609, it will be possible to keep the degradation in video playback quality, influenced by an error or loss in the transmission path, to a minimum.
Additionally, when the I picture encoded data of the video data 1 and the P picture encoded data of the video data 1 are transmitted in multicast or broadcast, the intra macroblock encoded data is transmitted to the receiver C in the unicast system, the I picture encoded data of the video data 2 is transmitted to both the receivers A and B in the unicast system, and the P picture encoded data of the video data 2 is transmitted to the receiver A in the unicast system, a similar difference in the stability of the video quality occurs as well. In this case, since the bandwidth of the video encoded data transmitted in the unicast system is smaller, it is more preferable for suppressing the increase in traffic.
Further, the similar effects can be achieved by encrypting the I picture, P picture, and the intra macroblock encoded data of the video data 1 for instance, notifying the information for restoring the encrypted video data 1 to the video data reception device of the receiver A, notifying only the information for restoring the I picture encoded data of the video data 1 to the video data reception device of the receiver B, notifying only the information for restoring the intra macroblock encoded data to the video data reception device of the receiver C, and not notifying any information regarding the encryption to the video data reception device of the receiver D.
In the present embodiment, when the video data reception device 611 is being used in an environment where the available power is limited such as in battery operation, and/or when it is used in a good reception environment where it can receive the video encoded data with almost no error and loss, the receptions of the intra macroblock data and/or the P picture encoded data of the video data 2, or further the I picture encoded data of the video data 2 can be stopped in order to maximize the operation time of the device. On the other hand, when it is connected to a battery with a large capacity or AC power, it can be set to receive the encoded data of all the pictures. As described, according to the available power and reception environment, it can control which data to receive. Further, it is also possible to formulate such that the receiver can be allowed to set the picture encoded data that the video data reception device 611 receives.
Further, for instance, in case where the video encoded data is being received accurately and it is interframe (reference frame) encoded data, even if an error and loss occur in the transmission path, the similar effects as those of CIR (Cyclic Intra Refresh) in the MPEG-4 encoding method will be able to be achieved by having the encoded data reconstructing unit 618 select the intra macroblock encoded data periodically, and the degradation in video playback quality caused by the error and loss in the transmission path can be quickly recovered, suppressing the propagation of the quality degradation over the following frames.
It should be noted that the following structure is also possible, i.e., a structure having a plurality (multiple) transmission paths, and transmitting the I picture encoded data of the video data 1, the P picture encoded data of the video data 1, the intra macroblock encoded data, the I picture encoded data of the video data 2, the P picture encoded data of the video data 2, and the session information through different transmission paths.
Further, when encrypting and distributing specific video encoded data, as shown in
In case where the sequence number of RTP header or information corresponding thereto is added to each output of the video data distribution device 601 for all the picture types, the data are reconstructed into one piece (stream) of encoded data by rearranging according to the sequence number and the discrimination information on the video data type, without adding the discrimination information regarding the picture type, therefore the similar effects can be achieved.
As for the packet receiver-separator unit 1302, it does the similar processing as that of the packet receiver-separator unit 702 in the first embodiment.
Further, not only the structure where all the sessions are multiplexed into one session, but also the following structure is possible where a plurality of packet multiplex transmitter units multiplexing some or several sessions are provided, and the similar discrimination information is added.
As in the previous examples, it should be noted that each function and processing of each unit in the video data distribution device 601 and the video data reception device 611 can be realized by program control executed on a computer constituting the video data distribution device and the video data reception device in the present example as well.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2003-031174 | Feb 2003 | JP | national |
| 2003-391933 | Nov 2003 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP04/01176 | 2/5/2004 | WO | 12/5/2005 |
