Patent Grant
7627184
This application claims priority from PCT Application No. PCT/JP2004/017051 filed Nov. 17, 2004, and from Japanese Patent Application No. 2003-392617 filed Nov. 21, 2003, which applications are incorporated herein by reference.
The present invention relates to contents distributing and receiving apparatuses, a contents transmitting/receiving system, contents distributing and receiving methods, and contents distribution and reception programs, which allow improvements in the resistance to data error and data loss of data encoded by hierarchical encoding in network distribution.
Recently, there has been a spread of contents distribution which distributes contents including moving pictures and audios via a network. Under such circumstances, as a method of efficiently transmitting moving picture data, there is often used a method of transmitting data that is encoded by high-efficient compression based on inter-frame prediction, for example. In these methods, data of prediction parameter obtained by predicting encoded picture from the preceding and following frames in terms of time and the prediction residual picture data are encoded so as to reduce the information amount of the moving picture data which have high correlation in the time direction. Furthermore, the prediction residual picture data is compression-encoded highly efficiently through conversion-encoding and quantization, thereby enabling transmission of contents data with less transmission band.
Typical examples thereof are methods which use compression encoding systems such as MPEG (Moving Picture Experts Group)-1, MPEG-2, and MPEG-4. In these compression encoding systems, inter-frame prediction by motion compensation is performed on the input picture frames by a rectangular area unit of a constant size called a macro block. The obtained motion vector data and signal data that is obtained by compressing the prediction residual picture data through application of two-dimensional discrete cosine conversion and quantization are variable-encoded.
Moreover, for audios, there is also an audio compression method directed to the data encoded by high-efficient compression based on inter-frame prediction such as AAC (Advanced Audio Codec). By employing this method, it becomes possible to distribute the contents data by efficiently utilizing the transmission band of the network.
There are a great number of methods for distributing such contents compression-encoded information to IP (Internet Protocol) network that utilizes a packet switching system. Further, it is expected in the future that such methods be developed into distribution of contents in a mobile communication system. The mobile communication system is a system where mobile stations constituted with PHS (Personal Handyphone Systems), portable telephones, and portable terminals utilizing those as a means for communication are connected through radio stations and radio channels.
There is also known scalable encoding which is capable of varying resolution and picture quality multi-hierarchically (for example, Patent Literature 1)
Patent Literature 1: Japanese Unexamined Patent Publication 9-98434
However, it is not possible with the conventional contents distribution method to properly decode the contents data with an error on the receiver side, if there is generated a transmission data error or transmission packet fault with a long burst characteristic that cannot be decoded even with an error correcting code.
Further, it is common to employ a method of requesting retransmission of the lost data when there is generated a packet loss. However, it is not possible with the contents distribution of multicast/broadcast to employ a method which transmits the error of transmitted data and fault information of the transmission packet from the receiver side to the transmitter side.
When there is generated a packet loss in the case of picture data, for example, there is an error concealment method taken as a measure for the receiver side to generate picture data so as to conceal the error as much as possible from the pictures of the properly-decoded preceding and following frames in terms of time or the peripheral picture data within the same frame. However, it is impossible to eliminate the disturbance of the decoded picture. Furthermore, since the inter-frame prediction is employed, the disturbances of the picture and audio generated once may propagate to the following frames.
In addition, when the error information is returned from the receiver side and transmitted to the transmitter side, the transmission band of the Internet is occupied by this feedback information and the retransmitted data.
The present invention is designed in view of the aforementioned circumstance. The first object of the present invention is to provide contents distributing and receiving apparatuses, a contents transmitting/receiving system, contents distributing and receiving methods, and contents distribution and reception programs, which can suppress as much as possible the significant disturbance of contents reproduced on the receiver side, which is generated due to the transmission error of the encoded data.
Further, the second object of the present invention is to provide contents distributing and receiving apparatuses, a contents transmitting/receiving system, contents distributing and receiving methods, and contents distribution and reception programs, which allow users to set trade-off between the transmission band used for contents transmission and the quality.
Furthermore, the third object of the present invention is to provide contents distributing and receiving apparatuses, a contents transmitting/receiving system, contents distributing and receiving methods, and contents distribution and reception programs, which can achieve the above-described first object of the present invention without sending the feedback information from the receive side to the transmitter side.
Moreover, the fourth object of the present invention is to provide contents distributing and receiving apparatuses, a contents transmitting/receiving system, contents distributing and receiving methods, and contents distribution and reception programs, which allow the transmitter side to control the quality and/or stability of the receive-side contents through controlling whether or not to perform encryption at the time of transmission and/or controlling information to be sent to the receiver side by call connection processing.
Further, the fifth object of the present invention is to provide contents distributing and receiving apparatuses, a contents transmitting/receiving system, contents distributing and receiving methods, and contents distribution and reception programs, which can achieve the aforementioned objects while preventing an increase in the operation amount required for decoding the compression-encoded data.
Furthermore, the sixth object of the present invention is to provide contents distributing and receiving apparatuses, a contents transmitting/receiving system, contents distributing and receiving methods, and contents distribution and reception programs, which can control the power consumption of the receiving apparatus in accordance with the power that can be used on the receiving apparatus side.
In order to achieve the aforementioned objects, a contents distributing apparatus according to the present invention comprises
a data output device for outputting encoded data, and
a transmitting device for transmitting the encoded data outputted from the data output device, wherein:
the data output device outputs data encoded by hierarchical encoding as the encoded data; and
the transmitting device transmits at least a part of data of at least one layer among the data encoded by the hierarchical encoding.
Further, the data output device may output the data encoded by the hierarchical encoding by separating it to respective encoded data of each layer, and the transmitting device may transmit each of the encoded data individually by each layer. Furthermore, the transmitting device may transmit at least a part of referred-information encoded data of at least one layer among the data encoded by the hierarchical encoding.
Further, the transmitting device may transmit, among the data encoded by the hierarchical encoding: a) at least a part of encoded data of one layer; and b) at least a part of referred-information encoded data of at least one layer among remaining layers. Furthermore, the transmitting device may transmit, among the data encoded by the hierarchical encoding: a) at least a part of referred-information encoded data of one layer; and b) at least a part of encoded data of at least one layer among remaining layers.
Also, the transmitting device may comprise a transmission managing unit, wherein the transmission managing unit controls quality and stability and/or confidentiality of distribution contents on the contents distributing apparatus side through controlling at least one of: number of the encoded data to be transmitted; hierarchy of the encoded data; distribution target of a cipher key; encryption method; and degree of encryption. Further, when performing data transmission of a plurality of pieces of encoded data with a time difference provided therebetween, the transmission managing unit may change a compression rate of following encoded data with respect to a compression rate of preceding encoded data with the time difference. Furthermore, the transmission managing unit may select the compression rate in accordance with a distribution rate and/or condition of a transmission line. Moreover, the transmission managing unit may select whether or not to transmit at least a part of the encoded data in accordance with the distribution rate and/or the condition of the transmission line.
Further, the transmitting device may perform data transmission including encoded data of layers higher than hierarchy of the data to be transmitted. Furthermore, the transmission managing unit may control the quality and stability of the distribution contents on a contents distributing side through controlling informing target of session information of a session by which the encoded data is transmitted.
A contents receiving apparatus according to the present invention comprises: a device for receiving encoded data transmitted by a plurality of sessions; a device for receiving the encoded data received by the receiving device, and discriminating and separating individual encoded data units therefrom; and a reconstruction device which extracts encoded data received without a transmission error and a fault from the discriminated and separated encoded data, and reconstructs and outputs encoded data from the extracted encoded data.
Further, when reconstructing the encoded data, the reconstruction device may judge duplication of the encoded data from identifiers given to encoded data transmission units. Furthermore, the reconstruction device may judge a compression rate and/or hierarchy of the encoded data from at least one of: a) an encoded data receiving session determined in advance; b) encoded data identifying information determined in advance, which is given to the encoded data transmission unit; c) an encoded data receiving session informed by call connection processing; and d) the encoded data identifying information which is given to the encoded data transmission unit and informed by the call connection processing. Moreover, there may be provided a report transmitting unit for transmitting a receiving state report to inform condition of a transmission line.
A contents transmitting/receiving system according to the present invention comprises a contents distributing apparatus, a contents receiving apparatus, and a communication network for connecting the contents distributing apparatus and the contents receiving apparatus, wherein:
the contents distributing apparatus comprises
a data output device for outputting encoded data, and
a transmitting device for transmitting at least a part of data of at least one layer among the data encoded by the hierarchical encoding outputted from the data output device; and
the contents receiving apparatus comprises
a device for receiving encoded data transmitted by a plurality of sessions;
a device for receiving the encoded data received by the receiving device, and discriminating and separating individual encoded data units therefrom; and
a reconstruction device which extracts encoded data received without a transmission error and a fault from the discriminated and separated encoded data, and reconstructs and outputs encoded data from the extracted encoded data.
A contents distributing method according to the present invention comprises the steps of:
an output step of outputting encoded data; and
a transmission step of transmitting the encoded data outputted in the output step, wherein
data encoded by hierarchical encoding is outputted as the encoded data in the output step, and
at least a part of data of at least one layer among the data encoded by the hierarchical encoding is transmitted in the transmission step.
In the output step, the data encoded by the hierarchical encoding may be separated to respective encoded data of each layer to be outputted and, in the transmission step, each of the encoded data may be transmitted by each layer. Further, in the transmission step, there may be transmitted at least a part of referred-information encoded data of at least one layer among the data encoded by the hierarchical encoding. Furthermore, in the transmission step, among the data encoded by the hierarchical encoding, there may be transmitted: a) at least a part of encoded data of a first layer; and b) at least a part of referred-information encoded data of at least one layer among remaining layers.
Moreover, in the transmission step, among the data encoded by the hierarchical encoding, there may be transmitted: a) at least a part of referred-information encoded data of the first layer; and b) at least a part of encoded data of at least one layer among remaining layers.
The quality and stability and/or confidentiality of distribution contents may be controlled on the contents distributing apparatus side through controlling, on the contents distributing side, the number of the encoded data to be transmitted, hierarchy of the encoded data, distribution target of a cipher key and/or encryption method and/or degree of encryption. Further, when performing data transmission of a plurality of pieces of encoded data with a time difference provided therebetween, a compression rate of following encoded data may be changed with respect to a compression rate of preceding encoded data with the time difference. Furthermore, the compression rate may be selected in accordance with a distribution rate and/or condition of a transmission line. Moreover, data transmission may be performed including encoded data of layers higher than hierarchy of the data to be transmitted. Further, the quality and stability of the distribution contents may be controlled on a contents distributing side through controlling informing target of session information of a session by which the encoded data is transmitted.
The output step may comprise at least one of the steps of: (a) an input step of first to N-th encoded data; (b) an input step of encoded data, and a generating step of the first to N-th encoded data from data of at least one layer among the encoded data inputted in the input step;(c) an input step of a first encoded data, and a generating step of second to N-th encoded data from data of at least one layer of the first encoded data inputted in the input step; (d) an encoding step of encoding an inputted signal into the first to N-th encoded data; and (e) an encoding step of encoding the inputted signal into the first encoded data, and a generating step of the second to N-th encoded data from data of at least one layer of the first encoded data inputted in the input step.
(N+1)-th encoded data may be outputted in addition to the N-th encoded data. Identifiers for identifying data may be given to the first to (N+1)-th encoded data. Each of the first to (N+1)-th encoded data may be transmitted by a different session. The first to (N+1)-th encoded data may be multiplexed to be transmitted. Among the first to (N+1)-th encoded data, at least two pieces of encoded data may be multiplexed, and the multiplexed encoded data and remaining encoded data without multiplexing may be transmitted, respectively, by different sessions. A contents distributing side may control an informing target of session information of a session by which the encoded data is transmitted. The first to (N+1)-th encoded data may be distributed with a time difference provided therebetween. The time difference may be set in accordance with condition of a transmission line and/or an encoding compression rate and/or a distribution rate and/or a rule determined in advance.
A contents receiving method according to the present invention comprises the steps of: a receiving step of receiving encoded data transmitted by a plurality of sessions; a step of receiving the encoded data received in the receiving step, and discriminating and separating individual encoded data units therefrom; and a reconstruction step of extracting encoded data received without a transmission error and a fault from the discriminated and separated encoded data, and reconstructing and outputting encoded data from the extracted encoded data.
When reconstructing the encoded data, duplication of the encoded data may be judged from identifiers given to encoded data transmission units. Further, a compression rate and/or hierarchy of the encoded data may be judged from at least one of: a) an encoded data receiving session determined in advance; b) encoded data identifying information determined in advance, which is given to the encoded data transmission unit; c) an encoded data receiving session informed by call connection processing; and d) the encoded data identifying information which is given to the encoded data transmission unit and informed by the call connection processing. A receiving state report may be transmitted for informing condition of a transmission line.
In the receiving step, the encoded data may be received by securing a buffer size determined by at least one of: (a) a receiving buffer size determined in advance; (b) a buffer size informed by call connection processing; and (c) a buffer size calculated based on a contents distributing rate and time-difference setting information, which is set in advance and/or informed by call connection.
A contents transmitting/receiving method, comprising the steps of: an output step of outputting data encoded by hierarchical encoding; a transmission step of transmitting at least a part of data of at least one layer among the data encoded by the hierarchical encoding; a receiving step of receiving encoded data transmitted by a plurality of sessions; a step of discriminating and separating individual encoded data units from the received encoded data; and a reconstruction step of extracting encoded data received without a transmission error and a fault from the discriminated and separated encoded data, and reconstructing and outputting the extracted encoded data.
A contents distribution program according to the present invention allows a computer that constitutes a contents distributing apparatus to execute: a data processing function for outputting encoded data; and a transmission processing function for transmitting the encoded data from the data section, wherein
data encoded by hierarchical encoding is outputted as the encoded data through execution of the data processing function, and at least a part of data of at least one layer among the data encoded by the hierarchical encoding is transmitted through execution of the transmission processing function.
The data encoded by the hierarchical encoding may be separated to respective encoded data of each layer to be outputted through execution of the data processing function, and each of the encoded data may be transmitted individually by each layer through execution of the transmission processing function. There may be provided a function of transmitting at least a part of referred-information encoded data of at least one layer among the data encoded by the hierarchical encoding.
There may be provided a function of transmitting, among the data encoded by the hierarchical encoding: a) at least a part of encoded data of a first layer; and b) at least a part of referred-information encoded data of at least one layer among remaining layers. There may be provided a function of transmitting, among the data encoded by the hierarchical encoding: a) at least a part of referred-information encoded data of a first layer; and b) at least a part of encoded data of at least one layer among remaining layers.
There may be provided a function of controlling quality and stability and/or confidentiality of distribution contents on the contents distributing apparatus side through controlling the number of the encoded data to be transmitted, hierarchy of the encoded data, distribution target of a cipher key and/or encryption method and/or degree of encryption. When performing data transmission of a plurality of pieces of encoded data with a time difference provided therebetween, a compression rate of following encoded data may be changed with respect to a compression rate of preceding encoded data with the time difference. There may be provided a function of selecting the compression rate in accordance with a distribution rate and/or condition of a transmission line. There may be provided a function of performing data transmission including encoded data of layers higher than hierarchy of the data to be transmitted. There may be provided a function of controlling the quality and stability of the distribution contents on a contents distributing side through controlling informing target of session information of a session by which the encoded data is transmitted.
A contents reception program according to the present invention allows a computer that constitutes a contents receiving apparatus to execute: a reception processing function for receiving encoded data transmitted by a plurality of sessions; a processing function for receiving the encoded data received in the receiving step, and discriminating and separating individual encoded data units therefrom; and a reconstruction processing function for extracting encoded data received without a transmission error and a fault from the discriminated and separated encoded data, and reconstructing and outputting encoded data from the extracted encoded data.
There may be provided a function of judging duplication of the encoded data from identifiers given to encoded data transmission units when reconstructing the encoded data. Further, there may be provided a function of judging a compression rate and/or hierarchy of the encoded data from at least one of: a) an encoded data receiving session determined in advance; b) encoded data identifying information determined in advance, which is given to the encoded data transmission unit; c) an encoded data receiving session informed by call connection processing; and d) the encoded data identifying information which is given to the encoded data transmission unit and informed by the call connection processing.
With the present invention as described above, when distributing the contents from the transmitter side to the receiver side, the transmitter side of the contents distributing apparatus transmits, by a session, at least a part of the data of at least one layer among the data encoded by hierarchical (scalable) encoding. Therefore, when distributing the contents via a network, it is possible to improve the reliability of the data received by the receiver even under distribution via an unstable transmission line such as a radio transmission line, while suppressing an increase in the amount of distribution data to be small. In addition, the contents can be provided with the stable quality. Moreover, it is possible to provide the contents with the quality that is best suited for the receiving environment of the contents receiver.
Furthermore, by changing the compression rates of the preceding and following distribution data with a time difference provided therebetween, expansion of the transmission band of the Internet network can be suppressed. Thus, increases in the processing amount for improving the reliability of the distribution data can be suppressed as small on both the transmitter and receiver sides.
Through individually transmitting the transmission data by a plurality of sessions when transmitting picture data, there is provided the low possibility for the entire data to receive an influence of the error and loss, thereby decreasing the deterioration in the picture due to the error and loss.
Further, the user can set the trade-off between the transmission band used for contents data transmission and the quality by setting the compression rate of the picture data (encoded data) of the hierarchical structure in accordance with the transmission band. For the compression rate, when there is no margin in the transmission band, for example, the compression rate of the following picture data is set higher. However, if there is a fault in the preceding picture data in this case, it is refilled with the following picture data. Thus, the picture quality of that section can be made better than the case of decoding it with the fault remained. The compression rate can be set in accordance with such trade-off between the transmission band and the quality to be maintained.
In general, when there is a fault in the received data, the receiver side requests (feedback) the retransmission of the fault data to the transmitter side. In the invention, however, the fault data is refilled by using the data transmitted in parallel, so that a retransmission request for the transmitter side becomes unnecessary.
Furthermore, the quality of the receiver side can be controlled on the distributing side by informing, from the distributing side to the receiver side, only the setting (multicast address, port number, used encoding tool for the case of multicast distribution) of the distributing session of the data qualified to be received among the picture data (encoded data) of a hierarchical structure. Thus, the quality and the stability of the contents on the receiver side can be controlled on the transmitter side.
For example, both the above-described setting of the session for distributing the data with a low compression rate and the setting of the session for distributing the data with a high compression rate are informed to a receiver A, and only the above-described setting of the session for distributing the data with a high compression rate is informed to a receiver B. With this, the quality (picture quality, loss resistance) of the received data of the receiver A and the receiver B can be controlled.
For the received data, detection of the fault and refilling the fault data is carried out before decoding, and the data that has been refilled and reconstructed at last is decoded. Therefore, it is possible to prevent an increase in the operation amount required for decoding the compression-encoded data. That is, unlike the technique which decodes all of a plurality of pieces of received data and refills the fault data with the decoded result, the present invention can suppress the increase in the operation amount to be small.
Moreover, the power consumption on the receiver side can be suppressed to be small by controlling the receiving number of encoded data in accordance with the power that can be used on the receiver side. With this, the receivable time of the contents under a battery-use environment can be extended dramatically.
[
[
[
[
[
[
[
[
[
[
[
[
[
[
Embodiments of the present invention will be described hereinafter by referring to the accompanying drawings.
As the fundamental structure, a contents distributing apparatus according to the present invention comprises data output devices (104, 804, 1104, 1204, 1404) for outputting encoded data and transmitting devices (106-109, 806-809, 1106-1109, 1206-1209, 1407-1409, 1506-1509) for transmitting the encoded data outputted from the data output devices. The data output devices output the data encoded by hierarchical encoding as the encoded data. The transmitting devices transmit at least a part of data of at least one layer among the data encoded by the hierarchical encoding.
A contents distributing method for distributing contents by utilizing the content distributing apparatus according to the present invention executes an output step of outputting the encoded data by the data output devices, and a transmission step of transmitting the encoded data outputted from the data output devices in the output step by the transmitting device. In the output step, the data encoded by hierarchical encoding is outputted as the encoded data and, in the transmission step, at least a part of data of at least one layer is transmitted among the data encoded by the hierarchical encoding.
As the fundamental structure, a contents receiving apparatus according to the present invention for receiving the contents data distributed from the contents distributing apparatus comprises: devices (114-117, 820, 1120, 1117, 1214-1217, 1420, 1520) for receiving the encoded data transmitted by a plurality of sessions; devices (118, 818, 1118, 1218, 1418, 1518) for receiving the encoded data received in the receiving units and discriminating and separating the individual encoded data units therefrom; and reconstruction devices (118, 818, 1118, 1218, 1418, 1518) for extracting the encoded data received without transmission error with no fault from the discriminated and separated encoded data, and reconstructing and outputting the encoded data from the extracted encoded data.
A contents receiving method for receiving the contents distributed by using the contents distributing apparatus according to the present invention executes: a receiving step of receiving the encoded data transmitted by a plurality of sessions by the receiving devices; a step of discriminating and separating the individual encoded data units from the encoded data received in the receiving step; and reconstruction step of extracting the encoded data received without transmission error with no fault from the discriminated and separated encoded data, and reconstructing and outputting the encoded data from the extracted encoded data.
In order to build a contents transmitting/receiving system for distributing contents between the transmitter side and the receiver side, the present invention comprises, as the fundamental structure, the above-described contents distributing apparatus, the above-described contents receiving apparatus, and communication networks (130, 830, 1130, 1230, 1430, 1530) for connecting the contents distributing apparatus and the contents receiving apparatus, wherein at least a part of the data of at least one layer among the data encoded by the hierarchical encoding is transmitted from the contents distributing apparatus, and the transmitted encoded data is reconstructed by the contents receiving apparatus.
As the fundamental structure, a contents transmitting/receiving method for performing contents distribution between the transmitter side and the receiver side, the present invention executes each step of the contents distributing method on the contents distributing side and executes each step of the contents receiving method on the contents receiver side, wherein the contents distributing side transmits at least a part of the data of at least one layer among the data encoded by the hierarchical encoding, and the contents receiver side reconstructs the transmitted encoded data.
Further, a contents distribution program for executing the contents distributing method by the contents distributing apparatus according to the present invention, as the fundamental structure, allows a computer that constitutes the contents distributing apparatus to execute a data processing function for outputting the encoded data and a transmission processing function for transmitting the encoded data from the data section, thereby comprising the functions of: outputting the data encoded by the hierarchical encoding as the encoded data through executing the data processing function; and transmitting at least a part of the data of at least one layer among the data encoded by the hierarchical encoding by executing the transmission processing function.
Furthermore, a contents reception program for executing the contents receiving method by the contents receiving apparatus according to the present invention, as the fundamental structure, allows a computer that constitutes the contents receiving apparatus to execute the functions of: a reception processing function for receiving the encoded data transmitted by a plurality of sessions; a processing function for receiving the encoded data received by the receiving devices and discriminating and separating the encoded data unit therefrom; and a reconstruction processing function for extracting the encoded data received without transmission error with no fault from the discriminated and separated encoded data, and reconstructing and outputting the encoded data from the extracted encoded data.
Next, the present invention will be described in detail by referring to specific examples. A first embodiment shown in
The embodiment comprises, on the transmitter side, first to N-th encoded data transmitting devices and an N+1-th encoded data transmitting device, and each transmitting device transmits the encoded data by a different session from each other. The receiver side comprises first to N-th encoded data receiving devices and an N+1-th encoded data receiving device, which extract the encoded data received without transmission error with no fault and performs reconstruction and output thereof.
In the embodiment, the contents distributing apparatus may comprise a device for setting routing priority control in the transmission lines and power control in the radio transmission lines for each session in at least one of the sessions for transmitting the encoded data, and a picture data receiving apparatus may comprise a device for selecting reception of at least one session based on at least one of: the error/loss rate of the received data; power that can be used in the receiving apparatus; and the setting determined in advance.
As shown in
The contents receiving apparatus 111 is a client terminal connected to the transmission line 130. For simplifying the explanation, it is assumed that the contents to be distributed herein are moving picture data.
The contents distributing apparatus 101 transmits the N-number of moving-picture encoded data encoded by the scalable encoding method of M-layers with “M” and “N” being integers of 2 or larger from the first to N+1-th transmitting units 106-109 to the contents receiving apparatus 111 via the transmission line 130 based on the transmission setting set in a transmission managing unit 103. The contents receiving apparatus 111 receives the moving-picture encoded data by the first to N+1- the receiving units 114-117 according to call processing performed between a call connection processing unit 112 of the contents receiving apparatus 111 and a call connection processing unit 102 of the contents distributing apparatus 101, and reconstructs and decode one piece of the moving-picture encoded data among the moving-picture encoded data received without data error and fault in accordance with the compression rate.
In the followings, the contents distributing apparatus 101 and the contents receiving apparatus 111 will be described in detail.
The transmission managing unit 103 of the contents distributing apparatus 101 sets at least one of:
The data reading/generating/converting unit 104 outputs the N-number of moving-picture scalable encoded data constituted with M-layers to the first to N+1-th transmitting units 106-109, which are obtained through one of:
A data reading/generating/converting unit 200 of
A data reading/generating/converting unit 200 of
A data reading/generating/converting unit 200 of
As in the cases of
Further, like the second to N-th encoded data conversion units 405-406 as shown in
The above-described first to N-th encoded data outputted from the data reading/generating/converting units 200 shown in
The first to N+1-th transmitting units 106-109 transmit the first to N-th encoded data to the contents receiving apparatus 111 via the transmission line 130 based on the setting in the transmission managing unit 103 and/or the setting determined in advance between the contents distributing apparatus 101 and the contents receiving apparatus 111. This processing will be described by referring to
A transmitting unit 500 shown in
The contents receiving apparatus Ill can grasp the compression rate and/or the layer of the encoded data based on the call connection processing and/or the correspondence between the compression rate and the identifying information determined in advance between the contents distributing apparatus 101 and the contents receiving apparatus 111. An error detection code adding unit 505 adds the check sum of the UDP header or the information corresponding thereto. There is added delay to the packet outputted from the packet processing unit 501 according to the delay amount set in a delay adding unit 506 such that the packet is outputted with a time difference provided between other transmitting units.
The encoded data dividing unit 502 divides the inputted encoded data into the encoded data transmission units to be the data of the same layer, which is obtained by scalable-encoding the information of the same frame and same part by the first to N+1-th transmitting units 106-109, like the video packet of the MPEG-4 encoding, for example. In this case, the encoded data identifier adding unit 504 adds, to the same transmission units of each transmitting unit, the sequence number of the RTP header or the same identification number corresponding thereto. Thereby, the contents receiving apparatus 111 can select the data by the transmission unit when there is a plurality of pieces of duplicated encoded data being received. Furthermore, it is possible to rearrange the encoded data correctly even if the order of the received encoded data is switched.
Through performing encryption of the first to N+1-th transmitting units 106-109 only on the specific hierarchy or lower or only by the specific transmitting unit and thereafter and controlling the distribution of the cipher key by each contents receiving apparatus 111, it is possible on the contents distributing apparatus 101 side to control the degree of the quality (picture quality) and the quality stability of the contents reproduced by the contents receiving apparatus 111. Furthermore, when the RTCP (RTP Control Protocol) receiving-state report or the information corresponding thereto can be obtained from the contents receiving apparatus 111, the delay amount added in the first to N+1-th transmitting units 106-109 can be kinetically altered even during the distribution of contents, e.g., increases the delay amount when the packet loss rate is high and/or burst length of the packet loss is long. In such a case, it is desirable to inform the altered delay amount to the contents receiving apparatus 111 from the call connection processing unit 102 of the contents distributing apparatus 101.
For delaying, any one of the first to N-th encoded data may be transmitted at first. There may be employed a method in which a time difference is provided between the encoded data transmitted at first and the data transmitted thereafter or a method in which the encoded data is transmitted successively by providing a time difference therebetween. A transmission data selecting unit 507 selects the packet of the encoded data to be transmitted based on the transmission data selection setting, and transmits the packet from a transmitting unit 508 to the transmission line 130 by each session in accordance with the routing priority in the transmission line and the setting of the transmission power of the radio transmission line.
Regarding the first to N-th encoded data transmitted from the first to N+1-th transmitting units 106-109, the followings may be performed:
The N+1-th transmitting unit 109 transmits the encoded data of the referring information of the first layer of the first encoded data where prediction in the time direction is performed, and the encoded data of the second layer and thereafter.
As has been described above, among the encoded data transmitted in any of the above-described ways, the packet to be transmitted is appropriately selected further in accordance with the characteristic of the picture and the condition of the transmission line. For this selection, the packet may be selected and transmitted by each specific cycle (once for every n-packets), for example. Alternatively, the packet to be transmitted may be determined appropriately by referring to the characteristic parameter within the encoded data, e.g., the motion vector, which is a parameter having a large influence on the decoded picture by the bit error or the packet loss, or may be selected in accordance with a rule that is, for example, to select the referred frame (intra-frame) at all times. This selecting method is capable of changing the selection setting kinetically even during distribution of the contents.
Moreover, for keeping the high quality (picture quality) and the high stability of the contents to be reproduced in the contents receiving apparatus 111, it is desirable to set such that the data of the higher layer among the first to N-th scalable encoded data transmitted at this time is transmitted in more transmitting units and the data of the higher layer is transmitted without fail. Therefore, in the case where it is necessary to reduce the amount of the transmission encoded data, the transmission data selecting unit 507 stops the transmission of the encoded data from the lower layer among the encoded data transmitted from the transmitting units.
Furthermore, for efficiently using the band of the network, either of the following methods may be employed:
Further, the contents distributing apparatus 101 can control the degree of the quality (picture quality) and the quality stability of the contents, to be reproduced by each contents receiving apparatus 111 through controlling the session information that is informed by the call connection processing among the first to N+1-th distributing sessions.
Furthermore, the first to N+1-th transmitting units 106-109 can employ either of the following methods:
Next, the contents receiving apparatus 111 will be described in detail.
A call connection processing unit 112 establishes a call by exchanging at least one of the followings between the call connection processing unit 102 of the contents distributing apparatus 101 via the transmission line 130:
When necessary, the report transmitting unit 113 transmits the RR (Receiver Report) of RTCP or the information corresponding thereto to a report receiving unit 105 of the contents distributing apparatus 101 via the network 130 for reporting the receiving state to the contents distributing apparatus 101.
The first to N+1-th receiving units 114-117 receive the encoded data from the respective sessions. Each receiving unit checks whether or not there is an influence of the data error being imposed during the transmission by the check sum of UDP or a corresponding function. When there is an error, the packet is discarded. When there is no error, the packet is outputted to the encoded data reconstruction unit 118. Even if there is an error contained in the received packet, it is possible to use the extracted encoded data without discarding it provided that: the information corresponding to the data identifying information can be obtained form the received session or the like; the encoded data can be extracted; and it is found that there is only an error in an unimportant part of the encoded data.
At this time, when the picture data receiving apparatus 111 is used under the environment with limited useable electric power like the case where it is operated by batteries/cells, for example, and/or when the receiving environment is fine and the encoded data of the highest quality can be received with almost no error and fault, there may be performed a control for stopping the reception of other encoded data and extending the use time of the apparatus as much as possible. Inversely, when the picture data receiving apparatus 111 has a large-capacity battery or an AC power source connected thereto, all the encoded data may be received. Like the manner described above, reception of the data can be controlled in accordance with the useable power and the receiving environment. Alternatively, it may be structured such that the number of receiving encoded data can be set in the picture data receiving apparatus 111 by the user.
Next, the encoded data reconstruction unit 118 will be described by referring to
A decoder 119 decodes the reconstructed encoded data and reproduces the media. Decoding is carried out only on one piece of the reconstructed encoded data, so that there is no increase in the processing amount required for decoding in the present invention.
A plurality of the transmission lines 130 maybe provided so that each distribution session of the first to N+1-th encoded data transmission units, the call connection processing information, and the receiving state report information are transmitted through the different transmission lines.
Furthermore, it is needless to say that each function and processing of the call connection processing unit 102, the transmission managing unit 103, the data reading/generating/converting unit 104, the report receiving unit 105, and the first to N+1-th transmitting units 106-109 of the contents distributing apparatus 101 may be achieved by the program control executed by a computer that constitutes the contents distributing apparatus 101. It is also needless to say that each function and processing of the call connection processing unit 112, the report transmitting unit 113, the first to N+1-th receiving units 114-117, the encoded data reconstruction unit 118, the decoder 119 of the contents receiving apparatus 111 may be achieved by the program control executed by a computer that constitutes the contents receiving apparatus 111.
In the manner described below, a second embodiment of the present invention performs, between the transmitter side and the receiver side, contents distribution of the N-number of encoded data of the same contents of the scalable encoding constituted with M-layers where, “M” and “N” are integers of 2 or larger.
The transmitter side comprises the first to N+1-th data transmitting devices and a device for performing multiplexing and transmission of the outputs thereof, and transmits the multiplexed data by the same session. The receiver side comprises a device for receiving the multiplexed encoded data, which extracts the encoded data that is received without a transmission error and a fault, and performs reconstruction and output thereof.
Further, like the first embodiment, the picture data transmitting apparatus according to the second embodiment of the present invention may have a structure with a device for setting the routing priority control of the transmission lines in the session for transmitting the encoded data and the power control of the radio transmission lines.
A contents distributing apparatus 801, a contents receiving apparatus 811, and a transmission line 830 according to the second embodiment of the present invention shown in
The first to N+1-th transmitting units 806-809 of
The packet multiplexing/transmitting unit 810 multiplexes the packet data from the first to N+1-th transmitting units 806-809 and transmits it to the contents receiving apparatus 811 via the transmission line 830. In the embodiment, there is a single distributing session. Thus, there is no control of the quality and the stability of the contents performed by controlling the transmitting target of the session information, and the transmitting method is one of multicast, broadcast, or unicast transmission.
The packet receiving unit 820 performs the same processing as that of the first to N+1-th receiving units 114-117 of the first embodiment. The encoded data reconstruction unit 818 performs the same processing as that of the encoded data reconstruction unit 118 of the first embodiment. However, unlike the first embodiment, the encoded data reconstruction unit 818 cannot obtain the information corresponding to the encoded data identifying information form the received session.
Except for those described above, each part is the same as the corresponding part of the first embodiment.
A plurality of the transmission lines 830 maybe provided so that the distribution session of the multiplexed first to N-th encoded data, the call connection processing information, and the receiving state report information are transmitted through the different transmission lines.
Furthermore, it is needless to say that each function and processing of the call connection processing unit 802, the transmission managing unit 803, the data reading/generating/converting unit 804, the report receiving unit 805, the first to N+1-th transmitting units 806-809, and the packet multiplexing/transmitting unit 810 of the contents distributing apparatus 801 may be achieved by the program control executed by a computer that constitutes the contents distributing apparatus 801. It is also needless to say that each function and processing of the call connection processing unit 812, the report transmitting unit 813, the encoded data reconstruction unit 818, the decoder 819, and the packet receiving/separating unit 820 of the contents receiving apparatus 811 may be achieved by the program control executed by a computer that constitutes the contents receiving apparatus 811.
In a third embodiment of the present invention, the transmitter side and the receiver side operate in the following manner for the N-number of encoded data of the same contents encoded by the scalable encoding constituted with M-layers where, “M” and “N” are integers of 2 or larger.
The transmitter side comprises the first to N+1-th data transmitting devices and at least one device for multiplexing and transmitting at least two of the outputs therefrom, which transmits the multiplexed data and the data without multiplexing by the respective sessions. The receiver side comprises a device for receiving the multiplexed data and the data without multiplexing by the respective sessions, which extracts the encoded data received without transmission error and a fault for performing reconstruction and output thereof.
Further, like the first embodiment, the picture data transmitting apparatus according to the third embodiment of the present invention also comprises a device for setting, by each session, the routing priority control of the transmission lines in at least one session for transmitting the encoded data and the power control of the radio transmission lines. The picture data receiving apparatus may be in a structure having a device for selecting the reception of at least one session based on at least one of: the error/loss rate of the received data; power that can be used in the receiving apparatus; and the setting set in advance.
In the third embodiment of the present invention shown in
Among the first to N+1-th transmitting units 1106-1109, the first to N-th transmitting units 1106-1108 perform the same processing as that of the first to N-th transmitting units 806-808 of
The N+1-th receiving unit 1117 performs the same processing as that of the N+1-th receiving unit 117 of
The encoded data reconstruction unit 1118 performs the same processing as that of the encoded data reconstruction unit 818 of
Except for those described above, each part is the same as the corresponding part of the first embodiment.
In the embodiment, there has been described that the outputs of the first to N-th transmitting units 1106-1108 are multiplexed and transmitted, which are then received and separated. However, it is possible to multiplex and separate the outputs of the first to N-th transmitting units 1106-1108 in arbitrary combinations.
In the embodiment, there has been described that there are one each of the packet multiplexing/transmitting unit 1110 and the packet receiving/separating unit 1120. However, the same effect can be achieved with a plurality of those units. Further, with a plurality of transmitting/receiving units for transmitting/receiving the packets by the respective separate sessions without interposing the packet multiplexing/transmitting unit 1110 and the packet receiving/separating unit 1120, it is also possible to achieve the same effect. In this case, it is needles to say that the contents distributing apparatus 1101 side can control the quality and the stability of the reproduced contents by each contents receiving apparatus 1111 through control of the session information informed by the call connection processing as in the first embodiment. Furthermore, it is needless to say that the multicast, broadcast, or unicast can be selected for each distributing session.
A plurality of the transmission lines 1130 may be provided so that the session of distributing the multiplexed output of the first to N-th transmitting units 1106-1108, the session of distributing the output of the N+1-th transmitting unit 1109, the call connection processing information, and the receiving state report information are transmitted through the different transmission lines.
Furthermore, it is needless to say that each function and processing of the call connection processing unit 1102, the transmission managing unit 1103, the data reading/generating/converting unit 1104, the report receiving unit 1105, the first to N+1-th transmitting units 1106-1109, and the packet multiplexing/transmitting unit 1110 of the contents distributing apparatus 1101 may be achieved by the program control executed by a computer that constitutes the contents distributing apparatus 1101. It is also needless to say that each function and processing of the call connection processing unit 1112, the report transmitting unit 1113, the encoded data reconstruction unit 1118, the decoder 1119, and the packet receiving/separating unit 1120 of the contents receiving apparatus 1111 may be achieved by the program control executed by a computer that constitutes the contents receiving apparatus 1111.
In a fourth embodiment of the present invention, the transmitter side and the receiver side operate in the following manner for the N-number of encoded data of the same contents encoded by the scalable encoding constituted with M-layers where, “M” and “N” are integers of 2 or larger, and for the error correction code data.
The transmitter side comprises the first to N-th encoded data transmitting devices and an FEC data transmitting device, and each transmitting device transmits the encoded data by the different session from each other. The receiver side comprises the first to N-th encoded data receiving devices and an FEC data receiving device, which extracts the encoded data received without transmission error with no fault and, when there is an error or fault generated in the data, restores the data by the error correction code for reconstructing it to be outputted.
Further, in the fourth embodiment of the present invention, the picture data transmitting apparatus comprises a device for setting, by each session, the routing priority control of the transmission lines in at least one session for transmitting the data and the power control of the radio transmission lines. The picture data receiving apparatus may be in a structure having a device for selecting the reception of at least one session based on at least one of: the error/loss rate of the received data; power that can be used in the receiving apparatus; and the setting set in advance.
A contents distributing apparatus 1201, a contents receiving apparatus 1211, and a transmission line 1230 according to the fourth embodiment of the present invention shown in
The transmission managing unit 1203 performs setting of the items set by the transmitting managing unit 103 of the first embodiment and at least one of:
In addition to the same processing as that of the data reading/generating/converting unit 104 of the first embodiment, the data reading/generating/converting unit 1204 generates the error correction (FEC; Forward Error Correction) code data from the first encoded data. As the error correction codes, there are Reed-Solomon codes, LDPC (LowDensity parity check), convolution codes, etc. The present invention uses these existing methods, so that the detailed explanation is omitted herein. For simplifying the description, the error correction code is to be generated from the first encoded data herein. However, it is needless to say that the error correction code maybe generated from other encoded data or the data of a specific layer of the encoded data.
Like the first to N-th transmitting units 105-108 of the first embodiment, the FEC data transmitting unit 1209 transmits the error correction code data generated by the data reading/generating/converting unit 1204. At the time of transmission, the payload type of the RTP header, SSRC, or CSRC, for example, is used or the identifying information corresponding thereto is added so that it is recognized as the error correction code data. Addition of delay and selecting whether or not to transmit the data in accordance with the distribution band is also performed in the same manner as that of the first embodiment.
The FEC data receiving unit 1217 performs the same processing as that of the first to N-th receiving units 114-116 of the first embodiment. It is also the same as the case of the first embodiment in respect that the reception of data is controlled in accordance with the power which can be used in the contents receiving apparatus 1211.
Next, the encoded data reconstruction unit 1218 will be described by referring to
Except for those described above, each part is the same as the corresponding part of the first embodiment.
A plurality of the transmission lines 1230 may be provided so that the distribution session of the first to N+1-th encoded data and the error correction code data, the call connection processing information, and the receiving state report information are transmitted through the different transmission lines.
Furthermore, it is needless to say that each function and processing of the call connection processing unit 1202, the transmission managing unit 1203, the data reading/generating/converting unit 1204, the report receiving unit 1205, the first to N-th transmitting units 1206-1208, and the FEC data transmitting unit 1209 of the contents distributing apparatus 1201 may be achieved by the program control executed by a computer that constitutes the contents distributing apparatus 1201. It is also needless to say that each function and processing of the call connection processing unit 1212, the report transmitting unit 1213, the first to N-th receiving units 1214-1216, the FEC data receiving unit 1217, the encoded data reconstruction unit 1218 and the decoder 1219 of the contents receiving apparatus 1211 may be achieved by the program control executed by a computer that constitutes the contents receiving apparatus 1211.
In a fifth embodiment of the present invention, the transmitter side and the receiver side operate in the following manner for the N-number of encoded data of the same contents encoded by the scalable encoding constituted with M-layers where, “M” and “N” are integers of 2 or larger, and for the error correction code data.
The transmitter side comprises the first to N-th data transmitting devices, an FEC data transmitting device, and a device for performing multiplexing and transmission of the outputs thereof, which transmits the multiplexed data by the same session. The receiver side comprises a device for receiving the multiplexed data, which extracts the encoded data that is received without a transmission error and a fault and, when there is an error or a fault generated in the data, restores it by the error correction code for performing reconstruction and output thereof.
Further, like the second embodiment, the picture data transmitting apparatus according to the fifth embodiment of the present invention may be in a structure with a device for setting the routing priority control of the transmission lines in the session for transmitting the multiplexed data and the power control of the radio transmission lines.
A contents distributing apparatus 1401, a contents receiving apparatus 1411, and a transmission line 1430 according to the fifth embodiment of the present invention shown in
The transmission managing unit 1403 performs the same processing as that of the transmission managing unit 1203 of the fourth embodiment.
The data reading/generating/converting unit 1404 performs the same processing as that of the data reading/generating/converting unit 1204 of the fourth embodiment.
The FEC data transmitting unit 1409 performs the same processing as that of the transmitting unit 900 shown in
The encoded data reconstruction unit 1418 performs the same processing as that of the encoded data reconstruction unit 1218 of the fourth embodiment.
Except for those described above, each part is the same as the corresponding part of the second embodiment.
A plurality of the transmission lines 1430 may be provided so that the distribution session of the data obtained by multiplexing the first to N-th encoded data and the error correction code data, the call connection processing information, and the receiving state report information are transmitted through the different transmission lines.
Furthermore, it is needless to say that each function and processing of the call connection processing unit 1402, the transmission managing unit 1403, the data reading/generating/converting unit 1404, the report receiving unit 1405, the first to N-th transmitting units 1406-1408, the FEC data transmitting unit 1409 and the packet multiplexing/transmitting unit 1410 of the contents distributing apparatus 1401 may be achieved by the program control executed by a computer that constitutes the contents distributing apparatus 1401. It is also needless to say that each function and processing of the call connection processing unit 1412, the report transmitting unit 1413, the encoded data reconstruction unit 1418, the decoder 1419 and the packet receiving unit 1420 of the contents receiving apparatus 1411 may be achieved by the program control executed by a computer that constitutes the contents receiving apparatus 1411.
In a sixth embodiment of the present invention, the transmitter side and the receiver side operate in the following manner for the N-number of encoded data of the same contents encoded by the scalable encoding constituted with M-layers where, “M” and “N” are integers of 2 or larger, and for the error correction code data.
The transmitter side comprises the first to N-th data transmitting devices, an FEC data transmitting device, and a device for multiplexing and transmitting at least two of the outputs thereof, which transmits the multiplexed data and the data without multiplexing by the respective sessions. The receiver side comprises a device for receiving the multiplexed data and the data without multiplexing by the respective sessions, which extracts the encoded data that is received without a transmission error and a fault and, when there is an error or a fault generated in the data, restores it by the error correction code for performing reconstruction and output thereof.
Further, like the third embodiment, the picture data transmitting apparatus according to the fifth embodiment of the present invention also comprises a device for setting, by each session, the routing priority control of the transmission lines in at least one session for transmitting each piece of encoded data and the power control of the radio transmission lines. The picture data receiving apparatus may be in a structure having a device for selecting the reception of at least one session based on at least one of: the error/loss rate of the received data; power that can be used in the receiving apparatus; and the setting set in advance.
A contents distributing apparatus 1501, a contents receiving apparatus 1511, and a transmission line 1530 according to the sixth embodiment of the present invention shown in
The transmission managing unit 1503 performs the same processing as that of the transmission managing unit 1203 of the fourth embodiment.
The data reading/generating/converting unit 1504 performs the same processing as that of the data reading/ generating/converting unit 1204 of the fourth embodiment.
The FEC data transmitting unit 1509 performs the same processing as that of the FEC data transmitting unit 1209 of the fourth embodiment.
The FEC data receiving unit 1517 performs the same processing as that of the FEC data receiving unit 1217 of the fourth embodiment.
The encoded data reconstruction unit 1518 performs the same processing as that of the encoded data reconstruction unit 1218 of the fourth embodiment.
Except for those described above, each part is the same as the corresponding part of the third embodiment.
A plurality of the transmission lines 1530 may be provided so that the distribution session of the multiplexed outputs of the first to N-th transmitting units 1506-1508, the session of distributing the output of the FEC data transmitting unit 1509, the call connection processing information, and the receiving state report information are transmitted through the different transmission lines.
Furthermore, it is needless to say that each function and processing of the call connection processing unit 1502, the transmission managing unit 1503, the data reading/generating/converting unit 1504, the report receiving unit 1505, the first to N-th transmitting units 1506-1508, the FEC data transmitting unit 1509 and the packet multiplexing/transmitting unit 1510 of the contents distributing apparatus 1501 may be achieved by the program control executed by a computer that constitutes the contents distributing apparatus 1501. It is also needless to say that each function and processing of the call connection processing unit 1512, the report transmitting unit 1513, the encoded data reconstruction unit 1518, the decoder 1519 and the packet receiving unit 1520 of the contents receiving apparatus 1511 may be achieved by the program control executed by a computer that constitutes the contents receiving apparatus 1511.
As described above, the present invention can provide contents distributing and receiving apparatuses, a contents transmitting/receiving system, contents distributing and receiving methods, and contents distribution and reception programs, which can suppress as much as possible the significant disturbance of the reproduced contents on the receiver side caused due to the transmission error of the encoded data. Furthermore, it is possible to provide contents distributing and receiving apparatuses, a contents transmitting/receiving system, contents distributing and receiving methods, and contents distribution and reception programs, which allow users to set trade-off between the transmission band used for contents data transmission and the quality. Moreover, it is possible to suppress as much as possible the significant disturbance of the reproduced contents on the receiver side caused due to the transmission error of the encoded data without sending the feedback information from the receiver side to the transmitter side.
Further, the transmitter side can control the quality and/or the stability of the contents on the receiver side through controlling whether or not to perform encryption at the time of actual transmission and/or controlling the information to be informed to the receiver side by the call connection processing. Furthermore, the aforementioned objects can be achieved while preventing an increase in the operation amount required for decoding the compression-encoded data. Moreover, the power consumption of the receiving apparatus can be controlled in accordance with the power that can be used on the receiving apparatus side.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2003-392617 | Nov 2003 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2004/017051 | 11/17/2004 | WO | 00 | 4/28/2006 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2005/050346 | 6/2/2005 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5959672 | Sasaki | Sep 1999 | A |
| 6028634 | Yamaguchi et al. | Feb 2000 | A |
| 6275988 | Nagashima et al. | Aug 2001 | B1 |
| 6289485 | Shiomoto | Sep 2001 | B1 |
| 7313386 | Kondo et al. | Dec 2007 | B2 |
| Number | Date | Country |
|---|---|---|
| 1 311 125 | May 2003 | EP |
| 1 355 473 | Oct 2003 | EP |
| 05-103214 | Apr 1993 | JP |
| 06-339130 | Dec 1994 | JP |
| 2000-295608 | Oct 2000 | JP |
| 2003-134045 | May 2003 | JP |
| 2003-152544 | May 2003 | JP |
| 2003-152626 | May 2003 | JP |
| 2003-152794 | May 2003 | JP |
| 2003-241799 | Aug 2003 | JP |
| 2003-244676 | Aug 2003 | JP |
| 03071801 | Aug 2003 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20070036447 A1 | Feb 2007 | US |
