Patent Application
20230283816
The present invention relates to a system that delivers a screen to a remote location through a network.
Conventionally, image data has been transferred to a display device such as a display and a monitor by using a connection scheme on the premise of connection in a short distance of about several meters to several tens of meters. For example, HDMI (High-Definition Multimedia Interface) (R) [Non-Patent Literature 1] is a scheme that is widely used for connecting a computer and a monitor or connecting a television and relevant AV (Audio/Visual) equipment, and cannot directly connect equipment at a remote location several kilometers away, to transfer image data.
A device for converting a signal for the HDMI into a signal appropriate for communication is used for transferring image data to a remote location. A scheme of performing the IP (Internet Protocol) packetization of the HDMI signal in a predetermined form, and the like are known [Non-Patent Literatures 2 and 3].
For transferring the image data to the remote location, there is a case where one image source is concurrently delivered not only to one display device but also to a plurality of display devices. Particularly, for achieving a low-delay delivery, it is necessary to transfer uncompressed image data.
In conventional schemes of transferring the uncompressed image data through a network, as exemplified by HDMI, it is not possible to achieve an efficient delivery of one image source to a plurality of display devices. For example, by multicast, an identical image data is merely delivered to all display devices. Such a system has two problems.
One problem is that there may in some cases be no network bandwidth for transmitting sufficient uncompressed image data to all receiving devices. When the delivery is uniformly executed at a high image quality and a high frame rate, uncompressed image data leads to a large amount of data. Therefore, image contents cannot be displayed in a network with an insufficient bandwidth. In addition, the network bandwidth is suppressed, and other communications are also disturbed. Further, when the delivery is performed so as to be suitable for a low network bandwidth, there is a problem in that it is not possible to perform the delivery while keeping the quality of the image source with a high image quality and a high frame rate.
The other problem is that there may in some cases be no display device that can display an image corresponding to the pixel and frame rate of the image source. In the case where the image source is dynamically changed each time instead of a static system or in the case where the delivery is performed to many display devices, it is difficult to cause all display devices to support all image source formats.
Accordingly, an object of the present disclosure is to make it possible to transfer a large volume of image data to display devices that are at remote locations and that have different display performances, through a plurality of networks having different available bandwidths.
The sending side sends image data as packets having different identifiers for a frame or scanning line, and the receiving side selectively receives packets having a predetermined identifier depending on the network bandwidth or displayable image quality.
Specifically, in an image delivery system in the present disclosure,
Specifically, in an image delivery method in the present disclosure,
Specifically, a sending device in the present disclosure includes:
A sending program in the present disclosure is a program that realizes a computer as each functional unit included in the sending device according to the present disclosure, and is a program that causes the computer to execute each step included in a sending method to be executed by the sending device according the present disclosure.
According to the present disclosure, it is possible to transfer a large volume of image data to display devices that are at remote locations and that have different display performances, through a plurality of networks having different available bandwidths.
Embodiments of the present disclosure will be described below in detail with reference to the drawings. The present disclosure is not limited to the embodiments shown below. The embodiments are examples only, and the present disclosure can be carried out in any form in which various modifications and improvements are made based on the knowledge of those in the art. In the specification and drawings, identical reference characters denote identical constituent elements.
(Specific Description of Developmental Technology)
A schematic configuration of an image delivery system in the present disclosure will be described with reference to
The identifier to be used for the delivery is an arbitrary identifier written in the header of the packet that can be read by general network equipment such as a router and a switch, and for example, is exemplified as follows.
In the case where an address is used as the identifier, the address may be any or both of a source address and a destination address. Further, as the destination address, a multicast address can be used. Further, other than the address, a different identifier included in the header of the packet or the like may be used. For example, in the case of a TCP/IP packet, a port number can be used. In this way, an arbitrary identifier corresponding to a protocol used for the packet can be employed.
Furthermore, a case where an IP address is used as the identifier will be described. For example, in the data of one screen, 239.0.0.1 that is a multicast address is used as the destination address for packets containing data of pixels on an even-number-th running line, 239.0.0.2 that is a multicast address is used as the destination address for packets containing data of pixels on an odd-number-th running line, and the receiving side can select the receiving of only 239.0.0.1, the receiving of only 239.0.0.2, and the receiving of both 239.0.0.1 and 239.0.0.2. Further, in the case of the multicast address, by separately transferring packets from the receiving side only to a route on which the network side is a receiving terminal by MLD (Multicast Listener Discovery) or IGMP (Internet Group Management Protocol), it is possible to restrain a useless packet transmission to a route with no receiving terminal.
That is, by the method, without using a device that deeply understands the interior of data, as exemplified by a DPI (Deep Packet Inspection) device, it is possible to achieve a selective delivery of contents with general network equipment that can read the header information in the packet, as exemplified by a router and a switch. Hereinafter, in the present disclosure, the case where the identifier is an address will be described.
Suppose that the delivery is performed from a sending device 101 to a receiving device 102 through a network 121, a network border device 112 and a network 122 and concurrently the delivery is performed to a receiving device 103 through the network 121, a network border device 113 and a network 123. The sending device 101 delivers one image source using different addresses A and B depending on the pixel and frame.
In the case where the network 122 and the receiving device 102 can perform receiving at a high image quality and a high frame rate, the network 122 and the receiving device 102 can reproduce contents using the one original image source, by receiving both the address A and the address B. On the other hand, in the case where the network 123 and the receiving device 103 can perform transfer and receiving only at a low image quality and a low frame rate, the receiving device 103 receives the data with only one address.
When the address is a multicast address, only the data with the address selected by the receiving device 103 is transferred from the network border device 113 to the network 123, and therefore, load is avoided from being needlessly put on the network 123 and the receiving device 103.
In the case where the display device 132 has a display performance of a high image quality and a high frame rate, the receiving device 102 can reproduce contents using the one original image source, by receiving both the address A and the address B. On the other hand, in the case where the display device 133 has a display performance of a low image quality and a low frame rate, the receiving device 103 receives the data with only one address of the addresses A and B.
A case where an address control is performed using the frame will be described with reference to
A receiving side that can perform the receiving at a high frame rate receives the address A and the address B. A receiving side that can perform only the receiving at a low frame rate selectively receives only the address A or the address B. For example, in the case where the image data to be sent has 60 fps, when the addresses A and B are received, data having 60 fps can be received and when only the address A or B is received, the data amount is reduced by half, so that data having 30 fps can be received.
The case where the data amount and the frame rate are reduced to ½ has been described above, but the present disclosure is not limited to this.
For example, in the case of {A, B, A, C}, data with the original frame rate can be received when all of the addresses A, B and C are received, data with ½ of the data amount and frame rate can be received when only the address A or the addresses B and C are received, and data with ¼ of the data amount and frame rate can be received when only the address B or C is received.
For example, in the case of {A, B, C}, data with the original frame rate can be received when all of the addresses A, B and C are received, data with ⅔ of the data amount and frame rate can be received when the addresses A and B, the addresses B and C or the addresses A and C are received, and data with ⅓ of the data amount and frame rate can be received when only the address A, B or C is received.
As for the image data that is used in the present disclosure, image data in which one image at a certain time includes all information is assumed, and an uncompressed image or an image having a relatively low compression ratio, as exemplified by motion JPEG (Joint Photographic Experts Group) in which a technique of compression between frames is not used, is intended. Further, as for the selective use of identifiers in one screen, a progressive uncompressed image is assumed.
A case where an address control is performed using a scanning line will be descried with reference to
A receiving side that can perform the receiving at a high image quality receives the address A and the address B. A receiving side that can perform only the receiving at a low image quality selectively receives only the address A or the address B. For example, in the case where the line number of scanning lines in the sending image is 2160, when the addresses A and B are received, 2160 scanning lines can be received, and when only the address A or B is received, the data amount is reduced by half, so that 1080 scanning lines can be received.
The case where the data amount and the line number are reduced to ½ has been described above, but the present disclosure is not limited to this.
For example, in the case of {A, B, A, C}, data with the original line number can be received when all of the addresses A, B and C are received, data with ½ of the data amount and line number can be received when only the address A or the addresses B and C are received, and data with ¼ of the data amount and line number can be received when only the address B or C is received.
For example, in the case of {A, B, C}, data with the original line number can be received when all of the addresses A, B and C are received, data with ⅔ of the data amount and line number can be received when the addresses A and B, the addresses B and C or the addresses A and C are received, and data with ⅓ of the data amount and line number can be received when only the address A, B or C is received.
A case where the address is controlled using data other than picture (pixel) data will be described with reference to
Pattern 1
Information such as audio can be received as the same data, regardless of the image quality and frame rate in the receiving. Therefore, for the data in the region 401, a different address from the address used for the region 402 can be used. It is possible to receive the audio and others without interruption, by commonly receiving the address only for the region 401 regardless of the image quality and frame rate in the receiving.
Pattern 1
Information such as audio can be received as the same data, regardless of the image quality and frame rate in the receiving. Therefore, the region 401 can be controlled with the same address as the region 402, and further can be moved to a data region in which the audio information and others are commonly received. For example, in the case where the image is divided by {A, B} and where there are a receiving device that receives (A, B) and a receiving device that receives only (A), the audio data and others originally included in the address B can be moved to a data region for the address A. Thereby, it is possible to receive the audio and others without interruption.
An exemplary system configuration of the present disclosure will be described with reference to
(Sending Device)
Reference numeral 501 denotes an input buffer, and the image data is read by a signal analysis unit 502 and a data adjustment unit 503 at necessary timings.
Reference numeral 502 denotes a signal analysis unit, and the signal analysis unit 502 reads the image data from the input buffer 501, performs the identification of the frame, the identification of the scanning line, the identification of the audio data and the identification of other control signals, and gives notice to the data adjustment unit 503 and a packetization unit 504. Data itself may be given to the data adjustment unit 503.
Reference numeral 503 denotes a data adjustment unit, and the data adjustment unit 503 reads the image data from the input buffer 501 or the signal analysis unit 502, executes the swapping and rewriting of some data such as the audio data based on the information from the signal analysis unit 502, and gives the data to the packetization unit 504. On this occasion, marker data showing the rewriting may be put in.
Reference numeral 504 denotes a packetization unit, and the packetization unit 504 reads the image data from the data adjustment unit 503, and perform packetization using the above-described address, based on the information from the signal analysis unit 502.
Reference numeral 505 denotes a data transmission unit, and the data transmission unit 505 transmits the image data at a speed appropriate for the network. An FEC function may be included.
The appropriate speed is such a speed that the transmission speed of the packet is restricted to a certain speed or lower and the transmission interval of the packet is kept at a certain interval in the rate such that the packet loss in the network or the delay by the buffer in the network device is not excessively large. For example, even when the network bandwidth is 10 Gbps, if the signal rate for the image is 1 Gbps, the transmission rate of the packet is restricted to about 1 Gbps and the packet transmission interval is kept at a constant interval of about 1 Gbps, so that the packet loss in the network and an excessive delay can be restrained.
(Receiving Device)
Reference numeral 601 denotes a data reception unit, and the data reception unit 601 receives the packet from the network. At this time, image data sent using a predetermined address is selectively received depending on the network bandwidth of the data reception unit or the image quality that can be displayed by the display device. Packet loss data may be restored by the FEC function.
Reference numeral 602 denotes a depacketization unit, and the depacketization unit 602 takes data out of the packet.
Reference numeral 603 denotes a signal analysis unit, and the signal analysis unit 603 reads data from the depacketization unit 602, and determines whether the relocation of the audio data and others has been performed. The determination may be performed based on the marker data.
Reference numeral 604 denotes a data adjustment unit, and the data adjustment unit 604 executes the swapping and rewriting of some data such as the audio data based on the information from the signal analysis unit 603, builds image data that can be reproduced by the display device, and gives the data to a data output unit 605. Further, in the case where it is necessary to decrease the resolution in the lateral direction, the decimation or integration process of the data is executed. In the case where the address is changed for each scanning line, it is possible to decimate the resolution in the portrait direction, but it is difficult to decimate the resolution in the lateral direction, and therefore the decimation in the lateral direction is executed on the receiving side.
Reference numeral 605 denotes a data output unit, and the data output unit 605 performs the buffering of the data and the conversion of an IF. In the case where a response to a signal from the display device is necessary, for example, in the case of HDMI, the response is executed by the data output unit.
In the present disclosure, the receiving device and the display device may be included in one device. For example, in
(Effect Produced by Invention)
It is possible to provide means for transferring one image having a large volume to a plurality of display devices that are at remote locations and that have different display performances, through a plurality of networks having different available bandwidths.
(Point of Invention)
Conventionally, for uncompressed image, an interface for connection in a short distance of about several meters to several tens of meters has been used. A system that performs a low-delay transfer by transferring this through a network has been proposed, but for the concurrent delivery to many receiving devices, only the multicast is used, and the concurrent delivery cannot be performed to display devices that have small network bandwidths or have different display performances.
In the present invention, the sending side transmits one image with different identifiers being used for delivery depending on the kind and order of data such as the pixel and the frame, and the receiving side selectively receives the packet depending on the identifier and thereby can obtain the image with the necessary numbers of pixels and frames, so that it is possible to selectively receive and display the image depending on the network bandwidth or the displayable image quality on the receiving side, by the transmission of only one image on the sending side. Particularly, in the case where there are a plurality of or a large number of networks or receiving devices on the receiving side, a plurality of image sources are not necessary in the present invention, and therefore the system is easily extended. Furthermore, an identifier in an existing network packet header is used for traffic selection on the network, and thereby it is possible to realize the system with existing network equipment.
The present disclosure can be applied to the information communication industry.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2020/025310 | 6/26/2020 | WO |
