Patent Application
20250150551
The present invention relates to technology of an information processing system, an information processing method, and a program.
User interface (UI) devices such as display devices, keyboards, mice, and operation controllers are connected to computers, game consoles (hereinafter referred to as “information apparatuses”), and the like, and operated. In an information apparatus, a UI device at the hand of a user is connected to the information apparatus installed at the hand of the user in a wired or wireless manner. As a device that allows a UI device to be installed at a location physically separated from an information apparatus, an extension device (for example, KVM extender (KVM=Keyboard, Video, Mouse)) is used (refer to Non Patent Document 1, for example). For example, as shown in
Non Patent Document 1: Monitoring and control of access to PC! Dual-link DVI-DIP-KVM extender, new release, 2020.7.6, The Daily Industrial News, [Internet retrieval] <https://www.nikkan.co.jp/releases/view/110893> (retrieved on 2021.7.26)
Non Patent Document 2: Nishijima Takamichi, Nakai Yuto, Ohsaki Hiroyuki, et al., “On the Impact of Network Environment on Remote Desktop Protocols,” IEICE Technical Report CQ2012-21 (2012-7), Electronic Information Communication Society, 2012, p23-28
However, in the case of connecting an information apparatus and a UI device for a plurality of users, it is necessary to provide extension devices corresponding to the number of users on a transmitting side and a receiving side, respectively, as shown in
In view of the above circumstances, an object of the present invention is to provide a technique capable of connecting an information apparatus and a UI device for a plurality of users without using an extension device for each UI signal standard.
One aspect of the present invention is an information processing system including: an information apparatus configured to process and transmit a signal; and a reception device configured to process the signal transmitted by the information apparatus and to output the processed signal to a UI device, wherein the information apparatus includes one or a plurality of mapping units configured to generate and map a signal having a frame structure suitable for communication with respect to a signal to be transmitted, a multiplexing unit configured to multiplex the signal mapped by the mapping units through a predetermined method, and a transmission unit configured to transmit the signal multiplexed by the multiplexing unit to the reception device, and the reception device includes a reception unit configured to receive the signal transmitted by the information apparatus, a separation unit configured to separate the multiplexed signal received by the reception unit, and one or a plurality of demapping units configured to generate a data signal from the frame structure.
One aspect of the present invention is an information processing method in an information processing system including an information apparatus configured to process and transmit a signal and a reception device configured to process the signal transmitted by the information apparatus and to output the processed signal to a UI device, the information processing method including: by the information apparatus, generating and mapping a signal having a frame structure suitable for communication with respect to a signal to be transmitted, multiplexing the mapped signal through a predetermined method, and transmitting the multiplexed signal; and by the reception device, receiving the signal transmitted by the information apparatus, separating the received multiplexed signal, and generating a data signal from the frame structure.
One aspect of the present invention is a program in an information processing system including an information apparatus configured to process and transmit a signal and a reception device configured to process the signal transmitted by the information apparatus and to output the processed signal to a UI device, the program causing a computer of the information apparatus to generate and map a signal having a frame structure suitable for communication with respect to a signal to be transmitted, to multiplex the mapped signal through a predetermined method, and to transmit the multiplexed signal and causing a computer of the reception device to receive the signal transmitted by the information apparatus, to separate the received multiplexed signal, and to generate a data signal from the frame structure. Advantageous Effects of Invention
According to the present invention, it is possible to connect an information apparatus and a UI device for a plurality of users without using an extension device for each UI signal standard.
Embodiments of the present invention will be described in detail with reference to the drawings.
In the network NW, at least one of an electrical cross-connect or an optical cross-connect may be provided. The electrical cross-connect is, for example, an optical-channel data unit (ODU) cross-connect. The optical cross-connect is, for example, a reconfigurable optical add/drop multiplexer (ROADM).
The information apparatus 12 includes, for example, a computer, a game device, a portable terminal, a smartphone, a tablet terminal, a television, a set-top box, a wearable device, and the like. As will be described later, the information apparatus 12 includes one or more mapping units, a multiplexing unit, and a transmission unit. In addition, the information apparatus 12 includes a central processing unit (CPU) and a graphic processing unit (GPU), as will be described later. Further, the information apparatus 12 transmits a signal obtained by mapping RGB (Red, Green, Blue) data that does not conform to specific video signal standards to a frame structure suitable for communication to the plurality of video signal reception devices 14 via the network NW. The video signal standards are, for example, high-definition multimedia interface (HDMI) (registered trademark), DisplayPort, and the like. A configuration example and a processing example of the information apparatus 12 will be described later with reference to
As will be described later, the video signal reception device 14 includes a reception unit, a separation unit, and one or more demapping units. The video signal reception device 14 separates the signal received from the information apparatus 12, for example, by an ODU, generates RGB data, a synchronizing signal, and a clock signal as necessary, and displays a signal conforming to the video signal standards on the display device 16. A configuration example and a processing example of the video signal reception device 14 will be described later with reference to
The display device 16 is, for example, a liquid crystal display device, an organic electroluminescence (EL) image display device, or the like.
Configuration and Operation Example of Information Apparatus
The clock generation unit 1211 generates a clock signal and outputs the clock signal to the timing control unit 1212.
The timing control unit 1212 generates a signal for controlling various timings of video data using the clock signal output by the clock generation unit 1211. The timing control unit 1212 outputs a read timing Rt to the video memory. The timing control unit 1212 outputs a horizontal synchronization signal (Hsync) and a vertical synchronization signal (Vsync) to the mapping unit 1216.
The CPU 1213 and the GPU 1214 exchange commands and data.
The GPU 1214 writes video data in the video memory 1215 for drawing on a screen. The video memory 1215 is, for example, a dual port random access memory (RAM). The video memory 1215 outputs RGB data to the mapping unit 1216.
The mapping unit 1216 maps (agnostic maps) RGB data that does not conform to a specific video signal standard to a frame structure suitable for communication as it is. The frame structure is, for example, an optical transport network (OTN). Each mapping unit 1216 outputs the mapped signal to the multiplexing unit 122. The plurality of mapping units 1216 may be present as physical bodies or may be realized virtually.
The multiplexing unit 122 performs ODU multiplexing processing in an OTN, for example, on signals input from the plurality of mapping units 1216. Accordingly, in the present embodiment, multiplexing processing is performed in the information apparatus 12. The multiplexing unit 122 inputs the multiplexed signals to the transmission unit 123 such that the signals are transmitted to the outside of the information apparatus 12.
The transmission unit 123 transmits the signals input from the multiplexing unit 122 to the plurality of video signal reception devices 14 via the network NW. The signals from the transmission unit 123 are, for example, OTU signals in the OTN.
Here, processing performed by the information apparatus 12 is further described.
Data corresponding to each dot of a screen, for example, RGB, for screen display is stored in the video memory 1215. The information processing system 1 updates display contents of the display device 16 by continuously rewriting the data stored in the video memory 1215. In the video memory 1215, the correspondence relationship between addresses on the video memory 1215 and dot positions on the screen has been determined.
The information apparatus 12 stores such RGB data in an optical-channel payload unit (OPU) payload area defined by the OTN, for example. The information apparatus 12 uses an OPU having a payload area greater than an RGB data transmission rate from the video memory 1215. Since the clock signal of the RGB data and the clock signal of the OPU are generally independent, the information apparatus 12 matches the RGB data with the capacity of the OPU payload by performing stuffing processing or idle signal insertion on the RGB data and then stores the RGB data in the OPU payload.
Although an example of using an OPU having a payload area greater than the RGB data transmission rate from the video memory 1215 has been described, the present invention is not limited thereto. The information apparatus 12 may reduce a required payload area by performing compression processing on the RGB data.
The configuration of the information apparatus 12 illustrated in
Further, the output of the information apparatus 12 is cross-connected to the video signal reception device 14 through the network NW.
The reception signal processing unit 143 includes a demapping unit 1431 (1431-1, 1431-2, 1431-3, . . . ), a synchronization signal encoding unit 1432 (1432-1, 1432-2, 1432-3, . . . ), a video data encoding unit 1433 (1433-1, 1433-2, 1433-3, . . . ), an output unit 1434 (1434-1, 1434-2, 1434-3, . . . ) (data encoding unit), an output unit 1435 (1435-1, 1435-2, 1435-3, . . . ), an output unit 1436 (1436-1, 1436-2, 1436-3, . . . ), an output unit 1437 (1437-1, 1437-2, 1437-3, . . . ), and a video signal output unit 1438 (1438-1, 1438-2, 1438-3, . . . ).
The reception unit 141 receives a signal (for example, an OTU signal) transmitted by the information apparatus 12 and outputs the received signal to the separation unit 142.
The separation unit 142 performs, for example, ODU separation processing on the signal (for example, an OTU signal) output from the reception unit 141. The separation unit 142 outputs the separated signal to one or a plurality of demapping units 1431.
The demapping unit 1431 generates RGB data and a synchronization signal from the frame structure of the signal output from the separation unit 142. The demapping unit 1431 generates a clock signal when the video signal standard is, for example, HDMI (transition minimized differential signaling (TMDS)). The demapping unit 1431 outputs the RGB data to the video data encoding unit 1433 and outputs the synchronization signal to the synchronization signal encoding unit 1432. When the video signal standard is, for example, HDMI, the demapping unit 1431 outputs the clock signal to the output unit 1437.
The synchronization signal encoding unit 1432 encodes the synchronization signal from the demapping unit 1431 and outputs the encoded synchronization signal to the video data encoding unit 1433.
The video data encoding unit 1433 generates a signal conforming to the video signal standard using the RGB data from the demapping unit 1431 and the synchronizing signal from the synchronization signal encoding unit and inputs the generated signal to the output units 1434 to 1436. In the present embodiment, the video data encoding unit 1433 encodes the video signal into a desired video signal standard for each user. As described above, the video signal standard is, for example, HDMI (TMDS), fixed rate link (FRL), DisplayPort (bit distribution), or the like.
The output units 1434 to 1437 output the signal output from the demapping unit 1431 or the signal output from the video data encoding unit 1433 to the video signal output unit 1438. The output units 1434 to 1437 performs serialization processing on the signals, controls the signals to a specified voltage, and input them to the video signal output unit. The specified voltage is, for example, low voltage differential signaling (LVDS). The video signal output unit 1438 outputs a video signal conforming to a video signal standard for each user to the display devices 16-1 to 16-3.
The configuration of the video signal reception device 14 illustrated in
For example, the video signal reception device 14 may include, for example, a compression unit, and the compression unit is, for example, VESA display stream compression (DSC), VESA display compression for mobile (VDC-M), MPEG, or the like.
Further, the video signal reception device 14 may not include the separation unit 142 if it is cross-connected in the network NW and signals become one system.
The information apparatus 12 and the video signal reception device 14 may be configured using a processor such as a central processing unit (CPU) and a memory. The information apparatus 12 may serve as the transmission signal processing unit 121, the multiplexing unit 122, and the transmission unit 123 by the processor executing a program. In addition, the video signal reception device 14 may serve as the reception unit 141, the separation unit 142, and the reception signal processing unit 143 by the processor executing the program. All of some functions of the information apparatus 12 and the video signal reception device 14 may be realized using hardware such as an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA). The aforementioned program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a storage device such as a portable medium such as a flexible disc, a magneto-optical disc, a ROM, a CD-ROM, or a semiconductor storage device (e.g., solid state drive (SSD)), or a hard disk or a semiconductor storage device built in a computer system. The aforementioned program may be transmitted via a telecommunication line.
Next, a processing procedure of the information processing system will be described. First, an example of a processing procedure performed by the information apparatus 12 will be described.
(Step S1) The clock generation unit 1211 generates a clock signal.
(Step S2) The timing control unit 1212 generates a read timing and synchronization signals (horizontal synchronization signal and vertical synchronization signal) using the clock signal output from the clock generation unit 1211.
(Step S3) The GPU 1214 writes video data in the video memory 1215 for drawing on a screen.
(Step S4) The mapping unit 1216 maps, for example, RGB data to a frame structure suitable for communication. Each transmission signal processing unit 121 performs processing of steps S1 to S4.
(Step S5) The multiplexing unit 122 multiplexes signals input from the plurality of mapping units 1216 by a predetermined method (for example, ODU multiplexing in OTN).
(Step S6) The transmission unit 123 transmits the signal (for example, an OTU signal) input from the multiplexing unit 122 to the plurality of video signal reception devices 14 via the network NW.
Next, an example of a processing procedure performed by the video signal reception device 14 will be described.
(Step S11) The reception unit 141 receives a signal (for example, an OTU signal) transmitted by the information apparatus 12.
(Step S12) The separation unit 142 performs, for example, ODU separation processing on the signal (for example, OTU signal) output from the reception unit 141.
(Step S13) The demapping unit 1431 generates RGB data and a synchronization signal from the frame structure of the signal output from the separation unit 142 (performs demapping).
(Step S14) The video data encoding unit 1433 generates a signal conforming to a video signal standard using the RGB data from the demapping unit 1431 and the synchronization signal from the synchronization signal encoding unit.
(Step S15) The output units 1434 to 1437 output the signal output from the demapping unit 1431 or the signal output from the video data encoding unit 1433 to the video signal output unit 1438.
(Step S16) The video signal output unit 1438 outputs a video signal conforming to a video signal standard for each user to the display device 16 for each user.
Each reception signal processing unit 143 performs processing of steps S13 to S16.
Here, the present embodiment is compared with the prior art.
The clock generation unit 9041 generates a clock signal. The timing control unit 9042 generates a signal for controlling various timings of video data using the clock signal.
The timing control unit 9042 outputs a read timing to the video memory 9045, outputs synchronization signals (horizontal synchronization signal and vertical synchronization signal) to the synchronization signal encoding unit 9046, and outputs clock information to the output unit 9051 as necessary.
The CPU 9043 and the GPU 9044 are connected and exchange commands and data. The GPU 9044 writes video data in the video memory 9045 for drawing on a screen.
The video memory 9045 outputs RGB data to the video data encoding unit 9047 in accordance with the read timing. The synchronization signal encoding unit 9046 encodes the synchronization signals and outputs the synchronization signals to the video data encoding unit 9047.
The video data encoding unit 9047 generates a signal conforming to a video signal standard on the basis of the RGB data from the video memory 9045 and the synchronization signals from the synchronization signal encoding unit 9046 and outputs the generated video signal to output units 9048 to 9050.
The output units 9048 to 9051 output the signal output from the timing control unit 9042 or the signal output from the video data encoding unit 9047 to the video signal output unit 9052. The output units 9048 to 9051 perform serialization processing on the signals, control the signals to a specified voltage, and input them to the video signal output unit. The specified voltage is, for example, LVDS.
The video signal output unit 9052 outputs a signal to the outside of the information apparatus 904 via a specified connector. The specified connector is, for example, a connector of HDMI or DisplayPort. The output signal is transmitted to the extension device 902 via the extension device 903 and the network NW as shown in
In this manner, the video data encoding unit 9047 generates a signal conforming to a specific video signal standard. Further, in the prior art, it is necessary to provide extension devices corresponding to the number of users on the transmitting side and the receiving side, as shown in
On the other hand, in the present embodiment, RGB data that does not conform to a specific video signal standard is mapped as it is to a frame structure suitable for communication, and UI signals (for example, video signals) of a plurality of users are collectively transmitted from the information apparatus 12. Therefore, according to the present embodiment, UI signals of a plurality of users can be collectively transmitted in a form independent of a signal standard in the information apparatus 12, and thus installation of extension devices for signal standards corresponding to the number of users on the side of the information apparatus 12 becomes unnecessary, reducing the cost of equipment.
Although HDMI and DisplayPort have been described as an example of video signals in the above-described example, video signals of other video signal standards may be used. Further, although an example in which the video memory 1215 outputs RGB signals to the mapping unit 1216 in the transmission signal processing unit 121 has been described in the above-described example, the present invention is not limited thereto. A signal output from the video memory 1215 may be YCbCr or the like. In this case, a signal output from the demapping unit 1431 to the video data encoding unit 1433 may be YCbCr or the like in the reception signal processing unit 143.
Further, although an example in which the information apparatus 12 transmits an OTU signal has been described in the above-described example, signals of other standards may be transmitted. In this case, the information apparatus 12 performs mapping according to a transmission signal, and the video signal reception device 14 performs demapping according to the transmitted signal.
Further, although a video signal is used as an example of a UI signal in the above-described example, the present invention is not limited thereto. The information apparatus 12 can collectively transmit UI signals of a plurality of users by the same method as the above-described method for operation signals or the like. Furthermore, the information processing system 1 may transmit and receive at least one of a video signal or a UI signal such as an operation signal.
Although the embodiments of the present invention have been described in detail with reference to the drawings, a specific configuration is not limited to this embodiment, and design within the scope of the gist of the present invention, and the like are included.
The present invention is applicable to, for example, a gaming system, an eSports system, a remote desktop system, an information apparatus rental system, and the like.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2021/029297 | 8/6/2021 | WO |
