Patent Application
20240146633
The present disclosure relates to a technique for measuring a delay time in a device that receives or outputs a video or an audio.
With the spread of remote work, the importance of communication using a video and an audio is increasing. In order to smoothly advance communication using a video and an audio, it is important to shorten a video delay and an audio delay. For a video delay and an audio delay, not only a communication delay occurring in a network but also a device delay occurring in a device such as a camera, a microphone, a speaker, or a display need to be considered.
As a method of measuring the device delay, there is a measurement method utilizing an LED and an optical sensor (refer to, for example, Non Patent Literature 1). In this method, a delay time occurring in the camera can be measured by measuring a time difference between a time when the LED is turned on and a time when light is observed by the optical sensor by using an oscilloscope.
In order to use the technique described in Non Patent Literature 1, dedicated devices such as an LED, an optical sensor, and an oscilloscope for measurement are required. For this reason, it is not realistic to widely implement the technique.
Non Patent Literature 1: “2. Measuring Delay Time”, Technologies, Optical Path Communications that connect high definition images by optical technology, https://h-path.co.jp/technologies/2-how-to-measure-latency/
An object of the present disclosure is to measure a delay time occurring in a device that receives or outputs a video or an audio without requiring a special apparatus or precise inter-apparatus time synchronization.
According to the present disclosure, there is provided a system including:
According to the present disclosure, there is provided a method for a system including a remote apparatus connected to a device that executes processing of outputting or receiving data of a video or an audio via a communication network and a user terminal that transmits and receives a signal to and from the remote apparatus via the communication network,
According to the present disclosure, there is provided an apparatus included in a system including a remote apparatus connected to a device that executes processing of outputting or receiving data of a video or an audio via a communication network and a user terminal that transmits and receives a signal to and from the remote apparatus via the communication network, the apparatus functioning as the user terminal,
The apparatus according to the present disclosure can also be implemented by a computer and a program, and the program can be provided by being recorded in a recording medium or via a network.
According to the present disclosure, it is possible to measure a delay time occurring in a video device without requiring a special apparatus or precise inter-apparatus time synchronization.
Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments described below. These examples are merely examples, and the present disclosure can be carried out in a form with various modifications and improvements based on the knowledge of those skilled in the art. Note that components having the same reference numerals in the present specification and the drawings denote the same components.
A delay time in a communication network can be measured. Therefore, the present disclosure includes a remote apparatus connected to a device that executes processing of receiving or outputting data of a video or an audio via a communication network. In addition, by causing the device to respond via the communication network and subtracting a delay time in the communication network from a response time of the device via the communication network, the response time of the device alone can be measured in a user terminal.
In the present disclosure, the device is any device that executes processing of receiving or outputting media data of a video or an audio, and is, for example, a camera, a monitor, a microphone, or a speaker. The camera converts a video into video data and outputs the video data. The monitor receives video data, and outputs a video. The microphone converts an audio into audio data, and outputs the audio data. The speaker receives audio data, and outputs an audio. In the present disclosure, a method for measuring a response time of a device alone in a case where media data is video data will be described.
In the present disclosure, processing of outputting or receiving a video having a predetermined pattern is performed in the device and the remote apparatus,
For example, in a case where the device is a camera, the camera and a response apparatus are connected to each other via a communication network. In this state, a screen of the user terminal is changed in a predetermined pattern, a change in the screen is imaged by the camera, and a response from the response apparatus is received by the user terminal. The user terminal detects a timing when the screen is changed in the pattern. In addition, the user terminal measures a time from when the received screen is changed to when the response signal is received, and subtracts the delay time in the communication network from the measured time. Thereby, the user terminal can measure a delay time in the camera.
For example, in a case where the device is a display, the display and a control apparatus are connected to each other via a communication network. In this state, a request signal is transmitted from the user terminal to the control apparatus, and a video on the display is changed in a predetermined pattern. The user terminal detects a change in the pattern of the video on the display. In addition, the user terminal measures a time from when the request signal is transmitted to when the video on the display is changed, and subtracts a delay time in the communication network from the measured time. Thereby, the user terminal can measure a delay time in the display.
The video data is any data that can be processed by the device, and is, for example, a high-definition multimedia interface (HDMI, HDMI is a registered trademark) signal. Hereinafter, an example in which the video data is an HDMI signal will be described in detail.
The camera 21 is connected to the video analysis unit 12 and the response unit 13 via a communication network 24. In the present embodiment, media converters (MC) 22 and 26 for transmitting an HDMI signal from the camera 21 via the communication network 24 are provided.
A video delay measurement system according to the present embodiment executes the following steps.
S101: A delay time TN of the communication network 24 is measured in advance using Ping or the like.
S102: The user terminal 11 displays a predetermined video pattern on a screen. The video pattern is a certain video of which a change on a screen can be detected by the video analysis unit 12. The camera 21 captures an image of a screen in a changing state. The HDMI signal from the camera 21 is transmitted by using the MCs 22 and 26 via the communication network 24.
S103: The video analysis unit 12 acquires the HDMI signal from the MC 26, and issues a signal to the response unit 13 at a timing when white is detected. Thereby, the communication network 24 detects a change on the screen.
S104: In a case where the signal from the video analysis unit 12 is received, the response unit 13 returns a response signal. A transmission destination of the response signal can be set to a certain destination, and is, for example, the user terminal 11.
S105: The user terminal 11 measures a round-trip time from when the screen is changed to when a response signal is received, and measures a delay in the camera 21 by using the round-trip time. For example, the delay in the camera 21 is measured by subtracting the delay time TN in the communication network 24 from the round-trip time.
In step S102, the user terminal 11 displays a screen under a control of the measurement application. For example, the video pattern is a video pattern in which screen display is switched in order of black, black, white, white, and white in a certain period of time. The response unit 13 detects the pattern in which screen display is switched in order of black, black, white, white, and white, and generates a response signal at a time when screen display is switched from black to white.
Further, in step S104, the response unit 13 may transmit the time when the signal from the video analysis unit 12 is received to the user terminal 11 using the response signal. In this case, in step S105, the user terminal 11 measures the delay in the camera 21 by using the time from when the video pattern is displayed on the screen to the time when the response signal is notified instead of the round-trip time from when the screen is changed to when the response signal is received.
In the present embodiment, an example in which the measurement target of the delay time is the camera 21 is described. On the other hand, the present disclosure is not limited thereto. For example, a microphone may be used instead of the camera 21. In this case, an audio analysis unit is provided instead of the video analysis unit 12. In addition, the user terminal 11 may change an output audio in step S102, the microphone may output audio data in step S103, and the audio analysis unit may detect a change in the audio in step S104.
The HDMI signal from the camera 21 is input to the video encoder 22E using an HDMI cable. The video encoder 22E converts the HDMI signal into a format that can be transmitted via the communication network 24. For example, the HDMI signal is converted into an IP packet. Thereby, the HDMI signal from the video encoder 22E is input to the router 23 via a LAN, is transmitted from the router 23 to the router 25 via the communication network 24, and is input from the router 25 to the video decoder 26D via a LAN.
The video decoder 26D converts the IP packet into an HDMI signal, and outputs the HDMI signal to the video analysis unit 12. The video analysis unit 12 detects that screen display is switched from black to white by using the HDMI signal. In a case where the video analysis unit 12 detects that screen display is switched from black to white, the response unit 13 transmits a response signal to the router 25 at that timing. The router 25 transmits the response signal to the router 23, and the router 23 transmits the response signal to the user terminal 11. Here, the response signal does not include a video signal, and thus any data transmission method can be used. For example, a LAN can be used between the response unit 13 and the router 25, and a wireless LAN such as WiFi can be used between the router 23 and the user terminal 11.
The screen control function 111 performs a certain display that can be detected by the video analysis unit 12, and notifies the user of a timing when the display is performed as a control timing. As the display, for example, the screen display is switched in order of black, black, white, white, and white in a certain period of time.
The response reception function 112 receives a response signal from the response unit 13, and notifies the delay calculation function 114 of the timing when the response signal is received.
The other delay acquisition function 113 acquires delay time information from the delay database, and notifies the delay calculation function 114 of the delay time information.
The delay calculation function 114 subtracts the delay time acquired by the other delay acquisition function 113 from the timing when the response signal is received, and obtains a delay time in the camera 21.
The display 41 is connected to the video output unit 32 and the control unit 33 via a communication network 44. In the present embodiment, media converters (MC) 46 and 42 for transmitting an HDMI signal from the video output unit 32 via the communication network 44 are provided.
A video delay measurement system according to the present embodiment executes the following steps.
S201: A delay time TN of the network 44 is measured in advance using Ping or the like.
S202: The user terminal 11 transmits a request signal for changing the output video.
S203: In a case where the request signal is received, the video output unit 32 outputs an HDMI signal for changing the output video. The HDMI signal from the video output unit 32 is transmitted by using the MCs 46 and 42 via the communication network 44.
S204: The display 41 acquires and displays the HDMI signal from the MC 42.
S205: The user terminal 31 detects a change in the video on the display 41, the change being occurred over the network. In addition, the user terminal 31 measures a time from the transmission of the request signal to the detection of the change in the video on the display 41, and measures a delay time of the display 41 using the time.
In step S203, the video output unit 32 outputs a video of which the screen display is switched to white under a control of the control unit 33. For example, the video output unit 32 outputs an HDMI signal of which the screen display is switched in order of black, black, white, white, and white in a certain period of time. Thereby, in step S203, a video of which the screen display is switched in order of black, black, white, white, and white in a certain period of time is displayed on the display 41.
In the present embodiment, an example in which the measurement target of the delay time is the display 41 is described. On the other hand, the present disclosure is not limited thereto. For example, a speaker may be used instead of the display 41. In this case, an audio output unit is provided instead of the video output unit 32. In addition, the audio output unit may change an output audio in step S203, the speaker may output audio data in step S204, and the user terminal 31 may detect a change in the audio in step S205.
The video encoder 46E converts the HDMI signal into a format that can be transmitted via the communication network 44. Thereby, the HDMI signal from the video encoder 46E is input to the router 45 via a LAN, is transmitted from the router 45 to the router 43 via the communication network 44, and is input from the router 43 to the video decoder 42D via a LAN.
The video decoder 42D converts the format that can be transmitted via the communication network 44 into an HDMI signal, and outputs the HDMI signal to the display 41. The display 41 displays a video of which the screen display is switched in order of black, black, white, white, and white in a certain period of time. The user terminal 31 detects a change in the video on the display 41, the change being occurred via the communication network 44.
The illuminance change detection function 211 acquires illuminance information from the illuminance sensor, detects that the illuminance is switched from black to white, and notifies the delay calculation function 214 of a timing when the illuminance change is detected.
The request transmission function 212 transmits a request signal to the control unit 33, and notifies the delay calculation function 214 of a transmission timing of the request signal.
The other delay acquisition function 213 acquires delay time information from the delay database, and notifies the delay calculation function 214 of the delay time information.
The delay calculation function 214 obtains a delay in the display 41 from the timing when the illuminance change is detected, using the timing when the request signal is transmitted and the delay time information acquired by the other delay acquisition function 213.
According to the present disclosure, the user terminal 11 can measure a delay occurring in a device such as the camera 21 or the display 41 without requiring a special apparatus or precise inter-apparatus time synchronization.
In the above-described embodiments, an example in which the measurement target of the delay time is the camera 21 or the display 41 is described. On the other hand, the present disclosure is not limited thereto. For example, a microphone may be used instead of the camera 21, a speaker may be used instead of the display 41, or a combination thereof may be used.
In addition, an example in which the remote apparatus is the response apparatus or the control apparatus is described. On the other hand, the remote apparatus may have both a response function corresponding to the response apparatus and a control function corresponding to the control apparatus.
Furthermore, in addition to the applications included in the user terminals 11 and 31, certain functions of the camera 21, the video analysis unit 12, the response unit 13, the MCs 22, 26, 42, and 46, the video encoders 22E and 46E, the video decoders 26D and 42D, the display 41, the video output unit 32, and the control unit 33 can also be realized by a computer and a program, and the program can be provided by being recorded in a recording medium or via a certain communication network.
The present disclosure can be applied to information and communication industries.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2021/024154 | 6/25/2021 | WO |
