The present disclosure relates to a video distribution system and a video distribution method for generating a distribution video from a captured video obtained by capturing an event venue and delivering the distribution video to a user terminal device via a network, and a user terminal device that receives a distribution video to be distributed from a video distribution device via a network and displays the distribution video.
In recent years, an image distribution service has been widely used to distribute a video obtained by capturing a venue, where various events of a concert and the like are held, by a camera, from a video distribution server to a user terminal. In such a video distribution service, there is a possibility that a favorite person is reflected on an edge of a video or hidden behind another person, so that it is difficult to view the favorite person. Therefore, there is a demand for a user to view the video that matches his/her demand.
In the related art, there is known a technology in which in response to the demand of the user in such a video distribution service, a plurality of cameras are installed so that a person can be captured at the same time from different directions, and the user can select a camera, so that videos with different capturing angles can be viewed (see PTL 1). In addition, there is known a technology in which an authority to operate a camera is given to a specific user and a capturing state such as a camera posture or a viewing angle is remotely controlled by an operation of the user (see PTL 2).
PTL 1: Japanese Patent Unexamined Publication No. 2011-103522
PTL 2: Japanese Patent Unexamined Publication No. 2003-339038
In the technology disclosed in PTL 1, the user can view a video captured from a desired direction. However, for example, there is a problem that even if there is a demand for viewing an attention portion in the video in a close view, such as a face of a favorite person, a video matching such a demand cannot be displayed. In the technology disclosed in PTL 2, there is a problem that a specific user having authority to operate the camera can view a video matching his/her demand, but all users cannot view an image matching his/her demand.
A main object of the present disclosure is to provide a video distribution system, a user terminal device, and a video distribution method that efficiently generate various videos matching diverse demands of a user and enable the user to view a video matching the demand of the user his/herself.
A video distribution system of the present disclosure that generates a distribution video from a captured video obtained by capturing an event venue and distributes the distribution video to a user terminal device via a network, the system including: a camera that is installed at a predetermined position of the event venue and captures the event venue in a predetermined direction; a captured video storage that accumulates the captured video output from the camera; a video processing computer that cuts out regions respectively corresponding to a plurality of viewpoints with respect to a subject from the captured video, and generates a plurality of distribution videos different in viewpoint; and a video distribution server that distributes the distribution video corresponding to a distribution request to the user terminal device when receiving the distribution request including information on the viewpoint selected by a user from the user terminal device.
A user terminal device of the present disclosure that receives a distribution video distributed from a video distribution device via a network and displays the distribution video, the device including: an operation input unit that performs an operation of selecting an event desired to be viewed and a viewpoint with respect to a subject by a user; a transmitter that transmits a distribution request including information on the event and the viewpoint which are selected by the user to the video distribution device; a receiver that receives the distribution video corresponding to the distribution request distributed from the video distribution device; and a processor that controls reproduction of the received distribution video and displays the distribution video on a video display unit.
A video distribution method of causing an information processing device to perform a process of generating a distribution video from a captured video obtained by capturing an event venue and distributing the distribution video to a user terminal device via a network, the method including: cutting out regions respectively corresponding to a plurality of viewpoints with respect to a subject from the captured video, and generating a plurality of distribution videos different in viewpoint; and distributing the distribution video corresponding to a distribution request to the user terminal device when receiving the distribution request including information on the viewpoint selected by a user from the user terminal device.
According to the present disclosure, it is possible to efficiently generate various videos matching diverse demands of the user by cutting out the regions respectively corresponding to the plurality of viewpoints from the captured video and generating the plurality of viewpoints different in viewpoint. The user can view the video matching his/her demand by selecting the viewpoint by the user.
In order to solve the problem, a first invention has a configuration of a video distribution system of the present disclosure that generates a distribution video from a captured video obtained by capturing an event venue and distributes the distribution video to a user terminal device via a network, the system including: a camera that is installed at a predetermined position of the event venue and captures the event venue in a predetermined direction; a captured video storage that accumulates the captured video output from the camera; a video processing computer that cuts out regions, respectively corresponding to a plurality of viewpoints with respect to a subject from the captured video, and generates a plurality of distribution videos different in viewpoint; and a video distribution server that distributes the distribution video corresponding to a distribution request to the user terminal device when receiving the distribution request including information on the viewpoint selected by a user from the user terminal device.
According to the configuration, it is possible to efficiently generate various videos matching diverse demands of the user by cutting out the regions respectively corresponding to the plurality of viewpoints from the captured video and generating a plurality of distribution videos different in viewpoint. The user can view the video matching his/her demand by selecting the viewpoint by the user.
A second invention has a configuration that the video processing computer detects a person from the captured video, and generates the plurality of distribution videos different in target person as the viewpoints based on a result of the person detection.
According to the configuration, when the user selects the target person as the viewpoint, it is possible to view a video in which a person to which the user attends is mainly reflected.
A third invention has a configuration that a plurality of the cameras are installed at different positions in the event venue, and the video processing computer generates the plurality of distribution videos different in capturing angle as the viewpoints from the captured video of the plurality of respective cameras.
According to the configuration, when the user selects the capturing angle as the viewpoint, the user can view videos captured from various angles.
A fourth invention has a configuration that the video processing computer generates the plurality of distribution videos different in apparent zoom magnification by changing a size of the region cut out from the captured video.
According to the configuration, the user can view videos different in zoom magnification.
A fifth invention has a configuration that the video processing computer generates the plurality of distribution videos different in attention portion of the subject as the viewpoint.
According to the configuration, the user can view the video in which the attention portion is mainly reflected by selecting the attention portion of the subject as the viewpoint by the user.
A sixth invention has a configuration that the attention portion is at least one of a whole body, a face, a hand, and a feet of a person, and a musical instrument played by the person.
According to the configuration, the user can view an image in which the whole body, the face, the hand, and the feet of the person, and the musical instrument played by the person are reflected.
A seventh invention has a configuration that at a time of starting distribution, during distribution of the distribution video in response to a first distribution request transmitted from the user terminal device, when a second distribution request including information on the viewpoint selected by the user is received, the video distribution server starts preparation for distribution of the distribution video corresponding to the second distribution request, and when the preparation for distribution is completed, the video distribution server starts distribution of the distribution video corresponding to the second distribution request.
According to the configuration, while the user views the video on the user terminal device, it is possible to switch the video to another video different in viewpoint.
An eighth invention has a configuration of a user terminal device of the present disclosure that receives a distribution video distributed from a video distribution device via a network and displays the distribution video, the device including: an operation input unit that performs an operation of selecting an event desired to be viewed and a viewpoint with respect to a subject by a user; a transmitter that transmits a distribution request including information on the event and the viewpoint which are selected by the user to the video distribution device; a receiver that receives the distribution video corresponding to the distribution request distributed from the video distribution device; and a processor that controls reproduction of the received distribution video and displays the distribution video on a video display unit.
According to the configuration, similar to the first invention, it is possible to efficiently generate various videos matching diverse demands of the user and to view the video matching the demand of the user his/herself.
A ninth invention has a configuration of a video distribution method of causing an information processing device to perform a process of generating a distribution video from a captured video obtained by capturing an event venue and distributing the distribution video to a user terminal device via a network, the method including: cutting out regions respectively corresponding to a plurality of viewpoints with respect to a subject from the captured video, and generating a plurality of distribution videos different in viewpoint; and distributing the distribution video corresponding to a distribution request to the user terminal device when receiving the distribution request including information on the viewpoint selected by the user from the user terminal device.
According to the configuration, similar to the first invention, it is possible to efficiently generate various videos matching diverse demands of the user and to view the video matching the demand of the user his/herself.
Hereinafter, embodiments will be described with reference to the drawings.
The video distribution system generates a distribution video from a captured video obtained by capturing a concert venue (event venue) by camera 1 and distributes the distribution video to user terminal 2 (user terminal device) via a network, and includes camera 1, captured video storage 3, video processing computer 4, distribution video storage 5, video distribution server 6, and user terminal 2.
A plurality of cameras 1 are installed in the concert venue. Each camera 1 is connected to captured video storage 3. The captured video may be stored in captured video storage 3 on the network, or may be locally stored in the capturing venue using an optical disk, a portable HDD, or the like.
User terminal 2 is a tablet terminal, a smartphone, a PC, or the like, and is connected to video distribution server 6 via a network.
Captured video storage 3 accumulates captured videos output from the plurality of cameras 1.
Video processing computer 4 performs a video process with respect to the captured video accumulated in captured video storage 3 to generate a distribution video. The video process performed by video processing computer 4 is to automate the process of generating the distribution video, and this can eliminate a need for editor's work, thereby reducing an editing cost. Video processing computer 4 is loaded with editing software for the distribution video, and can confirm or correct the generated distribution video.
Distribution video storage 5 accumulates the distribution video output from video processing computer 4. In the present embodiment, store distribution is performed, the distribution video generated by video processing computer 4 is accumulated in distribution video storage 5, and after the concert is completed, the distribution video accumulated in distribution video storage 5 is distributed to user terminal 2.
In response to a distribution request from user terminal 2, video distribution server 6 performs streaming distribution in which data of the distribution video accumulated in distribution video storage 5 is sequentially distributed to user terminal 2.
Next, a disposition status of cameras 1 will be described.
Each camera 1 is installed at a predetermined position in the concert venue, and the installation position is fixed (fixed point capturing). Each camera 1 is set to capture the stage from a predetermined direction, and the capturing angle (capturing direction) is also fixed. In each camera 1, wide-angle capturing is performed so that the entire stage can be substantially captured, and thereby a person moving on the stage can be captured by each camera 1 without omission. In each camera 1, capturing with high resolution (for example, 4K or more) is performed, thereby enabling a video with high definition to be distributed even when software zoom is performed. A stereo image process can be performed using the captured video by two cameras set in advance among respective cameras 1, and 3D distribution video can also be created.
Respective cameras 1 are installed at different positions so as to surround the stage. In the example illustrated in
As described above, in the present embodiment, camera 1 is installed at a predetermined position in the event venue, and camera 1 constantly captures the event venue in a predetermined direction, and thereby a need for a cameraman is eliminated and the capturing cost can be reduced.
Next, schematic configurations of video processing computer 4 and video distribution server 6 will be described.
Video processing computer 4 includes communication unit 11, processor 12, and storage unit 13.
Communication unit 11 communicates with captured video storage 3 and distribution video storage 5, receives the captured video transmitted from captured video storage 3, and transmits the distribution video generated by processor 12 to distribution video storage 5.
Storage unit 13 stores the captured video or the like received from captured video storage 3. Storage unit 13 also stores a program to be executed by processor 12.
Processor 12 executes various processes by executing programs stored in storage unit 13. In the present embodiment, each of processes of person detection, cutout region setting, image cutout, and distribution video generation is performed.
In the person detection process, a person (moving object) is detected from each frame of the captured video, and position information on a person region in each frame is acquired. In this case, as illustrated to
In the cutout region setting process, a position and a size of the region cut out from the captured video are set based on the position information on the person region acquired in the human detection process. In the present embodiment, a distribution video in which a person detected from a captured video is reflected at a center is generated for each person, and the position of the cutout region is set so that the center of the person region (person frame) becomes the center of the cutout region.
In the present embodiment, so-called software zoom is performed, the subject is zoomed in and displayed in large and by reducing the cutout region, and the subject is zoomed out and displayed in small by enlarging the cutout region. Therefore, in the present embodiment, the size of the cutout region is set in accordance with the zoom magnification.
In the present embodiment, regions respectively corresponding to a plurality of viewpoints with respect to the subject are cut out from the captured video to generate a plurality of distribution videos different in viewpoint, and in particular, to generate a plurality of distribution videos different in attention portion of the persons as the viewpoint. Specifically, a distribution video in which the whole body, the face, the hand, and the feet of the person, and the musical instrument played by the person are at the center is generated, and the position of the cutout region is set so that the center of the attention portion becomes the center of the cutout region.
The example illustrated in
Here, in a case where the attention portion is the whole body of the person, the center of the person region (person frame) may be set at the center of the cutout region.
In a case where the attention portion is the face, an upper region of the person region may be set as the region of the face and the center of the region may be set at the center of the cutout region. A Ω shape formed by the head and the shoulder may be detected, the head region may be specified based on the Ω shape, and the center of the region may be set at the center of the cutout region.
In a case where the attention portion is the hand, the hand may be detected for the person region as a target and the center of the hand region may be set at the center of the cutout region. A lower region of the head region specified by the Ω shape may be set as the hand region, and the center of the region may be set at the center of the cutout region.
In a case where the attention portion is the foot, the foot may be detected for the person region as a target, and the center of the foot region may be set at the center of the cutout region. A lower region of the person region may be set as the foot region, and the center of the region may be set at the center of the cutout region.
In a case where the attention portion is the musical instrument, the musical instrument may be detected for a periphery of the hand region as a target, and the center of the musical instrument region may be set at the center of the cutout region. In the detection of the musical instrument, the musical instrument may be detected paying attention to a shape and a color thereof.
In the present embodiment, the attention portions are the whole body, the face, the hand, and the feet of the person, and the musical instrument, but the upper body or the lower body of the person may be the attention portion.
In the image cutout process, an image (cutout image) of the cutout region is cut out from the captured video based on the position and the size of the cutout region set in the cutout region setting process.
In the distribution video generation process. a distribution video is generated based on the cutout image acquired in the image cutout process. In this case, the distribution video is generated based on the specification of video distribution and, for example, resampling is performed by the resolution (for example, 2K) of the distribution video.
Video distribution server 6 includes first communication unit 21, second communication unit 22, processor 23, and storage unit 24.
First communication unit 21 communicates with distribution video storage 5 and receives the distribution video transmitted from distribution video storage 5.
Second communication unit 22 communicates with user terminal 2, receives a distribution request transmitted from user terminal 2, and transmits the distribution video to user terminal 2.
Storage unit 24 stores the distribution video or the like received from distribution video storage 5. Storage unit 24 stores a program to be executed by processor 23.
Processor 23 executes programs stored in storage unit 24 to execute various processes. In the present embodiment, each process of distribution request reception and distribution control is performed.
In the distribution request reception process, a distribution request transmitted from user terminal 2 and received by second communication unit 22 is received.
In the distribution control process, in response to the distribution request received in the distribution request reception process, the distribution video corresponding to the distribution request, that is, the distribution video corresponding to the viewpoint selected by the user is acquired from distribution video storage 5, and streaming distribution, in which distribution video data is sequentially transmitted to user terminal 2, is controlled.
Next, a schematic configuration of user terminal 2 will be described.
User terminal 2 includes communication unit 31, processor 32, storage unit 33, video display unit 34, and operation input unit 35.
Communication unit 31 communicates with video distribution server 6, transmits a distribution request to video distribution server 6, and receives the distribution video transmitted from video distribution server 6.
Storage unit 33 stores the distribution video received from video distribution server 6. Storage unit 33 stores a program to be executed by processor 32.
Video display unit 34 is configured of a liquid crystal display panel or the like. Operation input unit 35 is configured of a touch panel, a mouse, or the like. Video display unit 34 and operation input unit 35 can be configured of a touch panel display.
Processor 32 performs various processes by executing programs stored in storage unit 33. In the present embodiment, the distribution request is transmitted from communication unit 31 to video distribution server 6 based on an operation of the user in operation input unit 35. The reproduction of the distribution video received by communication unit 31 is controlled, and the distribution video is displayed on video display unit 34.
In video distribution server 6, an audio is distributed together with the video, and user terminal 2 includes an audio output unit (not illustrated) configured of a speaker or the like, and in the audio output unit, the audio distributed from video distribution server 6 is output.
Here, in the present embodiment, when video distribution server 6 is caused to start video distribution, a first distribution request is transmitted from user terminal 2 to video distribution server 6 in accordance with the operation of the user, and a distribution video corresponding to the first distribution request is distributed from video distribution server 6 to user terminal 2. In the present embodiment, when starting the distribution, the user can select a person as a viewpoint, and the first distribution request including the selection information is transmitted from user terminal 2 to video distribution server 6.
In the present embodiment, a second distribution request is transmitted from user terminal 2 to video distribution server 6 in response to an operation of the user for selecting the viewpoint during the video distribution, and the distribution video corresponds to the second distribution request, that is, the distribution video corresponding to the viewpoint selected by the user is distributed from video distribution server 6 to user terminal 2. In the present embodiment, as the viewpoint, the capturing angle, the attention portion (full body, face, hand, feet, or musical instrument), and the target person can be selected, and the second distribution request including the selection information is transmitted from user terminal 2 to video distribution server 6. As a result, it is possible to switch the video to another video different in viewpoint at an arbitrary timing during the video distribution.
Next, the accumulation status of the captured video in captured video storage 3 will be described.
In the present embodiment, the captured video output from each camera 1 is accumulated in captured video storage 3. In captured video storage 3, a video ID (video identification information) with a serial number or the like is added to the captured video (video binary data), and the captured video is managed based on the video ID. In captured video storage 3, the captured video is managed based on a camera ID (camera identification information) of camera 1 of the capturing source which is attribute information of the captured video.
Although the example illustrated in
Next, the accumulation status of the distribution video in distribution video storage 5 will be described.
In video processing computer 4, a plurality of distribution videos different in viewpoint with respect to the subject are generated, and the plurality of distribution videos are accumulated in distribution video storage 5. In distribution video storage 5, the video ID (video identification information) with a serial number or the like is given to the distribution video (video binary data), and the distribution video is managed based on the video ID. In distribution video storage 5, the distribution video is managed based on the viewpoint (target person, attention portion, or capturing angle) and the zoom magnification, which are the attribute information of the distribution video.
In the example illustrated in
Next, the screen displayed on user terminal 2 will be described.
As illustrated in
In video distribution server 6, user authentication is performed based on the input information, and when the user authentication is successful, a success notification for permitting use is transmitted to user terminal 2.
In user terminal 2, when the success notification is received from video distribution server 6, as illustrated in
As illustrated in
On the person selection screen, as illustrated in
In video distribution server 6, when the first distribution request from user terminal 2 is received, the distribution video corresponding to the first distribution request, that is, the distribution video corresponding to the event and the person selected by the user is acquired from distribution video storage 5, and a stream distribution for sequentially transmitting data of the distribution video to user terminal 2 is started.
In user terminal 2, when the stream distribution is started by video distribution server 6, the screen changes to the reproduction screen.
On the reproduction screen, as illustrated in
“Setting” button 49 is provided on the reproduction screen. When the user operates button 49, as illustrated in
On these setting screens during the reproduction, as illustrated in
As illustrated in
As illustrated in
As illustrated in
Here, when icon 61 of the whole body is operated, a video, in which the whole body (entire person) of the target person is reflected, is displayed. When face icon 62 is operated, a video centered on the face of the target person is displayed. When hand icon 63 is operated, a video centered on the hand of the target person is displayed. When foot icon 64 is operated, a video centered on the foot of the target person is displayed. When musical instrument icon 65 is operated, a video centered on the musical instrument being played by the target person is displayed.
As described above, in the present embodiment, since the attention portion of the target person can be changed, it is possible to observe the movement of the whole body or observe the step of the foot in detail, for example, when the person is dancing, and to observe fingering in detail when a person is playing the musical instrument.
As illustrated in
Each of these reproduction setting screens (zoom magnification setting screen, capturing angle setting screen, attention portion setting screen, and target person setting screen) is provided with reproduction operation unit 69. Reproduction operation unit 69 can perform operations such as start and stop of the reproduction, adjustment of a sound volume, and full screen display.
As described above, in the reproduction setting screens (zoom magnification setting screen, capturing angle setting screen, attention portion setting screen, and target person setting screen), when an operation for setting the zoom magnification and the viewpoint (capturing angle, attention portion, and target person) is performed, as illustrated in
When video distribution server 6 receives the second distribution request from user terminal 2, the distribution video corresponding to the second distribution request, that is, the distribution video corresponding to the zoom magnification and the viewpoint (capturing angle, attention portion, and target person) set by the user is acquired from distribution video storage 5 to start preparation for the stream distribution of the distribution video. When the preparation for the stream distribution is completed, stream distribution of a new distribution video is started, and in user terminal 2, the video displayed on the reproduction screen is switched to a new video based on the zoom magnification and the viewpoint set by the user.
As described above, in the present embodiment, the user can switch the video to another video different in viewpoint by performing an operation to change the viewpoint by the user while the user views the video on user terminal 2, and the user can view the video matching his/her demand.
Next, a second embodiment will be described. Points not particularly mentioned here are the same as those of the above embodiment.
In the first embodiment (see
Although the schematic configurations of video processing computer 4 and video distribution server 6 are the same as those in the first embodiment, video processing computer 4 generates all the distribution videos regardless of the distribution request from user terminal 2, and video distribution server 6 selects the distribution video corresponding to the distribution request, that is, the distribution video corresponding to the viewpoint selected by the user, from the distribution video output from video processing computer 4 in response to the distribution request from user terminal 2 to distribute the distribution video to user terminal 2.
As described above, the embodiments are described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to the embodiments, and can be applied to embodiments in which changes, replacements, additions, omissions, and the like are performed. It is also possible to combine respective configuration components explained in the above embodiments to set those as a new embodiment.
For example, in the above embodiments, the video processing computer (video processing device) and the video distribution server (video distribution device) are provided, but the video processing computer and the video distribution server may be configured of one information processing device.
Although the captured video storage and the distribution video storage are provided, the captured video storage and the distribution video storage may be configured of one storage device.
In the above embodiments, although an example in which the subject of the video is a person is described, the subject of the video is not limited to the person, and may be a video in which a moving object other than the person such as an animal is the subject.
The video distribution system, the user terminal device, and the video distribution method according to the present disclosure have the effects that various videos matching diverse demands of the user can be efficiently generated and a video that matches the needs of the user his/herself can be viewed, and are useful as a video distribution system and a video distribution method for distributing the distribution video to the user terminal device via a network by generating the distribution video from the captured video obtained by capturing the event venue, and a user terminal device that displays the distribution video by receiving the distribution video distributed from the video distribution device via a network.
1 CAMERA
2 USER TERMINAL (USER TERMINAL DEVICE)
3 CAPTURED VIDEO STORAGE
4 VIDEO PROCESSING COMPUTER
5 DISTRIBUTION VIDEO STORAGE
6 VIDEO DISTRIBUTION SERVER
31 COMMUNICATION UNIT (TRANSMITTER, RECEIVER)
32 PROCESSOR
34 VIDEO DISPLAY UNIT
35 OPERATION INPUT UNIT
Number | Date | Country | Kind |
---|---|---|---|
2017-008318 | Jan 2017 | JP | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2017/045412 | 12/19/2017 | WO | 00 |