An aspect of the present invention relates to a generation apparatus of structured data, a reconstruction apparatus of a virtual viewpoint image, a generation method of structured data, a control program, and a recording medium in which the control program is described.
In recent years, various techniques have been developed for streaming distribution of a large capacity of video, such as a multi-viewpoint image, an entire celestial sphere image, or the like. Examples of such techniques include Dynamic Adaptive Streaming over HTTP (DASH) which is being standardized by Moving Picture Experts Group (MPEG). In DASH, formats of description information called Media Presentation Description (MPD) data and Segment data are defined.
NPL 1: ISO/IEC 23009-1 Second edition 2014-05-15
However, the existing technique as described above has a problem that, in a case that a reconstruction apparatus that performs image reconstruction processing acquires a virtual viewpoint image from a generation apparatus or the like that performs image generation processing, and generates an image with a viewpoint position, it is difficult to determine the virtual viewpoint image to be acquired from the generation apparatus or the like.
An aspect of the present invention has been made in view of the problem described above, and an object of the present invention is to provide a generation apparatus that generates structured data with which a reconstruction apparatus can easily specify and acquire a preferable virtual viewpoint image.
In order to solve the above-described problem, a generation apparatus according to an aspect of the present invention includes: a virtual viewpoint image generation unit configured to generate a virtual viewpoint image on a per section basis in a virtual space; and a data generation unit configured to generate structured data indicating an acquisition source of a piece of data constituting the virtual viewpoint image, in which the data generation unit includes, in the structured data, subset data defining one or more prescribed ranges identifiable from each other, each of the one or more prescribed ranges including one or more virtual viewpoint positions in the virtual space.
According to an aspect of the present invention, it is possible to implement a generation apparatus that generates structured data with which a reconstruction apparatus can easily specify and acquire an image at a preferable virtual viewpoint position.
Embodiments of the present invention will be described as follows with reference to
An embodiment of the present invention will be described below with reference to
In the present embodiment, a configuration of an image reconstruction system that defines a sub-space in a virtual viewpoint space will be described. Note that in the present specification, unless otherwise stated, a still image and a moving image are not distinguished, and both are referred to as images.
A configuration of an image reconstruction system 1 according to the present embodiment will be described with reference to
The generation apparatus 2 is an apparatus configured to generate each virtual viewpoint image in a virtual space and Media Presentation Description (MPD) data, which are structured data indicating an acquisition source of a piece of data (media segment) constituting the virtual viewpoint image. Note that the virtual viewpoint image refers to a viewpoint image at a viewpoint position in the virtual space. Additionally, details of the MPD data will be described later.
The generation apparatus 2 includes a controller 3, a storage unit 4, and a communication unit 5. The controller 3 is a control device that integrally controls the generation apparatus 2 as a whole, and includes a virtual viewpoint image generation unit 11, a data generation unit 12, and a communication controller 13.
The virtual viewpoint image generation unit 11 generates a virtual viewpoint image at a virtual viewpoint position in the virtual space. The data generation unit 12 generates the MPD data described above. The communication controller 13 performs control related to communication processing by the communication unit 5.
The storage unit 4 is a storage device configured to store various data. The communication unit 5 performs communication processing with an external device via a network.
The reconstruction apparatus 20 is an apparatus configured to acquire a piece of data constituting the virtual viewpoint image with reference to the MPD data described above, and reconstruct the virtual viewpoint image. The reconstruction apparatus 20 includes a controller 21, a storage unit 22, a display unit 23, and a communication unit 24.
The controller 21 is a control device that integrally controls the reconstruction apparatus 20 as a whole, and includes a reconstruction processing unit 31, a data acquiring unit 32, a position acquiring unit 33, a display controller 34, and a communication controller 35.
The reconstruction processing unit 31 performs processing for generating the virtual viewpoint image from the media segment. The data acquiring unit 32 acquires the MPD data and the media segment. The position acquiring unit 33 acquires a position and posture (orientation) of a user in the virtual space. A display acquiring unit performs control related to picture display processing by the display unit 23. The communication controller 35 performs control related to communication processing by the communication unit 24.
The storage unit 22 is a storage device configured to store various data. The display unit 23 is a display panel configured to perform display processing of a moving image or a still image. The communication unit 24 performs communication processing with an external device via a network.
The virtual viewpoint image and the MPD data generated by the generation apparatus 2 in the present embodiment will be described with reference to
As illustrated in
Here, one virtual viewpoint image as included in
In the image reconstruction system 1, in the virtual space, a prescribed range including one or more virtual viewpoint positions is defined as a sub-space.
The data generation unit 12 generates the MPD data including the subset data defining one or more prescribed ranges identifiable from each other, each of the one or more prescribed ranges including one or more virtual viewpoint positions in the virtual space as illustrated in Subset S1 and Subset S2 in
The flow of processing according to the virtual viewpoint image reconstruction according to the present embodiment will be described for each step with reference to
In step S101, the reconstruction apparatus 20 receives the MPD data. The communication controller 13 included in the generation apparatus 2 transmits, via the communication unit 5, the MPD data including the subset data described above as illustrated in
Next, in step S102, the position acquiring unit 33 acquires a position and posture of the user in the virtual space.
Next, in step S103, the controller 21 selects a subset in which the user is positioned with reference to the MPD data stored in the storage unit 22 in step S101 and the position of the user acquired by the position acquiring unit 33 in step S102.
Next, in step S104, the controller 21 determines whether or not the sub-space (subset) in which the user is positioned has been changed. In a case that the sub-space in which the user is positioned has been changed or in a case that the process transitions to present step S104 for the first time, the process transitions to step S105, and in a case that there is no change in the sub-space in which the user is positioned, the process transitions to step S107.
In step S105, the data acquiring unit 32 acquires, with reference to AdaptationSet corresponding to each of the virtual viewpoint positions included in the sub-space in which the user is positioned, the media segments constituting each virtual viewpoint image via the communication unit 24, and stores them in the storage unit 22.
Next, in step S106, the data acquiring unit 32 stops acquisition of the media segments included in AdaptationSet corresponding to each of the virtual viewpoint positions included in the subset other than the subset selected by the controller 21 in step S103.
Next, in step S107, the position acquiring unit 33 determines whether or not the user is positioned in a common range of multiple sub-spaces, that is, an overlap range. In a case that the user is positioned in the common range of multiple sub-spaces, the process transitions to step S108, and in a case of not being positioned, the process transitions to step S109.
In step S108, the reconstruction processing unit 31 generates the virtual viewpoint image at the virtual viewpoint position of the user, by composing the media segments included in AdaptationSet corresponding to each of the virtual viewpoint positions included in the common range of the sub-space in step S107 in which the user is positioned. Furthermore, in this step, the data acquiring unit 32 may stop acquisition of the media segments included in AdaptationSet other than each of the AdaptationSet elements described above. Processing in this step is performed and then the process transitions to step S110.
In step S109, the reconstruction processing unit 31 generates the virtual viewpoint image at the virtual viewpoint position of the user, from the media segments corresponding to each of the virtual viewpoint position included in the sub-space in which the user is positioned.
In step S110, the display controller 34 causes, with reference to the virtual viewpoint image generated by the reconstruction processing unit 31 in step S108 or step S109, and the position and posture of the user acquired by the position acquiring unit 33 in step S102, the display unit 23 to display an image corresponding to a field of view from the virtual viewpoint position of the user.
Next, in step S111, the controller 21 determines whether or not a prescribed time has elapsed since the count has started in this step. In a case that the prescribed time has elapsed, the process transitions to step S102, and in a case of the time has not elapsed, the process transitions to step S107.
In other words, the reconstruction apparatus 20 iterates a flow from the processing in which the position acquiring unit 33 acquires the position and posture of the user in the virtual space in step S102 to the processing in which the display processing unit causes the display unit 23 to display the image corresponding to the field of view from the virtual viewpoint position of the user in step S110, until the user performs a prescribed termination operation, or until the moving image indicated by the MPD data stored in the storage unit 22 in step S101 is terminated. The flow of processing according to the flowchart of
As described above, the generation apparatus 2 according to the present embodiment includes: the virtual viewpoint image generation unit 11 configured to generate a virtual viewpoint image on a per section basis in a virtual space; and the data generation unit 12 configured to generate structured data indicating an acquisition source of a piece of data constituting the virtual viewpoint image, in which the data generation unit 12 includes, in the structured data, subset data defining one or more prescribed ranges identifiable from each other, each of the one or more prescribed ranges including one or more virtual viewpoint positions in the virtual space.
According to the above-described configuration, it is possible to implement the generation apparatus 2 that generates structured data with which the reconstruction apparatus 20 can easily specify and acquire an image of a preferable virtual viewpoint position.
Furthermore, the reconstruction apparatus 20 according to the present embodiment includes: the data acquiring unit 32 configured to acquire structured data indicating an acquisition source of a piece of data constituting a virtual viewpoint image generated on a per section basis in a virtual space, the structured data including subset data defining one or more prescribed ranges identifiable from each other, each of the prescribed ranges including one or more virtual viewpoint positions in the virtual space; and the position acquiring unit 33 configured to acquire a position of a user in the virtual space, in which the data acquiring unit 32 acquires, with reference to the structured data and the position of the user acquired by the position acquiring unit 33, a piece of data constituting the virtual viewpoint image corresponding to a virtual viewpoint position of the one or more virtual viewpoint positions included in a prescribed range of the one or more prescribed ranges in which the user is present.
According to the above-described configuration, it is possible to implement the reconstruction apparatus 20 that can easily specify and acquire an image of a preferable virtual viewpoint position.
A first modification of Embodiment 1 will be described with reference to
The virtual viewpoint image generation unit 11 generates the virtual viewpoint image on a per section basis in the virtual space, and stores the media segments constituting each virtual viewpoint image in the storage unit 4 or in a storage device which is not illustrated in
The data acquiring unit 32 included in the reconstruction apparatus 20 acquires the MPD data as illustrated in
In the present modification, in processing corresponding to step S103 of Embodiment 1, the controller 21 selects a subset in which the user is positioned with reference to the MPD data stored in the storage unit 22 and the position and posture (orientation) of the user in the virtual space acquired by the position acquiring unit 33.
According to the above-described configuration, since the data acquiring unit 32 does not acquire the virtual viewpoint image in a direction in which the user's position is not oriented, an effect of reducing the amount of communication processing can be achieved.
A second modification of Embodiment 1 will be described with reference to
The data generation unit 12 generates the MPD data including the subset data, which indicate the sub-space, defining one or more prescribed ranges identifiable from each other, each of the one or more prescribed ranges including one or more virtual viewpoint positions in the three-dimensional virtual space as illustrated in Subset S1 in
In the present modification as well, in the same manner as Embodiment 1, the reconstruction apparatus 20 refers to the MPD data, receives the media segments corresponding to the subset in which the user is positioned, and uses them for subsequent processing.
According to the above-described configuration, even in a case that the user is positioned in the virtual three-dimensional space, the reconstruction apparatus 20 can easily specify an image of a preferable virtual viewpoint position.
Furthermore,
Note that the above-described Modification 1 and Modification 2 can be appropriately applied to the embodiments described below.
A second embodiment of the present invention will be described with reference to
The configuration illustrated in
The virtual viewpoint image and the MPD data generated by the generation apparatus in the present embodiment will be described with reference to
The virtual viewpoint image generation unit 11 generates the primary virtual viewpoint images from the multiple zero-order virtual viewpoint images, and generates the secondary virtual viewpoint images from the multiple zero-order or primary virtual viewpoint images.
As illustrated in a virtual viewpoint composition level in
The flow of processing according to the virtual viewpoint image reconstruction according to the present embodiment will be described for each step with reference to
In step S201, the reconstruction apparatus 20 receives the MPD data. The communication controller 13 included in the generation apparatus 2 transmits, via the communication unit 5, the MPD data including the composition level data described above as illustrated in
Next, in step S202, the position acquiring unit 33 acquires a position and posture of the user in the virtual space. Additionally, the controller 21 calculates a velocity at which the user moves in the virtual space by referring to position information of the user acquired by the position acquiring unit 33 as needed.
Next, in step S203, the data acquiring unit 32 determines, with reference to the composition level data included in AdaptationSet in the MPD data stored in the storage unit 22 in step S201, and the position of the user acquired by the position acquiring unit 33 in step S202 and the velocity of the user calculated by the controller 21, the virtual viewpoint image to be acquired. For example, the data acquiring unit 32 may be configured such that, in a case that the user moves at high velocity, even in a case that the virtual viewpoint image corresponds to the virtual viewpoint position on the movement path of the user, the virtual viewpoint image is not acquired in a case of being the secondary virtual viewpoint image described above.
Alternatively, in a case that the communication speed in the network between the generation apparatus 2 and the reconstruction apparatus 20 is slow, and the overall processing speed in the image reconstruction system 1 is faster in a case that the reconstruction apparatus 20 side generates the primary virtual viewpoint image, the secondary virtual viewpoint image, or both of them, the data acquiring unit 32 may be configured so as not to acquire the primary virtual viewpoint image, the secondary virtual viewpoint image, or both of them from the generation apparatus 2.
Next, in step S204, the data acquiring unit 32 acquires, with reference to the MPD data stored in the storage unit 22 in step S201, the media segments constituting the virtual viewpoint image that has been determined to be acquired in step S203, and stores them in the storage unit 22.
Next, in step S205, the reconstruction processing unit 31 generates the virtual viewpoint image at the virtual viewpoint position on the movement path of the user, by composing the media segments stored in the storage unit 22 at step S204.
Next, in step S206, the display controller 34 causes, with reference to the virtual viewpoint image generated by the reconstruction processing unit 31 in step S205, and the position and posture of the user acquired by the position acquiring unit 33 in step S202, the display unit 23 to display an image corresponding to a field of view from the virtual viewpoint position of the user. The above description is processing in step S206.
The reconstruction apparatus 20 iterates processing from the processing in which the position acquiring unit 33 acquires the position and posture of the user in the virtual space in step S202 to the processing in which the display processing unit causes the display unit 23 to display the image corresponding to the field of view from the virtual viewpoint position of the user in step S206, until the user performs a prescribed termination operation, or until the moving image indicated by the MPD data stored in the storage unit 22 in step S201 is terminated.
As described above, the generation apparatus 2 according to the present embodiment includes: the virtual viewpoint image generation unit 11 configured to generate a virtual viewpoint image on a per section basis in a virtual space; and the data generation unit 12 configured to generate structured data indicating an acquisition source of a piece of data constituting the virtual viewpoint image, in which the virtual viewpoint image generation unit 11 recursively generates a virtual viewpoint image from a generated virtual viewpoint image, and the data generation unit 12 includes, in a data group constituting the virtual viewpoint image of the structured data, composition level data indicating the number of times of recursive composition through which the virtual viewpoint image is generated.
According to the above-described configuration, for example, in the reconstruction apparatus 20, in a case that the user moves in the virtual space at high velocity, the generation apparatus 2 can generate the MPD data that enable only a virtual viewpoint image minimum necessary for viewpoint composition to be acquired. Furthermore, the reconstruction apparatus 20 according to the present embodiment includes: the data acquiring unit 32 configured to acquire structured data indicating an acquisition source of a piece of data constituting a virtual viewpoint image generated on a per section basis in a virtual space, the structured data including, in a data group constituting a virtual viewpoint image, composition level data indicating the number of times of recursive composition through which the virtual viewpoint image is generated, in which the data acquiring unit 32 determines, with reference to the composition level data, the virtual viewpoint image to be acquired.
According to the above-described configuration, for example, in a case that the user moves in the virtual space at high velocity, the reconstruction apparatus 20 can acquire only a virtual viewpoint image minimum necessary for viewpoint composition and use it for image reconstruction, and an effect of reducing the amount of communication data and the like is achieved.
A third embodiment of the present invention will be described with reference to
The configuration illustrated in
The virtual viewpoint image and the MPD data generated by the generation apparatus 2 in the present embodiment will be described with reference to
The flow of processing according to the virtual viewpoint image reconstruction according to the present embodiment will be described for each step with reference to
In step S301, the data acquiring unit 32 acquires the MPD data according to the present embodiment generated by the generation apparatus 2 via the communication unit 24, and stores the acquired data in the storage unit 22.
Next, in step S302, the position acquiring unit 33 acquires a position and posture of the user in the virtual space.
Next, in step S303, the data acquiring unit 32 determines, with reference to the arrangement pattern data included in the MPD data stored in the storage unit 22 in step S301, and the position of the user acquired by the position acquiring unit 33 in step S302 as appropriate, an AdaptationSet to acquire the corresponding virtual viewpoint image in reconstruction of a moving image corresponding to Period 0.
Next, in step S304, the data acquiring unit 32 acquires, with reference to the MPD data stored in the storage unit 22 in step S301, the media segments constituting the virtual viewpoint image that has been determined to be acquired in step S303, and stores them in the storage unit 22.
Next, in step S305, the reconstruction processing unit 31 generates the virtual viewpoint image at the virtual viewpoint position on the movement path of the user, by composing the media segments stored in the storage unit 22 at step S304.
Next, in step S306, the display controller 34 causes, with reference to the virtual viewpoint image generated by the reconstruction processing unit 31 in step S305, and the position and posture of the user acquired by the position acquiring unit 33 in step S302 as appropriate, the display unit 23 to display, for example, an image corresponding to a field of view from the virtual viewpoint position of the user.
Next, in step S307, in a case that a next Period is present, step 303 and subsequent processing are iterated. In the example of the MPD data illustrated in
The generation apparatus 2 according to the present embodiment includes: the virtual viewpoint image generation unit 11 configured to generate a virtual viewpoint image on a per section basis in a virtual space; and the data generation unit 12 configured to generate structured data indicating an acquisition source of a piece of data constituting the virtual viewpoint image, in which the data generation unit 12 generates the structured data including arrangement pattern data indicating a virtual viewpoint position at which each of the virtual viewpoint images is arranged.
According to the above-described configuration, by preparing multiple kinds of arrangement pattern data and switching between Periods, it is possible to implement the generation apparatus 2 that allows an image to be reconstructed from a recommended viewpoint for each Period while the user is in a fixed position.
Furthermore, the reconstruction apparatus 20 according to the present embodiment includes: the data acquiring unit 32 configured to acquire structured data indicating an acquisition source of a piece of data constituting a virtual viewpoint image generated on a per section basis in a virtual space, the structured data including arrangement pattern data indicating a virtual viewpoint position at which each of the virtual viewpoint images is arranged, in which the data acquiring unit 32 determines, with reference to the arrangement pattern data, the virtual viewpoint image to be acquired.
According to the above-described configuration it is possible to implement the reconstruction apparatus 20 that reconstructs an image from a viewpoint recommended by the generation apparatus 2 side for each Period while the user is in a fixed position.
A modification of Embodiment 3 will be described with reference to
The control blocks (in particular, the data generation unit 12) of the generation apparatus 2 and the control blocks (in particular, the reconstruction processing unit 31) of the reconstruction apparatus 20 may be implemented with a logic circuit (hardware) formed as an integrated circuit (IC chip) or the like, or may be implemented with software.
In the latter case, each of the generation apparatus 2 and the reconstruction apparatus 20 includes a computer that performs instructions of a program that is software for implementing each function. The computer includes at least one processor (control device), for example, and includes at least one computer-readable recording medium having the program stored thereon. On the computer, the processor reads the program from the recording medium and performs the program to achieve the object of the present invention. A Central Processing Unit (CPU) can be used as the processor, for example. As the above-described recording medium, a “non-transitory tangible medium” such as a tape, a disk, a card, a semiconductor memory, and a programmable logic circuit, for example, in addition to a Read Only Memory (ROM) and the like, can be used. Furthermore, a Random Access Memory (RAM) or the like for deploying the above-described program may be further provided. The above-described program may be supplied to the above-described computer via a transmission medium (such as a communication network and a broadcast wave) capable of transmitting the program. Note that one aspect of the present invention may also be implemented in a form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.
A generation apparatus 2 according to Aspect 1 of the present invention includes: a virtual viewpoint image generation unit 11 configured to generate a virtual viewpoint image on a per section basis in a virtual space; and a data generation unit 12 configured to generate structured data indicating an acquisition source of a piece of data constituting the virtual viewpoint image, in which the data generation unit 12 causes the structured data to include subset data defining one or more prescribed ranges identifiable from each other, each of the one or more prescribed ranges including one or more virtual viewpoint positions in the virtual space.
According to the above-described configuration, it is possible to implement the generation apparatus 2 that generates structured data with which the reconstruction apparatus 20 can easily specify and acquire an image of a preferable virtual viewpoint position.
A generation apparatus 2 according to Aspect 2 of the present invention includes: a virtual viewpoint image generation unit 11 configured to generate a virtual viewpoint image on a per section basis in a virtual space; and a data generation unit 12 configured to generate structured data indicating an acquisition source of a piece of data constituting the virtual viewpoint image, in which the virtual viewpoint image generation unit 11 recursively generates a virtual viewpoint image from the virtual viewpoint image that has been generated, and the data generation unit 12 causes a data group constituting the virtual viewpoint image of the structured data to include composition level data indicating the number of times of recursive composition through which the virtual viewpoint image is generated.
According to the above-described configuration, for example, in the reconstruction apparatus 20, in a case that the user moves in the virtual space at high velocity, the generation apparatus 2 can generate the MPD data that enable only a virtual viewpoint image minimum necessary for viewpoint composition to be acquired.
A generation apparatus 2 according to Aspect 3 of the present invention includes: a virtual viewpoint image generation unit 11 configured to generate a virtual viewpoint image on a per section basis in a virtual space; and a data generation unit 12 configured to generate structured data indicating an acquisition source of a piece of data constituting the virtual viewpoint image, in which the data generation unit 12 generates the structured data including arrangement pattern data indicating a virtual viewpoint position at which each of a plurality of the virtual viewpoint images is arranged.
According to the above-described configuration, by preparing multiple kinds of arrangement pattern data and switching between Periods, it is possible to implement the generation apparatus 2 that allows an image to be reconstructed from a recommended viewpoint for each Period while the user is in a fixed position.
A reconstruction apparatus 20 according to Aspect 4 of the present invention includes: a data acquiring unit 32 configured to acquire structured data indicating an acquisition source of a piece of data constituting a virtual viewpoint image generated on a per section basis in a virtual space, the structured data including subset data defining one or more prescribed ranges identifiable from each other, each of the one or more prescribed ranges including one or more virtual viewpoint positions in the virtual space; and a position acquiring unit 33 configured to acquire a position of a user in the virtual space, in which the data acquiring unit 32 acquires, with reference to the structured data and the position of the user acquired by the position acquiring unit 33, a piece of data constituting the virtual viewpoint image corresponding to a virtual viewpoint position of the one or more virtual viewpoint positions included in a prescribed range of the one or more prescribed ranges in which the user is present.
According to the above-described configuration, it is possible to implement the reconstruction apparatus 20 that can easily specify and acquire an image of a preferable virtual viewpoint position.
A reconstruction apparatus 20 according to Aspect 5 of the present invention includes: a data acquiring unit 32 configured to acquire structured data indicating an acquisition source of a piece of data constituting a virtual viewpoint image generated on a per section basis in a virtual space, the structured data including, in a data group constituting the virtual viewpoint image, composition level data indicating the number of times of recursive composition through which the virtual viewpoint is generated, in which the data acquiring unit 32 determines, with reference to the composition level data, the virtual viewpoint image to be acquired.
According to the above-described configuration, for example, in a case that the user moves in the virtual space at high velocity, the reconstruction apparatus 20 can acquire only a virtual viewpoint image minimum necessary for viewpoint composition and use it for image reconstruction, and thus achieving an effect of reducing the amount of communication data and the like.
A reconstruction apparatus 20 according to Aspect 6 of the present invention includes: a data acquiring unit 32 configured to acquire structured data indicating an acquisition source of a piece of data constituting a virtual viewpoint image generated on a per section basis in a virtual space, the structured data including arrangement pattern data indicating a virtual viewpoint position at which each of a plurality of the virtual viewpoint images is arranged, in which the data acquiring unit 32 determines, with reference to the arrangement pattern data, the virtual viewpoint image to be acquired.
According to the above-described configuration it is possible to implement the reconstruction apparatus 20 that reconstructs an image from a viewpoint recommended by the generation apparatus 2 side for each Period while the user is in a fixed position.
A generation method according to Aspect 7 of the present invention is a method for generating data, the method being performed by an apparatus, the method including: a virtual viewpoint image generation step configured to generate a virtual viewpoint image on a per section basis in a virtual space; and a data generation step configured to generate structured data indicating an acquisition source of a piece of data constituting the virtual viewpoint image, in which, in the step of generating the structured data, the structured data is caused to include subset data defining one or more prescribed ranges identifiable from each other, each of the one or more prescribed ranges including one or more virtual viewpoint positions in the virtual space.
A generation method according to Aspect 8 of the present invention is a method for generating data, the method being performed by an apparatus, the method including: a virtual viewpoint image generation step configured to generate a virtual viewpoint image on a per section basis in a virtual space; and a data generation step configured to generate structured data indicating an acquisition source of a piece of data constituting the virtual viewpoint image, in which, in the step of generating the virtual viewpoint image, a virtual viewpoint image is recursively generated from the virtual viewpoint image that has been generated, and in the step of generating the structured data, a data group constituting the virtual viewpoint image of the structured data is caused to include composition level data indicating the number of times of recursive composition through which the virtual viewpoint image is generated.
A generation method according to Aspect 9 of the present invention is a method for generating data, the method being performed by an apparatus, the method including: a virtual viewpoint image generation step configured to generate a virtual viewpoint image on a per section basis in a virtual space; and a data generation step configured to generate structured data indicating an acquisition source of a piece of data constituting the virtual viewpoint image, in which, in the data generation step, the structured data including arrangement pattern data indicating a virtual viewpoint position at which each of a plurality of the virtual viewpoint images is arranged is generated.
A control program for causing a computer to operate as a generation apparatus 2 according to Aspect 10 of the present invention, the control program for causing the computer to operate as the generation apparatus 2 may be a control program configured to cause a computer to operate as the generation apparatus 2 according to any one of Aspects 1 to 3 described above, and may have a configuration to cause the computer to operate as the generation apparatus 2.
A control program for causing a computer to operate as a reconstruction apparatus 20 according to Aspect 11 of the present invention, the control program for causing the computer to operate as the reconstruction apparatus 20 may be a control program configured to cause a computer to operate as the reconstruction apparatus 20 according to any one of Aspects 4 to 6 described above, and may have a configuration to cause the computer to operate as the reconstruction apparatus 20.
A recording medium according to Aspect 12 of the present invention may be a computer-readable recording medium having recorded therein the control program according to Aspect 10.
A recording medium according to Aspect 13 of the present invention may be a computer-readable recording medium having recorded therein the control program according to Aspect 11.
The generation apparatus 2 and the reconstruction apparatus 20 according to each of the aspects of the present invention may be implemented by a computer. In this case, a control program of the generation apparatus 2 and the reconstruction apparatus 20 configured to cause a computer to operate as each unit (software component) included in the generation apparatus 2 and the reconstruction apparatus 20 to implement the generation apparatus 2 and the reconstruction apparatus 20 by the computer and a computer-readable recording medium configured to record the control program are also included in the scope of the present invention.
The present invention is not limited to each of the above-described embodiments. It is possible to make various modifications within the scope of the claims. An embodiment obtained by appropriately combining technical elements each disclosed in different embodiments falls also within the technical scope of the present invention. Further, combining technical elements disclosed in the respective embodiments makes it possible to form a new technical feature.
Number | Date | Country | Kind |
---|---|---|---|
2018-033661 | Feb 2018 | JP | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2019/006377 | 2/20/2019 | WO | 00 |