Patent Grant
11895444
The present application claims priority under 35 U.S.C. § 119 to Japanese Patent Application No. 2021-126129, filed on Jul. 30, 2021. This Japanese Patent Application is hereby expressly incorporated by reference, in its entirety, into the present application.
The present invention relates to a control device, a control method, a projection system, and a computer readable medium storing a control program.
JP2015-026992A discloses capturing calibration projection images projected from a plurality of projectors separately in a plurality of regions by a camera and calculating various correction coefficients for performing registration, scale matching, distortion correction, brightness correction of an overlapping region, and the like for each projector.
JP2016-204068A1 discloses capturing a projection region of a projector separately in a plurality of regions in a partially overlapping manner and estimating a projective transformation matrix for connecting adjacent partial images to each other based on a captured image.
JP2012-047849A discloses, in stack projection, projecting overlapping test patterns at the same time and capturing the test patterns, projecting patterns obtained by changing a wavelength region such as R, and B or patterns obtained by changing polarization characteristics for each projector, and retrospectively separating the overlapping patterns.
One embodiment according to the disclosed technology provides a control device, a control method, a projection system, and a computer readable medium storing a control program that can easily adjust projection of a projection apparatus with respect to a wide projection range.
A control device according to an aspect of the present invention is a control device of a projection system including one or more projection apparatuses that project a first image including a plurality of marker images, and an imaging apparatus that captures at least a part of the first image, the control device comprising a processor, in which the processor is configured to perform a control of projecting a second image including a plurality of marker images from the projection apparatus based on a capturing result of at least the part of the first image by the imaging apparatus.
A control method according to another aspect of the present invention is a control method by a control device of a projection system including one or more projection apparatuses that project a first image including a plurality of marker images, and an imaging apparatus that captures at least a part of the first image, the control device including a processor, the control method comprising performing, by the processor, a control of projecting a second image including a plurality of marker images from the projection apparatus based on a capturing result of at least the part of the first image by the imaging apparatus.
A projection system according to still another aspect of the present invention is a projection system comprising one or more projection apparatuses that project a first image including a plurality of marker images, an imaging apparatus that captures at least a part of the first image, and a control device, in which the control device includes a processor, and the processor is configured to perform a control of projecting a second image including a plurality of marker images from the projection apparatus based on a capturing result of at least the part of the first image by the imaging apparatus.
A control program according to still another aspect of the present invention causes a processor of a control device of a projection system to execute a process, the projection system including one or more projection apparatuses that project a first image including a plurality of marker images, and an imaging apparatus that captures at least a part of the first image, the process comprising performing a control of projecting a second image including a plurality of marker images from the projection apparatus based on a capturing result of at least the part of the first image by the imaging apparatus.
According to the present invention, a control device, a control method, a projection system, and a control program that can easily adjust projection of a projection apparatus with respect to a wide projection range can be provided.
Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.
Projection System 100 of Embodiment
The computer 50 can communicate with the projection apparatus 10 and the imaging apparatus 90. In the example illustrated in
The projection apparatus 10 is a projection apparatus that can perform projection to a projection target object 6. The imaging apparatus 90 is an imaging apparatus that can capture an image projected to the projection target object 6 by the projection apparatus 10.
The projection target object 6 is an object such as a screen having a projection surface on which a projection image is displayed by the projection apparatus 10. In the example illustrated in
A projection range 11 illustrated by a dot dashed line is a region that is irradiated with projection light by the projection apparatus 10 in the projection target object 6. In the example illustrated in
Projection Apparatus 10
The control portion 4 controls the projection performed by the projection apparatus 10. The control portion 4 is a device including a control portion configured with various processors, a communication interface (not illustrated) for communicating with each portion, and a storage medium 4a such as a hard disk, a solid state drive (SSD), or a read only memory (ROM) and generally controls the projection portion 1. Examples of the various processors of the control portion of the control portion 4 include a central processing unit (CPU) that is a general-purpose processor performing various processing by executing a program, a programmable logic device (PLD) such as a field programmable gate array (FPGA) that is a processor having a circuit configuration changeable after manufacturing, or a dedicated electric circuit such as an application specific integrated circuit (ASIC) that is a processor having a circuit configuration dedicatedly designed to execute specific processing.
More specifically, a structure of these various processors is an electric circuit in which circuit elements such as semiconductor elements are combined. The control portion of the control portion 4 may be configured with one of the various processors or may be configured with a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA).
The operation reception portion 2 detects an instruction (user instruction) from a user by receiving various operations from the user. The operation reception portion 2 may be a button, a key, a joystick, or the like provided in the control portion 4 or a reception portion or the like that receives a signal from a remote controller for remotely operating the control portion 4.
The communication portion 5 is a communication interface that can communicate with the computer 50. The communication portion 5 may be a wired communication interface that performs wired communication as illustrated in
The projection portion 1, the control portion 4, and the operation reception portion 2 are implemented by, for example, one device (for example, refer to
Internal Configuration of Projection Portion 1
The light modulation portion 22 is configured with three liquid crystal panels (light modulation elements) that emit each color image by modulating, based on image information, each color light which is emitted from the light source 21 and is separated into three colors of red, blue, and green by a color separation mechanism, not illustrated, and a dichroic prism that mixes each color image emitted from the three liquid crystal panels and emits the mixed color image in the same direction. Filters of red, blue, and green may be mounted in each of the three liquid crystal panels, and each color image may be emitted by modulating the white light emitted from the light source 21 in each liquid crystal panel.
The light from the light source 21 and the light modulation portion 22 is incident on the projection optical system 23. The projection optical system 23 includes at least one lens and is composed of, for example, a relay optical system. The light that has passed through the projection optical system 23 is projected to the projection target object 6.
In the projection target object 6, a region irradiated with the light transmitted through the entire range of the light modulation portion 22 is the projectable range within which the projection can be performed by the projection portion 1. In the projectable range, a region that is actually irradiated with the light transmitted through the light modulation portion 22 is the projection range 11. For example, in the projectable range, a size, a position, and a shape of the projection range 11 are changed by controlling a size, a position, and a shape of a region through which the light is transmitted in the light modulation portion 22.
The control circuit 24 projects an image based on display data to the projection target object 6 by controlling the light source 21, the light modulation portion 22, and the projection optical system 23 based on the display data input from the control portion 4. The display data input into the control circuit 24 is configured with three constituents of red display data, blue display data, and green display data.
In addition, the control circuit 24 enlarges or reduces the projection range 11 (refer to
In addition, the projection apparatus 10 comprises a shift mechanism that mechanically or optically moves the projection range 11 while maintaining an image circle of the projection optical system 23. The image circle of the projection optical system 23 is a region in which the projection light incident on the projection optical system 23 appropriately passes through the projection optical system 23 in terms of a light fall-off, color separation, edge part curvature, or the like.
The shift mechanism is implemented by at least one of an optical system shift mechanism that performs optical system shifting, or an electronic shift mechanism that performs electronic shifting.
The optical system shift mechanism is, for example, a mechanism (for example, refer to
The electronic shift mechanism is a mechanism that performs pseudo shifting of the projection range 11 by changing a range through which the light is transmitted in the light modulation portion 22.
In addition, the projection apparatus 10 may comprise a projection direction changing mechanism that moves the image circle of the projection optical system 23 and the projection range 11. The projection direction changing mechanism is a mechanism that changes a projection direction of the projection portion 1 by changing a direction of the projection portion 1 by mechanical rotation (for example, refer to
Mechanical Configuration of Projection Apparatus 10
As illustrated in
The optical unit 106 comprises a first member 102 supported by the body part 101. The optical unit 106 may be configured to be attachable to and detachable from the body part 101 (in other words, interchangeably configured).
As illustrated in
As illustrated in
As illustrated in
As illustrated in
The first member 102 is a member having, for example, a rectangular cross-sectional exterior, in which an opening 2a and an opening 2b are formed in surfaces parallel to each other. The first member 102 is supported by the body part 101 in a state where the opening 2a is arranged at a position facing the opening 15a of the body part 101. The light emitted from the light modulation portion 22 of the light modulation unit 12 of the body part 101 is incident into the hollow portion 2A of the first member 102 through the opening 15a and the opening 2a.
An incidence direction of the light incident into the hollow portion 2A from the body part 101 will be referred to as a direction X1. A direction opposite to the direction X1 will be referred to as a direction X2. The direction X1 and the direction X2 will be collectively referred to as a direction X. In addition, in
In addition, a direction perpendicular to the direction X and the direction Z will be referred to as a direction Y. In the direction Y, an upward direction in
The projection optical system 23 illustrated in
The first optical system 121 includes at least one lens and guides the light that is incident on the first member 102 from the body part 101 and travels in the direction X1, to the lens 34.
The lens 34 is arranged in an end part of the first member 102 on the direction X1 side in the form of closing the opening 2b formed in this end part. The lens 34 projects the light incident from the first optical system 121 to the projection target object 6.
The first shift mechanism 105 is a mechanism for moving the optical axis K of the projection optical system (in other words, the optical unit 106) in a direction (direction Y in
The first shift mechanism 105 may be a mechanism that moves the light modulation portion 22 in the direction Y instead of moving the optical unit 106 in the direction Y. Even in this case, the image G1 projected to the projection target object 6 can be moved in the direction Y.
Hardware Configuration of Computer 50
For example, the processor 51 is a circuit performing signal processing and is a CPU that controls the entire computer 50. The processor 51 may be implemented by other digital circuits such as an FPGA and a digital signal processor (DSP). In addition, the processor 51 may be implemented by combining a plurality of digital circuits.
For example, the memory 52 includes a main memory and an auxiliary memory. For example, the main memory is a random access memory (RAM). The main memory is used as a work area of the processor 51.
For example, the auxiliary memory is a non-volatile memory such as a magnetic disk, an optical disc, or a flash memory. The auxiliary memory stores various programs for operating the computer 50. The programs stored in the auxiliary memory are loaded into the main memory and executed by the processor 51.
In addition, the auxiliary memory may include a portable memory that can be detached from the computer 50. Examples of the portable memory include a memory card such as a universal serial bus (USB) flash drive or a secure digital (SD) memory card and an external hard disk drive.
The communication interface 53 is a communication interface that communicates with an outside (for example, the projection apparatus 10 and the imaging apparatus 90) of the computer 50. The communication interface 53 is controlled by the processor 51. The communication interface 53 may be a wired communication interface that performs wired communication or a wireless communication interface that performs wireless communication or may include both of the wired communication interface and the wireless communication interface.
For example, the user interface 54 includes an input device that receives an operation input from a user, and an output device that outputs information to the user. For example, the input device can be implemented by a pointing device (for example, a mouse), a key (for example, a keyboard), or a remote controller. For example, the output device can be implemented by a display or a speaker. In addition, the input device and the output device may be implemented by a touch panel or the like. The user interface 54 is controlled by the processor 51.
Projection of First Image by Projection Apparatus 10 and Imageable Range of Imaging Apparatus 90
An imageable range 72 is a range that can be imaged by the imaging apparatus 90 in the projection target object 6. As illustrated in
For example, the computer 50 performs a control of outputting a message that prompts a user of the imaging apparatus 90 to perform imaging by including as many marker images as possible among the marker images of the first image 71. The user of the imaging apparatus 90 may be the same as or different from the user of the computer 50.
The user of the imaging apparatus 90 captures at least a part of the first image 71 by the imaging apparatus 90. In the example in
The computer 50 calculates the imageable range 72 based on the captured image transmitted from the imaging apparatus 90. Specifically, the calculation of the imageable range 72 is calculation of a relative size of the imageable range 72 with respect to the first image 71 projected to the projection target object 6. For example, the computer 50 calculates the number of marker images (in the example in
For example, the computer 50 determines the number of marker images included in the captured image from the imaging apparatus 90 by image recognition. In the example in
Projection of Second Images by Projection Apparatus 10
The second image 81 is obtained by extracting an upper left part of 4×7=28 marker images from the first image 71. That is, the second image 81 is an image in which 28 rectangular marker images are arranged in a 4×7 matrix. In addition, sizes and intervals of the marker images included in the second image 81 are the same as sizes and intervals of the marker images included in the first image 71. Accordingly, the user of the imaging apparatus 90 can perform the imaging by the imaging apparatus 90 by causing the 28 marker images included in the second image 81 to fall within the imageable range 72. The imaging apparatus 90 transmits a captured image of the second image 81 obtained by the imaging to the computer 50.
In the state illustrated in
Next, for example, as illustrated in
In the state illustrated in
Next, for example, as illustrated in
In the state illustrated in
Next, for example, as illustrated in
In the state illustrated in
As described using
For example, the computer 50 generates an image equivalent to the captured image of the entire first image 71 by combining the received captured images of the second images 81 to 84. The computer 50 detects distortion of a projection image within the projection range 11 based on distortion in shape or arrangement of marker images included in the generated image and performs distortion correction of the projection image based on a detection result.
The computer 50 causes the imaging apparatus 90 to capture at least a part of the first image 71 by projecting the first image 71 including a plurality of marker images from the projection apparatus 10 and repeats the control of projecting the second image (for example, the second images 81 to 84) including a plurality of marker images from the projection apparatus 10 based on a capturing result.
Specifically, the computer 50 calculates the imageable range 72 of the imaging apparatus 90 based on a captured image of at least a part of the first image 71 and performs the control of projecting the second images 81 to 84 from the projection apparatus 10 based on the calculated imageable range 72. At this point, the computer 50 projects the second images 81 to 84 from the projection apparatus 10 while changing a projection position of each of the second images 81 to 84. The computer 50 performs the control of adjusting the projection of the projection apparatus 10 based on capturing results of the second images 81 to 84 by the imaging apparatus 90.
Accordingly, even in a case where the entire first image 71 projected to the projection range 11 cannot be captured by the imaging apparatus 90, the second images 81 to 84 that can be captured by the imaging apparatus 90 can be captured by the imaging apparatus 90 by projecting the second images 81 to 84, and the projection of the projection apparatus 10 can be adjusted based on the capturing results. Thus, the projection of the projection apparatus 10 with respect to a wide projection range 11 can be easily adjusted.
In addition, in repeating the control of projecting the second image from the projection apparatus 10, the projection apparatus 10 may perform a control of changing the second image projected from the projection apparatus 10. For example, the projection apparatus 10 may set the marker images included in the second image 81 as rectangles, set the marker images included in the second image 82 as circles, set the marker images included in the second image 83 as triangles, and set the marker images included in the second image 84 as x marks. Accordingly, in receiving the captured images of the second images 81 to 84 from the imaging apparatus 90, the computer 50 can securely determine to which of the second images 81 to 84 the received captured images correspond by determining the shapes of the marker images included in the received captured images.
While processing of performing the imaging by sequentially projecting the second images 81 to 84 is described, the present invention is not limited thereto. For example, the computer 50 may project only the second image 81 among the second images 81 to 84 from the projection apparatus 10. In this case, the computer 50 estimates the distortion of the projection image within the entire projection range 11 based on the distortion in shape or arrangement of the marker images included in the second image 81 and performs the distortion correction of the projection image based on an estimation result.
In addition, while a case where the second images 81 to 84 are obtained by extracting a part of the first image 71 is described, the second images 81 to 84 are not limited thereto. For example, in the examples illustrated in
Processing by Computer 50
First, the computer 50 projects the first image 71 from the projection apparatus 10 by transmitting a control signal to the projection apparatus 10 (step S1201). For example, the computer 50 projects the first image 71 illustrated in
Next, the computer 50 performs the control of prompting the user of the imaging apparatus 90 to capture the first image 71 projected in step S1201 by the imaging apparatus 90 (step S1202). For example, this control is performed by controlling the projection apparatus 10 to project the message to the projection range 11.
Next, the computer 50 receives the captured image of the first image 71 obtained by the imaging prompted in step S1202 from the imaging apparatus 90 (step S1203). The transmission of the captured image by the imaging apparatus 90 may be automatically performed by the imaging apparatus 90 with the imaging of the imaging apparatus 90 as a trigger, or may be performed by a user operation after the imaging of the imaging apparatus 90.
Next, the computer 50 determines whether or not all marker images of the first image 71 are included in the captured image received in step S1203 (step S1204). In step S1204, in a case where all marker images are included (step S1204: Yes), the computer 50 performs the control of adjusting the projection of the projection apparatus 10 based on the captured image of the first image 71 received in step S1203 (step S1205) and finishes the series of processing.
In step S1204, in a case where at least any of the marker images is not included (step S1204: No), the computer 50 generates a plurality of second images (step S1206). For example, the computer 50 calculates the imageable range 72 based on the captured image of the first image 71 received in step S1203 and generates second images (for example, the second images 81 to 84) that can cover the projection range 11 based on the calculated imageable range 72.
Next, the computer 50 projects a non-projected second image from the projection apparatus 10 among the second images generated in step S1206 (step S1207). Next, the computer 50 performs the control of prompting the user of the imaging apparatus 90 to capture the second image projected in step S1207 by the imaging apparatus 90 (step S1208).
Next, the computer 50 receives the captured image of the second image obtained by the imaging prompted in step S1208 from the imaging apparatus 90 (step S1209). Next, the computer 50 determines whether or not all second images generated in step S1206 are completely projected in step S1207 (step S1210).
In step S1210, in a case where at least any of the second images is not projected (step S1210: No), the computer 50 returns to step S1207. In a case where all second images are projected (step S1210: Yes), the computer 50 performs the control of adjusting the projection of the projection apparatus 10 based on the captured images of the plurality of second images received in step S1209 (step S1211) and finishes the series of processing.
Another Example of First Image 71 Projected by Projection Apparatus 10
In addition, the computer 50 stores a correspondence table in which each number included in the first image 71 as the marker image is associated with a position at which the number is arranged in the projection range 11. That is, the marker images of the first image 71 illustrated in
Accordingly, even in a case where only a part of the marker images of the first image 71 is included in the captured image received from the imaging apparatus 90, the computer 50 can determine at which position the marker images of the part are present within the projection range 11.
For example, in a case where marker images of a predetermined ratio or more of the marker images of the first image 71 are included in the captured image of the first image 71 received from the imaging apparatus 90, the computer 50 determines the position of each marker image included in the captured image of the first image 71 within the projection range 11 based on the correspondence table, detects the distortion of the projection image within the entire projection range 11 based on a determination result, and performs the distortion correction of the projection image based on the detection result. In this case, the computer 50 may not perform the control of projecting the second image (for example, the second images 81 to 84) from the projection apparatus 10.
Another Example of Second Image 81 Projected by Projection Apparatus 10
In addition, the computer 50 stores a correspondence table in which each number included in the second image 81 as the marker image is associated with a position at which the number is arranged in the projection range 11. This correspondence table may be the same as the correspondence table described using
Accordingly, even in a case where only a part of the second image 81 is included in the captured image received from the imaging apparatus 90, the computer 50 can determine at which position the marker images of the part are present within the projection range 11.
While the second image 81 is described, the second images 82 to 84 may also be images in which different numbers or the like for each position are arranged as the marker images.
For example, in a case where only a part of the marker images of the second image 81 is included in the captured image of the second image 81 received from the imaging apparatus 90, the computer 50 determines the position of each marker image included in the captured image of the second image 81 within the projection range 11 based on the correspondence table and combines the captured image of the second image 81 with the captured images of the second images 82 to 84 based on the determination result. Accordingly, while a part of a region of the second image 81 is missing, an image equivalent to the captured image of the entire first image 71 can be generated. The computer 50 detects the distortion of the projection image within the projection range 11 based on the distortion in shape or arrangement of the marker images included in the generated image and performs the distortion correction of the projection image based on the detection result.
In the examples in
In addition, the marker images associated with the positions within the projection range 11 may be different Quick Response (QR) codes (registered trademark), ArUco markers, or the like for each position. In this case, information indicating the positions of the marker images within the projection range 11 can be included in the marker images. Accordingly, even in a case where the correspondence table is not stored, the computer 50 can determine the positions of the marker images within the projection range 11 by reading the information of the QR codes or ArUco markers included in the captured image as the marker images.
Example of State Before Adjustment of Stack Projection by Plurality of Projection Apparatuses
The projection apparatus 10A has the same configuration as the projection apparatus 10 and performs projection to the projection target object 6 together with the projection apparatus 10. The computer 50 can communicate with the projection apparatus 10A. In the example illustrated in
A projection range 11A illustrated by a double dot dashed line is a region that is irradiated with projection light by the projection apparatus 10A in the projection target object 6. In the example illustrated in
In this example, a case of performing the stack projection for improving a dynamic range or gradation representation by making the entire projection range 11 of the projection apparatus 10 overlap with the entire projection range 11A of the projection apparatus 10A and projecting the same image from the projection apparatuses 10 and 10A will be described.
Processing Based on Imaging Condition by Computer 50
First, the computer 50 acquires an imaging condition of the imaging apparatus 90 (step S161). This imaging condition of the imaging apparatus 90 includes whether the imaging by the imaging apparatus 90 is fixed imaging or handheld imaging. The fixed imaging is imaging in a state where the imaging apparatus 90 is fixed to an object such as a tripod or a seat that does not shake. The handheld imaging is imaging in a state where the imaging apparatus 90 is held in hands by the user, and a shake occurs.
For example, the computer 50 acquires the imaging condition of whether the imaging is the fixed imaging or the handheld imaging by receiving a user operation with respect to the computer 50. Alternatively, the computer 50 may acquire the imaging condition of whether the imaging is the fixed imaging or the handheld imaging based on a state of the shake of the imaging apparatus 90 depending on a gyro sensor or the like in the imaging apparatus 90, by communicating with the imaging apparatus 90. For example, in a case where the shake of the imaging apparatus 90 detected by the gyro sensor or the like is less than a threshold value, the computer 50 determines that the imaging is the fixed imaging. In a case where the shake of the imaging apparatus 90 detected by the gyro sensor or the like is greater than or equal to the threshold value, the computer 50 determines that the imaging is the handheld imaging.
Alternatively, the computer 50 may acquire the imaging condition of whether the imaging is the fixed imaging or the handheld imaging based on an ON/OFF state of a camera shake correction function in the imaging apparatus 90, by communicating with the imaging apparatus 90. For example, in a case where the camera shake correction function in the imaging apparatus 90 is in the OFF state, the computer 50 determines that the imaging is the fixed imaging. In a case where the camera shake correction function in the imaging apparatus 90 is in the ON state, the computer 50 determines that the imaging is the handheld imaging.
Next, the computer 50 determines whether or not the imaging of the imaging apparatus 90 is the handheld imaging based on the imaging condition acquired in step S161 (step S162). In a case where the imaging is not the handheld imaging (step S162: No), the computer 50 individually performs projection adjustment controls of the projection apparatuses 10 and 10A (step S163) and finishes the series of processing. The individual projection adjustment controls in step S163 will be described using
In step S162, in a case where the imaging is the handheld imaging (step S162: Yes), the computer 50 performs the projection adjustment controls of the projection apparatuses 10 and 10A at the same time (step S164) and finishes the series of processing. The projection adjustment controls in step S164 will be described using
Individual Projection of Projection Apparatuses 10 and 10A
In the examples in
As illustrated in
In addition, since the first images 71 and 71A are projected and captured at different timings, the computer 50 can receive each of the captured image of the first image 71 and the captured image of the first image 71A from the imaging apparatus 90. Thus, since the computer 50 may not perform processing of separately extracting each of the first images 71 and 71A from one captured image, the relative positional relationship between the projection ranges 11 and 11A can be securely determined by simple processing.
Projection of Projection Apparatuses 10 and 10A at Same Time
While illustration is not provided, in a case where step S1207 is executed in the processing in
Next, the second image 82 from the projection apparatus 10 illustrated in
In step S1211 of the processing in
As illustrated in
Stack Projection by Making Projection Ranges 11 and 11A Overlap
The computer 50 adjusts relative projection positions between the projection apparatus 10 and the projection apparatus 10A based on a result of the determination such that the entire projection range 11 overlaps with the entire projection range 11A as illustrated in
For example, by controlling the shift mechanism of the projection apparatus 10A to adjust the projection range 11A based on the projection range 11 of the projection apparatus 10, the computer 50 enables the stack projection by making the entire projection range 11 overlap with the entire projection range 11A.
For example, in a case of using the electronic shift mechanism, the computer 50 calculates a conversion parameter for correcting the projection range 11A such that the projection range 11A matches the projection range 11. For example, the conversion parameter includes a projective transformation (homography) matrix. The computer 50 can match the projection range 11A to the projection range 11 by correcting an input image of the projection apparatus 10A using the calculated conversion parameter and performing the projection from the projection apparatus 10A.
In a case of performing the stack projection by making the projection ranges of the projection apparatuses 10 and 10A (the plurality of projection apparatuses) overlap, the computer 50 adjusts the overlapping between the projection range 11 and the projection range 11A by adjusting the projection range 11A of the projection apparatus 10A based on the projection range 11 of the projection apparatus 10.
While a case of performing the projection adjustment controls for the projection of the projection apparatuses 10 and 10A in the stack projection of projecting the same image from the projection apparatuses 10 and 10A by making the entire projection range 11 of the projection apparatus 10 overlap with the entire projection range 11A of the projection apparatus 10A is described, a form of performing the projection adjustment controls for the projection of the projection apparatuses 10 and 10A is not limited thereto.
For example, blending projection for obtaining a large screen of projection may be performed by making an end part of the projection range 11 of the projection apparatus 10 overlap with an end part of the projection range 11A of the projection apparatus 10A and projecting each of divided images obtained by dividing a large image from the projection apparatuses 10 and 10A.
State Before Adjustment of Blending Projection by Plurality of Projection Apparatuses
Even in this case, as in the case of the stack projection, the computer 50 can perform the projection adjustment controls in performing registration or the distortion correction of the projection ranges 11 and 11A. For example, in the state illustrated in
However, in this case, in adjusting the projection ranges 11 and 11A, the computer 50 adjusts a relative position between the projection ranges 11 and 11A such that a specific region (for example, a right end region having a constant width) of the projection range 11 overlaps with a specific region (for example, a left end region having a constant width) of the projection range 11A. The specific region of the projection range 11 and the specific region of the projection range 11A have the same size.
Furthermore, the computer 50 performs blending processing such as dividing brightness of each of the projection images from the projection apparatuses 10 and 10A in half for an overlapping portion between the projection ranges 11 and 11A. Accordingly, incongruity such as bright display of only the overlapping portion between the projection ranges 11 and 11A can be reduced.
Blending Projection by Making Projection Ranges 11 and 11A Overlap
The computer 50 adjusts the relative projection positions between the projection apparatus 10 and the projection apparatus 10A based on a result of the determination such that the specific region of the projection range 11 overlaps with the specific region of the projection range 11A as illustrated in
For example, by controlling the shift mechanisms of the projection apparatuses 10 and 10A to adjust the projection ranges 11 and 11A, the computer 50 enables the blending projection by making the specific region of the projection range 11 overlap with the specific region of the projection range 11A.
For example, in a case of using the electronic shift mechanisms, the computer 50 calculates a conversion parameter for correcting the projection ranges 11 and 11A such that the specific region of the projection range 11 matches the specific region of the projection range 11A. For example, the conversion parameter includes a projective transformation (homography) matrix. The computer 50 can match the specific region of the projection range 11 to the specific region of the projection range 11A by correcting input images of the projection apparatuses 10 and 10A using the calculated conversion parameter and performing the projection from the projection apparatuses 10 and 10A.
In a case of performing the blending projection by making a part of the projection ranges of the projection apparatuses 10 and 10A (the plurality of projection apparatuses) overlap, the computer 50 adjusts the overlapping between the specific region of the projection range 11 and the specific region of the projection range 11A by adjusting each of the projection ranges 11 and 11A. Accordingly, by performing the blending processing such as adjusting brightness of the specific regions, the blending processing is applied to only the overlapping portion between the projection range 11 and the projection range 11A, and incongruity of appearance can be reduced.
Another Example of Processing Based on Imaging Condition by Computer 50
First, the computer 50 acquires the imaging condition of the imaging apparatus 90 (step S231). This imaging condition of the imaging apparatus 90 includes the resolution of the imaging of the imaging apparatus 90. The resolution is definition of the imaging and is decided by the number of pixels of an imaging sensor of the imaging apparatus 90, the angle of view of the imaging of the imaging apparatus 90, and the like.
For example, the computer 50 acquires the resolution of the imaging by receiving the resolution by the user operation with respect to the computer 50. Alternatively, the computer 50 may acquire the resolution of the imaging by communicating with the imaging apparatus 90 to receive information such as the number of pixels and the angle of view from the imaging apparatus 90.
Next, the computer 50 sets a size of the second image based on the resolution of the imaging included in the imaging condition acquired in step S231 (step S232). For example, in a case where the resolution of the imaging is greater than or equal to a threshold value, the computer 50 sets the size of the second image to “normal”. In a case where the resolution of the imaging is less than the threshold value, the computer 50 sets the size of the second image to “small”.
Next, the computer 50 performs the projection adjustment control of the projection apparatus 10 (step S233) and finishes the series of processing. In step S233, for example, the computer 50 adjusts the projection of the projection apparatus 10 by executing the processing illustrated in
For example, in a case where the size of the second image is set to “normal” in step S232, the computer 50 generates the second images 81 to 84 illustrated in
Second Image in Case where Resolution of Imaging of Imaging Apparatus 90 is Low
The second image 81 illustrated in
While the second image 81 projected in step S1206 of the first execution of the processing in
As illustrated in
While the control of projecting the second image having the size corresponding to the resolution of the imaging from the projection apparatus 10 in a case of performing the projection of only the projection apparatus 10 is described, a control of projecting the second image having the size corresponding to the resolution of the imaging from the projection apparatuses 10 and 10A may also be performed in a case of performing the projection of the projection apparatuses 10 and 10A.
Still Another Example of Processing Based on Imaging Condition by Computer 50
First, the computer 50 acquires the imaging condition of the imaging apparatus 90 as in step S231 in
Next, the computer 50 performs the projection adjustment control of the projection apparatus 10 (step S253) and finishes the series of processing. In step S253, for example, the computer 50 adjusts the projection of the projection apparatus 10 by executing the processing illustrated in
For example, in a case where the sizes of the marker images are set to “normal” in step S252, the computer 50 generates the second images 81 to 84 illustrated in
Marker Images of Second Image in Case where Resolution of Imaging of Imaging Apparatus 90 is Low
The marker images of the second image 81 illustrated in
While the second image 81 projected in step S1206 of the first execution of the processing in
As illustrated in
While the control of projecting the second image including the marker images having the sizes corresponding to the resolution of the imaging in a case of performing the projection of only the projection apparatus 10 is described, a control of projecting the second image including the marker images having the sizes corresponding to the resolution of the imaging from the projection apparatuses 10 and 10A may also be performed in a case of performing the projection of the projection apparatuses 10 and 10A.
While a configuration in which the optical axis K is not bent is described as a configuration of the projection apparatus 10 in
As illustrated in
As illustrated in
In the examples in
The first optical system 121 guides the light that is incident on the first member 102 from the body part 101 and travels in the direction X1, to the reflective member 122. The reflective member 122 reflects the light incident from the first optical system 121 in the direction Y1. The reflective member 122 is configured with, for example, a mirror. In the first member 102, the opening 2b is formed on the optical path of the light reflected by the reflective member 122, and the reflected light travels to the hollow portion 3A of the second member 103 by passing through the opening 2b.
The second member 103 is a member having an approximately L-shaped cross-sectional exterior, in which an opening 3a is formed at a position facing the opening 2b of the first member 102. The light that has passed through the opening 2b of the first member 102 from the body part 101 is incident into the hollow portion 3A of the second member 103 through the opening 3a. The first member 102 and the second member 103 may have any cross-sectional exterior and are not limited to the above.
The second optical system 31 includes at least one lens and guides the light incident from the first member 102 to the reflective member 32. The reflective member 32 guides the light incident from the second optical system 31 to the third optical system 33 by reflecting the light in the direction X2. The reflective member 32 is configured with, for example, a mirror. The third optical system 33 includes at least one lens and guides the light reflected by the reflective member 32 to the lens 34.
The lens 34 is arranged in an end part of the second member 103 on the direction X2 side in the form of closing the opening 3c formed in this end part. The lens 34 projects the light incident from the third optical system 33 to the projection target object 6.
The projection direction changing mechanism 104 is a rotation mechanism that rotatably connects the second member 103 to the first member 102. By the projection direction changing mechanism 104, the second member 103 is configured to be rotatable about a rotation axis (specifically, the optical axis K) that extends in the direction Y. The projection direction changing mechanism 104 is not limited to an arrangement position illustrated in
While the computer 50 is illustratively described as an example of the control device according to the embodiment of the present invention, the control device according to the embodiment of the present invention is not limited thereto. For example, the control device according to the embodiment of the present invention may be the projection apparatus 10 (or the projection apparatus 10A). In this case, each control of the computer 50 is performed by the projection apparatus 10. The projection apparatus 10 may communicate with the imaging apparatus 90 through the computer 50 or may communicate with the imaging apparatus 90 without passing through the computer 50. In a case where the projection apparatus 10 communicates with the imaging apparatus 90 without passing through the computer 50, the computer 50 may be configured to be omitted from the projection system 100.
Alternatively, the control device according to the embodiment of the present invention may be the imaging apparatus 90. In this case, each control of the computer 50 is performed by the imaging apparatus 90. The imaging apparatus 90 may communicate with the projection apparatuses 10 and 10A through the computer 50 or may communicate with the projection apparatuses 10 and 10A without passing through the computer 50. In a case where the imaging apparatus 90 communicates with the projection apparatuses 10 and 10A without passing through the computer 50, the computer 50 may be configured to be omitted from the projection system 100.
While the projection apparatuses 10 and 10A are illustratively described as an example of the plurality of projection apparatuses, the plurality of projection apparatuses may be three or more projection apparatuses.
At least the following matters are disclosed in the present specification.
(1) A control device of a projection system including one or more projection apparatuses that project a first image including a plurality of marker images, and an imaging apparatus that captures at least a part of the first image, the control device comprising a processor, in which the processor is configured to perform a control of projecting a second image including a plurality of marker images from the projection apparatus based on a capturing result of at least the part of the first image by the imaging apparatus.
(2) The control device according to (1), in which the processor is configured to perform a control of adjusting the projection of the projection apparatus based on a capturing result of the second image by the imaging apparatus.
(3) The control device according to (1) or (2), in which one or more marker images among the marker images are images associated with positions of the marker images within a projection range of the projection apparatus.
(4) The control device according to any one of (1) to (3), further comprising a correspondence table in which the marker images are associated with positions of the marker images within a projection range of the projection apparatus, in which the processor is configured to determine the positions of the marker images within the projection range of the projection apparatus based on the correspondence table.
(5) The control device according to any one of (1) to (4), in which the processor is configured to calculate an imageable range of the imaging apparatus based on the capturing result of at least the part of the first image and perform the control of projecting the second image based on the imageable range from the projection apparatus.
(6) The control device according to (5), in which the processor is configured to repeat the control of projecting the second image from the projection apparatus by generating the second image different from the first image based on the imageable range.
(7) The control device according to (6), in which the processor is configured to, in a case of repeating the control of projecting the second image from the projection apparatus, perform a control of changing a projection position of the second image.
(8) The control device according to (6) or (7), in which the processor is configured to, in a case of repeating the control of projecting the second image from the projection apparatus, perform a control of changing the second image projected from the projection apparatus.
(9) The control device according to any one of (1) to (8), in which the processor is configured to acquire an imaging condition of the imaging apparatus and perform the control of projecting the second image from the projection apparatus based on the acquired imaging condition.
(10) The control device according to (9), in which the projection apparatus includes a plurality of projection apparatuses, the imaging condition includes whether the imaging of the imaging apparatus is fixed imaging or handheld imaging, and the processor is configured to, in a case where the imaging is the fixed imaging, perform the control of projecting at least any of the first image or the second image from the plurality of projection apparatuses at different timings.
(11) The control device according to (10), in which the processor is configured to, in a case where the imaging is the handheld imaging, perform the control of projecting distinguishable images of at least any of the first image or the second image from the plurality of projection apparatuses at the same time.
(12) The control device according to any one of (9) to (11), in which the imaging condition includes a resolution of the imaging, and the processor is configured to perform the control of projecting the second image having a size corresponding to the resolution from the projection apparatus.
(13) The control device according to any one of (9) to (11), in which the imaging condition includes a resolution of the imaging, and the processor is configured to perform the control of projecting the second image including the marker images having sizes corresponding to the resolution from the projection apparatus.
(14) The control device according to any one of (1) to (13), in which the projection apparatus includes a plurality of projection apparatuses of which at least parts of projection ranges overlap, and the processor is configured to perform a control of adjusting the overlapping.
(15) The control device according to (14), in which the processor is configured to, based on a projection range of a first projection apparatus included in the plurality of projection apparatuses, perform the control of adjusting the overlapping by adjusting a projection range of a second projection apparatus that is included in the plurality of projection apparatuses and is different from the first projection apparatus.
(16) The control device according to (15), in which the processor is configured to, in a case of making the projection ranges of the plurality of projection apparatuses overlap, perform the control of adjusting the overlapping by adjusting the projection range of the second projection apparatus based on the projection range of the first projection apparatus.
(17) The control device according to (14), in which the processor is configured to perform the control of adjusting the overlapping by adjusting projection images of the plurality of projection apparatuses.
(18) The control device according to (17), in which the processor is configured to, in a case of making parts of the projection ranges of the plurality of projection apparatuses overlap, perform the control of adjusting the overlapping by adjusting the projection ranges of the plurality of projection apparatuses.
(19) A control method by a control device of a projection system including one or more projection apparatuses that project a first image including a plurality of marker images, and an imaging apparatus that captures at least a part of the first image, the control device including a processor, the control method comprising performing, by the processor, a control of projecting a second image including a plurality of marker images from the projection apparatus based on a capturing result of at least the part of the first image by the imaging apparatus.
(20) The control method according to (19), in which the processor performs a control of adjusting the projection of the projection apparatus based on a capturing result of the second image by the imaging apparatus.
(21) The control method according to (19) or (20), in which one or more marker images among the marker images are images associated with positions of the marker images within a projection range of the projection apparatus.
(22) The control method according to any one of (19) to (21), in which a correspondence table in which the marker images are associated with positions of the marker images within a projection range of the projection apparatus is further provided, and the processor determines the positions of the marker images within the projection range of the projection apparatus based on the correspondence table.
(23) The control method according to any one of (19) to (22), in which the processor calculates an imageable range of the imaging apparatus based on the capturing result of at least the part of the first image and performs the control of projecting the second image based on the imageable range from the projection apparatus.
(24) The control method according to (23), in which the processor repeats the control of projecting the second image from the projection apparatus by generating the second image different from the first image based on the imageable range.
(25) The control method according to (24), in which the processor performs, in a case of repeating the control of projecting the second image from the projection apparatus, control of changing a projection position of the second image.
(26) The control method according to (24) or (25), in which the processor performs, in a case of repeating the control of projecting the second image from the projection apparatus, a control of changing the second image projected from the projection apparatus.
(27) The control method according to any one of (19) to (26), in which the processor acquires an imaging condition of the imaging apparatus and performs the control of projecting the second image from the projection apparatus based on the acquired imaging condition.
(28) The control method according to (27), in which the projection apparatus includes a plurality of projection apparatuses, the imaging condition includes whether the imaging of the imaging apparatus is fixed imaging or handheld imaging, and the processor performs, in a case where the imaging is the fixed imaging, the control of projecting at least any of the first image or the second image from the plurality of projection apparatuses at different timings.
(29) The control method according to (28), in which the processor performs, in a case where the imaging is the handheld imaging, the control of projecting distinguishable images of at least any of the first image or the second image from the plurality of projection apparatuses at the same time.
(30) The control method according to any one of (27) to (29), in which the imaging condition includes a resolution of the imaging, and the processor performs the control of projecting the second image having a size corresponding to the resolution from the projection apparatus.
(31) The control method according to any one of (27) to (29), in which the imaging condition includes a resolution of the imaging, and the processor performs the control of projecting the second image including the marker images having sizes corresponding to the resolution from the projection apparatus.
(32) The control method according to any one of (19) to (31), in which the projection apparatus includes a plurality of projection apparatuses of which at least parts of projection ranges overlap, and the processor performs a control of adjusting the overlapping.
(33) The control method according to (32), in which the processor performs, based on a projection range of a first projection apparatus included in the plurality of projection apparatuses, the control of adjusting the overlapping by adjusting a projection range of a second projection apparatus that is included in the plurality of projection apparatuses and is different from the first projection apparatus.
(34) The control method according to (33), in which the processor performs, in a case of making the projection ranges of the plurality of projection apparatuses overlap, the control of adjusting the overlapping by adjusting the projection range of the second projection apparatus based on the projection range of the first projection apparatus.
(35) The control method according to (32), in which the processor performs the control of adjusting the overlapping by adjusting projection images of the plurality of projection apparatuses.
(36) The control method according to (35), in which the processor performs, in a case of making parts of the projection ranges of the plurality of projection apparatuses overlap, the control of adjusting the overlapping by adjusting the projection ranges of the plurality of projection apparatuses.
(37) A projection system comprising one or more projection apparatuses that project a first image including a plurality of marker images, an imaging apparatus that captures at least a part of the first image, and a control device, in which the control device includes a processor, and the processor is configured to perform a control of projecting a second image including a plurality of marker images from the projection apparatus based on a capturing result of at least the part of the first image by the imaging apparatus.
(38) A control program causing a processor of a control device of a projection system to execute a process, the projection system including one or more projection apparatuses that project a first image including a plurality of marker images, and an imaging apparatus that captures at least a part of the first image, the process comprising performing a control of projecting a second image including a plurality of marker images from the projection apparatus based on a capturing result of at least the part of the first image by the imaging apparatus.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2021-126129 | Jul 2021 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 11477422 | Masuda | Oct 2022 | B2 |
| 20080266321 | Aufranc | Oct 2008 | A1 |
| 20120050698 | Kotani | Mar 2012 | A1 |
| 20150029465 | Ishikawa et al. | Jan 2015 | A1 |
| 20180232855 | Fukui | Aug 2018 | A1 |
| 20210152796 | Boud | May 2021 | A1 |
| 20210306604 | Kubota | Sep 2021 | A1 |
| 20220264063 | Inoue | Aug 2022 | A1 |
| 20220368872 | Shishido | Nov 2022 | A1 |
| Number | Date | Country |
|---|---|---|
| 2012-47849 | Mar 2012 | JP |
| 2015-26992 | Feb 2015 | JP |
| WO2016204068 | Apr 2018 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20230031450 A1 | Feb 2023 | US |
