The present application claims priority from Japanese application JP 2017-212336, filed on Nov. 2, 2017, the contents of which is hereby incorporated by reference into this application.
The present invention relates to an installation technique of a range image camera.
Japanese Unexamined Patent Publication No. 2012-247226 is disclosed as an installation technique of a range image camera.
In Japanese Unexamined Patent Publication No. 2012-247226 describes that it provides a range image camera and a range image synthesizing method capable of acquiring a range image having a wider angle of view than an angle of view of a single photographing part and having a high range accuracy.
Japanese Unexamined Patent Publication No. 2012-247226 provides the following technique: that is, in a range image camera system provided with a plurality of range image cameras arranged in such a way that range images to be acquired overlap each other, by subjecting range images acquired by the respective camera units to processing of searching, selecting, computing, replacing, and synthesizing, there can be acquired a range image having a wider angle of view than an angle of view of a single photographing part and having a high range accuracy. However, as for an installation of the plurality of camera units, Japanese Unexamined Patent Publication No. 2012-247226 does not consider a checking operation of an installation state (position and direction) of the plurality of camera units. For this reason, there is presented such a problem that an installation operation of the plurality of camera units will require a troublesome repetitive operation including an installation state checking operation by a visual check of the range image and an adjustment operation based on a qualitative shift of the installation state acquired by the visual check.
In view of the problem described above, an object of the present invention is to provide a quantification technique of an installation state of a range image camera in a range image camera system provided with a plurality of range image cameras.
In view of the background art described above, as an example of the present invention, the present invention is a range image camera including: a light emitting part that emits an irradiation light to a photographing space; a light receiving part that receives a reflected light of the photographing space; a range image generation part that generates a range image from a time difference between a light emitting timing of the irradiation light by the light emitting part and a light receiving timing of the reflected light by the light receiving part; a luminance image generation part that generates a luminance image from an intensity of the reflected light by the light receiving part; and a control part that controls the light emitting part, the light receiving part, the range image generation part, and the luminance image generation part, wherein the control part gives at least one instruction of an image generation execution of: an instruction of a light emission execution to the light emitting part; an instruction of a light reception execution to the light receiving part; an instruction of a range image generation execution to the range image generation part; and an instruction of a luminance image generation execution to the luminance image generation part, as an image generation execution mode. Here, the control part is composed so as to have at least one mode of: an image generation execution mode for generating installation information; and a light emission execution mode for generating installation information, and so as to give at least one instruction of: an instruction of a light emission stop to the light emitting part; an instruction of a light reception execution to the light receiving part; and an instruction of a luminance image generation execution to the luminance image generation part, as the image generation execution mode for generating installation information, and so as to give at least one instruction of: an instruction of a light emission execution to the light emitting part; an instruction of a light reception stop to the light receiving part; and an instruction of a luminance image generation stop to the luminance image generation part, as the light emission execution mode for generating installation information.
According to the present invention, in a range image camera system provided with the plurality of range image cameras, it is possible to provide a quantification technique of an installation state of the range image cameras.
Embodiments of the present invention will be described with reference to the figures.
Further, the range image camera processing device 12 gives a mode instruction of an image generation execution mode that generates an image to the two range image cameras 11 (11a and 11b) as a cooperative image generation execution mode that generates a cooperative image such as a mode of synthesizing images generated by the two range image cameras 11.
Still further, as a special feature of the range image camera system in the present embodiment, the range image camera processing device 12 has two modes of a first cooperative installation information generation execution mode and a second cooperative installation information generation execution mode as a cooperative installation information generation execution mode for generating cooperative installation information, and gives a mode instruction of the image generation execution mode to the range image cameras 11 (11a, 11b) as the first cooperative installation information generation execution mode, and gives a mode instruction of an image generation execution mode for generating installation information, which generates an image for generating installation information to one of the range image cameras 11 (11a or 11b), as the second cooperative installation information generation execution mode. Here, in the first cooperative installation information generation execution mode, first installation information of the range image cameras 11 (11a, 11b) is generated from at least one image of a range image and an luminance image of the range image cameras 11 (11a, 11b) to instruct the image generation execution mode, and in the second cooperative installation information generation execution mode, second installation information of the range image cameras 11 (11a, 11b) to instruct the image generation execution mode for generating installation information is generated from the luminance image of the range image cameras 11 (11a, 11b) to instruct the image generation execution mode for generating installation information. Although the first installation information and the second installation information will be described later in detail, to put it simply, the first installation information is information of whether or not an installation position of a camera, which is generated from an image from a single range image camera, is shifted, and the second installation information is information of whether or not an installation position of a camera, which is generated from a cooperation of images from a plurality of range image cameras, is shifted.
In this regard, the image generation execution mode of the range image cameras 11, the image generation execution mode for generating installation information, the cooperative image generation execution mode of the range image camera system 1, the first cooperative installation information generation execution mode and the second cooperative installation information generation execution mode of the cooperative installation information generation execution mode will be described later in detail.
Further, as the image generation execution mode, the control part 116 gives an instruction of executing a light emission to the light emitting part 111, an instruction of executing a light reception to the light receiving part 112, an instruction of executing a range image generation to the range image generation part 113, and an instruction of executing a luminance image generation to the luminance image generation part 114.
Still further, as a special feature of the range image camera in the present embodiment, as the image generation execution mode for generating installation information, the control pat 116 gives an instruction of stopping the light emission to the light emitting part 111, an instruction of a light reception execution to the light receiving part 112, and an instruction of a luminance image generation execution to the luminance image generation part 114.
Next, an operation of the image generation execution mode of the range image camera in the present embodiment will be described.
Here, an operation flow and an operation timing are controlled by the control part 116 that receives an instruction of a mode from the range image camera cooperative processing device 12 via the communication part 115. Further, so as to show an operation timing in
In S210 (T210), a measurement of an elapsed time is started and a light emission execution is instructed to the light emitting part 111 and a light reception execution is instructed to the light receiving part 112.
In S220, it is determined whether or not the light receiving signal generated by the light receiving part 112 is shifted from an invalid state to a valid state, and in a case where the light receiving signal is shifted to the valid state, the process proceeds to S230, whereas in a case where the light receiving signal is not shifted to the valid state, the process proceeds to S240.
In S230 (T230), an elapsed time at that timing is made a delay time from a light emission timing of the irradiation light by the light emitting part 111 to a light reception timing of the reflected light by the light receiving part 112.
In S240 (T240), it is determined from the elapsed time at that timing whether or not a specified light emission execution time by the light emitting part 111 elapses, and in a case where it is determined that the specified light emission execution time elapses, the process proceeds to S250, whereas in a case where it is determined that the specified light emission execution time does not elapse, the process returns to S220.
In S250 (T250), a light emission stop is instructed to the light emitting part 111.
In S260, it is determined from the elapsed time at that timing whether or not a specified light reception execution time by the light receiving part 112 elapses, and in a case where it is determined that the specified light reception execution time elapses, the process proceeds to S270, whereas in a case where it is determined that the specified light reception execution time does not elapse, the process returns to S260.
In S270 (T270), a measurement of the elapsed time is stopped and a light reception stop is instructed to the light receiving part 112 and a range image generation execution is instructed to the range image generation part 113 and a luminance image generation execution is instructed to the luminance image generation part 114.
In S280 (T280), a range image communication execution and a luminance image communication execution are instructed to the communication part 115.
Next, an operation of the image generation execution mode for generating installation information of the range image camera in the present embodiment will be described.
Here, the image generation execution mode for generating installation information of the range image camera is used in the second cooperative installation information generation execution mode of the cooperative installation information generation execution mode as the range image camera system and generates installation information of relative installation positions of these range image cameras according to an overlapping state of the range image camera, which is an object to be instructed in the image generation execution mode for generating installation information, and the other range image camera arranged adjacently to the range image camera in the photographing space in the range image camera system. This installation information is information for urging a worker of installing the range image camera to adjust the relative installation positions of the range image cameras.
Specifically, a reflected light by an irradiation light emitted as the image generation execution mode in an adjacent range image camera is received as a reflected light of a photographing space of the range image camera to be objected and generates a luminance image from an intensity of the reflected light. A region in which the intensity is significant in this luminance image can be confirmed as an overlapping space where a photographing space of the range image camera to be objected and a photographing space of the adjacent range image camera overlap each other. In this regard, the second cooperative installation information generation execution mode of the cooperative installation information generation execution mode as the range image camera system will be described later in detail.
In other words, the image generation execution mode for generating installation information of the range image camera is different from the image generation execution mode described above in that, in the image generation execution mode for generating installation information, an operation related to a light emission and an operation related to a range image generation are not made.
In this regard,
In S310 (T310), a measurement of an elapsed time is started and a light reception execution is instructed to the light receiving part 112. Here, when compared with S210 in the operation flow chart at the time of the image generation execution mode shown in
S260 is the same as S260 in
In S370 (T370), the measurement of the elapsed time is stopped and a light reception stop is instructed to the light receiving part 112, and a luminance image generation execution is instructed to the luminance image generation part 114. Here, when compared with S270 of the operation flow chart at the time of the image generation execution mode shown in
In S380 (T380), the communication part 115 is instructed to execute a luminance image communication.
Next, an operation of the cooperative image generation execution mode of the range image camera system in the present embodiment will be described.
Here, the present operation flow chart and the present operation timing chart are controlled by the range image camera cooperative processing device 12.
In S410 (T410), the image generation execution mode is instructed to all range image cameras 11 (11a and 11b).
In this regard, in S410 (T410), each of all range image cameras 11 (11a and 11b) that receives an instruction of the image generation execution mode needs to prevent errors from being caused in a delay time and an intensity of the light receiving signal by an interference of the reflected lights by their respective irradiation lights. For this reason, each of the range image cameras 11a, 11b gives an instruction of a light reception execution to the light receiving part 112 at a specified timing in such a way that a period in which a light emission execution is instructed to the light emitting part 111 does never even partially overlap a period in which a light reception execution is instructed to the light receiving part 112. For example, a starting timing of the image generation mode operation of the range image camera 11a is shifted from a starting timing of the image generation mode operation of the range image camera 11b by a delay time of starting the image generation mode described in
In S420, it is determined whether or not the range images and the luminance images of all range image cameras 11 (11a and 11b) are received, and in a case where all images are received, the process proceeds to S430, whereas in a case where all images are not received, the process returns to S420.
In S430, a cooperative image processing of the range image and the luminance image from the respective range image cameras 11 (11a and 11b) is performed. In this regard, the cooperative image processing of the range image and the luminance image is, for example, a synthesizing processing of the range image and the luminance image but may be other processing.
Next, an operation of the cooperative installation information generation execution mode of the range image camera system in the present embodiment will be described.
In
S510 (T510) and S520 are same as S410 (T410) and S420 shown in
In S530, a generation processing of first installation information of each of the range image cameras 11 (11a and 11b) is performed from the range image from each of the range image cameras 11 (11a and 11b). In this regard, the generation processing of the first installation information of each of the range image cameras 11 (11a and 11b) in S530 will be described later in detail.
In S540 (T540), the image generation execution mode is instructed to an arbitrary range image camera 11 (for example, 11a), whereas the image generation execution mode for generating installation information is instructed to the other range image camera 11 (for example, 11b).
In this regard, in S540 (T540), a reflected light by an irradiation light emitted by the range image camera 11 (for example 11a) to instruct the image generation execution mode is received as the reflected light in the photographing space of the range image camera (for example, 11b) to instruct an image generation execution mode for generating installation information and generates a luminance image from an intensity of the reflected light. Hence, a starting timing of an operation of an image generation mode of the range image camera 11 (for example, 11a) to instruct the image generation execution mode is not shifted from, that is, is the same as a starting timing of an operation of the image generation execution mode for generating installation information of the range image camera 11 (for example, 11b) to instruct the image generation execution mode for generating installation information.
In S550, it is determined whether or not the luminance image of the range image camera 11 (for example, 11a) to instruct the image generation execution mode for generating installation information is received, and in a case where it is determined that the luminance image is received, the process proceeds to S560, whereas in a case where it is determined that the luminance image is not received, the process returns to S550.
In S560, a generation processing of second installation information of the respective range image cameras 11 (11a and 11b) is performed from the luminance image of the range image camera 11 (for example, 11b) to instruct the image generation execution mode for generating installation information. In this regard, the generation processing of the second installation information of the respective range image cameras 11 (11a and 11b) in S560 will be described later in detail.
As described above, in the range image camera system provided with the plurality of range image cameras and the range image camera cooperative processing device for cooperatively processing the plurality of range image cameras, the installation information of the range image cameras is generated from a range distribution of the range image photographed by the range cameras and a luminance distribution of the luminance image to photograph the reflected light of the irradiation light of the range image camera arranged adjacently.
In this way, in the range image camera system provided with the plurality of range image cameras, it is possible to provide a quantification technique of an installation state of the range image cameras.
The present embodiment provides an example of a specific installation state of the range image camera in a range image camera system and describes a specific image regarding an operation of a generation processing of first installation information and an operation of a generation processing in an installation state in which the range image cameras are installed as intended and in an installation state in which the range image cameras are not installed as intended.
First, an example of the installation state in which the range image camera in the range image camera system is installed as intended will be described.
Here, solid lines with arrows show X, Y, and Z axes in both of world space coordinates and camera space coordinates. Further, dotted lines and broken lines with arrows are an angle of view 12a of the range image camera 11a and an angle of view 12b of the range image camera 11b. Still further, parallelograms shown by dotted lines and broken lines are photographing planes 13a and 13b on a floor (not shown in the figure) of the range image cameras 11a and 11b. Hence, a space, which is surrounded by the dotted lines with arrows and by the broken lines with arrows and is surrounded by the parallelograms shown by the dotted lines and by the broken lines, becomes a photographing space of the range image cameras 11a and 11b (hereinafter denoted by a camera photographing space).
On the other hand, a rectangular parallelepiped shown by single dot and dash lines is a photographing space (hereinafter denoted by a system photographing space 2) of a person or the like that is intentionally caught in the range image camera system. Hence, a height in the system photographing space 2 (in a Z-axis direction) becomes, for example, a height of a finger from the floor when a standing person raises his hand.
Here, in
As shown in
As for more specific installation states of the two range image cameras 11 (11a and 11b), in the installation positions in the world space coordinates, both Z positions are identical to each other and a position of a ceiling surface (not shown in the figure), both Y positions are same specified positions, and both X positions are specified positions which are separate from each other by a specified distance. Further, as for installation directions, all of an X axis, a Y axis, and a Z axis of the world space coordinates are identical in directions to those of the camera space coordinates.
Next, an operation of a generation processing of the first installation information in the range image camera system of the present embodiment will be described.
First, an operation of a generation processing of the first installation information in a case where the range image cameras 11 (11a and 11b) in the range image camera system of the present embodiment are installed as intended will be described.
Here,
As shown in
In the S530 of the generation processing of the first installation information described above, the first installation information of the information of whether or not the range image cameras 11a and 11b are shifted from their intended positions are generated as quantitative information. In this regard, the generation of the first installation information can be realized by optics and geometry of the publicly known techniques and is not specific to the present embodiment and hence its detailed description will be omitted.
Hence, the range image camera system 1 can provide an installation worker with the range images 14a and 14b of the range image cameras 11a and 11b and the first installation information.
In this way, as for the first installation information of the range image cameras 11a and 11b, the installation worker can quantitatively get the installation state where the range image cameras 11a and 11b are installed as intended and hence can easily confirm that an installation work related to the first installation information is finished.
Next, an operation of a generation processing of the first installation information in a case where the range image camera in the range image camera system of the present embodiment is inclined from an intended installation direction will be described.
In
In this way, as for the first installation information of the range camera 11a, the installation worker can quantitatively get the installation state where the range image cameras 11a is installed in a state inclined by a certain angle in a right direction in a Y axis rotation angle, in which the Y axis is defined as an axis of rotation, from an intended installation direction and hence can easily perform an installation adjustment work related to the first installation information.
In this regard, although the installation state where the range image camera 11a is installed in the inclined state with the Y axis defined as the axis of rotation has been described in
Further, there will be described an operation of a generation processing of the first installation information in a case of an installation state where the range image camera in the range image camera system of the present embodiment is installed at a position shifted from an intended installation position.
In
In other words, in
In this way, the installation worker can quantitatively get the installation state where the range image camera 11a is shifted by a certain distance in a positive direction of the Z axis of the world coordinates from an intended installation position as the first installation information of the range image camera 11a and hence can easily perform an installation adjustment work related to the first installation information.
Next, an operation of a generation processing of second installation information in the range image camera system of the present embodiment will be described.
First, the operation of the generation processing of the second installation information in a case where the range image cameras 11 (11a and 11b) in the range image camera system of the present embodiment are installed as intended will be described.
Here,
As shown in
In the S560 of the generation processing of the second installation information described above, the second installation information of the information of whether or not the range image cameras 11a and 11b are shifted from the intended positions are generated as quantitative information. In this regard, the generation of the second installation information can be realized by optics and geometry of the publicly known techniques and is not specific to the present embodiment and hence its detailed description will be omitted.
Hence, the range image camera system 1 can provide the installation worker with the luminance images 15a and 15b of the range image cameras 11a and 11b and the second installation information.
In this way, as for the second installation information of the range image cameras 11a and 11b, the installation worker can quantitatively get the installation state where the range image cameras 11a and 11b are installed as intended and hence can easily confirm that an installation work related to the second installation information is finished.
Next, the operation of the generation processing of the second installation information in a case where the range image camera in the range image camera system of the present embodiment is inclined from an intended installation direction will be described.
In
In this way, as for the second installation information of the range camera 11a, the installation worker can quantitatively get the installation state where the range image cameras 11a is installed in a state inclined by a certain angle in the counterclockwise direction with the Z axis defined as an axis of rotation from an intended installation direction and hence can easily perform an installation adjustment work related to the second installation information.
In this regard, although the installation state where the range image camera 11a is installed in an inclined state with the Z axis defined as the axis of rotation has been described in
In this regard, the luminance image 15b generated by receiving the reflected light by the irradiation light emitted from the range image camera installed adjacently does not become a parallelogram in a case where an irradiation range of the range image camera installed adjacently is a point light source. Hence, it is difficult to detect the luminance image 15b to be rotated but, by employing a plurality of point light sources or by making the irradiation range a parallelogram by the use of a mask, it is possible to detect the luminance image 15b to be rotated.
Further, there will be described the operation of the generation processing of the second installation information in a case of an installation state where the range image camera in the range image camera system of the present embodiment is shifted from an intended installation position.
In
In this way, as for the second installation information of the range image camera 11a, the installation worker can quantitatively get the installation state where the range image camera 11a is shifted by the certain distance in the positive direction of the X axis of the world coordinates from the intended installation position and hence can easily perform an installation adjustment work related to the second installation information.
In this regard, although the installation state where the range image camera 11a is shifted in the X direction has been described in
As described above, according to the present embodiment, the installation information of the range image cameras can be quantitatively generated, the installation worker can easily check an adjustment content of the installation work and that an adjustment work is finished.
In the present embodiment, a modification of a configuration described in the embodiments 1, 2 will be described.
First, as a configuration of the range image camera, the present embodiment may have a 3D image generation part that generates a 3D image from a range image generated by the range image generation part, and the control part may control the 3D image generation part and may instruct the 3D image generation part to execute a generation of a 3D image or to stop the generation of the 3D image in conjunction with an instruction of a generation execution or a generation stop of the range image to the range image generation part in the image generation execution mode.
Further, an original image used for generating the first installation information generated in the first cooperative installation information generation execution mode may be the 3D image or the luminance image in place of the range image.
Still further, as the configuration of the range image camera, the present embodiment may have at least one of: a first installation information generation part that generates the first installation information of the range image camera from at least one of the range image, the 3D image, and the luminance image; and a second installation information generation part that generates the second installation information of the range image camera from the luminance image, and the control part may control at least one of the first installation information generation part and the second installation information generation part, and the control part may have at least one of an image generation execution mode for generating installation information, a first installation information generation execution mode, and a second installation information generation execution mode as an installation information generation execution mode; and the control part may generate the first installation information of the range image camera from at least one image of the range image, the 3D image, and the luminance image as the first installation information generation execution mode and may generate the second installation information of the range image camera from the luminance image of the range image camera as the second installation information generation execution mode.
Still further, a mode instruction to an arbitrary range camera in a second cooperative installation information generation execution mode may be a light emission execution mode for generating installation information that gives at least one instruction of: an instruction of a light emission execution to the light emitting part; an instruction of a light emission stop to the light receiving part; and an instruction of a luminance image generation stop to the luminance image generating part, in place of the image generation execution mode.
Still further, at least installation information of the first installation information and the second installation information, which are generated by the range image camera cooperative processing device 12, may be generated by the range image camera and may be communicated with an exterior portion.
Up to this point, the embodiments have been described, and the present invention is not limited to the embodiments described above but includes various kinds of modifications. In other words, the above-mentioned embodiments have been described in detail so as to explain the present invention clearly and the present invention is not necessarily limited to an embodiment including all compositions described above. Further, a portion of a composition of a certain embodiment can be replaced by a composition of the other embodiment, and a composition of the other embodiment can be added to a composition of the certain embodiment. Still further, a composition of each embodiment can be added to, removed from, or replaced by a portion of a composition of the other embodiment. Still further, a portion or all of each composition, each function, each processing part, and each processing means described above may be realized, for example, by hardware designed by means of an integrated circuit or the like. Still further, each composition, each function, and the like may be realized by means of software in which a processor interprets and executes a program for realizing each function.
Number | Date | Country | Kind |
---|---|---|---|
2017-212336 | Nov 2017 | JP | national |