The present disclosure relates to a 3D depth camera, computer vision, depth perception, and a three-dimensional rebuilding technology, and specifically relates to a method and apparatus for coding structured light that is projected in an active stereovision for marking a space, and more particularly to a method and an apparatus generating a random coded pattern for coding structured light.
Vision is the most direct and effective approach for mankind to perceive external environment. The purpose of studying computer vision is to expect that computers can perceive various states (e.g., color, shape, surface textual information of an object, environmental position of the object, and movement of the object, etc.) in an environment in a way like what the human vision does. However, in application fields of computer vision, the information that may be easily parsed out by mankind and animals from one or more images possibly goes wrong with the computer vision. One technology of capturing distance information in a scene is referred to as a depth perception technology, which may facilitate a computer to better perceive a surrounding environment and plays a critical role in fields like virtual reality, three-dimensional rebuilding, human-machine interaction, industrial automation, robot navigation, and medical imaging. For example, in the field of consumer electronics, the depth perception technology can facilitate an electronic product to recognize different actions of human beings to thereby make corresponding reactions, which brings a fresh, convenient and smart experience to users and thus plays a significant role in enhancing the interaction capability and smart level of the electronic product. In the industrial field, a high-precision and high-resolution depth information technology is urgently needed in sectors such as 3D printing, robot manipulation, and etc.
The depth perception technologies may be categorized into an active type and a passive type based on whether an active light source exists in a depth perception device. The active-type depth perception technology captures a pattern projected by a projector on a spatial object using an image sensor and then obtains spatial position information of the object by analyzing and processing the pattern (e.g., the approach of structured light coding, the approach of ToF (Time of Flight)). The active-type depth perception technology is characterized in that the depth information is stable, reliable, and less affected by ambient light; besides, its matching process is not affected by textual information of the object. One of the core content of a structured light coding-based depth perception method is to develop a coded pattern projecting device, and the coded pattern as designed will directly affect the computational complexity of depth decoding as well as the precision and spatial resolution of the depth information; besides, it also has a relatively large impact on the ant-noise and anti-distortion capability during the decoding process as well as the depth decoding capability in a complex scene.
A method and an apparatus for generating a random coded pattern for coding a structured light provided by the present disclosure at least can solve part of the above problems, which facilitates depth recognition by generating a random coded pattern satisfying a requirement of performing high-performance depth decoding in a complex scene, the random coded pattern being projected by a projecting device on a three-dimensional space or a target object for spatial coding and feature calibration.
A method of generating a random coded pattern for coding structured light comprises steps of:
S100. determining a resolution of a random coded pattern to obtain a random probability distribution map of a same resolution size;
S200. obtaining a position of a point with the largest probability value in the random probability distribution map;
S300. marking the position as a central point position of a coding primitive in the random coded pattern and the random probability distribution map;
S400. marking coding primitives in the random probability distribution map according to a shape and size of the coding primitive;
marking points that cannot be other coding primitives surrounding the coding primitives in the random probability distribution map according to a distribution rule of the coding primitives;
S500. determining whether all points in the random probability distribution map are completely marked; if not yet, performing step S600;
S600. obtaining a position of a point with a largest probability value from among unmarked points in the random probability distribution map, and returning to step S300.
For the method aforementioned, if it is determined that all points in the random probability distribution map have been completely marked in the step S500, the following steps are performed:
S700. determining whether a structured light coded pattern satisfies a requirement of window uniqueness distribution, the structured light coded pattern being constituted by random coded patterns; in the case of not satisfying the requirement, performing step S800;
S800. determining a re-marked zone and re-generating a random probability distribution map that has a same size as the re-marked zone; and returning to step S200.
For the aforementioned or the following method, a shape of the coding primitive includes a feature point, a square, a circle, an oblique strip, an S shape, or any other shape consisting of a plurality of feature points.
For the aforementioned or the following method, the step S700 further comprises a step before the determining:
S701. splicing and expanding the random coded pattern as a basic image element.
For the aforementioned or the following method, the splicing and expanding includes a regular array manner, a staggered array manner, and an array rotating manner.
For the aforementioned or the following method, probability values of points in the random probability distribution map range between (0, 1).
For the aforementioned or the following method, in the steps S300-S600, a probability mark map instead of the random probability distribution map is used to mark; a resolution size of the probability mark map is identical to the random probability distribution map, where an initial value of each point is identical and not 0; when marking a point in the probability mark map, a value of the point is made 0;
Moreover, the step S600 may be replaced by the following step: S600′: replacing the random probability distribution map with a resultant map from point-to-point multiplication of the probability mark map and the random probability distribution map marked in step S400, and then returning to step S200.
For the aforementioned or the following method, after marking the random coded pattern in the step S300, the method further comprises a step of:
marking coding primitives in the random coded pattern using the marked point according to a shape and size of the coding primitive.
For the aforementioned method, a distribution rule of the coding primitives includes one or a combination of the following rules: satisfying an isolation principle, and satisfying a spacing requirement between coding primitives.
In another aspect, the present disclosure provides an apparatus for generating a random coded pattern for coding structured light, the apparatus comprising a pattern generating module and a projecting module;
the pattern generating module generates a random coded pattern using any method aforementioned; and the projecting module determines a corresponding light-emitting source according to a coding primitive in the random coded pattern.
Compared with the prior art:
the method and apparatus in the present disclosure may quickly and accurately generate a random coded pattern satisfying a requirement of performing high-performance depth decoding in a complex scene; and the coding primitives constituting the random coded pattern have a good stochastic characteristic while satisfy a window unique identification characteristic, i.e., when the random coded pattern is projected, by the projecting module, on a three-dimensional space or a target object according to a certain angle of view, each feature point in the projected three-dimensional space or on a surface of the projected target object may be uniquely identified within a certain range, wherein each coding primitive corresponds to one or more feature points; in this way, the identification issues of monocular, binocular or multi-view matching during a depth decoding process may be solved.
Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.
In an embodiment, a random coded pattern may be generated using a schematic flow diagram of
S100. determining a resolution of a random coded pattern to obtain a random probability distribution map of a same resolution size;
S200. obtaining a position of a point with the largest probability value in the random probability distribution map;
S300. marking the position as a central point position of a coding primitive in the random coded pattern and the random probability distribution map;
S400. marking coding primitives in the random probability distribution map according to a shape and size of the coding primitive;
marking points that cannot be other coding primitives surrounding the coding primitives in the random probability distribution map according to a distribution rule of the coding primitives;
S500. determining whether all points in the random probability distribution map are completely marked; if not yet, performing step S600;
S600. obtaining a position of a point with a largest probability value from among unmarked points in the random probability distribution map, and returning to step S300.
S700. determining whether a structured light coded pattern satisfies a requirement of window uniqueness distribution, the structured light coded pattern being constituted by random coded patterns; in the case of not satisfying the requirement, performing step S800;
S800. determining a re-marked zone and re-generating a random probability distribution map that has a same size as the re-marked zone; and returning to step S200.
In this embodiment, the random probability distribution map has a same resolution as the random coded pattern, and positions of points in the random probability distribution map are identical to the positions of points in the random coded pattern, a value of a point in the random probability distribution map indicating an appearing probability of a central point of the coding primitive at the position of the point. Values of the points in the random probability distribution map are random numbers generated by a random array generating function inherent in a programming software or generated by a customized random array generator, which may be any decimals ranging between 0-1. Central points of the coding primitives in the random coded pattern are determined according to random numbers in the random probability distribution map. In order to guarantee uniqueness of the position of the coding primitive generated in each selection, values of the points in the random probability distribution map are different from one another. For the values of points in each map, specifically, for example, the initial values of the points are all 0, and the initial values of the points in the random probability distribution map correspond to random numbers ranging from 0-1. The values of the points in each map are used for marking, such that they may not be limited to what are listed above. For the random coded pattern whose initial values are all 0, in order to find corresponding positions during marking, the pixel values of the positions are changed from 0 to 1.
The resolution of the randomly encoded pattern may be set dependent on the requirements of making the coded pattern projecting module and a ranging scope of a 3D depth camera, where the initial value of each pixel is uniformly 0. The coded pattern projecting module is fabricated in association with the shape, size, number, and distribution restraining rule of the coding primitives. In the random coded pattern, when the coding primitives are distributed in identical numbers, the larger the pixel size representing a coding primitive, the greater the distance between coding primitives is, and therefore the larger the resolution of the designed random coded pattern is. The ranging scope of the 3D depth camera also affects the resolution size of the coded pattern, and meanwhile it is required that the designed random coded pattern must guarantee a minimum resolution size of window uniqueness. For example, if a searching scope required by the ranging scope is 20 pixels respectively thereabove and thereunder, and 200 pixels respectively to the left and the right, the resolution of the randomly encoded pattern should not be lower than 41×401, and in the scope, a window of a certain size consisting of coding primitives must satisfy the uniqueness requirement.
The number of coding primitives may be plural in a random coded pattern; by controlling the distribution rule of the coding primitives, the number of coding primitives may be adjusted and then the density of coding primitives may be controlled. When the number of coding primitives is plural, after the coding primitives are generated at step S400, points that surround the marked coding primitive but cannot act as other coding primitives are marked in the random probability distribution map; these points may highlight the coding primitive shape. The coding primitives may be of any shape consisting of a plurality of pixels, such as a dot, a line segment, a circle, a rectangle, an oblique strip, a parallelogram, a diamond, a trapezoid, a triangle, an S shape, etc.
The structured light coded pattern may be a single image generated according to the method above, or may be spliced and expanded with the single pattern generated above as a basic image element according to a certain rule. The basic image element refers to a random coded pattern containing a certain number of coding primitives with a relatively low resolution; due to the limitations of manufacturing techniques and volume restraints of the projecting module, only a smaller resolution can be adopted. A high-resolution random coded pattern is expanded by splicing the basic image elements. The splicing and expanding method of basic image elements includes a regular array manner, a staggered array manner, and an array rotating manner, etc.; the size of the expanded random coded pattern may be voluntarily set. In the regular array manner, the basic image elements are regularly arranged according to horizontal and vertical directions, which are spliced into a randomly coded pattern with K×L blocks, where K and L are positive integers, which may be identical or different. The staggered array manner, as illustrated in
In the three-dimensional depth perception technology based on structured light coding, the output structured light coded pattern has to satisfy the window uniqueness, i.e., the characteristic requirement of window uniqueness identification: in a certain search scope of the structured light coded pattern (e.g., obtaining a coded pattern block with a range of r×v arrays, where r and v are positive integers), if the coded pattern block consists of k1×k2 matrixes (which may include a plurality of coding primitives, where k1 and k2 are both positive integers), the coded pattern block is unique if the coded pattern block only appears once, the coded pattern block is unique so as to be distinguished from other coded pattern blocks of the same size. The size of the window is k1×k2, i.e., the size of the coded pattern block. The present disclosure has no limitation regarding the size of the coded pattern block, i.e., the size of the coded pattern block is adjustable. The larger the window, the easier the window uniqueness is satisfied; the smaller the window, the more detailed the obtained depth map is.
Based on the window uniqueness requirement above, the compulsory precondition for outputting the structured light coded pattern (regardless of whether it consists of a single random coded pattern or it is spliced and expanded from a plurality of random coded patterns) is that the requirement of window uniqueness distribution should be satisfied. For a generated random coded pattern, if it cannot satisfy the window uniqueness distribution, the random coded pattern may be re-generated locally or entirely so as to cause it to satisfy the window uniqueness requirement. Here, the “locally” may refer to one local part or several local parts; the one or more local parts may be artificially designated or determined randomly. The present disclosure does not limit the size, shape, and position of the re-marked zone. Therefore, the present disclosure is not limited to the aforementioned implementation steps. Any modification and improvement within the spirit and scope of the present disclosure should be included in the scope of the claims.
For a random coded pattern with the initial values being all 0, pixel value 1 in the outputted random coded pattern corresponds to a central point of respective coding primitive. In order to obtain a complete random coded pattern, after the random coded pattern is marked in step S300, the following step may be executed: marking coding primitives in the random coded pattern using the marked points according to the shape and size of coding primitives. In this way, the random coded pattern finally outputted in step S700 has complete coding primitives, and the coding primitives have a certain distribution rule; the pattern may be applied to encode the structured light through a projecting module.
In one embodiment, for the steps S300, S400, and S500, it is the probability mark map instead of the random probability distribution that is marked. The resolution of the probability mark map is identical to the random probability map, the initial value of each point being identical and not 0. Upon marking points in the probability mark map, the values of the points are made 0. For generating a structured light random coded pattern with a resolution size of 3×10,
In the case of marking using the probability mark map, steps S300-S600 are changed to:
S300′. marking the position as a central point position of a coding primitive in the random coded pattern and the random probability distribution map, and changing the probability mark value corresponding to the position from 1 to 0.
S400′. marking coding primitives in the random probability distribution map according to a shape and size of the coding primitive; and meanwhile marking points that cannot be other coding primitives surrounding the coding primitives in the random probability distribution map according to a distribution rule of the coding primitives;
S500′. determining whether all points in the random probability distribution map are completely marked; if not yet, performing step S600;
Correspondingly, the step S600 in the step S500′ actually refers to the step S600′ below:
S600′: point-to-point multiplying the random probability distribution map with the marked probability mark map in step S400′ and replacing the random probability distribution map with the resultant map, and then returning to step S200. Through the point-to-point multiplication, positions where coding primitives will not be generated any more are excluded.
In another embodiment, an apparatus for generating a random coded pattern for coding structured light is provided to generate the random coded pattern and project the generated random coded pattern, the apparatus comprises a random coded pattern using any method above; the projecting module determines corresponding light-emitting sources according to the coding primitives in the structured light random coded pattern, wherein the coding primitives may one-to-one correspond to the light-emitting sources. The projecting module projects the random coded pattern onto a three-dimensional space or a target object to implement feature marking of the three-dimensional space or the target object; each feature point on the surface of the projected three-dimensional space or target object may be uniquely identified within a certain range, wherein each coding primitive corresponds to one or more feature points on the three-dimensional space or the target object, which facilitates depth identification. The projecting module is not limited to a laser projecting module, an LED/LCD projector, a DLP, an MEMS projector, as well as other stationary or mobile projection devices. The supported projection pattern optical waves include optical waves of various wave lengths, e.g., an X-ray, an ultraviolet ray, a visible light, and an infrared ray, etc. Its light-emitting sources may be various kinds of laser sources such as VECSEL, LD, LED, and etc.
The present disclosure is not limited to the preceding embodiments. Therefore, modifications and improvements without departing from the spirit and scope of the present disclosure should be included in the scope of the claims.
Number | Date | Country | Kind |
---|---|---|---|
2017104058683 | Jun 2017 | CN | national |