Patent Application
20180231378
This application claims priority to and the benefit of Korean Patent Application No. 10-2017-0018854 filed in the Korean Intellectual Property Office on Feb. 10, 2017, the entire contents of which are incorporated herein by reference.
The present invention relates to an apparatus for obtaining depth information using a digital micro-mirror device, and a method thereof.
Depth information is useful information that is utilized in various application fields such as a 3D movie, a 3D TV, 3D graphics, holography, an autonomous driving unmanned vehicle, a guided weapon, a robot visual device, factory automation, user interaction, a game, and the like.
A method for obtaining depth information can be broadly classified into a direct measurement method and an indirect measurement method. The direct measurement method includes a calculation method using triangulation and a time-of-flight measurement method. The indirect measurement method includes a method using focusing information, a shadow analysis method using lighting, and a moiré method. The most accurate and useful method is the direct measurement method. A stereoscopic method is a passive method of the direct measurement method, and a structured light illumination method is an active method of the direction measurement method. The stereoscopic method finds a corresponding point using at least two sheets of images, calculates binocular disparity from the corresponding point, and calculates a depth by triangulation method. However, the passive method has a drawback of imperfection in passive selection of a corresponding point.
The structured light illumination method uses a light pattern having a predetermined structure in order to overcome such an imperfection in the passive selection of the corresponding point. Such a method mostly uses a straight-line type of light pattern, and an image is obtained through the illumination light pattern, and then an illumination angle of a light source and a main angle viewed by each pixel of a camera are applied to the triangulation method such that depth information can be obtained.
As the structured light illumination method, an illumination method that scans light in the form of a vertical stripe from left to right or from right to left is mostly used. In such a case, a polygon mirror or a rotation mirror that rotates a mirror mounted to a vertical axis or a horizontal axis or a Galvano mirror is used as a mechanical device in the structured light illumination method.
However, such a conventional method has problems of high volume, generation of noise during rotation of the mirror, and limitation in high-speed scanning. In addition, an optical axis of the rotation mirror cannot be easily aligned, and an error may occur. The thickness of a projected optical scan line may be changed or intensity of reflected light is changed depending on depth so that image may not be clear when reflected light is obtained through a camera.
In addition, when scanned laser light is reflected at the surface of the object, a camera for obtaining an image of a laser beam or a dedicated sensor array is required. In this case, a large amount of image data to be processed is required and when a dedicated sensor, which is separately manufactured for distance information processing, is used, production cost is increased.
The above information disclosed in this Background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art.
The present invention has been made in an effort to provide an apparatus for obtaining depth information by using a digital micro-mirror device (DMD), and a method thereof.
According to an exemplary embodiment of the present invention, a depth information acquiring apparatus can be provided. The depth information obtaining apparatus includes: a first digital micro-mirror device (DMD) that generates a first line light and irradiates the first line light to an object; a second DMD that receives a second line light reflected from the object and reflects light corresponding to the second line light; a sensor that senses light reflected by the second DMD; and a controller that controls operations of the first and second DMDs, and calculates depth information of the object by using information with respect to light sensed by the sensor.
The first DMD may include a plurality of element mirrors, and element mirrors arranged in one column among the plurality of element mirrors may be simultaneously driven such that the first line light is generated.
The second DMD may include a plurality of element mirrors, and the plurality of element mirrors may be driven one by one to reflect light corresponding to the second line light to the sensor.
For a first period during which the first line light is irradiated to the object, all element mirrors of the second DMD may be driven one by one to reflect light corresponding to the second line light to the sensor.
The controller may calculate the depth information by using an irradiation angle of the first line light to the object and a location at which the second line light is received at the second DMD.
The second DMD may include a plurality of element mirrors arranged in n columns and m rows, and the controller may sequentially drive the element mirrors of a predetermined column that is smaller than an m-th column and then may detect all light from n element mirrors, and may omit driving of other columns excluding the predetermined column.
The second DMD may include a plurality of element mirrors, and when light is detected from one of the plurality of element mirrors while sequentially driving element mirrors arranged in the first row, the controller may omit driving of the rest of the element mirrors in the first row and may sequentially drive element mirrors arranged in the second row.
The depth information obtaining apparatus may further include an optical system that is disposed between the first DMD and the second DMD and focuses the first line light to the second DMD.
The depth information obtaining apparatus may further include an optical system that is disposed between the second DMD and the sensor and focuses light reflected by the second DMD to the sensor.
According to another exemplary embodiment of the present invention, a method for obtaining depth information of an object by irradiating a first line light to the object may be provided. The method includes: generating the first line light and irradiating the first line light to the object by using the first digital micro-mirror device (DMD); reflecting a second line light reflected from the object by using the second DMD; detecting a location where the second line light is reflected in the second DMD; and calculating the depth information by using the detected location.
The detecting the location may include sensing light reflected by the second DMD; and detecting the location by using a driving time of the second DMD and the sensed light.
The first DMD may include a plurality of element mirrors, and the irradiating the first line light may include generating the first line light by simultaneously driving element mirrors arranged in one column among the plurality of element mirrors and irradiating the first line light to the object.
The second DMD may include a plurality of element mirrors, and the reflecting the second line light may include reflecting light corresponding to the second line light by driving the plurality of element mirrors one by one.
For a first period during which the first line light is irradiated to the object, all element mirrors of the second DMD may be driven one by one to reflect light corresponding to the second line light.
The calculating the depth information may calculate the depth information by using an angle at which the first line light is irradiated to the object and the detected location.
According to still another exemplary embodiment of a depth information obtaining apparatus can be provided. The depth information obtaining apparatus includes: a first digital micro-mirror device (DMD) that includes a plurality of first element mirrors and transmits a first line light to an object; a second DMD that includes a plurality of second element mirrors, receives a second line light reflected from the object, and reflects light corresponding to the second line light; a sensor that senses light reflected by the second DMD; and a controller that controls the first line light to be generated by simultaneously driving element mirrors arranged in one column among the plurality of first element mirrors for a first period, and controls light corresponding to the second line light to be reflected to the sensor by driving the plurality of second element mirrors one by one, wherein the controller calculates depth information of the object by using information with respect to light sensed by the sensor
The controller may calculate the depth information by using an irradiation angle of the first line light to the object and a location at which the second line light is received at the plurality of second element mirrors.
According to the exemplary embodiments of the present invention, since a DMD is used as a means to generate light line light rather than using a galvano mirror, high mechanical stability can be provided and no rotation noise is generated.
According to the exemplary embodiments of the present invention, a DMD is used as a means for receiving reflection light so that a certain type of reflected light can be obtained.
According to the exemplary embodiments of the present invention, since the DMD is used, the entire size of the depth information obtaining apparatus can be reduced and depth information can be stably obtained.
In addition, according to the exemplary embodiments of the present invention, the depth information can be obtained by using two DMDs and one simple sensor, thereby simply acquiring depth information.
In the following detailed description, only certain exemplary embodiments of the present invention have been shown and described, simply by way of illustration. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive. Like reference numerals designate like elements throughout the specification.
Throughout this specification and the claims that follow, when it is described that an element is “coupled” to another element, the element may be “directly coupled” to the other element or “electrically coupled” to the other element through a third element. In addition, unless explicitly described to the contrary, the word “comprise” and variations such as “comprises” or “comprising” will be understood to imply the inclusion of stated elements but not the exclusion of any other elements.
Hereinafter, a depth information obtaining apparatus using a digital micro-mirror device, and a method thereof according to an exemplary embodiment of the present invention will be described.
As shown in
The transmitting DMD 110 is controlled by the controller 130 and sequentially irradiates (transmits) line light to an object of which a depth needs to be measured. The transmitting DMD 110 is formed of a digital micro-mirror device (DMD).
The DMD is a device of which micro-mirrors are arranged using a semiconductor process and image information of a pixel is controlled by adjusting an angle of each element mirror, and has merits of high contrast, rapid driving speed, inexpensive cost, and the like. One of the plurality of element mirrors arranged in the DMD may have the following three states: flat, on, and off. When no power is applied, the element mirror is in a flat state. An element mirror corresponding to a pixel at a position to be modulated is electrically inclined into a state of (+/−)Ω°. Cases that the element mirror is inclined in the state of (+/−)Ω° respectively correspond to an on state and an off state. The on/off state of each element mirror is programmed such that line light can be generated.
In the exemplary embodiment of the present invention, light from the laser surface light source is wholly irradiated to the transmitting DMD 110, and transmitting DMDs 110 are sequentially turned on per each column such that line light is generated. In this case, the transmitting DMDs 110 are sequentially turned on per column by control of the controller 130.
As shown in
The receiving DMD 120 is controlled by the controller 130, and receives line light reflected from the object and reflects the received line light to the sensor 140. The receiving DMD 120 is also formed of DMDs. If the line light generated in the transmitting DMD 110 is reflected to the object, the line light becomes bent line light depending on surface roughness of the object. Here, the bent line light implies a depth change from a 3-dimensional reference point, and includes depth information. The receiving DMD 120 reflects light corresponding to such bent line light to the sensor 140. In this case, the receiving DMD 120 turns on element mirrors one by one and each turned-on element mirror reflects light to the sensor 140.
As shown in
As shown in
During t2, the receiving DMD 120 is sequentially driven from the first row to the n-th row and thus the respective element mirrors are sequentially turned on. In this case, when light exists in each element mirror, the light is reflected to be transmitted to the sensor 140. During t2, line light is generated from the second column of the transmitting DMD 110, and the second line light is reflected to the object and is thus incident on the receiving DMD 120. In this case, during t2, the respective element mirrors of the receiving DMD 120 are sequentially turned on and light corresponding to bent line light is reflected to the sensor 140.
As described, the receiving DMD 120 according to the exemplary embodiment of the present invention sequentially turns on the respective element mirrors so as to detect bent line light.
The sensor 140 senses light reflected by the receiving DMD 120. The sensor 140 may be implemented as a sensor that senses light, such as a photodiode. When sensing light, the sensor 140 transmits data of the sensed light to the controller 130. That is, the sensor 140 according to the exemplary embodiment of the present invention receives light corresponding to bent line light from the receiving DMD 120 and senses the received light, and transmits data of the sensed light to the controller 130. The controller 130 obtains the location of the receiving element mirror of the receiving DMD 120 by checking the counter number of the operation pulse using the sensed light data.
Meanwhile, the controller 130 controls operation of the transmitting DMD 110 and operation of the receiving DMD 120. The controller 130 controls sequential driving per column of the transmitting DMD 110, and controls sequential driving of the receiving DMD 120. That is, the controller 130 performs a control operation to control the transmitting DMD 110 and the receiving DMD 120 to operate as shown in
In addition, the controller 130 according to the exemplary embodiment of the present invention receives light data from the sensor 140 and calculates depth information by using the received light data. As described above, the bent line light reflected to the object includes depth information, and the controller 130 calculates depth information by using the light data received from the sensor 140. A depth information calculation method of the controller 130 will be described in detail with reference to
Meanwhile, as shown in
The first optical system 150 is provided between the object and the receiving DMD 120, and serves to focus bent line light reflected to the object to the receiving DMD 120. In addition, the first optical system 150 also serves to remove noise. The first optical system 150 may be provided as a convex lens and the like.
The second optical system 160 is disposed between the receiving DMD 120 and the sensor 140, and serves to focus light reflected from the receiving DMD 120 to the sensor 140. In addition, the second optical system 160 also serves to remove noise. The second optical system 160 may be provided as a convex lens and the like. In addition, a projection optical system composed of a convex lens or the like may be positioned between the transmitting DMD 110 and the object so that the transmitting DMD 110 can clearly illuminate the object with the line light.
In
When a trigonometric function is applied, tamp is as given in Equation 1.
tan ψ=u/f (Equation 1)
In addition, in
X1=z·tan(90−θ) (Equation 2)
X2=z·tan (ψ) (Equation 3)
The reference line b is as given in Equation 4.
b=X1+X2 (Equation 4)
When Equation 2 and Equation 3 are put to Equation 4, Equation 5 can be given as follows.
b=z·tan(90−θ)+z·tan (ψ) (Equation 5)
In Equation 5, when Equation 1 is inserted instead of tamp and z is solved, z can be calculated as given in Equation 6.
z=b·f/(u+f cot θ) (Equation 6)
In Equation 6, z is depth information that the controller 130 needs to calculate. In Equation 6, b, f, and θ are values that are predetermined in a setup stage of the depth information obtaining apparatus 100. Thus, the controller 130 calculates depth information only when u that corresponds to the bent line light is obtained in Equation 6.
The controller 130 can obtain u by using light data received from the sensor 140. Since the controller 130 controls the receiving DMD 120, the controller 130 is aware of on-timing of the respective element mirrors of the receiving DMD 120. Accordingly, the controller 130 determines which element mirror receives line light through the timing of the light data received from the sensor 140 and accordingly obtains the value of u.
The method for the controller 130 to calculate the value u will be described in further detail with reference to
As shown in
As shown in
As shown in
As described above, the depth information obtaining apparatus according to the exemplary embodiment of the present invention uses a DMD rather than using a galvanometer mirror as means for generating line light, and therefore no rotation noise is generated. The depth information obtaining apparatus according to the exemplary embodiment of the present invention uses a DMD as a means for receiving reflection light so that a certain type of reflected light can be obtained. Meanwhile, according to the exemplary embodiment of the present invention, since the DMD is used, the entire size of the depth information obtaining apparatus can be reduced and depth information can be stably obtained.
In addition, in the exemplary embodiment of the present invention, the depth information can be obtained by using two DMDs and one simple sensor, thereby saving cost.
While this invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2017-0018854 | Feb 2017 | KR | national |
