Patent Application
20190061152
The present invention relates to a measuring method, a program, a measuring apparatus, and a method of manufacturing an article.
Robots having gripping parts configured to grip objects are now performing complex tasks (such as the assembly of industrial products) which have been done by humans. Gripping parts are controlled on the basis of the measurement results of measuring apparatuses configured to measure the arrangement of objects (for example, positions and postures).
Sizes or materials of objects handled by robots are various. Generally, there is a distribution of machining accuracy in each of a plurality of constituent elements constituting an object. In addition, there is a case in which a position or a posture of an object is not determined before and after a gripping part grips the object or a case in which the entire object does not fit into an image used by a measuring apparatus in some cases.
There is a measuring apparatus which acquires the disposition of an object with high accuracy by a two-stage measurement method of roughly acquiring the disposition thereof by processing an image of an object and accurately acquiring the disposition thereof by further processing the image on the basis of the acquired disposition. For example, Japanese Patent Laid-Open No. 2011-22991 includes detection of a specimen and calculation of a rough position and posture by performing voting using a pre-learned classification tree for an acquired image. Japanese Patent Laid-Open No. 2011-27623 includes calculation of a position and posture of a workpiece with high accuracy by correcting the position and posture so that a three-dimensional shape model of the workpiece and an acquired image fit. There is an apparatus which acquires the disposition of an object with high accuracy by performing the calculation in Japanese Patent Laid-Open No. 2011-22991 and performing the calculation in Japanese Patent Laid-Open No. 2011-27623 on the result.
In a measuring apparatus for performing two-stage image processing, it is important to combine an object portion in an image processed at the time of roughly acquiring the disposition and an object portion used at the time of accurately acquiring the disposition. For example, when most of the object portion in the image processed at the time of roughly acquiring the disposition is photographed in the image, it is assumed that an approximate disposition has been calculated. However, when the object portion used at the time of accurately acquiring the disposition is far away from the object portion in the image processed at the time of roughly acquiring the disposition and the object portion used at the time of accurately acquiring the disposition is not photographed in the image, the disposition cannot be estimated with high accuracy.
In existing image processing apparatuses including the image processing apparatus of Japanese Patent Laid-Open No. 2015-199155, separate region images and processing means are in one-to-one correspondence. Therefore, even when the image processing apparatus of Japanese Patent Laid-Open No. 2011-22991 is used in a measuring apparatus, a combination of a part which is subjected to rough processing and a part which is subjected to accurate processing cannot be flexibly set. Considering the situation described above, the fact that only a part used at the time of acquiring one accurate disposition is registered for a part used at the time of acquiring one rough disposition leads to inconvenience.
The present invention proposes a measuring apparatus which is advantageous in terms of measurement accuracy.
According to an aspect of the invention, a measuring method measures a position and a posture of an object. The method comprises: a first calculation step of processing a first image obtained by imaging the object as a first processing object; and a second calculation step of determining a position and a posture of the object by processing a plurality of constituent elements in the image as a second processing object on the basis of the result of the first calculation step. The second processing object is associated with the first processing object.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Embodiments of the present invention will be described below with reference to the drawings or the like.
(Measuring Apparatus)
The image acquisition unit 110 captures an image of an object W serving as an object to be inspected. The image captured by the image acquisition unit 110 is sent to an image storage unit 121 provided inside the processing unit 120. The image acquisition unit 110 includes, for example, a projection unit (not shown) and an imaging unit (not shown). The projection unit projects pattern light onto the object W. It should be noted that the projection unit may project pattern light onto a part of the object W or may project uniform light onto the object W instead of pattern light. When pattern light is projected, the projection unit includes a pattern generation unit (not shown) and a projection optical system (not shown).
The pattern generation unit generates pattern light projected using the projection optical system. Examples of pattern light include a periodic line pattern (stripe pattern) in which bright portions formed by bright lines and dark portions formed by dark lines are alternately arranged. The imaging unit images an object onto which light is projected.
The imaging unit images the object W within a field of view range 111 in which light is projected. The imaging unit includes an imaging optical system (not shown) and an imaging element (not shown). The imaging unit acquires an image by receiving light reflected from the object using the imaging element via the imaging optical system. The imaging element may use an optical sensor such as a complementary metal-oxide semiconductor (CMOS) or a charge coupled device (CCD). The imaging element may be an imaging element with a color filter or a monochrome imaging element.
The processing unit 120 realizes processing of a main algorithm of the measuring apparatus 100 according to this embodiment using a computer and an electrical circuit. The processing unit 120 performs a process of acquiring an accurate position and posture (inclination or aspect) of the object W from an image captured by the imaging unit. The processing unit 120 includes the image storage unit 121, a calculation unit 122, and a storage unit 123.
The image storage unit 121 stores an image of the object W captured by the image acquisition unit 110. The storage unit 123 stores settings information concerning calculation which will be described later. A program causing a computer to execute the measuring method according to the embodiment may be stored in the storage unit 123. The calculation unit 122 acquires a position and a posture of the object W using the settings information stored in the storage unit 123 with respect to the image stored in the image storage unit 121.
In the embodiment, the calculation unit 122 processes an image using calculations of two types including a first calculation and a second calculation to acquire the position and the posture of the object W. In the embodiment, the first calculation includes estimating an approximate position and posture of the object W The second calculation includes acquiring a specific position and posture of the object W using the result (processing result) of the first calculation. In addition to this, calculations of two types in which calculation times are different may be performed in the first calculation and the second calculation.
The first calculation is estimated, for example, by matching learning information which will be described later and three-dimensional information acquired from an image acquired by the imaging unit. The second calculation is performed by matching model information of the object W or constituent elements (components, structure members) of the object W and three-dimensional information acquired from the image acquired by the imaging unit. The model information will be described in detail later. An initial position of the object W at the time of performing matching in the second calculation uses an approximate position and posture result in the first calculation.
In the embodiment, manufacturing accuracies acceptable for the plurality of constituent elements 202 to 205 are different. For example, when the object W is connected to another object, if the constituent elements 203 and 204 are set as constituent elements serving as connecting portions, manufacturing accuracies for the constituent elements 203 and 204 are higher than those of the constituent elements 202 and 205.
In the measuring method according to the embodiment, an approximate position and posture of the object W is estimated on the basis of a first portion (first processing object) constituting the object W in an image acquired by the image acquisition unit 110. Moreover, the object in an image is matched with the model information on the basis of a plurality of second processing objects associated with the estimation results and a first processing object. The matching results obtained for the plurality of second processing objects are compared and a specific position and posture of the object W is determined. At the time of matching with the model information, contour information included in the second processing object, distance information of the housing 201, or the like is used. Here, both of the first and second processing objects may be interpreted as a part of the object W (relative characteristic part) or as a part of an image obtained by photographing the object W (corresponding to a part of the image).
(Measuring Method)
First, in Step S301, the storage unit 123 stores learning information and model information. The learning information is information obtained by processing a plurality of images obtained by photographing the object W in a plurality of directions (imaging angles) in advance. The model information is created, for example, using a previously created computer aided design (CAD) model of the object W.
(Learning Information Used in First Calculation)
As illustrated in
Referring back to
In Step S303, the user selects a second processing object. The second processing object may or may not include any constituent element including any of the first processing objects. In the embodiment, the second processing object includes a constituent element included in the first processing object associated with the second processing object. In the second calculation, it is required to perform measurement with higher accuracy than in the first calculation. As described above, the second calculation is performed by matching the model information of the object W and contour information or three-dimensional information obtained from an image acquired by the image acquisition unit 110. However, when manufacturing accuracies of constituent elements constituting the object W are low, deviation from the model information increases and thus accuracy of the second calculation deteriorates.
Thus, it is desirable that the second calculation be performed on a constituent element of the object W having a high manufacturing accuracy. In the embodiment, as described above, manufacturing accuracies of the constituent elements 203 and 204 are higher than those of other constituent elements. Therefore, the second calculation is preferably performed on the constituent element 203 or 204. Here, since there is also a case in which the constituent elements 203 and 204 are not included in an image in some cases, another constituent element may be selected as an object used in the second calculation. The selected constituent element is stored in the storage unit 123.
An example of a relationship between the first processing object stored in Step S302 and the second processing object stored in Step S303 is illustrated in
(Calculation of Position and Posture)
The calculation unit 122 performs the first calculation in Step S703 on the basis of the settings information acquired in Step S701 and then performs the second calculation in Step S704. It should be noted that a step of determining whether to use any of the calculation sets selected in Step S701 in Step 5704 may be provided between Step S703 and Step S704. That is to say, a portion used in the second calculation in Step S704 may be selected on the basis of the result of the first calculation in Step S703 so that the second calculation in Step S704 is performed using the selected portion. For example, if it is determined from the result of the first calculation in Step S703 that an estimation accuracy of a position and a posture in a case in which the first portion 51 is used is higher than that of a case in which the second portion 52 is used, the second calculation may be performed using the constituent elements 202 and 203 of the second processing object associated with the first portion 51.
In Step S705, the calculation unit 122 determines a position and a posture of the object W on the basis of scores (results of matching) indicating matching (degree of matching, concordance rate) of an image with the model information calculated by the second calculation. According to the settings in the embodiment, since the second calculation is performed using the constituent element 202 or 203, the number of results of matching is two. The calculation unit 122 uses a result of greater matching between the two results of matching to determine a position and posture.
According to the measuring apparatus 100 in the embodiment, a combination of a portion to be subjected to the first calculation and a portion to be subjected to the second calculation can be flexibly set. Therefore, the measuring apparatus 100 is advantageous, for example, in terms of measurement availability and measurement accuracy regardless of an imaged portion of the object W. Furthermore, the measuring apparatus 100 can also be advantageous in terms of a calculation time and the number of processes.
In the first embodiment, no duplication of constituent elements is assumed among a plurality of portions which can be used in the first calculation. However, common constituent elements can also be provided among the plurality of portions which can be used in the first calculation in some cases.
For example, a case in which the first processing object as the object W illustrated in
This embodiment is characterized by each first processing object including a common second processing object. It should be noted that an apparatus configuration and a measurement flow are the same as in the first embodiment.
A measuring method according to a third embodiment has features concerning a method of storing settings information. This embodiment is characterized by a first processing object and a second processing object being designated at different timings in a stepwise manner when settings information is stored in a storage unit 123.
The display unit 301 is connected to the processing unit 120. The input unit 302 is connected to the display unit 301. For example, the display unit 301 displays required information when storing settings information in the storage unit 123. The user gives instructions to the display information by the input unit 302. The display unit 301 stores settings information in the storage unit 123 on the basis of an output from the input unit 302.
In an upper part of the display unit 301, as illustrated in
A window illustrated on the left of
Also, all combinations of a set designated by selecting a checkbox in the list are displayed in a calculation combination column. The set designated by selecting the checkbox is set to be a calculation set actually used in calculation and thus a combination for performing calculations (settings information) is stored.
When the display unit 301 in
Also, for example, portions of the object W which a first processing object and a second processing object indicate may be displayed on the display unit 301 by CAD data and additional information such as the number of calculations performed in accordance with set conditions may be indicated. Thus, convenience when the user performs setting is improved.
Also, second processing objects associated with the first processing object may be sorted to some extent before the above-described screen is displayed, the sorted second processing objects may be displayed when a first processing object has been selected, and any among them may be registered. In the example of
It should be noted that an input of information indicated on the display unit 301 may be designated by an electronic message of a command or the like.
(Embodiment Related to Method of Manufacturing Article)
The above-described measuring apparatus can be used while being supported by a certain support member. In this embodiment, for example, a control system provided and used in a robot arm 180 (gripping apparatus) like in
Note that, although calculations having different degrees of refinement are used as a plurality of calculation methods used for estimating a position and posture in the embodiment, the calculations may be calculations with different calculation speeds. Furthermore, in the second calculation, different constituent elements (for example, constituent elements 203 and 204) may be selected as one set.
Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments and various modifications are possible without departing from the gist of the present invention.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2017-165323 filed on Aug. 30, 2017, which are hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2017-165323 | Aug 2017 | JP | national |
