Patent Application
20180262739
The present disclosure relates to an object detection system, such as an object detection system for vehicles that performs three-dimensional reconstruction of select objects of interest.
This section provides background information related to the present disclosure, which is not necessarily prior art.
Some vehicle safety systems and autonomous driving systems use three-dimensional scene reconstruction of an entire environment around a vehicle. While current three-dimensional scene reconstruction systems are suitable for their intended use, they are subject to improvement. For example, current systems three-dimensionally reconstruct an entire scene captured by a camera, which requires an extensive amount of processing power and processing time making it sometimes difficult for the system to operate optimally when the vehicle is traveling at high speed. The present teachings address these issues with current three-dimensional systems, as well as numerous other issues, and provide numerous advantages as set forth herein and as one skilled in the art will appreciate.
This section provides a general summary of the disclosure, and is not a comprehensive disclosure of its full scope or all of its features.
The present teachings include a three-dimensional imaging system for imaging an object of interest present in an area about a vehicle. The system includes a camera and a control module. The camera is configured to capture an image of the area about the vehicle including the object of interest. A control module of the system compares the captured image to previously captured model images including examples of the object of interest. The control module also identifies the object of interest in the captured image based on the comparison, and builds a three-dimensional reconstruction of the object of interest.
Further areas of applicability will become apparent from the description provided herein. The description and specific examples in this summary are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.
The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations, and are not intended to limit the scope of the present disclosure.
Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings.
Example embodiments will now be described more fully with reference to the accompanying drawings.
With initial reference to
The system 10 is able to prepare a three-dimensional reconstruction of any suitable object of interest, such as, for example, any suitable road sign, traffic light, pedestrian, and/or any suitable type of infrastructure, such as an overpass, bridge, toll booth, construction zone, etc. The camera 20 can be any type of camera or sensing device capable of capturing images of one or more of such objects of interest present in an area about the vehicle 40. For example, the camera 20 can be a visible light camera, an infrared camera, etc. The camera 20 can be mounted at any suitable position about the vehicle 40, such as on a roof of the vehicle 40, at or near a front end of the vehicle 40, on a windshield of the vehicle 40, etc. The system 10 can include any suitable number of cameras 20, although the exemplary system described herein includes a single camera 20.
As explained further herein, the control module 30 receives an image taken by the camera 20 including an object of interest, and builds a three-dimensional image of the object of interest. In this application, including the definitions below, the term “module” may be replaced with the term “circuit.” The term “module” may refer to, be part of, or include processor hardware (shared, dedicated, or group) that executes code and memory hardware (shared, dedicated, or group) that stores code executed by the processor hardware. The code is configured to provide the features of the control module 30 described herein.
The control module 30 will now be described in conjunction with method 210 of
The control module 30 is configured to compare the image captured by the camera 20 of the object of interest to previously captured model images including examples of the object of interest (e.g., objects that are similar to, or the same as, the object of interest). The previously captured model images including the objects of interest can be created and supplied in any suitable manner. For example, the previously captured model images can be captured by a manufacturer, distributor, or general provider of the system 10. The previously captured model images can be loaded to the control module 30 by the manufacturer, seller, or provider of the system 10, or can be obtained and loaded by a user of the system 10, such as by downloading the previously captured model images from any suitable source in any suitable manner, such as by way of an internet connection.
With reference to block 212 of the method 210, the control module 30 can compare the captured images to the previously captured model images including examples of the object of interest in any suitable manner. For example and with reference to block 214, the control module 30 can segment the captured image into regions having similar pixel characteristics, such as with respect to pixel brightness, color, etc.
With reference to block 216, the control module 30 obtains image statistics for each one of the segmented regions of the segmented image. Any image statistics suitable for identifying the object of interest can be obtained. For example, the mean and standard deviation of pixel values of each one of the segmented regions can be obtained by the control module 30. The control module 30 then compares the image statistics obtained from the captured images with model image statistics of segmented areas of the previously captured model images that are known to include examples of the object of interest, such as set forth at block 218.
With reference to block 220 of the method 210, the control module 30 identifies the object of interest in the captured image based on the comparison of the captured image to the previously captured model images that include examples of the object of interest. For example, the control module 30 can identify the object of interest in the captured image by identifying the segmented region of the captured image having image statistics that are most similar to, or the same as, the image statistics of the segment(s) of the previously captured model image(s) including an example of the object of interest, as set forth at block 222. In other words, if the object of interest is a road sign, the control module 30 identifies the segment(s) of the model image(s) having an exemplary road sign and the image characteristics of the segment(s). The control module 30 then determines which segment(s) of the captured image has image statistics that are most similar to, or the same as, the segment of the model image that is known to include a road sign, and identifies that segment of the captured image as having a road sign.
The control module 30 assigns a confidence value to each segment identified as including the object of interest, such as a road sign, as illustrated in
Advantageously, the control module 30 builds a three-dimensional model of only the object(s) of interest. The control module 30 does not create a three-dimensional model of other objects in the image captured by the camera 20, which advantageously saves time and processing power. Thus when the vehicle 40 is traveling at a high rate of speed, the control module 30 can quickly identify objects of interest and create a three-dimensional reconstruction thereof.
With reference to block 226 of the method of
The foregoing description of the embodiments has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but, where applicable, are interchangeable and can be used in a selected embodiment, even if not specifically shown or described. The same may also be varied in many ways. Such variations are not to be regarded as a departure from the disclosure, and all such modifications are intended to be included within the scope of the disclosure.
Example embodiments are provided so that this disclosure will be thorough, and will fully convey the scope to those who are skilled in the art. Numerous specific details are set forth such as examples of specific components, devices, and methods, to provide a thorough understanding of embodiments of the present disclosure. It will be apparent to those skilled in the art that specific details need not be employed, that example embodiments may be embodied in many different forms and that neither should be construed to limit the scope of the disclosure. In some example embodiments, well-known processes, well-known device structures, and well-known technologies are not described in detail.
The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms “a,” “an,” and “the” may be intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms “comprises,” “comprising,” “including,” and “having,” are inclusive and therefore specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order discussed or illustrated, unless specifically identified as an order of performance. It is also to be understood that additional or alternative steps may be employed.
When an element or layer is referred to as being “on,” “engaged to,” “connected to,” or “coupled to” another element or layer, it may be directly on, engaged, connected or coupled to the other element or layer, or intervening elements or layers may be present. In contrast, when an element is referred to as being “directly on,” “directly engaged to,” “directly connected to,” or “directly coupled to” another element or layer, there may be no intervening elements or layers present. Other words used to describe the relationship between elements should be interpreted in a like fashion (e.g., “between” versus “directly between,” “adjacent” versus “directly adjacent,” etc.). As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.
Although the terms first, second, third, etc. may be used herein to describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections should not be limited by these terms. These terms may be only used to distinguish one element, component, region, layer or section from another region, layer or section. Terms such as “first,” “second,” and other numerical terms when used herein do not imply a sequence or order unless clearly indicated by the context. Thus, a first element, component, region, layer or section discussed below could be termed a second element, component, region, layer or section without departing from the teachings of the example embodiments.
Spatially relative terms, such as “inner,” “outer,” “beneath,” “below,” “lower,” “above,” “upper,” and the like, may be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. Spatially relative terms may be intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as “below” or “beneath” other elements or features would then be oriented “above” the other elements or features. Thus, the example term “below” can encompass both an orientation of above and below. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.
