Patent Grant
11948371
This application is a U.S. National Phase of International Patent Application No. PCT/JP2019/020378 filed on May 23, 2019, which claims priority benefit of Japanese Patent Application No. JP 2018-108328 filed in the Japan Patent Office on Jun. 6, 2018. Each of the above-referenced applications is hereby incorporated herein by reference in its entirety.
The present technique relates to an information processing apparatus, an information processing method, a program, and a mobile body and, particularly, to an information processing apparatus, an information processing method, a program, and a mobile body which are suitably used when performing detection processing with respect to a passage around a mobile body.
Conventionally, methods of detecting roads and lanes using a camera and LiDAR (Light Detection and Ranging) have been proposed (for example, refer to NPL 1).
[NPL 1]
On the other hand, configurations of roads vary depending on a shape (for example, a straight line or a curve), lane markings, the number of lanes, the presence or absence of a sidewalk or a shoulder, and the like. In addition, even a same road may appear significantly different depending on circumstances (for example, weather or time of day). Therefore, the emergence of a technique that improves accuracy of detection processing of a configuration or the like of a road regardless of differences in configurations, circumstances, and the like is eagerly anticipated.
The present technique has been devised in consideration of the circumstances described above and an object thereof is to improve accuracy of detection processing related to a passage (for example, a road) around a mobile body.
An information processing apparatus according to a first aspect of the present technique includes: a transforming unit configured to respectively transform a plurality of photographed images with partially overlapping photographing ranges into bird's eye images; a compositing unit configured to generate a composite image by compositing a plurality of the bird's eye images; and a detecting unit configured to perform detection processing related to a passage around a mobile body based on the composite image.
An information processing method according to the first aspect of the present technique includes the steps of: respectively transforming a plurality of photographed images with partially overlapping photographing ranges into bird's eye images; generating a composite image by compositing a plurality of the bird's eye images; and performing detection processing related to a passage around a mobile body based on the composite image.
A program according to the first aspect of the present technique causes a computer to execute processing for: respectively transforming a plurality of photographed images with partially overlapping photographing ranges into bird's eye images; generating a composite image by compositing a plurality of the bird's eye images; and performing detection processing related to a passage around a mobile body based on the composite image.
A mobile body according to a second aspect of the present technique includes: a photographing unit configured to photograph a plurality of photographed images with partially overlapping photographing ranges; a transforming unit configured to respectively transform a plurality of the photographed images into bird's eye images; a compositing unit configured to generate a composite image by compositing a plurality of the bird's eye images; a detecting unit configured to perform detection processing related to a passage around a mobile body based on the composite image; and a motion control unit configured to control motion based on a result of the detection processing.
In the first aspect of the present technique, a plurality of photographed images with partially overlapping photographing ranges are respectively transformed into bird's eye images, a composite image is generated by compositing a plurality of the bird's eye images, and detection processing related to a passage around a mobile body is performed based on the composite image.
In the second aspect of the present technique, a plurality of photographed images with partially overlapping photographing ranges are photographed, a plurality of the photographed images are respectively transformed into bird's eye images, a composite image is generated by compositing a plurality of the bird's eye images, detection processing related to a passage around a mobile body is performed based on the composite image, and control of motion is performed based on a result of the detection processing.
According to the first aspect or the second aspect of the present technique, accuracy of detection processing related to a passage around a mobile body improves.
It should be noted that the advantageous effects described above are not necessarily restrictive and any of the advantageous effects described in the present disclosure may apply.
Hereinafter, modes for implementing the present technique will be described. The descriptions will be given in the following order.
First, a first embodiment of the present technique will be described with reference to
<Configuration Example of Vehicle 10>
The vehicle 10 includes a photographing unit 11, an image processing unit 12, and a motion control unit 13.
For example, the photographing unit 11 includes a photographing apparatus which includes a zoom lens and which is capable of changing an angle of view and a focal length. In addition, the photographing unit 11 is provided at a position where the front of the vehicle 10 can be photographed. The photographing unit 11 photographs the front of the vehicle 10 while alternately switching between two angles of view and supplies the images having been photographed (hereinafter, referred to as photographed images) to a transforming unit 31 of a composite image generating unit 21 of the image processing unit 12.
Hereinafter, among the photographed images that are taken by the photographing unit 11, a photographed image with a wider angle of view will be referred to as a wide angle image and a photographed image with a smaller angle of view will be referred to as a telescopic image. The wide angle image and the telescopic image have partially overlapping photographing ranges.
Based on the wide angle image and the telescopic image photographed by the photographing unit 11, the image processing unit 12 performs detection processing of a lane marking (for example, a white line) of a road in front of the vehicle 10. The image processing unit 12 includes the composite image generating unit 21 and a detecting unit 22.
The composite image generating unit 21 generates a composite image obtained by compositing the wide angle image and the telescopic image. The composite image generating unit 21 includes the transforming unit 31 and a compositing unit 32.
The transforming unit 31 transforms the wide angle image and the telescopic image into bird's eye images and supplies the compositing unit 32 with the transformed bird's eye images. Hereinafter, the bird's eye image having been transformed from the wide angle image will be referred to as a wide angle bird's eye image and the bird's eye image having been transformed from the telescopic image will be referred to as a telescopic bird's eye image.
The compositing unit 32 generates a composite image obtained by compositing the wide angle bird's eye image and the telescopic bird's eye image and supplies the detecting unit 22 with the composite image. When necessary, the compositing unit 32 composites two bird's eye images using a detection result of a lane marking by the detecting unit 22.
Based on the composite image, the detecting unit 22 performs detection processing of a lane marking on the road in front of the vehicle 10. The detecting unit 22 supplies the compositing unit 32 and the motion control unit 13 with data indicating a detection result of a lane marking.
The motion control unit 13 controls motion of the vehicle 10 based on the detection result of a lane marking on the road in front of the vehicle 10 and the like. For example, the motion control unit 13 controls acceleration, deceleration, stopping, steering, automated driving, and the like of the vehicle 10. In addition, the motion control unit 13 supplies the transforming unit 31 with data indicating a speed of the vehicle 10.
<Configuration Example of Detecting Unit 22>
The detecting unit 22 includes a filtering unit 101, a binarizing unit 102, a line segment detecting unit 103, a spline processing unit 104, and a lane marking detecting unit 105.
The filtering unit 101 performs filtering of the composite image using a prescribed filter and extracts an edge in the composite image. The filtering unit 101 supplies the binarizing unit 102 with an image (hereinafter, referred to as an edge image) obtained by extracting an edge of the composite image.
The binarizing unit 102 performs binarization processing using a prescribed threshold with respect to the edge image and supplies the line segment detecting unit 103 with an obtained binarized image.
The line segment detecting unit 103 performs detection processing of a line segment inside the binarized image. The line segment detecting unit 103 supplies the spline processing unit 104 with an image (hereinafter, referred to as a line segment-detected image) indicating a detection result of a line segment.
The spline processing unit 104 performs spline processing with respect to the line segment-detected image and detects a curve in the line segment-detected image. The spline processing unit 104 supplies the lane marking detecting unit 105 with an image (hereinafter, referred to as a spline image) indicating a detection result of a curve.
Based on the curve inside the spline image, the lane marking detecting unit 105 detects a lane marking on the road in front of the vehicle 10. The lane marking detecting unit 105 supplies the compositing unit 32 and the motion control unit 13 with data indicating a detection result of a lane marking.
<Lane Marking Detection Processing>
Next, lane marking detection processing to be executed by the vehicle 10 will be described with reference to the flow chart shown in
For example, the processing is started when the vehicle 10 is started and an operation for starting driving is performed such as when an ignition switch, a power switch, a start switch, or the like of the vehicle 10 is turned on. In addition, for example, the processing is ended when an operation for ending driving is performed such as when the ignition switch, the power switch, the start switch, or the like of the vehicle 10 is turned off.
In step S1, the photographing unit 11 photographs a wide angle image. Specifically, the photographing unit 11 photographs the front of the vehicle 10 in a state where the focal length of the zoom lens is set short and an angle of view thereof is set wide. The photographing unit 11 supplies the transforming unit 31 with a wide angle image obtained as a result of the photography.
In step S2, the photographing unit 11 photographs a telescopic image. Specifically, the photographing unit 11 photographs the front of the vehicle 10 in a state where the focal length of the zoom lens is set long and an angle of view thereof is set narrow. The photographing unit 11 supplies the transforming unit 31 with a telescopic image obtained as a result of the photography.
In step S3, the transforming unit 31 transforms the wide angle image and the telescopic image into bird's eye images.
For example, the transforming unit 31 transforms the wide angle image and the telescopic image into bird's eye images using a function f1 (α, β, γ, s) which is a transformation function and a function f2 (v, Δt).
Hereinafter, the function f1 (α, β, γ, s) may be simply referred to as the function f1, the function f2 (v, Δt) may be simply referred to as the function f2, and function f1 (α, β, γ, s)+function f2 (v, Δt) may be simply referred to as function f1+f2.
The function f1 is a function used in the transformation from a photographed image into a bird's eye image.
A pitch angle α, a yaw angle β, and a roll angle γ which are variables of the function f1 indicate a mounting angle (an Euler angle) of the photographing unit 11. A scale s indicates an enlargement ratio or a reduction ratio of a bird's eye image after the transformation.
Definitions of the pitch angle α, the yaw angle β, and the roll angle γ will now be described with reference to
A point O in
The world coordinate system is defined by an Xw axis, a Yw axis, and a Zw axis which are mutually perpendicular. The Xw axis and the Yw axis are axes parallel to ground and the Zw axis is an axis that is perpendicular to ground.
The camera coordinate system is defined by an Xc axis, a Yc axis, and a Zc axis which are mutually perpendicular. The Xc axis is an axis in a lateral direction (a left-right direction) of the photographing unit 11, the Yc axis is an axis in a longitudinal direction (an up-down direction) of the photographing unit 11, and the Zc axis is an axis that coincides with an optical axis of the photographing unit 11.
The pitch angle α indicates an orientation of the Zc axis (the optical axis of the photographing unit 11) in a direction perpendicular to ground. Specifically, the pitch angle α indicates an angle of the Zc axis with respect to the Yw axis on a plane which is constituted by the Yw axis and the Zw axis of the world coordinate system and which is perpendicular to ground.
The yaw angle β indicates an orientation of the Zc axis (the optical axis of the photographing unit 11) in a direction parallel to ground. Specifically, the yaw angle β indicates an angle of the Zc axis with respect to the Yw axis on a plane which is constituted by the Xw axis and the Yw axis of the world coordinate system and which is parallel to ground.
The roll angle γ is an angle in a direction of rotation of the Zc axis (the optical axis of the photographing unit 11) and indicates an angle of the Xc axis with respect to the Xw axis on a plane which is constituted by the Xw axis and the Zw axis of the world coordinate system and which is perpendicular to ground.
For example, the function f1 includes processing for performing coordinate transformation from a photographed image to a bird's eye image using a transformation matrix T represented by expression (1) below.
A height h indicates a height of the optical center O of the photographing unit 11 from ground as shown in
In addition, c1=cos α, c2=cos β, s1=sin α, and s2=sin β. α and β denote the pitch angle α and the yaw angle β indicating the mounting angle of the photographing unit 11 described above.
A focal length fu and a focal length fv indicate a focal length in the horizontal direction and a focal length in the perpendicular direction of the photographing unit 11.
cu and cv indicate coordinates of a center Pc of a photographed image in an image coordinate system constituted by a u axis in the horizontal direction and a v axis in the perpendicular direction as shown in
Regarding a method of transforming a photographed image into a bird's eye image, details are disclosed in, for example, M. Aly, “Real time detection of lane markers in urban streets”, In Intelligent Vehicles Symposium, 2008 IEEE, pages 7-12, IEEE, February 2008.
The function f2 (v, Δt) is a function for shifting a bird's eye image in a direction of travel of the vehicle 10. The function f2 (v, Δt) is mainly used to correct a deviation in photographic timings of a wide angle image and a telescopic image.
A vehicle speed v indicates a speed of the vehicle 10. A time difference ΔT indicates a time difference between a photographic timing (a photographic time point) of the wide angle image and a photographic timing (a photographic time point) of the telescopic image.
In addition, the transforming unit 31 performs an adjustment (optimization processing) of function f1+f2 every time or at a prescribed timing such that a similarity between a wide angle bird's eye image and a telescopic wide angle image is increased (for example, maximized).
For example, first, the transforming unit 31 detects feature points of a wide angle image and a telescopic image.
A detection method of the feature points is not particularly limited.
Next, the transforming unit 31 detects at least three or more pairs of mutually corresponding feature points or, in other words, at least three or more pairs of feature points at same positions in the real world between the wide angle image and the telescopic image.
A detection method of the pairs of feature points is not particularly limited. For example, pairs of feature points are detected using ZNCC (Zero-means Normalized Cross-Correlation), SIFT (Scale-Invariant Feature Transform), or the like.
Next, using function f1+f2, the transforming unit 31 calculates positions of feature points that constitute each pair on the wide angle bird's eye image and the telescopic bird's eye image.
In addition, based on a positional relationship of corresponding feature points (feature points that constitute a pair) of the wide angle bird's eye image and the telescopic bird's eye image, the transforming unit 31 adjusts the pitch angle α, the yaw angle β, the roll angle γ, the scale s, and the vehicle speed v which are variables of function f1+f2. For example, the transforming unit 31 adjusts the pitch angle α, the yaw angle β, the roll angle γ, the scale s, and the vehicle speed v using a least-square method or the like so as to minimize a difference between coordinates of corresponding feature points of the wide angle bird's eye image and the telescopic bird's eye image.
The pitch angle α, the yaw angle β, and the roll angle γ are obtained as design values upon mounting the photographing unit 11 to the vehicle 10. However, since the position or an orientation of the photographing unit 11 may change due to motion of the vehicle 10, impact from the outside, and the like, a fine adjustment is desirably performed as described above.
In addition, the vehicle speed v is set based on data supplied from the motion control unit 13.
Furthermore, the transforming unit 31 supplies the compositing unit 32 with the wide angle bird's eye image and the telescopic bird's eye image.
As described above, by respectively transforming the wide angle image and the telescopic image into bird's eye images, two images photographed according to different viewpoints and different focal lengths are transformed onto a single domain. As a result, compositing of the two images can be readily performed.
In addition, while lane markings become line segments that head toward a vanishing point in a wide angle image and a telescopic image, the lane markings become parallel lines in a wide angle bird's eye image and a telescopic bird's eye image. As a result, detection accuracy of a lane marking improves.
In step S4, the compositing unit 32 composites two bird's eye images. Specifically, the compositing unit 32 generates a composite image by positioning and subsequently compositing the wide angle bird's eye image and the telescopic bird's eye image. The compositing unit 32 supplies the filtering unit 101 with the composite image.
For example, in a loop of the lane marking detection processing, the compositing unit 32 may perform feedback control using a detection result of a lane marking obtained in processing (to be described later) of step S9 performed prior to processing of step S4. For example, the compositing unit 32 positions the wide angle bird's eye image and the telescopic bird's eye image based on the number of lane markings, a dispersion of positions thereof, or the like detected in the processing of step S9. Accordingly, a compositing accuracy of the two bird's eye images improves.
In step S5, the filtering unit 101 performs filtering of the composite image. Specifically, the filtering unit 101 performs filtering of the composite image using a prescribed filter to extract an edge of the composite image. The filtering unit 101 supplies the binarizing unit 102 with an edge image obtained by extracting an edge of the composite image.
An arbitrary filter capable of extracting an edge of an image can be used as the filter to be used by the filtering unit 101.
In step S6, the binarizing unit 102 performs binarization processing. In other words, the binarizing unit 102 binarizes the edge image using a predetermined threshold. Accordingly, a binarized image indicating an edge portion of the composite image is generated. The binarizing unit 102 supplies the line segment detecting unit 103 with the binarized image.
In step S7, the line segment detecting unit 103 performs line segment detection processing. Specifically, the line segment detecting unit 103 detects a line segment inside the binarized image. The line segment detecting unit 103 supplies the spline processing unit 104 with a line segment-detected image indicating a detection result of a line segment.
An arbitrary method can be used in the line segment detection processing.
In step S8, the spline processing unit 104 performs spline processing. Specifically, the spline processing unit 104 performs spline processing with respect to the line segment-detected image to fit each line segment in the line segment-detected image to a curve model. Accordingly, a curve is detected in the line segment-detected image. The spline processing unit 104 supplies the lane marking detecting unit 105 with a spline image indicating a detection result of a curve.
For example, an arbitrary method such as RANSAC (Random Sampling Consensus) can be used in the spline processing.
In step S9, the line marking detecting unit 105 performs detection processing of a lane marking. For example, based on a shape of each curve, a width between curves, and the like, the lane marking detecting unit 105 extracts a curve constituting a lane marking among the curves in the spline image. In addition, the line marking detecting unit 105 detects a lane marking constituted by the detected curve. The lane marking detecting unit 105 supplies the compositing unit 32 and the motion control unit 13 with data indicating a detection result of a lane marking.
An arbitrary method can be used in the detection processing of a lane marking.
The motion control unit 13 controls motion of the vehicle 10 based on the detection result of a lane marking on the road in front of the vehicle 10 and the like. For example, the motion control unit 13 detects a lane of the road in front based on lane markings on the road in front. In addition, for example, based on a detection result of a lane, the motion control unit 13 performs control (lane control) in a direction of travel of the vehicle 10 such that the vehicle 10 travels inside a lane in a stable manner. Furthermore, for example, based on a detection result of a lane, the motion control unit 13 performs control of a lane change by the vehicle 10 and detection processing of a vehicle in an adjacent lane.
Subsequently, processing returns to step S1 and processing subsequent to step S1 is executed.
In this manner, detection accuracy of a lane marking on the road in front of the vehicle 10 can be improved.
For example,
As described above, in the vehicle 10, a lane marking can be detected using information on a wider range as compared to a case where a wide angle bird's eye image or a telescopic bird's eye image is used independently. Therefore, due to an increased amount of information, detection accuracy of a lane marking by the detecting unit 22 improves. In addition, lane markings of a wider range can be detected at once. For example, a lane marking at a greater distance or a lane marking of a lane adjacent to a lane being traveled by the vehicle 10 can be detected.
In addition, when lane markings from a wide angle bird's eye image and a telescopic bird's eye image are individually detected and detection results are subsequently composited, errors and the like may create an inconsistency between the two detection results. In addition, complex arithmetic processing is required to composite the detection results. On the other hand, compositing the two bird's eye image and then detecting a lane marking prevents the occurrence of an inconsistency between the detection results described above. In addition, since compositing processing of images requires a smaller amount of calculations than compositing processing of detection results, the amount of calculations can be reduced.
Furthermore, detection accuracy of a lane marking can be further improved by, for example, optimizing a transformation function from a photographed image to a bird's eye image so as to maximize a similarity between a wide angle bird's eye image and a telescopic wide angle image.
Next, a second embodiment of the present technique will be described with reference to
<Configuration Example of Vehicle 200>
A vehicle 200 differs from the vehicle 10 in that the vehicle 200 includes a photographing unit 201 instead of the photographing unit 11. The photographing unit 201 includes a visible image photographing unit 211 and an infrared image photographing unit 212 of which wavelength regions to be a photographic target differ from each other.
The visible image photographing unit 211 includes, for example, a photographing apparatus having sensitivity to visible light. The visible image photographing unit 211 photographs the front of the vehicle 200 and supplies the transforming unit 31 with an image (hereinafter, referred to as a visible image) obtained as a result of the photography.
The infrared image photographing unit 212 includes, for example, a photographing apparatus having sensitivity to infrared light. The infrared image photographing unit 212 photographs the front of the vehicle 200 and supplies the transforming unit 31 with an image (hereinafter, referred to as an infrared image) obtained as a result of the photography.
A visible image and a captured image have partially overlapping photographing ranges.
<Lane Marking Detection Processing>
Next, lane marking detection processing to be executed by the vehicle 200 will be described with reference to the flow chart shown in
For example, the processing is started when the vehicle 200 is started and an operation for starting driving is performed such as when an ignition switch, a power switch, a start switch, or the like of the vehicle 200 is turned on. In addition, for example, the processing is ended when an operation for ending driving is performed such as when the ignition switch, the power switch, the start switch, or the like of the vehicle 200 is turned off.
In step S101, the visible image photographing unit 211 and the infrared image photographing unit 212 photographs a visible image and an infrared image. Specifically, the visible image photographing unit 211 photographs the front of the vehicle 200 and supplies the transforming unit 31 with a visible image obtained as a result of the photography. The infrared image photographing unit 212 photographs the front of the vehicle 200 and supplies the transforming unit 31 with an infrared image obtained as a result of the photography.
In step S102, the transforming unit 31 transforms the visible image and the infrared image into bird's eye images. In other words, the transforming unit 31 transforms the visible image and the infrared image into bird's eye images by processing similar to that of step S3 in
Hereinafter, the bird's eye image transformed from the visible image will be referred to as a visible bird's eye image and the bird's eye image transformed from the infrared image will be referred to as an infrared bird's eye image.
The transforming unit 31 supplies the compositing unit 32 with the visible bird's eye image and the infrared bird's eye image.
Since a visible image and an infrared image are photographed by different photographing units, values of variables given to the transformation function f1+f2 differ between the visible image and the infrared image. Therefore, when performing optimization processing of the transformation function f1+f2 described earlier, optimal values of the variables assume different values between the visible image and the infrared image.
In step S103, the compositing unit 32 composites the two bird's eye images, namely, the visible bird's eye image and the infrared bird's eye image by processing similar to that of step S3 in
Subsequently, in step S104 to step S108, processing similar to that of step S5 to step S9 in
Subsequently, processing returns to step S101 and processing subsequent to step S101 is executed.
As described above, by detecting a lane marking on the road in front of the vehicle 200 based on the composite image obtained by compositing the visible bird's eye image and the infrared bird's eye image, detection accuracy of a lane marking improves particularly during nighttime or at dark places such as the inside of a tunnel. For example, when only a visible image is used, a lane marking can only be accurately detected in a region (hereinafter, referred to as an illuminated region) that is illuminated by lighting such as a headlight of the vehicle 200 or a street light. In contrast, by also using an infrared image, a lane marking in regions (hereinafter, referred to as non-illuminated regions) other than the illuminated region can also be detected with accuracy.
In addition, since using the composite image obtained by compositing the visible bird's eye image and the infrared bird's eye image enables the detecting unit 22 to perform detection processing of a lane marking based on a larger amount of information, detection accuracy improves. For example, since an infrared image normally contains a large amount of noise, it is difficult to detect a lane marking with accuracy when only an infrared image is used. In contrast, for example, by adopting a detection result of a lane marking inside the illuminated region of the visible bird's eye image as a clue, detection processing of a lane marking in a non-illuminated region of the infrared bird's eye image can be performed. Accordingly, detection accuracy of a lane marking of a non-illuminated region improves.
Furthermore, by detecting a lane marking after compositing the visible bird's eye image and the infrared bird's eye image, an occurrence of an inconsistency between detection results based on the two images can be prevented and an amount of calculations can be reduced in a similar manner to the first embodiment described earlier.
In addition, detection accuracy of a lane marking can be further improved by optimizing a transformation function from a photographed image to a bird's eye image.
Hereinafter, modifications of the embodiments of the present technique described above will be described.
While examples of using the present technique in detection processing of a lane marking in front of a vehicle have been described above, the present technique can also be applied to detection processing of a lane marking in directions other than the front of the vehicle.
In addition, the present technique can also be applied to other detection processing related to a road around the vehicle (for example, the front of the vehicle).
For example, the present technique can be applied to detection processing of a shoulder, a margin, or a sidewalk of a road around the vehicle.
In addition, for example, the present technique can be applied to detection processing of a travelable region of a road around the vehicle (for example processing for separating a driveway from a sidewalk).
Furthermore, while examples in which two photographed images are respectively transformed into bird's eye images and a composite image obtained by compositing the bird's eye images is used in detection processing have been described above, three or more photographed images may be respectively transformed into bird's eye images and a composite image obtained by compositing the bird's eye images may be used in detection processing.
In addition, for example, configurations of photographing units may be modified as deemed appropriate.
For example, in the first embodiment, a wide angle image and a telescopic image may be photographed by different photographing apparatuses (for example, a wide angle camera and a telescopic camera).
In addition, a plurality of photographing apparatuses may be provided such that various directions (for example, the front, the side, the rear, and the like) of the vehicle are photographed and that a part of a photographing range of one photographing apparatus overlaps with that of at least one of the other photographing apparatuses. Furthermore, photographed images obtained by photographing the respective directions may be converted into bird's eye images and a composite image obtained by compositing the bird's eye images may be used in detection processing.
Furthermore, types of vehicles to which the present technique can be applied are not particularly limited. In addition, besides the vehicle exemplified earlier, the present technique can also be applied to other mobile bodies that travel on a road surface such as personal mobility, construction machinery, agricultural and farm machinery (tractors), and robots. Furthermore, the present technique can also be applied to mobile bodies that are remotely operated (manipulated) without carrying a user such as a robot and mobile bodies that travel autonomously.
When applying the present technique to a mobile body that travels a passage other than a road, for example, detection processing related to a passage around the mobile body (for example, detection processing of a lane marking such as a corridor inside a building) can be performed.
<Configuration Example of Computer>
The series of processing described above can be executed by hardware or by software. When the series of processing is to be executed by software, a program constituting the software is installed in a computer. Examples of the computer in this case include a computer built into dedicated hardware and a general-purpose personal computer or the like capable of executing various functions when various programs are installed therein.
In a computer 500, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503 are connected to each other by a bus 504.
An input/output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input/output interface 505.
The input unit 506 is constituted by an input switch, a button, a microphone, an imaging element, or the like. The output unit 507 is constituted by a display, a speaker, or the like. The recording unit 508 is constituted by a hard disk, a non-volatile memory, or the like. The communication unit 509 is constituted by a network interface or the like. The drive 510 drives a removable medium 511 that is a magnetic disk, an optical disk, a magneto optical disk, a semiconductor memory, or the like.
In the computer 500 configured as described above, the series of processing described above is performed as the CPU 501 loads a program recorded in the recording unit 508 onto the RAM 503 via the input/output interface 505 and the bus 504 and executes the program.
For example, the program executed by the computer 500 (the CPU 501) can be provided by being recorded on the removable medium 511 as a packaged medium or the like. Alternatively, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
In the computer 500, the program can be installed in the recording unit 508 via the input/output interface 505 by mounting the removable medium 511 to the drive 510. In addition, the program can be received by the communication unit 509 via a wired or wireless transmission medium and installed in the recording unit 508. Alternatively, the program can be installed in the ROM 502 or the recording unit 508 in advance.
The program to be executed by the computer may be a program which causes processing to be time-sequentially performed along an order described in the present specification or a program which causes processing to be performed in parallel or at necessary timings such as when a call is made.
In addition, in the present specification, a system signifies a set of a plurality of components (apparatuses, modules (parts), and the like), and whether or not all of the components are present inside a same casing does not matter. Therefore, a plurality of apparatuses which are housed in separate casings and which are connected to each other via a network and a single apparatus in which a plurality of modules are housed in a single casing are both considered systems.
Furthermore, embodiments of the present technique are not limited to the embodiments described above and various modifications can be made without departing from the gist of the present technique.
For example, the present technique may adopt a configuration of cloud computing in which a single function is shared among and cooperatively processed by a plurality of apparatuses via a network.
In addition, each step explained in the flow charts described above can be executed in a shared manner by a plurality of apparatuses in addition to being executed by a single apparatus.
Furthermore, when a single step includes a plurality of processing steps, the plurality of processing steps included in the single step can be executed in a shared manner by a plurality of apparatuses in addition to being executed by a single apparatus.
<Examples of Configuration Combinations>
The present technique can also be configured as follows.
It should be noted that the advantageous effects described in the present specification are merely exemplary and are not restrictive, and other advantageous effects may be produced.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2018-108328 | Jun 2018 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2019/020378 | 5/23/2019 | WO |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2019/235227 | 12/12/2019 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 20090097708 | Mizuta | Apr 2009 | A1 |
| 20170006234 | Higuchi et al. | Jan 2017 | A1 |
| 20170184396 | Tateishi et al. | Jun 2017 | A1 |
| 20190026918 | Gomezcaballero et al. | Jan 2019 | A1 |
| 20190094040 | Lewis | Mar 2019 | A1 |
| Number | Date | Country |
|---|---|---|
| 112016000689 | Oct 2017 | DE |
| 2004-297405 | Oct 2004 | JP |
| 2015-192198 | Nov 2015 | JP |
| 2016-149613 | Aug 2016 | JP |
| 2017-116450 | Jun 2017 | JP |
| 2017145543 | Aug 2017 | WO |
| Entry |
|---|
| Hillel, et al, “Recent progress in road and lane detection: A survey”, Machine Vision and Applications, vol. 25, No. 3, Apr. 2014, pp. 727-745. |
| International Search Report and Written Opinion of PCT Application No. PCT/JP2019/020378, dated Jun. 18, 2019, 10 pages of ISRWO. |
| Number | Date | Country | |
|---|---|---|---|
| 20210209379 A1 | Jul 2021 | US |
