Patent Application
20230375668
The present disclosure relates to radar technologies, and is applied to the field of fusion perception of autonomous driving or intelligent driving, and in particular, to a detection apparatus, a control method, a fusion detection system, and a terminal.
With the development of society, intelligent vehicles are gradually entering people's daily lives. An advanced driver assistance system (ADAS) plays a very important role in the intelligent vehicles. The ADAS system uses various sensors installed on a vehicle to sense an ambient environment, collect data, and identify, detect, and trace a still object and a moving object in a driving process of the vehicle, and the ADAS system performs a system operation and system analysis based on map data of a navigator, so that a driver detects a potential danger in advance, thereby effectively increasing driving comfort and safety of the vehicle. In conclusion, in an intelligent vehicle, a sensor plays a very important role in assisted driving and autonomous driving of the intelligent vehicle, and a sensing layer is used as an “eye” of the vehicle, including a visual sensor such as a vehicle-mounted camera and a radar sensor such as a vehicle-mounted lidar and a vehicle-mounted ultrasonic radar. The lidar has comparatively high resolution, good detection performance, and strong concealment, and is one of important sensors in vehicle sensing.
However, compared with an image obtained by using a camera, a detection result obtained by the lidar still has lower resolution, and image-level high-resolution information of a detected target cannot be obtained. This restricts actual application of the lidar. However, a camera sensor usually can capture an image with higher resolution. The lidar and the camera are fused to perform depth completion, so that spatial resolution of the lidar can be effectively improved. This is an important development direction in the autonomous driving field. Specifically, the lidar may obtain point cloud data through detection, and the camera sensor may obtain image data through photographing. After calibration and field-of-view registration are performed on the point cloud data and the image data by using a fusion technology, data fusion is performed to obtain a detection result. However, in a fusion detection process, the lidar and the camera have problems such as a large size, low integration, and complex registration, which affect detection quality.
Therefore, how to resolve the foregoing problem is a research focus of a person skilled in the art.
Embodiments of the present disclosure disclose a detection apparatus, a control method, a fusion detection system, and a terminal. The present disclosure provides a fusion detection apparatus with high integration, to improve accuracy of a detection result and reduce computational complexity.
According to a first aspect, an embodiment of the present disclosure discloses a detection apparatus. The detection apparatus includes a laser transmitter, a laser detector, a beam optical splitter, and an image detector.
The laser transmitter is configured to emit a laser signal. A first laser signal from the laser transmitter is transmitted to a detection area through the beam optical splitter.
The laser detector is configured to receive a first signal from the detection area through the beam optical splitter. The first signal includes a reflected signal corresponding to the first laser signal.
The beam optical splitter is further configured to provide a second signal from the detection area for the image detector.
The image detector is configured to perform imaging by using the second signal. Specifically, the first signal and/or the second signal may be from a first target object in the detection area.
In this embodiment of the present disclosure, the laser transmitter, the laser detector, and the image detector are integrated into one detection apparatus. This improves integration and reduces an overall size, and reduces impact caused by motion, vibration, and the like on a detection result. Further, in this embodiment of the present disclosure, optical paths of the laser signal, the first signal, and the second signal can be coaxial through the beam optical splitter. The laser detector can correspondingly receive a laser signal reflected by the detection area irradiated by the laser transmitter, thereby improving accuracy of the detection result. In addition, the image detector and the laser detector synchronously obtain point cloud information and image information of the detection area, to facilitate subsequent fusion of the point cloud information and the image information that are of a lidar, and therefore complex calibration, registration, and the like do not need to be performed during fusion, and computational complexity is reduced.
With reference to the first aspect, in a possible implementation of the first aspect, the laser transmitter includes a flash light source. Further optionally, the laser transmitter may be a flash array light source and a flash flood light source.
The flash light source can irradiate one field of view at a time, and can achieve an effect of surface irradiation. An array light source can irradiate to obtain an illumination lattice, and can provide a uniform irradiation field of view, thereby improving detection effect. In addition, the array light source has higher energy utilization.
With reference to the first aspect, in another possible implementation of the first aspect, the beam optical splitter includes a first beam optical splitter and a second beam optical splitter.
The first beam optical splitter is configured to transmit the laser signal to the second beam optical splitter.
The second beam optical splitter is configured to transmit the laser signal to the detection area.
The second beam optical splitter is further configured to split an optical signal from the detection area into the first signal and the second signal, provide the second signal for the image detector, and provide the first signal for the first beam optical splitter.
The first beam optical splitter is further configured to provide the first signal for the laser detector.
With reference to the first aspect, in another possible implementation of the first aspect, the first beam optical splitter is a polarization beam splitter or a semi-transmissive semi-reflective beam splitter.
The polarization beam splitter can split polarized incident light into two beams of polarized light for transmission.
Further, the laser transmitter and the laser detector may be coaxial through the polarization beam splitter. For example, the laser signal emitted by the laser transmitter is in a first polarization direction, and may be provided for the detection area after passing through the polarization beam splitter. A polarization direction of an optical signal from the detection area may be changed by using another optical device. After passing through the polarization beam splitter, the optical signal whose polarization direction is changed is provided for the laser detector, and is not transmitted to (or a small part of the optical signal is transmitted to) the laser transmitter (or most of the optical signal is provided for the laser detector, and a small part of the optical signal is absorbed or transmitted to another direction). Therefore, the laser transmitter and the laser detector can be coaxial, and a loss of an echo signal can be reduced.
Alternatively, the semi-transmissive semi-reflective beam splitter can split, within a specific wavelength range, one beam of light into two beams of light whose spectral components are approximately the same, and can also enable the laser transmitter and the laser detector to be coaxial.
With reference to the first aspect, in another possible implementation of the first aspect, a wavelength range of the first laser signal falls inside a first wavelength range, the first signal includes an optical signal whose wavelength range falls inside the first wavelength range, and the second signal includes an optical signal whose wavelength range falls inside a second wavelength range. Further, the first wavelength range is different from the second wavelength range.
A possible case is used as an example herein: The laser transmitter may emit infrared light (for example, a mid-infrared ray at a wavelength of 3 micrometers (μm) to 5 μm), the laser detector may respond to the infrared light, and the image detector may respond to visible light. The second beam optical splitter is configured to split the optical signal from the detection area into the first signal and the second signal, where the first signal includes an optical signal whose wavelength range falls inside a wavelength range of the infrared light, and the second signal includes an optical signal whose wavelength range falls inside a wavelength range of the visible light, so that the laser detector receives the infrared light (including reflected light obtained based on the emitted infrared light), and subsequently obtains point cloud information based on the received infrared light. The image detector receives the visible light, performs imaging, and subsequently obtains image information.
With reference to the first aspect, in another possible implementation of the first aspect, the second beam optical splitter is a dichroic beam splitter or a semi-transmissive semi-reflective beam splitter.
The dichroscope may split the optical signal from the detection area into two parts of signals. The first signal includes a signal at a wavelength range required by the laser detector, and the second signal includes a signal at a wavelength range required by the image detector.
A possible case is used as an example herein: The laser transmitter may emit infrared light (for example, a mid-infrared ray at a wavelength of 3 μm to 5 μm), the laser detector may respond to the infrared light, and the image detector is an image sensor for imaging with visible light. The second beam optical splitter is configured to split the optical signal from the detection area into the first signal and the second signal. Most signals included in the first signal are optical signals whose wavelength ranges are greater than 800 nanometers (nm), and most signals included in the second signal are optical signals whose wavelength ranges are less than 750 nm. Therefore, the first signal includes a reflected signal corresponding to an emitted laser signal, and the second signal includes the visible light. The first signal is provided for the laser detector, and the second signal is provided for the image detector, so that the two detection units respectively obtain valid optical signals, thereby improving signal utilization.
Alternatively, the semi-transmissive semi-reflective beam splitter can split, within a specific wavelength range, one beam of light into two beams of light whose spectral components are approximately the same. In this case, the laser detector, the laser transmitter, and the image detector may operate in a same band.
With reference to the first aspect, in another possible implementation of the first aspect, the first beam optical splitter includes a polarization beam splitter and a wave plate, and the wave plate is disposed between the polarization beam splitter and the second beam optical splitter. Optionally, the wave plate may be a quarter-wave plate, an eighth-wave plate, or the like.
The wave plate may be configured to change a polarization state of passing light, so that a polarization state of the optical signal emitted by the laser transmitter is different from a polarization state of the optical signal from the detection area. The wave plate may be used with the polarization beam splitter to allow the laser transmitter and the laser detector to be coaxial.
With reference to the first aspect, in another possible implementation of the first aspect, a polarizer is disposed between the laser transmitter and the beam optical splitter, and a polarization direction of a laser signal passing through the polarizer is a first polarization direction.
For example, an optical signal from the laser detector is non-polarized light, and the non-polarized light is linearly polarized light after passing through the polarizer. P-polarized light (or S-polarized light) may be obtained when the polarized light passes through the polarization beam splitter, and is subsequently provided for the detection area. The polarizer may be used with the polarization beam splitter and the wave plate to allow optical signals of the laser transmitter, the laser detector, and the image detector to be coaxial, to reduce an optical signal loss.
With reference to the first aspect, in another possible implementation of the first aspect, the laser transmitter is configured with a first polarization film coating, and a polarization direction of a laser signal passing through the polarization film coating is a first polarization direction.
It can be learned that the polarized light is emitted through a polarization film coating, so that a structure of the detection apparatus is further compact, impact caused by motion, vibration, and the like on fusion detection can be reduced, and stability can be improved.
With reference to the first aspect, in another possible implementation of the first aspect, the laser detector is configured with a second polarization film coating, and the second polarization film coating is configured to transmit a signal whose polarization direction is a preset polarization direction in the first signal. In this manner, a light filtering effect can be achieved, and signal validity can be improved.
With reference to the first aspect, in another possible implementation of the first aspect, the image detector includes at least one of a color camera, a grayscale camera, or a multidimensional camera, and the multidimensional camera includes at least one of a grayscale polarization camera, a color polarization camera, and a multispectral polarization camera.
The multidimensional camera may obtain more information about the detection area in original dimensions such as energy strength and a wavelength of an optical signal in combination with another dimension (another attribute of light, for example, one or more of a polarization direction of light and energy density of light).
With reference to the first aspect, in another possible implementation of the first aspect, the apparatus further includes a controller, and the controller is configured to control at least one of the laser transmitters, the image detector, or the laser detector.
In a design, the controller may generate a control signal and send the control signal to at least one unit in the detection apparatus, to control the at least one unit. For example, the controller may send a first control signal to the laser transmitter, and the laser transmitter generates a laser signal in response to the first control signal. For another example, the controller may further send a second control signal to the laser detector, to control the laser detector to convert an optical signal into an electrical signal.
With reference to the first aspect, in another possible implementation of the first aspect, in the detection apparatus, a center point of a laser emission field of view, a center point of a laser detection field of view, and a center point of an image detection field of view overlap with each other.
The laser emission field of view is an area that can be irradiated by the laser signal emitted by the laser transmitter, the laser detection field of view is an area that can be detected by the laser detector, and the image detection field of view is an area that can be detected by the image detector.
It may be understood that a point corresponding to a principal optical axis is a center point of a field of view. Because the optical paths of the laser signal, the first signal, and the second signal are coaxial, the center point of the laser emission field of view, the center point of the laser detection field of view, and the center point of the image detection field of view in the detection apparatus overlap with each other. It should be noted that the overlapping herein means that the center points of the fields of view are also coaxial because the optical paths are coaxial, but the center points are not necessarily precisely overlapping. For example, the overlapping may be not necessarily accurate due to different process designs. However, the method in the present disclosure can reduce complexity of registration and calibration as much as possible.
When the center points overlap, the laser detector may correspondingly receive an optical signal returned by the area irradiated by the laser transmitter, thereby improving accuracy of a laser detection result.
If the laser detection field of view overlaps with the image detection field of view, complexity of registration and calibration during subsequent fusion can be reduced (for example, no extrinsic parameter calibration is required), computation consumption can be reduced, and efficiency of obtaining a detection result can be improved.
In addition, if the image detector supports operating in a laser illumination mode, the laser transmitter and the image detector are coaxial, so that an imaging effect of the image detector can be further improved.
With reference to the first aspect, in another possible implementation of the first aspect, the laser detector is configured to obtain first detection data based on the first signal. Further, the first detection data is used to generate point cloud information of the detection area.
Optionally, the laser detector may obtain an electrical signal based on the first signal. The first detection data may include the electrical signal or further data obtained after the electrical signal is processed.
Further optionally, the first detection data is the point cloud information, or the point cloud information may be obtained by processing the first detection data. The point cloud information may be used to obtain information such as a distance, a speed, a reflectivity, and reflection intensity of the detection area.
With reference to the first aspect, in another possible implementation of the first aspect, the image detector is configured to obtain second detection data based on the second signal. Further, the second detection data is used to generate image information of the detection area.
Specifically, the image detector may obtain pixel information (or an electrical signal) based on the second signal. The second detection data may include the pixel information or further data obtained after the pixel information is processed.
Further optionally, the second detection data includes the image information, or the image information may be obtained by processing the second detection data. The image information may be used to obtain a color, a pattern, a point of interest, or the like of the detection area, or may be used to analyze and identify the detection area.
With reference to the first aspect, in another possible implementation of the first aspect, the apparatus further includes a processing apparatus, and the processing apparatus is configured to fuse the first detection data and the second detection data to obtain a detection result of the detection area.
Optionally, the detection result of the detection area may be a high-resolution fused image of the detection area, or a 3D image of the detection area. Based on the detection result, a distance of an object in the detection area may be determined, and object recognition, analysis, and the like may be performed by using a high-resolution pixel.
Optionally, the processing apparatus and the foregoing controller may be integrated or independent. It may also be understood that the processing apparatus may be located in a lidar, and the lidar implements fusion processing; or the processing apparatus may be located outside the lidar, and an external processing apparatus implements fusion processing.
With reference to the first aspect, in another possible implementation of the first aspect, the apparatus further includes an optical device, and the optical device includes at least one of a lens, a filter, a beam shaper, and a light uniformizer.
With reference to the first aspect, in another possible implementation of the first aspect, the apparatus further includes a first optical device, and the first optical device is disposed between the beam optical splitter and the detection area.
The first optical device is configured to transmit light within a third wavelength range, and the third wavelength range includes the wavelength range of the first laser signal and an operating wavelength range of the image detector. Optionally, the optical device may be specifically a lens or a filter.
During specific implementation, the first optical device may be a wide-spectrum optical device. The wide spectrum indicates that an optical signal with a large wavelength range can pass through the optical device. For example, in a scenario in which the laser transmitter is infrared light, and the image detector responds to visible light, the wide-spectrum optical device can pass through at least a band in which the infrared light is located and a band in which the visible light is located, and can be reused during transceiving.
It can be learned that the wide-spectrum optical device can reuse optical systems of the laser transmitter, the laser detector, and the image detector, to reduce a volume of the detection apparatus and improve compactness.
In addition, in some possible examples, because the laser detector and the image detector share a receiving optical device, synchronous zooming of laser detection and image detection can be implemented, thereby reducing complexity of registration and calibration during subsequent fusion, reducing computation consumption, and improving efficiency of obtaining a detection result.
With reference to the first aspect, in another possible implementation of the first aspect, the apparatus further includes at least one of a second optical device, a third optical device, and a fourth optical device.
The second optical device is disposed between the laser transmitter and the beam optical splitter; the third optical device is disposed between the laser detector and the beam optical splitter; and/or the fourth optical device is disposed between the image detector and the beam optical splitter.
With reference to the first aspect, in another possible implementation of the first aspect, the apparatus further includes a fifth optical device and a sixth optical device.
The fifth optical device is disposed between the first beam optical splitter and the second beam optical splitter.
The sixth optical device is disposed between the image detector and the second beam optical splitter.
According to a second aspect, an embodiment of the present disclosure discloses a control method, including: controlling a laser transmitter to emit a first laser signal, where the first laser signal is transmitted to a detection area through a beam optical splitter; controlling a laser detector to receive a first signal from the detection area, where the first signal is provided for the laser detector through the beam optical splitter, and the first signal includes a reflected signal corresponding to the first laser signal; and controlling an image detector to receive a second signal from the detection area, where the second signal is provided for the image detector through the beam optical splitter.
It can be learned that, according to the foregoing control method, a detection apparatus can be controlled to detect the detection area, so that both laser detection and image detection can be performed on the detection area. The detection apparatus includes the laser transmitter, the laser detector, and the image detector, and the device has high compactness and good stability. Further, in this embodiment of the present disclosure, optical paths of the laser signal, the first signal, and the second signal can be coaxial through the beam optical splitter. The image detector can correspondingly receive a laser signal reflected by the detection area irradiated by the laser transmitter, thereby improving accuracy of a detection result. In addition, the image detector and the laser detector synchronously obtain point cloud information and image information of the detection area, to facilitate subsequent fusion of the point cloud information and the image information without performing complex calibration and registration, thereby reducing computational complexity, and improving resolution of the detection result.
With reference to the second aspect, in a possible implementation of the second aspect, the method further includes: obtaining the point cloud information of the detection area based on first detection data from the laser detector; obtaining the image information of the detection area based on second detection data from the image detector; and fusing the point cloud information and the image information to obtain the detection result of the detection area.
Optionally, the detection result of the detection area may be a high-resolution fused image of the detection area, or a 3D image of the detection area. Based on the detection result, a distance of an object in the detection area may be determined, and object recognition, analysis, and the like may be performed by using a high-resolution pixel. It should be noted that “from” herein is not limited to “directly from” or “indirectly from”. A person skilled in the art may understand that detection data used to obtain the point cloud information and the image information is from a corresponding detection unit.
With reference to the second aspect, in another possible implementation of the second aspect, the beam optical splitter specifically includes a first beam optical splitter and a second beam optical splitter.
The first laser signal is specifically transmitted to the detection area through the first beam optical splitter and the second beam optical splitter.
An optical signal from the detection area is split into the first signal and the second signal through the second beam optical splitter, the first signal is provided for the laser detector through the first beam optical splitter, and the second signal is provided for the image detector.
With reference to the second aspect, in another possible implementation of the second aspect, the first beam optical splitter includes a polarization beam splitter or a semi-transmissive semi-reflective beam splitter.
With reference to the second aspect, in another possible implementation of the second aspect, a wavelength range of the first laser signal falls inside a first wavelength range, the first signal includes an optical signal whose wavelength range falls inside the first wavelength range, and the second signal includes an optical signal whose wavelength range falls inside a second wavelength range. Further, the first wavelength range is different from the second wavelength range.
With reference to the second aspect, in another possible implementation of the second aspect, the second beam optical splitter includes a dichroic beam splitter or a semi-transmissive semi-reflective beam splitter.
With reference to the second aspect, in another possible implementation of the second aspect, the laser transmitter includes a flash array light source.
With reference to the second aspect, in another possible implementation of the second aspect, the first beam optical splitter includes a polarization beam splitter and a wave plate, and the wave plate is disposed between the polarization beam splitter and the second beam optical splitter. Optionally, the wave plate may be a quarter-wave plate, an eighth-wave plate, or the like.
With reference to the second aspect, in another possible implementation of the second aspect, the laser transmitter is configured with a first polarization film coating, and a polarization direction of a laser signal passing through the polarization film coating is a first polarization direction.
With reference to the second aspect, in another possible implementation of the second aspect, the laser transmitter is specifically a flash array light source configured with the first polarization film coating, and the polarization direction of the laser signal passing through the polarization film coating is the first polarization direction.
With reference to the second aspect, in another possible implementation of the second aspect, the laser detector is configured with a second polarization film coating, and the second polarization film coating is configured to transmit a signal whose polarization direction is a preset polarization direction in the first signal.
With reference to the second aspect, in another possible implementation of the second aspect, the image detector includes at least one of a color camera, a grayscale camera, or a multidimensional camera, and the multidimensional camera includes at least one of a grayscale polarization camera, a color polarization camera, and a multispectral polarization camera.
With reference to the second aspect, in another possible implementation of the second aspect, the apparatus further includes an optical device, and the optical device includes at least one of a lens, a filter, a beam shaper, and a light uniformizer.
With reference to the second aspect, in another possible implementation of the second aspect, the apparatus further includes a first optical device, and the first optical device is disposed between the beam optical splitter and the detection area. Optionally, the first optical device may be a wide-spectrum optical device.
With reference to the second aspect, in another possible implementation of the second aspect, the apparatus further includes at least one of a second optical device, a third optical device, and a fourth optical device.
The second optical device is disposed between the laser transmitter and the beam optical splitter; the third optical device is disposed between the laser detector and the beam optical splitter; and/or the fourth optical device is disposed between the image detector and the beam optical splitter.
With reference to the second aspect, in another possible implementation of the second aspect, the apparatus further includes a fifth optical device and a sixth optical device.
The fifth optical device is disposed between the first beam optical splitter and the second beam optical splitter.
The sixth optical device is disposed between the image detector and the second beam optical splitter.
According to a third aspect, an embodiment of the present disclosure provides a control apparatus, including a control unit. Optionally, the control apparatus further includes a processing unit. The control apparatus is configured to implement the method described in any one of the second aspect or the possible implementations of the second aspect. There may be one or more control units and processing units.
In a possible implementation, the control unit is configured to control a laser transmitter to emit a first laser signal. The first laser signal is transmitted to a detection area through a beam optical splitter.
The control unit is further configured to control a laser detector to receive a first signal from the detection area. The first signal is provided for the laser detector through the beam optical splitter, and the first signal includes a reflected signal corresponding to the first laser signal.
The control unit is further configured to control an image detector to receive a second signal from the detection area. The second signal is provided for the image detector through the beam optical splitter.
With reference to the third aspect, in a possible implementation of the third aspect, the processing unit is configured to: obtain point cloud information of the detection area based on first detection data from the laser detector; obtain image information of the detection area based on second detection data from the image detector; and fuse the point cloud information and the image information to obtain a detection result of the detection area.
According to a fourth aspect, an embodiment of the present disclosure discloses a control apparatus, including at least one processor and a communications interface. The communications interface is configured to provide an input and/or output for the at least one processor, and the processor is configured to execute a computer program to implement the method described in any one of the second aspect or the possible implementations of the second aspect.
According to a fifth aspect, an embodiment of the present disclosure discloses a fusion detection system. The fusion detection system includes a detection apparatus and a processing apparatus. The detection apparatus includes the apparatus described in any one of the first aspect or the possible implementations of the first aspect, and the processing apparatus is configured to execute a computer program to fuse first detection data and second detection data, to obtain a detection result of a detection area.
The first detection data comes from a laser detector of the detection apparatus, and the second detection data comes from an image detector of the detection apparatus.
According to a sixth aspect, an embodiment of the present disclosure discloses a terminal, and the terminal includes the detection apparatus described in any one of the first aspect or the possible implementations of the first aspect.
In a possible implementation of the sixth aspect, the terminal may be a transportation tool or an intelligent terminal such as a vehicle, an uncrewed aerial vehicle, a roadside unit, an intersection radar, or a robot.
According to a seventh aspect, an embodiment of the present disclosure discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is run on one or more processors, the method described in any one of the second aspect or the possible implementations of the second aspect is implemented.
According to an eighth aspect, an embodiment of the present disclosure discloses a computer program product. When the computer program product runs on one or more processors, the method described in any one of the second aspect or the possible implementations of the second aspect is implemented.
It should be noted that some possible implementations of the second aspect and the third aspect of the present disclosure have a same concept as some implementations of the first aspect. For beneficial effects brought by the possible implementations of the second aspect and the third aspect, refer to the beneficial effects of the first aspect. Therefore, details are not described again.
The following describes accompanying drawings used in embodiments of the present disclosure.
The following describes embodiments of the present disclosure with reference to accompanying drawings in embodiments of the present disclosure. It should be noted that in the present disclosure, a word such as “example” or “for example” is used to represent giving an example, an illustration, or a description. Any embodiment or design solution described by using “example” or “for example” in the present disclosure shall not be construed as being more preferred or more advantageous than another embodiment or design solution. Exactly, use of the word “example” or “for example” is intended to present a related concept in a specific manner.
In embodiments of the present disclosure, “at least one” means one or more, and “a plurality of” means two or more. “At least one of the following items (pieces)” or a similar expression thereof means any combination of these items, including any combination of one item (piece) or a plurality of items (pieces). For example, at least one item (piece) of a, b, or c may represent a, b, c, (a and b), (a and c), (b and c), or (a, b, and c), where a, b, and c may be singular or plural. The term “and/or” describes an association relationship between associated objects and represents that three relationships may exist. For example, A and/or B may represent the following three cases: Only A exists, both A and B exist, and only B exists, where A and B may be singular or plural. The character “/” generally represents an “or” relationship between the associated objects.
In addition, unless otherwise stated, in embodiments of the present disclosure, ordinal numbers such as “first” and “second” are used to distinguish between a plurality of objects, and are not intended to limit an order, a time sequence, a priority, or a degree of importance of the plurality of objects. For example, a first beam optical splitter and a second beam optical splitter are merely intended to distinguish between different beam optical splitters, but do not indicate different structures, detection principles, importance degrees, and the like of the two apparatuses.
For ease of understanding, the following first explains some terms in embodiments of the present disclosure.
1. Detection Apparatus
The detection apparatus mentioned in the present disclosure may be a lidar, or may be another optical detection apparatus, for example, a fusion detection apparatus. An operating principle thereof is to detect a corresponding target object by emitting an optical signal and receiving an optical signal from the target object.
The detection apparatus in embodiments of the present disclosure can be applied to various fields such as intelligent transportation, autonomous driving, atmospheric environment monitoring, geographic mapping, and uncrewed aerial vehicle, and can complete one or more functions of target detection, distance measurement, speed measurement, target tracking, imaging recognition, and the like.
The detection apparatus in embodiments of the present disclosure can be applied to a roadside detection apparatus (for example, an intersection fusion detection apparatus), an in-vehicle detection apparatus (for example, an in-vehicle fusion detection apparatus), or the like, but can also be applied to another detection apparatus, for example, a detection apparatus installed on an apparatus such as an uncrewed aerial vehicle, a robot, a railcar, a bicycle, a signal light, a speed measurement apparatus, or a base station. An installation location of the detection apparatus is not limited in the present disclosure.
2. Polarization
Polarized light means that a vibration direction of an electric vector of light has a specific rule. Polarization is an inherent attribute of light, and a polarization state is a parameter for the light. The polarization state may be classified into linear polarization and elliptic polarization (circular polarization in a special case). When the polarized light is linearly polarized light, the polarization state further includes an angle and an amplitude. When the polarized light is circularly polarized light, the polarization state further includes a polarization direction (clockwise or counterclockwise). For example, an electric vector of the linearly polarized light vibrates back and forth in one direction. However, an electric vector of non-polarized light such as natural light vibrates disorderly and in no same direction, and there is no fixed time correspondence (no fixed phase) when the non-polarized light vibrates. Therefore, the vibration of the non-polarized light is random and has no fixed rule. To describe a relationship between vibration directions, basic linearly polarized light (the linearly polarized light may be combined into elliptically polarized light) is usually distinguished by using P-polarized light and S-polarized light. The P-polarized light represents linearly polarized light whose vibration direction is parallel to a plane of incidence, and the S-polarized light represents linearly polarized light whose vibration direction is perpendicular to the plane of incidence.
The following embodiments of the present disclosure provide a detection apparatus, a control method, a fusion detection system, and a terminal, to improve accuracy of a detection result and reduce computational complexity.
An example of the detection apparatus provided in the following embodiments of the present disclosure is first described.
(1) The laser transmitter 101 is configured to generate a laser signal. Specifically, the laser transmitter may include one or more laser transmitters. The laser transmitter may be a light emitting device such as a laser diode (LD), a vertical-cavity surface-emitting laser (VCSEL), a distributed feedback laser diode (DFB-LD), a distributed Bragg reflector laser diode (DBR-LD), a grating coupled sampling reflection laser diode (GCSR-LD), or a micro-opto-electro-mechanical system laser diode (MOEMS-LD).
Optionally, the laser transmitter 101 is a flash light source, and the flash light source can irradiate one field of view at a time, and can achieve an effect of surface irradiation.
Optionally, the laser transmitter 101 may be an array light source including a plurality of laser transmitters. The array may be an array with any quantity of rows and any quantity of columns, for example, an array of specifications such as a 1×2 array, a 2×2 array, and a 3×3 array.
Optionally, the laser transmitter 101 may alternatively include one laser transmitter. For example, refer to
In a possible implementation, the detection apparatus may further adjust, by using a beam shaper or the like, an optical signal emitted by the laser transmitter 101. For example, refer to (b) in
(2) The beam optical splitter 104 is configured to transmit a first laser signal emitted by the laser transmitter 101 to the detection area. As shown in
The target object 105 may reflect an optical signal of the laser transmitter, and may reflect an optical signal of another light source, or may generate an optical signal (for example, the target object includes a vehicle lamp, and the vehicle lamp may generate an optical signal). The detection apparatus 10 may correspondingly receive a signal from the target object 105.
Correspondingly, in the optical signal received by the detection apparatus 10 from the target object 105 in the detection area, a part of the optical signals is reflected signals of the laser signal, and a part of the optical signals are related signals of an ambient light source (for example, may be optical signals directly from the ambient light source, or may be reflected signals obtained by reflecting the optical signals of the ambient light source). The ambient light source may include one or more of a natural light source (for example, the sun) and an artificial light source (for example, a street lamp or a vehicle lamp).
As shown in
The first signal includes a reflected signal corresponding to the first laser signal, for example, a reflected signal obtained after the first target object reflects the first laser signal emitted by the laser transmitter 101. The reflected signal (or a part of the emitted signal) may be included in the first signal, and is provided for the laser detector 102 through the beam optical splitter 104.
The beam optical splitter 104 is further configured to provide a second signal from the detection area for the image detector 103. A part or all of the second signal may come from the first target object in the detection area.
(3) The laser detector 102 is an optoelectronic device configured to receive an optical signal, and can obtain an electrical signal based on the optical signal. Optionally, the laser detector 102 may obtain first detection data based on the electrical signal. For example, the first detection data may include the electrical signal or further data obtained after the electrical signal is processed. Further optionally, the first detection data is point cloud information, or point cloud information may be obtained by processing the first detection data.
The point cloud information may be used to obtain information such as a distance, a speed, a reflectivity, and reflection intensity of the detection area. Further optionally, the first detection data may be output to a controller and/or a processor (not shown in the figure), and the controller or the processor may be inside the detection apparatus 10 or outside the detection apparatus.
In a possible design, the laser detector 102 may be a detection device array arranged in rows and columns. For example, refer to
Optionally, based on different detection devices, the laser detector 102 may be specifically one of a semiconductor avalanche photodiode (APD) array, a single-photon avalanche diode (SPAD) array, or an electron multiplying charge-coupled device (EMCCD) array. According to an arrangement rule of a detection unit, the detection unit may be an array of specifications such as a 1×2 array, a 2×3 array, and a 3×3 array. This is not limited in the present disclosure.
The present disclosure provides two example designs:
Design 1: The laser detector 102 may respond to reflected light of the first laser signal, and may further respond to the related signal of the ambient light source.
Design 2: The laser detector 102 may respond to reflected light of the first laser signal, but does not respond to the related signal of the ambient light source. Alternatively, the laser detector 102 is sensitive to an optical signal in a wavelength range of the first laser signal, and is insensitive to an optical signal in another wavelength range. In this case, it may be considered that the laser detector 102 reduces impact of the optical signal in another wavelength range on reflected light of the first laser signal, thereby improving signal validity. For example, the first laser signal is infrared light. The laser detector may be sensitive to the infrared light, but is insensitive to non-infrared light (for example, visible light or ultraviolet light). When receiving an optical signal from the detection area, the laser detector may respond to the infrared light (including the reflected light of the first laser signal), but does not respond to the non-infrared light (or an electrical signal obtained through response is weak). Therefore, impact of the non-infrared light on the reflected light of the first laser signal is reduced.
(4) The image detector 103 is an array including a plurality of photosensitive devices, and obtains second detection data by converting an optical signal into an electrical signal. Specifically, the image detector 103 may obtain pixel information (or an electrical signal) based on the optical signal from the detection area, and the second detection data may include the pixel information or further data obtained after the pixel information is processed. Optionally, the second detection data includes image information, or image information may be obtained by processing the second detection data. The image information may be used to obtain a color, a pattern, a point of interest, or the like of the detection area, or may be used to analyze and identify the detection area. Further optionally, the second detection data may be output to the controller (for example, the controller 106) and/or the processor (not shown in the figure), and the controller or the processor may be inside the detection apparatus 10 or outside the detection apparatus.
The photosensitive device may include at least one of a charge-coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS), a live metal-oxide-semiconductor (MOS), and the like. It should be noted that, in addition to a device having a photosensitive effect, the image detector 103 may further include another optical device to achieve higher image detection effect. For example, the image detector 103 may further include at least one of the following: a micro lens, a color filter (for example, a red, green, and blue color filter), or a low-pass filter.
In an example design of the present disclosure, the image detector 103 may respond to the reflected light of the first laser signal, and may further respond to the related signal of the ambient light source. In this case, it may be considered that the image detector 103 supports operating in a laser illumination mode, and the image detector 103 may be applied to a scenario without the ambient light source (certainly, may alternatively be applied to a scenario with the ambient light source).
Optionally, the image detector 103 may be specifically at least one of a color camera, a grayscale camera, an infrared camera, a multispectral camera, or a multidimensional camera. It should be noted that the camera herein does not necessarily have a camera structure in a broad sense. To describe a feature that can be used for imaging, the camera is described as a camera, and may also be referred to as an image sensor during specific implementation.
The color camera usually uses a filter array or another vision-related technology to record a color of the detection area by using information such as a wavelength and a refractive index of an optical signal. The grayscale camera can count a quantity of electrons obtained by converting photons in an optical signal, to obtain a grayscale image that reflects strength of the optical signal. The infrared camera can obtain an infrared image of the detection area through an infrared photosensitive device. In some implementations, the infrared camera can further obtain other properties of some objects in the detection area, for example, one of moisture content, temperature, chemical composition, and the like. For another example, some infrared cameras support observation of an inner structure through a cloth, a plastic sheet, or the like. Further optionally, the infrared camera may be specifically one of a short-wave infrared camera, a medium- and long-wave infrared camera, or the like.
The multispectral camera can obtain images of a plurality of bands. A conventional color camera usually classifies visible light into three reference colors: red-green-blue (RGB), and light with different spectral content may present same RGB color coordinates. Consequently, original spectral content of an optical signal is lost. A multispectral imaging technology can provide images of a plurality of wavelengths, so that accurate spectral information can be obtained.
The multidimensional camera is a camera that obtains more information about the detection area in combination with another dimension in one or more original dimensions such as a wavelength and strength of light. It may be understood that the grayscale camera can generate a grayscale pixel by using a quantity of photons, and the color camera can determine a corresponding color by using a wavelength of light. The multidimensional camera can obtain another attribute of the light, for example, one or more attributes of a polarization direction of the light, an amplitude of the light, energy density, and the like.
For example, the multidimensional camera may include a polarization camera, for example, may be at least one of a grayscale polarization camera, a color polarization camera, or a multispectral polarization camera.
Specifically, a polarization array is disposed above a photosensitive device of the polarization camera, and a polarizer in the polarization array may be configured to transmit light in a preset polarization direction, to obtain polarization information of reflected light. The preset polarization direction is related to a property of the polarizer. A polarization direction that can pass through the polarizer may be preset during manufacturing of the polarizer, or an effect of transmitting light in a polarization direction may be achieved through circuit control. For example, refer to
It may be understood that the color polarization camera is a color camera configured with a polarizer. The grayscale polarization camera is a grayscale camera configured with a polarizer. The multispectral polarization camera is a multispectral camera configured with a polarizer.
(5) Optionally, the detection apparatus 10 may further include the controller 106. The controller 106 is configured to control some or all components in the detection apparatus 10. Optionally, the controller 106 may include one or more of the following apparatuses: a processor, a field-programmable gate array (FPGA), an application processor (AP), a graphics processing unit (GPU), an image signal processor (ISP), a digital signal processor (DSP), a neural-network processing unit (NPU), or the like.
It should be understood that the controller 106 may include a plurality of sub-control units. For example, the controller 106 may include two sub-control units. One sub-control unit is configured to control the laser transmitter and the laser detector, and the other sub-control unit is configured to control the image detector.
In a design, the controller 106 may generate a control signal and send the control signal to at least one unit in the apparatus 10, to control the at least one unit. For example, the controller 106 may send a first control signal to the laser transmitter 101, and the laser transmitter 101 generates a laser signal in response to the first control signal. For another example, the controller 106 may further send a second control signal to the laser detector 102, to control the laser detector 102 to convert an optical signal into an electrical signal. Similarly, the controller 106 may further send a third control signal to the image detector 103, to control the image detector 103 to convert an optical signal into an electrical signal.
Alternatively, the controller 106 may be further connected to an external processor (or an external controller). For example, the detection apparatus 10 is a vehicle-mounted radar. The controller 106 in the detection apparatus 10 may be connected to a multi domain controller (MDC) in a vehicle. The MDC may send an instruction to the controller 106, and the controller 106 generates a control signal according to the instruction.
Further, the controller may send information and/or data to the MDC, so that the MDC performs fusion and/or decision processing.
In another possible design, the detection apparatus 10 may further include a processing apparatus (not shown in the figure). The processing apparatus may obtain the point cloud information of the detection area based on the first detection data from the laser detector 102, and/or obtain the image information of the detection area based on the second detection data from the image detector 103. Further, the processing apparatus may further obtain a detection result of the detection area based on the point cloud information and the image information.
It should be noted that, when the detection apparatus 10 includes the controller 106 and the processing apparatus, the processing apparatus and the controller 106 may be integrated or independent.
It can be learned that, in the detection apparatus shown in
In a possible design, in the detection apparatus shown in
When the center points overlap, the laser detector may correspondingly receive an optical signal returned by the area irradiated by the laser transmitter, thereby improving accuracy of a laser detection result.
If the laser detection field of view overlaps with the image detection field of view, complexity of registration and calibration during subsequent fusion can be reduced, computation consumption can be reduced, and efficiency of obtaining the detection result can be improved.
It may be understood that a structure shown in this embodiment of the present disclosure does not constitute a specific limitation on the detection apparatus 10. In some other embodiments of the present disclosure, the detection apparatus 10 may combine some units, or split some units, or have different unit arrangements, or further include another unit. The units shown in
The following describes some possible designs in the present disclosure. Each design is used to describe a possible case. A plurality of designs that are not mutually exclusive may be combined. Some combined cases are not described again in embodiments.
In a possible design, the beam optical splitter 104 in the detection apparatus 10 shown in
When receiving a signal, the second beam optical splitter splits the optical signal from the detection area into the first signal and the second signal, provides the first signal for the first beam optical splitter, and provides the second signal for the image detector. The first beam optical splitter further provides the first signal for the laser detector.
For example, refer to
Refer to
Refer to
It should be noted that positions of the laser transmitter 101 and the laser detector 102 of the detection apparatus 70 may be exchanged, and positions of other components may also be exchanged.
In the embodiment shown in
In another possible design, the first signal includes a signal whose wavelength range falls inside a first wavelength range, and the second signal includes a signal whose wavelength range falls inside a second wavelength range. A wavelength range is a band between a maximum value (upper limit) and a minimum value (lower limit) of a wavelength (optionally including the maximum value and/or the minimum value).
A width of the first wavelength range may be different from a width of the second wavelength range. For example, the first wavelength range includes wavelengths of 780 nm and above, and the second wavelength range includes wavelengths below 780 nm.
Specifically, the first wavelength range and the second wavelength range may overlap or do not overlap. For example, the first wavelength range includes wavelengths of 700 nm and above, and the second wavelength range includes wavelengths of 800 nm and below. In this case, the first wavelength range has an overlapping part between 700 nm and 800 nm.
Refer to Table 1. Table 1 shows possible light colors and wavelength ranges corresponding to the light colors provided in this embodiment of the present disclosure.
It should be noted that the wavelength range shown in Table 1 is merely an example of a possible wavelength range. For an upper limit and a lower limit of each wavelength range, refer to one or more of the following standard documents or solutions recommended by organizations: a standard document of ophthalmic optics and ophthalmic devices, a standard document of ISO 20473:2007 optics and photonics-spectral bands, an International Commission on Illumination classification system, or an astronomy classification solution.
A possible case is used as an example in the present disclosure: The laser transmitter may emit infrared light (for example, mid-infrared ray at a wavelength of 3 μm to 5 μm), the laser detector may respond to the infrared light, and the image detector may respond to visible light. The second beam optical splitter is configured to split the optical signal from the detection area into the first signal and the second signal. The first signal includes an optical signal whose wavelength range falls inside a wavelength range of the infrared light, and the second signal includes an optical signal whose wavelength range falls inside a wavelength range of the visible light. The laser detector may receive the infrared light (including reflected light obtained based on the emitted infrared light), and obtain the point cloud information based on the received infrared light. In addition, the image detector receives the visible light, and performs imaging to obtain the image information.
In still another possible design, the second beam optical splitter may be one or a combination of a dichroscope, a semi-transmissive semi-reflective beam splitter (or referred to as a semi-transmissive semi-reflective mirror), or a medium beam splitter (one of a dielectric beam splitter, a metal film beam splitter, and the like). The present disclosure provides two possible implementations.
Implementation 1: The second beam optical splitter includes a dichroscope. The dichroscope is an optical device having different reflectivity and transmittance properties for light at different wavelengths, and may also be referred to as a dichroic beam splitter, a dichroic filter, a dichroic splitter, or the like. For example, refer to
It can be learned that the dichroscope having the reflection performance shown in
It may be understood that a reflectivity and a refractive index of the dichroscope shown in
A possible implementation is as follows: The laser transmitter may emit the infrared light (for example, the mid-infrared ray at the wavelength of 3 μm to 5 μm), the laser detector may respond to the infrared light, and the image detector is an image sensor for imaging with the visible light. The second beam optical splitter (for example, the dichroscope shown in
For example, refer to
Implementation 2: The second beam optical splitter may be a semi-transmissive semi-reflective beam splitter. The semi-transmissive semi-reflective beam splitter can split, in a specific wavelength range, one beam of light into two beams of light whose spectral components are approximately the same, that is, in a specific wavelength area, for example, in a wavelength range from 300 nm to 100 μm, transmittance and reflectivity for wavelengths are the same (being the same herein are not necessarily totally the same, and there may be fluctuations within a specific range but the fluctuations are small). Generally, a neutral beam splitter in which transmitted light and reflected light respectively account for 50% is common. However, the present disclosure is also applicable to a beam splitter in another transmission proportion (reflection proportion).
In another example, a filter may be disposed between the laser detector 102 and the first beam optical splitter 1041, and the filter may filter other stray light.
For example, the detection apparatus 90 shown in
A possible case is as follows: The laser signal emitted by the laser transmitter 101 is infrared light, and the first beam optical splitter splits the optical signal from the detection area into the first signal and the second signal whose spectral components are approximately the same. The first signal includes both infrared light (the infrared light includes reflected light obtained based on the emitted infrared light) and an optical signal at another wavelength. However, light in another wavelength range in the first signal can be filtered through an optical device 901, and the infrared light is provided for the laser detector 102, so that validity of a signal received by the laser detector 102 is improved.
In this case, if the image detector includes an infrared camera, an infrared image of the detection area may be obtained.
It should be noted that the optical device 901 may be further disposed at any position between the laser detector 102 and the second beam optical splitter 1042.
In yet another possible design, the first beam optical splitter may be one or a combination of a polarization beam splitter (PBS) or a semi-transmissive semi-reflective beam splitter (or referred to as a semi-transmissive semi-reflective mirror). For ease of understanding, the present disclosure provides two possible implementations.
Implementation 1: The first beam optical splitter is a semi-transmissive semi-reflective beam splitter. For example, refer to
Further, because a part of reflected light is transmitted to the laser transmitter, the part of reflected light may interfere with laser detection. The detection apparatus may perform detection in a form of a pulse or by using a detection frame, and reduce impact of an interference signal in a time division manner. The detection apparatus may further configure a filter or an apparatus for absorbing the reflected light in a direction of the laser transmitter, to reduce the impact of the interference signal.
Implementation 2: The first beam optical splitter is a polarization beam splitter. The polarization beam splitter can split polarized incident light into two beams of polarized light for transmission.
The polarization beam splitter may perform light splitting based on a polarization state. Therefore, the polarization beam splitter can be used with a device that can change the polarization state to allow a laser transmitter and a laser detector to be coaxial. For example, a laser signal emitted by the laser transmitter is in a first polarization direction, and may be provided for a detection area after passing through the polarization beam splitter. A polarization direction of an optical signal from the detection area may be changed by using another optical device. After passing through the polarization beam splitter, the optical signal whose polarization direction is changed is provided for the laser detector, and is not transmitted to (or a small part of the optical signal is transmitted to) the laser transmitter (or most of the optical signal is provided for the laser detector, and a small part of light is absorbed or transmitted to another direction). Therefore, the laser transmitter and the laser detector can be coaxial, and a loss of an echo signal can be reduced.
Further optionally, the device that changes the polarization state may be a wave plate.
Refer to
Refer to
The wave plate is also referred to as a retarder, and can transmit light and change a polarization state of the light. The polarization state of the light is specifically changed by delaying one polarization component of a quadrature component. The wave plate may be specifically a quarter-wave plate, an eighth-wave plate, or the like, or may be formed by two eighth-wave plates. The quarter-wave plate delays a phase of one polarization component of the quadrature component by a quarter of a wavelength, and the eighth-wave plate delays a phase of one polarization component of the quadrature component by one-eighth of a wavelength.
The wave plate is usually made of a crystal material, but may also be made of another material. For example, a crystalline quartz polarization rotator is a monocrystal of quartz.
For another example, a Fresnel rhomb retarder uses internal reflection at a specific angle in a prism structure to delay incident polarized light. Each time the light is reflected, P-polarized light moves forward by λ/8, where λ is a polarized wavelength. When light is emitted from a Fresnel rhomb retarder having two surfaces for reflection, a total delay is λ/4.
It should be noted that optical paths shown in
It may be understood that, because an optical path is reversible, positions of the laser transmitter and the laser detector may be exchanged. For example, in the detection apparatus 120 shown in
In still yet another possible design, the laser emitting unit 101 may emit polarized light by configuring a polarizer or by configuring a polarization film coating. The present disclosure provides two possible implementations.
Implementation 1: A polarizer is disposed between a laser transmitter and a first beam optical splitter (PSB), and the polarizer may make a laser signal passing through the polarizer be a laser signal in a first polarization direction.
In an example, after a first laser signal emitted by a detection apparatus passes through the polarizer, a polarization direction of the first laser signal is the first polarization direction. Correspondingly, when receiving a signal, the laser detection apparatus may filter out a signal whose polarization direction is different from a second polarization direction. The second polarization direction is a polarization direction of a reflected signal of the first laser signal. Therefore, validity of a signal received by the laser detector can be improved, and a laser detection effect can be improved. The second polarization direction may be the same as or different from the first polarization direction. It should be understood that, when the second polarization direction is different from the first polarization direction, another optical device (for example, a wave plate) may be disposed in the apparatus, so that a polarization direction of an optical signal passing through the optical device is changed.
For example, refer to
Subsequently, the P-polarized light passes through the wave plate 1001 to obtain circularly polarized light, and the circularly polarized light is irradiated to a detection area. A reflected signal from the detection area includes the circularly polarized light. The circularly polarized light passes through the wave plate 1001 to obtain linearly polarized light. When the linearly polarized light passes through 1041, the linearly polarized light is reflected to obtain S-polarized light, and the S-polarized light is provided for the laser detector 102.
It should be noted that positions of the laser transmitter and the laser detector may be changed. For example, refer to
Implementation 2: The laser transmitter is specifically a flash array light source configured with a first polarization film coating, and a polarization direction of a laser signal passing through the polarization film coating is the first polarization direction.
Polarized light is emitted through the polarization film coating, so that a structure of the detection apparatus is further compact, impact caused by motion, vibration, and the like on fusion detection can be reduced, and stability can be improved.
It should be noted that this is implemented in a manner of coating on a single laser transmitter. In some other implementations, overall coating may be performed on an array light source, or the array light source may be divided into a plurality of laser transmitter groups, and coating is performed on each group.
In a further possible design, a laser detector is configured with a second polarization film coating, and the second polarization film coating is configured to transmit a signal whose polarization direction is a preset polarization direction in a first signal. The preset polarization direction is related to a property of a polarization film coating. A polarization direction of an optical signal that can pass through the polarization film coating may be preset during manufacturing or configuration of the polarization film coating, or an effect of transmitting an optical signal in a polarization direction may be achieved through circuit control.
Optionally, the polarization film coating may be implemented by coating a single detection device, or overall coating may be performed on the laser detector, or the laser detector may be divided into a plurality of detection device groups, and coating is performed on each group. The present disclosure provides two possible implementations.
Implementation 1: The polarization film coating is implemented by coating a single detection device. For example, refer to
It should be noted that the foregoing lens is merely used as a carrier of a film coating for description. During specific implementation, the polarization film coating may be configured on another carrier.
In a possible implementation process, the lens array 1603 is configured to filter light whose wavelength range is different from a wavelength range of an emitted laser signal, to improve signal validity. For example, refer to
Implementation 2: Overall coating is performed on the laser detector.
It should be noted that the foregoing lens is merely used as a carrier of a film coating for description. During specific implementation, the polarization film coating may be configured on another carrier, for example, directly plated on a surface of a receiving unit.
In some possible implementation processes, the lens 1702 may be configured to filter light whose wavelength range is different from a wavelength range of an emitted laser signal, to improve signal validity.
Alternatively, in addition to the film coating, a polarizer may be further used to achieve polarization light filtering effect. The polarization film coating layer 1703 shown in
In a still further possible design, a detection apparatus further includes an optical device, for example, includes at least one of a lens, a beam shaper, a filter, a light uniformizer, and a collimation apparatus. The present disclosure provides the following several possible implementations.
Implementation 1: The detection apparatus further includes a first optical device, and the first optical device is disposed between a beam optical splitter and a detection area. The first optical device is configured to transmit light within a third wavelength range, and the third wavelength range includes a wavelength range of a first laser signal and an operating wavelength range of an image detector. Optionally, the optical device may be specifically a lens or a filter. The operating wavelength range of the image detector is used to indicate a wavelength range of a signal to which the image detector is sensitive, or a wavelength range in which a response can be generated. For example, an operating wavelength range of a visible light camera is a wavelength range of visible light, and an operating wavelength range of a short-wave near-infrared camera is approximately 400 nm to 1700 nm (only used as an example).
For example, in a scenario in which a laser transmitter responds to infrared light, and the image detector responds to the visible light, the first optical device can pass through at least a band of the infrared light and a band of the visible light, and can be reused during transceiving.
The optical device that can be reused during transceiving is disposed, to reuse optical systems of the laser transmitter, a laser detector, and the image detector, to reduce a volume of the detection apparatus and improve compactness.
During specific implementation, the first optical device may include a wide-spectrum optical device. The wide spectrum indicates that an optical signal with a large wavelength range can pass through the optical device. In this embodiment of the present disclosure, a band through which the wide-spectrum optical device can pass includes at least a wavelength range and an operating wavelength range of an emitted signal. Further optionally, the band through which the wide-spectrum optical device can pass may further include an operating wavelength range of the laser detector.
The present disclosure provides a possible case.
It can be learned that a laser signal emitted by the laser transmitter 101 may be irradiated to the detection area through the wide-spectrum optical device 1801, and an optical signal from the detection area may be provided for the beam optical splitter 104 through the wide-spectrum optical device 1801.
Implementation 2: The detection apparatus further includes at least one of a second optical device, a third optical device, and a fourth optical device. The second optical device is disposed between the laser transmitter and the beam optical splitter; the third optical device is disposed between the laser detector and the beam optical splitter; and/or the fourth optical device is disposed between the image detector and the beam optical splitter.
The second optical device can be configured to transmit a laser signal, and the second optical device may also be referred to as an emission optical system. For example, if the laser transmitter emits infrared light, the laser transmitter may be referred to as an infrared emission optical system. Optionally, the second optical device may include a lens configured to transmit a laser signal. Alternatively, optionally, the second optical device may include a filter configured to enable an emitted laser signal to be a signal within a preset wavelength range. Alternatively, optionally, the second optical device may further include a beam shaper. For example, the second optical device may include a collimation apparatus configured to adjust a direction, a shape, and the like of a laser signal.
The third optical device is configured to receive a reflected signal obtained based on the emitted laser signal. The third optical device may also be referred to as a receiving optical system. For example, if the third optical device is configured to receive echo light returned based on emitted infrared light, the third optical device may be referred to as an infrared receiving optical system. Optionally, the third optical device may include a lens configured to transmit an optical signal. Alternatively, optionally, the third optical device includes a filter configured to filter stray light. Alternatively, optionally, the third optical device includes a light uniformizer, and the light uniformizer is configured to even a returned optical signal, to prevent energy of a first signal from being excessively concentrated in an area.
The fourth optical device is configured to transmit an optical signal required by the image detector, and the fourth optical device may also be referred to as a receiving optical system. For example, if the fourth optical device is configured to receive visible light, the fourth optical device may also be referred to as a visible light receiving optical system. Optionally, the fourth optical device may include a lens configured to transmit an optical signal. Alternatively, optionally, the fourth optical device includes a filter configured to filter stray light.
The present disclosure provides a possible case.
The second optical device 1901 can be configured to transmit a laser signal, the third optical device 1902 is configured to filter stray light in an optical signal transmitted to the laser detector 102, and the fourth optical device 1903 is configured to transmit an optical signal required by the image detector 103.
Implementation 3: The detection apparatus further includes a fifth optical device and a sixth optical device. The fifth optical device is disposed between the first beam optical splitter and the second beam optical splitter, and the sixth optical device is disposed between the image detector and the second beam optical splitter.
The fifth optical device may be configured to transmit the laser signal and/or filter the stray light, and the fifth optical device is configured to transmit the optical signal required by the image detector. It may be understood that the optical signal emitted by the laser transmitter and an optical signal required by the laser detector have a same wavelength. Therefore, the fifth optical device can be reused during laser transceiving, to reduce a volume of the detection apparatus and improve compactness. Optionally, the fifth optical device may include a lens configured to transmit the optical signal. Alternatively, optionally, the fifth optical device may include a filter configured to filter the stray light.
The sixth optical device is configured to transmit a signal required by the image detector, and the sixth optical device may also be referred to as a receiving optical system. For example, if the sixth optical device is configured to receive visible light, the sixth optical device may also be referred to as a visible light receiving optical system. Optionally, the sixth optical device may include a lens configured to transmit the optical signal. Alternatively, optionally, the sixth optical device includes a filter configured to filter the stray light. The present disclosure provides a possible case.
Optionally, positions of the fifth optical device 2001 and the wave plate 1201 in
The fifth optical device 2001 may be configured to transmit the laser signal and/or filter the stray light, and the sixth optical device 2002 is configured to transmit the optical signal required by the image detector 103.
Step S2101: Control a laser transmitter to emit a first laser signal.
Specifically, step S2101 may be implemented by using a control apparatus. Optionally, the control apparatus may be included in the foregoing detection apparatus, or may be deployed outside the detection apparatus. For example, the control apparatus may be the controller 106 in the detection apparatus 10 shown in
The control apparatus may include one or more of the following apparatuses: a controller, a processor, a FPGA, an AP, a modem processor, a GPU, an ISP, a DSP, a NPU, or the like. Optionally, the laser transmitter is a flash light source, and the flash light source can irradiate one field of view at a time, and can achieve an effect of surface irradiation. Optionally, the laser transmitter may be an array light source including a plurality of laser transmitters, or may be a flood light source including one laser transmitter.
The controller may send a first control signal to the laser transmitter, and the laser transmitter generates a laser signal in response to the first control signal.
Step S2102: Control a laser detector to receive a first signal from a detection area.
The first signal is provided for the laser detector through a beam optical splitter, and the first signal includes a reflected signal corresponding to the first laser signal.
Specifically, the beam optical splitter may provide the first signal from the detection area (which may be specifically from a first target object in the detection area) for the laser detector. The first signal includes the reflected signal corresponding to the first laser signal.
Optionally, the beam optical splitter may include a first beam optical splitter and a second beam optical splitter. The second beam optical splitter is configured to split an optical signal from the detection area into the first signal and a second signal. The first beam optical splitter is configured to provide the first signal for the laser detector.
In a possible design, the second beam optical splitter may be one or a combination of a dichroscope, a semi-transmissive semi-reflective beam splitter (or referred to as a semi-transmissive semi-reflective mirror), or a medium beam splitter (one of a dielectric beam splitter, a metal film beam splitter, and the like).
In yet another possible design, the first beam optical splitter may be one or a combination of a PBS or a semi-transmissive semi-reflective beam splitter (or referred to as a semi-transmissive semi-reflective mirror). Further optionally, the beam optical splitter further includes a wave plate configured to change a polarization direction of an optical signal.
Specifically, step S2102 may be implemented by using the control apparatus. Optionally, the control apparatus may control the laser detector by using a control signal. For example, the control apparatus generates a second control signal. The second control signal is used by the laser detector to start or stop receiving the first signal from the detection area. Further, the control apparatus may further generate threshold indication information or the like, to indicate a threshold of a signal to which the laser detector can generate a response.
In some embodiments, a polarizer on the laser detector may transmit, through circuit control, an optical signal in a preset polarization direction. In this case, the control apparatus may generate a control signal to indicate the preset polarization direction.
Step S2103: Control an image detector to receive the second signal from the detection area.
The second signal is provided for the image detector through the beam optical splitter.
Optionally, the beam optical splitter provides the second signal for the image detector and provides the first signal for the laser detector. The first signal and the second signal are coaxial.
Specifically, step S2103 may be implemented by using the control apparatus. The control apparatus may control the image detector by using a control signal. For example, the control apparatus generates a third control signal. The third control signal is used by the image detector to start or stop receiving the second signal from the detection area.
In some embodiments, a polarizer on the image detector may transmit, through circuit control, the optical signal in the preset polarization direction. In this case, the control apparatus may further generate a control signal to indicate the preset polarization direction.
Optionally, the control method shown in
Step S2104: Obtain a detection result of the detection area based on first detection data from the laser detector and second detection data from the image detector.
Specifically, step S2104 may be performed by the control apparatus, or may be performed by another control apparatus different from the control apparatus, for example, an external processing apparatus. For example, steps S2101 to S2103 may be performed by a processor in the detection apparatus, and step S2104 may be performed by a processor in a terminal in which the detection apparatus is located, for example, the MDC in the vehicle.
Optionally, when the control apparatus is included in the detection apparatus, the control apparatus, the laser transmitter, the image detector, and the laser detector may be interconnected through a bus or interconnected in a wireless manner, so that the control apparatus can receive the first detection data and the second detection data.
Alternatively, the control apparatus may be integrated into the laser detection unit. In this case, the control apparatus receives the first detection data from the laser detector, and obtains the detection result of the detection area based on the first detection data and the second detection data from the image detector.
Similarly, the control apparatus may also be integrated into the image detector.
Specifically, the laser detector may obtain an electrical signal based on the first signal. The first detection data may include the electrical signal or further data obtained after the electrical signal is processed.
The image detector may obtain pixel information (or an electrical signal) based on the second signal. The second detection data may include the pixel information (or the electrical signal) or further data obtained after the pixel information (or the electrical signal) is processed.
The control apparatus fuses the first detection data and the second detection data to obtain the detection result of the detection area. In a possible design, the control apparatus may obtain point cloud information of the detection area based on the first detection data, obtain image information of the detection area based on the second detection data, and obtain the detection result of the detection area based on the point cloud information and the image information.
The detection result of the detection area may be a high-resolution fused image of the detection area, or a three-dimensional (3-dimensional, 3D) image of the detection area. Based on the detection result, a distance of an object in the detection area may be determined, and object recognition, analysis, and the like may be performed by using a high-resolution pixel.
For example, the first signal and the second signal are specifically from the first target object in the detection area, and the detection result may be used to describe one or more pieces of information of a color, a distance relative to the detection apparatus, a speed, a contour, a pattern, or the like of the first target object. The detection result may be further used to identify or analyze the first target object.
It can be learned that, according to the control method shown in
The foregoing describes in detail the method in this embodiment of the present disclosure. The following provides possible control apparatuses in embodiments of the present disclosure.
It should be noted that for implementation of each unit, refer to the corresponding description in the method embodiment shown in
In a possible implementation, the control unit 2201 is configured to control a laser transmitter to emit a first laser signal. The first laser signal is transmitted to a detection area through a beam optical splitter.
The control unit 2201 is further configured to control a laser detector to receive a first signal from the detection area. The first signal is provided for the laser detector through the beam optical splitter, and the first signal includes a reflected signal corresponding to the first laser signal.
The control unit 2201 is further configured to control an image detector to perform imaging by using a second signal from the detection area. The second signal is provided for the image detector through the beam optical splitter.
In a possible implementation, the processing unit 2202 is configured to: obtain point cloud information of the detection area based on first detection data from the laser detector; obtain image information of the detection area based on second detection data from the image detector; and fuse the point cloud information and the image information to obtain a detection result of the detection area.
For specific implementation of the foregoing units, refer to detailed descriptions in the method embodiment shown in
It should be understood that, in the control apparatus 220, division into a plurality of units or modules is merely logical division based on functions, and is not intended to limit a specific structure of the apparatus. During specific implementation, some function modules may be subdivided into more fine function modules, and some function modules may be combined into one function module. However, regardless of whether the function modules are subdivided or combined, general procedures in a process of implementing the control method are the same. For example, the control unit in the control apparatus 220 may be subdivided into a first control unit and a second control unit. The first control unit is configured to control the laser transmitter and the laser detector, and the second control unit is configured to control the image detector. Usually, each unit corresponds to program code (or a program instruction). When the program code corresponding to the unit is run on at least one processor, the unit executes a corresponding procedure to implement a corresponding function.
Optionally, the control apparatus 230 may further include a bus 2304. One or more of the at least one processor 2301, the communications interface 2302, and the at least one memory 2303 may be connected through the bus 2304.
The processor 2301 is a module that performs an arithmetic operation and/or a logic operation, and may be specifically one or a combination of an analog-to-digital converter (ADC), a DSP, a central processing unit (CPU), a GPU, a microprocessor unit (MPU), an application-specific integrated circuit (ASIC), an FPGA, a complex programmable logic device (CPLD), a coprocessor (assisting the central processing unit in completing corresponding processing and application), and a microcontroller unit (MCU).
The communications interface 2302 may be configured to provide an information input or output for the at least one processor. Alternatively, the communications interface 2302 may be configured to receive data sent from the outside and/or send data to the outside, and may be a wired link interface including an Ethernet cable or the like, or may be a wireless link (Wi-Fi, Bluetooth, universal wireless transmission, an in-vehicle short-distance communications technology, or the like) interface. Optionally, the communications interface 2302 may further include a transmitter (for example, a radio frequency transmitter or an antenna), a receiver, or the like coupled to the interface.
The memory 2303 is configured to provide storage space, and the storage space may store data such as an operating system and a computer program. The memory 2303 may be one or a combination of a cache, a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), or the like. The RAM may also be one or a combination of a static random-access memory (SRAM), a dynamic random-access memory (DRAM), and the like.
The processor 2301 in the control apparatus 230 is configured to read the computer program stored in the memory, and is configured to perform the foregoing control method, for example, one or more steps of the detection method described in
An embodiment of the present disclosure further provides a fusion detection system. The fusion detection system includes a detection apparatus and a processing apparatus. The detection apparatus may be the detection apparatus described in the embodiment of
An embodiment of the present disclosure further provides a terminal. The terminal includes the foregoing detection apparatus, for example, the detection apparatus shown in
Optionally, the terminal may be a transportation tool or an intelligent terminal such as a vehicle, an uncrewed aerial vehicle, a roadside unit, an intersection radar, or a robot.
An embodiment of the present disclosure further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is run on one or more processors, the method in the embodiment shown in
An embodiment of the present disclosure further provides a computer program product. When the computer program product runs on one or more processors, the method in the embodiment shown in
An embodiment of the present disclosure further provides a chip system. The chip system includes a communications interface and at least one processor. The communications interface is configured to provide an information input/output for the at least one processor. The processor is configured to invoke computer instructions from the communications interface, to implement the method in the embodiment shown in
According to the context, the term “when” used in the foregoing embodiments may be interpreted as a meaning of “if”, “after”, “in response to determining”, or “in response to detecting”.
All or some of the foregoing embodiments may be implemented by using software, hardware, firmware, or any combination thereof. When software is used to implement embodiments, all or some of the embodiments may be implemented in a form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the procedure or functions according to embodiments of the present disclosure are all or partially generated. The computer may be a general-purpose computer, a dedicated computer, a computer network, or another programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or may be transmitted from a computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a web site, computer, server, or data center to another website, computer, server, or data center in a wired (for example, a coaxial cable, an optical fiber, or a digital subscriber line) or wireless (for example, infrared, radio, or microwave) manner. The computer-readable storage medium may be any usable medium accessible by a computer, or a data storage device, such as a server or a data center, integrating one or more usable media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, or a magnetic tape), an optical medium (for example, a digital video disc (DVD)), a semiconductor medium (for example, a solid-state drive), or the like.
A person of ordinary skill in the art may understand that all or some of the procedures of the methods in the embodiments may be implemented by a computer program instructing related hardware. The program may be stored in the computer-readable storage medium. When the program is executed, the procedures in the method embodiments may be included. The foregoing storage medium includes any medium that can store program code, such as a ROM, a RAM, a magnetic disk, or an optical disc.
This application is a continuation of International Patent Application No. PCT/CN2021/074937, filed on Feb. 2, 2021, the disclosure of which is hereby incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/CN2021/074937 | Feb 2021 | US |
| Child | 18364222 | US |
