Patent Application
20200264312
The present invention is generally related to the field of light ranging systems as used in vehicles.
In the present invention vehicles comprising a light ranging system are considered whereon one or more sensors are mounted.
In optical sensor applications the range and performance of the sensing system are often limited by the power consumption of the illumination technology, e.g. LED or VCSEL (Vertical Cavity Surface Emitting Laser). The range and performance may further also be affected by eye safety considerations, which impose an average illumination level.
In some cases, a sensor may be used for more than one sensing function (sensor fusion) and the sensor then operates at a performance level high enough to fulfil the different function requirements.
KR101848312 reveals a sensor fusion system for an autonomous emergency braking (AEB) system of a vehicle, comprising a camera module to photograph surroundings of the vehicle; a light detection and ranging (LiDAR) module scanning the surroundings of the vehicle to recognize an object and to detect a distance from the recognized object; a camera module fault monitoring unit to monitor whether or not the camera module is normally operated, a LiDAR module fault monitoring unit to monitor operation of the LiDAR module and a fault monitoring unit receiving a signal from the camera module fault monitoring unit and the LiDAR module fault monitoring unit to monitor the overall operation of a sensor fusion system, and outputting result data thereof to the outside including an AEB system. The proposed system enhances driving safety of an autonomous driving vehicle including the AEB system.
There is a need in the art for solutions allowing improved control of the light ranging system of a vehicle.
It is an object of embodiments of the present invention to provide for a method for controlling the configuration of a light ranging system of a vehicle in a deterministic way based on the available information.
The above objective is accomplished by the solution according to the present invention.
In a first aspect the invention relates to a method for controlling a light ranging system mounted on a vehicle, said vehicle provided with receiving means for receiving external data and a plurality of sensors, said light ranging system arranged to be configured by a plurality of configurable parameters and comprising storage means for storing a plurality of predetermined operational schemes. The method comprises:
The proposed solution indeed allows for deterministic control of the light ranging system configuration. From the collected information an operational scheme is selected. The choice of operational scheme is communicated to the light ranging system that comprises a memory wherein for each operational scheme a table of parameter settings is provided. The light ranging system is arranged for converting the parameter settings into actual control instructions to steer the operation of the light ranging system.
In a preferred embodiment the configurable parameters comprise one or more of {illumination power, field of view, wavelength, number of active light emitting elements, illumination location, illumination frame rate}.
Preferably one or more sensors of the plurality are arranged for sensing one or more of {vehicle speed, steering angle, acceleration, tyre pressure, vehicle battery voltage, battery charge level}. Advantageously, one or more sensors of the plurality are arranged for sensing one or more of {a weather condition, a light condition}.
In one embodiment the receiving means is arranged for receiving traffic information and/or location information, wherein the traffic information and/or location information is considered when determining the operational scheme.
In a preferred embodiment each operational scheme mode corresponds to a different parameter table. In other embodiments at least two operational schemes modes of said plurality correspond to a same parameter table. In certain embodiments at least one parameter of the parameter table is calculated on the fly.
In a typical embodiment the light ranging system is a LIDAR system or a Time-of-Flight system.
In embodiments the data is collected each time a cycle of measurements relating to said vehicle's current operation has been performed.
Advantageously, the plurality of predetermined operational schemes has been derived from operational data stored in said vehicle.
In one embodiment the operational scheme also sets parameters of sensors not related to said light ranging system. It may be related for example to vehicle speed.
In one aspect the invention relates to a program, executable on a programmable device containing instructions, which when executed, perform the method as in previously described.
In another aspect the invention relates to a vehicle comprising
In a preferred embodiment the light ranging system of the vehicle comprises a control unit arranged for compiling the one or more control instructions.
For purposes of summarizing the invention and the advantages achieved over the prior art, certain objects and advantages of the invention have been described herein above. Of course, it is to be understood that not necessarily all such objects or advantages may be achieved in accordance with any particular embodiment of the invention. Thus, for example, those skilled in the art will recognize that the invention may be embodied or carried out in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other objects or advantages as may be taught or suggested herein.
The above and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.
The invention will now be described further, by way of example, with reference to the accompanying drawings, wherein like reference numerals refer to like elements in the various figures.
The present invention will be described with respect to particular embodiments and with reference to certain drawings but the invention is not limited thereto but only by the claims.
Furthermore, the terms first, second and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a sequence, either temporally, spatially, in ranking or in any other manner. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the invention described herein are capable of operation in other sequences than described or illustrated herein.
It is to be noticed that the term “comprising”, used in the claims, should not be interpreted as being restricted to the means listed thereafter; it does not exclude other elements or steps. It is thus to be interpreted as specifying the presence of the stated features, integers, steps or components as referred to, but does not preclude the presence or addition of one or more other features, integers, steps or components, or groups thereof. Thus, the scope of the expression “a device comprising means A and B” should not be limited to devices consisting only of components A and B. It means that with respect to the present invention, the only relevant components of the device are A and B.
Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment, but may. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments.
Similarly it should be appreciated that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Furthermore, while some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention, and form different embodiments, as would be understood by those in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.
It should be noted that the use of particular terminology when describing certain features or aspects of the invention should not be taken to imply that the terminology is being re-defined herein to be restricted to include any specific characteristics of the features or aspects of the invention with which that terminology is associated.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Apart from these sensors in the light ranging system, the vehicle comprises other sensors communicating with the vehicle control unit (12), either one-way or two-way. These other sensors can for instance relate to conditions surrounding the vehicle, like e.g. a temperature sensor or a rain-light sensor to detect the amount of rain and the amount of light, or as in the case of the light ranging system (100), objects in the field-of-view of the system. Other sensor systems may relate to vehicle speed, acceleration, pressure of the tyres etc. that can be measured within the vehicle. It is also possible to receive data from outside the car, for instance traffic information through e.g. vehicle-to-vehicle information or GPS/GNSS data to calculate the location.
All the sensor data is collected from the sensors in the vehicle control unit (12), where they are processed and may be used to indicate e.g. to the driver the conditions ahead or to indicate if the vehicle is not functioning correctly using measurements coming from meters. It is also possible to use the information for instance to control the operation of the motor e.g. by shutting it down in case certain conditions are met. These types of sensors and the use of their information are commonly known in the art. The vehicle control unit may apply central computing or distributed computing.
In operation, the light source (112) emits pulses or modulated light into the scene in its Field of View (FOV). The FOV is defined by the light source and its relation to optics (130). The light source may comprise one or more light emitting devices, e.g. LED or VCSEL. Different parameters of the light source can be controlled as well as the number of active light emitting devices. Thereby it is possible to control e.g. illumination power or wavelength, or the light pattern and thereby the FOV.
In case there is an object in the FOV, the emitted light is partially reflected back. This is shown in
As it can be seen, there are several parameters that control the operation of the light ranging system (100). These parameters regulate among other things the illumination power, illumination Field of View, illumination location, illumination wavelength, illumination frame rate or other pixel settings. It is possible to set parameters relating to the light source side of the system (i.e. relating to the illumination configuration) and/or relating to the pixels at the detector side of the system. Concerning the pixel configuration distinction can be made between settings that are common for the whole array of pixels, e.g. frame rate, integration time, and settings relating to only part of the array, e.g. to apply pixel binning (i.e. combining of pixels) to implement some high dynamic range strategies relating to pixel by pixel operation.
The parameters and the way they are controlled are device type specific, i.e. depending on the type of light ranging system; the control signals are specific to that system.
The light ranging system is controlled by a control block (TOF CTRL 120), as also shown in
The control instruction signals are derived in the control block from deterministic signals according to a set of parameters stored in the memory (204) and corresponding to a schedule selected from a set of predetermined schedules. In this way the configuration of the light ranging system is only changed in a deterministic way based on predetermined vehicle and vehicle environment information which has been validated to perform as such. So a system always works in “the same” way (i.e. its operation is predictable) and can therefore be validated to meet safety system requirements while still optimizing operation of the system within the many different constraints and utilizing the many parameters/variables in the control system.
The predetermined schedules each describe an operational scheme, indicated in
The selection of said schedule among the various possible schedules is in the present invention performed in the vehicle control unit (12) based on the input data collected from the sensors and the light ranging system output and from the receiving means of the vehicle. The VCU is thus capable of determining which of the available schedules fits best with the current conditions wherein the light ranging system is operating. Selection of the schedule may be based on information representative of the vehicle operation, the environmental conditions/information or a combination of both. The information can be in the form of parameter values obtained from measurement (i.e. raw ion data or filtered detection data), analysed distance data, text data, light ranging system data etc.
Basically the available information can be categorized into two types of data: information related to the operation of the vehicle and environmental information concerning the external conditions wherein the vehicle is operating.
Information related to the vehicle operation may e.g. be vehicle speed data, a steering angle, data related to the vehicle trajectory, the braking system or a stability control system. Also other sensors on the vehicle can provide useful information, e.g. a rain/light sensor. Further information may relate to the vehicle battery voltage and/or the charge level.
Information related to the environment wherein the vehicle is operating, may for example be data on the roadway type, high density map data, external light conditions, daytime/night time, time of the day, atmospheric conditions e.g. precipitation. Other info that can be taken into account in the control block may for example be the planned routing.
Another input that can be considered is for example an emergency vehicle notification. Suppose a received Dedicated Short Range Communications (DSRC) information (vehicle-to-vehicle or vehicle-to-infrastructure) whereby location and speed vector are transmitted, indicates a fast approaching vehicle is approaching. Such information can be used after processing in the VCU as an emergency notification, resulting in the light ranging system preparing a detection of the vehicle from a certain direction. Yet another input that can be taken into account comprises optical feedback (including lens cleanliness). It could be that the light ranging system informs the VCU that the optics are not clean. The VCU can then, e.g. in a situation where the power is already maximum, make sure that the light ranging system starts operating in limited power mode.
The idea behind the invention is to provide in the database (or any other appropriate storage means) a number of parameter tables corresponding to different operational scenarios, i.e. a number of operational schemes. Regulating the light ranging system in the vehicle is performed by first detecting actual conditions wherein the vehicle/light ranging system is operating and then selecting in the vehicle control unit, based on the gathered information, one of the available operational schemes (referred to as ‘Modes’ in
The proposed solution allows dynamically adapting the illumination configuration to changing application conditions. This leads to more appropriate settings during operation and thus more energy efficiency. Different parts of the scene/field of illumination in front of the light source can so be illuminated differently. For example, the use of high power illumination can so be limited to situations wherein it is really required given the number of frames per second and/or the field of view. Alternatively, only a window, i.e. a part of the field of view, is considered wherein a part of the detector array is read out more often than the rest of the detector array. Further, the eye safety is enhanced, as a lower average illumination power can be used and certain zones in the field of view can operate with reduced average power levels.
In any application example there is optical sensing technology (such as ToF or LIDAR) available in the vehicle with which the invention may be applied. The solution of this invention may rely on the optical sensors mounted on different parts of the vehicle to provide sensor information for the zones adjacent to the sensor mounting locations. The sensed data can be used to control amongst other things the power consumption and/or field of view.
In each location and in each vehicle mode of operation, the sensor performance can be optimized in either a single or multiple operating mode by use of a schedule with associated parameter table. The operational schedule and corresponding parameter table to be applied is dynamically selectable based on the operating conditions (e.g. vehicle movement), operation limitations (e.g. minimized power consumption, heat generation of the illumination technology) and environmental operating limitations (e.g. eye safety).
Suppose the light ranging system under consideration is a time-of-flight system or LIDAR system integrated in a chipset. Different schemes of operation with corresponding parameter tables can be stored in the system or external to it, from which one is selected by the vehicle control unit given the current operational conditions. The selected parameter table can be fixed (e.g. in a ROM) or uploaded at system initiation (e.g. vehicle ignition on). Alternatively, the parameter table in accordance with the operational scheme to be used may be continuously communicated to registers used within the chipset for management and execution.
Control signals are generated in the light ranging system based on the parameter settings corresponding to the selected operational schedule received from the vehicle control unit, which can be processed locally in the chipset comprising the light ranging system to set parameters of both the illumination (or illumination array) and the pixel array. The control signals convey instructions to set parameters according to the dynamic parameter table selected in the vehicle control unit. The vehicle control unit receives external (vehicle, environmental and/or other) information and selects accordingly the most appropriate mode and communicates by serial data to the chipset comprising the light ranging system which executes in the form of a schedule table.
In an alternative implementation the chipset comprising the light ranging system is preloaded (into on-board memory) with a number of “preferred” schedule tables and the required operating mode of the system chipset (as defined in the schedule table) is selected by an external microcontroller which itself uses algorithms and additional data to determine the correct operating mode for the light ranging system.
A practical example is now provided. The following schedule tables are stored in the memory 204 in the light ranging system 100 in
The front facing sensor as described in the first and the second schedule table is used in a stationary vehicle situation. In Schedule table 1 maximum power is used for the illumination array every tenth sample, which allows a reach of 15 meter. The full field-of-view is illuminated. To obtain maximum sensitivity pixels are grouped for reading out, i.e. binning (combining of read-outs from several pixels), is applied. For the other 9 samples out of the group of 10 samples a reduced power level is applied which reduces the illumination range to 2 meter. To obtain maximal time/distance resolution each pixel in the detector is read out separately in the low power mode. In this case only the low array part of the detector is employed, so that only a window of the full field of view is considered. In the first schedule table a repetition rate is provided of 100 msec/sample. Schedule table 2 differs from table 1 by the repetition rate, which is 200 msec/sample in Table 2. Due to the doubling of the repetition period, the frame rates are halved to 0.5 frame/s wherein the range is 15 m and to 5 frames/s with the 2 m range.
Schedule table 3 is for a side facing sensor of a fast moving vehicle. A sample with maximum power, full field of view illumination and a reading out with pixel grouping in order to achieve maximum sensitivity is now alternated with a sample with medium power, illumination of low and mid-level arrays only, while still applying read-out with pixel grouping to keep maximum sensitivity. When maximum power is applied, an illumination range of 15 m is reached, which reduces to a range of 8 m when the medium power level is used. The repetition rate is 50 msec/sample.
Schedule table 4 is for all surround sensors of a low speed vehicle (e.g. when self-parking). Here every fifth sample gets maximum power and full field of view illumination, and a read-out with pixel grouping (binning) in order to obtain maximum sensitivity. In this case the full field-of-view is illuminated. The other 4 samples of each group of five samples have medium power, illumination of low and mid-level arrays only and read-out of each pixel separately for maximum resolution. The repetition rate is set at 200 msec/sample.
Having a light ranging system with those four schedule tables allows manufacturers to program vehicle control units in several possible ways. For example, a first manufacturer can provide a vehicle control unit operable according to the following table:
In case 3 it is possible to switch between Schedule Table 1 and Schedule Table 2.
Another manufacturer may opt for a vehicle control unit wherein both steering and GNSS information is exploited. The VCU is then operable e.g. according to:
Yet another manufacturer may use the following scheme, wherein DSRC is applied:
The proposed approach based on a set of parameter tables corresponding to a set of scenarios can also be applied to perform some self-monitoring at system level. When it is observed certain operational limitations are being approached (e.g. a temperature limit of the illumination array), this potential performance degradation can be diagnosed and the status is communicated back to the vehicle, allowing the vehicle to transition into a mode where the operational limit is not a factor.
The deterministic behaviour of the proposed system is important in the specification and validation of Safety Systems and when providing a Safety Element Out Of Context (SEOOC) ASIL capability for a module for ISO26262 where the system components have to contribute to the System Safety Goals and performance assumptions need to be made for the elements of the system.
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The foregoing description details certain embodiments of the invention. It will be appreciated, however, that no matter how detailed the foregoing appears in text, the invention may be practiced in many ways. The invention is not limited to the disclosed embodiments.
Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfil the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.
| Number | Date | Country | Kind |
|---|---|---|---|
| 19158170.1 | Feb 2019 | EP | regional |
