Patent Application
20220120874
This application claims priority to German Patent Application No. DE 10 2019 202 327.4, filed on Feb. 21, 2019 with the German Patent and Trademark Office. The contents of the aforesaid patent application are incorporated herein for all purposes.
The invention relates to a sensor assembly and a method for a vehicle, for example for a motor vehicle such as for example a passenger car or a truck for detecting distance information. The detection principle may generally be based on the detection of light (i.e., electromagnetic radiation) within the visible or invisible range.
This background section is provided for the purpose of generally describing the context of the disclosure. Work of the presently named inventor(s), to the extent the work is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.
In the context of vehicles, it is known to detect scenes within a vehicle, or in the environment of the vehicle, by means of optical sensors. For example it is known to shine light with predetermined properties into the scene and to receive light reflected from the scene by means of a light-sensitive sensor. According to the received light, measuring signals may be generated, and these may be evaluated to obtain distance information or in other words proximity information.
This information may then be used by various driver assistance systems. Lighting systems of a vehicle, in particular LED-based lighting systems, may be used as the light source for generating and radiating light which may then be detected by sensors in a reflected form. Including in the context of the present invention, the lighting system may for example include daytime running lights, high beams, low beams, a turn signal light, fog lights, or the like.
To shorten the evaluation time and reduce requirements on the sensors, strip-shaped areas may be formed for which distance histograms are determined. Based thereupon, distance information may then be determined for intersecting points of the strip-shaped areas.
The teaching of DE 10 2013 002 671 A1 is incorporated herein by way of reference in its entirety.
It was recognized that optimum distance detection is not achievable in the prior art.
A need exists to further improve the strip-shaped light detection. The need is addressed by the subject matter of the independent claims. Embodiments of the invention are described in the dependent claims, the following description, and the drawings.
The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features will be apparent from the description, drawings, and from the claims.
In the following description of embodiments of the invention, specific details are described in order to provide a thorough understanding of the invention. However, it will be apparent to one of ordinary skill in the art that the invention may be practiced without these specific details. In other instances, well-known features have not been described in detail to avoid unnecessarily complicating the instant description.
In contrast to the solution from DE 10 2013 002 671 A1 that provides sequential measured value detection for the individual strip-shaped areas as results from the measuring time data at the end of paragraph [0067], the teachings herein provide detecting measured values of the strip-shaped areas simultaneously, or respectively in parallel at least in part. For example, all strip-shaped areas, however at least 10% of all the provided strip-shaped areas, may be detected in parallel, or in other words read out.
A measured value may be understood to be a measured value generated at a predetermined point in time, and for example a measured signal value that is obtained for the corresponding strip-shaped area. The measured value, or respectively the measured signal value, may also be a value distribution relative to a specific point in time. The measured value is for example one of the above-described distance histograms, or respectively may be used to form such.
A benefit of the disclosed solution is that the observed scene may be detected at least partially and for example completely all at once. The simultaneously detected light, or respectively the simultaneously generated measured values based thereupon, accordingly form the scene at the same time. This reduces motion blur which may arise during sequential detection and simultaneous relative movement between the sensor and scene. Expressed otherwise, with the solutions proposed herein, motion blur may occur at most during the comparatively short parallel illumination time, or respectively readout time, whereas with the known solutions, movement blur may arise during the cumulative illumination times for all the sequentially evaluated strip-shaped areas.
Another benefit results from the fact that the shone light may be better exploited due to the at least partially parallel measured value detection over a plurality of the strip-shaped areas. In comparison to the previous solution, comparatively short shone light pulses or pulse trains are sufficient, between which comparatively long pauses are also possible. This enables potential savings in terms of hardware and for example in association with utilized lighting sources.
It should be noted that to practically implement the teaching of DE 10 2013 002 671 A1, several detection cycles including all associated illumination cycles may have to be provided in certain circumstances, for example one for first strip-shaped areas (for example in the row direction), and others for second strip-shaped areas (for example in the column direction). With the present solution, one detection and illumination cycle may contrastingly be sufficient since the light from several strip-shaped areas is detected in parallel.
In detail and a first exemplary aspect, an (optical) sensor assembly for a vehicle for detecting distance information is proposed comprising:
wherein the strip-shaped areas of the scene each correspond with strip-shaped areas of the sensor, and
wherein the sensor assembly (and for example its processing unit) is configured to simultaneously determine the light received therein of a plurality of the corresponding strip-shaped areas (or in other words, to read out, and/or simultaneously determine measured values for the corresponding strip-shaped areas).
It may therefore be provided that an assignment, or respectively correspondence, exists between the areas of a scene, the strip-shaped areas with which the scene is detected, and the strip-shaped areas of a, or respectively on a, sensor. For example, it may be provided that a scene is detected by means of (for example virtual) strip-shaped areas of the aforementioned type, and these strip-shaped areas are formed, or respectively provided by corresponding areas of the sensor, and for example the sensor elements available therein that will be described below. The first and second strip-shaped areas may therefore be as it were virtual, but attributable to a corresponding grouping, or respectively arrangement of sensor elements within the sensor that divide the scene into correspondingly detectable strip-shaped areas. For example, the strip-shaped areas may correspond to rows and columns of a sensor subdivided like a matrix (i.e., the corresponding strip-shaped areas of the sensor may be its rows and columns).
In general, the sensor may enable a scene to be depictable by means of a two-dimensional detection area that is divided into correspondingly strip-shaped areas, or respectively that may be assigned the aforementioned strip-shaped areas with which the scene is to be detected.
The sensor may include several sensor elements that each have at least one photodetector (or photodetector element as well). The sensor elements may be SiPMs (silicone photomultipliers) that for example may be constructed of several smaller photodetectors (such as from so-called SPADs—single photon avalanche diodes). Alternatively, the sensor elements may be formed from so-called PIN diodes.
The photodetectors may be arranged in a matrix, or respectively grid and accordingly may have a row and a column direction that are perpendicular to each other as well as generally speaking the sensor, or respectively a detection area defined thereby.
The determination of received light for the corresponding strip-shaped areas may also be termed reading out or evaluating these areas, wherein the latter for example also includes forming the corresponding distance histograms (or in other words area-wise measured value generation).
All measured values or measuring signals may generally be determined with time resolution. The sensor assembly may also include a memory apparatus in which the measuring signals may be saved (for example time-resolved). The time-resolved signals that are for example received per pixel and/or sensor element may also receive a time-resolved sum signal per row and column, for example at the same time. This may then be converted into a distance-resolved signal, or respectively into a distance histogram using the methods described herein.
The parallelism, or respectively simultaneity of reading out may be achieved in that each strip-shaped area is assigned its own (electrical) line into which all sensor elements feed their signals within this strip-shaped area (or respectively within a corresponding strip-shaped area of the sensor). These may be the row and column lines explained below.
It may be provided for all corresponding strip-shaped areas (of the sensor) to be able to be read out simultaneously, at least however all strip-shaped areas (of the sensor) that correspond to the first or second strip-shaped areas.
According to some embodiments, it may be provided to simultaneously determine the light received therein of at least 50% (and for example at least 25% or at least 10%) of the corresponding strip-shaped areas.
In general, it may furthermore be provided that the corresponding strip-shaped areas of the sensor are (for example virtually) divided into areas in which the light received therein is determined simultaneously, and into areas in which the light received therein is not determined simultaneously. In the last-mentioned areas, a sequential evaluation, or respectively a readout may be performed instead.
In this context, it may furthermore be provided that the areas (at least) partially lie within an edge area of the sensor without simultaneously determining the received light. The edge area of the sensor may for example be understood to be a row and/or column area (or generally an area) that includes up to 10% or up to 5% of the total number and/or the total surface area of the rows, and/or columns (or generally the total number and/or the total surface area of the corresponding strip-shaped areas) of the sensor, and that for example lies to one side of a geometric center of the sensor, and for example also includes a row and/or column forming the outermost edge (or generally a strip-shaped area forming the outermost edge).
A benefit of this version is that the simultaneous and more precise detection of a central area of the sensor may be reserved, and less precise sequential detection may occur in the edge areas in which fewer events relevant to the vehicle will probably be observed.
As already noted, the sensor may include a sensor matrix with sensor elements arranged in rows and columns that are each configured to receive light, wherein the rows with the first strip-shaped areas correspond to the scene, and the columns with the second strip-shaped areas correspond to the scene.
In general, the sensor and for example the sensor elements may each detect a (light) intensity of the received light. The sensor elements may supply pixel values and/or define individual pixels, or respectively picture elements of the sensor matrix. Simultaneously recorded pixel values as well as sequentially recorded pixel values that refer to the same detection process may be considered together and for example used to derive common distance histograms.
The sensor elements in the rows and columns are for example each interconnected (for example by being connected to common electrical (signal) lines, and for example to common row, or respectively column lines), and the distance histograms are for example determined from an overall signal of correspondingly interconnected sensor elements. The corresponding first and second strip-shaped areas of the sensor may be formed by being interconnected.
In some embodiments, at least those sensor elements whose received light is determined simultaneously each include a current mirror arrangement that is connected to a row line and to a column line to which other current mirror arrangements of other sensor elements are also connected. This makes it possible to combine the measuring signals obtained per sensor element by row, or respectively by column and then also form the corresponding distance histograms therefrom. Furthermore, this version is for example suitable if the sensor elements are SiPMs. In general, this is a reliable and positive option for forming, or respectively detecting the strip-shaped areas. The current mirror arrangement may be formed conventionally by two e.g. parallel-connected (semiconductor) transistors and will be explained below in an example with reference to the FIGS.
Another version provides that the sensor elements each include at least one PIN detector that is connected to two resistors in order to feed the current falling there into lines (for example into a signal line and a row line) that are assigned to the strip-shaped areas. The PIN detector may be connected to a transimpedance amplifier, and its output voltage may be applied to the corresponding resistors.
Moreover, it may be provided that at least those sensor elements whose received light is simultaneously determined (i.e., that are simultaneously read out) include at least two photodetector elements that each receive light (i.e., each may be read out, or respectively each provide a measuring signal), and wherein one of the photodetector elements is connected to a row line, and the other to a column line. The photodetector elements may be designed as aforementioned SPADS, for example if the sensor element is a SiPM. For example, but not restricted to the latter case, a sensor element may include up to 16 or up to 32 photodetector elements.
By means of this version, additional hardware elements such as for example the aforementioned current mirror may be spared, and a reliable ability of the strip-shaped areas to be read out in parallel may nonetheless be achieved.
Moreover, it may be provided that more than two photodetector elements per sensor element are provided (such as 16 or 32, see above) and these are combined into two for example equal size groups, wherein the photodetector elements of the one group are connected to a row line, and the photodetector elements of the one group are connected to a column line, and wherein at least two photodetector elements of the one group include at least one photodetector element of the other group between themselves. Expressed otherwise, the photodetector elements of the two groups may be arranged nested in each other and/or alternatingly (for example in the row and in the column direction). In general, the photodetector elements of the two groups may thus be arranged like a chessboard pattern. By means of a corresponding group-shaped arrangement, a high resolution may be ensured despite subdividing individual sensor elements into separate detection areas (i.e., into separate photodetector elements).
A second exemplary aspect relates to a method for a vehicle for detecting distance information, comprising:
wherein the strip-shaped areas of the scene each correspond with strip-shaped areas of the sensor, and
wherein, for a plurality of the corresponding strip-shaped areas, the light received therein is determined simultaneously (or in other words, a plurality of the corresponding strip-shaped areas of the sensor are read out simultaneously).
All of the explanations above and below of the features of the sensor assembly may also apply to the identical features of the methods. For example, the method may include any additional step in any additional feature for providing all or some of the functions, operating states or effects described herein that are associated with the sensor assembly. For example, the method may be executed with a sensor assembly according to any of the above and below embodiments.
According to some embodiments of the sensor assembly and the method, the scene is illuminated with an light emitting diode (LED) light source of the vehicle. The LED light source may include a lighting apparatus of the vehicle for illuminating an environment or an interior the vehicle. The lighting apparatus may for example include daytime running lights, high beams, low beams, a turn signal light, fog lights, etc. The LED light source may be actuated using a modulation method, and the distance histogram may be determined depending on the modulation signal and a receive signal of the sensor matrix. The modulation method may for example include a frequency-modulated continuous wave method in which a frequency with which the LED light source is modulated is modified over a certain time from a starting frequency to an end frequency. The modulation frequency is for example continuously modified from the starting frequency to the end frequency. Moreover, a random frequency modulation method may be used as the modulation method in which a frequency with which the LED light source is modulated is modified randomly or pseudo-randomly. Moreover, the LED light source may be actuated using a single frequency modulation method in which a frequency for modulating the LED light source is constant. Finally, the LED light source may be actuated using a pulse modulation method. Depending on the employed modulation method, various evaluation methods may be used for determining the distance histograms. For example, correlation methods may be used that correlate a signal which was used to modulate the LED light source with a receive signal of the sensor matrix. In another evaluation method, the modulation signal may be mixed with the receive signal, and the distance may be determined depending on the mixing frequency. The distance histograms represent in principle distance-resolved echograms that are generated by an object or several objects in the strip-shaped detection area. These distance-resolved echograms may be processed using a method similar to that of a CAT scan into a spatially resolved image, wherein each location of the image is assigned a corresponding distance. All of the aforementioned methods are also applicable for simultaneously reading out at least individual strip-shaped areas.
Since the distance histograms each include an entire row or an entire column of the scene, fewer sensors with a coarse resolution (row or column sensors) are necessary or, if a matrix sensor is used, only a few row and column measurements are needed instead of measurements for each picture element. Accordingly, a high resolution with few sensors, or respectively few measurements is possible.
According to the present aspect, a device for detecting distance information for a vehicle may moreover be provided. The device includes a light source that is equipped for illuminating a scene in an environment or within the vehicle. The light source is for example a lighting apparatus of the vehicle. Moreover, the lighting apparatus for example includes LEDs for generating the light since LEDs may be modulated at a sufficiently high frequency to provide light that may be used for the determination of distance histograms described below. The device furthermore comprises a sensor assembly of the aforementioned type for receiving light that originates from the light source and was reflected by the scene. Finally, the device includes a processing unit that actuates the light source and determines several first distance histograms and several second distance histograms of the aforementioned type depending on the received light.
The present invention will be described further in detail below with reference to the drawings.
Specific references to components, process steps, and other elements are not intended to be limiting. Further, it is understood that like parts bear the same or similar reference numerals when referring to alternate FIGS. It is further noted that the FIGS. are schematic and provided for guidance to the skilled reader and are not necessarily drawn to scale. Rather, the various drawing scales, aspect ratios, and numbers of components shown in the FIGS. may be purposely distorted to make certain features or relationships easier to understand.
So that the above-described method may be used in a vehicle for different driver assistance systems, it is necessary to use different transmitting and receiving methods for the different applications. These methods may for example be chosen depending on the required distance or application. To accomplish this, for example an operating state of the vehicle 10 may be determined, and a corresponding transmission and reception method may be selected depending on the operating state of the vehicle, i.e., a corresponding modulation method for generating the modulated signal and a corresponding evaluation method (such as mixing or correlating) is selected depending on the operating state. The modulation method may for example include a frequency modulated continuous wave method, a random frequency modulation method, a single frequency modulation method, or a pulse modulation method. These methods will be described below in detail. The operating state of the vehicle may for example include a speed of the vehicle, an activation state of a light source of the vehicle that indicates whether the light source is or is not turned on to illuminate an environment of the vehicle or to output an optical signal, a driving direction of the vehicle, previously determined position information or distance information of an object in the environment of the vehicle, weather conditions in the environment of the vehicle, or a type of assistance device of the vehicle that is provided the distance information.
In the frequency modulated continuous wave method, also termed FMCW or the chirp method, a modulation frequency is modified over a certain time from a starting frequency to an end frequency. For example, the modulation frequency is continuously modified from the starting frequency to the end frequency. As will be shown later, the method may be used not just for distance measurement, but also for a speed measurement of the object 17. The generation of modulation signals in which a modulation frequency is continuously modified over a certain time from a starting frequency to an end frequency is known in the prior art, and the method may therefore be easily implemented, for example by blanking a synthetically generated waveform. With the method, the distance 18 to the object 17 may be measured continuously, whereby the method is particularly suitable for light sources 11 that are continuously turned on. A frequency ramp results by continuously modifying the modulation frequency and hence the transmission frequency of the light source 11 from a starting frequency to an end frequency. By mixing the transmit signal with the receive signal that is received by the sensor 12, both the distance 18 as well as the speed of the object 17 may be measured directly. When using light emitting diodes (LEDs) as a light source 11 that have typical response times of 5-10 ns, modulation frequencies may be used up to for example a maximum of 100 MHz. The FMCW modulation may accordingly for example use the transmission frequency of 10 MHz to 100 MHz continuously or over a time period of for example 0.5-400 μs. The distance may optionally be measured when using the frequency modulated continuous wave method (FMCW) by means of frequency mixing or a correlation method.
When the frequency modulated continuous wave method is used, the transmit and the receive signal may be compared using a frequency mixer. An object at a certain distance generates a mixed frequency that is proportional to the distance. The local resolution of several objects is a function of the resolution of the frequency measurement and therefore the measuring time. Such a frequency measuring method may for example be realized as an analog circuit in an integrated circuit. If for example a distance between 0 m and 40 m is to be measured, the light needs about 3.3 ns/m×40 m×2=264 ns for this distance there and back along the arrows 15 and 16 in
In measuring distance using the frequency modulated continuous wave method, the transmit signal and the receive signal may also be correlated with each other to determine the distance to the object 17 on the basis of a correlation signal generated in this manner. To accomplish this, the modulated signal that was transmitted is correlated at a delay with the receive signal. A correlation maximum to the shift time results that is proportional to the distance 18 of the object 17. Since basically distorted signals are evaluated, the level of the correlation maximum is a measure of the signal strength so that different objects 17 may be distinguished. The resolution in the distance may for example be determined by the sampling frequency of the receive signal. To generate the correlation signal, several correlation coefficients may be generated. In so doing, each correlation coefficient of the several correlation coefficients are assigned a particular shift time. Each correlation coefficient is formed by correlating the modulated signal delayed by the particular assigned shift time with the receive signal. The distance to the object is determined depending on the several correlation coefficients, for example by determining an absolute or local maximum and the assigned shift times. In doing so, it is beneficial to use a high signal significance in the time domain by the continuously modulated FMCW signal that contains many frequencies that differ from each other. To achieve a high sampling rate of the receive signal, a one bit conversion may for example be beneficial. To generate the binary receive signal, the receive signal may be generated with a first signal value if a level of the received light falls below a certain intensity, and a receive signal is generated with a second signal value when the level of the received light reaches or exceeds the certain intensity. To accomplish this, for example an amplitude-limiting amplifier may be used that generates a receive signal with distinct levels depending on the received light, for example a binary sequence of zeros and ones. Since it is the time that is significant, scarcely any information is lost by this binary conversion since the significance in the amplitude may be unreliable given the amplitude modulation from the object 17 to be expected. With the receive signal reduced to binary signals, a corresponding correlator may be fabricated comparatively easily and may be suitable for processing long sequences. This improves the correlation result. If the receive signal is digital or binary, it is beneficial for the reference sample of the modulated transmit signal to also be digital. This may be achieved for example in that a synthesized digital signal is used to modulate the light source. This may for example be generated with consistent quality and only depending on a transmission cycle.
If the receive cycle is the same as the transmission cycle, arising errors, for example from temperature drifts, may be compensated. By using the correlation method, long signal sequences may be used. The useful frequency deviation is accordingly not restricted to the runtime of the signal for the distance to be measured. As described above, the method may be realized in a purely digital matter and may therefore be economically constructed. For example, a modulated signal with a length of 50 μs-500 μs may be transmitted, and the frequency may be increased during this time from 10 MHz to 100 MHz. Such a modulated signal may for example be generated using a shift register in which a synthetically generated signal is stored. A clock frequency with which the transmit signal may be synchronously clocked out and the receive signal may be clocked in may for example be 1 GHz, and is therefore realizable with comparatively little effort. The measuring time of 50 μs-500 μs is fast enough for most applications of driver assistance systems so that multiplex methods in multichannel sensors are also possible. Moreover, several measurements may be performed and averaged to further improve the signal quality.
The modulated signal with which the light source 11 is actuated may moreover be generated using a random frequency modulation method. In doing so, a transmit frequency from a frequency band is randomly varied over a specific time. This method is also termed random frequency modulation (RFM). To determine the distance from the object 17, the previously described correlation method may be used in a comparable manner. The random frequency modulation method offers very strong interference resistance, for example against scattered light and other measuring methods. Moreover, several measuring channels may be measured simultaneously since corresponding crosstalk from other measuring channels are suppressed by the correlation evaluation. The modulation frequencies and the length of the time of the transmit signal may be selected to be comparable with those of the frequency-modulated continuous wave method. The random frequency modulation method may accordingly be used for example when several light sources simultaneously illuminate a scene or a space and all are to be measured simultaneously. For example, measurements may be performed simultaneously with all of the headlights of the vehicle 10 using the random frequency modulation method. In doing so, each light source is given its own significant signature that may then be distinguished by using the correlation method. Moreover, data that are encoded in the modulation signal may be simultaneously transmitted to other vehicles or receivers at the roadside. For continuous distance measurement, continuous actuation of the light source 11 is necessary, so that this method is particularly suitable for light sources which are continuously on, such as for example daytime running lights or a headlight during night travel. The above-described frequency-modulated continuous wave method and the above-described random frequency modulation method may also be used in combination. For example due to the improved signal quality, the frequency-modulated continuous wave method may be used initially. If interference sources are detected such as light sources from other vehicles, there may be a switch to the random frequency modulation method. Likewise there may be at least a temporary switch to the random frequency modulation method when data transmission is necessary. The light source 11 and the sensor 12 may equally be used for the frequency-modulated continuous wave method as well as for the random frequency modulation method.
In the method for determining the distance to the object 17, furthermore a single frequency modulation method for generating the modulated signal may be used to actuate the light source 11. The single frequency modulation method uses a constant modulation frequency and is therefore very easy to realize. Contrastingly, it may however be interfered with comparatively easily by fog, spray, dust or foreign light sources, and may therefore be used for example in applications where such interference cannot arise, for example due to the installation site, or if a temporary failure may be tolerated, such as for example in a distance measurement in the interior, or with park assists with which measuring too closely does not have any negative consequences and the necessary distances are small due to the low speed of the vehicle, or the development of spray is unproblematic. For continuous distance measurement, a permanently active light source is also necessary in the single frequency method, so the single frequency method may for example be used in conjunction with for example daytime running lights or low beams of the vehicle. A determination of the distance, i.e., an evaluation of the single frequency modulation method, may for example be reduced to a phase measurement in which a phase difference is determined between the modulated signal and the receive signal. For example, the phase may be digitally measured via an and-link by a comparison of the receive signal with the modulated signal. Suitable typical modulation frequencies are for example within a range of 5-20 MHz. Within this range, clarity of the phase evaluation based on the single frequency modulation may be ensured.
Finally, a pulse modulation method may be used to generate the modulated signal for actuating the light source 11. The pulse modulation method may for example also be used to measure when the light source 11 is turned off. The short light pulses of the pulse modulation method may be configured so that they are invisible or scarcely visible to the viewer. If a light source is turned on, the pulse modulation method may also be used by configuring the light source for the pulse duration or, expressed otherwise, by generating “negative” light pulses. The pulse modulation method is therefore particularly attractive when measuring is to be with a low measuring frequency of for example 10 to 100 Hz, and the light for measuring should not be discernible. Light sources such as for example low beams, a turn signal light, tail lights, brake lights or backup lights that are not turned on at the time of measurement may be turned on with brief pulses having a length of for example 10 to 100 ns that are not noticed by a human observer due to the low average output. When light sources are turned on, the light may for example be turned off for a short time period of for example 10 to 100 ns, whereby a negative light pulse arises that may also be detected by the sensor 12. The distance 18 to the object 17 may for example be determined using the above-described correlation method when using the pulse modulation method. For example, a pulse modulation may be used that consists of a pulse sequence which has a high time significance over nonuniform pulse spacings. The receive signal generated depending on the received light may in turn be correlated with the modulated signal, or alternatively, a mathematical description of the pulse may serve as a correlation pattern for the pulse modulation. The receive signal may be sampled over the measuring distance. By using an oversampling method, several such echograms may be recorded and added up as a distance histogram. Echo pulses may be recognized using pulse evaluation, and for example a precise distance 18 may be determined for example by centroid determination. The method is suitable both for positive pulses as well as for negative pulses.
As already mentioned above in conjunction with the frequency-modulated continuous wave method, a speed of the object 17 may also be determined in addition to the distance 18 to the object 17. This will be described in detail below with reference to
An example of the method will be described below in detail with reference to a modulation with the assistance of a frequency-modulated continuous wave method (FMCW). In FMCW modulation, a continuous frequency deviation of for example 10 MHz to 100 MHz is modulated over 40 μs. With a distance of 200 m between the vehicle 10 and the object 17, a shift of 1.32 μs or 2.97 MHz results. With a relative speed of v, another mixing sequence according to the Doppler formula results:
where f is the modulation frequency, f0 is the frequency of the receive signal, and c is the speed of light. The following table shows the Doppler frequency shift for various speeds of the object.
The table shows that the Doppler frequency depends on the modulation frequency. A higher modulation frequency also leads to a higher Doppler frequency. The FMCW modulation may therefore be modified for example so that, for example over 20 μs, the frequency of 10 MHz is modulated to 100 MHz, and the frequency of 100 MHz is held for another 20 μs. The Doppler frequency may then for example be measured at 100 MHz. The Doppler frequency may for example be determined directly by mixing the transmit frequency within the receive frequency. For practical reasons, the Doppler frequency may however be alternatively determined by mixing the receive signal with another signal that has a frequency deviating by a predetermined value from the frequency of frequency-modulated transmit signal. For example, the receive signal may be compared or mixed with a signal that has a frequency less by 100 kHz than the frequency-modulated transmit signal. Consequently frequencies between 100,000 and 100,024 Hz are obtained for the Doppler frequency in the example shown in the table for speeds between 0 and 260 km/h. These clearly higher frequencies may be measured more easily and may arise within the measuring period for example of 20 μs.
As described above, the light source 11 of the vehicle 10 should for example be modulated within a frequency range of 10 MHz to 100 MHz. To accomplish this, for example LED light sources are suitable that use semiconductor LEDs to generate the light 15. For example, LEDs that generate ultraviolet or blue light have such a large modulation bandwidth. To change the color to white light or differently colored light components such as for example red or green light, these LEDs may additionally have phosphorus coatings that convert ultraviolet light or blue light into differently colored light. The high-frequency light for distance measurement or speed measurement is for example the blue light of the LEDs. The currents through the LEDs lie within a range of a few amperes to achieve corresponding light ranges. To achieve efficient modulation, a corresponding actuation of the LED must be correspondingly designed.
In the previously described methods and devices, available lighting apparatuses of the vehicle such as for example headlights of low beams, fog lights, turn signal lights, brake lights or backup lights are used to generate a modulated light signal that is reflected by an object in the environment of the vehicle and is received by a sensor in the vehicle. A distance or a speed of the object may be determined from the receive signal of the sensor and the awareness of the modulated signal with which the lighting apparatus of the vehicle was actuated. Since the primary function of the lighting apparatus is to illuminate an environment of the vehicle or output a light signal such as for example a flashing signal or a brake signal, a method 60 will be described in the following that simultaneously ensures a determination of distance information.
To accomplish this, first an operating state of the vehicle is detected in step 61. The operating state of the vehicle may for example be a target state for the lighting apparatus of the vehicle that indicates whether the lighting apparatus should be turned on or off. Detecting the operating state may furthermore include determining an environmental brightness in an environment or within the vehicle, or determining a distance measuring area for which the distance information is to be determined. Depending on the operating state determined in this manner, a modulated transmit signal is generated in step 62. For example, a first modulated transmit signal may be generated when the target state of the lighting apparatus indicates that the lighting apparatus should be turned on. Moreover, a second modulated transmit signal may be generated that is inverted to the first modulated transmit signal when the target state indicates that the lighting apparatus should be turned off. Accordingly for example when the lighting apparatus is turned off, a modulated transmit signal may be generated that includes short light pulses whose energy is insufficient to be seen by an observer. Conversely if the lighting apparatus is to be turned on, a modulated transmit signal may be generated that turns off the lighting apparatus for short pulses which are so short that they are not noticed by an observer, and the lighting apparatus therefore appears to be continuously turned on. In step 63, the lighting apparatus 11 of the vehicle 10 is actuated by the generated transmit signal. In step 64, reflected light 16 is received which was emitted as light 15 from the lighting apparatus 11 and reflected by the object 17. Depending on the received light 16, a receive signal is generated in step 65. In step 66, the receive signal is combined with the transmit signal, and in step 67, the distance of the object 17 is determined from the combination.
The amount of light that cannot be seen by an observer depends inter alia on an overall brightness of the environment of the vehicle and a contrast in the transmission domain. In the daytime, significantly larger amounts of light may be emitted by the lighting apparatus that are not noticed by an observer than at night. Typically, a signal-to-noise ratio is significantly worse in the daytime from the interfering light of the sun so that higher transmission outputs are needed in the daytime than at night. In the daytime, for example outputs of up to 2 mJ may be emitted that are not noticed by an observer. In the method, an average output of the modulated signal may therefore be adjusted depending on the operating state, for example an environmental brightness. Moreover, the transmission energy may be adjusted depending on a distance measuring area for which the distance measuring information is to be determined. This depends for example on the need of an application which uses the distance information. A driver assistance system for a distance regulation or a collision avoidance system may require a larger distance measuring area than a parking system.
The modulated transmit signal may for example include a pulse modulated signal. The pulse-modulated signal may have a pulse duration within a range of 1 to 500 ns, for example 10 to 100 ns. A frequency with which the pulses of the pulse-modulated signal are repeated may be within a range of 1 to 1000 Hz, for example 10 to 100 Hz.
The lighting apparatus of the vehicle may for example include the above-described LED light source, or several LEDs. With white LEDs, the primary blue light component may be used as a modulation carrier. This is modulated at a high frequency with the modulated transmit signal and remains in the spectrum of the white LED. The phosphorus of the LED cannot follow the fast modulations since it is generally sluggish. This yields a white, continuously shining light to human perception, whereas its blue component has the desired modulation.
Depending on the operating state of the vehicle and the modulated transmit signal, another lighting apparatus of the vehicle may be actuated. The vehicle 10 is for example driving on a country road, and a driver assistance system such as for example an adaptive cruise control is turned on. The headlights of the vehicle are turned off. Consequently, a modulated transmit signal is generated that includes brief light pulses. Distance information to an object in front of the vehicle may therefore be provided for the adaptive cruise control system. It is therefore unnecessary to turn on a driving light of the vehicle, i.e., the entire energy for all LED illuminants of the headlights of the vehicle does not have to be provided, which for example may be beneficial for an electric vehicle. For example the adaptive cruise control system requires a large measuring range. If, as described above, the headlights are turned off during the day, the high beams may for example be used with high energy to transmit measuring pulses that have a long range. If contrastingly the vehicle is driving in the dark, the high beams are modulated by briefly reducing the brightness to enable the large measuring range. If an oncoming vehicle is approaching in the dark, it is however no longer possible to operate the high beams so as not to blind the driver of the oncoming vehicle. In this case, LEDs of the low beams may be modulated by briefly reducing the brightness in order to determine distance information. At the same time, LEDs of the high beams may be modulated with short pulses to determine distance information without blinding the oncoming traffic. Expressed otherwise, some LEDs are briefly turned on (in this case, LEDs of the turned off high beams), and other LEDs are briefly turned off (in this case, LEDs of the low beams). This may enable a large measuring range without the LEDs for the high beams blinding or disturbing the oncoming vehicle.
In the above-described methods and devices, a distance of the object 17 or a speed of the object 17 was determined using a lighting apparatus 11 that is already available in the vehicle such as for example low beams, daytime running lights, or high beams of the vehicle 10. In the following, it will be described how, by using the above-described methods, position information, i.e., additionally direction information, of the object 17 may also be determined with respect to the vehicle 10.
In some embodiments, the sensor 12 of the vehicle 10 includes at least two first sensors for receiving light that was generated by a light source 11 of the vehicle, and that was reflected by a scene that includes the object 17 in the environment of the vehicle. Each of the at least two first sensors is assigned a particular first detection area of the scene. The first detection areas are arranged in a row in a first direction. FIG. shows 15 first detection areas that are assigned 15 first sensors. The 15 first detection areas are arranged in a horizontal direction. Two of the 15 first detection areas are identified with reference signs 71 and 72. The sensor 12 moreover includes at least two second sensors for receiving light reflected by the scene, wherein each of the at least two second sensors is assigned a particular second detection area of the scene. The second detection areas are arranged in a row in a second direction. The second direction differs from the first direction.
The determination of the position of the object 17 with the assistance of the overlapped detection areas as described above will be described below in detail. In comparison, it is noted at this juncture that, with nonoverlapping detection areas having for example five detection areas, only five different position areas for the object 17 may be distinguished. Given the overlap of the detection areas as shown in
The second detection areas may for their part also overlap in the vertical direction, and may also overlap in the horizontal direction with additional detection areas such as the third overlapping areas 83, 84. This may for example be achieved by a so-called “blur” of the assigned sensors.
The detection areas in
In conjunction with
The lighting apparatus 11 of the vehicle 10 has at least one first light source and one second light source. The first and second light source may be actuated independent of each other. The first light source is designed to illuminate a first lighting area of a scene in an environment or within the vehicle 10. The second light source is designed to illuminate a second lighting area of the scene. The first lighting area is different from the second lighting area. A plurality of lighting areas 121-127 is shown in
The detection areas may for example be arranged oriented toward the lighting areas, i.e., the detection area 131 corresponds substantially to the lighting area 121, the detection area 132 corresponds substantially to the lighting area 122, etc. Each of the detection areas may have a predetermined angular range, for example 10 or, as shown in
From the arrangement of lighting areas and detection areas described above in conjunction with
To illuminate the environment of the vehicle 10 in a segmented manner, several light sources are needed as described above. To accomplish this, for example a plurality of LEDs of for example low beams or for example daytime running lights that have a linear structure may be used. To achieve a uniform appearance for example with linear daytime running lights, LEDs arranged spaced apart for example in the daytime running lights may be interconnected into groups and may illuminate a particular lighting area. LEDs lying therebetween may illuminate other lighting areas. Expressed otherwise, the first light source that is used to generate the lighting area 121 may for example include at least one first LED and one second LED. A second light source that illuminates the lighting area 122 may also include at least one LED or a plurality of LEDs. The first and the second LED of the first light source and the LED of the second light source are arranged in a row, wherein the LED of the second light source is arranged between the first and second LED of the first light source. Since the brightness of the LEDs may vary in the distance measurement, this offset arrangement may allow these differences in brightness to be imperceptible to an observer. Alternatively, an effect that is more interesting in terms of design may also be thereby created if the differences in brightness are visible to an observer.
Different assistance systems of a vehicle may require image information of the environment of the vehicle that provide a high resolution of a depiction of a scene in front of the vehicle from the perspective of the vehicle, wherein each area or picture element of the image information is assigned a corresponding distance value to an object in this area. This image information may be necessary in order for example to be able to detect obstacles above or below a certain area, such as for example obstacles located on the roadway such as barriers that may not be run over.
Per row, or respectively column, a distance-resolved echogram, a so-called distance histogram, is now available. This may be processed using a method similar to that of a CT scan into a pixel-resolved image. To reduce the processing effort, a certain area of interest may be selected using the same method. For this area, the corresponding receive elements of the receiver matrix are interconnected, and only this area is observed and evaluated.
The switch between different areas to be evaluated may be done dynamically and may therefore be adapted to different driving situations.
The above-described method will be described below with reference to
From the totality of the echograms, corresponding distance information to objects in the scene may be determined for each of the 266 individual areas of the scene in
The distance-resolved echograms in
Both in the pulse modulation as well as in the random frequency modulation method (RFM), it is possible to encode information in the transmitted signal 15 that may be decoded by a receiver. This information may for example be used for communication between vehicles, so-called car-to-car communication, or for communication between the vehicle 10 and an infrastructure object, for example a stoplight or a traffic management system.
In step 221, a modulated signal is generated depending on transmit data that are to be sent by the vehicle 10. In step 222, the light source 11 of the vehicle 10 is actuated using the modulated signal. In step 223, light 16 which was emitted as light 15 from the light source 11 and reflected by the vehicle 210 or another object in the environment of the vehicle 10 is received. In step 224, a receive signal is generated depending on the received light. The receive signal may for example include an analog electrical signal or a digital electrical signal. In step 225, the receive signal is combined with the modulated signal, for example with the assistance of the previously described correlation method, and in step 226, the distance between the vehicle 10 and the vehicle 210 is determined from a combination signal from this combination. The modulation method for generating the modulated signal may for example include a random frequency modulation method or a pulse modulation method. In the frequency modulation method, a modulation frequency is modified depending on the transmit data. In the pulse modulation method, a pulse distance or a pulse length is modified depending on the transmit data. The modulated signal may additionally be generated depending on random data.
The data that are to be transmitted from the vehicle 10 are accordingly transmitted in the modulation of the transmit signal. For example as shown in
In the pulse modulation method, light pulses are sent with a pulse repetition rate. This is typically long in comparison with the pulse length of the light pulses. Since it may be problematic for distance measurement when the pulse repetition rate is constant, the distance between the pulses may be varied within a certain range in order to avoid for example beating states. In order to transmit data, this variation may for example divide the distance between the pulses into a static component and into a systematic component. For example, pulses may be used with a length of 50 ns and a pulse repetition rate of 25 kHz, i.e., 40 μs. To measure a distance within a range of for example up to 250 m, a pulse interval of 250 m×6.6 ns/m×2=3.3 μs may not be undershot. Accordingly, it is possible to vary the pulse interval between 3.3 μs and 76 μs. In a system with a transit time distance measurement and a base timing of 25 ns, there are 2,936 possible variations. Of these, for example, 512 may be used to transmit 9 bits. Of these, for example, 6 bits may include the transmission data to be transmitted, and the remaining 3 bits may be varied statistically. Accordingly, the distance between the pulses fluctuates by 12.8 μs from 33.6 to 46.6 μs. Accordingly, 6 bits of transmission data may be transmitted every 40 μs, whereby a net data rate of 150 Kbit/sec is achieved.
For example, in random frequency modulation (RFM), frequencies from 10 MHz to 100 MHz may be varied within 40 μs. In a random frequency modulation method without data transmission, several frequencies are randomly selected statistically from this frequency band, which are then modulated sequentially and therefore result in a frequency train which is significant for the measurement. To transmit the transmission data, the frequency selection is no longer performed randomly but rather includes at least one systematic component. For example, from the band of 10 to 100 MHz, frequencies may be synthesized in frequency steps of 10 kHz. 9,000 different frequencies are therefore possible. Of these, for example, 512 may in turn be used as significant frequencies so that there is a frequency spacing of about 175 kHz for each piece of information. A typical frequency modulation receiver may easily distinguish frequencies of 50 kHz so that the transmitted information may be easily decoded if a frequency spacing of 50 kHz or more is maintained. For the random variation to reduce interference, 125 kHz, or respectively ±62.5 kHz still remain.
The sensor elements 300 are arranged like a matrix. There is therefore one row and column direction indicated per coordinate system, wherein the rows in
Each row Z and each column S forms a strip-shaped area of the type shown herein so that distance histograms are correspondingly formed by row and column. Furthermore, each sensor element 300 supplies a measuring signal that may be read out both in the row as well as in the column direction in the manner described below, for example to form the distance histograms formed by row and column.
The sensor elements 300 are built in the same way. The components of the sensor elements 300 explained below are however not provided with a separate reference sign for each sensor element 300.
Each sensor element 300 includes a photodetector element 302 in the form of a SiPM 302. The SiPM 302 generates an electrical measuring signal according to a received light, or respectively a received light intensity. Per sensor element 300, the measuring signal is to be taken into account both by row as well as by column (i.e., contribute to the distance histograms formed by row and column). For this purpose, each SiPM 302 is connected to a row line 304-308 and to a column line 310-314.
The signals applied simultaneously to the row lines 304-308 and to the column lines 310-314 may be used to form the distance histograms by row and column.
The connection to the row lines 304-308 and to the column lines 310-314 is established by a current mirror 316, the adjacently shown example is formed by a conventional circuit of two semiconductor transistors 318. The row lines 310-314 are supplied by one of the semiconductor transistors 318, and the row lines 304-308 are supplied by the other one.
This design represents a simple and reliable version for enabling the desired simultaneous determination of received light for sensor elements 300 in the strip-shaped areas, or more precisely, in the individual rows Z and columns S.
It is shown that each sensor element 300 includes a plurality of photodetector elements 320, 340 that are combined into two groups. Stated more precisely, a first light group of photodetector elements 320 and a second dark group of photodetector elements 340 are shown, wherein each group for example may comprise 16 individual photodetector elements 320, 340. The photodetector elements 320, 340 may be formed in the form of SPADs described above that are combined into one or more SIPMs or are comprised thereof.
It may be seen that the photodetector elements 320, 340 of the individual groups are arranged like a chessboard so that two sequential photodetector elements 320, 340 in the column and row direction of a group each include one photodetector element 320, 340 of the other group. The photodetector elements 320, 340 of the individual groups are therefore arranged alternatingly in the column and row direction.
The photodetector elements 320 of the one group supply a measuring signal that is applied to a row line (see for example the marked row line 304). The photodetector elements 340 of the other group supply a measuring signal that is applied to a column line (see for example the marked row line 310).
This ensures that without additional hardware components in the form of the current mirror or other amplification circuits (that may however optionally also be provided), a measuring signal of a sensor element 300 may be read out both by row as well as by column and for example simultaneously.
Finally,
For the sake of completeness, it should be noted that the outermost sensor elements 300 of the rows P1,1 to Pn,1 and P1,m to Pn,m as well as the columns Pn,1 to Pn,m and P1,1 to P1,m (and optionally also other rows and columns adjacent thereto) are edge areas in which, instead of simultaneous detection or triggering, sequential detection, or respectively triggering of the corresponding sensor elements 300 could also take place. This is based on the idea that relevant objects will probably be detectable less frequently in these areas than, for example, in a central area of the matrix-shaped sensor element arrangement.
The invention has been described in the preceding using various exemplary embodiments. Other variations to the disclosed embodiments may be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single processor, module or other unit or device may fulfil the functions of several items recited in the claims.
The term “exemplary” used throughout the specification means “serving as an example, instance, or exemplification” and does not mean “preferred” or “having advantages” over other embodiments.
The mere fact that certain measures are recited in mutually different dependent claims or embodiments does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2019 202 327.4 | Feb 2019 | DE | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2020/050941 | 1/15/2020 | WO | 00 |
