The present disclosure relates to the field of autonomous vehicle control, especially within confined areas such as mining sites, logistics centers or ports. In particular, it discloses a method for controlling an autonomous vehicle on the basis of a model (world model) that associates areas of a drivable surface with a probabilistic occupancy score.
Autonomous vehicles (AVs) in confined areas typically have a simpler sensor setup than those used on public roads, to reduce the expenditure on product cost as well as development cost. Since the environment is known to a further extent in a confined area, the foreseen obstacles that need detection can be narrowed down significantly, as opposed to public road applications where all types of physical obstacles, other traffic participants etc. need to be detected by the sensors. Thanks to the narrowing-down, sensors can be more adapted to the particular use case. Also, the applications can often be implemented with simpler algorithms for object tracking, reducing the need for more complex sensor setups. For example, in some applications it is enough to determine that an object exists in the path of the AV and to stop, whereas for a public-road use case, the object needs to be detected, classified and its intentions need to be predicted, in order to keep the productivity high. With the simpler sensor setup, a simpler representation of the data can be used, for example an occupancy grid can be used as a representation of the world surrounding the AV.
US2020225622A1, which targets a generic AV use case, discloses a method for generating a non-uniform occupancy grid map, in which the cell size depends on a distance from an object to an ego autonomous vehicle. It is argued that the precision requirements decrease further away from the autonomous vehicle, which justifies using a lower grid-cell density there. The grid size may further be adjusted based on the speed of the ego vehicle.
Further, US2022095871A1 discloses a method for operating an AV operating in a space, such as an indoor space such as a room, facility, terminal, warehouse of similar. The method assigns a weight to an object detected by an AV onboard sensor relating to how static the detected object is. When the AV has been operated in the space multiple times and the same static object has been detected, the weight can be updated (increased). It is disclosed in US2022095871 that the method may use an occupancy grid, by which the space is divided into a grid of map cells, where each cell contains an estimate of the probability that that cell is occupied by a landmark or feature. If an occupancy probability is high, that position is more trustworthy.
When operation of the AV is started in this or a similar technical context, all cells in the occupancy grid typically have an uncertain occupancy probability, since the sensors have not registered any hits at all. Once the sensors are up and running, the occupancy probability will be lowered for free cells and raised for occupied cells. However, this is only for cells where the sensors have coverage. In some sensor installations, e.g., the area closest to the AV (cf. area 504 in
A frequently used, though not always cost-efficient, solution may be to add sensors, so that the total FOV is extended. This may well solve the problem (except the startup deadlock issue) but adds complexity and cost. Not all confined applications are foreseen to require full sensor coverage since other mechanisms can ensure the path to be clear of obstacles, such as manual checks, site-mounted sensors, fenced-off areas, etc.
One objective of the present disclosure is to make available a method and vehicle controller suitable for more efficiently controlling an AV on the basis of a model having an occupancy score for each area of a drivable surface on which the AV can move. It is a further objective to improve the productivity of the AV thus controlled without sacrificing safety. It is a further objective to make such improvements in a case where the model is obtained or updated based on measurement data from one or more sensors carried by the AV. It is a further objective to present measures for counteracting the deadlock scenario outlined above. It is a still further objective to provide accurate speed control for the AV and reliable estimation of the time needed to complete a driving mission.
At least some of these objectives are achieved by the present invention as defined by the independent claims. The dependent claims relate to advantageous embodiments of the invention.
In a first aspect of the invention, there is provided a method of controlling an AV which is movable on a surface. The method may be a computer-implemented method, and the various steps of the method may be carried out by processing circuitry of a computer system. The method comprises: obtaining a model of the surface, by which each area of the surface is associated with a probabilistic occupancy score; determining an occupancy threshold to be applied to an area of the surface; enabling movement of the AV into the area if the associated occupancy score is less than the determined occupancy threshold, and otherwise disabling movement into the area. According to the first aspect, the occupancy threshold is determined on the basis of the area's position.
By the use of a distance-dependent threshold for determining whether grid cells are occupied, the method according to the first aspect could for instance allow the AV to move into relatively more distant cells under a relatively more lenient condition. Indeed, noise and other spurious contributions to the occupancy score will be disregarded to a greater extent if the occupancy threshold is set higher in distant cells, so that the noise and other spurious contributions to the occupancy score are less likely to exceed the occupancy threshold. The effect is achieved regardless of whether the model has a uniform or non-uniform spatial resolution throughout the drivable surface; instead, the spatial resolution of the model can be constant over time and need not be revised when the AV moves to a new position, which seems to be required in the prior art solution described in US2020225622A1. With the method according to the first aspect, the vehicle control function will typically see a finite free path ahead of the AV, which is shorter in regions with more uncertain cells, so that the AV's speed will be inherently limited. The proposed solution is able to account for the fact that measurements on locations more distant from the AV are typically less accurate and/or less reliable, especially if the measurements are made from the viewpoint of the AV. This effect flows from the use of different thresholds, which will limit the impact of any errors that affect the relatively more distant locations.
In some embodiments, the occupancy threshold is determined to be relatively higher if the area is relatively closer to the AV and relatively lower if the area is relatively farther from the AV. This determination presupposes knowledge of the AV's current position (e.g., relative to the model), which must be determined unless already available. It is important to note that although the occupancy threshold is determined dynamically in view of the AV position, it is independent from the model. The occupancy threshold can be understood as a desired safety level at a certain position relative to the AV; it typically does not represent a topographic feature (or local feature) associated with a definite location on the drivable surface.
In some embodiments, where the model is obtained and/or updated based on measurement data from one or more sensors carried by the AV, the occupancy threshold is determined to be relatively higher if the area is outside a field of view (FOV) of the sensors carried by the AV and relatively lower if the area is inside the field of view. The FOV is a simple yet efficient criterion for setting the occupancy threshold. Especially for measuring techniques relying on propagating electro-magnetic or acoustic waves, it accurately distinguishes the areas where fresh measurement data is available and the areas where older measurement data has to be used. The use of older measurement data could bring a greater degree of uncertainty.
In some embodiments, the method further comprises generating a route of the AV towards a destination (in particular, up to a destination), said route passing only via such areas into which movement is enabled, and estimating a time of arrival at the destination. This embodiment improves on such prior art approaches where a time of arrival is estimated without sufficient attention given to obstacles in the driving environment and/or without considering spatial limitations of the sensorics used. Such prior art approaches may initially produce vehicle routes with too high speed, and the speed might later have to be reduced on short notice when new obstacles appear, e.g., as a result of a refresh of the model or when more reliable measurement data becomes available. Such sudden changes could also have negative secondary effects on a vehicle fleet to which the routed vehicle belongs, more precisely, as an original plan for coordinating vehicle movements within the fleet could suddenly become obsolete and have to be made up anew. In the present embodiment, the use of a variable occupancy threshold, which increases away from the vehicle, will produce conservative velocity profiles, which are typically a more realistic basis for vehicle routing and vehicle coordination.
In a second aspect of the invention, there is provided a vehicle controller configured to control at least one AV movable on a surface on the basis of a model of the surface. The vehicle controller has processing circuitry configured to determine an occupancy threshold to be applied to an area of the surface, to enable movement of the AV into the area if the associated occupancy score is less than the determined occupancy threshold and otherwise to disable movement into the area. In this process, the processing circuitry shall determine the occupancy threshold on the basis of this area's position.
The second aspect of the invention generally shares the effects and advantages of the first aspect, and it can be implemented with a corresponding degree of technical variation.
The invention further relates to a computer program containing instructions for causing a computer system, or the vehicle controller in particular, to carry out the above method. The computer program may be stored or distributed on a data carrier. As used herein, a “data carrier” may be a transitory data carrier, such as modulated electromagnetic or optical waves, or a non-transitory data carrier. Non-transitory data carriers include volatile and non-volatile memories, such as permanent and non-permanent storage media of magnetic, optical or solid-state type. Still within the scope of “data carrier”, such memories may be fixedly mounted or portable.
In the present disclosure, the concept of an “occupancy score” could overlap with the scope of the term occupancy probability in the literature, where it is sometimes defined as an estimate of the probability that that cell is occupied by a landmark or feature, or not drivable for other reasons. An occupancy score should be understood in a somewhat broader sense, to include also indicators that are not necessarily probabilistically true in a strict sense. A higher occupancy score for an area should be understood as a higher probability that the area is occupied. The condition of being occupied can refer to various aspects of the area not being possible to drive, e.g., a physical obstacle is present, the road surface is missing, the road is bumped or recessed (e.g., pothole), or is polluted (e.g., oil, grease, debris, crushed glass).
Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to “a/an/the element, apparatus, component, means, step, etc.” are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.
Aspects and embodiments are now described, by way of example, with reference to the accompanying drawings, on which:
The aspects of the present disclosure will now be described more fully hereinafter with reference to the accompanying drawings, on which certain embodiments of the invention are shown. These aspects may, however, be embodied in many different forms and should not be construed as limiting; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and to fully convey the scope of all aspects of the invention to those skilled in the art. Like numbers refer to like elements throughout the description.
The AV 110 is assumed to be under the control of a vehicle controller 130. The vehicle controller 130 may be any programmable general-purpose computer with suitable input/output interfaces. It can be mounted in the vehicle AV 110 or be external to the AV 110. As shown in detail
The sensor interface 134 is adapted for at least receiving data from sensors 112, such as a camera, a lidar and a radio receiver. The sensors 112 may use one or more of the following measuring principles: optical, electromagnetic reflection, electromagnetic scattering, electromagnetic diffraction, lidar, color-depth sensing, millimeter-wave radar, ultra-wideband radar. For these measuring principles, which rely on rectilinear propagation of energy, a sensor's FOV can usually be accurately estimated by tracing straight lines to or from the sensor. The sensors 112 may be carried by (e.g., installed in) the AV 110 or they may be external to the AV 110. The external sensors 112 may be fixedly mounted on a building or another structure, or they may be mobile, in particular carried by other vehicles.
The control interface 136 is compatible with the communication interface 114 of the AV 110. It may be used for conveying instructions suitable for execution by actuators in the AV 110, either directly (machine-level commands) or after conversion into machine-level commands by a processor carried in the AV 110.
The operator interface 138 is configured to receive higher-level or long-term commands, such as transport missions to be carried out by the AV 110 or strategic instructions, from a human operator or from an executing software application authorized to control the AV 110.
It is seen in
Functionally, the occupancy grid is an example of a “model” in the sense of the appended claims. In particular, the occupancy grid may be characterized as a “world model”, i.e., a data structure to be used as guidance for vehicle control, including routing and vehicle coordination, which is a (simplified) representation of a region of the physical world. Specifically, the cells of the occupancy grid constitute “areas” each associated with an occupancy score. As an alternative to an occupancy grid, sufficient information for supporting the vehicle control could be provided by a map or a map-like structure in computer-readable form, an object list, or any other suitable data structure which associates areas of the surface 110 with locally valid occupancy score values.
Two example embodiments of a method 200 suitable for controlling an AV of the type illustrated in
In a first embodiment of the method 200, an initial step 202 includes obtaining a model (world model). The model associates an occupancy score, denoted p(mi), with each area mi of the surface 100. The vehicle controller 130 can obtain the model by receiving a data structure representing the model, or retrieving it from a shared memory. Alternatively, the vehicle controller 130 can obtain the model, or update it, based on measurement data from the sensors 112. In particular, measurement data from sensors 112 in the AV 110 itself can be used to generate the model.
In a next step 206, for at least one area mi of the surface 100, an occupancy threshold t(mi) is determined. The occupancy threshold t(mi) is determined on the basis of the position of the area mi where the occupancy threshold is to be applied. The position dependence of the occupancy threshold can be defined according to one of the following options.
In such implementations of the method 200 where the position of the AV 110 is not determined, a static division into zones of the surface 100 may be used. One option is to define, with reference to the AV's 110 normal area of operation, one ‘near’ zone and one ‘far’ zone, wherein a relatively lower occupancy threshold is applied in the ‘far’ zone. It is recalled that an area with a lower occupancy threshold represents a stricter condition to fulfil in order to enable movement into the area. Another option is to define one ‘well-known’ and one ‘not yet explored’ zone, wherein a relatively lower occupancy threshold is applied in the ‘not yet explored’ zone. Yet another option is to define one ‘low-risk’ and one ‘high-risk’ zone, e.g. based on the presence of vulnerable road users, sensitive equipment or the like, wherein a relatively lower occupancy threshold is applied in the ‘high-risk’ zone.
In such implementations of the method 200, where a position or orientation of one of the sensors 112 is determined, a dynamic division into zones of the surface 100 dependent on the sensor position/orientation may be used. For example, a FOV of said one of the sensors 112 can be determined in relation to the surface 100. The FOV may correspond to those areas on the surface 110 from which straight lines can impinge on an aperture of the sensor 112. Geometrically, said areas can be determined by an inverse approach, namely, by projecting a solid angle corresponding to the sensor's 112 aperture and collocated with the sensor 112 onto the surface 110. The projection thus obtained can then be converted into a set of areas based on a conversion rule, e.g., that an area mi shall be included in the set of areas if at least 50% or at least 75% or some other percentage overlaps with the projection. Regardless of the method by which the FOV is determined, the FOV can be defined to be one zone of the surface 100 where the occupancy threshold is relatively lower (corresponding to a more lenient condition on movement), whereas the complement, outside the FOV, is defined as a zone where the occupancy threshold is relatively higher (corresponding to a stricter condition on movement).
It is important to note, for each of these static and dynamic options, that the determination 206 of the occupancy threshold t(mi) in no way interferes—let alone modifies—the data of the model data. The occupancy threshold is independent from the model, and independent from the occupancy scores p(mi) in particular.
With reference to the flowchart in
p(mi)<t(mi)
If the inequality is true, then movement of the AV 110 into the area mi is enabled 210a. Otherwise, if it is found that
p(mi)≥t(mi)
then movement of the AV 110 into the area mi is disabled 210b.
The occupancy threshold determination 206 and assessment 208 may be repeated as desired for sufficiently many areas mi to cover a region of interest of the surface 100. For example, the region of interest may be the region currently under consideration for route planning. Alternatively, the region of interest may be a corridor extending from the AV 110 to a next destination assigned by a transport mission that is in progress; the width of the corridor can be chosen based on earlier runs, e.g., the smallest width that has been found to allow efficient route planning in normal circumstances.
The decision to enable 210a or disable 210b movement may remain in force until the assessing step 208 is repeated. The assessing step 208 can be repeated in response to an updating of the model and/or an updating of the occupancy thresholds. To repeat the assessing step 208 in these cases, it may or may not be necessary to execute all of the preceding steps anew.
In a second embodiment, the method 200 comprises a further step 204 in which a current position of the AV 110 is determined. The position may be determined based on data from a global navigation satellite system (GNSS) receiver, cellular-network positioning, inertial measurements, an optical or radio-frequency (RF) fiducial deployed at a known reference location, or the like. Using the current position of the AV 110, it is possible to define a dynamic division of the surface 100 into two or more zones.
A simple option is to split the surface 100 into two zones Mfar, Mnear and to define the occupancy threshold as follows:
where the constants t1, t2 satisfy t1<t2. The near zone Mnear may be defined as all points on the surface 100 which are closer than a predefined distance to a reference point on the AV 110, such as an approximate center, or a driver position. Alternatively, the near zone Mnear may be defined as all points on the surface 100 which are closer than a predefined distance to a nearest point on an approximate horizontal contour of the AV 110. The horizontal contour may alternatively be described as the ground projection of the AV 110. Ways of converting such point sets into equivalent sets of areas mi in the model have been suggested above in connection with sensor FOVs.
A somewhat more complex option is to divide the surface 100 into three or more zones M1, M2, M3, . . . . The zones may be concentric on a reference point on the AV 100, and they may have a circular-annular shape or a shape approximating the horizontal contour of the AV 110. In the case of three zones, the occupancy threshold can be defined as follows:
where t1<t2<t3 and M1 represents the farthest, M2 the intermediate and M3 the nearest zone. A variation is illustrated in
A still further option is to define the occupancy threshold continuously, as per:
t(mi)=ϕ(d(mi,X110)),
where d is a distance function, X110 denotes the position of the AV 110, and j is a nonincreasing function. For example, one may use ϕ(s)=min{t0, 1/s}, where t0 is a preset maximum occupancy threshold to be applied.
In each of the first and second embodiments, the method 200 may include one or more of the following optional steps after the enabling 210a or disabling 210b of movement into areas mi of the surface 100.
In one optional step 212 of the method 200, the speed of the AV 110 is controlled such that the AV 110 is at least one stopping distance away from any areas mi into which movement is disabled. It is known that the stopping distance is a (generally superlinear) function of speed, which may have a further dependence on road conditions. The criterion of having at least one stopping distance away from any areas mi into which movement is disabled may be relaxed so that it applies only for areas mi ahead of the AV 110, e.g., areas mi belonging to a frontal halfplane relative to the AV 110.
In another optional step 214, a route of the AV 110 towards a destination is generated, wherein the route passes only via such areas into which movement has been enabled 210a. Further a time of arrival at the destination is estimated. The estimation may take into account a velocity profile of the route; the velocity profile may observe one of the criteria stated in step 212.
In a further optional step 216, the current position of the AV 110 is determined and the occupancy threshold t) is updated. The execution of step 216 may end after the current position of the AV 110 has been determined and found to differ by less than a predefined threshold distance from the previous position of the AV 110; in this case the new occupancy threshold t) may be set equal to the previous occupancy threshold t).
In a further development of step 216, the updating of the occupancy threshold t) takes into account a current orientation of the AV 110. The orientation may for example affect the FOV of a vehicle-carried sensor 112 relative to the surface 110. Further, a change of AV orientation may further lead to a corresponding change in the orientation of a contour that moves with the AV 110, such as one of the contours 722, 724 illustrated in
The present disclosure has proposed various ways of safeguarding a desired deterministic behavior of an AV that could also circumvents issues with possible blind spots of the sensor setup. It does so without strict requirements as to how a model (world model) shall be represented, but rather uses an interpretation of the uncertainties inherent to the representation so as to combine a pessimistic behavior (which, in isolation, could lead to low productivity) with an optimistic behavior (which, in isolation, might produce an unsafe system). The invention reaps the benefits of these two behaviors into a balanced model where the interpretation of the model—and thus the behavior of the ego vehicle—is be made dependent on the closeness to the ego vehicle. As seen in some embodiments, a pessimistic behavior is applied close to the ego vehicle, whereas further away a more optimistic behavior is configured. This leads to higher productivity while maintaining a safety-oriented pessimism in respect of areas close to the ego vehicle.
As mentioned, the example methods disclosed herein may be carried out by the vehicle controller 130, which can be implemented as a computer system. The computer system is adapted to execute instructions from a computer-readable medium to perform these and/or any of the functions or processing described herein. The computer system may be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, or the Internet. While only a single device is illustrated, the computer system may include any collection of devices that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein. Accordingly, any reference in the disclosure and/or claims to a computer system, computing system, computer device, computing device, control system, control unit, electronic control unit (ECU), processor device, processing circuitry, etc., includes reference to one or more such devices to individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein. For example, control system may include a single control unit or a plurality of control units connected or otherwise communicatively coupled to each other, such that any performed function may be distributed between the control units as desired. Further, such devices may communicate with each other or other devices by various system architectures, such as directly or via a Controller Area Network (CAN) bus, etc.
The computer system may comprise at least one computing device or electronic device capable of including firmware, hardware, and/or executing software instructions to implement the functionality described herein. The computer system may include processing circuitry 132 (e.g., processing circuitry including one or more processor devices or control units), a memory, and a system bus (solid lines). The computer system may include at least one computing device having the processing circuitry. The system bus provides an interface for system components including, but not limited to, the memory and the processing circuitry. The processing circuitry may include any number of hardware components for conducting data or signal processing or for executing computer code stored in memory. The processing circuitry 132 may, for example, include a general-purpose processor, an application specific processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a circuit containing processing components, a group of distributed processing components, a group of distributed computers configured for processing, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The processing circuitry 132 may further include computer executable code that controls operation of the programmable device.
The system bus may be any of several types of bus structures that may further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and/or a local bus using any of a variety of bus architectures. The memory may be one or more devices for storing data and/or computer code for completing or facilitating methods described herein. The memory may include database components, object code components, script components, or other types of information structure for supporting the various activities herein. Any distributed or local memory device may be utilized with the systems and methods of this description. The memory may be communicably connected to the processing circuitry (e.g., via a circuit or any other wired, wireless, or network connection) and may include computer code for executing one or more processes described herein. The memory may include non-volatile memory (e.g., read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.), and volatile memory (e.g., random-access memory (RAM)), or any other medium which can be used to carry or store desired program code in the form of machine-executable instructions or data structures and which can be accessed by a computer or other machine with processing circuitry. A basic input/output system (BIOS) may be stored in the non-volatile memory and can include the basic routines that help to transfer information between elements within the computer system.
The computer system may further include or be coupled to a non-transitory computer-readable storage medium such as the storage device, which may comprise, for example, an internal or external hard disk drive (HDD) (e.g., enhanced integrated drive electronics (EIDE) or serial advanced technology attachment (SATA)), HDD (e.g., EIDE or SATA) for storage, flash memory, or the like. The storage device and other drives associated with computer-readable media and computer-usable media may provide non-volatile storage of data, data structures, computer-executable instructions, and the like.
Computer-code which is hard- or soft-coded may be provided in the form of one or more modules. The module(s) can be implemented as software and/or hard-coded in circuitry to implement the functionality described herein in whole or in part. The modules may be stored in the storage device and/or in the volatile memory, which may include an operating system and/or one or more program modules. All or a portion of the examples disclosed herein may be implemented as a computer program stored on a transitory or non-transitory computer-usable or computer-readable storage medium (e.g., single medium or multiple media), such as the storage device, which includes complex programming instructions (e.g., complex computer-readable program code) to cause the processing circuitry to carry out actions described herein. Thus, the computer-readable program code of the computer program can comprise software instructions for implementing the functionality of the examples described herein when executed by the processing circuitry. In some examples, the storage device may be a computer program product (e.g., readable storage medium) storing the computer program thereon, where at least a portion of a computer program may be loadable (e.g., into a processor) for implementing the functionality of the examples described herein when executed by the processing circuitry. The processing circuitry may serve as a controller or control system for the computer system that is to implement the functionality described herein.
The computer system may include an input device interface configured to receive input and selections to be communicated to the computer system when executing instructions, such as from a keyboard, mouse, touch-sensitive surface, etc. Such input devices may be connected to the processing circuitry through the input device interface(s) 134, 138 coupled to the system bus but can be connected through other interfaces, such as a parallel port, an Institute of Electrical and Electronic Engineers (IEEE) 1394 serial port, a Universal Serial Bus (USB) port, an IR interface, and the like. The computer system may include an output device interface 136 configured to forward output, such as to a display, a video display unit (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system may include a communications interface suitable for communicating with a network as appropriate or desired.
The operational actions described in any of the exemplary aspects herein are described to provide examples and discussion. The actions may be performed by hardware components, may be embodied in machine-executable instructions to cause a processor to perform the actions, or may be performed by a combination of hardware and software. Although a specific order of method actions may be shown or described, the order of the actions may differ. In addition, two or more actions may be performed concurrently or with partial concurrence. The aspects of the present disclosure have mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.
Number | Date | Country | Kind |
---|---|---|---|
22190589.6 | Aug 2022 | EP | regional |