Patent Application
20240326694
The present disclosure is generally related to road vehicles with large blind spots, and, in particular, safety features in large road vehicles.
Road vehicles of many types have traditionally imposed certain challenges to operators in enabling clear views of the surrounding areas and objects within those areas. For instance, the structure and shape of vehicles and windshield/windows typically imposes an obstruction for a driver with respect to the turning side field of view. The obstruction in the field of view is often referred to as a blind spot, and its location and/or scope (e.g., in area and/or distance from the vehicle) may vary depending on the size of the vehicle. In vehicles with large blind spots, such as large vehicles and especially large commercial vehicles such as tractor trailers, placards are often placed on the trailer warning trailing vehicles that the inability by the driver of a trailing vehicle to see a reflection of the driver of the tractor in the rear view mirror of the tractor means that the driver of the tractor is unable to see the trailing vehicle because the trailing vehicle is in the tractor-trailer blind spot. In some commercial vehicles, for instance those involved in commercial excavation sites such as large haul or dump trucks, the blind spot area is much more extensive than the blind spot area for smaller, passenger vehicles, particularly when making hard or sharp turns. The blind spot issue is not limited to large commercial vehicles, though, as operators of large recreational vehicles may face similar challenges.
When maneuvering the vehicle into turns, there is a risk of personal injury and/or damage to all proximally located vehicles, property, or people (hereinafter, objects) in a blind spot intersected by the path or trajectory of the large vehicle. What is needed is a way to detect the presence of objects that are located within the blind spot of the large vehicle while also alerting the driver of the large vehicle of the presence of the vehicle in the blind spot with sufficient time to avoid collision.
In one embodiment, a blind spot detection system for a vehicle, the system comprising: one or more sensors configured to detect vehicle movements; a memory comprising instructions; and a controller configured by the instructions to: predict an impending sharp turn of the vehicle based on receiving a signal or signals from the one or more sensors; adjust a detection range from a first setting based on the prediction; and return to the first setting of the detection range upon termination of a sharp turn by the vehicle.
Other systems, methods, features, and advantages of the present invention will be or become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that such additional systems, methods, features, and advantages be included within this description, be within the scope of the present invention, and be protected by the accompanying claims.
Many aspects of the invention can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present invention. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.
Certain embodiments of a blind spot detection system and method are disclosed that uses adaptive sensing to boost the safety and maneuverability of large vehicles, or more generally, vehicles with large blind spots, by predicting an impending sharp turn and assisting an operator (e.g., driver) with increased visibility or reduced blind spot, and early warning to avoid a collision with objects located within the blind spot while in the sharp turn. In one embodiment, the blind spot detection system uses sensor input (e.g., location and/or inertial sensors) to predict whether the vehicle is about to negotiate a sharp turn, and accordingly, adjusts a detection range relative to a first setting (e.g., detection settings used during relatively straight driving or wide but not sharp turns) based on the prediction for detection of objects during the sharp turn maneuver. The adjusted-to setting, or second setting, comprises an extended protection zone or range that includes extending detection at the turning side of the vehicle. Upon the vehicle ceasing the sharp turn (e.g., returning to a wide turn, proceeding in a straight path, or stopping), the blind spot detection system returns from the extended protection zone setting to the first setting of the detection range while continuously monitoring the sensor input to anticipate and respond to any subsequent sharp turns.
Digressing briefly, large vehicles (e.g., mining or hauling trucks, at weights of approximately 26,000 pounds or more) possess a larger blind spot than smaller vehicles (e.g., passenger sedans, small pickup trucks, etc.), and particularly during sharp turning maneuvers, risk collision with other vehicles (or generally, objects) that are undetected (e.g., in the blind spot) by the operator. Large vehicles, given their weight, also require a longer distance to stop than smaller vehicles. Technology, such as radar, helps to effectively reduce this blind spot (e.g., expand detection), but depending on the surrounding terrain, may provide false alerts, such as from boulders or other objects not on a collision course with the vehicle. In contrast, certain embodiments of a blind spot detection system use location and/or inertial sensors and knowledge-base/heuristics (e.g., gathering a large quantity of raw data and developing rules) to predict a sharp turn (also referred to as a hard turn) and extend the detection range on the turning side (and in some embodiments, forward and rearward of) the vehicle during the sharp turn, mitigating the risk of collision due to an effectively reduced blind spot and expanded sensing for enabling timely alerts to the operator.
Having summarized certain features of a blind spot detection system of the present disclosure, reference will now be made in detail to the description of a blind spot detection system as illustrated in the drawings. While a blind spot detection system will be described in connection with these drawings, there is no intent to limit it to the embodiment or embodiments disclosed herein. For instance, in the description that follows, there is an emphasis on large commercial vehicles, including a hauler used as an illustrative example, though other types of large commercial vehicles may be used. Further, the blind spot detection system may be used in other types of vehicles, including large non-commercial vehicles (e.g., recreational vehicles), which may also have a larger blind spot than smaller, passenger vehicles. The vehicle described herein may utilize any one of a plurality of different existing steering systems, including the typical hydraulic steering gear, (powered) rack and pinion steering mechanisms, a steer-by-wire steering system that uses a joystick or steering wheel as the operator interface, etc. Since emphasis herein is not on the type or control of steering, as such systems exist today, description of the same herein is omitted to avoid obfuscating the focus of the disclosure. Further, though the vehicles described herein are generally equipped with a cab in which an operator controls the vehicle, in some embodiments, the vehicle may be operated with or without an operator in the cab in an autonomous, semi-autonomous, or remote-control manner in which cameras are used in place of, or as a supplement to, proximal and direct vehicle operational control (e.g., where the operator sits in the cab and controls the steering wheel, accelerator pedals, brakes, etc.). Further, although the description identifies or describes specifics of one or more embodiments, such specifics are not necessarily part of every embodiment, nor are all various stated advantages necessarily associated with a single embodiment or all embodiments. On the contrary, the intent is to cover alternatives, modifications and equivalents included within the scope of the invention as defined by the appended claims. Further, it should be appreciated in the context of the present disclosure that the claims are not necessarily limited to the particular embodiments set out in the description.
Note that reference herein to steering or steering control is to be understood as steering control characterized primarily by direct operator intervention, as opposed to what may be characterized primarily as machine or satellite guided control, though as explained above, some embodiments may use autonomous or semi-autonomous control of the vehicle proximal to, or remote from, the vehicle and benefit from a blind spot detection system as described herein.
Note also that references hereinafter made to certain directions, such as, for example, “front”, “rear”, “left” and “right”, are made as viewed from the rear of a vehicle looking forwardly.
Additionally, a sharp or hard turn is distinguished from one that is not based on, for instance, the speed of the vehicle and turning angle. For instance, a vehicle may turn in a complete circle, where the turn in a span of one minute may not rise to a level of force (e.g., acceleration or centripetal force) to be characterized as a sharp or hard turn than the same turn in, say, five seconds. As another example, at a given speed, a vehicle turning may experience greater force at an angle of eighty (80) degrees than the same vehicle at an angle of one hundred, twenty (120) degrees. A hairpin turn may qualify as a sharp turn based on the speed at which the vehicle travels. Additionally, a vehicle moving quickly through a turn poses a greater risk of collision than one moving slowly through the turn, since a driver requires a quicker time to react and the vehicle needs more time to stop or more distance to stop. The decision as to what qualifies as a sharp turn or a fast turn may be based on analysis of data for any given application and vehicle or vehicle type, as explained further below.
Referring now to
Explaining further, and referring now to
Certain embodiments of a blind spot detection system adjusts a detection range from that provided by the detection zone 26 to the extended detection zone 28, the latter which extends the range at least to the turning side of the vehicle 10. By extending the detection to the turning side of the vehicle 10, the object 24 is detectable during a sharp turn and, because of the prediction, within sufficient time for the operator (or collision avoidance technology within the vehicle 10) to avoid a collision with the object 24. Stated otherwise, the prediction mechanisms of the blind spot detection system enables an adjustment (extension, or the extended detection zone 28) in the detection range (e.g., from the first setting reflected by the detection zone 26) with sufficient time to deploy collision avoidance measures while in the sharp turn. It should be appreciated by one having ordinary skill in the art, in the context of the present disclosure, that the extended detection zone 28 may cover additional areas. For instance, though the extended detection zone 28 is shown in
In some embodiments, the blind spot detection system detects and enters the sharp turn at approximately eighty-degrees (80 degrees), and adjusts (e.g., extends) the detection range to approximately fifteen (15) meters (m) or approximately forty-nine (49) ft. in front of the vehicle 10 and 15 meters on the turning side of the vehicle 10. In one embodiment, detection of the object 24 may be achieved via camera sensor in conjunction with image processing (e.g., existing object detection/recognition software). In some embodiments, the detection of the object 24 may be achieved with the camera/software and also other sensing technology. For instance, radar technology (e.g., a radar sensor) may also be used, where the range of the radar sensor may be at or less than the range of the camera sensor (e.g., to avoid false alerts caused by detection of surrounding objects, such as boulders, trees, etc.). In some embodiments, the blind spot detection system may disable radar detection in the side (turning side) zone during a sharp turn to prevent false alerts (e.g., from boulders in the extended range). In some embodiments, one or more additional sensors may be used. In one embodiment, the blind spot detection system may also receive feedback of the steering wheel (or whatever navigating device the operator uses to control the movement of the vehicle) during the prediction and/or detection process. For instance, the direction in which the operator maneuvers the steering wheel may provide preliminary insight to the blind spot detection system as to whether the operator intends to straighten the vehicle or turn the vehicle (e.g., make a hard turn). Based on these example values, the blind spot detection system provides for an alert to the operator within one (1) second from the time of detection (e.g., assuming a built-in 0.3 second lag reaction time of the operator). If the speed of the vehicle is, say, five (5) miles per hour (MPH), the vehicle 10 has approximately 7.8 meters (approximately twenty-five (25) ft.) to come to a full stop.
It is noted that prediction of the sharp turn is performed during negotiation of the turn, and in one embodiment, before or at the time of entering the latter half of the turn (e.g., at approximately ninety-degrees), and not delayed until after the vehicle is already in the turn, to enable sufficient reaction time for detection and collision avoidance. In one embodiment, a threshold may be established for the turning angle that balances the need for sufficient reaction time while avoiding unnecessary activation of the extended detection range (e.g., for normal, ninety-degree (90 degree) turns). In one embodiment, the threshold is set at, or approximately at, seventy-degrees (70 degrees). In some embodiments, the threshold is set at, or approximately at, eighty-degrees (80 degrees). In some embodiments, the threshold is set at, or approximately at, ninety-degrees (90 degrees). In some embodiments, such as to account for a reduced approach angle, the threshold may be set at less than ninety-degrees (90 degrees). Note that the determination of threshold angle may also be set based on the approach angle. For instance, curved line 34 may correspond to a trajectory in which the operator maneuvers the vehicle along a wider angle of approach (e.g., hugging the left side of the road before entering the turn), which may alter the point in which a hard or sharp turn is predicted.
In some embodiments, filtering and timeout mechanisms may be used to eliminate and/or reduce the time spent in a sharp turn mode of operation.
In the S2 state, the state machine determines whether, even though in a large turn mode, the turn is a hard or sharp turn. The !HardTurn indicates that the turn is not a hard turn (and if not a hard turn, the state machine remains in the not hard turn mode), and the !FastTurn indicates that the vehicle is not making a fast turn (if the vehicle turn is not fast, the machine state remains in the not fast mode). In one embodiment, a fast turn may relate to an average turning speed of the entire turning process, whereas a sharp/hard turn may relate to travel speed during a later phase of the turning process. The thresholds may be set (e.g., based on historical data, testing, vehicle and/or terrain parameters, etc.) accordingly to categorize a turning process as falling into one, both or neither the fast and sharp/hard turn types. If the vehicle is not making a hard turn, and not making a fast turn, then then the state machine transitions to a watch state S2′ or large-turn keep mode. For instance, during a large turn, the operator may pause the vehicle, and then continue, such as to participate in another task or as a precaution before proceeding. The angle accumulation stops, but the turn has not ended yet, and thus instead of returning to S0 or S1, the state machine remains at the S2′ or large-turn keep mode.
The state machine may remain in the S2′ state until there is a time out (e.g., ImuSharpTurnKepTimeexpired). For instance, when the time out arrives, and the operator is still driving very slowly, then there is an assumption that the operator does not need the extended protection since the operator is cautious. Accordingly, the state machine exits to the turning mode (monitoring state transitions between S0 and S1). On the other hand, if the operator begins to turn fast while in the large turning mode, then the state machine proceeds to the S2 mode and determines whether the operator is maneuvering the vehicle in a hard turn or not.
Returning again to the S2 state, if the angle becomes very sharp (e.g., in the latter portion of the large turn), then the state machine transitions to the S3 state. For instance, the state machine may determine whether the turn is sharp or not using the last, say, 20% of the current turn, and based on the turn being within a certain amount of time and within a certain distance, the turn is considered sharp. For instance, and as explained above, if the time for the vehicle to make a ninety-degree turn is one minute, the turn is a large turn, though not a sharp turn. On the other hand, if the 90 degree turn is achieved in, say, five seconds, then it may be considered a sharp turn, which causes the extended detection zone to be deployed.
The S3′ state (the so-called sharp-turn keep mode) is analogous to the S2′ state—i.e., it is a watching or suspended state. For instance, once in an S3 turn (middle of a sharp or hard turn), the operator may cause the vehicle to slow down, giving rise to the S3′ state that provides the operator with an ability to observe and/or take precautions while still in the sharp turn.
Sensor input may be used to perform computations (e.g., by an inertial measurement unit), and the computations are in turn used by a controller to make predictions of sharp turns. The following equations or formulas are set forth below and identified with the various states from the state machine depicted by the state diagram 36 in
MA25(angular-speed)≥ImuTurnSpeedThreshold (Eq. 1)
Average Angle≥ImuSharpTurnAverageAngleSpeedThreshold (Eq. 2)
(Accumulated Angle≥ImuSharpTurnAngleThreshold)) (Eq. 3a)
Without the FastTurn predicate, a vehicle making a sharp turn at a very slow turning speed (!FastTurn) may enter the LargeAngle state prematurely, then after keep time goes back to the Turning state, which resets the Accumulated Angle.
Alternatively, Eq. 3b, immediately below, may be used:
(Accumulated Angle≥ImuTurnAngleDeterminateSharpThreshold)
The operator may not see the blind spot regardless of turning speed. This condition gives a vehicle making a sharp turn at a very slow turning speed a window of ImuSharpTurnKeepTime seconds to detect an object with an extended detection zone after reaching ImuTurnAngleDeterminateSharpThreshold.
HardTurn (Eq. 4 immediately below):
Traveling distance during last {ImuSharpTurnAngleThreshold−CalculateTurnDistanceStartAngle} degrees<SharpturnNeedToLowerDistance
SharpTurnNeedToLowerThanDistance>2.5*30*Pi*(ImuSharpTurnAngleThreshold−CalculateTurnDistanceStartAngle)/360 for CAT-775.
In a filtering step, the distance traveled between CalculateTurnDistanceStartAngle and ImuSharpTurnAngleThreshold is compared to SharpTurnNeedToLowerThanDistance to determine whether the turn is a sharp turn (true positive) or simply a large turn (false positive).
SlowTurn OR (!HardTurn) (Eq. 5)
SharpTurnMode may be S3 or S3′.
Attention is now directed to operator feedback mechanisms used by certain embodiments of a blind spot detection system when entering and in a sharp turn. Referring to
As another example,
In one embodiment, the controller 50 comprises one or more processors, such as processor 51, input/output (I/O) interface(s) 52, and memory 53, all coupled to one or more data busses, such as data bus 54.
The memory 53 may include any one or a combination of volatile memory elements (e.g., random-access memory RAM, such as DRAM, and SRAM, etc.) and nonvolatile memory elements (e.g., ROM, hard drive, tape, CDROM, etc.). The memory 53 may store a native operating system, one or more native applications, emulation systems, or emulated applications for any of a variety of operating systems and/or emulated hardware platforms, emulated operating systems, etc. In the embodiment depicted in
The processor 51 may be embodied as a custom-made or commercially available processor, a central processing unit (CPU) or an auxiliary processor among several processors, a semiconductor based microprocessor (in the form of a microchip), a macroprocessor, one or more application specific integrated circuits (ASICs), a plurality of suitably configured digital logic gates, and/or other existing electrical configurations comprising discrete elements both individually and in various combinations to coordinate the overall operation of the controller 50.
The I/O interfaces 52 provide one or more interfaces to a network comprising a communication medium 60, which may be a wired medium (e.g., controller area network (CAN) bus) as depicted in
The user interface device(s) 62 may include a keyboard, mouse, microphone, touch-type display device, head-set, smart phone, tablet, and/or other devices (e.g., switches) that enable input by an operator and/or feedback to the operator (e.g., presentation or display of the user interfaces 38A and 38B of
The location device 64 may include a global navigation satellite system (GNSS), including global positioning system (GPS), or the like, providing location, distance traveled, speed, etc.
The inertial measurement unit 66 comprises existing technology, including accelerometer(s), gyroscope(s), and other circuitry to compute angular speed, pitch, roll, yaw, speed (e.g., based on accumulated acceleration over time), vehicle acceleration, and relative positioning relative of the vehicle.
The image acquisition device (camera) 68 may include a camera for providing visualization to the operator of detection zones and objects within the detection zones.
The sensors 70 may include additional sensors used in the navigation and/or sensing for the vehicle, including radar, lidar, acoustic sensors, steering wheel angle sensors, etc.
The blind spot detection software 58 comprises executable code/instructions that, when executed by the processor 51, receives input from the location device 64, inertial measurement unit 66, camera 68, and sensors 70, and predicts/detects a sharp turn and adjusts a detection zone (determined according to base settings) to an extended detection zone or range. The blind spot detection software 58 may present feedback of the detection zone, including any detected objects and camera views, on the user interface device 62 via a user interface with alerts and views (e.g., user interfaces 38A, 38B of
Execution of the blind spot detection software 58 is implemented by the processor 51 under the management and/or control of the operating system 56. In some embodiments, the operating system 56 may be omitted and a more rudimentary manner of control implemented.
In some embodiments, functionality of the blind spot detection software 58 may be distributed among plural controllers (and hence, plural processors). For instance, each controller may be similarly configured in hardware and/or software (e.g., one or more processors, memory comprising executable code/instructions, etc.) as the controller 50, with the control strategy including a peer-to-peer or primary-secondary control arrangement.
When certain embodiments of the controller 50 are implemented at least in part with software (including firmware), as depicted in
The software may be embedded in a variety of computer-readable mediums for use by, or in connection with, an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
When certain embodiments of the controller 50 are implemented at least in part with hardware, such functionality may be implemented with any or a combination of the following technologies, which are all well-known in the art: a discrete logic circuit(s) having logic gates for implementing logic functions upon data signals, an application specific integrated circuit (ASIC) having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), etc.
In view of the above description, it should be appreciated within the context of the present disclosure that one embodiment of an example blind spot detection method, denoted as method 78 (e.g., as implemented at least in part by the blind spot detection software 58,
Any process descriptions or blocks in flow diagrams should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the embodiments in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present disclosure.
In this description, references to “one embodiment”, “an embodiment”, or “embodiments” mean that the feature or features being referred to are included in at least one embodiment of the technology. Separate references to “one embodiment”, “an embodiment”, or “embodiments” in this description do not necessarily refer to the same embodiment and are also not mutually exclusive unless so stated and/or except as will be readily apparent to those skilled in the art from the description. For example, a feature, structure, act, etc. described in one embodiment may also be included in other embodiments, but is not necessarily included. Thus, the present technology can include a variety of combinations and/or integrations of the embodiments described herein. Although the control systems and methods have been described with reference to the example embodiments illustrated in the attached drawing figures, it is noted that equivalents may be employed and substitutions made herein without departing from the scope of the disclosure as protected by the following claims.
This application claims the benefit of U.S. Provisional Application No. 63/493,323 filed Mar. 31, 2023, which is hereby incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 63493323 | Mar 2023 | US |
