Patent Application
20180225964
The present disclosure generally relates to vehicles, and more particularly relates to systems and methods for presenting situation awareness information to occupants in a vehicle.
An autonomous vehicle is a vehicle that is capable of sensing its environment and navigating with little or no user input. An autonomous vehicle senses its environment using sensing devices such as radar, lidar, image sensors, and the like. The autonomous vehicle system further uses information from global positioning systems (GPS) technology, navigation systems, vehicle-to-vehicle communication, vehicle-to-infrastructure technology, and/or drive-by-wire systems to navigate the vehicle.
Vehicle automation has been categorized into numerical levels ranging from Zero, corresponding to no automation with full human control, to Five, corresponding to full automation with no human control. Various automated driver-assistance systems, such as cruise control, adaptive cruise control, and parking assistance systems correspond to lower automation levels, while true “driverless” vehicles correspond to higher automation levels. At any level, information obtained by the vehicle and relied upon to perform the autonomous control is not typically presented to a driver or passenger of the vehicle. Such information, referred to as situation awareness information, is defined as information associated with the perception of environmental elements with respect to time or space, the comprehension of their meaning, and the projection of their status after some variable has changed, such as time, or some other variable, such as a predetermined event.
If a driver or passenger were presented with situation awareness information they may be more comfortable riding in the autonomous vehicle. Accordingly, it is desirable to provide systems and methods that provide situation awareness information to occupants in a way that excites the human visceral processing system in an autonomous vehicle or other type of vehicle. It is further desirable to provide systems and methods that alert and warn an occupant about the situation around the vehicle and potential threats. It is further desirable to provide methods and systems that advise an occupant regarding the feasibility and probability of possible trajectories. It is still further desirable to provide methods and systems that inform occupants about trajectories taken. Furthermore, other desirable features and characteristics of the present invention will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and the foregoing technical field and background.
Systems and method are provided for generating an integrated interface in a vehicle. In one embodiment, a method includes: receiving data associated with an environment of the vehicle; encoding the environment data into a plurality of layers; extracting data from the plurality of layers into a point cloud; projecting the point cloud onto a meta-template defining a basic element and manipulable features of the basic element; and generating display data based on the projecting.
The exemplary embodiments will hereinafter be described in conjunction with the following drawing figures, wherein like numerals denote like elements, and wherein:
The following detailed description is merely exemplary in nature and is not intended to limit the application and uses. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, brief summary or the following detailed description. As used herein, the term module refers to any hardware, software, firmware, electronic control component, processing logic, and/or processor device, individually or in any combination, including without limitation: application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and memory that executes one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality.
Embodiments of the present disclosure may be described herein in terms of functional and/or logical block components and various processing steps. It should be appreciated that such block components may be realized by any number of hardware, software, and/or firmware components configured to perform the specified functions. For example, an embodiment of the present disclosure may employ various integrated circuit components, e.g., memory elements, digital signal processing elements, logic elements, look-up tables, or the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices. In addition, those skilled in the art will appreciate that embodiments of the present disclosure may be practiced in conjunction with any number of systems, and that the systems described herein is merely exemplary embodiments of the present disclosure.
For the sake of brevity, conventional techniques related to signal processing, data transmission, signaling, control, and other functional aspects of the systems (and the individual operating components of the systems) may not be described in detail herein. Furthermore, the connecting lines shown in the various figures contained herein are intended to represent example functional relationships and/or physical couplings between the various elements. It should be noted that many alternative or additional functional relationships or physical connections may be present in an embodiment of the present disclosure.
With reference to
As depicted in
In various embodiments, the vehicle 10 is an autonomous vehicle and the situation awareness information system 100 is incorporated into the autonomous vehicle 10 (hereinafter referred to as the autonomous vehicle 10). The autonomous vehicle 10 is, for example, a vehicle that is automatically controlled to carry passengers from one location to another. The vehicle 10 is depicted in the illustrated embodiment as a passenger car, but it should be appreciated that any other vehicle including motorcycles, trucks, sport utility vehicles (SUVs), recreational vehicles (RVs), marine vessels, aircraft, public transportation, etc., can also be used. In an exemplary embodiment, the autonomous vehicle 10 is a so-called Level Four or Level Five automation system although the present disclosure may be implemented for any level of automation. A Level Four system indicates “high automation”, referring to the driving mode-specific performance by an automated driving system of all aspects of the dynamic driving task, even if a human driver does not respond appropriately to a request to intervene. A Level Five system indicates “full automation”, referring to the full-time performance by an automated driving system of all aspects of the dynamic driving task under all roadway and environmental conditions that can be managed by a human driver.
As shown, the autonomous vehicle 10 generally includes a propulsion system 20, a transmission system 22, a steering system 24, a brake system 26, a sensor system 28, an actuator system 30, at least one data storage device 32, at least one controller 34, and a communication system 36. The propulsion system 20 may, in various embodiments, include an internal combustion engine, an electric machine such as a traction motor, and/or a fuel cell propulsion system. The transmission system 22 is configured to transmit power from the propulsion system 20 to the vehicle wheels 16-18 according to selectable speed ratios. According to various embodiments, the transmission system 22 may include a step-ratio automatic transmission, a continuously-variable transmission, or other appropriate transmission. The brake system 26 is configured to provide braking torque to the vehicle wheels 16-18. The brake system 26 may, in various embodiments, include friction brakes, brake by wire, a regenerative braking system such as an electric machine, and/or other appropriate braking systems. The steering system 24 influences a position of the of the vehicle wheels 16-18. While depicted as including a steering wheel for illustrative purposes, in some embodiments contemplated within the scope of the present disclosure, the steering system 24 may not include a steering wheel.
The sensor system 28 includes one or more sensing devices 40a-40n that sense observable conditions of the exterior environment and/or the interior environment of the autonomous vehicle 10. The sensing devices 40a-40n can include, but are not limited to, radars, lidars, global positioning systems, optical cameras, thermal cameras, ultrasonic sensors, and/or other sensors. The actuator system 30 includes one or more actuator devices 42a-42n that control one or more vehicle features such as, but not limited to, the propulsion system 20, the transmission system 22, the steering system 24, and the brake system 26. In various embodiments, the vehicle features can further include interior and/or exterior vehicle features such as, but are not limited to, doors, a trunk, and cabin features such as air, music, lighting, etc. (not numbered).
The communication system 36 is configured to wirelessly communicate information to and from other entities 48, such as but not limited to, other vehicles (“V2V” communication,) infrastructure (“V2I” communication), remote systems, and/or personal devices (described in more detail with regard to
The data storage device 32 stores data for use in automatically controlling the autonomous vehicle 10. In various embodiments, the data storage device 32 stores defined maps of the navigable environment. In various embodiments, the defined maps may be predefined by and obtained from a remote system (described in further detail with regard to
The controller 34 includes at least one processor 44 and a computer readable storage device or media 46. The processor 44 can be any custom made or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the controller 34, a semiconductor based microprocessor (in the form of a microchip or chip set), a macroprocessor, any combination thereof, or generally any device for executing instructions. The computer readable storage device or media 46 may include volatile and nonvolatile storage in read-only memory (ROM), random-access memory (RAM), and keep-alive memory (KAM), for example. KAM is a persistent or non-volatile memory that may be used to store various operating variables while the processor 44 is powered down. The computer-readable storage device or media 46 may be implemented using any of a number of known memory devices such as PROMs (programmable read-only memory), EPROMs (electrically PROM), EEPROMs (electrically erasable PROM), flash memory, or any other electric, magnetic, optical, or combination memory devices capable of storing data, some of which represent executable instructions, used by the controller 34 in controlling the autonomous vehicle 10.
The instructions may include one or more separate programs, each of which comprises an ordered listing of executable instructions for implementing logical functions. The instructions, when executed by the processor 44, receive and process signals from the sensor system 28, perform logic, calculations, methods and/or algorithms for automatically controlling the components of the autonomous vehicle 10, and generate control signals to the actuator system 30 to automatically control the components of the autonomous vehicle 10 based on the logic, calculations, methods, and/or algorithms. Although only one controller 34 is shown in
In various embodiments, one or more instructions of the controller 34 are embodied in the situation awareness information system 100 and, when executed by the processor 44, cause situation awareness information to be selectively presented to an occupant of the vehicle 10 in a manner that does not overload the occupant. For example, environment information is processed to selectively present certain situation awareness information in an integrated and holistic interface 50; and the environment information is processed to selectively present the situation awareness information in the integrated interface 50 in a consistent manner for any use case scenario (e.g., as presented in an uncertain an dynamic environment).
In various embodiments, the integrated interface 50 includes an ego-centric representation of the host vehicle and regions of operation around the representation. For example, the ego-centric representation can be in the form of one or more basic elements such as a circle, a square, a rectangle, an octagon, a hexagon, etc. the represents the vehicle 10. The basic element includes features that are manipulable (shaped or distorted) to illustrate elements of the exterior environment of the vehicle 10. The features can be manipulated based on a mapping to a cloud of potential points that represent anything that can influence on the decisions that the automated system makes.
In various embodiments, the influence may be either positive or negative. For example, center of lanes are sampled into positive points that attracts the host vehicle, and road boundaries are sampled into negative points that repel the host vehicle. Similarly, other obstacles can be represented by negative points. And there may be other considerations such as, but not limited to, time, gasoline consumption, aggressive-defensive driving behavior that can all be represented by appropriate potential points with an associated weight (that can take any positive or negative value). At any time point, just after all the relevant potential points are assigned around the host vehicle, a physical equation calculates the overall impact of all the potential points around the host vehicle allowing for the shaping or distortion.
In various embodiments, as shown in
For example, changing a value of the length or the width of a bar 54 or a cell 56 may change the geometry of the overall basic element 52 to illustrate a value of a force such as a detected object acting on or that may act on the vehicle 10. In another example, changing a value of the length or width may change a size of the basic element 52 to illustrate a value of a speed or other forces of the vehicle 10. In another example, changing a value of the pattern may change a connectivity of an outline to illustrate a value of danger to the vehicle 10. In another example, changing a value of the color may change a color of the basic element 52 to illustrate a threat level. In another example, changing a value of transparency may change a color saturation or transparency to illustrate a value of probability of an action. As can be appreciate, the parameters and associations are merely exemplary as any number of parameters can be associated with a variety of information in various embodiments.
With reference now to
As can be appreciated,
Referring now to
In various embodiments, the information awareness system 100 includes an encoder module 70, an extraction module 72, and a projection module 74. The functions of the modules 70-74 are performed continuously on received information to shape the display according to the dynamically changing environment.
For example, the encoder module 70 receives sensed or perceived data 76 associated with the exterior environment of the vehicle and encodes the sensed or perceived information into layers 78 that represent driving objectives and constraints. The layers 78 may include, for example, a geometry layer, a kinematic layer, a safety layer, object meaning layer, etc. The geometry layer encodes, for example, road geometry (i.e., road boundaries, intersection structures, etc.). The kinematic layer, for example, encodes the vehicle's kinematics and dynamic state and ability (i.e., the vehicle's momentum, the vehicle's kinematic envelope, etc.). The safety layer, for example, encodes surrounding obstacles (i.e., other vehicles, pedestrians, etc.) The encoding can be implicit representations and/or explicit representations of the objects. The object meaning layer, for example, encodes meanings and/or importance levels of objects.
The extraction module 72 receives the layers 78 and extracts data from the layers 78 to a cloud of spatio-temporal potential points (i.e., that have an effect on the vehicle) 80. For example, the data from the layers 78 are extracted to the spatio-temporal cloud of potential points where each point has a position and a time with respect to the host vehicle. The data can include all of the data from the layer or a selected some of the data. For example, the data can include a potential weight that reflects its positive (attraction) or negative (repulsion) quality of an object.
The projection module 74 receives the cloud of points 80 and a meta-template or templates 82 of a basic element having manipulable features (e.g., as discussed above) and projects the potential field defined by the cloud of potential points on the meta-template(s) of the display design. For example, when the data includes a weight reflecting an attraction, a force field analysis is performed that is a physical-based analysis that uses the forces derived from the cloud of potential points in order to shape (e.g., adjust the length and/or width parameters) a quasi-static instance of the display as defined by the meta-template(s) 82. As can be appreciated, other data can be analyzed to cause changes in other parameters of the meta-template(s) 82 as this is merely an example.
The projection module 74 then generates display data 84 to display the projection of the cloud of points 80 onto the meta-template on a display in the vehicle for viewing by an occupant.
Referring now to
In various embodiments, the method 400 may begin at 405. The sensed and/or perceived environment data 76 is received at 410. The environment data 76 is then processed at 420 and extracted into layers 78. The layers 78 are then extracted into a point cloud 80 at 430. The point cloud 80 is then projected onto the meta-template(s) 82 at 440 and display data 84 is generated. Thereafter, the method may end at 450.
While at least one exemplary embodiment has been presented in the foregoing detailed description, it should be appreciated that a vast number of variations exist. It should also be appreciated that the exemplary embodiment or exemplary embodiments are only examples, and are not intended to limit the scope, applicability, or configuration of the disclosure in any way. Rather, the foregoing detailed description will provide those skilled in the art with a convenient road map for implementing the exemplary embodiment or exemplary embodiments. It should be understood that various changes can be made in the function and arrangement of elements without departing from the scope of the disclosure as set forth in the appended claims and the legal equivalents thereof.
