Patent Application
20200294414
The present disclosure generally relates to a vehicle driving simulator. More specifically, the present disclosure relates to driving simulator integrated with a mobility computer-aided experience (CAE).
Vehicle driving simulators are used to provide driving simulations for various scenarios. Professional drivers such as bus drivers may be trained using driving simulators before operating real vehicles on public roads. However, driving simulators may be unrealistic as the driving conditions may not accurately resemble real conditions. In addition, this simulation environment helps with digitally prototyping a mobility service to save time, cost and resources.
In one or more illustrative embodiment of the present disclosure, a driving simulation platform includes one or more controllers of a driving simulator, programmed to perform a driving simulation for a pre-designed use case selected by a user via a web-based configuration interface, the driving simulation using road data imported from a cloud server; receive a signal to provide to an external device in communication with the driving simulation platform, the external device providing additional information in support of the simulation; and responsive to receiving a response from the external device, record the response as a simulation record.
In one or more illustrative embodiment of the present disclosure, a method for a driving simulator includes responsive to receiving a user input via a web-based configuration application, importing a 3D city model and road network data into the driving simulator from a database; starting a driving simulation for a pre-designed use case selected by a user via the web-based configuration application; responsive to receiving a functionality input via the web-based configuration application, adjusting a functionality control for the simulation during a process of the simulation; and responsive to receiving a message from an external device, recording the message as a simulation record in a storage.
In one or more illustrative embodiment of the present disclosure, a non-transitory computer readable medium includes instructions, when executed by a driving simulator, cause the driving simulator to: responsive to receiving a user input via a web-based configuration application, import a 3D city model into the driving simulator from a cloud server; starting a driving simulation for a pre-designed use case selected by a user via the web-based configuration application; responsive to receiving a signal for an external device in communication with the driving simulator, send the signal to the external device; and responsive to receiving feedback from an external device responding the signal, recording the feedback as a simulation record in a storage.
For a better understanding of the invention and to show how it may be performed, embodiments thereof will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which:
As required, detailed embodiments of the present invention are disclosed herein; however, it is to be understood that the disclosed embodiments are merely exemplary of the invention that may be embodied in various and alternative forms. The figures are not necessarily to scale; some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present invention.
The present disclosure generally provides for a plurality of circuits or other electrical devices. All references to the circuits and other electrical devices, and the functionality provided by each, are not intended to be limited to encompassing only what is illustrated and described herein. While particular labels may be assigned to the various circuits or other electrical devices, such circuits and other electrical devices may be combined with each other and/or separated in any manner based on the particular type of electrical implementation that is desired. It is recognized that any circuit or other electrical device disclosed herein may include any number of microprocessors, integrated circuits, memory devices (e.g., FLASH, random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), or other suitable variants thereof) and software which co-act with one another to perform operation(s) disclosed herein. In addition, any one or more of the electric devices may be configured to execute a computer-program that is embodied in a non-transitory computer readable medium that is programed to perform any number of the functions as disclosed.
The present disclosure, among other things, proposes a vehicle driving simulator. More specifically, the present disclosure proposes a driving simulator integrated with CAE based on internet-of-things (IoT) platform.
Referring to
The driving simulator 102 may be provided with various features allowing users to interface with the driving simulator 102. For instance, the driving simulator 102 may receive input from human-machine interface (HMI) controls 110 configured to provide for user interaction with the driving simulator 102. As an example, the driving simulator 102 may interface with an input/output (I/O) controller 112 or other controllers via the HMI controls 110. The I/O controller 112 may include a steering wheel, a gear shifter, pedals or the like configured to provide the user with driving inputs to simulate a vehicle driving environment.
The driving simulator 102 may also send signals to or otherwise communicate with one or more displays 114 configured to provide visual output to a user by way of a video controller 116. In some cases, the display 114 may be provided with touch screen features configured to receive user touch input via the video controller 116, while in other cases the display 114 may be a display only, without touch input capabilities. The display 114 may be a liquid-crystal display (LCD), active-matric organic light-emitting diode display (AMOLED), a head up display (HUD), a projector, virtual reality (VR) glasses, augmented reality (AR) glasses, or mixed reality (MR) glasses as a few non-limiting examples. The driving simulator 102 may also drive or otherwise communicate with one or more speakers 118 configured to provide audio output to the user by way of an audio controller 120.
The simulator applications 106 may include various applications or software configured to perform various features. For instance, the simulator applications 106 may include a simulation engine 122 configured generate driving simulations for the user to simulate driving environment include street, city, signals, traffics or the like. The simulator applications 106 may further include a configuration application 124 configured to provide an interface to allow the user to configure and adjust parameters for driving simulations. The configuration application 124 may be configured to support a web-based input from a web (to be introduced below). Digital data used to perform simulations may be stored in the storage 108 as a part of simulator data 126. For instance, the simulator data 126 may include data models simulating streets, traffics, and different vehicles, to provide a variety of simulation options. The simulator data 126 may further include user profiles associate with one or more users configure to provide driving records of the users.
The driving simulator 102 may be further provided with a network controller 128 configured to communicate with a cloud 130 e.g. using a modem (not shown). The term cloud is used as a general term in the present disclosure and may include any computing network involving computers, servers, controllers or the like configured to perform data processing and storage functions and facilitate communication between various parties. The driving simulator 102 may be configured download and upload simulator applications 106 and simulation data 126 from and to the cloud.
The driving simulator 102 may be further configured to wirelessly communicate with an external device 132 via a wireless transceiver 134 through a wireless connection 136. The external device 132 may be any of various types of portable computing device, such as cellular phones, tablet computers, wearable devices, smart watches, laptop computers, vehicle scan tool, or other device capable of communication with the driving simulator 102. A wireless transceiver 134 may be in communication with a Wi-Fi controller 136, a Bluetooth controller 138, a radio-frequency identification (RFID) controller 140, a near-field communication (NFC) controller 142, and other controllers such as a Zigbee transceiver, an IrDA transceiver (not shown), and configured to communicate with a compatible wireless transceiver (not shown) of the external device 132. Additionally or alternatively, the driving simulator 102 may be configured to communicate with the external device via a wired connector 144 through a cable 146. The wired connector may be configured to support various connection protocols including universal serial bus (USB), Ethernet, or on-board diagnostics 2 (OBD-II) as a few non-limiting examples.
Referring to
The inputs 204 may further include road network data 208 configured to provide road network data to simulate roads. For instance, the road network data may include various map and road application programming interfaces (APIs) such as Google Maps®, Mapbox®, Here®, or the like associated with one or more third parties, to provide the user with a more realistic road simulation environment. The inputs 204 may further include signal timing data 210 configured to provide street signals data for simulation purposes. Some cities use adaptive or coordinated traffic signal schemes to improve traffic conditions. The signal timing data 210 may include traffic signal data, timer control data, and/or other signal time data to provide more accurate simulations to various traffic schemes. The inputs 204 may further include a use-case specific inputs 212, such as stop locations, delivery targets, and/or origin-destination (OD) pairs to provide specific inputs for each simulation use case.
Since the UI layer 202 may be configured to support inputs 204 in various formats/forms, the mobility CAE platform 200 may further include a data ingestion layer 214 configured to convert the inputs 204 received via the UI layer 202 into a universal standardized format. For instance, the 3D city model 206 as discussed above may be from various sources and include various models (e.g. OSM and CAD). The models in those formats may not be immediately usable by the driving simulator 102. The data ingestion layer 214 may be configured to process the 3D city models 206 and convert the models into a standardized format/form which is supported throughout the mobility CAE platform 200.
The mobility CAE platform 200 may further include a toolkit layer 216 configured to process the data/models having been converted via the toolkit layer 216 to provide the user with driving simulations. The toolkit layer 216 may include multiple groups of modules for simulation. For instance, the toolkit layer 216 may include a simulation control group 218 configured to operate vehicle driving simulation controls of the driving simulator 102. The simulation control group 218 may include a vehicle dynamics model 220 configured to define performance and capabilities of a subject vehicle using various parameters. The vehicle dynamics model 220 may define various types of vehicle for simulations to provide users with different needs. For instance, the vehicle dynamics model 220 may include vehicle models for passenger vehicles, sport vehicles, racing vehicles, sport utility vehicles (SUVs), pickup trucks, semi-trucks, emergency vehicles (e.g. ambulance, police vehicle, or fire engines), or the like configured to allow the user to simulate driving experience with those vehicles. Although driving simulations may be performed as the user driving alone without any traffic, a more realistic simulation would include ambient vehicles operated by computer. The simulation control group 218 may further include an ambient traffic artificial intelligent (AI) model 218 configured to define the driving behavior of ambient vehicles. The ambient traffic AI model 218 may include parameters to simulate various ambient traffic driving behavior with multiple levels of aggressiveness, traffic density or the like.
The simulation control group 218 may further include a communication/vehicle-to-everything (V2X) model 224 configured to simulate intra-simulation communication and V2X interactions. For instance, the communication/V2X model 224 may allow a user in a simulation for an emergency vehicle to communicate with a virtual control center and change the traffic signals to simulate emergency response situation. The simulation control group 218 may further include a view camera controller module 226 configured to enable controls for a subject camera. The view camera controller module 226 may be used to move the camera to different positions to simulate sitting in different types of vehicles (e.g. cars, trucks). The simulation control group 218 may further include a weather control module 228 configured to control weather and time of the day for simulations. The simulation control group 218 may further include an ambient pedestrian AI model 230 configured to define the behavior of pedestrians for simulations. Similar to the operations of the ambient traffic AI model 222, the ambient pedestrian AI model may be configured to control the number of pedestrians, speed of movement, different levels of aggressiveness (e.g. jaywalking) to provide a more realistic simulation environment. The simulation control group 218 may further include an aerial vehicle control module 232 configured to support modelling and control of aerial vehicles (e.g. drones). For instance, the driving simulator 102 may be configured to simulate specific use-cases related to aerial vehicle-based goods delivery or other unmanned aerial vehicle (UAV) use-cases through the aerial vehicle control module 232. The simulation control group 218 may further include a generic city traffic model 234 configured to define traffic pattern/flow in a generic city where road network data is not available.
The toolkit layer 216 may further include a scenario control group 236 having multiple entries configured to control simulation scenarios of the driving simulator 102. The scenario control group 236 may include a behavior control module 238 configured to provide a scenario-specific behavior control for a target. For instance, a target may include a pedestrian crossing the road in from of the simulating vehicle in which case the user is required to take actions to avoid an accident. The scenario control group 236 may further include a timer control module 240 configured to provide for one or more timers to keep a check of virtual time or to create scenarios, as some scenarios may have time requirements (e.g. a shuttle driving simulation). The scenario control group 236 may further include an infrastructure control module 242 configured to allow controls over various infrastructures such as traffic lights, railway signals, or the like. The scenario control group 236 may further include one or more vehicle add-on models 244 configured to put add-on items on a simulating vehicle such as a snow plow, a trailer or the like, by modifying parameters of the vehicle dynamics model 220.
The toolkit layer 216 may further include a visual control group 246, configured to provide visual images to the user via the display 114 by way of the video controller 116. The visual control group 246 may include a 3D rendering module 248 configured to render 3D graphics for the Display 114 via the video controller 116. The visual control group 246 may include various generic models. For instance, the visual control group 246 may include a generic 3D Pedestrian model 250 configured to provide a generic or default visual model for pedestrians in case that the user does not provide a specific visual model for pedestrian. The visual control group 246 may further include a generic 3D vehicle model 252 configured to provide a generic 3D model for vehicles in case that the user does not provide a specific visual model for vehicles. The visual control 246 may further include a generic 3D city model 254 configured to provide a generic 3D model of the city utilized for simulations in case that the user does not provide a 3D model for the city of preference.
The toolkit layer 216 may further include a communication control group 256 configured to control the communication between the driving simulator 102 and external devices or services. The communication control group 256 may include an external communication module 258 configured to enable bi-directional communications with the external device 132 via applications through the cable 146 and/or the wireless connection 136. In addition to the external device 132, the driving simulator 102 may be connected to a vehicle infotainment system 262 to provide a more realistic driving simulation environment. For instance, the infotainment system 262 may include the SYNC system manufactured by The Ford Motor Company of Dearborn, Michigan. Therefore, the communication control group 256 may further include an infotainment integration module 260 configured to enable communication between the driving simulator 102 and an infotainment system 262 through various types of wired or wireless connections.
The toolkit layer 216 may further include a data control module 264 which has a data storage module 266 configured to load and store simulation data including data analytics and simulation results from and to a database 268. The database 268 may be implemented locally using a server managed via software such as SQLite in communication with the driving simulator 102. Alternatively, the database 268 may be implemented in the storage 108 of the driving simulator 102. Alternatively, the database 268 may be implemented on the cloud 130 in communication with the driving simulator 102 via the network controller 128.
Referring to
The user interface 300 may further provide the user with one or more option buttons 324 to trigger various actions. As illustrated in
As illustrated in
Referring to
In reality, many drivers may use an external device (e.g. a smart phone, or a tablet) to perform various operations such as communication and navigation, while operating the vehicle. To accommodate that particular training need, the mobility CAE platform 200 may be configured to support a connection to the external device 132 via a Wi-Fi connection. Additionally, the manager controlling the simulation may access the external device via the web-based configuration application through a connection (e.g. a router 606) to provide communication and instructions. For instance, in case that the user is simulating a shuttle driver UX 310, the manager may send new pickup and drop off locations to the user via the external device using the web-based configuration application 604 to simulator dynamic real-life situations.
Referring to
At operation 708, the mobility CAE platform 200 starts the simulation based on the imported inputs 204 and user customization. Depending on the specific use-case functionality settings, different control modules within the toolkit layer 216 of the mobility CAE platform 200 may be enabled or disabled. Taking the shuttle driver UX 310 for instance, the following modules/models of the toolkit layer 216 may be enabled by default: the vehicle dynamics model 220, the ambient traffic AI model 222, the view camera controller module 226, the weather control module 228, the timer control module 240, the infrastructure control module 242, the 3D rendering module 248, the generic 3D vehicle model 252, the external communication module 258 and the data storage module 266. As discussed above, the mobility CAE platform 200 may be configured to allow a user or manager to modify the enabling/disabling of the toolkit layer controls to adjust the simulation. At operation 710, the mobility CAE platform 200 receives an input to change simulation parameters via the web-based configuration application 604. For instance, responsive to detecting the weather functionality 412 is unchecked, the mobility CAE platform 200 may disable the weather control module 228 to reduce the difficulty of the simulation as needed.
At operation 712, the mobility CAE platform 200 receives an input via the web-based configuration application 604 for the external device 132 connected via the external communication module 258. For instance, while simulating a shuttle driver UX 310 use-case, the mobility CAE platform 200 may dynamically receive updates for new pickup and drop off locations for new passengers via the web-based configuration application 604. Such new updates are sent to the external device 132 to inform the user performing the simulation. The user may drive the simulating vehicle based on the instructions from the external device 132. After each successful pickup and/or drop off, the user may provide a feedback/response via the external device. At operation 714, the mobility CAE platform 200 receives the user response from the external device and record the response as a simulation data. Continuing to use the shuttle driver UX 310 use-case for example, the mobility CAE platform 200 may record the timing of each user response indicative of a successful pickup or drop off to monitor the performance of the user.
While exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the invention. Additionally, the features of various implementing embodiments may be combined to form further embodiments of the invention.
