The present disclosure is drawn to a system providing virtual- or tele-presence of a person inside a motor vehicle.
Although traditionally motor vehicles are operated by a driver situated inside the subject vehicle, recent developments have established ability of motor vehicles to operate semi and fully autonomously. Such “autonomous vehicles” typically employ sensors operable to sense the environment surrounding the vehicle together with a computing device operable to control various aspects of vehicle operation.
Autonomy in vehicles is often categorized in discrete levels, such as Level 1—Driver assistance—where the vehicle may control either steering or speed autonomously in specific circumstances to assist the driver; Level 2—Partial automation—where the vehicle may control both steering and speed autonomously in specific circumstances to assist the driver; Level 3—Conditional automation—where the vehicle may control both steering and speed autonomously under normal environmental conditions, but requires driver oversight; Level 4—High automation—where the vehicle may complete a prescribed trip autonomously under normal environmental conditions, not requiring driver oversight; and Level 5—Full autonomy—where the vehicle may complete a prescribed trip autonomously under any environmental conditions.
In addition to transporting passengers, autonomous vehicles may be used for such duties as pick-up and delivery of freight. Although semi-autonomous vehicles (Levels 2-3) generally require some operator input, oversight, and/or control, fully and highly autonomous vehicles (Levels 4-5) are capable of operating in their environment without human intervention or presence inside the vehicle. Accordingly, where the computing device with the aid of sensors controls all aspects of vehicle operation, the vehicle may completely eschew the presence of an individual capable of interacting with the external environment.
A two-way audio-video transmission telepresence system for a motor vehicle having a vehicle exterior, a vehicle interior, and a panel separating the vehicle interior from the vehicle exterior. The system includes a first telecommunication device arranged on the motor vehicle and configured to receive audio and video data from a second telecommunication device situated remotely from the motor vehicle. The system also includes an audio speaker arranged on the vehicle exterior and configured to reproduce the audio data received from the second telecommunication device. The audio speaker may be located proximate the panel. The system additionally includes a visual display device in communication with the first telecommunication device, arranged on the panel and facing the vehicle exterior, and configured to display the video data received from the second telecommunication device toward an environment surrounding the motor vehicle.
The system also includes a microphone arranged on the vehicle exterior proximate the visual display, configured to capture sound proximate the motor vehicle, and communicate the captured sound to the first telecommunication device. The system further includes at least one camera mounted to the motor vehicle and configured to generate an image of at least a portion of the environment surrounding the motor vehicle and communicate the generated image to the first telecommunication device. The first telecommunication device is additionally configured to communicate the captured sound and the generated image to the second telecommunication device to thereby establish real-time communication between the environment surrounding the motor vehicle and the second telecommunication device.
The at least one camera may include a plurality of cameras. Each such camera may be configured to capture a distinct partial view of the environment surrounding the motor vehicle and a corresponding section of the vehicle exterior. The subject plurality of cameras may further stitch together a composite view of the vehicle interior and the environment surrounding the motor vehicle. In such an embodiment, the generated image may include the stitched together composite view.
The second telecommunication device may be configured to display the generated image, including the stitched together composite view.
Alternatively, the at least one camera may include an accessory camera mounted to the motor vehicle and facing the environment surrounding the motor vehicle. The accessory camera may be configured to capture a view of a portion of the environment surrounding the motor vehicle proximate one of a left-hand side and a right-hand side of the vehicle exterior, such as near a typical seating position of a vehicle operator. The generated image displayed by the second telecommunication device may then include the view of the portion of the environment captured by the accessory camera.
The panel may be a vehicle window providing a view of the vehicle interior from the vehicle exterior and vice versa. The captured view of the portion of the environment surrounding the motor vehicle may be proximate the vehicle window.
The above-described accessory camera may be arranged inside the vehicle and capture images through the window.
The second telecommunication device may be configured to communicate the audio and video data from a remote user, such as a custodian or an operator, of the motor vehicle.
The first telecommunication device and the second telecommunication device may be configured to communicate wirelessly via a 5G wireless network, a satellite uplink, or a similar wireless medium.
The motor vehicle may be configured as a self-guided autonomous vehicle.
The remote user may be enabled by the telepresence system to operate various systems of the motor vehicle, such as controlling access into the vehicle by locking/unlocking of vehicle doors, and modifying the vehicle's routing/destination.
A motor vehicle employing the telepresence system for establishing real-time communication between the environment surrounding the motor vehicle and a source of audio and video data, e.g., a user of the vehicle, situated remotely from the motor vehicle is also disclosed.
A method of establishing real-time communication between a remote user of a motor vehicle and an environment surrounding the motor vehicle, such as via the two-way audio-video transmission telepresence system, is further disclosed.
The above features and advantages, and other features and advantages of the present disclosure, will be readily apparent from the following detailed description of the embodiment(s) and best mode(s) for carrying out the described disclosure when taken in connection with the accompanying drawings and appended claims.
Referring to the drawings, wherein like reference numbers refer to like components throughout the several views,
As shown in
The motor vehicle 10 also includes an electronic controller 28 (shown in
Volatile media of each of the controller's memory may include, for example, dynamic random-access memory (DRAM), which may constitute a main memory. Such instructions may be transmitted by one or more transmission medium, including coaxial cables, copper wire and fiber optics, including the wires that comprise a system bus coupled to the sensors 27. Memory of the electronic controller 28 may also include a flexible disk, hard disk, magnetic tape, other magnetic medium, a CD-ROM, DVD, other optical medium, etc. The electronic controller 28 may be equipped with a high-speed primary clock, requisite Analog-to-Digital (A/D) and/or Digital-to-Analog (D/A) circuitry, input/output circuitry and devices (I/O), as well as appropriate signal conditioning and/or buffer circuitry. Algorithms required by the electronic controller 28 or accessible thereby may be stored in the memory and automatically executed to provide the required functionality.
The motor vehicle 10 may be part of a two-way, audio-video transmission telepresence system 30 (shown in
The data communication 34 is intended to be implemented by the source 36 via a second telecommunication device 38 (shown in
With continued reference to
The telepresence system 30 additionally includes a microphone 46 (shown in
In the embodiment of the telepresence system 30 having a plurality of cameras 50 (shown in
Specifically, the stitched-together exterior image 52A and the static image of the interior 16 may be combined on the vehicle side, as a function of the first telecommunication device 32 and then communicated to the second telecommunication device 38 for displaying thereon. Alternatively, the stitched-together exterior image 52A may be communicated (sans overlaid image of the interior 16) to the second telecommunication device 38, with the second telecommunication device overlaying the static interior 16 image programmed therein to generate the image 52. Accordingly, in such an embodiment, the second telecommunication device 38 may be configured to superimpose the video data received from the vehicle mounted camera(s) 50 over an existing or saved image of the vehicle interior 16. Each of the above options would, therefore, result in the same image 52 being viewed on the remote device 38, as shown in
In a separate embodiment, a single accessory camera 50, as shown in
The first telecommunication device 32 is additionally configured to communicate the captured sound and the generated image 52 to the second telecommunication device 38. The second telecommunication device 38 may then display the generated image 52, such as including the stitched together composite view 52A or include the view of the portion of the environment 44 captured by the accessory camera 50. The remote user of the motor vehicle 10 may therefore receive to view the stitched composite view 52A on the second telecommunication device 38, as shown in
The telepresence system 30 may employ the above-described real-time communication in combination with algorithms programmed into the electronic controller 28 and assistance from the vehicle sensors 27 to enable further interaction of the motor vehicle 10 with its surroundings. For example, the telepresence system 30 may enable the motor vehicle 10 user to remotely operate various vehicle systems, for example, by locking/unlocking of the door 20 to control access into the vehicle, and control vehicle operation, such as by pausing vehicle forward progress or revising the vehicle's routing/destination, etc.
Following frame 102, the method proceeds to frame 104, where the method includes receiving, via the first telecommunication device 32 audio and video data from a remotely located user of the motor vehicle 10 via the second telecommunication device 38. After frame 104, the method advances to frame 106. In frame 106 the method includes reproducing via the audio speaker 40 the audio data received from the second telecommunication device 38. Following frame 106, the method proceeds to frame 108. In frame 108 the method includes displaying via the visual display device 42 the video data received from the second telecommunication device 38 toward environment 44. Following frame 108, the method may proceed to frame 110. In frame 110 the method includes capturing, via the microphone 46, sound proximate the motor vehicle 10 and communicating the captured sound to the first telecommunication device 32.
After frame 110, the method advances to frame 112 for generating via one or more cameras 50 the image 52 of relevant portion(s) of the environment 44 and communicating the generated image to the first telecommunication device 32. As described with respect to
Alternatively, in frame 112 the camera 50 may be an accessory camera mounted to the motor vehicle, such as to the exterior 14, and facing the environment 44 to capture images through the window 18. In such an embodiment, in frame 112 the method may include capturing, via the subject accessory camera 50, a view of a portion of the environment 44 next to the driver's position, e.g., proximate either the left-hand side 14-2 and the right-hand side 14-4 of the vehicle exterior 14. Following frame 112, the method proceeds to frame 114. In frame 114 the method includes communicating via the first telecommunication device 32 the captured sound and the generated image 52 to the second telecommunication device 38 to thereby establish real-time communication between the environment 44 and the second telecommunication device. After frame 114, the method may advance to frame 116. In frame 116 the method may include enabling the vehicle user to remotely operate vehicle systems, for example, to grant or restrict access into the vehicle by locking/unlocking the door 20, or to revise the vehicle's routing or destination.
After frame 116, the method may loop back to frame 104 for continued communication between the remote vehicle user and various individuals situated in the vehicle environment 44. The method 100 may conclude in frame 118 once the needed interaction of the motor vehicle's remote user with the environment 44 has been completed, allowing the motor vehicle to proceed along its route toward its destination. Accordingly, the method 100 permits a self-guided autonomous motor vehicle 10 to accomplish various tasks which may require real-time decision making as a consequence of communication between individuals.
The detailed description and the drawings or figures are supportive and descriptive of the disclosure, but the scope of the disclosure is defined solely by the claims. While some of the best modes and other embodiments for carrying out the claimed disclosure have been described in detail, various alternative designs and embodiments exist for practicing the disclosure defined in the appended claims. Furthermore, the embodiments shown in the drawings or the characteristics of various embodiments mentioned in the present description are not necessarily to be understood as embodiments independent of each other. Rather, it is possible that each of the characteristics described in one of the examples of an embodiment may be combined with one or a plurality of other desired characteristics from other embodiments, resulting in other embodiments not described in words or by reference to the drawings. Accordingly, such other embodiments fall within the framework of the scope of the appended claims.