High phone BLE or CPU burden detection and notification

Information

  • Patent Grant
  • 11195344
  • Patent Number
    11,195,344
  • Date Filed
    Friday, March 15, 2019
    5 years ago
  • Date Issued
    Tuesday, December 7, 2021
    2 years ago
Abstract
Method and apparatus are disclosed for monitoring of communication for vehicle remote motive control. An example vehicle includes an autonomy unit for remote motive control, a communication module, and a controller. The controller is to send a diagnostic signal to and receive a return signal from a mobile device via the communication module, determine a time period between sending of the diagnostic signal and receipt of the return signal, and prevent the autonomy unit from causing vehicle movement responsive to determining the time period is greater than a threshold.
Description
TECHNICAL FIELD

The present disclosure generally relates to remote motive control and, more specifically, to monitoring of communication for remote motive control of a vehicle.


BACKGROUND

Many vehicles include motive functions that are at least partially autonomously controlled by the vehicle. For example, some vehicles include cruise control in which the vehicle controls acceleration and/or deceleration of the vehicle so that a speed of the vehicle is maintained. Further, some vehicles include adaptive cruise control in which the vehicle controls acceleration and/or deceleration of the vehicle so that a speed of the vehicle and a following distance behind a lead vehicle are maintained. Additionally, some vehicles include park-assist features in which the vehicle autonomously controls motive functions of the vehicle to park the vehicle into a parking spot.


SUMMARY

The appended claims define this application. The present disclosure summarizes aspects of the embodiments and should not be used to limit the claims. Other implementations are contemplated in accordance with the techniques described herein, as will be apparent to one having ordinary skill in the art upon examination of the following drawings and detailed description, and these implementations are intended to be within the scope of this application.


Example embodiments are shown for monitoring of communication and processing for vehicle remote motive control. An example disclosed vehicle includes a communication module and an autonomy unit comprising an autonomy unit controller configured to send a diagnostic signal including a unique identifier, via the communication module, to a mobile device. The mobile device includes a device controller configured to generate a return signal including the unique identifier upon detecting an input responsive to receipt of the diagnostic signal. Furthermore, the autonomy unit controller is configured to receive the return signal including the unique identifier from the mobile device, via the communication module. The autonomy unit controller determines a time period between sending the diagnostic signal and receiving the return signal, and wherein responsive to a determination that the time period is greater than a threshold, the autonomy unit controller disables the autonomy unit to prevent movement of the vehicle during a remote motive control session of the vehicle.


In some examples, the autonomy unit controller generates a first diagnostic signal time stamp when the diagnostic signal is sent to the mobile device and the autonomy unit controller generates a second diagnostic signal time stamp when the return signal is received form the mobile device. The autonomy unit controller determines the time period between the first diagnostic signal time stamp and the second diagnostic signal time stamp.


In some examples, the communication module is configured to wirelessly communicate via a Bluetooth® low energy protocol, a Classic Bluetooth® protocol, a Wi-Fi® protocol, a Wi-Fi® low power protocol, and an Ultra-Wide Band (UWB) protocol. In some examples, the communication module sends the counter signal to trigger sending of the return signal to the communication module.


In some examples, responsive to determining the time period is less than or equal to a threshold, the autonomy unit controller enables the autonomy unit to cause movement of the vehicle during remote motive control of the vehicle.


In some examples, the controller sends a notification to the mobile device including a plurality of potential sources responsible for the time period being greater than the threshold. In some examples, the notification further includes an acknowledgment input confirming investigation of one or more of the plurality of potential sources responsible for the time period being greater than the threshold, and a confirmation response is received by the autonomy unit controller to trigger sending a second diagnostic signal to the mobile device.


In some examples, after the mobile device detects the input, the device controller is to process a task associated with the input and generate a second return signal. The device controller sends the second return signal to the autonomy unit controller and the autonomy unit controller determines a second time period between receiving the return signal and receiving the second return signal.


In some examples, responsive to determining the second time period is less than or equal to a threshold, the autonomy unity is enabled to cause movement during remote motive control of the vehicle. In some examples, responsive to determining the second time period is greater than a threshold, the autonomy unit is to disabled to prevent movement during remote motive control of the vehicle.


An example disclosed method performed by a vehicle includes sending, via a communication module of a vehicle, a diagnostic signal including a first diagnostic signal time stamp from an autonomy unit controller to a mobile device and detecting, via the mobile device, an input responsive to receipt of the diagnostic signal. The example disclosed method also includes generating, by the mobile device, a return signal including the first diagnostic signal time stamp upon detecting the input. Furthermore, the method includes receiving, via the communication module, the return signal including the first diagnostic signal time stamp by the autonomy unit controller of the vehicle. The example disclosed method also includes generating, by the autonomy unit controller, a second diagnostic signal time stamp upon receiving the return signal. The method further includes determining, by a processor of the autonomy unit controller, a time period between the first diagnostic signal time stamp and the second diagnostic signal time stamp. The example disclosed method further includes disabling, via the processor of the autonomy unit controller, an autonomy unit from causing movement of the vehicle responsive to determining the time period is greater than a threshold.


In some examples, a time of the autonomy unit controller is synchronized with a time of the mobile device. The example disclosed method further discloses, generating, by the mobile device, a first device time stamp upon detecting the input, including the first device time stamp in the return signal, and determining, via the processor of the autonomy unit controller, the time period between the first diagnostic signal time stamp and the first device time stamp.


In some examples, detecting the input by the mobile device triggers the mobile device to process a task associated with the input and generate a second return signal. Furthermore, the second return signal is sent to the autonomy unit controller and the autonomy unit controller generates a third diagnostic signal time stamp upon receiving the second return signal. The autonomy unit controller determines a second time period between the second diagnostic signal time stamp and the third diagnostic signal time stamp.





BRIEF DESCRIPTION OF THE DRAWING

For a better understanding of the invention, reference may be made to embodiments shown in the following drawings. The components in the drawings are not necessarily to scale and related elements may be omitted, or in some instances proportions may have been exaggerated, so as to emphasize and clearly illustrate the novel features described herein. In addition, system components can be variously arranged, as known in the art. Further, in the drawings, like reference numerals designate corresponding parts throughout the several views.



FIG. 1 illustrates an example vehicle in accordance with the teachings herein.



FIG. 2 illustrates an example mobile device initiating remote parking of the vehicle of FIG. 1.



FIG. 3A depicts signals sent to monitor communication and processing between the vehicle of FIG. 1 and the mobile device of FIG. 2.



FIG. 3B depicts a comparison of a time period and a threshold to monitor communication and processing between the vehicle of FIG. 1 and the mobile device of FIG. 2.



FIG. 3C depicts a comparison of a time period and a threshold to monitor communication and processing between the vehicle of FIG. 1 and the mobile device of FIG. 2.



FIG. 4A depicts signals sent to monitor communication and processing of the vehicle of FIG. 1.



FIG. 4B depicts a comparison of a time period and a threshold to monitor communication and processing of the vehicle of FIG. 1.



FIG. 5 is a block diagram of a display of the mobile device of FIG. 2



FIG. 6 is a block diagram of electronic components of the vehicle of FIG. 1.



FIG. 7 is a flowchart for monitoring of communication and processing for remote motive control in accordance with the teachings herein.



FIG. 8 is a flowchart for monitoring of communication and processing for remote motive control in accordance with the teachings herein.





DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

While the invention may be embodied in various forms, there are shown in the drawings, and will hereinafter be described, some exemplary and non-limiting embodiments, with the understanding that the present disclosure is to be considered an exemplification of the invention and is not intended to limit the invention to the specific embodiments illustrated.


Many vehicles include motive functions that are at least partially autonomously controlled by the vehicle. For example, some vehicles include cruise control in which the vehicle controls acceleration and/or deceleration of the vehicle so that a speed of the vehicle is maintained. Further, some vehicles include adaptive cruise control in which the vehicle controls acceleration and/or deceleration of the vehicle so that a speed of the vehicle and a following distance behind a lead vehicle are maintained.


Additionally, some vehicles include park-assist systems, such as remote park-assist systems, in which the vehicle autonomously controls motive functions of the vehicle to park the vehicle into a parking spot. Remote park-assist systems are configured to autonomously park a vehicle when a driver has already exited the vehicle. Some remote park-assist systems require the driver to provide a continuous input via a mobile device in wireless communication with the vehicle to instruct the vehicle to autonomously park in the parking spot. In some instances, there may be latency in communication between the mobile device and the vehicle that causes the vehicle to continue to receive an input from the mobile device when the operator is no longer providing an input to the mobile device. In such instances, the vehicle may potentially continue to perform remote park-assist functions when the operator no longer desires to perform such functions.


Example apparatus and methods disclosed herein diagnose and mitigate latency and/or other communication errors between a vehicle and a mobile device that is configured to initiate remote motive control features of the vehicle. Examples disclosed herein include a vehicle that wirelessly transmits diagnostic signals and other system control signals to a mobile device at predetermined time intervals. In some examples, the vehicle system utilizes timers and diagnostics to monitor for delay in communication between the vehicle and the mobile device. In some examples, the vehicle system timers and diagnostics determine whether the communication delay is caused by the mobile device streaming other data, the mobile device being overburdened with additional background processing and/or a communication module of the vehicle being overburdened. In some examples, the vehicle system sends a notification to communicate one or more potential causes of the delay to the user of vehicle system. The system will allow the user to correct the communication delay if the delay is detected while the vehicle system is in a pre-maneuver state. Alternatively, the system will stop performance of the maneuver if the delay is detected while a remote motive control maneuver is being performed.


As used herein, “remote motive control,” and “vehicle remote motive control” refer to controlling motive functions of the vehicle without direct steering or velocity input (i.e., direct input on steering wheel and/or accelerator) from an operator to autonomously control movement of the vehicle. In one non-limiting example, a remote park assist-system of an autonomy unit controls the motive functions of the vehicle upon initiation from a driver to remotely park the vehicle into a parking spot. In some embodiments, the user provides an input on a mobile device (e.g., a smart phone, a tablet computer, a smart watch, a fob, any other such mobile device and/or combinations therein) that is received by the autonomy unit. The autonomy unit controls the motive functions of the vehicle (e.g., steering, velocity increase, velocity decrease) based on the user provided input to the mobile device.


As used herein, to “tether” refers to enabling a mobile device to cause a vehicle to perform remote parking or other remote motive control maneuver. In one non-limiting example, a vehicle is configured to perform remote parking upon receiving instruction(s) to do so from a mobile device when the mobile device is tethered to the vehicle and is configured to not perform remote parking when the mobile device is untethered from the vehicle. As used herein, a “tethered” device refers to a mobile device that is enabled to send instructions to a vehicle to perform remote parking. For example, a mobile device is tethered to a vehicle responsive to the mobile device being wirelessly communicatively coupled to the vehicle and located within a predetermined tethering range (e.g., 6 meters) of the vehicle. In such examples, a mobile device that sends instructions to a vehicle to perform remote parking is untethered from the vehicle if the mobile device is beyond the tethering range of the vehicle. In some examples, a mobile device of an operator is tethered via communication between the vehicle and another device (e.g., a key fob) carried by the operator. In such examples, the vehicle may detect that the key fob is within a predetermined tethering range of the vehicle, presume that the operator is carrying both the key fob and the mobile device associated with the operator, and subsequently tether the mobile device to the vehicle. Further, in cases where the key fob shares wireless technology with the mobile device (e.g., BLE, UWB or other such communication protocol) the mobile device may confirm the key fob is within a specified distance of the phone (e.g., 1 meter) via received signal strength indicator(s) (RSSI), Time-of-Flight or other such methods of validating the assumption that the operator controlling the mobile device is also in possession of the key fob.


Turning to the figures, FIG. 1 illustrates an example vehicle 100 in accordance with the teachings herein. The vehicle 100 may be a standard gasoline powered vehicle, a hybrid vehicle, an electric vehicle, a fuel cell vehicle, and/or any other mobility implement type of vehicle. The vehicle 100 includes parts related to mobility, such as a powertrain with an engine, a transmission, a suspension, a driveshaft, and/or wheels, etc. The vehicle 100 may be non-autonomous, semi-autonomous (e.g., some routine motive functions controlled by the vehicle 100), or autonomous (e.g., motive functions are controlled by the vehicle 100 without direct driver input). In the illustrated example, the vehicle 100 includes an engine 102, a transmission 104, and a transmission gear selector 106.


For example, the engine 102 includes an internal combustion engine, an electric motor, a fuel-cell engine, and/or any other type of engine or motor that is configured to propel the vehicle 100. For example, an internal combustion engine generates mechanical power to propel the vehicle 100 by combusting fuel (e.g., gasoline, petrol, etc.), and an electric motor generates mechanical power to propel the vehicle 100 by converting electrical energy stored in a battery (e.g., of a battery cell and/or a battery pack) into mechanical energy.


The transmission 104 controls an amount of power generated by the engine 102 that is transferred to other components of the vehicle 100, such as a powertrain, wheels, etc. For example, the transmission 104 includes a gearbox that controls the amount of power transferred to the components of the vehicle 100. The transmission gear selector 106 enables an operator (e.g., a driver) of the vehicle 100 to control a setting of the transmission 104 that affects how the engine 102 propels the vehicle 100. For example, the transmission gear selector 106 enables the operator to position the transmission 104 in park, neutral, reverse, drive, first gear, etc.


Further, the vehicle 100 of the illustrated example includes an engine sensor 108 and a transmission position sensor 110. The engine sensor 108 detects whether the engine 102 is activated or deactivated. The transmission position sensor 110 detects a current position (e.g., park, neutral, reverse, drive, first gear, etc.) of the transmission 104 and/or the transmission gear selector 106. In some embodiments, the vehicle 100 does not include a gear box. Accordingly, in these embodiments, the transmission position sensor 110 is not required to detect the current position of the transmission 104 and/or the transmission gear selector 106.


As illustrated in FIG. 1, the vehicle 100 also includes a vehicle speed sensor 112, external lamps 114, and a horn 116. For example, the vehicle speed sensor 112 detects a speed at which the vehicle 100 is travelling. In some examples, the vehicle speed sensor 112 detects an acceleration or deceleration of the vehicle 100 by monitoring the speed at which the vehicle 100 is traveling over a period of time. Further, the external lamps 114 emit light to facilitate operation of the vehicle 100 in low-light environments and/or to alert nearby vehicle(s), driver(s), pedestrian(s), and/or other people of functions being performed by the vehicle 100. For example, the external lamps 114 include headlamps, such as high-beam headlamps and low-beam headlamps, to illuminate an area in front of the vehicle 100 in low-light environments. Additionally or alternatively, the external lamps 114 include brake lamps to alert others that the vehicle 100 is about to turn and/or turn-signal lamps to alert others that the vehicle 100 is braking. Additionally, the horn 116 emits a loud audio signal. For example, an operator (e.g., a driver) of the vehicle 100 activates the horn 116 (e.g., by pressing a button located within a driving wheel of the vehicle 100) to alert pedestrians, vehicle operators, and/or other people of a presence of the vehicle 100 and/or of a nearby hazard.


The vehicle 100 of the illustrated example also includes a communication module 118 that is configured to communicatively connect to a mobile device of a user of the vehicle 100 (e.g., a mobile device 204 of a user 206 of FIG. 2). The communication module 118 includes hardware and firmware to establish a wireless connection with the mobile device. For example, the communication module 118 is a wireless personal area network (WPAN) module that wirelessly communicates with mobile device(s) of user(s) via short-range wireless communication protocol(s). In some examples, the communication module 118 implements the Classic Bluetooth®, Bluetooth® and/or Bluetooth® Low Energy (BLE) protocols. The Classic Bluetooth®, Bluetooth® and BLE protocols are set forth in Volume 6 of the Bluetooth® Specification 4.0 (and subsequent revisions) maintained by the Bluetooth® Special Interest Group. Additionally, or alternatively, the communication module 118 is configured to wirelessly communicate via Wi-Fi®, Wi-Fi® low power, Near Field Communication (NFC), Ultra-Wide Band (UWB), and/or any other short-range and/or local wireless communication protocol (e.g., IEEE 802.11 a/b/g/n/ac) that enables the communication module 118 to communicatively couple to a mobile device. In some examples, the communication module 118 wirelessly communicates via Wi-Fi® and/or Wi-Fi® low power protocols (e.g., Wi-Fi HaLow™ operating at 900 MHz) to facilitate communication with a mobile device when a line-of-sight between the communication module 118 and the mobile device is interrupted. Further, in some examples, the vehicle utilizes a Li-Fi protocol (e.g., a light-based LAN communication protocol) and/or a mid-frequency UHF protocol (e.g., at 433 MHz or 902 MHz) to communicate with the mobile device.


As illustrated in FIG. 1, the vehicle 100 includes an autonomy unit 120. The autonomy unit 120 is an electronic control unit (ECU) (e.g., one of a plurality of electronic control units 606 of FIG. 6) of the vehicle 100 that performs autonomous and/or semi-autonomous functions of the vehicle 100. For example, the autonomy unit 120 autonomously controls motive functions of the vehicle 100 to perform remote parking of the vehicle 100 into an available parking spot (e.g., a parking spot 202 of the FIG. 2) and/or to otherwise autonomously and/or semi-autonomously drive the vehicle 100. For example, the autonomy unit 120 controls motive functions of the vehicle 100 based on data collected from sensor(s) (e.g., radar sensor(s), LIDAR sensor(s), ultrasonic sensor(s), etc.) of the vehicle 100 and/or camera(s) of the vehicle 100.


The vehicle 100 of the illustrated example also includes an autonomy unit controller 122 that monitors for latency in communication between the vehicle 100 and a mobile device (e.g., the mobile device 204) that is utilized by a user (e.g., the user 206) to initiate remote parking. For example, the autonomy unit controller 122 of the illustrated examples initiates one or more timers and sends a series of pings to the mobile device via the communication module 118 to monitor for latency in communication with mobile device over a period of time while the autonomy unit 120 performs remote parking. Each of the pings sent by the communication module 118 includes a counter signal that facilitates identification of latency in communication between the mobile device and the vehicle 100.


In another such example, the autonomy unit controller 122 sends a diagnostic signal to the mobile device via the communication module 118. Further, the autonomy unit controller 122 is configured to receive a corresponding return signal that includes a time stamp from the mobile device via the communication module 118. The mobile device generates the time stamp based on receiving an initial input, from a user of the mobile device, in response to receipt of the diagnostic signal. To determine whether there is an acceptable or an unacceptable amount of latency in communication between the mobile device and the vehicle 100, the autonomy unit controller 122 determines a time period between the sending of the diagnostic signal and the generation of the time stamp generated by the mobile device in response to receipt of the diagnostic signal. For example, an acceptable amount of latency is less than a predetermined threshold amount of latency, and an unacceptable amount of latency is greater than or equal to the predetermined threshold amount of latency. Further, the autonomy unit controller 122 compares the time period to a predetermined threshold (e.g., a latency threshold). The autonomy unit controller 122 detects that there is unacceptable latency in communication between the mobile device and the vehicle 100 if the time period is greater than the predetermined threshold, and the autonomy unit controller 122 detects that there is acceptable latency if the time period is less than or equal to the predetermined threshold. In response to detecting unacceptable latency, the autonomy unit controller 122 sends a notification via the communication module 118 to the mobile device identifying the communication latency between the vehicle 100 and the mobile device. In some examples, the notification includes a plurality of potential causes for the detected latency between the vehicle 100 and the mobile device to help the user correct the issue.


In another such example, the autonomy unit controller 122 includes a time information or other unique identifier (e.g., time stamp) with the diagnostic signal sent to the mobile device via the communication module 118. Alternatively, the autonomy unit controller 122 sends the time information to the mobile device, via the communication module 118, independent from the diagnostic signal. The mobile device incorporates the time information into the corresponding return signal sent from the mobile device, via the communication module 118, to the autonomy unit controller 122. In another such example, the mobile device incorporates the time information associated with the diagnostic signal sent from autonomy unit controller 122 with a time information (e.g., time stamp) generated by the mobile device. As such, the mobile device incorporates the time information associated with the diagnostic signal and the time information generated by the mobile device into the corresponding return signal sent from the mobile device, via the communication module 118, to the autonomy unit controller 122.


For example, if the detected latency is during a pre-maneuver portion of the remote motive control, the autonomy unit controller 122 pauses the vehicle and prevents the autonomy unit 120 from allowing or otherwise enabling the vehicle 100 to move during the remote motive control maneuver. In some examples, the user is provided a predetermined amount of time to investigate and/or correct the potential causes for the latency. If the user is successful, the autonomy unit controller 122 enables remote motive control of the vehicle 100 and allows the user and the autonomy unit controller 122 to proceed with executing the maneuver during the remote motive control of the vehicle 100. If the user is not successful, the autonomy unit controller 122 and/or the autonomy unit 120 aborts the remote motive control. Additionally, or alternatively, if the detected latency is during a maneuver, the autonomy unit controller 122 instructs the autonomy unit 120 to bring the vehicle 100 to a halt and sends an audio alert emitted via the horn 116 (e.g., traffic horn, electronic chirper, external speaker, and other such acoustic sound devices) and/or a visual alert emitted via one or more of the external lamps 114. The autonomy unit controller 122 further sends a notification to the mobile device to also emit an alert (e.g., an audio alert such as a chirp, a visual alert presented via a display, a haptic alert such as a vibration). The notification also includes the plurality of potential causes for the detected latency between the vehicle 100 and the mobile device to help the user correct the issue. In some examples, the user is provided a predetermined amount of time to investigate and/or correct the potential causes for the latency. If the user is successful, the autonomy unit controller 122 instructs the autonomy unit 120 to resume the remote motive control, otherwise the autonomy unit controller 122 and/or the autonomy unit 120 aborts the remote motive control. Further, in response to detecting that there is acceptable latency, the autonomy unit controller 122 enables the autonomy unit 120 to cause the vehicle 100 to move during remote parking.


In some examples, the autonomy unit controller 122 sends one or more signals via the communication module 118 in response to determining that remote parking is active. The communication module 118 does not send signals to the mobile device when remote parking is inactive, for example, to reduce an amount of communication signals being sent between the vehicle 100 and the mobile device. The autonomy unit controller 122 of the illustrated example detects remote motive control of the vehicle is active in response to the communication module 118 determining that the mobile device is within a tethering range for remote motive control (e.g., a tethering range 208 of FIG. 2), the communication module 118 receiving a remote motive control instruction from the mobile device, the engine sensor 108 detecting that the engine 102 is active, and/or the transmission position sensor 110 detecting that the transmission 104 is not in park. While FIG. 1 shows the autonomy unit 120 and the autonomy unit controller 122 as separate components, it will be understood that the autonomy unit controller 122 may be incorporated into the autonomy unit 120.



FIG. 2 illustrates one non-limiting example of remote motive control where the vehicle 100 is remotely parked into a parking spot 202 via a mobile device 204 (e.g., a smart phone, a tablet computer, a smart watch, a fob, any other such mobile device and/or combinations therein) of a user 206 (e.g., an operator of the vehicle 100). In the illustrated example, the parking spot 202 is a parallel parking spot. In other examples, the parking spot 202 for which the user 206 utilizes remote park-assist is a perpendicular parking spot or a diagonal parking spot.


As illustrated in FIG. 2, an exemplary remote motive control (e.g., remote park-assist) of the autonomy unit 120 of the vehicle 100 performs remote parking when the user 206 is located outside of a cabin of the vehicle 100. The user 206 utilizes the mobile device 204 to send an instruction to the vehicle 100 to perform remote parking. For example, the user 206 provides an input to cause a communication module of the mobile device 204 to wirelessly send an instruction to the communication module 118 of the vehicle 100. The autonomy unit controller 122 receives the instruction to perform remote parking from the mobile device 204 via the communication module 118.


Further, the autonomy unit controller 122 determines whether the autonomy unit 120 is to perform remote parking upon receiving the instruction to do so based upon, at least in part, whether the mobile device 204 is tethered to the vehicle 100. For example, the mobile device 204 is tethered to the vehicle 100 responsive to the mobile device 204 being (i) wirelessly communicatively coupled to the communication module 118 of the vehicle 100 and (ii) located within a tethering range 208 of the vehicle 100. That is, the autonomy unit 120 is configured to perform remote parking of the vehicle 100 responsive to the autonomy unit controller 122 determining that the mobile device 204 is within the tethering range 208 while sending the remote parking instruction to the vehicle 100. Further, the autonomy unit 120 is configured to not perform remote parking of the vehicle 100 responsive to the autonomy unit controller 122 determining that the mobile device 204 is beyond the tethering range 208 while sending the remote parking instruction to the vehicle 100. That is, if the mobile device 204 moves from within to beyond the tethering range 208, the autonomy unit 120 (temporarily) disables performance of remote parking. To reactivate remote parking, the user 206 is to move the mobile device 204 back into the tethering range and resend a remote parking instruction to the vehicle 100 while within the tethering range 208.


In the illustrated example, the tethering range 208 is defined based upon a predetermined distance (e.g., 6 meters) from an exterior surface of the vehicle 100. That is, the mobile device 204 is within the tethering range 208 of the vehicle 100 if a distance between the mobile device 204 and the exterior surface of the vehicle 100 is less than or equal to the predetermined distance. In some examples, the autonomy unit controller 122 determines the distance between the mobile device 204 and the exterior surface of the vehicle 100 based upon received signal strength indicator(s) (RSSI) of signal(s) that are communicated between the mobile device 204 and the communication module 118. For example, the autonomy unit controller 122 determines the distance between the mobile device 204 and the vehicle 100 based upon the RSSI(s) of the instruction(s) sent to the vehicle 100 by the mobile device 204 to perform remote parking. In some examples, the autonomy unit controller 122 determines the distance between the mobile device 204 and the vehicle 100 via GPS locations of the mobile device 204 and the vehicle 100. Additionally, or alternatively, the autonomy unit controller 122 may utilize any other manner (e.g., time-of-flight, angle-of-arrival, etc.) for determining the distance between the mobile device 204 and the vehicle 100 during remote parking.



FIG. 3A, depicts signals sent to monitor the latency in communication and processing between the vehicle 100 and the mobile device 204. As illustrated in FIG. 3A, the communication module 118 of the vehicle sends a diagnostic signal 302 to the mobile device 204. In one example, a user (e.g., user 206 of FIG. 2) initiates a remote motive control session by activating a remote motive control application on the mobile device 204. Upon receiving the remote motive control request, the autonomy unit controller 122 prepares the autonomy unit 120 to perform remote parking by sending the diagnostic signal 302, via the communication module 118, to the mobile device 204 to monitor communication during the current remote motive control session. For example, when the user activates the remote motive control application on the mobile device 204, the mobile device 204 displays an interface of the remote motive control application, such as an unlock screen or other such interface. As such, the communication module 118 sends the diagnostic signal 302 to correspond with presentation of the unlock screen by the mobile device 204 such that latency between the communication module 118 and the mobile device 204 can be determined before starting a maneuver of the remote motive control session (e.g., remote parking). While it is disclosed that the diagnostic signal 302 corresponds to the presentation of the unlock screen of the remote motive control application, it will be understood that the diagnostic signal 302 may correspond with other portions of the remote motive control application, such as but not limited to, presentation of the input screen during performance of a remote motive control maneuver and other such portions of the remote motive control application.


In the illustrated example, the communication module 118 sends the diagnostic signal 302 from the autonomy unit controller 122 to the mobile device 204. In response to the mobile device 204 receiving the diagnostic signal 302 and detecting an input 304 in response to the diagnostic signal 302 (e.g., input to interface presented on display 502 of the mobile device 204), the mobile device 204 generates and sends a first return signal 306 (e.g., input on mobile device in response to diagnostic signal) to the autonomy unit controller 122 via the communication module 118 of the vehicle 100. That is, the communication module 118 sends the diagnostic signal 302 to the mobile device 204, and detection of the input 304 on the mobile device 204 triggers the mobile device 204 to send the return signal 306 to the autonomy unit controller 122 via the communication module 118.


Subsequently, the autonomy unit controller 122 determines a time period 308 (e.g., a first time period) between the sending of the diagnostic signal 302 and the receipt of the return signal 306. In some examples, the diagnostic signal 302 includes a unique identifier generated by the autonomy unit controller 122 (i.e., a number, a letter, a unique character, any other unique identifier or combination thereof) and the return signal 306 includes the unique identifier associated with the received diagnostic signal 302. As a result, the autonomy unit controller 122 determines when the communication module 118 sends the diagnostic signal 302 including the unique identifier and receives the corresponding return signal 306 including the unique identifier associated with the diagnostic signal 302.


In some examples, the autonomy unit controller 122 generates a first time stamp (i.e., first diagnostic signal time stamp) upon sending the diagnostic signal 302 including the unique identifier, via the communication module 118, to the mobile device 204. Furthermore, the autonomy unit controller 122 generates a second time stamp (i.e., second diagnostic signal time stamp) upon receiving the return signal including the unique identifier, via the communication module 118, from the mobile device 204. The autonomy unit controller 122 determines the time period 308 between the first time stamp and the second time stamp.


In some examples, during a continuous communication session the autonomy unit controller 122 sends a plurality of diagnostic signals 302 (i.e., first diagnostic signal, second diagnostic signal, third diagnostic signal, etc.) to the mobile device 204. The autonomy unit controller 122 receives a plurality of return signals 306 (i.e., first return signal, second return signal, third return signal, etc.) from the mobile device 204. Each of the plurality of diagnostic signals 302 are sent in consecutive order and include a different unique identifier (i.e., sequential numbers, sequential letters, different unique characters, any other such unique identifiers or combinations thereof). Each of the plurality of return signals 306 include the unique identifier that corresponds to a specific diagnostic signal 302 (i.e., a first unique identifier corresponds with a first diagnostic signal and corresponding first return signal). In other words, the autonomy unit controller 122 uses the unique identifier to confirm the receipt of the return signal 306 corresponds with a specific diagnostic signal 302 (i.e., the first return signal corresponds with the first diagnostic signal, the second return signal corresponds with the second diagnostic signal, the third return signal corresponds with the third diagnostic signal etc.). Additionally, or alternatively, the diagnostic signal 302 and the return signal 306 each include a respective time stamp to enable the autonomy unit controller 122 to determine when the diagnostic signal 302 was sent and the return signal 306 was received. In some examples the time stamps are relative to the start or initiation of the active remote motive control session.


In some examples, the diagnostic signal 302 includes the first time stamp (i.e., first diagnostic signal time stamp) generated by the autonomy unit controller 122 and the return signal 306 includes the first time stamp associated with the received diagnostic signal 302. Furthermore, the autonomy unit controller 122 generates the second time stamp (i.e., second diagnostic signal time stamp) upon receiving the return signal 306 including the first time stamp, via the communication module 118, from the mobile device 204. The autonomy unit controller 122 determines the time period 308 between the first time stamp and the second time stamp. In such examples, the first time stamp (i.e., first diagnostic signal time stamp) is included in the diagnostic signal 302 without the unique identifier. Alternatively, the first time stamp and the unique identifier may be included in the diagnostic signal 302.


In some examples, a device controller or other such component of the mobile device 204 generates a first time stamp (i.e., first device time stamp) upon detecting the input 304 in response to the diagnostic signal 302 and includes the first device time stamp in the return signal 306. In some examples, the autonomy unit controller 122 determines the time period 308 between the first diagnostic signal time stamp and the first device time stamp. In some examples, a time generated by the autonomy unit controller 122 is synchronized with a time generated by the device controller of the mobile device 204. In some examples the time stamps are relative to the start or initiation of the active remote motive control session.


The autonomy unit controller 122 also compares the time period 308 to a threshold 310 (e.g., a communication latency threshold). In some examples, the threshold 310 corresponds to a predefined time limit that is associated with latency in communication between the mobile device 204 and the communication module 118. A predetermined value for the threshold 310 is stored in a memory (i.e., memory 612 of FIG. 6) of the autonomy unit controller 122. For example, as illustrated in FIG. 3B, in response to determining that time period 308 is greater than the threshold 310, the autonomy unit controller 122 pauses, disables, and/or prevents the autonomy unit 120 from moving the vehicle 100 during the active remote motive control session. Additionally, in some examples the autonomy unit controller 122 sends a notification (e.g., notification 504 of FIG. 5) to the user of the mobile device 204. For example, the notification includes a suggestion that the mobile device 204 is communicating (e.g., streaming over BLE or other communication protocol) for non-remote motive control purposes. In some examples, the notification includes a list of potential sources of non-remote motive control communications, such as but not limited to, streaming video data, streaming audio data, and the like. The notification further includes an acknowledgement input (e.g., acknowledgment input 506 of FIG. 5) to confirm that the user received the notification and has taken corrective action to address the detected time period 308 being greater than the threshold 310. Otherwise, in response to detecting that the time period 308 is less than or equal to the threshold 310, the autonomy unit controller 122 enables the autonomy unit 120 to cause the vehicle 100 to move during remote parking.


In some examples, the autonomy unit 120 and/or the autonomy unit controller 122 are configured to monitor the processing latency of the mobile device 204 during the active remote motive control session. For example, in response to the mobile device 204 detecting the input 304, the mobile device 204 begins processing a task of the remote motive control application associated with the input 304, such as but not limited to, performing the unlock procedure to initiate the current remote motive control session, performing a remote motive control maneuver, and other such task. The mobile device 204 generates a second return signal 312 (e.g., task output signal) when the mobile device 204 finishes processing the current task and is ready to transmit an output signal or other such message back to the autonomy unit controller 122. That is, responsive to receiving the input 304, the mobile device 204 begins processing an instruction or task of the remote motive control application. For the given communication cycle between the mobile device 204 and the vehicle 100, the mobile device 204 generates the second return signal 312 upon completion of processing the instruction or task and sends the second return signal 312 to the autonomy unit controller 122 via the communication module 118.


In some examples, the autonomy unit controller 122 generates a third time stamp (i.e., third diagnostic signal time stamp) upon receiving the second return signal 312. The autonomy unit controller determines a time period 314 (e.g., a second time period) between the second time stamp and the third time stamp (e.g., completion of processing the task associated with the input 304). In other words, the autonomy unit controller 122 determines the time period 314 between receiving the return signal 306 and receiving the second return signal 312. In some examples, the second return signal 312 includes a time stamp, generated by the mobile device 204, associated with the detection of the input 304 and a time stamp associated with the generation of the second return signal 312 to enable the autonomy unit controller 122 to determine a processing time (i.e., time period 314) of the task associated with the input 304. In some examples the time stamps are relative to the start or initiation of the active remote motive control session. In some examples, the second return signal 312 includes the unique identifier associated with the received diagnostic signal 302. In some examples, the second return signal 312 includes the first time stamp (i.e., first diagnostic signal time stamp) associated with the sending of the diagnostic signal 302 to the mobile device.


The autonomy unit controller 122 also compares the time period 314 to a threshold 316 (e.g., a mobile device processing threshold or second threshold). For example, the threshold 316 corresponds to a predefined time limit that is associated with latency in processing of an instruction or task by the mobile device 204. A predetermined value for the threshold 316 is stored in a memory (i.e., memory 612 of FIG. 6) of the autonomy unit controller 122. In one such example as illustrated in FIG. 3C, in response to determining the time period 314 is greater than the threshold 316, the autonomy unit controller 122 pauses, disables and/or prevents the autonomy unit 120 from moving the vehicle 100 during the active remote motive control session. Additionally, in some examples the autonomy unit controller 122 sends a notification (e.g., notification 504 of FIG. 5) to the user of the mobile device 204. For example, the notification includes a suggestion that the mobile device 204 is overburdened and not able to process instructions within the threshold 316. In some examples, a determination of the type of burden includes a comparison of a total communication time with a processing time on the mobile device 204 and the vehicle 100. In one such example, a short local processing time by the mobile device 204 and a long communication period between the mobile device 204 and the vehicle 100 would imply a communication latency or other such communication issue. In another such example, a long local processing time on the mobile device 204 and a short communication period between the mobile device 204 and the vehicle 100 would imply a processing burden on the mobile device 204. In some examples, the notification includes a list of potential sources of mobile device 204 processing overburden, such as but not limited to, number of background applications running on the mobile device 204, a list of non-remote motive control applications running in the background of the mobile device 204, and other such sources. The notification further includes an acknowledgement input (e.g., acknowledgment input 506 of FIG. 5) to confirm that the user received the notification and has taken corrective action to address the detected time period 314 being greater than the threshold 316. Otherwise, in response to detecting that the time period 314 is less than or equal to the threshold 316, the autonomy unit controller 122 enables the autonomy unit 120 to cause the vehicle 100 to move during remote parking. While FIG. 3 shows the autonomy unit controller 122 incorporated with the autonomy unit 120, it will be understood that the autonomy unit 120 and the autonomy unit controller 122 may be separate components that are communicably coupled to one another within the vehicle 100.



FIG. 4A illustrates timestamps sent to the autonomy unit controller 122 to monitor the latency in processing of the autonomy unit 120 during a remote motive control session. The autonomy unit 120 of the vehicle sends a first timestamp 402 that corresponds with the start of a task performed by the autonomy unit 120 (e.g., autonomy unit task and/or instruction output signal). In some examples, the timestamp 402 marks or otherwise represents the start of each iteration of the remote motive control application (e.g., each iteration of the state machine while loop). Additionally, the autonomy unit 120 sends a second timestamp 404 that corresponds with the completion of the task performed by the autonomy unit 120 (e.g., autonomy unit task and/or instruction output signal). In some examples, the timestamp 404 marks or otherwise represents the end of the current iteration and a new state value (e.g., pre-maneuver state, maneuver state, etc.) of the remote motive control application being returned. For example, the user opens the remote motive control application on the mobile device 204 to initiate the remote motive control session. Provided the communication module 118 detects the mobile device 204 within tethering range (e.g., tethering range 208 of FIG. 2), the unlock screen (e.g., unlock screen presented on display 502 of FIG. 5) is presented on the mobile device 204 as part of initiating the remote motive control session. As such, the first timestamp 402 marks the beginning of the pre-maneuver state and the second timestamp 404 marks when the pre-maneuver state is completed and the active maneuver portion (e.g., remote parking of vehicle) is ready to begin.


Subsequently, the autonomy unit controller 122 determines an autonomy unit time period 406 between the generation of the first timestamp 402 and the generation of the second timestamp 404. In some examples, the autonomy unit controller 122 detects when the autonomy unit 120 generates and/or otherwise sends the first timestamp 402 and the second timestamp 404. Additionally, or alternatively, an output signal from the autonomy unit 120 may include the respective first and second time stamps 402, 404 to enable the autonomy unit controller 122 to determine the autonomy unit time period 406 between generation of a first output signal and a second output signal or other such output of the autonomy unit. In some examples, the first and second time stamps 402, 404 are relative to the start or initiation of the active remote motive control session.


The autonomy unit controller 122 also compares the autonomy unit time period 406 to a threshold 408 (e.g., autonomy unit delay threshold). For example, the threshold 408 corresponds to a predefined time limit that is associated with latency or delay in processing performed by the autonomy unit 120. A predetermined value for the threshold 408 is stored in a memory (i.e., memory 612 of FIG. 6) of the autonomy unit controller 122. In one such example as illustrated in FIG. 4B, if during a maneuver state (i.e., the pre-maneuver state) of the active remote motive control session the autonomy unit controller 122 determines the autonomy unit time period 406 is greater than the threshold 408, the autonomy unit controller 122 sends a notification to the user of the mobile device 204 communicating that the vehicle is experiencing a processing delay. The user is then asked to stand-by while the autonomy unit 120 performs a self-check in attempt to correct the issue. The autonomy unit controller 122 will send a follow-up notification to the user of the mobile device 204 when the self-check is complete. Alternatively, during another maneuver state (i.e., performing a parking maneuver) of remote motive control, in response to determining the autonomy unit time period 406 is greater than the threshold 408, the autonomy unit controller 122 instructs the autonomy unit 120 to pause remote motive control of the vehicle. In some examples, if the delay can be resolved prior to stopping or pausing the vehicle and/or prior to reaching a threshold change in vehicle speed, then the autonomy unit 120 can resume normal operation. Alternatively, if the delay cannot be resolved and the vehicle is unable to resume processing tasks associated with the autonomy unit 120 processing delay, the autonomy unit controller 122 disables the autonomy unit 120 to prevent the vehicle 100 from moving until the delay can be resolved. The autonomy unit controller 122 sends a notification (e.g., notification 504 of FIG. 5) to notify the user the remote motive control will remain paused for a predetermined amount of time in attempt to resolve the autonomy unit 120 processing delay. The autonomy unit controller 122 will abort the current remote motive control maneuver if the delay cannot be resolved in the predetermined amount of time. Alternatively, if during the pre-maneuver state of the active remote motive control session the autonomy unit controller determines the autonomy unit time period 406 is less than or equal to the threshold 408, the autonomy unit controller 122 enables the autonomy unit 120 to move the vehicle during the active remote motive control session.



FIG. 5 is a block diagram of the display 502 of the mobile device 204. As illustrated in FIG. 5 the display 502 includes a notification 504 and acknowledgement input 506. In some examples, during use of the remote motive control application the display 502 presents a user interface to the user of the mobile device 204 that corresponds to the active remote motive control session. For example, the display 502 presents a plurality of different user interfaces during a remote motive control session such as but not limited to, an unlock screen during a pre-maneuver state, a maneuver input screen during a maneuver state, a pause screen during a pause state, and other screens associated with states of the active remote motive control session.


In some examples, in addition to presenting screens associated with the active remote motive control session, the display 502 presents the notification 504 and/or the acknowledgment input 506 to provide information to the user of the mobile device 204 related to the communication between the vehicle and the mobile device 204. In one such example, the notification 504 includes a suggestion that the mobile device 204 is communicating (e.g., streaming over BLE or other communication protocol) for non-remote motive control purposes. Determination of this notification may include verifying that the local processing time of the mobile device 204 is within allowable latency, such that the detected latency is likely caused by sharing of communication resources by the mobile device 204. The notification may further include a list of potential sources of non-remote motive control communications, such as but not limited to, streaming video data, streaming audio data, and the like. In some examples, the display 502 further presents the acknowledgement input 506 to confirm that the user received the notification 504 and has taken corrective action to address the detected communication delay between the vehicle and the mobile device 204.


In some examples, the display 502 presents the notification 504 and/or the acknowledgment input 506 to provide information to the user of the mobile device 204 related to processing of the mobile device 204. In one such example, the notification 504 includes a suggestion that the mobile device 204 is overburdened and not able to process instructions within a predefined threshold. In such examples, the notification 504 includes a list of potential sources of mobile device 204 processing overburden, such as but not limited to, the number of background applications running on the mobile device 204, a list of non-remote motive control applications running in the background of the mobile device 204, and other such sources. In some examples, the display 502 further presents the acknowledgement input 506 to confirm that the user received the notification 504 and taken corrective action to address the detected processing delay on the mobile device 204.


In some examples, the display presents the notification 504 and/or the acknowledgment input 506 to provide information to the user of the mobile device 204 related to processing of the autonomy unit (e.g., autonomy unit 120 of FIG. 1). In one such example, the notification includes a suggestion the autonomy unit is overburdened and unable to process instructions within a predefined threshold. In such examples, the notification 504 includes an instruction to stand-by while the autonomy unit 120 performs a self-check in attempt to correct the issue. The autonomy unit controller 122 will send a follow-up notification to the user of the mobile device 204 when the self-check is complete. Alternatively, if the self-check is not successful in correcting the issue, the display 502 presents the notification 504 to notify the user that the active remote motive control session will remain paused for a predetermined amount of time in attempt to resolve autonomy unit processing delay. Furthermore, the notification 504 communicates that the active remote motive control session is aborted if the autonomy unit processing delay is not resolved within the predetermined amount of time.



FIG. 6 is a block diagram of electronic components 600 of the vehicle 100. As illustrated in FIG. 6, the electronic components 600 include an on-board computing platform 602, the external lamps 114, the horn 116, the communication module 118, sensors 604, electronic control units (ECUs) 606, and a vehicle data bus 608.


The on-board computing platform 602 includes a microcontroller unit, controller or processor 610 and memory 612. In some examples, the processor 610 of the on-board computing platform 602 is structured to include the autonomy unit controller 122. Alternatively, in some examples, the autonomy unit controller 122 is incorporated into another electronic control unit (ECU) with its own processor 610 and memory 612. The processor 610 may be any suitable processing device or set of processing devices such as, but not limited to, a microprocessor, a microcontroller-based platform, an integrated circuit, one or more field programmable gate arrays (FPGAs), and/or one or more application-specific integrated circuits (ASICs). The memory 612 may be volatile memory (e.g., RAM including non-volatile RAM, magnetic RAM, ferroelectric RAM, etc.), non-volatile memory (e.g., disk memory, FLASH memory, EPROMs, EEPROMs, memristor-based non-volatile solid-state memory, etc.), unalterable memory (e.g., EPROMs), read-only memory, and/or high-capacity storage devices (e.g., hard drives, solid state drives, etc.). In some examples, the memory 612 includes multiple kinds of memory, particularly volatile memory and non-volatile memory.


The memory 612 is computer readable media on which one or more sets of instructions, such as the software for operating the methods of the present disclosure, can be embedded. The instructions may embody one or more of the methods or logic as described herein. For example, the instructions reside completely, or at least partially, within any one or more of the memory 612, the computer readable medium, and/or within the processor 610 during execution of the instructions.


The terms “non-transitory computer-readable medium” and “computer-readable medium” include a single medium or multiple media, such as a centralized or distributed database, and/or associated caches and servers that store one or more sets of instructions. Further, the terms “non-transitory computer-readable medium” and “computer-readable medium” include any tangible medium that is capable of storing, encoding or carrying a set of instructions for execution by a processor or that cause a system to perform any one or more of the methods or operations disclosed herein. As used herein, the term “computer readable medium” is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals.


The sensors 604 are arranged in and around the vehicle 100 to monitor properties of the vehicle 100 and/or an environment in which the vehicle 100 is located. One or more of the sensors 604 may be mounted to measure properties around an exterior of the vehicle 100. Additionally, or alternatively, one or more of the sensors 604 may be mounted inside a cabin of the vehicle 100 or in a body of the vehicle 100 (e.g., an engine compartment, wheel wells, etc.) to measure properties in an interior of the vehicle 100. For example, the sensors 604 include accelerometers, odometers, tachometers, pitch and yaw sensors, wheel speed sensors, microphones, tire pressure sensors, biometric sensors and/or sensors of any other suitable type.


In the illustrated example, the sensors 604 include the engine sensor 108, the transmission position sensor 110, one or more proximity sensors 614, and/or any other sensor (e.g., the vehicle speed sensor 112) that monitors a property of the vehicle 100 and/or the surrounding area. For example, the engine sensor 108 detects whether the engine 102 is activated or deactivated, and the transmission position sensor 110 detects a position (e.g., park, neutral, reverse, drive, first gear, etc.) of the transmission 104 and/or the transmission gear selector 106. Further, the proximity sensors 614 monitor a surrounding area of the vehicle 100 to collect data that detects and identifies location(s) of object(s) near the vehicle 100. For example, the proximity sensors 614 collect the data to facilitate the autonomy unit 120 in performing autonomous and/or semi-autonomous driving maneuvers (e.g., remote parking) of the vehicle 100. The proximity sensors 614 include radar sensor(s), LIDAR sensor(s), ultrasonic sensor(s), and/or any other sensor(s) that detects the presence and location of nearby objects. For example, a radar sensor detects and locates an object via radio waves, a LIDAR sensor detects and locates the object via lasers, and an ultrasonic sensor detects and locates the object via ultrasound waves.


The ECUs 606 monitor and control the subsystems of the vehicle 100. For example, the ECUs 606 are discrete sets of electronics that include their own circuit(s) (e.g., integrated circuits, microprocessors, memory, storage, etc.) and firmware, sensors, actuators, and/or mounting hardware. The ECUs 606 communicate and exchange information via a vehicle data bus (e.g., the vehicle data bus 608). Additionally, the ECUs 606 may communicate properties (e.g., status of the ECUs 606, sensor readings, control state, error and diagnostic codes, etc.) to and/or receive requests from each other. For example, the vehicle 100 may have seventy or more of the ECUs 606 that are positioned in various locations around the vehicle 100 and are communicatively coupled by the vehicle data bus 608.


In the illustrated example, the ECUs 606 include the autonomy unit 120 and a body control module 616. The autonomy unit 120 controls performance of autonomous and/or semi-autonomous driving maneuvers (e.g., remote parking) of the vehicle 100 based upon, at least in part, image(s) and/or video captured by camera(s) and/or data collected by one or more of the sensors 604 of the vehicle 100. The body control module 616 controls one or more subsystems throughout the vehicle 100, such as power windows, power locks, an immobilizer system, power mirrors, etc. For example, the body control module 616 includes circuits that drive one or more of relays (e.g., to control wiper fluid, etc.), brushed direct current (DC) motors (e.g., to control power seats, power locks, power windows, wipers, etc.), stepper motors, LEDs, etc.


The vehicle data bus 608 communicatively couples the external lamps 114, the horn 116, the communication module 118, the on-board computing platform 602, the sensors 604, and the ECUs 606. In some examples, the vehicle data bus 608 includes one or more data buses. The vehicle data bus 608 may be implemented in accordance with a controller area network (CAN) bus protocol as defined by International Standards Organization (ISO) 11898-1, a Media Oriented Systems Transport (MOST) bus protocol, a CAN flexible data (CAN-FD) bus protocol (ISO 11898-7) and/a K-line bus protocol (ISO 9141 and ISO 14230-1), and/or an Ethernet™ bus protocol IEEE 802.3 (2002 onwards), etc.



FIG. 7 is a flowchart of an example method 700 to monitor communication for vehicle remote motive control. The flowchart of FIG. 7 is representative of machine readable instructions that are stored in memory (such as the memory 612 of FIG. 6) and include one or more programs which, when executed by a processor (such as the processor 610 of FIG. 6), cause the vehicle 100 to implement the example autonomy unit controller 122 of FIGS. 1 and 6. While the example program is described with reference to the flowchart illustrated in FIG. 7, many other methods of implementing the example autonomy unit controller 122 may alternatively be used. For example, the order of execution of the blocks may be rearranged, changed, eliminated, and/or combined to perform the method 700. Further, because the method 700 is disclosed in connection with the components of FIGS. 1-6, some functions of those components will not be described in detail below.


Initially, at block 702, the autonomy unit controller 122 determines whether a remote motive control session (e.g., remote parking) of the vehicle 100 is active. For example, the autonomy unit controller 122 determines that remote parking is active for the vehicle 100 in response to receiving a remote motive control instruction from the mobile device 204, determining that the mobile device 204 is within the tethering range 208 of the vehicle 100, detecting via the engine sensor 108 that the engine 102 is active, and/or detecting via the transmission position sensor 110 that the transmission 104 is not in park. In response to the autonomy unit controller 122 determining that remote parking is not activated, the method 700 remains at block 702. Otherwise, in response to the autonomy unit controller 122 determining that remote parking is activated, the method 700 proceeds to block 704.


At block 704, the autonomy unit controller 122 sends, via the communication module 118 of the vehicle 100, a diagnostic signal 302 to the mobile device 204 that is being utilized to initiate the remote motive control. For example, the user 206 opens a remote motive control application on the mobile device 204 to initiate the remote motive control session. An interface such as an unlock screen is presented on the display 502 of the mobile device 204 as part of initiating the remote motive control session. As such, the communication module 118 sends the diagnostic signal 302 to correspond with presentation of the interface on the display 502 such that latency between the communication module 118 and the mobile device 204 can be determined before a maneuver of the remote motive control session (e.g., remote parking) begins. In some examples, the diagnostic signal 302 includes a unique identifier (i.e., time information or other such unique identifier).


At block 706, the mobile device 204 determines whether an input is detected responsive to the presentation of the interface on the display 502. In response to the mobile device 204 detecting an input 304, at block 708, the mobile device 204 generates and sends a return signal 306 to the communication module 118 of the vehicle 100. In some examples, the return signal 306 includes the unique identifier included with the diagnostic signal 302. That is, upon detection of an initial input response (e.g., input 304), the mobile device 204 generates and sends the return signal 306 including the unique identifier associated with the diagnostic signal 302 to the autonomy unit controller 122. Thus, the autonomy unit controller 122 uses the unique identifier to confirm the received return signal 306 corresponds to the diagnostic signal 302. In another such example, the return signal 306 may include the unique identifier and a time stamp or other such information to the communication module 118 of the vehicle 100. Alternatively, in response to the mobile device 204 not detecting the input 304, the method 700 returns to block 702.


At block 710, the autonomy unit controller 122 receives, via the communication module 118, the return signal 306 sent by the mobile device 204. At block 712, the autonomy unit controller 122 determines a time period 308 between a sending of the diagnostic signal 302 and a receipt of the return signal 306. In some examples, the autonomy unit controller 122 detects when the communication module 118 sends the diagnostic signal 302 and receives the return signal 306 to determine the time period 308 between the sending of diagnostic signal 302 and the receipt of the return signal 306. Additionally, or alternatively, the diagnostic signal 302 and the return signal 306 include respective time stamps to enable the autonomy unit controller 122 to determine the time period 308 between the diagnostic signal 302 and the return signal 306.


At block 714, the autonomy unit controller 122 determines whether the time period 308 between the diagnostic signal 302 and the return signal 306 is greater than a latency threshold 310. In response to the autonomy unit controller 122 determining that the time period 308 is greater than the latency threshold 310, the method 700 proceeds to block 716 at which the autonomy unit controller 122 prevents the autonomy unit 120 from causing movement of the vehicle 100 during the remote motive control session. At block 718, the autonomy unit controller 122 sends a notification 504 to the mobile device 204 to indicate to the user 206 that the autonomy unit controller 122 has detected an unacceptable amount of latency in communication between the mobile device 204 and the vehicle 100. For example, the notification 504 sent to the mobile device includes a suggestion to the user 206 that the mobile device 204 is communicating (e.g., streaming over BLE) for non-remote motive control purposes. In some examples, the notification 504 includes a list of potential sources of non-remote motive control communications, such as but not limited to, streaming video data, streaming audio data, streaming GPS data, and the like. The notification 504 further includes an acknowledgement input 506 to provide confirmation to the autonomy unit controller 122 that the user received the notification 504 and has taken corrective action to address the unacceptable amount of latency in communication between the mobile device 204 and the vehicle 100.


At block 720, the autonomy unit controller 122 determines if the acknowledgment input 506 was received from the user. If the acknowledgement input was received, the method 700 returns to block 704 to determine if corrective action taken by the user (e.g., stop streaming non-remote motive control applications) was successful in addressing the detected unacceptable amount of latency in communication between the mobile device 204 and the vehicle 100. In some examples, upon return to block 704, the autonomy unit 120 waits a predetermined time (e.g., 1 to 5 seconds) to repeat the process in order to provide the user an opportunity to look for potential sources of non-remote motive control communication. In some examples, if the unacceptable amount of latency communication delay between the vehicle 100 and mobile device 204 persists after a predetermined number of attempts, the method 700 proceeds to block 722 and the current remote motive control session is aborted. Alternatively, if the acknowledgment input was not received from the user at block 720, the method 700 proceeds to block 722 and the current remote motive control session is aborted. To restart remote motive control of the vehicle 100, the user 206 subsequently is to reinitiate remote parking of the vehicle 100.


Returning to block 714, in response to the autonomy unit controller 122 determining that the time period is less than or equal to the latency threshold, the method 700 proceeds to block 724 at which the autonomy unit controller 122 enables the autonomy unit 120 to cause movement of the vehicle 100 during the remote motive control session. For example, in a remote motive control session such as remote parking, when the time period is less than or equal to the latency threshold, the autonomy unit 120 autonomously controls motive functions of the vehicle 100 to perform remote parking of the vehicle 100 into an available parking spot (e.g., a parking spot 202) and/or to otherwise autonomously and/or semi-autonomously drive the vehicle 100.


At block 726, the autonomy unit controller 122 continues to monitor the communication between the mobile device 204 and the vehicle 100 during the remote motive control session. Accordingly, the method 700 returns to block 702 to monitor communication between the mobile device 204 and the vehicle 100 and ensure there is an acceptable amount of latency between the mobile device 204 and vehicle 100 during the active remote motive control session.



FIG. 8 is a flowchart of an example method 800 to monitor processing on the mobile device during vehicle remote motive control. The flowchart of FIG. 8 is representative of machine readable instructions that are stored in memory (such as the memory 612 of FIG. 6) and include one or more programs which, when executed by a processor (such as the processor 610 of FIG. 6), cause the vehicle 100 to implement the example autonomy unit controller 122 of FIGS. 1 and 6. While the example program is described with reference to the flowchart illustrated in FIG. 8, many other methods of implementing the example autonomy unit controller 122 may alternatively be used. For example, the order of execution of the blocks may be rearranged, changed, eliminated, and/or combined to perform the method 800. Further, because the method 800 is disclosed in connection with the components of FIGS. 1-6, some functions of those components will not be described in detail below.


Initially, at block 802, the autonomy unit controller 122 determines whether remote parking of the vehicle 100 is active. For example, the autonomy unit controller 122 determines that remote parking is active for the vehicle 100 in response to receiving a remote parking instruction from the mobile device 204, determining that the mobile device 204 is within the tethering range 208 of the vehicle 100, detecting via the engine sensor 108 that the engine 102 is active, and/or detecting via the transmission position sensor 110 that the transmission 104 is not in park. In response to the autonomy unit controller 122 determining that remote parking is not activated, the method 800 remains at block 802. Otherwise, in response to the autonomy unit controller 122 determining that remote parking is activated, the method 800 proceeds to block 804.


At block 804, the mobile device receives an input 304 associated with an instruction or task performed by the mobile device 204 during the active remote motive control session. Following detection of the input 304, at block 806, the mobile device 204 begins processing the task responsive to receipt of the input 304. For example, during initiation of the remote motive control session, the mobile device 204 processes or otherwise performs an unlock procedure to initiate the active remote motive control session. The mobile device 204 generates a second return signal 312 (e.g., task output signal) upon completion of processing and/or otherwise performing the task. In some examples, the mobile device 204 determines a time period (e.g., second time period 314) between detection of the input 304, which represents a start of processing the task, and generation of the second return signal 312, which represents the completion of processing the task associated with the input 304. In such examples, the time period 314 is included in the second return signal 312.


At block 808, the autonomy unit controller 122 receives, via the communication module 118, the second return signal 312 sent by the mobile device 204. At block 810, the autonomy unit controller 122 compares the time period 314 with a mobile device processing threshold (e.g., threshold 316). In some examples, the mobile device processing threshold corresponds with a lower limit of time that is associated with latency in processing of an instruction or task by the mobile device 204.


At block 812, the autonomy unit controller 122 determines whether the time period 314 between detection of the input 304 (e.g., start of task processing) and the generation of the second return signal 312 (e.g., completion of task processing) is greater than the threshold 316. In response to the autonomy unit controller 122 determining that the time 314 period is greater than the threshold 316, the method 800 proceeds to block 714 at which the autonomy unit controller 122 prevents the autonomy unit 120 from causing movement of the vehicle 100 during the remote motive control session. At block 816, the autonomy unit controller 122 sends a notification to the mobile device 204 to indicate to the user 206 that the autonomy unit controller 122 has detected an unacceptable amount of latency in processing by the mobile device 204. For example, the notification sent to the mobile device includes a suggestion to the user 206 that the mobile device 204 is overburdened and not able to process tasks and/or instructions within the threshold 316. In some examples, the notification includes a list of potential sources of the detected overburdening of the mobile device 204, such as but not limited to, the number of applications running in the mobile device 204 background, a list of non-remote motive control applications running in the mobile device 204 background, and other such sources. The notification further includes an acknowledgement input provide confirmation to the autonomy unit controller 122 that the user received the notification and has taken corrective action to address the unacceptable amount of latency in processing by the mobile device 204.


At block 818, the autonomy unit controller 122 determines if the acknowledgment input was received from the user. If the acknowledgement input was received, the method 800 returns to block 804 to repeat the steps of the method 800 and determine if corrective action taken by the user (e.g., close non-remote motive control applications running in the mobile device 204 background) was successful in addressing the detected unacceptable amount of latency in processing by the mobile device 204. In some examples, upon return to block 704, the autonomy unit 120 waits a predetermined time (e.g., 1 to 5 seconds) to repeat the process in order to provide the user an opportunity to look for potential sources of overburden to the mobile device 204. In some examples, if the unacceptable amount of latency in processing by the mobile device 204 persists after a predetermined number of attempts, the method 800 proceeds to block 820 and the current remote motive control session is aborted. Similarly, if the acknowledgment input was not received from the user at block 818, the method 800 proceeds to block 820 and the current remote motive control session is aborted. To restart remote motive control of the vehicle 100, the user 206 subsequently is to reinitiate remote parking of the vehicle 100.


Returning to block 812, in response to the autonomy unit controller 122 determining that the time period is less than or equal to the threshold 316, the method 800 proceeds to block 822 at which the autonomy unit controller 122 enables the autonomy unit 120 to cause movement of the vehicle 100 during the remote motive control session. For example, the remote motive control includes remote parking of the vehicle 100. When the time period is less than or equal to the latency threshold, the autonomy unit 120 autonomously controls motive functions of the vehicle 100 to perform remote parking of the vehicle 100 into an available parking spot (e.g., a parking spot 202) and/or to otherwise autonomously and/or semi-autonomously drive the vehicle 100.


At block 824, the autonomy unit controller 122 continues to monitor the processing of the mobile device 204 during the remote motive control session. Accordingly, the method 800 returns to block 802 to monitor the processing of the mobile device 204 and ensure there is an acceptable amount of latency on processing performed on the mobile device 204 during the active remote motive control session.


In this application, the use of the disjunctive is intended to include the conjunctive. The use of definite or indefinite articles is not intended to indicate cardinality. In particular, a reference to “the” object or “a” and “an” object is intended to denote also one of a possible plurality of such objects. Further, the conjunction “or” may be used to convey features that are simultaneously present instead of mutually exclusive alternatives. In other words, the conjunction “or” should be understood to include “and/or”. The terms “includes,” “including,” and “include” are inclusive and have the same scope as “comprises,” “comprising,” and “comprise” respectively. Additionally, as used herein, the terms “module,” “unit,” and “node” refer to hardware with circuitry to provide communication, control and/or monitoring capabilities, often in conjunction with sensors. A “module,” a “unit,” a “node” may also include firmware that executes on the circuitry.


The above-described embodiments, and particularly any “preferred” embodiments, are possible examples of implementations and merely set forth for a clear understanding of the principles of the invention. Many variations and modifications may be made to the above-described embodiment(s) without substantially departing from the spirit and principles of the techniques described herein. All modifications are intended to be included herein within the scope of this disclosure and protected by the following claims.

Claims
  • 1. A vehicle comprising: a communication module; andan autonomy unit comprising an autonomy unit controller configured to send a diagnostic signal including a unique identifier, via the communication module, to a mobile device, wherein the mobile device includes a device controller configured to generate a return signal including the unique identifier upon detecting an input responsive to receipt of the diagnostic signal; andwherein the autonomy unit controller is configured to receive the return signal including the unique identifier from the mobile device, via the communication module, and determine a time period between sending the diagnostic signal and receiving the return signal, and wherein responsive to a determination that the time period is greater than a threshold, the autonomy unit controller disables the autonomy unit to prevent movement of the vehicle during a remote motive control session of the vehicle and sends a notification to the mobile device including a first plurality of potential causes for the time period being greater than the threshold.
  • 2. The vehicle of claim 1, wherein the autonomy unit controller generates a first diagnostic signal time stamp when the diagnostic signal including the unique identifier is sent to the mobile device and the autonomy unit controller generates a second diagnostic signal time stamp when the return signal including the unique identifier is received from the mobile device, and wherein the autonomy unit controller determines the time period between the first diagnostic signal time stamp and the second diagnostic signal time stamp.
  • 3. The vehicle of claim 1, wherein, the communication module is configured to wirelessly communicate via a Bluetooth® low energy protocol, a Classic Bluetooth® protocol, a Wi-Fi® protocol, a Wi-Fi® lower power protocol, and an Ultra-Wide Band (UWB) protocol.
  • 4. The vehicle of claim 1, wherein responsive to determining the time period is less than or equal to the threshold, the autonomy unit controller enables the autonomy unit to cause movement during the remote motive control session of the vehicle.
  • 5. The vehicle of claim 1, wherein the notification further includes an acknowledgement input confirming investigation by a user of one or more of the first plurality of potential causes responsible for the time period being greater than the threshold, and wherein a confirmation response is received by the autonomy unit controller to trigger sending of a second diagnostic signal to the mobile device.
  • 6. The vehicle of claim 1, wherein after the mobile device detects the input, the device controller is to: process a task associated with the input;generate a second return signal including the unique identifier; andsend the second return signal to the autonomy unit controller, wherein the autonomy unit controller determines a second time period between receiving the return signal and receiving the second return signal.
  • 7. The vehicle of claim 6, wherein responsive to determining the second time period is greater than a second threshold, the autonomy unit controller disables the autonomy unit to prevent movement during the remote motive control session of the vehicle.
  • 8. The vehicle of claim 7, wherein the autonomy unit controller sends a notification to the mobile device including a second plurality of potential causes responsible for the second time period being greater than the second threshold.
  • 9. The vehicle of claim 8, wherein the notification further includes an acknowledgement input confirming investigation by a user of one or more of the second plurality of potential causes responsible for the second time period being greater than the second threshold, and wherein a confirmation response is received by the autonomy unit controller to trigger sending of a second diagnostic signal to the mobile device.
  • 10. The vehicle of claim 6, wherein responsive to determining the second time period is less than or equal to a second threshold, the autonomy unit controller enables the autonomy unit to cause movement during the remote motive control session of the vehicle.
  • 11. A method performed by a vehicle comprising: sending, via a communication module of a vehicle, a diagnostic signal including a first diagnostic signal time stamp from an autonomy unit controller of the vehicle to a mobile device;detecting, by the mobile device, an input responsive to receipt of the diagnostic signal;generating, by the mobile device, a return signal including the first diagnostic signal time stamp upon detecting the input;receiving, via the communication module, the return signal including the first diagnostic signal time stamp by the autonomy unit controller of the vehicle;generating, by the autonomy unit controller, a second diagnostic signal time stamp upon receiving the return signal;determining, via a processor of the autonomy unit controller, a time period between the first diagnostic signal time stamp and the second diagnostic signal time stamp;disabling, via the processor of the autonomy unit controller, an autonomy unit from causing movement of the vehicle responsive to determining the time period is greater than a threshold; andsending, via the communication module, a notification to the mobile device including a first plurality of potential causes for the time period being greater than the threshold.
  • 12. The method of claim 11, wherein a time of the autonomy unit controller is synchronized with a time of the mobile device, and the method further includes, generating, by the mobile device, a first device time stamp upon detecting the input, including the first device time stamp in the return signal, and determining, via the processor of the autonomy unit controller, the time period between the first diagnostic signal time stamp and the first device time stamp.
  • 13. The method of claim 11, wherein responsive to determining the time period is less than or equal to the threshold, enabling, via the processor of the autonomy unit controller, the autonomy unit to cause movement of the vehicle.
  • 14. The method of claim 13, wherein the notification further includes an acknowledgement input confirming investigation by a user of one or more of the first plurality of potential causes responsible for the time period being greater than the threshold.
  • 15. The method of claim 14, further includes receiving a confirmation response, by the autonomy unit controller, confirming investigation by the user of one or more of the first plurality of potential causes responsible for the time period being greater than the threshold, wherein the confirmation response triggers sending a second diagnostic signal to the mobile device.
  • 16. The method of claim 11, wherein detecting the input, by the mobile device, triggers the mobile device to: process a task associated with the input;generate a second return signal;send the second return signal to the autonomy unit controller, wherein the autonomy unit controller generates a third diagnostic signal time stamp upon receiving the second return signal, and wherein the autonomy unit controller determines a second time period between the second diagnostic signal time stamp and the third diagnostic signal time stamp.
  • 17. The method of claim 16, wherein responsive to determining the second time period is greater than a second threshold, the autonomy unit is disabled to prevent movement of the vehicle during remote motive control of the vehicle.
  • 18. The method of claim 17, wherein responsive to determining the second time period is greater than the second threshold, the autonomy unit controller sends a notification to the mobile device including a second plurality of potential causes responsible for the second time period being greater than the second threshold.
US Referenced Citations (354)
Number Name Date Kind
5959724 Izumi Sep 1999 A
6275754 Shimizu Aug 2001 B1
6356828 Shimizu Mar 2002 B1
6452617 Bates Sep 2002 B1
6476730 Kakinami Nov 2002 B2
6477260 Shimomura Nov 2002 B1
6657555 Shimizu Dec 2003 B2
6683539 Trajkovic Jan 2004 B2
6724322 Tang Apr 2004 B2
6744364 Wathen Jun 2004 B2
6768420 McCarthy Jul 2004 B2
6801855 Walters Oct 2004 B1
6850844 Walters Jan 2005 B1
6850148 Masudaya Feb 2005 B2
6927685 Wathen Aug 2005 B2
6997048 Komatsu Feb 2006 B2
7042332 Takamura May 2006 B2
7123167 Staniszewski Oct 2006 B2
7307655 Okamoto Dec 2007 B1
7663508 Teshima Feb 2010 B2
7737866 Wu Jun 2010 B2
7813844 Gensler Oct 2010 B2
7825828 Watanabe Nov 2010 B2
7834778 Browne Nov 2010 B2
7847709 McCall Dec 2010 B2
7850078 Christenson Dec 2010 B2
7924483 Smith Apr 2011 B2
8035503 Partin Oct 2011 B2
8054169 Bettecken Nov 2011 B2
8098146 Petrucelli Jan 2012 B2
8126450 Howarter Feb 2012 B2
8164628 Stein Apr 2012 B2
8180524 Eguchi May 2012 B2
8180547 Prasad May 2012 B2
8224313 Howarter Jul 2012 B2
8229645 Lee Jul 2012 B2
8242884 Holcomb Aug 2012 B2
8335598 Dickerhoof Dec 2012 B2
8401235 Lee Mar 2013 B2
8493236 Boehme Jul 2013 B2
8538408 Howarter Sep 2013 B2
8542130 Lavoie Sep 2013 B2
8552856 McRae Oct 2013 B2
8587681 Guidash Nov 2013 B2
8594616 Gusikhin Nov 2013 B2
8599043 Kadowaki Dec 2013 B2
8618945 Furuta Dec 2013 B2
8645015 Oetiker Feb 2014 B2
8655551 Danz Feb 2014 B2
8692773 You Apr 2014 B2
8706350 Talty Apr 2014 B2
8725315 Talty May 2014 B2
8742947 Nakazono Jun 2014 B2
8744684 Hong Jun 2014 B2
8780257 Gidon Jul 2014 B2
8787868 Leblanc Jul 2014 B2
8825262 Lee Sep 2014 B2
8933778 Birkel Jan 2015 B2
8957786 Stempnik Feb 2015 B2
8994548 Gaboury Mar 2015 B2
8995914 Nishidai Mar 2015 B2
9008860 Waldock Apr 2015 B2
9014920 Torres Apr 2015 B1
9078200 Wuergler Jul 2015 B2
9086879 Gautama Jul 2015 B2
9141503 Chen Sep 2015 B1
9147065 Lauer Sep 2015 B2
9154920 O'Brien Oct 2015 B2
9168955 Noh Oct 2015 B2
9193387 Auer Nov 2015 B2
9225531 Hachey Dec 2015 B2
9230439 Boulay Jan 2016 B2
9233710 Lavoie Jan 2016 B2
9273966 Bartels Mar 2016 B2
9275208 Protopapas Mar 2016 B2
9283960 Lavoie Mar 2016 B1
9286803 Tippelhofer Mar 2016 B2
9302675 Schilling Apr 2016 B2
9318022 Barth Apr 2016 B2
9379567 Kracker Jun 2016 B2
9381859 Nagata Jul 2016 B2
9429657 Sidhu Aug 2016 B2
9429947 Wengreen Aug 2016 B1
9454251 Guihot Sep 2016 B1
9469247 Juneja Oct 2016 B2
9493187 Pilutti Nov 2016 B2
9506774 Shutko Nov 2016 B2
9511799 Lavoie Dec 2016 B2
9522675 You Dec 2016 B1
9529519 Blumenberg Dec 2016 B2
9557741 Elie Jan 2017 B1
9563990 Khan Feb 2017 B2
9595145 Avery Mar 2017 B2
9598051 Okada Mar 2017 B2
9606241 Varoglu Mar 2017 B2
9616923 Lavoie Apr 2017 B2
9637117 Gusikhin May 2017 B1
9651655 Feldman May 2017 B2
9656690 Shen May 2017 B2
9666040 Flaherty May 2017 B2
9688306 McClain Jun 2017 B2
9701280 Schussmann Jul 2017 B2
9712977 Tu Jul 2017 B2
9715816 Adler Jul 2017 B1
9725069 Krishnan Aug 2017 B2
9731714 Kiriya Aug 2017 B2
9731764 Baek Aug 2017 B2
9754173 Kim Sep 2017 B2
9809218 Elie Nov 2017 B2
9811085 Hayes Nov 2017 B1
9842444 Van Wiemeersch Dec 2017 B2
9845070 Petel Dec 2017 B2
9846431 Petel Dec 2017 B2
9914333 Shank Mar 2018 B2
9921743 Bryant Mar 2018 B2
9946255 Matters Apr 2018 B2
9959763 Miller May 2018 B2
9971130 Lin May 2018 B1
9975504 Dalke May 2018 B2
10019001 Dang Van Nhan Jul 2018 B2
10032276 Liu Jul 2018 B1
10040482 Jung Aug 2018 B1
10043076 Zhang Aug 2018 B1
10131347 Kim Nov 2018 B2
10192113 Liu Jan 2019 B1
10246055 Farges Apr 2019 B2
10268341 Kocienda Apr 2019 B2
10974717 Golgiri Apr 2021 B2
20030060972 Kakinami Mar 2003 A1
20030098792 Edwards May 2003 A1
20030133027 Itoh Jul 2003 A1
20050030156 Alfonso Feb 2005 A1
20050068450 Steinberg Mar 2005 A1
20050099275 Kamdar May 2005 A1
20060010961 Gibson Jan 2006 A1
20060227010 Berstis Oct 2006 A1
20060235590 Bolourchi Oct 2006 A1
20070230944 Georgiev Oct 2007 A1
20080027591 Lenser Jan 2008 A1
20080154464 Sasajima Jun 2008 A1
20080154613 Haulick Jun 2008 A1
20080238643 Malen Oct 2008 A1
20080306683 Ando Dec 2008 A1
20090096753 Lim Apr 2009 A1
20090098907 Huntzicker Apr 2009 A1
20090115639 Proefke May 2009 A1
20090125181 Luke May 2009 A1
20090125311 Haulick May 2009 A1
20090128315 Griesser May 2009 A1
20090146813 Nuno Jun 2009 A1
20090174574 Endo Jul 2009 A1
20090241031 Gamaley Sep 2009 A1
20090289813 Kwiecinski Nov 2009 A1
20090309970 Ishii Dec 2009 A1
20090313095 Hurpin Dec 2009 A1
20100025942 Mangaroo Feb 2010 A1
20100061564 Clemow Mar 2010 A1
20100114471 Sugiyama May 2010 A1
20100114488 Khamharn May 2010 A1
20100136944 Taylor Jun 2010 A1
20100152972 Attard Jun 2010 A1
20100156672 Yoo Jun 2010 A1
20100245277 Nakao Sep 2010 A1
20100259420 Von Rehyer Oct 2010 A1
20110071725 Kleve Mar 2011 A1
20110082613 Oetiker Apr 2011 A1
20110190972 Timmons Aug 2011 A1
20110205088 Baker Aug 2011 A1
20110253463 Smith Oct 2011 A1
20110309922 Ghabra Dec 2011 A1
20120007741 Laffey Jan 2012 A1
20120072067 Jecker Mar 2012 A1
20120083960 Zhu Apr 2012 A1
20120173080 Cluff Jul 2012 A1
20120176332 Fujibayashi Jul 2012 A1
20120271500 Tsimhoni Oct 2012 A1
20120303258 Pampus Nov 2012 A1
20120323643 Volz Dec 2012 A1
20120323700 Aleksandrovich Dec 2012 A1
20130021171 Hsu Jan 2013 A1
20130024202 Harris Jan 2013 A1
20130043989 Niemz Feb 2013 A1
20130073119 Huger Mar 2013 A1
20130109342 Welch May 2013 A1
20130110342 Wuttke May 2013 A1
20130113936 Cohen May 2013 A1
20130124061 Khanafer May 2013 A1
20130145441 Mujumdar Jun 2013 A1
20130211623 Thompson Aug 2013 A1
20130231824 Wilson Sep 2013 A1
20130289825 Noh Oct 2013 A1
20130314502 Urbach Nov 2013 A1
20130317944 Huang Nov 2013 A1
20140052323 Reichel Feb 2014 A1
20140095994 Kim Apr 2014 A1
20140096051 Boblett Apr 2014 A1
20140121930 Allexi May 2014 A1
20140147032 Yous May 2014 A1
20140156107 Karasawa Jun 2014 A1
20140188339 Moon Jul 2014 A1
20140222252 Matters Aug 2014 A1
20140240502 Strauss Aug 2014 A1
20140282931 Protopapas Sep 2014 A1
20140297120 Cotgrove Oct 2014 A1
20140300504 Shaffer Oct 2014 A1
20140303839 Filev Oct 2014 A1
20140320318 Victor Oct 2014 A1
20140327736 DeJohn Nov 2014 A1
20140350804 Park Nov 2014 A1
20140350855 Vishnuvajhala Nov 2014 A1
20140365108 You Dec 2014 A1
20140365126 Vulcano Dec 2014 A1
20150022468 Cha Jan 2015 A1
20150039173 Beaurepaire Feb 2015 A1
20150039224 Tuukkanen Feb 2015 A1
20150048927 Simmons Feb 2015 A1
20150066545 Kotecha Mar 2015 A1
20150077522 Suzuki Mar 2015 A1
20150088360 Bonnet Mar 2015 A1
20150091741 Stefik Apr 2015 A1
20150109116 Grimm Apr 2015 A1
20150116079 Mishra Apr 2015 A1
20150123818 Sellschopp May 2015 A1
20150127208 Jecker May 2015 A1
20150149265 Huntzicker May 2015 A1
20150151789 Lee Jun 2015 A1
20150153178 Koo Jun 2015 A1
20150161890 Huntzicker Jun 2015 A1
20150163649 Chen Jun 2015 A1
20150197278 Boos Jul 2015 A1
20150203111 Bonnet Jul 2015 A1
20150203156 Hafner Jul 2015 A1
20150210317 Hafner Jul 2015 A1
20150217693 Pliefke Aug 2015 A1
20150219464 Beaurepaire Aug 2015 A1
20150220791 Wu Aug 2015 A1
20150226146 Elwart Aug 2015 A1
20150274016 Kinoshita Oct 2015 A1
20150286340 Send Oct 2015 A1
20150329110 Stefan Nov 2015 A1
20150344028 Gieseke Dec 2015 A1
20150346727 Ramanujam Dec 2015 A1
20150360720 Li Dec 2015 A1
20150365401 Brown Dec 2015 A1
20150371541 Korman Dec 2015 A1
20150375741 Kiriya Dec 2015 A1
20150375742 Gebert Dec 2015 A1
20160012653 Soroka Jan 2016 A1
20160012726 Wang Jan 2016 A1
20160018821 Akita Jan 2016 A1
20160055749 Nicoll Feb 2016 A1
20160153778 Singh Feb 2016 A1
20160062354 Li Mar 2016 A1
20160068158 Elwart Mar 2016 A1
20160068187 Hata Mar 2016 A1
20160075369 Lavoie Mar 2016 A1
20160090055 Breed Mar 2016 A1
20160107689 Lee Apr 2016 A1
20160112846 Siswick Apr 2016 A1
20160114726 Nagata Apr 2016 A1
20160117926 Akavaram Apr 2016 A1
20160127664 Bruder May 2016 A1
20160139244 Holtman May 2016 A1
20160144857 Ohshima May 2016 A1
20160152263 Singh Jun 2016 A1
20160170494 Bonnet Jun 2016 A1
20160185389 Ishijima Jun 2016 A1
20160189435 Beaurepaire Jun 2016 A1
20160207528 Stefan Jul 2016 A1
20160224025 Petel Aug 2016 A1
20160229452 Lavoie Aug 2016 A1
20160236680 Lavoie Aug 2016 A1
20160249294 Lee Aug 2016 A1
20160257304 Lavoie Sep 2016 A1
20160272244 Imai Sep 2016 A1
20160282442 O'Mahony Sep 2016 A1
20160284217 Lee Sep 2016 A1
20160288657 Tokura Oct 2016 A1
20160300417 Hatton Oct 2016 A1
20160304087 Noh Oct 2016 A1
20160304088 Barth Oct 2016 A1
20160349362 Rohr Oct 2016 A1
20160321445 Turgeman Nov 2016 A1
20160321926 Mayer Nov 2016 A1
20160334797 Ross Nov 2016 A1
20160347280 Daman Dec 2016 A1
20160355125 Herbert Dec 2016 A1
20160357354 Chen Dec 2016 A1
20160358474 Uppal Dec 2016 A1
20160368489 Aich Dec 2016 A1
20160371607 Rosen Dec 2016 A1
20160371691 Kang Dec 2016 A1
20170001650 Park Jan 2017 A1
20170008563 Popken Jan 2017 A1
20170026198 Ochiai Jan 2017 A1
20170028985 Kiyokawa Feb 2017 A1
20170030722 Kojo Feb 2017 A1
20170032593 Patel Feb 2017 A1
20170072947 Lavoie Mar 2017 A1
20170073004 Shepard Mar 2017 A1
20170076603 Bostick Mar 2017 A1
20170097504 Takamatsu Apr 2017 A1
20170116790 Kusens Apr 2017 A1
20170123423 Sako May 2017 A1
20170129537 Kim May 2017 A1
20170129538 Stefan May 2017 A1
20170132482 Kim May 2017 A1
20170144654 Sham May 2017 A1
20170144656 Kim May 2017 A1
20170147995 Kalimi May 2017 A1
20170168479 Dang Jun 2017 A1
20170192428 Vogt Jul 2017 A1
20170200369 Miller Jul 2017 A1
20170203763 Yamada Jul 2017 A1
20170208438 Dickow Jul 2017 A1
20170297385 Kim Oct 2017 A1
20170297620 Lavoie Oct 2017 A1
20170301241 Urhahne Oct 2017 A1
20170308075 Whitaker Oct 2017 A1
20170336788 Iagnemma Nov 2017 A1
20170357317 Chaudhri Dec 2017 A1
20170371514 Cullin Dec 2017 A1
20180015878 McNew Jan 2018 A1
20180024559 Seo Jan 2018 A1
20180029591 Lavoie Feb 2018 A1
20180029641 Solar Feb 2018 A1
20180039264 Messner Feb 2018 A1
20180043884 Johnson Feb 2018 A1
20180056939 van Roermund Mar 2018 A1
20180056989 Donald Mar 2018 A1
20180082588 Hoffman, Jr. Mar 2018 A1
20180088330 Giannuzzi Mar 2018 A1
20180093663 Kim Apr 2018 A1
20180105165 Alarcon Apr 2018 A1
20180105167 Kim Apr 2018 A1
20180148094 Mukaiyama May 2018 A1
20180174460 Jung Jun 2018 A1
20180189971 Hildreth Jul 2018 A1
20180194344 Wang Jul 2018 A1
20180196963 Bandiwdekar Jul 2018 A1
20180224863 Fu Aug 2018 A1
20180236957 Min Aug 2018 A1
20180284802 Tsai Oct 2018 A1
20180286072 Tsai Oct 2018 A1
20180339654 Kim Nov 2018 A1
20180345851 Lavoie Dec 2018 A1
20180364731 Liu Dec 2018 A1
20190005445 Bahrainwala Jan 2019 A1
20190042003 Parazynski Feb 2019 A1
20190066503 Li Feb 2019 A1
20190103027 Wheeler Apr 2019 A1
20190137990 Golgiri May 2019 A1
20190152472 Aksyutina May 2019 A1
20190230471 Golgiri Jul 2019 A1
Foreign Referenced Citations (101)
Number Date Country
101929921 Dec 2010 CN
103818204 May 2014 CN
104183153 Dec 2014 CN
104485013 Apr 2015 CN
104691544 Jun 2015 CN
103049159 Jul 2015 CN
105513412 Apr 2016 CN
105588563 May 2016 CN
105599703 May 2016 CN
105774691 Jul 2016 CN
106027749 Oct 2016 CN
205719000 Nov 2016 CN
106598630 Apr 2017 CN
106782572 May 2017 CN
106945662 Jul 2017 CN
104290751 Jan 2018 CN
3844340 Jul 1990 DE
19817142 Oct 1999 DE
19821163 Nov 1999 DE
102005006966 Sep 2005 DE
102006058213 Jul 2008 DE
102009024083 Jul 2010 DE
102016224529 Mar 2011 DE
102016226008 Mar 2011 DE
102010034129 Feb 2012 DE
102009060169 Jun 2013 DE
102011080148 Jul 2013 DE
102012200725 Sep 2013 DE
102009051055 Oct 2013 DE
102011122421 Jun 2014 DE
102012008858 Jun 2014 DE
102013016342 Jan 2015 DE
102013019904 Feb 2015 DE
102012215218 Apr 2015 DE
102012222972 May 2015 DE
102013004214 May 2015 DE
102013019771 Dec 2015 DE
102013213064 Feb 2016 DE
102014007915 Feb 2016 DE
102014011802 Feb 2016 DE
102014009077 Apr 2016 DE
102014226458 Jun 2016 DE
102014011864 Dec 2016 DE
102014015655 May 2017 DE
102014111570 Jun 2017 DE
102016214433 Jun 2017 DE
102015209976 Jul 2017 DE
102015221224 Dec 2017 DE
102016011916 Feb 2018 DE
102016125282 Jun 2018 DE
102016211021 Jun 2018 DE
2653367 Jun 2000 EP
2768718 Jun 2011 EP
2289768 Oct 2013 EP
2620351 Dec 2015 EP
2295281 Mar 2016 EP
2135788 Jun 2016 EP
3021798 Dec 2012 FR
2534471 Oct 2000 GB
2344481 Dec 2012 GB
2497836 Sep 2014 GB
2481324 Mar 2015 GB
2517835 May 2016 GB
2491720 Jul 2016 GB
5586450 May 2004 JP
5918683 Oct 2004 JP
2000293797 Jul 2005 JP
2004142543 Apr 2009 JP
2016119032 Apr 2009 JP
2018052188 Jan 2010 JP
2004287884 Jul 2014 JP
2005193742 Jul 2014 JP
2009090850 Jun 2016 JP
2014134082 Jul 2016 JP
2014125196 Apr 2018 JP
20130106005 Jun 2006 KR
20160039460 May 2008 KR
20160051993 Jan 2010 KR
101641267 Sep 2013 KR
20090040024 Apr 2016 KR
20100006714 May 2016 KR
WO 2017112444 Dec 2010 WO
WO 2017118510 Jun 2011 WO
WO 2006064544 Nov 2011 WO
WO 2017125514 Jan 2013 WO
WO 2008055567 Apr 2013 WO
WO 2010006981 Aug 2013 WO
WO 2011141096 Jul 2014 WO
WO 2013056959 May 2015 WO
WO 2013123813 Dec 2015 WO
WO 2014103492 Mar 2016 WO
WO 2015068032 Aug 2016 WO
WO 2015193058 Sep 2016 WO
WO 2016046269 Apr 2017 WO
WO 2016128200 May 2017 WO
WO 2016134822 Jun 2017 WO
WO 2017062448 Jun 2017 WO
WO 2017073159 Jun 2017 WO
WO 2017096307 Jun 2017 WO
WO 2017096728 Jun 2017 WO
WO 2017097942 Jun 2017 WO
Non-Patent Literature Citations (23)
Entry
US 9,772,406 B2, 09/2017, Liu (withdrawn)
Alberto Broggi and Elena Cardarelli, Vehicle Detection for Autonomous Parking Using a Soft-Cascade ADA Boost Classifier, Jun. 8, 2014.
Al-Sherbaz, Ali et al., Hybridisation of GNSS with other wireless/sensors technologies on board smartphones to offer seamless outdoors-indoors positioning for LBS applications, Apr. 2016, 3 pages.
Automatically Into the Parking Space—https://www.mercedes-benz.com/en/mercedes-benz/next/automation/automatically-into-the-parking-space/; Oct. 27, 2014.
Bill Howard, Bosch's View of the Future Car: Truly Keyless Entry, Haptic Feedback, Smart Parking, Cybersecurity, Jan. 9, 2017, 8 Pages.
ChargeItSpot Locations, Find a Phone Charging Station Near You, retrieved at https://chargeitspot.com/locations/ on Nov. 28, 2017.
Core System Requirements Specification (SyRS), Jun. 30, 2011, Research and Innovative Technology Administration.
Daimler AG, Remote Parking Pilot, Mar. 2016 (3 Pages).
Jingbin Liu, IParking: An Intelligent Indoor Location-Based Smartphone Parking Service, Oct. 31, 2012, 15 pages.
Land Rover develops a smartphone remote control for its SUVs, James Vincent, Jun. 18, 2015.
Land Rover, Land Rover Remote Control Via Iphone RC Range Rover Sport Showcase—Autogefühl, Retrieved from https://www.youtube.com/watch?v=4ZaaYNaEFio (at 43 seconds and 1 minute 42 seconds), Sep. 16, 2015.
Perpendicular Parking—https://prezi.com/toqmfyxriksl/perpendicular-parking/.
Safecharge, Secure Cell Phone Charging Stations & Lockers, retrieved at https://www.thesafecharge.com on Nov. 28, 2017.
Search Report dated Jan. 19, 2018 for GB Patent Application No. 1711988.4 (3 pages).
Search Report dated Jul. 11, 2017 for GB Patent Application No. 1700447.4 (3 Pages).
Search Report dated May 21, 2018 for Great Britain Patent Application No. GB 1800277.4 (5 Pages).
Search Report dated Nov. 22, 2018 for GB Patent Application No. GB 1809829.3 (6 pages).
Search Report dated Nov. 27, 2018 for GB Patent Application No. GB 1809112.4 (3 pages).
Search Report dated Nov. 28, 2017, for GB Patent Application No. GB 1710916.6 (4 Pages).
Search Report dated Nov. 28, 2018 for GB Patent Application No. GB 1809842.6 (5 pages).
Search Report dated Oct. 10, 2018 for GB Patent Application No. 1806499.8 (4 pages).
Tesla Model S Owner's Manual V2018.44. Oct. 29, 2018.
Vehicle's Orientation Measurement Method by Single-Camera Image Using Known-Shaped Planar Object, Nozomu Araki, Takao Sato, Yasuo Konishi and Hiroyuki Ishigaki, 2010.
Related Publications (1)
Number Date Country
20200294325 A1 Sep 2020 US