Patent Application
20240132102
This application claims the benefit under 35 USC § 119(a) of Korean Patent Application Nos. 10-2022-0135815, filed on Oct. 20, 2022, and 10-2023-0117376, filed on Sep. 5, 2023, in the Korean Intellectual Property Office, the entire disclosures of which are incorporated herein by reference for all purposes.
Disclosed embodiments relate to a technology for providing autonomous driving safety map service.
Recently, as the number of vehicles equipped with an autonomous driving function has increased, the number of cases in which various kinds of autonomous vehicles drive on roads together with regular manual driving vehicles (vehicles driven directly by drivers) is increasing.
In order to implement the autonomous driving function, various kinds of sensors capable of recognizing the current state of a vehicle and the surrounding environment thereof, as well as hardware and software capable of controlling the driving of the vehicle based on the current state and the surrounding environment are essential. Depending on the vehicle model, vehicle manufacturer, model year, etc., the characteristics and performance of the autonomous driving function that may be implemented in a vehicle vary. In addition, at this point in time, most autonomous vehicles are not fully autonomous, but offer a level of autonomous driving that requires driver intervention in certain situations. For example, if it is difficult to recognize lanes properly due to poor road conditions, if an accident has occurred or construction is underway on a road, or if a sensor's visibility is limited due to bad weather, it is not possible for the autonomous driving function to operate properly. In this case, it is impossible for an autonomous vehicle to perform autonomous driving, and thus, the autonomous vehicle requests a driver to directly operate the vehicle, or outputs an error message.
If an anomaly of the autonomous driving function is caused by the surrounding environment rather than a problem of a vehicle itself (e.g., when lanes are erased due to the aging of a road, or when it is difficult for an autonomous driving module to properly recognize the road situation due to the characteristics of the road), the above case is highly likely to occur simultaneously not to a specific vehicle, but to most autonomous vehicles passing on a corresponding road. On the contrary, if an anomaly of the autonomous driving function is caused by a problem with a specific sensor installed in a vehicle, the problem is highly likely to occur only in vehicles of the vehicle model equipped with the corresponding sensor. Therefore, servicing, in the form of a map, information such as the environment, location, or vehicle model in which anomalies of the autonomous driving function frequently occur may be of great help in safe driving of autonomous vehicles.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
The disclosed embodiments are to provide a technology for calculating the degree of risk of autonomous driving on a road and servicing the same in the form of a map.
In one general aspect, there is provided a system for providing autonomous driving safety map service, wherein the system includes, an autonomous vehicle configured to transmit anomaly information of an autonomous driving system acquired during the driving of the vehicle, and a server for providing safety map service, configured to receive the anomaly information of the autonomous driving system from one or more of the autonomous vehicles driving within a specific area, generate autonomous driving danger zone information based on one or more of the location of occurrence of the anomaly information or identification information of the autonomous vehicle, and provide the generated autonomous driving danger zone information in conjunction with a map of the specific area.
In an embodiment, the anomaly information of the autonomous driving system may be generated based on one or more of whether a takeover request message has been generated in the autonomous vehicle, whether driver intervention has occurred in the driving of the autonomous vehicle, or whether an error message of the autonomous driving system has been generated.
In an embodiment, the identification information of the autonomous vehicle may include a vehicle identification number (VIN) of the autonomous vehicle.
In an embodiment, when the autonomous vehicle is located within a predetermined information transmission restricted area, the autonomous vehicle may stop transmitting the anomaly information.
In an embodiment, when the autonomous vehicle is located within a predetermined range from the origin or destination of a route set by a user of the autonomous vehicle, the autonomous vehicle may stop transmitting the anomaly information.
In an embodiment, the autonomous vehicle may transmit the anomaly information based on a predetermined information transmission rate.
In an embodiment, the autonomous vehicle may encrypt the anomaly information of the autonomous driving system by using an encryption-based privacy assurance technique and then configured to transmit the encrypted anomaly information to the server for providing autonomous driving safety map service.
In an embodiment, the encryption-based privacy assurance technique may include one or more of homomorphic encryption or functional encryption.
In an embodiment, the server may be configured to classify the encrypted anomaly information based on one or more of the location of occurrence of the anomaly information or identification information of the autonomous vehicle.
In an embodiment, only when the number of the encrypted anomaly information classified according to a specific location or specific identification information is greater than a preset threshold value, the server for autonomous driving safety map service may decrypt the corresponding anomaly information to generate the autonomous driving danger zone information.
In an embodiment, the server may generate the autonomous driving danger zone information by further reflecting environmental information related to the location of occurrence of the anomaly information in the anomaly information.
In an embodiment, the server may receive destination information from the user of the autonomous vehicle to generate a route plan to the destination based on the autonomous driving danger zone information.
In another general aspect, there is provided a server for providing autonomous driving safety map service, wherein the server includes a vehicle information collecting module configured to receive anomaly information of an autonomous driving system from an autonomous vehicle driving within a specific area, a danger zone identifying module configured to generate autonomous driving danger zone information based on one or more of the location of occurrence of the anomaly information or identification information of the autonomous vehicle, and an information providing module configured to provide the generated autonomous driving danger zone information in conjunction with a map of the specific area.
In an embodiment, the anomaly information of the autonomous driving system may be generated based on one or more of whether a takeover request message has been generated in the autonomous vehicle, whether driver intervention has occurred in the driving of the autonomous vehicle, or whether an error message of the autonomous driving system has been generated.
In an embodiment, the identification information of the autonomous vehicle may include a vehicle identification number (VIN) of the autonomous vehicle.
In an embodiment, the autonomous vehicle may encrypt the anomaly information of the autonomous driving system by using an encryption-based privacy assurance technique and then configured to transmit the encrypted anomaly information to the server for providing autonomous driving safety map service.
In an embodiment, the encryption-based privacy assurance technique may include one or more of homomorphic encryption or functional encryption.
In an embodiment, the danger zone identifying module may classify the encrypted anomaly information based on one or more of the location of occurrence of the anomaly information or identification information of the autonomous vehicle.
In an embodiment, only when the number of the encrypted anomaly information classified according to a specific location or specific identification information is greater than a preset threshold value, the danger zone identifying module may decrypt the corresponding anomaly information to generate the autonomous driving danger zone information.
In still another general aspect, there is provided, as a method performed on a computer, a method for providing autonomous driving safety map service, wherein the method includes receiving anomaly information of an autonomous driving system from an autonomous vehicle driving within a specific area, generating autonomous driving danger zone information based on one or more of the location of occurrence of the anomaly information or identification information of the autonomous vehicle, and providing the generated autonomous driving danger zone information in conjunction with a map of the specific area.
Hereinafter, specific embodiments of the present invention will be described with reference to the accompanying drawings. The following detailed description is provided to assist in a comprehensive understanding of the methods, devices and/or systems described herein. However, the detailed description is only illustrative, and the present invention is not limited thereto.
In describing embodiments of the present invention, when a specific description of known technology associated with the present invention is deemed to make the gist of the present invention unnecessarily vague, the detailed description thereof will be omitted. The terms used below are defined in consideration of functions in the present invention, but may vary in accordance with the customary practice or the intention of a user or an operator. Therefore, the terms should be defined based on whole content throughout the present specification. The terms used herein are only for describing the embodiments of the present invention, and should not be construed as limited. A singular expression includes a plural meaning unless clearly used otherwise. In the present description, expressions such as “include” or “have” are for referring to certain characteristics, numbers, steps, operations, components, and some or combinations thereof, and should not be construed as excluding the presence or possibility of one or more other characteristics, numbers, steps, operations, components, and some or combinations thereof besides those described.
In the following description, the terms “transmission,” “communication,” and “reception” of a signal or information and other similar terms may include a meaning in which the signal or information is directly transmitted from one element to another element and transmitted from one element to another element through an intervening element. In particular, “transmission” or “sending” of the signal or information to one element may indicate a final destination of the signal or information and may not imply a direct destination. The same applies to “reception” of the signal or information. In addition, in the present specification, a meaning in which two or more pieces of data or information are “related” indicates that when any one piece of data (or information) is obtained, at least a portion of other data (or information) may be obtained based thereon.
Furthermore, the terms “first,” “second,” and the like may be used for describing various elements, but the elements should not be construed as being limited by the terms. These terms may be used for distinguishing one element from another element. For example, a first element could be termed a second element and vice versa without departing from the scope of the present invention.
There are various criteria for classifying the level of an autonomous driving technology, but in general, six levels of subdivided contents from Level 0 to Level 5 defined by the Society of Automotive Engineers (SAE International) are internationally accepted. For example, Level 1 is a driver assistance level, in which only specific functions of a level that assists a driver, such as speed and lane keeping systems, etc., are automated, and the driver should always control the speed and direction. Level 2 is a partial automation level that assists a driver in maintaining lane spacing, steering, accelerating, and decelerating under a constant situation such as a highway, and the driver always keeps an eye on the surrounding situation and actively intervenes in driving when necessary. Level 3 is a conditional automation level that enables autonomous driving under limited conditions, in which active intervention from a driver is not required during driving, but the driver intervenes when limit conditions for autonomous driving such as unexpected situations are met while keeping an eye on the surrounding situation. Level 4 is a high automation level that enables all safety controls under specific road conditions, in which autonomous driving is possible in all situations under specified road conditions, but which requires a driving control device since a driver intervenes in specific situations such as bad weather. Lastly, Level 5 is a full automation level, which does not require a driver and enables driving only with a passenger. At this level, when the passenger simply inputs a destination, a system takes charge of all the driving, which does not require a controller for steering or accelerating/decelerating.
The autonomous vehicle 102 is a vehicle with an autonomous driving function. The autonomous vehicle 102 includes an autonomous driving system. In the disclosed embodiments, the autonomous vehicle 102 may include, without limitation, a vehicle in which any one of the above-described levels of autonomous driving functions are implemented.
The autonomous vehicle 102 transmits autonomous driving-related information collected during the driving of the vehicle to the server 104. At this time, the autonomous driving-related information includes anomaly information of the autonomous driving system, which is acquired while the autonomous vehicle 102 is driving in an autonomous driving mode.
In an embodiment, the anomaly information of the autonomous driving system may be generated based on one or more of whether a takeover request message has been generated in the autonomous vehicle 102, whether driver intervention has occurred in the driving of the autonomous vehicle 102, or an error message of the autonomous driving system has been generated.
For example, an autonomous vehicle 102 of Level 5 or lower may require driver intervention during autonomous driving depending on the situation. In this case, the autonomous vehicle 102 may output a takeover request message to guide a driver to manual driving. The takeover request message may be delivered to the driver in a visual (e.g., dashboard messages or warning light) or audible (e.g., warning sound) form. In addition, in some cases, even if there is no take over request from the autonomous vehicle 102, the driver may intervene in driving of the vehicle by directly operating a steering system or a braking system if it is determined that the driver intervention is required. Depending on the embodiment, the driver intervention may be divided into weak intervention (e.g., slight adjustment to the steering device or slight additional braking in a situation in which braking by the autonomous driving system is performed, etc.) or strong intervention (e.g., the steering device is operated contrary to the autonomous driving system or strong braking is performed in a section in which the autonomous driving system determines that braking is not required, etc.). In this case, the autonomous vehicle 102 may determine that an anomaly has occurred in the autonomous driving system only when the strong intervention has occurred.
In addition, when it is determined that an error has occurred in the autonomous driving system for various reasons, the autonomous vehicle 102 may output an error message corresponding thereto. For example, when the autonomous vehicle 102 detects a malfunction or error in the autonomous driving system caused by poor road conditions (e.g., erased lanes, lanes that cannot be processed into images due to road crack repair work) or bad weather, the autonomous vehicle 102 may notify the server 104 of the malfunction or error. In this case, depending on the vehicle manufacturer or vehicle model, detailed information on the malfunction may vary.
The autonomous vehicle 102 may generate anomaly information of the autonomous driving system and transmit the same to the server 104 when various abnormal situations related to autonomous driving as described above occur.
In an embodiment, the anomaly information of the autonomous driving system may include information on a location at which an anomaly has occurred, identification information of the autonomous vehicle 102, road driving videos, information on a module in which an error has occurred, type and content of the anomaly, etc.
In an embodiment the identification information of the autonomous vehicle 102 may include a vehicle identification number (VIN) of the autonomous vehicle 102. When the vehicle identification number is used, various information such as the vehicle model, model year, manufacturer, etc., of the autonomous vehicle 102 that has transmitted the anomaly information may be obtained, so that various types of autonomous driving danger zone information may be generated by using the same.
In an embodiment, the location information may be a global navigation satellite system (GNSS)-based information, such as a GPS, a GLONASS, a Galileo, a Beidou-2, or the like. In other embodiments, the location information may be precise location information. For example, the location information may be location information based on a satellite based augmentation system (SBAS). The SBAS is a system that provides correction information and integrity information on a signal error provided by the GNSS through a geostationary satellite. When precise location information such as the SBAS is used, not only the type of a road on which the autonomous vehicle 102 is driving, but also information on a lane on which the autonomous vehicle 102 is currently driving may be identified. In this case, the server 104 may acquire additional information on whether an anomaly of the autonomous driving system occurs on the entire road at the corresponding location or only on a specific lane.
In some embodiments, when a recognized location of the autonomous vehicle 102 is within a predetermined information transmission restricted area, the autonomous vehicle 102 may stop transmitting the anomaly information of the autonomous driving system. In some other embodiments, when the location of the autonomous vehicle 102 is within a predetermined range from the origin or destination of a route set by a user, the autonomous vehicle 102 may stop transmitting the anomaly information. For example, when a driver moves from home to a workplace using the autonomous vehicle 102, the autonomous vehicle 102 may not transmit the anomaly information within a predetermined range from the home (origin) or the workplace (destination).
In many cases, a vehicle travels between specific places, such as residences, workplaces, schools, or the like. Therefore, a third party with malicious intent may use the transmission location of anomaly information transmitted from the autonomous vehicle 102 to infer personal information such as the residence of an owner of the autonomous vehicle 102. In the same way, there is a possibility of finding out personal information related to the autonomous vehicle 102 through information such as an origin, destination, or the like set in a navigation system in the autonomous vehicle 102. In order to solve the above problem, in the disclosed embodiments, a specific area such as a residence or workplace, or an area within a predetermined range from the origin or destination of a route set by a user is set as an information transmission restricted area, and the transmission of anomaly information of an autonomous driving system is restricted in the corresponding area to prevent the possibility of leakage of personal information of a vehicle owner, etc.
Meanwhile, in some embodiments, the autonomous vehicle 102 may be configured to transmit the anomaly information based on a predetermined information transmission rate. For example, when the information transmission rate is set to 40%, the autonomous vehicle 102 may transmit only 40% of the total anomaly information generated in the autonomous driving system to the server 104. When the information transmission rate is lowered, the amount of collection of autonomous driving anomaly information may be reduced from the perspective of the server 104, but the risk of leakage of personal information may also be reduced. In order to compensate for the above, the server 104 may grant incentives, such as points or a reduction in service usage fees, to a user who has set a high information transmission rate.
Meanwhile, in some embodiments, the autonomous vehicle 102 may encrypt the anomaly information of the autonomous driving system by using an encryption-based privacy assurance technique and then transmit the encrypted anomaly information to the server 104. At this time, the encryption-based privacy assurance technique refers to a technique that allows calculations in an encrypted state. For example, the encryption-based privacy assurance technique may include one or more of homomorphic encryption or functional encryption. When the above technique is used, the server 104 may add up the number of anomalies for each specific location or vehicle of a specific manufacturer without decrypting individual anomaly information received, so that it is possible to minimize leakage of personal information that may occur during a process of collecting anomaly information.
The server 104 receives anomaly information of an autonomous driving system from one or more of the autonomous vehicles 102 driving within a specific area. The illustrated embodiment shows an example in which the server 104 receives anomaly information of an autonomous driving system from n autonomous vehicles 102 (n is a natural number), which is are respectively referred to as 102-1, 102-2, . . . , and 102-n.
The server 104 generates autonomous driving danger zone information based on one or more of the location of occurrence of the received anomaly information of the autonomous driving system or identification information of the autonomous vehicle 102 that has transmitted the corresponding anomaly information, and provides the generated autonomous driving danger zone information in conjunction with a map of the specific area.
The vehicle information collecting module 202 receives anomaly information of an autonomous driving system from one or more of the autonomous vehicles 102 driving on a road within a preset area.
The danger zone identifying module 204 generates autonomous driving danger zone information based on one or more of the location of occurrence of the anomaly information or identification information of the autonomous vehicle 102.
The information providing module 206 provides the generated autonomous driving danger zone information in conjunction with a map of the specific area. For example, the information providing module 206 may provide information on a section in which a malfunction of the autonomous driving module frequently occurs. According to an embodiment, the information providing module 206 may provide information on a section in which all autonomous driving modules malfunction or a section in which an autonomous driving module of a particular manufacturer malfunctions.
Meanwhile, in some embodiments, the server 104 may further include a route generation module 208. The route generation module 208 may receive destination information from a user (e.g., driver) of the autonomous vehicle 102, and generate a route plan to the destination based on the autonomous driving danger zone information. For example, the route generation module 208 may generate the route plan such that the proportion of the autonomous driving danger zone in possible routes to the destination is minimized.
As described above, in some embodiments, the autonomous vehicle 102 may encrypt the anomaly information of the autonomous driving system by using an encryption-based privacy assurance technique and then transmit the encrypted anomaly information to the server 104. In this case, the danger zone identifying module 204 may classify the encrypted anomaly information based on one or more of the location of occurrence of the anomaly information or identification information of the autonomous vehicle without decrypting the anomaly information. Here, the identification information of the autonomous vehicle may include, without limitation, various information that may be used to classify the autonomous vehicle, such as the vehicle model, model year, and manufacturer of the autonomous vehicle, type of the autonomous vehicle module, etc. For example, when homomorphic encryption or functional encryption is used, the server 104 may add up the number of pieces of information on autonomous driving anomalies occurring in a specific location (e.g., a vicinity of an intersection A) or a specific vehicle type without decrypting individual anomaly information. Only when the number of the encrypted anomaly information classified according to a specific location or specific identification information is greater than a preset threshold value, the danger zone identifying module 204 may decrypt the corresponding anomaly information to generate the autonomous driving danger zone information. When only the anomaly information greater than the threshold value is decrypted as described above, all anomaly information is not required to be decrypted, so that the computing resource of the server 104 may be efficiently used and the risk of personal information leakage due to decryption may be reduced.
In an embodiment, the danger zone identifying module 204 may generate the autonomous driving danger zone information by further reflecting environmental information related to the location of occurrence of the anomaly information in the anomaly information. For example, if weather conditions such as heavy rain or haze in the corresponding area at the time an anomaly has occurred to the autonomous driving function, or time information such as day/night is further reflected, the cause of the autonomous driving anomaly may be more closely analyzed.
In Step 602, the vehicle information collecting module 202 receives anomaly information of an autonomous driving system from the autonomous vehicle 102 driving within a specific area.
In Step 604, the danger zone identifying module 204 generates autonomous driving danger zone information based on one or more of the location of occurrence of the anomaly information or identification information of the autonomous vehicle 102.
In Step 606, the information providing module 206 provides the generated autonomous driving danger zone information in conjunction with a map of the specific area.
In Step 702, the danger zone identifying module 204 collects encrypted anomaly information of the autonomous driving system by using an encryption-based privacy assurance technique.
In Step 704, the danger zone identifying module 204 classifies the encrypted anomaly information based on one or more of the location of occurrence of the anomaly information or identification information of the autonomous vehicle without decrypting the anomaly information.
In Step 706, the danger zone identifying module 204 determines whether the number of pieces of anomaly information corresponding to a specific location or specific identification information is greater than a threshold value.
When it is determined that the number is greater than the threshold value, in Step 708, the danger zone identifying module 204 decrypts anomaly information greater than the threshold value to generate autonomous driving danger zone information.
On the contrary, when it is determined that the number is not greater than the threshold value, the danger zone identifying module 204 returns to Step 702 and further collects anomaly information of the autonomous driving system.
The illustrated computing environment 10 includes a computing device 12. In some embodiments, the computing device 12 may be any one of an autonomous driving system mounted on the autonomous vehicle 102 or the server 104 for providing autonomous driving safety map service. The computing device 12 includes at least one processor 14, a computer-readable storage medium 16, and a communication bus 18. The processor 14 may enable the computing device 12 to operate according to the exemplary embodiment mentioned above. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16. The one or more programs may include one or more computer-executable commands, and when executed by the processor 14, the computer-executable command may be configured to enable the computing device 12 to perform operations according to the exemplary embodiment.
The computer-readable storage medium 16 is configured to store computer-executable commands or program codes, program data, and/or other suitable types of information. A program 20 stored in the computer-readable storage medium 16 includes a set of commands executable by the processor 14. In an embodiment, the computer-readable storage medium 16 may be a memory (a volatile memory such as a random access memory, a non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, other types of storage media accessed by the computing device 12 and capable of storing desired information, or any suitable combination thereof.
The communication bus 18 includes the processor 14 and the computer-readable storage medium 16 to interconnect various other components of the computing device 12.
The computing device 12 may also include one or more input/output interfaces 22 which provide an interface for one or more input/output devices 24, and one or more network communication interfaces 26. The input/output interface 22 and the network communication interface 26 are connected to the communication bus 18. The input/output device 24 may be connected to other components of the computing device 12 through the input/output interface 22. The exemplary input/output device 24 may include a pointing device (such as a mouse or track pad), a keyboard, a touch input device (such as a touch pad or touch screen), a voice or sound input device, an input device such as various types of sensor devices and/or photographing devices, and/or an output device such as a display device, a printer, a speaker, and/or a network card. The exemplary input/output device 24 may be included inside the computing device 12 as one component constituting the computing device 12, or may be connected to the computing device 12 as a separate device distinct from the computing device 12.
According to the disclosed embodiments, if a malfunction of an autonomous driving system is detected while an autonomous vehicle is driving on a road, one or more of corresponding location information and a road driving video are transmitted to a server, and the server analyzes a factor that interfere with autonomous driving based on the collected information and services the same in the form of a map to allow an autonomous driving vehicle and a driver of the corresponding vehicle to visually and easily determine the degree of risk of autonomous driving on a road on which the vehicle and the driver are driving or scheduled to drive.
In addition, according to the disclosed embodiments, the stability of autonomous driving may be increased by setting a route to the destination of an autonomous vehicle based on the degree of risk of autonomous driving.
Although representative embodiments of the present invention have been described in detail above, those skilled in the art to which the present invention belongs will understand that various modifications may be made to the above-described embodiments without departing from the scope of the present invention. Therefore, the scope of rights of the present invention should not be limited to the described embodiments, but should be defined not only by claims set forth below but also by equivalents of the claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2022-0135815 | Oct 2022 | KR | national |
| 10-2023-0117376 | Sep 2023 | KR | national |
