Patent Application
20240126283
The present technology relates to robotic vehicles and, specifically, to a method for generating a path for navigation of an autonomous system in populated environments.
Various techniques may be used for providing navigation paths for robotic vehicles. These systems may use Global Positioning System (GPS) or SLAM technologies or the like, computer vision algorithms, and/or artificial intelligence, among other technologies. The deployment of robotic vehicles as assistive systems in populated environments has recently gained more interest because of increased maturity and capabilities of those technologies. Guide robots in airports, server robots, cleaning robots, and assisting robots for the elderly or sick persons are only a few examples of assistive systems. However, such autonomous systems may need to respect social and physical norms by adapting their navigation paths within the environment in order to provide a harmonious coexistence with the people in its surroundings.
Even though the recent developments identified above may provide benefits, improvements are still desirable.
The subject matter discussed in the background section should not be assumed to be prior art merely as a result of its mention in the background section. Similarly, a problem mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the background section merely represents different approaches.
Embodiments of the present technology have been developed based on developers' appreciation of shortcomings associated with the prior art.
In particular, such shortcomings may comprise (1) power-draining and time-consuming algorithms; (2) invariability of interaction with a person and a group of persons; and/or (3) definition of a destination depending whether the autonomous system has to interact with a person or not.
In a first broad aspect of the present technology, there is provided a method for generating a navigation path for an autonomous system, wherein the autonomous system comprises a processor. The method comprises receiving, by the processor, first data comprising characteristics of entities, the entities defining an environment in which the autonomous system is configured to operate, receiving, by the processor, first instructions causing the autonomous system to identify a destination in the environment, and generating a global navigation path comprising waypoints to be followed by the autonomous system to reach the destination from a current position of the autonomous system. The waypoints are generated based on the characteristics of the entities and defining segmental paths. The method comprises executing, by the processor, second instructions causing the autonomous system to navigate along the navigation path. The method comprises upon navigation of the autonomous system from a first waypoint to a second waypoint, accessing second data comprising updates of the characteristics of the entities located in a vicinity of a corresponding segmental path. The method comprises generating, by the processor, a local sub-path between the first waypoint and the second waypoint based on second data and executing, by the processor, third instructions causing the autonomous system to navigate along the sub-path to reach the second waypoint.
In some embodiments of the method, the first data and the second data are provided by two distinct sources of data.
In some embodiments of the method, entities are partitioned into human entities and non-human entities, characteristics of one of the human entities being selected in a group of characteristics comprising: a formation of the human entity, a number of persons comprised in the human entity, an orientation of the persons of the human entity, a location of the human entity, a level of business of the human entity, or a combination thereof.
In some embodiments of the method, upon receiving first instructions causing the autonomous system to identify a destination in the environment, the method further comprises determining whether the autonomous system has to interact with a human entity based on the first instructions. The method comprises, if determination is made based on the first instructions that the autonomous system has to interact with a human entity, accessing characteristics of the human entity and determining a destination in a vicinity of the human entity based on the characteristics of the human entity.
In some embodiments of the method, determining a destination in a vicinity of the human entity is further based on social and physical norms associated with the human entity.
In some embodiments of the method, locations of the waypoints are determined based on social and physical norms associated with the entities, the social and physical norms defining location costs, each location of one waypoint being associated with a location cost, each segmental path being associated with a location cost, and generating the navigation path comprises minimizing a global cost, the global cost comprising a combination of the location cost of each of the waypoints and each segmental path.
In some embodiments of the method, generating a sub-path between the first waypoint and the second waypoint comprises generating one or more sub-waypoints to be followed by the autonomous system to reach the consecutive waypoint, the sub-waypoints being generated based on social and physical norms associated with the entities, the social and physical norms defining location costs, each location of one sub-waypoint being associated with a location cost, and generating the sub-path comprises minimizing a sub-cost, the sub-cost comprising a combination of the location cost of each of the one or more sub-waypoints.
In some embodiments of the method, a first set of social and physical norms are associated with human entities and a second set of social and physical norms are associated with non-human entities.
In some embodiments of the method, the method comprises, concurrently to executing, by the processor, second instructions causing the autonomous system to navigate along the navigation path, updating the characteristics of entities in the environment is received by the processor and adjusting the locations of the waypoints based on the update of the characteristics of entities.
In some embodiments of the method, the autonomous system comprises a perception unit configured to provide the characteristics of entities in the vicinity of the autonomous system to the processor and the second data is provided by the perception unit.
In a second broad aspect of the present technology, there is provided an autonomous system configured for navigating in an environment. The autonomous system comprises a processor comprising at least one processor and a memory storing a plurality of executable instructions which, when executed by the at least one processor, cause the processor to receive first data comprising characteristics of entities in the environment, the entities defining an environment in which the autonomous system is configured to operate. The system is configured to receive first instructions causing the autonomous system to identify a destination in the environment and generate a navigation path comprising waypoints to be followed by the autonomous system to reach the destination from a current position of the autonomous system. The waypoints are generating based one the characteristics of the entities and defining segmental paths between two consecutive waypoints. The processor is configured to cause the autonomous system to navigate along the navigation path. The processor is configured to, upon navigation of the autonomous system from a first waypoint to a second waypoint, access second data comprising updates of the characteristics of the entities located in a vicinity of a corresponding segmental path, generate a sub-path between the first waypoint and the second waypoint based on second data and follow the sub-path to reach the second waypoint.
In some embodiments of the system, the system comprises a network device communicably connected to a network and configured for receiving the characteristics of the entities and the updates of the characteristics of the entities and a perception unit configured to provide the characteristics and the updates of the characteristics of the entities in the vicinity of the autonomous system.
In some embodiments of the system, locations of the waypoints are determined based on social and physical norms associated with the entities, the social and physical norms defining location costs, each location of one waypoint being associated with a location cost, each segmental path being associated with a location cost, and generating the navigation path comprises minimizing a global cost, the global cost comprising a combination of the location cost of each of the waypoints and each segmental path.
In some embodiments of the system, upon generating a sub-path between the first waypoint and the consecutive waypoint, the processor is further configured to generate one or more sub-waypoints to be followed by the autonomous system to reach the consecutive waypoint, the sub-waypoints being generated based on social and physical norms associated with the entities, the social and physical norms defining location costs, each location of one sub-waypoint being associated with a location cost, and generating the sub-path comprises minimizing a sub-cost, the sub-cost comprising a combination of the location cost of each of the one or more sub-waypoints.
In some embodiments of the system, entities are partitioned into human entities and non-human entities, characteristics of one of the human entities being selected in a group of characteristics comprising: a formation of the human entity, a number of persons comprised in the human entity, an orientation of the persons of the human entity, a location of the human entity, a level of business of the human entity, or a combination thereof.
In some embodiments of the system, upon receiving first instructions causing the autonomous system to identify a destination in the environment, the processor is further configured to, if determination is made based on the first instructions that the autonomous system has to interact with a human entity, access characteristics of the human entity and determine a destination in a vicinity of the person based on the characteristics of the human entity and the social and physical norms.
In some embodiments of the system, entities are partitioned into human entities and non-human entities, and a first set of social and physical norms are associated with human entities and a second set of social and physical norms are associated with non-human entities.
In a third broad aspect of the present technology, there is provided a method for generating a navigation path for an autonomous system, wherein the autonomous system comprises one or more sensing devices configured to observe an environment in which the autonomous system is configured to navigate, and a processor configured to execute the method. The method comprises receiving, by the processor, first data comprising characteristics of entities defining the environment, receiving, by the processor, first instructions causing the autonomous system to identify a destination in the environment, and generating a navigation path comprising waypoints to be followed by the autonomous system to reach the destination from a current position of the autonomous system, the waypoints defining segmental paths. Each waypoint is associated with a location cost indicative of a safeness of the autonomous system and entities in its surroundings if the autonomous system is located at said waypoint, the location cost being determined based on the first data, the generating the navigation path comprising computing a global cost, the global cost being a combination of the location costs of the waypoints. The method further comprises executing, by the processor, second instructions causing the autonomous system to navigate along the navigation path and, during navigation of the autonomous system from a first one of the waypoints to a second one of the waypoints along a corresponding segmental path, accessing, by the processor, second data generated by the one or more sensing devices during navigation of the autonomous system comprising updates of the characteristics of the entities located in a vicinity of a corresponding segmental path, generating, by the processor, a sub-path between a current position of the autonomous system and the second one of the waypoints based on the second data and the corresponding segmental path and executing, by the processor, third instructions causing the autonomous system to navigate along the sub-path to reach the second waypoint.
In some embodiments of the method, each segmental path between a given waypoint and a consecutive waypoint is associated with a location cost indicative of a safeness of the autonomous system and entities in its surroundings when the autonomous system is navigating from the given waypoint to the consecutive waypoint along the segmental path, the location cost of the segmental paths being determined based on the first data, and the global cost further rely upon the location costs of the waypoints and the location costs of segmental paths.
In some embodiments of the method, a location of the second one of the waypoints is adjusted during navigation of the autonomous system if determination is made, based on the second data, that the autonomous system being located at the second one of the waypoints corresponds to a lack of safety for surrounding entities or for the autonomous system.
In some embodiments of the method, computing the global cost comprises minimizing the global cost.
In a fourth aspect, various implementations of the present technology provide a system for generating an action strategy to be executed by an autonomous system, the system comprising a processor and a memory configured to store instructions which, upon being executed by the processor, cause the system to perform the methods.
In a fifth aspect, various implementations of the present technology provide a computer-readable support comprising instructions which, upon execution of the instructions, cause a computer to carry out the steps of the methods.
In the context of the present specification, unless expressly provided otherwise, a computer system may refer, but is not limited to, an “electronic device”, an “operation system”, a “system”, a “computer-based system”, a “controller unit”, a “monitoring device”, a “control device” and/or any combination thereof appropriate to the relevant task at hand.
In the context of the present specification, unless expressly provided otherwise, the expression “computer-readable medium” and “memory” are intended to include media of any nature and kind whatsoever, non-limiting examples of which include RAM, ROM, disks (CD-ROMs, DVDs, floppy disks, hard disk drives, etc.), USB keys, flash memory cards, solid state-drives, and tape drives. Still in the context of the present specification, “a” computer-readable medium and “the” computer-readable medium should not be construed as being the same computer-readable medium. To the contrary, and whenever appropriate, “a” computer-readable medium and “the” computer-readable medium may also be construed as a first computer-readable medium and a second computer-readable medium.
In the context of the present specification, unless expressly provided otherwise, the words “first”, “second”, “third”, etc. have been used as adjectives only for the purpose of allowing for distinction between the nouns that they modify from one another, and not for the purpose of describing any particular relationship between those nouns.
In the context of the present specification, the expression “information” includes information of any nature or kind whatsoever capable of being stored in a database. Thus, information includes, but is not limited to audio-visual works (images, movies, sound records, presentations etc.), data (location data, numerical data, etc.), text (opinions, comments, questions, messages, etc.), documents, spreadsheets, etc.
Implementations of the present technology each have at least one of the above-mentioned object and/or aspects, but do not necessarily have all of them. It should be understood that some aspects of the present technology that have resulted from attempting to attain the above-mentioned object may not satisfy this object and/or may satisfy other objects not specifically recited herein.
Additional and/or alternative features, aspects and advantages of implementations of the present technology will become apparent from the following description, the accompanying drawings and the appended claims.
For a better understanding of the present technology, as well as other aspects and further features thereof, reference is made to the following description which is to be used in conjunction with the accompanying drawings, where:
The examples and conditional language recited herein are principally intended to aid the reader in understanding the principles of the present technology and not to limit its scope to such specifically recited examples and conditions. It will be appreciated that those skilled in the art may devise various arrangements which, although not explicitly described or shown herein, nonetheless embody the principles of the present technology and are included within its spirit and scope.
Furthermore, as an aid to understanding, the following description may describe relatively simplified implementations of the present technology. As persons skilled in the art would understand, various implementations of the present technology may be of a greater complexity.
In some cases, what are believed to be helpful examples of modifications to the present technology may also be set forth. This is done merely as an aid to understanding, and, again, not to define the scope or set forth the bounds of the present technology. These modifications are not an exhaustive list, and a person skilled in the art may make other modifications while nonetheless remaining within the scope of the present technology. Further, where no examples of modifications have been set forth, it should not be interpreted that no modifications are possible and/or that what is described is the sole manner of implementing that element of the present technology.
Moreover, all statements herein reciting principles, aspects, and implementations of the technology, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof, whether they are currently known or developed in the future. Thus, for example, it will be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the present technology.
Similarly, it will be appreciated that any flowcharts, flow diagrams, state transition diagrams, pseudo-code, and the like represent various processes which may be substantially represented in computer-readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
The functions of the various elements shown in the figures, including any functional block labelled as a “processor”, may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared.
Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read-only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional and/or custom, may also be included.
Software modules, or simply modules which are implied to be software, may be represented herein as any combination of flowchart elements or other elements indicating performance of process steps and/or textual description. Such modules may be executed by hardware that is expressly or implicitly shown.
With these fundamentals in place, we will now consider some non-limiting examples to illustrate various implementations of aspects of the present technology.
With reference to
In some embodiments, the computer system 100 may also be a subsystem of one of the above-listed systems. In some other embodiments, the computer system 100 may be an “off-the-shelf” generic computer system. In some embodiments, the computer system 100 may also be distributed amongst multiple systems. The computer system 100 may also be specifically dedicated to the implementation of the present technology. As a person in the art of the present technology may appreciate, multiple variations as to how the computer system 100 is implemented may be envisioned without departing from the scope of the present technology.
Those skilled in the art will appreciate that processor 110 is generally representative of a processing capability. In some embodiments, in place of or in addition to one or more conventional Central Processing Units (CPUs), one or more specialized processing cores may be provided. For example, one or more Graphic Processing Units 111 (GPUs), Tensor Processing Units (TPUs), and/or other so-called accelerated processors (or processing accelerators) may be provided in addition to or in place of one or more CPUs.
System memory will typically include random access memory 130, but is more generally intended to encompass any type of non-transitory system memory such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), or a combination thereof. Solid-state drive 120 is shown as an example of a mass storage device, but more generally such mass storage may comprise any type of non-transitory storage device configured to store data, programs, and other information, and to make the data, programs, and other information accessible via a system bus 160. For example, mass storage may comprise one or more of a solid-state drive, hard disk drive, a magnetic disk drive, and/or an optical disk drive.
Communication between the various components of the computer system 100 may be enabled by a system bus 160 comprising one or more internal and/or external buses (e.g., a PCI bus, universal serial bus, IEEE 1394 “Firewire” bus, SCSI bus, Serial-ATA bus, ARINC bus, etc.), to which the various hardware components are electronically coupled.
The input/output interface 150 may provide networking capabilities such as wired or wireless access. As an example, the input/output interface 150 may comprise a networking interface such as, but not limited to, a network port, a network socket, a network interface controller and the like. Multiple examples of how the networking interface may be implemented will become apparent to the person skilled in the art of the present technology. For example, the networking interface may implement specific physical layer and data link layer standards such as Ethernet, Fibre Channel, Wi-Fi or any other radio data link protocol, Token Ring or Serial communication protocols. The specific physical layer and the data link layer may provide a base for a full network protocol stack, allowing communication among small groups of computers on the same local area network (LAN) and large-scale network communications through routable protocols, such as Internet Protocol (IP).
The input/output interface 150 may be coupled to a touchscreen 190 and/or to the one or more internal and/or external buses 160. The touchscreen 190 may be part of the display. In some embodiments, the touchscreen 190 is the display. The touchscreen 190 may equally be referred to as a screen 190. In the embodiments illustrated in
According to some implementations of the present technology, the solid-state drive 120 stores program instructions suitable for being loaded into the random-access memory 130 and executed by the processor 110 for executing acts of one or more methods described herein. For example, at least some of the program instructions may be part of a library or an application.
The autonomous system 200 may comprise a computer system 210. The computer system 210 may be the computer system 100 and/or may include some or all of the components of the computer system 100. The computer system 210 comprises a processor that may control various aspects of the autonomous system 200, such as by determining a navigation path, or simply “path”, for the autonomous system 200 and causing the autonomous system 200 to travel along the navigation path. The computer system 210 may store the navigation path and/or any other information in a memory 206. The memory 206 may be any type of storage medium, such as an SSD or flash memory. The computer system 210 may control any aspect of the autonomous system 200, such as lights on the autonomous system 200, speakers of the autonomous system 200, a speed of the autonomous system 200, etc.
The environment 230 may include entities such as walls, doors, shops, offices, desks, tables, plants, persons, groups of persons, etc. Therefore, the entities may be partitioned into “human entities” and “non-human entities”. A human entity may be a person or a group of persons.
The autonomous system 200 may comprise a chassis 208, which may include and/or be connected to any number of wheels. The chassis 208 may comprise six 3-axis wheels or any other suitable mechanism for causing the autonomous system 200 to travel, such as continuous tracks, skis, a mechanism for traveling on rails, and/or any other suitable mechanism for navigating within the environment 230.
The autonomous system 200 may comprise a perception unit 202. The perception unit 202 may include image sensors, charge-coupled device (CCD) sensors, complementary metal oxide semiconductor (CMOS) sensors, digital cameras, light detection and ranging (LiDAR) sensors, radar sensors, time of flight (TOF) 3D cameras, stereo vision sensors comprising two or more cameras, 3D sonar, such as 3D ultrasound sensors, structured-light 3D cameras and/or scanners, and/or any other type of sensors. The perception unit 202 may be communicably connected with the computer system 210 and may be configured to gather data regarding the environment 230. The perception unit 202 may be, for instance and without limitation, mounted in a dedicated enclosure mounted on the top of the autonomous system 200.
The perception unit 202 may include one or more cameras configured to capture images and/or video of the environment 230. The cameras may be configured to capture a portion of the environment 230 surrounding the autonomous system 200, such as a 90-degree view or a 180-degree view of the environment 230 in the direction of travel of the autonomous system 200. The perception unit 202 may comprise one or more light detection and ranging (LiDAR) sensors and/or one or more radio detection and ranging (radar) sensors. The radar sensors may comprise long-range, medium-range, and/or short-range radar sensors. The perception unit 202 may be used for adaptive cruise control, automatic emergency braking, and forward collision warning, park assist, cross-traffic alert, junction assist, blind side detection, and/or other functions of the robotic vehicle 200. The perception unit 202 may comprise CCD/CMOS sensors and/or other sensors.
The perception unit 202 may comprises ultrasonic SONAR sensors, odometry sensors including accelerometers and gyroscopes, mechanical tilt sensors, and/or a magnetic compass.
The chassis controller 220 may be communicably connected to the computer system 210. The chassis controller 220 may receive a navigation path and/or other instructions from the computer system 210. The chassis controller 220 may control the chassis 208 based on the instructions from the computer system 210. For example, the chassis controller 220 may receive a navigation path from the computer system 210 and control the chassis 208 to cause the autonomous system 200 to navigate within the environment 230 according to the navigation path.
A map of the environment 230 may be stored in the memory 206 and/or accessed by the computer system 210. The map may be used by the computer system 210 to determine a current location of the autonomous system 200 in the environment 230. The autonomous system 200 may include a global positioning system (GPS) sensor (not depicted), SLAM algorithms and/or other positioning technologies for determining a location of the autonomous system 200 in the environment 230 using the map.
The autonomous system 200 may comprise a networking system 212. The networking system 212 may be used by the computer system 210 to communicate with one or more remote servers and/or other devices. The networking system 212 may communicate a Local Area Network (LAN), a wireless connection such as a Wireless Local Area Network (WLAN), a non-local network (WAN) and/or any other type of network. The networking system 212 may communicate via Wi-Fi, cellular network, and/or any other type of network. The computer system 210 may download update of the map of the environment 230, instructions, indication of a destination, and/or any other information using the networking system 212. The networking system 212 may be used to upload a current position of the robotic vehicle 200, speed of the robotic vehicle 200, generated navigation path of the robotic vehicle, data measured by the perception unit 202, and/or any other information regarding the autonomous system 200. The computer system 210 may be configured to upload information to a server managing the autonomous system 200 using the networking system 212.
The map of the environment 230 may comprise a representation of the environment 230 and/or indication of entities of the environment 230. The map may include multiple layers, where each layer provides a different type of information. For example, a first layer of the map may include structures of the environment 230, such as corridors, stairs, doors, etc; a second layer may comprise characteristics of obstacles, such as benches, plants, tables, etc; and a third layer may comprise characteristics of human entities in the environment 230. The different layers of the map may provide a different resolution and/or level of detail. The computer system 210 may update one or more layer of the map using the perception unit 202. For instance, the computer unit may determine characteristics of non-human entities and/or human entities located in a vicinity of the autonomous system 200 based on the perception unit 202 and further update the map of the environment 230. The characteristics of a non-human entity may comprise a location or any other information of said non-human entity. The characteristics of a human entity may comprise a location, an orientation, a level of business or any other information of said non-human entity. The computer system 210 may receive information from the networking system 212 to update characteristics of non-human entities and/or characteristics of human entities of the environment 230 and that may possibly be out of detection reach of the perception unit 202.
The computer system 210 may generate a navigation path. The navigation path may be determined based on the map of the environment 230 and on social and physical norms that may be stored in the memory 206. The social and physical norms may be populated in the memory 206 by an operator of the autonomous system 200 prior or during operation of the autonomous system 200. The social and physical norms may comprise one or more sets of computer-readable rules and/instructions causing the autonomous system 200 to navigate in accordance with social rules that may be associated with the environment 230. The social and physical norms may be populated and/or updated by the computer system 210 based on instructions send by the operator and received by the networking device 212. Therefore, the social and physical norms may be adjusted to the environment 230. Adjusting the social and physical norms may enable the autonomous system 200 to navigate seamlessly with navigation standards of the environment 230. Notably, adjusting the social and physical norms may cause the autonomous system to navigate in the environment 230 without inconveniencing, offending or disturbing human entities of the environment 230. For instance, the social and physical norms may include instructions causing the autonomous system not to navigate too close to a human entity's back, or in a middle of a group of persons. Therefore, it can be said that the navigation path is a “social navigation path” as it takes into consideration social and physical norms to provide the autonomous system with a path in populated environments. Therefore, the social and physical norms comprise rules for social human interaction and avoidance. The social and physical norms may also ensure the safety if the autonomous system 200, causing, for instance, the autonomous system 200 to navigate at a reasonable distance from the walls, doors, angles, obstacles, etc. Social and physical norms may be associated with entities of the environment 230. For instance, a first set of social and physical norms may be associated with the human entities and a second set of social and physical norms may be associated with the non-human entities.
The navigation path may define a path to be travelled by the autonomous system 200 to reach a destination. The navigation path may comprise one or more waypoints in the format of a series of coordinates, a series of instructions, and/or any other suitable format. Determining location of the one or more waypoints is described in greater details hereinafter.
The computer system 210 may receive instructions to interact with a human entity associated with a location. In this situation, the computer system 210 may determine a destination in a vicinity of the human entity based on the map of environment 230 and the social and physical norms.
The navigation path may be stored in the memory 206 and/or transmitted to the chassis controller 220. The computer system 210 may generate instructions for the chassis controller 220 based on the navigation path.
In this illustrative embodiment, the navigation path comprises one or more waypoints 52, the one or more waypoints 52 being determined by the computer system 210 based on the map and the social and physical norms. The one or more of waypoints 52 define one or more segmental paths 54, each segmental path being defined between two consecutive waypoints 52. The autonomous system 200 may be configured to navigate on the one or more segmental paths 54 to reach the destination 50. Besides, the computer system 210 may determine the location of the one or more waypoints 52 so that the segmental paths 54 do not intersect the private areas 12 and 14.
Location of waypoints may be updated and/or adjusted by the computer system 210 based on information provided by the networking system 212 and/or the perception unit 202. For instance, if determination is made that a waypoint 52 and/or the destination 50 is located within a private area such as private areas 12 and 14, the computer system may update a location of said waypoints 53 and or destination 50. The navigation path may be stored in the memory 206 by the computer system 206.
In the same or another embodiment, each waypoint 54 is associated with a location cost, the location cost of a waypoint being an indication of a safeness of the autonomous system 200 and entities in its surroundings when the autonomous system is located at said waypoint. Therefore, the location cost may depend on characteristics of the entities when the location of the waypoint is determined. For instance, a location cost of a waypoint located in a middle of a corridor may be higher than a location cost of a waypoint located near a wall of the corridor, as it may be riskier for the autonomous system 200 to navigate in the middle of the corridor due to higher chance of encountering and interfering with a person. Similarly, a location cost of a waypoint located near a human entity may be higher than a location cost of a waypoint located further from said human entity, as it may be riskier for the autonomous system 200 to navigate in the middle of the corridor due to higher chance of encountering and interfering with a person.
The location costs may be determined by the computer system 210 based on the map of the environment 230. Alternatively or additionally, a location of the segmental paths between the waypoints may be associated with location costs based on social and physical norms combined with the map of the environment 230 comprising the entities. For instance, a segmental path intersecting a private area may have a higher location cost than a segmental path that does not intersect said private area. Determining the navigation path may comprise determining locations of the waypoints that minimize a global cost, the global cost comprising a combination of the location cost of each waypoint and each segmental path. The location cost of the waypoints may be determined based on location of the waypoints and/or location of the segmental paths between the waypoints. For instance, two consecutive waypoints having a corresponding segmental path intersecting a private area such as private areas 12 and 14 may be associated with a high location cost. The global cost of a navigation path may also comprise a length cost, based on a total length of the navigation path.
Upon navigating between the two consecutive waypoints 510 and 520, the computer system 210 may generate a sub-path between the two consecutive waypoints 510 and 520 so that the autonomous system navigates in accordance with the social and physical norms associated with the entities located in a vicinity of the segmental path 54. The sub-path may comprise one or more sub-waypoints 515 in the format of a series of coordinates, a series of instructions, and/or any other suitable format, the computer system 210 causing the autonomous system to follow the one or more sub-waypoints 515 to reach the second waypoint 520. A location of the second waypoint 520 may be adjusted in a similar manner if determination is made by the computer system 210 that it is located within a private area such as private area 16. The sub-waypoints 515 may define segmental paths 56 between two consecutive sub-waypoints 515.
In the same or another embodiment, each sub-waypoint 515 is associated with a location cost using the same approach as the waypoints 54. The location costs of sub-waypoints 515 may be determined by the computer system 210 based on the map of the environment 230 upon navigating from the first waypoint 510 to the second waypoint 520. Generating the sub-path may comprise determining locations of the sub-waypoints 515 that minimize a sub-cost, the sub-cost comprising a combination of the location cost of each sub-waypoints 515 determined between the first waypoint 510 and the second waypoint 520. The sub-cost of a sub-path may also comprise a length cost, based on a total length of the sub-path. In this embodiment, it can thus be said that the determination of a path of the autonomous system 200 is hierarchical given that waypoints are firstly determined based on entities within the map of the environment thereby optimizing the path in terms of length and safety, and sub-waypoints are further determined during navigation of the autonomous system 200, thereby increasing safety of the autonomous system 200 and entities therearound by adjusting portions of the path between consecutive waypoints. Developers of the present technology have realized that said adjustments increase a safety of the autonomous system 200 and entities therearound in highly dynamic environment such as, without limitations, malls, airports, hospitals, warehouses and train stations.
Alternatively or additionally, a location of the segmental paths between the sub-waypoints may be associated with location costs based on social and physical norms combined with the map of the environment 230 comprising the entities. For instance, two consecutive sub-waypoints having a corresponding segmental path intersecting a private area such as private area 16 may be associated with a high location cost. In the illustrative example of
In this illustrative situation, the segmental path 54 does not intersect with the private area 65. Therefore, a sub-path between the first waypoint 510 and the second waypoint 520 may be defined by the segmental path 54.
Human entities comprising a higher number of persons are also contemplated. The computer system 210 may be configured to identify characteristics of the human entities, notably a number of persons comprised in the human entities and a formation of the human entities and determined the private areas and the ideal interaction destination associated thereto.
The private area associated with a human entity may comprise a plurality of subareas populated in the map of the environment 230. Each subarea may be associated with one or more corresponding social and physical norms. Therefore, segmental paths intersecting the subareas may be associated with a distinct location cost depending on the subareas they are intersecting. A subarea may have a different shape than its corresponding private area and/or than other subareas of its corresponding private area. A subarea may extend outside its corresponding private area. As an example,
Definition of the private areas and the ideal interaction destinations based on characteristics of the human entities may be stored as instructions and/or functions as disclosed in Dynamic proxemia modeling formal framework of social navigation and interaction (Abir Bellarbi, Abdel-illah Mouaddib, Noureddine Ouadah and Nouara Achour), published in 25 Aug. 2020, the content of which being hereby incorporated by reference. Said instructions and/or functions may be stored in the memory 206.
At step 805, characteristics of entities of an environment may be received. The environment may be the environment 230 in which the autonomous system 200 may be configured to operate. The entities may define the environment 230. The computer system 210 of the autonomous system 200 may receive the characteristics and populate a map of the environment 230, the map comprising the characteristics. A current position of the autonomous system 200 may be determined using the perception unit 202 and/or using a positioning system, such as a global positioning system (GPS) sensor and/or other positioning technologies.
The entities may be human entities or non-human entities. Characteristics of human entities may be selected in a group of characteristics comprising: a formation of the human entity, a number of persons comprised in the human entity, an orientation of the persons of the human entity, a location of the human entity, a level of business of the human entity, or a combination thereof. The computer system 210 may be configured to process the received characteristics and determine a location of the autonomous system in the environment 230.
Characteristics may be received from a networking system, such as networking system 212, and/or from a perception unit, such as perception unit 202. Characteristics may be updated at any time by the networking system 212 and/or the perception unit 202. The map comprising the characteristics may be updated as new information is received, such as additional information relating to characteristics of the entities from the perception unit 202 and/or the networking system 212, at a predetermined frequency, and/or at any other interval. Characteristics and/or the map populated with the characteristics may be stored in a memory such as memory 206.
Social and physical norms may be stored in the memory 206 and further associated with entities. Social and physical norms may be computer-readable sets of instructions defining costs and representing riskiness of a navigation of the autonomous system 200 nearby corresponding entities. For instance, social and physical norms associated with a human entity may be used by the computer system 210 to define a private area around the human entity such as private areas 12, 14 and 16.
At step 810, instructions to identify a destination are received by the computer system 210. The instructions may be transmitted by an operator of the autonomous system via the networking system 212. The instructions may be generated by the computer system 210 based on information provided by the perception unit 202.
If determination is made based on the instructions that the autonomous system has to interact with a human entity, the computer system 210 may determine a destination in a vicinity of the human entity based on the characteristics of the human entity and on the social and physical norms associated thereto. Therefore, the destination may depend on characteristics of the human entity, such as a number of persons comprised in the human entity, an orientation of the persons, the level of business of the persons, etc.
If determination is made based on the instructions that the autonomous system does not have to interact with a person, the instructions may comprise the destination.
At step 815, the computer system 210 may generate a navigation path using a hierarchical path planning. The navigation path may be defined between a current location of the autonomous system 210 and the destination determined at step 810 using the map of the environment 230. The navigation path may comprise one or more waypoints to be followed by the autonomous system 200 to reach the destination. Each waypoint may comprise coordinates relative to a current location of the autonomous system 200. The one or more waypoints may define segmental paths between two consecutive waypoints. Therefore, the autonomous system 200 may navigate on the segmental paths to reach the destination.
Location of the one or more waypoints may be based on the characteristics of the entities, such as location and/or orientation of the entities comprised in the map of the environment 230, and the social and physical norms associated thereto. Indeed, the one or more waypoints and the segmental path may be associated with location costs based on the characteristics of the entities and the social and physical norms associated thereto, the map of the environment 230 being indicative of characteristics of human and non-human entities. A global cost may be associated with the navigation path. The global cost may comprise a combination of the location costs of the one or more waypoints and the location costs of the segmental paths. The global cost may comprise a length cost corresponding to a length of the navigation path. Upon generating the navigation path, the computer system 210 may minimize the global cost of the navigation path. This may allow to maximize an accordance of the navigation of the autonomous system 200 with the social and physical norms of the entities. More precisely, this may allow the autonomous system 200 to navigate safely and/or without inconveniencing and/or disturbing entities of the environment 230.
At step 820, the computer system 210 may execute instructions to cause the autonomous system to navigate along the navigation path. The instructions may be generated by the computer system 210 and further transmitted to a chassis of the autonomous system 200, such as the chassis 208 via a chassis controller, such as the chassis controller 220. Although described as being sent to the chassis controller 220, it should be understood that the instructions may be sent to any other system or systems operating the autonomous system 200, and/or may be used by the computer system 210 to operate the autonomous system 200.
The navigation path may be adjusted by the computer system 210 during navigation. For instance, the computer system 210 may receive update of the characteristics of the entities from the networking device 212 during navigation, and further modify a location of one or more of the waypoints and/or a location of the destination according to the characteristics and the social and physical norms associated thereto. An adjustment of the navigation path may be performed in accordance with the following sub-steps.
At sub-step 822, upon navigation of the autonomous system 200 from a first waypoint to a consecutive waypoint, the computer system may receive updates of the characteristics of the entities of the environment 230. The updates may correspond to entities located in a vicinity of the autonomous system 200. The updates may be provided by the perception unit 202. A speed of the autonomous system 200 may be adjusted based on the location costs of the first waypoints, the consecutive waypoint and/or the segmental path defined between the two.
At sub-step 824, a sub-path between the first waypoint and the consecutive waypoint may be generated. The computer system 210 may generate the sub-path. The sub-path may comprise sub-waypoints to be followed by the autonomous system 200 to reach the consecutive waypoint. The sub-waypoints may define segmental paths between two consecutive sub-waypoints. Therefore, the autonomous system 200 may navigate on the segmental paths to reach the consecutive waypoint.
Location of the sub-waypoints may be based on the updates of the characteristics of the entities, such as location and/or orientation of the entities, and the social and physical norms associated thereto. Indeed, the sub-waypoints and the segmental path between sub-waypoints may be associated with location costs based on the updates of the characteristics of the entities and the social and physical norms associated thereto. A sub-cost may be associated with the sub-path. The sub-cost may comprise a combination of the location costs of the one or more sub-waypoints and the location costs of the segmental paths. The sub-cost may comprise a length cost corresponding to a length of the sub-path. Upon generating the sub-path, the computer system 210 may minimize the sub-cost of the sub-path. This may allow to maximize an accordance of the navigation of the autonomous system 200 with the social and physical norms of the entities between the first waypoint and the consecutive waypoint. The computer may adjust a location of the consecutive waypoint based on the updates of the characteristics of the entities and the social and physical norms associated thereto.
At sub-step 826, the computer system 210 may cause the autonomous system 200 to follow the sub-path and reach the consecutive waypoint. The computer system 210 may generate instructions and further transmit said instructions to the chassis 208 via the chassis controller 220. Sub-steps 822 to 826 may be repeated until the autonomous system 200 reaches the destination of the navigation path.
It should be expressly understood that not all technical effects mentioned herein need to be enjoyed in each and every embodiment of the present technology.
Modifications and improvements to the above-described implementations of the present technology may become apparent to those skilled in the art. The foregoing description is intended to be exemplary rather than limiting. The scope of the present technology is therefore intended to be limited solely by the scope of the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 21305893.6 | Jun 2021 | EP | regional |
| 21305894.4 | Jun 2021 | EP | regional |
| 21305895.1 | Jun 2021 | EP | regional |
| 21305896.9 | Jun 2021 | EP | regional |
The present patent application is a continuation application of the International Application No. PCT/IB2022/056018 filed on Jun. 28, 2022, which claims priority from European Patent Application Number 21305893.6 filed on Jun. 29, 2021, European Patent Application Number 21305894.4 filed on Jun. 29, 2021, European Patent Application Number 21305895.1 filed on Jun. 29, 2021, and European Patent Application Number 21305896.9 filed on Jun. 29, 2021, an entirety of contents of each of which is incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/IB2022/056018 | Jun 2022 | US |
| Child | 18393425 | US |
