Patent Application
20210191415
Mobile robotic devices have reduced the human effort required to perform many everyday tasks. For example, automatic cleaning devices help maintain and clean surfaces, such as hardwood floors, carpet and the like. An example of such a device is disclosed in U.S. Pat. No. 9,028,617, titled “Cleaning Device with Single Tank Recycling System,” the disclosure of which is fully incorporated into this document by reference.
Users who use such devices to clean floors wish to have the machines perform their cleaning tasks in the shortest possible amount of time, while also avoiding collisions with barriers such as walls, storage/shelving units, machinery, in-store displays, and/or other obstacles. Customers also need to identify differences between adjacent areas, such as differences in floor types, as a robotic device may only be permitted to enter an area having a floor type that corresponds to the cleaning mode in which the device is operating. This often requires the mobile robotic device to be pre-loaded with a detailed map of the area(s) to be cleaned within a facility. The device will rely on this map identify and avoid known obstacles and follow a predetermined cleaning pattern. However, it is not always possible or practical for a device to obtain a detailed map before entering the facility.
In addition, learning an unknown environment can be a complex process that requires the robotic device to collect and analyze a large volume of data. Accordingly, methods and systems are needed for enabling robotic devices to more efficiently generate a map of a facility while also navigating through it.
This document describes methods and systems that are intended to provide a robotic device, such as a robotic cleaning device, the ability to more efficiently develop a map for a facility as it moves through the facility.
In various embodiments, a robotic device is used to generate a map of a facility. A processor that is part of and/or external to the robotic device will receive a facility description that includes an initial path and one or both of the following: a first number of navigable openings that are positioned along the initial path, or a first number of barriers that are positioned along the initial path. The robotic device will move along the initial path. As this happens, one or more sensors of the robotic device will capture a data set of sensed characteristics of the facility. The processor will analyze the data set to detect navigable openings along the initial path, and it will assign an identifier to each navigable opening. The processor will use the identifiers of the navigable openings to develop a map of the facility that includes the navigable openings. The processor will save the map to a data store.
In some embodiments, the facility description also includes a set of instructions for the initial path. Causing the robotic device to move along the initial path may include causing the robotic device to move around the facility according to the set of instructions.
In some embodiments, the sensed characteristics of the facility include one or more detected openings. The system may analyze the data set to detect which of the detected openings are navigable openings in the facility by, for each opening: determining a width of the opening; and determining whether the opening is navigable by determining whether the width of the opening exceeds a threshold that is larger than a width dimension of the robotic device. Identifying the width of each detected opening may include identifying a width dimension for the detected opening that is either (a) a distance from an adjacent endpoint of the initial path to a barrier that is adjacent to the navigable opening, or (b) a distance between two barriers that are adjacent to the navigable opening. Alternatively, identifying the width of each detected opening may include (i) identifying a distance between a first end of the navigable opening and a start position that is either (a) a starting point of the initial path or (b) a second end of a previous navigable opening; and (ii) identifying a distance between a second end of the navigable opening and either (a) the start position or (b) the first end of the navigable opening.
In some embodiments, the system also may analyze the data set to detect the barriers that are positioned along the initial path. For each of the barriers, the system may identify a width of the barrier. When developing the map of the facility, the system may include the barriers in the map of the facility.
In some embodiments, for one or more of the navigable openings, the system may determine whether the system has received an instruction to map an area within the navigable opening. If (and only if) the system has received an instruction to map the area within the navigable opening, the system may cause the robotic device to move into the area, As the robotic device moves within the area, one or more sensors of the robotic device may capture sensed characteristics of the area. The system may then include structural information corresponding to the sensed characteristics of the area in the map.
In some embodiments, the facility description may include the first number of barriers that are positioned along the initial path. If so, then analyzing the data set to detect each navigable opening along initial path may include: (i) detecting each barrier along the initial path; and (ii) for each detected barrier, identifying an opening that begins or ends at an endpoint of the detected barrier.
In some embodiments, the robotic device may be a robotic cleaning device. If so, the robotic cleaning device may use the map to navigate within and clean the facility. If so or alternatively, the robotic device may clean the facility while moving along the initial path and capturing the data set of sensed characteristics.
In other embodiments, in a method of generating a map of a facility using a mobile robotic device, a processor will receive a facility description that includes a number of navigable openings, a number of barriers, or both. One or more sensors of the robotic device will capture a data set of sensed characteristics of the facility while the robotic device remains in place or rotates. The processor will then analyze the data set to detect navigable openings in the facility and assign an identifier to each navigable opening. The processor will use the identifiers of the navigable openings to develop a map of the facility that includes the navigable openings, and it will save the map to a data store.
In this document, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. In this document, the term “comprising” means “including, but not limited to.” Unless defined otherwise, all technical and scientific terms used in this document have the same meanings as commonly understood by one of ordinary skill in the art.
When used in this document, terms such as “top” and “bottom,” “upper” and “lower”, or “front” and “rear,” are not intended to have absolute orientations but are instead intended to describe relative positions of various components with respect to each other. For example, a first component may be an “upper” component and a second component may be a “lower” component when a device of which the components are a part is oriented in a first direction. The relative orientations of the components may be reversed, or the components may be on the same plane, if the orientation of the structure that contains the components is changed. The claims are intended to include all orientations of a device containing such components.
In this document, the terms “communication link” and “communication path” mean a wired or wireless path via which a first device sends communication signals to and/or receives communication signals from one or more other devices. Devices are “communicatively connected” if the devices are able to send and/or receive data via a communication link. “Electronic communication” refers to the transmission of data via one or more signals between two or more electronic devices, whether through a wired or wireless network, and whether directly or indirectly via one or more intermediary devices.
The terms “memory,” “memory device,” “computer-readable medium,” “data store,” “data storage facility” and the like each refer to a non-transitory device on which computer-readable data, programming instructions or both are stored. Except where specifically stated otherwise, the terms “memory,” “memory device,” “data store,” “data storage facility” and the like are intended to include single device embodiments, embodiments in which multiple memory devices together or collectively store a set of data or instructions, as well as individual sectors within such devices.
The terms “processor” and “processing device” refer to a hardware component of an electronic device that is configured to execute programming instructions. Except where specifically stated otherwise, the singular terms “processor” and “processing device” are intended to include both single-processing device embodiments and embodiments in which multiple processing devices together or collectively perform a process.
A “robot” or “robotic device” is a system that is capable of autonomous movement and that performs both physical and computational activities. The physical activities may be performed using a wide variety of movable parts including various tools, such as scrub brushes, liquid applicators and vacuum systems in a robotic cleaning device. The computational activities may be performed using a suitable processor and computer readable memory devices that are part of the robot, that are external to the robot on a system that communicates with the robot by a communication signal delivered via communication technologies such as Wi-Fi, short range communication, or near field communication. The computational activities may include processing information input from various sensors or other inputs of the robotic device to perform commanded functions such as vacuuming, scrubbing, or applying water or cleaning solution to a floor. Furthermore, while robotic cleaning devices are described throughout the present disclosure, it is to be understood that the “robot” or “robotic device” is not necessarily limited to a robotic cleaning device, and may be any mobile robotic device capable of covering an area, including devices covering the area for purposes other than (or in addition to) cleaning operations.
The sensor assembly 110 may include various components such as a first sensor assembly 112 and a second sensor assembly 114, which assemblies may include a sonar assembly, a LIDAR assembly, and/or other various sensor mechanisms such as other laser-based sensors, global positioning system (GPS) devices, microwave sensors, cameras, gyroscopes, infrared (IR) sensors, structured light sensors such as structured light cameras and 3D structured light scanners, stereo cameras and other sensors that are capable of capturing information about the environment in which the robot is operating. Any combinations of these sensors may be used, and the sensors may be placed in various locations on the robotic device, such as its front, rear, or sides. The sensors will collect information about the mobile robotic device's surroundings such as coordinates, navigational information, visual information and/or the like. This document may refer to data collected by one or more of the sensors as “sensor data.” For example, if the sensor assembly includes a sonar assembly, the system may include a tunable ultrasonic piezoelectric transducer configured to transmit sound at a desired pressure level, and to receive reflected sound so that the processor may use the reflected signals to calculate a distance between the robot and an obstacle such as a wall.
If the sensor assembly includes a camera, the device may process received images to locate one or more substantially straight, curved or otherwise shaped lines in the image that may be representative of a boundary that is formed by a wall, a step up or down, a transition between flooring types, or another obstacle or edge. For example, a video camera may be able to detect a corner where an aisle of a store or storage facility meets an intersecting aisle. Similarly, a video camera may be able to detect an edge based on a variation in floor type, such as by color contrast. The mobile robotic device may use this information, optionally with information from other sensors of the device to determine its position relative to the detected edge. For example, a mobile robotic device may compare information about its current position, such as its coordinates, heading and/or the like, with information received from the video camera regarding the edge. The mobile robotic device may use this comparison to determine its position relative to the edge, its distance away from the edge and/or the like.
The controller 120 may include a processor 122 and a computer-readable medium 124 such as a hard drive, an optical drive, a flash drive, a read-only memory or a random access memory containing programming instructions that are configured to cause the processor 122 to process information and control the other elements of the robot. In addition or alternatively to an on-board memory with programming instructions, the controller may be connected to a transceiver that is capable of receiving commands from one or more external systems.
The transport assembly 130 includes a mechanical system that is configured to move the robotic device from one place to another, such as a motor, drive system, and set of wheels, rollers, or the like. The speed of the robotic device may be determined via, for example, data received from optical encoders associated with the wheels, rollers, etc. However, other means of determining the speed of the robotic device are also within the scope of this disclosure.
The cleaning assembly 140 may include cleaning elements that are configured to contact, dislodge and/or draw particles from the floor to clean the floor. Examples of such cleaning elements include a cleaning head with and a scrubber, a squeegee, a vacuum system, and one or more motors to activate these elements.
The fluid delivery system 150 may include one or more tanks for holding fresh water and/or cleaning solution that is directed to the floor scrubber and/or other components of the cleaning assembly, and/or used water and/or cleaning solution that the cleaning assembly withdraws from the floor after application. The fluid delivery system 150 also will include elements for directing fluid to, and withdrawing fluid from, the floor such as filter, a discharge line, and a pump. For example, the fluid delivery system 150 may include a fluid dispensing tank, a pump for directing fluid from the dispensing tank to the floor cleaning devices via one or more conduits, a suction assembly such as a suction pump and conduit leading from the underside of the robotic device, and a fluid receiving tank.
The components of the robotic device 100 as described may be configured to perform a wide variety of operations. The robotic device may include a processor 122 that is configured to monitor and control various operations of the robotic device 100. The robotic device also may include a memory device 124 configured to store a variety of programming instructions and/or data used by the processor 122 as well as the other components of the robotic device 100.
The block diagram of
In step 302 the system will identify an initial path in the facility (such as path 201 in
The system may receive the initial path as part of the basic site description (in other words, concurrently with step 301 above) in a data file, by using one or more sensors (such as cameras, radar or LIDAR), or by manual input. Alternatively, the system may develop the initial path using one or more sensors to identify a first open area (such as that associated with path 201) along with the robotic device may travel for at least a minimum threshold distance. Such sensors may be any of the components of an onboard sensor assembly 110 as described in
In step 303, the processor will then cause the robotic device to move along the initial path. As the robotic device moves along the initial path, at step 304 one or more sensors of the robotic device will capture a data set of sensed characteristics of the facility on one or both sides of the initial path. The processor will analyze the data set at step 305 to detect the navigable openings for one or both sides, along with one or more barriers that are positioned between any of the navigable openings. The system will do this using any now or to be known object detection techniques. For example, the processor may analyze digital image streams from a camera and using any now or to be known edge detection technique to detect the presence or absence of boundaries within a certain distance from the path. The processor also may analyze radar or sonar data received from a radar or sonar sensor to identify locations where reflected signals are returned to the sensor within a time period that is below a threshold that corresponds to an expected time period for a boundary to be present. In this situation, the location of any boundary may be associated with a barrier, and the location where any boundary is not present may be associated with an opening.
To determine whether an opening is navigable, the system will determine a width for each detected opening (step 310). The width may simply be a width dimension from start to finish (i.e., distance between two adjacent barriers as with areas 203-205 of
For each of the navigable openings, the system will assign an identifier to the navigable opening (step 306). The identifier may be a code that serves to identify the opening's position along the initial path with respect to the starting point. For example, referring to
Optionally, the system also may determine a width for each barrier (step 311). As with the width determination for navigable openings, the system may determine the width of each barrier simply as a width dimension from end to end, or distance between an adjacent opening and the closest endpoint of the initial path if the barrier overlaps the endpoint.
After identifying each navigable opening on a side of the path and assigning the identifier to the opening, the system may determine whether the system has reached the end of its path or if the identifier corresponds to the received number of openings for the side (step 312). In other words, the system will determine whether the opening is the final expected opening along the initial path for that side. In the example shown in
If the opening is the final opening in step 312, the system may continue along the initial path until the path's ending condition is met (such as until reaching a certain distance, or until detecting an obstacle). Optionally, detection of the final opening may be the ending condition. Optionally, multiple passes along the openings may be required before the end of the path is reached. After the end of the path has been reached (or incrementally while traversing the path), the system will develop a map by saving the positions and widths of each of the openings to a map file, and saving the map file in a data store (step 313). The map will thus include data that represents the facility layout (such as that of
Optionally, the system also may determine whether it should map areas within each opening (step 307). The system may do this incrementally, as it encounters each navigable opening. Alternatively, the system may go back and determine whether to map areas within navigable openings after it has completed traversing the initial path. Either way, when the system determines that it should map an area within an opening, it will move the robotic device into the opening and perform a defined map acquisition process in the area. For example, its sensors may collect characteristics of the area to identify structural details for the area (openings, barriers, boundaries, and the like as discussed above) for inclusion in the map the area discovered during the acquisition process defined by the acquisition method in the facility description dataset (step 308). The system may do this using the same processes as are described in this document, or using any other data collection and map generation methods. When it has completed mapping that area, the system will return to the initial path (step 309) and then continue moving along the initial path, either to the next navigable opening or to the end of the initial path if no more openings exist.
To determine whether to map areas within an opening (step 307), the system will analyze the initial site description to determine whether the site description indicates that the area within the opening should be mapped. In some embodiments, a site description may include an identifier for each opening indicating whether it should be mapped. Alternatively, the system may default to either (a) mapping each opening's area unless instructed otherwise, or (b) not mapping any opening's area unless instructed otherwise. Either way, by using the site description to determine that only certain corridors need to be mapped, the system can more quickly map the important corridors of a facility and not bother with corridors for which the site description indicated that a map is not required. For example, in
Optionally, some areas may be designated for only partial mapping. If so, an area within an opening may be designated with an “offset,” which is a distance from an endpoint of the area to an interior position in the area that should be excluded from the mapping process. In
A site description may use any suitable format. For example, the site description may be a sequence of numbers, an example of which is:
(1, (0 (2,1), (3,0), (4,1), (5,1,0,0.5), (6, ((7,1))), (1 (8,0), (9,0), (10,0))).
In this example, the number 1 in the first order indicates that the initial path is area 1. In the second order, 0 means left and 1 means right. A third order indicates whether each opening should be mapped (with 1 being “map” and 0 being “don't map”), either in full or in part. Therefore, in the example above, along the left side (0) there are openings designated as areas 2, 3 and 4. The (2,1) and (4,1) indicate that areas 2 and 4 should be mapped with no offsets, while the (3,0) indicates that area 3 should not be mapped. The (5,1,0.5), indicates that area 5 should be mapped, but that during operation the device should maintain a 0.5 offset from the area's right side wall. The (6, (7,1))), indicates that areas 6 and 7 should be mapped. Finally, (1 (8,0), (9,0), (10,0)) indicates that to the right (1) of area 1, areas 8, 9 and 10 exist but none of them should be mapped.
Returning to
In step 403, the processor will then cause the robotic device to move along the initial path. As the robotic device moves along the initial path, at step 404 one or more sensors of the robotic device will capture a data set of sensed characteristics of the facility on one or both sides of the initial path. The processor will analyze the data set at step 405 to detect the barriers for one or both sides, along with one or more navigable openings that are adjacent to each barrier. The system will do this using any now or to be known object detection techniques, such as those described above in the discussion of
The process then continues as it did for
After identifying each barrier on a side of the path and assigning the identifier to the opening, the system will determine whether the end of the initial path has been reached (such as, for example, if the system has detected a number of barriers that match the received number of barriers in the facility description data set) (step 412). If all barriers have not yet been detected along a side, or if the initial path's ending condition is not yet reached, the system will return to step 403 and continue to move the robotic device along the initial path to capture data and identify additional barriers and openings.
For example, one acquisition method may be to drive around a room in a counterclockwise direction. This would provide an initial path having four components (one for each sidewall of the room. As the robot travels along the walls, it will detect openings, always to the right side of the wall when moving in the counterclockwise direction. The first pass would be the openings to the right of the area, the second pass would be the openings at the end of the area, the third pass would be the openings to the left of the area and the fourth pass would be the openings at the front of the area. Other acquisition method examples include following each wall along the right side of the robotic device until the device returns to its starting point.
If step 412 determines that the endpoint of the initial path has been reached (such as by determining that all barriers have been detected), or even incrementally as the process proceeds, the system may develop a map by saving the positions and widths of each of the openings to a map file, and saving the map file in a data store (step 413). The map will thus include data that represents the facility.
Optionally, as with the embodiment of
As with the previous embodiment, if the system's robotic device is a robotic cleaning device, the system also may then use the map to clean the facility (step 414), While developing the map, the robotic cleaning device also may clean the facility.
In the embodiments described above, with only minimal initial information (specifically, the initial path, and the number of openings or barriers on the right and/or left side of the path) the system can use the processes described above to more quickly and efficiently develop a map of the facility.
In step 852 the system will identify an initial instruction set for sensing data about the area of facility around the robotic device. Instead of causing the robotic device to travel along the initial path (or before starting along the initial path, or at any point along the initial path), the initial instruction set will cause the robotic device to use one or more sensors to capture (at step 854) a data set of sensed characteristics of the facility around the robotic device, especially along the sidewalls or other barriers. In step 854, while capturing the data the robotic device may remain in place, or the robotic device may rotate to capture data from multiple angles around the robot. (For example,
For each of the navigable openings, the system will assign an identifier to the navigable opening (step 856). The identifier may be a code that serves to identify the opening's position along the initial path with respect to the starting point. Other identification formats may be used.
The system will repeat this process until all navigable openings have been identified (step 861). The system may determine whether the system identified all navigable openings if the number of openings detected corresponds to the number in the facility description data set, or if the robotic device has sensed the entire environment that is within its sensors' field of view, or by other methods.
Optionally, the system also may determine whether it should map areas within each opening (step 857). The system may do this incrementally, as it encounters each navigable opening. Alternatively, the system may go back and determine whether to map areas within navigable openings after it has identified all openings. Either way, when the system determines that it should map an area within an opening, it will move the robotic device into the opening and perform a defined map acquisition process in the area, as described in previous embodiments. When it has completed mapping an area, the system may return to the starting point or move to the next navigable opening (step 859).
When all navigable openings have been identified and area maps completed, the system may develop a map by saving the positions and widths of each of the openings to a map file, and saving the map file in a data store (step 863). The map will thus include data that represents the facility layout. Example map generation processes were discussed above.
Optionally, as with previous embodiments, if the system's robotic device is a robotic cleaning device, the system may then use the map to clean the facility (step 864). The robotic device also may clean the facility while moving into the navigable openings to develop the map.
Referring to
An optional display interface 830 may permit information from the bus 800 to be displayed on a display device 835 in visual, graphic or alphanumeric format. An audio interface and audio output (such as a speaker) also may be provided. Communication with external devices may occur using various communication devices such as a wireless antenna 844, an RFID tag and/or short-range or near-field communication transceiver 842, each of which may optionally communicatively connect with other components of the system. The communication device 840 may be configured to be communicatively attached to a communications network, such as the Internet, a local area network or a cellular telephone data network.
The hardware may also include a user interface sensor 845 that allows for receipt of data from input devices 850 such as a keyboard, a mouse, a joystick, a touchscreen, a touch pad, a remote control, a pointing device and/or microphone. Digital image frames also may be received from a camera (image sensor 820) that can capture video and/or still images. The system also may include a positional sensor 860 and/or motion sensor 870 to detect position and movement of the device. Examples of motion sensors 870 include gyroscopes or accelerometers. Examples of positional sensors 860 include a GPS sensor device that receives positional data from an external GPS network, and/or the other sensors described above such as a sonar assembly or LIDAR assembly.
The following U.S. patent documents are fully incorporated into this patent document by reference: (1) U.S. Pat. No. 8,958,937, titled “Cleaning Machine with Collision Prevention”; (2) U.S. Pat. No. 8,679,260, titled “Method and System for Movement of an Automatic Cleaning Device using Video Signal”; (3) U.S. Pat. No. 8,532,860, titled “Method and System for Automatic Yield to High Priority Traffic”; and (4) U.S. Pat. No. 8,423,225, titled Method and System for Movement of Robotic Device using Video Signal.
It will be appreciated that the above-disclosed and other features and functions may be combined into many other different systems or applications. All such applications and alternatives are also intended to be encompassed by the disclosure of this patent document.
