Patent Application
20240119630
The embodiments described herein relate to autonomous and semi-autonomous cleaning devices and more particularly, to a system and method for detecting the status of one or more components and/or systems in a semi-autonomous cleaning device to for improved cleaning of surfaces.
This disclosure relates to cleaning devices or robots that utilize Calibration Health Monitor (CHM) systems for monitoring and making minor adjustments to camera calibration over time. Cleaning devices or robots may use 3D sensors to detect obstacles and cliffs. The accuracy of 3D sensors' calibration is critical for the performance of obstacle and cliff detection.
All robots are statically calibrated before shipping to customers with acceptable calibration error, and a dynamic calibration is applied on the software level to address small calibration errors. The calibration error could increase due to the deformation of the sensor mount, loose mounting or other related problems. Once the error is out of the range of dynamic calibration, the robot will be running with large calibration errors and having more chances of hitting obstacles or stopping by a false obstacle/cliff.
Furthermore, an Occupancy Grid Filter (OGF) can also be used wherein a 3D occupancy grid for probabilistically observing obstacles with 3D sensors that are susceptible to noise or other inaccuracies is employed. An obstacle filtering is applied in the obstacle/cliff detection for noise reduction purposes. The current filtering algorithm is a combination of spatial filters in 3D space, which is very computationally expensive. The high CPU/processor load makes adding another high computational feature (e.g., cliff detection) very difficult.
There is a desire to provide an improved system and method for obstacle and cliff detection techniques. A calibration health monitor (CHM) would be very helpful to reduce the risk of collision, stopping and falling. A better filtering algorithm is required to reduce the CPU consumption to allow adding new features.
A system and method of obstacle and cliff detection for an autonomous or a semi-autonomous cleaning device utilizing a calibration health monitor and occupancy grid filters. A calibration health monitor is used for monitoring and making minor adjustments to camera calibration over time. An occupancy grid filter is a 3D occupancy grid for probabilistically observing obstacles with 3d sensors that are susceptible to noise or other inaccuracies.
An exemplary embodiment of an autonomous or semi-autonomous cleaning device is shown in
The frame 102 of cleaning device 100 can be any suitable shape, size, and/or configuration. For example, in some embodiments, the frame 102 can include a set of components or the like, which are coupled to form a support structure configured to support the drive system 104, the cleaning assembly 108, and the electronic system 106. Cleaning assembly 108 may be connected directly to frame 102 or an alternate suitable support structure or sub-frame (not shown). The frame 102 of cleaning device 100 further comprises strobe light 110, front lights 112, a front sensing module 114 and a rear sensing module 128, rear wheels 116, rear skirt 118, handle 120 and cleaning hose 122. The frame 102 also includes one or more internal storage tanks or storing volumes for storing water, disinfecting solutions (i.e., bleach, soap, cleaning liquid, etc.), debris (dirt), and dirty water. More information on the cleaning device 100 is further disclosed in U.S. utility patent application Ser. No. 17/650,678, entitled “APPARATUS AND METHODS FOR SEMI-AUTONOMOUS CLEANING OF SURFACES” filed on Feb. 11, 2022, the disclosure which is incorporated herein by reference in its entirety.
More particularly, in this embodiment, the front sensing module 114 further includes structured light sensors in a vertical and horizontal mounting position, an active stereo sensor and a RGB camera. The rear sensing module 128, as seen in
The back view of a semi-autonomous cleaning device 100, as seen in
According to this disclosure, there are two calibration health monitor (CHM) calibration parameters. These parameters include static calibration and dynamic calibration.
The static calibration is processed before using the 3D sensors of the robot. This so-called factory calibration can only happen once, and the parameters are saved permanently. The robot must be statically calibrated before all 3D sensors' data can be used for navigation. The static calibration is an individual process that is not included in the autonomous navigation loop; all parameters are calculated separately and set on an online command center portal. The robot syncs the value from the Avidbots Command Center fleet management website (ACC) for every reboot. Normally the static calibration results are fairly good with small errors (e.g., a few cm or a few degrees) at the beginning, but the errors could be increasing as the robot continues to operate.
Dynamic calibration is part of the perception stack, to address small factory calibration errors by using the detected floor plane when the robot is on. Dynamic calibration values are only used for every power-on cycle of the robot software wherein dynamic calibration values are saved in memory, and will be reset for every reboot or service restart. The dynamic calibration can only be used for addressing small calibration errors, but by comparing the static calibration values and dynamic calibration values, bad miscalibration could be detected automatically and at an early stage.
Dynamic calibration is a module in the obstacle/cliff detection pipeline to address small miscalibration when the robot is running autonomously. It updates some of the calibration parameters while the obstacle/cliff detection pipeline is running by detecting the floor plane. The updated values are only used for the power-on cycle of the software, it won't modify the static calibration values on ACC.
According to
According to
For the invalid status, a static calibration validator is introduced to report this status and prevent executing any downstream processing based on an ‘invalid’ situation. In this case, the whole obstacle/cliff pipeline won't be brought up successfully.
For “Good”, “Bad” and “Very bad” miscalibration levels, a calibration health monitor is introduced to report the status as described on above table, with limitation of reporting ‘very bad’.
According to
The main reason for high CPU usage of 3D filtering is that all individual 3D points are being dealt with in 3D space. The main idea of occupancy grid filtering (OGF) is that the filtering decision can be made based on counting the number of points that are projected onto the same cell on the occupancy grid.
To address OGF, an occupancy counting grid algorithm is implemented.
According to
According to the disclosure, the implementation can be broken down into two parts:
occupancy_counting_grid.cpp
Parameters are managed by a floor estimation file under the device's description package.
The solution in this disclosure comprises several differences to linked solutions, including:
According to embodiments of this disclosure, a computer-implemented method for optimized 3D filtering for obstacle and cliff detection on a semi-autonomous cleaning device is provided. The cleaning device has a processor, a camera and one or more sensors. The method comprising the steps of acquiring a depth image with the camera and sensor, converting the depth image to 3D coordinate points, projecting the 3D coordinate points onto an occupancy grid, increasing the count on the projected cells of the occupancy grid, processing the threshold of the occupancy grid based on count, processing the occupancy grid based with large enough contours, combining the count occupancy grid and contour occupancy grid using a filtered occupancy grid, converting the filtered occupancy grid as a point cloud and providing an output of the point cloud of an occupancy grid image.
According to the disclosure, the sensor of the method is a 3D sensor. The step of processing the threshold of the occupancy grid based on count is done by the occupancy counting grid module. The step of processing the occupancy grid based with large enough contours is done by the occupancy grid filtering module.
According to the disclosure, the output of point cloud provided to a user or a remotely to central server. Furthermore, the method for optimized 3D filtering is configured to minimize high CPU usage.
According to the disclosure, a computer-implemented method using a calibration health monitor module for monitoring obstacle and cliff detection on a semi-autonomous cleaning device is disclosed. The method has a processor, a camera and one or more sensors. The method comprising the steps of receiving data at a static calibration loader module, the static calibration loader module configured to determine whether the values are true, if true, sending the data to the static calibration validator module, receiving depth data from a depth streaming module, receiving depth data from the depth streaming module and data from the static calibration validator module at a dynamic calibration module, the dynamic calibration module configured to generate dynamic calibration values, receiving at the calibration health monitor module, static calibration values from the static calibration loader module and dynamic calibration values from the dynamic calibration module and generating a calibration status at the calibration health monitor module. The step of providing an output of the calibration status is to the semi-autonomous cleaning device.
According to the disclosure, a system for obstacle and cliff detection for a semi-autonomous cleaning device is provided. The system comprises a processor, a camera, one or more 3D sensors, a calibration health monitor module and an occupancy grid filter configured to reduce CPU consumption. The calibration health monitor module is configured for monitoring and making minor adjustments to camera calibration over time. The occupancy grid filter is a 3D occupancy grid configured for probabilistically observing obstacles with the 3D sensors that are susceptible to noise or other inaccuracies.
According to disclosure, the occupancy grid filter of the system is optimized 3D filtering is configured to minimize high CPU usage. The system is further configured for processing the threshold of the occupancy grid based on count is done by the occupancy counting grid module.
According to the disclosure, the system is further configured for processing the occupancy grid based with large enough contours is done by the occupancy grid filtering module. The system is configured to provide an output of the point cloud of an occupancy grid image. Furthermore, the output of point cloud is provided to a user or a remotely to central server.
The functions described herein may be stored as one or more instructions on a processor-readable or computer-readable medium. The term “computer-readable medium” refers to any available medium that can be accessed by a computer or processor. By way of example, and not limitation, such a medium may comprise RAM, ROM, EEPROM, flash memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be noted that a computer-readable medium may be tangible and non-transitory. As used herein, the term “code” may refer to software, instructions, code or data that is/are executable by a computing device or processor. A “module” can be considered as a processor executing computer-readable code.
A processor as described herein can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor can be a microprocessor, but in the alternative, the processor can be a controller, or microcontroller, combinations of the same, or the like. A processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Although described herein primarily with respect to digital technology, a processor may also include primarily analog components. For example, any of the signal processing algorithms described herein may be implemented in analog circuitry. In some embodiments, a processor can be a graphics processing unit (GPU). The parallel processing capabilities of GPUs can reduce the amount of time for training and using neural networks (and other machine learning models) compared to central processing units (CPUs). In some embodiments, a processor can be an ASIC including dedicated machine learning circuitry custom-build for one or both of model training and model inference.
The disclosed or illustrated tasks can be distributed across multiple processors or computing devices of a computer system, including computing devices that are geographically distributed.
The methods disclosed herein comprise one or more steps or actions for achieving the described method. The method steps and/or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is required for proper operation of the method that is being described, the order and/or use of specific steps and/or actions may be modified without departing from the scope of the claims.
As used herein, the term “plurality” denotes two or more. For example, a plurality of components indicates two or more components. The term “determining” encompasses a wide variety of actions and, therefore, “determining” can include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure), ascertaining and the like. Also, “determining” can include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory) and the like. Also, “determining” can include resolving, selecting, choosing, establishing and the like.
The phrase “based on” does not mean “based only on,” unless expressly specified otherwise. In other words, the phrase “based on” describes both “based only on” and “based at least on.”
While the foregoing written description of the system enables one of ordinary skill to make and use what is considered presently to be the best mode thereof, those of ordinary skill will understand and appreciate the existence of variations, combinations, and equivalents of the specific embodiment, method, and examples herein. The system should therefore not be limited by the above-described embodiment, method, and examples, but by all embodiments and methods within the scope and spirit of the system. Thus, the present disclosure is not intended to be limited to the implementations shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The application claims priority to and the benefit of U.S. Provisional Patent Application Ser. No. 63/413,580, entitled “SYSTEM AND METHOD OF OBSTACLE AND CLIFF DETECTION FOR A SEMI-AUTONOMOUS CLEANING DEVICE” filed on Oct. 5, 2022, the disclosure of which is incorporated herein by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 63413580 | Oct 2022 | US |
