Patent Application
20220304546
This invention relates generally to cleaning a dish by localizing its dirty regions and targeting the dirty regions for deep cleaning.
Conventional dishwashers wash a large number of dishes at once. Such batch washing comes with two major problems. Firstly, when washing a large number of dishes together, conventional dishwashers prioritize the average cleanliness of a group of dishes over the thorough cleanliness of every individual dish. Conventional dishwashers typically employ a turbidity detector to measure the quantity of dirt present in water during the cleaning process. When the turbidity detector senses the dirt level to be under a threshold, conventional dishwashers assume that dishes are clean. Such an assumption arrived based on the average state of a batch of dishes often overlooks the state of each individual dish in the batch. Conventional dishwashers do not have the means to ensure every dish is thoroughly cleaned during the cleaning process. This is the reason why dishes often do not come out clean even after hours of washing in a conventional dishwasher. A slew of preparatory work such as scraping, rinsing, soaking, optimal loading and positioning of dishes according to their shape, size, and material becomes necessary to maximize the chances of dirty dishes coming out clean in a conventional dishwasher, albeit without any guarantee of success.
Secondly, conventional dishwashers found in homes are substantially slower than washing dishes by hand in a kitchen sink. Conventional dishwashers also consume much more water and energy than washing dishes by hand. This is because batch dishwashing lacks the perception and ability to focus on dirty regions of an individual dish. As a result, it spends much more resources on clean regions of dishes than what is necessary, in an attempt to maximize the chances of all dirty regions of dishes to come out clean. Batch dishwashing fundamentally suffers from a tradeoff between the duration of the dishwashing cycle and cleanliness of each dish. Without the perception to evaluate the cleanliness of each dish, batch dishwashing resorts to longer dishwashing cycles, spanning hours, to increase the chances of removing dirt from all dishes, thereby causing a significant wastage of time, energy, and water.
Accordingly, there is a need for an improved system and method to clean a dirty dish thoroughly and efficiently. One that could clean each dish with individual attention to ensure every dish comes out clean after cleaning; one that could locate dirty regions of dishes in three dimensions; one that could target dirty regions for deep cleaning; one that could conserve energy and water; and one that could be as fast as hand washing of dishes.
The invention is a system and method for cleaning a dish by localizing its dirty regions and targeting the dirty regions for deep cleaning.
In some embodiments, the invention is a system for cleaning a dish, comprising: at least one camera for capturing at least one image of said dish; a processor configured to: compute a bounding polygon for a dirty region in said image, wherein said dirty region is a region of said dish containing unwanted material that needs to be removed; compute a three dimensional representation of said bounding polygon by estimating three dimensional locations of multiple points within said bounding polygon; compute a spray pattern within said three dimensional representation of said bounding polygon such that the spray pattern substantially covers all regions of said three dimensional representation; a nozzle for spraying a fluid on said dirty region of said dish, wherein said nozzle is reoriented or relocated such that said fluid reaches at least one location within said three dimensional representation of said bounding polygon according to said spray pattern, whereby dirty regions of said dish are targeted for a fast and efficient cleaning of said dish.
In some embodiments, the invention is a method for cleaning a dish, comprising: capturing at least one image of said dish using at least one camera; computing a bounding polygon for a dirty region in said image, wherein said dirty region is a region of said dish containing unwanted material that needs to be removed; computing a three dimensional representation of said bounding polygon by estimating three dimensional locations of multiple points within said bounding polygon; computing a spray pattern within said three dimensional representation of said bounding polygon such that the spray pattern substantially covers all regions of said three dimensional representation; spraying a fluid on said dirty region of said dish with a nozzle, wherein said nozzle is reoriented or relocated such that said fluid reaches at least one location within said three dimensional representation of said bounding polygon according to said spray pattern, whereby dirty regions of said dish are targeted for a fast and efficient cleaning of said dish.
A light source 5 illuminates dish 6, while cameras 3 and 4 capture one or more images of dish 6. A nozzle 1 sprays a fluid on dish 6 with a predetermined spray distribution 2. Fluids include liquids and gases such as water, soap, rinsing agent, sanitizing agent, cleaning agent, or air. The nozzle 1 can reorient or relocate to spray fluid on any region of dish 6 visible in one or more images captured by cameras 3 and 4.
A processor configured to compute a bounding polygon 10 for a dirty region in a camera image. A three dimensional representation of the bounding polygon 10 is then computed by estimating three dimensional locations of multiple points within the bounding polygon 10. A spray pattern 11 within the three dimensional representation of the bounding polygon 10 is computed such that the spray pattern 11 substantially covers all regions of the three dimensional representation. In some embodiments, the bounding polygon 10 fully encompasses the dirty region in a camera image.
In some embodiments, a bounding polygon 10 is computed from the edges of the dirty region 7 of dish 6. In some embodiments, each edge of the bounding polygon 10 is substantially parallel to its closest edge of dirty region 7. In some embodiments, the bounding polygon 10 is computed by computing multiple feature points along the edges of the dirty region 7 of dish 6. In some embodiments, feature points along edges are points that are substantially different from at least some of its surrounding regions.
In some embodiments, the bounding polygon 10, the three dimensional representation, or the spray pattern 11 is estimated using a deep learning model such as a neural network. In some embodiments, a deep learning model takes a camera image of a dish as an input and returns one or more bounding polygon 10 as output. In some embodiments, a deep learning model is trained with multiple camera images and their corresponding bounding polygons 10.
In some embodiments, a deep learning model takes a camera image of a dish as an input and returns a three dimensional representation as output. In some embodiments, a deep learning model is trained with multiple camera images and their corresponding three dimensional representations. In some embodiments, a deep learning model takes a bounding polygon 10 as an input and returns a three dimensional representation as output. In some embodiments, a deep learning model is trained with multiple bounding polygons 10 and their corresponding three dimensional representations.
In some embodiments, a deep learning model takes a camera image of a dish as an input and returns a spray pattern 11 as output. In some embodiments, a deep learning model is trained with multiple camera images and their corresponding spray patterns 11. In some embodiments, a deep learning model takes a three dimensional representation as an input and returns a spray pattern 11 as output. In some embodiments, a deep learning model is trained with multiple three dimensional representations and their corresponding spray patterns 11. In some embodiments, a deep learning model takes a bounding polygon 10 as an input and returns a spray pattern 11 as output. In some embodiments, a deep learning model is trained with multiple bounding polygons 10 and their corresponding spray patterns 11.
In some embodiments, the three dimensional locations of multiple points within the bounding polygon 10 are computed from a depth map of dish 6. In some embodiments, the depth map is estimated using stereo matching from at least two camera images. In some embodiments, the depth map is estimated by projecting a structured illumination pattern on the dish, recording an image of dish 6 with said camera, computing deformations to the illumination pattern from the image, and estimating depth map from said deformations. In some embodiments, the three dimensional locations are computed by estimating the location of the bounding polygon 10 within a known three dimensional model of dish 6. In some embodiments, spray pattern 11 comprises a plurality of waypoints such that each waypoint corresponds to a specific position or orientation of nozzle 1.
The nozzle 1 sprays a fluid 2 on the dirty region 7 of dish 6. The nozzle 1 is reoriented or relocated such that the fluid reaches at least one location within the three dimensional representation of the bounding polygon 10 according to the spray pattern 11. Accordingly, dirty regions of said dish are targeted for a fast and efficient cleaning of said dish. The dish 6, nozzle 1, cameras 3 and 4, and light source 5 are enclosed in a module 12.
In some embodiments, a processor is further configured to compute a plurality of bounding polygon 10 for a plurality of dirty regions in a camera image. In some embodiments, a processor then computes a plurality of three dimensional representations of the plurality of bounding polygon 10. Further, in some embodiments, a processor computes a plurality of spray patterns 11 within the plurality of three dimensional representations.
Some embodiments comprise a light source for illuminating the dish 6. In some embodiments, the light source emits a structured pattern of light such as dots or lines. In some embodiments, the light source is either configured as a ring that surrounds a camera or configured as a diffuse illumination panel. In some embodiments, the light source emits infrared light. Some embodiments further comprise an ultraviolet light source to disinfect dish 6.
In some embodiments, a camera is designed to capture infrared images. In some embodiments, a camera captures images when a dish is placed and is ready for cleaning.
