Patent Application
20240360609
The present subject matter relates generally to connected home appliances, such as a laundry appliance and a separate secondary appliance, and more particularly to systems and methods of coordinated action between connected home appliances.
Generally, modern laundry appliances (e.g., washing machine or dryer appliances) are made up of multiple components that include or are monitored by one or more electronic assemblies (e.g., an assembly or subsystem formed from one or more electrically driven or signal-generating components). For instance, one or more electronically controlled motors, valves, temperature sensors, may be provided.
One of the issues that arises, even with modern laundry appliances, is coordinating cleaning within a household. For instance, it may often be the case that a laundry article (e.g., clothing, towel, sheet, etc.) is intended to be placed back in a specific room or region (e.g., for storage or later use) after it is cleaned or dried in a laundry appliance. However, it is very often possible for that specific room or region to be in a dirty or unkempt state. As an example, laundered towels may be intended to go into a dirty bathroom. Thus, a user must remember to clean the dirty room before placing the laundry article in that room. Often, this needs to occur within the span of a laundry cycle since a user may wish to remove dirty laundry articles from the dirty room and return them to the same room after the laundry articles are clean. This can be very tedious, time-consuming, and frustrating for users.
In recent years, it has become increasingly popular to provide features and methods that allow such laundry appliances to communicate with owners. For instance, an owner may configure a virtual account to pair with a particular laundry appliance. Subsequently, the virtual account may be able to receive notifications regarding the status or completion of a laundry cycle (e.g., wash cycle, dry cycle, etc.).
Although such features can allow an owner to monitor the status of a laundry appliance even when the owner is apart from the appliance, they have drawbacks. For instance, this still requires a user to remember and clean the dirty room (e.g., before the laundry cycle is completed). Some users may have automated cleaning appliances (e.g., robotic vacuums or mops), but such appliances generally have no way of responding to the laundry appliance or anticipating a specific area of need of cleaning (e.g., without explicit or continuous direction from a user).
As a result, there is a need for improved coordination between household appliances (e.g., not relying on or needing direct user intervention or direction). For instance, it would be useful for two or more appliances to improve cleaning of one or more regions of a residence or user-defined space.
Aspects and advantages of the invention will be set forth in part in the following description, or may be obvious from the description, or may be learned through practice of the invention.
In one exemplary aspect of the present disclosure, a method of operating a laundry appliance and a secondary appliance is provided. The method may include detecting a load attribute for a load of articles at the laundry appliance. The method may also include initiating a laundry cycle at the laundry appliance for the load of articles. The method may further include directing a load signal corresponding to the detected load attribute to the secondary appliance and initiating a secondary operation at the secondary appliance in a predefined region based on the load signal during the laundry cycle.
In another exemplary aspect of the present disclosure, a laundry appliance is provided. The laundry appliance may include a cabinet, a tub, an agitation element, a motor, and a controller. The tub may define a laundry chamber to receive a load of articles therein. The agitation element may be disposed within the laundry chamber. The motor may be connected to the agitation element to motivate rotation thereof. The controller may be in communication with the motor. The controller may be configured to initiate a multi-appliance operation. The multi-appliance operation may include detecting a load attribute for a load of articles at the laundry appliance, directing a laundry cycle at the laundry appliance for the load of articles, and transmitting a load signal corresponding to the detected load attribute to a secondary appliance spaced apart from the laundry appliance. The load signal may be configured to initiate a secondary operation at the secondary appliance in a predefined region based on the load signal.
These and other features, aspects and advantages of the present invention will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
A full and enabling disclosure of the present invention, including the best mode thereof, directed to one of ordinary skill in the art, is set forth in the specification, which makes reference to the appended figures.
Repeat use of reference characters in the present specification and drawings is intended to represent the same or analogous features or elements of the present invention.
Reference now will be made in detail to embodiments of the invention, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the invention, not limitation of the invention. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope of the invention. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present invention covers such modifications and variations as come within the scope of the appended claims and their equivalents.
As used herein, the terms “first,” “second,” and “third” may be used interchangeably to distinguish one component from another and are not intended to signify location or importance of the individual components. The terms “includes” and “including” are intended to be inclusive in a manner similar to the term “comprising.” Similarly, the term “or” is generally intended to be inclusive (i.e., “A or B” is intended to mean “A or B or both”). In addition, here and throughout the specification and claims, range limitations may be combined or interchanged. Such ranges are identified and include all the sub-ranges contained therein unless context or language indicates otherwise. For example, all ranges disclosed herein are inclusive of the endpoints, and the endpoints are independently combinable with each other. The singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
Approximating language, as used herein throughout the specification and claims, may be applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “generally,” “about,” “approximately,” and “substantially,” are not to be limited to the precise value specified. In at least some instances, the approximating language may correspond to the precision of an instrument for measuring the value, or the precision of the methods or machines for constructing or manufacturing the components or systems. For example, the approximating language may refer to being within a 10 percent margin (i.e., including values within ten percent greater or less than the stated value). In this regard, for example, when used in the context of an angle or direction, such terms include within ten degrees greater or less than the stated angle or direction (e.g., “generally vertical” includes forming an angle of up to ten degrees in any direction, such as, clockwise or counterclockwise, with the vertical direction V).
The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” In addition, references to “an embodiment” or “one embodiment” does not necessarily refer to the same embodiment, although it may. Any implementation described herein as “exemplary” or “an embodiment” is not necessarily to be construed as preferred or advantageous over other implementations. Moreover, each example is provided by way of explanation of the invention, not limitation of the invention. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope of the invention. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present invention covers such modifications and variations as come within the scope of the appended claims and their equivalents.
Referring now to the figures, an exemplary communications assembly 190 that includes a laundry appliance (e.g., washing machine appliance 100) and secondary appliance 194 (e.g., robotic cleaner 214, air conditioner appliance, etc.) that may be used to implement aspects of the present subject matter will be described. Specifically,
As illustrated, washing machine appliance 100 generally defines a vertical direction V, a lateral direction L, and a transverse direction T, each of which is mutually perpendicular, such that an orthogonal coordinate system is generally defined. Washing machine appliance 100 includes a cabinet 102 that extends between a top 104 and a bottom 106 along the vertical direction V, between a left side 108 and a right side 110 along the lateral direction, and between a front 112 and a rear 114 along the transverse direction T. Cabinet 102 is generally configured for containing or supporting various components of washing machine appliance 100 and which may also define one or more internal chambers or compartments of washing machine appliance 100. In this regard, as used herein, the terms “cabinet,” “housing,” and the like are generally intended to refer to an outer frame or support structure for washing machine appliance 100, e.g., including any suitable number, type, and configuration of support structures formed from any suitable materials, such as a system of elongated support members, a plurality of interconnected panels, or some combination thereof. It should be appreciated that cabinet does not necessarily require an enclosure and may simply include open structure supporting various elements of laundry appliance 100. By contrast, cabinet 102 may enclose some or all portions of an interior of cabinet 102. It should be appreciated that cabinet 102 may have any suitable size, shape, and configuration while remaining within the scope of the present subject matter.
In some embodiments, a wash basket 120 is rotatably mounted within cabinet 102 such that it is rotatable about an axis of rotation A. A motor 122, e.g., such as a pancake motor, is in mechanical communication with wash basket 120 to selectively rotate wash basket 120 (e.g., during an agitation or a rinse cycle of washing machine appliance 100). Wash basket 120 is received within a wash tub 124 and defines a wash chamber 126 that is configured for receipt of articles for washing. The wash tub 124 holds wash and rinse fluids for agitation in wash basket 120 within wash tub 124. As used herein, “wash fluid” may refer to water, detergent, fabric softener, bleach, or any other suitable wash additive or combination thereof. Indeed, for simplicity of discussion, these terms may all be used interchangeably herein without limiting the present subject matter to any particular “wash fluid.”
Wash basket 120 may define one or more agitator features that extend into wash chamber 126 to assist in agitation and cleaning articles disposed within wash chamber 126 during operation of washing machine appliance 100. For example, as illustrated in
Cabinet 102 may include a front panel 130 which defines an opening 132 that permits user access to wash basket 120 of wash tub 124. More specifically, washing machine appliance 100 includes a door 134 that is positioned over opening 132 and is rotatably mounted to front panel 130. In this manner, door 134 permits selective access to opening 132 by being movable between an open position (not shown) facilitating access to a wash tub 124 and a closed position (
In certain embodiments, window 136 in door 134 permits viewing of wash basket 120 when door 134 is in the closed position, e.g., during operation of washing machine appliance 100. Door 134 also includes a handle (not shown) that, e.g., a user may pull when opening and closing door 134. Further, although door 134 is illustrated as mounted to front panel 130, it should be appreciated that door 134 may be mounted to another side of cabinet 102 or any other suitable support according to alternative embodiments.
Referring again to
A drain pump assembly 144 is located beneath wash tub 124 and is in fluid communication with sump 142 for periodically discharging soiled wash fluid from washing machine appliance 100. Drain pump assembly 144 may generally include a drain pump 146 which is in fluid communication with sump 142 and with an external drain 148 through a drain hose 150. During a drain cycle, drain pump 146 urges a flow of wash fluid from sump 142, through drain hose 150, and to external drain 148. More specifically, drain pump 146 includes a motor (not shown), which is energized during a drain cycle such that drain pump 146 draws wash fluid from sump 142 and urges it through drain hose 150 to external drain 148.
A spout 152 is configured for directing a flow of fluid into wash tub 124. For example, spout 152 may be in fluid communication with a water supply 154 (
As illustrated in
In addition, a water supply valve 158 may provide a flow of water from a water supply source (such as a municipal water supply 154) into detergent dispenser 156 and into wash tub 124. In this manner, water supply valve 158 may generally be operable to supply water into detergent dispenser 156 to generate a wash fluid, e.g., for use in a wash cycle, or a flow of fresh water, e.g., for a rinse cycle. It should be appreciated that water supply valve 158 may be positioned at any other suitable location within cabinet 102. In addition, although water supply valve 158 is described herein as regulating the flow of “wash fluid,” it should be appreciated that this term includes, water, detergent, other additives, or some mixture thereof.
Washing machine appliance 100 may include a control panel 160 (e.g., attached to cabinet 102, such as being coupled to front panel 130) that may represent a general-purpose Input/Output (“GPIO”) device or functional block for washing machine appliance 100. In some embodiments, control panel 160 may include or be in operative communication with one or more user input devices 162, such as one or more of a variety of digital, analog, electrical, mechanical, or electro-mechanical input devices including rotary dials, control knobs, push buttons, toggle switches, selector switches, and touch pads. Additionally, washing machine appliance 100 may include a display 164, such as a digital or analog display device generally configured to provide visual feedback regarding the operation of washing machine appliance 100. For example, display 164 may be provided on control panel 160 and may include one or more status lights, screens, or visible indicators. According to exemplary embodiments, user input devices 162 and display 164 may be integrated into a single device, e.g., including one or more of a touchscreen interface, a capacitive touch panel, a liquid crystal display (LCD), a plasma display panel (PDP), a cathode ray tube (CRT) display, or other informational or interactive displays. Control panel 160 and input selectors 162 or display 164 collectively form a user interface input for operator selection of machine cycles and features.
Operation of washing machine appliance 100 is generally controlled by a controller or processing device 166 (
Controller 166 may include a memory and microprocessor, such as a general or special purpose microprocessor operable to execute programming instructions or micro-control code associated with a cleaning cycle. The memory may represent random access memory such as DRAM, or read only memory such as ROM or FLASH. In one embodiment, the processor executes programming instructions stored in memory. The memory may be a separate component from the processor or may be included onboard within the processor. Alternatively, controller 166 may be constructed without using a microprocessor, e.g., using a combination of discrete analog or digital logic circuitry (such as switches, amplifiers, integrators, comparators, flip-flops, AND gates, and the like) to perform control functionality instead of relying upon software. Control panel 160 and other components of washing machine appliance 100 may be in communication with controller 166 via one or more signal lines or shared communication busses.
During operation of washing machine appliance 100, laundry items are loaded into wash basket 120 through opening 132, and a washing operation (e.g., laundry cycle) is initiated through operator manipulation of input selectors 162. Wash tub 124 is filled with water, detergent, or other fluid additives, e.g., via spout 152 or detergent drawer 156. One or more valves (e.g., water supply valve 158) can be controlled by washing machine appliance 100 to provide for filling wash basket 120 to the appropriate level for the amount of articles being washed or rinsed. By way of example for a wash mode, once wash basket 120 is properly filled with fluid, the contents of wash basket 120 can be agitated (e.g., with ribs 128) for washing of laundry items in wash basket 120.
After the agitation phase of the wash cycle is completed, wash tub 124 can be drained. Laundry articles can then be rinsed by again adding fluid to wash tub 124, depending on the particulars of the cleaning cycle selected by a user. Ribs 128 may again provide agitation within wash basket 120. One or more spin cycles may also be used. In particular, a spin cycle may be applied after the wash cycle or after the rinse cycle in order to wring wash fluid from the articles being washed. During a final spin cycle, basket 120 is rotated at relatively high speeds and drain assembly 144 may discharge wash fluid from sump 142. After articles disposed in wash basket 120 are cleaned, washed, or rinsed, the user can remove the articles from wash basket 120, e.g., by opening door 134 and reaching into wash basket 120 through opening 132.
In some embodiments, one or more electronic attribute sensors 168 are included on or within appliance 100. Such sensors 168 may generally be configured to automatically detect one or more load attributes (e.g., of a load of articles or clothes to be treated by the appliance 100). Sensors 168 may include or be provided as weight sensors 168 (e.g., configured to detect a weight of articles within laundry appliance), speed sensors (e.g., configured to detect a rotational velocity or acceleration of basket), temperature sensors (e.g., configured to detect a temperature of air or wash fluid within cabinet 102), motor sensors (e.g., configured to detect an energy or power draw from the motor 122), camera sensors (e.g., configured to capture one or more images within or proximal to cabinet 102), or other suitable sensors in operable communication with (e.g., electrically or wirelessly coupled to) controller 166 and utilized in a laundry appliance to obtain load attribute or performance data for a laundry appliance, as would be understood.
In exemplary embodiments, at least electronic attribute sensor is provided with or as a camera assembly 170 having a camera 178 mounted to cabinet 102 (e.g., directly or indirectly, such as at door 134). As shown, camera assembly 170 is generally positioned and configured for obtaining images of wash chamber 126 or a load of articles (e.g., as identified schematically by reference numeral 172) within wash chamber 126 of washing machine appliance 100. Specifically, according to the illustrated embodiment, door 134 of washing machine appliance 100 comprises and inner window 174 that partially defines wash chamber 126 and an outer window 176 that is exposed to the ambient environment. According to the illustrated exemplary embodiment, camera assembly 170 includes a camera 178 that is mounted to inner window 174. Specifically, camera 178 is mounted such that is faces toward a bottom side of wash tub 124. In this manner, camera 178 can take images or video of an inside of wash chamber 126 and remains unobstructed by windows that may obscure or distort such images. Nonetheless, it is understood that camera 178 may be mounted or disposed at another suitable location (e.g., to capture images of a load of articles 172), as would be understood in light of the present disclosure.
Separate from or in addition to camera assembly 170, appliance 100 may include at least one electronic attribute sensor 168 that is provided with or as a basket speed sensor 186 (
Referring especially to
Communications assembly 190 permits controller 166 of washing machine appliance 100 to communicate with external devices either directly or through a network 192.
As noted above, appliance 100 includes a controller 166 in communication (e.g., electric or wireless communication) with various components (e.g., the motor 122, pump 146, sensor(s) 168, input(s) 162, etc.) of appliance 100. Controller 166 may include one or more processors and one or more storage or memory devices (i.e., memory). The one or more processors can be any suitable processing device (e.g., a processor core, a microprocessor, a CPU, an ASIC, a FPGA, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory device can include one or more non-transitory computer-readable storage mediums, such as RAM, DRAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., or combinations thereof. The memory device 116 may be a separate component from the processor or may be included onboard within the processor.
Generally, the storage or memory devices can store data and instructions (e.g., non-transitory programming instructions) that are executed by the processors to cause laundry appliance 100 to perform operations. In certain embodiments, the instructions include a software package configured to operate appliance 100 or execute an operation or routine (e.g., the exemplary method 400 described below with reference to
In some embodiments, appliance 100 includes a network interface 198 that couples laundry appliance 100 (e.g., controller 166) to a network 192 such that laundry appliance 100 can transmit and receive information over network 192. Network 192 can be any wired or wireless network such as a WAN, LAN, or HAN.
In some embodiments, controller 166 includes a network interface 198 such that laundry appliance 100 can connect to and communicate over one or more networks (e.g., network 192) with one or more network nodes. Network interface 198 can be an onboard component of controller 166 or it can be a separate, off board component. Controller 166 can also include one or more transmitting, receiving, or transceiving components for transmitting/receiving communications with other devices communicatively coupled with laundry appliance 100. Additionally or alternatively, one or more transmitting, receiving, or transceiving components can be located off board controller 166.
Network 192 can be any suitable type of network, such as a local area network (e.g., intranet), wide area network (e.g., internet), low power wireless networks [e.g., Bluetooth Low Energy (BLE)], radio field wireless networks [e.g., Near Field Communications (NFC) pairing], cellular communications network, or some combination thereof and can include any number of wired or wireless links. In general, communication over network 192 can be carried via any type of wired or wireless connection, using a wide variety of communication protocols (e.g., TCP/IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), or protection schemes (e.g., VPN, secure HTTP, SSL).
In some embodiments, the one or more remote servers 196 (e.g., web servers) are in operable communication with laundry appliance 100. The remote server 196 can be used to host a service platform or cloud-based application. Additionally or alternatively, remote server 196 can be used to host an information database (e.g., a machine-learned model, received data, or other relevant service data—optionally including intermediate processing data products). Remote server 196 can be implemented using any suitable computing device(s). Each remote server 196 generally includes a remote controller 166B having one or more processors and one or more memory devices (i.e., memory). The one or more processors can be any suitable processing device (e.g., a processor core, a microprocessor, a CPU, an ASIC, a FPGA, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory device can include one or more non-transitory computer-readable storage mediums, such as RAM, DRAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., or combinations thereof. The memory devices can store data and instructions (e.g., non-transitory programming instructions) that are executed by the processors to cause remote server 196 to perform operations. For example, instructions could be instructions for receiving/transmitting component signals (e.g., including data or information), analyzation results, machine-learned models, etc.
The memory devices of remote server 196 may also include data, such as data logs of appliance performance, analyzation results, machine-learned models, etc., that can be retrieved, manipulated, created, or stored by processors. The data can be stored in one or more databases. The one or more databases can be connected to remote server 196 by a high bandwidth LAN or WAN, or through one or more secondary networks. Optionally, the one or more databases can be split up so that they are located in multiple locales.
Additionally or alternatively, memory can store data that can be obtained (e.g., received, accessed, written, manipulated, generated, created, stored, etc.) for further analysis of appliance performance, such as data received from the electronic components, sensor data, processed sensor data, input data, output data, cycle history data, usage profile data, recorded fault data, fault table/sequence data, data indicative of machine-learned model(s) or other data/information described herein.
In some embodiments, remote controller 166B can store or include one or more machine-learned models. As examples, the machine-learned model(s) can be or can otherwise include various machine-learned models such as, for example, neural networks (e.g., deep neural networks, etc.), support vector machines, decision trees, ensemble models, k-nearest neighbors models, Bayesian networks, or other types of models including linear models or non-linear models. Example neural networks include feed-forward neural networks (e.g., convolutional neural networks, etc.), recurrent neural networks (e.g., long short-term memory recurrent neural networks, etc.), or other forms of neural networks. The machine-learned models of the remote server 196 may be used by the laundry appliance 100 (e.g., by analyzing received sensor data or images). Additionally or alternatively, remote server 196 can train the machine-learned models through use of a model trainer (e.g., training algorithm), as would be understood. Optionally, such a model trainer may train machine-learned models based on a set of training data compiled from a plurality of different appliances.
Remote server 196 includes a network interface 198B such that interactive remote server 196 can connect to and communicate over one or more networks (e.g., network 192) with one or more network nodes. Network interface 198B can be an onboard component or it can be a separate, off board component. In turn, remote server 196 can exchange data with one or more nodes over the network 192.
As noted above, a secondary appliance 194 may be in communication with laundry appliance 100. For instance, secondary appliance 194 may communicate directly with laundry appliance 100 via network 192 [e.g., via a low energy wireless connection, such as Bluetooth Low Energy (BLE)]. Alternatively, secondary appliance 194 can communicate indirectly with laundry appliance 100 by communicating via network 192 with remote server 196 (e.g., directly or indirectly through one or more intermediate remote servers 196), which in turn communicates with laundry appliance 100 via network 192.
Secondary appliance 194 is generally unattached to and spaced apart from laundry appliance (e.g., for independent movement relative to the same). In some embodiments, secondary appliance 194 includes or is provided as a robotic cleaner 214, such as a domestic robotic vacuum appliance, robotic mopping device, etc. Such robotic cleaner 214s are generally known and may include one more motors (e.g., for propelling the robotic cleaner 214, actuating one or more sweep/scrubber arms, activating a vacuum pump, etc.), as well as one or more inputs (e.g., buttons, knobs, or touchscreens for receiving user commands). Additionally, one or more internal cleaner sensors may be provided, such as for guiding, mapping, or redirecting the robotic cleaner 214 during cleaner operations.
In additional or alternative embodiments, secondary appliance 194 includes or is provided as an air conditioner appliance. Such air conditioner appliances are generally known and may include one or more motors (e.g., for fans, compressors, etc.) and heat exchangers, as well as one or more inputs (e.g., buttons, knobs, or touchscreens for receiving user commands). Additionally, one or more internal AC sensors may be provided, such as for detecting air or refrigerant temperatures, air speed, etc. during conditioning operations.
Secondary appliance 194 can include one or more device controllers 166C. Device controller 166C can include one or more processors and one or more memory devices. The one or more processors can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory device (i.e., memory) can include one or more non-transitory computer-readable storage mediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory can store data and instructions which are executed by the processor to cause secondary appliance 194 to perform secondary operations (e.g., independent cleaning, vacuuming, or mopping operations). Device controller 166C may include a network interface 198C such that secondary appliance 194 can connect to and communicate over one or more networks (e.g., network 192) with one or more network nodes. Network interface 198C can be an onboard component of device controller 166C or it can be a separate, off board component. Device controller 166C can also include one or more transmitting, receiving, or transceiving components for transmitting/receiving communications with other devices communicatively coupled with secondary appliance 194. Additionally or alternatively, one or more transmitting, receiving, or transceiving components can be located off board for device controller 166C.
External communication system 190 is described herein according to an exemplary embodiment of the present subject matter. However, it should be appreciated that the exemplary functions and configurations of external communication system 190 provided herein are used only as examples to facilitate description of aspects of the present subject matter. System configurations may vary, other communication devices may be used to communicate directly or indirectly with one or more laundry appliances, other communication protocols and steps may be implemented, etc. These variations and modifications are contemplated as within the scope of the present subject matter.
Now that the construction of a laundry appliance (e.g., appliance 100 (and the configuration of controller 166 according to exemplary embodiments have been presented, an exemplary method 400 of operating a laundry appliance will be described. In exemplary embodiments, the various method steps as disclosed herein may be performed by controller 166 or a separate, dedicated controller.
Advantageously, methods within the scope of the present disclosure may provide improved coordination between household appliances (e.g., without relying on or needing direct user intervention or direction). Additionally or alternatively, one or more regions of a residence or user-defined space may be advantageously and efficiently cleaned.
Turning especially to
In some embodiments, the load attribute is provided, at least in part, directly by a user (e.g., at the control panel or user interface of the laundry appliance). Thus, 410 may include receiving a user-specified attribute input from a user interface of the laundry appliance. Such an attribute input may be included, for instance, with the initial inputs provided to the laundry appliance to start or initiate a laundry cycle, such as to wash or dry the load of articles or clothes.
In additional or alternative embodiments, the load attribute is provided, at least in part, automatically (e.g., without direct user interaction or direction) based on one or more sensor signals. Thus, 410 may include receiving a detection signal from one or more electronic attribute sensors. Once received, the detection signal may be further analyzed, such as to determine one or more load attributes
As an example, one or more detection signals may include or be provided as image signals obtained from the camera assembly. In some such embodiments, 410 includes obtaining one or more images from the camera assembly. For instance, the camera may be aimed at a front of the laundry appliance or the wash basket. In addition to the cabinet or basket of the laundry appliance, such images may include a load of articles that are to be washed during a wash cycle of a laundry appliance. In this regard, continuing the example from above, load of articles may be placed within chamber of the laundry appliance prior to closing the door and implementing a laundry (e.g., wash) cycle.
It should be appreciated that obtaining the images may include obtaining more than one image, a series of frames, a video, or any other suitable visual representation of the load of articles using the camera assembly. Thus, 410 may include receiving a video signal from the camera assembly. Separate from or in addition to the video signal, the images obtained by the camera assembly may vary in number, frequency, angle, resolution, detail, etc. in order to improve the clarity of the load of articles. In addition, the obtained images may also be cropped in any suitable manner for improved focus on desired portions of the load of articles.
The one or more images may be obtained using the camera assembly at any suitable time prior to completing the wash cycle. As an example, these images may be obtained when the door is in the open position (e.g., such that the field of view of the camera can capture at least a portion of the wash chamber through the front opening). As an alternative example, these images may be obtained when the door is in the closed position (e.g., wherein the camera assembly is positioned within the cabinet such that the field of view of the camera can capture at least a portion of the wash chamber).
Once obtained, the images may be further analyzed. Optionally, 410 may thus include analyzing one or more obtained images using a machine learning image recognition process to estimate a load attribute of the load of articles within the washing machine appliance based on the analysis. As noted above, it should be appreciated that the load attribute may be an approximation or best fit representation of a load of articles. For example, a controller may be programmed with thresholds for determining whether a load qualifies as a white load, such as greater than 70% whites, greater than 80% whites, greater than 90% whites, greater than 95% whites, etc.
As used herein, the terms image recognition, object detection, and similar terms may be used generally to refer to any suitable method of observation, analysis, image decomposition, feature extraction, image classification, etc. of one or more image or videos taken within a wash chamber of a washing machine appliance. It should be appreciated that any suitable image recognition software or process may be used to analyze images taken by the camera assembly and a controller may be programmed to perform such processes and take corrective action.
In certain embodiments, the image analysis may include utilizing artificial intelligence (“AI”), such as a machine learning image recognition process, a neural network classification module, any other suitable artificial intelligence (AI) technique, or any other suitable image analysis techniques, examples of which will be described in more detail below. Moreover, each of the exemplary image analysis or evaluation processes described below may be used independently, collectively, or interchangeably to extract detailed information regarding the images being analyzed to facilitate performance of one or more methods described herein or to otherwise improve appliance operation. According to exemplary embodiments, any suitable number and combination of image processing, image recognition, or other image analysis techniques may be used to obtain an accurate analysis of the obtained images.
In this regard, the image recognition process may use any suitable artificial intelligence technique, for example, any suitable machine learning technique, or for example, any suitable deep learning technique. According to an exemplary embodiment, controller may implement a form of image recognition called region based convolutional neural network (“R-CNN”) image recognition. Generally speaking, R-CNN may include taking an input image and extracting region proposals that include a potential object, such as an item of clothing (e.g., jeans, socks, etc.) or an undesirable article (e.g., a belt, a wallet, etc.). In this regard, a “region proposal” may be regions in an image that could belong to a particular object. A convolutional neural network is then used to compute features from the regions proposals and the extracted features will then be used to determine a classification for each particular region.
According to still other embodiments, an image segmentation process may be used along with the R-CNN image recognition. In general, image segmentation creates a pixel-based mask for each object in an image and provides a more detailed or granular understanding of the various objects within a given image. In this regard, instead of processing an entire image—i.e., a large collection of pixels, many of which might not contain useful information—image segmentation may involve dividing an image into segments (e.g., into groups of pixels containing similar attributes) that may be analyzed independently or in parallel to obtain a more detailed representation of the object or objects in an image. This may be referred to herein as “mask R-CNN” and the like. It should be appreciated that any other suitable image recognition process may be used while remaining within the scope of the present subject matter.
According to still other embodiments, the image recognition process may use any other suitable neural network process. For example, 410 may include using Mask R-CNN instead of a regular R-CNN architecture. In this regard, Mask R-CNN is based on Fast R-CNN which is slightly different than R-CNN. For example, R-CNN first applies CNN and then allocates it to zone recommendations on the covn5 property map instead of the initially split into zone recommendations. In addition, according to exemplary embodiments standard CNN may be used to analyze the image and estimate load size or main load fabric type of the load within the wash basket. In addition, a K-means algorithm may be used for dominant color analysis to find individual color of fabrics to serve with warnings.
According to exemplary embodiments the image recognition process may further include the implementation of Vision Transformer (ViT) techniques or models. In this regard, ViT is generally intended to refer to the use of a vision model based on the Transformer architecture originally designed and commonly used for natural language processing or other text-based tasks. For example, ViT represents an input image as a sequence of image patches and directly predicts class labels for the image. This process may be similar to the sequence of word embeddings used when applying the Transformer architecture to text. The ViT model and other image recognition models described herein may be trained using any suitable source of image data in any suitable quantity. Notably, ViT techniques have been demonstrated to outperform many state-of-the-art neural network or artificial intelligence image recognition processes.
According to still other embodiments, the image recognition process may use any other suitable neural network process while remaining within the scope of the present subject matter. For example, the step of analyzing the one or more images may include using a deep belief network (“DBN”) image recognition process. A DBN image recognition process may generally include stacking many individual unsupervised networks that use each network's hidden layer as the input for the next layer. According to still other embodiments, the step of analyzing one or more images may include the implementation of a deep neural network (“DNN”) image recognition process, which generally includes the use of a neural network (computing systems inspired by the biological neural networks) with multiple layers between input and output. Other suitable image recognition processes, neural network processes, artificial intelligence analysis techniques, and combinations of the above described or other known methods may be used while remaining within the scope of the present subject matter.
In addition, it should be appreciated that various transfer techniques may be used but use of such techniques is not required. If using transfer techniques learning, a neural network architecture may be pretrained such as VGG16/VGG19/ResNet50 with a public dataset then the last layer may be retrained with an appliance specific dataset. In addition, or alternatively, the image recognition process may include detection of certain conditions based on comparison of initial conditions, may rely on image subtraction techniques, image stacking techniques, image concatenation, etc. For example, the subtracted image may be used to train a neural network with multiple classes for future comparison and image classification.
It should be appreciated that the machine learning image recognition models may be actively trained by the appliance with new images, may be supplied with training data from the manufacturer or from another remote source, or may be trained in any other suitable manner. For example, according to exemplary embodiments, this image recognition process relies at least in part on a neural network trained with a plurality of images of the appliance in different configurations, experiencing different conditions, or being interacted with in different manners. This training data may be stored locally or remotely and may be communicated to a remote server for training other appliances and models. According to exemplary embodiments, it should be appreciated that the machine learning models may include supervised or unsupervised models and methods. In this regard, for example, supervised machine learning methods (e.g., such as targeted machine learning) may help identify problems, anomalies, or other occurrences which have been identified and trained into the model. By contrast, unsupervised machine learning methods may be used to detect clusters of potential failures, similarities among data, event patterns, abnormal concentrations of a phenomenon, etc.
It should be appreciated that image processing and machine learning image recognition processes may be used together to facilitate improved image analysis, object detection, color detection, or to extract other useful qualitative or quantitative data or information from the one or more images that may be used to improve the operation or performance of the appliance. Indeed, the methods described herein may use any or all of these techniques interchangeably to improve image analysis process and facilitate improved appliance performance and consumer satisfaction. The image processing algorithms and machine learning image recognition processes described herein are only exemplary and are not intended to limit the scope of the present subject matter in any manner.
At 420, the method 400 includes initiating a laundry cycle. For instance, a wash cycle or dry cycle may be prompted (e.g., before or after 410), such as would generally be understood in light of the present disclosure. Such a cycle may optionally be based on the detected load attribute or one or more user inputs.
In exemplary embodiments, 420 includes directing a laundry (e.g., wash) cycle within the washing machine appliance based on the estimated load attribute (e.g., from 410). Such direction may require adjusting one or more operating parameters of the washing machine appliance (e.g., as part of the wash cycle, which may then be initiated). Thus, 420 may include selecting an operating cycle parameter, adjusting a water or detergent fill amount, etc. As used herein, an “operating parameter” of the washing machine appliance is any cycle setting, operating time, component setting, spin speed, part configuration, or other operating characteristic that may affect the performance of the washing machine appliance. In turn, references to operating parameter adjustments or “adjusting at least one operating parameter” are intended to refer to control actions intended to improve system performance based on the load characteristics. For example, adjusting an operating parameter may include adjusting an agitation time or an agitation profile, adjusting a water level, limiting a spin speed of the wash basket, etc. Other operating parameter adjustments are possible and within the scope of the present subject matter.
For example, according to an exemplary embodiment, the mask R-CNN image recognition process may be used on one or more images obtained at 410 to determine that the load of clothes is primarily delicate garments. As a result, it may further be determined that cool water should be used (e.g., below a certain temperature), that the agitation profile should be gentle, and that the total wash time should be decreased. One or more of the corresponding controllers may automatically detect and implement such a wash cycle without requiring user input. By contrast, if a load of sheets or towels is detected, a large volume of hot water may be used with more detergent and an aggressive agitation profile. It should be appreciated that the exemplary load characteristics and the exemplary operating parameters described herein are only exemplary and not intended to limit the scope of the present subject matter in any manner.
At 430, the method 400 includes directing a load signal to the secondary appliance. Such a load signal may be directed as a transmission, for instance, subsequent to 410 or 420. In some embodiments, 430 includes transmitting a load signal corresponding to the detected load attribute (e.g., a load signal having data that indicates or details the detected load attribute for the secondary appliance). Generally, the transmitted load signal may be configured to prompt or initiate a particular secondary operation at the secondary appliance, such as that described below with respect to 440.
At 440, the method 400 includes initiating a secondary operation at the secondary appliance (e.g., based on the load signal and, thus, load attribute). Specifically, the secondary appliance may be prompted or directed to execute the secondary operation at a predefined region (e.g., of the building, home, or residence within which the laundry appliance or secondary appliance are installed). For instance, the predefined region may generally correspond to an area, zone, or room in which the secondary appliance is to move (e.g., under its own power) and execute the secondary operation. As noted above, the secondary appliance may be provided as a robotic cleaner, such as a vacuum. In turn, the secondary operation may include a cleaning operation, such as a vacuum operation. Additionally or alternative, the secondary appliance may be provided as an air conditioner appliance. In turn, the secondary operation may include a conditioning operation.
Optionally, the predefined region may be configured to limit or focus the secondary operation to a region in which the laundry articles being treated have been or will be. Thus, the predefined region may be less than the total area of a residence or set cleaning region of the secondary appliance (e.g., of a scheduled secondary/cleaning/conditioning operation). As an example, if the load attribute indicates a bulk or towel load, the secondary appliance may be directed to execute a secondary cleaning operation within a designated bathroom in which towels are predetermined or predicted to be stored within. As an additional or alternative example, if the load attribute indicates a large or bedding load, the secondary appliance may be directed to execute a secondary cleaning operation within one or more designated bedrooms in which bedding or sheets are predetermined or predicted to be stored within. As another additional or alternative example, if the load attribute indicates articles predetermined to require air drying or to release large volumes of moisture, the secondary appliance may be configured to initiate a humidity reduction or cooling AC operation.
In certain embodiments, the predefined region is programmed within the controller of the laundry appliance or secondary appliance. For instance, the predefined region may a user-defined region spaced apart from the laundry appliance corresponding to a location of laundry storage. Thus, the secondary appliance may prepare (e.g., clean) the region in which articles are likely to be stored. Additionally or alternatively, the predefined region may extend proximal to or directly in front of the laundry appliance. Thus, the secondary appliance may clean an area in which articles are likely to have been moved or temporarily placed (e.g., by a user) prior to being loaded to the laundry appliance.
Upon completion of the secondary operation at the predefined region, the secondary operation may be halted, such as by directing the secondary appliance back to a “home” or docking station or returning the secondary appliance to a programmed scheduled operation. In some embodiments, 440 overlaps with at least a portion of the laundry cycle. Thus, the secondary operation may be in tandem with the laundry cycle. Optionally, the secondary operation may be configured to finish during the laundry cycle (e.g., to provide the secondary appliance back at the “home” or docking station or to return the secondary appliance back to a scheduled operation before the laundry cycle ends).
This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.
