Patent Application
20250151970
The present invention relates to a system with a water-bearing household appliance and to a method for operating a water-bearing household appliance.
Known water-bearing household appliances, for example dishwashers, typically have a number of treatment programs, like cleaning programs or washing programs for washing items, like dishes.
Conventionally, treatment programs are developed for a huge plurality of household appliances. For example, a household appliance manufacturer develops treatment programs and stores the developed treatment programs in a memory of the household appliance. In operation, the user of the household appliance may select one of the predefined and pre-stored treatment programs. But, the users or consumers of household appliances are very different in their habits when using a household appliance. The predefined treatment programs cannot cover these different habits of the plurality of different users, disadvantageously. Moreover, conventional treatment programs are represented by a manually and explicitly generated program code that is particularly generated and delivered during development.
It is one objective of the invention to provide an improved water-bearing household appliance.
According to a first aspect, a system with a water-bearing household appliance, in particular a dishwasher, is suggested. The system comprises:
For example, the water-bearing household appliance is implemented as a dishwasher or a washing machine. The treatment program may be a cleaning program or washing program for washing items. For example, washing items are items to be washed or rinsed, e.g. cutlery, plates, pots and the like.
Advantageously, the present system has the ability to provide adapted treatment programs. Because the present system uses rewards being dependent on observation results and deep reinforcement learning process (DRL), the provided adapted treatment programs are user-specific and user-tailored. In particular, the use of deep reinforcement learning process provides an adapted, intelligent, self-learning and consumer-tailored mechanism for adapting treatment programs. This new type of treatment program, i.e. the adapted treatment program, is no longer generated by empirical procedures, like expert knowledge and trial and error, as such.
Using the present mechanism including deep reinforcement learning, the present self-learning system is able to keep the device process so dynamic that it optimally adapts to the needs of the costumer during use. Using the present scheme, each household appliance may develop independently over its entire product lifetime, in particular it may adapt to specific habits of the place the specific household appliance is used. Deep reinforcement learning (deep RL) combines reinforcement learning (RL) and deep learning.
During executing of the certain treatment program, detergents, e.g. detergent tablets, may be used. The detergents preferably comprise one or more active ingredients for an automatic cleaning or washing process. As will be appreciated by the skilled person, the nature of the active ingredient(s) used in the detergents will vary depending on the desired application. When used inside a dishwasher, the detergents may, for example, comprise an active ingredient performing a dishwasher detergent, rinse aid, machine cleaner or dishwasher deodorizing function or any further additional chemistry which supports the cleaning process, or further physical or chemical processes. In the context of laundry washing machines, the detergents may, for example, comprise an active ingredient performing a laundry detergent or fabric softener function. Suitable active ingredients are known to the skilled person; examples include bleach, bleach activator, bleach catalyst, enzyme, surfactant, builder, pH-adjusting agent, corrosion inhibitor, and fragrance.
According to an embodiment, the deep reinforcement learning process is configured to adapt the treatment policy using the provided reward as input.
According to a further embodiment, the treatment policy includes a vector of treatment results being a function of a vector of operation parameters of the treatment program. In particular, the vector of treatment results may include a plurality of vector components. For example, the vector components include a first vector component for a desired cleaning result, a second vector component for a desired drying result, a third vector component for a desired runtime result, and/or a fourth vector component for a desired energy consumption for the treatment program.
Each of the vector components may include or may be represented by a certain interval. For example, a desired drying result may be represented by the interval 92% to 95%.
According to a further embodiment, the operation parameters of the treatment program include a temperature of the water in a washing chamber of the household appliance, a pump speed of a pump of the household appliance, an amount of water in the washing chamber, a commodities amount of the certain treatment program, particularly including a detergent amount of the certain treatment program, a rinsing agent amount of the certain treatment program, a salt amount of the certain treatment program, and/or a fragrance amount of the certain treatment program, a number of water changes of the water in the washing chamber during the treatment program, a control parameter for the regeneration of the water softener, and/or a control parameter of the share between softened and tap water.
According to a further embodiment, the adjustable program parameters include a program duration, a cleaning intensity and/or a drying intensity.
According to a further embodiment, the system includes a user interface for adjusting the adjustable program parameters by a user.
According to a further embodiment, the treatment result includes a runtime result, a cleaning result and/or a drying result.
According to a further embodiment, the sub-program steps of the respective treatment program include pre-rinsing, cleaning, rinsing and/or drying. In particular, the sub-program steps are executed sequentially. In embodiments, the sub-program steps of the respective treatment program include at least two different temperatures. In particular, the at least two different temperatures are all above the ambient temperature of the household appliance.
According to a further embodiment, the observation unit is configured to provide the observation result including performance parameters and/or consumption parameters of performance and/or consumption of the household appliance during the execution of the certain treatment program.
In embodiments, the performance parameters include:
In embodiments, the consumption parameters particularly include:
According to a further embodiment, the reward includes a runtime reward, a cleaning reward, a drying reward, a reward for removing spots at washing items being washed by the certain treatment program, a reward for a hygiene of the certain treatment program, a reward for the acoustics of the certain treatment program, a reward for a glass corrosion of glass of the washing items, a reward for a power consumption of the certain treatment program, a reward for a water consumption of the certain treatment program, a reward for a detergent amount of the certain treatment program, and/or a reward for a CO2-consumption of the certain treatment program.
According to a further embodiment, the providing unit is configured to provide the adapted treatment program using the treatment policy, the deep reinforcement learning process and environment data for the household appliance.
In embodiments, the environment data include user data associated to the household appliance, sensor data associated to the household appliance, test data generated by testing the household appliance, simulation data generated by simulating the household appliance using a digital twin of the household appliance, and/or environmental data describing a local environment of the household appliance, particularly including temperature and humidity.
According to a further embodiment, the providing unit is configured to provide the adapted treatment program using the observation result provided by the observation unit and a status information indicating a current status of the household appliance additionally. Using this addition information, the provision of adapted treatment programs for a specific user of a specific household appliance may be further improved.
According to a further embodiment, the household appliance includes the control device, the observation unit, the interpreter unit, the providing unit and the receiver unit. In this embodiment, the control device may integrate the observation unit, the interpreter unit, the providing unit and the receiver unit.
According to a further embodiment, the system comprises the household appliance and an agent device being external to the household applicant, wherein the household appliance integrates the control device, the observation unit, the interpreting unit and the receiver unit, and wherein the agent device integrates the providing unit. In this embodiment, the control device of the household appliance may integrate the observation unit, the interpreter unit and the receiver unit.
According to a further embodiment, the system further comprises a checking unit, the checking unit being configured to check if the reward provided by the interpreter unit reaches a first predefined threshold or not, wherein the checking unit is particularly configured to trigger the deep reinforcement learning process with the reward if said reward is below the first predefined threshold.
According to a further embodiment, the checking unit is configured to calculate a ratio between a difference of the provided reward and the first predefined threshold and a number of deep reinforcement learning processes applied to the certain treatment program for determining a progress of learning, wherein the checking unit is further configured to adapt the treatment policy and/or the deep reinforcement learning process, if the calculated ratio is greater than a second predefined threshold. In particular, the second predefined threshold is determined by a threshold function or by a target function.
According to a second aspect, a computer-implemented method for operating a water-bearing household appliance, in particular a dishwasher, is proposed. The method includes:
In particular, the respective sub-program includes at least one water change.
In embodiments, the step of providing an adapted treatment program may include receiving an adapted treatment program by a receiver unit of the household appliance and transferring the received adapted treatment program to the control device of the household appliance for execution.
The embodiments and features according to the first aspect are also embodiments of the second aspect.
According to a third aspect, a computer program product is proposed, the computer program product comprising machine readable instructions, that when executed by one or more processing units, cause the one or more processing units to perform the method of the second aspect or of any embodiment of the second aspect.
A computer program product, such as a computer program means, may be embodied as a memory card, USB stick, CD-ROM, DVD or as a file which may be downloaded from a server in a network. For example, such a file may be provided by transferring the file comprising the computer program product from a wireless communication network.
According to a fourth aspect, a computer readable medium is proposed on which program code sections of a computer program are saved, the program code sections being loadable into and/or executable in a system to make the system execute the method of the second aspect or of any embodiment of the second aspect when the program code sections are executed in the system.
The embodiments and features according to the first aspect are also embodiments of the fifth aspect.
According to a fifth aspect, a computer-implemented device for operating a water-bearing household appliance, in particular a dishwasher, is proposed, the computer-implemented device comprising:
The respective unit, for example the processing unit, the observation unit, the interpreter unit and the providing unit, may be implemented in hardware and/or in software. When implemented in hardware, the respective unit may be implemented as a computer, a CPU (central processing unit), an ASIC (application specific integrated circuit) or a PLC (programmable logic controller). When implemented in software, the respective unit may be configured as a computer program product, a function, an algorithm, a routine, as part of a programming code or as an executable object
Further possible implementations or alternative solutions of the invention also encompass combinations—that are not explicitly mentioned herein—of features described above or below with regard to the embodiments. The person skilled in the art may also add individual or isolated aspects and features to the most basic form of the invention.
Further embodiments, features and advantages of the present invention will become apparent from the subsequent description and dependent claims, taken in conjunction with the accompanying drawings, in which:
In the Figures, like reference numerals designate like or functionally equivalent elements, unless otherwise indicated.
The system 100 of
The control device 15 is adapted to execute a certain treatment program from a plurality of treatment programs. A treatment program may be a cleaning program or a washing program for washing items. Washing items are items to be washed or rinsed, like cutlery, plates, pots and the like, for example. Each of the treatment programs has a number of sub-programs and a number of water changes. Moreover, the sub-program steps of the respective treatment program particularly include at least two different temperatures. Moreover, the sub-program steps of the respective treatment program include pre-rinsing, cleaning, rinsing and/or drying. In particular, the sub-program steps are executed sequentially.
Moreover, each of the treatment programs is determined by a number of program parameters. The program parameters are particularly adjustable program parameters, wherein a user may adjust them. The adjustable program parameters particularly include a program duration, a cleaning intensity and/or a drying intensity. For adjusting the adjustable program parameters, the system 100 may include a user interface (not shown).
The observation unit 16 may be coupled to the control device 15. The observation unit 16 is adapted to provide an observation result O by observing the execution of the certain treatment program. In particular, the observation unit 16 is adapted to provide the observation result including performance parameters and/or consumption parameters of performance and/or consumption of the dishwasher 1 during the execution of the certain treatment program.
In this regard, the performance parameters may include a parameter indicating a cleaning result of the certain treatment program, a parameter indicating a drying result of the certain treatment program, a parameter indicating a runtime result of the certain treatment program, a parameter indicating spots at washing items being washed by the certain treatment program, a parameter indicating a hygiene of the certain treatment program, a parameter indicating the acoustics of the certain treatment program, and/or a parameter indicating a glass corrosion of glass of the washing items.
Moreover, the consumption parameters particularly include a parameter indicating a power consumption of the certain treatment program, a parameter indicating a water consumption of the certain treatment program, a parameter indicating a detergent amount of the certain treatment program, and/or a parameter indicating a CO2-consumption of the certain treatment program.
The observation unit 16 may be coupled to the interpreter unit 17. The interpreter unit 17 is adapted to provide a reward R by interpreting the provided observation result O. In particular, the reward R includes a runtime reward R1, a cleaning reward R2 and a drying reward R3.
Additionally, the reward R may include a reward for removing spots at washing items being washed by the certain treatment program, a reward for a hygiene of the certain treatment program, a reward for the acoustics of the certain treatment program, a reward for a glass corrosion of glass of washing items, a reward for a power consumption of the certain treatment program, a reward for a water consumption of the certain treatment program, a reward for a detergent amount of the certain treatment program, and/or a reward for a CO2-consumption of the certain treatment program.
The interpreter unit 17 may be coupled to the providing unit 18. The providing unit 18 is configured to provide an adapted treatment program AT by adapting the certain treatment program using a treatment policy TP and a deep reinforcement learning process DRL, said deep reinforcement learning DRL having the provided reward R as input. In particular, the deep reinforcement learning process DRL is configured to adapt the treatment policy TP using the provided reward R as input. The treatment policy TP may include a vector of treatment results being a function of a vector of operation parameters of the treatment program. In particular, the vector of treatment results may include a plurality of vector components. For example, the vector components may include a first vector component for a desired cleaning result, a second vector component for a desired drying result, a third vector component for a desired runtime result, and/or a fourth vector component for a desired energy consumption for the treatment program. The treatment result may include a runtime result, a cleaning result and/or a drying result.
Moreover, the operation parameters of the treatment program may include the temperature of the water in a washing chamber 4 (see
In particular, the providing unit 18 is configured to provide the adapted treatment program AT using the treatment policy TP, the deep reinforcement learning process DRL and environment data for the dishwasher 1. In particular, the environment data includes user data associated to the dishwasher 1, sensor data associated to the dishwasher 1, test data generated by testing the dishwasher 1, simulation data generated by simulating the dishwasher 1 using a digital twin of the dishwasher 1, and/or environmental data describing a local environment of the dishwasher 1. For example, the environmental data includes temperature and humidity.
The providing unit 18 is coupled, in particular temporarily coupled, to the receiver unit 19. The receiver unit 19 is configured to receive the adapted treatment program AT. Moreover, the receiver unit 19 provides the received adapted treatment program AT to the control device 15. Then, the control device 15 executes the received adapted treatment program AT.
In embodiments, as exemplarily shown in
Further, the domestic dishwasher 1 of
In
The domestic dishwasher 1 further has at least one rack 12, 13, 14 on which dishes to be washed can be placed. Preferably, more than one rack 12, 13, 14 is used, wherein rack 12 can be lower rack, rack 13 can be an upper rack and rack 14 can be a rack specific for cutlery. As is shown in
Furthermore,
The checking unit 20 is configured to check if the reward R provided by the interpreter unit 17 reaches a first predefined threshold or not. If said reward R is below the first predefined threshold, the checking unit 20 triggers the deep reinforcement learning process DRL with the reward R.
Moreover, the checking unit 20 may calculate a ratio between a difference of the provided reward R and the first predefined threshold and a number of deep reinforcement learning processes DRL applied to the certain treatment program for determining a progress of learning. The checking unit 20 is further configured to adapt the treatment policy TP and/or the deep reinforcement learning process DRL, if the calculated ratio is greater than a second predefined threshold.
In step S1, a certain treatment program from a plurality of treatment programs is executed, each of the treatment programs having a number of sub-programs and a number of water changes and is determined by a number of program parameters.
In step S2, the execution of the certain treatment program is observed for providing an observation result O.
In step S3, a reward R is provided by interpreting the provided observation result O.
In step S4, an adapted treatment program AT is provided by adapting the certain treatment program using a treatment policy TP and a deep reinforcement learning process DRL. The deep reinforcement learning process DRL has the provided reward R as an input.
In step S5, the adapted treatment program AT is executed by the household appliance 1.
In embodiments, the step S4 of providing an adapted treatment program AT may include receiving an adapted treatment program AT by the receiver unit 19 of the household appliance 1 and transferring the received adapted treatment program AT to the control device 15 of the household appliance 1 for execution.
Although the present invention has been described in accordance with preferred embodiments, it is obvious for the person skilled in the art that modifications are possible in all embodiments.
| Number | Date | Country | Kind |
|---|---|---|---|
| 22159151.4 | Feb 2022 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2022/083506 | 11/28/2022 | WO |
