Patent Application
20190059133
Numerous trends in modern technology have led to the widespread adoption of machine learning and artificial intelligence algorithms across a broad spectrum of industries, but the application of artificial intelligence to certain sectors is sometimes held up by a lack of available training data. A human brain can learn to recognize a pattern or conduct a classification task with a limited amount of training data. For example, a human brain can learn what a steak looks and feels like when it is fully cooked after just a few samples. However, modern machine intelligence algorithms are lacking in this regard and require large volumes of digitized training data before they are fully trained for a given task. As a result, fields where training data is not readily available in digital form present a stumbling block for machine intelligence applications.
In certain applications, the lack of available training data can be overcome because the training data itself can be easily generated using a computer. For example, using an evaluative feedback approach to train an artificial intelligence to play chess can generate training data for the artificial intelligence as fast as a move in the chess game can be digitally modeled. As entire chess matches can be simulated in milliseconds, generating a large volume of training data for the artificial intelligence is a trivial task. However, training an artificial intelligence using a training environment that includes physical reactions taking place in the real world is not as easy. For example, to train an artificial intelligence how to cook an item, the item must actually be cooked, which is a task that occurs on the scale of minutes instead of milliseconds. These applications are therefore thousands of times slower than those that exist in a purely digital realm.
Furthermore, in some applications, the training data generated may be an incomplete picture of the environment. In other words, what is observable in the training data might not fully capture what is happening with the actual process that the machine intelligence is trying to learn. For example, heating an item in an oven and using the surface temperature as training data suffers from not knowing what is happening on the inside of the item. The same observed condition for an item could reflect two totally different real, but unobserved, states for the system. This is referred to as the hidden states problem, and can be difficult to overcome when a limited number of sensors are used to interrogate the condition of a real-world process.
For the above reasons, training an artificial intelligence to conduct actions that are constrained by physical reactions such as cooking food is difficult because of a lack of accurate and readily available training data.
Various methods and systems that relate to electronic ovens and energy responsive sensors within electronic ovens are disclosed.
One such system is an electronic oven that has a control system with a machine learning system. The electronic oven also includes a set of radio frequency (RF) responsive sensors that obtain training data for the machine learning system. The electronic oven has a heating chamber and an energy source that is coupled to an injection port in the heating chamber for introducing an application of energy into the chamber. The control system adjusts a distribution of the application of energy in the heating chamber. Each RF responsive sensor in the set of RF responsive sensors directly responds to the application of energy in the heating chamber. The control system uses data from each RF responsive sensor in the set of RF responsive sensors to create a set of training data. The control system uses the set of training data to train the machine learning system.
One such method is a method for training a machine learning system for an electronic oven. The method includes applying an application of energy to a heating chamber of the electronic oven from an energy source via an injection port in the heating chamber. The method also includes adjusting a distribution of the application of energy in the heating chamber using a control system. The method also includes sensing the distribution of the application of energy in the heating chamber using a set of RF responsive sensors. The set of RF responsive sensors are located in the heating chamber and include at least two elements. The method also includes creating a set of training data for the machine learning system using data from each RF responsive sensor in the set of RF responsive sensors. The method also includes training the machine learning system using the set of training data.
A system of sensors and associated methods are disclosed that improve the functionality of an intelligent electronic oven. The disclosed approaches improve the functionality of an electronic oven that is augmented with a machine learning system by directly observing a distribution of energy in the electronic oven and using the obtained data to train or guide a machine intelligence system in the intelligent electronic oven. Some of the approaches disclosed herein can be beneficially applied to electronic ovens that alter the distribution of energy within their heating chambers by altering their configurations between a large number of potential configurations. The electronic ovens of
The system described with reference to
Electronic oven 110 provides control system 114 with a more direct measurement of the distribution of energy in the chamber than electronic oven 100. Specifically, one that is not masked by an observation of the item's idiosyncratic heating response. Furthermore, if the energy responsive sensors are placed proximate to the item, they can provide an estimate of how much energy is being applied to the item. Energy 116 is applied to the chamber, is sensed directly by energy responsive sensors 113, and generates data 117 that is provided to the control system. As such, training data regarding the distribution of energy in the chamber can be gathered more quickly and more accurately in electronic oven 110 than in electronic oven 100. Indeed, electronic oven 110 could be run without an item located in the chamber at all such that the data observed was used only to inform control system 114 as to how the various configurations of the oven varied the distribution of energy in the chamber.
The approaches of electronic oven 100 and 110 were described separately to accentuate the benefit of observing the distribution of energy in the chamber directly. However, the approach of electronic oven 110 can be augmented with a method for observing the response of the item to heat such as by utilizing both data 106 and 117. Machine intelligence applications often improve by being provided with numerous types of data regarding the task they are being trained for. As a basic example, an electronic oven could include both illustrated feedback paths (i.e., those carrying data 106 and 117). The training data for the machine intelligence system would then include both: (i) data concerning the response of the item to an application of energy; and (ii) a direct measurement of the distribution of that application of energy. Indeed, the data obtained by the set of energy responsive sensors can be combined with data obtained from a multitude of sensor and control sources to train and assist the machine intelligence system of an intelligent electronic oven.
Electronic ovens in accordance with this disclosure can include control systems that adjust the distribution of energy in the chamber in various ways. For example, electronic oven 110 includes an energy source, such as a magnetron, that is coupled to injection port 112 in heating chamber 111. The magnetron can be coupled to the injection port using a waveguide. The energy source introduces RF energy into heating chamber 111. The electronic oven could also be configured to adjust the distribution of energy in the heating chamber using control system 114. The intensity of the application of energy can be fixed by, or varied using, control system 114. For instance, the energy source could be augmented with inverter technology to vary the amplitude of the electromagnetic waves introduced to the chamber. The frequency of the electromagnetic waves introduced to the chamber could also be fixed or variant. For example, the application of energy can be microwave energy fixed and centered at a frequency of either 2.45 GHz or 915 MHz. Control system 114 could adjust the distribution of energy in the chamber by altering the frequency and/or intensity of the application of energy. Control system 114 could adjust the frequency of the electromagnetic waves to target interior portions of the item in the chamber.
The control system of an electronic oven in accordance with certain approaches in this disclosure could also adjust the distribution of energy in the chamber by altering the state of a set of variable reflectance elements in the chamber. Examples of such approaches are described in U.S. patent application Ser. No. 15/619,390, filed Jun. 9, 2017, and entitled “Electronic Oven with Reflective Energy Steering,” 62/434,179, filed Dec. 14, 2016, and entitled “Electronic Oven with Reflective Energy Steering,” and 62/349,367, filed Jun. 13, 2016, and entitled “Electronic Oven with Reflective Beam Steering Array,” all of which are incorporated by reference herein in their entirety for all purposes. The manner in which the variable reflectance elements in the chamber were altered could be controlled by a machine intelligence system such as the ones described in U.S. patent application Ser. No. 15/467,975, filed Mar. 23, 2017, and entitled “Electronic Oven with Infrared Evaluative Control,” which is incorporated by reference herein in its entirety for all purposes.
The examples above benefit from direct sensing of the energy in the heating chamber for several reasons. In these examples, the number of potential configurations of the electronic oven can be immense, such that it would be difficult to model the distribution of energy in the chamber for all of those potential configurations to generate training data for every potential configuration. Furthermore, the issue of having too many configurations to model is compounded by the fact that each unique item placed in the electronic oven will alter the distributions of each of those potential configurations. Therefore, these approaches benefit from a source of training data that measures the energy distribution resulting from a given configuration of the electronic oven directly.
The energy responsive sensors can be positioned within the electronic oven in numerous ways. The sensors can be located in the heating chamber and can include at least two elements. The sensors could be arranged in a regular array or irregular array. The sensors can be located in a single array or multiple arrays. The array or arrays could be designed to sample the distribution of energy along a single chamber wall of the electronic oven. The term wall, used in this context, is meant to include the floor and ceiling of the chamber. The array or arrays could be built directly into the walls of the chamber. The sensors could be integrated into a wall of the chamber or attached to a wall of the chamber. Alternatively, the array or arrays could be implemented on a substrate that was separate from the walls of the chamber such as a rigid printed circuit board or flex circuit. The substrate could be wholly transparent to RF except for the RF-responsive sensors.
The array or arrays could be fixed within the chamber, removable from the chamber as a unit, or adjustable within the chamber using an actuator. The arrays could also be designed to be swept through the chamber volume, such as by lifting an array up in a direction normal to a floor of the chamber, or by rotating an array in the chamber. As such, the same distribution of energy in the chamber could be sampled multiple times as the array was being swept or rotated through the chamber to produce a single snapshot of that distribution of energy. For example, tomography sampling of the chamber volume could be achieved by sweeping an array of sensors that was the size of the base of the chamber through a range of Z positions towards the ceiling of the chamber. Alternatively, if the sensors were located on a flex circuit substrate, the sensors could be positioned around a phantom item to obtain irregular tomographic samples of the chamber that are directly applicable to items of a particular shape.
The sensors could be designed to be easily removed from the chamber such as by being placed on a test board with fastening devices to hold the test board in place during testing, while allowing it to be easily removed when the electronic oven was deployed in the field. As a specific example, the sensors could be located on a removable printed circuit board and could be attached to a wall of the chamber via a hook and loop connection or other connecting means. As another example, the sensors could be located on a removable printed circuit board and could be removably attached to a permanent portion of the electronic oven responsible for moving items placed in the heating chamber during heating. In particular, the board could be connected in place of a movable tray that would otherwise be located in the electronic oven for moving food through the chamber during heating. The sensor array could be kept in the electronic oven only during calibration, only in the factory for research and development, or it could be an integral part of the electronic as deployed in the field. For example, an array of sensors could be integrated behind a floor panel or coincident with walls of the chamber to provide real-time feedback to the control system.
The array or arrays of sensors could be designed with a density sufficient to get an accurate sample of the distribution of energy in the chamber. For example, the sensors could be spaced such that there was at least one sensor for every 10 cm2 of one or more of the chamber's interior surfaces. However, the sensors would not generally need to be spaced tighter than a spacing equal to one half of the wavelength of the shortest wavelength of energy that the energy source of the electronic oven was designed to introduce to the chamber. The sensors could be located behind a false wall of the chamber. For example, the sensors could be located below a false floor and/or above a false ceiling of the chamber. The false wall can be referred to as “false” in that it is a physical barrier but is transparent to the electromagnetic waves provided to the heating chamber by the energy source.
The sensors can directly sense the distribution of energy in the chamber in various ways. The sensors can respond directly to the energy applied to the chamber such that each of them individually generated a detectable analog signal in response to the applied energy that varied along with the distribution of the application of energy to the chamber. The sensors can also be selected from variant different types of sensors such as a combination of light emitting diodes with a visible light camera. The sensors could also be antennas configured to capture information and channel it out of the chamber via a wired connection to an analog digital converter integrated circuit located outside of the heating chamber. The sensors can also be designed to have different characteristics for detecting electromagnetic signals of a specific frequency, range of amplitudes, and polarization, and the characteristics of the sensors can vary across the chamber of the electronic oven. For example, the set of sensors could be divided into different subsets where each of the subsets was designed to detect to detect electromagnetic radiation with a particular polarization, frequency, or amplitude.
The control system of the electronic oven can include a machine learning system. The machine learning system could include a classifier, an evaluative feedback loop, a reinforcement learning system, a deterministic planner, a neural network, a support vector machine, or any other machine learning system, algorithm, or architecture. The training data can also be the data generated during a discovery phase of a machine intelligence system that alternates between exploitation and discovery phases. The electronic oven could create a set of training data for the machine learning system using data from each sensor in the electronic oven. The training data could be used to update a set of weights in a neural network, be used to derive a plan cost for a deterministic planner, or to evaluate a reward function for a reinforcement learning system. The training data could be used in real time to guide an online machine learning system or it could be collected during a training session and used to guide later heat jobs. The training data could include data from the energy responsive sensors and corresponding data regarding a configuration of the electronic oven. For example, if the electronic oven could place itself in different configurations to adjust the distribution of energy in the electronic oven that result from a given application of heat, the commands used to adjust that distribution, or a state used to describe that configuration, could be paired with the corresponding sensor data to create an entry in the training data for the machine intelligence system.
In step 220, the distribution of the application of energy in the chamber is sensed using a set of sensors. The sensors are located in the heating chamber and include at least two elements. In the illustrated case, the sensors include a regular array of sensors 221. The individual elements of the array are light emitting diodes (LEDs) that are electrically coupled to antennas. Information regarding the distribution of energy is obtained by a visible light camera 222 with a view of the array. The dynamic range of the application of energy in the chamber can be used to set the electrical properties of the antennas and LEDs such that the intensity of light output by the LEDs is indicative of where on the range from minimum to maximum the intensity of the electromagnetic field is at the point occupied by the sensor. The visible light camera can then obtain information regarding the relative intensity of the distribution of the application of energy to the chamber by sensing the intensity of light generated by each element of the array.
Flow chart 200 loops back to step 210 from step 220 because the distribution in the electronic oven could be adjusted and measured numerous times. In each iteration, the item in the electronic oven could be modified and/or the configuration of the electronic oven could be altered to modify the distribution. For example, one iteration of steps 210 and 220 could be conducted while the heating chamber was sealed and a variable reflectance element in the chamber was moved from one orientation to another. As another example, one iteration of steps 210 and 220 could be conducted by opening the chamber and replacing the item inside with another item.
In step 230, data from each sensor in the set of sensors is used to create a set of training data for the machine learning system. The set of training data can include a set of configurations for the electronic oven and a corresponding set of measurements from the set of sensors. Alternatively, the set of training data can include a set of commands from the control system and a corresponding set of measurements from the set of sensors. The commands could be a sequence of commands used by the control system to adjust the distribution of the energy in the chamber. The sensors can be linear time invariant sensors with no memory such that their current state is an accurate snapshot of the current distribution of energy in the chamber. Sensors with this characteristic will help to ensure that the training data provides an accurate uncorrupted correspondence between commands given by the control system to affect the distribution of energy in the chamber and the actual response of the distribution of energy in the chamber to those commands.
In the illustrated example, an element of training set data 231 represents both the physical configuration of a set of variable reflectance elements used to alter the distribution of energy in the chamber as set in step 210, and a corresponding set of sensor values reflecting the distribution of energy in the chamber as measured in step 220. For example, element 231 could be a set of samples captured by the set of sensors and a corresponding command in the sequence of commands used to alter the configuration of the electronic oven. In such an example, the set of samples would have been taken while the corresponding command was applied to adjust the distribution of energy in the chamber. As another example, element 231 could be a set of samples captured by the set of sensors and a corresponding configuration of the electronic oven. The corresponding configuration could be one that was selected and applied in step 210 from a set of at least five fixed configurations for the electronic oven. In such an approach, adjusting the distribution of the application of energy could involve altering a physical configuration of the oven between a set of at least five fixed configurations. The set of at least five fixed configurations could be defined by the state of a set of variable reflectance elements in the chamber, such as the orientation of a reflective element relative to a dominant polarization of an electromagnetic wave in the chamber. Regardless, the combined data of element 231 describes the functional relationship between the control outputs of the control system of the electronic oven and the resulting distribution of energy in the electronic oven. The methods illustrated by flow chart 200 allow for the rapid discovery of which configurations of the electronic oven will provide the most useful distributions of energy in the chamber for heating a particular surface or volume in the electronic oven.
In addition to the data obtained by the energy responsive sensors located in the chamber, the training data used to train the machine intelligence system can include numerous other values and information regarding the state of the electronic oven, and any item in the oven. For example, the data could represent an overall temperature measurement in the chamber, the visible appearance of an item in the chamber, the surface and core temperature of an item in the chamber, an auditory output from the item in the chamber, an identity or description of the item provided by an operator, etc. The sensors used to obtain the additional training data could include temperature sensors, auditory sensors, RF parameter sensors, humidity sensors, particulate concentration sensors, altitude sensors, ultrasound sensors, ultraviolet or IR sensors, a weight sensor such as a scale, and any other sensors that can be used to obtain information regarding the state of the item, chamber, or oven. For example, the oven could include sensors to detect the power applied to the chamber via the energy source, the return loss from the chamber, an impedance match between the energy source and item or chamber, and other physical aspects of the energy source. In particular, the return loss can be measured to determine a phase change in the item as certain items absorb energy at a much greater degree when they are melted compared to when they are frozen. Impedance matching or return loss measurements could also be applied to detect more subtle changes in the physical characteristics of the item being heated. Additional sensors could detect the humidity of the air exiting the chamber via a ventilation system or within the chamber. Additional sensors could detect a particulate concentration within those volumes to determine if the items were smoking. Additional sensors could detect the weight of the item.
In step 240, the training data is used to train a machine learning system. The data can be used in real time if the machine learning system engages in online learning or discovery. However, the data can also be stored and recalled to train the system at a later time. As illustrated, element 231 is used as a tagged example for training a neural network 241 through back-propagation. The stored data can also be used in combination with online learning approaches. For example, the data describing the distribution of energy in the chamber obtained during a prior run in which no item was in the chamber can be applied as feedback for the training of a neural network while the neural network is being used to heat an item in the chamber. This later use of the neural network could involve information gleaned from the reaction of the item during the present heat job in combination with the prerecorded data. Although the actual distribution of energy in the chamber will be effected by the presence of the item, the added information would assist the neural network as there will still be a relationship between the distribution of energy in the chamber with the item in the chamber and without the item in the chamber.
The set of RF responsive sensors could include an analog sensing means. The input range of the analog sensing means could be set by a physical dimension of the analog sensing means. The analog sensing means could be an antenna or set of antennas located in the electronic oven and configured as described herein for sampling the distribution of energy in the chamber. The antennas could be monopole, dipole, slot, or patch antennas. The antennas could be sized to correspond with a potential dynamic range of the distribution of the application of energy in the chamber. The antennas could be used to generate a rectified RF signal which would then be converted into a DC voltage indicative of the field strength at a specific location within the electronic oven. Alternatively, the antennas could be used to provide power to forward bias an LED which would generate light with a brightness indicative of the field strength at a specific location in within the electronic oven. The execution of step 220 would then comprise forward biasing a set of LEDs using the application of energy to the chamber and detecting an intensity of light and a distribution of light from the set of LEDs using a visible light camera. In any such approach, the input range of the analog sensing mean could be configured to be larger than a potential dynamic range of the distribution of energy in the chamber.
The set of sensors could also include an analog to digital conversion means for generating a set of digital data points from a set of samples captured by the set of energy responsive sensors. The set of digital data points could be represented in the training data of the machine intelligence system of the electronic oven. For example, the digital data points could be used directly as the set of samples of the distribution in the electronic oven to form training set data element 231. In approaches that are in keeping with those described with reference to
The different kinds of sensors mentioned above can work in combination with different potential locations for the sensors as mentioned above. For example, sensors in which the analog sensing means was an LED and the analog to digital converter means was a visible light camera could be used in combination with an approach in which the array of sensors was placed below a false floor of the chamber by making the false floor of the chamber transparent to visible light as well as radio frequency radiation. In such an approach, the floor of the chamber would appear to light up as the heating process was in progress which would add an interesting visible element to the use of the electronic oven. Also, any of the approaches disclosed could be used in combination with a board that was only located in the electronic oven during calibration at the factory or during research and development.
The analog sensing means could be antenna arrangements augmented with diodes or other forms of active antennas. The analog sensing means could be net absorbers of energy in the electronic oven. For example, each combination of electrically coupled diodes and antennas could absorb 10 mW to 100 mW of power from the distribution of energy. The analog sensing means could produce a DC voltage or current proportional to the field intensity picked-up by an antenna probe. The antennas could be printed circuit board traces on a printed circuit board. The length of those antennas for purposes of configuring the input range of the analog sensing means would include the effect of parasitic inductances and capacitances on the substrate. The antennas could alternatively be pins extending in a direction normal to a substrate, such as a printed circuit board, on which the diodes were located. The antennas could also have a mix of orientations.
The analog sensing means could produce a rectified signal from the energy in the chamber. The analog sensing means could produce a rectified and filtered signal from the energy in the chamber. The analog sensing means could be configured using low frequency components. The rectified signal could be generated by sampling the signal produced by the antenna arrangement, filtering out any signal with a frequency in the vicinity of twice the frequency of the energy applied to the chamber, and measuring the DC output of the resulting signal. The resulting DC signal would be proportional to the strength of the RF field at that portion of the chamber. The analog sensing means could exhibit various cell topologies involving a rectifying circuit located between and electrically connecting two portions of an antenna.
The LED could be a III-V material LED. For example, the LED could be an InGaN LED. The LED could be chosen such that it does not emit light until the electric field strength reaches a predetermined value that results in substantial heating, such as greater than 6 volts per centimeter. The dipole antenna could be anywhere from 0.5 cm to 2 cm in length. The forward diode bias voltage and the length of the dipole antenna could be configured such that the maximum brightness of the LED matched the maximum expected field strength for any particular point in the heating chamber of the microwave. The LED could be a white LED such as a Luxeon 2835 2D LP with two LED junctions in series and a forward bias voltage of 6 volts.
Waveform plot 420 includes two waveforms plotted against time on x-axis 421. The magnitude of electric field 406 across antennas 402 and 403 is illustrated by waveform 412 where the x-intercept represents zero. The voltage across LED 401 in response to electric field 406 is illustrated by waveform 423 where the x-intercept represents zero and the same scale is used for both waveform 422 and waveform 423. The two waveforms overlap until the field strength is strong enough to forward bias voltage of LED 401 which is marked by reference line 414 in waveform plot 410. While the field strength exceeds the forward bias voltage, light is emitted by the LED. If the frequency is high enough the light will appear continuous to a visible light camera viewing the LED and the apparent brightness of the LED will increase with the magnitude of waveform 422.
Waveform plot 510 includes two waveforms plotted against time on x-axis 511. The magnitude of electric field 504 across the antenna formed by portions 502 and 503 is illustrated by waveform 512 where the x-intercept represents zero. The voltage across LEDs 501 in response to electric field 504 is illustrated by waveform 513 where the x-intercept represents zero and the same scale is used for both waveform 512 and waveform 513. The two waveforms overlap until the magnitude of the field strength exceeds the forward bias voltage of LEDs 501 which is marked by reference line 514 in waveform plot 510. While the magnitude of the field strength exceeds the forward breakdown voltage of the two LEDs, light is emitted by the LEDs. If the frequency is high enough the light will appear continuous to a visible light camera viewing the LEDs and the apparent brightness of the LEDs will increase with the magnitude of waveform 512. The resulting signal is fully rectified as LEDs 501 are electrically coupled to antenna portions 502 and 503 via a fully rectifying diode bridge 506. The diode bridge could be a Schottky diode bridge such as an HSMS-2828-TR1G with a 15-volt tolerance and 1 pico-Farad of parasitic capacitance.
Different variations of the structures provided in
As mentioned previously, the set of sensors can be divided into subsets that can respond to electromagnetic energy with different characteristics. For example, sensors that have the approximate configuration described with reference to
The electronic oven can include numerous subsets of sensors with different physical orientations and/or different physical compositions. The sensors physical compositions can be selected such that they generate different kinds of signals or respond to incident energy differently. The different physical orientations and different physical compositions can both affect the type of incident energy that the sensors respond to. Electronic oven 610 provides a specific example of subsets of sensors with different physical orientations to respond to different kinds of incident energy while also having a different physical composition to respond differently to incident energy. In electronic oven 610, sensors 601 and 602 are in subsets that differ as to both physical orientation and physical composition. The subset of sensors that have the same orientation as sensor 601 are configured to generate light in a first spectrum 603. The subset of sensors that have the same orientation as sensor 602 are configured to generate light in a second spectrum 604. For example, the first subset could generate green light and the second subset could generate red light. The two subsets of sensors both produce light that is detectable by a visible light camera with a view of both subsets via viewport 103. Therefore, the visible light camera is able to obtain information regarding the distribution of field strength components in the electronic oven without requiring any preprogrammed information concerning the subsets.
While the specification has been described in detail with respect to specific embodiments of the invention, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing, may readily conceive of alterations to, variations of, and equivalents to these embodiments. Any of the method steps discussed above can be conducted by a processor operating with a computer-readable non-transitory medium storing instructions for those method steps. The computer-readable medium may be memory within a personal user device or a network accessible memory. Although examples in the disclosure where generally directed to electronic ovens, the same approaches could be utilized in any application in which a machine intelligence system is in the feedback loop of a system that applies electromagnetic energy to affect an item located in a volume subject to that electromagnetic energy. These and other modifications and variations to the present invention may be practiced by those skilled in the art, without departing from the scope of the present invention, which is more particularly set forth in the appended claims.
