Patent Application
20160005329
The present disclosure relates to an information processing device and a storage medium.
Recently, devices that assist dietary lifestyle management are being proposed.
Further, Patent Literature 2 below discloses technology that reduces the user workload of recording meal content for efficient management. Specifically, if a food image is sent together with time and date information from a personal client to a center server, an advisor (expert) at the center server analyzes the image of food, and inputs and sends advice.
Also, Patent Literature 2 below discloses a navigation system that includes a nutritional data storage unit (for example, an IC tag attached to a packaging container of a food product), a nutritional data reading unit (for example, an IC tag reader), and a data output unit, which reduces the long time and effort required to memorize food composition.
Patent Literature 1: JP 2010-33326A
Patent Literature 2: JP 2006-195943A
However, with Patent Literature 1 above, since analysis is conducted on the basis of a captured image of an already-prepared dish, it is difficult to recognize what kinds of ingredients and how much were used in the cooking process.
Also, Patent Literature 2 likewise does not discuss the computation of calories or nutritional components during the cooking process.
Accordingly, the present disclosure proposes a new and improved information processing device and storage medium capable of computing indices during a cooking process.
According to the present disclosure, there is provided an information processing device including: an estimation unit configured to estimate a used quantity of at least one of an ingredient to be cooked and a seasoning used in cooking, on the basis of a signal detected by a sensor; an index computation unit configured to compute a prescribed cooking index according to the estimated result by the estimation unit; and a notification control unit configured to perform control to issue a notification of the cooking index computed by the index computation unit.
According to the present disclosure, there is provided a storage medium having a program stored therein, the program causing a computer to function as: an estimation unit configured to estimate a used quantity of at least one of an ingredient to be cooked and a seasoning used in cooking, on the basis of a signal detected by a sensor; an index computation unit configured to compute a prescribed cooking index according to the estimated result by the estimation unit; and a notification control unit configured to perform control to issue a notification of the cooking index computed by the index computation unit.
According to the present disclosure as described above, it becomes possible to compute indices during a cooking process.
Hereinafter, (a) preferred embodiment(s) of the present disclosure will be described in detail with reference to the appended drawings. Note that, in this specification and the drawings, elements that have substantially the same function and structure are denoted with the same reference signs, and repeated explanation is omitted.
Hereinafter, the description will proceed in the following order.
1. Overview of cooking indices display according to an embodiment of the present disclosure
2. Basic configuration and operational process of HMD
3. Screen display examples
4. Supplemental remarks 5. Conclusion
<<1. Summary of Cooking Indices Display According to Embodiment of Present Disclosure>>
First, a cooking indices display according to an embodiment of the present disclosure will be summarized with reference to
Also, the HMD 1 is configured such that, in the worn state, a pair of display units 2 for the left eye and the right eye are placed immediately in front of either eye of the user 8, or in other words at the locations where the lenses of ordinary eyeglasses are positioned. A captured image of a real space captured with an image capture lens 3a, for example, is displayed on the display units 2. The display units 2 may also be transparent, and by having the HMD 1 put the display units 2 in a see-through state, or in other words a transparent or semi-transparent state, ordinary activities are not impaired even if the user 8 wears the HMD 1 continuously like eyeglasses.
Also, as illustrated in
Also, although only illustrated on the left eye side in
Note that the external appearance of the HMD 1 illustrated in
Also, although the image capture lens 3a and the light emitter 4a that provides illumination are placed facing forward on the side of the right eye in the example illustrated in
It is also acceptable to provide a single earphone speaker 5a to be worn in only one ear, rather than as left and right stereo speakers. Likewise, a microphone may be one of either the microphone 6a or 6b.
Furthermore, a configuration not equipped with the microphones 6a and 6b or the earphone speakers 5a is also conceivable. A configuration not provided with the light emitter 4a is also conceivable.
The above thus describes an external configuration of the HMD 1 illustrated in
Herein, as discussed earlier, with Patent Literature 1 above, it is difficult to analyze what kinds of ingredients and how much were used in the cooking process on the basis of a captured image of an already-prepared dish. Also, if cooking indices indicating the calories and mass of ingredients and seasonings could be computed in real-time during the cooking process and including in a notification to the user, dietary lifestyle support would be possible from the cooking stage.
Particularly, in cases in which improvements in dietary lifestyle are demanded due to problems of lifestyle-related diseases or the like, the intake and numerical values of calories, fat, sugar, purines, cholesterol, and the like become problematic, but there is a large amount of information related to food substances that are preferred and non-preferred for improving dietary lifestyle, which becomes a burden on the user. In addition, it has been difficult for a user to continually take in preferred food substances, as in some cases the user may forget when eating or cooking, or an unexpected food substance may be non-preferred.
Accordingly, focusing on the above circumstances led to the creation of an HMD 1 (information processing device) according to the embodiments of the present disclosure. An HMD 1 according to the embodiments of the present disclosure is able to compute indices during the cooking process.
Specifically, the HMD 1 (information processing device) according to the present embodiment captures a cooking state with an image capture lens 3a, estimates the quantities of used ingredients or used seasonings in the captured image, computes cooking indices of each ingredient or seasoning on the basis of the estimation results, and notifies the user of the computed cooking indices in real-time. In this specification, cooking indices are the values of the mass or calorific value (calories) of ingredients or seasonings, for example. Also, the values of the mass or calorific value may also be the values of the mass or calorific value of respective nutritional components included in ingredients or seasonings. The respective nutritional components may be vitamins, fat, protein, carbohydrates, calcium, magnesium, dietary fiber, potassium, iron, retinol, salt, sugar, purines, cholesterol, or alcohol, for example. In addition, the quantities of used ingredients and used seasonings may be estimated on the basis of not only a captured image, but also results detected from a smell sensor or various concentration sensors (such as a salt sensor or sugar sensor). The various concentration sensors are provided in a cooking utensil such as a measuring spoon or pot, and detected measurement data is transmitted to the HMD 1.
For example, in the example illustrated in
For the method of notifying the user of cooking indices, a display notification by the display units 2 or an audio notification by the earphone speakers 5a is possible. For example, as illustrated in
Additionally, the HMD 1 may also compare a cooking index to a prescribed value, and present a warning notification to the user when the cooking index approaches the prescribed value. A prescribed value refers to a limit index, and specifically is a prescribed value prioritizing health, or a prescribed value prioritizing the user's taste preferences. Additionally, the HMD 1 may also compare a cooking index to a prescribed value for when cooking is complete, and present a warning notification to the user according to the comparison result.
The above thus summarizes a cooking indices display according to the present embodiment. Next, a basic configuration and operational process of an HMD 1 that presents a cooking indices display according to the present embodiment will be described with reference to
<<2. Basic Configuration and Operational Process of HMD>>
<2-1. Basic Configuration of HMD>
The main controller 10 is made up of a microcomputer equipped with a central processing unit (CPU), read-only memory (ROM), random access memory (RAM), non-volatile memory, and an interface unit, for example, and controls the respective components of the HMD 1.
As illustrated in
The used quantity estimation unit 10a estimates the used quantity of at least one of an ingredient to be cooked and a seasoning used in cooking, on the basis of a signal detected by a sensor, and supplies an estimation result to the index computation unit 10c. A signal detected by a sensor refers to information such as a captured image captured by an image capture sensor (image capture unit 3), numerical values detected by various concentration sensors, and smell data detected by a smell sensor, for example.
The various concentration sensors (such as a salt concentration sensor, sugar concentration sensor, ion concentration sensor, and pH sensor) are provided in a cooking utensil such as a measuring spoon or pot, and detected measurement data is transmitted to the HMD 1, as discussed earlier.
In addition, a smell sensor is provided in the HMD 1 or externally (such as in a cooking utensil or in the kitchen), and detects smell components in an ingredient or seasoning. Herein, a smell sensor may be configured using multiple types of metal-oxide-semiconductor sensor elements, for example. Ordinarily, a metal-oxide-semiconductor is in a state of low conductivity, in which oxygen present in the air is adsorbed on the surface of crystal grains, and this oxygen traps electrons in the crystals which are the carriers. In this state, if smell components adhere to the surface of the metal-oxide-semiconductor, oxidation of the smell components takes away adsorbed oxygen on the surface, and the conductivity increases. Since the change in conductivity differs according to differences in the type and grain size of the metal-oxide-semiconductor, and the catalyst to be added, smell components are identified by utilizing this property. in the case of estimating on the basis of a captured image, the used quantity estimation unit 10a estimates the quantity of ingredients and seasonings used by using an analysis result by the captured image analyzer 13 discussed later. For example, the used quantity estimation unit 10a is able to recognize an ingredient/seasoning on the basis of character recognition or by reading a barcode or marker on a label affixed to a container or bag holding an ingredient or seasoning, and estimate the quantity of recognized ingredient/seasoning used. At this point, an ingredient/seasoning quantity estimation method by the used quantity estimation unit 10a will be described with reference to
In addition, the used quantity estimation unit 10a is also able to reference an analysis result of a captured image, and on the basis of features such as the color, shape, and size of the analyzed ingredient/seasoning, recognize an ingredient/seasoning and estimate the quantity of recognized ingredient/seasoning used. In this case, the used quantity estimation unit 10a may also use used quantity estimation data stored in the storage unit 22 (a database associating features such as color, shape, and size with ingredients/seasonings).
Furthermore, the used quantity estimation unit 10a is also able to combine captured image analysis results from the captured image analyzer 13, smell data detected by a smell sensor, and various measurement data to comprehensively estimate the quantity of ingredient/seasoning used.
The cooking method identification unit 10b determines a cooking method during the cooking process (such as stir-fried, grilled, boiled, fried, steamed, raw, or dressed), and supplies a determined result to the index computation unit 10c. Specifically, the cooking method identification unit 10b may identify a cooking method on the basis of a captured image acquired by an image capture sensor (the image capture unit 3), and measurement data detected by sensors such as an infrared sensor (thermal imaging sensor), a temperature sensor, and a humidity sensor. For example, the cooking method identification unit 10b may also identify a cooking method according to a cooking utensil recognized from a captured image (such as a frying pan, pot, tempura pot, or steamer). In addition, the cooking method identification unit 10 may also identify a cooking method in conjunction with various measurement data (such as temperature data and humidity data). In this case, the cooking method identification unit 10b may also use cooking method identification data stored in the storage unit 22 (a database associating cooking utensils, temperature data, and the like with cooking methods).
The index computation unit 10c computes prescribed cooking indices according to the quantity of ingredient/seasoning used that is estimated by the used quantity estimation unit 10a. In addition, the index computation unit 10c is also able to convert computed cooking indices according to the cooking method identified by the cooking method identification unit 10b. The index computation unit 10c supplies computed/converted cooking indices to the index comparison unit 10d and the notification control unit 10e. Also, the computed/converted cooking indices are accumulated in the storage unit 22.
The index computation unit 10c may also compute cooking indices by using index computation data stored in the storage unit 22 (a database associating ingredient/seasoning quantities with cooking indices). For example, the index computation unit 10e computes a teaspoon of salt as “table salt 6 g”, and a pinch of salt (the quantity picked up with the three fingers of the thumb, index finger, and middle finger) as “table salt 1 g”.
Also, since cooking indices (calories and mass) change according to cooking method in some cases depending on the nutritional components, the index computation unit 10c may also convert computed indices according to a cooking method identified by the cooking method identification unit 10b. For example, the index computation unit 10c converts cooking indices using cooking index conversion data stored in the storage unit 22 (a database associating cooking methods with changes in respective cooking indices).
In addition, the index computation unit 10c is also able to total computed/converted cooking indices and cooking indices accumulated in the storage unit 22.
The index comparison unit 10d compares cooking indices supplied from the index computation unit 10c to prescribed values, and supplies comparison results to the notification control unit 10e. Specifically, the index comparison unit 10d compares a cooking index to a limit index prioritizing health, and judges whether or not the cooking index approaches the limit index (is within a prescribed range based on the limit index). A limit index prioritizing health refers to an upper limit value on the intake of a cooking index that is generally acceptable from a health perspective, or alternatively, an upper limit value on the intake of a cooking index that is acceptable on the basis of information about the user 8 such as medical information (including disease history and medication history), health information (including current physical condition information), genetic information, and predisposition information (including allergy information). Information about the user 8 such as medical information and health information may be extracted from the storage unit 22, or acquired from a designated server via the communication unit 21. In addition, current physical condition information about the user 8 may be detected with various biological sensors that detect biological information (such as blood pressure, body temperature, pulse, or brain waves) provided on the HMD 1, or detected with a biological information detection device (not illustrated) separate from the HMD 1 and possessed by the user.
On the other hand, depending on the preferences of the user 8, a limit index prioritizing health may not necessarily be optimal, and thus the index comparison unit 10d may also compare a cooking index to a limit index prioritizing the preferences of the user 8, and judge whether or not the cooking index approaches the limit index (is within a prescribed range based on the limit index).
In addition, when a limit index (upper limit value) for when cooking is complete may be acquired, the index comparison unit 10d may also compare a cooking index to a limit index (upper limit value) for when cooking is complete. For example, to avoid situations in which the salt content would exceed the upper limit value when adding cheese, dry-cured ham, or a sauce to finish cooking, the HMD 1 notifies the user of the comparison result between the limit index (upper limit value) for when cooking is complete and the current cooking index (a cumulative dish index) during the cooking process. Consequently, the user 8 is able to avoid adding too much salt while cooking. Note that a limit index (upper limit value) for when cooking is complete may be acquired on the basis of recipe data for the dish currently being cooked, or a dish name specified by the user 8.
The notification control unit 10e applies control to notify the user 8 of cooking indices supplied from the index computation unit 10c and comparison results supplied from the index comparison unit 10d. Notification to the user 8 may be a display notification or an audio notification. In the case of a display notification, the notification control unit 10e generates and outputs to the display controller 17 image data indicating cooking indices (for example, the cooking indices display 30 illustrated in
In addition, the notification control unit 10e applies control to present a warning notification when a cooking index approaches a limit index (is within a prescribed range based on the limit index) on the basis of a comparison result supplied from the index comparison unit 10d. A warning notification to the user 8 may be a display notification, an audio notification, a vibration notification, or the like. In the case of a display notification, the notification control unit 10e generates and outputs to the display controller 17 image data indicating a warning (for example, the warning display 35 illustrated in
The image capture unit 3 includes a lens subsystem made up of the image capture lens 3a, a diaphragm, a zoom lens, a focus lens, and the like, a driving subsystem that causes the lens subsystem. to conduct focus operations and zoom operations, a solid-state image sensor array that generates an image capture signal by photoelectric conversion of captured light obtained with the lens subsystem, and the like. The solid-state image sensor array may be realized by a charge-coupled device (CCD) sensor array or a complementary metal-oxide-semiconductor (CMOS) sensor array, for example.
The image capture controller 11 controls operations of the image capture unit 3 and the image capture signal processor 12 on the basis of instructions from the main controller 10. For example, the image capture controller 11 controls the switching on/off of the operations of the image capture unit 3 and the image capture signal processor 12. The image capture controller 11 is also configured to apply control (motor control) causing the image capture unit 3 to execute operations such as autofocus, automatic exposure adjustment, diaphragm adjustment, and zooming. The image capture controller 11 is also equipped with a timing generator, and controls signal processing operations with timing signals generated by the timing generator for the solid-state image sensors as well as the sample and hold/AGC circuit and video A/D converter of the image capture signal processor 12. In addition, this timing control enables variable control of the image capture frame rate.
Furthermore, the image capture controller 11 controls image capture sensitivity and signal processing in the solid-state image sensors and the image capture signal processor 12. For example, as image capture sensitivity control, the image capture controller 11 is able to conduct gain control of signals read out from the solid-state image sensors, set the black level, control various coefficients for image capture signal processing at the digital data stage, control the correction magnitude in a shake correction process, and the like.
The image capture signal processor 12 is equipped with a sample and hold/automatic gain control (AGC) circuit that applies gain control and waveform shaping to signals obtained by the solid-state image sensors of the image capture unit 3, and a video analog/digital (AID) converter. Thus, the image capture signal processor 12 obtains an image capture signal as digital data. The image capture signal processor 12 also conducts white balance processing, luma processing, chrome signal processing, shake correction processing, and the like on an image capture signal.
The captured image analyzer 13 is an example of a configuration for acquiring external information. Specifically, the captured image analyzer analyzes image data (a captured image) that has been captured by the image capture unit 3 and processed by the image capture signal processor 12, and obtains information on an image included in the image data.
Specifically, the captured image analyzer 13 conducts analysis such as point detection, line/edge detection, and area segmentation on image data, for example, and outputs analysis results to the used quantity estimation unit 10a and the cooking method identification unit 10b of the main controller 10.
The illumination unit 4 includes the light emitter 4a illustrated in
The audio input unit 6 includes the microphones 6a. and 6b illustrated in
The display controller 17, according to control from the main controller 10, conducts driving control for displaying image data supplied from the notification control unit 10c on the display units 2. The display controller 17 may be made up of a pixel driving circuit for causing display in display units 2 realized as liquid crystal displays, for example. The display controller 17 is also able to control the transparency of each pixel of the display units 2, and put the display units 2 in a see-through state (transparent state or semi-transparent state). In addition, the display controller 17 may include components such as a video processor, a digital signal processor, and a D/A converter, for example, and may conduct luma level adjustment, color correction, contrast adjustment, sharpness (edge enhancement) adjustment, and the like on image data to display.
The display units 2 display image data under control by the display controller 17. Specifically, the display units 2 according to the present embodiment present a cooking indices display, a warning display; and the like.
The audio controller 18, according to control from the main controller 10, applies control to output audio signal data supplied from the notification control unit 10e from the audio output unit 5. In addition, the audio controller 18 conducts volume adjustment, sound quality adjustment, acoustic effects, and the like on audio signal data to output,
The audio output unit 5 includes the pair of earphone speakers 5a illustrated in
The storage unit 22 is a member that records or plays back data with respect to a designated recording medium. The storage unit 22 is realized by a hard disk drive (HDD), for example. Obviously, various media such as flash memory or other solid-state memory, a memory card housing solid-state memory, an optical disc, a magneto-optical disc, and holographic memory are conceivable as the recording medium, and it is sufficient to configure the storage unit 22 to be able to execute recording and playback in accordance with the implemented recording medium.
Also, the storage unit 22 according to the present embodiment stores data such as used quantity estimation data used by the used quantity estimation unit 10a, cooking method identification data used by the cooking method identification unit 10b, and index computation data and index conversion data used by the index computation unit 10c. Also, the storage unit 22 stores information about the user 8, such as medical information, health information, genetic information, and predisposition information.
Furthermore, the storage unit 22 stores cooking indices computed/converted by the index computation unit 10c. In this way, by accumulating cooking indices computed during the cooking process, the HMD 1 is able to generate more accurate cooking indices for the finished dish. In the storage unit 22, the finished dish and its cooking indices (monitor results) are stored in association with each other.
The communication unit 21 sends and receives data to and from external equipment. The communication unit 21 communicates wirelessly with external equipment directly or via a network access point, according to a scheme such as a wireless local area network (LAN), Wi-Fi (Wireless Fidelity, registered trademark), infrared communication, or Bluetooth (registered trademark).
The communication unit 21 according to the present embodiment receives measurement data from a cooking utensil provided with a concentration sensor, for example.
The above thus describes in detail an internal configuration of an HMD 1 according to the present embodiment. Note that although the audio output unit 5, audio input unit 6, audio signal processor 15, and audio controller 18 are illustrated as an audio-related configuration, it is not strictly necessary to provide all of the above. Also, although the communication unit 21 is illustrated as part of the configuration of the HMD 1, it is not strictly necessary to provide the communication unit 21.
According to the above configuration, during the cooking process, the HMD 1 according to the present embodiment is able to estimate the quantities of ingredients to be cooked or seasonings used in cooking, compute cooking indices of the ingredients or seasonings on the basis of the estimation results, and notify the user of the computed cooking indices in real-time. In addition, by accumulating cooking indices computed during the cooking process, the HMD 1 is able to generate more accurate cooking indices for the finished dish. Next, an operational process of an HMD 1 according to the present embodiment will be described.
As discussed above, the HMD 1 according to the present embodiment is worn by the user 8, and notifies the user of cooking indices in real-time while the user 8 is cooking. A cooking indices notification by such an HMD 1 will be specifically described hereinafter with reference to
Next, in step S 106, the used quantity estimation unit 10a of the HMD 1 estimates the quantities of ingredients to be cooked or seasonings used in cooking, on the basis of a captured image of the cooking process captured by the image capture unit 3.
Subsequently, in step S109, the index computation unit 10c generates a cooking index A of an ingredient or seasoning, according to a quantity of ingredient or seasoning used that was estimated by the used quantity estimation unit 10a.
Next, in step S112, if a cooking method is identified by the cooking method identification unit 10b, the index computation unit 10c converts the computed cooking index A according to the cooking method.
Subsequently, in step S115, the notification control unit 10e instructs the display controller 17 to display on the display units 2 the cooking index supplied from the index computation unit 10c (the current value of the cooking index A). Alternatively, if the cooking index A and a prescribed limit index (upper limit value) are compared by the index comparison unit 10d, the notification control unit 10e may instruct the display controller 17 to display on the display units 2 the comparison result (the ratio of the current value of the cooking index A versus the upper limit value; P %).
Next, in step S118, the notification control unit 10e judges whether or not the above P % (the ratio of the current value of the cooking index A versus the upper limit value) is 90% or greater.
In the case of being below 90% (S118/No), in step S121 the notification control unit 10e applies control to display P % normally.
On the other hand, in the case of being 90% or greater (S118/Yes), in step S124 the notification control unit 10e judges whether or not P % is 100+a (alpha) % or greater. In other words, the notification control unit 10e judges whether or not the current value of the cooking index A has exceeded the upper limit value+a (alpha).
In the case of being below 100+a % (S124,No), in step S130 the notification control unit 10e instructs the display controller 17 or the audio controller 18 to produce a warning display from the display units 2 or a warning announcement from the audio output unit 5. Thus, the HMD 1 is able to warn the user when the current value of the cooking index A is between 90% and 100+a %.
On the other hand, in the case of 100+a % or greater (S124/Yes), in step S127 the notification controller 10e instructs the display controller 17 or the audio controller 18 to produce a stop display from the display units 2 or a stop announcement from the audio output unit 5.
A stop notification (prohibition notification) has a higher alert level than a warning notification. For example, the notification control unit 10e may cause the display units 2 to display “DO NOT ADD” in large letters, or cause the audio output unit 5 to output a warning sound until the user stops adding the ingredient or seasoning (until the ingredient or seasoning is removed from the angle of view of the image capture unit 3). In addition, if the lid of a seasoning container is motorized, the notification control unit 10e may also transmit a command from the HMD 1 to the seasoning container causing the lid to close, thereby closing the lid of the seasoning container automatically, and preventing the seasoning from being added to the dish.
Subsequently, in step S133, the main controller 10 judges whether or not an ingredient or seasoning has been added. For example, if an action of the user being about to add an ingredient or seasoning to a cooking utensil such as a frying pan is extracted on the basis of a captured image captured by the image capture lens 3a, it is judged that an ingredient or seasoning is added.
If an ingredient or seasoning is added (S133/Yes), in step S106 the used quantity estimation unit 10a estimates the quantity of the added ingredient or seasoning, and in the following step S109, the index computation unit 10c generates a cooking index A of the ingredient or seasoning. At this point, the index computation unit 10c generates the current value of the cooking index A by adding the total value of the currently computed cooking index to the cumulative (total) cooking index since cooking started. Thereafter, the above is repeated from S112 to S133.
If an ingredient or seasoning is not added (S133/No), it is judged that cooking is finished, and the cooking index notification process ends.
The above thus specifically describes a cooking indices display process according to the present embodiment.
Next, screen display examples of cooking indices according to the present embodiment will be described with reference to
The cooking indices display 31 illustrated in
Subsequently, as cooking proceeds and other ingredients are added to the frying pan, the index computation unit 10c computes cooking indices for each ingredient, and totals the computed dish indices.
Next, the display of cooking indices for a seasoning will be described with reference to
Next, if an action of being about to add the salt scooped up with the measuring spoon obj7 to the frying pan obj5 is extracted on the basis of image analysis results for the captured image P6 illustrated in the lower part of
Subsequently, the notification control unit 10e instructs the display controller 17 to display on the display units 2 a cooking indices display 33 including a display of the computed cooking indices.
As described above, the HMD 1 according to the present embodiment is able to continually observe the cooking process, compute cooking indices for ingredients to be cooked and seasonings used in the cooking, and notify the user 8. Note that a notification by the HMD 1 is not limited to a notification of computed cooking indices, and may also be a warning notification or a comparison result notification based on comparison results from comparing the computed cooking indices to prescribed values. Hereinafter, as an example, a screen display based on a comparison result of a cooking index and a prescribed value will be described with reference to
Subsequently, the index comparison unit 10d compares the cooking index computed by the index computation unit 10c to a prescribed value. The prescribed value to compare against is a prescribed upper limit value prioritizing health, or a prescribed upper limit value prioritizing taste preferences, for example, as discussed earlier.
Subsequently, on the basis of a comparison result supplied from the index comparison unit 10d, if the cooking index indicating the quantity of salt computed by the index computation unit 10c approaches the prescribed value, the notification control unit 10e causes the display units 2 to display a warning display 35 indicating a warning of “The amount of salt is over the limit!”, as illustrated in
Also, if the cooking index approaches and then exceeds the prescribed value, and specifically reaches a ratio equal to or greater than 100+a % versus the upper limit value, the notification control unit 10e may also cause the display units 2 to display a prohibition display indicating a warning of “NO MORE SALT ALLOWED”.
As described above, when a prescribed cooking index approaches a prescribed value, the notification control unit 10e is able to present a warning display and warn the user 8. However, some cases are anticipated in which, even though the user may be notified that the salt content or sugar content exceeds an upper limit value, not adding an ingredient is undesirable in terms of the completed dish, such as when adding an important ingredient last to finish cooking. Consequently, the index comparison unit 10d of the HMD 1 according to the present embodiment is also able to compare a cooking index computed by the index computation unit 10c to a limit index (upper limit value) for when cooking is complete, and by notifying the user 8 of the comparison result with the notification control unit 10e, is able to present a notification that accounts for the upper limit value at the time of completion,: Hereinafter, an example of such a notification of a comparison result will be described with reference to
Consequently, the user 8 is able to grasp the current ratio of the cooking index, and make sure to not add too much of a specific seasoning or ingredient while cooking. Note that in the example illustrated in
As discussed earlier, during the cooking process, the HMD 1 according to the present embodiment is able to compute cooking indices for ingredients to be cooked and seasonings used in the cooking, and notify the user 8 of the computed cooking indices in real-time. In addition, by continually accumulating the computed cooking indices, the HMD 1 is able to generate more accurate cooking indices for the completed dish. At this point, usage examples of cooking indices generated in this way will be described as supplemental remarks.
Cooking indices for a dish are used to notify the user of the calories and nutritional components of a dish when eating and drinking, for example. By notifying the user of the calories and nutritional components of a dish when eating and drinking, the user is able to intuitively grasp the calories and nutritional components consumed.
Such a notification of calories and nutritional components when eating and drinking may also be conducted by the notification control unit 10e of the HMD 1. Specifically, by including a main controller 10′ illustrated in
As illustrated in
The cooking index acquisition unit 10g acquires cooking indices of a dish determined by the dish determination unit 10f. At this point, suppose a case in which cooking indices computed by the index computation unit 10c illustrated in
The consumption index computation unit 10h computes indices of food and drink consumed by the user 8. Specifically, the consumption index computation unit 10h computes indices of the portion of food or drink consumed by the user 8, on the basis of an analysis result of a captured image that is captured by the image capture unit 3. Indices of food or drink consumed refer to indicates that indicate intake calories in other words, and the masses of nutritional components that are taken in.
The consumption index comparison unit 10i compares a consumption index computed by the consumption index computation unit 10h to a prescribed value. The prescribed value refers to an upper limit value on calorie intake in one day, an upper limit value on the intake of a nutritional component in one day, or the like, for example.
The notification control unit 10e instructs the display controller 17 or the audio controller 18 to notify the user 8 of consumption indices computed by the consumption index computation unit 10h and comparison results from the consumption index comparison unit 10i.
The above thus specifically described a functional configuration of the main controller 10′ of the HMD 1. Next, an operational process for the case of notifying a user of the calories and nutritional components of a dish when eating and drinking on the basis of the cooking indices of the dish will be described with reference to
Next, in step S206, the main controller 10′ of the HMD recognizes that the consumption index AE=0 at the start of eating and drinking.
Next, in step S209, the dish determination unit 10f of the main controller 10′ determines the number of dishes on the basis of an analysis result of a captured image that is captured by the image capture unit 3, and the cooking index acquisition unit 10g acquires a cooking index A of each dish. The dish determination unit 10f may also assign a number to each dish when determining the number of dishes. Also, in the case of five dishes numbered from 1 to 5, for example, the cooking index acquisition unit 10g acquires cooking indices A1 to A5 respectively for each dish.
Next, in step S212, the consumption index computation unit 10h specifies the number i of the dish that the user 8 is currently consuming, on the basis of an analysis result of a captured image that is captured by the image capture unit 3, and totals the cooking index Ai of the corresponding dish as the intake consumption index AEi. In other words, the consumption index computation unit 10h computes the consumption index AE (current value) as being equal to AEi (the consumption index just consumed). In addition, when the intake consumption index AEi is additional consumption, the consumption index computation unit 10h computes the consumption index AE current value) as being equal to AE (the previous total of the consumption index) plus AEi (the consumption index just consumed).
Next, in step S215, the notification control unit 10e of the main controller 10 instructs the display controller 17 to display on the display units 2 the current value of the consumption index AE computed by the consumption index computation unit 10h. Alternatively, if the current consumption index AE and a prescribed value (intake upper limit value) are compared by the consumption index comparison unit 10i, the notification control unit 10e may instruct the display controller 17 to display on the display units 2 the comparison result (the ratio of the current value of the consumption index AE versus the upper limit value; (Q %). The intake upper limit value is an upper limit value on calorie intake in one day, an upper limit value on calorie intake in one week, or an upper limit value on cholesterol in one day, or the like, for example. Such an upper limit value may also be set on the basis of a user's medical information and health information.
Next, in step S218, the notification control unit 10e judges whether or not the above Q % (the ratio of the current value of the consumption index AE versus the intake upper limit value) is 90% or greater.
In the case of being below 90% (S218/No), in step S221 the notification control unit 10e applies control to display Q % normally.
On the other hand, in the case of being 90% or greater (S218/Yes), in step S224 the notification control unit 10e judges whether or not Q % is 100+a (alpha) % or greater. In other words, the notification control unit 10e judges whether or not the current value of the consumption index AE has exceeded the intake upper limit value+a (alpha).
In the case of being below 100+a % (S224/No), in step S230 the notification controller 10e instructs the display controller 17 or the audio controller 18 to produce a warning display from the display units 2 or a warning announcement from the audio output unit 5. Thus, in the case where the current value of the consumption index AE is between 90% and 100+a %, the HMD 1 issues a warning to the user.
On the other hand, in the case of 100+a % or greater (S224/Yes), in step S227 the notification controller 10e instructs the display controller 17 or the audio controller 18 to produce a stop display from the display units 2 or a stop announcement from the audio output unit 5.
A stop notification (prohibition notification) has a higher alert level than a warning notification. For example, the notification controller 10e may cause the display units 2 to display “STOP EATING” in large letters, or cause the audio output unit 5 to output a warning sound until the user stops eating.
Subsequently, in step S233, the main controller 10′ judges whether or not there is additional consumption. For example, if an action of the user being about to continue eating or drinking is extracted on the basis of a captured image captured by the image capture lens 3a, it is judged that there is additional consumption.
If there is additional consumption (S233/Yes), in step S212 the consumption index computation unit 10h computes, on the basis of the cooking index Ai of the dish i being additionally consumed, the consumption index AE (current value) as being equal to AE (the previous total of the consumption index) plus AEi (the consumption index just consumed). Thereafter, the above is repeated from S215 to S233.
If there is no additional consumption (S233/No), it is judged that eating and drinking is finished, and the consumption index notification process ends.
As discussed above, with an HMD 1 according to the present embodiment, it is possible to compute indices during the cooking process, and notify the user of the cooking indices in real-time while cooking. In addition, cooking indices may be converted according to the cooking method for the dish.
Also the HMD 1 is able to present a warning notification or a prohibition notification based on a comparison result from comparing a current cooking index to a prescribed value, and notify the user of the ratio of the current value of the cooking index versus the prescribed value or the like.
Furthermore, since the HMD 1 continually observes the cooking process, computes cooking indices of ingredients to be cooked and seasonings used in the cooking, and accumulates the computed cooking indices, the HMD 1 is able to generate more accurate cooking indices for the finished dish.
The foregoing thus describes preferred embodiments of the present technology in detail and with reference to the attached drawings. However, the present disclosure is not limited to such examples. It is clear to persons ordinarily skilled in the technical field of the present disclosure that various modifications or alterations may occur insofar as they are within the scope of the technical ideas stated in the claims, and it is to be understood that such modifications or alterations obviously belong to the technical scope of the present disclosure.
For example, it is possible to create a computer program for causing hardware such as a CPU, ROM, and RAM built into the HMD 1 to exhibit the functionality of the HMD 1 discussed earlier. A computer-readable storage medium made to store such a computer program is also provided.
Also, in the above respective embodiments, although an HMD 1 is used as an example of an information processing device, an information processing device according to the present embodiment is not limited to an HMD 1, and may also be a display control system formed from a smartphone and an eyeglasses-style display, for example. The smartphone (information processing device) is connectable to the eyeglasses-style display in a wired or wireless manner, and is able to transmit and receive data.
Herein, the eyeglasses-style display includes a wearing unit having a frame structure that wraps halfway around the back of the head from either side of the head, and is worn by a user by being placed on the pinna of either ear, similarly to the HMD 1 illustrated in
Also, the eyeglasses-style display is provided with an image capture lens for capturing the user's gaze direction while in the worn state, similarly to the HMD 1 illustrated in
The smartphone (information processing device) includes functions similar to the main controller 10, and estimates quantities of ingredients to be cooked and seasonings used from a captured image, and generates a cooking indices display image indicating the estimated cooking indices. Additionally, the smartphone (information processing device) transmits a generated cooking indices display image to the eyeglasses-style display, and the cooking indices display image is displayed on the display units of the eyeglasses-style display.
Application is also conceivable to an eyeglasses-style device that, although similar in shape to an eyeglasses-style display, does not include display functions. In this case, the cooking process is captured by a camera, provided on the eyeglasses-style device, that captures the wearer's (the user's) gaze direction, and a captured image is transmitted to the smartphone (information processing device). Subsequently, the smartphone (information processing device) generates a cooking indices display image indicating cooking indices on the basis of the cooking process depicted in the captured image, which is displayed on a display of the smartphone.
Furthermore, although the foregoing embodiments described the used quantity estimation unit 10a determining quantities of ingredients and seasonings used and the cooking method identification unit 10b identifying the cooking method on the basis of a captured image analysis result from the captured image analyzer 13 of the HMD 1, such a captured image analyzing process may also be conducted in the cloud. The HMD 1 sends a captured image of the cooking process to the cloud via the communication unit 21, receives a result that has been analyzed in the cloud (on an analysis server, for example), and on the basis thereof, conducts used quantity estimation and cooking method identification with the used quantity estimation unit 10a and the cooking method identification unit 10b.
Additionally, the present technology may also be configured as below.
(1)
An information processing device including:
an estimation unit configured to estimate a used quantity of at least one of an ingredient to be cooked and a seasoning used in cooking, on the basis of a signal detected by a sensor;
an index computation unit configured to compute a prescribed cooking index according to the estimated result by the estimation unit; and
a notification control unit configured to perform control to issue a notification of the cooking index computed by the index computation unit.
(2)
The information processing device according to (1), further including
a cooking method identification unit configured to identify a cooking method,
wherein the index computation unit converts the computed cooking index according to the cooking method identified by the cooking method identification unit.
(3)
The information processing device according to (1) or (2), further including
a display unit configured to display the cooking index according to the control by the notification control unit.
(4)
The information processing device according to any one of (1) to (3), wherein
the index computation unit accumulates the computed cooking index.
(5)
The information processing device according to any one of (1) to (4), wherein
when the cooking index computed by the index computation unit approaches a prescribed value, the notification control unit performs control to present a warning notification to a user.
(6)
The information processing device according to (5), wherein
the prescribed value is a limit index, and
the cumulative dish index is compared to a plurality of limit indices.
(7)
The information processing device according to (6), wherein
the plurality of limit indices include a prescribed value prioritizing health and a prescribed value prioritizing taste.
(8)
The information processing device according to (4), wherein
the notification control unit performs control to present a warning notification to a user according to a comparison result of the cooking index accumulated by the index computation unit and a prescribed value for when cooking is complete.
(9)
The information processing device according to any one of (1) to (8), wherein
the cooking index is a value of mass or calorific value of the ingredient to be cooked or the seasoning.
(10)
The information processing device according to (9), wherein
the value of mass or the calorific value is a value of mass or calorific value per nutritional component.
(11)
The information processing device according to any one of (1) to (10), wherein
the sensor is a concentration sensor, an imaging sensor, or a smell sensor.
(12)
The information processing device according to any one of (1) to (11), wherein
the information processing device is a head-mounted-style or eyeglasses-style head-mounted display.
(13)
The information processing device according to any one of (1) to (12), wherein
the information processing device is a smartphone, a mobile phone, a tablet, a PC, a digital camera, or a digital camcorder.
(14)
A storage medium having a program stored therein, the program causing a computer to function as:
an estimation unit configured to estimate a used quantity of at least one of an ingredient to be cooked and a seasoning used in cooking, on the basis or a signal detected by a sensor;
an index computation unit configured to compute a prescribed cooking index according to the estimated result by the estimation unit; and
a notification control unit configured to perform control to issue a notification of the cooking index computed by the index computation unit.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2013-039356 | Feb 2013 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2013/083229 | 12/11/2013 | WO | 00 |
