Patent Application
20070299807
These and/or other aspects and advantages of the invention will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
Reference will now be made in detail to the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments are described below to explain the present invention by referring to the figures.
The term ‘module’, as used herein, means, but is not limited to, a software or hardware component, such as a Field Programmable Gate Array (FPGA) or Application Specific Integrated Circuit (ASIC), which performs certain tasks. A module may advantageously be configured to reside on the addressable storage medium and configured to execute on one or more processors. Thus, a module may include, by way of example, components, such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided for in the components and modules may be combined into fewer components and modules or further separated into additional components and modules.
An apparatus to organize a user's life pattern according to an embodiment of the present invention collects data indicating a user's life pattern, and provides a cartoon diary that sums up the user's experiences based on the collected data. In order to determine the user's life pattern, the apparatus to organize a user's life pattern may use a variety of data, for example, data received from an external apparatus, data internally generated by the apparatus to organize a user's life pattern, or data stored in an external storage. In detail, examples of the data used by the apparatus to organize a user's life pattern include data provided by websites such as weather, atmospheric temperature, and wind velocity data, data provided by personal information managers (PIMs) such as age, sex, occupation, hobby, habit, address, and anniversary data, and log data regarding making phone calls, sending/receiving Short Message Service (SMS) messages, taking photos, and playing back music files.
The apparatus to organize a user's life pattern may be realized as a digital apparatus. Here, the digital apparatus is an apparatus equipped with a digital circuit capable of processing digital data. Examples of the digital apparatus include a computer, a digital camera, a digital home appliance, a digital telephone, a digital projector, a home server, a digital video recorder, a digital satellite broadcast receiver, a set-top box, and a digital TV broadcast receiver. It will hereinafter be assumed that the apparatus to organize a user's life pattern is realized as a mobile phone, for example.
The input module 110 receives a command from a user and may include a plurality of keys, e.g., a power key and a plurality of letter keys. Each of the keys included in the input module 110 generates a key signal when being hit by the user.
The storage module 115 stores a geographic information table which is illustrated in
The storage module 115 also stores a plurality of panels needed to create an image corresponding to landmarks to be included in a diary, as illustrated in
The storage module 115 also stores a first panel information mapping table including panel information regarding landmarks, and a second panel information including panel information regarding times and places. The first panel mapping information and the second panel mapping information will hereinafter be described in detail with reference to
Referring to
The storage module 115 may also store location data and various log data collected by the data collection module 120 and images corresponding to landmarks. The storage module 115 may be realized as a non-volatile memory device such as a read only memory (ROM), a programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory, may be realized as a volatile memory device such as a random access memory (RAM), or may be realized as a storage medium such as a hard disc drive (HDD). But it is not limited thereto.
The data collection module 120 collects data indicating the user's life pattern. In other words, the data collection module 120 collects data regarding the use of the apparatus 100, for example, log data regarding making phone calls, sending/receiving SMS messages, taking photos, and playing back multimedia content. In detail, when the user transmits a text message, the data collection module 120 collects data regarding, for example, the content of the text message, the recipient of the text message, and the time of transmission of the text message. When the user makes a call, the data collection module 120 collects data regarding, for example, the recipient of the call, the length of the call, and call traffic. When the user plays back a music file (a DMB file, a video file, and etc), the data collection module 120 collects data regarding, for example, the genre and title of the song or music, the name of the singer (the names of actors/actresses in the case of movie files), the number of times the music file has been played back, and the length of the song or music.
The data collection module 120 may also collect location information of the user. For this, the data collection module 120 may include a Global Positioning System (GPS). The GPS receives a coordinate value corresponding to a current location of the user. The data collection module 120 may also collect various data such as weather, atmospheric temperature, wind velocity, and news data from websites.
The analysis module 130 statistically analyzes the data collected by the data collection module 120. For this, the analysis module 130 may include a location information analysis unit 131 and a log data analysis unit 132.
The location information analysis unit 131 analyzes location data provided by the data collection module 120. In detail, when the location information analysis unit 131 is provided with the coordinate value corresponding to the current location of the user by the data collection module 120, the location information analysis unit 131 searches the geographic information table illustrated in
The log data analysis module 132 creates log context by statistically analyzing the log data provided by the data collection module 120. For this, the log data analysis module 132 may use various preprocessing functions, for example, a daily frequency function, a time interval function, an instant impact function, a daily impact function, an event time span function, a daily time portion function, and a daily priority function. The definitions of the daily frequency function, the time interval function, the instant impact function, the daily impact function, the event time span function, the daily time portion function, and the daily priority function are presented in Table 1 below.
For example, in order to analyze log data regarding the playback of a music file, the log data analysis unit 132 may use the preprocessing functions presented in Table 1 to perform impact analysis, and can thus determine how many times the music file has been played back during one day, how much time has elapsed since the least recent playback of the music file, a time span between the time when the music file begins to be played back and the time when the playback of the music file ends (in other words, for how many hours the music file has been played back), and whether the playback of the music file has been performed intensively within a short period of time. Impact analysis will hereinafter be described in detail with reference to
Log context illustrated in
The landmark probability estimating module 140 statistically estimates landmarks based on the results of the analysis performed by the location information analysis unit 131 and the log context provided by the log data analysis module 132. In other words, the landmark probability estimating module 140 estimates landmarks associated with the user's action, emotional state, the circumstances of the user, and an event.
In order to reason landmarks associated with the user's action, emotional state, the circumstances of the user, and an event, the landmark probability reasoning module 140 may use a Bayesian network. A Bayesian network is a graph of nodes and arcs representing the relations among variables included in data. Nodes of a Bayesian network represent random variables, and arcs represent connections among the nodes.
A Bayesian network may be designed as a module in order to efficiently perform computation needed for landmark estimating. In detail, the user's actions may include taking a rest, sleeping, having a meal, studying, exercising, attending school, going home from school, taking classes, enjoying entertainment, having a get-together, taking a trip, climbing a mountain, taking a walk, go shopping, and/or dining out. The user's emotions may be classified into positive emotions such as joy and negative emotions such as anger and irritability. The circumstances of the user may be classified into time circumstances, spatial circumstances, the weather, the state of a device, and the circumstances of people around the user. A Bayesian network may be designed as a module for each of the aforementioned classifications, wherein the Bayesian network may be a hierarchical Bayesian network having a hierarchical structure.
The landmark probability estimating module 140 estimates landmarks using one or more hierarchical Bayesian networks. For this, the landmark probability estimating module 140 inputs log context currently being discovered regarding, for example, photos, music file playback records, call records, SMS records, weather information, current location information, information indicating whether the user is currently on the move, the movement speed of the user, and the user's previous actions, to a Bayesian network, thereby reasoning landmarks. This will hereinafter be described in further detail with reference to
Assume that log context currently being discovered is as indicated by Table 2 below.
The landmark probability estimating module 140 inputs as evidence the log context presented in Table 2 to the hierarchical Bayesian network corresponding to the item ‘dining out’, thereby calculating the probabilities of the input nodes of the hierarchical Bayesian network. In other words, referring to
Referring to
Likewise, the landmark probability estimating module 140 calculates the probability that the user is dining out based on the probability that the user is in a place where the user visits ordinarily, the probability that the user is in a fancy restaurant, and the probability that the user is having a meal.
If log context currently being discovered is as indicated by Table 3, results obtained by inputting as evidence the current log context to the hierarchical Bayesian network corresponding to the item ‘dining out’ are illustrated in
In detail, referring to the hierarchical Bayesian network corresponding to the item ‘dining out’, the probability that the user is drinking tea is affected by the probability that the user has already had a meal, the probability that the user has already taken a walk, the probability that the user is in a coffee shop, and the probability that it is mealtime. Table 3 indicates that the user is currently in a coffee shop and that it is not mealtime. Accordingly, the probability that the user is drinking tea is determined to be 95% based on the evidence input to the hierarchical Bayesian network corresponding to the item ‘dining out’.
Likewise, the probability that the user is having a snack is affected by the probability that the user is in a fast food restaurant, the probability that it is lunch time, and the probability that it is dinner time. Table 3 indicates that the user is currently in a coffee shop and that it is not mealtime. Accordingly, the probability that the user is in a fast food restaurant, the probability that it is lunch time, and the probability that it is dinner time are all 0%. Thus, the probability that the user is having a snack is as low as 10%.
Likewise, the landmark probability estimating module 140 calculates the probability that the user is having Korean food and the probability that the user is having western food. Thereafter, the landmark probability estimating module 140 calculates the probability that the user is dining out based on the probability that the user is having a meal, the probability that the user is in a place where the user visits ordinarily, and the probability that the user is in a fancy restaurant. By referencing the evidence presented in Table 3, the landmark probability estimating module 140 determines the probability that the user is dining out to be 26%.
The landmark probability estimating module 140 inputs log context currently being discovered to a hierarchical Bayesian network corresponding to each item as evidence in the aforementioned manner, thereby estimating landmarks.
Thereafter, the landmark probability estimating module 140 inputs the landmarks and the log context to each Bayesian network as evidence, thereby secondarily estimating landmarks. In this case, the landmark probability estimating module 140 may use a virtual node method to precisely reflect the landmarks to be input as evidence to each Bayesian network. The virtual node method is a method involving the adding of virtual nodes to reflect statistical evidence to a Bayesian network and applying the probability of evidence using the conditional probability values (CPVs) of the virtual nodes. The virtual node method is well taught by E. Horvitz, S. Dumais, P. Koch, “Learning predictive models of memory landmarks,” CogSci 2004: 26th Annual Meeting of the Cognitive Science Society, 2004, which is incorporated herein by reference, and thus, a detailed description thereof will be omitted.
Thereafter, the landmark probability estimating module 140 calculates causal relationships between the landmarks obtained through the secondary estimating operation and the strengths of the connections between the landmarks obtained through the secondary estimating operation. In order to calculate the strengths of the connections between the landmarks obtained through the secondary estimating operation, the landmark probability estimating module 140 may use a NoisyOR weight. A NoisyOR weight represents the strength of a connection between conditional probabilities for each cause used in a NoisyOR Bayesian network model, which is a Bayesian probability table calculation method capable of reducing designing and learning costs. A NoisyOR weight can be obtained by converting an ordinary conditional probability table (CPT) into a NoisyOR CPT, and this will hereinafter be described with reference to
The ordinary CPT illustrated in
Referring to
Once the strengths of the connections between the landmarks ‘busy time’ and ‘spam message’ and the landmark ‘irritating SMS message’ are determined in the aforementioned manner, the landmark probability estimating module 140 extracts a meaningful connection path by referencing the strengths of the connections between the landmarks ‘busy time’ and ‘spam message’ and the landmark ‘irritating SMS message’. In other words, if the strength of a connection between a pair of nodes is less than a predefined threshold, the landmark probability estimating module 140 deems the connection between the nodes less meaningful, and removes the nodes from a corresponding Bayesian network. For example, referring to
The landmark selection module 150 selects one or more landmarks to be included in a diary from the landmarks obtained by the landmark probability estimating module 140, and determines which of the selected landmarks are to be emphasized. The selection of landmarks will hereinafter be described in further detail with reference to
Thereafter, the landmark selection module 150 determines which of the selected landmarks are to be emphasized. For example, the landmark selection module 150 may select one or more landmarks corresponding to a climax from the landmarks included in the first and second groups 610 and 620 as landmarks to be emphasized. In other words, as illustrated in
According to an aspect of the present embodiment, various story lines can be obtained from the landmarks selected by the landmark selection module 150. For example, referring to
The coding module 160 describes one or more images corresponding to the landmarks to be included in a diary using a markup language such as eXtensible Markup Language (XML) with reference to the user profile information and the panel information mapping tables stored in the storage module 115.
The image generation module 170 extracts one or more panels from the storage module 115 with reference to the XML image description provided by the coding module 160, and synthesizes the extracted panels, thereby creating an image. For example, if the XML image description provided by the coding module 160 is as illustrated in
One or more panels associated with an emphasis effect may be chosen for landmarks to be emphasized, and the extracted panels may be synthesized. This will hereinafter be described in further detail with reference to
An image illustrated in
The image group creation module 175 arranges one or more images generated by the image generation module 170 according to predetermined rules, thereby creating a diary. The predetermined rules may include at least any one of a time rule, a space rule, and a correlation rule. For example, when using the correlation rule, the image group creation module 175 may arrange the images generated by the image generation module 170 on the basis of a place associated with a landmark. In other words, the image group creation module 175 may arrange only the images associated with a predetermined place according to a predetermined time order, thereby generating an image group.
The display module 180 visually displays results of executing a command input by the user. For example, the display module 180 may display an image generated by the image generation module 170. The display module 180 may be realized as a flat panel display device such as a liquid crystal display (LCD) device. However, it is not limited thereto.
The control module 190 connects and controls the input module 110, storage module 115, data collection module 120, analysis module 130, landmark probability estimating module 140, landmark selection module 150, coding module 160, image generation module 170, the image group creation module 175, and the display module 180 in response to a key signal provided by the input module 110.
In operation S712, the analysis module 130 statistically analyzes the log data collected by the data collection module 120 using various preprocessing functions. For example, the analysis module 130 may analyze log data regarding the playback of a music file, thereby determining how many times the music file has been played back during one day, for how long the music file has been played back at a time, and for how many hours the music file has been played back during one day.
Log context is generated as a result of the analysis performed by the analysis module 130. In operation S713, the landmark probability estimating module 140 performs a primary landmark estimating operation by inputting the log context to each Bayesian network. For example, if the log context presented in Table 1 is input to the Bayesian network illustrated in
In operation S714, the landmark probability estimating module 140 performs a secondary landmark estimating operation by inputting the primary landmarks and the log context to each Bayesian network.
In operation S715, the landmark probability estimating module 140 determines the connections among a plurality of secondary landmarks obtained as the results of the secondary landmark estimating operation and calculates the strengths of the connections among the secondary landmarks. In order to calculate the strengths of the connections among the secondary landmarks, the landmark probability estimating module 140 may convert a CPT created based on the connections among the secondary landmarks into a NoisyOR CPT.
In operation S716, once the strengths of the connections among the secondary landmarks are determined based on the NoisyOR CPT, the landmark probability estimating module 140 extracts one or more landmarks that are meaningful from the secondary landmarks by referencing the strengths of the connections among the secondary landmarks. In other words, the landmark probability estimating module 140 selects those of the secondary landmarks corresponding to a connection strength greater than a predetermined threshold.
Referring to
In operation S730, the coding module 160 describes one or more images corresponding to the landmarks to be included in a diary, including the landmarks to be emphasized, using a markup language with reference to user profile information and panel information mapping tables. As a result, the coding module 160 may provide the XML image description presented in
In operation S740, the image generation module 170 extracts one or more panels needed to create images from the storage module 115 with reference to the XML image description provided by the coding module 160, and synthesizes the extracted panels, thereby creating one or more images corresponding to the landmarks to be included in a diary. In this case, the image generation module 170 may choose a panel appropriate for each of the landmarks to be emphasized, and synthesizes the chosen panel with other panels. The image generation module 170 may provide the image illustrated in
In operation S750, the image group creation module 175 creates an image group, i.e., a diary, by arranging the images generated by the image generation module 170 according to predetermined rules. The image group generated by the image group creation module 175 is displayed by the display module 180 in response to a command input to the input module 110 by the user.
As described above, the apparatus and method to organize a user's life pattern according to an embodiment of the present invention can summarize a user's life pattern into a small number of extraordinary events, systematically combine the results of the summarization using a small number of images, and visualize the result of the combination. Thus, the apparatus and method to organize a user's life pattern according to the present invention can help the user's memory, and satisfy the demand for emotion/life pattern-based estimating.
Although a few embodiments of the present invention have been shown and described, it would be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the claims and their equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2006-0049906 | Jun 2006 | KR | national |
