Patent Application
20160357774
Modern mobile devices (e.g., smartphones) may contain many applications. Each application may be designed to perform one or more specific functions. For instance, an application may be designed to play music or a video. As modern mobile devices become more integrated with modern day life, the number of applications stored on the mobile devices increases. It is not uncommon for modern mobile phones to have hundreds of applications. Having numerous applications may allow the mobile device to be particularly useful to the user; however, it may be difficult and time consuming for the user to find and run a desired application amongst all of the available applications.
Embodiments can provide systems, methods, and apparatuses for suggesting one or more applications to a user of a computing device based on an event. Examples of a computing device are a phone, a tablet, a laptop, or a desktop computer. Example events include connecting to an accessory device and changing a power state (e.g., to awake from off or sleeping).
A prediction model can correspond to a particular event. The suggested application can be determined using one or more properties of the computing device. For example, a particular sub-model can be generated from a subset of historical data that are about user interactions after occurrences of the event and that are gathered when the device has the one or more properties (e.g., user interactions of which application is selected after the event of connecting to one's car, with a property of a particular time of day). A tree of sub-models may be determined corresponding to different contexts of properties of the computing device. And, various criteria can be used to determine when to generate a sub-model, e.g., a confidence level in the sub-model providing a correct prediction in the subset of historical data and an information gain (entropy decrease) in the distribution of the historical data relative to a parent model.
Other embodiments are directed to systems, portable consumer devices, and computer readable media associated with methods described herein.
A better understanding of the nature and advantages of embodiments of the present invention may be gained with reference to the following detailed description and the accompanying drawings.
A “user interface” corresponds to any interface for a user to interact with a device. A user interface for an application allows for a user to interact with the application. The user interface could be an interface of the application when the application is running. As another example, the user interface can be a system interface that provides a reduced set of applications for users to select from, thereby making it easier for a user to use the application.
A “home screen” is a screen of a device that appears when a device is first powered on. For a mobile device, a home screen often shows an array of icons corresponding to various applications that can be run on the device. Additional screens may be accessed to browse other applications not appearing on the home screen.
A “lock screen” is a screen that is shown when a user has not been authenticated, and therefore the device is locked from most usage. Some functionality can be exposed, e.g., a camera. In some embodiments, if a user interface corresponding to a suggested application is exposed on a lock screen, then some functionality associated with the suggested application can be obtained. For example, the application could be run. The functionality may be limited if the application is run from a lock screen, and the limited functionality may be expanded when the user is authenticated.
“Contextual information” refers collectively to any data that can be used to define the context of a device. The contextual information for a given context can include one or more contextual data, each corresponding to a different property of the device. The potential properties can belong to different categories, such as a time category or a location category. We contextual data is used as a feature of a model (or sub-model), the data used to train the model can include different properties of the same category. A particular context can correspond to a particular combination of properties of the device, or just one property.
A “confidence level” corresponds to a probability that a model can make a correct prediction (i.e., at least one of the predicted application(s) was chosen after the event) based on the historical data. An example of a confidence level is the percentage of events where a correct prediction was made. Another example uses a cumulative distribution function (CDF) of a probability distribution (e.g., beta distribution) generated from the number of correct and incorrect predictions. The CDF can be computed by integrating the probability distribution. In various implementations, the confidence level can be the amount of increase in the CDF past an input value (e.g., between 0 and 1, with 1 corresponding to a correct prediction) or the input value providing a specified CDF past the input value. The probability of an application being selected can be required to be a threshold probability, which is the corollary of the model having a confidence level above a confidence threshold. The confidence level can be inversely proportional to a measure of entropy, and thus an increase in confidence level from a parent model to a sub-model can correspond to decrease in entropy.
Embodiments can provide a customized and personalized experience for suggesting an application to a user of a device, thereby making use of the device easier. A user can have an extensive set of interactions with the user device (e.g., which applications are launched or are running in association with an event) that occur after specific events. Examples of a computing device are a phone, a tablet, a laptop, or a desktop computer. Example events include connecting to an accessory device and changing a power state (e.g., to awake from off or sleeping).
Each data point in the historical data can correspond to a particular context (e.g., corresponding to one or more properties of the device), with more and more data for a particular context being obtained over time. This historical data for a particular event can be used to suggest an application to a user. As different users will have different historical data, embodiments can provide a personalized experience.
To provide an accurate personalized experience, various embodiments can start with a broad model that is simply trained without providing suggestions or that suggests a same set of application(s) for a variety of contexts. With sufficient historical data, the broad model can be segmented into sub-models, e.g., as a decision tree of sub-models, with each sub-model corresponding to a different subset of the historical data. Then, when an event does occur, a particular sub-model can be selected for providing a suggested application corresponding to a current context of the device. Various criteria can be used to determine when to generate a sub-model, e.g., a confidence level in the sub-model providing a correct prediction in the subset of historical data and an information gain (entropy decrease) in the distribution of the historical data relative to a parent model.
Accordingly, some embodiments can decide when and how to segment the user's historical data in the context of user recommendations. For example, after collecting a period of user activity, embodiments can accumulate a list of possible segmentation candidates (e.g. location, day of week, etc.). Embodiments can also train a model on the entire dataset and compute a metric of the confidence in the joint distribution of the dataset and the model. A set of models can be trained, one for each of the segmented datasets (i.e., subsets), and then measure the confidence of each of the data model distributions. If the confidence of all data model distributions is admissible, embodiments can perform the segmentation (split) and then recursively examine the segmented spaces for additional segmentations.
In this way, some embodiments can use inference to explore the tradeoff between segmentation and generalization, creating more complex models for users who have more distinct, complex patterns, and simple, general models for users who have noisier, simpler patterns. And, some embodiments can generate a tree of probabilistic models based on finding divergence distributions among potential candidate models.
Embodiments can suggest an application based upon an event, which may be limited to certain predetermined events (also called triggering events). For instance, a music application can be suggested when headphones are inserted into a headphone jack. In some embodiments, contextual information may be used in conjunction with the event to identify an application to suggest to a user. As an example, when a set of headphones are inserted into a headphone jack, contextual information relating to location may be used. If the device is at the gym, for instance, application A may be suggested when headphones are inserted into the headphone jack. Alternatively, if the device is at home, application B may be suggested when the headphones are inserted into the headphone jack. Accordingly, applications that are likely to be used under certain contexts may be suggested at an opportune time, thus enhancing user experience.
At block 110, an event is detected. In some embodiments, it can be determined whether the event is a triggering event for suggesting an application. In some implementations, a determination of a suggested application is only made for certain predetermined events (e.g., triggering events). In other implementations, a determination of the suggested application can be made for dynamic list of events, which can be updated based on historical user interactions with applications on the device.
In some embodiments, a triggering event can be identified as sufficiently likely to correlate to unique operation of the device. A list of events that are triggering events can be stored on the device. Such events can be a default list and be maintained as part of an operating system and may or may not be configurable by a user.
A triggering event can be an event induced by a user and/or an external device. For instance, the triggering event can be when an accessory device is connected to the mobile device. Examples include inserting headphones into a headphone jack, making a Bluetooth connection, turning on the device, waking the device up from sleep, arriving at a particular location (e.g., a location identified as being visited often), and the like. In this example, each of these events can be classified as a different triggering event, or the triggering event can collectively be any accessory device connecting to the mobile device. As other examples, a triggering event can be a specific interaction of the user with the device. For example, the user can move the mobile device in a manner consistent with running, where a running state of the device is a triggering event. Such a running state (or other states) can be determined based on sensors of the device.
At block 120, an application associated with the event is identified. As an example, a music application can be identified when the headphones are inserted into the headphone jack. In some embodiments, more than one application can be identified. A prediction model can identify the associated application, where the prediction model may be selected for the specific event. The prediction model may use contextual information to identify the application, e.g., as different application may be more likely to be used in different contexts. Some embodiments can identify applications only when there is a sufficient probability of being selected by a user, e.g., as determined from historical interactions of the user with the device.
The prediction model can be composed of sub-models, each for different combinations of contextual data. The different combinations can have differing amounts of contextual data. The sub-models can be generated in a hierarchical tree, with the sub-models of more specific combinations being lower in a hierarchical tree. In some embodiments, a sub-model can be generated only if the sub-model can predict an application with greater accuracy than a model higher in the tree. In this manner, a more accurate prediction can be made for which application the user will select. In some embodiments, the prediction model and sub-models may identify the top N applications (e.g., a fixed number of a percentage) that are chosen by the user after the event when there is a particular combination of contextual data.
Contextual information may specify one or more properties of the device for a certain context. The context may be the surrounding environment (type of context) of the device when the triggering event is received. For instance, contextual information may be the time of day that the event is detected. In another example, contextual information may be a certain location of the device when the event is detected. In yet another example, contextual information may be a certain day of year at the time the triggering event is detected. Such contextual information may provide more meaningful information about the context of the device such that the prediction engine may accurately suggest an application that is likely to be used by the user in that context. Accordingly, prediction engine utilizing contextual information may more accurately suggest an application to a user than if no contextual information were utilized.
At block 130, an action is performed in association with the application. In an embodiment, the action may be the displaying of a user interface for a user to select to run the application. The user interface may be provided in various ways, such as by displaying on a screen of the device, projecting onto a surface, or providing an audio interface.
In other embodiments, an application may run, and a user interface specific to the application may be provided to a user. Either of the user interfaces may be provided in response to identifying the application, e.g., on a lock screen. In other implementations, a user interface to interact with the application may be provided after a user is authenticated (e.g., by password or biometric), but such a user interface would be more specific than just a home screen, such as a smaller list of suggested applications to run.
As described herein, one aspect of the present technology is the gathering and use of data available from various sources to suggest applications to a user. The present disclosure contemplates that in some instances, this gathered data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data can include location-based data, home addresses, or any other identifying information.
The present disclosure recognizes that the use of such personal information data, in the present technology, can be used to the benefit of users. For example, the personal information data can be used to suggest an application that is of greater interest to the user. Accordingly, use of such personal information data enables calculated control of the delivered content. Further, other uses for personal information data that benefit the user are also contemplated by the present disclosure.
The present disclosure further contemplates that the entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal information data will comply with well-established privacy policies and/or privacy practices. In particular, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining personal information data private and secure. For example, personal information from users should be collected for legitimate and reasonable uses of the entity and not shared or sold outside of those legitimate uses. Further, such collection should occur only after receiving the informed consent of the users. Additionally, such entities would take any needed steps for safeguarding and securing access to such personal information data and ensuring that others with access to the personal information data adhere to their privacy policies and procedures. Further, such entities can subject themselves to evaluation by third parties to certify their adherence to widely accepted privacy policies and practices.
Despite the foregoing, the present disclosure also contemplates embodiments in which users selectively block the use of, or access to, personal information data. That is, the present disclosure contemplates that hardware and/or software elements can be provided to prevent or block access to such personal information data. For example, users can select not to provide location information for targeted content delivery services. In yet another example, users can select to not provide precise location information, but permit the transfer of location zone information.
Each time a particular event occurs (e.g., plugging in headphones or powering up the device), the device can track which application(s) is used in association with the event. In response to each occurrence of the particular event, the device can save a data point corresponding to a selected application, action performed with the application, and the event. In various embodiments, the data points can be saved individually or aggregated, with a count being determined for the number of times a particular application is selected, which may include a count for a specific action. Thus, different counts a determine for different actions for a same selected application. This historical data the previous user interactions with the device can be used as an input for determining the prediction model, and for determining whether and how many sub-models are to be created.
Once a particular event is detected, a prediction model corresponding to the particular event can be selected. The prediction model would be determined using the historical data corresponding to the particular event as input to a training procedure. However, the historical data might occur in many different contexts (i.e., different combinations of contextual information), with different applications being selected in different contexts. Thus, in aggregate, the historical data might not provide an application that will clearly be selected with a particular event occurs.
A model, such as a neural network or regression, can be trained to identify a particular application for a particular context, but this may be difficult when all of the corresponding historical data is used. Using all the historical data can result in overfitting the prediction model, and result in lower accuracy. Embodiments of the present invention can segment the historical data into different input sets of the historical data, each corresponding to different contexts. Different sub-models can be trained on different input sets of the historical data.
Segmentation can improve performance of a machine learning system. In one step of segmentation, the input space can be divided into two subspaces, and each of these subspaces can be solved independently with a separate sub-model. Such a segmentation process can increase the number of free parameters available to the system and can improve training accuracy, but at the cost of diluting the amount of data in each model, which can reduce the accuracy of the system when the system is shown new data, e.g., if the amount of data for a sub-model is small. Embodiments can segment the input space only when the joint distributions of the data and the model parameters created from the resulting subspaces are confident.
A. Different Models Based on Different Contextual Data
When a particular event occurs, the device could be in various contexts, e.g., in different locations, at different times, at different motion states of the device (such as running, walking, driving in a car, or stationary,), or at different states of power usage (such as being turned or transitioning from a sleep mode). The contextual information can be retrieved in association with the detected event, e.g., retrieved after the event is detected. The contextual information can be used to help predict which application might be used in connection with the detected event. Different motion states can be determined using motion sensors, such as an accelerometer, a gyrometer, or a GPS sensor.
Embodiments can use the contextual information in various ways. In one example, a piece of the contextual data (e.g., corresponding to one property of the device) can be used as a feature of a particular sub-model to predict which application(s) are most likely to be selected. For example, a particular location of the device can be provided as an input to a sub-model. These features are part of the composition of the sub-model.
In another example, some or all of the contextual data of the contextual information can be used in a segmentation process. A certain piece of contextual data can be used to segment the input historical data, such that a particular sub-model is determined only using historical data corresponding to the corresponding property of that piece of contextual data. For example, a particular location of the device would not be used as an input to the sub-model, but would be used to select which sub-model to use, and correspondingly which input data to use to generate the particular sub-model.
Thus, in some embodiments, certain contextual data can be used to identify which sub-model to use, and other contextual data can be used as input to the sub-model for predicting which application(s) that the user might interact with. A particular property (e.g., a particular location) does not correspond to a particular sub-model, that particular property can be used as a future (input) to the sub-model that is used. If the particular property does correspond to a particular sub-model, the use of that property can become richer as the entire model is dedicated to the particular property.
One drawback of dedicating a sub-model to a particular property (or combination of properties) is that there may not be a large amount of the historical data corresponding to that particular property. For example, the user may have only performed a particular event (e.g., plugging in headphones) at a particular location a few times. This limited amount of data is also referred as data being sparse. Data can become even more sparse when combinations of properties are used, e.g., a particular location at a particular time. To address this drawback, embodiments can selectively determine when to generate a new sub-model as part of a segmentation process.
B. Segmenting as More Data is Obtained
When a user first begins using a device, there would be no historical data for making predication about actions the use might take with an application after a particular event. In an initial mode, historical data can be obtained while no predictions are provided. As more historical data to obtained, determinations can be made about whether to segment the prediction model into sub-models. With even more historical data, sub-models can be segmented into further sub-models. When limited historical data is available for user interactions with the device, no actions can be taken or more general model can be used, as examples.
In this example, event model 205 can correspond to a particular event (e.g., connecting to a particular device, such as a car). Event model 205 can correspond to a top level of a prediction engine for the particular event. At the beginning, there can be just one model for the particular event, as minimal historical data is available. At this point, event model 205 may just track the historical data for training purposes. Event model 205 can make predictions and compared those predictions to the actual results (e.g., whether user to interact with predicted application within a specified time after the event is detected). If no applications have a probability greater than a threshold, no action may be performed when the particular event occurs.
In some embodiments, event model 205 only uses data collected for the particular device. In other embodiments, event model 205 can be seeded with historical data aggregated from other users. Such historical data may allow event model 205 can provide some recommendations, which can then allow additional data points to be obtained. For example, it can be tracked whether a user interacts with a suggested application via a user interface, which can provide more data points than just whether a user does select an application.
As more data is collected, a determination can be made periodically as to whether a segmentation should occur. Such a determination can be based on whether greater accuracy can be achieved via the segmentation. The accuracy can be measured as a level of probability that a prediction can be made, which is described in more detail below. For example, if an application can be predicted with a higher level of probability for a sub-model than with event model 205, then a segmentation may be performed. One or more other criteria can also be used to determine whether a sub-model should be created as part of segmentation process. For example, a criterion can be that a sub-model must have a statistically significant amount of input historical data before the sub-model is implemented. The requirement of the amount of data can provide greater stability to the sub-model, and ultimately greater accuracy as a model trained on a small amount of data can be inaccurate.
At segmentation 201, it is determined to segment event model 205 into gym sub-model 210 and another sub-model 240. This segmentation can occur the user has definitive behavior for a particular context. In this example, there is definitive behavior when the context is that the device is located at the gym, which may be a specific gym or any gym, as can be determined by cross-referencing a location restored locations of businesses. Such a cross-referencing can use external databases stored on servers. The definitive behavior can be measured when gym sub-model 210 can predict a correct application that is selected by the user with greater probability than event model 205.
As part of segmentation 201, the input historical data is used for generating gym sub-model 210 is used for generating a sub-model 240, which corresponds to all other contexts besides the gym. Other sub-model 240 can be used to predict applications that the user might interact with when the context is something other than the gym.
At segmentation 202 after more data has been gathered, it is determined that a further segmentation can be made from event model 205 to generate supermarket model 220. This determination may be made after a sufficient number of data points have been obtained at a supermarket such that supermarket model 220 can make a prediction with sufficient confidence. A sufficient confidence can be measured relative to the confidence obtained from other sub-model 240. Once supermarket sub-model 220 can predict an application greater confidence and the other sub-model 240, the segmentation can be performed. After segmentation 202, a sub-model 240 would correspond to any other context besides the gym and the supermarket.
At segmentation 203 after even more data has been gathered, it is determined that a segmentation can be made of gym sub-model 210. In this instance, it is determined that an application can be predicted with higher confidence in the historical data for the gym is segmented into specific times, specifically afternoon times (e.g., 12-4). Thus, when a user is at the gym in the afternoon, afternoon gym sub-model 211 can be used to predict which application(s) the user might interact with. If the user is the gym for any other times, gym sub-model 210 can be used, which is equivalent to having some other sub-model at a position in the tree, i.e., in a similar manner as other sub-model 240 is depicted.
At segmentation 204 after even more data has been gathered, it is determined that a further segmentation can be made of gym sub-model 210 to generate morning gym sub-model 212. In this instance, sufficient historical data has been gathered for morning times that an application can be predicted with greater accuracy than using a more general gym sub-model 210 (which would only use data not corresponding to afternoon gym sub-model 211).
1. Default Model
When a device is first obtain (e.g., brought) by a user, a default model can be used. The default model could apply to a group of events (e.g., all events designated as triggering events). As mentioned above, the default model can be seeded is aggregate data from other users. In some embodiments, the default model can simply pick the most popular application, regardless of the context, e.g., as not enough data is available for any one context. Once more data is collected, the default model can be discarded.
In some embodiments, the default model can have hardcoded logic that specifies predetermined application(s) to be suggested and actions to be performed. In this manner, a user can be probed for how the user responds (e.g., a negative response is a user does not select a suggested application), which can provide additional data that simply tracking for affirmative responses are user. In parallel with such a default model, a prediction model can be running to compare its prediction against the actual result. A prediction model can then be refined in response to the actual result. When the prediction model has sufficient confidence, the switch can be made from the default model to the prediction model. Similarly, the performance of a sub-model can be tracked. When the sub-model has sufficient confidence, the sub-model can be used for the given context.
2. Initial Training
A prediction model (e.g., event model 205) can undergo initial training using historical data collected so far, where the model does not provide suggestions to a user. This training can be called initial training. The prediction model can be updated periodically (e.g., every day) as part of the background process, which may occur when the device is charging and not in use. The training may involve optimizing coefficients of the model so as to optimize the number of correct predictions and compared to the actual results in historical data. In another example, the training may include identifying the top N (e.g., a predetermined number a predetermined percentage) applications actually selected. After the training, the accuracy of the model can be measured to determine whether the model should be used to provide a suggested application (and potential corresponding action) to the user.
Once a model is obtaining sufficient accuracy (e.g., top selected application is being selected with a sufficiently high accuracy), then the model can be implemented. Such an occurrence may not happen for a top-level model (e.g., event model 205), but may occur when sub-models are tested for specific contexts. Accordingly, such an initial training can be performed similarly for a sub-model.
As historical information accumulates through use of the mobile device, prediction models may be periodically trained (i.e., updated) in consideration of the new historical information. After being trained, prediction models may more accurately suggest applications and actions according to the most recent interaction patterns between the user and the mobile device. Training prediction models may be most effective when a large amount of historical information has been recorded. Thus, training may occur at intervals of time long enough to allow the mobile device to detect a large number of interactions with the user. However, waiting too long of a period of time between training sessions may hinder adaptability of the prediction engine. Thus, a suitable period of time between training sessions may be between 15 to 20 hours, such as 18 hours.
Training prediction models may take time and may interfere with usage of the mobile device. Accordingly, training may occur when the user is most unlikely going to use the device. One way of predicting that the user will not use the device is by waiting for a period of time when the device is not being used, e.g., when no buttons are pressed and when the device is not moving. This may indicate that the user is in a state where the user will not interact with the phone for a period of time in the near future, e.g., when the user is asleep. Any suitable duration may be used for the period of time of waiting, such as one to three hours. In a particular embodiment, the period of time of waiting is two hours.
At the end of the two hours, prediction models may be updated. If, however, the user interacts with the mobile device (e.g., presses a button or moves the device) before the end of the two hours, then the two hour time period countdown may restart. If the time period constantly restarts before reaching two hours of inactivity, then the mobile device may force training of prediction models after an absolute period of time. In an embodiment, the absolute period of time may be determined to be a threshold period of time at which user friendliness of the mobile device begins to decline due to out-of-date prediction models. The absolute period of time may range between 10 to 15 hours, or 12 hours in a particular embodiment. Accordingly, the maximum amount of time between training may be between 28 hours (18+10 hours) to 33 hours (18+15 hours). In a particular embodiment, the maximum amount of time is 30 hours (18+12 hours).
A prediction model and any sub-models can be organized as a decision tree, e.g., as depicted in
A. Traversing Decision Tree
A first hierarchal level of decision tree 300 corresponds to the location category. Node 310 corresponds to location 1, which may be defined as a boundary region (e.g., within a specified radius) of location 1. Node 320 corresponds to location 2. Node 330 corresponds to location 3. Node 340 corresponds to any other locations.
Each of nodes 310, 320, and 330 can be generated if the sub-model can predict an application with greater confidence when the contextual information corresponds to the particular location than the more general node 340 can. Nodes 310 and 320 have further children nodes while node 330 does not.
Embodiments can traverse decision tree 300 by searching whether any of the nodes 310, 320, and 330 match the contextual information for the particular occurrence. If the contextual information of the user device for a particular occurrence of the event indicates a context including location 3, then a match is found for node 330. Since node 330 does not have any further children nodes, the sub-model for node 330 can be used.
Node 310 has two children nodes: node 311 and node 312. Node 311 corresponds to a particular time (time 1), and node 312 corresponds to all other times that do not match to time 1. If the contextual information for a current occurrence of the event includes location 1 (and thus a match to node 310), then a search can be performed to determine whether the contextual information includes time 1 (i.e., matches to node 311). If the contextual information includes time 1 (i.e., in combination with location 1), then the sub-model for node 311 can be used to make the prediction. If the contextual information does not include time 1, then the sub-model for node 312 can be used to make the prediction.
Node 320 has two children nodes: node 321 and node 322. Node 321 corresponds to whether the user device is connected to a particular device (device 1), and node 322 corresponds to when the user device is not connected to device 1. If the contextual information for a current occurrence of the event includes location 2 (and thus match to node 310), then a search can be performed to determine whether the contextual information includes a connection to device (i.e., matches to node 321). If the contextual information includes a connection to device 1 (i.e., in combination with location 2), then the sub-model for node 321 can be used to make the prediction. If the contextual information does not include a connection to device 1, then the sub-model for node 322 can be used to make the prediction.
Accordingly, once a bottom of the tree is detected, the sub-model of the final node can be used to make the prediction. All of the branches of tree 300 can be deterministic with a final node always being selected for the same contextual information. Having all the nodes of a same hierarchal level of decision tree 300 correspond to a same category can avoid conflicts in selecting an applicable node. For example, there could be a conflict if a child node of event model 305 corresponded to time 1, as that might conflict with node 311. In such embodiments, nodes of the same level but underneath different parent nodes can correspond to different categories, as is the case for the set of nodes 311 and 312 and a set of nodes 321 and 322.
Once a sub-model has been selected based on the detected event and the contextual information, the selected sub-model can be used to predict what a more applications and any corresponding actions. In some embodiments, which action to take for a predicted application can depend on a level of confidence that the application is predicted.
B. Method
At block 410, the device detects an event at an input device. Examples of an input device are a headphone jack, a network connection device, a touch screen, buttons, and the like. The event may be any action where the mobile device interacts with an external entity such as an external device or a user. The event can be of a type that recurs for the device. Thus, historical, statistical data can be obtained for different occurrences of the event. Models and sub-models can be trained using such historical data.
At block 420, a prediction model corresponding to the event is selected. The selected prediction model may depend on the event. For instance, a prediction model designed for Bluetooth connections may be selected when the event relates to establishing a Bluetooth connection with an external device. As another example, a prediction model designed for headphone connections may be selected when the event relates to inserting a set of headphones into a headphone jack.
At block 430, one or more properties of the computing device are received. The one or more properties may be received by an application suggestion engine executing on the device. As mentioned herein, the properties can correspond to time, location, a motion state, a current or previous power state (e.g., on, off, or sleep), charging state, current music selection, calendar events, and the like. Such one or more properties can correspond to contextual data that defines a particular context of the device. The one or more properties can be measured at a time around the detection of the event, e.g., within some time period. The time period can include a time before and after the detection of the event, a time period just before the detection of the event, or just a time after the detection of the event.
At block 440, the one or more properties are used to select a particular sub-model of the prediction model. For example, a decision tree can be traversed to determine the particular sub-model. The particular sub-model can correspond to the one or more properties, e.g., in that the one or more properties can uniquely identify the particular sub-model. This may occur when the decision tree is defined to not have properties of different categories under a same parent node.
The particular sub-model can be generated using a particular subset of historical interactions of the user with the device. The particular subset can result from a segmentation process that increases accuracy by creating sub-models. The particular subset of historical interactions can be obtained by tracking user interactions with the device after occurrences of the event. The computing device has the one or more properties when the particular subset is obtained. Thus, a current context of the device corresponds to the context of the device within which the particular subset of historical interactions was obtained.
At block 450, the particular sub-model identifies one or more applications to suggest to the user. The one or more applications can have at least a threshold probability of at least one of the one or more applications being accessed by the user in association with the event. Predicting one of the one or more applications in the historical data can be identified as a correct prediction. The threshold probability can be measured in a variety of ways, and can use a probability distribution determined from the historical data, as is described in more detail below. For example, an average (mean) probability, a median probability, or a peak value of a probability distribution can be required to be above the threshold probability (e.g., above 0.5, equivalent to 50%). Thus, a confidence level can be an average value, median value, or a peak value of the probability distribution. Another example is that the area for the probability distribution above a specific value is greater than the threshold probability.
At block 460, a user interface is provided to the user for interacting with the one or more applications. For example, the device may display the identified applications to the user via an interface with which the user may interact to indicate whether the user would like to access the identified applications. For instance, the user interface may include a touch-sensitive display that shows the user one or more of the identified applications, and allows the user to access one or more of the applications identified by the device by interacting with the touch-sensitive display. The user interface can allow interactions on a display screen with fewer applications than provided on a home screen of the computing device.
As an example, one or more suggested applications can be provided on a lock screen. The user can select to open the applications from the lock screen, thereby making it easier for the user to interact with the application. The user interface can be provided on other screen, which may occur after activating a button to begin use of the device. For example, a user interface specific to the application can appear after authenticating the user (e.g., via password or biometric).
C. Example Models
In some embodiments, a model can select the top N applications for a given set (or subset) of data. Since the N application has been picked most in the past, it can be predicted that future behavior will mirror past behavior. N can be a predetermined number (e.g., 1, 2, or 3) or a percentage of applications, which may be the percentage of applications actually used in association with the event (i.e., not all applications on the device). Such a model can select the top N applications for providing to the user. Further analysis can be performed, e.g., to determine a probability (confidence) level for each of the N applications to determine whether to provide them to the user, and how to provide them to the user (e.g., an action), which may depend on the confidence level.
In an example where N equals three, the model would return the top three most launched apps when the event occurs with contextual information corresponding to the particular sub-model.
In other embodiments, a sub-model can use a composite signal, where some contextual information is used in determining the predicted application, as opposed to just using the contextual information to select the sub-model. For example, a neural network or a logistic regression model can use a location (or other features) and build sort of a linear weighted combination of those features to predict the application. Such more complex models may be more suitable when an amount of data for a sub-model is significantly large. Some embodiments could switch the type of sub-model used at a particular node (i.e., particular combination of contextual data) once more data is obtained for that node.
In some embodiments, the decision tree can be regenerated periodically (e.g., every day) based on the historical data at the time of regeneration. Thus, the decision tree can have different forms on different days. The generation of a child node (a further sub-model) can be governed by the confidence for predicting an application(s) is increased, also referred to as information gain. The generation of a child node can be also governed by whether the data for the child node is statistically significant. In some embodiments, all of the children at a given level (e.g., gym sub-model 210 and other sub-model 240) can be required to be statistically significant and provide information gain relative to the parent model.
In determining the nodes of the decision tree, segmentation can be performed in various ways to result in different decision trees. For example, a particular location and a particular time could both be used. In some embodiments, the properties of provides the highest increase in information gain (confidence) for predicting an application can be generated higher in the decision. Such a segmentation process can ensure a highest probability of predicting the correct application that a user will interact with.
A. Accuracy Distribution of a Model
The accuracy of a model can be tested against the historical data. For a given event, the historical data can identify which application(s) were used in association with the event (e.g., just before or just after, such as within a minute). For each event, the contextual data can be used to determine the particular model. Further, contextual data can be used as input features to the model.
In an example where the model (or sub-model) selects the top application, a number of historical data points where the top application actually was selected (launched) can be determined as a correct count, and a number of historical data points where the top application was not selected can be determined as an incorrect count. In an embodiment where N is greater than one for a model that selects the top N, the correct count can correspond to any historical data point where one of the top N applications was launched.
The correct count and the incorrect count can be used to determine a distribution specifying how accurate the model is. A binomial distribution can be used as the accuracy distribution. The binomial distribution with parameters m and p is the discrete probability distribution of the number of successes in a sequence of m independent yes/no experiments. Here the yes/no experiments are whether one of the predicted N applications is correct. For example, if the model predicted a music application would be launched, and a music application was launched, then the data point adds to the number of yes (True) experiments. If the music application was not launched (e.g., another application was launched or no application was launched), then the data point adds to the number of no (False) experiments
Under Bayes theorem,
B is the event of getting a specified determined correct count T and incorrect count F. A is the event of the predicted application being correct. P(A) is a prior (expected) probability of randomly selecting the correct application, which may be assumed to be 1, as no particular application would be expected more than any other, at least without the historical data. P(B) is the probability of the model being correct (which corresponds to the correct count divided by total historical events). P(B|A) is the likelihood function of getting the correct count T and the incorrect count F for a given probability r (namely event A, which can be taken to be 0.5 for equal probability of getting correct or incorrect). P(A|B) is the posterior probability is to be determined, namely the probability of one of the prediction application(s) being selected given the historical data B.
If there is a uniform prior, P(A) disappears and one is left with P(A|B)/P(B), which is equal to Beta[#correct, #incorrect], i.e., the beta distribution with parameters alpha=#correct and beta=#incorrect. Because the beta function is ill-defined for alpha=0 or beta=0, embodiments can assume an initial value of 1 for #correct and #incorrect. Beta[1+#correct, 1+#incorrect] is the binomial distribution.
For Bayesian statistics, the posterior probability p(θ|X) is the probability of the parameters θ (e.g., the actual selected application is one of the predicted application) given the evidence X (e.g., correct count and incorrect count of historical data). It contrasts with the likelihood function p (x|θ), which is the probability of the evidence X (e.g., correct count and incorrect count of historical data) given the parameters (e.g., the predicted application is selected for an event). The two are related as follows: Let us have a prior belief that the probability distribution function is P(θ) (e.g., expected probability that the selected application would be correct and observations X with the likelihood p(x|θ), then the posterior probability is defined as
The posterior probability can be considered as proportional to the likelihood times the prior probability.
Other accuracy distributions can be used. For example, one could use a Dirichlet distribution, which is a multivariate generalization of the beta distribution. The Dirichlet distribution is the conjugate prior of the categorical distribution and multinomial distribution, in a similar manner as the beta distribution is the conjugate prior of the binomial distribution. The Dirichlet has its probability density function returns the belief that the probabilities of K rival events are xi given that each event has been observed αi−1 times. The Dirichlet distribution can be used generate the entire histogram of app launches (i.e., predicted number of app launches for a particular event) as a multinomial distribution.
Instead, embodiments can separate them into two classes (correct and incorrect) so use a binominal distribution and do not have to provide the entire histogram. Other embodiments could use a Dirichlet distribution (the conjugate prior of the multinomial distribution) to try to solve the harder problem of describing the whole histogram, but this would take more data to be confident since more data needs to be explained.
B. Example Binomial Distributions
C. Statistically Significant
A model can be considered statistically significant if the model can accurately separate the cases where it is correct and wrong with sufficient confidence. The posterior probability distribution determined based on the number of incorrect and correct predictions can be used to determine whether the model is sufficiently accurate with enough confidence.
The required confidence level for statistical significance can be provided in various ways and can have various criteria. The average accuracy (#correct/#total) for the distribution, the peak of the distribution, or median of the distribution can be required to have a certain value. For example, the model can be required to be correct at least 50% of the time, e.g., as measured by the average of the distribution (i.e., greater than 0.5). The #correct/#total is also called the maximum likelihood estimation.
A further criterion (confidence level) can be for the confidence of the accuracy. The confidence can be measured by an integral of the distribution that is above a lower bound (e.g., area of the distribution that is above 0.25 or other value). The area under the distribution curve is also called the cumulative distribution function. In one embodiment, the criteria can be that 95% of the area of the PDF is above 0.25. The point at which the interval [x,1.0] covers 95% of the area under the PDF is called the “lower confidence bound”. Thus, if you were right twice and wrong once, you were right 66 percent of the time, but that's not statistically significant because the distribution is very broad, as in in
Some embodiments will only begin to use a model (e.g., the top-level model or a sub-model) when the model is sufficiently accurate and there is enough confidence in knowing the accuracy. For example, an initial model might get trained for a while before it is used. Only once the accuracy and confidence are above respective thresholds, then might an embodiment begin to use the model to provide suggestions to a user. In some embodiments, a requirement of a certain amount of the area of the PDF can provide a single criterion for determining whether to use the model, as the accuracy can be known to be sufficiently high if the area is sufficiently shifted to the right.
In some embodiments, an initial model could use data from other people to provide more statistics, at least at first. Then, once enough statistics are obtained, then only the data for the specific person can be used. Further, the data specific to the user can be weighted higher, so as to phase out the data from other people.
D. Information Gain (Entropy)
A comparison can be made between a first probability distribution of a model and a second probability distribution of a sub-model to determine whether segment the model. In some embodiments, the comparison can determine whether there is an information gain (e.g., Kullback-Leibler divergence), or equivalently a decrease in entropy. High entropy would have many applications having similar probability of being selected, with maximum entropy having the same probability for all applications. With maximum entropy the likelihood of selecting the correct application is the smallest, since all of the applications have an equal probability, and no application is more probable than another.
Such difference metrics can be used to determine whether a more accurate prediction (including confidence) can be made using the sub-model for the given context that the sub-model would be applied to. If the difference metric is greater than a difference threshold, then a segmentation can be performed. The difference metric can have a positive sign to ensure information is gained. Kullback-Leibler divergence can be used as the difference metric. Other example metrics include Gini impurity and variance reduction.
For example, if there was one model for everything, the model would only pick the top application (e.g., a music application) for all contexts. The music application would be the prediction for all contexts (e.g., the gym, for driving to work, etc.). As sub-models are generated for more specific contexts, then the predictions can become more specific, e.g., when the user goes to the gym a single app dominates, or a particular playlist dominates. Thus, there can be a peak in the number of selections for one application, and then everything else is at zero. Thus, a goal with the decision tree is to maximize the information gain (minimize the entropy).
Further sub-models can be identified when more specific contexts can provide more information gain. For example, the gym in the morning can be a more specific context for when a particular playlist dominates. As another example, connected to the car in the morning can provide for a more accurate prediction of a news application, since the historical data organizes more (decrease in entropy) to have selections of predominantly the news application (or a group of news applications).
E. When to Segment
As mentioned above, various embodiments can use one or more criteria for determining whether to segment a model to generate a sub-model. One criterion can be that a confidence level for making a correct prediction (one of a group of one or more predicted application is selected) is greater than a confidence threshold. For example, the average probability of a correct prediction is greater than an accuracy threshold (example of a confidence threshold). As another example, the CDF of the distribution above a specific value can be required to be above a confidence level.
Another criterion can be that using the sub-model, instead of the model, provides an information gain (decrease in entropy). For example, a value for the Kullback-Leibler divergence can be compared to a difference threshold. The one or more criteria for segmentation can guarantee that the sub-models will outperform the base model. The one or more criteria can be required for all of the sub-models of a parent model, e.g., gym sub-model 210 and other sub-model 240.
In some instances, the lower confidence bounds can decrease for two sub-models versus the parent model, but still have an information gain and the lower confidence bound above a threshold. The lower confidence bound could increase as well. As long all of the sub-models have a high enough confidence bounds and the information gain is sufficiently positive, embodiments can choose to segment (split) the more general model.
In some embodiments, any accuracy and information gain criteria can be satisfied by ensuring that a confidence level increases as a result of the segmentation. For example, a first property of the device can be selected for testing a first sub-model of a first context, which could include other properties, relative to a parent model. A first subset of the historical interactions that occurred when the computing device had the first property can be identified. The first subset is selected from the set of historical interactions for the parent model and is smaller than the set of historical interactions.
Based on the first subset of historical interactions, the first sub-model can predict at least one application of a first group of one or more applications that the user will access in association with the event with a first confidence level. The first sub-model can be created at least based on the first confidence level being greater than the initial confidence level at least a threshold amount, which may be 0 or more. This threshold amount can correspond to a difference threshold. In some implementations, the first sub-model can be created may not always be created when this criterion is satisfied, as further criteria may be used. If the confidence level is not greater than the initial confidence level another property can be selected for testing. This comparison of the confidence levels can correspond to testing for information gain. The same process can be repeated for determining a second confidence level of a second sub-model (for a second property) of the first sub-model for predicting a second group of one or more applications. A second subset of the historical interactions can be used for the second sub-model. A third property or more properties can be tested in a similar manner.
F. Regeneration of Decision Tree
Embodiments can generate a decision tree of the models periodically, e.g., daily. The generation can use the historical data available at that time. Thus, the decision tree can change from one generation to another. In some embodiments, the decision tree is built without knowledge of previous decision trees. In other embodiments, a new decision tree can be built from such previous knowledge, e.g., knowing what sub-models are likely or by starting from the previous decision tree.
In some embodiments, all contexts are attempted (or a predetermined listed of contexts) to determined which sub-models provide a largest information gain. For example, if location provides the largest information gain for segmenting into sub-models, then sub-models for at least one specific location can be created. At each level of segmentation, contexts can be tested in such a greedy fashion to determine which contexts provide a highest increase in information gain.
In other embodiments, a subset of contexts are selected (e.g., a random selection, which include pseudorandom) for testing whether segmentation is appropriate. Such selection can be advantageous when there are many contexts that could be tested. The contexts can be selected using Monte Carlo based approach, which can use probabilities for which contexts will likely result in a segmentation. A random number can be generated (an example of a random process) and then used to determine which context (for a particular property) to test.
The probabilities can be used as weights such that contexts with higher weights are more likely to be selected in the “random” selection process. The probabilities can be determined based on which sub-models have been generated in the past. For example, if the gym (and potentially a particular time of day was very successful before), then the generation process pick that context with a 90%, 95%, or 99% likelihood, depending on how often it had been picked in the past, and potentially also depending on how high the information gain had been in the past. A certain number of splits would be attempted for each level or for an entire tree generation process.
The prediction model can test not only for the selected application but a specific action, and potentially media content (e.g., a particular playlist). In some embodiments, once the probability of selecting an application is sufficiently accurate, a more aggressive action can be provided than just providing an option to launch. For example, when the application is launched, content can automatically play. Or, the application can automatically launch.
When selecting an application is predicted with sufficient probability (e.g., confidence level is above a high threshold), then the prediction can begin testing actions. Thus, the testing is not just for prediction of an application, but testing whether a particular action can be predicted with sufficient accuracy. The different possible actions (including media items) can be obtained from the historical data. A plurality of actions can be selected to be performed with the one application. Each of the plurality of actions can correspond to one of a plurality of different sub-models of the first sub-model. A confidence level of each of the plurality of different sub-models can be tested to determine whether to generate a second sub-model for at least one of the plurality of actions.
Accordingly, embodiments can be more aggressive with the actions to be performed when there is greater confidence. The prediction model may provide a particular user interface if a particular action has a high probability of being performed. Thus, in some embodiments, the higher the probability of use, more aggressive action can be taken, such as automatically opening an application with a corresponding user interface (e.g., visual or voice command), as opposed to just providing an easier mechanism to open the application.
For example, a base model can have a certain level of statistical significance (accuracy and confidence) that the action might be to suggest the application(s) on the lock screen. As other examples, a higher level of statistical significance can cause the screen to light up (thereby brining attention to the application, just one application can be selected, or for a user interface (UI) of the application can be provided (i.e., not a UI of the system for selecting the application). Some embodiments may take into account the actions being taken when determining whether to segment, and not segment if an action would be lost, which generally would correspond to having an information gain.
The action can depend on whether the model predicts just one application or a group of application. For example, if there is an opportunity to make three recommendations instead of one, then that also would change the probability distribution, as a selection of any one of the three would provide a correct prediction. A model that was not confident for recommendation of one application might be sufficiently confident for three. Embodiments can perform adding another application to a group of application being predicted by the model (e.g., a next most used application not already in the group), thereby making the model more confident. If the model is based on a prediction of more than one application, the user interface provided would then provide for an interaction with more than application, which can affect the form for the UI. For example, all of the applications can be provided on a lock screen, and one application would not automatically launch.
There can also be multiple actions, and a suggestion for different actions. For example, there can be two playlists at the gym as part of the sub-model (e.g., one application is identified but two actions are identified in the model when the two actions have a similar likelihood of being selected). Together the two actions can have statistically significance, whereas separately they did not.
As an example, when the model for an event (e.g., plugging in the headphones) is first being trained, the model may not be confident enough to perform any actions. At an initial level of confidence, an icon or other object could be displayed on a lock screen. At a next higher level of confidence, the screen might light up. At a further level of confidence, a user interface specific to a particular functionality of the application can be displayed (e.g., controls for playing music or a scroll window for accessing top stories of a new application). A next higher level can correspond to certain functionality of the application automatically being launched. The action could be even to replace a current operation of the application (e.g., playing one song) to playing another song or playlist. These different levels could be for various values used to define a confidence level.
Other example actions can include changing a song now playing, providing a notification (which may be front and center on the screen). The action can occur after unlocking the device, e.g., a UI specific to the application can display after unlocking. The actions can be defined using deep links to start specific functionality of an application.
Some embodiments may display a notice to the user on a display screen. The notice may be sent by a push notification, for instance. The notice may be a visual notice that includes pictures and/or text notifying the user of the suggested application. The notice may suggest an application to the user for the user to select and run at his or her leisure. When selected, the application may run. In some embodiments, for more aggressive predictions, the notification may also include a suggested action within the suggested application. That is, a notification may inform the user of the suggested application as well as a suggested action within the suggested application. The user may thus be given the option to run the suggested application or perform the suggested action within the suggested application. As an example, a notification may inform the user that the suggested application is a music application and the suggested action is to play a certain song within the music application. The user may indicate that he or she would like to play the song by clicking on an icon illustrating the suggested song. Alternatively, the user may indicate that he or she would rather run the application to play another song by swiping the notification across the screen.
Other than outputting a suggested application and a suggested action to the user interface in one notification, a prediction engine may output two suggested actions to the user interface in one notification. For instance, prediction engine may output a suggested action to play a first song, and a second suggested action to play a second song. The user may choose which song to play by clicking on a respective icon in the notification. In embodiments, the suggested actions may be determined based on different criteria. For instance, one suggested action may be for playing a song that was most recently played regardless of contextual information, while the other suggested action may be for playing a song that was last played under the same or similar contextual information. As an example, for the circumstance where a user enters into his or her car and the triggering event causes the prediction engine to suggest two actions relating to playing a certain song, song A may be a song that was last played, which happened to be at home, while song B may be a song that was played last time the user was in the car. When the user selects the song to be played, the song may continue from the beginning or continue from where it was last stopped (e.g., in the middle of a song).
In order for a prediction engine to be able to suggest an action, a prediction engine 302 may have access to a memory device that stores information about an active state of the device. The active state of a device may represent an action that is performed following selection of the suggested application. For instance, an active state for a music application may be playing a certain song. The active state may keep track of when the song last stopped. In embodiments, historical database may record historical data pertaining to the active state of the device. Accordingly, the prediction engine may suggest an action to be run by the suggested application.
At the top are UI elements. As shown, there is a lock screen 710, a search screen 720, and a voice interface 725. These are ways that a user interface can be provided to a user. Other UI elements can also be used.
At the bottom, are data sources. An event manager 742 can detect events and provide information about the event to an application suggestion engine 740. In some embodiments, event manager can determine whether an event triggers a suggestion of an application. A list of predetermined events can be specified for triggering an application suggestion. Location unit 744 can provide a location of the user device. As examples, location unit 744 can include GPS sensor and motion sensors. Location unit 744 can also include other applications that can store a last location of the user, which can be sent to application suggestion engine 740. Other contextual data can be provided from other context unit 746.
Application suggestion engine 740 can identify one or more applications, and a corresponding action. At a same level as application suggestion engine 740, a contacts suggestion engine 750 can provide suggested contacts for presenting to a user.
The suggested application can be provided to a display center 730, which can determine what to provide to a user. For example, display center 730 can determine whether to provide a suggested application or a contacts. In other examples, both the application(s) and contact(s) can be provided. Display center can determine a best manner for providing to a user. The different suggestions to a user may use different UI elements. In this manner, display center 730 can control the suggestions to a user, so that different engines do not interrupt suggestions provided by other engines. In various embodiments, engines can push suggestions (recommendations) to display center 730 or receive a request for suggestions from display center 730. Display center 730 can store a suggestion for a certain amount of time, and then determine to delete that suggestion if the suggestion has not been provided to a user, or the user has not interacted with the user interface.
Display center 730 can also identify what other actions are happening with the user device, so as to device when to send the suggestion. For example, if the user is using an application, a suggestion may not be provided. Display center 730 can determine when to send the suggestion based on a variety of factors, e.g., motion state of device, whether lock screen is one or whether authorized access has been provided, whether user is using the device, etc.
Input/Output (I/O) subsystem 806, wireless circuitry 808, and audio circuitry 810 including speaker 850 and microphone 852. These components may be coupled by one or more communication buses or signal lines 803. Device 800 can be any portable electronic device, including a handheld computer, a tablet computer, a mobile phone, laptop computer, tablet device, media player, personal digital assistant (PDA), a key fob, a car key, an access card, a multi-function device, a mobile phone, a portable gaming device, a car display unit, or the like, including a combination of two or more of these items.
It should be apparent that the architecture shown in
Wireless circuitry 808 is used to send and receive information over a wireless link or network to one or more other devices' conventional circuitry such as an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, memory, etc. Wireless circuitry 808 can use various protocols, e.g., as described herein.
Wireless circuitry 808 is coupled to processing system 804 via peripherals interface 816. Interface 816 can include conventional components for establishing and maintaining communication between peripherals and processing system 804. Voice and data information received by wireless circuitry 808 (e.g., in speech recognition or voice command applications) is sent to one or more processors 818 via peripherals interface 816. One or more processors 818 are configurable to process various data formats for one or more application programs 834 stored on medium 802.
Peripherals interface 816 couple the input and output peripherals of the device to processor 818 and computer-readable medium 802. One or more processors 818 communicate with computer-readable medium 802 via a controller 820. Computer-readable medium 802 can be any device or medium that can store code and/or data for use by one or more processors 818. Medium 802 can include a memory hierarchy, including cache, main memory and secondary memory.
Device 800 also includes a power system 842 for powering the various hardware components. Power system 842 can include a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)) and any other components typically associated with the generation, management and distribution of power in mobile devices.
In some embodiments, device 800 includes a camera 844. In some embodiments, device 800 includes sensors 846. Sensors can include accelerometers, compass, gyrometer, pressure sensors, audio sensors, light sensors, barometers, and the like. Sensors 846 can be used to sense location aspects, such as auditory or light signatures of a location.
In some embodiments, device 800 can include a GPS receiver, sometimes referred to as a GPS unit 848. A mobile device can use a satellite navigation system, such as the Global Positioning System (GPS), to obtain position information, timing information, altitude, or other navigation information. During operation, the GPS unit can receive signals from GPS satellites orbiting the Earth. The GPS unit analyzes the signals to make a transit time and distance estimation. The GPS unit can determine the current position (current location) of the mobile device. Based on these estimations, the mobile device can determine a location fix, altitude, and/or current speed. A location fix can be geographical coordinates such as latitudinal and longitudinal information.
One or more processors 818 run various software components stored in medium 802 to perform various functions for device 800. In some embodiments, the software components include an operating system 822, a communication module (or set of instructions) 824, a location module (or set of instructions) 826, an application suggestion module 828, and other applications (or set of instructions) 834, such as a car locator app and a navigation app.
Operating system 822 can be any suitable operating system, including iOS, Mac OS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks. The operating system can include various procedures, a plurality of instructions, software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.
Communication module 824 facilitates communication with other devices over one or more external ports 836 or via wireless circuitry 808 and includes various software components for handling data received from wireless circuitry 808 and/or external port 836. External port 836 (e.g., USB, FireWire, Lightning connector, 60-pin connector, etc.) is adapted for coupling directly to other devices or indirectly over a network (e.g., the Internet, wireless LAN, etc.).
Location/motion module 826 can assist in determining the current position (e.g., coordinates or other geographic location identifier) and motion of device 800. Modern positioning systems include satellite based positioning systems, such as Global Positioning System (GPS), cellular network positioning based on “cell IDs,” and Wi-Fi positioning technology based on a Wi-Fi networks. GPS also relies on the visibility of multiple satellites to determine a position estimate, which may not be visible (or have weak signals) indoors or in “urban canyons.” In some embodiments, location/motion module 826 receives data from GPS unit 848 and analyzes the signals to determine the current position of the mobile device. In some embodiments, location/motion module 826 can determine a current location using Wi-Fi or cellular location technology. For example, the location of the mobile device can be estimated using knowledge of nearby cell sites and/or Wi-Fi access points with knowledge also of their locations. Information identifying the Wi-Fi or cellular transmitter is received at wireless circuitry 808 and is passed to location/motion module 826. In some embodiments, the location module receives the one or more transmitter IDs. In some embodiments, a sequence of transmitter IDs can be compared with a reference database (e.g., Cell ID database, Wi-Fi reference database) that maps or correlates the transmitter IDs to position coordinates of corresponding transmitters, and computes estimated position coordinates for device 800 based on the position coordinates of the corresponding transmitters. Regardless of the specific location technology used, location/motion module 826 receives information from which a location fix can be derived, interprets that information, and returns location information, such as geographic coordinates, latitude/longitude, or other location fix data.
Application suggestion module 828 can include various sub-modules or systems, e.g., as described above in
The one or more applications 834 on the mobile device can include any applications installed on the device 800, including without limitation, a browser, address book, contact list, email, instant messaging, word processing, keyboard emulation, widgets, JAVA-enabled applications, encryption, digital rights management, voice recognition, voice replication, a music player (which plays back recorded music stored in one or more files, such as MP3 or AAC files), etc.
There may be other modules or sets of instructions (not shown), such as a graphics module, a time module, etc. For example, the graphics module can include various conventional software components for rendering, animating and displaying graphical objects (including without limitation text, web pages, icons, digital images, animations and the like) on a display surface. In another example, a timer module can be a software timer. The timer module can also be implemented in hardware. The time module can maintain various timers for any number of events.
The I/O subsystem 806 can be coupled to a display system (not shown), which can be a touch-sensitive display. The display displays visual output to the user in a GUI. The visual output can include text, graphics, video, and any combination thereof. Some or all of the visual output can correspond to user-interface objects. A display can use LED (light emitting diode), LCD (liquid crystal display) technology, or LPD (light emitting polymer display) technology, although other display technologies can be used in other embodiments.
In some embodiments, I/O subsystem 806 can include a display and user input devices such as a keyboard, mouse, and/or track pad. In some embodiments, I/O subsystem 806 can include a touch-sensitive display. A touch-sensitive display can also accept input from the user based on haptic and/or tactile contact. In some embodiments, a touch-sensitive display forms a touch-sensitive surface that accepts user input. The touch-sensitive display/surface (along with any associated modules and/or sets of instructions in medium 802) detects contact (and any movement or release of the contact) on the touch-sensitive display and converts the detected contact into interaction with user-interface objects, such as one or more soft keys, that are displayed on the touch screen when the contact occurs. In some embodiments, a point of contact between the touch-sensitive display and the user corresponds to one or more digits of the user. The user can make contact with the touch-sensitive display using any suitable object or appendage, such as a stylus, pen, finger, and so forth. A touch-sensitive display surface can detect contact and any movement or release thereof using any suitable touch sensitivity technologies, including capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display.
Further, the I/O subsystem can be coupled to one or more other physical control devices (not shown), such as pushbuttons, keys, switches, rocker buttons, dials, slider switches, sticks, LEDs, etc., for controlling or performing various functions, such as power control, speaker volume control, ring tone loudness, keyboard input, scrolling, hold, menu, screen lock, clearing and ending communications and the like. In some embodiments, in addition to the touch screen, device 800 can include a touchpad (not shown) for activating or deactivating particular functions. In some embodiments, the touchpad is a touch-sensitive area of the device that, unlike the touch screen, does not display visual output. The touchpad can be a touch-sensitive surface that is separate from the touch-sensitive display or an extension of the touch-sensitive surface formed by the touch-sensitive display.
In some embodiments, some or all of the operations described herein can be performed using an application executing on the user's device. Circuits, logic modules, processors, and/or other components may be configured to perform various operations described herein. Those skilled in the art will appreciate that, depending on implementation, such configuration can be accomplished through design, setup, interconnection, and/or programming of the particular components and that, again depending on implementation, a configured component might or might not be reconfigurable for a different operation. For example, a programmable processor can be configured by providing suitable executable code; a dedicated logic circuit can be configured by suitably connecting logic gates and other circuit elements; and so on.
Any of the software components or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perl or Python using, for example, conventional or object-oriented techniques. The software code may be stored as a plurality of instructions or commands on a computer readable medium for storage and/or transmission. A suitable non-transitory computer readable medium can include random access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a compact disk (CD) or DVD (digital versatile disk), flash memory, and the like. The computer readable medium may be any combination of such storage or transmission devices.
Computer programs incorporating various features of the present invention may be encoded on various computer readable storage media; suitable media include magnetic disk or tape, optical storage media such as compact disk (CD) or DVD (digital versatile disk), flash memory, and the like. Computer readable storage media encoded with the program code may be packaged with a compatible device or provided separately from other devices. In addition program code may be encoded and transmitted via wired optical, and/or wireless networks conforming to a variety of protocols, including the Internet, thereby allowing distribution, e.g., via Internet download. Any such computer readable medium may reside on or within a single computer product (e.g. a hard drive, a CD, or an entire computer system), and may be present on or within different computer products within a system or network. A computer system may include a monitor, printer, or other suitable display for providing any of the results mentioned herein to a user.
Although the invention has been described with respect to specific embodiments, it will be appreciated that the invention is intended to cover all modifications and equivalents within the scope of the following claims.
This application is related to commonly owned and concurrently filed U.S. patent application entitled “Application Recommendation Based On Detected Triggering Events” by Gauci et al. (attorney docket number 90911-P26710US1-938986), the disclosure of which is incorporated by reference in its entirety for all purposes.
