Patent Application
20190253519
Various embodiments described herein relate to user interaction and more particularly, but not exclusively, to a service for determining opportune times for providing notifications, messages, or other interactions to a user.
Mobile devices such as smart phones and tablets have become the prime point of contact for most people, not just for person-to-person communications but for communications from applications and services as well. These devices can be particularly powerful for pushing information to a user via notifications or other messages rather than requiring the user to first solicit input. For example, a coaching program is able to push suggestions and information to a user to help them achieve the goal of the program throughout the day. This frees the user to go about their day without actively following the coaching program. At any point in the day, the user may receive a message from the coaching service and read the information or take suggested actions (e.g., walk to lunch rather than driving).
While the freedom to push messages to users can be beneficial to the goals of an application, it does not help to push notifications to a user when they are not free or otherwise open to participate in a user interaction. For example, if the user is currently driving their car, they are unable to read incoming messages. As another example, if the user is in a work meeting, they are unable to leave and participate in a suggested physical activity (but may be able to read a message, depending on the meeting).
It would be beneficial to provide a method and system for identifying opportune times to interact with a user. As will be described in greater detail below, various embodiments leverage the usage reported by the mobile device (e.g., text messaging, application switching, types of application being used) to gauge whether a user is currently available to act on a suggestion or receptive to read an informative message.
Accordingly, various embodiments described herein relate to a method performed by a prediction device for determining an opportune time for user interaction, the method including: receiving a request from a client application for an indication of whether a user is open to participate in a user interaction; obtaining usage information regarding the user's recent activity on a user device; applying at least one trained predictive model to the usage information to identify the user's current contextual state, wherein the current contextual state includes at least one of: an availability measure representative of the user's current ability to perform a physical action associated with the user interaction, and a receptiveness measure representative of the user's current ability to pay attention to the user interaction; determining an opportunity indication based on the user's contextual state; and providing the opportunity indication to the client application.
Various embodiments described herein relate to a prediction device for determining an opportune time for user interaction, the prediction device including: a memory configured to store at least one trained predictive model for identifying a user's current contextual state, wherein the current contextual state includes at least one of: an availability measure representative of the user's current ability to perform a physical action associated with the user interaction, and a receptiveness measure representative of the user's current ability to pay attention to the user interaction; and a processor in communication with the memory, the processor being configured to: receive a request from a client application for an indication of whether a user is open to participate in a user interaction; obtain usage information regarding the user's recent activity on a user device; apply the at least one trained predictive model to the usage information; determine an opportunity indication based on the user's contextual state; and provide the opportunity indication to the client application.
Various embodiments described herein relate to a non-transitory machine-readable medium encoded with performed by a prediction device for determining an opportune time for user interaction, the non-transitory machine-readable medium including: instructions for receiving a request from a client application for an indication of whether a user is open to participate in a user interaction; instructions for obtaining usage information regarding the user's recent activity on a user device; instructions for applying at least one trained predictive model to the usage information to identify the user's current contextual state, wherein the current contextual state includes at least one of: an availability measure representative of the user's current ability to perform a physical action associated with the user interaction, and a receptiveness measure representative of the user's current ability to pay attention to the user interaction; instructions for determining an opportunity indication based on the user's contextual state; and instructions for providing the opportunity indication to the client application.
Various embodiments are described wherein the opportunity indication includes the current contextual state.
Various embodiments are described wherein at least part of the usage information is obtained via an operating system application programmer interface (API) of the user device.
Various embodiments are described wherein the steps of receiving, obtaining, applying, determining, and providing are performed by a processor of the user device.
Various embodiments additionally include receiving the at least one trained predictive model from a remote predictive model training device.
Various embodiments additionally include subsequent to providing the opportunity indication to the client application: obtaining feedback information regarding the user's reaction to the client application; discerning from the feedback information a label regarding at least one of availability and receptiveness of the user; generating a training example by associating the label with the usage information; updating an existing training set by at least adding the training example to the existing training set to generate an updated training set; and retraining the at least one trained predictive model based on the updated training set
Various embodiments are described wherein the step of discerning includes applying a trained feedback interpretation model to the feedback information to receive the label.
Various embodiments are described wherein the feedback information describes the user activity on the user device subsequent to providing the opportunity indication to the client application.
Various embodiments are described wherein the step of discerning includes analyzing the feedback information together with the usage information to determine whether the user changed their usage behavior.
In order to better understand various example embodiments, reference is made to the accompanying drawings, wherein:
The description and drawings presented herein illustrate various principles. It will be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody these principles and are included within the scope of this disclosure. As used herein, the term, “or,” as used herein, refers to a non-exclusive or (i.e., and/or), unless otherwise indicated (e.g., “or else” or “or in the alternative”). Additionally, the various embodiments described herein are not necessarily mutually exclusive and may be combined to produce additional embodiments that incorporate the principles described herein.
As shown, a client application device 110 may be a device that, according to its own operation and purposes, engages in interactions with a user such as sending notifications with information or action suggestions to a user via their smart device or other channel (e.g., via an app, SMS, email, phone call, wearable notification, etc.). The client application device may be, for example, the user's smart device (e.g., running the client application as an app), a server that is remote to the user (e.g., a virtual machine running the client application as a web or other service), or any device capable of executing a client application that engages in interactions with a user that are capable of delay, flexible delivery, or other ability to adjust the timing thereof.
The interruption service device 120 may be virtually any physical device (e.g., the user's smart device, remote server running a virtual machine, etc.) capable of interpreting measures of a user's contextual state provided by the prediction device and providing such indication back to the client application device 110. For example, the interruption service device 120 may interpret multiple such measures and provide a single opportunity indication (e.g., a token correlated to notions such as “not open,” “receptive but not available,” “both receptive and available,” etc.) or may simply forward such measures as the opportunity indication. The interruption service device 120 may provide such opportunity indications on demand to the client application device 110 (e.g., as a query service) or upon a relevant change in the contextual state to a client application device 110 that has previously indicated a desire to receive such updates (e.g., as a subscription service). The client application 110 may then use these received indications to gauge when it is appropriate to initiate the user interaction.
Where the client application device 110 and interruption service device 120 are implemented in separate physical devices, the query or subscription communications may occur, e.g., via one or more networking channels. For example, if the two devices are physically close to each other (e.g., if the client application device 110 is a wearable wrist-watch device and the interruption service device 120 is the user's mobile phone), the communications may occur according to a short range wireless communication protocol such as near field communication (NFC), Bluetooth (including Bluetooth low energy), Wi-Fi, etc. In embodiments wherein the two devices are more remote from each other, the communications may traverse one or more wireless or wired networks such as Wi-Fi, 3G, 4G, LTE, or Ethernet networks. These networks may include or form part of, for example, a mobile carrier network, a cloud computing data center network, the Internet, etc. Where the two devices are implemented in the same physical hardware, the communications described herein may occur via on or more inter-process communications which may be provided by an operating system of the local device. Such approaches may include recording a file to be read by the other process, transmitting data to sockets associated with the respective sockets, establishing one or more pipes between the processing, writing data to shared memory, pushing events through an event bus, etc. In some embodiments where the two devices are implemented as part of the same process, the communications described herein may be achieved by calling or passing data to functions associated with the other functional device, writing data to a common location such as a configuration file or memory set aside for the process. Various other approaches to communication between functional devices will be apparent in the various physical and procedural contexts in which the functional devices may be implemented. Additionally, any of these approaches may be applied to communication amongst the other functional devices 130-170 described herein or other devices not illustrated depending on the particular embodiment of the system 100 being implemented.
The smart device monitor 130 is a device that monitors one or more types of raw device (e.g., mobile phone or tablet) usage information such as, for example, current device screen (e.g., locked, unlocked, home screen, in an app, in app switcher, etc.), ringer setting (e.g., high volume, low volume, high vibrate, low vibrate, silent, etc.), time since last unlock, historical log of locking/unlocking, battery state (e.g., charging/not charging, battery level, time to discharge), messages (e.g., pattern of messages/notifications], number of messages/notifications, number of recipients, frequency of communications in and out in a given window of time, phone calls (e.g., in call/not in call, pattern of calls made/taken, pattern of missed/dismissed calls, number of calls made/taken/missed/dismissed, number of other parties if calls made/taken/missed/dismissed in a given window of time, etc.), current application in use (e.g., name, category [productivity, game, utility, etc.], time in current app, etc.), pattern of application usage (e.g., number and category of apps used in a given window of time), network connection (e.g., none, Wi-Fi, cellular, specific carrier/frequency, etc.), current Wi-Fi network (e.g., SSID name, signal strength, frequency, etc.), signature of current Wi-Fi networks in range, signature of current connected devices (e.g., via Bluetooth, NFC, etc.), etc. In some embodiments, some or more of this information may not be available directly from the OS or other applications (i.e., in raw form) and instead are be extracted (as will be described in greater detail below with respect to the feature extraction device 140) from that raw usage information that is available.
In some embodiments, the raw usage information may be gathered in whole or in part by the operating system (OS) of the smart device itself which may thus constitute a smart device monitor 130. In some such embodiments, the OS may make such raw usage information directly available to the feature extraction device 130 via an application programmer interface (API) which as will be understood may operate according to push or pull protocols. In some embodiments, another process or app running on the smart device may gather some or all of the raw usage information through direct observation of user interaction with the app or by polling other raw usage information from the OS or other apps via respective APIs. For example, an OS API may provide an instantaneous view of the current network connection to another processor or app which may then compile a log of all network connection states seen over a time period. Various other approaches to compiling raw usage information will be apparent.
The feature extraction device 140 may be any device capable of extracting additional useful information from the raw usage information. This “extracted usage information” may be virtually any information that is not already included in the raw usage information (e.g., not already available through logging or observing OS states) but derivable therefrom. For example, a frequency of network switching, a median time spent in a particular text conversation, or a classification of a current call (e.g., important, casual, unwelcome). Such features may be obtained according to any approach including statistical algorithms or trained machine learning models.
In some embodiments, the feature extraction device 140 may be extensible post-deployment to change the features that are extracted. For example, the feature extraction device 140 may store one or more scripts, compiled instructions, or other items defining algorithms for feature extraction. The feature extraction device 140 may execute or interpret each such available item when extracting features and proved all resulting data downstream to the training set creation device 150 or prediction device 170. To modify which features are extracted, the set of these items may be modified, pruned, or supplemented.
The feature extraction device 140 provides both raw and extracted usage information (i.e., the features for use with one or more trained model, or at least a subset thereof) to the training set creation device 150 for training one or more models and to the prediction device 170 for application of the trained models for generating measures of the current contextual state of the user. The feature extraction device 140 may continually transmit the information (e.g., in a consistent stream or as is available) or may gather data for batch transmission at more spaced-out times. Where usage information from multiple time slots are to be sent in a batch, the feature extraction device 140 may aggregate and compress these features for transmission. The method of transmission need not be the same to both the training set creation device 150 and the prediction device 170. For example, the feature extraction device 140 may send aggregated and compressed sets of usage information to the training set creation device 150 every hour, but may send usage information immediately to the prediction device as the usage information becomes available. This may be particularly beneficial where the training set creation device 150 is located remotely from the feature extraction device 140 but the prediction device is local to or collocated in the same hardware as the feature extraction device 140.
The training set creation device 150 may be any device capable of constructing one or more training sets for training one or more machine learning models from received usage information, feedback, and any other useful contextual information. Thus, the training set creation device 150 may be implemented in, for example, the user's smart device or a remote server hosting a VM. It will be apparent that the operation of the training set creation device 150 and the arrangement of the produced training set(s) will be dependent on the machine-learning approaches and models employed by the predictive model training device 160. For example, where the predictive model training device 160 uses a supervised (or semi-supervised) learning approach, the training set creation device 150 may generate a labeled training set (e.g., wherein each feature set is paired with a label indicating a Boolean or numerical value representing the user's availability, receptiveness, etc. found or presumed to correspond to that feature set). To provide such labels, a human operator (e.g., the user or another operator) may review the feature set and manually provide these labels. Alternatively, the client application 110 may return such labels as feedback based on the actual observed user behavior after delivering a user interaction. In some embodiments, a semi-supervised approach may be followed rather than providing the labels directly. For example, one or more separately trained machine-learning models (e.g., a logistic regression model for each measure of contextual state) may be applied to the feature set or feedback information (which may constitute usage information or other information observed about the user before, during, or after user interaction delivery) to determine one or more labels (e.g., discretely or as a likelihood of each potential label being applicable) for each training example. Some useful feedback for labeling feature sets may include information describing how the user interacts with a delivered notification (or other initiated user interaction) (e.g., ignoring, reading for 2 seconds and dismissing, etc.), or the data transferred after a user interaction. Through application of such models, the training set creation device 150 may be able to determine whether and to what degree a user's behavior has changed in response to an initiated user interaction, thereby gauging the success of that user interaction.
In some embodiments, the training set creation device 150 may also prune old or non-personalized training examples from the training set as fresh, personalized examples are gathered. For example, when a client application device 110 first begins using the interruption service device 120 for a particular user, that user's models 172, 174 may be generated based on training examples drawn from the population at large or from a population that is demographically or otherwise similar to that user. As training examples specific to that user are gathered, the old examples may be phased out of the training set (or weighted comparatively less than the new examples) to provide more fully customized models. Similarly, personalized-yet-old training examples may also be phased out (or weighted comparatively less) in favor of newer training examples to account of changes in user behavior or attitude. The resulting labeled data set may then be provided to the predictive model training device 160.
The predictive model training device 160 may be any device capable of training one or more models based on a training set. For example, the predictive model training device 160 may be a mobile device or remote server. Various trained models and machine learning algorithms for providing measures of contextual states may be used. For example, the predictive model training device 160 may train a regression (linear or logistic) or neural network model using a version of gradient descent for analyzing the training set. Accordingly, the resulting model may include a set of learned weights (e.g., theta values) that can be transmitted to the prediction device 170 for application to current feature sets for real-time prediction of the user's contextual state.
The prediction device 170 may be any device capable of applying one or more trained models to incoming feature sets from the one or more feature extraction devices 140. The prediction device 170 may store one or more models received from the predictive model training device 160 (e.g., a receptiveness model 172 and an availability model 174). Upon receiving a feature set from the feature extraction device 140, the prediction device 170 applies the models 172, 174 to the feature sets (e.g., by inputting the features into the trained regression or neural network function) The outputs of the models 172, 174 (e.g., the contextual state measures) may then be provided to the interruption service device 120 for use in providing an opportunity indication to the client application device 110 in the manner described above.
It will be apparent that, while in various embodiments described herein some functional devices are described as coordinating with a smart device operating system (e.g., the smart device monitor 130), in other embodiments, one or more of the functional devices may be implemented as part of the operating system. For example, the smart device monitor 130 or feature extraction device 140 may operate as operating system components for reporting features to external devices/services (e.g., via an operating system API). As additional examples, all of the functional devices 110-170 or the real-time interruption service pipeline 130, 140, 170, 120 may be implemented as components of the operating system itself.
The processor 220 may be any hardware device capable of executing instructions stored in memory 230 or storage 260 or otherwise processing data. As such, the processor may include a microprocessor, field programmable gate array (FPGA), application-specific integrated circuit (ASIC), or other similar devices. It will be apparent that, in embodiments where the processor includes one or more ASICs (or other processing devices) that implement one or more of the functions described herein in hardware, the software described as corresponding to such functionality in other embodiments may be omitted.
The memory 230 may include various memories such as, for example L1, L2, or L3 cache or system memory. As such, the memory 230 may include static random access memory (SRAM), dynamic RAM (DRAM), flash memory, read only memory (ROM), or other similar memory devices.
The user interface 240 may include one or more devices for enabling communication with a user such as an administrator. For example, the user interface 240 may include a display, a mouse, and a keyboard for receiving user commands. In some embodiments, the user interface 240 may include a command line interface or graphical user interface that may be presented to a remote terminal via the communication interface 250.
The communication interface 250 may include one or more devices for enabling communication with other hardware devices. For example, the communication interface 250 may include a wired or wireless network interface card (NIC) configured to communicate according to the Ethernet protocol. Additionally, the network interface 250 may implement a TCP/IP stack for communication according to the TCP/IP protocols. Additionally or alternatively, the communication interface 250 may include hardware for communication with nearby devices such as hardware for communication according to NFC, Bluetooth, Wi-Fi or other local wireless or wired protocols. Various alternative or additional hardware or configurations for the communication interface 250 will be apparent.
The storage 260 may include one or more machine-readable storage media such as read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash-memory devices, or similar storage media. In various embodiments, the storage 260 may store instructions for execution by the processor 220 or data upon with the processor 220 may operate.
For example, where the device 200 implements a user's smart device (e.g., a mobile phone or tablet) the storage 260 may store a smart device operating system 261 for controlling various basic operations of the hardware 200. For example, the smart device operating system 261 may provide an environment for executing and interacting with client applications, managing various network connectivity, enabling phone and messaging communication, etc. In the embodiment shown, the smart device operating system 261 includes usage information reporting instructions 262 for providing various information describing the usage of the smart device by a user (such as one or more of the metrics described above). Thus, in this example embodiment, the OS implements the smart device monitor 130 of
The smart device may also host and execute one or more client applications 266 (which may implement the client application device 110 of
Where the device 200 implements a supporting server, the storage 260 may store a server operating system. In some embodiments wherein the server is deployed in a cloud computing architecture, the server operating system may include a hypervisor for coordinating one or more virtual machines which, in turn, may include additional instructions and data 272-279. Training set creation instructions 272 may implement the training set creation device 150 and, as such, may receive feature sets from the feature reporting device 130 and create new training examples therefrom, forming a training set 274. Feedback interpretation instructions 273 may interpret feedback from the client application device 110 or other sources (e.g., using a trained feedback interpretation model, not shown) to label the training examples for use in training the predictive models.
Predictive model training instructions 275 may implement the predictive model training device 160 and, as such, may train one or more predictive models 276 based on the training set 274 or portions thereof. For example, the predictive model training instructions 275 may implement a form of gradient descent to learn one or more theta weight values for instantiating a regression or neural network (including deep learning) predictive model 276. With the trained models 277, the model application instructions 277 may implement the prediction device 170 by receiving features from the feature extraction device 140 then applying the model(s) 276 to obtain one or more measures of the user's contextual state. The query service instructions 278 or subscription service instructions may then implement the interruption service device 120 by providing opportunity indications (e.g., “available,” “receptive but not available” or the contextual state measures themselves) to the client application device 110 in a pull or push manner, respectively.
It will be apparent that various information described as stored in the storage 260 may be additionally or alternatively stored in the memory 230. In this respect, the memory 230 may also be considered to constitute a “storage device” and the storage 260 may be considered a “memory.” Various other arrangements will be apparent. Further, the memory 230 and storage 260 may both be considered to be “non-transitory machine-readable media.” As used herein, the term “non-transitory” will be understood to exclude transitory signals but to include all forms of storage, including both volatile and non-volatile memories.
While the host device 200 is shown as including one of each described component, the various components may be duplicated in various embodiments. For example, the processor 220 may include multiple microprocessors that are configured to independently execute the methods described herein or are configured to perform steps or subroutines of the methods described herein such that the multiple processors cooperate to achieve the functionality described herein. Further, where the device 200 is implemented in a cloud computing system, the various hardware components may belong to separate physical systems. For example, the processor 220 may include a first processor in a first server and a second processor in a second server.
As shown, the smart device 320 includes a usage monitoring process 322 for performing usage monitoring (e.g., within or via an API of the smarty device OS) and a feature extraction process 324 for extracting additional features from the raw usage information gathered by the usage monitoring process 324 (e.g., within or via an API of the smarty device OS). The smart device 320 transmits the relevant usage information features (whether raw or extracted) to the cloud VM 330, where a training set construction process 332 commits the features as one or more new training examples in the training set 334. Whether these records are initially unlabeled, the training set construction process 332 may subsequently label the examples once sufficient feedback is obtained to discern an appropriate label. A model training process 336 may then use the training set 334 (immediately or at a later time) to create or update one or more predictive models 338 as described above.
Meanwhile, the model application process 342 applies the ten-current version of the predictive model 338 to the incoming feature set to determine one or measures of the user's contextual state an makes this information available to the query service 344 and subscription service 346 processes. Client application 1326 would like to initiate a user interaction and transmits a query to the query service 344. In response, the query service 344 transmits an opportunity indication back to the client application 1326 based on the measure(s) provided by the model application process 342. The client application 1326 may then use this indication to determine whether or not it is an opportune time to initiate the user interaction. If the user interaction is initiated, the client application 1326 may then report feedback back to the training set construction process 332.
The client application 2328, on the other hand, may have previously indicated to the subscription service process 346 a desire to receive push notifications about updates to the user availability. For example, the client application may subscribe to all changes to the opportunity indication or to any changes of the opportunity indication to a defined value or group or range thereof. Upon a change to the opportunity indication based on the measure(s) reported by the model application process 342, the subscription service 346 may then push the new opportunity indication to the client application 2328. The client application 2328 may then use this indication to determine whether or not it is an opportune time to initiate the user interaction. If the user interaction is initiated, the client application 2328 may then report feedback back to the training set construction process 332.
As shown, in this second embodiment, the majority of the functional devices are embodied in the smart device 420 itself. A context monitoring process 422 gathers raw usage data and a feature extraction process 424 extracts additional features therefrom. A training set construction process 426 uses these features and feedback information to create a training set 428 from which a model training process 432 creates one or more predictive models 434 for determining one or more measures of a user's current contextual state (e.g., availability and receptiveness). A model application process 436 applies the model(s) 434 to the new feature set to determine these measures and, then, a subscription service process 438 forwards an opportunity indication based on these measures to the VM1440 which had previous subscribed to receive push notifications for changes to opportunity indications.
The VM1440 is established as an intermediary between the smart device 420, which determines the user's contextual state periodically, and VM2450, which hosts an client application 452 that pushes notifications (or other user interactions) to a notification receiving app (or OS module) 439 running on the smart device 420. For example, the notification pushing application 452 may be a server side of a coaching service, while the notification receiving application 439 may be a client side (e.g., dashboard or message center) of the coaching service. The intermediary VM1440 may subscribe to opportunity indication updates from the smart device 420 and provide such opportunity indications to the VM2450 as requested. Thus, the VM1440 provides the VM2450 with a query service for determining opportune times for user interaction without requiring the smart device 420 to provide such on-demand availability. In some embodiments, the VM1440 may receive updates from multiple smart devices 420 or for multiple users and provide a single point of query for VM2450 or other servers hosting client applications. As such, the VM1440 includes a subscription client process 442 that receives opportunity indications from the smart device 420, memory dedicated to storing reported opportunity indications 444, and a query service process 446 for forwarding these opportunity indications to the VM2450 upon request. As noted above, the VM2450 includes a notification pushing application 452 which, itself implements an opportunity query software module 454. For example, the opportunity query module 454 may handle requesting opportunity indications at the request of other portions of the application 452 code according to the appropriate procedures and formats required by an API of the query service 446.
It will be understood that the examples of
The method 500 begins in step 505 where the method receives an indication of one or more OS events (e.g., via an event bus provided by the OS). In step 510, the method 500 determines whether the received event indicates that the user has unlocked their phone. The specific approach to determining what information a received event conveys will be specific to the OS with which the method 500 interacts; specific steps for implementing this decision 510 (and subsequent decisions) will be apparent in view of the OS. If the event does relate to an unlock event, the method 500 proceeds to step 515 where the device updates a tracked variable storing the time of the last device unlock.
Next, in step 520, the device determines whether the OS event indicates that a message (e.g., a text or multimedia message) has been sent or received. If so, the device logs the message 525 (e.g., the message content, time, sender, or recipient) and updates a running count of the messages received in the last 5 minutes (or some other useful time window) by referring to the message log created through successive executions of step 525. Both the message log and the count may be stored for later use as features or for extracting additional features therefrom. In step 535 the device determines whether the OS event indicates that the user has switched to a different application. If so, the device logs the switch (e.g., the time, previous application, and new application) in a running log in step 540, recounts the number of application switches in the preceding 5 minutes (or other time window) in step 545, and recounts the number of application switches initiated by the user in the preceding 5 minutes (or other time window) in step 545, and recounts the number unique applications activated by the user in the preceding 5 minutes (or other time window) in step 550. The switch log and the counts may be stored for later use as features or for extracting additional features therefrom. The method 500 may then proceed to end in step 555.
It will be apparent that method 500 is merely one example of a method for gathering raw usage information and that various alternative approaches may be followed. For example, in some embodiments, a dedicated algorithm may be defined for each type of OS events or multiple groupings thereof. For example, upon registering with an event bus for each type of OS event, the device may register a dedicated algorithm for processing that type of OS event. As such, the event bus in effect does the work of steps 510, 520, 535 because only the algorithm for processing the specific OS event is called.
The method 600 begins in step 605 (e.g., based on execution as a schedule task configured to occur periodically) and proceeds to step 610 where the device creates an empty feature set data object. In step 615, the device copies any current values for raw usage information that has been tracked by monitoring the OS or other sources (e.g., according to method 500 or a method similar thereto.) into the empty feature set. For example, the time of the last unlock, a message log, and an application log may be copied into the feature set. Next, in step 620, the device polls the OS for any raw usage information to be used as features that are directly available from the OS (or other app or other source). For example, the OS may be able to provide information such as the current connection information (e.g., network, data usage) or current device status. This information may then also be copied into the feature set in step 625.
In step 630, the device determines whether there are any algorithms available to be applied for extracting features from the usage information that is present. For example, the device may store a collection of scripts, JAR files, etc. that are to be executed in sequence for each feature set. Various alternatives, such as simply coding the feature extraction into the method 600 itself are also possible. If there are feature extracting algorithms to apply, each of these are performed (e.g., called as functions or executed as scripts) in step 635 and the results thereof are copied into the feature set in step 640. The method then proceeds to end in step 645. The features may then be transmitted to the training set creation device 150 or prediction device 170 at some point in the future or as a final step (not shown) before the method 600 ends in step 645. In some embodiments the device ay gather multiple feature sets (e.g., a new feature set every 5 minutes) and transmit them together as a batch to one of the other device (e.g., every hour or other time period, or on request by the other device). The transmission schedule need not be the same for transmitting to the training set creation device 150 and prediction device 170. For example, the feature extraction device 140 may transmit batches of features to the training set creation device on an hourly basis but may transmit each new feature set to the prediction device as it becomes available to enable the most up-to-date opportunity indications. Various other modifications will be apparent.
As shown, the training set 700 includes a set of features 710 and a set of labels 720. The set of features 710 may include various fields for storing reported features for each training example. As an example, the training set 700 includes a device status field 712 for storing a then-current device status, a communications field 714 for storing information about communications (e.g., calls and text messages) engaged in by the user, an app usage field 716 for storing information about the apps used by the user, and a connectivity field 718 for storing then-current connection information of the device. The labels 720 include an available field 722 for storing a label of whether the user was deemed “Available” to physically engage user interaction and a receptive field 724 for storing a label of whether the user was deemed “Receptive” to mentally engaging with a user interaction. While Boolean labels are shown (and may be useful for training logistic regression models), in other embodiments, numerical values may be provided as labels of these two measures (e.g., for use in training a linear regression model). These numerical values may correspond to a probability that the label applies or a degree to which the label applies (e.g., an availability of 100 may indicate that the user can go for a jog now, while an availability of 20 may indicate that the user can't go for a jog but can take a brief walk around the office). It will also be apparent that various alternative measures may be provided. For example, availability may be broken down into multiple labels that each correspond to a different type, grouping, or degree of physical activity for which the user is available, thereby enabling the training of a different models for each of multiple degrees of availability, receptiveness, or other measures of a user's contextual state. Virtually any of the approaches to training set creation, model training, and model application described herein with respect to the predictive models may also be employed to enable operation of the labeling models.
As an example, training example 730 indicates that for a specific feature set (including connection to a WiFi access point called “Work SSID” and currently using the “Web Browser App”), the user was judged to be receptive to incoming user interactions but not available to act on them immediately. Training example 740 may indicate that for a different feature set (including the user currently being in a call that has lasted 35 minutes so far) the user has been judged to be neither receptive nor available for receiving a user interaction. A third training example stores another reported feature set (including that the device is currently locked and no apps have been used in the last 5 minutes) has yet to be labeled. After feedback information has been received, the training set creation device 150 may return to this example 750 to provide labels. For example, where the feedback is manual input from the user, another human user, or the client application the feedback may include the label itself to be attached to the example 750. Where the feedback is subsequent usage information or other information that does not already include the label, the training set creation device 150 may interpret the feedback to generate the label(s) for attachment to the example 750. For example, the training set may include a trained model (e.g., a regression model or neural network) for each potential label that is applied to the features in the data structure or the received feedback to determine whether each respective label applies to an example, These labeling models, not to be confused with the predictive models applied by the prediction device 170, may be trained based on yet another training set (not shown) that may have been manually labeled by a human user. Various learning approaches, such as a version of gradient descent, may be applied to achieve such training.
The method 800 begins in step 805 (e.g., in response to obtaining a new feature set from a feature extraction device or based on a scheduled task) and proceeds to step 810 where the device determines whether training is currently enabled for the user associated with the received feature set (for example, in some embodiments, training may be turned off at times when the predictive models are deemed to be sufficiently trained to conserve memory and processing resources). If training is turned off, the method 800 may skip ahead to step 825; otherwise, the device creates a new record of a training example that stores the received feature set and stores the new training example as part of the existing training set. As will be understood, the feature set may be obtained from various sources depending on the device that is executing the instructions. Obtaining may include receiving the feature set from another device (e.g., where the feature extraction device 140 and prediction device 170 are implemented in physically separate devices) or receiving the feature set from another local process (e.g., via sockets or shared memory) (e.g., where the feature extraction device 140 and prediction device 170 are implemented as separate processes or OS modules in the same device).
In step 825, the device applies one or more predictive models to the received feature set to obtain one or more measures of the user's current contextual state. In step 830, the device determines whether the determined contextual state is different from the previously determined contextual state (e.g., if the user is now available where previously unavailable, or if receptiveness has decreased from a value of 60 to 20) and, if so, proceeds to step 835 where the device stores the determined contextual state as the current contextual state (potentially overwriting the previous contextual state).
In step 840, the subscription service device determines whether any client applications had previously subscribed to updates about this user and, if so, pushes the new contextual state to the subscribed applications in step 845. This approach may be sufficient for embodiments wherein client applications simply subscribe to all updates. In other embodiments wherein client applications may specify one or more criteria for when an update should be pushed (e.g., when the user becomes available, when receptiveness or availability increases, or when receptiveness is at least 50), an additional conditional blocks may be implemented between steps 840 and 845 to determine which client applications' criteria have been met to determine to which client applications the update will be pushed. The method then proceeds to end in step 850.
The method begins in step 905 (e.g., in response to receiving feedback information or based on a scheduled task) and proceeds to step 910 where the device interprets one or more labels from the received feedback. For example, step 910 may include simply reading a manually provided label from the feedback or applying a separate trained model (e.g., a classification model such as logistic regression) to the feedback (and, in some embodiments, features stored in the relevant training records to be labeled). In step 915, the device may locate any records that are contemporary to the feedback (e.g., based on timestamps of when the features sets stored therein were created or received). In some embodiments, only as-yet unlabeled training examples may be located in the step while in other embodiments, both unlabeled and labeled examples may be obtained so as to update the previously labeled examples based on the additional feedback. In step 920, the device labels each retrieved training example record in accordance with the interpreted label(s).
In step 925, the device may determine whether training set decay is enabled. For example, in some embodiments, old training examples may be pruned from the training set (at least with respect to the user to which the current feedback applies) as new records are added. This may be particularly useful where a general training set based on examples drawn from a population of users is used at first and gradually supplemented and replaced with examples created based on the specific user, thereby personalizing operation of the system to the user's behaviors and habits. If decay is enabled, the device removes the oldest records from the training set in step 935 (e.g., by deleting them or flagging them as not to be used for training models for the current user). In some embodiments, one old training example may be removed for each new training example when decay is enabled, although other proportions are possible.
In step 935, the device determines how many new records have been added (or how many records have been labeled) since the last time the predictive model(s) were trained. If this number exceed 5 (or some other threshold deemed appropriate), the method 900 proceeds to step 945 to retrain the predictive model(s) on the current iteration of the training set. Various alternative methods for identifying appropriate times for model retraining, such as retraining the models at scheduled times, will be apparent, The method 900 then proceeds to end in step 950.
The method begins in step 1002 and proceeds to step 1004 where the device obtains a labeled data set (e.g., the training set 700 for training a predictive model or another training set (not shown) for training a labeling model) for a given parameter for which a model is to be created. Various alternative approaches for training a model from an unlabeled training set will be apparent. In various embodiments, the training set may include a number of records of training examples that specify one or more features (e.g., various usage information or feedback information as described above, etc.) and the appropriate conclusion to be drawn from that feature set.
In step 1006, the device identifies the number of features identified in the data set (or, where the model does not utilize every feature in the data set, the number of features relevant to the model being trained) and, in step 1008, initializes a set of coefficients to be used in the resulting model. According to various embodiments, a coefficient is created for each feature along with one additional coefficient to serve as a constant. Where the model is being trained to output a numerical value, a linear regression approach may be utilized, wherein the final model function may take the form of
h(X)=θ0+θ1x1+θ2x2
where X is the set of features {x1, x2, . . . } and the coefficients {θ0, θ1, θ2, . . . } are to be tuned by the method 1100 to provide as output an appropriate relevance estimation, consistent with the trends learned from the training data set. In some embodiments the final model function may incorporate a sigmoid function as follows:
where tuning of the coefficients results in the function h(X) outputting a value between 0 and 1 that serves as an estimation of the relevance of an offering. According to various embodiments, the coefficients are all initialized to values of zero. It will be apparent that in some embodiments, additional features for inclusion in h(X) (and associated coefficients) may be constructed from the features in the training set such as, for example, x12 or x1x2.
The method begins to train the coefficients by initializing two loop variables, i and p, to 0 in steps 1010, 1012 respectively. Then, in step 1014, the device obtains a partial derivative of the cost function, J(θ), on the current coefficient, θp, where the cost function may be defined in some embodiments as
where m is the number of training examples in the training data set, hθ(x) is the trained function using the current coefficient set θ, x(j) is the set of features for the jth training example, and y(i) is the desired output (i.e., the label) for the jth training example. Thus, following a batch gradient descent approach, the partial derivative on coefficient p (θr) may be
where xp(i) is the pth feature in the jth training example (or when p=0, xp(j)=1).
In step 1016, the device increments p and, in step 1018, the device determines whether all coefficients have been addressed in the current loop by determining whether p now exceeds the total number of features to be included in h(X). If not, the method loops back around to step 1014 to find the next partial derivative term.
After all partial derivatives are found for the current iteration, the method 1000 proceeds to reset the loop variable p to zero in step 1020. Then, in step 1022, the device updates the pth coefficient, θp, based on the corresponding partial derivative found in step 1114 and based on a preset learning rate. For example, the device may apply the following update rule:
where α is a learning rate such as, for example, 0.1, 0.3, 1 or any other value appropriately selected for the desired rate of change on each iteration.
In step 1024, the device increments p and, in step 1026, the device determines whether all coefficients have been addressed in the current loop by determining whether p now exceeds the total number of features to be included in h(X). If not, the method loops back around to step 1022 to update the next coefficient. Note that according to the method 1000, all partial derivatives are found in a first loop prior to actually modifying the coefficients in a second loop so that the partial derivatives are not taken based on the partially updated values. Other embodiments may not implement such a “simultaneous” update of the coefficients.
After all coefficients are updated, the method proceeds to step 1028 where the variable i is incremented. In step 1030, the device determines whether i now exceeds a predefined maximum number of iterations to ensure that the method 1000 does not loop indefinitely. A sufficiently high maximum number of iterations may be chosen such as 1000, 5000, 100000, etc. If the maximum iterations has not been reached, the method 1000 proceeds to step 1032 where the device computes the current cost, using the cost function J(θ), based on the training set. In step 1034, the device determines whether the function h(X) has converged to an acceptable solution by determining whether the change in the cost from the last iteration to the present iteration fails to meet a minimum threshold. If the change surpassed the threshold the method loops back to step 1012 to perform another coefficient update loop. If, on the other hand, the maximum iterations is reached or the cost change is below the minimum threshold, the method 1000 proceeds to step 1036, where the device stores the coefficients as part of the new model for extracting the parameter and the method 1000 proceeds to end in step 1038.
It will be apparent that, in addition to following approaches other than regression, other embodiments may utilize different methods for tuning coefficients in a regression approach other than batch gradient descent. For example, some embodiments may use stochastic gradient descent, wherein each coefficient update is performed based on a single training example (thereby removing the summation from the partial derivative), and the method additionally iterates through each such example. In other embodiments, the normal equations for regression may be used to find appropriate coefficients, using a matrix-based, non-iterative approach where the set of coefficients is computed as
θ=(XTX)−1XTy
where X is a matrix of features from all training examples and y is the associated vector of labels.
According to the foregoing, various embodiments provide a means for client applications to determine opportune times for initiating user interactions. By monitoring smart phone or other device usage, an operating system or other service may identify times when a user is open to receiving various types of interactions. Further, by training different machine learning models for different types of opportunity (e.g., mental vs. physical opportunity, or gradations thereof), the identification of opportune times for interaction can be further tuned to the specific type, format, medium, or content of the user interaction proposed or desired to be initiated by a client application. Various additional benefits will be apparent in view of the foregoing.
It should be apparent from the foregoing description that various example embodiments of the invention may be implemented in hardware or firmware. Furthermore, various exemplary embodiments may be implemented as instructions stored on a machine-readable storage medium, which may be read and executed by at least one processor to perform the operations described in detail herein. A machine-readable storage medium may include any mechanism for storing information in a form readable by a machine, such as a personal or laptop computer, a server, or other computing device. Thus, a machine-readable storage medium may include read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash-memory devices, and similar storage media.
It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the invention. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in machine readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
Although the various exemplary embodiments have been described in detail with particular reference to certain exemplary aspects thereof, it should be understood that the invention is capable of other embodiments and its details are capable of modifications in various obvious respects. As is readily apparent to those skilled in the art, variations and modifications can be affected while remaining within the spirit and scope of the invention. Accordingly, the foregoing disclosure, description, and figures are for illustrative purposes only and do not in any way limit the invention, which is defined only by the claims.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2017/065468 | 6/22/2017 | WO | 00 |
| Number | Date | Country | |
|---|---|---|---|
| 62353601 | Jun 2016 | US |
