Patent Application
20120209907
With the widespread adoption of the Internet, more people have access to more information than ever before. For example, people can track topics of interest by navigating a variety of online resources like social networking sites, websites, online communities, web blogs, feeds in syndication formats such as RSS or ATOM, e-mails, and Internet search engines. Generally, the user manages accessing each online resource individually, and the user actively searches the content received from different online resources for information pertinent to the user's current circumstances. Although the user can reach out for information on their topics of interest, the user can still miss out on information about events, and experiences of others which he or she would have liked to have known about.
Technology is presented for providing content related to a user to another user based on a context of the other user. For ease of description, the user is hereafter referred to as a broadcaster or broadcast user, and the other user is hereafter referred to as a subscriber or subscriber user. An example of a broadcast user is a celebrity. Other examples include an organization, a brand, a work project, a band or other entity which has relevance in different contexts.
In one embodiment, the broadcaster and the subscriber each have accounts with a context relevant content aggregation and distribution service system which can be implemented in a cloud computing system or environment. Content is automatically and continuously gathered about topics of interest to the broadcast user. Some examples of topics of interests are people, places, things, activities, ideas, fields of study, research topics and one or more categories of or related to each topic. Some examples of content are a recommendation, an advertisement, a search result, a search term, a photo, a video, an audio clip, and comments entered in a social networking site or web blog, a text message, or an e-mail. Content can be derived or created based on content in different forms. For example, a message may be formulated based on data received from a purchase transaction application of a merchant system or a user device providing information about what a broadcaster bought in a location or type of store.
The context of the subscriber is determined on a continuous basis by the system based on context information received from one or more computer devices associated with the subscriber. Distribution of the content of the broadcaster is based on the context of the subscriber. In many embodiments, the distribution is based on the context of the subscriber and is performed independently of the current context of the broadcast user. Some examples of context information which may be used to determine a context for a user are actual physical location as may be provided by GPS, a WiFi connection point, or a cellular transmission point radius, IP address of a computer to which the user has logged in, a photograph to which an image recognition technique is applied, photo metadata, time of day and time of year, a computer network connection mode associated with a computer device, for example, work, home, public, etc., who is with the subscriber, an activity the subscriber is performing, an availability of the subscriber based on information stored by a scheduling software application, or what a subscriber is inputting to a user input device of an associated computer device communicatively coupled to the aggregation and distribution system.
The broadcaster and the subscriber can control distribution and reception of content based on subscription settings.
In one embodiment, the technology provides a method for providing content related to a user to another user based on a context of the other user. The method comprises subscribing a user account of a subscriber user to a user account of a broadcast user, and automatically gathering content about one or more topics of interest for the broadcast user from online resources. Context information is received from one or more client modules executing on one or more computer devices associated with the subscriber user. Content gathered for the broadcast user is filtered based upon contextual relevance to the subscriber user, and the filtered content is reported to one or more client modules executing on the one or more devices associated with the subscriber user.
In another embodiment, the technology provides a system for providing content related to a broadcast user to a subscriber user based on a context of the subscriber. The system comprises a first set of one or more computers comprising a radar service that is in communication with one or more client modules executing on computer devices associated with a broadcast user and a subscriber user, and a second set of one or more computers comprising an intelligence system that is in communication with the first set of one or more computers comprising the radar service.
The first set of one or more computers receives indications of topics of interest and context information from a first client module associated with the broadcast user and from a second client module associated with the second user. The second set of one or more computers stores one or more indexes of resources to search for information available for aggregation. The first set of one or more computers uses the indications of topics of interest from the first client module and the one or more indexes to automatically and continuously gather content about the topics of interest to the broadcast user from the information available for aggregation. Additionally, the first set of one or more computers filters the gathered content based on the topics of interest of the broadcast user and the context information provided by the second client module for the subscription user. The first set of one or more computers pushes the filtered content to one or more client modules executing on computer devices associated with the subscriber.
The technology also provides one or more processor readable storage devices having processor readable code embodied thereon, which when executed by one or more processors causes the one or more processors to perform a method for providing content related to a user to another user based on a context of the other user. The method comprises subscribing a user account of a subscriber user to a user account of a broadcast user. Content is automatically gathered about one or more topics of interest for the broadcast user from online resources, and a context is determined based on context information received from one or more client modules executing on one or more computer devices associated with the subscriber user. The content gathered for the broadcast user is filtered based on relevance to the context of the subscriber user, and the filtered content is reported to one or more client modules executing on one or more user computer devices in accordance with device limitations indicated by the subscriber user for receiving the filtered content.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.
Technology described herein provides content based on a broadcast user's interests (e.g. celebrity's interests) to a subscriber based on the context of the subscriber. As mentioned above, a broadcast user to whose account others subscribe may be other than a human being. For example, the broadcast user account may be for an organization, a celebrity, a brand, a work project, a band or other entity which has relevance in different contexts. For example, human beings change their contexts constantly by traveling, performing different activities and associating with different groups of people. Organizations may have offices, programs and transactions in many different contexts. A work project may be supported by workers, online resources and materials having different contexts and contexts which change.
Relevance of content to topics of interest to a broadcast user may be determined from information typically stored in a user profile such as the user's demographics data, for example age, gender, and place of residence and identified topics of interests. Topics of interests may be identified by a user explicitly, and the content aggregation and distribution service system may identify topics passively, for example by monitoring a user's browsing history and social networking history.
Content is information which is in a form accessible by a computer system. For example, as permitted by a user the recording and streaming of a conversation by a smartphone results in streamed audio data which can be processed by a computer. Other examples include video, photographic and audio data stored in computer readable formats as well as text data and computer readable data representing physical sensations (e.g. vibrating smartphone) and movements. Content can also be generated which is derived from other content items.
The technology provides embodiments of a context relevant, content aggregation and distribution service which constantly gathers content relevant to a broadcast user from a variety of online resources, filters the content relevant to a context of a subscriber and reports the filtered content to the subscriber. The service presents content about a broadcast user to a subscriber based on the subscriber's context so the subscriber can act upon the information in the content.
The service may be embodied to leverage the power of cloud computing systems. A cloud computer system or environment of systems is able to communicate over communication networks using different protocols. For example, a cloud computing system may be implemented by computers which can interface with different types of communication systems and make the information available in formats for the other computers to access. Some examples of different types of communication systems are various wireless telecommunication systems to receive and forward telephone calls and Short Message Service (SMS) messages, Plain Old Telephone Service (POTS) and ISDN lines as well as computer networking standards such as TCP/IP, Ethernet and the wireless 802.11 series connection protocols.
The service provides advantages over other data discovery systems in that content is gathered from many different types of online resources such as e-mail, social networking sites, websites, and other data accessible via online communication networks as permitted by a user. Additionally, the content can be provided across many of a user's different types of computing devices.
The exemplar computing devices for exemplar user A include a cellular telephone 2, a television or set top box 4, a desktop computing device 6, a mobile computing device such as a laptop 8, a game console 12 and an automobile based computer device 14. Some examples of computing devices for an exemplar user B are also illustrated including a smartphone 16, a TV/STB 18, a desktop computer 22, a tablet computer 24 as well as a game console 26 and an automobile based computing device 28. Computing devices 2-28 are examples of multiple computing devices that can be operated by a particular user. Other computing devices can also be used with the technology described herein. The user can interact with various entities via cloud 10 using any of the devices 2-28.
The context relevant content aggregation and distribution service system 30, automatically and continuously finds and aggregates relevant and current information about respective topics of interest based on the current respective contexts of user A and user B. The information is reported to the respective user, typically ubiquitously on a user interface of one or more of the respective user's devices 2-28. In one embodiment, system 30 includes one or more servers 201, 202 . . . 20N. The number of servers used to implement system 30 is not a requirement, and is based on bandwidth, demand, performance of the servers and other factors that are implementation specific. In one example, system 30 is implemented by multiple server farms.
In operation, a user will use any of devices e.g. 2-14, 16-28 (at different times or concurrently) in order to perform various tasks (e.g., work, entertainment, social, etc.). These devices have software (e.g. a client module for the service as discussed in
Additionally, by gathering information from sources such as IP addresses, computer network connection mode designations (e.g. work, home, public), Global Positioning System (GPS) modules, WiFi connection nodes, which computers a user is logged into, time of day, calendars, schedules, a subscriber identification module (SIM) for a mobile phone, applications being executed on a user computer device (e.g. a document processing application or a game) and the like, the client module software executing on the device can determine contextual information for the user and communicate this contextual information to the service in the cloud. The cloud based service system 30 may determine a context for the user based on contextual information derived from information from a variety of devices and online resources.
The topics of information and context information are received for further searching, aggregation and/or filtering of data in order to identify that data to a respective user (e.g. User A or User B) via any of the devices 2-28. The information sent to system 30 is also used to create a digital history for the user. In addition, it is used to make the system smarter, so the system can learn more about the user and improve the relevancy of the information that is pushed. Users A and B may each be subscribers of each other and broadcasters to each other.
User computer device 38 illustrates some examples of software components which may be embodied in a client module 32A. A notification control software unit 31 interacts with a communication interface 42, for example an application programming interface (API), of the service 30 for sending context information collected by a context unit 39 to the service 30 and receiving a notification of available contextually relevant content.
As illustrated in
An output select software unit 33 identifies which output device is to provide the output to the user. This selection may be identified from the notification of content, for example in a message or metadata of the notification. As in the illustrated example of the communicatively coupled output device 37, via a wireless connection (e.g. Bluetooth) in this example, the output select unit 33 directs the notification or content to the output device 37 which has been selected for the content to be received. The selection of device may be based on the topic of the content and/or the context of the user. An output interface unit 35 formats the notification or content in a format which may be processed by the selected output device. A display or other audiovisual output device which is integrated in user computer device 38 may have been the selected output device in other examples.
In this embodiment, exemplar software components of the service 30 which provide content and determine the context for provision to the client modules 32 comprise magnets 44, a controller 46, one or more recommendation engine(s) 48, and one or more search engine(s) 50. Additionally, the software components comprise a communication interface 42 for communicating with computers of online resources via a variety of different communication protocols and with one or more client modules 32 executing on one or more user computer devices 36, 38. The software components of the service 30 further comprise a subscription manager 52 which filters content of a broadcast user based on relevance to the context of a subscriber second user subscribed to the broadcast user account. The manager 52 causes the filtered content to be reported to a client module 32 of a user device associated with the subscriber user. In some examples, the subscription manager 52 may also base the filtering on common topics of interest between the first and second users in addition to the context of the second user. The common topics may be identified passively by monitoring the online history of actions of each user or via active selection by each user.
The magnets 44 are software processes that automatically and continuously collect content about a topic of interest (e.g., person, place or thing). For example, magnets look for content (via cloud 10) relative to a seed (e.g., topic or specific content). For example, if a magnet is provided with a seed topic (e.g. a type of shoes, musical group, a subject matter, etc.) then the magnet will search the Internet (or other source) to find content available that is related to that seed. What makes the magnet different from a typical Internet search engine is that magnets are persistent, personalized based on information the magnet retrieves about a user as may be stored in a user profile, and other user information stored by other services and databases to which the system 42 has access via the interface 42. For example, another service may be a social networking site or an online gaming service such as XBOX® Live which stores a user's gaming statistics and preferences. A magnet may also use context in its searches and can interact with other magnets, can show many types of content (games, music, tweets, search terms) from both public and private indexes, and the content that is returned has context (such as returning a place and saying who else is there).
One or more magnets may be defined to search for content of a broadcast account which is relevant to the context of a subscriber user. Additionally, one or more magnets may be defined to search for an intersection of topics of interest for both broadcaster and subscriber and which are context relevant to the subscriber.
User interface representations for magnets allow a user to interact with them (e.g. to get information on a specific topic). For example, a user can interact with a magnet via a user interface (e.g., touch screen) to set up, configure and see results of a magnet. If a users pulls two magnet representations near each other (e.g., on a user interface), they'll effect each other. For example, a person pulls their restaurants magnet near their New York magnet on a touch screen display, and in response the two magnets interact and output an intersection of the topics being searched (e.g., restaurants in New York). The client module 32 reports the user input dragging the magnets to the service 30 which updates search parameters to limit restaurant results for restaurants in New York. Magnets may also be “pruned.” For example, a user may personalize a magnet about a singer to show fashion, music, and gossip, but never anything jail-related.
In one embodiment, magnets are virtual objects that attract similar particles, customized for a broadcast user and the subscriber user's context. A magnet is typically centered on a topic of interest and attracts a collection of particles related to that topic, filtered and prioritized based on the profile of the user who owns the magnet and the subscriber user's current context, for example the time, location, device, activity, and living things like pets and people the subscriber is with.
A particle is a statement or suggestion which may be displayed or outputted. A statement particle has a small amount of textual information such as “Stocks are down right now” or “Mom's birthday is tomorrow.” A suggestion particle may have a link to some digital content such as “Karma Club DJ Night” or “Halo® 2.” Suggestion particles like (i.e. find out the full Karma event details, or go play the game) can be acted upon. For example, a particle may include a Uniform Resource Locator (URL) or magnetic link which a user can select to find details for the event at the Karma Club or sign in online to the game of Halo 2. A user can magnetize a particle, give feedback on a particle, hide, or save a particle.
Particles can have value and actions. Examples of values include a pointer to the actual content, metadata (type, format, thumbnail), pizzazz (by surpassing a relevance threshold, a particle gets pizzazz, which draws attention to it), and relevance (the system 30 figures out the relevance and applies these values). Some examples of actions include “Go to the content,” “Magnetize this (have it attract similar particles, basically turning it into a temporary magnet),” “Give Feedback on this (thumbs up, thumbs down),” “Hide this (make this go away)” and “Pin this, so I can get back to it.” Other values and actions can also be included. Magnets attract particles related to a particular subject or interest. A digital history may include a trail of particles that a person acted upon or viewed at some point in the past. Aggregation system 30 shows the most relevant particles based on current context.
One or more search engines 50 are used to search various sources available via cloud 10. In one embodiment, a magnet 44 uses one or more search engines 50 to search. In other embodiments, magnets 44 and search engines 50 will search independently for information. Recommendation engine 48 will provide recommendations of information for a user based on the results from magnets 44 and search engines 50.
Controller 46 acts as the central brain that coordinates the operation of interface 42, magnets 44, recommendation engines 48, search engines 50 and subscription manager 52. In operation, topics of interest and context information is received at interface 42 and provided to controller 46. In one embodiment, the controller 46 provides the topics/and or context information to magnets 44, recommendation engine 48, search engines 50 and subscription manager 52 in order to obtain additional data and/or to filter data already found. In some embodiments, magnets 44, recommendations engines 48, search engines 50 and subscription manager 52 provides all of the data to the controller 44 which filters the data based on the context information and provides the result of the filtering to the appropriate client module (36 or 38) via interface 42. In some situations, the information is reported to the subscriber user on the same device/client that provided context information to system 30. In other situations, the context information is provided to system 30 from a first device associated with the subscriber (e.g., device 2 for User A) and the information from system 30 is pushed to a second device associated with the subscriber user (e.g., device 4 for User A) because the subscriber has changed the device the subscriber is working with or the second device is a more appropriate platform to report the information.
The subscription manager 52 interacts with a user computer system via the interface 42 to set a user account to be subscribed to or to associate a user account as a subscriber to another user account. After the account subscription settings have been updated, in one example, an instance of the subscription manager 52 may execute for filtering the content of a broadcast user account based on one or more contexts of subscriber accounts. The controller 46 may send a message when content is available for different users or the subscription manager may perform periodic checks of one or more database(s) (e.g. 21) to identify new content for the broadcast account. The context of the subscriber users associated with the broadcast user account is monitored. Additionally, the subscription manager 52 may use one or more magnets to track mutual topics of interest for filtering the broadcast content which may be forwarded to a subscriber. Furthermore, the broadcaster, subscriber or both may have selected settings which are also criteria for filtering the broadcast content. The subscription manager 52 may notify the controller 46 of the filtered content to be sent, the interface 42 or both. In some examples, the controller 46 directs the user profile data of the broadcast user and subscribers be updated to reflect the content sent to the subscribers.
In other embodiments, some of the functions of the software components of the service 30 may be processed by the client modules 32 or both software instances of one or more components (e.g. 42, 46, 44, 50, 48 and 52) operating on servers (e.g. 20) and the client modules 32.
For illustrative purposes and ease of description only, reference is made to the software components of
In step 63, the service 30 automatically gathers content about one or more topics of interest to the broadcaster from online resources and receives in step 64 context information from one or more client modules for the subscriber user. The subscription manager 52 in step 65 filters content gathered for the broadcaster based on relevance to the context of the subscriber and works with the service 30 components to report in step 66 the filtered content to one or more computer devices associated with the subscriber. In some embodiments, the filtering is based on the current context of the subscriber and is independent of the current context of the broadcast user.
The subscription manager 52 may in step 67 obtain one or more subscription settings from the broadcast user related to content distributed to subscribers. The subscription manager 52 may also in step 68 obtain one or more subscription settings from a subscriber for receiving content from the broadcast user account. In some examples, the subscriber may have settings indicating to display the content without waiting for a user response to the notification. In step 69, the service 30 reports the content from the broadcast account filtered based on a context of the subscriber and the subscriptions settings of the broadcast user and the subscriber.
In step 72, the subscription manager 52 receives a selection of one or more topics of interest to a broadcast user available for content sharing with subscribers and stores in step 74 the broadcaster's selection of the one or more topics in subscription settings for the broadcaster. The selectable topics of interest may have been identified or recommended or both by the service 30 for the user for selection to be shared with subscribers. For example, a famous celebrity chef may only want to share content related to cooking and kitchen design, but not his personal life or financial deals. In this way, a broadcast user can indicate which topics he or she wants to be shared with subscribers of the service independent of the online source for the content. The chef may have a Facebook® page and a Twitter® page and can enter information, e.g. a recipe or restaurant review, on either one of these social networking sites and know it will be distributed to his subscribers who have an account with the service by the service 30 whether or not his subscribers have a Facebook or Twitter account.
In step 76, the subscription manager 52 receives a selection of one or more authorization criteria for content to be shared with subscribers. A broadcast user may set an authorization criteria that he or she approve every selection or item of content to be sent to subscribers. In another example, the broadcast user may designate only items initiated by the broadcast user as being permitted to be shared with subscribers. In the famous chef example, the chef may not want content of reviews of his recipes or TV program by critics sent to all subscribers, but permits a review or recipe he enters to be shared automatically. In another example, the chef may limit by authorization criteria the types of content to be shared. For example, he may not wish his search terms for a prime rib recipe shared with subscribers, but only items he designates as recommendations like his “likes” and “dislikes” lists from one of his social networking sites and comments he initiates. Subscribers may be grouped into categories and different authorization criteria applied to the different categories. For example, demographic data from user profiles and other identifying characteristics (e.g. approximate weight class, or relationship to the user) may be used to group subscribers. The subscriber manager 52 in step 78 stores the selected authorization criteria in a subscription settings datastore (e.g. in a database 21 controlled by the service) for the broadcast user account. The broadcast user account settings may be stored or linked as part of user profile data for the broadcast user as well.
Additionally, the subscriber manager 52 in step 79 receives from the broadcast user a selection of one or more context limitations on content to be shared with subscribers. A celebrity or members of an organization may not want children receiving content during school hours or when a subscriber is driving or at work. For example, through GPS and device settings, the user computer device of a subscriber can provide this context information so the controller 46 and/or the magnets 44 and the subscription manager 52 determine a subscriber's context includes respectively “school” “driving” or “work.” In another example, the broadcast user can designate a time delay on content distributions. For example, a context limitation may be set for privacy considerations such that context information on the location of the broadcaster or an activity he or she performed at a location is not published to subscribers until a set time period (e.g. a day, a week, two hours) after the broadcast user has left the location. In step 90, the subscription manager 52 stores the broadcast user's selection of one or more context limitation selections in the subscription settings for the user. The subscription settings for each broadcast user account associated with a user may also be stored or linked as part of the user's profile data.
Analogous to the user providing the content to the subscriber, the subscriber can indicate for which topics he or she wants to receive content from the broadcast account independent of the online source for the content. In the chef example above, the chef may enter a comment on his Facebook or Twitter page, and the subscriber need not have accounts with either of these social networking entities and still receive the comment in his or her service account. If the chef says one thing on Facebook and another on Twitter, a subscriber with accounts to these does not need to log in to see the comments; both are pushed to the subscriber via the service 30.
The subscription manager 52 also presents the subscriber the opportunity to limit the context in which he or she receives broadcast content. For each account to be broadcast, in step 86, the subscription manager receives from the subscriber a selection of one or more context limitations for content from each broadcast user account and stores in step 87 data indicating the one or more context limitations selections in the subscription settings for the subscriber.
An example of a context limitation is a limitation to not receive content when the subscriber is physically at work, or connected to his or her work computer network (e.g. work mode) or driving or while the subscriber is at the gym or meditation class. Additionally, context limitations may limit content by topic. Another set of limitations, which may be considered a subset of context limitations, are device limitations. For example, work project account or a subscriber to that project account may set the subscription settings so that subscribers, those working on the project, may only access content for the work project on any subscriber device logged in via a secure connection to a designated work computer network. However, a subscriber may set subscription settings that content about a celebrity's social life is limited to one or more particular user computer devices and may also be limited to which network (e.g. home, school or work) a device is connected.
For each broadcast account to be subscribed, in step 88, the subscription manager receives from the subscriber a selection of one or more device limitations for content from each broadcast user account and stores in step 89 data indicating the one or more device limitations selections in the subscription settings for the subscriber.
Subscription settings for both the broadcast user and the subscriber may be linked to the respective user profile data.
Below is an example of how content may be generated, captured and distributed by a subscription service for providing content related to a user to another user based on the context of the other user. Returning to the celebrity chef for this example, the celebrity sets up his user account with the context relevant content aggregation and distributions service (e.g. 30). As part of this, the celebrity chef may enter demographic data about himself, select topics of interest, link external accounts (e.g. Facebook, Twitter, Mint.com) provide access privileges to the service 30 as desired, and set his subscription settings for subscribers to his account, and for himself for any accounts to which he subscribes.
The celebrity chef has lunch at a new restaurant in Vancouver. He may be located there by the GPS in his mobile telephone or by information he uploads to his service account or an external account linked to his service account. For example, the chef launches his client module 32 and enters a ‘likes’ recommendation for the restaurant and adds a comment “try the pretzel with welsh rarebit!” The cloud service may create a ‘place card’ display for the restaurant with information like its name, address, hours, phone, and the celebrity's comments. The subscription manager 52 monitors content received for the celebrity, for example by being registered with an API to receive notice of updates to the celebrity's account or is continuously checking the account records for new content. The subscription manager 52 identifies the recommendation and place card storage location in a database 20. The subscription manager 52 looks at contexts currently identified with subscribers to the celebrity chef's account.
A user who subscribes to the celebrity chef account launches his client module 32 on his smartphone (e.g. 16). The user may actively indicate he is ready to receive updates from the celebrity chef's account. The smartphone 16 provides context information (e.g. SIM card location or GPS status) to the service 30 which the subscription manager 52 accesses. The user walks within a present distance of the Vancouver restaurant for which the celebrity chef provided a recommendation. The subscription manager 52 may also use the subscriber's preferences in his user profile data for determining if content is to be made available to the user. For example, the subscription manager 52 may determine not to share a celebrity chef's comments on a crab house restaurant when the subscriber's profile indicates he is allergic to shell fish or does not want information on such types of restaurants.
Assuming the type of restaurant is not a limitation on sharing content, the subscription manager 16 sends a notification to the notification controller 31 of the smartphone which displays via the output interface 35 a notification for indicating to the user that he is in proximity to the restaurant that the celebrity recently visited. The user taps on the notification on his smartphone user interface and is able to view the restaurant's place card including the celebrity's comment about the pretzel which has also been downloaded from the cloud based service. The user may then decide to walk up to the restaurant to check out the interior and a menu.
If such an intervening communication device is available, the subscription manager 52 determines whether the selected user device is available for receiving content in step 94. For example, the subscription manager 52 may check context information from the intervening device which besides things it may contain like a network connection mode, IP address and physical location (e.g. based on GPS, cell tower connection, WiFi Hotspot connection) may also include other devices connected or communicatively coupled to it wired or wirelessly. Additionally, the subscription manager 52 may send messages to the client module on the intervening device to obtain the connection status of the selected device. Again, if the selected device is determined not to be available, for example, it is not connected to the intervening device, then in step 96, the subscription manager does not send the filtered content for the selected device. If, however, the selected device is available, the subscription manager 52 in step 95 sends or has sent the filtered content to the intervening user device communicatively coupled to the selected user device with an indication to output the filtered content via the selected user device.
In response to an event, an intelligent processing module 174 will obtain appropriate data in data store 172, user profile data 176 and context data 178. The user profile data can be information about a user such as demographic information, behavioral information, web browsing history, search topics, friends, user accounts subscriptions, etc. Context data is gathered for identifying a current context or context in a defined time period for a subscriber user and can also be gathered for a broadcast user to be stored as data upon which context relevance for a subscriber is determined Examples of context data include the location of a user, what a user is doing, what device the user is interacting with, who the user is interacting with, what the user is saying, time of day for the user, current persona of the user (e.g., parent, employee, spouse, coach, commuter, etc.). The intelligent processing module 174 evaluates the information described above and identifies information of interest to a subscriber based on that evaluation. Additionally, intelligent processing 174 may utilize one or more recommendation engines and/or search engines. In one example, the process embodiments of
In one embodiment, there's a secondary form of intelligent processing that takes the user profile data, the history, behaviors, (and possibly everything known about a user from using properties like email, messenger, Internet searches) for the broadcaster and the subscriber, and combines that data together to make intelligent inferences. For example, the inference engine may determine that both are financial gurus, but the subscriber is not that interested in sports. Even if the broadcaster has selected sports as a sharable topic, the intelligent processing engine may implement a rule to only share sports information if the subscriber is at a sporting event or viewing one on TV or the Internet. For financial information, the system may follow a rule to forward financial information content from the broadcaster in many more contexts allowed by the subscriber and deemed relevant by the service. Some inferences receive higher confidence values based on user actions. For example, finance as a topic of interest may receive a high confidence value and a magnet is created for finance. Each of these inferences about the broadcaster and subscriber is assigned a confidence level value for how much mutual interest there is in the topic.
The intelligent processing module 174 sends information identified as contextually relevant content for a subscriber to an output adaptor 180 which adapts the data for the particular target user computing device. Some examples of the target devices shown in
Any one or more of the components of
In this embodiment, an example of a client module 32 identified as a radar client 250 is software running on any of the physical computing devices of a user which is in communication via a software interface such as the exemplar application programming interface (API) 254 with one or more applications of a cloud based, contextually relevant content aggregation and distribution service, referred to here as a “radar” service 252. Like a radar, the service 252 and client modules 250 search for information and track it. A radar client 250 provides context information (and, optionally, information about a user's interest) to radar service 252 and receives back content such as recommendations from a broadcast user account.
The received content, e.g. recommendations, can be reported to the subscriber as discussed below (see e.g.
The cloud based radar service 252 includes an API for a set of recommendation engines 254, 256, 258 and 260, each of which makes different types of recommendations for different types of content. For example, one recommendation engine may make recommendations for music, another for shopping, another for parties, another for restaurants, etc. A system can have more than four recommendation engines. The radar service 252 includes magnets 265, which are discussed above, and a relevancy engine 264. Magnets make use of the recommendation engines to identify content for a user. Additionally, recommendation engines can make recommendations without a magnet involved. Magnets use an API 272 to an intelligence system 270 to mine the world of knowledge for information related to one or more topics of interest to a broadcast user or of mutual interest to a broadcaster and a subscriber. Magnets use the recommendation engines to make interesting recommendations for the subscriber about the one or more topics based on the current context of the subscriber. All information and recommendations made by both magnets and recommendation engines are then sorted by the relevancy engine 264 and provided to the radar client 250.
In one embodiment, magnets make use of the recommendation engines to identify content for a user and the one or more instances of relevancy engines scores each item of content based on its perceived relevancy to the current context of a subscriber. In one embodiment, the relevancy engine 264 provides a numerical relevancy score based on how pertinent the item content is to the current context of the subscriber. For example, a restaurant recommendation may get a higher relevancy score between 3-5 PM, then at midnight. Similarly, information about a football team may receive a higher relevancy score during the football season than between seasons. Information about a band the subscriber is currently listening to, will get a higher score, than another band or other content related to work.
Additionally, in this embodiment, the relevancy engine 264 evaluates gathered content for relevance to the subscription settings of the broadcaster and the subscriber. The relevancy engine 264 identifies content for output processing to the subscriber by the other components in the system such as the radar service 252 and its API 254.
As noted above, the radar service 252 communicates with the intelligence system 270 via an API 272. The intelligence system 270 creates, obtains and stores information about data available to the radar service 252. In one embodiment, the radar service 252 is implemented by a set of one or more computers (e.g., a server farm) and the intelligence system 270 is implemented by a different set of one or more computers (e.g., a server farm). In other embodiments, one or more of the same computers can implement both the radar service 252 and the intelligence system 270.
In one example, database management software 216 may be contacted by the radar service via the API 272 to access data from the intelligence system 270 which includes a public index 274, a datastore or index for ads and offers 278, a private index 280 and user profile data 282. The public index 274 may be an Internet Search index of web sites on the World Wide Web. The private index 280 includes a separate index of private sites for each user. For example, each user will have a private index which provides information about their social networking pages, email, contacts, etc. In some embodiments, the user logins to the radar service via a single sign-on procedure allowing the radar service to maintain sessions with the user's private accounts. The ads and offers datastore 278 provides a list (e.g. with a pointer to or the actual content) of advertisements and offers to consumers that are available. User profile data 282 stores information known, e.g. knowledge base, and inferred, e.g. inferences, for each user as well as context data and history data including a digital history for the respective user.
In one embodiment, the intelligence system 270 includes an inference engine 283, which is software that guesses information about users based on their behavior. The inference engine can provide context information for a subscriber that includes an intent or state of being of a subscriber derived from an inference based on an action of the subscriber. For example, if a subscriber is reading about a restaurant, the inference engine may assume the user is desiring to eat (e.g., the user is hungry). The response may be to show the user comments from a celebrity broadcaster of a restaurant nearby that serves the food the user was observed to eat in the past. The public index 274, ads and offers store (or index) 278, and the private index 280 are created based on crawling or searching on the World Wide Web, social networking systems, collaborative services, multiplayer game services, search engines, movie and music distribution services, purchase systems, text messages, call history, e-mail accounts, examples of which are illustrated as feeds 284 and/or other networks or sources.
In operation, magnets 265 will utilize the recommendation engines 254, 256, 258, 260, the public index 274, the ads and offers store (or index) 278, the private index 280 and the user profile data 282 to identify content of interest to the user. The relevancy engine 264 provides a relevancy score for the items found based on the subscriber's current context. Items of content closest to the subscriber's current context will be given a higher score. Items with the higher scores will be pushed to the radar client module 250 on a subscriber device by the cloud based radar service 252.
The processes, software components or the functionality they provide may be embodied or stored as code, software, instructions, firmware, or the like on processor readable storage media and which can be executed by a processor. An example of processor readable storage media are a computer readable storage medium such as memory, whether in a volatile or non-volatile form, removable or non-removable. Furthermore, the particular naming and division of the modules, applications, software components, and other aspects are not mandatory, and are used for illustrative purposes. The mechanisms that implement the technology or its features may have different names, divisions and/or formats.
There are many ways for alerting the subscriber that the context relevant aggregation and distribution service (e.g. 30, 150, 254) has information to report.
In response to receiving the topics in step 202, the system 30, 150, 252 will create a new magnet or update an existing magnet (or other software component) in step 204 to automatically and continually search and filter content for the topics of interest without the broadcast user requesting that the content be gathered. If the topic of interest is something new, a new magnet is created by creating new criteria and starting a process to search the internet or other space for content related to that new criteria. If the topic received in step 202 is similar to the focus of an existing magnet, the existing magnet can be changed to modify its focus based on the new topic. In step 206, the magnets automatically and repeatedly search and gather content related to the seed for each magnet. For example, the magnets may identify events or news relevant to one or more of the topics. Although
In step 208, one or more client modules will obtain context information for a subscriber associated with those client modules. The context information can be obtained interactively, passively or actively (concurrently or at separate times). The context information obtained in step 208 is provided to the content aggregation and distribution service system 30, 150, 252 in step 210. For example, the information is transmitted to a server that is part of the service system 30, 150, 252. In step 212, system 30, 150, 252 will filter content from one or more magnets based on the current context information provided in step 210. It is contemplated that the various magnets associated with a broadcast user will search for and identify a large amount of information. The current context information can be used to filter that information to a smaller subset. For example, if a magnet is searching for information about shoes a celebrity wore, and current context information indicates the location of the subscriber, the information about shoes can be filtered to only provide information about shoes sold in a store geographically close to the subscriber. In step 214, the information identified in step 212 is used to create an output message and that output message is sent to a client for the subscriber user.
The content is sent to whatever client(s) is/are active for the user. In some instances, the client that sent the context information is the client that is currently active and, therefore, the result data will be sent to the same client. In other embodiments, the subscriber will be interacting with a first client device when the context information is sent to the contextually relevant content aggregation and distribution service system 30, 150, 252 and interacting with a second device when the result is reported back. Therefore, the result was reported to a different client device which may be a completely different type of device. For example, a subscriber may be playing a game executing on a console when context information is sent to the service system 30, 150, 252 and be using the cellular telephone when content still contextually relevant is sent back. Therefore, the information will be packaged for the user's cellular telephone rather than the game console. In step 218, the client device that received the data will report the new content in step 218, as described above or in any other suitable manner. More information about many of the steps of
In step 400 of
As described above, magnets are software processes that search for content relative to a seed. When content is found, the magnets will create a database entry and generate an event associated with that database entry. The database entry is stored and the event will trigger the filtering described above.
Computing system 710 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computing system 710 and includes both volatile and nonvolatile media, removable and non-removable media, including RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by computing system 710.
The system memory 730 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 731 and random access memory (RAM) 732. A basic input/output system 733 (BIOS), containing the basic routines that help to transfer information between elements within computer 710, such as during start-up, is typically stored in ROM 731. RAM 732 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 720. By way of example, and not limitation,
The computer 710 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
The drives and their associated computer storage media discussed above and illustrated in
The computer 710 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 780. The remote computer 780 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computing device 710, although only a memory storage device 781 has been illustrated in
When used in a LAN networking environment, the computer 710 is connected to the LAN 771 through a network interface or adapter 770. When used in a WAN networking environment, the computer 710 typically includes a modem 772 or other means for establishing communications over the WAN 773, such as the Internet. The modem 772, which may be internal or external, may be connected to the system bus 721 via the user input interface 760, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 710, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
Head mounted display device 802, which in one embodiment is in the shape of glasses, is worn on the head of a user so that the user can see through a display and thereby have an actual direct view of the space in front of the user. The use of the term “actual and direct view” refers to the ability to see the real world objects directly with the human eye, rather than seeing created image representations of the objects. For example, looking through glass at a room allows a user to have an actual direct view of the room, while viewing a video of a room on a television is not an actual direct view of the room. More details of the head mounted display device 802 are provided below. Although the device shown in
In one embodiment, processing unit 804 is worn on the user's wrist and includes a portion of the computing power used to operate head mounted display device 802. Processing unit 804 communicates wirelessly (e.g., WiFi, Bluetooth, infra-red, or other wireless communication means) to one or more computer devices such as in this example a mobile device of a smartphone 850. A client module 32 such as a radar service client module 250 is executing on the smartphone 850 to which the display device 802 is wirelessly coupled. The client module 32 provides connection information and operating status information on the display device 802 to the service 30. The smartphone display 852 displays application tiles including one for a Facebook application instance 8563. The other tiles 8561 and 8562 are overlaid with a particle or message 8541 from a contextually relevant content aggregation and distribution service system. In this example, the user is walking during lunchtime in downtown San Francisco on Market Street, and receives the contextually relevant content message of “$5.99 lunch combo at Sal's (0.1 mi. ahead on left on Market). Additionally, the user's smartphone is not logged in to her work network, but the radar service 252 sends her a message which is displayed as an overlay 8542 “E-mail from Bob Subj: Merger Failed” as content indicating she has received an e-mail in her work e-mail account. The user does not need to log in and check her work e-mail during her lunchtime to track her e-mails; a service application (e.g. 252) notifies her instead. The user has designated in her device limitations that content received from her husband's user account and a celebrity chef, Chef Bob, are to be displayed during work hours to her mixed-reality display system 802.
A portion of the frame of head mounted display device 802 will surround a display (that includes one or more lenses). In order to show the components of head mounted display device 802, a portion of the frame surrounding the display is not depicted. The display includes a light guide optical element 825, opacity filter 829, see-through lens 816 and see-through lens 818. In one embodiment, the opacity filter 829 is behind and aligned with see-through lens 816, lightguide optical element 825 is behind and aligned with opacity filter 829, and see-through lens 818 is behind and aligned with lightguide optical element 825. See-through lenses 816 and 818 are standard lenses used in eye glasses and can be made to any prescription (including no prescription). In one embodiment, see-through lenses 816 and 818 can be replaced by a variable prescription lens. In some embodiments, head mounted display device 802 will include only one see-through lens or no see-through lenses. In another alternative, a prescription lens can go inside light guide optical element 825. Opacity filter 829 filters out natural light (either on a per pixel basis or uniformly) to enhance the contrast of the virtual imagery. Light guide optical element 825 channels artificial light to the eye.
Mounted to or inside temple 812 is an image source, which (in one embodiment) includes micro display assembly 820 for projecting a virtual image and lens 822 for directing images from micro display 820 into light guide optical element 825. In one embodiment, lens 822 is a collimating lens.
More details of control circuits 836 are provided below with respect to
Micro display 820 projects an image through lens 822. There are different image generation technologies that can be used to implement micro display 820. For example, micro display 820 can be implemented in using a transmissive projection technology where the light source is modulated by optically active material, backlit with white light. These technologies are usually implemented using LCD type displays with powerful backlights and high optical energy densities. Micro display 820 can also be implemented using a reflective technology for which external light is reflected and modulated by an optically active material. The illumination is forward lit by either a white source or RGB source, depending on the technology. Digital light processing (DLP), liquid crystal on silicon (LCOS) and Mirasol® display technology from Qualcomm, inc. are all examples of reflective technologies which are efficient as most energy is reflected away from the modulated structure and may be used in the system described herein. Additionally, micro display 820 can be implemented using an emissive technology where light is generated by the display. For example, a PicoP™ display engine from Microvision, Inc. emits a laser signal with a micro mirror steering either onto a tiny screen that acts as a transmissive element or beamed directly into the eye (e.g., laser).
Light guide optical element 825 transmits light from micro display 820 to the eye 840 of the user wearing head mounted display device 802. Light guide optical element 825 also allows light from in front of the head mounted display device 802 to be transmitted through light guide optical element 825 to eye 840, as depicted by arrow 842, thereby allowing the user to have an actual direct view of the space in front of head mounted display device 802 in addition to receiving a virtual image from micro display 820. Thus, the walls of light guide optical element 825 are see-through. Light guide optical element 825 includes a first reflecting surface 824 (e.g., a mirror or other surface). Light from micro display 820 passes through lens 822 and becomes incident on reflecting surface 824. The reflecting surface 824 reflects the incident light from the micro display 820 such that light is trapped inside a planar, substrate comprising light guide optical element 825 by internal reflection. After several reflections off the surfaces of the substrate, the trapped light waves reach an array of selectively reflecting surfaces 826. Note that only one of the five surfaces is labeled 826 to prevent over-crowding of the drawing. Reflecting surfaces 826 couple the light waves incident upon those reflecting surfaces out of the substrate into the eye 840 of the user. As different light rays will travel and bounce off the inside of the substrate at different angles, the different rays will hit the various reflecting surface 826 at different angles. Therefore, different light rays will be reflected out of the substrate by different ones of the reflecting surfaces. The selection of which light rays will be reflected out of the substrate by which surface 826 is engineered by selecting an appropriate angle of the surfaces 826. More details of a light guide optical element can be found in United States Patent Application Publication 2008/0285140, Ser. No. 12/214,366, published on Nov. 20, 2008, “Substrate-Guided Optical Devices” incorporated herein by reference in its entirety. In another embodiment, there can be one light guide optical element which reflects light into both eyes.
Opacity filter 829, which is aligned with light guide optical element 825, selectively blocks natural light, either uniformly or on a per-pixel basis, from passing through light guide optical element 825. In one embodiment, the opacity filter can be a see-through LCD panel, electro chromic film, PDLC (Polymer dispersed Liquid Crystal) or similar device which is capable of serving as an opacity filter.
Head mounted display device 802 also includes a system for tracking the position of the user's eyes as a user's eyes will be directed at a subset of the environment. Head mounted display device 802 includes eye tracking assembly 134 which includes an eye tracking illumination device 134A (see
Via a user input mechanism for the see-through display 802, for example an input device such as a button or trackball on the wristband which interfaces with the processing unit 804, the user is able to select the content for Mike and the Celebrity.
In some embodiments, a broadcaster may allow subscribers to link in real-time to a video or images feed, with or without sound, of what the room or front facing camera 813 is capturing while the broadcaster is wearing them. For example, a musician is performing a concert at a stadium and is wearing a near-eye or head mounted display 802. Before the concert or during, the musician has updated his subscription settings in his broadcast account to allow subscribers identified as having a context of being physically in the stadium to be able to view on displays accessible by the service what he is seeing during the concert. The stadium computer system may be a subscriber and may be able to display on a large screen in the stadium. Subscribers may also view the feed on their mobile device displays or near-eye displays.
In other embodiments, a broadcaster may allow a video or images captured by the front facing camera 813 to be captured and stored for his broadcast account. When a subscriber is in the same context, e.g. walking on the same street in a city, or just in the same city, a notification of the available feed is received from the service by a client module and is displayed on a client device available to the subscriber. The subscriber can then view the feed on his or her mobile device or near-eye display. If the mobile device or near-eye display can access GPS or other location finding technology which can provide position information to within a street block or two, the video or images can track its output to the user's progress along the route of the video or images. Besides celebrities, this allows many opportunities to communicate with users who permit their views to be broadcast to another user to explain their surroundings in their own languages in different localities. Some examples of services which may take advantage of such capability are tourist services, emergency services, and directions services.
Power management circuit 902 includes voltage regulator 934, eye tracking illumination driver 936, audio DAC and amplifier 938, microphone preamplifier audio ADC 940, temperature sensor interface 942 and clock generator 944. Voltage regulator 934 receives power from processing unit 804 via band interface 932 and provides that power to the other components of head mounted display device 802. Eye tracking illumination driver 936 provides the IR light source for eye tracking illumination 834A, as described above. Audio DAC and amplifier 238 receive the audio information from earphones 830. Microphone preamplifier and audio ADC 940 provide an interface for microphone 810. Temperature sensor interface 942 is an interface for temperature sensor 838. Power management unit 902 also provides power and receives data back from three axis magnetometer 832A, three axis gyro 832B and three axis accelerometer 832C.
Power management circuit 1006 includes clock generator 1060, analog to digital converter 1062, battery charger 1064, voltage regulator 1066, head mounted display power source 1076, and temperature sensor interface 1072 in communication with temperature sensor 1074 (located on the wrist band of processing unit 804). Analog to digital converter 1062 is connected to a charging jack 1070 for receiving an AC supply and creating a DC supply for the system. Voltage regulator 1066 is in communication with battery 1068 for supplying power to the system. Battery charger 1064 is used to charge battery 1068 (via voltage regulator 1066) upon receiving power from charging jack 1070. HMD power source 1076 provides power to the head mounted display device 802.
Mobile device 1100 may include, for example, processors 1112, memory 1110 including applications and non-volatile storage. The processor 1112 can implement communications, as well as any number of applications, including the interaction applications discussed herein. Memory 1110 can be any variety of memory storage media types, including non-volatile and volatile memory. A device operating system handles the different operations of the mobile device 1100 and may contain user interfaces for operations, such as placing and receiving phone calls, text messaging, checking voicemail, and the like. The applications 1130 can be any assortment of programs, such as a camera application for photos and/or videos, an address book, a calendar application, a media player, an internet browser, games, an alarm application, other third party applications, the interaction application discussed herein, and the like. The non-volatile storage component 1140 in memory 1110 contains data such as web caches, music, photos, contact data, scheduling data, and other files.
The processor 1112 also communicates with RF transmit/receive circuitry 1106 which in turn is coupled to an antenna 1102, with an infrared transmitted/receiver 1108, with any additional communication channels 1160 like Wi-Fi or Bluetooth, and with a movement/orientation sensor 1114 such as an accelerometer. Accelerometers have been incorporated into mobile devices to enable such applications as intelligent user interfaces that let users input commands through gestures, indoor GPS functionality which calculates the movement and direction of the device after contact is broken with a GPS satellite, and to detect the orientation of the device and automatically change the display from portrait to landscape when the phone is rotated. An accelerometer can be provided, e.g., by a micro-electromechanical system (MEMS) which is a tiny mechanical device (of micrometer dimensions) built onto a semiconductor chip. Acceleration direction, as well as orientation, vibration and shock can be sensed. The processor 1112 further communicates with a ringer/vibrator 1116, a user interface keypad/screen 1118, a speaker 1120, a microphone 1122, a camera 1124, a light sensor 1126 and a temperature sensor 1128.
The processor 1112 controls transmission and reception of wireless signals. During a transmission mode, the processor 1112 provides a voice signal from microphone 1122, or other data signal, to the transmit/receive circuitry 1106. The transmit/receive circuitry 1106 transmits the signal to a remote station (e.g., a fixed station, operator, other cellular phones, etc.) for communication through the antenna 1102. The ringer/vibrator 1116 is used to signal an incoming call, text message, calendar reminder, alarm clock reminder, or other notification to the user. During a receiving mode, the transmit/receive circuitry 1106 receives a voice or other data signal from a remote station through the antenna 1102. A received voice signal is provided to the speaker 1120 while other received data signals are also processed appropriately.
Additionally, a physical connector 1188 can be used to connect the mobile device 800 to an external power source, such as an AC adapter or powered docking station. The physical connector 1188 can also be used as a data connection to a computing device. The data connection allows for operations such as synchronizing mobile device data with the computing data on another device.
A GPS receiver 1165 utilizing satellite-based radio navigation to relay the position of the user applications is enabled for such service.
The technology may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Likewise, the particular naming and division of modules, routines, features, attributes, methodologies and other aspects are not mandatory, and the mechanisms that implement the technology or its features may have different names, divisions and/or formats. Furthermore, as will be apparent to one of ordinary skill in the relevant art, the modules, routines, features, attributes, methodologies and other aspects of the embodiments disclosed can be implemented as software, hardware, firmware or any combination of the three. Of course, wherever a component, an example of which is a module, is implemented as software, the component can be implemented as a standalone program, as part of a larger program, as a plurality of separate programs, as a statically or dynamically linked library, as a kernel loadable module, as a device driver, and/or in every and any other way known now or in the future to those of ordinary skill in the art of programming
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. It is intended that the scope of the invention be defined by the claims appended hereto.
