Patent Application
20120215652
1. Technical Field
The present invention relates to polling and processing sensor data, and more specifically, to polling and processing sensor data of mobile devices according to context for the purposes of buying and selling high-level sensor data.
2. Description of the Related Art
The problem of establishing real-time high-level knowledge of the context of mobile device (e.g., cell phone) users from the sensor data of their mobile devices is considered. High-level context information may be useful for businesses to, e.g., create targeted advertisements or develop marketing strategies tailored to end users. For example, mobile device sensor data may be polled and processed to provide high-level interpretations such as “who will be the next president of the United States,” which may be useful to businesses who perform voter surveys, or “how many people are running in the park,” which may be useful for healthcare companies or athletic shoe companies. However, gaining high-level context information from low-level sensor data requires many steps, and many end-users may have little motivation to share their personal information.
A system for the exchange of sensor data and its abstractions includes one or more service hubs, including a first service hub. A query input is configured to receive a query that makes requests for sensor data on at least one of the one or more service hubs and receives processed sensor data as a context of at least one user of a mobile device. In response to a request for processed sensor data, the first service hub is configured to request sensor data from one or more mobile devices by sending a request to be scheduled by an application executed on the one or more mobile devices. The first service hub is further configured to process the sensor data and return the processed sensor data to at least one of the query and a higher-level service hub.
A system for the exchange of sensor data and its abstractions includes one or more service hubs, including a first service hub. The one or more service hubs receive sensor data from at least one of a lower-level service hub and a mobile device in exchange for compensation. A query input is configured to receive a query and receive compensation, wherein the query makes requests for sensor data on at least one of the one or more service hubs and receives processed sensor data as a context of at least one user of a mobile device in exchange for compensation. In response to a request for processed sensor data, the first service hub is configured to request sensor data from one or more mobile devices by sending a request to be scheduled by an application executed on the one or more mobile devices. The request is scheduled such that compensation received is maximized and in accordance with user preferences. The first service hub is further configured to process the sensor data and return the processed sensor data to at least one of the query and a higher-level service hub.
A method for the exchange of sensor data and its abstractions includes providing one or more service hubs, including a first service hub. A query is received that makes requests for sensor data on at least one of the one or more service hubs. In response to a request for processed sensor data, the first service hub requests sensor data from one or more mobile devices by sending a request to be scheduled by an application executed on the one or more mobile devices. The first service hub processes the sensor data on the first service hub and returns the processed sensor data to at least one of the query and a higher-level service hub. Processed sensor data is received to the query as a context of at least one user of a mobile device.
These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.
The disclosure will provide details in the following description of preferred embodiments with reference to the following figures wherein:
In accordance with the present principles, systems and methods are provided for a marketplace for buying and selling information gleaned from mobile device sensors. In one embodiment, interested parties construct a query in the marketplace to determine high-level information of mobile device users. Queries can be broken down into sub-queries that are serviced by service nodes. For example, an interested party can construct the query, “who is running in the park?” which can be broken down into the sub-queries “who is running?” and “who is in the park?” Service nodes will retrieve and process the information from other processing nodes or the mobile device sensors directly, adding value to the data at each node. Advantageously, in this manner, the present principles can poll for specific information that relates to a particular type of sensor data or complies with a constraint (i.e., no more than 5 minutes old) to provide a specific amount of data, as opposed to collecting a fixed amount of data and uploading it to a database at regular intervals. Queries can then combine information obtained from different service nodes to provide an answer to the query.
Advantageously, in a preferred embodiment, the marketplace includes a system of compensation (e.g., money). Some form of compensation is injected by the interested party upon execution of the query, and is used by service nodes to purchase processed information from other sensor nodes or sensor data directly from the mobile device. In this way, mobile device users are compensated for their personal information. The marketplace may, in one embodiment, autonomously sell raw sensor data in order to receive as much compensation for the user as possible, within the constraints of the user preferences.
In yet another embodiment, third-parties may enter and compete in the marketplace to provide service nodes with specialized analysis technology. These third parties add value to data coming from mobile device sensors or other service nodes. Third parties may advertise their services, accept orders, pay for data, and sell their processed data.
Embodiments described herein may be entirely hardware, entirely software or including both hardware and software elements. In a preferred embodiment, the present invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.
Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. A computer-usable or computer readable medium may include any apparatus that stores, communicates, propagates, or transports the program for use by or in connection with the instruction execution system, apparatus, or device. The medium can be magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. The medium may include a computer-readable storage medium such as a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk, etc.
A data processing system suitable for storing and/or executing program code may include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code to reduce the number of times code is retrieved from bulk storage during execution. Input/output or I/O devices (including but not limited to keyboards, displays, pointing devices, etc.) may be coupled to the system either directly or through intervening I/O controllers.
Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.
Referring now to the drawings in which like numerals represent the same or similar elements and initially to
It is to be noted that the present principles are not limited to mobile devices, but may in fact be applied to any type of device to process its context for any type of use. For example, the present principles may be applied to a television device to poll and process its sensors, which may include a microphone or camera, to obtain feedback as to the ratings of a particular advertisement or television show. Other applications are also contemplated.
Referring back to
High-level queries can be broken down into lower-level sub-queries. For example, the query “which mobile device user is running in the park?” can be broken down into the sub-queries: “which mobile device users are running?” and “which mobile device users are in the park?” The high-level query connects to middleware service nodes of marketplace 120 to break down the high-level query and make requests on the middleware service nodes for the sub-queries.
The middleware service nodes of marketplace 120 may make requests on other service nodes in marketplace 120, which ultimately make different requests for data 150 on sensors 116 of mobile device 110. Sensors 116 may include, for example, but are not limited to, a Global Positioning Systems (GPS), Bluetooth™, accelerometers, cameras, microphones, etc. Users of mobile device 110 who are willing to share their personal data 150 can set a premium, such as an “ask” price, for each type of sensor data that their mobile device 110 provides. The “ask” price can be abstracted in the form of preset user preferences 112 of the user's mobile device 110. The “bid” price is determined by the market place, where businesses compete to obtain sensory data 150 from mobile devices 110. The market price for sensory data is expected to vary based on demand (i.e., requests for data analytics and interpretations from businesses 130) and supply (i.e., the number of users willing to share data 150 of a particular sensor 116 and their “ask” price).
It is noted that in order for a mobile device 110 to provide data 150 from sensors 116, mobile device 110 uses resources 114 to sample the data 150 from sensors 116 and upload it to the marketplace 120. Users of mobile device 110 may set a limit on the use of mobile device 110 resources 114 by configuring user preferences 112. Resources 114 may include, for example, battery, processing components, memory, etc. In the event that there are too many interested parties requesting data 150 from mobile device 110, such that user preferences 112 is overloaded, mobile device 110 schedules the requests for processing in a manner that maximizes compensation 140 for the user.
High-level assessment of contexts requires many levels of processing and abstraction, from the creation and processing of low-level features, such as image processing on pixels and audio filters on audio samples, to high-level semantic methods, such as object recognition and text analysis/search. There are numerous processing services contemplated, including, but not limited to, e.g., facial recognition, object recognition, action detection, location analysis, speech recognition, voice recognition, music recognition, optical character recognition, translation analysis, etc. It would be difficult for one lab or company to build a system covering all areas of expertise. It is contemplated that third parties with special analysis technology enter the marketplace 120 by creating service nodes to provide middleware processing services to extract data 150 in exchange for compensation 140. These service nodes add value to data 150 from sensors 116 or other service nodes. Third parties may advertise their service, accept orders, pay for data and sell the processed data.
Many of these processing services require a lot of processing power and system resources. While it is contemplated that some of the processing services run on mobile device 110, performance may be low and the impact on battery life, processing and memory (i.e., resources 114) may make some types of analysis prohibitive. It is, therefore, contemplated that processing services be hosted remotely via, e.g., cloud computing. However, it is also contemplated that processing services are performed on mobile device 110.
Referring now to
Businesses 130, as depicted in
Referring now to
Application 305 also handles requests for pricing and sensor data from marketplace 120 through interface 310 to respond to requests for sensor data, retrieve sensor data, deliver results and accept payment. Application 305 displays compensation received from marketplace 120. Application interface 402 includes sensor queues 315 and queue scheduler 320. Sensor queues 315 may be a set of queues to hold and process sensor requests, including queues for slow, regular and express processing. Queue scheduler 320 accepts requests for sensor data from marketplace 120 and places requests in queues 315. In one embodiment, scheduler 320 schedules requests in queues 315 such that the user receives the largest compensation, given the type of request and user preferences 112. It is noted that the use of queues 315 and scheduler 320 is one method to schedule many requests in order to optimize some objective, such as to maximize the financial return for the user. Other methods are also contemplated.
Marketplace 120 includes queries 220, which are constructed by interest parties, such as businesses 130 (of
Service nodes 210 includes service nodes 330 that request sensor data from mobile device 110, and services nodes 335 that request services from other service nodes 335 and 330. Service nodes 335 that use other service nodes: receive requests for pricing and data from higher level service nodes 335 or queries 220; make requests and payments to other service nodes 335 for the results of their analysis; process acquired data to perform analysis; and return the results of the analysis to higher level service nodes 335 or queries 220. Service nodes 330 that request sensor data from mobile device 110: receive requests from service nodes 335 for a description and cost of a “data product” that it can generate; make requests on mobile device 110 in the form of messages 344 for sensor data; process sensor data; and send the product to a higher level service node 335 in exchange for compensation. In one embodiment, requests made onto service nodes 330 and 335 are in the form of a service level agreement indicating that if the information requested is returned within a specified time, the service node will receive compensation.
As noted above, messages 301 represent communication between marketplace 120 and mobile device 110. This may include: a request from service node 330 for the cost and characteristics of sensor information available on mobile device 110; a request from service node 330 for a specific kind of sensor data, accompanied by some form of compensation; and the requested sensor data from queue scheduler 320 of mobile device 110.
Marketplace 120 also includes messages 350 between services in marketplace 120, including service nodes 210 and queries 220. Similar to messages 345 between marketplace 120 and mobile device 110, messages 350 may include: a request from service node 335 or query 220 for the cost and characteristics of information available from the service; a request from service node 335 or query for data, accompanied by some form of compensation; and the requested data from service node 330 or service node 335, which may be processed data pertaining to one or more mobile devices 110.
Payment of the requests may be conditioned on receiving results within a set period of time or a minimum recall. For example, if a request is for an “action classification” on a subset of 1,000 mobile devices in a system, then a minimum recall requirement might be “get me the action classification of at least 500 mobile devices.”
In one embodiment, marketplace 120 includes an isolated datacenter. The datacenter may include a cluster of computers and/or virtual machines. The datacenter connects service nodes 210, which are, e.g., in the cloud, with a secure and trusted network. Information into and out of the secure network may be tightly controlled to protect user privacy. For example, allowed communication may include: 1) between the cloud (i.e., service nodes 210) and the mobile device 110; 2) for passing sensor requests to mobile device 110; 3) for passing personal data from mobile devices 110 to service nodes 210 in the cluster; 4) for passing compensation information to mobile device 110; 5) for the insertion of a service node 210 into the marketplace 120; 6) for the insertion of queries into marketplace 120, where queries may include query logic, compensation, and an advertising payload; 7) information about the advertisements delivered to the users from the query nodes 220; and 8) information about compensation payments to service nodes 210. For 1), 2), 3) and 4), a common protocol may be declared to ensure only approved communications (e.g., sensor requests and sensor data) only to the intended mobile devices 110. For 5) and 6), information is only permitted to enter the datacenter. For 7) and 8), this information would have to be tightly controlled and monitored to prevent information that may identify a user from being delivered to an advertiser. It is assumed that the datacenter is a trusted network and that data coming out of it does not hold any information that might identify a user.
It is further contemplated that queries 220 and service nodes 210 written by third parties would have to be submitted, in the form of code, to the marketplace 120. Marketplace 120 may require compliance with a licensing agreement to protect users' rights. This code may be reviewed before it is installed and executed on the system.
Referring now to
For example, in
It should be noted that a mobile device 110 with limited resources 114 will attempt to accommodate requests for sensor data 150 with higher compensation 140 before other requests. Therefore, if a query requires high recall (i.e., results from every phone) in a short time, then the budget for that query should also be high. In the event that the number of services 210 in the marketplace 120 grows and the number of real-time queries 220 grows, the demand for mobile device 110 data 150 will translate into higher compensation cost and, thus, efficiency in the marketplace leading to fewer direct demands on the sensor data will be a key factor for many businesses or other interested parties. If a service 210 is able to make an intelligent assessment of context from cheaply available data 150 before it commits to purchasing more expensive data on a case-by-case basis, then it can reduce its own running costs and improve the overall efficiency of the marketplace.
Caching services that specialize in storing historical data 150 may also be important. Service nodes 210 may be interested in a piece of sensor data for a particular mobile device 110 that is, e.g., “no more than 5 minutes old.” The service node 210 may ask a caching service for the sensor data instead of asking the mobile device 110 directly. If the caching service does not have the data, it can retrieve it from the mobile device 110 for some compensation 140, send it to the requesting service 210, and also store it in a database. Future requests for the same data 150 from the same mobile device 110 (e.g., requests for data no more than 10 minutes old) can be served from the database at no additional cost to the caching service. The caching service would be able to offer the data at a lower price to many services 210 than the price paid and still make a profit.
Referring now to
In block 520, each query or sub-query makes requests on one or more service nodes. In one embodiment, the service nodes are created by third-parties who enter and compete in the marketplace by providing special analysis technology. The third-parties may advertise their service, accept orders, pay for data and sell processed data. In another embodiment, the service nodes may include a caching service that specializes in storing historical data. The caching service may provide historical data to a requesting service node at a lower cost than what it paid.
In block 525, if applicable, a service node may request processed sensor data from one or more lower-level service nodes. Alternatively, in block 530, a service node may request raw sensor data from a sensor of a mobile device. Requests may be in the form of a service level agreement indicating that if data can be delivered within the specified time limit, the service node or mobile device will be compensated. In one embodiment, a query or service node will request data from a lower-level service node or the mobile device directly by first requesting cheaper data and assessing context relevance of mobile devices before requesting more expensive data. In this manner, a service node can limit the number of expensive requests.
Requests for raw sensor data of a mobile device, in block 530, are made to an application executed on the mobile device, in block 535. In one embodiment, the application schedules the requests in queues, which may include slow, regular and express queues. Other methods are also contemplated. In yet another embodiment, the requests are scheduled in way to maximize compensation for the user, while conforming to the constraints of the user preference and resources of the mobile device.
In block 540, sensor data is returned from the mobile device or a lower-level service node to a high-level service node up to the query in exchange for compensation. Compensation for raw sensor data is set by the user in the user preferences of the mobile device. In block 545, the query combines the results of the two or more sub-queries to provide high-level context information satisfying the query.
Having described preferred embodiments of systems and methods of a marketplace for sensor data from mobile devices and its abstractions (which are intended to be illustrative and not limiting), it is noted that modifications and variations can be made by persons skilled in the art in light of the above teachings. It is therefore to be understood that changes may be made in the particular embodiments disclosed which are within the scope of the invention as outlined by the appended claims. Having thus described aspects of the invention, with the details and particularity required by the patent laws, what is claimed and desired protected by Letters Patent is set forth in the appended claims.
This application claims priority to provisional application Ser. No. 61/444,350 filed Feb. 18, 2011 and provisional application Ser. No. 61/474,792 filed Apr. 13, 2011, both incorporated herein by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 61444350 | Feb 2011 | US | |
| 61474792 | Apr 2011 | US |
