Patent Application
20160321543
The technical field of predictive analytics includes the application of computational techniques, such as machine learning and data mining, to typically large data sets. Previously unknown patterns, such as similarities, differences, and relationships between different elements of a data set, can be discovered by these computational techniques. Predictive algorithms, such as statistical methods, can be used to identify trends and predict future outcomes based on patterns that are found in the data set. Many predictive analytics techniques are model-based. For example, a mathematical model may be constructed that represents relationships between elements in a data set and conclusions about those data elements. The model can be “trained” using a “trainer” data set for which those relationships are already known.
Predictive analytics can be provided as a service running on a computer network. Face detection (e.g., the ability to, using a computer, identify a human face in a digital photograph) is an application of predictive analytics that can be provided as a service to multiple different types of computing devices and applications, over a network. In automated face detection, the trainer data set may include a very large number of sample photographs of human faces coupled with descriptive tags indicating features shown in the photographs. The trainer data is used to train the model, for instance to establish probabilities that certain combinations of features shown in the photographs are representative of certain types of people (e.g., male, female, young, old). The model and the trainer data can then be used to classify new input photographs (e.g., photographs that are not already in the existing data set).
The concepts described herein are illustrated by way of example and not by way of limitation in the accompanying figures. For simplicity and clarity of illustration, elements illustrated in the figures are not necessarily drawn to scale. Where considered appropriate, reference labels have been repeated among the figures to indicate corresponding or analogous elements.
While the concepts of the present disclosure are susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and will be described herein in detail. It should be understood, however, that there is no intent to limit the concepts of the present disclosure to the particular forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives consistent with the present disclosure and the appended claims.
References in the specification to “one embodiment,” “an embodiment,” “an illustrative embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may or may not necessarily include that particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described. Additionally, it should be appreciated that items included in a list in the form of “at least one A, B, and C” can mean (A); (B); (C); (A and B); (B and C); (A and C); or (A, B, and C). Similarly, items listed in the form of “at least one of A, B, or C” can mean (A); (B); (C): (A and B); (B and C); (A and C); or (A, B, and C).
The disclosed embodiments may be implemented, in some cases, in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on a transitory or non-transitory machine-readable (e.g., computer-readable) storage medium, which may be read and executed by one or more processors. A machine-readable storage medium may be embodied as any storage device, mechanism, or other physical structure for storing or transmitting information in a form readable by a machine (e.g., a volatile or non-volatile memory, a media disc, or other media device).
In the drawings, some structural or method features may be shown in specific arrangements and/or orderings. However, it should be appreciated that such specific arrangements and/or orderings may not be required. Rather, in some embodiments, such features may be arranged in a different manner and/or order than shown in the illustrative figures. Additionally, the inclusion of a structural or method feature in a particular figure is not meant to imply that such feature is required in all embodiments and, in some embodiments, may not be included or may be combined with other features.
Predictive analytics services can introduce vulnerabilities that raise user privacy and/or intellectual property rights management issues. For instance, when an end user's digital photograph is uploaded to a network, a face recognition service may have access to private information associated with the photograph. At the same time, proprietary information about the model and/or training data used by the face recognition service may be exposed to the client device. Further, many current predictive analytics products are installed as monolithic, vertical software stacks in which the predictive models are tightly coupled with the training data. This can result in the user needing to install many different vertical software stacks to solve different types of predictive analytics problems, or even to solve the same problem in different ways. The situation is exacerbated by the proliferation of many different types of networked computing devices (e.g., smartphones, tablets, wearable devices, laptops, desktops, etc.), each of which may require a different version of the model description, or may require a different version of a model that is optimized for a particular platform.
Referring now to
In an example scenario, the user-level application 118 may be a camera application, a photo uploading service, or a front end to a social media service such as FACEBOOK or PINTEREST. The user computing device 110 is configured with a trust execution subsystem 120. The trust execution subsystem 120 may be embodied as a hardware- or software-implemented Trusted Platform Module (TPM) or using a TrustZone by ARM, for example. When a user-level application 118 requests access to the predictive analytics service (e.g., “detector”) 194, the trust execution subsystem 120 launches the trusted predictive analytics middleware 166 in a trusted execution environment. By running in a trusted execution environment and instantiating an executable trusted predictive analytics service (e.g., “detector”) 194 that is based on a model description that is created using the model description language 160, the trusted predictive analytics middleware 166 provides a common trusted execution environment in which the predictive analytics service (e.g., “detector”) 194 can be executed and sensitive data can be isolated. The illustrative predictive analytics service (e.g., “detector”) 194 provides a data analytics service, such as a machine learning-based data analysis service or a “big data” analytics service. Some example implementations of the predictive analytics service (e.g., “detector”) 194 are shown in
Using the model description language 160, the trusted middleware 166 avoids the need to have multiple different predictive analytics services (e.g., “detectors”) 194 running on the middleware platform. Instead, the middleware 166 replaces the different predictive analytics services (e.g., “detectors”) 194 with different instances of a predictive analytics service (e.g., “detector”) 194, where each of the instances is created from an execution primitive whose operation is supported by the trusted middleware 166. The trusted middleware 166 decrypts and instantiates a predictive analytics service (e.g., “detector”) 194 using a model description 162. The model description 162 is created by the middleware 166 using the model description language 160. Digital rights management functionality of the trusted middleware 166 enforces license agreement terms and restrictions on the use of the predictive analytics service (e.g., “detector”) 194. User data (e.g., user content 124) is protected because it is accessed by the middleware primitive (e.g., the predictive analytics service or “detector” 194 instantiated using the model description 162) inside the trust execution environment and thus, the user data is not directly exposed to the predictive analytics service (e.g., “detector”) 194.
Referring now in more detail to
The user computing device 110 also includes memory 114, an input/output subsystem 116, a data storage device 122, a camera 132, one or more sensors 134, a user interface (UI) subsystem 136, and a communication subsystem 140. The user computing device 110 may include other or additional components, such as those commonly found in a mobile and/or stationary computer, in other embodiments. Additionally, in some embodiments, one or more of the illustrative components may be incorporated in, or otherwise form a portion of, another component. Each of the components of the user computing device 110 may be embodied as software, firmware, hardware, or a combination of software, firmware, and/or hardware.
The processor 112 may be embodied as any type of processor capable of performing the functions described herein. For example, the processor 112 may be embodied as a multi-core processor or other multiple-CPU processor or processing/controlling circuit. The memory 114 of the user computing device 110 may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. In operation, the memory 114 may store various data and software used during operation of the user computing device 110, as well as operating systems, applications, programs, libraries, and drivers.
The memory 114 is communicatively coupled to the processor 112, e.g., via the I/O subsystem 116. The I/O subsystem 116 may be embodied as circuitry and/or components to facilitate input/output operations with the processor 112, the memory 114, and other components of the user computing device 110. For example, the I/O subsystem 116 may be embodied as, or otherwise include, memory controller hubs, input/output control hubs, firmware devices, communication links (i.e., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.) and/or other components and subsystems to facilitate the input/output operations. In some embodiments, the I/O subsystem 116 may form a portion of a system-on-a-chip (SoC) and be incorporated, along with the processor 112, the memory 114, and/or other components of the user computing device 110, on a single integrated circuit chip.
The data storage device 122 may be embodied as any type of physical device or devices configured for short-term or long-term storage of data such as, for example, memory devices and circuits, memory cards, hard disk drives, solid-state drives, flash memory or other read-only memory, memory devices that are combinations of read-only memory and random access memory, or other data storage devices. User content 124 (e.g., digital content, such as photographs, videos, music files, and documents) and detector models 190 are stored in the data storage device 122. Portions of the user content 124 and/or the detector models 190 may be copied to the memory 114 from time to time during operation of the computing device 110, e.g., for faster processing.
The camera 132 may be embodied as any type of camera capable of performing the functions described herein, e.g., capturing still and/or video images using camera hardware, software, or a combination of hardware and software. The sensor(s) 134 may be embodied as any suitable type of sensor capable of performing the functions described herein, including one or more of motion sensors, proximity sensors, location sensors, and eye tracking devices.
The user interface subsystem 136 may include a number of additional devices to facilitate user interaction with the user computing device 110, including physical or virtual control buttons or keys, a microphone, a speaker, a display device, and/or others. For example, a display device may be embodied as any type of display capable of displaying digital information such as a liquid crystal display (LCD), a light emitting diode (LED), a plasma display, a cathode ray tube (CRT), or other type of display device. In some embodiments, the display device may be coupled to a touch screen or other human computer interface device to allow user interaction with the user computing device 110. The user interface subsystem 136 may also include other devices, such as motion sensors, proximity sensors, and eye tracking devices, which may be configured to detect, capture, and process various other forms of human interactions involving the user computing device 110.
The user computing device 110 further includes a communication subsystem 140, which may be embodied as any communication circuit, device, or collection thereof, capable of enabling communications between the user computing device 110 and other electronic devices. The communication subsystem 140 may be configured to use any one or more communication technology (e.g., wireless, optical, or wired communications) and associated protocols (e.g., Ethernet, Bluetooth®, Wi-Fi®, WiMAX, 3G/LTE, etc.) to effect such communication. The communication subsystem 140 may be embodied as a network adapter, including a wireless network adapter.
The illustrative user computing device 110 also includes a number of computer program components, such as the user-level application 118, the trust execution subsystem 120, and one or more detector models 190, described below. The user-level application 118 may be embodied as any computer application (e.g., software, firmware, hardware, or a combination thereof) that interacts directly or indirectly with an end user via, for example, a display device or another component of the UI subsystem 136. Some examples of user-level applications 118 include word processing programs, document viewers/readers, web browsers, electronic mail programs, messaging services, social media services, content sharing services, computer games, camera and video applications, etc. While not specifically shown, the user computing device 110 includes a privileged system component that facilitates communication between the user-level applications 118 and the hardware components of the user computing device 110. Portions of the privileged system component 142 may be embodied as any operating system capable of performing the functions described herein, such as a version of WINDOWS by Microsoft Corporation, ANDROID by Google, Inc., and/or others. Alternatively or in addition, portion of the privileged system component 142 may be embodied as any type of virtual machine monitor capable of performing the functions described herein (e.g., a type I or type II hypervisor).
The trusted predictive analytics middleware computing device 150 and the predictive analytics provider computing device 180 may each be embodied as any type of electronic device for performing the functions described herein. For example, the computing devices 150, 180 may be embodied as, without limitation, a server computer, a workstation, a distributed computing system, a multiprocessor system, a consumer electronic device, a smart phone, a tablet computer, a wearable computing device, a laptop computer, a notebook computer, a mobile computing device, a cellular telephone, a handset, a messaging device, a vehicle telematics device, and/or any other computing device configured to perform the functions described herein. As shown in
As noted above, the trusted middleware 166 includes a model description language 160 and a trusted predictive analytics middleware subsystem 164. The model description language 160 includes a model specification and parameter information. The model description language 160 is interpreted using components of the middleware subsystem 164, as described in more detail below. Referring now to
A highly optimized PA execution primitive 256 is implemented on top of one or more platform-specific execution backends 260 (where the execution backends 260 may include: a specific implementation based on a platform-specific instruction set (e.g., the Intel AVX instruction set), an implementation based on a specific graphics processing unit (GPU), or an implementation based on platform-specific acceleration technology (e.g., a floating point gate array (FPGA) in a customized processor (e.g., the Xeon processor by Intel). The trust management subsystem 220 is responsible for monitoring the execution of the framework 200 in a trusted way. The trust management subsystem 220 can prevent the modification of framework 200, to ensure that the framework's behavior acts as intended and cannot be modified by a malfunctioning process running on the same system. The trust management subsystem 220 can protect the access of privacy data and prevent the access of sensitive data, like decrypted modeling descriptions, a PA task's code page in memory, etc.
The DRM subsystem 222 is responsible for managing digital rights access and protection of the model 190's owner. The illustrative DRM subsystem 222 can play two roles. The first role is to manage the key and cryptographic operations as defined by the DRM protocol. The second role is to control the framework 200 to permit access to the content of the model 190 only in the way licensed by the model owners, for example, at fixed times, on a fixed size of input data, etc. The DRM subsystem 222 operates to ensure that framework 200 protects the model owner's digital rights in the model 190. Each of the components 218, 220, 222 may be embodied as computer hardware, firmware, software, or a combination thereof.
Referring to
The model description assembler module 252 takes as input a detector model 190 of a predictive analytics service (e.g., “detector”) 194 (e.g., M1, M2, M3, or M4 described below), interprets the detector model 190 using the model description language 160, and thereby generates a model description 162 of the detector model 190 in the model description language 160. In some embodiments, the generation of the model description 162 (e.g., the conversion of the detector 194 to the model description 162 is performed as a one-time event, which may be performed on the predictive analytics provider computing device 180. Once the model description 162 is created (e.g., on the provider computing device 180), the detector 194 can be deployed to the trusted PA middleware computing device 150 or to another computing device (e.g., by the trusted predictive analytics middleware subsystem 164). As a result of the model interpretation, the assembler module 252 obtains information about the structure of the detector model 190 and the associated parameters. With this information, the assembler module 252 creates an executable instance of the predictive analytics service (e.g., “detector”) 194 based on the model description 162 and using the predictive analytics execution primitive 256, which is supplied by the middleware 166. To do this, the assembler module 252 maps the node of the model structure to the predictive analytics execution primitive 256 (e.g., by code logic) and maps execution variables to model parameters embedded in the model description 162. The assembler module 252 may apply one or more optimization algorithms (e.g., to remove any redundant operations). In this way, the assembler module 252 creates a “trusted” version of the predictive analytics service (e.g., “detector”) 194 that has a common execution primitive with other predictive analytics services and specifies the model description 162 using the common description language 160.
The model description 162 provides a minimal set of information that can be used to re-build an executable detector, assuming the detector will run on the middleware. The model description 162 includes model structures, model parameters, and model meta information about the model. References to “model structure” herein may refer to, among other things, a graph or tree structure, which is commonly used to represent analytical models, as well as nodes and the network structure (e.g., arrangement of nodes and edges). As used herein, “node” may refer to, among other things, a primitive connection. Connections and network structure determine a composition rule, which establishes the control flow through the model network. Some examples of model structures include acyclic graphs, probabilistic graphical models, Bayesian networks, multi-layer network structures, and/or others. Model parameters provide coefficients, such as primitive coefficients, connection coefficients, and network coefficients. Meta information provides information about the detector model (e.g., “comments”). For example, the meta information may indicate the kind of problem or application for which the detector is suitable, the inputs required by the detector, and information about how the detector is trained. Meta information can typically be released publicly, e.g., by a directory service. Meta information enables the user-level application 118 to run queries using a search engine to find detectors that are suitable for particular tasks (e.g., detectors whose meta information matches business requirements specified by the user-level application 118). Some examples of model meta information include trained data size (e.g., the size of a training data set) and input format (e.g., formatting requirements for input data, if any).
Once the assembler module 252 creates an optimized executable of the predictive analytics service (e.g., “detector”) 194, the executable is submitted to the detector scheduler module 254. The detector scheduler module 254 schedules and executes detectors that are instantiated by the assembler module 252, and interacts with the resource management module 258, the trust execution monitor module 262, and the DRM manager module 264. The scheduler module 254 handles data distribution and identifies and eliminates redundant data copy if possible. The scheduler module 254 manages the life-cycle of an instantiated detector and frees the resource when the detector finishes execution.
The predictive analytics execution primitive 256 is an executable that performs a common predictive analytics task in an efficient way. As such, the predictive analytics execution primitive 256 can form the basis of many types of detectors. The resource management module 258 manages resources such as data storage and memory allocation. The trust execution monitor module 262 and the DRM manager module 264 expose the standard trust execution and DRM management components to the layers above (e.g., the scheduler module 254 and/or management interface 214). The platform-specific execution backend 260 enables the middleware 166 to interface with platform-specific capabilities across many different types of devices and computing platforms.
In more detail, the trust management subsystem 220 (
Referring now in more detail to the DRM subsystem 222 (
Referring now to
With the detector (e.g., detector 310, 322, 342, 352) integrated in the manner shown in
The trusted middleware 166 addresses the issues raised by the traditional implementations shown in
Referring now to
If the computing system 100 determines in block 418 that the user request is permitted, the computing system 100 decrypts the model description selected in block 414 (using, e.g., a component of the trust execution environment). In block 424, the computing system 100 stores the decrypted model description in a trusted memory region (e.g., a hardware or software isolated region of the user computing device 110). In block 426, the computing system 100 instantiates the decrypted model. In block 428, the computing system 100 executes the decrypted model to process the input data, and in block 430, the computing system 100 outputs the results of executing the model on the input data.
Referring now to
In block 514, the computing system 100 creates the model structure (e.g., nodes, network, etc.), based on the model description prepared in block 512. In block 516, the computing system 100 converts the model structure to an executable, using the execution primitive created or selected in block 510. In block 518, the computing system 100 applies one or more optimizations to the executable created in block 516, as needed. In block 520, the computing system 100 submits the executable to a schedule module of the trusted middleware (e.g., the detector scheduler module 254).
Illustrative examples of the technologies disclosed herein are provided below. An embodiment of the technologies may include any one or more, and any combination of, the examples described below.
An Example 1 includes a computing system including one or more computing devices, the computing system to provide a trusted predictive analytics service, the computing system including: a trusted predictive analytics middleware subsystem to, in response to a user-level application request for a predictive analytics service, in a trust execution environment of the computing system, cause the computing system to: determine a model description for a predictive analytics model, the model description created with a predictive analytics model description language, wherein the predictive analytics model description language is to describe a plurality of different predictive analytics models using a common language; compare data associated with the user-level application request with data indicative of digital rights permissions associated with the model description; and if, based on the comparison of the data associated with the user-level application request with data indicative of digital rights permissions associated with the model description, the user-level application request is permitted, instantiate an executable associated with the model description.
Example 2 includes the subject matter of Example 1, wherein the predictive analytics model description language includes data indicative of a predictive analytics model structure, one or more model parameters, and meta information about a predictive analytics model.
Example 3 includes the subject matter of Example 1 or Example 2, wherein the trusted predictive analytics middleware subsystem is launched, by a user computing device of the computing system, in the trust execution environment.
Example 4 includes the subject matter of Example 1 or Example 2, wherein the trusted predictive analytics middleware subsystem is obtain input data associated with the user-level application request for a predictive analytics service, and store the input data in a trusted memory region of the trust execution environment.
Example 5 includes the subject matter of Example 1 or Example 2, wherein the trusted predictive analytics middleware subsystem is to decrypt the model description if, based on the comparison of the data associated with the user-level application request with data indicative of digital rights permissions associated with the model description, the user-level application request is permitted.
Example 6 includes the subject matter of Example 5, wherein the trusted predictive analytics middleware subsystem is to store the decrypted model description in a trusted memory region of the trust execution environment.
Example 7 includes the subject matter of Example 1 or Example 2, wherein the trusted predictive analytics middleware subsystem is to create the executable based on a predictive analytics execution primitive and the model description.
Example 8 includes the subject matter of Example 1 or Example 2, wherein the trusted predictive analytics middleware subsystem includes a digital rights management (DRM) subsystem to verify digital rights associated with the predictive analytics service.
Example 9 includes the subject matter of Example 1 or Example 2, wherein the trusted predictive analytics middleware subsystem includes a model description assembler module to interpret the model description using the model description language, create a model structure for the predictive analytics service based on the model description, and convert the model structure to an executable based on a predictive analytics execution primitive.
An Example 10 includes a method for providing a trusted predictive analytics service, the method including, with a computing system: describing a plurality of different predictive analytics models using a common model description language; and in response to a user-level application request for a predictive analytics service, in a trust execution environment of the computing system: determining a model description for a predictive analytics model, the model description created with the predictive analytics model description language; comparing data associated with the user-level application request with data indicative of digital rights permissions associated with the model description; and if, based on the comparison of the data associated with the user-level application request with data indicative of digital rights permissions associated with the model description, the user-level application request is permitted, instantiating an executable associated with the model description.
Example 11 includes the subject matter of Example 10, including launching the trusted predictive analytics middleware subsystem in the trust execution environment.
Example 12 includes the subject matter of Example 10, including obtaining input data associated with the user-level application request for a predictive analytics service, and storing the input data in a trusted memory region of the trust execution environment.
Example 13 includes the subject matter of Example 10, including decrypting the model description if, based on the comparison of the data associated with the user-level application request with data indicative of digital rights permissions associated with the model description, the user-level application request is permitted.
Example 14 includes the subject matter of Example 13, including storing the decrypted model description in a trusted memory region of the trust execution environment.
Example 15 includes the subject matter of Example 10, including creating the executable based on a predictive analytics execution primitive and the model description.
Example 16 includes the subject matter of Example 10, including, by a digital rights management (DRM) subsystem, verifying digital rights associated with the predictive analytics service.
Example 17 includes the subject matter of Example 10, including interpreting the model description using the model description language, creating a model structure for the predictive analytics service based on the model description, and converting the model structure to an executable based on a predictive analytics execution primitive.
Example 18 includes the subject matter of Example 10, including describing the predictive analytics models using a description language that includes data indicative of a predictive analytics model structure, one or more model parameters, and meta information about a predictive analytics model.
An Example 19 includes or more non-transitory machine readable storage media including a plurality of instructions stored thereon that, in response to being executed, cause a computing device to perform the method of any of Examples 10-18.
An Example 20 includes a computing system for providing trusted predictive analytics services, the system including means for performing the method of any of claims 10-18.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2013 10754861.4 | Dec 2013 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2014/093660 | 12/11/2014 | WO | 00 |
