Patent Grant
9609020
Field
The present invention is related to mobile device security. More particularly, the present invention is in the technical field of distributed policy enforcement in mobile communications devices.
Description of the Related Art
Software and data related security of current devices, especially mobile devices, rely on a variety of features including virtual machines, inter-process communication, package managers, mobile device management systems, touch screen software components, shared memory, relational databases, device configuration signature checking, specialized debugging interfaces (e.g. Android Debug Bridge, and the like), trusted daemon processes, and the like. In an example, Android mobile devices use checks on inter-process communication to determine if an application should gain access to a particular system resource, such as the user's contact list. Virtual machine security checks, such as determining whether or not a specific native library should be loaded, are also employed.
A key challenge with mobile devices is that it is difficult to ensure the integrity of the very software that is relied upon to provide security (e.g. virtual machine, device I/O, inter-process communication, specialized debugging interfaces, and other components). A need exists for systems that provide security, such as by enforcing data provenance, protecting against malware and enforcing security policies via inter-process communications mechanisms.
In embodiments, method and systems of filtering access to a resource having one or more inter-process control paths between two or more objects that are controlled by one or more object firewalls in the device may use a processor to control access to the resource from an application based on a policy. Controlling access to the resource from an application based on a policy may comprise providing the policy and filtering access through the inter-process control path to the object providing access to the resource.
In embodiments, methods and systems may further comprise using a computer processor to generate a device-based context based on one or more of the current date and time, the current location of the device, the identity of a current user of the device, the identity of each application currently executing on the device, and the identity of each resource currently in use.
In embodiments, controlling access to the device resource from an object associated with an application based upon a policy may comprise providing a context-aware policy engine to control access to the resource. In embodiments, the context-aware policy may be enabled to communicate with a policy server to communicate one or more policies.
In embodiments, the policy governing the inter-process communications to the object may be a context-related policy.
In embodiments, the resource may be one of a network connection, a cellular connection, a keyboard, a touch interface, an operating system, an application, a part of an application programming interface, a software driver, a database, a port, a wireless communication interface, and a secured area in memory.
In embodiments, the object firewall may record resource access attempts.
In embodiments, the object firewall may be stored in a centralized object registry.
In embodiments, the objects providing access to the resource are stored in a centralized registry.
In embodiments, the policies may be authored using a graphical user interface, which may be stored in a first format, translated into a second format for transmission to a device, and may be parsed by a receiving device in order to determine how to configure one or more object firewalls.
In embodiments, a computer processor may be used to control and configure the object firewalls in a single process associated with a device security system.
In embodiments, methods and systems of enforcing distributed policies in a mobile network may comprise at least one processor adapted to provide an inter-process communications firewall on a device to enforce one or more rules governing communication between at least two systems that may communicate via the device, wherein the at least one processor is adapted to generate, via a policy engine associated with the inter-process communications firewall, an indicator of the context of at least one of the systems that communicate via the device, and said at least one processor adapted to determine whether a communication between the at least two systems is permitted by the inter-process communications firewall based on a policy and the context of at least one of the systems.
In embodiments, the at least one processor may be adapted to provide a plurality of inter-process communications firewalls on a device. In embodiments, said firewalls may be provided to enforce one or more rules governing communication between at least two systems that may communicate via the device.
In embodiments, the at least one processor may be adapted to generate the policy engine wherein the policy engine is enabled to communicate with a policy server to communicate one or more policies.
In embodiments, the policy may be comprised of one or more of a black list, a white list a signing policy, a naming policy, a checksum analysis policy, a library analysis policy, and a permission for one or more of an application, a process, a user, and a group of users.
In embodiments, the inter-process communications firewall may be an object-oriented firewall. In embodiments, the object-oriented firewall may be associated with an application executing on the device.
In embodiments, the at least one processor adapted to determine whether a communication between the at least two systems is permitted by the inter-process communications firewall may be adapted to determine, based on the content of the communication, whether a communication between the at least two systems is permitted by the inter-process communications firewall.
In embodiments, methods and systems of securing a mobile device from malware may comprise using a computer processor to pass a remote procedure call from a first application to an inter-process control data bus, requesting, from a policy engine by the inter-process control data bus, a policy validation for the remote procedure call, determining by the policy engine whether to approve the remote procedure call based on the context of the remote procedure call and a stored policy, communicating the determination from the policy engine back to the inter-process control data bus, and at least one of permitting and blocking the remote procedure call by the inter-process control data bus in response to the determination.
In embodiments, passing the remote procedure call may comprise passing a remote procedure call for a second object associated with a second application from first object associated with a first application to an inter-process control data bus.
In embodiments, the policy engine may be enabled to communicate with a policy server to communicate one or more policies.
In embodiments, the methods and systems of securing a mobile device from malware may comprise providing an inter-process controller for installing new object firewalls as new objects are created.
In embodiments, the methods and systems of securing a mobile device from malware may comprise at least one of permitting and blocking the remote procedure call by the inter-process control data bus in response to the determination, wherein the at least one of permitting and blocking the remote procedure call by the inter process control data bus in response to the determination may further comprise at least one of permitting the remote procedure call without modification, permitting the remote procedure call with modified contents of the remote procedure call, permitting the remote procedure with a modified return value of the data sent from a resource in response to the remote procedure call, blocking the remote procedure call, logging the remote procedure call, ignoring the remote procedure call, modifying one or more firewall rules, adding one or more firewall policies, and removing one or more firewall policies.
In embodiments, the methods and systems of securing a mobile device from malware may comprise a context-aware policy engine. In embodiments, the context-aware policy engine may be further enabled to generate a system-specific context, said system-specific context comprising one or more of the current date, the current time, the location of the mobile device, the identity of the device user, and applications currently executing on the mobile device.
In embodiments, methods and systems may include enforcing policies associated with the loading, linking and/or execution of native code by an application. In embodiments, the method may comprise taking an application executing, through the use of a computer processor, in a first process on a device, providing a policy engine executing, through the use of a computer processor, in a second process on the device, taking a request for a native code library by the application, determining by the policy engine whether the request from the application may be allowed based on a policy, and permitting access to the native code library upon determining that the application is allowed to access the native library.
In embodiments, the methods and systems may comprise facilitating interaction between the application and an operating system via the native code library.
In embodiments, the methods and systems may comprise providing a policy engine enabled to communicate with a remote policy server to obtain the policy. In embodiments, the remote policy server may comprise a policy server at least one policy.
In embodiments, the application may be an application running inside of a virtual machine.
In embodiments, the methods and systems may comprise taking requests from a plurality of applications.
In embodiments, the device may be one of a mobile phone, a tablet, a laptop and a smartphone.
In embodiments, the policy may comprise one or more of a black list, a white list, a signature, a name check, a checksum, a library analysis check, a check for permission for an application, a process check, a user check, and a group check.
In embodiments, requesting a native code library by the application may comprise communicating via a local cross-process communication mechanism. In embodiments, the local cross-process communication mechanism may comprise one of an inter-process communication mechanism, a Unix domain socket, and a shared memory.
In embodiments, the application may be selected from the group consisting of one or more of a game, a utility, a phone application, a web browser, a music player, a tool, and an operating system.
These and other systems, methods, objects, features, and advantages of the present invention will be apparent to those skilled in the art from the following detailed description of the preferred embodiment and the drawings. All documents mentioned herein are hereby incorporated in their entirety by reference.
In the drawings, which are not necessarily drawn to scale, like numerals may describe substantially similar components throughout the several views. Like numerals having different letter suffixes may represent different instances of substantially similar components. The drawings illustrate generally, by way of example, but not by way of limitation, a detailed description of certain embodiments discussed in the present document.
Detailed embodiments of the present invention are disclosed herein; however, it is to be understood that the disclosed embodiments are merely exemplary of the invention, which may be embodied in various forms. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present invention in virtually any appropriately detailed structure. Further, the terms and phrases used herein are not intended to be limiting, but rather to provide an understandable description of the invention.
Mobile devices, such as smartphones, tablets and other web-connected devices are proliferating, both for use as business tools and for personal use. Such mobile devices may provide a platform for collecting, storing, processing and communicating data. In many cases, such data may be personal and/or confidential, such as personal contacts, financial information, and business materials.
Consequent to the proliferation of mobile devices, mobile security is an increasing area of concern in the field of mobile computing. Mobile security may be implemented in a variety of ways. As disclosed herein, several ways of providing mobile security may include protecting the data stored and communicated by the mobile devices and controlling the ability of the software on the devices to access other resources.
In embodiments, methods of securing a device may include filtering access to a device or system resource by controlling access based upon a policy, wherein the policy may be applied by a firewall to filter and/or control the inter-process control paths by which messages may be delivered between the objects that control the system resources, based on a policy governing the inter-process communication between the objects. In some embodiments, the device may be a cellular phone, such as an iPhone, a Motorola Droid Razr Maxx, a HTC One X, a Samsung Focus 2, a Samsung Gusto 2, or some other cellular phone. In other embodiments, the device may be a tablet, such as an iPad, an Asus Eee Pad Transformer Prime, a Sony Tablet S, a Samsung Galaxy Tab 10.1, or some other tablet. The device resource may be a network connection, a cellular connection, a keyboard, a touch interface, an operating system, an application, or some other resource. A system resource may be a software driver, a database, a method of an application programming interface, a port, a wireless communication interface, a secured area in memory or some other resource. The inter-process communication may be provided by any inter-process communication mechanism, such as the Android Binder, Unix Domain Sockets, or shared memory. Prior art, such as Android's permission system for applications, does not provide object firewalls and requires that the receiving object providing access to the system resource enforces its own policies on received inter-process communications.
The policy may state that a request to access a resource should be filtered and/or modified based on one or more criteria. In some embodiments, the policy may state that a request should be filtered based on the source of the request to access the resource. For example, the policy may state that requests to access the resource should be filtered based on the name or type of application making the request. In embodiments, the policy may state that a request should be filtered based on the resource. For example, the policy may state that any requests to use the cellular connection should be filtered. In other embodiments, the policy may state that a request should be filtered based on the requested outcome of or data included in the access. For example, the policy may state that a request to access the network connection to send data to www.google.com should be filtered.
The inter-process control (IPC) path between the objects may be controlled by one or more object-oriented firewalls. In some embodiments, there may be one object firewall per object associated with an application. The object firewalls may implement the policy, for example, by controlling the inter-process communication from one object to a receiving object that provides access to a system resource. In embodiments, controlling access to the resource from an application may be based on a policy, and may comprise filtering access to one or more objects providing access to the resource, wherein said access is through the inter-process control path. Further, in embodiments, said filtering may be based on a policy governing inter-process communications to said one or more objects providing access to the resource. The object firewalls may obtain the policy from the policy engine. The policy may be translated, for example, by a policy engine, into one or more specific settings on a particular object firewall. As new objects are created, the IPC controller may install new object firewalls as needed. The object firewall may respond to a request of a resource in one or more ways, including without limitation, that the object firewall may block the request of the resource, the object firewall may allow the request of the resource, the object firewall may modify the contents of the request, the object firewall may modify the return value of the data sent from the resource, the object firewall may change the resource requested, the object firewall may log the request, the object firewall may ignore the request, the object firewall may change one or more firewall rules, and/or the object firewall may add or remove object firewall rules. In embodiments, the object firewall may record resource access attempts. The object firewalls may be stored in a centralized registry. Similarly, the objects providing access to the device and system resources may also be stored in a centralized registry.
For security purposes, a single process may be associated with the device security system. In some embodiments, this process may be enabled to control and configure the object firewalls.
In embodiments, a secure computing device may include a device-based context-aware policy engine to enforce polices relating to the provenance of data between an application executing on the computing device and another application computing on the computing device. In some embodiments, the computing device may be a portable computing device, such as a laptop, a cellular phone or a tablet. In some embodiments, one of the applications may be a game, such as Angry Birds, Smash Cops, Words with Friends or some other game. In some embodiments, one of the applications may be a utility, such as the phone application, Skype, a web browser, a music player or some other utility. In some embodiments, one of the applications may be a tool, such as Twitter, ESPN ScoreCenter, Google Translate or some other tool. In an embodiment, the second application may be the operating system.
In embodiments, an authoring tool may be provided for authoring one or more policies by a user. The authoring tool may have a browser-based interface. The authoring tool may have a GUI. The authoring tool may be installed on the device and may be used to control and/or create object firewalls on the device. In embodiments, the authoring tool may be installed on a remote system. Policies authored may be stored in one format (e.g. a set of objects and methods stored in a database), translated to a second format for transmission to a device (e.g. XML), and parsed by a receiving device to determine how to configure one or more object firewalls.
The policy engine may generate system-specific context, which may include one or more of the current date and time, the computing device location, the identity of the device user, which applications are executing on the computing device, which applications are consuming which device resources, and other data related to the context in which the system resides. In some embodiments, the policy engine may be connected to a policy server, which may push one or more policies to the policy engine.
In embodiments, the policy engine may control access to a resource. For example, enforcing policies related to the data provenance between the applications may include evaluating, by the policy engine, a call from the first application to the second application. The policy engine may evaluate the call based on one or more policies, and one or more of the system context, application context and the context of the call. The policies may include, for example, system policies, application policies, and other policies. The policy engine may use the one or more policies to evaluate the call, including, without limitation, whether the source of the data is a trusted source, a permitted source, or the like, and/or whether the nature of the data is of a type permitted to be relayed to or used by the second application. In some embodiments, the policy engine may also determine, based on the evaluation of the call, whether any data to be transferred by way of the call is authorized.
For example, a call from one application to a web browser to transfer a secured contact list may be evaluated by the policy engine on mobile phone. The policy engine may include a policy prohibiting the transmission of any data from the contact list. Upon evaluating the call, the policy engine would reject the call and may report a failure to the first application.
In embodiments, methods of securing a computing device by may include providing a device-based context-aware policy engine to enforce a policy relating to data provenance between a first application executing on the computing device and a second application executing on the computing device; reviewing, by the device-based context-aware policy engine, a data transfer from the first application to the second application; and determining, by the device-based context-aware policy engine based on a policy, whether the data transfer is permitted. In some embodiments, the computing device may be a portable computing device, such as a laptop, a cellular phone or a tablet. In some embodiments, one of the applications may be a game, such as Angry Birds, Smash Cops, Words with Friends or some other game. In some embodiments, one of the applications may be a utility, such as the phone application, Skype, a web browser, a music player or some other utility. In some embodiments, one of the applications may be a tool, such as Twitter, ESPN ScoreCenter, Google Translate or some other tool. In an embodiment, the second application may be the operating system.
A device-based context-aware policy engine may be enabled to identify the device's context and state, and may generate a system-specific context. The system-specific context may include one or more of the current date and time, the computing device location, the identity of the device user, the applications currently executing on the device and other context-related data. In some embodiments, the policy engine may be connected to a policy server, which may push one or more policies to the policy engine.
Enforcing data provenance policies between the applications may include evaluating, by the policy engine, a call from the first application to the second application. The policy engine may evaluate the call based on one or more policies, and one or more of the system context, application context and the context of the call. The policies may include, for example, system policies, application policies, and other policies. The policy engine may use the one or more policies to evaluate the call. In some embodiments, the policy engine, may also determine, based on the evaluation of the call, whether any data to be transferred by way of the call is authorized.
Reviewing a data transfer, by the device-based context-aware policy engine, may include generating a context specific to the received remote procedure call. In some embodiments, the context may include the identity of the first application.
Determining whether the data transfer is permitted may include evaluating the data transfer request subject to one or more available policies. The determination may be based on a comparison of the context against a policy. Such policies may include, for example, a system policy, an application policy, a system context-related policy, an application context-related policy, a policy regarding the content of the requested data transfer, or some other policy.
In embodiments, methods of enforcing distributed policies in mobile networks may include providing an inter-process communications firewall on a device to enforce rules governing communication between two systems and/or subsystems; generating, by a policy engine associated with the inter-process communications firewall, a system context; and determining, by the inter-process communications firewall, whether the communication is permitted. In some embodiments, the determination of whether the communication is permitted by the inter-process communications firewall may be based on one or more of a policy, a system context, and/or the content of the communication.
In some embodiments, the distributed policies may include one or more policies such as a black/white list, a signing and/or naming policy, a checksum/library analysis policy, a permission for one or more of an application, a process, a user, a group of users, and other policies. In some embodiments, the policy may be stored on a policy server connected to the mobile network. The policy may also be stored in a policy engine on the device. A black list may identify one or more prohibited actions. For example, an application black list may comprise a list of application IDs for applications prohibited from executing on the device. A white list may identify one or more allowed actions. For example, an application white list may comprise a list of application ids for applications that are permitted to execute on the device.
The inter-process communications firewall may be an object-oriented firewall related to one or more objects in an application. In some embodiments, the inter-process communications firewall may communicate with an IPC controller to control communications between the object related to the inter-process communications firewall and a second object. The second object may be related to a second application.
In some embodiments, generating a system context by the policy engine may include the current date and time, the device location, identity of the device user or some other context.
In embodiments, a secure computing system may include an operating system adapted to secure the system's processes by filtering the processes using inter-process communications (IPC). The computing system may be a mobile device, such as a cellular phone, an MP3 player, a tablet and a laptop. In some embodiments, the device may be a cellular phone, such as an iPhone, a Motorola Droid Razr Maxx, a HTC One X, a Samsung Focus 2, a Samsung Gusto 2, or some other cellular phone. In other embodiments, the device may be a tablet, such as an iPad, an Asus Eee Pad Transformer Prime, a Sony Tablet S, a Samsung Galaxy Tab 10.1, or some other tablet. Examples of operating systems include, but are not limited to, Android, BlackBerry OS, iOS, Symbian OS, Windows Phone and Chrome OS.
The way in which the filtering of the processes using IPC may be implemented may depend on the particular operating system. In some embodiments, the operating system may use a universal resource identifier (URI) instead of the inter-process communications, for example, in iOS.
In embodiments, a secure computing system may include an operating system adapted to secure the computing system's processes by commanding and controlling processes using inter-process communications (IPC). The computing system may be a mobile device, such as a cellular phone, an MP3 player, a tablet and a laptop. Examples of operating systems include, but are not limited to, Android, BlackBerry OS, iOS, Symbian OS, Windows Phone and Chrome OS. The way in which the filtering of the processes using IPC may be implemented may depend on the particular operating system. In some embodiments, the operating system may use URI instead of the inter-process communications, for example, in iOS.
Using the IPC, the command and control processes may be used to securely control functions of the computing system. For example, the IPC may be used to command and control web browsing, phone calls, text messaging and other computing system functions. In other embodiments, using the IPC, the command and control process may be used to filter inter-process communications. For example, the inter-process communications may be filtered according to a rule or policy to prevent a particular class of applications from sending private data. In another example, the inter-process communications may be filtered according to a rule or policy to prevent a particular class of applications from connecting to any computers outside of a defined network.
In embodiments, methods for protecting against malware in a mobile communications device may include passing a remote procedure call from a first application to a data bus; requesting a policy validation for the remote procedure call from the data bus to a policy engine; determining whether to approve the remote procedure call by the policy engine, based on the context of the remote procedure call and a stored policy; communicating the determination from the policy engine back to the data bus; and either permitting or blocking the remote procedure call by the data bus, based on the determination. The data bus may be an inter-process communications bus. Embodiments may have included passing signatures at a file level. Embodiments of the present disclosure may be that passing the procedure call may include passing the process signature between the processes rather than at the file level.
In embodiments, methods for using a policy engine to enforce distributed policies on the loading, linking, and execution of native code may include providing an application running inside of a virtual machine on a mobile device; providing a policy engine running on the mobile device; and adapting the rules for loading, linking, and executing code in native libraries in the virtual machine, in response to an input from the policy engine and based on a policy factor.
In some embodiments, the application may run inside a virtual machine. Examples of virtual machines include, but are not limited to, Java Virtual Machines, Perl virtual machines, an Oracle Virtual Machine, a Parallels virtual machine, a Sun xVM, and a VMware virtual machine.
In some embodiments, methods for allowing security policies to be applied to existing APIs may be through aspect-oriented programming and may be applied to existing APIs without modifying the internal logic of APIs. An existing API may be wrapped with one or more layers of security using aspect-oriented programming methods and techniques.
In embodiments, methods for securing a mobile device may include using an inter-process communication to distribute a policy or other data needed to apply aspect-oriented security to a plurality of processes on a mobile device. Security-related data may be distributed via an inter-process communications mechanism, for example an IPC controller, Android Binder, or Unix Domain Sockets, to one or more target processes. Once such security-related data is distributed, aspect-oriented security techniques may be applied to intercept and manage security related to invocations of methods, functions, and services in the target processes.
In some embodiments, methods for securing a device may include using contextual information to alter how policies are applied to the device and consequently how aspect-oriented security techniques are applied across one or more processes. Such contextual information may include geographic, accelerometer, camera, microphone, wireless network, application usage, user interaction, running processes, disk state, nearby wireless signals/networks, pairing state with external devices, websites being visited, device network traffic, battery level, types of data resident on a device, or other device hardware or software detectable context information. Device context may be either real-world, such as geographic location, virtual, such as data resident on the device, applications currently executing, or input/output of data to/from a network or a disk, or arbitrary combinations of the two. For example, a security policy may be triggered by connection to a specific wireless network, the launch of one or more applications, or the downloading of specific datasets.
In some embodiments, methods for securing a device may include tracking which processes are functioning on the device are covered by some form of aspect-oriented security and/or determining processes that are candidates for aspect-oriented security programming. This tracking may be centralized, distributed, or a hybrid combination of the two.
In embodiments, methods securing a device may include storing aspect-related data that may be stored on the device. In some embodiments, the data may be redistributed to processes when the device is turned back on. A non-volatile storage system may capture the needed policy and/or aspect-oriented programming information. When the device is powered on, either a distributed or centralized mechanism may be used for input/output of policy and/or aspect-oriented programming data into processes to enforce security policies.
In embodiments, methods for securing a device may include combining non aspect-oriented programming logic may be coupled with aspect-oriented programming to bring a device to a desired state. In some embodiments, securing the device may include securing specific device functions. For example, non aspect-oriented programming logic may turn off wireless network access before an aspect-oriented programming technique is used to restrict which applications may turn wireless network access on or off. In another example, non aspect-oriented programming logic can automatically shut down a malware application before an aspect-oriented programming technique is used to prevent relaunch of the malware.
In embodiments, methods for securing a device may include adapting an IPC mechanism so that a request over an IPC bus from an application in a normal zone for another application or service may be automatically redirected to a trusted version of the requested application or service.
In embodiments, methods for authenticating an unspoofable context on a device by providing a context detection engine on a server that verifies the context on the device and, in response to the verification, provides access to secure data. In embodiments, the server may be a gateway server to a network.
In embodiments, methods for composing a policy may include combining a plurality of policies from one or more sources to provide a single, coherent policy for a policy engine by reconciling any inconsistent rules. A policy may be a security policy. The plurality of policies may be comprised of, for example, a phone policy, an IT administrator policy, a cellphone carrier policy, an enterprise policy, a department policy or some other policy. The sources of policies may include, for example, a cellphone carrier, a government, a device provider, a device support provider, a device user, the enterprise who supplied the device to the user or some other policy provider. Reconciling inconsistent rules may include comparing two or more rules and selecting the most restrictive rule. Reconciling inconsistent rules may, in some embodiments, include comparing two or more rules and selecting the least restrictive rule. Reconciling inconsistent rules may, in some embodiments, include comparing two or more rules and selecting one of the rules based on some other set of rules, for example, based on to what resource(s) the inconsistent rules apply.
Embodiments of methods and systems for securing a device are depicted in
One or more applications 110 may execute locally on the system 102. In some embodiments, the application 110 may be a game, such as Angry Birds, Smash Cops, Words with Friends or some other game. In some embodiments, the application may be a utility, such as the phone application, Skype, a web browser, a music player or some other utility. In some embodiments, the application may be a tool, such as Twitter, ESPN ScoreCenter, Google Translate or some other tool. The application 110 may be downloaded to the system from a legitimate market place, for example iTunes. However, in some cases, the application 110 may be obtained from a malware system 108. In some other cases, the application 110 may be made available from a malware system 108 via a legitimate market place. In embodiments, the application may attempt to execute one or more of privileged code (e.g. code that only may be accessed once permission is granted by a privileged code service 140), code in a trusted code zone 146, or code protected by an object firewall 144.
In embodiments, one or more applications 110 may execute in one or more virtual machines 112. Examples of virtual machines include, but are not limited to, Java Virtual Machines, Perl virtual machines, an Oracle Virtual Machine, a Parallels virtual machine, a Sun xVM, and a VMware virtual machine. To load, link, and execute code in native libraries 114, an application 110 may send a library request to the respective virtual machine 112. The virtual machine 112 may communicate with the policy engine 118 to determine if the request is allowed. In some embodiments, the virtual machine 112 may also use a local policy to determine if the request is allowed. If the request is allowed, the virtual machine 112 may facilitate application 110 access to the native library 114, which facilitates interacting with an operating system 116. The virtual machines 112 may signal library access allowance, such as to a native library 114, to the applications 110.
The native library 114 may facilitate an interaction between an application 110 and an operating system 116. The operating system 116 of the system 102 is the software that manages the system 102. Examples of operating systems include, but are not limited to, Android, BlackBerry OS, iOS, Symbian OS, Windows Phone and Chrome OS.
The policy engine 118 may enforce policies, for example, on the loading, linking, and execution of code by an application 110, and on remote procedure calls. The policy engine 118 may also generate system-specific context, which may include the current date and time, the device location, and identity of the device user. In some embodiments, the policy engine 118 may enforce a distributed policy on the loading, linking, and execution of native code by an application 110 running inside of a virtual machine 112. In embodiments, the policy engine 118 may be resident in a second process, and dynamically send and adapt one or more rules for loading, linking, and executing code in one or more native libraries 114. Having the policy engine in a second process that is resident on the same system 102 as a first process, may provide higher speed communication to transfer policies to the virtual machine 112 processes, allowing the policies 130 to be dynamically changed based on a number of policy factors. The second process, in which the policy engine 118 may be resident, may isolate the policy engine 118 from attack, allow it to access external services that might not be accessible from the first process, and may allow the policy engine 118 to be resident in memory both before and after the execution of the first process.
In the context of a remote procedure call, the policy engine 118 may approve or disapprove the transaction and may communicate this result back to a data bus. If this remote procedure call involves a system service, the data bus may pass the request to the operating system 116. The operating system 116 may execute the remote procedure call and return the result to the source application 110 via the data bus. If instead this remote procedure call involves an interaction with another application 110, the data bus may pass the call to the destination application 110. The result of that remote procedure call may then be returned via the data bus to the source application 110.
The system 102 may be connected, via a communication facility 150, to a policy server 106 through the cloud or other networking 104. The communication facility 150 may be a network interface controller, a wireless network interface controller, a Wi-Fi adapter and the like. The policy server 106 may manage a policy repository. The policy server 106 may serve policies upon request from the policy engine 118. The policy server 106 may serve such policies by performing a policy repository access to determine policy aspects such as black/white lists 120, signing and/or naming 122, checksum/library analyses 124, permissions for applications, processes, users, groups, and other policy checks 128. The policy server 106 may receive policy repository responses and provide a policy request response to the policy engine 118. Alternatively, the policy engine 118 may serve virtual machines 112 inquiries regarding application 110 access of native libraries 114 based on policy information known to or accessible by the policy engine 118.
In various embodiments, various elements of system 102 may communicate directly or indirectly with communication facility 150. By way of example only and not to limit the sentence above, application 110 and/or operating system 116 may communicate directly with communication facility 150.
The application 110 may include one or more objects that are capable of inter-process communication. In the prior art, these objects were connected directly to the IPC bus 132. Here, the objects may be mediated using object firewalls 144 and/or IPC controllers 138 A and/or B. Here, each object may have an independent object firewall 144 that may connect to an IPC controller 138 A and/or B. An IPC controller 138 A and/or B may connect to the IPC bus 132. The policy engine 118 may communicate with the object firewall 144 and the IPC controllers 138A and 138B to implement one or more policies 130. In some embodiments, the policy engine 118 may translate a high-level firewall rule into a specific setting on one or more object firewalls 144. As new inter-process communication capable objects are created, the IPC controller 138 A and/or B in each process may install additional object firewalls 144 as needed.
In embodiments, the IPC controller 138A may manage the installation and removal of object firewalls 144 as new inter-process communication capable objects are created and destroyed. This controller may eliminate the overhead of performing additional inter-process communications with an IPC controller 138B in another process on each object creation and may improve performance (e.g. by dynamically managing the instances of the object firewalls and IPC controllers associated with each object; by enabling inter-process communications among the objects associated with a single application, as opposed to communicating with a single global controller and/or firewall for all applications and objects; etc.). The IPC controller 138A and/or B may send an IPC call from one inter-process communication capable object to a second inter-process communication capable object's object firewall 144. The second inter-process communication capable object's object firewall 144 may determine, based on a policy 130 implemented as object firewall rules, whether to authorize the call.
The IPC bus 132 may be a data bus. In some embodiments, the IPC bus 132 may enable inter-process communications. In embodiments, the IPC bus 132 may perform inter-process communications via a shared data bus instantiated as a remote procedure call service, protocol handler system call table, or any other function or object broker. For example, the IPC bus 132 may enable inter-process communications as a remote procedure call from an IPC controller 138A associated with an object in one application 110 to another object firewall 144 associated with an object in a second application.
In embodiments, a trusted code zone 146 may exist on the system 102 as a zone of a processor and one or more of the system's 102 specialized debugging interfaces and/or remote auditing tools (e.g. Android™ ADB) may be placed in the trusted code zone 146. A trusted zone of a processor may ensure through a cryptographic chain of trust that code executing within it has not been tampered with. Once an element is placed within the trusted processor zone for execution, the output from operations performed on it may be considered tamper-free, correct, and trusted. An example of commercial software providing trusted zone functionality is TrustZone™ by ARM Limited.
By placing the entire specialized debugging interface and/or tools into the trusted code zone 146, a remote computer can be used to audit the integrity of the system 102 or to securely control the system's 102 execution or configuration with confidence that commands provided remotely are being handled by the correct and trusted debugging software on the system 102. Alternatively, parts of these specialized debugging elements may be placed into the trusted code zone 146 (e.g. file system components and USB I/O components).
In embodiments, the system's 102 inter-process communication mechanism may be placed into the trusted code zone 146. Such an inter-process communication mechanism is intended to govern communication between user-space applications (e.g. not in the operating system) and services (e.g. including system services running in user-space) on the system 102. The inter-process communication mechanism may be, for example, an object firewall 144, an IPC controller 138 A and/or B, or some other inter-process communication mechanism. Once the inter-process communication mechanism is placed into the trusted processor zone, the control of the communication between the user-space applications and services on the device may be considered protected because the software executing in the trusted zone will be tamper-free. Moreover, an inter-process communication mechanism that is secured by a trust zone may be used as a supplemental security control point on the device by intercepting, inspecting, blocking, filtering, or otherwise adapting communications between user-space applications and services. Because the inter-process communication mechanism is within the trusted processor zone, it may be considered a secure point of control over inter-application/service communication.
A system controller 134 may execute a system call 136 in response to a request from an application 110. In embodiments, the system controller 134 may be adapted to send a request to the IPC controller 138 A and/or B, in response to a request from the application 110. By establishing a security policy verification path between the system controller 134 and the IPC subsystem via the IPC controller 138 A and/or B, the system controller 134 may directly verify security permissions via a path that is distinct from the caller application (e.g. based on a query to a policy engine 118). Therefore the query and its result cannot be influenced or manipulated by the caller application or any other application type code. The security of the IPC process itself may further ensure independence of the security permissions query. In embodiments, the subsystem may include the object firewall, IPC controller, and IPC bus. In embodiments the IPC subsystem may include the object firewall, IPC controller, IPC bus, and policy engine.
In embodiments, an application 110 seeking to execute a privileged code service 140 may attempt to make such a privileged code service 140 execution attempt by interfacing with the system controller 134. Rather than simply allowing execution of the code, the system controller 134 may send a request to an IPC controller 138A which may request over an IPC bus 132 to a system service IPC controller 138B for a system service that governs access control for privileged code service execution 140. This service may make an access decision request of the privileged code policy engine 118 to facilitate determining whether the originating application is authorized to execute the requested privileged code. This determination may be made based on a variety of factors, to include without limitation the identity of the calling application, the identity of the device user, the time of day, the physical location of the device, the current device configuration, and the like. An indication of the result of the system call policy determination may then be returned via the IPC controllers 138A and 138B as connected by the IPC bus 132 to the system controller 134, which then may enforce the determination and may either allow or disallow the execution of privileged code service 140. Regardless of the policy determination, information about the execution attempt, conditions used in making the determination, and resulting action may be logged for use by the user and device administrator.
A malware system 108 may attempt to compromise the security on the system 102. The malware system 108 may connect to the system 102 through the cloud or other networking 104. The malware system 108 may communicate malicious software to the system 102. The malicious software may be a computer virus, a worm, a Trojan horse, spyware, adware, a rootkit, or some other malicious program or script. The malicious software may be communicated to the system 102 via an email, a webpage, an application 110, a text message, a SIM card, or in some other fashion.
Networking 104 may communicate via cloud-based networking. In an embodiment, networking 104 may communicate via cloud-based networking via a network, such as, but not limited to the Internet, an intranet, a personal area network, a VPN, a local area network, a wide area network, a metropolitan area network, or some other network.
Referring still to
A jail-broken mobile device as described in various embodiments may be a device where the operating system 116 on the device is broken out of or bypassed so that the user of the device may be able to access files outside of chroot-like restrictions. For example, a user may jailbreak an iPhone to install Cydia, a third party application marketplace alternative to Apple's App Store, which the user would not otherwise be able to do on an iPhone that is not jail-broken.
The privileged code may be code that only may be accessed once permission is granted by a privileged code service 140. For example, the privileged code may be kernel code. A privilege may be, for example to access and run code in supervisor or administrator mode.
In some embodiments, the application 110 may be a game, such as Angry Birds, Smash Cops, Words with Friends or some other game. In some embodiments, the application 110 may be a utility, such as the phone application, Skype, a web browser, a music player or some other utility. In some embodiments, the application 110 may be a tool, such as Twitter, ESPN ScoreCenter, Google Translate or some other tool.
In some embodiments, the policy engine 118 determines whether a call, by an application 110, to execute privileged code may be executed. The determination may be based on one or more of the type of application 110 making the call, the name of the application 110 making the call, the location of the application 110 making the call, the system context, the device location, the current date, the current time, the identity of the device user, the type of the privileged code, the content of the call or some other criteria.
Enforcing the determination of the policy engine 118 may include comparing the determination against a policy 130. The policy engine 118 may enforce the determination based on one or more policies 130. The policies 130 may include, for example, system policies, application policies, and other policies. The policy engine 118 may use the one or more policies 130 to evaluate the call. In some embodiments, the policy engine 118, may also determine, based on the evaluation of the call, whether any data to be transferred by way of the call is authorized.
In embodiments, methods for enforcing security and access control policies on privileged code execution on mobile devices may include calling, by an application 110 to a system controller 134, to execute privileged code; requesting, by the system controller 134 to an inter-process communications controller 138A, for a permission to access the privileged code; requesting, by the system controller 134 to a privileged code policy engine 118, a determination whether the application 110 is permitted to access the privileged code; determining, by the privileged code policy engine 118, whether the application 110 may execute the privileged code; and enforcing, by the system controller 134, the determination by the privileged code policy engine 118. The mobile device may be, for example, a cellular phone, an MP3 player, a tablet and a laptop. Examples of operating systems 116 include, but are not limited to, Android, BlackBerry OS, iOS, Symbian OS, Windows Phone and Chrome OS. The way in which the filtering of the processes using IPC may be implemented may depend on the particular operating system 116. In some embodiments, the operating system 116 may use URI instead of the inter-process communications, for example, in iOS.
A jail-broken mobile device as described in various embodiments may be a device where the operating system 116 on the device is broken out of or bypassed so that the user of the device may be able to access files outside of chroot-like restrictions. For example, a user may jailbreak an iPhone to install Cydia, a third party application marketplace alternative to Apple's App Store.
The privileged code may be code that only may be accessed once permission is granted by a privileged code service 140. For example, the privileged code may be kernel code. A privilege may be, for example to access and run code in supervisor or administrator mode.
In some embodiments, the application 110 may be a game, such as Angry Birds, Smash Cops, Words with Friends or some other game. In some embodiments, the application 110 may be a utility, such as the phone application, Skype, a web browser, a music player or some other utility. In some embodiments, the application 110 may be a tool, such as Twitter, ESPN ScoreCenter, Google Translate or some other tool.
The system controller 134, in response to call to execute privileged code from the application 110, may request permission to access privileged code. In the prior art, the system controller 134 would execute the privileged code in response to the call from the application 110. However, here, the system controller 134 may request permission to access such privileged code from an inter-process communications controller 138A. The inter-process communications controller 138A, in response to the request from the system controller 134, may pass the request to a policy engine 118. In some embodiments, the inter-process communications controller 138A, in response to the request from the system controller 134, may pass the request to a policy engine 118 via an object firewall 144.
In some embodiments, the policy engine 118 determines whether a call, by an application 110, to execute privileged code may be executed. In some embodiments, the policy engine 118 may be a privileged code policy engine. The determination may be based on one or more of the type of application 110 making the call, the name of the application 110 making the call, the location of the application 110 making the call, the system context, the device location, the current date, the current time, the identity of the device user, the type of the privileged code, the content of the call or some other criteria.
Enforcing the determination of the policy engine 118 may include comparing the determination against a policy 130. The policy engine 118 may enforce the determination based on one or more policies 130. The policies 130 may include, for example, system policies, application policies, and other policies. The policy engine 118 may use the one or more policies 130 to evaluate the call. In some embodiments, the policy engine 118, may also determine, based on the evaluation of the call, whether any data to be transferred by way of the call is authorized.
One of the advantages of the present invention may include, without limitation, the fact that the calling application 110 need not be aware of the security policy infrastructure that is responsible for making these decisions about access control. In particular, the execution environment in which the application 110 is operating can be instrumented to support these features in a way that is transparent to the application developer. This may allow for seamless backward compatibility with existing apps that operate using jailbreak tools and no need for development of future application programmer interfaces for new applications 110 that leverage this infrastructure.
One mechanism of using a trusted processor zone to improve mobile device security may be to place a device's specialized debugging interfaces and/or remote auditing tools, such as Android™ ADB, into the trusted zone. These debugging interfaces and tools may provide mechanisms via USB, wireless, or other wired communication to audit, configure, or control one or more of the processes, file systems, applications, and other components of a mobile device. By placing the entire specialized debugging interface and/or tools into the trustzone, a remote computer may be used to audit the integrity of a device or securely control its execution or configuration with confidence that commands provided remotely are being handled by the correct and trusted debugging software on the device. Alternatively, parts of these specialized debugging elements may be placed into the trustzone (e.g. file system components and USB I/O components).
Another mechanism for using a trusted processor zone to improve mobile device security may be to place the device's inter-process communication mechanism into the trusted zone. Such an inter-process communication mechanism is intended to govern communication between user-space applications (e.g. not in the operating system) and services (e.g. including system services running in user-space) on a mobile device. Once the inter-process communication mechanism is placed into the trusted processor zone, the control of the communication between the user-space applications and services on the device may be considered protected because the software executing in the trusted zone will be tamper-free (e.g. because the software executing in the trusted zone may execute independently of the software in all other zones). Moreover, an inter-process communication mechanism that is secured by a trust zone may be used as a supplemental security control point on the device by intercepting, inspecting, blocking, filtering, or otherwise adapting communications between user-space applications and services. Because the inter-process communication mechanism is within the trusted processor zone, it may be considered a secure point of control over inter-application/service communication.
Secure processes with enhanced permissions, such as daemon user-space processes, may be used to spawn and control the execution of other processes on a device. For example, on Android™, the Zygote is responsible for launching and adapting the permissions of the processes for applications. In embodiments, these secure daemons may be moved inside of a trusted processor zone to ensure that they cannot be tampered with to launch, configure, or control other processes maliciously. Further, when a secure daemon is moved within a trusted processor zone along with a secure inter-process communication mechanism, other user-space processes may securely interact with this daemon process.
User-space application permissions, code, and configuration are typically managed by a package manager on a mobile device. The package manager installs, configures, uninstalls, and responds to queries regarding application artifacts, configuration, and permissions. If the package manager on a mobile device is compromised, an attacker can use the package manager to falsely report application permissions, configuration settings, code locations, or other critical parameters. In embodiments, this may allow the package manager to be moved inside of the trusted processor zone in order to ensure that package manager and all of its functions (e.g. package installation, configuration, uninstallation, application info querying, and the like) are not tampered with. By moving a package manager (e.g. Android Package Manager service, and the like) into the trusted processor zone, these critical application packaging services may be protected.
Virtual machines, such as the Dalvik Virtual Machine™, are used to execute code on mobile devices. Since virtual machines control execution of key application code, if they are tampered with, severe security holes can be opened that allow applications to run arbitrary code. By moving the entire virtual machine into the trusted processor zone, the device may ensure that virtual machine execution is not compromised. Likewise, even if core parts of the Dalvik Virtual Machine™, such as instruction dispatching, virtual dispatch tables, socket and I/O code, file system interaction code, class bytecode caches, symbol tables, or class loading mechanisms are moved to a trusted zone, one may ensure that these critical components are not compromised.
Many configuration functions on a mobile device operate via reading XML, querying a relational database (e.g. SQLite), or loading other configuration files and then changing system execution parameters. For example, XML or Java bytecode files (e.g. Android Manifest.dex/class/java/xml) may be used to store mappings of application user IDs to Linux user IDs and permission groups. By moving the I/O, reading, and interpretation of these configuration data sources into the trusted processor zone, the mobile device may ensure that these information sources are properly checked cryptographically for provenance and integrity, read and interpreted properly, and not altered to incorrectly perform their function. Relational database components, configuration loading routines (e.g. Android LayoutInflater, Manifest reader, and the like) may be moved into the trust zone as needed to protect these core functions.
Enterprises use mobile device management systems to control policies governing the usage/security of mobile devices. If a mobile device management system is compromised, an attacker may use these mobile device management systems to steal sensitive data or perform other nefarious actions. By moving one or more parts of the mobile device management system inside of the trusted processor zone one may ensure that they are not compromised. Once inside of the trusted processor zone, these mobile device management functions may be considered secure and not exploitable by attackers.
User input on a device may leverage a touch screen software component to receive touch events from the hardware; translate these events to movement, key presses, or other user input; dispatch the events through shared memory or inter-process communication to a target process; and deliver the events to application software components. If these touch screen software components are tampered with, they can be used as an attack vector to siphon off pin numbers, banking information, and other secure credentials. The trusted zone methods and systems described herein may counteract this threat on mobile devices by moving one or more parts of the software touch screen event dispatching, shared memory reading/writing, inter-process communication dispatch, and intra-application dispatch code into the trusted processor zone. Moreover, the parts moved into the trusted processor zone may include a software input method, such as code for controlling a virtual on-screen keyboard and/or its configuration data, into the trusted processor zone.
A geo-localization, proximity detection, position estimation, or proximity authentication component can be used to determine or validate a device's location. However, these mechanisms can be attacked and/or the result spoofed to make applications on a device detect a different location that the actual location of the device. This may be used to circumvent location-based policies or attack systems that rely on precise localization (e.g. car navigation). To thwart this possible exploit vector, one or more of these systems may be moved into a trusted processor zone to prevent tampering.
While examples of use of a trust zone for enhancing mobile device software and data security have been described herein, there may be other beneficial uses of a trust zone in addition to these examples that are contemplated and therefore included herein. In addition, while TrustZone by ARM Limited uses as an example trust zone facility, any facility that provides a trust zone with robust protection of software and/or data through cryptographic or other tamper-proof means may be used with the methods, systems, and applications described herein.
Referring now to
Some modern mobile devices may use virtual machines to execute applications within controlled execution environments. A key challenge with such systems may be that current approaches may not have fine-grained and/or adaptive mechanisms for determining what libraries of native code may be loaded and used by applications running inside of virtual machines. Approaches may not adapt native libraries that are allowed to be loaded, linked, and executed based on the device context or policies that reside in processes outside of the virtual machine's process. Further, some approaches may have downloaded policies from processes running on other computing systems, but such approaches may be slow since policies may be transferred from remote locations. Further, due to orders of magnitude greater execution speed for local data transfer compared to remote data transfer, downloading approaches may limit the speed and frequency at which native library loading, linking, and execution rules can be adapted.
The primary prior approach to enforcing restrictions on the loading, linking and execution of native library code from within virtual machines may have been to use static policy files stored on disk and loaded into the memory of the virtual machine which may run in a first process. Given the static nature of such approach, which may require native library policies to be resident in the virtual machine's process at startup, the policies may not be changed from a second process running a policy engine.
A more effective and flexible approach to controlling the loading, linking and execution of native code may be to have a policy engine, resident in a second process, dynamically send and adapt the rules for loading, linking, and executing code in native libraries. Because the second process may be resident on the same mobile device as the first process, higher speed communication may be used to transfer policies to the virtual machine processes, which may allow the policies to be dynamically changed based on a number of policy factors. The second process of the policy engine may isolate the policy engine from attack, may allow it to access external services that might not be accessible from the first process, and may allow the policy engine to be resident in memory both before and after the execution of the first process.
In embodiments, systems and methods may comprise using an external policy server resident in a process other than the virtual machine's process to use local cross-process communication means to control the rules governing the loading, linking, and execution of mobile application code running inside of a virtual machine.
Referring now to
The policy engine 118 may optionally exchange policy requests (622, 624) with a policy server 106 that may manage a policy repository 628. The policy server 106 may serve policy requests 622 from the policy engine 118 by performing a policy repository access 630 to determine policy aspects such as black/white lists 120, signing and/or naming 122, checksum/library analyses 124, permissions for applications, processes, users, groups 126, and other policy checks 128. The policy server 106 may receive policy repository responses 632 and itself provide a policy request response 624 to the policy engine 118. Alternatively, the policy engine 118 may serve virtual machines 112 inquiries regarding application 110 access of native libraries 114 based on policy information known to or accessible by the policy engine 118.
Referring now to
Referring now to
In more detail, still referring to
In more detail, still referring to the invention in
In embodiments, applications 110 may be modularly installed on a smart phone and able to perform inter-process communication via the shared data bus 202 which may be instantiated as a remote procedure call service, protocol handler, system call table, or any other function or object broker. The policy engine 118 may be instantiated as extensions to this broker service whereby inter-process communications requests may be evaluated against the available policies. These requests may either be approved or denied based on the outcome of the policy evaluation.
Turning now to
Data transfer authorization may be obtained by ensuring proper data context is obtained 308 and maintained after the transfer 320. Policies used to evaluate whether the transfer is authorized 312 may use the data, the data's context, and the overall system's context to make authorization decisions. Embodiments of this process may ensure that sensitive data is not transferred to an application that is not authorized to receive that data, and/or that data is only transferred between applications and/or individuals that are authorized to send and receive information between each other.
A specific instantiation of this process may be shown in
The advantages of embodiments include, without limitation, the ability to enforce rigorous, detailed security policies on all remote procedure calls, inter-process communication, and system calls that occur on mobile devices. By implementing a system-wide policy engine, device administrators may deploy policies that allow applications can more easily protect themselves against other potentially malicious applications. When applied to data provenance, the movement of all data within a mobile device can be authorized based on parameters such as the source, destination, and sensitivity of the data. This provides significant advantages over prior art that relied on applications to individually approve/disapprove individual transactions without a common policy set.
Referring now to
The policy engine 118 may translate high-level firewall rules into specific settings of a plurality of IPC object firewalls 504 A-D. In embodiments, as new IPC-capable objects 502 A, B, C, and/or D are created, the local IPC controller 138 A and/or B in each process may install one or more IPC object firewalls 504 A-D into the IPC-capable objects 502 A-D as needed.
An application 110 A may initiate an inter-process communication call from an object (e.g. 502 A) to a second object (e.g. 502 D) in a second application 110 B. Optionally, an IPC object firewall 504A on the first application may determine if the outbound IPC call is allowed based on the current IPC firewall rules. The inter-process communication call may be sent to the second application 110 B IPC controller 138 B via the IPC bus 132. The IPC controller 138 B may send the IPC call onto the second object's IPC firewall 504 D. The second object's IPC firewall 504 D may make an access determination based on the IPC firewall rules, the target object 502 D of the call, the data provided with the call, the current state of the target object 502 D, and the current state of the target application 110 B.
The processing of the IPC call by the IPC firewall 504 D of the target object 502 D of the target application 110 B may involve any of the following. The target object IPC firewall 504 D may block the IPC call to the target object 502 D. The target object IPC firewall 504 D may modify the contents of the data sent with the call to the target object 502 D. The target object IPC firewall 504 D may modify the return value of the data sent from the target object 502 D to the initiating object 502 A in response to the inter-process communication call. The target object IPC firewall 504 D may change the target object 502 D of an IPC call. The target object IPC firewall 504 D may log the call. The target object IPC firewall 504 D may change one or more IPC firewall rules or add/remove IPC firewall rules.
When the IPC call returns to the initiating object 502 A, the initiating object's IPC firewall 504 A may determine, based on one or more of the IPC firewall rules, the target object of the call, the data provided with the call, the data provided in the return value of the call, the current state of the initiating object 502 A, and the current state of the initiating application 110 A, how to process the IPC call. The processing may include any one or more of the following: the initiating object 502 A may throw an exception rather than proceeding; the initiating object firewall 504 A may modify the return value of the IPC call; the initiating object firewall 504 A may send additional IPC calls to the initiating object 502 A or other objects (e.g. 502B); the initiating object firewall 504 A may modify one or more IPC firewall rules or adding/removing IPC Firewall rules.
The advantage of present embodiments may include, without limitation, the ability to enforce rigorous, detailed security policies on all IPCs that occur on mobile devices. By implementing a system-wide policy engine, device administrators may deploy policies that allow applications to more easily protect themselves against other potentially malicious applications. When implemented as an IPC firewall, the invention may achieve policy enforcement in an efficient, scalable way that may enforce a broad range of system policies.
In embodiments, referring now to
The system call authorization may be obtained by ensuring proper application context is obtained 408 and updated 420 after the transfer. Policies used to evaluate whether the system call is authorized 410 may use the call, the application's context, and the overall system's context to make authorization decisions. Various embodiments may allow device administrators to push policies to devices that can identify and disable malware based on known system call patterns and application context.
A specific instantiation of this process may be shown in
The advantages of the present embodiments may include, without limitation, is the ability to enforce rigorous, detailed security policies on all remote procedure calls, inter-process communication, and system calls that occur on mobile devices. By implementing a system-wide policy engine, device administrators can deploy policies that allow applications can more easily protect themselves against other potentially malicious applications. When applied to malware detection and prevention, known system call patterns can be recognized, intercepted, and stopped prior to execution. Offending applications can then be disabled and device administrators notified of the malicious activity. This provides significant advantages over prior art that relied on applications to individually approve/disapprove individual system calls without a common policy set. Additionally, it allows device administrators to implement device policies that protect against emerging threats without needing to wait for a vendor-supplied patch to become available.
A further aspect discussed herein is the use of inter-process communication to distribute policy or other data needed to apply aspect-oriented security to a plurality of processes on a mobile device.
A challenge with existing mobile security solutions is that they require modifications to the application programming interfaces, system libraries, or operating system in order to enforce security policies. For example, in order to restrict access to wireless networks or cutting/pasting of data, the APIs related to these features must be modified to allow security policies to change their behavior. For rapidly developing mobile systems, modifying the APIs of the platform to support security features and maintain them requires substantial effort.
Embodiments may address mobile device security issues by allowing security policies to be applied to existing APIs through aspect-oriented programming and applied to existing APIs without modifying the internal logic of APIs. Instead, an existing API may be wrapped with one or more layers of security using the aspect-oriented programming methods and techniques described herein. Although aspect-oriented programming has been used to apply security policies to a single process in non-mobile operating environments, mobile devices, however, use a multi-process architecture and inter-process communication to operate. Therefore, single-process application of security policies may not satisfy mobile device operating security requirements. Inter-process communications may be used to distribute one or more policies or other data needed to apply aspect-oriented security to a plurality of processes on a mobile device. Once the security-related data is distributed via an inter-process communication mechanism, such as the Android Binder or Unix Domain Sockets, to the target processes, aspect-oriented security techniques may be applied to intercept and manage security related to invocations of methods, functions, and services in these target processes.
Aspect-oriented programming may manifest in numerous forms on mobile platforms. An aspect-oriented programming approach may be a modification to an object class to invoke a specific segment of code before, after, in place of, or any combination of these in relation to an object-oriented method execution. An aspect-oriented programming approach may include: a Java Dynamic Proxy; an interceptor applied to a method, service, system, or other function call; a modification of the loading of classes into a virtual machine to change their default behavior; a binary code patch, such as Java JAR or Android DEX files; modification to a method dispatch table to alter code execution for specific functions or methods; and other suitable approaches.
In various embodiments, the ability to use contextual information to alter how policies are applied to the device and consequently how aspect-oriented security techniques are applied across one or more processes may be provided. Such contextual information may include geographic, accelerometer, camera, microphone, wireless network, application usage, user interaction, running processes, disk state, nearby wireless signals/networks, pairing state with external devices, websites being visited, device network traffic, battery level, types of data resident on a device, or other device hardware or software detectable context information. Device context may be either real-world, such as geographic location, virtual, such as data resident on the device, applications currently executing, or input/output of data to/from network or disk, or arbitrary combinations of the two. For example, a security policy may be triggered by connection to a specific wireless network, the launch of one or more applications, or the downloading of specific datasets.
Aspect-oriented security for mobile devices may include tracking which processes that are functioning on the device are covered by some form of aspect-oriented security and/or determining processes that are candidates for aspect-oriented security programming being applied, such as to enforce a security policy. This tracking may be centralized, distributed, or a hybrid combination of the two.
A mechanism for such tracking may determine how to distribute policy and/or aspect-oriented programming data to processes in order to apply security policies to a set of desired functions or device capabilities. Such a mechanism may either reside in the operating system or outside of the operating system in user space.
Since devices may be shut down and restarted, policy and/or aspect-related data may be stored on the device so that it can be redistributed to processes when a device is turned back on. A non-volatile storage system may capture the needed policy and/or aspect-oriented programming information. When a device is powered on, either a distributed or centralized mechanism may be used for input/output of policy and/or aspect-oriented programming data into processes to enforce security policies.
Security policies may encompass restrictions on the execution of application, operating system, middleware, or other code. A security policy may include restrictions on how the user may interact with the system, what operations they may perform, what data they can access, how they can use data, and the like. A security policy may also govern input/output or other operations related to physical hardware.
Additionally, non aspect-oriented programming logic may be coupled with aspect-oriented programming to bring a device to a desired state before securing specific device functions or capabilities. For example, non aspect-oriented programming logic may turn off wireless network access before an aspect-oriented programming technique is used to restrict which apps can turn wireless network access on/off. In another example, non aspect-oriented programming logic may automatically shut down a malware application before an aspect-oriented programming technique is used to prevent re-launch of the malware.
Referring now to
In an AspectJ (Java) example of enforcing mobile device security policy via aspect oriented programming, Secure Setting fields in a mobile operating system may be accessed by a plurality of system functions that may enable setting a field that would result in allowing non-market applications to be installed. A non-market application is an application obtained by a means other than the official market for the operating system on the device (e.g. an Android application obtained from a third party, but not through the official Android Market). To the extent that non-market applications are often unsigned and therefore more likely to present security risks (e.g. may be a form of malware), a security policy may be established that limits the conditions under which a non-market application can be permitted to be installed. Such system functions might appear throughout the system application but may all include names that begin with the word “update” (e.g. updateSecureSettingsInfo) and may take SettingsField Object and Value arguments. Therefore the various occurrences of “updateSecureSettingsInfo” may be a cross-cutting concern that is suitable for employing a security policy via aspect oriented programming. The security policy may specifically target the SettingsField InstallNonMarketApps to prevent changes that would allow installation of non-market apps. A join point may be defined for the secure setting update method and for the SettingsField object that incorporates name elements such as “update”, “info” and “SettingsField”. Based on these join points, AspectJ pointcuts may be prepared for enforcing the security policy that would ensure that any use of a method that begins with “update” and ends with “Info” or any use of the “SettingsField” object may be controlled to comply with the security policy. The pointcuts may be included in an aspect method type along with code to address the security policy. In this example, the accompanying code may detect the “InstallNonMarketApps” access and perform a function after such access to restore the setting to the proper value that does not allow installation of non-market apps. This can be done in AspectJ using an “after” type advice to invoke the security policy enforcing code.
In embodiments, methods and systems for enforcing security in mobile computing may comprise synchronizing data to a mobile device based on device usage context.
Modern mobile devices often store data that is synchronized with a remote system, such as a server. Because of its finite resources compared to the remote system, usually only a partial image of the data stored on the remote system is replicated on the mobile device. This is often accomplished by passing incremental updates between the two systems. For example, a user's email inbox, sent folder and other saved folders may all be stored on a remote email server, and only the most recent 25 emails in the inbox may be stored on the user's mobile device. The emails residing on the mobile device may be updated as the user drafts additional emails from the device or as new emails received at the mail server are pushed to the mobile device. Changes made at the mobile device may be recorded at the mail server as the user, for example, sends emails via the mail server.
Embodiments described below may address security, bandwidth and energy efficiency concerns associated with the current art for synchronizing data on mobile device by intelligently organizing and prioritizing the synchronization of higher priority data. In a system where data is synchronized between two computing systems, such as a server and a mobile device, it may be more secure and more efficient (both with respect to bandwidth and energy usage) to only synchronize said data when it will be of use to one of the computing systems. For example, when synchronizing data to a mobile device from a central server, the mobile device only needs the data when the user is actively using the data or when the data will be immediately usable, not when the mobile device is sitting idle.
These security and efficiency concerns may be addressed by defining multiple classes of data with different synchronization priorities, by defining and monitoring the device's context (e.g. whether the device is idle, whether the user is attempting to unlock the device, whether the user is starting the email client, etc.) and synchronizing one or more classes of data based on the existing classes and the system context.
The methods and systems of the present disclosure may benefit existing applications or enable new ones, including but not limited to, communications applications, such as enhanced features of chat, sharing, social networking, contact management, messaging, email, web browsing and the like; games and entertainment content applications (video games, music, video content, online content, etc.); command and control applications and features (operating system control, phone control, restricted/secured data access control, etc.); enterprise IT management applications, such as device imaging and device wiping; automotive applications, such as navigation, driver support and safety systems; and advanced security tools, such as anti-virus, firmware integrity, operating system integrity, boot loader integrity, firewalls, intrusion detection systems, and intrusion prevention systems, and the like.
Referring to
In embodiments, adaptive synchronization may include adapting a synchronization facility 164 on a system 102 to determine when to synchronize a plurality of classes of data 158, 160 and 130B with data on a server 104.
In a system where data is synchronized between two computing systems, such as a server 1102 and a system 102, it may be advantageous to only synchronize said data when it will be of use to one of the computing systems. For example, when synchronizing data to a mobile device from a central server, the device may only need the data when the device user is actively using the data or when the data will be immediately usable, not when the mobile device is sitting idle.
In one embodiment, a user interaction with the system 102 may initiate a synchronization event. The user interaction with the system 102 may be, for example, an input to the device UI 154. The input to the device UI 154 may one or more of locking the system 102, unlocking the system 102, starting an application 110, stopping an application 110, using an application 110, booting the system 102, shutting down the system 102, sending information to a remote computer, requesting information from a remote computer, or some other input, and the like.
In other embodiments, the synchronization event may be initiated by the system 102 or software executing on the system 102. For example, the power management facility 162 may initiate a synchronization event when the battery of the system 102 reaches a certain charge.
In one example, the user may provide an input to the device UI 154 to lock the screen, and, based on that input, the synchronization facility 164 may determine the system's state (i.e. the user is not intending to use the system 102 for a period of time) and, based on the state, begin synchronizing data on the system 102.
It may be advantageous to adjust the data synchronization process based on current usage state because it may allow the system 102 to realize the full power consumption benefits in low-power states, such as when the system 102 display is turned off, and perform more power-intensive tasks, such as network operations, when the system 102 is in already in use.
In some instances, it may be necessary to define multiple classes of data to be synchronized between the computing systems. One class may be low priority data 160. In some embodiments, the low priority data 160 may be synchronized only when the device is active. Types of data that may be in the class of low priority data may include, for example, personal emails, tweets, contact information, music files, and image files.
Another class of data may be high priority data 158. In some embodiments, the high priority data 158 may be synchronized regardless of the current usage state of the device. In some embodiments, there may be additional classes of data, such as medium priority data, medium-low priority data, highest priority data, and other classes of data. Types of data that may be in the class of high priority data may include, for example, confidential business emails, text messages, voicemail notifications, instructions to wipe data on the device, and classified data.
In embodiments, the data being synchronized may be policy data, such as a policy 130, for a policy engine 118, which may use the policy data to control aspects or features of the system 102.
The policy engine 118 may generate a device-specific context, which may include one or more of the current date and time, the device location, the identity of the device user, and other context-related data. In some embodiments, the policy engine 118 may be connected to a server 1102, such as a policy server 106, which may push one or more policies 130 as policy data to the policy engine 118.
The policy engine 118 may be used to enforce one or more security policies on the system 102. In some embodiments, the policy data may include a policy 130 for the policy engine 118 to cause the system 102 to disable functionality. For example, the policy 130 may include a rule for disabling the camera 152 when the policy engine 124 determines that the system 102 is located in a building that prohibits the use of cameras 152, like a research lab. In other embodiments, the policy data may include a policy 130 for the policy engine 118 to cause the system 102 to perform operations like erasing the stored content on the system 102. For example, the policy 130 may include a rule for wiping all memory on the system 102 when the system user is not an authorized user or in response to an instruction from an authorized user who lost the system 102. In embodiments, a policy 130 that disables the camera 152, for instance, may need only be synchronized when the system 102 is in a high-power state, as the camera 152 cannot be used in a low-power state regardless. However, in the case of a stolen or compromised system 102, it would be necessary to erase any sensitive data stored on the system 102 immediately rather than when the system 102 is going to be interacted with.
In another embodiment, the data synchronization strategy could depend on the context of the receiving computing system. For example, the synchronization facility 164 may initiate data synchronization when events occur on the system 102 such as, when an application 110 is started or stopped. In the policy synchronization example, a synchronization of policies 130 between the computing systems may be triggered when an untrusted application 110 is launched on the system 102. In embodiments, data may be synchronized between a system 102 and a server 1102 based on the power usage state of the system and/or based on other considerations. In embodiments, synchronization may be based on various considerations described herein separately or together.
The synchronization could be made more or less complicated by adjusting the synchronization conditions. For example, the synchronization facility 164 may only use the network 104 while the system 102 is active and the network connection of the network 104 is idle. In another example, the synchronization facility 164 may only use the network 104 while the system 102 is active and in a particular geo-location. In still another example, the synchronization facility 164 may only use the network 104 while the system 102 is active and the user has permitted synchronization.
In embodiments, methods and systems for enforcing security in mobile computing may include securing short-range communications between a mobile device and another device to securely provide location and business identification information. Securing such communications may provide customer location information in addition to the customer identification information. Some embodiments may also use certain events sent over an inter-process communication (IPC) mechanism to securely trigger execution of an application on the device.
Referring to
The business server 1216 may be part of a business system 1204, which may transmit data to the system 102 for determining the location of the system 102 and/or for providing information to the system 102 based on the location of the device 102.
Providing a secure short-range proximity signal may include providing a system 102, wherein the device 102 includes a location-aware facility 1210 and a communication facility 150; and providing a business system 1204 to provide information to the system 102 based on the location of the system 102, wherein the business system 1204 may include one or more short-range proximity radios 1218A-C for identifying the location of the system 102, and a business server 1216 for providing the information. In embodiments, a short-range proximity radio 1218A may be enabled to emit a unique signal, which may be used by the location-aware facility 1210 to identify the location of the device.
The system 102 may be a mobile phone, a tablet, personal digital assistant, a watch, a laptop, or some other device. The system 102 may have one or more applications executing. In some embodiments, the applications may execute in one or more processes 1214A-B. The processes 1214A-B may be connected to an inter-process communications facility 1212 to facilitate communication between one or more processes 1214A-B, and between one or more processes 1214A-B and the location-aware facility 1210. In some embodiments, the inter-process communications facility 1212 may be an inter-process communications firewall 144 to enforce rules governing communication between two subsystems.
An aspect of the disclosure is that the use of Wi-Fi, cellular, Bluetooth, or Bluetooth Low Energy (Bluetooth LE) network events, which may indicate entrance or exit from a business location, may enable sending such events over the inter-process communication facility 1212 to automatically trigger the execution of logic contained within an application running in a process 1214 A and/or B. Such networking events indicating a business location change may be generated in a first process 1214A, transmitted over an inter-process communication facility 1212, and then delivered to a second process 1214B that is dynamically launched to execute logic from the business aiding application. This aspect of the disclosure allows the business aiding application's code to be dynamically loaded into memory and executed upon a networking event, such as a system 102 with a specific Wi-Fi SSID coming into range, which may indicate a business location has been entered or exited. Once this application code is loaded into memory, the application may interact with the user of the system 102 by doing one or more of the following: 1.) using business logic to devise and present personalized discounts based on the user's location in the business and their buying history, 2.) providing a mechanism for requesting help from a customer representative of the store, 3.) offering one or more personalized advertisements, and 4.) offering help and/or directions to a specific product.
The location-aware facility 1210 may be adapted to send and receive transmissions through a communication facility 150 via a network 104. The location aware facility 1210 may use GPS location. The location aware facility 1210 may access a database of stored location data, such as data on locations of devices or IP addresses connected to a network. The location-aware facility 1210 may use a hybrid positioning system, such as using triangulation, trilateration or multilateration using signals such as from a plurality of short-range proximity radios 1218A-C, wireless internet signals, Bluetooth sensors; and/or some other positioning system to identify the system 102 location.
The transmissions between the communication facility 150 and the network 104 may utilize one or more short-range proximity signals, such as, but not limited to, cellular, Bluetooth, Bluetooth LE, near-field communication, RFID, Wi-Fi, and ultrasonic sound. The transmissions may include short-range proximity information from one or more short-range proximity radios 1218A-C. Such transmissions may also include information associated with the location of the system 102 to and/or from the business server 1216. For example, the information may include customer loyalty information, store information, store navigation information, purchasing information, a coupon, barcode scanning information, product information, shopping information, browsing information (such as for products), shopping cart information, and/or other business-aiding information.
The business server 1216 may be part of a business system 1204. In some embodiments, the business server 1216 may include a location calculator 1220, a business operations system 1222, an advertising operations system 1224 and one or more other operations systems 1226. The location calculator 1220 may, in response to data associated with a customer system 102, and received via one or more short-range proximity radios 1218A-C, identify the location of the customer system 102. The advertising operations system 1224 may identify advertisements to be delivered to a customer system 102 based on a location identified by the location calculator 1220. The business operations system 1222 may process a business transaction in response to a location of a customer system 102 identified by the location calculator 1220. For example, the location calculator 1220 may identify that a customer device is standing in front of an end cap for some cookies that are on sale. In the same example, in response to the identification by the location calculator 1220, the advertising operations system 1224 may deliver a coupon for the cookies to the customer system 102. Continuing with the same example, in response to the same identification by the location calculator 1220, the business operations system 1222 may project that, based on the rate of cookie sales to people who have stood in the same location, the store should submit an order for more of the cookies. In another example, in response to an identification by the location calculator 1220, the business operations system 1222 may generate date/time specific suggestions/reminders based on the customer demographic. The other operations systems 1226 may be any other systems, such as, but not limited invoice printing, security, CRM, or other systems.
An aspect of the current disclosure is that the short-range proximity signal may transmit time-dependent cryptographic, identity, and/or session data that the system 102 may collect and use to indicate its location via one or more messages to the business server 1216. The system 102 may either directly transmit the data received over the short-range proximity signal to the business server 1216 to indicate location, or use the data to create derivative data that the system 102 may send to the business server 1216. Such derivative data may be a cryptographic hash, a signature, or other data.
Methods and systems for securing a device may include filtering access to the device resource using a device-based context-aware policy engine to enforce policies relating to the provenance of data. Such methods and systems may be associated with methods and systems for addressing malware threats. The foregoing may further be associated with methods and systems for enforcing distributed policies in mobile networks by providing an inter-process communications firewall on a device to enforce rules governing communication between two systems. For example, a device may be provided in which provenance of data and/or applications must be proven prior to installation/execution/storage on the device. If the provenance of some data and/or application cannot be proven, then the IPC firewall may prevent the installation/execution/storage of the data and/or application. Additionally, the IPC firewall may record the path the data and/or application uses to spread through the system. Such path information may be used by the device or another system to provide this provenance or to determine that the data may be corrupted or the result of a system compromise, such as a malware infection.
Methods and systems for enforcing distributed policies in mobile networks by providing an inter-process communications firewall on a device to enforce rules governing communication between two systems may be associated with other methods and systems. For example, such methods and systems may be associated with methods and systems for securing a device via aspect-oriented programming. For example, IPC firewalls may be used to determine the aspect of the current system, such by tracking the methods called and the payloads passed through the IPC firewalls. Additionally, modifications to or configurations of new IPC firewall rules may occur to change the behavior of the system based on the detected new system aspect.
Additionally, more complex combinations of methods and systems may be useful. For example and as described above, methods and systems for securing a device may include filtering access to the device resource using a device-based context-aware policy engine to enforce policies relating to the provenance of data, and may be associated with methods and systems for addressing malware threats and further associated with methods and systems for enforcing distributed policies in mobile networks by providing an inter-process communications firewall on a device to enforce rules governing communication between two systems. The foregoing may further be associated with methods and systems for enforcing distributed policies on the loading, linking, and execution of native code and with methods and systems for securing the device via aspect-oriented programming. By way of an example, a solution that monitors the content and/or use of IPC mechanisms may determine if the device has been compromised (e.g. infected with malware) based on the current aspect. Such a solution may monitor the device by checking data provenance to determine the origin and path of data transmission, which may be indicative of malware infection. This exemplary solution may also use the detection of malware indicative behavior to change the current aspect, wipe data from the device, or take other preventative measures for data exfiltration or additional malware infection. Such new aspect may include automated steps to remediate the detected threat, such as the enforcement of a security policy to remove applications that have been determined to potentially contain malware. Additionally, the new aspect may include steps to prevent additional infection, such as preventing the execution of native code or the instantiation of other IPC firewall rules.
A similar combination may associate methods and systems for securing a device may include filtering access to the device resource using a device-based context-aware policy engine to enforce polices relating to the provenance of data with methods and systems for enforcing distributed policies in mobile networks by providing an inter-process communications firewall on a device to enforce rules governing communication between two systems. Such combination may be further associated with methods and systems for enforcing distributed policies on the loading, linking, and execution of native code, methods and systems for using a trusted processor zone to improve mobile device security, and methods and systems for securing a device via aspect-oriented programming. For example, all trusted software and applications on a device may be signed with credentials stored in the Trusted Platform Module (TPM) of the device. In the event that the software cannot be verified with credentials originating from the TPM, then the aspect may be altered such that preventative measures can take effect. Such preventative measures may include preventing native code linking, loading and/or executing. In this example, the IPC firewall may record traffic, which may be signed using credentials and stored within the TPM. Access to the TPM can be arbitrated via the IPC firewall, as can any data passed to be stored in or retrieved from the TPM. This arbitration may take into account the current aspect of the system when determining the level of access to be granted.
Methods and systems for securing a device may include filtering access to the device resource using a device-based context-aware policy engine to enforce policies relating to the provenance of data. Such methods and systems may be associated with methods and systems for enforcing security and access control policies on privileged code execution on a jail-broken mobile device, with methods and systems for securing a device via aspect-oriented programming, together with methods and systems for securing short-range communications between a plurality of devices. By way of example, a solution may include setting the privilege level of the user based on cryptographic identification tokens received from transmissions of nearby proximity-based beacons. Such tokens or other data may only be received when with in physical proximity to the short-range proximity signal. Data, stored locally or remotely on a backend server may only be accessible through authentication using the cryptographic identification token received via the short-range transmission. The cryptographic identification token can be used to create a signature that definitively links data provenance to the appropriate user. The aspect of the system may also be changed based on the detected presence and verification of cryptographic identification tokens generated and transmitted by the short-range proximity signal creator, or be altered based on data received from a remote backend server once successful authentication is complete.
Methods and systems for enforcing security and access control policies on privileged code execution on a jail-broken mobile device, methods and systems for securing a device via aspect-oriented programming, and methods and systems for securing short-range communications between a plurality of devices may be associated with and combined with other methods and systems. For example, such methods and systems may be associated with methods and systems for enforcing security in mobile computing may comprise synchronizing data to a mobile device based on device usage context. For example, data synchronization may occur when the device is within close proximity to a short range signal emitter. In this example, the credentials transmitted to the mobile device may be used to authenticate with a remote backend server. Once this authentication is complete, the aspect of the mobile device may be changed so that secure and privileged data may be synchronized between the mobile device and the server. This process may also make use of credentials stored in the TPM to decrypt the data received from the remote backend server. The credentials needed to complete this decryption may differ from those received from the short range signal emitter to authenticate with the remote backend and may only be accessible if the privileged access has been granted for the current aspect.
While only a few embodiments of the present invention have been shown and described, it will be obvious to those skilled in the art that many changes and modifications may be made thereunto without departing from the spirit and scope of the present invention as described in the following claims. All patent applications and patents, both foreign and domestic, and all other publications referenced herein are incorporated herein in their entireties to the full extent permitted by law.
The methods and systems described herein may be deployed in part or in whole through a machine that executes computer software, program codes, and/or instructions on a processor. The present invention may be implemented as a method on the machine, as a system or apparatus as part of or in relation to the machine, or as a computer program product embodied in a computer readable medium executing on one or more of the machines. In embodiments, the processor may be part of a server, cloud server, client, network infrastructure, mobile computing platform, stationary computing platform, or other computing platform. A processor may be any kind of computational or processing device capable of executing program instructions, codes, binary instructions and the like. The processor may be or include a signal processor, digital processor, embedded processor, microprocessor or any variant such as a co-processor (math co-processor, graphic co-processor, communication co-processor and the like) and the like that may directly or indirectly facilitate execution of program code or program instructions stored thereon. In addition, the processor may enable execution of multiple programs, threads, and codes. The threads may be executed simultaneously to enhance the performance of the processor and to facilitate simultaneous operations of the application. By way of implementation, methods, program codes, program instructions and the like described herein may be implemented in one or more thread. The thread may spawn other threads that may have assigned priorities associated with them; the processor may execute these threads based on priority or any other order based on instructions provided in the program code. The processor, or any machine utilizing one, may include memory that stores methods, codes, instructions and programs as described herein and elsewhere. The processor may access a storage medium through an interface that may store methods, codes, and instructions as described herein and elsewhere. The storage medium associated with the processor for storing methods, programs, codes, program instructions or other type of instructions capable of being executed by the computing or processing device may include but may not be limited to one or more of a CD-ROM, DVD, memory, hard disk, flash drive, RAM, ROM, cache and the like.
A processor may include one or more cores that may enhance speed and performance of a multiprocessor. In embodiments, the process may be a dual core processor, quad core processors, other chip-level multiprocessor and the like that combine two or more independent cores (called a die).
The methods and systems described herein may be deployed in part or in whole through a machine that executes computer software on a server, client, firewall, gateway, hub, router, or other such computer and/or networking hardware. The software program may be associated with a server that may include a file server, print server, domain server, internet server, intranet server, cloud server, and other variants such as secondary server, host server, distributed server and the like. The server may include one or more of memories, processors, computer readable media, storage media, ports (physical and virtual), communication devices, and interfaces capable of accessing other servers, clients, machines, and devices through a wired or a wireless medium, and the like. The methods, programs, or codes as described herein and elsewhere may be executed by the server. In addition, other devices required for execution of methods as described in this application may be considered as a part of the infrastructure associated with the server.
The server may provide an interface to other devices including, without limitation, clients, other servers, printers, database servers, print servers, file servers, communication servers, distributed servers, social networks, and the like. Additionally, this coupling and/or connection may facilitate remote execution of program across the network. The networking of some or all of these devices may facilitate parallel processing of a program or method at one or more location without deviating from the scope of the disclosure. In addition, any of the devices attached to the server through an interface may include at least one storage medium capable of storing methods, programs, code and/or instructions. A central repository may provide program instructions to be executed on different devices. In this implementation, the remote repository may act as a storage medium for program code, instructions, and programs.
The software program may be associated with a client that may include a file client, print client, domain client, internet client, intranet client and other variants such as secondary client, host client, distributed client and the like. The client may include one or more of memories, processors, computer readable media, storage media, ports (physical and virtual), communication devices, and interfaces capable of accessing other clients, servers, machines, and devices through a wired or a wireless medium, and the like. The methods, programs, or codes as described herein and elsewhere may be executed by the client. In addition, other devices required for execution of methods as described in this application may be considered as a part of the infrastructure associated with the client.
The client may provide an interface to other devices including, without limitation, servers, other clients, printers, database servers, print servers, file servers, communication servers, distributed servers and the like. Additionally, this coupling and/or connection may facilitate remote execution of program across the network. The networking of some or all of these devices may facilitate parallel processing of a program or method at one or more location without deviating from the scope of the disclosure. In addition, any of the devices attached to the client through an interface may include at least one storage medium capable of storing methods, programs, applications, code and/or instructions. A central repository may provide program instructions to be executed on different devices. In this implementation, the remote repository may act as a storage medium for program code, instructions, and programs.
The methods and systems described herein may be deployed in part or in whole through network infrastructures. The network infrastructure may include elements such as computing devices, servers, routers, hubs, firewalls, clients, personal computers, communication devices, routing devices and other active and passive devices, modules and/or components as known in the art. The computing and/or non-computing device(s) associated with the network infrastructure may include, apart from other components, a storage medium such as flash memory, buffer, stack, RAM, ROM and the like. The processes, methods, program codes, instructions described herein and elsewhere may be executed by one or more of the network infrastructural elements. The methods and systems described herein may be adapted for use with any kind of private, community, or hybrid cloud computing network or cloud computing environment, including those which involve features of software as a service (SaaS), platform as a service (PaaS), and/or infrastructure as a service (IaaS).
The methods, program codes, and instructions described herein and elsewhere may be implemented on a cellular network having multiple cells. The cellular network may either be frequency division multiple access (FDMA) network or code division multiple access (CDMA) network. The cellular network may include mobile devices, cell sites, base stations, repeaters, antennas, towers, and the like. The cell network may be a GSM, GPRS, 3G, EVDO, mesh, or other networks types.
The methods, program codes, and instructions described herein and elsewhere may be implemented on or through mobile devices. The mobile devices may include navigation devices, cell phones, mobile phones, mobile personal digital assistants, laptops, palmtops, netbooks, pagers, electronic books readers, music players and the like. These devices may include, apart from other components, a storage medium such as a flash memory, buffer, RAM, ROM and one or more computing devices. The computing devices associated with mobile devices may be enabled to execute program codes, methods, and instructions stored thereon. Alternatively, the mobile devices may be configured to execute instructions in collaboration with other devices. The mobile devices may communicate with base stations interfaced with servers and configured to execute program codes. The mobile devices may communicate on a peer-to-peer network, mesh network, or other communications network. The program code may be stored on the storage medium associated with the server and executed by a computing device embedded within the server. The base station may include a computing device and a storage medium. The storage device may store program codes and instructions executed by the computing devices associated with the base station.
The computer software, program codes, and/or instructions may be stored and/or accessed on machine readable media that may include: computer components, devices, and recording media that retain digital data used for computing for some interval of time; semiconductor storage known as random access memory (RAM); mass storage typically for more permanent storage, such as optical discs, forms of magnetic storage like hard disks, tapes, drums, cards and other types; processor registers, cache memory, volatile memory, non-volatile memory; optical storage such as CD, DVD; removable media such as flash memory (e.g. USB sticks or keys), floppy disks, magnetic tape, paper tape, punch cards, standalone RAM disks, Zip drives, removable mass storage, off-line, and the like; other computer memory such as dynamic memory, static memory, read/write storage, mutable storage, read only, random access, sequential access, location addressable, file addressable, content addressable, network attached storage, storage area network, bar codes, magnetic ink, and the like.
The methods and systems described herein may transform physical and/or or intangible items from one state to another. The methods and systems described herein may also transform data representing physical and/or intangible items from one state to another.
The elements described and depicted herein, including in flow charts and block diagrams throughout the figures, imply logical boundaries between the elements. However, according to software or hardware engineering practices, the depicted elements and the functions thereof may be implemented on machines through computer executable media having a processor capable of executing program instructions stored thereon as a monolithic software structure, as standalone software modules, or as modules that employ external routines, code, services, and so forth, or any combination of these, and all such implementations may be within the scope of the present disclosure. Examples of such machines may include, but may not be limited to, personal digital assistants, laptops, personal computers, mobile phones, other handheld computing devices, medical equipment, wired or wireless communication devices, transducers, chips, calculators, satellites, tablet PCs, electronic books, gadgets, electronic devices, devices having artificial intelligence, computing devices, networking equipment, servers, routers and the like. Furthermore, the elements depicted in the flow chart and block diagrams or any other logical component may be implemented on a machine capable of executing program instructions. Thus, while the foregoing drawings and descriptions set forth functional aspects of the disclosed systems, no particular arrangement of software for implementing these functional aspects should be inferred from these descriptions unless explicitly stated or otherwise clear from the context. Similarly, it will be appreciated that the various steps identified and described above may be varied, and that the order of steps may be adapted to particular applications of the techniques disclosed herein. All such variations and modifications are intended to fall within the scope of this disclosure. As such, the depiction and/or description of an order for various steps should not be understood to require a particular order of execution for those steps, unless required by a particular application, or explicitly stated or otherwise clear from the context.
The methods and/or processes described above, and steps associated therewith, may be realized in hardware, software or any combination of hardware and software suitable for a particular application. The hardware may include a general-purpose computer and/or dedicated computing device or specific computing device or particular aspect or component of a specific computing device. The processes may be realized in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors or other programmable device, along with internal and/or external memory. The processes may also, or instead, be embodied in an application specific integrated circuit, a programmable gate array, programmable array logic, or any other device or combination of devices that may be configured to process electronic signals. It will further be appreciated that one or more of the processes may be realized as a computer executable code capable of being executed on a machine-readable medium.
The computer executable code may be created using a structured programming language such as C, an object oriented programming language such as C++, or any other high-level or low-level programming language (including assembly languages, hardware description languages, and database programming languages and technologies) that may be stored, compiled or interpreted to run on one of the above devices, as well as heterogeneous combinations of processors, processor architectures, or combinations of different hardware and software, or any other machine capable of executing program instructions.
Thus, in one aspect, methods described above and combinations thereof may be embodied in computer executable code that, when executing on one or more computing devices, performs the steps thereof. In another aspect, the methods may be embodied in systems that perform the steps thereof, and may be distributed across devices in a number of ways, or all of the functionality may be integrated into a dedicated, standalone device or other hardware. In another aspect, the means for performing the steps associated with the processes described above may include any of the hardware and/or software described above. All such permutations and combinations are intended to fall within the scope of the present disclosure.
While the disclosure has been disclosed in connection with the preferred embodiments shown and described in detail, various modifications and improvements thereon will become readily apparent to those skilled in the art. Accordingly, the spirit and scope of the present disclosure is not to be limited by the foregoing examples, but is to be understood in the broadest sense allowable by law.
The use of the terms “a” and “an” and “the” and similar referents in the context of describing the disclosure (especially in the context of the following claims) is to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms “comprising,” “having,” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate the disclosure and does not pose a limitation on the scope of the disclosure unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.
While the foregoing written description enables one of ordinary skill to make and use what is considered presently to be the best mode thereof, those of ordinary skill will understand and appreciate the existence of variations, combinations, and equivalents of the specific embodiment, method, and examples herein. The disclosure should therefore not be limited by the above described embodiment, method, and examples, but by all embodiments and methods within the scope and spirit of the disclosure.
All documents referenced herein are hereby incorporated by reference.
This application is a Non-Provisional Application of the following U.S. Provisional Applications, each of which is hereby incorporated by reference herein in its entirety: U.S. Provisional Application Ser. No. 61/621,865 entitled METHOD TO ENFORCE SECURITY POLICIES ON THE LOADING, LINKING, AND EXECUTION OF NATIVE CODE BY MOBILE APPLICATIONS RUNNING INSIDE OF VIRTUAL MACHINES, filed Apr. 9, 2012; U.S. Provisional Application Ser. No. 61/662,189 entitled APPLICATIONS OF A PROCESSOR TRUSTED ZONE FOR ENHANCED MOBILE DEVICE SECURITY, filed Jun. 20, 2012; U.S. Provisional Application Ser. No. 61/655,109, entitled METHOD TO ENFORCE ACCESS CONTROL POLICIES ON PRIVILEGED ACCESSES FOR MOBILE DEVICES, filed Jun. 4, 2012; U.S. Provisional Application Ser. No. 61/675,927 entitled MOBILE SECURITY VIA ASPECT ORIENTED PROGRAMMING, filed Jul. 26, 2012; U.S. Provisional Application Ser. No. 61/693,427 entitled RESTRICTING ACCESS TO NETWORK RESOURCES VIA IN-LOCATION ACCESS POINT PROTOCOL, filed Aug. 27, 2012; U.S. Provisional Application Ser. No. 61/780,408 entitled SYSTEMS AND METHODS TO SYNCHRONIZE DATA TO A MOBILE DEVICE BASED ON A DEVICE USAGE CONTEXT, filed Mar. 13, 2013; U.S. Provisional Application Ser. No. 61/781,252 entitled SYSTEMS AND METHODS TO SECURE SHORT-RANGE PROXIMITY SIGNALS, filed Mar. 14, 2013; U.S. Provisional Application Ser. No. 61/779,931 entitled SYSTEMS AND METHODS FOR SECURING THE BOOT PROCESS OF A DEVICE USING CREDENTIALS STORED ON AN AUTHENTICATION TOKEN, filed Mar. 13, 2013; U.S. Provisional Application Ser. No. 61/785,109 entitled SYSTEMS AND METHODS FOR SECURING AND LOCATING COMPUTING DEVICES, filed Mar. 14, 2013; and U.S. Provisional Application Ser. No. 61/790,728, entitled SYSTEMS AND METHODS FOR ENFORCING SECURITY IN MOBILE COMPUTING, filed Mar. 15, 2013. This application is a Continuation-in-Part of Co-pending U.S. application Ser. No. 13/735,885, ENTITLED SYSTEMS AND METHODS FOR ENFORCING SECURITY IN MOBILE COMPUTING, filed Jan. 7, 2013, which is hereby incorporated by reference herein in its entirety. U.S. application Ser. No. 13/735,885 claims the benefit of the following U.S. Provisional Applications, each of which is hereby incorporated by reference herein in its entirety: U.S. Provisional Application Ser. No. 61/583,605, entitled METHOD TO ENFORCE DATA PROVENANCE IN MOBILE COMPUTING, filed Jan. 6, 2012; U.S. Provisional Application Ser. No. 61/583,610, entitled METHOD TO PROTECT AGAINST MALWARE THREATS IN MOBILE COMPUTING, filed Jan. 6, 2012; and U.S. Provisional Application Ser. No. 61/584,284, entitled METHOD TO ENFORCE SECURITY POLICIES USING A DISTRIBUTED OBJECT-ORIENTED IPC FIREWALL, filed Jan. 8, 2012.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6317868 | Grimm et al. | Nov 2001 | B1 |
| 6467086 | Kiczales et al. | Oct 2002 | B1 |
| 7207041 | Elson et al. | Apr 2007 | B2 |
| 7461144 | Beloussov et al. | Dec 2008 | B1 |
| 7681226 | Kraemer et al. | Mar 2010 | B2 |
| 7966599 | Malasky | Jun 2011 | B1 |
| 8099596 | Rusakov et al. | Jan 2012 | B1 |
| 8225329 | Lynn | Jul 2012 | B1 |
| 8387048 | Grechishkin et al. | Feb 2013 | B1 |
| 8490191 | Kuegler et al. | Jul 2013 | B2 |
| 8505029 | Chanda | Aug 2013 | B1 |
| 8516095 | Eisener et al. | Aug 2013 | B2 |
| 8533860 | Grecia | Sep 2013 | B1 |
| 8584229 | Brutch | Nov 2013 | B2 |
| 8595832 | Yee et al. | Nov 2013 | B1 |
| 8613052 | Weiss | Dec 2013 | B2 |
| 8655307 | Walker et al. | Feb 2014 | B1 |
| 8874891 | Gattegno | Oct 2014 | B2 |
| 8887308 | Grecia | Nov 2014 | B2 |
| 9191382 | Hornung et al. | Nov 2015 | B1 |
| 20020052965 | Dowling | May 2002 | A1 |
| 20030115291 | Kendall et al. | Jun 2003 | A1 |
| 20030140088 | Robinson et al. | Jul 2003 | A1 |
| 20030149874 | Balfanz et al. | Aug 2003 | A1 |
| 20030194094 | Lampson et al. | Oct 2003 | A1 |
| 20040199763 | Freund | Oct 2004 | A1 |
| 20040255145 | Chow | Dec 2004 | A1 |
| 20050060365 | Robinson et al. | Mar 2005 | A1 |
| 20050071643 | Moghe | Mar 2005 | A1 |
| 20050136845 | Masuoka et al. | Jun 2005 | A1 |
| 20050138416 | Qian et al. | Jun 2005 | A1 |
| 20050240396 | Childs et al. | Oct 2005 | A1 |
| 20050240985 | Alkove | Oct 2005 | A1 |
| 20050246453 | Erlingsson | Nov 2005 | A1 |
| 20050246522 | Samuelsson et al. | Nov 2005 | A1 |
| 20050246718 | Erlingsson | Nov 2005 | A1 |
| 20060048226 | Rits et al. | Mar 2006 | A1 |
| 20060092037 | Neogi et al. | May 2006 | A1 |
| 20060224742 | Shahbazi | Oct 2006 | A1 |
| 20070088792 | Piper | Apr 2007 | A1 |
| 20070186274 | Thrysoe et al. | Aug 2007 | A1 |
| 20070271594 | Wobber et al. | Nov 2007 | A1 |
| 20080115011 | Codrescu et al. | May 2008 | A1 |
| 20080126800 | Guo et al. | May 2008 | A1 |
| 20080134347 | Goyal et al. | Jun 2008 | A1 |
| 20080235587 | Heie et al. | Sep 2008 | A1 |
| 20080278408 | Strickland et al. | Nov 2008 | A1 |
| 20090025011 | Neil | Jan 2009 | A1 |
| 20090030974 | Boudreau et al. | Jan 2009 | A1 |
| 20090138547 | Boudreau | May 2009 | A1 |
| 20090204815 | Dennis et al. | Aug 2009 | A1 |
| 20090282477 | Chen | Nov 2009 | A1 |
| 20090319782 | Lee | Dec 2009 | A1 |
| 20100023582 | Pedersen et al. | Jan 2010 | A1 |
| 20100031252 | Horwitz | Feb 2010 | A1 |
| 20100031325 | Maigne et al. | Feb 2010 | A1 |
| 20100037311 | He et al. | Feb 2010 | A1 |
| 20100121567 | Mendelson | May 2010 | A1 |
| 20100121597 | Steffens et al. | May 2010 | A1 |
| 20100153697 | Ford et al. | Jun 2010 | A1 |
| 20100306759 | Kohler | Dec 2010 | A1 |
| 20110023115 | Wright | Jan 2011 | A1 |
| 20110055890 | Gaulin et al. | Mar 2011 | A1 |
| 20110067107 | Weeks | Mar 2011 | A1 |
| 20110151955 | Nave | Jun 2011 | A1 |
| 20110197201 | Yoo | Aug 2011 | A1 |
| 20110225624 | Sawhney | Sep 2011 | A1 |
| 20120033678 | Page | Feb 2012 | A1 |
| 20120084381 | Alladi | Apr 2012 | A1 |
| 20120084792 | Benedek | Apr 2012 | A1 |
| 20120116602 | Vaswani et al. | May 2012 | A1 |
| 20120129503 | Lindeman et al. | May 2012 | A1 |
| 20120191677 | Lim | Jul 2012 | A1 |
| 20120197988 | Leppanen et al. | Aug 2012 | A1 |
| 20120210417 | Shieh | Aug 2012 | A1 |
| 20120215637 | Hermann | Aug 2012 | A1 |
| 20120254602 | Bhansali et al. | Oct 2012 | A1 |
| 20120255014 | Sallam | Oct 2012 | A1 |
| 20120258730 | Tinnakornsrisuphap et al. | Oct 2012 | A1 |
| 20120304310 | Blaisdell | Nov 2012 | A1 |
| 20130007873 | Prakash et al. | Jan 2013 | A1 |
| 20130054812 | DeCoteau | Feb 2013 | A1 |
| 20130067488 | Kumar et al. | Mar 2013 | A1 |
| 20130083722 | Bhargava et al. | Apr 2013 | A1 |
| 20130086202 | Connelly | Apr 2013 | A1 |
| 20130091543 | Wade et al. | Apr 2013 | A1 |
| 20130111039 | Gomes | May 2013 | A1 |
| 20130124840 | Diluoffo et al. | May 2013 | A1 |
| 20130145362 | Dawson | Jun 2013 | A1 |
| 20130179991 | White et al. | Jul 2013 | A1 |
| 20130227641 | White et al. | Aug 2013 | A1 |
| 20130232573 | Saidi et al. | Sep 2013 | A1 |
| 20130268997 | Clancy, III | Oct 2013 | A1 |
| 20130291056 | Gaudet et al. | Oct 2013 | A1 |
| 20130312058 | Thompson et al. | Nov 2013 | A1 |
| 20140059357 | Andersson et al. | Feb 2014 | A1 |
| 20140075451 | Anderson | Mar 2014 | A1 |
| 20140201807 | White et al. | Jul 2014 | A1 |
| 20140256251 | Caceres et al. | Sep 2014 | A1 |
| 20140273857 | White et al. | Sep 2014 | A1 |
| 20140282857 | White et al. | Sep 2014 | A1 |
| 20140282992 | Clancy, III et al. | Sep 2014 | A1 |
| 20140283136 | Dougherty et al. | Sep 2014 | A1 |
| Number | Date | Country |
|---|---|---|
| 2002-506247 | Feb 2002 | JP |
| 2005-184836 | Jul 2005 | JP |
| 2009-289198 | Dec 2009 | JP |
| WO-9945454 | Sep 1999 | WO |
| Entry |
|---|
| Debbabi et al., “Java for Mobile Devices: A Security Study”, 2005. |
| Gronli et al., “Android vs Windows Mobile vs Java ME”, “A comparative Study of Mobile Development Environments”, 2010. |
| Ion et al.,“Extending the Java Virtual Machine to Enforce Fine-Grained Security Policies in Mobile Devices”, 2007. |
| International Search Report and Written Opinion issued by the International Searching Authority for Application No. PCT/US14/33344 mailed on Sep. 11, 2014 (12 pages). |
| Bertino et al., “Location-Based Access Control Systems for Mobile Users—Concepts and Research Directions,” SPRINGL '11 ACM, Chicago, IL, Nov. 1, 2011, pp. 49-52 (4 pages). |
| Cogen, “Android 101: Rooting, Jailbreaking, and Unlocking,”http://theunlockr.com/2010/08/27/android-101-rooting-jailbreaking-and-unlocking Accessed Dec. 15, 2014 (8 pgs.). |
| Extended European Search Report issued by the European Patent Office for European Patent Application No. 13733798.6, dated May 6, 2015, 6 pages. |
| Fenzi, “Tuning Your SELinux Policy with Audit2allow,” http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/SA/v14/i08/a1.htm Accessed Dec. 15, 2014 (6 pgs.) |
| Garfinkel, “Traps and Pitfalls: Practical Problems in System Call Interposition Based Security Tools,” The 10th Annual Network and Distributed System Security Symposium, Feb. 2003 (14 pgs.) |
| International Search Report and Written Opinion issued by the U. S. Patent and Trademark Office as International Searching Authority for International Application No. PCT/US14/47826 dated Apr. 1, 2015 (8 pages). |
| International Search Report and Written Opinion issued by the United States Patent and Trademark Office as International Searching Authority for International Application No. PCT/US2015/020492 mailed Jun. 25, 2015 (7 pages). |
| International Search Report and Written Opinion issued by the U.S. Patent and Trademark Office as International Searching Authority in International Application No. PCT/US14/33344, mailed Sep. 11, 2014. 12 pages. |
| Kotrappa et al., “Multilevel Security Using Aspect Oriented Programming AspectJ,” 2010 International Conference on Advances in Recent Technologies in Communication and Computing, IEEE Computer Society, No Month 2010, (pp. 369-373) (5 pages). |
| Schreiber, “Android Binder Android Interprocess Communication,” Ruhr-Universitat Bochum, Seminarthesis, Oct. 5, 2011 (34 pgs.). |
| Shimko et al., “Securing Inter-process Communications in SELinux,” Tresys Technology, Mar. 12, 2007 (10 pgs.). |
| Proceedings of the 52nd (1996, the first term) Annual Convention(1) of Information Processing Society of Japan (IPSJ), Information Processing Society of Japan (IPSJ), Mar. 6, 1996, pp. 1-75-1-76 (4 pages)—Japanese Language. |
| Number | Date | Country | |
|---|---|---|---|
| 20130227641 A1 | Aug 2013 | US |
| Number | Date | Country | |
|---|---|---|---|
| 61621865 | Apr 2012 | US | |
| 61662189 | Jun 2012 | US | |
| 61655109 | Jun 2012 | US | |
| 61675975 | Jul 2012 | US | |
| 61693427 | Aug 2012 | US | |
| 61780408 | Mar 2013 | US | |
| 61781252 | Mar 2013 | US | |
| 61779931 | Mar 2013 | US | |
| 61785109 | Mar 2013 | US | |
| 61790728 | Mar 2013 | US | |
| 61583605 | Jan 2012 | US | |
| 61583610 | Jan 2012 | US | |
| 61584284 | Jan 2012 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13735885 | Jan 2013 | US |
| Child | 13858929 | US |
