Privilege management and revocation

Information

  • Patent Grant
  • 10515195
  • Patent Number
    10,515,195
  • Date Filed
    Monday, July 31, 2017
    7 years ago
  • Date Issued
    Tuesday, December 24, 2019
    5 years ago
Abstract
This disclosure relates to management of privileges associated with applications accessible by users of electronic devices. In one aspect, an electronic device detects that a privilege has been revoked, shuts down any application running on the electronic device that has previously accessed the privilege, and restarts any application that was shut down, the restarted application no longer having any access to the revoked privilege. In another aspect, an electronic device keeps a log of which applications have previously accessed which privileges, receives a new set of privileges associated with applications, determines that a privilege has been revoked, and if the log indicates that an application previously accessed the privilege, resets the electronic device. In a further aspect, a method sets privileges associated with applications, records which electronic devices have which applications, revokes a privilege, and instructs those electronic devices having applications to which the privilege is associated to reset themselves.
Description
FIELD OF THE INVENTION

The present invention relates generally to the management of privileges associated with certain applications that are accessible by users of electronic equipment, such as, for example, networked computers, mobile wireless communications devices, and the like. In particular, the disclosure is directed to systems and methods for managing privileges associated with particular applications and for revoking these privileges in a timely and robust manner.


RELATED ART

It is well known that certain electronic equipment, such as, for example, networked computers, mobile wireless communications devices, and the like, include applications resident on such equipment that may have access to certain privileges that enable the applications to perform various functions. Typically, a system administrator may use IT policy and application control to set the privileges associated with various applications present on the equipment that is subject to the administrator's control. Examples of privileges may include, for example, allowing an application to use inter-process communication (IPC), enabling the opening of internal and external connections, enabling the injection of browser filters, enabling Bluetooth™ functionality, enabling use of e-mail, enabling the use of personal information management (PIM) functionality, use of application program interface (API), etc. It is important for the system administrator be able to track which applications have access to which privileges, and to be able to revoke privileges on an as needed basis.


For example, if an application has access to a privilege, and the system administrator revokes that privilege, the application should immediately, or within a small window of time, be denied access to that privilege. In other words, the privilege should be revoked as soon as possible. Events that might trigger a revocation of privileges may include, for example, an application being loaded before the IT administrator/application control data is present on the device, an application is discovered to be a rogue application, or company policy changes, resulting in limiting the use or availability of certain applications and/or privileges associated therewith.


Regardless of the reason for privilege revocation, such revocation must be accomplished in a secure manner and in a manner that prevents possible work arounds by malicious applications or individuals. In general, according to current privilege revocation schemes, privilege checking is typically performed on the first access to a privilege. For example, applications communicate with IPC using the application registry. Once an application has a reference (e.g., pointer) to the application registry, it is difficult to take this reference away from the application. In another example, if an application has passed some if its privileges to another application using IPC, conventional systems can detect that the first application has access to IPC, but there is no way to detect that the other application has been passed the privilege(s).


Therefore, there remains a need for a system and method for effectively managing privileges associated with applications, and in particular, when privilege revocation is required, to revoke these privileges in a timely and robust manner.





BRIEF DESCRIPTION OF THE DRAWINGS

These and other embodiments along with their attendant advantages are described herein with reference to the following drawings in which like reference numerals refer to like elements, and wherein:



FIG. 1 is a block diagram showing a computer network and communication system in which electronic devices running applications having access to associated privileges are used;



FIG. 2 is a block diagram of a wireless mobile communication device as an example of an electronic device running applications having access to associated privileges;



FIG. 3 is a flow diagram illustrating a method of revoking privileges according to an exemplary embodiment;



FIG. 4 is a flow diagram illustrating a method of revoking privileges according to another exemplary embodiment; and



FIG. 5 is a flow diagram illustrating yet another method of revoking privileges according to another exemplary embodiment.





DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

In view of the foregoing, we have now identified an efficient, accurate, robust and easy to implement system and method for managing privileges associated with particular applications and for revoking these privileges in a timely and reliable manner


According to an exemplary embodiment an application having privileges that are to be revoked may be shut down. When this application is restarted, access to revoked privileges will be denied. In order to keep track of which applications need to be shut down and reset, the system must keep track of which applications have accessed which privileges. To enable monitoring of the privileges accessed by applications, each time an application uses a privilege, the system records the application identifier and which privilege has been accessed. This may be done in any of a number of conventional methods, such as, for example, a data table listing the application identifier and corresponding accessed privilege(s) associated with the application identifier. When the system administrator, or any other authority, institutes a change in privileges, the system accesses the record of which applications have accessed which privileges. A comparison of accessed privileges with the privileges to be revoked is performed. Each application that has accessed a privilege that is to be revoked is identified by the system and then shut down. When the application is restarted, the application will not have access to any of the revoked privileges.


In another embodiment, a device reset may be performed. By resetting a device, the system is brought to a known state. According to this embodiment, a system administrator, or any other authority, specifies that a device must be reset when a new policy that revokes (or changes) privileges is instituted. A device reset may be instituted at any time the administrator or authority deems it necessary to do so. For example, a device reset may be done whenever a new policy that includes privilege revocation is instituted. However, this may result in numerous unnecessary device reset events that are potentially inconvenient to the user and may interfere with use of the device. Alternatively, the system may keep track of which devices include which applications, and use this information to determine a less intrusive device reset schedule. For example, resetting only those devices having applications that may potentially be affected by the policy change.


In yet another advantageous embodiment, aspects of previously described embodiments are combined to provide effective privilege management and revocation. According to this exemplary embodiment, the system administrator or other authority has no actual control over when a device is reset. The administrator merely manages the privileges of the system and particular applications. The device itself is responsible for resetting when necessary. In this embodiment, the device keeps track of which applications get access to which privileges. When policies or application control changes, the system detects which privileges have been revoked for which applications. This can be accomplished by simply comparing the old set of privileges with the new set of privileges. For each revoked privilege for a given application, the system determines if the application has ever accessed that privilege in the past. As noted above, the system has been keeping track of these since the system was first started up. If an application has accessed a privilege that is now revoked at any time in the past, the device is reset. For some privileges (e.g., IPC), it still cannot be determined whether a privilege has been used as a result of being passed from another application. To overcome the potential for missing a revocation of privileges when applications pass privileges between themselves, if, for example, IPC (or any other privilege that is able to be passed between applications) is revoked from any application, regardless of if the system has detected that the application has accessed the privilege, the device must be reset. This reset will bring the device back to a known state. Device reset will only be done when necessary, thus limiting the number of resets and solving the problem associated with privileges that have been passed between applications.


Each of these embodiments is useful in a variety of privilege management environments ranging from a low-level low-priority where a modest level of privilege revocation security is needed, resulting in a less robust system, to a highly important ultra-robust environment where user inconvenience is secondary to the need for absolute certainty of privilege revocation.



FIG. 1 is a block diagram showing a computer network and communication system in which electronic devices running applications having access to associated privileges are used. The computer network 18 includes, for example, various networked computers 28 and, optionally, a message server 26, all linked via a Local Area Network (LAN) 30. The communication system includes a Wide Area Network (WAN) 12 coupled to a computer system 14, a wireless network gateway 16 and the LAN 30 of the computer network 18. The wireless network gateway 16 is also connected to a wireless communication network 20 in which a wireless mobile communication device 22 (hereinafter “mobile device”), is configured to operate. The entire system 10 is typically managed by, among others, a system administrator or like authority 32.


The computer system 14 may be a desktop or laptop personal computer that is configured to communicate with the WAN 12 or any other suitable network, such as, for example, the Internet. Personal computers, such as the computer system 14, typically access the Internet via an Internet Service Provider (ISP), Application Service Provider (ASP), or the like.


The LAN 30 is an example of a typical working computer network environment, in which multiple computers 28 are connected in a network. The computer network 18 is typically located behind a security firewall 24. Within the LAN 30, a message server 26, operating on a computer behind the firewall 24 may act as the primary interface for the owner of the computer network 18 to exchange messages both within the LAN 30 and with other external messaging clients via the WAN 12. Known message servers include, for example, Microsoft Outlook®, Lotus Notes®, Yahoo!® Messenger®, AOL Instant Messenger®, or any other client-server or peer-to-peer, or similar messaging clients with various architectures. Messages received by the message server 26 are distributed to mailboxes for user accounts addressed in the received messages, and are then accessed by a user through a messaging client operating on a computer system 28. The foregoing is merely an exemplary description illustrating a client-server architecture, and in no way implies that such architecture is necessary, as other suitable architectures known to those skilled in the art may be used.


Although only a message server 26 is shown in the LAN 30, those skilled in the art will appreciate that a LAN may include other types of servers supporting resources that are shared between the networked computer systems 28, and that the message server 26 may also provide additional functionality, such as dynamic database storage for data such as, but not limited to, calendars, to-do lists, task lists, e-mail and documentation. The message server 26 and electronic messaging are described for illustrative purposes only. Systems and methods for managing and revoking privileges are applicable to a wide range of electronic devices, and are in no way limited to electronic devices with messaging capabilities.


The wireless gateway 16 provides an interface to a wireless network 20, through which messages may be exchanged with a mobile device 22. Such functions as addressing of the mobile device 22, encoding or otherwise transforming messages for wireless transmission, and any other interface functions are performed by the wireless gateway 16. The wireless gateway 16 may be configured to operate with more than one wireless network 20, in which case the wireless gateway 16 also determines a most likely network for locating a given mobile device 22 and possibly track mobile devices as users roam between countries or networks.


The mobile device 22 is, for example, a data communication device, a voice communication device, a dual-mode communication device such as many modern cellular telephones having both data and voice communications functionality, a multiple-mode device capable of voice, data and other types of communications, a personal digital assistant (PDA) enabled for wireless communications, or a laptop or desktop computer system with a wireless modem.


Any computer system with access to the WAN 12 may exchange messages with the mobile device 22 through the wireless network gateway 16. Alternatively, private wireless network gateways such as wireless Virtual Private Network (VPN) routers could be implemented to provide a private interface to a wireless network. A wireless VPN router implemented in the LAN 30 provides a private interface from the LAN 30 to one or more mobile devices such as 22 through the wireless network 20. A private interface to a mobile device 22 may also effectively be extended to entities outside the LAN 30 by providing a message forwarding or redirection system that operates with the message server 26. Such a message redirection system is disclosed in U.S. Pat. No. 6,219,694, which is hereby incorporated into this application by reference. In this type of system, incoming messages received by the message server 26 and addressed to a user of a mobile device 22 are sent through the wireless network interface, either a wireless VPN router, the wireless gateway 16, or another interface, for example, to the wireless network 20 and to the user's mobile device 22. Another alternate interface to a user's mailbox on a message server 26 may be a Wireless Application Protocol (WAP) gateway. Through a WAP gateway, a list of messages in a user's mailbox on the message server 26, and possibly each message or a portion of each message, may be sent to the mobile device 22. A wireless network 20 normally delivers messages to and from communication devices such as the mobile device 22 via RF transmissions between base stations and devices. The wireless network 20 may, for example, be a data-centric wireless network, a voice-centric wireless network, or a dual-mode network that can support both voice and data communications over the same infrastructure. Recently developed networks include Code Division Multiple Access (CDMA) networks and General Packet Radio Service (GPRS) networks. So-called third-generation (3G) networks like Enhanced Data rates for Global Evolution (EDGE) and Universal Mobile Telecommunications Systems (UMTS) are currently under development. Older data-centric networks include, but are not limited to, the Mobitex™ Radio Network (“Mobitex”), and the DataTAC™ Radio Network (“DataTAC”). Voice-centric data networks such as Personal Communication System (PCS) networks, including Global System for Mobile Communications (GSM) and Time Division Multiple Access (TDMA) systems, have been available in North America and world-wide for several years.



FIG. 2 is a block diagram of an exemplary wireless mobile communication device as an example of an electronic device. However, it should be understood that the systems and methods disclosed herein may be used with many different types of devices, such as personal digital assistants (PDAs), desktop computers, or the like.


The mobile device 500 is preferably a two-way communication device having at least voice and data communication capabilities. The mobile device 500 preferably has the capability to communicate with other computer systems on the Internet. Depending on the functionality provided by the mobile device, the mobile device may be referred to as a data messaging device, a two-way pager, a cellular telephone with data messaging capabilities, a wireless Internet appliance, or a data communication device (with or without telephony capabilities). As mentioned above, such devices are referred to generally herein as mobile devices.


The mobile device 500 includes a transceiver 511, a microprocessor 538, a display 522, non-volatile memory 524, random access memory (RAM) 526, auxiliary input/output (I/O) devices 528, a serial port 530, a keyboard 532, a speaker 534, a microphone 536, a short-range wireless communications sub-system 540, and may also include other device sub-systems 542. The transceiver 511 preferably includes transmit and receive antennas 516, 518, a receiver (Rx) 512, a transmitter (Tx) 514, one or more local oscillators (LOs) 513, and a digital signal processor (DSP) 520. Within the non-volatile memory 524, the mobile device 500 includes a plurality of software modules 524A-524N that can be executed by the microprocessor 538 (and/or the DSP 520), including a voice communication module 524A, a data communication module 524B, and a plurality of other operational modules 524N for carrying out a plurality of other functions.


The mobile device 500 is preferably a two-way communication device having voice and data communication capabilities. Thus, for example, the mobile device 500 may communicate over a voice network, such as any of the analog or digital cellular networks, and may also communicate over a data network. The voice and data networks are depicted in FIG. 2 by the communication tower 519. These voice and data networks may be separate communication networks using separate infrastructure, such as base stations, network controllers, etc., or they may be integrated into a single wireless network. References to the network 519 should therefore be interpreted as encompassing both a single voice and data network and separate networks.


The communication subsystem 511 is used to communicate with the network 519. The DSP 520 is used to send and receive communication signals to and from the transmitter 514 and receiver 512, and also exchange control information with the transmitter 514 and receiver 512. If the voice and data communications occur at a single frequency, or closely-spaced set of frequencies, then a single LO 513 may be used in conjunction with the transmitter 514 and receiver 512. Alternatively, if different frequencies are utilized for voice communications versus data communications or the mobile device 500 is enabled for communications on more than one network 519, then a plurality of LOs 513 can be used to generate frequencies corresponding to those used in the network 519. Although two antennas 516, 518 are depicted in FIG. 2, the mobile device 500 could be used with a single antenna structure. Information, which includes both voice and data information, is communicated to and from the communication module 511 via a link between the DSP 520 and the microprocessor 538.


The detailed design of the communication subsystem 511, such as frequency band, component selection, power level, etc., is dependent upon the communication network 519 in which the mobile device 500 is intended to operate. For example, a mobile device 500 intended to operate in a North American market may include a communication subsystem 511 designed to operate with the Mobitex or DataTAC mobile data communication networks and also designed to operate with any of a variety of voice communication networks, such as AMPS, TDMA, CDMA, PCS, etc., whereas a mobile device 500 intended for use in Europe may be configured to operate with the GPRS data communication network and the GSM voice communication network. Other types of data and voice networks, both separate and integrated, may also be utilized with the mobile device 500.


Communication network access requirements for the mobile device 500 also vary depending upon the type of network 519. For example, in the Mobitex and DataTAC data networks, mobile devices are registered on the network using a unique identification number associated with each device. In GPRS data networks, however, network access is associated with a subscriber or user of the mobile device 500. A GPRS device typically requires a subscriber identity module (“SIM”), which is required in order to operate the mobile device 500 on a GPRS network. Local or non-network communication functions (if any) may be operable, without the SIM, but the mobile device 500 is unable to carry out functions involving communications over the network 519, other than any legally required operations, such as ‘911’ emergency calling.


After any required network registration or activation procedures have been completed, the mobile device 500 is able to send and receive communication signals, preferably including both voice and data signals, over the network 519. Signals received by the antenna 516 from the communication network 519 are routed to the receiver 512, which provides for signal amplification, frequency down conversion, filtering, channel selection, etc., and may also provide analog to digital conversion. Analog to digital conversion of the received signal allows more complex communication functions, such as digital demodulation and decoding, to be performed using the DSP 520. In a similar manner, signals to be transmitted to the network 519 are processed, including modulation and encoding, for example, by the DSP 520 and are then provided to the transmitter 514 for digital to analog conversion, frequency up conversion, filtering, amplification and transmission to the communication network 519 via the antenna 518. Although a single transceiver 511 is shown for both voice and data communications, in alternative embodiments, the mobile device 500 may include multiple distinct transceivers, such as a first transceiver for transmitting and receiving voice signals, and a second transceiver for transmitting and receiving data signals, or a first transceiver configured to operate within a first frequency band, and a second transceiver configured to operate within a second frequency band.


In addition to processing the communication signals, the DSP 520 also provides for receiver and transmitter control. For example, the gain levels applied to communication signals in the receiver 512 and transmitter 514 may be adaptively controlled through automatic gain control algorithms implemented in the DSP 520. Other transceiver control algorithms could also be implemented in the DSP 520 in order to provide more sophisticated control of the transceiver 511.


The microprocessor 538 preferably manages and controls the overall operation of the mobile device 500. Many types of microprocessors or microcontrollers could be used here, or, alternatively, a single DSP 520 could be used to carry out the functions of the microprocessor 538. Low-level communication functions, including at least data and voice communications, are performed through the DSP 520 in the transceiver 511. High-level communication applications, including the voice communication application 524A, and the data communication application 524B are stored in the non-volatile memory 524 for execution by the microprocessor 538. For example, the voice communication module 524A may provide a high-level user interface operable to transmit and receive voice calls between the mobile device 500 and a plurality of other voice devices via the network 519. Similarly, the data communication module 524B may provide a high-level user interface operable for sending and receiving data, such as e-mail messages, files, organizer information, short text messages, etc., between the mobile device 500 and a plurality of other data devices via the network 519.


The microprocessor 538 also interacts with other device subsystems, such as the display 522, RAM 526, auxiliary I/O devices 528, serial port 530, keyboard 532, speaker 534, microphone 536, a short-range communications subsystem 540 and any other device subsystems generally designated as 542. For example, the modules 524A-N are executed by the microprocessor 538 and may provide a high-level interface between a user of the mobile device and the mobile device. This interface typically includes a graphical component provided through the display 522, and an input/output component provided through the auxiliary I/O devices 528, keyboard 532, speaker 534, or microphone 536. Additionally, the microprocessor 538 is capable of running a variety of applications that may be present in the device non-volatile memory 524, including applications that have access to various privileges, as will be described in more detail herein.


Some of the subsystems shown in FIG. 2 perform communication-related functions, whereas other subsystems may provide “resident” or on-device functions. Notably, some subsystems, such as keyboard 532 and display 522 may be used for both communication-related functions, such as entering a text message for transmission over a data communication network, and device-resident functions such as a calculator or task list or other PDA type functions.


Operating system software used by the microprocessor 538 is preferably stored in a persistent store such as the non-volatile memory 524. In addition to the operating system and communication modules 524A-N, the non-volatile memory 524 may include a file system for storing data. The non-volatile memory 524 may also include data stores for owner information and owner control information. The operating system, specific device applications or modules, or parts thereof, may be temporarily loaded into a volatile store, such as RAM 526 for faster operation. Moreover, received communication signals may also be temporarily stored to RAM 526, before permanently writing them to a file system located in the non-volatile memory 524. The non-volatile memory 524 may be implemented, for example, with Flash memory, non-volatile RAM, or battery backed-up RAM.


An exemplary application module 524N that may be loaded onto the mobile device 500 is a PIM application providing PDA functionality, such as calendar events, appointments, and task items. This module 524N may also interact with the voice communication module 524A for managing phone calls, voice mails, etc., and may also interact with the data communication module 524B for managing e-mail communications and other data transmissions. Alternatively, all of the functionality of the voice communication module 524A and the data communication module 524B may be integrated into the PIM module.


The non-volatile memory 524 preferably provides a file system to facilitate storage of PIM data items on the device. The PIM application preferably includes the ability to send and receive data items, either by itself, or in conjunction with the voice and data communication modules 524A, 524B, via the wireless network 519. The PIM data items are preferably seamlessly integrated, synchronized and updated, via the wireless network 519, with a corresponding set of data items stored or associated with a host computer system, thereby creating a mirrored system for data items associated with a particular user.


The mobile device 500 is manually synchronized with a host system by placing the mobile device 500 in an interface cradle, which couples the serial port 530 of the mobile device 500 to a serial port of the host system. The serial port 530 may also be used to insert owner information and owner control information onto the mobile device 500 and to download other application modules 524N for installation on the mobile device 500. This wired download path may further be used to load an encryption key onto the mobile device 500 for use in secure communications, which is a more secure method than exchanging encryption information via the wireless network 519.


Owner information, owner control information and additional application modules 524N may be loaded onto the mobile device 500 through the network 519, through an auxiliary I/O subsystem 528, through the short-range communications subsystem 540, or through any other suitable subsystem 542, and installed by a user in the non-volatile memory 524 or RAM 526. Such flexibility in application installation increases the functionality of the mobile device 500 and may provide enhanced on-device functions, communication-related functions, or both. For example, secure communication applications may enable electronic commerce functions and other such financial transactions to be performed using the mobile device 500.


When the mobile device 500 is operating in a data communication mode, a received signal, such as a text message or a web page download, will be processed by the transceiver 511 and provided to the microprocessor 538, which preferably further processes the received signal for output to the display 522, or, alternatively, to an auxiliary I/O device 528. Owner information, owner control information, commands or requests related to owner information or owner control information, and software applications received by the transceiver 511 are processed as described above. A user of mobile device 500 may also compose data items, such as email messages, using the keyboard 532, which is preferably a complete alphanumeric keyboard laid out in the QWERTY style, although other styles of complete alphanumeric keyboards such as the known DVORAK style may also be used. User input to the mobile device 500 is further enhanced with the plurality of auxiliary I/O devices 528, which may include a thumbwheel input device, a touchpad, a variety of switches, a rocker input switch, etc. The composed data items input by the user are then transmitted over the communication network 519 via the transceiver 511.


When the mobile device 500 is operating in a voice communication mode, the overall operation of the mobile device 500 is substantially similar to the data mode, except that received signals are output to the speaker 534 and voice signals for transmission are generated by a microphone 536. In addition, the secure messaging techniques described above might not necessarily be applied to voice communications. Alternative voice or audio I/O devices, such as a voice message recording subsystem, may also be implemented on the mobile device 500. Although voice or audio signal output is accomplished through the speaker 534, the display 522 may also be used to provide an indication of the identity of a calling party, the duration of a voice call, or other voice call related information. For example, the microprocessor 538, in conjunction with the voice communication module 524A and the operating system software, may detect the caller identification information of an incoming voice call and display it on the display 522.


A short-range communications subsystem 540 is also included in the mobile device 500. For example, the subsystem 540 may include an infrared device and associated circuits and components, or a Bluetooth or 802.11 short-range wireless communication module to provide for communication with similarly-enabled systems and devices. Thus, owner information insertion, owner control information insertion, and application loading operations as described above may be enabled on the mobile device 500 via the serial port 530 or other short-range communications subsystem 540.



FIG. 2 represents a specific example of an electronic device in which owner control systems and methods described herein may be implemented. Implementation of such systems and methods in other electronic devices having further, fewer, or different components than those shown in FIG. 2 would occur to one skilled in the art to which this application pertains and are therefore considered to be within the scope of the present application.



FIG. 3 is a flow diagram illustrating a method of revoking privileges according to an exemplary embodiment. In this example, an application having privileges that are to be revoked may be shut down. When the application is restarted, access to revoked privileges will be denied. In order to keep track of which applications need to be shut down and reset, the system must keep track of which applications have access to which privileges. To accomplish this, for example, the system monitors and detects use of privileges by applications 300. The system may record an application identifier associated with a particular application and which privilege has been accessed by the application 302. This may be accomplished in any number of conventional methods that are readily apparent to those skilled in the art. For example, a data table listing the application identifier and having pointers directed to privilege(s) accessed by the application or associated with the application identifier. The system then continues to monitor the system for any change, for example, a change in IT policy, that might result in a revocation of privileges 304. So long as no such change is detected in decision block 304, the system continues to monitor and keep track of applications and associated privileges.


Upon a detection of a change that would result in revocation of privileges being accessed by applications in the system by the decision block 304, such as, for example, a change in IT policy, a comparison of the recorded data relating to the accessed privileges with a listing of the new privileges is performed 306. For example, the application identifiers and associated accessed privileges recorded in step 302 are compared to the new privilege list, or to a list of revoked privileges 306. The changes discussed herein are typically instituted and administered by a system administrator or other authority who has responsibility for operation and management of the system.


As a result of the comparison 306, each application that has accessed a privilege(s) to be revoked is identified 308 by the system. Upon identification of these applications, the system implements a shut down of these identified applications 310. When these applications are restarted 312, the applications will not have access to any of the revoked privileges. The system will continue to monitor and detect the accessing of privileges and associated applications as described above.


In another embodiment, as illustrated in the flow diagram of FIG. 4, a device reset may be performed. According to this example, all devices in the system are monitored 400. A system administrator or other authority specifies, for example, that whenever there is a change in system policy that requires revocation or changes in privileges 402, all devices in the system must be reset 404. Resetting the devices brings the system to a known state, i.e., a state in which the system knows which applications have access to which privileges throughout the system. Upon restarting these applications 406 after the device reset 404, the applications will no longer have access to any of the revoked privileges. According to this example, a device reset may be instituted any time the system administrator or authority deems it necessary to do so. For example, a device reset may be done whenever a new policy that includes privilege revocation is instituted. As a result, this solution may invoke numerous (potentially unnecessary) device reset events that may be intrusive and inconvenient for the users. However, this embodiment provides very robust and timely privilege revocation, and is thus suitable to highly secure systems where privilege management is more important than user convenience.


Turning now to FIG. 5, another advantageous embodiment implementing features of both embodiments described above with respect to FIGS. 3 and 4 is illustrated. According to this example, aspects of the previously described embodiments are combined to provide highly effective and timely privilege management and revocation. In this example, the system administrator or authority has no actual control over when a device is reset. The administrator or authority merely manages the privileges of the system and of particular applications. The device itself is responsible for resetting as needed.


In this example, the device monitors which applications of the device have access to which privileges 600, and a log of privileges for the device is kept 602. The device monitors whether policies or application control changes are made in the system 604. If there is no change detected 604, the device continues to monitor applications and keep a log of privileges for the device 600, 602. If a change in policy or application control is detected in step 604, the system determines which privileges have been revoked for which applications by comparing the old set of privileges in the log with the new set of privileges received from the system administrator 606. The device then determines if any revoked privileges are present on the device 608. If revoked privileges are detected, e.g., if an application has accessed a privilege at any time in the past that has now been revoked, the device will reset 610. As described above, resetting the device brings the system to a known state in which all applications and privileges are known. After the device is reset 610, it is restarted 612. Upon restart 612, the device applications will have access to the correct privileges. Advantageously, if no revoked privileges are detected in step 608, the device performs another check to ensure that no privileges that have been passed between application have been missed. As explained above, for some privileges, e.g., IPC, it cannot be determined whether a privilege has been used as a result of being passed from another application. To overcome the potential for missing revocation of privileges when applications pass privileges between themselves, the system checks for privileges that are able to be passed between applications (e.g., IPC) 614. If a privilege that is able to be passed between applications is revoked from any application, regardless of if the system has detected that the application has accessed the privilege 614, the device must be reset 610 to bring the system to a known state. After resetting, the device is restarted 612, and will now have the only have access to the correct privileges. In this manner, device reset will only be performed when necessary, thus limiting the number of resets and solving the problem associated with privileges that go undetected due to their ability to be passed between applications.


While this disclosure describes specific exemplary embodiments, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, the exemplary embodiments described herein, are intended to be illustrative, not limiting. Various changes may be made without departing from the true spirit and full scope of the invention, as defined in the following claims.

Claims
  • 1. A method for managing privileges in a computer system, the method comprising: setting privileges associated with applications capable of running on one or more electronic devices in the system;recording which of the electronic devices in the system have which applications;detecting a revocation of a privilege associated with at least one of the applications by comparing an old set of privileges associated with the applications to a new set of privileges associated with the applications; andresponsive to detecting the revocation of the privilege, automatically instructing those electronic devices having applications to which the privilege is associated to reset themselves.
  • 2. The method as recited by claim 1, wherein the privilege comprises allowing an application to use inter-process communication.
  • 3. The method as recited by claim 1, wherein the privilege comprises enabling opening of internal and external connections.
  • 4. The method as recited by claim 1, wherein the privilege comprises enabling use of communication tools.
  • 5. The method as recited by claim 1, wherein the privilege comprises enabling use of personal information management functionality.
  • 6. The method as recited by claim 1, wherein the privilege comprises enabling the use of an application program interface.
  • 7. A computer system comprising: a processor; anda memory storing instructions which, when executed by the processor, cause the processor to: set privileges associated with applications capable of running on one or more electronic devices in the system;record which of the electronic devices in the system have which applications;detect a revocation of a privilege associated with at least one of the applications by comparing an old set of privileges associated with the applications to a new set of privileges associated with the applications; andresponsive to detecting the revocation of the privilege, automatically instruct those electronic devices having applications to which the privilege is associated to reset themselves.
  • 8. The computer system as recited by claim 7, wherein the privilege comprises allowing an application to use inter-process communication.
  • 9. The computer system as recited by claim 7, wherein the privilege comprises enabling opening of internal and external connections.
  • 10. The computer system as recited by claim 7, wherein the privilege comprises enabling use of communication tools.
  • 11. The computer system as recited by claim 7, wherein the privilege comprises enabling use of personal information management functionality.
  • 12. The computer system as recited by claim 7, wherein the privilege comprises enabling the use of an application program interface.
  • 13. A non-transitory computer-readable medium storing instructions which, when executed by a processor of a computer system, result in: setting privileges associated with applications capable of running on one or more electronic devices in the system;recording which of the electronic devices in the system have which applications;detecting a revocation of a privilege associated with at least one of the applications by comparing an old set of privileges associated with the applications to a new set of privileges associated with the applications; andresponsive to detecting the revocation of the privilege, automatically instructing those electronic devices having applications to which the privilege is associated to reset themselves.
  • 14. The non-transitory computer-readable medium as recited by claim 13, wherein the privilege comprises allowing an application to use inter-process communication.
  • 15. The non-transitory computer-readable medium as recited by claim 13, wherein the privilege comprises enabling opening of internal and external connections.
  • 16. The non-transitory computer-readable medium as recited by claim 13, wherein the privilege comprises enabling use of communication tools.
  • 17. The non-transitory computer-readable medium as recited by claim 13, wherein the privilege comprises enabling use of personal information management functionality.
  • 18. The non-transitory computer-readable medium as recited by claim 13, wherein the privilege comprises enabling the use of an application program interface.
US Referenced Citations (228)
Number Name Date Kind
4815128 Malek Mar 1989 A
4837812 Takahashi et al. Jun 1989 A
4945556 Namekawa Jul 1990 A
4972457 O'Sullivan Nov 1990 A
4991197 Morris Feb 1991 A
5220604 Gasser et al. Jun 1993 A
5327560 Hirata et al. Jul 1994 A
5398233 Balmer et al. Mar 1995 A
5408520 Clark et al. Apr 1995 A
5606594 Register et al. Feb 1997 A
5675782 Montague et al. Oct 1997 A
5774551 Wu et al. Jun 1998 A
5802483 Morris Sep 1998 A
5826265 Van Huben et al. Oct 1998 A
5850515 Lo et al. Dec 1998 A
5864765 Barvesten Jan 1999 A
5881225 Worth Mar 1999 A
5924094 Sutter Jul 1999 A
5987440 O'Neil et al. Nov 1999 A
5987611 Freund Nov 1999 A
6052735 Ulrich et al. Apr 2000 A
6105132 Fritch et al. Aug 2000 A
6125447 Gong Sep 2000 A
6131136 Liebenow et al. Oct 2000 A
6219694 Lazaridis et al. Apr 2001 B1
6233446 Do May 2001 B1
6243756 Whitmire et al. Jun 2001 B1
6253326 Lincke et al. Jun 2001 B1
6285889 Nykanen et al. Sep 2001 B1
6292798 Dockter et al. Sep 2001 B1
6321334 Jerger et al. Nov 2001 B1
6343313 Salesky et al. Jan 2002 B1
6351816 Mueller et al. Feb 2002 B1
6360322 Grawrock Mar 2002 B1
6405202 Britton et al. Jun 2002 B1
6412070 Van Dyke et al. Jun 2002 B1
6490289 Zhang et al. Dec 2002 B1
6516421 Peters Feb 2003 B1
6647388 Numao et al. Nov 2003 B2
6668323 Challener et al. Dec 2003 B1
6745047 Karstens et al. Jun 2004 B1
6748543 Vilhuber Jun 2004 B1
6757821 Akiyama et al. Jun 2004 B1
6772350 Belani et al. Aug 2004 B1
6775536 Geiger et al. Aug 2004 B1
6785810 Lirov et al. Aug 2004 B1
6795688 Plasson et al. Sep 2004 B1
6795967 Evans Sep 2004 B1
6886038 Tabbara et al. Apr 2005 B1
6895502 Fraser May 2005 B1
6901429 Dowling May 2005 B2
6957330 Hughes Oct 2005 B1
6976241 Cruz et al. Dec 2005 B2
6978385 Cheston et al. Dec 2005 B1
6999562 Winick Feb 2006 B2
7076239 Kirkup et al. Jul 2006 B2
7146638 Malcolm Dec 2006 B2
7246374 Simon et al. Jul 2007 B1
7315750 Chou et al. Jan 2008 B2
7330712 Kirkup et al. Feb 2008 B2
7331058 Gladney Feb 2008 B1
7400878 Hassan et al. Jul 2008 B2
7546956 Adams et al. Jun 2009 B2
7574200 Hassan et al. Aug 2009 B2
7603466 Kilian-Kehr et al. Oct 2009 B2
7721087 Dipasquo May 2010 B1
7734284 Adams et al. Jun 2010 B2
7751331 Blair et al. Jul 2010 B1
7765185 Rangadass Jul 2010 B2
7793355 Little et al. Sep 2010 B2
7869789 Hassan et al. Jan 2011 B2
7886053 Newstadt et al. Feb 2011 B1
7921452 Ridlon et al. Apr 2011 B2
8074078 Brown et al. Dec 2011 B2
8208900 Adler et al. Jun 2012 B2
8495700 Shahbazi Jul 2013 B2
8495731 Mar et al. Jul 2013 B1
8503340 Xu Aug 2013 B1
8620297 Kirkup et al. Dec 2013 B2
20010047485 Brown et al. Nov 2001 A1
20010056549 Pinault et al. Dec 2001 A1
20020001288 Fukunaga et al. Jan 2002 A1
20020013815 Obradovich et al. Jan 2002 A1
20020019944 Kou Feb 2002 A1
20020031230 Sweet et al. Mar 2002 A1
20020087880 Rhoades Jul 2002 A1
20020095414 Barnett et al. Jul 2002 A1
20020095497 Satagopan et al. Jul 2002 A1
20020107943 Heath et al. Aug 2002 A1
20020112155 Martherus et al. Aug 2002 A1
20020184398 Orenshteyn Dec 2002 A1
20030005317 Audebert et al. Jan 2003 A1
20030014521 Elson et al. Jan 2003 A1
20030026220 Uhlik et al. Feb 2003 A1
20030031184 Cunetto et al. Feb 2003 A1
20030035397 Haller et al. Feb 2003 A1
20030037237 Abgrall et al. Feb 2003 A1
20030054860 Chen Mar 2003 A1
20030061087 Srimuang Mar 2003 A1
20030065676 Gbadegesin et al. Apr 2003 A1
20030070091 Loveland Apr 2003 A1
20030087629 Juitt et al. May 2003 A1
20030093698 Challener et al. May 2003 A1
20030120948 Schmidt et al. Jun 2003 A1
20030126437 Wheeler et al. Jul 2003 A1
20030163685 Paatero Aug 2003 A1
20030167405 Freund et al. Sep 2003 A1
20030177389 Albert et al. Sep 2003 A1
20030226015 Neufeld et al. Dec 2003 A1
20030233410 Gusler et al. Dec 2003 A1
20030236983 Mihm, Jr. Dec 2003 A1
20040001101 Trajkovic et al. Jan 2004 A1
20040083382 Markham et al. Apr 2004 A1
20040097217 McClain May 2004 A1
20040098610 Hraster May 2004 A1
20040100983 Suzuki May 2004 A1
20040121802 Kim et al. Jun 2004 A1
20040132449 Kowarzch Jul 2004 A1
20040139206 Claudatos et al. Jul 2004 A1
20040177073 Snyder et al. Sep 2004 A1
20040187029 Ting Sep 2004 A1
20040193738 Natu et al. Sep 2004 A1
20040199606 Brown et al. Oct 2004 A1
20040209608 Kouznetsov et al. Oct 2004 A1
20040260710 Marston et al. Dec 2004 A1
20050154935 Jin Jul 2005 A1
20050164687 Difazio Jul 2005 A1
20050182832 Chen Aug 2005 A1
20050192008 Desai et al. Sep 2005 A1
20050193080 Gold et al. Sep 2005 A1
20050210270 Rohatgi et al. Sep 2005 A1
20050213763 Owen et al. Sep 2005 A1
20050245272 Spaur et al. Nov 2005 A1
20050288903 Jackson et al. Dec 2005 A1
20060059556 Royer Mar 2006 A1
20060070114 Wood et al. Mar 2006 A1
20060090136 Miller et al. Apr 2006 A1
20060120526 Boucher et al. Jun 2006 A1
20060129848 Paksoy et al. Jun 2006 A1
20060129948 Hamzy et al. Jun 2006 A1
20060141985 Patel et al. Jun 2006 A1
20060128259 Spilotro et al. Jul 2006 A1
20060149846 Schuba Jul 2006 A1
20060156026 Utin Jul 2006 A1
20060168395 Deng et al. Jul 2006 A1
20060206931 Dillaway et al. Sep 2006 A1
20060212589 Hayer et al. Sep 2006 A1
20060274750 Babbar et al. Dec 2006 A1
20070019643 Shaheen Jan 2007 A1
20070050854 Cooperstein et al. Mar 2007 A1
20070073694 Picault et al. Mar 2007 A1
20070156766 Hoang et al. Jul 2007 A1
20070204153 Tome et al. Aug 2007 A1
20070204166 Tome et al. Aug 2007 A1
20070254631 Spooner Nov 2007 A1
20070277127 Carlson et al. Nov 2007 A1
20080002726 Haung et al. Jan 2008 A1
20080081609 Burgan et al. Apr 2008 A1
20080098237 Dung et al. Apr 2008 A1
20080125146 Bainbridge May 2008 A1
20080130524 Volach et al. Jun 2008 A1
20080132202 Kirkup et al. Jun 2008 A1
20080134347 Goyal et al. Jun 2008 A1
20080137593 Laudermilch et al. Jun 2008 A1
20080222711 Michaelis Sep 2008 A1
20080235041 Cashdollar et al. Sep 2008 A1
20080318616 Chipalkatti et al. Dec 2008 A1
20090031393 Denner et al. Jan 2009 A1
20090070181 Loeffen Mar 2009 A1
20090094668 Corbin et al. Apr 2009 A1
20090178107 Karjoth et al. Jul 2009 A1
20090181662 Fleischman et al. Jul 2009 A1
20090254753 De Atley et al. Oct 2009 A1
20090260052 Bathula et al. Oct 2009 A1
20100024016 Violleau et al. Jan 2010 A1
20100024020 Baugher et al. Jan 2010 A1
20100081417 Hickie Apr 2010 A1
20100088753 Ayres et al. Apr 2010 A1
20100107215 Bechtel et al. Apr 2010 A1
20100153969 Dyba et al. Jun 2010 A1
20100175104 Khalid et al. Jul 2010 A1
20100184440 Mao et al. Jul 2010 A1
20100222097 Gisby et al. Sep 2010 A1
20100242086 Adams et al. Sep 2010 A1
20100278162 Groux et al. Nov 2010 A1
20100281487 Schneider et al. Nov 2010 A1
20100299376 Batchu Nov 2010 A1
20100299394 Jania et al. Nov 2010 A1
20100319053 Gharabally Dec 2010 A1
20100325221 Cohen et al. Dec 2010 A1
20100325430 Denninghoff Dec 2010 A1
20100325710 Etchegoyen Dec 2010 A1
20110010699 Cooper et al. Jan 2011 A1
20110082808 Beykpour et al. Apr 2011 A1
20110099605 Cha et al. Apr 2011 A1
20110126214 O'Farrell et al. May 2011 A1
20110145833 De Los Reyes et al. Jun 2011 A1
20110179083 Galloway et al. Jul 2011 A1
20110195698 Pearce Aug 2011 A1
20110210171 Brown et al. Sep 2011 A1
20110252240 Freedman et al. Oct 2011 A1
20110270963 Saito et al. Nov 2011 A1
20110314467 Pearson Dec 2011 A1
20120005477 Wei et al. Jan 2012 A1
20120005723 Chaturvedi et al. Jan 2012 A1
20120023573 Shi Jan 2012 A1
20120054853 Gupta et al. Mar 2012 A1
20120079110 Brown et al. Mar 2012 A1
20120079586 Brown et al. Mar 2012 A1
20120079609 Bender et al. Mar 2012 A1
20120109826 Kobres May 2012 A1
20120124640 Bender et al. May 2012 A1
20120144196 Owen et al. Jun 2012 A1
20120202527 Obradovich et al. Aug 2012 A1
20120210443 Blaisdell et al. Aug 2012 A1
20120214442 Crawford et al. Aug 2012 A1
20120214503 Liu et al. Aug 2012 A1
20120278863 Wallace et al. Nov 2012 A1
20120278904 Perez et al. Nov 2012 A1
20120291140 Robert et al. Nov 2012 A1
20120309344 Ferrazzini et al. Dec 2012 A1
20120324067 Hari et al. Dec 2012 A1
20130074142 Brennan et al. Mar 2013 A1
20130097657 Cardamore et al. Apr 2013 A1
20130125198 Ferguson et al. May 2013 A1
20130174222 Ogle Jul 2013 A1
20130219465 Tse et al. Aug 2013 A1
20130346606 Ryerson et al. Dec 2013 A1
Foreign Referenced Citations (41)
Number Date Country
2505343 Jun 2003 CA
2547708 Jul 2010 CA
1897543 Jan 2007 CN
100466547 Mar 2009 CN
101523878 Sep 2009 CN
0605106 Jul 1994 EP
0973350 Jan 2000 EP
1043648 Oct 2000 EP
1168141 Jan 2002 EP
1471691 Oct 2004 EP
1596410 Nov 2005 EP
1624428 Feb 2006 EP
1806674 Jul 2007 EP
1739518 Oct 2007 EP
1868135 Dec 2007 EP
1563663 Oct 2008 EP
2337300 Jun 2011 EP
2378780 Feb 2003 GB
2408179 May 2005 GB
2440015 Jan 2008 GB
2000253241 Sep 2000 JP
200177811 Mar 2001 JP
2001203761 Jul 2001 JP
2002288087 Oct 2002 JP
128561 Nov 2008 SG
9625828 Aug 1996 WO
9905814 Feb 1999 WO
0059225 Oct 2000 WO
0060434 Oct 2000 WO
2004017592 Feb 2004 WO
2004043031 May 2004 WO
2005045550 May 2005 WO
2006130807 Dec 2006 WO
2007048251 May 2007 WO
2009012329 Jan 2009 WO
2009014975 Jan 2009 WO
2009021200 Feb 2009 WO
2012037656 Mar 2012 WO
2012037658 Mar 2012 WO
2013037657 Mar 2012 WO
2012109497 Aug 2012 WO
Non-Patent Literature Citations (33)
Entry
Providing secure remote access to legacy healthcare applications. Young et al. Computing and Control Engineering Journal. (Year: 2001).
A Technical Overview of the Lucent VPN Firewall White Paper Lucent Technologies, XX, XX, Aug. 2002, pp. 1-35, XP002271173, Chapter 1.
“Customizing User Interaction in Smart Phones”, Pervasive Computing, IEEE CS (2006) pp. 81-90, retrieved from http://www.idi.ntnu.no/grup-perlsu/bibliography/pdf/2006/Korpipaa2006pc.pdf.
“Secure Inter-Process Communication”, Apr. 4, 2004. Retrieved from https://web.archive.org/web/20040404015137/http://cr.yp.to/docs/secureipc.html on Jan. 20, 2014.
“Using Software Restriction Policies in Windows XP and Windows, NET server to Protect Against Unauthorized Software”, XP0029559638, Microsoft XP and Windows Net Technical Article, p. 1-52, Jan. 2002.
Basic Access Authentication, Retrieved from http://en.wikipedia.org/wiki/Basic_access_authentication on Jan. 23, 2010.
Cross-site request forgery, retrieved from http://en.wikipedia.org/wiki/cross-site_request_forgery, on Nov. 30, 2008.
Digest Access Authentication, Retrieved from http;//en.wikipedia.org/wiki/Digest_access_authentication on Dec. 23, 2009.
Introduction to using IRM for e-mail messages; Support/Outlook/Outlook 2007 Help and How-to. http://office.microsoft.com/en-us/outlook-help/introduction-to-using-irm-for-a-email-message-HA0100366.aspx?CTT=5 . . . pp. 1-6, retrieved on Feb 12, 2010.
Send an email message with restricted permission by using IRM; Support/outlook/Outlook2007 Help and How-to http://office.microsoft.com/en-us/outlook-help/send-an-e-mail-message-with-restricted-permission-by-using-irm-HA01024780 . . . p. 1-4, retrieved Feb. 12, 2010.
Smartphone Security Beyond Lock and Wipe, Retrieved from http://www.enterprisemobiletoday.com/article.php/3887006 on Jun. 10, 2010.
View messages with restricted permission sent by using IRM; Support/Outlook/Outlook2007 Help and How-to. http://office.microsoft.com/en-us/outlook-help/view-messages-with-restricted-permission-sent-by-using-irm-HA01246115.as . . . , pp. 1-2 retrieved Feb. 12, 2010.
Windows 7 Product Guide; Microsoft Corp. 2009; published in 2009; 140 pages.
Bellavista, et al., “Dynamic Binding in Mobile Applications a Middleware Approach”, IEEE Internet Computing, Mar.-Apr. 2003.
Chakravorty, “Architecture and Implementation of a Remote Management Framework for Dynamically Reconfigurable Device”, ICON, 2002.
Chen, “Java Card Technology for Smart Cards: Architecture and Programmer's Guide”; “Applet Firewall and Object Sharing;” XP002167366; Retrieved from http://developer.java.sun.com/developer/books/consumerproducts/javacard/ch09.pdf on Jun. 2, 2000.
Google Inc., Android 2.3.4 User's Guide; May 20, 2011; 384 pages.
Gupta, et al., “Using context-profiling to aid access control decisions in mobile devices”, Nokia Research Center (http://www.cerias.purdue.edu/assets.symposium/2011-posters1372-C24.pdf) 1 page., 2011.
Hauswirth, et al., “A Secure Execution Framework for Java”. ACM (2000).
Hedbom, et al., “Analysis of the Security of Windows NT”, XP002364916, Chalmers University of Technology, Goteberg, Sweden, p. 1-97, Mar. 1, 1999.
IETF RFC 3530, “Network File System (NFS) Version 4 Protocol”, Apr. 2003.
Microsoft Corp., Microsoft Outlook 2010; Released Jul. 15, 2010; 27 pages.
Microsoft Office, Microsoft Outlook 2010 Product Guide; Microsoft Corp. 2010; published in 2010; 65 pages.
Red Hat, “Red Hat Linux 7.2—The Official Red Hat Linux Reference Guide” Red Hat Linux Manuals, ‘Online!’ Oct. 22, 2001, XP002276029, pp. 145-155.
Research in Motion, “BlackBerry Bridge App 2.1 and BlackBerry PlayBook Tablet 2.1, Security Technical Overview”; Version: 2.1; Sep. 17, 2012; 90 pages.
Research in Motion, “BlackBerry Device Service 6.1 and BlackBerry PlayBook Tablet 2.1, Security Technical Overview”, Version: 6.1 Sep. 17, 2012; 90 pages.
Seifert, “Supporting Mobile Privacy and Security through Sensor-Based Context Detection” Second Inter'l Workshop on Security & Privacy in Spontaneous Interaction & Mobile Phone Use May 2010 http://www.medien.ifi.lmu.de/iwssi2010/papers/iwssi=spmu2010-seifert.pdf.
Sygate, “Sygate Personal Firewall PRO User Guide” Sygate Personal Firewall Pro User Guide, XX, XX, 2001, pp. 1-77 XP002248366.
Yang, et al., EagleVision; A pervasive Mobile Device Protection System, Iowa State University http://www.cs.iastate.edu/-wzhang/papers/eagleVision.pdf, in Proc. MobiQuitous'09, Jul. 13-16, 2009.
“Microsoft Windows 2000 Server Resource Kit”, Microsoft Corporation, 2000.
“Microsoft Internet Explorer Resource Kit”, 2000.
“Microsoft Windows 2000 Server Deployment Planning Guide”, 2000.
“Windows 2000 Server Resource Kit Group Policy”, 2000.
Related Publications (1)
Number Date Country
20170329940 A1 Nov 2017 US
Divisions (2)
Number Date Country
Parent 15063019 Mar 2016 US
Child 15664497 US
Parent 13960214 Aug 2013 US
Child 15063019 US
Continuations (2)
Number Date Country
Parent 12561370 Sep 2009 US
Child 13960214 US
Parent 11169302 Jun 2005 US
Child 12561370 US