Populating property values of event objects of an object-centric data model using image metadata

Abstract
A mobile data analysis system is provided that enables mobile device location tracking, secure messaging, and real-time sharing of intelligence information, among other features. In one embodiment, a method and apparatus is provided for creating data objects from one or more digital images captured by a mobile device. A computing device receives a first digital image file comprising first image metadata, wherein the first image metadata includes one or more image properties each having an image property value. The computing device transforms one or more of the image property values of the one or more image properties into one or more particular values. The computing device populates one or more data object property values of a data object with the one or more particular values. The computing device stores the data object in a data repository.
Description
TECHNICAL FIELD

The present disclosure generally relates to data analysis. The disclosure relates more specifically to a data analysis system that includes one or more mobile devices.


BACKGROUND

The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.


Many organizations frequently conduct operations that include organization members performing activities in a dispersed geographic area. For example, the operations of a law enforcement agency typically include police officers patrolling assigned geographic areas, responding to crime scenes, and interviewing suspects and witnesses. As another example, a disaster relief organization may respond to a natural disaster by sending out aid workers to a disaster area to locate and provide assistance to those in crisis. These types of operations may be referred to as field operations and may generally include monitoring specific geographic areas and subjects, interacting with persons of interest, responding to and reporting information about the occurrence of notable events, and any other activities that an organization member may perform in the field.


In order to better coordinate field operations, an organization may employ one or more other organization members at a centralized location, referred to herein as operations analysts, that help coordinate the activities of the organization members in the field, referred to herein as field analysts. For example, operations analysts may be responsible for instructing field analysts on particular locations to investigate or subjects to monitor. Similarly, field analysts may be expected to communicate certain information related to the field operations back to operations analysts.


Both field analysts and operations analysts face a number of challenges in efficiently conducting field operations. These challenges include enabling field analysts to maintain a situational awareness of the environment in which the field analysts are operating, including maintaining an awareness of the location and activities of other field analysts. Additionally, field analysts typically lack efficient access to information that may have been previously collected and shared by other field analysts and to real-time updates of such shared information. Operations analysts similarly lack ways of maintaining a meaningful awareness of the activities of a possibly large number of field analysts for whom the operation analysts are responsible and sharing detailed information with those field analysts.





BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings:



FIG. 1 illustrates an example networked computer system in accordance with an embodiment.



FIG. 2 illustrates an example mobile device system in accordance with an embodiment.



FIG. 3 illustrates an example operations center system in accordance with an embodiment.



FIG. 4 illustrates a process flow for generating one or more mobile device teams and mobile device team visibility settings.



FIG. 5 illustrates an example graphical user interface of a mobile device that is configured to display a location of one or more mobile devices.



FIG. 6 illustrates an example graphical user interface of an analyst workstation that is configured to display a location of one or more mobile devices.



FIG. 7 illustrates an example graphical user interface of a mobile device that is configured to enable messaging between field analysts and between field analysts and operations analysts.



FIG. 8A, FIG. 8B illustrate example graphical user interfaces of a mobile device that are configured to enable a user to capture multimedia content on a mobile device and to send and receive multimedia content in messages.



FIG. 9 illustrates a process flow for creating a data object from a digital image.



FIG. 10 illustrates an example graphical user interface that is configured to obtain user selection one or more digital images and enable a user to create one or more data objects from the selected digital images.



FIG. 11 illustrates an example graphical user interface that is configured to enable a user to modify information associated with a data object.



FIG. 12 illustrates an example graphical user interface of a mobile device that is configured to enable a user to submit geosearches.



FIG. 13 illustrates a computer system upon which an embodiment may be implemented.





DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention. Embodiments are described herein according to the following outline:


1.0 General Overview


2.0 Structural Overview


3.0 Architectural and Functional Overview

    • 3.1 Mobile Device System Architecture
    • 3.2 Operations Center System Architecture
    • 3.3 Configuring Mobile Device Teams
    • 3.4 Mobile Device Tracking
    • 3.5 Mobile Device Messaging
    • 3.6 Creating Data Objects from Images
    • 3.7 Mobile Device Search


4.0 Implementation Mechanisms—Hardware Overview


1.0 General Overview


The appended claims may serve as a summary of the invention.


According to various embodiments, a mobile data analysis system and methods are provided that enable mobile device location tracking, secure messaging, real-time data access and analysis, and other features described herein.


In an embodiment, a mobile data analysis system may comprise one or more mobile device user accounts. For example, mobile device user accounts may be created for field analysts within an organization and used by the field analysts to authenticate with the mobile data analysis system using a mobile or other computing device. In an embodiment, mobile device user accounts may be grouped into one or more mobile device teams. Mobile device user accounts may be grouped into mobile device teams based on organizational roles, areas of responsibility, or any other characteristics. In an embodiment, mobile device teams may be associated with visibility settings that control user access to information associated with mobile device user accounts of particular mobile device teams.


In an embodiment, systems and methods facilitate location tracking of mobile devices. For example, mobile devices of the mobile data analysis system may be configured to periodically report location data associated with the mobile devices to a centralized server. Field analysts using one or more mobile devices, and operations analysts using one or more workstations, may then use one or more graphical user interfaces to display the location of tracked mobile devices based on the location data sent from the mobile devices. In an embodiment, the display of tracked mobile devices in a graphical user interface for a particular user may be based on any configured mobile device teams and any associated visibility settings.


In an embodiment, systems and methods facilitate the exchange of messages between field analysts using mobile devices, and between field analysts and operations analysts using analyst workstations. Messages may be exchanged using one or more graphical user interfaces and may generally comprise any combination of text, multimedia objects, location data, data objects, and other information. In various embodiments, users may perform additional operations and data transformations based on information included in received messages.


In another embodiment, systems and methods facilitate the formulation of search requests by a user using a mobile device. In general, search requests may include requests for data objects or other information stored in the mobile data analysis system, including information generated by other users. In one embodiment, mobile device users may use one or more graphical user interfaces to formulate geosearch requests. In general, a geosearch request is a search request for data objects or other information that is associated with a user-specified geographic location or area.


According to some embodiments, systems and methods facilitate the creation of data objects from received digital images. In one embodiment, a computing device receives a first digital image file comprising first image metadata, wherein the first image metadata includes one or more image properties each having an image property value. The computing device transforms one or more of the image property values of the one or more image properties into one or more particular values. The computing device populates one or more data object property values of a data object with the one or more particular values. The computing device stores the data object in a data repository.


Other embodiments include, without limitation, a non-transitory computer-readable medium that includes processor-executable instructions that enable a processing unit to implement one or more aspects of the disclosed methods as well as a system configured to implement one or more aspects of the disclosed methods.


2.0 Structural Overview



FIG. 1 illustrates an example of a mobile data analysis system 100 in accordance with an embodiment. A mobile data analysis system 100 generally facilitates the communication and the exchange of data between one or more mobile devices (e.g., mobile devices 106), one or more analyst workstations (e.g., analyst workstations 114), and information stored in one or more data repositories (e.g., data repositories 110, 118). The example mobile data analysis system 100 is conceptually described herein as comprising a mobile device system 102 supporting one or more mobile devices 106 and that is interconnected over a network 112 to an operations center system 104 supporting one or more analyst workstations 114 and other computing resources; however, the mobile data analysis system 100 represents just one system arrangement and other system arrangements are possible.


In an embodiment, a mobile device system 102 comprises mobile devices 106, mobile device server 108, and data repository 110. Each of mobile devices 106 generally may comprise any mobile computing device including, without limitation, a smartphone, a cellular phone, a tablet, a laptop, and a personal digital assistant (PDA). Each of mobile devices 106 is communicatively coupled to mobile device server 108 via one or more wireless links that may include, for example, cellular, Wi-Fi, WiMAX, ZigBee, microwave, and other wireless network links. For the purposes of illustrating a clear example, four mobile devices 106 and one mobile device server 108 are shown in FIG. 1, but practical implementations may use hundreds or thousands of mobile devices and any number of mobile devices servers.


In an embodiment, mobile device server 108 may be communicatively coupled to resources of operations center system 104 via network 112, which broadly represents one or more local area networks, wide area networks, global interconnected internetworks such as the public internet, or a combination thereof. Mobile device server 108 generally may be configured to coordinate communication between mobile devices 106 and resources of operations center system 104 and to access and retrieve data stored in data repository 110. For example, mobile device server 108 may be configured to relay search requests, messages, and other data sent from mobile devices 106 to resources of operations center system 104, and to send information received from operations center system 104 to the appropriate mobile devices 106.


In an embodiment, operations center system 104 comprises one or more analyst workstations 114, application server 116, data repository 118, and web server 120. One or more components of operations center system 104 may, for example, be located in a centralized location that is remote from mobile device system 102 and mobile devices 106.


In an embodiment, analyst workstations 114 comprise one or more workstation computers, server computers, laptop computers, mobile devices, or combinations thereof. Analyst workstations 114 generally are configured to support one or more operations analysts that may request information provided by application server 116 and/or web server 120, send information to application server 116 to be stored in data repository 118, communicate with one or more field analysts using mobile devices 106, and perform other operations described herein.


In an embodiment, application server 116 generally is configured to access and retrieve data stored in data repository 118 in response to requests from mobile devices 106, mobile device server 108, analyst workstations 114, and web server 120. Application server 116 may perform data manipulations and other operations in response to receiving requests to access and/or store data in data repository 118.


3.0 Architectural and Functional Overview


3.1 Mobile Device System Architecture



FIG. 2 illustrates an example mobile device system architecture 200. In an embodiment, a mobile device system architecture 200 comprises one or more mobile devices 106, mobile device server 108, and data repository 110.


In the embodiment illustrated in FIG. 2, a mobile device 106, which may be implemented by one or more physical computing devices, is communicatively coupled to mobile device server 108, which may be implemented by one or more second physical computing devices, over one or more wireless networks. A mobile device 106 comprises a display 202, graphical user interface (GUI) logic 204, mobile application logic 206, location sensors 208, and camera 210.


In an embodiment, GUI logic 204 may be a set of program instructions which, when executed by one or more processors of a mobile device 106, are operable to receive user input and to display a graphical representation of one or more graphic constructs related to the mobile data analysis system approaches described herein. As an example, a mobile device 106 may be a smartphone and GUI logic 204 may be operable to receive touch screen signals and other user input from, and display the graphics constructs to, a graphical user interface that is provided on display 202. Touch screen signals may comprise selecting buttons, holding down buttons, selecting items displayed on the screen, dragging, or other gestures or selections. In general, GUI logic 204 is configured to receive user input and determine what user requests or commands are represented by the user input.


In an embodiment, a mobile device 106 includes mobile application logic 206, which may comprise firmware, hardware, software, or a combination thereof in various embodiments that is configured to implement the functions of a mobile data analysis system on a mobile device as described herein. In one embodiment, mobile application logic 206 may be implemented as part of an application program configured to execute on the Android operating system. In other embodiments, mobile application logic 206 may be implemented as a combination of programming instructions written in any programming language (e.g., C++ or Java) and hardware components (e.g. memory, CPU time) that have been allocated for executing the program instructions on a mobile device 106.


In an embodiment, location sensors 208 generally represent any sensors which may be used to determine information associated with a geographic location, spatial orientation, device movement, or any other information associated with the physical presence of a mobile device 106, and which may be referred to herein as location data. Location data may include, for example, latitudinal and longitudinal coordinates, elevation measures, cardinal direction information, movement information, etc. For example, location sensors 208 may comprise a Global Positioning System (GPS) component, motion sensors (e.g., an accelerometer), rotation sensors (e.g., a gyroscope), a compass, and a magnetometer. In an embodiment, mobile application logic 206 is operable to receive location data from location sensors 208 and to send the location data to a mobile device server 108, which in turn may send the location data to resources of operations center system 104. The receiving and sending of location data by mobile application logic 206 may be performed periodically, at user configured intervals, or based on any other schedule.


In an embodiment, camera 210 generally represents any component capable of capturing multimedia information such as images, video, and sound. Camera 210 may be integrated into a mobile device 106 or may be an external device communicatively coupled to a mobile device 106.


In an embodiment, mobile device server 108 comprises mobile device security logic 212 and repository access logic 214.


In an embodiment, mobile device security logic 212 provides processes for controlling access to the mobile data analysis system by mobile devices. For example, access by mobile devices 106 to mobile device server 108, and via mobile device server 108 to resources of operations center system 104 over network 112, may be restricted and/or secured. As such, access by a mobile device user to a mobile device 106 and/or mobile device server 108 may be based on the user supplying an authorized mobile device user account and associated passwords, secret questions, personal identification numbers (PINs), biometrics, and/or any other suitable authentication mechanism. Mobile device security logic 212 comprise a set of program instructions configured to process mobile device user login requests sent from a mobile device 106.


In one embodiment, user access to a mobile device 106, mobile device server 108, and one or more of the resources of operations center system 104 may be protected by separate authentication mechanisms. In another embodiment, mobile device security logic 212 may be configured to implement a Single Sign-On (SSO) access control system in order to provide a single point of authentication for mobile device users. An SSO access control system generally enables a system resource, such as mobile device server 108, to process access credentials supplied by a mobile device user and, if a successful login occurs, to grant an authenticated user account access to resources located on other system resources, such as the resources of operations center system 104, and without the mobile user manually authenticating with the other systems.


In an embodiment, communication between mobile devices 106, mobile device server 108, and resources in operations center system 104 may be secured using a cryptographic communication protocol such as, for example, the Secure Sockets Layer (SSL) protocol. For example, each of mobile devices 106 may be configured for secure communications by installing a public key security certificate on the mobile devices and a corresponding private key security certificate on mobile device server 108 and resources of operations center system 104. Mobile device security logic 212 may comprise instructions configured to send and receive encrypted network traffic based on the installed security certificates, whereby the mobile device security logic 212 encodes outgoing data with the public key security certificate, and mobile devices server 108 and/or resources of operations center system 104 decode received data with the installed private key security certificates.


In an embodiment, mobile device security logic 212 may comprise program instructions configured to restrict mobile device access to mobile device server 108 based on a whitelist of authorized mobile devices. A mobile device whitelist may be configured by a mobile data analysis system administrator and may include, for example, one or more entries that specify a unique identifier associated with approved mobile devices. The unique identifier may be, for example, a device serial number, an international mobile equipment identity (IMEI) number, a MAC address, or any other identifier that may be transmitted by a mobile device 106 to mobile device server 108. In an embodiment, mobile device security logic 212 may be configured to cause mobile device server 108 to ignore requests that are sent from a mobile device that does not supply an identifier on the whitelist. A mobile device whitelist may be stored in a database, a spreadsheet, or any other suitable format for storage in a data repository such as data repository 110.


In an embodiment, mobile device server 108 comprises repository access logic 214. Repository access logic 214 may comprise a set of instructions which, when executed by one or more processors, are operable to access and retrieve data from data repository 118. For example, repository access logic may be a database client or an Open Database Connectivity (ODBC) client that supports calls to a database server that manages data repository 118.


In an embodiment, data repository 110 generally represents any data storage device (e.g., local memory on mobile device server 108, shared memory, a database, etc.) known in the art which may be configured to store data. In an embodiment, data repository 110 may store, for example, configuration files, security information, and other data associated with mobile devices 106. In some embodiments, data stored in data repository 110 may be accessed by mobile device server 108 in order to avoid sending requests for the same information to resources of operations center system 104.


3.2 Operations Center System Architecture



FIG. 3 illustrates an example operations center architecture 300. In an embodiment, operations center architecture 300 comprises application server 116, web server 120, and one or more analyst workstations, such as analyst workstation 114.


In the embodiment illustrated in FIG. 3, analyst workstation 114, which may be implemented by one or more physical computing devices, is communicatively connected to application server 116 and web server 120, which may be implemented by one or more other physical computing devices, over a network. In some embodiments, each such physical computing device may be implemented as a separate computer system. For example, analyst workstation 114 may be implemented in a computer system as a set of program instructions recorded on a machine-readable storage medium, while application server 116 and web server 120 may be implemented in different computer systems.


Analyst workstation 114 comprises graphical user interface (GUI) logic 304. GUI logic 304 may be a set of program instructions which, when executed by one or more processors of the computer system, are operable to receive user input and display a graphical representation of one or more graphic constructs related to the mobile data analysis approaches described herein. GUI logic 304 may be operable to receive user input from, and display the graphic constructs to, a graphical user interface that is provided on display 302 by the computer system on which analyst workstation 114 executes.


Analyst workstation 114 may also interact with application server 116 to provide input, definition, editing instructions, and expressions related to a mobile data analysis system as described herein using a programmatic interface, and then the application server 116 may use, process, log, store, or otherwise interact with the received input according to application server logic.


In an embodiment, web server 120 is configured to provide one or more web-based interfaces to resources available from application server 116 and data repository 118. As an example, one or more of mobile devices 106 may comprise a browser that can access HTML documents that web server 120 generates. The web pages may include information about data stored in data repository 118. In other embodiments, web server 120 may use formats other than HTML for transmitting information to requesting devices.


In an embodiment, application server 116 may be implemented as a special-purpose computer system having the logical elements shown in FIG. 3. In an embodiment, the logical elements may comprise program instructions recorded on one or more machine-readable storage media. Alternatively, the logical elements may be implemented in hardware, firmware, or a combination.


When executed by one or more processors of the computer system, logic in application server 116 is operable to perform mobile data analysis system operations according to the techniques described herein. In one embodiment, logic in application server 116 may be implemented in a Java Virtual Machine (JVM) that is executing in a distributed or non-distributed computer system. In other embodiments, logic in application server 116 may be implemented as a combination of programming instructions written in any programming language (e.g., C++ or Visual Basic) and hardware components (e.g., memory, CPU time) that have been allocated for executing the program instructions.


In an embodiment, application server 116 comprises repository access logic 314. Repository access logic 314 may comprise a set of instructions which, when executed by one or more processors, are operable to access and retrieve data from data repository 118.


In an embodiment, data repository 118 may be a type of structured storage for storing data including, but not limited to, relational or object-oriented databases, data warehouses, directories, data files, and any other structured data storage. In one embodiment, data repository 118 is implemented as a revisioning database system configured to track changes made to data stored in the data repository. In an embodiment, a revisioning database system records metadata about changes to stored data, facilitates UNDO and REDO operations of data changes, can receive requests to subscribe to particular data and publish updates to such data for delivery to subscribers, and perform other functions.


In an embodiment, data stored in data repository 118 is conceptually structured according to an object-centric data model, the data model consisting of a collection of data objects. For example, a data object in the data model may represent an entity such as a person, a place, an organization, an event, a document, or a digital media item such as audio or video. A data object may have a type (e.g., Person, Event, Organization) and include any number of data property fields and corresponding data property values. For example, Event data objects may have data property fields for storing information associated with a particular events represented by the data objects such as, for example, a date and time of an event, a location of an event, etc.


In one embodiment, data objects in the data model may be represented as a data object graph consisting of nodes and edges. The nodes of the graph may represent data objects and the edges may represent relationships or other links between data objects. For example, a particular person, represented by a Person data object, may be known to have an affiliation with a particular organization, represented by an Organization data object. The relationship between the person and the organization may be represented by an edge in the data object graph between the Person data object and the Organization data object. An edge between two data object nodes may be represented and stored in various embodiments as a data object property value of one or more of the connected nodes, or as a separate data entity.


In an embodiment, application server 116 comprises mobile application base logic 306. Mobile application base logic 306 generally includes logic implementing mobile data analysis system operations that may be requested by an analyst workstation 114 and mobile devices 106 and comprises mobile helper logic 308 and geosearch logic 310.


In an embodiment, mobile helper logic 308 provides processes for assisting users to observe the location of one or more mobile devices on a map display. Mobile helper logic mobile device security logic 212 may comprise program instructions operable to receive and store locational and other data sent from mobile devices and to provide locational and other data in response to requests. The data received, stored, and sent may further include metadata. Mobile helper logic 308 may further comprise logic operable to transmit messages sent from mobile device and analyst workstation users, perform data object searches, and other functionality described herein.


In an embodiment, geosearch logic 310 provides processes for handling geosearch requests sent from a mobile device and analyst workstation users. In general, a geosearch request is a search request for data objects or other information that is associated with one or more specified geographic locations or areas. Examples of processing geosearch requests are described in a separate section herein.


3.3 Configuring Mobile Device Teams


For the purposes of clearly illustrating how the functions described herein operate, the following sections describe example graphical user interface displays for the described mobile data analysis system features. However, the graphical user interface displays described herein represent only selected examples of visualizations for the mobile data analysis system operations that are described herein. Thus, the disclosure broadly encompasses any methods of operating a mobile analysis system that are described herein.


Further, no particular graphical user interface is required and the disclosure is intended to encompass processing approaches for a mobile analysis system that are described independent of any graphical user interface, and it is not intended to be limited to any particular graphical user interface or other form of display. For example, the example graphical user interfaces merely represent one way for an analyst workstation user to view the location of one or more mobile devices on a map, to send and receive messages on a mobile device, and to view images received from a mobile device at an analyst workstation; in other embodiments, programmatic methods may be used to obtain the same information and other forms of data output may be used such as logging, reporting, storing in database tables, storing in spreadsheets, etc.


In an embodiment, mobile device user accounts of a mobile data analysis system may be grouped into one or more mobile device teams. In this context, mobile device user accounts may comprise information associated with a particular user in the mobile data analysis system including, for example, a user name, passwords, and other user settings, and enable users to authenticate with the mobile data analysis system. For example, a field analyst may provide an assigned user name and password at a mobile device in order to be granted access to use the resources of the mobile data analysis system from the mobile device. A mobile device team is a logical grouping of one or more mobile device accounts, and by extension the mobile device users associated with the mobile device user accounts. A mobile device team may be based on organizational, operational, or any other characteristics that define one or more groupings of users within an organization. For example, a law enforcement agency may group mobile device user accounts that have been created for police officers in the agency into one or more mobile device teams based on geographic areas of responsibility, organization roles (e.g., special weapons and tactics, bomb squad, K-9 unit, etc.), security access levels, or other such groupings.


In an embodiment, mobile device teams may be associated with one or more visibility settings. Visibility settings for a particular mobile device team may control which other users are able to view information associated with the particular mobile device team including associated location data, messages, and other team-centric information. FIG. 4 illustrates an example process flow 400 for generating one or more mobile device teams and mobile device team visibility settings. In an embodiment, one or more of the steps below may be omitted, repeated, or performed in a different order. The specific arrangement shown in FIG. 4 is not required.


In Step 402, one or more mobile device user accounts are generated. For example, a separate mobile device user account may be generated for each field analyst in an organization using an approved mobile device in the mobile data analysis system. In an embodiment, an input mechanism is provided for a system administrator or other user to enter commands for the purposes of generating mobile device user accounts. Here, the term “input mechanism” includes either a command line interaction mechanism or a graphical user interface based interaction mechanism, or a combination of the preceding two. For example, a command may be issued by a user at an analyst work station 114 and received by application server 116 and, in response, application server 116 may generate and store the one or more mobile device user accounts in a repository, such as data repository 118. In another embodiment, mobile device user account information for one or more mobile device user accounts may be received by an application server in the form of an account configuration file, for example, in an XML file or other structured document format.


In Step 404, a first mobile device team and a second mobile device team are generated. For example, an authorized user using an analyst workstation may issue one or more additional commands to create two new mobile device teams. The user may associate a label for each of the new teams, for example, the first mobile device team may be labeled the “Green” team and the second mobile device team may be labeled the “Blue” team. The mobile device team labels may be used for the purposes of identifying a mobile device team in other graphical user interfaces of the mobile device system. For the purposes of illustrating a clear example, only two mobile device teams are generated; however, in other embodiments any number of mobile device teams may be generated.


In Step 406, one or more of the mobile device user accounts are assigned to the first mobile device team and one or more of the mobile device user accounts are assigned to the second mobile device team. For example, a user may issue a command that specifies one or more of the generated mobile device user accounts and a mobile device team, the command indicating that the specified mobile device user accounts are to be assigned to the specified mobile device team. In an embodiment, a particular mobile device user account may be assigned to any number of different mobile device teams. The mobile device user account information and mobile device team information may be stored in one or more configuration files, database tables, or in any other suitable format in a data repository, such as data repository 118.


In Step 408, one or more mobile device team visibility settings are received. In this context, mobile device team visibility settings comprise one or more configuration settings indicating whether mobile device user accounts of particular mobile device teams are permitted access to view information associated with mobile device user accounts of other mobile device teams. For example, visibility settings may be used to manage a mobile device user's ability to view other mobile device user accounts and associated locational data on map displays, to send messages to other mobile device user accounts, and access other data associated with other mobile device user accounts. As used herein, indicating that first mobile device team is visible to a second mobile device team means that mobile device users of the second mobile device team are permitted to access and view information associated with mobile device user accounts of the first mobile device team.


In an embodiment, mobile device team visibility settings may be received by an application server as commands input by a user. Using the example mobile device teams Green and Blue generated above, a user may issue a first command that specifies that the Blue mobile team is visible to the Green mobile device team. The user may issue a second command that specifies that the Green mobile device team is not visible to the Blue mobile device team. As a result, mobile device user accounts associated with the Green mobile device team may be able to access and view information about mobile device user accounts associated with the Blue mobile device team. In contrast, mobile device user accounts associated with the Blue mobile device team may be prevented from viewing information associated with the Green mobile device team. In an embodiment, mobile device team visibility settings may also be configured on an individual mobile device user account basis. For example, a user may issue a command indicating that a particular mobile device team is or is not visible to a particular mobile device user account.


In Step 410, mobile device team visibility settings are stored in association with the first mobile device team and the second mobile device team, for example, in data repository 118.


3.4 Mobile Device Tracking


In one embodiment, a mobile data analysis system is configured to track the location and movement of one or more mobile devices and, by extension, the mobile device users using the tracked mobile devices. Tracking the location of mobile devices may be of interest to both mobile device users (e.g., field analysts) using the tracked mobile devices, and to analyst workstation users (e.g., operations analysts) that may be coordinating the activities of the mobile device users. As such, logic in both mobile devices 106 and analyst workstations 114 may be configured to receive and display location data associated with one or more tracked mobile devices.



FIG. 5 illustrates an example graphical user interface of a mobile device that is configured to display a location of one or more mobile devices. In an embodiment, GUI 500 comprises a mobile device user information panel 502 and a map display 506.


In an embodiment, map display 506 displays a portion of an interactive map including user location icons 504, 508. User location icons 504, 508 indicate an approximate location of two tracked mobile devices associated with mobile device user account indicated by the labels Brown and Smith. Although the example map display 506 displays two user location icons, map display 506 may display any number of mobile device user icons depending on the user's visibility settings, and the number of tracked mobile devices present in the displayed map area at the current map zoom level.


In an embodiment, information panel 502 comprises information related a mobile device user account associated with a currently selected mobile device in map display 506. In the current example, information panel 502 displays information associated with a mobile device user “Brown.” The information about mobile device user Brown may be displayed, for instance, in response to a user indicating input selecting user location icon 508 on map display 506 representing mobile device user Brown, selecting the mobile device user's name from a list, typing the mobile device user name into a search box, inputting a voice command, or otherwise indicating a selection of the mobile device user Brown. For example, a different mobile device and mobile device user account may be selected by indicating input selecting another user location icon that is visible on map display 506, such as user location icon 504 representing mobile device user Smith.


Information panel 502 includes additional information about mobile device user Brown, including “last update” information indicating that mobile device user Brown is assigned to a mobile device team identified by the team name “Red.” Information panel 502 further provides information indicating how recently location data was received by a mobile device server from the mobile device associated with mobile device user Brown. In this manner, the last update information may provide an indication of the accuracy of the displayed location for a selected mobile device.


In an embodiment, visibility setting information may determine which mobile device teams and mobile device users a particular user is able to view in GUI 500. For example, a user may be using a mobile device and logged in with a particular mobile device user account. The mobile device may send a request for location data of other mobile devices to mobile device server 108 or application server 116 in order to display the location data on map display 506. In response to receiving a request for location data of other mobile devices from the mobile device, mobile device server 108 and/or application server 116 may determine a set of mobile device teams and mobile device users accounts that the requesting mobile device user account has access to view based on mobile device user account visibility settings stored in data repository 118. Mobile device server 108 and/or application server 116 may send back location data for those mobile device teams and mobile device user accounts for which the visibility settings permit access. In this manner, map display 506 may display a user location icon for those mobile devices for which a user has visibility access, and not for those mobile devices for which visibility access has not been granted.



FIG. 6 illustrates an example graphical user interface of an analyst workstation that is configured to display a location of one or more mobile devices. GUI 600 comprises an assets list 602, mobile device team user lists 604, map display 606, and interface components 610. In an embodiment, GUI 600 generally is configured to display a location associated with one or more mobile device users on a map and the movement of those mobile device users. For example, as a particular mobile device user changes locations, the user's mobile device periodically sends updated location data of the mobile device to a mobile device server and/or application server. The mobile device server and/or application server may then send the updated location data to one or more mobile devices tracking the location of the particular user's mobile device. A map display 606 of the one of the mobile devices receiving the updated location data may be updated based on the received location data. For example, an icon may be displayed on the map for the particular user at an approximate location based on geographic coordinates or other location information included in the received location data.


In an embodiment, assets list 602 displays a list of mobile device teams and mobile device users that the user may view on map display 606. Each of the mobile device team user lists 604 in assets list 602 is configured to display the mobile device user accounts associated with the mobile device team. In an embodiment, the mobile device team user lists 604 shown in assets list 602 may be based on visibility settings stored in data repository 118. Assets list 602 may provide interface elements that enable a user to selectively hide or show particular mobile device teams on the map depending on the user's preferences.


In an embodiment, map display 606 displays one or more user location icons, such as user location icons 608, at a location most recently reported by the associated mobile devices. In the current example, user location icons 608 may correspond to the last known location of the mobile devices associated with the Red mobile device team, as indicated by the matching circle icons in assets list 602 and map display 606. In an embodiment, map display 606 may display information related to an estimated accuracy of the display location data associated with each tracked mobile device user. For example, if a particular mobile device fails to report location data within a specified period of time, an icon of the associated with mobile device user may change in assets list 602 and map display 606, for example, turning from green to yellow to red, and may further provide information indicating the lag time.


In an embodiment, map display 606 may be configured to select an appropriate map centering location and zoom level based on the mobile teams the user has visibility access to and/or teams currently selected from assets list 602. For example, map display 606 may display a map area that is large enough so that each of the selected mobile device team users in assets list 602 is displayed in the map area. An appropriate map zoom level may be determined, for example, by analyzing the location data (e.g., geographic coordinates) to determine a location associated with each of the mobile device user accounts to be displayed and selecting a map centering location and zoom level that includes each of the associated locations. For example, if a first mobile device team includes mobile device user accounts that are located in California, and a second mobile device team includes mobile device user accounts in New York, map display 606 may center on a location between California and New York and display a zoom level that is large enough to simultaneously display the mobile device users in both California and New York.


In FIG. 6, for example, the map currently is zoomed to display an area that includes user location icons for each of the mobile device teams and mobile device user accounts listed in asset list 602. If one or more of the currently displayed user accounts moves to an area that is outside of the currently displayed map view, a mobile device generating map display 606 may be configured to re-center or adjust the zoom level so as to maintain a display view of all selected mobile device teams. As another example, if a user selects one or more additional mobile device teams for viewing in assets list 602, or indicates input hiding one or more of the mobile device teams in assets list 602, map display 606 may be configured to re-center or adjust the zoom level so as to display all selected mobile device teams.


Interface components 610 may enable a user to adjust the current view of map display 606, for example, by zooming in or out, panning, annotating the display, or selecting particular mobile device users or teams to track.


3.5 Mobile Device Messaging


In an embodiment, mobile devices and analyst workstations of the mobile data analysis system may include logic operable to enable field analysts and operations analysts to exchange messages. In general, a message in the mobile data analysis system may comprise, without limitation, any combination of text, images, video, hyperlinks and other markup, other messages, and data objects. FIG. 7 illustrates a GUI 700 of a mobile device that is configured to enable messaging.


GUI 700 comprises messages 702, message input box 704, and message send button 706. In an embodiment, messages 702 provide a transcript of messages that a user of the mobile device has previously sent and received. Each of messages 702 may include information such as, for example, the message content, the name of the user that generated the message, and a timestamp indicating when the message was sent or received. In the current example, messages 702 depict a text conversation between mobile device user accounts labeled as Brown and Smith.


In an embodiment, to send a new message, a user may input message content into message input box 704. A user may input information into message input box 704 including text, multimedia content, and data objects stored on a data repository, or new multimedia content generated by the user using, for example, a camera 210 or voice input.


In an embodiment, selection of the send button 706 may cause message content from message input box 704 to be sent to one or more mobile devices, broadcast to one or more mobile device teams, and/or sent to one or more analyst workstations.



FIG. 8A, FIG. 8B illustrate GUIs 800, 802 that may be implemented on a mobile device for sending multimedia content, such as a digital image, in a message, according to an embodiment. GUI 800 comprises new photo button 804 and image gallery button 806.


A user may select new photo button 804, for example, in order to capture a new digital image using the mobile device using a camera 210. In another embodiment, a mobile device user may select one or more images previously captured and stored on the mobile device by selecting image gallery button 806. Once image gallery button 806 has been selected, for example, a user may be prompted to select one or more digital images stored on the mobile device. The image may also be selected from a data object stored on the data repository.


GUI 808 illustrates a graphical user interface configured to enable a user to send and receive messages including a digital image. GUI 808 includes, for example, a message 810 sent by user Smith. In the example, message 810 comprises a digital image 812 that may have been previously captured by user Smith and attached to message 810. For example, a user may attach a digital image, possibly associated with a data object, to a message using text input box 814 or any other input mechanisms.


3.6 Creating Data Objects from Images


In an embodiment, an operations analyst or other user may desire to create a data object from a digital image captured by a field analyst or other user using a mobile device. FIG. 9 illustrates an example process flow 900 for creating data objects from one or more digital images. In an embodiment, one or more of the steps below may be omitted, repeated, or performed in a different order. The specific arrangement shown in FIG. 9 is not required.


In Step 902, a computing device receives a digital image comprising image metadata. For example, the computing device may be an analyst workstation and the digital image may be sent to the analyst workstation from a mobile device via a network, such as network 112. The digital image may be sent from a mobile device to an analyst workstation as part of a message, attached to an email, as a direct upload, or using any other mechanism of transmitting a digital image. For example, a field analyst using a mobile device may capture an image of a person of interest or a particular location and send the captured image to an analyst workstation in a multimedia message, as described above. In other embodiments, the digital image may be received by querying a stored collection of digital images in a data repository, such as data repository 118.


In an embodiment, a digital image received by the computing device comprises image metadata. The image metadata generally may comprise information about the digital image and include one or more image properties each having an image property value. For example, the image properties may include, without limitation, date and time information, location information, camera manufacturer and camera model number, compression format, camera settings (e.g., exposure time, f-number, etc.), image thumbnails, and mobile device user information. The image metadata may be generated by a camera or mobile device when the digital image is captured and may be stored, for example, as part of a file representing the digital image or stored in a separate metadata file. In one embodiment, the image metadata may comprise data conforming to the exchangeable image file format (EXIF) standard.


In Step 904, the computing device transforms one or more of the image property values of the one or more image properties into one or more particular values. In an embodiment, transforming the one or more image property values may include, for example, reading the image metadata and extracting one or more image property values from the image metadata. Transforming may further comprise reformatting, converting units, combining values, or any other data transformations to one or more of the image property values. For example, transforming may include converting values representing a date and time in one format into another format suitable for storage in a data object of the mobile data analysis system.


In Step 906, one or more data object property values of a data object are populated with the one or more particular values. In an embodiment, the data object may represent a newly generated data object, or a data object already existing in the mobile data analysis system. Populating the one or more data object property values generally may include assigning the transformed particular values to corresponding data object property fields. For example, a particular data object property field may store a value indicating a geographical location and the particular data object property field may be assigned a value obtained from the image metadata indicating a geographical location where the digital image was captured.


In an embodiment, the populated data object may be associated with a particular data object type. For example, default settings may associate the data object with an Event data object type. Depending on the data object type, particular data object property fields may or may not be associated with the data object. For example, a data object of type Event may have a property field indicating an event time, whereas a data object of type Person may not. In an embodiment, a user may change the default data object type and/or modify the data object type currently associated with a particular data object. For example, a data object of type Event may be created from a digital image of an individual, and a user may desire that the data object be changed to a Person type. The user may specify input modifying the data object type to type Person, for example, using one or more graphical user interfaces described herein. Data objects created generally may be associated with any data object types defined in the mobile data analysis system and may include, for example, an event type, a person type, an organization type, a location type, an entity type, and an item type.


In an embodiment, creating a data object from a digital image may further comprise generating one or more links to one or more other existing data objects of a data object graph stored in a data repository, such as data repository 118. For example, using the steps described above in FIG. 9, a data object of type Person may be created from an image depicting an individual known to be affiliated with a particular organization. The particular organization may, for example, be represented by an Organization data object as part of a data object graph stored in data repository 118. An operations analyst or other user may desire to associate the created Person data object with the existing Organization data object in the data object graph. Accordingly, the user may specify input indicating an association between the two data objects, and as a result one or more links may be generated from the Person data object to the Organization data object and the links may be stored in the data repository. For example, the links may be stored as one or more data object property fields of the Person data object and/or Organization data object, or stored as separate link data entities. In one embodiment, the links between the created data object and other existing data objects may be specified by a mobile device user. For example, the mobile device user may specify the links as part of a message that includes the digital image from which the data object is created.


In one embodiment, a data object may be created from two or more digital images. For example, an analyst workstation may receive multiple images from one or more mobile devices that depict the same individual. An operations analyst using the analyst workstation may desire to create a single data object based on the multiple images. In an embodiment, transforming one or more image property values into one or more particular values further comprises generating summary information based on image metadata of the two or more digital images. For example, the metadata of each of the two or more digital images may include a property value indicating a location where the digital image was captured. In one embodiment, transforming the metadata image property values into particular values may include generating a particular location value, the particular location value derived from an average of the location values of the metadata for each of the digital images.


In Step 908, the data object is stored in a data repository. For example, the data object may be stored in a data repository such as repository 118 and made accessible to other field analysts and operations analysts using mobile devices 106 and analyst workstations 114 for further data manipulations.


In some embodiments, creating a data object from a digital image may be performed in response to a user selecting an image for processing, for example, using a GUI 1000 as shown in FIG. 10. GUI 1000 includes an image gallery 1002, a selected image display 1006, a map display 1008, and a data object creation button 1010. For example, GUI 1000 may be used by an operations analyst to view digital images at an analyst workstation that have been received from one or more mobile devices and/or from other sources, and to create data objects from the received digital images.


Image gallery 1002 displays a selection of digital images received by a computing device generating GUI 1000. Digital images displayed in the image gallery 1002 may be filtered and/or sorted to display those images most relevant to a user's interests. For example, GUI 1000 may provide input mechanisms that enable a user to filter the displayed digital images to those uploaded by one or more particular mobile device user accounts or by one or more particular mobile device teams. Digital images displayed in image gallery 1002 may also be sorted by other criteria, for example, by the date and time the digital images were captured, the date and time the images were uploaded, a location associated with the digital images, or any other desirable sorting order.


In an embodiment, a user may select one or more digital images displayed in image gallery 1002 and in response GUI 1000 may provide additional information related to the selected digital images in selected image display 1006 and map display 1008. For example, a digital image selected in image gallery 1002 may be displayed in selected image display 1006 in a larger or higher resolution format. In the present example, Photo 1 from the image gallery 1002 is selected, as indicated by the bolded outline of the image, and an enlarged display of Photo 1 is shown in selected image display 1006.


Map display 1008 includes a map that may display geographic information associated with a selected digital image from image gallery 1002. For example, map display 1008 may that include an icon representing a geographic location where a selected image was captured. For example, a geographic location where one or more selected images were captured may be determined based on information in the associated image metadata for each selected image. If multiple digital images are selected in image gallery 1002, map display 1008 may display multiple icons representing a geographic location associated with each of the selected images. Map display 1008 may be further configured to zoom to an area of the map that includes the location associated with each of the selected images.


In an embodiment, GUI 1000 includes a data object creation button 1010 which, when selected, may cause performance of one or more steps of creating a data object from the currently selected digital images, as described above in further detail with reference to FIG. 9.


In some embodiments, GUI 1100 of FIG. 11 may be optionally used to modify information or supply additional information related to a data object created from a digital image. FIG. 11 comprises data object list 1102, data object information panel 1104, and data object property interface elements 1106.


Data object list 1102 displays a list of selectable data objects stored in the mobile data analysis system. The data objects listed in data object list 1102 may include, for example, data objects created during a current user session, or data objects previously created by the user and retrieved from a data repository. The data objects displayed in data object list 1102 may be filtered and stored based on object type, a title associated with object, a user associated with the object, or any other criteria.


Data object information panel 1104 includes information associated with a currently selected data object from data object list 1102. As depicted, data object information panel 1104 displays information for an Event data object labeled “Suspect sighting.” Information displayed in data object information panel 1104 includes a display of a digital image associated with the selected data object, information about other related data objects, information about a user that created the data object, when the data object was created, and a location associated with the data object.


Interface elements 1106 are selectable to facilitate modification of the property values of a currently selected data object. For example, object type selector 1108 is a pull-down menu that allows a user to select between different object types that are defined in the mobile data analysis system. Users may modify other data object properties using the interface elements including a label of the data object, date and time information, location information, and data object link information. The interface elements 1106 may depend on the data object type for the selected data object and the particular data object property fields associated with the data object type.


3.7 Mobile Device Search


In an embodiment, mobile device users may search for data objects and other information stored in a centralized data repository, such as data repository 118. For example, a field analyst may desire to search for stored data objects to acquire information about the field analyst's current activities. The field analyst may, for example, be interviewing an individual in the field and desire to know if any information previously has been collected about the individual and stored in the data repository. In an embodiment, the field analyst may specify a search request for information about the individual using a mobile device 106 and the mobile device may send the search request to application server 116 via mobile device server 108. In an embodiment, a mobile device user may specify a search request, for example, by inputting one or more search terms or using a voice command.


In response to receiving a search request from a mobile device, application server 116 may retrieve one or more data object results from data repository 118 that are relevant to the search request. For example, application server 116 may locate data objects in data repository 118 that include one or more of the specified search terms. Application server 116 may send the resulting data objects or other information to the requesting mobile device for display on the mobile device.


In one embodiment, mobile device users may specify geosearch requests using a mobile device. In general, a geosearch request is a search request for data objects that are associated with a specified geographic location or area. FIG. 12 illustrates an example GUI 1200 of a mobile device that is configured to enable users to specify a geosearch request.


In an embodiment, GUI 1200 comprises search button 1202 which, when selected, may be configured to enable a user to specify a geosearch request. In one embodiment, mobile device users may specify a geosearch request using map display 1204. For example, a user may specify on map display 1204 a geographic bounding area of interest for the geosearch request. In the current example, bounding area 1206 illustrates an example bounding area for a geosearch request. In the example, the bounding area is depicted as a circle, but other bounding area shapes may be used. The specified geographic bounding area may be used to transmit as part of the geosearch request a set of geographic coordinates or other information that enables application server 116 to determine a geographic area within to search. A geosearch request may include other parameters including a time span desired for result data objects. For example, a user may desire that only Event data objects associated with an event that occurred in the past week be returned.


In response to receiving a geosearch request, application server 116 may determine one or more result data objects in data repository 118 that include location data specifying one or more locations that are within the bounding area specified by the geosearch request. Application server 116 may send the result data objects to the requesting mobile device and the mobile device may display the result data objects on map display 1204. For example, event data object icon 1208 illustrates an example geosearch result data object for a geosearch request specified by bounding area 1206. Event data object icon 1208 may be displayed at a location on the map based on location data stored in association with the data object result corresponding to the displayed icon. In an embodiment, a user may select one or more of the data object result icons on map display 1204 in order to view additional information associated with the data object.


4.0 Implementation Mechanisms—Hardware Overview


According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices. The special-purpose computing devices may be hard-wired to perform the techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) that are persistently programmed to perform the techniques, or may include one or more general purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices may also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to accomplish the techniques. The special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, networking devices or any other device that incorporates hard-wired and/or program logic to implement the techniques.


For example, FIG. 13 is a block diagram that illustrates a computer system 1300 upon which an embodiment may be implemented. Computer system 1300 includes a bus 1302 or other communication mechanism for communicating information, and a hardware processor 1304 coupled with bus 1302 for processing information. Hardware processor 1304 may be, for example, a general purpose microprocessor.


Computer system 1300 also includes a main memory 1306, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 1302 for storing information and instructions to be executed by processor 1304. Main memory 1306 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 1304. Such instructions, when stored in non-transitory storage media accessible to processor 1304, render computer system 1300 into a special-purpose machine that is customized to perform the operations specified in the instructions.


Computer system 1300 further includes a read only memory (ROM) 1308 or other static storage device coupled to bus 1302 for storing static information and instructions for processor 1304. A storage device 1310, such as a magnetic disk, optical disk, or solid-state drive is provided and coupled to bus 1302 for storing information and instructions.


Computer system 1300 may be coupled via bus 1302 to a display 1312, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 1314, including alphanumeric and other keys, is coupled to bus 1302 for communicating information and command selections to processor 1304. Another type of user input device is cursor control 1316, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 1304 and for controlling cursor movement on display 1312. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.


Computer system 1300 may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware and/or program logic which in combination with the computer system causes or programs computer system 1300 to be a special-purpose machine. According to one embodiment, the techniques herein are performed by computer system 1300 in response to processor 1304 executing one or more sequences of one or more instructions contained in main memory 1306. Such instructions may be read into main memory 1306 from another storage medium, such as storage device 1310. Execution of the sequences of instructions contained in main memory 1306 causes processor 1304 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions.


The term “storage media” as used herein refers to any non-transitory media that store data and/or instructions that cause a machine to operate in a specific fashion. Such storage media may comprise non-volatile media and/or volatile media. Non-volatile media includes, for example, optical disks, magnetic disks, or solid-state drives, such as storage device 1310. Volatile media includes dynamic memory, such as main memory 1306. Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid-state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge.


Storage media is distinct from but may be used in conjunction with transmission media. Transmission media participates in transferring information between storage media. For example, transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 1302. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.


Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 1304 for execution. For example, the instructions may initially be carried on a magnetic disk or solid-state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 1300 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 1302. Bus 1302 carries the data to main memory 1306, from which processor 1304 retrieves and executes the instructions. The instructions received by main memory 1306 may optionally be stored on storage device 1310 either before or after execution by processor 1304.


Computer system 1300 also includes a communication interface 1318 coupled to bus 1302. Communication interface 1318 provides a two-way data communication coupling to a network link 1320 that is connected to a local network 1322. For example, communication interface 1318 may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 1318 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 1318 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.


Network link 1320 typically provides data communication through one or more networks to other data devices. For example, network link 1320 may provide a connection through local network 1322 to a host computer 1324 or to data equipment operated by an Internet Service Provider (ISP) 1326. ISP 1326 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 1328. Local network 1322 and Internet 1328 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 1320 and through communication interface 1318, which carry the digital data to and from computer system 1300, are example forms of transmission media.


Computer system 1300 can send messages and receive data, including program code, through the network(s), network link 1320 and communication interface 1318. In the Internet example, a server 1330 might transmit a requested code for an application program through Internet 1328, ISP 1326, local network 1322 and communication interface 1318.


The received code may be executed by processor 1304 as it is received, and/or stored in storage device 1310, or other non-volatile storage for later execution.


In the foregoing specification, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. The sole and exclusive indicator of the scope of the embodiments, and what is intended by the applicants to be the scope of the embodiments, is the literal and equivalent scope of the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction.

Claims
  • 1. A computer-implemented method, comprising: maintaining, in a data repository of a computing device, an object-centric data model comprising a plurality of data objects of a plurality of data object types, each of the data object types representing a person, place, organization, or event;using digitally programmed logic of the computing device, generating a first data object of a first data object type, wherein the first data object is based on one or more image properties of a digital image;receiving, over a network at the computing device, input associating the first data object with a second data object of a second data object type, wherein the second data object preexists in the object-centric data model;using digitally programmed logic of the computing device, generating a data object link between the first data object and the second data object;storing the data object link in the computing device.
  • 2. The computer-implemented method of claim 1, wherein storing the data object link in the computing device comprises storing a separate link data entity in the computing device wherein the separate link data entity identifies the first data object and the second data object.
  • 3. The computer-implemented method of claim 1, wherein storing the data object link in the computing device comprises: populating one or more data object property values of the first data object with a link to the second data object; andpopulating one or more data object property values of the second data object with a link to the first data object.
  • 4. The computer-implemented method of claim 1, wherein the one or more links are generated in response to receiving data from a mobile device indicating the one or more links.
  • 5. The computer-implemented method of claim 1, further comprising: receiving a first digital image comprising first image metadata, wherein the first image metadata includes one or more image properties having an image property value;transforming one or more of the image property values of the image properties into one or more particular values;populating one or more data object property values of the first data object with the one or more particular values.
  • 6. The computer-implemented method of claim 5, further comprising populating the one or more data object property values of the data object in response to user selection of the digital image.
  • 7. The computer-implemented method of claim 5, further comprising: receiving a second digital image comprising second image metadata; andwherein transforming one or more of the image property values of the one or more image properties into one or more particular values includes generating summary information based on the first image metadata and the second image metadata.
  • 8. The computer-implemented method of claim 7, wherein the summary information includes a particular location value, the particular location value derived from an average of a first location value associated with the first image metadata and a second location value associated with the second image metadata.
  • 9. A non-transitory computer-readable media storing instructions which, when executed by one or more processors, causes performance of: maintaining, in a data repository of a computing device, an object-centric data model comprising a plurality of data objects of a plurality of data object types, each of the data object types representing a person, place, organization, or event;using digitally programmed logic of the computing device, generating a first data object of a first data object type, wherein the first data object is based on one or more image properties of a digital image;receiving, over a network at the computing device, input associating the first data object with a second data object of a second data object type, wherein the second data object preexists in the object-centric data model;using digitally programmed logic of the computing device, generating a data object link between the first data object and the second data object;storing the data object link in the computing device.
  • 10. The non-transitory computer-readable media of claim 9, wherein storing the data object link in the computing device comprises storing a separate link data entity in the computing device wherein the separate link data entity identifies the first data object and the second data object.
  • 11. The non-transitory computer-readable media of claim 9, wherein storing the data object link in the computing device comprises: populating one or more data object property values of the first data object with a link to the second data object; andpopulating one or more data object property values of the second data object with a link to the first data object.
  • 12. The non-transitory computer-readable media of claim 9, wherein the one or more links are generated in response to receiving data from a mobile device indicating the one or more links.
  • 13. The non-transitory computer-readable media of claim 9 wherein the instructions, when executed by the one or more processors, further cause performance of: receiving a first digital image comprising first image metadata, wherein the first image metadata includes one or more image properties having an image property value;transforming one or more of the image property values of the image properties into one or more particular values;populating one or more data object property values of the first data object with the one or more particular values.
  • 14. The non-transitory computer-readable media of claim 13, wherein the instructions, when executed by the one or more processors, further cause performance of populating the one or more data object property values of the data object in response to user selection of the digital image.
  • 15. The non-transitory computer-readable media of claim 13, wherein the instructions, when executed by the one or more processors, further cause performance of: receiving a second digital image comprising second image metadata; andwherein transforming one or more of the image property values of the one or more image properties into one or more particular values includes generating summary information based on the first image metadata and the second image metadata.
  • 16. The non-transitory computer-readable media of claim 15, wherein the summary information includes a particular location value, the particular location value derived from an average of a first location value associated with the first image metadata and a second location value associated with the second image metadata.
BENEFIT CLAIMS

This application claims the benefit under 35 U.S.C. § 120 as a continuation of application Ser. No. 15/598,752, filed May 18, 2017, which claims the benefit under 35 U.S.C. § 120 as a continuation of application Ser. No. 14/806,517, filed Jul. 22, 2015, which claims the benefit under 35 U.S.C. § 120 as a continuation of application Ser. No. 13/838,815, filed Mar. 15, 2013, which claims the benefit under 35 U.S.C. § 119(e) of provisional application 61/759,283, filed Jan. 31, 2013, the entire contents of which are hereby incorporated by reference as if fully set forth herein. The applicant(s) hereby rescind any disclaimer of claim scope in the parent application(s) or the prosecution history thereof and advise the USPTO that the claims in this application may be broader than any claim in the parent application(s).

US Referenced Citations (585)
Number Name Date Kind
5021792 Hwang Jun 1991 A
5109399 Thompson Apr 1992 A
5329108 Lamoure Jul 1994 A
5555503 Kyrtsos et al. Sep 1996 A
5632009 Rao et al. May 1997 A
5670987 Doi et al. Sep 1997 A
5781704 Rossmo Jul 1998 A
5798769 Chiu et al. Aug 1998 A
5845300 Corner Dec 1998 A
6057757 Arrowsmith et al. May 2000 A
6091956 Hollenberg Jul 2000 A
6141659 Barker et al. Oct 2000 A
6161098 Wallman Dec 2000 A
6189003 Leal Feb 2001 B1
6219053 Tachibana et al. Apr 2001 B1
6232971 Haynes May 2001 B1
6247019 Davies Jun 2001 B1
6272489 Rauch et al. Aug 2001 B1
6279018 Kudrolli et al. Aug 2001 B1
6341310 Leshem et al. Jan 2002 B1
6366933 Ball et al. Apr 2002 B1
6369835 Lin Apr 2002 B1
6456997 Shukla Sep 2002 B1
6549944 Weinberg et al. Apr 2003 B1
6560620 Ching May 2003 B1
6581068 Bensoussan et al. Jun 2003 B1
6594672 Lampson et al. Jul 2003 B1
6631496 Li et al. Oct 2003 B1
6642945 Sharpe Nov 2003 B1
6674434 Chojnacki et al. Jan 2004 B1
6714936 Nevin, III Mar 2004 B1
6775675 Nwabueze et al. Aug 2004 B1
6820135 Dingman Nov 2004 B1
6828920 Owen et al. Dec 2004 B2
6839745 Dingari et al. Jan 2005 B1
6877137 Rivette et al. Apr 2005 B1
6976210 Silva et al. Dec 2005 B1
6980984 Huffman et al. Dec 2005 B1
6985950 Hanson et al. Jan 2006 B1
7036085 Barros Apr 2006 B2
7043702 Chi et al. May 2006 B2
7055110 Kupka et al. May 2006 B2
7139800 Bellotti et al. Nov 2006 B2
7158797 Jayaraman et al. Jan 2007 B1
7158878 Rasmussen et al. Jan 2007 B2
7162475 Ackerman Jan 2007 B2
7168039 Bertram Jan 2007 B2
7171427 Witowski et al. Jan 2007 B2
7188100 De Bellis et al. Mar 2007 B2
7269786 Malloy et al. Sep 2007 B1
7278105 Kitts Oct 2007 B1
7290698 Poslinski et al. Nov 2007 B2
7333998 Heckerman et al. Feb 2008 B2
7370047 Gorman May 2008 B2
7379811 Rasmussen et al. May 2008 B2
7379903 Caballero et al. May 2008 B2
7383053 Kent et al. Jun 2008 B2
7426654 Adams et al. Sep 2008 B2
7441182 Beilinson Oct 2008 B2
7454466 Bellotti et al. Nov 2008 B2
7467375 Tondreau et al. Dec 2008 B2
7487139 Fraleigh et al. Feb 2009 B2
7502786 Liu et al. Mar 2009 B2
7523100 Bionda et al. Apr 2009 B1
7525422 Bishop et al. Apr 2009 B2
7529727 Arning et al. May 2009 B2
7529734 Dirisala May 2009 B2
7558677 Jones Jun 2009 B2
7574409 Patinkin Aug 2009 B2
7574428 Leiserowitz et al. Aug 2009 B2
7579965 Bucholz Aug 2009 B2
7596285 Brown et al. Sep 2009 B2
7614006 Molander Nov 2009 B2
7617232 Gabbert et al. Nov 2009 B2
7620628 Kapur et al. Nov 2009 B2
7627812 Chamberlain et al. Dec 2009 B2
7634717 Chamberlain et al. Dec 2009 B2
7652622 Hansen et al. Jan 2010 B2
7703021 Flam Apr 2010 B1
7706817 Banirah et al. Apr 2010 B2
7712049 Williams et al. May 2010 B2
7716077 Mikurak May 2010 B1
7725530 Sah et al. May 2010 B2
7725547 Albertson et al. May 2010 B2
7730082 Sah et al. Jun 2010 B2
7730109 Rohrs et al. Jun 2010 B2
7747648 Kraft et al. Jun 2010 B1
7760969 Silverbrook et al. Jul 2010 B2
7770100 Chamberlain et al. Aug 2010 B2
7805457 Viola et al. Sep 2010 B1
7809703 Balabhadrapatruni et al. Oct 2010 B2
7818291 Ferguson et al. Oct 2010 B2
7818658 Chen Oct 2010 B2
7870493 Pall et al. Jan 2011 B2
7894984 Rasmussen et al. Feb 2011 B2
7899611 Downs et al. Mar 2011 B2
7917376 Bellin et al. Mar 2011 B2
7920963 Jouline et al. Apr 2011 B2
7933862 Chamberlain et al. Apr 2011 B2
7945470 Cohen et al. May 2011 B1
7962281 Rasmussen et al. Jun 2011 B2
7962495 Jain et al. Jun 2011 B2
7962848 Bertram Jun 2011 B2
7970240 Chao et al. Jun 2011 B1
7971150 Raskutti et al. Jun 2011 B2
7984374 Caro et al. Jun 2011 B2
7971784 Lapstun Jul 2011 B2
8001465 Kudrolli et al. Aug 2011 B2
8001482 Bhattiprolu et al. Aug 2011 B2
8010545 Stefik et al. Aug 2011 B2
8015487 Roy et al. Sep 2011 B2
8024778 Cash et al. Sep 2011 B2
8028894 Lapstun et al. Oct 2011 B2
8036632 Cona et al. Oct 2011 B1
8042110 Kawahara et al. Oct 2011 B1
8103543 Zwicky Jan 2012 B1
8134457 Velipasalar et al. Mar 2012 B2
8145703 Frishert et al. Mar 2012 B2
8185819 Sah et al. May 2012 B2
8214361 Sandler et al. Jul 2012 B1
8214764 Gemmell et al. Jul 2012 B2
8225201 Michael Jul 2012 B2
8229947 Fujinaga Jul 2012 B2
8230333 Decherd et al. Jul 2012 B2
8271461 Pike et al. Sep 2012 B2
8280880 Aymeloglu et al. Oct 2012 B1
8285725 Bayliss Oct 2012 B2
8290926 Ozzie et al. Oct 2012 B2
8290942 Jones et al. Oct 2012 B2
8301464 Cave et al. Oct 2012 B1
8301904 Gryaznov Oct 2012 B1
8312367 Foster Nov 2012 B2
8312546 Alme Nov 2012 B2
8352881 Champion et al. Jan 2013 B2
8368695 Howell et al. Feb 2013 B2
8397171 Klassen et al. Mar 2013 B2
8402047 Mangini et al. Mar 2013 B1
8412707 Mianji Apr 2013 B1
8447722 Ahuja et al. May 2013 B1
8452790 Mianji May 2013 B1
8463036 Ramesh et al. Jun 2013 B1
8477994 Noshadi Jul 2013 B1
8489331 Kopf et al. Jul 2013 B2
8489641 Seefeld et al. Jul 2013 B1
8498984 Hwang et al. Jul 2013 B1
8510743 Hackborn et al. Aug 2013 B2
8514082 Cova et al. Aug 2013 B2
8515207 Chau Aug 2013 B2
8521135 Cryderman Aug 2013 B2
8554579 Tribble et al. Oct 2013 B2
8554653 Falkenborg et al. Oct 2013 B2
8554709 Goodson et al. Oct 2013 B2
8577911 Stepinski et al. Nov 2013 B1
8589273 Creeden et al. Nov 2013 B2
8595234 Siripuapu et al. Nov 2013 B2
8620641 Farnsworth et al. Dec 2013 B2
8639757 Zang et al. Jan 2014 B1
8646080 Williamson et al. Feb 2014 B2
8676857 Adams et al. Mar 2014 B1
8688069 Cazanas et al. Apr 2014 B1
8689108 Duffield et al. Apr 2014 B1
8713467 Goldenberg et al. Apr 2014 B1
8726379 Stiansen et al. May 2014 B1
8739059 Rabenold et al. May 2014 B2
8739278 Varghese May 2014 B2
8742934 Sarpy et al. Jun 2014 B1
8744890 Bernier Jun 2014 B1
8745516 Mason et al. Jun 2014 B2
8762870 Robotham et al. Jun 2014 B2
8781169 Jackson et al. Jul 2014 B2
8787939 Papakipos et al. Jul 2014 B2
8788407 Singh et al. Jul 2014 B1
8799799 Cervelli et al. Aug 2014 B1
8812960 Sun et al. Aug 2014 B1
8830322 Nerayoff et al. Sep 2014 B2
8832594 Thompson et al. Sep 2014 B1
8849254 Bolon Sep 2014 B2
8868537 Colgrove et al. Oct 2014 B1
8917274 Ma et al. Dec 2014 B2
8924872 Bogomolov et al. Dec 2014 B1
8937619 Sharma et al. Jan 2015 B2
8938686 Erenrich et al. Jan 2015 B1
9009171 Grossman et al. Apr 2015 B1
9009827 Albertson et al. Apr 2015 B1
9021260 Falk et al. Apr 2015 B1
9021384 Beard et al. Apr 2015 B1
9037407 Thompson May 2015 B2
9043696 Meiklejohn et al. May 2015 B1
9043894 Dennison et al. May 2015 B1
9116975 Shankar et al. Aug 2015 B2
9123086 Freeland et al. Sep 2015 B1
9262529 Colgrove et al. Feb 2016 B2
9275069 Garrod et al. Mar 2016 B1
9301103 Thompson Mar 2016 B1
9313233 Sprague et al. Apr 2016 B2
9380431 Freeland et al. Jun 2016 B1
20020033848 Sciammarella et al. Mar 2002 A1
20020065708 Senay et al. May 2002 A1
20020091707 Keller Jul 2002 A1
20020095658 Shulman Jul 2002 A1
20020116120 Ruiz et al. Aug 2002 A1
20020174201 Ramer et al. Nov 2002 A1
20020194119 Wright et al. Dec 2002 A1
20030028560 Kudrolli et al. Feb 2003 A1
20030039948 Donahue Feb 2003 A1
20030061211 Shultz et al. Mar 2003 A1
20030140106 Raguseo Jul 2003 A1
20030144868 MacIntyre et al. Jul 2003 A1
20030152277 Hall et al. Aug 2003 A1
20030163352 Surpin et al. Aug 2003 A1
20030225755 Iwayama et al. Dec 2003 A1
20030227746 Sato Dec 2003 A1
20030229848 Arend et al. Dec 2003 A1
20040032432 Baynger Feb 2004 A1
20040064256 Barinek et al. Apr 2004 A1
20040085318 Hassler et al. May 2004 A1
20040095349 Bito et al. May 2004 A1
20040111410 Burgoon et al. Jun 2004 A1
20040126840 Cheng et al. Jul 2004 A1
20040143602 Ruiz et al. Jul 2004 A1
20040143796 Lerner et al. Jul 2004 A1
20040163039 McPherson et al. Aug 2004 A1
20040193600 Kaasten et al. Sep 2004 A1
20040203380 Hamdi et al. Oct 2004 A1
20040221223 Yu et al. Nov 2004 A1
20040260702 Cragun et al. Dec 2004 A1
20040267746 Marcjan et al. Dec 2004 A1
20050027705 Sadri et al. Feb 2005 A1
20050028094 Allyn Feb 2005 A1
20050039119 Parks et al. Feb 2005 A1
20050065811 Chu et al. Mar 2005 A1
20050080769 Gemmell et al. Apr 2005 A1
20050086207 Heuer et al. Apr 2005 A1
20050125436 Mudunuri et al. Jun 2005 A1
20050125715 Franco et al. Jun 2005 A1
20050143096 Boesch Jun 2005 A1
20050162523 Darrell et al. Jul 2005 A1
20050166144 Gross Jul 2005 A1
20050180330 Shapiro Aug 2005 A1
20050182793 Keenan et al. Aug 2005 A1
20050183005 Denoue et al. Aug 2005 A1
20050210409 Jou Sep 2005 A1
20050246327 Yeung et al. Nov 2005 A1
20050251786 Citron et al. Nov 2005 A1
20060026120 Carolan et al. Feb 2006 A1
20060026170 Kreitler et al. Feb 2006 A1
20060059139 Robinson Mar 2006 A1
20060074881 Vembu et al. Apr 2006 A1
20060080619 Carlson et al. Apr 2006 A1
20060093222 Saffer et al. May 2006 A1
20060116991 Calderwood Jun 2006 A1
20060129746 Porter Jun 2006 A1
20060139375 Rasmussen et al. Jun 2006 A1
20060142949 Helt Jun 2006 A1
20060143034 Rothermel Jun 2006 A1
20060143052 Fotsch et al. Jun 2006 A1
20060149596 Surpin et al. Jul 2006 A1
20060161568 Dettinger et al. Jul 2006 A1
20060203337 White Sep 2006 A1
20060206235 Shakes et al. Sep 2006 A1
20060218637 Thomas et al. Sep 2006 A1
20060241974 Chao et al. Oct 2006 A1
20060242040 Rader et al. Oct 2006 A1
20060242630 Koike et al. Oct 2006 A1
20060250764 Howarth et al. Nov 2006 A1
20060271277 Hu et al. Nov 2006 A1
20060279630 Aggarwal et al. Dec 2006 A1
20070011150 Frank Jan 2007 A1
20070016363 Huang et al. Jan 2007 A1
20070038646 Thota Feb 2007 A1
20070038962 Fuchs et al. Feb 2007 A1
20070043744 Carro Feb 2007 A1
20070057966 Ohno et al. Mar 2007 A1
20070072591 McGary et al. Mar 2007 A1
20070078832 Ott et al. Apr 2007 A1
20070083541 Fraleigh et al. Apr 2007 A1
20070088596 Berkelhamer et al. Apr 2007 A1
20070094389 Nussey et al. Apr 2007 A1
20070118547 Gupta et al. May 2007 A1
20070130541 Louch et al. Jun 2007 A1
20070150369 Zivin Jun 2007 A1
20070174760 Chamberlain et al. Jul 2007 A1
20070192265 Chopin et al. Aug 2007 A1
20070198571 Ferguson et al. Aug 2007 A1
20070208497 Downs et al. Sep 2007 A1
20070208498 Barker et al. Sep 2007 A1
20070208736 Tanigawa et al. Sep 2007 A1
20070233709 Abnous Oct 2007 A1
20070240062 Christena et al. Oct 2007 A1
20070250491 Olszak et al. Oct 2007 A1
20070266336 Nojima et al. Nov 2007 A1
20070294643 Kyle Dec 2007 A1
20080007618 Yuasa Jan 2008 A1
20080025629 Obrador Jan 2008 A1
20080040684 Crump Feb 2008 A1
20080051989 Welsh Feb 2008 A1
20080052142 Bailey et al. Feb 2008 A1
20080077597 Butler Mar 2008 A1
20080077642 Carbone et al. Mar 2008 A1
20080082486 Lermant et al. Apr 2008 A1
20080104019 Nath May 2008 A1
20080126951 Sood et al. May 2008 A1
20080155440 Trevor et al. Jun 2008 A1
20080195417 Surpin et al. Aug 2008 A1
20080195608 Clover Aug 2008 A1
20080222295 Robinson et al. Sep 2008 A1
20080227473 Haney Sep 2008 A1
20080252419 Batchelor et al. Oct 2008 A1
20080263468 Cappione et al. Oct 2008 A1
20080267107 Rosenberg Oct 2008 A1
20080276167 Michael Nov 2008 A1
20080278311 Grange et al. Nov 2008 A1
20080288306 MacIntyre et al. Nov 2008 A1
20080301559 Martinsen et al. Dec 2008 A1
20080301643 Appleton et al. Dec 2008 A1
20080313281 Scheidl et al. Dec 2008 A1
20090002492 Velipasalar et al. Jan 2009 A1
20090005070 Forstall et al. Jan 2009 A1
20090006471 Richardson Jan 2009 A1
20090006474 Richardson Jan 2009 A1
20090027418 Maru et al. Jan 2009 A1
20090030915 Winter et al. Jan 2009 A1
20090037912 Stoitsev et al. Feb 2009 A1
20090055251 Shah et al. Feb 2009 A1
20090088964 Schaaf et al. Apr 2009 A1
20090119309 Gibson et al. May 2009 A1
20090119578 Relyea May 2009 A1
20090125369 Kloosstra et al. May 2009 A1
20090125459 Norton et al. May 2009 A1
20090132921 Hwangbo et al. May 2009 A1
20090132953 Reed et al. May 2009 A1
20090138790 Larcheveque et al. May 2009 A1
20090143052 Bates et al. Jun 2009 A1
20090144262 White et al. Jun 2009 A1
20090144274 Fraleigh et al. Jun 2009 A1
20090156231 Versteeg et al. Jun 2009 A1
20090164934 Bhattiprolu et al. Jun 2009 A1
20090171939 Athsani et al. Jul 2009 A1
20090172511 Decherd et al. Jul 2009 A1
20090177962 Gusmorino et al. Jul 2009 A1
20090179892 Tsuda et al. Jul 2009 A1
20090187464 Bai et al. Jul 2009 A1
20090222400 Kupershmidt et al. Sep 2009 A1
20090222759 Drieschner Sep 2009 A1
20090222760 Halverson et al. Sep 2009 A1
20090234720 George et al. Sep 2009 A1
20090249244 Robinson et al. Oct 2009 A1
20090265105 Davis et al. Oct 2009 A1
20090281839 Lynn et al. Nov 2009 A1
20090287470 Farnsworth et al. Nov 2009 A1
20090292626 Oxford Nov 2009 A1
20090315679 Bauchot et al. Dec 2009 A1
20100011282 Dollard et al. Jan 2010 A1
20100042922 Bradateanu et al. Feb 2010 A1
20100057716 Stefik et al. Mar 2010 A1
20100058212 Belitz et al. Mar 2010 A1
20100070523 Delgo et al. Mar 2010 A1
20100070842 Aymeloglu et al. Mar 2010 A1
20100070845 Facemire et al. Mar 2010 A1
20100070897 Aymeloglu et al. Mar 2010 A1
20100073315 Lee et al. Mar 2010 A1
20100082842 Lavrov et al. Apr 2010 A1
20100100963 Mahaffey Apr 2010 A1
20100103124 Kruzeniski et al. Apr 2010 A1
20100114887 Conway et al. May 2010 A1
20100121817 Meyer et al. May 2010 A1
20100122152 Chamberlain et al. May 2010 A1
20100131457 Heimendinger May 2010 A1
20100162176 Dunton Jun 2010 A1
20100173619 Hua et al. Jul 2010 A1
20100185984 Wright et al. Jul 2010 A1
20100191563 Schlaifer et al. Jul 2010 A1
20100191884 Holenstein et al. Jul 2010 A1
20100198684 Eraker et al. Aug 2010 A1
20100199225 Coleman et al. Aug 2010 A1
20100214117 Hazzani Aug 2010 A1
20100223543 Marston Sep 2010 A1
20100228812 Uomini Sep 2010 A1
20100250412 Wagner Sep 2010 A1
20100280857 Liu et al. Nov 2010 A1
20100281458 Paladino et al. Nov 2010 A1
20100293174 Bennett et al. Nov 2010 A1
20100306713 Geisner et al. Dec 2010 A1
20100313119 Baldwin et al. Dec 2010 A1
20100318924 Frankel et al. Dec 2010 A1
20100321399 Ellren et al. Dec 2010 A1
20100325526 Ellis et al. Dec 2010 A1
20100325581 Finkelstein et al. Dec 2010 A1
20100330801 Rouh Dec 2010 A1
20110022312 McDonough et al. Jan 2011 A1
20110029526 Knight et al. Feb 2011 A1
20110047159 Baid et al. Feb 2011 A1
20110060753 Shaked Mar 2011 A1
20110061013 Bilicki et al. Mar 2011 A1
20110066933 Ludwig Mar 2011 A1
20110074811 Hanson et al. Mar 2011 A1
20110078055 Faribault et al. Mar 2011 A1
20110078173 Seligmann et al. Mar 2011 A1
20110093327 Fordyce et al. Apr 2011 A1
20110093440 Asakura et al. Apr 2011 A1
20110111786 Rao May 2011 A1
20110117878 Barash et al. May 2011 A1
20110119100 Ruhl et al. May 2011 A1
20110137766 Rasmussen et al. Jun 2011 A1
20110153384 Horne et al. Jun 2011 A1
20110158469 Mastykarz Jun 2011 A1
20110161096 Buehler et al. Jun 2011 A1
20110167105 Ramakrishnan et al. Jul 2011 A1
20110170799 Carrino et al. Jul 2011 A1
20110173032 Payne et al. Jul 2011 A1
20110185316 Reid et al. Jul 2011 A1
20110202557 Atsmon et al. Aug 2011 A1
20110208724 Jones et al. Aug 2011 A1
20110213655 Henkin Sep 2011 A1
20110218934 Elser Sep 2011 A1
20110219450 McDougal et al. Sep 2011 A1
20110225198 Edwards et al. Sep 2011 A1
20110238495 Kang Sep 2011 A1
20110238553 Raj et al. Sep 2011 A1
20110251951 Kolkowtiz Oct 2011 A1
20110258158 Resende et al. Oct 2011 A1
20110270705 Parker Nov 2011 A1
20110276423 Davidson Nov 2011 A1
20110289397 Eastmond et al. Nov 2011 A1
20110289407 Naik et al. Nov 2011 A1
20110289420 Morioka et al. Nov 2011 A1
20110291851 Whisenant Dec 2011 A1
20110310005 Chen et al. Dec 2011 A1
20110314007 Dassa et al. Dec 2011 A1
20120010812 Thompson Jan 2012 A1
20120014560 Obrador et al. Jan 2012 A1
20120015673 Klassen et al. Jan 2012 A1
20120019559 Siler et al. Jan 2012 A1
20120032975 Koch Feb 2012 A1
20120036013 Neuhaus et al. Feb 2012 A1
20120036434 Oberstein Feb 2012 A1
20120050293 Carlhian et al. Mar 2012 A1
20120066296 Appleton et al. Mar 2012 A1
20120072825 Sherkin et al. Mar 2012 A1
20120079363 Folting et al. Mar 2012 A1
20120084118 Bai et al. Apr 2012 A1
20120106801 Jackson May 2012 A1
20120117082 Koperda et al. May 2012 A1
20120131512 Takeuchi et al. May 2012 A1
20120137235 TS et al. May 2012 A1
20120144335 Abeln et al. Jun 2012 A1
20120150578 Mangat et al. Jun 2012 A1
20120159307 Chung et al. Jun 2012 A1
20120159362 Brown et al. Jun 2012 A1
20120159399 Bastide et al. Jun 2012 A1
20120166929 Henderson et al. Jun 2012 A1
20120170847 Tsukidate Jul 2012 A1
20120173985 Peppel Jul 2012 A1
20120180002 Campbell et al. Jul 2012 A1
20120196557 Reich et al. Aug 2012 A1
20120196558 Reich et al. Aug 2012 A1
20120203708 Psota et al. Aug 2012 A1
20120208636 Feige Aug 2012 A1
20120216106 Casey Aug 2012 A1
20120221511 Gibson et al. Aug 2012 A1
20120221553 Wittmer et al. Aug 2012 A1
20120221580 Barney Aug 2012 A1
20120245976 Kumar et al. Sep 2012 A1
20120246148 Dror Sep 2012 A1
20120254129 Wheeler et al. Oct 2012 A1
20120268269 Doyle Oct 2012 A1
20120277914 Crow et al. Nov 2012 A1
20120284345 Costenaro et al. Nov 2012 A1
20120290879 Shibuya et al. Nov 2012 A1
20120296907 Long et al. Nov 2012 A1
20120311684 Paulsen et al. Dec 2012 A1
20120323888 Osann, Jr. Dec 2012 A1
20120330973 Ghuneim et al. Dec 2012 A1
20130005362 Borghei Jan 2013 A1
20130006426 Healey et al. Jan 2013 A1
20130006725 Simanek et al. Jan 2013 A1
20130006916 McBride et al. Jan 2013 A1
20130013642 Klein et al. Jan 2013 A1
20130018796 Kolhatkar et al. Jan 2013 A1
20130024268 Manickavelu Jan 2013 A1
20130046635 Grigg et al. Feb 2013 A1
20130046842 Muntz et al. Feb 2013 A1
20130060786 Serrano et al. Mar 2013 A1
20130061169 Pearcy et al. Mar 2013 A1
20130073377 Heath Mar 2013 A1
20130073454 Busch Mar 2013 A1
20130078943 Biage et al. Mar 2013 A1
20130086482 Parsons Apr 2013 A1
20130097482 Marantz et al. Apr 2013 A1
20130110877 Bonham et al. May 2013 A1
20130111320 Campbell et al. May 2013 A1
20130117651 Waldman et al. May 2013 A1
20130101159 Rosen Jun 2013 A1
20130143597 Mitsuya et al. Jun 2013 A1
20130150004 Rosen Jun 2013 A1
20130151148 Parundekar et al. Jun 2013 A1
20130151388 Falkenborg et al. Jun 2013 A1
20130157234 Gulli et al. Jun 2013 A1
20130165069 Nitta et al. Jun 2013 A1
20130166550 Buchmann et al. Jun 2013 A1
20130176321 Mitchell et al. Jul 2013 A1
20130179420 Park et al. Jul 2013 A1
20130196614 Pahlevani Aug 2013 A1
20130224696 Wolfe et al. Aug 2013 A1
20130225212 Khan Aug 2013 A1
20130226318 Procyk Aug 2013 A1
20130226953 Markovich et al. Aug 2013 A1
20130232045 Tai Sep 2013 A1
20130235749 Cho et al. Sep 2013 A1
20130238616 Rose et al. Sep 2013 A1
20130246170 Gross et al. Sep 2013 A1
20130251233 Yang et al. Sep 2013 A1
20130262171 Solodko et al. Oct 2013 A1
20130262497 Case et al. Oct 2013 A1
20130262527 Hunter et al. Oct 2013 A1
20130263019 Castellanos et al. Oct 2013 A1
20130267207 Hao et al. Oct 2013 A1
20130268520 Fisher et al. Oct 2013 A1
20130279757 Kephart Oct 2013 A1
20130282696 John et al. Oct 2013 A1
20130290011 Lynn et al. Oct 2013 A1
20130290825 Arndt et al. Oct 2013 A1
20130295970 Sheshadri et al. Nov 2013 A1
20130297619 Chandarsekaran et al. Nov 2013 A1
20130311375 Priebatsch Nov 2013 A1
20140019936 Cohanoff Jan 2014 A1
20140032506 Hoey et al. Jan 2014 A1
20140033010 Richardt et al. Jan 2014 A1
20140040371 Gurevich et al. Feb 2014 A1
20140047319 Eberlein Feb 2014 A1
20140047357 Alfaro et al. Feb 2014 A1
20140059038 McPherson et al. Feb 2014 A1
20140067611 Adachi et al. Mar 2014 A1
20140068487 Steiger et al. Mar 2014 A1
20140074855 Zhao et al. Mar 2014 A1
20140079340 Kawano Mar 2014 A1
20140081685 Thacker et al. Mar 2014 A1
20140093174 Zhang et al. Apr 2014 A1
20140095273 Tang et al. Apr 2014 A1
20140095509 Patton Apr 2014 A1
20140108068 Williams Apr 2014 A1
20140108380 Gotz et al. Apr 2014 A1
20140108985 Scott et al. Apr 2014 A1
20140129261 Bothwell et al. May 2014 A1
20140149436 Bahrami et al. May 2014 A1
20140156527 Grigg et al. Jun 2014 A1
20140157172 Peery et al. Jun 2014 A1
20140164502 Khodorenko et al. Jun 2014 A1
20140176606 Narayan et al. Jun 2014 A1
20140189536 Lange et al. Jul 2014 A1
20140195515 Baker et al. Jul 2014 A1
20140195887 Ellis et al. Jul 2014 A1
20140214579 Shen et al. Jul 2014 A1
20140222521 Chait Aug 2014 A1
20140244388 Manouchehri et al. Aug 2014 A1
20140258827 Gormish et al. Sep 2014 A1
20140267294 Ma Sep 2014 A1
20140267295 Sharma Sep 2014 A1
20140279824 Tamayo Sep 2014 A1
20140302783 Aiuto et al. Oct 2014 A1
20140310266 Greenfield Oct 2014 A1
20140316911 Gross Oct 2014 A1
20140333651 Cervelli et al. Nov 2014 A1
20140337772 Cervelli et al. Nov 2014 A1
20140344230 Krause et al. Nov 2014 A1
20140357299 Xu et al. Dec 2014 A1
20150005014 Huang et al. Jan 2015 A1
20150019394 Unser et al. Jan 2015 A1
20150046870 Goldenberg et al. Feb 2015 A1
20150080012 Sprague et al. Mar 2015 A1
20150089424 Duffield et al. Mar 2015 A1
20150100897 Sun et al. Apr 2015 A1
20150100907 Erenrich et al. Apr 2015 A1
20150134666 Gattiker et al. May 2015 A1
20150169709 Kara et al. Jun 2015 A1
20150169726 Kara et al. Jun 2015 A1
20150170077 Kara et al. Jun 2015 A1
20150178825 Huerta Jun 2015 A1
20150178877 Bogomolov et al. Jun 2015 A1
20150186821 Wang et al. Jul 2015 A1
20150187036 Wang et al. Jul 2015 A1
20150227295 Meiklejohn et al. Aug 2015 A1
20150331919 Freeland et al. Nov 2015 A1
20160110458 Colgrove et al. Apr 2016 A1
20170257743 Freeland Sep 2017 A1
Foreign Referenced Citations (39)
Number Date Country
102014103482 Sep 2014 DE
102014215621 Feb 2015 DE
1 672 527 Jun 2006 EP
2400448 Dec 2011 EP
2551799 Jan 2013 EP
2560134 Feb 2013 EP
2778977 Sep 2014 EP
2816513 Dec 2014 EP
2835745 Feb 2015 EP
2835770 Feb 2015 EP
2838039 Feb 2015 EP
2846241 Mar 2015 EP
2851852 Mar 2015 EP
2858014 Apr 2015 EP
2858018 Apr 2015 EP
2863326 Apr 2015 EP
2863346 Apr 2015 EP
2869211 May 2015 EP
2884439 Jun 2015 EP
2884440 Jun 2015 EP
2891992 Jul 2015 EP
2911078 Aug 2015 EP
2911100 Aug 2015 EP
2916276 Sep 2015 EP
2516155 Jan 2015 GB
2518745 Apr 2015 GB
2012778 Nov 2014 NL
2013306 Feb 2015 NL
624557 Dec 2014 NZ
WO 2000009529 Feb 2000 WO
WO 2002065353 Aug 2002 WO
WO 2004038548 May 2004 WO
WO 2005104736 Nov 2005 WO
WO 2008064207 May 2008 WO
WO 2009061501 May 2009 WO
WO 2010000014 Jan 2010 WO
WO 2010030913 Mar 2010 WO
WO 2013010157 Jan 2013 WO
WO 2013102892 Jul 2013 WO
Non-Patent Literature Citations (204)
Entry
Official Communication for European Patent Application No. 14180321.3 dated Apr. 17, 2015.
Official Communication for Australian Patent Application No. 2014201511 dated Feb. 27, 2015.
Official Commuincation for Australian Patent Application No. 2014202442 dated Mar. 19, 2015.
Official Communication for Great Britain Patent Application No. 1404457.2 dated Aug. 14, 2014.
Hansen et al., “Analyzing Social Media Networks with NodeXL: Insights from a Connected World”, Chapter 4, pp. 53-67 and Chapter 10, pp. 143-164, published Sep. 2010.
Official Communication for New Zealand Patent Application No. 628263 dated Aug. 12, 2014.
Gesher, Ari, “Palantir Screenshots in the Wild: Swing Sightings,” The Palantir Blog, Sep. 11, 2007, pp. 1- 12.
Official Communication for European Patent Application No. 15155846.7 dated Jul. 8, 2015.
Palmas et al., “An Edge-Bunding Layout for Interactive Parallel Coordinates” 2014 IEEE Pacific Visualization Symposium, pp. 57-64.
Manske, “File Saving Dialogs,” <http://www.mozilla.org/editor/ui_specs/FileSaveDialogs.html>, Jan. 20, 1999, pp. 7.
Huang et al., “Systematic and Integrative Analysis of Large Gene Lists Using DAVID Bioinformatics Resources,” Nature Protocols, 4.1, 2008, 44-57.
Definition “Identify”, downloaded Jan. 22, 2015, 1 page.
Official Communication for European Patent Application No. 14180142.3 dated Feb. 6, 2015.
Microsoft—Developer Network, “Getting Started with VBA in Word 2010,” Apr. 2010, <http://msdn.microsoft.com/en-us/library/ff604039%28v=office.14%29.aspx> as printed Apr. 4, 2014 in 17 pages.
Bugzilla@Mozilla, “Bug 18726—[feature] Long-click means of invoking contextual menus not supported,” http://bugzilla.mozilla.org/show_bug.cgi?id=18726 printed Jun. 13, 2013 in 11 pages.
Official Communication for Netherlands Patent Application No. 2013306 dated Apr. 24, 2015.
Official Communication for European Patent Application No. 14197879.1 dated Apr. 28, 2015.
“A Quick Guide to UniProtKB Swiss-Prot & TrEMBL,” Sep. 2011, pp. 2.
Chen et al., “Bringing Order to the Web: Automatically Categorizing Search Results,” CHI 2000, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, Apr. 1-6, 2000, The Hague, The Netherlands, pp. 145-152.
Official Communication for European Patent Application No. 14187996.5 dated Feb. 12, 2015.
Keylines.com, “An Introduction to KeyLines and Network Visualization,” Mar. 2014, <http://keylines.com/wp-content/uploads/2014/03/KeyLines-White-Paper.pdf> downloaded May 12, 2014 in 8 pages.
Dramowicz, Ela, “Retail Trade Area Analysis Using the Huff Model,” Directions Magazine, Jul. 2, 2005 in 10 pages, http://www.directionsmag.com/articles/retail-trade-area-analysis-using-the-huff-model/123411.
Official Communication for European Patent Application No. 14158861.6 dated Jun. 16, 2014.
Yang et al., “HTML Page Analysis Based on Visual Cues”, A129, pp. 859-864, 2001.
Li et al., “Interactive Multimodal Visual Search on Mobile Device,” IEEE Transactions on Multimedia, vol. 15, No. 3, Apr. 1, 2013, pp. 594-607.
Official Communication for Australian Patent Application No. 2014210614 dated Jun. 5, 2015.
Official Communication for European Patent Application No. 14159464.8 dated Oct. 8, 2014.
Kitts, Paul, “Chapter 14: Genome Assembly and Annotation Process,” The NCBI Handbook, Oct. 2002, pp. 1-21.
Official Communication for European Patent Application No. 14159464.8 dated Jul. 31, 2014.
Official Communication for Australian Patent Application No. 2014210604 dated Jun. 5, 2015.
Rouse, Margaret, “OLAP Cube,” <http://searchdatamanagement.techtarget.com/definition/OLAP-cube>, Apr. 28, 2012, pp. 16.
Goswami, Gautam, “Quite Writly Said!,” One Brick at a Time, Aug. 21, 2005, pp. 7.
Conner, Nancy, “Google Apps: The Missing Manual,” May 1, 2008, pp. 15.
Olanoff, Drew, “Deep Dive with the New Google Maps for Desktop with Google Earth Integration, It's More than Just a Utility,” May 15, 2013, pp. 1-6, retrieved from the internet: http://web.archive.org/web/20130515230641/http://techcrunch.com/2013/05/15/deep-dive-with-the-new-google-maps-for-desktop-with-google-earth-integration-its-more-than-just-a-utility/.
Official Communication for New Zealand Patent Application No. 628840 dated Aug. 28, 2014.
Map of San Jose, CA. Retrieved Oct. 2, 2013 from http://maps.bing.com.
Official Communication for Great Britain Patent Application No. 1413935.6 dated Jan. 27, 2015.
Official Communication for European Patent Application No. 14159464.8 dated Sep. 22, 2014.
Official Communication for Australian Patent Application No. 2014250678 dated Jun. 17, 2015.
Griffith, Daniel A., “A Generalized Huff Model,” Geographical Analysis, Apr. 1982, vol. 14, No. 2, pp. 135-144.
Kahan et al., “Annotea: An Open RDF Infrastructure for Shared Web Annotations”, Computer Networks, Elsevier Science Publishers B.V., vol. 39, No. 5, dated Aug. 5, 2002, pp. 589-608.
Official Communication for Australian Patent Application No. 2014213553 dated May 7, 2015.
Official Communication for New Zealand Patent Application No. 622513 dated Aug. 3, 2014.
Nierman, “Evaluating Structural Similarity in XML Documents”, 6 pages, 2002.
Official Communication for European Patent Application No. 14191540.5 dated May 27, 2015.
Official Communication for European Patent Application No. 14186225.0 dated Feb. 13, 2015.
Ananiev et al., “The New Modality API,” http://web.archive.org/web/20061211011958/http://java.sun.com/developer/technicalArticles/J2SE/Desktop/javase6/modality/ Jan. 21, 2006, pp. 8.
Chung, Chin-Wan, “Dataplex: An Access to Heterogeneous Distributed Databases,” Communications of the ACM, Association for Computing Machinery, Inc., vol. 33, No. 1, Jan. 1, 1990, pp. 70-80.
Official Communication for New Zealand Patent Application No. 627962 dated Aug. 5, 2014.
“A Word About Banks and the Laundering of Drug Money,” Aug. 18, 2012, http://www.golemxiv.co.uk/2012/08/a-word-about-banks-and-the-laundering-of-drug-money/.
Official Communication for Great Britain Patent Application No. 1404574.4 dated Dec. 18, 2014.
“Potential Money Laundering Warning Signs,” snapshot taken 2003, https://web.archive.org/web/20030816090055/http:/finsolinc.com/Anti-Money%20LAUNDERING%20TRAINING%20GUIDES.pdf.
Official Communciation for Great Britain Patent Application No. 1411984.6 dated Dec. 22, 2014.
GIS-NET 3 Public _ Department of Regional Planning. Planning & Zoning Information for Unincorporated LA County. Retrieved Oct. 2, 2013 from http://gis.planning.lacounty.gov/GIS-NET3_Public/Viewer.html.
Microsoft Office—Visio, “About connecting shapes,” <http://office.microsoft.com/en-us/visio-help/about-connecting-shapes-HP085050369.aspx> printed Aug. 4, 2011 in 6 pages.
Official Communication for European Patent Application No. 14189802.3 dated May 11, 2015.
Official Communication for European Patent Application No. 14189344.6 dated Feb. 20, 2015.
Definition “Overlay”, downloaded Jan. 22, 2015, 1 page.
Map of San Jose, CA. Retrieved Oct. 2, 2013 from http://maps.yahoo.com.
Manno et al., “Introducing Collaboration in Single-user Applications through the Centralized Control Architecture,” 2010, pp. 10.
Boyce, Jim, “Microsoft Outlook 2010 Inside Out,” Aug. 1, 2010, retrieved from the internet https://capdtron.files.wordpress.com/2013/01/outlook-2010-inside_out.pdf.
Wikipedia, “Federated Database System,” Sep. 7, 2013, retrieved from the internet on Jan. 27, 2015 http://en.wikipedia.org/w/index.php?title=Federated_database_system&oldid=571954221.
Official Communication for European Patent Application No. 14197895.7 dated Apr. 28, 2015.
Official Communication for European Patent Application No. 14199182.8 dated Mar. 13, 2015.
Huff et al., “Calibrating the Huff Model Using ArcGIS Business Analyst,” SSRI, Sep. 2008, pp. 33.
Official Communication for New Zealand Patent Application No. 624557 dated May 14, 2014.
Acklen, Laura, “Absolute Beginner's Guide to Microsoft Word 2003,” Dec. 24, 2003, pp. 15-18, 34-41, 308-316.
Celik, Tantek, “CSS Basic User Interface Module Level 3 (CSS3 UI),” Section 8 Resizing and Overflow, Jan. 17, 2012, retrieved from internet http://www.w3.org/TR/2012/WD-css3-ui-20120117/#resizing-amp-overflow retrieved on.
Liu, Tianshun, “Combining GIS and the Huff Model to Analyze Suitable Locations for a New Asian Supermarket in the Minneapolis and St. Paul, Minnesota USA,” Papers in Resource Analysis, 2012, vol. 14, pp. 8.
Official Communication for New Zealand Patent Application No. 622517 dated Apr. 3, 2014.
Keylines.com, “Visualizing Threats: Improved Cyber Security Through Network Visualization,” Apr. 2014, <http://keylines.com/wp-content/uploads/2014/04/Visualizing-Threats1.pdf> downloaded May 12, 2014 in 10 pages.
Official Communication for European Patent Application No. 14189347.9 dated Mar. 4, 2015.
Keylines.com, “KeyLines Datasheet,” Mar. 2014, <http://keylines.com/wp-content/uploads/2014/03/KeyLines-datasheet.pdf> downloaded May 12, 2014 in 2.
Official Communication for New Zealand Patent Application No. 628495 dated Aug. 19, 2014.
Umagandhi et al., “Search Query Recommendations Using Hybrid User Profile with Query Logs,” International Journal of Computer Applications, vol. 80, No. 10, Oct. 1, 2013, pp. 7-18.
Official Communication for New Zealand Patent Application No. 628161 dated Aug. 25, 2014.
Huff, David L., “Parameter Estimation in the Huff Model,” ESRI, ArcUser, Oct.-Dec. 2003, pp. 34-36.
Hogue et al., “Thresher: Automating the Unwrapping of Semantic Content from the World Wide Web,” 14th International Conference on World Wide Web, WWW 2005: Chiba, Japan, May 10-14, 2005, pp. 86-95.
Canese et al., “Chapter 2: PubMed: The Bibliographic Database,” The NCBI Handbook, Oct. 2002, pp. 1-10.
“Refresh CSS Ellipsis When Resizing Container—Stack Overflow,” Jul. 31, 2013, retrieved from internet http://stackoverflow.com/questions/17964681/refresh-css-ellipsis-when-resizing-container, retrieved on May 18, 2015.
Thompson, Mick, “Getting Started with GEO,” Getting Started with GEO, Jul. 26, 2011.
Delcher et al., “Identifying Bacterial Genes and Endosymbiont DNA with Glimmer,” Biolnformatics, vol. 23, No. 6, 2007, pp. 673-679.
Hardesty, “Privacy Challenges: Analysis: It's Surprisingly Easy to Identify Individuals from Credit-Card Metadata,” MIT News on Campus and Around the World, MIT News Office, Jan. 29, 2015, 3 pages.
Official Communication for New Zealand Patent Application No. 628585 dated Aug. 26, 2014.
Bluttman et al., “Excel Formulas and Functions for Dummies,” 2005, Wiley Publishing, Inc., pp. 280, 284-286.
Official Communication for European Patent Application No. 14180432.8 dated Jun. 23, 2015.
IBM, “Determining Business Object Structure,” IBM, 2004, 9 pages.
Official Communication for European Patent Application No. 14180281.9 dated Jan. 26, 2015.
Hibbert et al., “Prediction of Shopping Behavior Using a Huff Model Within a GIS Framework,” Healthy Eating in Context, Mar. 18, 2011, pp. 16.
Sirotkin et al., “Chapter 13: The Processing of Biological Sequence Data at NCBI,” The NCBI Handbook, Oct. 2002, pp. 1-11.
Madden, Tom, “Chapter 16: The BLAST Sequence Analysis Tool,” The NCBI Handbook, Oct. 2002, pp. 1-15.
Microsoft Office—Visio, “Add and glue connectors with the Connector tool,” <http://office.microsoft.com/en-us/visio-help/add-and-glue-connectors-with-the-connector-tool-HA010048532.aspx?CTT=1> printed Aug. 4, 2011 in 1 page.
Map of San Jose, CA. Retrieved Oct. 2, 2013 from http://maps.google.com.
Notice of Acceptance for Australian Patent Application No. 2014250678 dated Oct. 7, 2015.
Official Communication for Great Britain Patent Application No. 1408025.3 dated Nov. 6, 2014.
“A First Look: Predicting Market Demand for Food Retail using a Huff Analysis,” TRF Policy Solutions, Jul. 2012, pp. 30.
Amnet, “5 Great Tools for Visualizing Your Twitter Followers,” posted Aug. 4, 2010, http://www.amnetblog.com/component/content/article/115-5-grate-tools-for-visualizing-your-twitter-followers.html.
Sigrist, et al., “PROSITE, a Protein Domain Database for Functional Characterization and Annotation,” Nucleic Acids Research, 2010, vol. 38, pp. D161-D166.
“The FASTA Program Package,” fasta-36.3.4, Mar. 25, 2011, pp. 29.
Mizrachi, Ilene, “Chapter 1: Gen Bank: The Nuckeotide Sequence Database,” The NCBI Handbook, Oct. 2002, pp. 1-14.
Official Communication for European Patent Application No. 15157642.8 dated Jul. 20, 2015.
Official Communication for New Zealand Patent Application No. 622501 dated Apr. 1, 2014.
Official Communication for New Zealand Patent Application No. 622501 dated Jun. 5, 2014.
Official Communication for European Patent Application No. 14159447.3 dated Nov. 25, 2014.
Official Communication for European Patent Application No. 14159447.3 dated Jan. 8, 2015.
U.S. Appl. No. 13/247,987, filed Sep. 28, 2011, Office Action, dated Apr. 2, 2015.
U.S. Appl. No. 13/831,791, filed Mar. 15, 2013, Office Action, dated Mar. 4, 2015.
U.S. Appl. No. 13/835,688, filed Mar. 15, 2013, First Office Action Interview, dated Jun. 17, 2015.
U.S. Appl. No. 13/839,026, filed Mar. 15, 2013, Restriction Requirement, dated Apr. 2, 2015.
U.S. Appl. No. 14/102,394, filed Dec. 10, 2013, Notice of Allowance, dated Aug. 25, 2014.
U.S. Appl. No. 14/108,187, filed Dec. 16, 2013, Notice of Allowance, dated Aug. 29, 2014.
U.S. Appl. No. 14/135,289, filed Dec. 19, 2013, Notice of Allowance, dated Oct. 14, 2014.
U.S. Appl. No. 14/148,568, filed Jan. 6, 2014, Office Action, dated Oct. 22, 2014.
U.S. Appl. No. 14/148,568, filed Jan. 6, 2014, Office Action, dated Mar. 26, 2015.
U.S. Appl. No. 14/192,767, filed Feb. 27, 2014, Notice of Allowance, dated Dec. 16, 2014.
U.S. Appl. No. 14/196,814, filed Mar. 4, 2014, Office Action, dated Oct. 7, 2015.
U.S. Appl. No. 14/225,006, filed Mar. 25, 2014, First Office Action Interview, dated Sep. 10, 2014.
U.S. Appl. No. 14/225,006, filed Mar. 25, 2014, First Office Action Interview, dated Feb. 27, 2015.
U.S. Appl. No. 14/225,084, filed Mar. 25, 2014, First Office Action Interview, dated Sep. 2, 2014.
U.S. Appl. No. 14/225,084, filed Mar. 25, 2014, Notice of Allowance, dated May 4, 2015.
U.S. Appl. No. 14/225,084, filed Mar. 25, 2014, First Office Action Interview, dated Feb. 20, 2015.
U.S. Appl. No. 14/225,160, filed Mar. 25, 2014, Final Office, dated Feb. 11, 2015.
U.S. Appl. No. 14/225,160, filed Mar. 25, 2014, Advisory Action, dated May 20, 2015.
U.S. Appl. No. 14/225,160, filed Mar. 25, 2014, First Office Action Interview, dated Oct. 22, 2014.
U.S. Appl. No. 14/268,964, filed May 2, 2014, First Office Action Interview, dated Sep. 3, 2014.
U.S. Appl. No. 14/268,964, filed May 2, 2014, Notice of Allowance, dated Dec. 3, 2014.
U.S. Appl. No. 14/289,596, filed May 28, 2014, First Office Action Interview, dated Jul. 18, 2014.
U.S. Appl. No. 14/289,596, filed May 28, 2014, Final Office Action, dated Jan. 26, 2015.
U.S. Appl. No. 14/289,599, filed May 28, 2014, First Office Action Interview, dated Jul. 22, 2014.
U.S. Appl. No. 14/289,599, filed May 28, 2014, Final Office Action, dated May 29, 2015.
U.S. Appl. No. 14/294,098, filed Jun. 2, 2014, Final Office Action, dated Nov. 6, 2014.
U.S. Appl. No. 14/294,098, filed Jun. 2, 2014, First Office Action Interview, dated Aug. 15, 2014.
U.S. Appl. No. 14/294,098, filed Jun. 2, 2014, Notice of Allowance, dated Dec. 29, 2014.
U.S. Appl. No. 14/306,138, filed Jun. 16, 2014, Final Office Action, dated Feb. 18, 2015.
U.S. Appl. No. 14/306,138, filed Jun. 16, 2014, First Office Action Interview, dated Sep. 23, 2014.
U.S. Appl. No. 14/306,138, filed Jun. 16, 2014, Office Action, dated May 26, 2015.
U.S. Appl. No. 14/306,147, filed Jun. 16, 2014, First Office Action Interview, dated Sep. 9, 2014.
U.S. Appl. No. 14/306,147, filed Jun. 16, 2014, Final Office Action, dated Feb. 19, 2015.
U.S. Appl. No. 14/306,154, filed Jun. 16, 2014, First Office Action Interview, dated Sep. 9, 2014.
U.S. Appl. No. 14/306,154, filed Jun. 16, 2014, Final Office Action, dated Mar. 11, 2015.
U.S. Appl. No. 14/306,154, filed Jun. 16, 2014, Advisory Action, dated May 15, 2015.
U.S. Appl. No. 14/319,765, filed Jun. 30, 2014, First Office Action Interview, dated Feb. 4, 2015.
U.S. Appl. No. 14/319,765, filed Jun. 30, 2014, First Office Action Interview, dated Nov. 25, 2014.
U.S. Appl. No. 14/323,935, filed Jul. 3, 2014, Office Action, dated Jun. 22, 2015.
U.S. Appl. No. 14/323,935, filed Jul. 3, 2014, First Office Action Interview, dated Nov. 28, 2014.
U.S. Appl. No. 14/323,935, filed Jul. 3, 2014, First Office Action Interview, dated Mar. 31, 2015.
U.S. Appl. No. 14/326,738, filed Jul. 9, 2014, First Office Action Interview, dated Dec. 2, 2014.
U.S. Appl. No. 14/326,738, filed Jul. 9, 2014, First Office Action Interview, dated Mar. 31, 2015.
U.S. Appl. No. 14/473,552, filed Aug. 29, 2014, Interview Summary, dated Feb. 24, 2015.
U.S. Appl. No. 14/486,991, filed Sep. 15, 2014, Office Action, dated Mar. 10, 2015.
U.S. Appl. No. 14/504,103, filed Oct. 1, 2014, First Office Action Interview, dated Feb. 5, 2015.
U.S. Appl. No. 14/504,103, filed Oct. 1, 2014, Notice of Allowance, dated May 18, 2015.
U.S. Appl. No. 14/504,103, filed Oct. 1, 2014, First Office Action Interview, dated Mar. 31, 2015.
U.S. Appl. No. 14/579,752, filed Dec. 22, 2014, First Office Action Interview, dated May 26, 2015.
U.S. Appl. No. 14/616,080, filed Feb. 6, 2015, Notice of Allowance, dated Apr. 2, 2015.
U.S. Appl. No. 14/639,606, filed Mar. 5, 2015, First Office Action Interview, dated May 18, 2015.
U.S. Appl. No. 14/639,606, filed Mar. 5, 2015, First Action Interview, dated Jul. 24, 2015.
U.S. Appl. No. 14/306,147, filed Jun. 16, 2014, Office Action, dated Aug. 7, 2015.
U.S. Appl. No. 14/319,765, filed Jun. 30, 2014, Final Office Action, dated Jun. 16, 2015.
U.S. Appl. No. 14/289,596, filed May 28, 2014, Advisory Action, dated Apr. 30, 2015.
U.S. Appl. No. 14/225,006, filed Mar. 25, 2014, Final Office Action, dated Sep. 2, 2015.
U.S. Appl. No. 13/836,815, filed Mar. 15, 2013, Office Action, dated Oct. 24, 2015.
U.S. Appl. No. 14/579,752, filed Dec. 22, 2014, Final Office Action, dated Aug. 19, 2015.
U.S. Appl. No. 13/839,026, filed Mar. 15, 2013, Notice of Allowance, dated Jul. 6, 2015.
U.S. Appl. No. 14/306,154, filed Jun. 16, 2014, Office Action, dated Jul. 6, 2015.
U.S. Appl. No. 14/334,232, filed Jul. 17, 2014, Notice of Allowance, dated Nov. 10, 2015.
U.S. Appl. No. 14/306,138, filed Jun. 16, 2014, Final Office Action, dated Sep. 14, 2015.
U.S. Appl. No. 14/473,860, filed Aug. 29, 2014, Notice of Allowance, dated Jan. 5, 2015.
U.S. Appl. No. 13/831,199, filed Mar. 14, 2013, Office Action, dated May 9, 2016.
U.S. Appl. No. 14/690,905, filed Apr. 20, 2015, Office Action, dated Oct. 7, 2015.
U.S. Appl. No. 14/196,814, filed Mar. 4, 2014, First Office Action Interview, dated Apr. 16, 2014.
U.S. Appl. No. 14/813,749, filed Jul. 30, 2015, Office Action, dated Sep. 28, 2015.
U.S. Appl. No. 14/225,160, filed Mar. 25, 2014, Office Action, dated Aug. 12, 2015.
U.S. Appl. No. 13/831,199, filed Mar. 14, 2013, Final Office Action, dated Nov. 4, 2016.
U.S. Appl. No. 14/490,612, filed Sep. 18, 2014, Final Office Action, dated Aug. 18, 2015.
U.S. Appl. No. 14/486,991, filed Sep. 15, 2014, Notice of Allowance, dated May 1, 2015.
U.S. Appl. No. 14/027,118, filed Feb. 4, 2016, Notice of Allowance, dated Apr. 4, 2016.
U.S. Appl. No. 15/047,405, filed Feb. 18, 2016, Office Action, dated Apr. 1, 2016.
U.S. Appl. No. 14/726,353, filed May 29, 2015, First Office Action Interview, dated Sep. 10, 2015.
U.S. Appl. No. 13/247,987, filed Sep. 28, 2011, Office Action, dated Sep. 22, 2015.
U.S. Appl. No. 12/556,318, filed Sep. 9, 2009, Office Action, dated Jul. 2, 2015.
U.S. Appl. No. 14/487,342, filed Sep. 16, 2014, Notice of Allowance, dated Sep. 23, 2015.
U.S. Appl. No. 14/027,118, filed Feb. 4, 2016, Office Action, dated Sep. 16, 2015.
U.S. Appl. No. 14/631,633, filed Feb. 25, 2015, First Office Action Interview, dated Sep. 10, 2015.
U.S. Appl. No. 14/326,738, filed Jul. 9, 2014, Final Office Action, dated Jul. 31, 2015.
U.S. Appl. No. 13/831,199, filed Mar. 31, 2013, Final Office Action, dated Oct. 6, 2015.
U.S. Appl. No. 14/077,159, filed May 3, 2016, Office Action, dated Mar. 12, 2014.
U.S. Appl. No. 15/145,177, filed May 3, 2016, Office Action, dated Jul. 29, 2016.
U.S. Appl. No. 14/289,599, filed May 28, 2014, Advisory Action, dated Sep. 4, 2015.
U.S. Appl. No. 15/145,177, filed May 3, 2016, Final Office Action, dated Aug. 7, 2015.
U.S. Appl. No. 14/319,765, filed Jun. 30, 2014, Advisory Action, dated Sep. 10, 2015.
U.S. Appl. No. 14/077,159, filed Nov. 11, 2013, Notice of Allowance, dated Aug. 15, 2014.
U.S. Appl. No. 14/196,814, filed Mar. 4, 2014, Final Office Action, dated Dec. 14, 2016.
U.S. Appl. No. 14/196,814, filed Mar. 4, 2014, Notice of Allowance, dated Apr. 6, 2017.
U.S. Appl. No. 14/225,084, filed Mar. 25, 2014, Office Action, dated Sep. 11, 2015.
U.S. Appl. No. 13/839,026, filed Mar. 15, 2013, Office Action, dated Aug. 4, 2015.
U.S. Appl. No. 14/088,251, filed Nov. 22, 2013, Office Action, dated Aug. 26, 2015.
U.S. Appl. No. 14/196,814, filed Mar. 4, 2014, Final Office Action, dated Jun. 13, 2016.
U.S. Appl. No. 14/134,558, filed Dec. 19, 2013, Office Action, dated Oct. 7, 20165.
U.S. Appl. No. 13/831,791, filed Mar. 15, 2013, Final Office Action, dated Aug. 6, 2015.
U.S. Appl. No. 15/598,752, filed May 18, 2017, Final Office Action, dated Aug. 29, 2017.
U.S. Appl. No. 14/806,517, filed Jul. 22, 2015, First Office Action Interview, dated Oct. 26, 2016.
U.S. Appl. No. 14/806,517, filed Jul. 22, 2015, Notice of Allowance, dated Feb. 2, 2017.
U.S. Appl. No. 13/831,791, filed Mar. 15, 2013, Final Office Action, dated Mar. 4, 2015.
Related Publications (1)
Number Date Country
20180084383 A1 Mar 2018 US
Provisional Applications (1)
Number Date Country
61759283 Jan 2013 US
Continuations (3)
Number Date Country
Parent 15598752 May 2017 US
Child 15822621 US
Parent 14806517 Jul 2015 US
Child 15598752 US
Parent 13838815 Mar 2013 US
Child 14806517 US