Patent Application
20110137548
Daily life is complicated by a bewildering array of interactions, choices, fragments of information, and commitments. For example, when a person visits a place that she finds interesting, she might want to mark the location and share it with friends. Or, this same person may have a thought that she would like to discuss with someone else at a later time. Currently, users may enter these items into existing time-based calendars. But there are many cases that the exact time cannot be specified when scheduling an event, or where a particular time is not called for. Moreover, existing time-based calendars remind users based on a pre-determined time interval, despite the fact that it may take different amounts of time for the user to reach the location of the appointment.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Described herein is an architecture and system for implementing a life organizer that is geolocation-centric. A user may input geo-tagged multi-modal information, or notes, into mobile devices. In one implementation, acquisition of a note may be started and stopped by shaking the device. The current geolocation of the device may be associated with the note and uploaded to cloud computing resources.
After receiving the note with geolocation information, an implicit search based on the geolocation of the input may be undertaken by the cloud computing resources and results provided to the user. Input may comprise flags to designate a personal point of interest (PPOI), files for sharing with others, location or people-based reminders, and so forth. Reminders for upcoming activities may be made at intervals determined to allow sufficient time for the user to move from the current geolocation to the geolocation of the activity. The results from this implicit search as well as reminders may then be provided to the user, either via the mobile device or a stationary device such as a desktop computer. These results may include a map showing the location associated with results and reminders, and be presented using a quick zoom feature. Furthermore, a time-to-leave may be determined and presented, as well as a time-to-arrive.
The detailed description is set forth with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items.
This disclosure describes architectures, systems and methods for implementing a multi-modal, geolocation-centric life organizer. In one example described herein, a user generates a note with a mobile device, and the note is associated with the current geolocation of the mobile device. The user may then access the note and view the note contents as well as information resulting from an automatic search including geolocation-specific information.
The discussion begins with a section entitled “Illustrated Architecture,” which describes one non-limiting architecture in which the claimed techniques may be implemented. A section entitled “Example Interface and Search Results” depicts and describes search results presented to the user. A third section entitled “Example Process of Acquiring and Presenting a Note” follows. Finally, a brief conclusion ends the discussion.
This brief introduction, including section titles and corresponding summaries, is provided for the reader's convenience and is not intended to limit the scope of the claims, nor the proceeding sections. Furthermore, the techniques described in detail below may be implemented in a number of ways and in a number of contexts. One example implementation and context is provided with reference to the following figures, as described below in more detail. However, it is to be appreciated that this following implementation and context is but one of many.
Processor 104(1) may comprise one or more processors, and is configured to execute programmatic instructions. The memory 104(2) is configured to store instructions and data for use by the processor 104(1). Memory may include any computer readable storage media, including random access memory (RAM), non-volatile RAM (NVRAM), magnetic memory, optical memory, and so forth.
The shake detection module 104(3) is configured to determine relative motion of the device 102(1) and determine if the unit is being shaken. That is, the shake detection module 104(3) is configured to determine if device 102(1) is being moved briskly to and fro intentionally by the user. Shake detection module 104(3) may incorporate or access accelerometers, strain gauges, or other sensors to determine when the shaking motion occurs.
Device 102(1), meanwhile, may be configured to begin acquisition of a note in response to determining that the user has shaken the device 102(1). This acquisition may include recording audio, recording video, accepting speech for a speech-to-text speech recognition application, and so forth. Notes may also include appointment data comprising a call for action upon satisfying a particular condition or condition, such as a particular date, time, place, completion of other event, and so forth, or simply a flag indicating the user to follow up such as determine the date of the appointment at a later time. For instance, a user may be prompted to stop at a garage for an oil change when it has been six months since the last oil change, they are currently within one mile of the garage, and they have one hour unallocated for on their schedule.
In one implementation, shake detection module 104(3) may incorporate the following process to determine an intentional shake as opposed to an unintentional jostling.
To increase the robustness, additional constraints may be applied. For example, in some instances a user typically shakes a device at about the same frequency. Similarly, users in general tend to shake at approximately the same frequency. Therefore, the detected potential shakes may be further validated with using periodicity checking, either for a general frequency range or for a user-defined frequency range. Thus, shakes violating a valid frequency range may be discarded. Furthermore, the shake detection module 104(3) may be “trained” by the user to recognize a deliberate shake.
In some implementations, the “shake to acquire” may be available even while device 102(1) is “locked.” For example, device 102(1) may be in a “locked” state, that is, unresponsive to keypad inputs from a user. Device 102(1) may be placed into a locked state for many reasons, including prevention of accidental access to features. However, upon shaking device 102(1), shake detection module 104(3) determines a shake is taking place, and initiates acquisition of a note. Note acquisition may be discontinued by another shake, a pre-determined threshold such as time, recording length, and so forth.
Device 102(1) may also comprise a note input module 104(4), which is configured to accept note input. Note inputs may include recording audio, accepting a text entry, taking a picture, recording a video, or a combination of these.
A geolocation module 104(5) is configured to determine the geographic location, or geolocation, of the device 102(1). Geolocation module 104(5) may access a location service from a wireless communication service, global positioning system (GPS), accept manual input, and so forth. Location may be updated continuously, at pre-determined intervals, or upon occurrence of an event, such as taking a note. Notes or appointments may include prompts for a user to specify a geolocation. For example, a user within a building may not have access to GPS signals, and thus may designate a previously acquired geolocation as being proximate to the current location. Furthermore, by localizing to proximate areas, downloads of maps to the device 102(1) may be minimized, as the currently stored map in memory 104(2) may be sufficient.
Communication module 104(6) is configured to allow device 102 to communicate with other devices, servers, communication networks, and so forth. For example, communication module may include a wireless transceiver, physical connection, and so forth.
Stored within memory 104(2) may be one or more notes 106(1), . . . , 106(N). Memory 104(2) may also store applications. An application, such as a browser or other client application, running on device 102 may facilitate access cloud resources.
Non-portable computing devices such as desktop computer 104(D) may comprise a processor 104(1), a memory 104(2), and a communication module 104(6). Other computing devices 102 may comprise at least a processor 104(1), memory 104(2), and a communication module 104(6). Non-portable computing devices such as desktop computer 104(D) and limited portability devices such as laptop 102(2) may be used to retrieve notes and supplemental information, as described later in this application.
Devices 102(1)-(D) may access a cloud 108 of network resources via communication module 104(6) connecting to a network. The network may include any one or combination of multiple different types of networks, such as cable networks, the Internet, and wireless networks.
The cloud 108 of network resources may comprise one or more servers 110(1), . . . , 110(S). Servers 110(1)-(S) may accept notes 106(1)-(N) from devices 102(1)-(D), provide results 112 comprising a search based on geolocation, maps, and so forth to devices 102(1)-(D). The architecture and functions of servers 110(1)-(S) are discussed in
A storage module 202 is configured to accept notes 106(1)-(N) and associated data and direct them for storage in database module 204. Database module 204 may be within server 110, or in some implementations may be separate from server 110 but accessible thereto.
Query module 206 is configured to take geolocation data associated with the note 106 and execute a query using the geolocation data to find information which may be of interest to the user. This query module may execute the query without prior activation by the user. For example, suppose a user enters note 106(1) which includes the words “four score and seven years ago . . . ” Geolocation module 104(5) determines a geolocation of the device at the time that the note is input to the device and associates the determined geolocation with the note 106(1). Query module 206 runs a query for data which is not only specific to the user (such personal appointments) but information which is related to the geolocation such as what restaurants are proximate to the location of note 106(1).
Reminder module 208 is configured to use the geolocation information gathered from geolocation module 104(5) to determine when reminders should be issued for appointments or other time- or event-sensitive notes. A current geolocation of device 102 may be used by reminder module 208 to update reminders. These updates may occur continuously, in response to a change in geolocation, at pre-determined intervals, upon a change in event status such as a new appointment being added, upon demand, and so forth. For example, in one implementation reminders may be dynamically updated to reflect a changing geolocation of the portable electronic device Furthermore, in some implementations, filters may be placed on reminders, thus presenting reminders for today, or for a particular city, or with a particular person, and so forth.
For example, suppose a user has a reminder for a personal appointment for lunch at a pre-determined geolocation across town. This may be a reminder which has been recently entered as a note, or which was previously stored. For routable addresses, that is addresses for which a “to and from” path in geographic space can be determined, reminder module 208 may determine that the current travel time from the current location of device 102(1) to the location of the appointment. For example, the travel time may be 47 minutes, and thus a prompt may be sent to device 102(1) indicating the travel time, and an estimated time-to-leave (TTL) as to when the user should leave in order to reach the appointment on time. Note that this differs from a conventional calendar. For example, continuing our example above, the current time is 10:53 a.m., the lunch appointment may be at 12:00 p.m., and the user receives a prompt to leave the airport within 20 minutes, that is, by 11:13 a.m. in order to arrive on time at the appointment.
A user may define transportation preferences, such as walking, bicycle, public transportation, car, airline schedules, and so forth. In another implementation, the system may provide recommendations based on optimal timing. For example, it may be faster to walk than drive to a given appointment, even when driving is preferred.
Reminder module 208 may also be configured to provide a time-to-arrive (TTA) which calculates an estimated arrival time. TTA may be adjusted to include conditions such as transportation preferences, traffic congestion, time of day, road conditions, travel delays, and so forth.
Server 110 may include a mapping module 210 configured to use the geolocation information associated with note 106 and provide relevant maps, such as that of a city or neighborhood where the note 106 was taken. Maps may be provided with an overlaid quick zoom grid, which is described in more depth below with regards to
Note that in some implementations some or all of these modules may be distributed across multiple servers, be provided by an outside third party service, or a combination thereof.
Delivery module 212 is configured to provide query results, reminders, maps, and so forth to devices 102(1)-(D). Delivery may use a “push” model where content is sent to the device 102 without prior interrogation, a “pull” model where content is sent to the device 102 after an interrogation, or a combination of the two. Delivery module 212 may also format and arrange data in a manner commensurate with the type of display available on device 102. For example, delivery module 212 may reformat information to fit on the smaller screen of smartphone 102(1).
A geolocation service module 214 may also be available to server 110. Geolocation service module 214 may be configured to determine a geolocation of a device 102. Geolocation service module 214 may determine geolocation based on a network-address-to-physical-address lookup, interrogating a wireless service provider, interrogating device 102, or interrogating the user of device 102.
A reservation module 216 may also be present and configured to place reservations based on appointment data. For example, when reminder module 208 determines an appointment is located at a particular restaurant, conference room, or other location, reservation module 216 may generate reservation information for the user. Thus, a reminder to meet at restaurant “A” may include contact information or a one-click reservation option suitable for establishing a reservation for a table at the time of the appointment.
At 302, contents of the note 106(1) are shown, in this example the text “four score and seven years ago . . . .” As described above, a geolocation may be assigned to a note during acquisition of the note. Query module 206 may then query one or more databases to determine information which may be relevant to the user. At 304 search results from this query are presented. Note that in some implementations, the search was initiated by the acquisition of the note, not by a specific request from the user to search.
At 306, the geolocation information which was used by query module 206 is shown. In this example, note 106(1) was taken while at Austin-Bergstrom International Airport in Austin, Tex. at a geolocation of 30.19444° North and 97.67° West. A list of detailed search results 308 associated with this geolocation is provided. This association may be topical or based on proximity between the geolocation of the note and the geolocation of the information in the database, such as the address of restaurants at the airport.
Shown in the list of detailed search results 308 are various airport specific items including links for surface transportation options, airport services, current FAA delay information, restaurants, people knows to the user who are close by, and two reminders. One reminder indicates the user is scheduled to pick-up Kristi at Coast Airlines Gate G7. Another reminder indicates that a meeting with George is scheduled at the office, and provides a TTL estimate that the user should depart the airport within 20 minutes (that is, by 11:13 a.m.) in order to make this meeting on time.
A map 310 may also be shown which is associated with the points of interest. In this example, a large scale map of the state of Texas is depicted. Map 310 may be provided to device 102 by mapping module 210 which has also provided a quick zoom overlay grid 312. The quick zoom overlay grid allows the user to quickly zoom into a map without the cumbersome process of stepping through intermediate levels. In this example, when a user selects the lower right quadrant of the quick zoom overlay grid 312, the map 310 may be changed to display quick zoom view 314 showing the selected quadrant at greater magnification. The quick zoom overlay grid 312 may be presented on successive views, although in some implementations when the greatest available magnification has been achieved it may be suppressed from the display. The quick zoom overlay grid 312 may, but need not be, arranged into a rectilinear configuration. A clock showing local time 316, in this example “10:53 a.m.” may also be presented.
Process 400 includes operation 402 in which a user begins note acquisition on a portable electronic device 102(1) by shaking the device. In other implementations, activation of a control or other input may initiate note acquisition.
Operation 404 shows the determination of geolocation of the device 102(1). In this illustration, geolocation module 104(5) in device 102(1) uses GPS signals to determine that the device is located at 30.19444° North and 97.67° West. In other implementations, geolocation may be determined by geolocation service module 214 on server 110, such as through network address lookup against a geographic database or by interrogating a wireless service provider, and so forth. In cases where device 102(1) is in a low-power mode, geolocation may be deferred to a later time. Geolocation service module 214 may also take geolocation information provided by device 102(1) and provide additional lookups, for example, determining that 30.19444° North and 97.67° West is the site of Austin-Bergstrom International Airport in Austin, Tex.
Next, operation 406 involves assigning the determined geolocation to the note. In this example, note 106(1) was taken at geolocation 30.19444° North and 97.67° West.
Operation 408, query module 206 initiates a search based on geolocation of the note and associates the results with the note. Thus, database module 204 is queried for information associated with geolocation 30.19444° North and 97.67° West. In some implementations, the query may also include information contained within the note as well. For example, if the note mentions “Rob's BBQ House” the query module 206 may search for promotional offers sponsored by “Rob's BBQ House.” Search results may also comprise reminders from reminder module 208, maps generated by mapping module 210, or a combination of these.
Finally, operation 410 presents the note and associated search results to the user. This presentation may be implemented by way of delivery module 212 which provides the search results to device 102(1) for presentation to the user.
Block 502 begins acquisition of a note at a device upon a pre-determined user action. This user action may comprise a keypress, a shake of the unit, or other activating signal.
Block 504 determines geolocation of the device. Block 506 assigns the determined geolocation to the note. Block 508 initiates a search based on the determined geolocation. In some implementations the search may be initiated before acquisition of the note is concluded.
Block 510 generates a map associated with the search results which may incorporate search results, and overlays the map with a quick zoom grid. Block 512 presents the note, search results, and map on the device.
Although specific details of illustrative methods are described with regard to the figures and other flow diagrams presented herein, it should be understood that certain acts shown in the figures need not be performed in the order described, and may be modified, and/or may be omitted entirely, depending on the circumstances. As described in this application, modules and engines may be implemented using software, hardware, firmware, or a combination of these. Moreover, the acts and methods described may be implemented by a computer, processor or other computing device based on instructions stored on memory, the memory comprising one or more computer-readable storage media (CRSM).
The CRSM may be any available physical media accessible by a computing device to implement the instructions stored thereon. CRSM may include, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, compact disk read-only memory (CD-ROM), digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computing device.
