Patent Application
20120120100
Services such as Google Maps are capable of displaying street level images, known as “Street Views”, of geographic locations. A client computer requests a street level image from a particular location, and receives an image (often a digitized panoramic 360° photograph) in response. These images typically comprise photographs of buildings and other features taken at a time prior to the request (often by a vehicle equipped with cameras), and allow a user to view a geographic location from a person's perspective as compared to a top-down map perspective.
To aid users while viewing street level images, Google also provides image overlays on the street level image for the purpose of navigation. These overlays may include, for example, arrows that a user may click to navigate down a road to the next street level image.
A variety of websites also provide information about geographic locations. For example, the National Weather Service of the National Oceanic and Atmospheric Administration's provides the current weather conditions at various user-selectable locations. The service (e.g., http://www.weather.gov/xml/current_obs) also provides XML files containing values such as character strings that describe current weather conditions (e.g., “<weather>A Few Clouds </weather>”) at locations that may be identified by a user. The XML files also contain references to a small icon (e.g., 55×58 pixels) that is available from the site and illustrates the current weather conditions. Foe example, an icon showing a few clouds against a sunny sky may be retrieved by going to the location referenced in the XML file, such as “<icon_url_base> http://weather.gov/weather/images/fcicons/</icon_url_base><icon_url_name>few.jpg</icon_url_name>”).
Yet other websites provide traffic conditions (http://www.dot.ca.gov/cgi-bin/roads.cgi) and the time of day (http://www.time.gov) in response to a user identifying a location.
In one embodiment, this disclosure provides a system comprising a memory storing instructions, and a processor in communication with the memory in order to process data in accordance with the instructions. The system may also include a display in communication with, and displaying data received from, the processor. The processor may be configured by the instructions to provide data identifying a location, request and display an image depicting a geographic object corresponding to the location, and display, over a portion of the image, a text box that contains information provided by a person at the location after the image was taken and prior to the request for the image.
In another embodiment of the system, the text box contains text obtained from a server that allows users to upload live descriptions of an event.
In a further embodiment of the system, the text box displayed in the image is displayed relative to the geographic object based on where the information was received relative to the geographic object.
In yet another embodiment of the system, the processor is housed within a mobile device.
In another embodiment, this disclosure provides a computer-implemented method. The computer-implemented method may include providing, by a processor, data that identifies a location, requesting, by the processor, an image depicting a geographic object corresponding to the identified location, and providing data, by the processor, for display of a text box that contains information provided by a user at the identified location after the image was taken and prior to the request for the image.
In another embodiment of the method, the text box displayed in the image is displayed relative to the geographic object based on where the information was received relative to the geographic object.
In a further embodiment of the method, the method includes requesting, by the processor, the information from a server, wherein the information was provided by the user at the identified location.
In yet another embodiment of the method, the server allows users to upload live descriptions of an event.
Another embodiment for a system is also provided. In one embodiment, the system includes a memory storing instructions and at least one processor in communication with the memory. The processor may be configured by the instructions to receive data from a remote device, the data identifying a location, provide data for display of an image by the remote device, the image depicting a geographic object, and provide data for display of a text box by the remote device, the text box containing information provided by a user at the identified location after the image was taken and prior to the request for the image.
In one aspect, the system and method provides a modified image in response to a request for a street level image at a particular location, wherein the previously captured image is modified to illustrate the current conditions at the requested location. By way of example only, the system and method may use local weather, time of day, traffic or other information to update street level images.
As shown in
Memory 220 stores information accessible by processor 210, including instructions 240 that may be executed by the processor 210. It also includes data 230 that may be retrieved, manipulated or stored by the processor. The memory may be of any type capable of storing information accessible by the processor, such as a hard-drive, memory card, ROM, RAM, DVD, CD-ROM, write-capable and read-only memories. The processor 210 may be any well-known processor, such as processors from Intel Corporation or AMD. Alternatively, the processor may be a dedicated controller such as an ASIC.
The instructions 240 may be any set of instructions to be executed directly (such as machine code) or indirectly (such as scripts) by the processor. In that regard, the terms “instructions,” “steps” and “programs” may be used interchangeably herein. The instructions may be stored in object code form for direct processing by the processor, or in any other computer language including scripts or collections of independent source code modules that are interpreted on demand or compiled in advance. Functions, methods and routines of the instructions are explained in more detail below.
Data 230 may be retrieved, stored or modified by processor 210 in accordance with the instructions 240. For instance, although the system and method is not limited by any particular data structure, the data may be stored in computer registers, in a relational database as a table having a plurality of different fields and records, XML documents, or flat files. The data may also be formatted in any computer-readable format such as, but not limited to, binary values, ASCII or Unicode. Moreover, the data may comprise any information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations) or information which is used by a function to calculate the relevant data.
Although the processor and memory are functionally illustrated in
In one aspect, computer 110 is a server communicating with one or more client computers 150, 170 (only client 150 being shown in
Although the client computers 150 and 170 may comprise a full-sized personal computer, many aspects of the system and method are particularly advantageous when used in connection with mobile devices capable of wirelessly exchanging data with a server over a network such as the Internet. For example, client computer 170 may be a wireless-enabled PDA such as a Blackberry phone or an Internet-capable cellular phone. In either regard, the user may input information using a small keyboard (in the case of a Blackberry phone), a keypad (in the case of a typical cell phone), a touch screen (in the case of a PDA) or any other means of user input.
Client computers 150 and 170 may include a component, such as circuits, to determine the geographic location of the device. For example, mobile device 170 may include a GPS receiver 155. By way of further example, the component may include software for determining the position of the device based on other signals received at the mobile device 170, such as signals received at a cell phone's antenna from one or more cell phone towers if the mobile device is a cell phone.
The server 110 and client computers 150, 170 are capable of direct and indirect communication, such as over a network 295. Although only a few computers are depicted in
Although certain advantages are obtained when information is transmitted or received as noted above, other aspects of the system and method are not limited to any particular manner of transmission of information. For example, in some aspects, information may be sent via a medium such as a disk, tape or CD-ROM. In other aspects, the information may be transmitted in a non-electronic format and manually entered into the system. Yet further, although some functions are indicated as taking place on a server and others on a client, various aspects of the system and method may be implemented by a single computer having a single processor.
Map database 270 of server 110 stores map-related information, at least a portion of which may be transmitted to a client device. For example, map database 270 may store map tiles 272, where each tile is a map image of a particular geographic area. Depending on the resolution (e.g., whether the map is zoomed in or out), one tile may cover an entire region, such as a state, in relatively little detail. Another tile may cover just a few streets in high detail. The map information of the system and method is not limited to any particular format. For example, the images may comprise street maps, satellite images, or a combination of these, and may be stored as vectors (particularly with respect to street maps) or bitmaps (particularly with respect to satellite images). The various map tiles are each associated with geographical locations, such that the server 110 is capable of selecting, retrieving and transmitting one or more tiles based on a receipt of a geographical location or range of geographical locations.
The map database may also store street level images 274. Street level images comprise images of objects captured by cameras at particular geographical locations in a direction roughly parallel to the ground. By way of example only, a single street level image may show a perspective view of a street and its associated buildings, taken at a position a few feet above the ground (e.g., from a camera mounted on top of a vehicle and at or below the legal limit for typical vehicles in certain states (approximately 7-14 feet)) and in a direction roughly parallel to the ground (e.g., the camera view was generally pointed down the street into the distance)). Street level images are not limited to any particular height above the ground, for example, a street level image may be taken from the top of a building.
In one aspect of the system and method, the street level images are panoramic images, such as 360° panoramas centered at the geographic location associated with the image. The panoramic street-level view image may be created by stitching together a plurality of photographs representing different camera angles taken from the same location. In other aspects, only a single street level image pointing in a particular direction may be available at any particular geographical location. The street level images are thus typically associated with both a geographical location and information indicating the orientation of the image. For example, each image may be associated with both a latitude and longitude, and data that allows one to determine which portion of the image corresponds with facing north, south, east, west, northwest, etc. Many street level images may be sized in the range of 3,000 to 13,000 pixels wide by 1,600 to 6,000 pixels high; however, unless otherwise stated, it will be understood that the system and method is not limited to images of any particular size.
Street level images may also be stored in the form of videos, such as MPEG videos captured by an analog video camera or time-sequenced photographs that were captured by a digital still camera.
In one aspect of the system and method, the images are captured by a camera prior to a request by the user for the image. For example, the image may have been captured days or longer before the request.
As discussed in more detail, the street level images 274 may also be associated with data defining a portion of the street level image, where this portion may be modified to correspond with current conditions at the location captured in the street level image.
Data 230 may also include different images associated with potential environmental conditions at the locations captured in street level images. For example and as shown in
System 100 may further include a source that provides information about the current conditions at a geographical location. These sources may be stored at the server 110 or, as shown in
Another potential source of routinely-changing location-specific conditions comprises traffic server 292. For example, the server may track traffic at a number of different locations. When provided with a location, traffic server 292 returns a value indicative of the extent of traffic at the location.
The information provided by the servers may not precisely match conditions at the location. For example, the information stored in the servers may lag current conditions and the information may relate to locations proximate to the requested location (e.g., the nearest city with a weather station). Accordingly, and unless specifically stated to the contrary, it will be understood that references to current conditions at geographic locations actually refer to the current conditions at geographic locations as determined by the system, and not necessarily to the conditions existing at that precise current moment at that precise location.
In addition to the operations illustrated in
Server 110 retrieves the appropriate street level image based on the requested location. For example, if the street level images are stored based on the latitude/longitude coordinates of the camera that captured the image, the closest image to the requested latitude/longitude will be retrieved.
Upon identifying the location to be displayed, server 110 determines whether portions of the image should be modified to reflect current conditions at the location. For example, as shown in the street level image 501 of
The system and method also retrieves information reflecting the current conditions at the requested location. For example, server 110 may determine whether there is any weather information associated with a requested latitude/longitude position by transmitting the latitude/longitude to weather server 290. Alternatively, server 270 may translate and transmit the location requested by the user into a location format used by the server. For example, the server 110 may use the latitude/longitude of the requested position to determine and transmit the name of the nearest city known to the weather server. In response, weather server 290 returns an indication of the current weather conditions at the location, such as a character string indicating “partly cloudy” or “sunny” conditions.
Server 110 also determines whether it has access to image data associated with the current condition at the location. For example, if current weather conditions indicate that the sky is partly cloudy at the requested location, server 110 queries the condition-reflective images 260 for data representing an image corresponding with the partly cloudy condition (such as a bitmap or instructions for drawing a partly cloudy sky). In response, an image of a partly cloudy image 610 such as that shown in
If server 110 fails to obtain access to current conditions, or lacks information enabling it modify the image to reflect current conditions, the server may simply send the street level image.
An image is then created based on both the previously-captured street level image of the requested location and the image data associated with current conditions at the requested location. For example, as shown in
The resultant image 750 is transmitted to client 150 for display, such as part of a web page 760. The transmission may occur as a single bitmap prepared by the server, or as multiple images (such as multiple bitmap files) and sufficient information for the client computer to display the street level image and the condition-reflective images together. In that regard, the resultant image represents both actual objects at the location and actual weather conditions.
In one aspect of the system and method, the images of the condition-reflective images 260 are structured to create a visually pleasing image. For example, the condition-reflective images may comprise previously captured images of the sky in climates similar to the requested location rather than drawings or cartoon-style images as is common with many icons.
Moreover, the condition-reflective images may be selected or structured to correspond as much as possible with the street level images (or at least a sizeable plurality of such images). For example, the condition-reflective images may be stored in sizes that correspond with the size of the street level image in order to minimize distortion if the condition-reflective image needs to be enlarged to match the street level image, or to minimize processing time of the condition-reflective images if the condition-reflective image needs to be subsampled to match the street level image. Yet further, the condition-reflective images may be selected to correspond with the orientation of the camera angle of typical street level images. For example, condition-reflective images of skies may be selected to include sky images that were captured from a camera that is relatively close to, and oriented parallel to, the ground rather than a camera pointing straight up.
In that regard, in one aspect of the system and method, the condition-reflective images comprise images that were captured at locations unrelated to the location requested by the user. Even so, the condition-reflective images and the street level image may be combined to create the appearance that the newly added portion and previously-captured portion were captured at the same time and location.
The modified street level image is then displayed on the client computer. For example, as shown in more detail in
If the weather had been completely overcast, a condition-reflective image corresponding with overcast skies would have been used instead. The result would be a street level image showing an overcast sky even if the street level image stored in the map database captured a clear sunny sky.
The street level image 810 may be shown in the browser along with controls 840 for zooming the image and controls 850 for changing the orientation of the view (which may require another street level image being retrieved and modified with a condition-reflective image). Other navigation controls may be included as well, such as panning controls in the form of arrows disposed along the street. Such arrows may be selected by a user (by clicking or by dragging along the street line) to change the vantage point from up or down the street.
Although modifying previously-captured images with images associated with current weather conditions has particular advantages, the system and method allows for modification based on other types of data as well. For example, the sky of the street level image may be modified to reflect the time of day at the requested location. As shown in
The system and method also allows for modification of other portions of the image. For example, as shown in
The system and method may modify the street level image so that the amount of vehicles shown on the image's streets reflect the current traffic conditions of that street. When the desired location is received, server 110 may determine the name of the street(s) captured in the image (such as by using a geocoder on the latitude/longitude position of the street level image). The server may then use the name of the street to query traffic server 292 (
In one aspect of showing traffic, the server retrieves an image of a car. As shown in
As shown in
In other aspects of the system and method, the server compares the previously-captured image with the current conditions and only modifies the image if the conditions do not match. For example, in response to a request from a user for a street level image of a particular street, a processor may retrieve the currently-stored street level image and use image processing (e.g., pattern recognition) to determine the approximate number of vehicles on the street. This information, in turn, is used to obtain a value indicative of the amount traffic shown in the picture. The processor also obtains a value, from a source of traffic data, associated with the current amount of traffic on the street.
If the traffic value of the image is less than the source's traffic value, photo-realistic images of cars may be superimposed on those portions of the street lacking a car. These images may be obtained from the map database or by replicating images of cars in the image being analyzed. If the traffic value of the image is more than the source's traffic value, the processor may replace images of the cars with images of pavement (such as by replicating images of pavement in the image being analyzed). Regardless of the source of the condition-reflective images, the images may be sized and oriented to provide as much realism as possible.
In lieu of showing traffic, the street level image may also be modified to show additional information about current road conditions such as by adding traffic cones or road closure signs to the street if the street is closed.
Other aspects of the system and method do not identify particular areas of the street level image to be modified. For example, as shown in
In one alternative aspect, the system and method displays information that (1) would not have been captured by the camera that captured the street level image but (2) is associated with the location.
Yet other aspects of the system and method incorporate combinations of conditions at the requested location. By way of example, if the user requests a street level image of a location, and if the conditions at the location indicate that it is night-time with cloudy skies and relatively little traffic, and processor may take the existing image and replace a clear daytime sky with a cloudy nighttime sky and remove images of cars from a busy street.
In another aspect of the system and method, the server 110 periodically downloads and caches conditions at various locations, and uses the cached information to modify the requested image.
Most of the foregoing alternative embodiments are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. As these and other variations and combinations of the features discussed above can be utilized without departing from the invention as defined by the claims, the foregoing description of the embodiments should be taken by way of illustration rather than by way of limitation of the invention as defined by the claims. It will also be understood that the provision of examples of the invention should not be interpreted as limiting the invention to the specific examples; rather, the examples are intended to illustrate only one of many possible embodiments.
This application is a continuation of U.S. patent application Ser. No. 12/414,878, filed on Mar. 31, 2009, the disclosure of which is incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12414878 | Mar 2009 | US |
| Child | 13357884 | US |
