The present invention relates to mobile communication devices, and more particularly, to recognizing objects in a digitally captured image of a mobile device.
The use of portable electronic devices and mobile communication devices has increased dramatically in recent years. Moreover, mobile communication devices are offering more features to enhance the user experience. One feature is a mobile device camera which allows a user to take digital pictures. The user can send the picture to other users over a communications network for providing a shared user experience. The user can include a text to describe the picture or upload attachments to associate with the image. However, the picture may include objects with are unfamiliar to the user or other users. A need therefore exists for identifying objects within an image and providing descriptive information related to the objects.
One embodiment is a method for providing location-specific image information. The method can include receiving a captured an image of at least one object using a mobile device, recognizing the at least one object from both the image and a location of the mobile device, and retrieving location-specific information associated with the at least one object in the image in response to the recognizing. The method can further include identifying at least one business affiliated with the at least one object, retrieving an advertisement associated with the at least one business that can be overlaid onto the image. The method can further include identifying contact information associated with the at least one object from the location-specific information, and displaying the contact information on the mobile device. The step of recognizing can further include receiving the image and the location of the mobile device at an image server, recognizing the at least one object from the image and the location of the mobile device, and sending the location-specific information to the mobile device.
One embodiment is directed to a method for advertisement on a mobile device. The method can include receiving a captured image containing at least one object, determining a location of the mobile device, recognizing the at least one object from both the image and the location in a database of images, retrieving advertisements associated with the at least one object that can be overlaid onto the image. The method can further include recognizing a building in the image, identifying a business associated with the building, and retrieving an advertisement associated with the business. The method can further include identifying a coordinate of the business in the image, and overlaying the advertisement at the coordinate in the image. The method can further include receiving a camera zoom setting on the mobile device, generating a search radius for the at least one object in the image from the camera zoom setting, and performing the recognizing based on the search radius. The method can further include receiving a camera compass heading on the mobile device, generating a direction vector of the at least one object in the image from the camera compass heading, and performing the recognizing based on the viewing angle. The method can further include receiving a camera focus setting on the mobile device, generating a search arc for the viewing angle from the camera focus setting, and performing the recognizing based on the search arc. The method can further include adjusting the size of the overlay in proportion to an advertising revenue.
One embodiment is directed to a system for providing location-specific image information on a mobile device. The system can include an image server having a communication unit that receives the image and the location of the mobile device, an image database of objects to associate with the image at the location, and a recognition engine that recognizes the at least one object from the location and the image from the image database. The image server can retrieve location-specific information for the at least one object that is recognized and sends the location-specific information to the mobile device.
The system can further include an address server communicatively coupled to the image server that generates contact information for the location-specific information and that is associated with the at least one object. The system can further include an advertisement server communicatively coupled to the image server that retrieves advertisements associated with the location-specific information. The advertisement server can send the advertisements to the mobile device, and the mobile device can overlay the advertisements onto the image. The system can further include a mobility manager communicatively coupled to the address server that can monitor a location of users in a push-over-cellular (PoC) system and identify users that are at a location corresponding to the at least one object. In one aspect, the at least one object is a building, the image database contains a plurality of street-level images of buildings, and the image server recognizes a building from the image database and generates an address for the building. A list of contacts for the building can be generated and sent to the mobile device. In one arrangement, the mobility manager can sort the contact list in order of social activity level. A profile can also be included that determines the mobile device's displayed list of contacts based on a time or day.
One embodiment is an electronic apparatus for providing image specific information. The apparatus can include a camera that captures an image of at least one object, a locator coupled to the camera that identifies a location of the mobile device, a processor that recognizes the image; and a user interface that renders a composite of the original image and one or more sources of image specific information. Upon recognizing the image, the processor can identify image specific information associated with the image including contacts, advertisements, or messages that can be overlaid on the image in the user interface. The processor can also generate audio for the image specific information associated with the image.
The features of the system, which are believed to be novel, are set forth with particularity in the appended claims. The embodiments herein, can be understood by reference to the following description, taken in conjunction with the accompanying drawings, in the several figures of which like reference numerals identify like elements, and in which:
While the specification concludes with claims defining the features of the embodiments of the invention that are regarded as novel, it is believed that the method, system, and other embodiments will be better understood from a consideration of the following description in conjunction with the drawing figures, in which like reference numerals are carried forward.
As required, detailed embodiments of the present method and system are disclosed herein. However, it is to be understood that the disclosed embodiments are merely exemplary, which can be embodied in various forms. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the embodiments of the present invention in virtually any appropriately detailed structure. Further, the terms and phrases used herein are not intended to be limiting but rather to provide an understandable description of the embodiment herein.
The terms “a” or “an,” as used herein, are defined as one or more than one. The term “plurality,” as used herein, is defined as two or more than two. The term “another,” as used herein, is defined as at least a second or more. The terms “including” and/or “having,” as used herein, are defined as comprising (i.e., open language). The term “coupled,” as used herein, is defined as connected, although not necessarily directly, and not necessarily mechanically. The term “image” can be defined as a picture or scene represented digitally. The term “location specific information” can be defined as information related one or more objects within an image. The term “recognizing” can be defined as identifying an object from visual aspects of an image. The term “building” can be defined as a physical structure.
Broadly stated, embodiments of the invention are directed to a method and system for capturing an image on a mobile device, identifying a location of the mobile device, and sending the image with the location of the mobile device to an image server that can recognize at least one object in the image. Camera settings can also be sent with the location for narrowing a search of the object in an image database. The image server can respond with location specific information associated with the at least one object given the location. The mobile device can present the location specific information with the image. As one example, the object can be a building in a street-level image scene and the location specific information can be an advertisement that is overlaid on the building. Advertisements or messages can be overlaid on multiple objects that are recognized in the image. As another example, the object can be a building and the location specific information can identify a list of personal or business contacts in the building. A user can take a picture of a building and receive a list of contacts associated with the building, or businesses within the building.
Referring to
Mobile devices in the wireless communication system 100 can also connect amongst one another over a Wide Local Area Network (WLAN) within an ad-hoc group. The WLAN provides wireless access within a local geographical area. The mobile devices can communicate with the WLAN according to the appropriate wireless communication standard. In another arrangement, the mobile devices can communicate amongst themselves in a peer-to-peer ad-hoc network without infrastructure or WLAN support. For example, the mobile device can use short-range radio communication to engage in a group call in a peer-to-peer mode. In a typical WLAN implementation, the physical layer can use a variety of technologies such as 802.11b or 802.11g Wireless Local Area Network (WLAN) technologies. The physical layer may use infrared, frequency hopping spread spectrum in the 2.4 GHz Band, or direct sequence spread spectrum in the 2.4 GHz Band, or any other suitable communication technology.
Briefly, the mobile device 110 can capture a picture, such as a street-level image, and send the picture to the image server 120. The image server 120 can recognize objects, such as buildings, within the picture. The image database 125 can include a plurality of images, such as street-level images, images of buildings, or images of businesses. Notably, the street-level images may include one or more buildings, and one or more businesses. The buildings and the businesses can each have an associated address. The image server 120 can identify the address of a building from the image using the image database 125 to provide a mapped location of the building. The image server 120 can send the address of the recognized buildings or businesses to the mobile device 110.
The image server 120 can also send the address to an address server 130, which can provide a list of contacts in the building or business. The address server 130 can include a contact database 135 that associates contact information with a given address. For example, upon receiving a building address from the image server 120, the address server 130 can generate a list of personal contacts associated with the building or business contacts in the building.
The image server 120 can also send the address to an advertisement server 140. The advertisement server 140 can include an advertisement database 145 having advertisements associated with business addresses or business contacts. The advertisement server 140, upon receiving address information or contact information from the image server 120, can send advertisements or location-specific information associated with the addresses or contacts to the image server 120. The image server 120 can send the advertisements and location-specific information to the mobile device 110.
The mobile device 110 can present the advertisements and/or location-specific information with the image. As one example, the mobile device 110 can overlay the advertisements at locations in the image corresponding to the location of the building or business recognized in the image. The user can visually see the advertisements overlaid on the image. As another example, the mobile device 110 can present a list of contacts associated with a business or building in the image. The list of contacts can include phone numbers, dispatch numbers, group identification numbers, web site names, or any other contact communication information. The user can call the contacts directly upon receiving the contact information.
Referring to
At step 201, the method can start. At step 202, an image of at least one object can be captured. For example, referring to
The mobile device 110 can include a camera 112 that captures the image 121, a locator 114 coupled to the camera 112 that identifies a location of the mobile device 110, and a modem 116 coupled to the locator 114 that transmits the street-level image 121 and the location. The mobile device 110 can also include a processor 118 for coordinating a capturing of the image 121 and transmitting the image. The user can transmit the image 121 to the image server 120 to receive more location-specific information related to the building. The processor 118 can also produce an audio representation of the image-specific information. For example, the image specific information may be promotional audio and video advertisements associated with a business that is recognized in an image. The video advertisements can be overlaid on the image 121 in the user-interface 119 and an audio representation of the advertisements can be played through a speaker or headset of the mobile device 110. The user-interface 119 allows a user to receive visual feedback associated with the image specific information and to interact with the image specific information.
Returning back to
Returning back to
Returning back to
Returning back to
The location specific information may also include notes or messages left by other individuals. For example, upon receiving the advertisements, a user may provide comments regarding the advertisement, such as favorable or negative reviews of the advertisement. Referring back to
In yet another arrangement, the mobility manager 115 can support a blog community that allows users to discuss topics related to a certain building, business, or advertisement. In such regard, users can subscribe to the blog to keep posted of events occurring at the location, even if they are not physically at the location. For example, a user may receive a promotional advertisement associated with a business the user captures in a street-level image. The user can subscribe to the blog to receive updated promotional information, or group user feedback related to the advertisement. Returning back to
Another embodiment of the invention is directed to ad-hoc group call through listing and social network finding using street-level images for a push-to-talk (PTT) over Cellular device. Briefly, upon taking a picture of a street-level image, a user can be provided with contact information for businesses or people associated with recognized buildings or businesses in the street-level image. For example, referring to
Referring to
At step 301, the method 300 can start. At step 302, an image of a building can be captured using a mobile device. Consider that a user captures an image of an office building and wants to generate a list of dispatch or interconnect numbers along with corresponding individual and/or business names. The list can be of the individuals that are currently in the building and the businesses that are represented in the building. Most large office buildings will have numerous businesses that reside within the building so a listing of businesses that are relevant to user would be beneficial. For example, referring to
At step 304, a location of the mobile device can be determined. For example, referring back to
Referring to
Referring to
In yet another arrangement, a location watermark can be provided in the image to identify a search location and to narrow a field of search. For example, the user can overlay a watermark of a business logo or a text message to further identify the business or building. In one aspect, the user may already know the name of the business, but may want the mobile device to retrieve contacts associated with the business. This can further reduce the search scope for recognizing the building or business, as well as the contacts associated with the building or business. Referring back to
Returning back to
At step 308, an address of the building can be identified in response to the recognizing. For example, referring back to
At step 310, contact information can be retrieved from the address. Returning back to
In another aspect, a mobility manager can provide location information to the address server 130 for all users in the call group of a PoC system. The address server 130 can provide dynamic contact information to the user based on who is currently in the building. For example, referring to
For example, referring to
In one arrangement, the mobility manager 115 can inform users of a location of other users in a call group. For example, the mobility manager 115 can identify a location of a first user A 110 and a second user B 112. If User A 110 and User B 112 are registered to the same call group, the mobility manager 115 can inform each of the mobile devices the whereabouts of the other device. As another example, a user may keep a profile that determines the mobile device's displayed list of users based on the time of day, day of the week, calendar, or location. In particular, upon a user taking a picture of a building, the mobility manager 115 can determine other users in the call group that are currently in the building. As shown in
Returning back to
In another aspect, the mobile device 110 or address server 120 may further reduce the contact listing through social network analysis. The mobile device may promote candidates based on a history of contact information in the phone. For example, a mobile device may prioritize a list of individuals or businesses based on recent calls or calling activity to a particular user, business, or contact listed in the mobile device. Moreover, the address server 130 or mobility manager 115 can keep an account of social networking activity. The mobility manager 115 can order the contact list sent from the image server 120 to the mobile device in order of priority. The address server 130 may utilize a much larger scope of activity to determine the social network of influence. As an example, the address server 130 may promote contacts within a building based on the degrees of separation from the user a contact may be. Moreover, the user may create and utilize a profile that determines the device's displayed list of candidates based on the time of day, day of the week, calendar or location.
Where applicable, the present embodiments of the invention can be realized in hardware, software or a combination of hardware and software. Any kind of computer system or other apparatus adapted for carrying out the methods described herein are suitable. A typical combination of hardware and software can be a mobile communications device with a computer program that, when being loaded and executed, can control the mobile communications device such that it carries out the methods described herein. Portions of the present method and system may also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein and which when loaded in a computer system, is able to carry out these methods.
While the preferred embodiments of the invention have been illustrated and described, it will be clear that the embodiments of the invention is not so limited. Numerous modifications, changes, variations, substitutions and equivalents will occur to those skilled in the art without departing from the spirit and scope of the present embodiments of the invention as defined by the appended claims.