Patent Application
20070118509
The present invention relates to tagging digital images with one or more keywords, and more specifically relates to suggesting keywords for tagging the digital images.
Given the recent advances in digital camera technology, users may accumulate digital image collections having hundreds of digital images. As a result, it is becoming increasingly difficult for users to organize their digital image collections such that they can easily and quickly find desired images. One method for assisting a user in finding digital images within his collection is tagging the digital images with keywords such as “Christmas 2004,” “Italy,” “Vacation,” and the like. However, the typical the tagging process is manually intensive. In general, tagging typically occurs after the user has uploaded numerous digital images from his digital camera to his personal computer. Each digital image may then be tagged with one or more keywords by either manually entering the keywords or selecting the keywords from a static list of keywords previously created by the user.
One problem associated with the typical tagging process is that the process becomes prohibitively time consuming for the average user if the user must manually enter the desired keywords. Further, even in the systems that allow the user to select the desired keywords from a static list of keywords, the user may be forced to sort through many keywords that are not particularly relevant to find the desired keywords. In addition, the static list may not contain many relevant keywords. Thus, there remains a need for a system that automates much of the tagging process and that intelligently suggests keywords to the user.
The present invention provides a collaborative location keyword registry service for suggesting keywords for tagging a digital media such as a digital image. In general, the location keyword registry service is implemented on a network connected server. A plurality of users interact with the location keyword registry service to vote for keywords in a list of keywords for each of a number of locations, add keywords to the list of keywords for each of the locations, and optionally contest keywords in the list of keywords for each of the locations. As a result, the location keyword registry service maintains a list of keywords for each of the locations.
When a user desires to receive suggested keywords for a digital image, a request including positional information is sent from a requesting node, such as a digital camera or personal computer, to the location keyword registry service via a network. The positional information defines a location of a digital camera at the time of capturing the digital image. If the request is sent from the digital camera prior to capturing the digital image, the positional information defines the location of the digital camera at the time of sending the request. If the request is sent from the digital camera or other device to which the digital image is transferred after capturing the digital image, the positional information defines the location of the digital camera at the time the digital image was captured. In response to the request, the location keyword registry service provides a number of most popular keywords for the location and optionally a number of preferred keywords for the location previously defined by the user of the digital camera by interacting with the location keyword registry service and/or a number of less popular keywords for the location selected at random.
In one embodiment, the request is sent directly from the digital camera to the location keyword registry service via the network. In another embodiment, the digital image is transferred from the digital camera to a device such as a personal computer, and the device sends the request directly to the location keyword registry service via the network. In yet another embodiment, the request is sent from either the digital camera or the device to which the digital image is transferred to a keyword registry service via the network. The keyword registry service then sends the request to the location keyword registry service.
Those skilled in the art will appreciate the scope of the present invention and realize additional aspects thereof after reading the following detailed description of the preferred embodiments in association with the accompanying drawing figures.
The accompanying drawing figures incorporated in and forming a part of this specification illustrate several aspects of the invention, and together with the description serve to explain the principles of the invention.
The embodiments set forth below represent the necessary information to enable those skilled in the art to practice the invention and illustrate the best mode of practicing the invention. Upon reading the following description in light of the accompanying drawing figures, those skilled in the art will understand the concepts of the invention and will recognize applications of these concepts not particularly addressed herein. It should be understood that these concepts and applications fall within the scope of the disclosure and the accompanying claims.
The present invention relates to a system and method for suggesting keywords for tagging digital images based on positional information. While the description herein focuses on digital images, the present invention is equally applicable to suggesting keywords for tagging digital media such as, but not limited to, digital images, digital videos, and the like. As used herein, a digital media refers to a digital image, a digital video, or the like while “a plurality of digital media,” “a number of digital media,” or the like refers to multiple digital images, digital videos, or the like.
In general, a collaborative location keyword registry service for suggesting keywords for tagging a digital image is provided. The location keyword registry service is implemented on a network-connected server, and a plurality of users interact with the location keyword registry service to vote for keywords in a list of keywords for each of a number of locations, add keywords to the list of keywords for each of the locations, and optionally contest keywords in the list of keywords for each of the locations. As a result, the location keyword registry service maintains a list of keywords for each of the locations. When a user desires to receive suggested keywords for a digital image, a request including positional information is sent from a requesting node, such as a digital camera or personal computer, to the location keyword registry service via a network. In response, the location keyword registry service provides a number of most popular keywords for the location and optionally a number of preferred keywords for the location previously defined by the user of the digital camera by interacting with the location keyword registry service and/or a number of less popular keywords for the location selected at random.
Using the wireless communication interface 16, the digital camera 12 is coupled to a network 18 via an access point 20. Preferably, the network 18 is the Internet. The access point 20 may be a base station in cellular network. Note that there may be additional cellular network infrastructure coupling the access point 20 to the network 18, as will be apparent to one of ordinary skill in the art. Alternatively, the access point 20 may be an access point for a local wireless communication network.
The system 10 also includes a keyword service 22, a location prediction service 24, a history prediction service 26, a location keyword registry service 28, a photosharing service 30, and a user node 32. The keyword service 22 operates to provide suggested keywords in response to a request from the digital camera 12. The request from the digital camera 12 may be sent before capturing a digital image and includes positional information from the SPS receiver 14 defining the location of the digital camera 12 at the time of sending the request. The positional information may define the location of the digital camera 12 in terms of latitude and longitude. Alternatively, the request from the digital camera 12 may be sent after capturing a digital image and includes positional information from the SPS receiver 14 defining the location of the digital camera 12 at the time the digital image is captured. Note that the positional information may be stored in association with the digital image at the time of capturing the digital image. For example, the positional information may be stored as metadata in a digital image file corresponding to the digital image.
As described below in more detail, upon receiving the request from the digital camera 12, the keyword service 22 communicates with one or more of the location prediction service 24, the history prediction service 26, and the location keyword registry service 28 to obtain keywords based on the positional information. The keyword service 22 then combines the keywords from the services 24-28 to provide the suggested keywords to the digital camera 12. It should be noted that while the discussion herein focuses on the use of all of the services 24-28, the present invention is not limited thereto. Any combination of one or more of the services 24-28 may be used. In addition, rather than sending the request for suggested keywords to the keyword service 22, the request may be sent directly to one or more of the services 24-28.
The location prediction service 24 operates to obtain keywords previously used by users associated with the photosharing service 30 to tag digital images captured at the location or proximate to the location defined by the positional information from the digital camera 12. As described below, the location prediction service 24 may request that the photosharing service 30 search the digital images hosted by the photosharing service 30 for digital images captured at or proximate to the location defined by the positional information. In response, the photosharing service 30 searches for digital images captured at the location or proximate to the location defined by the positional information and returns statistics to the location prediction service 24. The statistics may include the keywords used to tag the digital images found during the search and the number of occurrences of times each of the keywords. The location prediction service 24 may then select a number of most popular keywords from the statistics from the photosharing service 30 and return the most popular keywords to the keyword service. The number of most popular keywords returned by the location prediction service 24 may be predetermined by the user of the digital camera 12, determined based on a priority assigned to the location prediction service 24 by the user, or determined based on feedback of ones of the suggested keywords selected by the user to tag the digital images. The number of most popular keywords returned may additionally or alternatively be based on a maximum number of keywords to be suggested to the digital camera 12.
The history prediction service 26 operates to obtain keywords previously used by the user of the digital camera 12 to tag digital images captured at the location or proximate to the location defined by the positional information from the digital camera 12. Note that the history prediction service 26 obtains keywords based on a search of the user's previously captured digital image, whereas the location prediction service 24 obtains keywords based on digital images hosted by the photosharing service 30 for a number of users of the photosharing service 30.
Based on the positional information and optionally a username, date, or time, the history prediction service 26 may communicate with the photosharing service 30 to obtain keywords previously used by the user to tag digital images captured at the location or proximate to the location defined by the positional information. In one embodiment, the photosharing service 30 searches the digital images hosted by the photosharing service 30 that are associated with the user of the digital camera 12 to find digital images previously captured by the user at the location or proximate to the location defined by the positional information. The photosharing service 30 then returns statistics to the history prediction service 26. The statistics may include the keywords previously used by the user to tag the digital images found during the search and the number of occurrences of times each of the keywords. The history prediction service 26 may then select a number of most popular keywords from the statistics from the photosharing service 30 and returns the most popular keywords to the keyword service. The number of most popular keywords returned by the history prediction service 26 may be predetermined by the user of the digital camera 12, determined based on a priority assigned to the history prediction service 26 by the user, or determined based on feedback of ones of the suggested keywords selected by the user to tag the digital images. The number of most popular keywords returned may additionally or alternatively be based on a maximum number of keywords to be suggested to the digital camera 12.
As described below in more detail, the location keyword registry service 28 operates to provide keywords for the location defined by the positional information from the digital camera 12 based on a collaborative process. More specifically, the location keyword registry service 28 preferably includes a webpage or website operating as a user interface of the location keyword registry service 28. Users associated with user nodes such as the user node 32 interact with the location keyword registry service 28 via a web browser 34 in order to nominate keywords for locations, vote on the keywords to define the most popular keywords for the locations, set user preferences, and optionally contest keywords previously nominated for the locations.
Thereafter, when the location keyword registry service 28 receives a request for keywords for a particular location, the location keyword registry service 28 may return the most popular keywords for the location and, if defined, preferred keywords for the location previously defined by the user. The location keyword registry service 28 may also return one or more less popular keywords for the location selected at random. The number of keywords returned may be defined by the user of the digital camera 12, determined by the keyword service 22 based on a priority assigned to the location keyword registry service 28 by the user, or determined based on feedback of ones of the suggested keywords selected by the user to tag the digital images. Note that the feedback of selected keywords may be used by the keyword service 22 to determine which of the services 24-28 are most popular with the user. The number of keywords returned may additionally or alternatively be based on a maximum number of keywords to be suggested to the digital camera 12.
The photosharing service 30 is part of a photosharing system such as a centrally hosted photosharing system or a peer-to-peer (P2P) photosharing system. In a centrally hosted photosharing system, the digital images for any number of users are hosted by a central server. The photosharing service 30 may be implemented on the central server. In a P2P photosharing system, the digital images are hosted by peer nodes, where the peer nodes may be a computer or the like associated with a user. The peer nodes may be communicatively coupled through a proxy as described in U.S. patent application Ser. No. 10/813,839, entitled METHOD AND SYSTEM FOR PROVIDING WEB BROWSING THROUGH A FIREWALL IN A PEER TO PEER NETWORK, filed on Mar. 31, 2004, currently pending, which is hereby incorporated by reference in its entirety. In the P2P photosharing system, the photosharing service 30 may be implemented on a central node along with the proxy such that the photosharing service 30 is communicatively coupled to each of the peer nodes in the P2P photosharing system.
While only one photosharing service 30 is illustrated in
Although the keyword service 22, the location prediction service 24, the history prediction service 26, the location keyword registry service 28, and the photosharing service 30 are illustrated in
First, the digital camera 12 sends a request to the keyword service 22 (step 200). The request includes positional information and optionally one or more of a username, date, and time. As examples, the request may be an Extensible Markup Language (XML) request or a Uniform Resource Locator (URL) including a query. The request may be sent before the user of the digital camera 12 captures a digital image, and the positional information defines the location of the digital camera 12 at the time of sending the request. If the request includes the date and time, they are the date and time of sending the request. Alternatively, the request may be sent at some point in time after capturing a digital image. More specifically, the request may be sent immediately after capturing the digital image or some time after capturing the digital image when the digital camera 12 establishes a connection with the network 18: The positional information defining the location of the digital camera 12 and optionally the date and time may be obtained from the SPS receiver 14 at the time of capturing the digital image and stored by the digital camera 12 in association with the digital image. For example, the positional information, the date, and the time may be stored in metadata associated with the digital image. More specifically, image file formats such as the Joint Photographic Experts Group (JPEG) format enable the storage of the positional information, the date, and the time within the image file for the digital image as metadata. Generally, metadata is any information describing the digital image and may include information about the digital camera 12 and its settings. Alternatively, the positional information, the date, and the time may be stored in the memory of the digital camera 12 in association with the digital image.
As stated above, the request may also include a username. The username may be entered by the user of the digital camera 12 or stored in the digital camera 12. Alternatively, the username may be some indicia identifying the digital camera 12 which may be associated with the user of the digital camera 12 at the keyword server 22. For example, if the digital camera 12 is a camera phone, the username may be a telephone number of the camera phone.
Upon receiving the request, the keyword server 22 may invoke the location prediction service 24 (step 202). Invoking the location prediction service 24 is preferably performed by sending a request including the positional information received from the digital camera 12 to the location prediction service 24. In response, the location prediction service 24 sends a request to the photosharing service 30 instructing the photosharing service 30 to search the digital images hosted by the photosharing service 30 for digital images captured at the location or proximate to the location defined by the positional information from the digital camera 12 (step 204). The photosharing service 30 then performs the search and returns statistics including the keywords used to tag the digital images found during the search and the number of occurrences of each of the keywords (step 206).
The location prediction service 24 then returns one or more of the keywords in the statistics to the keyword service 22 (step 208). In one embodiment, the location prediction service 24 selects a number of most popular keywords from the statistics from the photosharing service 30 and returns the most popular keywords to the keyword service 22. The number of keywords returned by the location prediction service 24 may be predefined by the user of the digital camera 12 or controlled based on a priority assigned to the location prediction service 24 by the user of the digital camera 12. Additionally or alternatively, the number of keywords returned by the location prediction service 24 may be controlled based on a maximum number of keywords to be suggested to the user at the digital camera 12.
The keyword service 22 may then invoke the history prediction service 26 (step 210). The history prediction service 26 operates similarly to the location prediction service 24. However, the history prediction service 26 searches only the previous digital images of the user of the digital camera 12, whereas the location prediction service 26 searches the digital images 12 captured by a number of users of the photosharing service 30.
The history prediction service 26 is preferably invoked by sending a request to the history prediction service 26 including the positional information and a username of the user of the digital camera 12. In response, the history prediction service 26 sends a request to the photosharing service 30 instructing the photosharing service 30 to search the digital images associated with the user of the digital camera 12 that are hosted by the photosharing service 30 to find digital images captured at the location or proximate to the location defined by the positional information from the digital camera 12 (step 212). The photosharing service 30 then performs the search and returns statistics including the keywords previously used by the user to tag the digital images found during the search and the number of occurrences each of the keywords was used (step 214).
The history prediction service 26 then returns one or more keywords from the statistics to the keyword service 22 (step 216). In one embodiment, the history prediction service 26 selects a number of most popular keywords from the statistics from the photosharing service 30 and returns the most popular keywords to the keyword service 22. The number of keywords returned by the history prediction service 26 may be predefined by the user of the digital camera 12 or controlled based on a priority assigned to the history prediction service 26 by the user of the digital camera 12. Additionally or alternatively, the number of keywords returned by the history prediction service 26 may be controlled based on a maximum number of keywords to be suggested to the user at the digital camera 12.
The keyword service 22 may also invoke the location keyword registry service 28 (step 218). The location keyword registry service 28 may be invoked by sending a request including the positional information and optionally a username, date, and time to the location keyword registry service 28. Based on the positional information and optionally username, date, and time, the location keyword registry service 28 returns keywords to the keyword service 22 (step 220). The date and time may be beneficial where the relevant keywords for a particular location vary depending on the date and time. For example, the Mile High Stadium in Denver, Colo., may host numerous sporting events. The teams playing in the sporting events may vary depending on the date and time. As such, the location keyword registry service 28 may suggest different keywords depending on the date and time to reflect the different events held at the stadium.
The location keyword registry service 28 determines the most popular keywords for the location or for locations proximate to the location defined by the positional information. The location keyword registry service 28 may optionally select one or more less popular keywords for the location at random. By selecting one or more keywords at random, the location keyword registry service 28 gives credence to less popular keywords that may be relevant but are less popular because they are newly nominated. In addition, the location keyword registry service 28 determines whether the user of the digital camera 12 has previously interacted with the location keyword registry service 28 to define preferred keywords for the location or for locations proximate to the location defined by the positional information, as described below. The most popular keywords, and optionally the user's preferred keywords and one or more keywords selected at random, are returned to the keyword service 22 (step 220).
The keyword service 22 then generates a list of suggested keywords by combining the keywords returned in steps 208, 216, and 220 (step 222). Thereafter, the list of suggested keywords is provided to the digital camera 12 (step 224). The user of the digital camera 12 may then select one or more keyword from the list of suggested keywords for the digital image, thereby tagging the digital image with the selected keywords. Optionally, the digital camera 12 may provide the selected keywords and optionally the positional information to the location keyword registry service 28 (steps 226 and 228). The location keyword registry service 28 may use the selected keywords in determining the popularity of the keywords for the location defined by the positional information.
The keyword service 22 begins the process of combining the keywords by examining the keywords used to tag a number of previous images, IMAGE[N-1]-IMAGE[N-4], captured by the user of the digital camera 12. Although this example uses four previous images, any number of previous images may be used. The previous images are examined to determine which of the keywords H1-H4, L1-L2, and R1-R4 were used to tag the previous images. This process is illustrated by table 36, where each row corresponds to one of the keywords H1-H4, L1-L2, and R1-R4 and each of the columns corresponds to one of the previous images IMAGE[N-1]-IMAGE[N-4]. A value of “0” indicates that a particular keyword was not used to tag the previous image, and a value of “1” indicates that a particular keyword was used to tag the previous image.
The previous images IMAGE[N-1]-IMAGE[N4] may then be weighted by weighting table 38. The first value in the weighting table 38 is the weight assigned to the most recent image IMAGE[N-1], the second value in the weighting table 38 is the weight assigned to the next most recent image IMAGE[N-2], the third value in the weighting table 38 is the weight assigned to the image IMAGE[N-3], and the fourth value in the weighting table 38 is the weight assigned to the image IMAGE[N-4]. In this example, the most recent image is assigned a greater weight than less recent images. The table 36 and the weighting table 38 are combined to provide scores for each of the keywords H1-H4, L1-L2, and R1-R4. The scores for each of the keywords are stored in a score table 40.
The keywords H1-H4, L1-L2, and R1-R4 are then sorted based on the scores to provide a prioritized list of keywords 42. Thereafter, one or more preferred keywords P1-P2 previously defined by the user of the digital camera 12 from the location keyword registry service 28 are added to the prioritized list of keywords 42 to provide a list of keywords 44. In this example, the preferred keywords P1-P2 are assigned a highest priority. Note that in an alternative embodiment, the preferred keywords P1-P2 may be scored like the other keywords H1-H4, L1-L2, and R1-R4.
In one embodiment, the list of keywords 44 is the list of suggested keywords provided to the digital camera 12, and a number of the keywords from the list of keywords 44 having priority above a threshold are automatically selected when provided to the digital camera 12. For example, the two highest priority keywords may be automatically selected when the list of suggested keywords is displayed to the user at the digital camera 12. In another embodiment, only a number of the keywords from the list of keywords 44 are provided to the digital camera 12 as the list of suggested keywords. For example, the eight highest priority keywords from the list of keywords 44 may be provided to the digital camera 12 as the list of suggested keywords.
More specifically, the collaboration process begins when a user at the user node 32 requests a webpage corresponding to the user interface of the location keyword registry service 28 via the web browser 34 (step 300). In response, the server hosting the location keyword registry service 28 returns the webpage to the user node 32 (step 302). Thereafter, the user at the user node 32 may interact with the location keyword registry service 28 via the web browser 34 in order to select a location (step 304), vote for keywords for the location (step 306), nominate or add one or more keywords for the location (step 308), and/or contest one or more keywords previously nominated for the location (step 310). Voting for keywords for the location enables the location keyword registry service 28 to determine the popularity of each nominated keyword for location. By accumulating votes from each of a plurality of users, the popularity of the keywords for each location can be determined. The popularity of the keywords may also be determined based on feedback from the digital camera 12 regarding the ones of the suggested keywords actually selected by the user at the digital camera 12 to tag a digital image.
Voting for the keywords may also serve to select the preferred keywords for the location for the user at the user node 32. More specifically, the location keyword registry service 28 may store the keywords for which the user at the user node 32 has voted as preferred keywords for the location for the user. Thereafter, if the user captures a digital image at the location with a digital camera operating according to the present invention, the location keyword registry service 28 may provide the preferred keywords for the user, one or more of the most popular keywords for the location, and optionally one or more of the less popular keywords for the location selected at random.
Nominating or adding keywords for the location allows the user to add keywords for the location, which may thereafter be voted on by the users of the location keyword registry service 28 to define the popularity of the added keywords. Contesting a keyword may decrease the popularity of keywords the user feels are not particularly relevant to the location. Alternatively, contesting a keyword may notify an operator of the location keyword registry service 28 that the contested keyword should be reviewed. If the user contesting a keyword is also the user who added the keyword, the keyword may simply be removed from the list of keywords for the location.
Once the location is selected, a list of keywords 70 for the location is displayed on the webpage 60. As illustrated, the list of keywords 70 may include the most popular keywords at the top of the list of keywords 70 and the less popular keywords at the bottom of the list of keywords 70. The list of keywords 70 may also include a first column of checkboxes 72 and a second column of checkboxes 74.
The user visiting the webpage 60 may vote for keywords for the location by selecting the desired keywords using the first column of checkboxes 72 and then selecting an “OK” button 76. Voting for the keywords enables the location keyword registry service 28 to determine the popularity of keywords for the location. Voting for the keywords may also set the selected keywords as the preferred keywords for the user for the location.
The user visiting the webpage 60 may also contest keywords for the location by selecting the keywords he desires to contest from the list of keywords 70 using the second column of checkboxes 74 and then selecting the “OK” button 76. Contesting a keyword may be beneficial when the user feels that the selected keywords are not particularly relevant to the location and enable the location keyword selection service 28 to reduce the popularity of the contested keywords. Alternatively, an operator associated with the location keyword registry service 28 may be notified that the particular keyword has been contested. In response, the operator may review the contested keyword and determine whether the keyword should remain or be removed from the list of keywords 70 for the location. If the user visiting the webpage 60 is the one who added a particular contested keyword, the keyword may simply be deleted.
The user visiting the webpage 60 may nominate new keywords for the selected location by selecting an “add” button 78. In response, the web browser 34 at the user node 32 may be redirected to a second webpage where the user enters the keywords to add for the location. In this example, the keywords “Skiing” and “Camera Phone” were previously added by the user, as indicated by the underlining of the keywords. However, it is not necessary that the keywords added by the user be underlined.
The present invention provides substantial opportunity for variation without departing from the spirit or scope of the present invention. For example, while the present invention is described above with respect to the digital camera 12 having the wireless communication interface 16, the present invention is equally applicable to images taken by a digital camera not having a wireless communication interface to the network 18. More specifically, the digital camera may include an SPS receiver 14 and operate to store positional information from the SPS receiver 14 in association with digital images at the time of capture. Thereafter, the digital images may be uploaded to the user's personal computer or the like being communicatively coupled to the network 18. The personal computer, rather than the digital camera, may then request suggested keywords from the keyword service 22 as described above.
As another example, the digital camera 12 may obtain the positional information by using a system other than the SPS receiver 14. For example, the digital camera 12 may obtain the positional information from a nearby mobile device equipped with an SPS receiver via a wireless communication interface such as a Bluetooth transceiver, Radio Frequency Identification (RFID) tag reader, a transceiver operating according to one of the IEEE 802.11 standards or the Wi-Fi standard, or the like. Alternatively, the digital camera 12 may obtain the positional information from a mobile base station in a cellular network.
As yet another example, a digital camera not including an SPS receiver or the like may be used together with the location keyword registry service 28 to obtain suggested keywords for digital images. More specifically, once the digital image is uploaded to the user's personal computer or the like or while the digital image resides on the digital camera, the location keyword registry service 28 is accessed via the network 18. Since the positional information is not known, the user may interact with the location registry service 28 to select a location in a manner similar to that described above with respect to
Those skilled in the art will recognize improvements and modifications to the preferred embodiments of the present invention. All such improvements and modifications are considered within the scope of the concepts disclosed herein and the claims that follow.
This U.S. patent application is related to the concurrently filed U.S. patent application entitled SYSTEM AND METHOD FOR TAGGING IMAGES BASED ON POSITIONAL INFORMATION, by Hugh Svendsen, which is hereby incorporated herein by reference in its entirety.
