Patent Application
20090123090
On the Internet, advertisements are often presented that are based on the content displayed on a webpage. The goal is to match the subject matter of the advertisement with the subject matter of the webpage's content. The subject matter of the advertisement is often determined based upon keywords that are submitted with and/or extracted from the textual content of the advertisement. The subject matter of a webpage's content is often automatically determined by a computer program that evaluates words and/or phrases within the textual content of the webpage. The subject matter of a visual media object, such as an image or a video, may not be automatically determined as readily, because the visual media object often lacks textual content. This makes it difficult to automatically match the subject matter of a visual media object with the subject matter of an advertisement.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
Embodiments of the present invention generally relate to matching advertisements with visual media objects, e.g., digital photographs and videos. An advertisement is selected to be associated with a visual media object by matching the subject matter of the advertisement with the subject matter of the visual media object at the same level of specificity. In general, advertisements are submitted with detailed keywords that may be used to define a very specific subject matter. On the other hand, image recognition technologies utilized to determine the subject matter of visual media objects may only be capable of placing visual media objects into more general subject-matter categories. Thus, in accordance with embodiments of the present invention, advertisements may be grouped into categories that correlate with the categories utilized by the relevant image recognition technology. In this way, visual media objects may be received and assigned a subject matter. An advertisement, having been assigned the same subject matter as the visual media object, may then be matched with the visual media object and the two may be presented together, if desired.
The present invention is described in detail below with reference to the attached drawing figures, wherein:
The subject matter of the present invention is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.
Accordingly, in one embodiment, the present invention relates to computer storage media having computer-executable instructions embodied thereon for performing a method of selecting advertisements to be associated with visual media objects. The method includes receiving a visual media object and assigning a subject-matter category to the visual media object using automated image recognition. The method also includes, retrieving an advertisement that is assigned the same subject-matter category as the visual media object to be associated with the visual media object. The method further includes, storing the advertisement in association with the visual media object.
In another embodiment, the present invention relates to a computerized method for selecting advertisements having the same context as visual media objects. The method includes categorizing one or more advertisements into a set of subject-matter categories, and assigning an importance value to each subject-matter category. The method further includes, classifying one or more of the visual media objects into a first subject-matter category that has an importance value of at least a threshold value, retrieving a categorized advertisement from the first subject-matter category, and storing the categorized advertisement in association with the appropriate subject-matter category.
In yet another embodiment, the present invention relates to a computerized system for retrieving advertisements having the same subject matter as visual media objects. The system includes an advertisement receiving component for receiving a plurality of advertisements and one or more keywords associated with each of the advertisements and for storing the advertisements and associated keywords in association with one another. The system also includes an advertisement categorizing component for categorizing the plurality of advertisements into subject-matter categories using the respectively associated keywords. The system further includes, a media receiving component for receiving a plurality of visual media objects and a media categorizing component for categorizing the plurality of visual media objects into the subject-matter categories. Finally, the system includes an advertisement retrieval component for retrieving a categorized advertisement having the same subject-matter category as one or more of the plurality of visual media objects.
Having briefly described an overview of embodiments of the present invention, an exemplary operating environment suitable for use in implementing embodiments of the present invention is described below.
Referring to the drawings in general, and initially to
The invention may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program components, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program components including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks, or implement particular abstract data types. Embodiments of the present invention may be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general-purpose computers, specialty computing devices, etc. Embodiments of the invention may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.
With continued reference to
Computing device 100 typically includes a variety of computer-readable media. By way of example, and not limitation, computer-readable media may comprise Random Access Memory (RAM); Read Only Memory (ROM); Electronically Erasable Programmable Read Only Memory (EEPROM); flash memory or other memory technologies; CDROM, digital versatile disks (DVDs) or other optical or holographic media; magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to encode desired information and be accessed by computing device 100.
Memory 112 includes computer storage media in the form of volatile and/or nonvolatile memory. The memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc. Computing device 100 includes one or more processors that read data from various entities such as memory 112 or I/O components 120. Presentation component(s) 116 present data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc. I/O ports 118 allow computing device 100 to be logically coupled to other devices including I/O components 120, some of which may be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc.
Turning now to
Computing system architecture 200 includes system input data 210, a computing environment 220, and output data 230. Computing environment 220 may be a single computing device, such as computing device 100 shown in
The system input data 210 shown in
As shown in
The media receiving component 222 is configured for receiving visual media objects. Visual media objects may be in the form of images, video or a combination thereof. In one embodiment, the media receiving component 222 receives one or more training images 213 and/or training videos 214 that are used to train the media categorizing component 224. In another embodiment, the media receiving component 222 receives one or more un-categorized images 211 and/or un-categorized videos 212. When either un-categorized videos 212 or training videos 214 are received, the media receiving component 222 may perform a key frame extraction process that finds key frames that are distinctive and representative of the video subject matter. The key frame extractions are performed as a preparatory step for the media categorizing component 224 that analyzes the key frames rather than every frame in the video. In embodiments, other preparatory steps may also be performed by the media receiving component 222 as needed.
The media categorizing component 224 is configured to categorize un-categorized images 211 or un-categorized videos 212 into subject-matter categories. Before the media categorizer component 224 may be used to classify un-categorized images 211 or un-categorized videos 212, it generally will first be trained using training images 213 and training videos 214. Generally, the training images 213 and training videos 214 are manually pre-classified into subject-matter categories. The media categorizing component 224 may break down the image into various descriptors. Descriptors include sharp edges, soft edges, colors, large blocks of colors, and other such identifying features associated with the image. The media categorizing component 224 may process the visual media object through shape extraction, space partitioning and object recognition algorithms and determine a subject-matter category relevant to the visual media object utilizing one or more of a set of rules and learning algorithms. Using the training images 213 and/or training videos 214, the media categorizing component 224 learns what descriptors are common to the respective subject-matter categories.
Once the media categorizing component 224 is trained, it may then be used to classify un-categorized images 211 or un-categorized videos 212 into subject-matter categories based on the descriptors identified in the un-categorized visual media object. Once in operation, the media categorizing component 224 may be continually improved through additional training methods. Subject-matter categories may be added or removed as desired. The media categorizer component 224 may receive the unclassified images 211 or un-classified videos 212 from the media receiving component 222 which may perform preparatory steps on the videos as described above.
In one embodiment, the media categorizer component 224 is only trained to classify un-categorized images 211 or un-categorized videos 212 into subject-matter categories that have an objective importance, for instance, a sufficient commercial demand, assigned by importance value determining component 227, above a pre-determined threshold value. The operation of the importance value determining component 227 is explained in more detail hereafter. In addition, if a sufficient number of training images 213 or training videos 214 are not available for a particular subject-matter category, then that subject-matter category may be abandoned or combined into a related category. Related subject-matter categories may also be combined if the media categorizer component 224 is unable to reliably distinguish between two similar subject-matter categories.
The advertisement receiving component 229 is configured to receive advertisements with associated keywords 216 or advertisements with associated subject-matter categories 218. The advertisements may be received over an Internet connection or other method. In one embodiment, advertisements are submitted to the advertisement receiving component 229 along with associated keywords and/or categories through a user interface provided by the advertisement receiving component 229. In some embodiments, the advertisement receiving component 229 stores the advertisements along with their associated keywords and categories in a database, for instance, database 240. The advertisement receiving component 229 may also pass the advertisements directly to the advertisement categorizing component 228.
The advertisement categorizing component 228 is configured to categorize advertisements into subject-matter categories using the keywords that are submitted with the respective advertisements. The subject-matter categories may be organized hierarchically. By way of example, and not limitation, the advertisement categorizing component 228 may use classification algorithms such as the support vector machine (SVM), trigram, the Bayesian Algorithm, or other suitable classification algorithm. The advertisement categorizing component 228 may receive the advertisements and keywords directly from the advertisement receiving component 229 or may retrieve the advertisements and keywords from the database 240. The advertisement categorizing component 228 may output the advertisements sorted by subject-matter categories into a list 236 that may then be stored in a table within database 240 or otherwise accessed by users directly, for example through a user interface or print out.
The importance value determining component 227 is configured to determine an importance value for each of the subject-matter categories relative to one another and/or a predetermined threshold value. Such determination may be made, for instance, based upon commercial advertising demand. Commercial advertising demand may be determined by weighing a variety of factors including, but not limited to, the number of advertisers submitting advertisements within a particular subject-matter category assigned thereto, the probability that an advertisement having a particular category assigned thereto will be selected by a viewer, the number of advertisements within a particular subject-matter category, and an objective importance of advertisements having a particular category assigned thereto, such importance being based, for instance, on the amount of money that advertisers are willing to pay to have the advertisements within a subject-matter category presented or a monetization value associated with advertisements having the particular category assigned thereto. The importance value determining component 227 is configured to determine an importance value for each subject-matter category used by the advertisement categorizing component 228. In one embodiment, the importance value reflects the commercial value of the subject-matter category. Thus, a high importance value would be assigned to a subject-matter category that has a lot of advertisers that are willing to pay relatively high amounts of money for there advertisements to be presented. Subject-matter categories with an importance value above a threshold value may be classified as highly important. In one embodiment, visual media objects are categorized only into highly important categories. The importance value determining component 227 may receive the subject-matter categories from advertisement categorization component 228, directly from database 240, or though any other available method within the scope of embodiments hereof. The output from the importance value determining component 227 may be a list ranking the subject-matter categories 234 with respect to one another, a list indicating those subject-matter categories that have an importance value only above and/or below a threshold value, or any combination thereof. The output may be stored in the database 240, or output as a file directly to a user or other application.
The advertisement retrieving component 226 receives the subject-matter classification assigned to the un-categorized visual media object by the media categorizing component 224, and retrieves a categorized advertisement 232 from the same subject-matter category. In one embodiment, the advertisement retrieving component 226 retrieves the categorizedl advertisement from the database 240.
The database 240 is configured to store advertisements along with associated keywords and subject-matter categories. In some embodiments, un-categorized images 211, un-categorized videos 212, training images 213, and training videos 214 are stored in association with the database 240. The database 240 maybe accessed by components 222, 224, 226, 227, 228, and 229 as needed. All system input data 210 and output data 230, as well as intermediary data, maybe stored temporarily or permanently in the database 240. In some embodiments, the database 240 is configured to be searchable for one or more of the items stored in association therewith. It will be understood and appreciated by those of ordinary skill in the art that the information stored in the database 240 may be configurable and may include any information relevant to the advertisements, visual media objects, components 222, 224, 226, 227, 228, and 229, system input data 210 and output data 230. The content and volume of such information are not intended to limit the scope of embodiments of the present invention in any way. Further, though illustrated as a single independent component, the database 240 may, in fact, be a plurality of databases, for instance, a database cluster, portions of which may reside on a plurality of computing devices.
Referring next to
At step 320, the visual media object is assigned a subject-matter category using an automated image recognition application. The visual media object is classified into a subject-matter category for which advertisements are available for presentation. Prior to operation, the automated image recognition application is trained to classify un-categorized visual media objects into subject-matter categories using training images 213 and training videos 214 that have been manually assigned a subject-matter category. In one embodiment, the visual media object is assigned to a subject-matter category that has an objective importance value, for instance, a commercial demand rank, above a designated threshold value.
At step 330, an advertisement assigned the same subject-matter category as the visual media object, is retrieved. The advertisement may be retrieved from a database, such as database 240, that contains advertisements that have been assigned subject-matter categories. At step 340, the advertisement is stored in association with the visual media object (or an identifier thereof) for later use. In one embodiment, advertisements are continually received in association with relevant keywords and/or subject-matter categories. If the advertisement is associated with only keywords, the advertisement is then categorized into a subject-matter category using the keywords. The categorization process may be performed automatically using any state of the art classifying algorithm such as SVM (Support Vector Machine), a trigram, or other method. The advertisements should be categorized into subject-matter categories no more specific than can accurately be identified by the image recognition application. As image recognition applications improve, the subject-matter categories may become more specific.
In one embodiment, the subject-matter categories are ranked relative to one another and/or a pre-determined threshold value based on one or more indicators of objective importance, for instance, commercial advertising demand. Commercial advertising demand may be calculated based on a number of factors that include, but are not limited to, the number of advertisements within a subject-matter category, the number of advertisers submitting advertisements with a subject-matter category, the probability of an advertisement having a particular subject-matter category assigned thereto being clicked by a viewer, a frequency with which an advertisement with the subject-matter category assigned thereto will be clicked by a viewer, a frequency with which viewers convert in response to seeing an advertisement with the subject-matter category assigned thereto, and an objective importance of advertisements within a particular category, for instance, determined by the amount of money advertisers are willing to pay to have advertisements within the subject-matter category presented. In one embodiment, the un-categorized visual media object is only classified into categories that have been assigned an importance value (e.g., a commercial demand rank) above a threshold value. The threshold value may be set, for instance, at a point where the expense to set up, and operate embodiments of the present invention, is more than offset by the increase in advertising revenue likely to result from matching advertisements with previously un-categorized visual media objects. As the cost of setting up and operating embodiments of the present invention decreases, more and more subject-matter categories may become commercially viable.
In one embodiment, an un-categorized visual media object on a webpage or within a web server is selected for pre-classification. The un-categorized visual media object is retrieved and assigned a subject-matter category, as in step 320. The subject-matter category assigned to the visual media object may be tracked using a table or other method. The table may be accessed when a visual media object is received to see if the visual media object has already been assigned a subject-matter category. In another embodiment, the visual media object is categorized in real time as it is displayed.
Referring next to
At step 420, an importance value is assigned to each subject-matter category. Assigning an importance value, for instance, based upon a commercial demand rank, to a subject-matter category was explained hereinabove with reference to
The present invention has been described in relation to particular embodiments, which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill-in-the-art to which the present invention pertains without departing from its scope.
From the foregoing, it will be seen that this invention is one well adapted to attain all the ends and objects set forth above, together with other advantages which are obvious and inherent to the system and method. It will be understood that certain features and sub-combinations are of utility and may be employed without reference to other features and sub-combinations. This is contemplated by and is within the scope of the claims.
