Patent Application
20110082735
Digital communication technologies have seen explosive growth over the past few years. This growth has been fueled by new content delivery technologies, including the Internet and new wireless services such as mobile broadcast television. With this new content delivery technologies have come increased consumer demand for audio and video content, as well as new opportunities for marketing products and responding to consumer demand.
The various embodiments provide methods and systems that enable users of still images or video content to select products appearing within the images displayed on a computing device and request information regarding such products. The still images or video content may be received from a variety of content delivery systems including, for example, mobile broadcast television, cable television services, satellite television services, the Internet, stored video (e.g., DVD or Tivo®), and video shot by the user. Information regarding the selected parts of a still or video image in the form of selected portions of the video image or image coordinates within the image sufficient to enable a server to obtain the selected portion of the image may be included in a product query message that the user's computing device transmits to a processing server (referred to herein as a “Transaction Server”). The image information received in a product query message may be processed by the Transaction Server to recognize objects or particular products within the selected portion of the image. Recognized objects or products may be compared to a database of available merchandise to determine their commercial availability. Information regarding commercially available products may be included in a product information message that the Transaction Server generates and transmits to the user's computing device. Recommended products may be presented in a user interface display so that users can select a product for purchase. Users may then initiate a purchase transaction for one or more of the recommended products, such as by interacting with the user interface. Product purchase transactions may be accomplished according to any known transaction method. In the embodiments in which the still or video image content is broadcast, such as by a mobile TV broadcast network, the product query messages may be processed by a Transaction Server that is part of the broadcast provider. In embodiments in which the still or video image content is broadcast or unicast over an Internet protocol (IP) network, such as the Internet, the Transaction Server may be a Web server. In embodiments in which the video content is broadcast by a cable or satellite television network, the Transaction Server may be a Web Server accessible via an IP network.
The accompanying drawings, which are incorporated herein and constitute part of this specification, illustrate exemplary embodiments of the invention, and together with the general description given above and the detailed description given below, serve to explain the features of the invention.
The various embodiments will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. References made to particular examples and implementations are for illustrative purposes, and are not intended to limit the scope of the invention or the claims.
The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any implementation described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other implementations.
As used herein, the term “computing device” refers generally to any device including a processor that may be programmed and configured to accomplish any of the various embodiments. As used herein, the terms “mobile device” and “handheld device” refer to any one or all of cellular telephones, personal data assistants (PDA's), palm-top computers, wireless electronic mail receivers (e.g., the Blackberry® and Treo® devices), multimedia Internet enabled cellular telephones (e.g., the Blackberry Storm®), Global Positioning System (GPS) receivers, wireless gaming controllers, and similar computing devices which include a programmable processor and memory and receiver circuitry for receiving and processing still and video image content. In an embodiment a mobile device includes receiver circuitry for receiving and processing mobile broadcast television services.
The term “unicast network” is used herein to refer to communication networks which transmit data to a single destination. Examples of a unicast network include WiFi and cellular data communication networks. Examples of unicast transmissions include simple message service (SMS), multimedia message service (MMS), and electronic mail messages as may be carried via a cellular telephone data communication network.
The word “broadcast” is used herein to mean the transmission of data (information packets) so that it can be received by a large number of receiving devices. Examples of a broadcast message are mobile television (TV) broadcast transmissions as well as digital television, IP multicast programs, cable television cablecasts and satellite television broadcasts. Although broadcast content delivery system is a type of content delivery system, the embodiments are not limited to broadcast content delivery systems as the embodiments may also be implemented with stored video images, such as stored in a digital video disc (DVD), a digital television storage device (e.g., Tivo®) or a web server accessed via the Internet (e.g., youtube.com). The embodiments may further be implemented with video images shot by the user, such as video shot on a mobile device like a cellular telephone.
The various embodiments utilize a number of process modules, software programs or analysis engines that may be implemented within one or more server computers to accomplish the embodiment methods. While such process modules, software programs or analysis engines may be implemented in a variety of architectures, including as a single processing module, for ease of description herein reference is made to separate processing modules whose names and functions defined in the following paragraphs.
As used herein, “Transaction Server” refers to a server or network of servers to which receive product query messages, process included or referenced images to identify products, and accomplish other product recommendation processes described herein. A Transaction Server may include a “Transaction Gateway” which may be a module that facilitates transaction communications with computing devices via a network, such as the Internet.
As used herein, “Image Selection” refers to a act of an end user (e.g., a video consumer) selecting a portion of an image (e.g., within a frame of video content) that contains an object of interest, and “Image Selection Information” refers to the selected portion of the image (i.e., image data such as pixel values) or to information about the Image Selection that can be used to obtain the selected portion of the image from memory or an accessible database (e.g., a file identifier for the image or a frame number of the video and image coordinates within the frame). The Image Selection Information is transmitted to a Transaction Server in a Product Query message so the Transaction Server either receives the selected portion of the image or can obtain the Image Selection from memory or an accessible database.
As used herein, “Annotation Information” refers to optional information that an end user could give to qualify the object of interest that is sent along with or in addition to the Image Selection Information in a Product Query message. Annotation Information may be in the form of text inputs, a voice memo, menu selections, etc.
As used herein, “User Profile” refers to information about a particular end user, such as age, gender, salary range, purchase history, personal preferences, etc. User Profile information may be provided by a user and/or observed by a computing device or the Transaction Server based upon the user's activities (e.g., purchases, Product Queries, mobile device uses, etc.). The User Profile may also include information about the end user's computing device or mobile device, such as delivery or display capabilities, music download accounts, etc., that may be relevant to recommending products for purchase.
As used herein, “Product Query” messages are messages that convey Image Selection Information along with Annotation Information to the Transaction Server.
As used herein, “Product Correlation Engine” refers to a process module or software module implemented on a server, such as within the Transaction Server which matches objects of interest to products from merchandisers' databases. As used herein, “Product Correlation” refers to the methods implemented by the Product Correlation Engine's based upon information received in a Product Query.
As used herein, “Recommendation Engine” refers to a process module or software module implemented on a server, such as within the Transaction Server, which recommends products to an end user based on a Product Query, a User Profile or other information.
As used herein, “Product Information message” refers to a message produced by the processing of the Product Correlation Engine and Recommendation Engine that contains product pricing/coupon information for products that match the Product Query and User Profile. The Product Information message need not always be sent in response to a Product Query, and may be generated at a later time. For example similar to advertisements, a Product Information may be “pushed” to end users based on information in a User Profile, such as a previous Product Query or purchase transaction. For example, if a user requests a black leather jacket in a Purchase Query including Image Selection Information selected from the Terminator movie and in response is sent Product Information message including jacket information on one day, another Product Information message may be sent the next day including information about sunglasses featured in Terminator movie.
Still images and video programs are being delivered to consumers in a wide variety of formats now. Consumers now receive visual content from satellite and cable television networks, broadcast digital television networks, and the Internet. Noteworthy is the recent develop of mobile TV broadcast services which have begun delivering video content to mobile users. A number of different mobile TV broadcast services and broadcast standards are available or contemplated in the future, all of which may implement and benefit from the various embodiments. Such services and standards include Open Mobile Alliance Mobile Broadcast Services Enabler Suite (OMA BCAST), MediaFLO, Digital Video Broadcast IP Datacasting (DVB-IPDC), and China Multimedia Mobile Broadcasting (CMMB).
Mobile TV users and mobile Internet users are different from the conventional home television audience in that they use viewing devices (i.e., mobile devices) that they carry with them which received mobile TV broadcast services or Internet multicast program and can communicate via unicast messages. Additionally, mobile devices can be personalized to users since in the majority of instances only one person uses a mobile device. Still further, mobile devices now typically include video cameras that enable users to film video images, such as images of products of interest to them. Typically, users carry their mobile devices with them everywhere, including while shopping. Thus, the mobility and access to the Internet and mobile TV broadcasting services mean that mobile devices may be highly valuable marketing and electronic transaction tools.
The various embodiments enable end users to select image portions or image data that correlates with merchandise displayed within broadcast programs, multicast programs, stored video images (e.g., from a DVD, Blu-ray disc player, or Tivo®) or a user video, and initiate order transactions for such products or related merchandise on their computing device, such as a personal computer or mobile devices. A client application running on the end user's computing device enables the user to identify products of interest by designating a portion of a displayed image containing a product or products of interest. The user may designate the selected portion of the still or video image using a pointing device, such as by drawing a circle around the image portion with a computer mouse or with a finger tracing a circle on a touchscreen display. The selected portion of the image including the object of interest may be packaged as Image Selection Information within a Product Query message that the mobile device transmits to a Transaction Server. Instead of transmitting the image data itself, the Image Selection Information within the Product Query may include information such as a frame number or image file name and coordinates within the image that define the selected portion so that the Transaction Server can obtain the image from an accessible database and determine the selected portion. The Product Query message forwarding the Image Selection Information may also include additional information (“Annotation Information”), such as voice notes or typed comments from the user regarding the product of interest. Such a Product Query message may be received in a Transaction Server where the message is parsed to obtain the Image Selection Information, Annotation Information, and other additional information, such as an identifier (ID) of the mobile device. The Image Selection Information may be processed by a Product Correlation engine which may be a software module within the Transaction Server or another server that is configured to process images in order to recognize particular product characteristics, such as shape, color and configuration. Image objects identified within the image selection may be compared to a database of merchandise to determine if the object(s) in the image selection matches or corresponds to available merchandise. If a match is found, information regarding the make, model, source, cost, and other purchasing details for the product may be obtained from the merchandise database, and assembled into a Product Information message that is transmitted to the mobile device. The Product Information message may also include other related products beyond those that match, such as products which a Recommendation Engine predicts will be of interest to the end user based upon the matched product and information in the user's User Profile. If a direct match between the image selection and a product available in the marketplace is not found, the image selection may correspond to a product such as a later or replacement model of the imaged product, an equivalent competitor's product, or a replica of the imaged product. Therefore, if a product match or correspondence is not found, the Recommendation Engine may process the recognized image object(s) along with any received Annotation Information, User Profile information about the user and the mobile device, and other information to generate a recommendation of alternative merchandise that might be of interest to the user. As in the case of a matched product, recommended alternative merchandise information may be assembled into a Product Information message that is transmitted to the end user's computing device.
When a computing device receives a Product Information message, a client application running on the device may parse the message to obtain information regarding the selected product and/or recommended alternative products, and generate a user interface display to enable the user to initiate a purchase transaction. This user interface display may be positioned outside the video or image content portion of the display to avoid interrupting the user's video content consumption. If a user decides to purchase or order any recommended product, the transaction may be promptly accomplished using a variety of electronic transaction methods. For example, when the user has transmitted a Product Query message from a mobile device, that device may place a data call to an Internet connection and access a merchant server to purchase the selected product online. As another example, the user's computing device may connect to a merchant server or Transaction Server to receive a coupon towards purchase of the product at a merchant's “brick and mortar” storefront. As a third example, information regarding the location of stores where the recommended product(s) can be purchased may be stored within a geographic information services (GIS) application so that when the user is in the vicinity of such a store the user's mobile device can alert the user and provide driving or walking directions. Regardless of the manner in which a transaction is completed, the user's computing device may report information to the Transaction Server or the merchant that links the transaction to the Product Information message and/or the Product Query message. Products within image selections provided in Product Query messages may be recognized using a variety of techniques, including for example image recognition algorithms, matching image information to product placement information supplied by content providers, and humans viewing the images and recognizing products manually.
An example communication system including a mobile TV broadcast network 100 suitable for use with the various embodiments is illustrated in
Within or coupled to the mobile TV broadcast network 100 may be a transaction module 110, which may be coupled to other components within the broadcast network 100 via a local area network 102. The purchase transaction module 110 may include a Transaction Server 112 and a merchandise database 106. The Transaction Server 112 may be coupled via a network or the Internet to a wireless network provider 122 in order to receive unicast messages from mobile devices 118. The Transaction Server 112 may also be coupled to product manufacturers and merchants 124 via a network or the Internet in order to receive information regarding available merchandise, track transactions related to the user Product Queries, complete transactions conducted through the Transaction Server 112, and inform merchants 124 of user interests in various products. Information regarding merchandise available for purchase, and other merchandise-related information necessary to support the functionality of the Transaction Server 112 may be stored within a merchandise database 106. Merchants and manufacturers 124 may also be able to store merchandise information in the merchandise database 106.
While
In operation, still and video image content is broadcast by the mobile TV broadcast network 100 via broadcast sites 116 and received by mobile devices 118. Users can use their mobile devices 118 to view selected broadcast programs. When a user sees an object of interest within a broadcast program, the user may select the portion of a video image containing the object of interest, such as by tracing a circle around it with a fingertip on a touchscreen display. A client application operating in a processor within the mobile device 118 may use the indicated portion of the video image to generate a Product Query message. The Product Query message may be transmitted via a unicast network, such as a wireless data network 122, to the Transaction Server 112. In such a transmission, wireless data messages from the mobile device 118 may be received by a wireless node antenna 120 and forwarded by the wireless data network 122 to another network, such as the Internet, to the Transaction Server 112. The Product Query message from a mobile device 118 to the Transaction Server 112 may be relayed using well-known communication methods and systems, such as cellular data networks and wireless wide area networks (e.g., WiFi), as well as wired network communications, such as the Internet.
The various embodiments may also be used in connection with non-broadcast content, such as unicast and multicast still and video image content available via the Internet as illustrated in
The functional components illustrated in
As discussed above, the various embodiments enable users to request information regarding products seen within still and video images received from a variety of content delivery systems, with such Product Queries processed in a Transaction Server 112 in order to reply with a Product Information message.
Referring to
If the computing device determines that the user chooses not to purchase any displayed products (i.e., determination 436=“No”), the computing device may store information contained in the Product Information message in memory for later reference by the user, step 450. For example, if the computing device is a mobile device it may store the product name and information regarding the location of stores carrying the product. Then, in the future, when the mobile device determines that the user is close by a store selling the product (e.g., based on GPS coordinates determined by a GPS receiver in the mobile device), a display may be generated to inform the user that the product may be purchased nearby. In an embodiment the Transaction Server or another server may store information about products that the user has declined to purchase in the past and use this information in preparing future purchase recommendations to avoid recommending the same product over and over. In another embodiment such information may also or alternatively be stored in the user's computing device.
Examples of messages which may be passed among various components in the embodiment methods are illustrated in
If a user of the mobile device 118 decides to act on the Product Information (e.g., purchase one of the recommended the products), corresponding user inputs may be processed by the mobile device 118 to generate a transaction request message that may be transmitted to the Transaction Server 112, message 514. The Transaction Server 112 may use the information in the transaction request message 514 to initiate a transaction with a merchant 124 by sending a transaction initiation message 516 that may be sent by a network, such as the Internet. In response, the merchant 124 may reply with a transaction response message 518 sent by the network, such as the Internet. As an example, the transaction response message 518 may include an electronic coupon for delivery to the mobile device 118. If the transaction response message 518 is returned to the Transaction Server 112, a transaction information message 520 may be forwarded by the Transaction Server 112 to the mobile device via the unicast network. Alternatively, the merchant 124 may respond directly to the mobile device 118, such as by transmitting a coupon or information via the unicast network, message 522.
If a user of the mobile device 118 decides to purchase a product, a purchase transaction may be completed directly between the mobile device 118 and the merchant 124, such as by transmitting a transaction request message via a unicast network, message 524. In response, a merchant 124 may reply with a transaction response message sent via the unicast network, message 526.
Periodically, the Transaction Server 112 may send transaction summary information to the merchant 124 such as information regarding consumer interest in certain products demonstrated in Product Query messages, as well as product recommendations sent to consumers in Product Information messages, message 528. Similarly, the Transaction Server 112 may send summary transaction information to the broadcaster 100 and/or content providers (not shown in
The processing of the various embodiments may be illustrated by way of an example. If a user is watching the movie Terminator on a mobile device and suddenly has a desire to purchase the black leather jacket worn by Arnold Schwarzenegger, the user may highlight the portion of a video image containing the jacket, such as by circling the image portion with a finger on a touchscreen display. A mobile device processes that user input to generate a Product Query message which is transmitted to a Transaction Server within the mobile TV broadcast network (or elsewhere). This Product Query message may be transmitted via a unicast network, such as a cellular data communication network. The Transaction Server receives the Product Query message and processes the image selection in a Product Correlation Engine to recognize the particular product of interest. The Product Correlation Engine determines that the most likely product is the black leather jacket within the image selection. This information may be used to compare the black leather jacket image to available merchandise within a merchandise database. Not surprisingly, the particular black leather jacket worn by Arnold Schwarzenegger in the Terminator is no longer commercially available, so the particular product cannot be recommended for purchase. Instead, the Recommendation Engine may use information about the broadcast content (i.e., that the program is the Terminator), the user's prior purchasing behavior (e.g., information that may be stored within a database regarding the user's purchasing behavior, user account information, or information provided by the mobile device itself), and the identified product to develop a recommendation of other products that the user may be interested in purchasing. As part of developing a Product Information, the Recommendation Engine may also consider comments or additional information provided by the user as Annotation Information in the Product Query, such as jacket size, color preference, or other expressions of interest. For example, the Recommendation Engine may select two or three available black leather jacket designs that are similar to the jacket that appears in the movie. The Recommendation Engine may also recommend other merchandise, such as dark sunglasses similar to the model of worn by Arnold Schwarzenegger in the movie. The Product Information message may then be transmitted to the mobile device via a unicast network, such as the cellular data communication network that carried the Product Query message. The mobile device receives the Product Information message and a client application uses the information contained therein to generate a user interface display. A user may then use the user interface on the mobile device to indicate whether any of the recommended products should be purchased. If the user chooses to purchase one of the recommended products, the mobile device and/or the Transaction Server may initiate a transaction using any of a number of known transaction methods.
While the foregoing example concerned a mobile television broadcast content delivery system, the user and device operations would be similar with most other types of content delivery systems.
An example embodiment method that may be implemented within a computing device to enable a user to inquire about a product shown within a video image is illustrated in
The client application may be configured to interpret the user inputs as selecting a portion of the displayed image and store coordinates of the selected portion in a manner that can be communicated to a Transaction Server 112. For example, the client application may record the particular pixels selected by the user as image data. As another example, the client application may record boundary coordinates of the image selection relative to particular coordinate axes, such as a corner of the video image. As a further example, the client application may record the pixel numbers encompassed within the image selection but not the image data. In a further example embodiment, the client application may record an image identifier and coordinates of a user touch to the image, such as the frame number of the touched image and the pixel or distance coordinates (e.g., X and Y distances from a coordinate axis like a corner). Other methods may also be used for identifying the location of user touch or an area encircled within a user designated image selection in a manner that will enable the Transaction Server to obtain the image from an accessible database and determine the portion of the image selected by the user. For ease of reference, any data that identifies a location or area of an image selection or includes the selected image data are collectively referred to herein as Image Selection Information. It is noted that Image Selection Information is information specific to an image and is not intended to encompass information appended or linked to an image such as a hyperlink.
The client application may also generate a prompt on the display prompting the user to provide additional input regarding the Product Query. For example, the user may be prompted to type in a description or quantity for the desired product, such as a size, color, or number of units the individual is interested in purchasing. Additionally, the prompt may invite the user to include product descriptors, such as “jacket,” “sweater,” “surfboard,” etc. Further, the prompt may invite the user to provide user input as spoken words which the mobile device may record for inclusion in the Product Query message. The more information provided by the user in step 608 the better the Transaction Server 112 may be able to identify the product of interest and provide relevant Product Informations to the user. For example, the user may be prompted to speak a Product Query, such as “how much is that sweater?” Such additional input from the user may enable the Transaction Server 112 to determine that the user is interested in a sweater and that the user wants to know the price. As another example, a user may say “does that jacket come in size extra large?” Such information would help the Transaction Server 112 determine that the user is interested in the jacket within the image selection and determine the user's desired size.
The client application operating in the mobile device may receive the additional user input as Image Selection Information, step 610, and generate a Product Query message that includes the Image Selection Information (i.e., the information regarding the user selected portion of the video image) along with Annotation Information and any additional inputs, step 612. The Product Query message may then be transmitted by the computing device to the Transaction Server 112 via a unicast network 122, such as the Internet or a cellular data communication network, step 614. Once the Product Query message has been transmitted, the computing device may return to displaying the broadcast content, step 618. In so doing the computing device may begin playing content stored in memory during the Product Query process so that the user continues viewing the content from the point that the image was frozen on the display. Alternatively, the computing device may simply begin receiving and displaying the content at the current point in the broadcast program.
After transmitting a Product Query message, a computing device may receive a response from the Transaction Server 112 in the form of a Product Information message. An example embodiment method for receiving such a message that may be implemented within a computing device, such as in a client application running on the computing device, is illustrated in
When the computing device receives a user response to the purchase display prompt, step 712, it may determine whether the user indicates that a purchase should be initiated, determination 714. If the user indicates that no purchase should be initiated (i.e., determination 714=“No”), the computing device may be configured to store the product information from the Product Information message in memory, step 716, and return to the operation underway before the message was received, returning to step 702. For example, the computing device may be configured to store information regarding the recommended product(s) and sources (e.g., stores where such products may be purchased) in memory so that the user can recall such information or the computing device may later remind the user about the availability of such products, such as when the user is within the vicinity of a store selling the product.
If the user indicates that a purchase should be initiated (i.e., determination 714=“Yes”), the computing device may generate another user interface display providing the user with options for conducting such a transaction, step 718. A transaction may be accomplished using any known electronic or conventional purchase transaction method. The display of transaction options may list alternative ways that the user may initiate a transaction, or alternatively begin a transaction according to previously selected transaction process.
In a first embodiment transaction method, a user may opt to receive a coupon that entitles the user to a discounted price for the particular product, in which case the user may transmit a request for such a coupon in a message addressed to the Transaction Server 112 or the merchant, such as a request to access a website addressed to the merchant URL, step 720. Such a message may be any form of addressable message, such as SMS, e-mail or a TCP/IP message addressed to a URL, and may be transmitted via a unicast wireless network, such as the Internet, a cellular data communication network or a WiFi network. The address for a coupon request message may be included within the Product Information message, and the computing device may be configured to use that such address information when generating and transmitting a coupon request message. The computing device may further be configured to receive a coupon from a merchant and store the coupon in memory for use in a later transaction, step 722. Such a merchant coupon may be transmitted via the unicast network used to send the coupon request message or may be transmitted by another content delivery system. For example, methods and systems for transmitting coupons to mobile devices via mobile TV broadcast transmissions are described in U.S. patent application Ser. No. 12/417,493 entitled “Systems and Methods for Distributing and Redeeming Credits on a Broadcast System” filed Apr. 2, 2009, the entire contents of which are hereby incorporated by reference.
In a second embodiment transaction method, a user may opt to access a merchant website in order to conduct an online transaction, in which case the computing device may initiate an Internet connection or a data call to the merchant's URL, step 724. The merchant URL may be included within the Product Information message, and the computing device may be configured to use the provided URL when initiating the merchant server access. Once an online connection is established to a merchant server, the user may complete an online purchase transaction in an ordinary manner.
In a third embodiment transaction method, a user may opt to receive information regarding stores or merchants that carry the product of interest, in which case the computing device may be configured to transmit a request for such information to the merchant URL, step 726. Such a message may involve accessing the merchant website so that the user can obtain more information about the product and the merchant as well, as identify nearby store locations and request driving directions. Alternatively, the v device may format a data request message specifying the information desired in a format that can be processed by the merchant server. The computing device may then receive and store the product and merchant information, step 728. For example, the computing device could store a website image downloaded from the merchant URL. As another example, the computing device may receive and store an electronic brochure regarding the product or the merchant.
In a fourth embodiment transaction method, a user may opt to retrieve store location information for stores or merchants that carry the product of interest, in which case the computing device may be configured to transmit a location request message to the merchant URL, step 730. The computing device may format a data request message specifying the location information desired in a format that the merchant server can process. The computing device may then receive and store the received store location information, step 732. For example, the merchant location information may be in the form of GPS coordinates or geographic information service (GIS) data that a mobile device can implement in a navigation or GIS application that can assist the user in locating the nearest store offering the particular product for sale.
As mentioned above, the Transaction Server 112 may receive and automatically process Product Query messages received from computing devices. An example embodiment method by which a Transaction Server 112 may respond to a Product Query message is illustrated in
The received image selection may be processed in a Product Correlation Engine to recognize outlines of objects and parse the image into recognized objects, step 808. A variety of known image processing methods may be used to identify objects within the image selection that may be products and ignore elements within the image selection that are not relevant to a Product Query, such as background scenery, human features and common structures. Image objects may be further recognized by comparing recognized outlines to a database of standard or known objects. Further, objects may be recognized by comparing recognized outlines to a database of images of known product configurations. Still further, objects may be recognized by comparing recognized outlines to a database of image of products known to be present in the broadcast content, such as images related to product placement advertising. A database of images related to product placement advertising may be obtained from the content provider.
Identified image objects may be compared to known patterns of products in order to further correlate objects within the image selection with purchasable products, step 810. If the image selection includes multiple image objects which compare favorably with known product patterns or one or more image objects compares favorably to a plurality of different product patterns, the Product Correlation Engine may use information obtained from user's Annotation Information to select a product of interest to the user, step 812. For example, if an image selection includes the torso of an actor wearing a leather jacket and sunglasses, the Product Correlation Engine might identify the leather jacket and the sunglasses as potential product image objects. To select the image objects of most interest to the user, the Product Correlation Engine (or other module within the Transaction Server 112) may determine whether the user's Annotation Information referred to a jacket, sunglasses, a clothing size, color or a style that would further clarify the user's interest. Thus, if the Annotation Information included “in size 44,” the Product Correlation Engine (or other module within the Transaction Server 112) may conclude that the user is interested in the leather jacket, and thus select one image object (i.e., the leather jacket) of most likely interest to the user in step 812.
Information regarding a most likely image object may then be processed in a Product Correlation Engine (or other module within the Transaction Server 112) in order to formulate a Product Information message, step 814. A Product Correlation Engine may compare the selected image object to a database of available merchandise, step 816. In doing so, the image object may be compared to images of available merchandise stored within a merchandise database. As part of this process, the Product Correlation Engine may also consider the user Annotation Information, especially regarding style, size, color and other distinguishing characteristics that may be matched to available merchandise in order to better address the user's Product Query. In conducting such a comparison, the Product Correlation Engine may determine whether the image object of interest to the user matches or corresponds to a particular merchandise product available in the marketplace, determination 820. This determination may be limited to the merchandise of a particular one or few merchants or suppliers, or may be practically unlimited encompassing any product available or mentioned on the Internet (e.g., a Google search on matching products).
It should be appreciated that image processing, product recognition and comparisons to available merchandise may be accomplished in a single process, such as be comparing the image selection to a database of images of available products or to a database of images of product placement merchandise.
If a match or correspondence between the selected image object and a product or products available in the marketplace is found (i.e., determination 820=“Yes”), the Recommendation Engine may recall information regarding the matched merchandise and available sources (e.g., suppliers and merchandisers) from a merchandise database, step 822. The recalled information may be used to generate a Product Information message, step 824, which is transmitted to the computing device, step 826. The Product Information message may be transmitted to the computing device in any known unicast method, such as SMS, e-mail, or TCP/IP data message. Alternatively, a Product Information message may be broadcast in a format that can only be processed by the destination computing device, such as encrypted or tagged in a manner that the destination computing device can receive and process and other computing devices will ignore.
The Transaction Server 112 may also record information regarding transmitted Product Information messages, step 828, such as maintaining a database of Product Queries and corresponding Product Information messages transmitted to particular computing devices. Such information may be of value to manufacturers and merchandisers, as well as content providers. For example, such information may be used for future product placement guidance. Further, maintaining a database of Product Information messages may facilitate completing corresponding purchase transactions. The Transaction Server 112 may also inform a merchant when a Product Information message is transmitted so that the merchant can be prepared to receive or recognize a subsequent purchase transaction by the particular mobile device, optional step 830. In addition to enabling the merchant to complete a transaction, such information may be useful to merchants for advertising and market research purposes. Additionally, the Transaction Server may inform the provider of the broadcast content that a user has expressed an interest in a product and a Product Information message has been sent, optional step 832. Content providers may find such information to be useful for generating advertising revenue and developing sponsors for their programs. Once a Product Information message has been transmitted and saved, and interested parties have been informed, the process for responding to a Product Query message may end, step 834. It should be noted that merchants and content providers may not be sent information regarding a particular transaction at the time the transaction is conducted. Instead, such information may be sent periodically, such as once daily, weekly or monthly and in a summary format, since the merchants and content providers may be more interested in general trends and consumer responses than particular transaction details.
If no match or correspondence is found between the image object and a product or products available in the marketplace (i.e., determination 820=“No”), the Recommendation Engine may use information regarding the identified image object, the user comments, and other available and relevant information in order to recommend one or more alternative products that may be of interest to the user, step 836. In identifying a recommended alternative product, the Recommendation Engine may consider a variety of sources of information that may provide insights regarding the user's interests, such as the nature or genre of the broadcast program, products or styles associated with actors within the broadcast program, the particular user's prior purchasing history, the user's demographic information, etc. For example, if a particular product is no longer commercially available, the Recommendation Engine (or other module within the Transaction Server 112) may identify a later model or similar product that is commercially available. Such alternative product recommendations, as well as information regarding the unavailability of the indicated product, may be used in generating the Product Information message, steps 822, 824.
Even if a match or correspondence is found between and a product or products available in the marketplace (i.e., determination 820=“Yes”), the Recommendation Engine may also use information regarding the matched product, the user's Annotation Information, the User Profile, and other available and relevant information in order to recommend one or more additional products that may be of interest to the user, step 836. In identifying recommended additional products, the Recommendation Engine may also consider a variety of sources of information that may provide insight into the user's interests, such as the nature or genre of the broadcast program, products or styles associated with actors within the broadcast program, the particular user's prior purchasing history, the user's demographic information, etc. For example, if the identified and matched product is an article of clothing being worn by an actor in the broadcast program, the Recommendation Engine (or other module within the Transaction Server 112) may identify other products also being worn by the actor, such as sunglasses or a hat. As another example, the Recommendation Engine may recommend products associated with or related to the program or program genre. As a further example, the Recommendation Engine may recommend products based upon prior purchases made by the user, such as refills or additional quantities. Such additional product recommendations, as well as matched product information may be used in generating the Product Information message, steps 822, 824.
While the embodiment process 800 illustrated in
When multiple image objects compare favorably to multiple product placements, or one image object compares favorably to multiple product placements, the Product Correlation Engine may compare the product matches to user Annotation Information to select a most likely product or products of interest, step 904. These comparisons to product placement information may be used to determine whether there is a match or correspondence between the user image selection and product placements, determination 906. If an image object is matched to a product placement (i.e., determination 906=“Yes”), this information may be used to generate the Product Information message, steps 822, 824, which is transmitted to the inquiring computing device, step 826.
If there is no match between the image object and products placed within the broadcast content (i.e., determination 906=“No”), the image object and other information may be provided to the Recommendation Engine (or other module within the Transaction Server 112), step 908, to develop a Product Information. The Recommendation Engine may compare the image object to a database of available merchandise, step 816, in a manner similar to that described above with reference to
Since product placement advertising is an increasingly important form of advertising and content providers can know for certain the products that appear in the video content, content providers may supply a transaction server 112 with more detailed information regarding product placements. The example of a product placement image database that can be used for image comparison algorithm is mentioned above with reference to
An example method for generating Product Information messages utilizing detailed product placement information to obviate the need for image processing is illustrated in
Another embodiment is illustrated in
A Transaction Server 112 may await the response from such operators, step 1104, and when such a response is received, step 1106, compare the received product information to a database of available merchandise, step 1108, to determine if there is a match or correspondence, determination 1110. If the recognized product matches or corresponds to available merchandise (i.e., determination 1110=“Yes”), this information may be used to generate the Product Information message, steps 822, 824, in a manner similar to that described above with reference to
As mentioned above, one method of facilitating a transaction involves transmitting a coupon for a recommended product to the computing device which the user can redeem at the time of purchase.
For a variety of reasons, a Transaction Server 112 may be configured to keep a record of completed transactions resulting from responses to Product Query messages. Such information may be very valuable for merchandisers and content providers as records of actual transactions prompted by the broadcast content, product placements and the services provided by the various embodiments.
While the foregoing description of users selecting products within broadcast content referred to non-advertising content, the embodiments and processes may apply equally well to advertising (i.e., commercial) content. Thus, if a user submits a Product Query including an image from a commercial, the various embodiments will perform in a similar manner to result in a Product Information message related to the commercial. Thus, the various embodiments may provide a direct purchasing option for broadcast commercials.
As mentioned above, an embodiment method for completing a transaction may include transmitting information regarding a merchant or store where the product of interest may be purchased.
In the alert generated in step 1412, the processor may provide the location information of the merchant or store to a navigation or GIS application so that the application can provide the user with driving or walking directions. The alert may also include any other information provided by the merchant or the Transaction Server 112 that may facilitate a purchase transaction, such as store hours, a store telephone number, an advertising display, a listing of additional products that may be of interest to the user carried by the store, and any other similar marketing or useful information.
Typical mobile devices 118 suitable for use with the various embodiments will have in common the components illustrated in
In some mobile devices 118 global positioning system (GPS) receiver circuitry 1509 may be coupled to the processor 1501 and to the antenna 1504. In some implementations, the GPS receiver circuitry 1509 may be incorporated within a part of the wireless transceiver 1505 as illustrative. In other implementations, the GPS receiver circuitry may be a separate module coupled to the processor 1501.
The embodiments described above may also be implemented on any of a variety of computing devices, such as a notebook computer 260 illustrated in
The embodiments described above may be implemented with any of a variety of general purpose computers or server devices, such as the server 1700 illustrated in
The processors in the various computing devices 1501, 1601, 1701 may be any programmable microprocessor, microcomputer or multiple processor chip or chips that can be configured by software instructions (applications) to perform a variety of functions, including the functions of the various embodiments described herein. In some mobile devices, multiple processors 1501, 1601, 1701 may be provided, such as one processor dedicated to communication functions and one processor dedicated to running other applications. Typically, software applications may be stored in the internal memory 1502, 1602, 1702 before they are accessed and loaded into the processor 1501, 1601, 1701. In some mobile devices, the processor 1501 may include internal memory sufficient to store the application software instructions. In some computing devices, the memory may be in a separate memory chip coupled to the processor 1501, 1601, 1701. In many computing devices 118, 206, the internal memory 1502 may be a volatile or nonvolatile memory, such as flash memory, or a mixture of both. For the purposes of this description, a general reference to memory refers to all memory accessible by the processor 1501, 1601, 1701, including internal memory 1502, 1602, 1702, removable memory plugged into the computing device, and memory within the processor 1501, 1601, 1701 itself.
The foregoing method descriptions and the process flow diagrams are provided merely as illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the order presented. As will be appreciated by one of skill in the art the order of steps in the foregoing embodiments may be performed in any order. Words such as “thereafter,” “then,” “next,” etc. are not intended to limit the order of the steps; these words are simply used to guide the reader through the description of the methods. Further, any reference to claim elements in the singular, for example, using the articles “a,” “an” or “the” is not to be construed as limiting the element to the singular.
The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The hardware used to implement the various illustrative logics, logical blocks, modules, and circuits described in connection with the aspects disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but, in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Alternatively, some steps or methods may be performed by circuitry that is specific to a given function.
In one or more exemplary aspects, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. The steps of a method or algorithm disclosed herein may be embodied in a processor-executable software module executed which may reside on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that may be accessed by a computer. By way of example, and not limitation, such computer-readable media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to carry or store desired program code in the form of instructions or data structures and that may be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a machine readable medium and/or computer-readable medium, which may be incorporated into a computer program product.
The preceding description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.
