Patent Application
20080187188
This invention relates in general to delivered content identification, and more particularly to systems, apparatuses and methods for facilitating efficient recognition of delivered content.
When originally introduced into the marketplace, analog mobile telephones used exclusively for voice communications were viewed by many as a luxury. Today, mobile communication devices are highly important, multi-faceted communication tools. A substantial segment of society now carries their mobile devices with them wherever they go. These mobile devices include, for example, mobile phones, Personal Digital Assistants (PDAs), laptop/notebook computers, and the like. The popularity of these devices and the ability to communicate “wirelessly” has spawned a multitude of new wireless systems, devices, protocols, etc. Consumer demand for advanced wireless functions and capabilities has also fueled a wide range of technological advances in the utility and capabilities of wireless devices. Wireless devices not only facilitate voice communication, but also messaging, multimedia communications, e-mail, Internet browsing, and access to a wide range of wireless applications and services.
More recently, wireless communication devices are increasingly equipped with other media capabilities such as radio receivers. Thus, a mobile phone can be equipped to receive amplitude modulated (AM) radio and/or frequency modulated (FM) radio signals, which can be presented to the device user via a speaker or headset. With the processing power typically available on such a mobile communication device, broadcast radio can be a more rich experience than with traditional radios. For example, a terminal (e.g., mobile phone, PDA, computer, laptop/notebook, etc.) is often equipped with a display to present images, video, etc. Terminals are also often capable of transmitting and/or receiving data, such as via GSM/GPRS systems or otherwise. These technologies enable such terminals to present images, video, text, graphics and/or other visual effects in addition to presenting the audio signal received via the radio broadcast. For example, the song title, artist name and/or other information relating to a song broadcast from a radio station can be provided to a terminal for visual presentation in addition to the audio presentation.
Currently, such a “visual radio service” is provided by a limited number of radio stations that are integrated with the visual radio content creation tools. A first problem involves the inability to provide visual radio content (e.g., song title, artist name, etc.) for any radio station that the broadcast radio-equipped terminal is capable of listening to. One current approach is that such a service has to be “integrated with” each radio station separately, and great effort is required to keep such a visual service running. Only those radio stations where visual radio is integrated with the radio automation system can deliver such a service. It is difficult to provide tight synchronizations in the case of a last minute change in a schedule of radio station.
One manner of addressing such a problem is to utilize song identification techniques. If a terminal can identify the song that is being played on the radio, this knowledge can be used to gather additional information relating to the song. However, such identification can be extremely processor intensive, which consumes processing power and adversely affects terminal battery life. Further, all of the song identification data created by every mobile device may unnecessarily consume a substantial quantity of bandwidth if sent from the terminals, which may also cost the terminal user financially for data communications volumes and/or times. Additionally, if the song identification takes a significant amount of time to develop, and/or takes a significant amount of time en route on a network, an unacceptable delay in presenting any visual radio information may occur.
Accordingly, there is a need in the industry for a manner of reducing the load on terminals, network elements and/or the network generally where accompanying data is provided in connection with radio and/or other media broadcasts. The present invention fulfills these and other needs, and offers other advantages over the prior art.
To overcome limitations in the prior art described above, and to overcome other limitations that will become apparent upon reading and understanding the present specification, the present invention discloses systems, apparatuses and methods for enhancing media fingerprint calculations by distributing the fingerprinting task among multiple terminals.
In accordance with one embodiment, a method is provided including distributing a task of calculating a plurality of fingerprint portions corresponding to a media stream among a plurality of terminals. The plurality of calculated fingerprint portions is aggregated to create a stream(s) of fingerprints. Content corresponding to the media stream is identified using at least a portion of the stream of fingerprints.
According to one embodiment of such a method, one or more of the fingerprint portions include partial fingerprints forming less than a complete fingerprint, and aggregating the plurality of calculated fingerprint portions involves deriving at least one complete fingerprint based on an aggregation of a plurality of the partial fingerprints. In another embodiment, one or more of the fingerprint portions include complete fingerprints each capable of identifying the media stream.
In another embodiment of the method, aggregating the plurality of calculated fingerprint portions involves forming an end-to-end chain of the calculated fingerprint portions from the plurality of terminals to create a substantially continuous stream of the fingerprints. In a more particular embodiment, identifying content corresponding to the media stream involves using the substantially continuous stream of fingerprints to identify changes in the media stream. In still another particular embodiment, using the substantially continuous stream of fingerprints to identify changes in the media stream involves identifying a change from one media item to another media item based on a change in the substantially continuous stream of fingerprints provided by the plurality of terminals.
Another embodiment of such a method involves distributing the task of calculating a plurality of fingerprint portions by distributing the calculation task of an over-the-air radio broadcast among the plurality of terminals, and where identifying content corresponding to the media stream involves identifying visual information associated with an audio track of the radio broadcast being presented on the plurality of terminals. In one particular embodiment, it is further determined which terminals are tuned to the radio broadcast to identify the plurality of terminals that will calculate the plurality of fingerprint portions.
One embodiment of such a method further includes transmitting the plurality of calculated fingerprint portions in a single fingerprint stream :for remote identification of the content associated with the media stream. In an alternative embodiment, the method involves transmitting the plurality of calculated fingerprint portions in a plurality of fingerprint streams facilitate parallel identification of the content associated with the media stream. In one particular embodiment, transmitting multiple fingerprint streams in parallel involves temporally overlapping the calculated fingerprint portions of the plurality of fingerprint streams.
In another embodiment of such a method, the identified content is transmitted to each of the plurality of terminals involved in the calculation of the fingerprint portions. In still another embodiment, calculating a plurality of fingerprint portions of different content fingerprint portions involves each of the plurality of terminals generating one or more different digital packets of information indicative of respective audio segments of the media stream occurring at different time intervals. In one embodiment, one or more of the different fingerprint portions includes at least some overlapping fingerprint data.
In accordance with another embodiment of the invention, a method is provided that includes receiving an over-the-air media stream including at least an audio component. A subset of the audio component that has been allocated for processing is identified. At least one digital fingerprint is calculated for the identified subset of the audio component, and the digital fingerprint(s) is transmitted.
In one embodiment of such a method, identifying a subset of the audio component that has been allocated for processing involves identifying the subset of the audio component in response to receipt of a fingerprint distribution notification. In a more particular embodiment, the fingerprint distribution notification is received from a server via a network. One particular embodiment involves transmitting the calculated digital fingerprint(s) to a processing system capable of recognizing a fingerprint stream including the calculated digital fingerprint(s) and other calculated digital fingerprints based on other subsets of the audio component.
In one embodiment, receiving an over-the-air media stream including at least an audio signal comprises receiving a radio broadcast signal of a song, and wherein identifying a subset of the audio component comprises identifying one or more time intervals of the song in which a respective one or more digital fingerprints are to be calculated. One embodiment involves transmitting one or more of the calculated digital. fingerprints as time multiplexed portions of a single fingerprint stream, while another embodiment involves transmitting one or more of the calculated digital fingerprints as time multiplexed portions of multiple fingerprint streams.
In one particular embodiment of such a method, at least the audio component of the media stream is audibly presented, content identified in response to the transmission of the at least one digital fingerprint is received, and the received content is presented during at least some of the audible presentation of the audio component of the media stream.
In another embodiment, the method is carried out via a mobile terminal, and a radio landscape data set is provided including information indicative of a location of the mobile terminal. In another embodiment, it is determined when to create the digital fingerprint(s). One embodiment of such a method includes transmitting location parameters, receiving an indication of a current location generated in response to the location parameters, and identifying a globally-unique radio station identifier to which a radio receiver is tuned based on the current location and a frequency to which the radio receiver is tuned.
In accordance with another embodiment, a method is provided that includes receiving a plurality of content fingerprint portions from a plurality of mobile terminals, where each content fingerprint portion representative of a portion of a media stream. Digital information is located using one or more of the plurality of content fingerprint portions, and the located digital information is transmitted for use by the plurality of mobile terminals.
In one embodiment of such a method, one or more of the content fingerprint portions comprise partial fingerprints forming less than a complete fingerprint, and aggregating the plurality of calculated fingerprint portions involves deriving at least one complete fingerprint based on an aggregation of a plurality of the partial fingerprints. In another embodiment, one or more of the content fingerprint portions comprise complete fingerprints each capable of identifying the media stream.
One embodiment of the method involves notifying the mobile terminals to which portion of the media stream it should create a partial content fingerprint. In one embodiment, receiving a plurality of partial content fingerprints involves: receiving the plurality of partial content fingerprints via a single data stream, while in another embodiment receiving a plurality of partial content fingerprints involves receiving the plurality of partial content fingerprints via multiple parallel data streams. In one embodiment, receiving the plurality of partial content fingerprints via multiple parallel data streams involves receiving a first data stream of concatenated fingerprint samples, and receiving one or more second data streams different concatenated fingerprint samples. In a particular embodiment, the concatenated fingerprint samples from the first and second data streams are temporally overlapping.
Other embodiments of such a method include aggregating at least some of the partial content fingerprints, and determining a radio station to which each of the plurality of terminals is tuned.
In accordance with one embodiment, an apparatus is provided that includes a radio receiver to receive an over-the-air media stream, a fingerprint extraction module configured to sample a subset of the media stream, a fingerprint calculation module to generate fingerprints for each of the portions sampled, and a transmitter to transmit the generated fingerprints.
In one embodiment, a data receiver is provided to receive content related to the media stream and identified using the transmitted fingerprints. In a more particular embodiment, a display visually presents the received content related to the media stream, and in another embodiment a speaker audibly presents the received media stream contemporaneously with the visual presentation of the received content. One embodiment includes the transmitter further transmitting radio landscape data including information indicative of a geographic location of the apparatus.
In accordance with one embodiment, an apparatus is provided that includes a receiver to receive a plurality of fingerprints from a respective plurality of terminals, where each fingerprint at least partly representative of a media stream. The apparatus also includes a processing module configured to locate digital information in a database based on the plurality of fingerprints, and a transmitter to transmit the digital information for use by the plurality of terminals.
The above summary of the invention is not intended to describe every embodiment or implementation of the present invention. Rather, attention is directed to the following figures and description which sets forth representative embodiments of the invention.
The invention is described in connection with the embodiments illustrated in the following diagrams.
In the following description of exemplary embodiments, reference is made to the accompanying drawings which form a part hereof, and in which is shown by way of illustration various manners in which the invention may be practiced. It is to be understood that other embodiments may be utilized, as structural and operational changes may be made without departing from the scope of the present invention.
Generally, the present invention provides systems, apparatuses and methods for enhancing media fingerprint calculations by distributing the fingerprinting task among multiple terminals. Media such as radio or other audio may be transmitted via a transmission frequency or channel, where multiple mobile terminals may located such that they can be tuned or otherwise capable of recognizing the media via that frequency/channel. For example, a radio station may transmit a radio signal, and a plurality of mobile terminals within a transmission range are tuned to the same station to receive that radio signal. In such cases, media fingerprint calculations may be distributed among a plurality of the receiving terminals in accordance with the invention.
The description provide herein often refers to radio content (e.g., broadcast radio such as AM/FM radio) as a media type, but it should be recognized that the present invention is equally applicable to any type of transmitted media. In one embodiment, the invention provides approaches to content generation that allows a visual radio service (e.g., NOKIA Visual Radio Service™) for any radio station that is received by a mobile terminal. These radio stations may be any type, such as frequency modulated (FM), amplitude modulated (AM), etc. As used herein, visual radio (or analogously, visual media) involves any visually presented information associated with the audio transmission, such as the song title, artist, album cover art, advertiser/product, and/or other information that may correlate to the provided audio transmission.
Presently, a visual radio service can be provided for a limited number of stations that are equipped with visual radio content creation tools. However, it would be desirable to provide such visual radio content for any radio/media station and not only for those that have been equipped with specific visual radio content tools. The present invention provides, among other things, manners for providing data such as visual radio content to any mobile terminal equipped with a receiver module(s) capable of receiving and presenting the audio and visual content. If each receiving terminal is responsible for assisting in song/media recognition in the radio/media program, there is duplication of such efforts that consumes bandwidth, battery power, etc. The present invention addresses manner of reducing the load on the terminal, network, server and/or other such components of the system.
One embodiment of the invention proposes manners for enabling content generation for services such as visual radio services, without the otherwise required integration with radio station content automation systems. One embodiment involves using song recognition technology, where the mobile terminal calculates the audio fingerprint and provides it to a server(s) for recognition and content creation. Generally, “fingerprinting” is a technique used for song identification. Fingerprints are smaller than the actual digital content but contain enough information to uniquely identify the song or other media item. Each audio fingerprint is unique and can be used to precisely identify a song or other media item. Any known “fingerprinting” technology may be used in connection with the invention.
After receiving the fingerprint and identifying the music piece or other audio, the visual radio server can send content that matches the currently broadcast song or other media item to the terminal. In accordance with the invention, the fingerprint calculation is distributed among multiple terminals based on the fact that there can be several mobile terminals tuned to the same station in the area.
In order to generate the visual content with the radio or other media broadcast, the fingerprinting task is performed relatively continuously, or at least repetitively, in accordance with one embodiment of the invention. By continuously and/or repeatedly recognizing the broadcast content, a song (or other media itern) change can be readily determined. By recognizing the song change, visual content for the terminated song can be discontinued, and a new portion of visual content can be created for the new song. Under normal circumstances, a single fingerprint from a terminal may be sufficient to identify the song/media item. Thus, as described more fully below, the fingerprint(s) received from each one of the plurality of collaborating terminals may be sufficient to identify the song or other media item.
Based on the received radio signal, each terminal 102, 104 can invoke a fingerprint calculation module 110, 112 which will collectively serve as the fingerprint calculation functionality 114. For example, the fingerprint calculation module 110 associated with the terminal 102 can calculate a first fingerprint portion-A 116, and an nth terminal 104 can calculate an nth fingerprint portion 118. Collectively, the fingerprint portions 116, 118 can provide sufficient fingerprint information to enable a server or other module to identify the media and return the visual information or other related data. The portions 116, 118 may be provided to the server or other module via a network 120, and/or may be provided in other known manners including but not limited to infrastructure-based networks (e.g., Internet, LAN, etc.), proximity networks (e.g., Bluetooth, WLAN, peer-to-peer networking, etc.), cellular networks (e.g., GSM/GPRS, etc.), direct connections (e.g., USB, firewire, etc.) and the like. While the fingerprint calculation modules 110, 112 need not be physically part of their respective terminals 102, 104, one embodiment of the invention involves physically embedding these modules/functionality 110, 112 within their respective terminals.
A stream of fingerprints is created 202. For example, one terminal can create a first fingerprint, and a second terminal can create a second fingerprint. A stream of fingerprints can be formed from the first and second fingerprints. In a more particular example, two or more terminals may calculate fingerprint portions at alternating time intervals of an audio signal, where the resulting calculations are used to derive an aggregate fingerprint capable of identifying the audio signal to a downstream entity (e.g., a music recognition server). While in one embodiment the “stream” of fingerprints is represented by an end-to-end chain of the fingerprints, a “serial” stream is not required, as parallel streams may additionally or alternatively be used.
In another embodiment, a fingerprint may be derived based on an aggregation of calculated fingerprint portions, where at least some of the portions represent an incomplete portion of a complete fingerprint. Such a derivation may be performed by, for example, assembling the resulting incomplete fingerprint portions in an order corresponding to the original audio stream from which the fingerprint portions were calculated. Thus, by concatenating the incomplete fingerprint portions from the plurality of terminals in the proper order, the resulting derived fingerprint can be substantially the same as if a single terminal generated the entire, complete fingerprint. In one embodiment, a server or other processing system receives the fingerprint portions and derives the resulting fingerprint, although the terminal or other entity can create the resulting fingerprint.
Using the received fingerprints, content associated with the media stream can be identified 204. By repeatedly sending such fingerprints, it can be determined when the media stream changes such that any returned content should be changed. For example, by continuing to calculate fingerprints, it can be determined when a radio broadcast song has ended and a new song has begun. In one embodiment, a database of content is stored, where the fingerprints are used to ultimately locate the data in the database that corresponds to that fingerprint. This database can be stored in a stand-alone terminal, server, etc. This database can alternatively be stored in a distributed server system, and/or in other distributed systems including any one or more of the terminal, server, etc. In one embodiment, the content is stored in a database associated with a server, where the derived fingerprint is used to index the database to obtain the associated content. This content may be any content associated with the media stream. For example, where a radio broadcast represents the media stream, the “content” may be visual radio content such as a song title, author, album cover art, artist photos or images, related trivia, artist biographies, and/or any other information that may pertain to the current media stream item. In other embodiments, the content may not specifically relate to the current media stream item (e.g., song), but may represent other content such as advertisements, coupons or discounts, “next song” indications, etc.
As indicated above, the fingerprint(s) can ultimately be used to identify the desired content, such as the desired visual content associated with a song received at a terminal via a radio broadcast. For example, the fingerprint(s) can be used to identify a song or other media identifier (ID), where that ID is internal to the recognition system. Then, song/media metadata or other such data can be identified, such as an artist name, title, album name, etc. Using the song/media metadata, the related content in a database can be identified. In another embodiment, the related content may be directly linked to the song/media ID. In another embodiment, the content may be directly obtained using the fingerprint. Any manner of ultimately identifying the desired content using the fingerprint(s) may be used in accordance with the present invention. Thus, where the present description indicates that content may be obtained or otherwise identified using the fingerprint, this does not imply that the fingerprint is used to directly obtain the desired content (e.g., visual radio content). Rather, the use of the fingerprint to obtain or otherwise identify the desired content can be direct or indirect use of the fingerprint, and reference herein to using the fingerprint to obtain/identify the data is not limited to any particular direct or indirect way of using the fingerprint in this manner.
A recipient device may use the information to create an aggregate fingerprint from the various fingerprint portions, and identify the song or other media item to which the aggregate fingerprint is associated with. When the song or other media item is identified, then content associated therewith (e.g., song title, artist, etc,) can be provided.
By distributing the fingerprint calculation task, the recognition can be more reliable since several fingerprint streams can be used which can overcome the limitation of recognition algorithms that can start the identification after the fingerprint data collection period. Further, fingerprint calculation reduces stress on the batteries of each of the calculating devices since the processor-intensive fingerprinting operations will be reduced. Fingerprint distribution also potentially provides for better overall quality fingerprinting because some devices can obtain a better signal than others. For example, a terminal remote from the radio station may benefit by having other closer terminals provide a portion of the aggregate fingerprint. The distribution of the fingerprinting task can also result in less network traffic, which conserves network capacity/bandwidth, and results in less cost for users that pay by transmission time and/or transmitted data volume. The receiving server(s) can also benefit because, for example, it theoretically only has to handle one (or at least fewer) distributed stream of fingerprints per station rather than N streams (where N represents the number of terminals tuned to the same radio station).
It should be noted that the collaboration in generating the fingerprint does not imply or require precision in the allocation of this task; i.e., there cart be gaps or overlap in the calculation. For example, one mobile terminal can begin calculating at an approximate point in a radio station-received song, and another mobile terminal can begin calculating at another point although some overlap or gap occurs, as long as the resulting fingerprint provides enough data for a receiving module to use the fingerprint portions to identify the song, advertisement or other media that was received by the mobile terminals.
In the embodiment illustrated in
Each mobile terminal that is involved in the distributed fingerprint calculation may determine the fingerprints at certain times or in connection with a certain event(s). For example, the terminals tuned to and within a reasonable receiving range of a common station can periodically calculate the fingerprints, such as every X seconds. The distributed fingerprint calculation may also be initiated at each participating terminal upon occurrence of a triggering event. One such triggering event can be a time, such as 09:10:15, 09:10:20, 09:10:25, etc. Such triggering times or durations may be adjusted depending on the number of mobile terminals tuned to the particular station. For example, the more participating terminals, the fewer times a particular terminal needs to perform its calculation. Other triggering events may include, for example, receipt of a triggering signal, recognized events such as recognizing a gap in the audio that could indicate changing from one song to another, etc. Many other events may be implemented to initiate the distributed calculation in each of the participating devices.
In the illustrated embodiment of
It should be recognized that the “servers” 320, 324, 328 represent any entity capable of providing the noted services. The servers 320, 324, 328 can be discrete elements, or can be partially or completely combined into one or more servers. The servers may be accessible via any known manner, such as by way of a network(s) and/or via direct wired or wireless connections. The servers may be stand-alone or distributed. Accordingly, the illustrated representation of the servers 320, 324, 328 is intended to facilitate an understanding of one representative embodiment of a content delivery mechanism, but the invention is clearly not limited to any particular arrangement or structure of servers.
As shown in
An example of the user's interaction to select a station in this manner is depicted in
One embodiment recognizes that the user can enter/select: an incorrect location 408. There is always the possibility of the user erroneously entering the wrong location, or that the user simply does not know the location at the level of detail being requested. For example, if the location entry 408 requires a ZIP code, the user may not know the ZIP code for the current location, particularly if the user is traveling. There are numerous reasons why a location can be entered or selected incorrectly it the user terminal. In such cases, other options can be alternatively or additionally used to assist in determining the user's current location. In one embodiment, the user may be tuned to a radio station frequency, and the location of the mobile terminal itself can be determined rather than entered by the user. Location data may be obtained using, for example, global positioning system (GPS) information if the terminal is so equipped. Such data may also be obtained using cell identification (ID) information, since a cellular network will know the location of the terminal for purposes of locating the terminal if an incoming call occurs. Because a radio station transmission range is relatively large and does not typically involve precise transmission range boundaries, great precision in the terminal's location is not required, and locations such as the location based on cell ID is suitable in one embodiment. Other location services may alternatively/additionally be implemented. In any event, the radio station frequency and location data is known for a terminal which can be sent to the SDS 332 to obtain the station name and visual channel ID.
Another embodiment is an RDS-assisted embodiment, where Radio Data System (RDS) information is utilized. As is known, RDS or other analogous services involve sending small amounts of data via radio broadcasts, such as FM broadcasts. Such systems may include standard information, such as time and station identification. For any such system providing a station identification/name, the station name can be used to ensure that the terminal is tuned to the correct station. An RDS (or analogous) server 412 is shown in
Still another representative embodiment for identifying the radio station to which a terminal is tuned is to statistically identify whether a terminal is tuned to the station that it professes to be tuned. For example, assume that one hundred terminals indicate that they are tuned to Radio Station-A at frequency-A. If some designated majority (e.g., 90%) of those terminals that are supposedly tuned to Radio Station-A are reporting the same song/content, then it can be assumed that any other terminals that are supposedly tuned to Radio Station-A, but are reporting a different song than the majority, are in fact not tuned to Radio Station-A. This may be because, for example, the minority terminals incorrectly identified their location in the first place, or the terminal roamed out of the area such that another radio station at the same frequency became the dominant radio signal. In such cases, the visual radio system can ignore the fingerprint data provided by those seemingly deviant terminals, and rely on the fingerprint data provided by the terminals that statistically suggest location accuracy.
The fingerprinting task may be distributed substantially evenly among the terminals, or may be distributed in a weighted manner. For example, the task may be weighted more heavily to terminals having better reception for any reason such as proximity to the radio station, design, battery power, etc. In one embodiment, the fingerprinting task is distributed substantially evenly among the terminals tuned to the station frequency in the relevant area.
As noted above, the invention enables content such as visual content to be obtained for related content provided by any media station, by generating fingerprints and using those fingerprints to directly or indirectly identify the desired content in a content database. The frequency or other channel that each terminal is tuned to should be determined so the fingerprint task can be distributed appropriately. There is no prior art solution to reliably determine which channel a terminal is tuned to, as the same frequency may be used in a different geographical location. Thus, frequency tuning is not enough, as the location or other parameters must be ascertained to reliably determine which station a device is tuned to.
One embodiment of the invention enables reliable determination of the tuned channel by collecting information available in the radio landscape and comparing that to a database 410 populated with radio landscape information. In the context of broadcast radio, each radio station is associated with a globally-unique identifier. One aspect of the invention contemplates manners of determining this globally-unique identifier, the knowledge of which provides a reliable indication of the terminal's current location. Knowing the frequency that the terminal is tuned to, and knowing the terminal's current location, a reliable determination of the radio station to which a terminal is tuned can be made. The fingerprinting task can then be properly distributed among those terminals tuned to the same station (and thereby listening to the same song).
In one embodiment information available regarding the radio channel landscape is used to reliably determine the globally-unique radio channel identity. This information may include any one or more of the following representative types of information. For example, the radio receiver at a terminal can detect the current frequency that the receiver is tuned to, so the tuned frequency represents one type of information usable to determine the globally-unique radio channel identity. Another example is that the radio receiver may detect the RDS identity of the current channel. Further, the radio receiver can scan and detect all the frequencies of the channels that it could tune into at it's current location. If desired, scanning can be effected with different levels of sensitivity to determine the large landscape and/or the local landscape. The radio receiver may scan and detect the RDS identity of the currently-available channels with RDS enabled. The radio receivers may provide their geographic position, such as via GPS technology. In another embodiment, the receiver can record an audio sample of the current broadcast. The radio receiver may recognize the current audio element, for example the currently playing song, the time and the position within the audio element. A mobile terminal can provide additional information that can be used to limit the geographical area and possible set of radio stations. This information may include, for example, the positioning data or information relating to coordinates and/or mobile cell ID, operator name operator ID, etc.
This and/or other such information can be used in determining the globally-unique identity of the channel. Not all of the information need be provided, but rather numerous subsets of the information can be sufficient to arrive at the globally-unique identity of the channel. Some information may be easier for the terminal to acquire than others, such as the current channel frequency. The current channel frequency is a piece of information that is typically available to a radio receiver. The radio receiver can then scan and automatically detect other radio stations that exist in area. Some information may be lacking due to missing hardware or service support; for example a positioning system such as GPS.
Information relative to a particular mobile terminal, such as the list of recognized radio frequencies and corresponding signal strength data for all of the radio stations in the terminal's vicinity, can be referred to as the radio landscape fingerprint. This list of received stations and signal strength can in a relatively unique fashion identify the position of the radio receiver. For example, if a radio receiver can pick up a particular eight radio stations, the radio receiver must be within a certain transmission range of each of the radio stations and thus the radio receiver's approximate location can be determined. In accordance with one embodiment of the invention, the database 410 may be provided with such radio channel landscape information. The radio landscape fingerprint from a radio receiver can be matched against the database 410 to determine the radio receiver's approximate location, and the globally-unique identity of the channel that it is tuned into. In one embodiment, the terminal 400 can obtain the list of radio frequencies at particular geographic locations from a database such as the station directory service 332. Such a database 332 may include, for example, a database of radio stations around the world and their corresponding frequencies at particular geographic locations (e.g., cities, approximate coordinate boundaries, etc.).
A mobile terminal equipped with a radio receiver can compile a collection of available landscape information such as the current frequency, other available frequencies, the RDS information of all available frequencies, current location, the currently playing song, etc. These and other types of information available for a radio receiver are shown in
When at least some of such information is collected, a query is made by the radio receiver to the radio landscape database 410 with the compilation of information. In one embodiment the database 410 is available via a network(s) 402, but can alternatively be a database locally at the radio receiver; i.e., at the mobile terminal. The radio landscape database 410 may be associated with a server that tries to match the provided compilation of information with the information stored at the database 410. Representative examples of the type of information that may be stored at the database 410 is shown in
Regardless of the particular information used, the information available for a radio receiver (e.g., examples shown in column 420) is mapped against the information in the database (e.g., examples shown in column 428) to determine the globally-unique identity of the channel. If a match is found, the identify of the channel is returned to the mobile terminal. In this manner, a high degree of confidence is achieved in the actual radio station to which the terminal is tuned. On the terminal side, one exemplary implementation is a C++ or java application that utilizes services in the mobile terminal, such as an FM tuner that is RDS enabled, positioning services of the terminal, etc.
Thus, an improved manner of acquiring radio channel identity is provided, which allows the fingerprinting task to be allocated among those terminals determined to be tuned to the same radio channel. This provides an automated method that does not require terminal user actions, and is not prone to intentional or unintentional incorrect location selections by terminal users. The solution does not require RDS for the radio stations, nor does the solution require proprietary extensions of radio stations or globally unique RDS identifiers for the RDS data elements. The solution also does not require that radio stations maintain up-to-date RDS data elements.
The use of a control channel and corresponding control protocol to distribute the fingerprinting task among a plurality of terminals is generally illustrated in
In one embodiment, the use of the control channel 514, 516 and the passing of messages 510, 512 is server controlled. For example, in one embodiment the visual radio server 504 is aware of the number of terminals 500, 502 listening to the same station, and can make a decision regarding calculation start times and intervals for each of the terminals so that it obtains a sufficient quantity of fingerprint data to identify the media (e.g., song). In such an embodiment, the server 504 can send to each terminal 500, 502 a message 510, 512 with the period for the fingerprint calculation and the starting time. A programming example is shown in Example 1 below:
The server 504 can later change the calculation timing for some of the terminals by re-sending a command. An example is shown in Example 2 below:
In response to such messages, content 518, 520 is returned from the server 504 to the client 506, 508. In a visual radio embodiment, such content may include any one or more of a song title, artist, length of song, etc.
It is possible that some terminals may disconnect from the distributed fingerprinting task while other terminals are joining. In one embodiment, the server 504 has the responsibility to maintain the fingerprint distribution as evenly as possible, although there is no need for precision and superfluous data will only improve the recognition quality. Thus, if the number of participating terminals is reduced, the server 504 may, for example, add time to the fingerprinting calculation interval for the terminals.
In one embodiment, the use of the control channel 514, 516 and the passing of messages 510, 512 is terminal controlled. For example, in one embodiment the terminal itself can make a decision on the start time and interval of the fingerprint calculation. As an example, the start time can be a random value, and the period can be s;et to the time synchronization interval. If number of terminals is large enough, then statistically there will be sufficient fingerprint data provided to the server to for adequate song/media recognition, such as in the case of Gaussian distribution. The terminal can send one or more complete or partial fingerprints in a discrete message(s), or may send the one or more complete or partial fingerprints to the server together with another message(s) already being sent via the control channel (e.g., time synchronization message) in order to conserve bandwidth. For example, the time synchronization message or “keep alive” message is part of existing radio communication protocols for a visual radio control channel, and is sent periodically to ensure that the terminal is still connected and operational. The fingerprint(s) may be sent with this or other existing traffic, or may be sent independently of existing traffic.
In another embodiment, a combination of server control and terminal control may be utilized. For example, in one embodiment the terminal can select a random value for the start interval using the “keep alive” interval. The server can determine the success of the results, and if the server is not satisfied it can set a new interval by sending an appropriate command (e.g., <fingerprint action=“restart”>). Memory can also be implemented to store the previous interval for later use.
As can be seen from
The sharing of the fingerprint distribution task is shown in
In another embodiment, multiple streams of fingerprints can be provided to facilitate faster recognition at the recognition backend. For example,
In one embodiment, the samples are overlapping as shown in
A representative system in which the present invention may be implemented or otherwise utilized is illustrated in
The representative terminal 800A utilizes computing systems to control and manage the conventional device activity as well as the device functionality provided by the present invention. For example, the representative wireless terminal 800B includes a processing/control unit 810, such as a microprocessor, controller, reduced instruction set computer (RISC), or other central processing module. The processing unit 810 need not be a single device, and may include one or more processors. For example, the processing unit may include a master processor and one-or more associated slave processors coupled to communicate with the master processor.
The processing unit 810 controls the basic functions of the terminal 800B as dictated by programs available in the program storage/memory 812. The storage/memory 812 may include an operating system and various program and data modules associated with the present invention. In one embodiment of the invention, the programs are stored in non-volatile electrically-erasable, programmable read-only memory (EEPROM), flash ROM, etc., so that the programs are not lost upon power down of the terminal. The storage 812 may also include one or more of other types of read-only memory (1ROM) and programmable and/or erasable ROM, random access memory (RAM), subscriber interface module (SIM), wireless interface module (WIM), smart card, or other fix,ed or removable memory device/media. The programs may also be provided via other media 813, such as disks, CD-ROM, DVD, or the like, which are read by the appropriate interfaces and/or media drive(s) 814. The relevant software for carrying out terminal operations in accordance with the present invention may also be transmitted to the terminal 800B via data signals, such as being downloaded electronically via one or more networks, such as the data network 815 or other data networks, and an intermediate wireless network(s) 816 in the case where the terminal 800A/800B is a wireless device such as a mobile phone.
For performing other standard terminal functions, the processor 810 is also coupled to user input interface 818 associated with the terminal 800B. The user input interface 818 may include, for example, a keypad, function buttons, joystick, scrolling mechanism (e.g., mouse, trackball), touch pad/screen, or other user entry mechanisms (not shown).
A user interface (UI) 820 may be provided, which allows the user of the terminal 800A/B to perceive information visually, audibly, through touch, etc. For example, one or more display devices 820A may be associated with the terminal 800B. The display 820A can display web pages, images, video, text, links, visual radio information and/or other information. A speaker(s) 820B may be provided to audibly present instructions, information, radio or other audio broadcasts, etc. Other user interface (UI) mechanisms can also be provided, such as tactile 820C or other feedback.
The exemplary mobile device 800B of
In one embodiment, the storage/memory 812 stores the various client programs and data used in connection with the present invention. For example, a fingerprint extractor module 830 can be provided at the device 800B to sample an audio stream received by way of a broadcast receiver, such as the radio receiver/tuner 840. The device 800B includes a fingerprint calculation module 832 to generate the fingerprint portions previously described. These and other modules may be separate modules operable in connection with the processor 810, may be a single module performing each of these functions, or may include a plurality of such modules performing the various functions. In other words, while the modules are shown as multiple software/firmware modules, these modules may or may not reside in the same software/firmware program. It should also be recognized that one or more of these functions may be performed using hardware. For example, a compare function may be performed by comparing the contents of hardware registers or other memory locations using hardware compare functions. These modules are representative of the types of functional and data modules that may be associated with a terminal in accordance with the invention, and are not intended to represent an exhaustive list. Also, other functions not specifically shown may be implemented by the processor 810.
The illustrated computing system 850 also includes DSP circuitry 866, and at least one transceiver 868 (which of course is intended to also refer to discrete transmitter/receiver components). While the server 850 may communicate with the data network 815 via wired connections, the server may also/instead be equipped with transceivers 868 to communicate with wireless networks 816 whereby an antenna 870 may be used.
Hardware, firmware, software or a combination thereof may be used to perform the functions and operations in accordance with the invention. Using the foregoing specification, some embodiments of the invention may be implemented as a machine, process, or article of manufacture by using standard programming and/or engineering techniques to produce programming software, firmware, hardware or any combination thereof. Any resulting program(s), having computer-readable program code, may be embodied within one or more computer-usable media such as memory devices or transmitting devices, thereby making a computer program product, computer-readable medium, or other article of manufacture according to the invention. As such, the terms “computer-readable medium,” “computer program product,” or other analogous language are intended to encompass a computer program existing permanently, temporarily, or transitorily on any computer-usable medium such as on any memory device or in any transmitting device.
From the description provided herein, those skilled in the art are readily able to combine software created as described with appropriate general purpose or special purpose computer hardware to create a computing system and/or computing subcomponents embodying the invention, and to create a computing system(s) and/or computing subcomponents for carrying out the method(s) of the invention.
The foregoing description of the exemplary embodiment of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not with this detailed description, but rather determined by the claims appended hereto.
