Patent Application
20240404535
In order to measure exposure of people to media content such as radio, television, online media, and/or commercial media, a media-monitoring company can arrange for a representative population of users to implement media-monitoring devices, i.e., meters, designed to detect hidden identification tones embedded in the media content and to log and report their findings.
These meters could take various forms and could be provided and operated in various locations. By way of example, some meters may be configured for use in a fixed location such as a household, possibly to be attached by cables to devices such as televisions, set top boxes, or routers, and/or provided as separate devices, and arranged to monitor media signals flowing to or from those devices and/or to monitor acoustic audio output and/or wirelessly transmitted media signals. Further, other meters may be portable, arranged to be worn or otherwise carried by users as they move about and to passively monitor acoustic audio and/or other media signals in the users' environment. In addition, some meters may be integrated with devices, such as mobile phones, televisions, set top boxes, routers, or the like, and similarly configured to monitor media content in various ways.
In an example system, the hidden identification tones could define audio watermarks. For instance, each of various broadcast stations (e.g., stations or channels provided through over the air broadcast, cable or satellite TV transmission, or over-the-top streaming, among other possibilities) could be watermarked with a respective station identification or “SID” value, to enable meters to detect and report media presentation on a per station basis. Further, to allow meters to regularly monitor media exposure, the audio media of each broadcast station could be repeatedly watermarked with the station's respective SID value.
Each instance of such a watermark could be encoded in the audio media by adding into the audio media a supplemental audio signal that is preferably outside the range of human hearing but that is detectable and interpretable by a suitably-configured meter to represent the SID value and possibly other information such as a watermark timestamp.
For example, each instance of the watermark could be encoded into the audio media as a sequence of values over a respective watermark time duration, by adding into the audio media a sequence of symbols defining tone-combinations that correspond with the sequence of values. In particular, each value in the sequence of values could be selected from a predefined set of values, and each value in the predefined set of values could have predefined correspondence with a respective combination of tones or frequency components different than each other value in the set. The process of encoding each successive value of the watermark's sequence of values of into the audio media could thus involve adding into the audio media a respective symbol in the form of that value's corresponding tone-combination.
As a result, the watermark as encoded in the audio media could span the watermark time duration divided into a sequence of symbol time segments, encoding a respective identifier value by including a sequence of symbols respectively in the sequence of symbol time segments, each symbol having a respective tone combination that corresponds with a respective symbol value.
The sequence of symbols that therefore make up a representative instance of the watermark in the audio media could include one or more synchronization symbols that help demarcate or define the structure of the watermark, as well as one or more payload symbols, such as symbols representing digits of the SID and symbols representing digits of an encoding timestamp. The specific configuration of the watermark could take any of a variety of forms.
Without limitation,
In this example watermark, symbol time segments S1 and S5 could respectively encode values that function as synchronization markers, illustrated in the figure as values X and Y. Further, symbol time segments S2, S3, and S4 could respectively encode the first three digits of the 4-digit SID, and symbol time segments S6, S7, and S8 could respectively encode values that are offset respectively from the first three digits of the SID by an offset value equal to the fourth digit of the SID, thus representing the fourth digit of the SID by that offset while also functioning to facilitate error correction. Still further, the remaining four symbol time segments of the watermark could respectively encode the example four digits of an encoding timestamp (e.g., indicating when the watermark was encoded into the audio media).
A meter, which may be associated with a panelist (i.e., a person who would consume media) and may be positioned in the panelist's household and/or carried as a mobile device by the panelist among other possibilities as noted above, may then be operate alone or in combination with one or more other devices or systems to detect such watermarks as an indication that the panelist was exposed to associated broadcast stations, and to log and report those findings.
For instance, the meter may include one or more microphones for receiving acoustic audio in the panelist's environment. Further, the meter may include a computing system having one or more processors for processing the received audio, to monitor for presence of such watermarks and for reading the SID from the detected watermark and logging and reporting that the panelist was exposed to the broadcast station having that SID.
Alternatively or additionally, a representative computing system could be distributed between the meter and a separate device such as a cloud server, with the meter functioning to evaluate one or more aspects of the received audio and to pass results of that evaluation along to the separate device, and with the separate device functioning to analyze the raw data to ascertain the SID and to log and report that the panelist was exposed to the broadcast station having that SID. For simplicity and without limitation, the following description will discuss a scenario where the meter conducts all of this processing and may report the results as measurement data to a cloud server or other system. It should be understood that numerous variations are possible as well.
To detect the presence of the example watermark in the audio received from the panelist's environment, the meter may analyze the received audio to determine what frequency components the audio contains over time and what the energy levels (e.g., signal to noise ratio (SNR)) of those frequency components are. For instance, the meter could digitize the received audio signal and compute a discrete Fourier transform in order to determine the frequency components respectively in each 0.4-second symbol segment of the audio.
In the example where each symbol time segment respectively encodes a value with a corresponding set of 10 tones, for instance, the meter could determine for each 0.4-second symbol segment the aggregate SNR respectively of each of various possible 10-tone combinations in that symbol time segment, and the meter could then select as the winning 10-tone combination the 10-tone combination that has the highest determined SNR. For each such symbol time segment, the meter could then designate as the winning symbol value, i.e., as the value represented by the symbol time segment, the value that corresponds with the selected winning 10-tone combination. Through this process, the meter may first detect the presence of synchronization symbol time segment S1 (e.g., a symbol having winning value X), and the meter may then evaluate succeeding 0.4-second segments of the audio in order to decode the four digits of the SID.
With the example 4.8-second watermark having a sequence of twelve 0.4-second symbol time segments as shown in
As shown in the example table of
Thus, in the example shown, the meter may first detect the presence of the synchronization symbol time segment S1 by determining that a 0.4-second symbol time segment has symbol value X, based on a determination that the strongest 10-tone combination in that symbol time segment is the 10-tone combination that corresponds with symbol value X. (Namely, the meter may compare its determined SNR values for all of the possible 10-tone combinations in that time segment, and thus for all of the possible symbol values, and may find that the symbol value X has the highest SNR (i.e., 86)—thus making symbol value X the winning symbol value for that symbol time segment.) The meter may therefore fill in the top row of the first column of the table with value “X”, which indicates that that 0.4-second segment of audio is synchronization symbol time segment S1 having value X.
In turn, the meter may conduct the same analysis for each subsequent 0.4-second segment of the audio, finding for each 0.4-second segment the strongest 10-tone combination and thus the winning symbol value, and filling in these winning symbol values in the first row of the table. As a result, the example table that the meter establishes for a given watermark may indicate in its first row the sequence of values that the meter determines to be represented by the watermark.
In line with the discussion above, this determined sequence of values could indicate in a coded manner the SID represented by the watermark. For instance, as discussed above, the symbols in time segments S2, S3, and S4 could represent the first three digits of the SID, and the symbols in time segments S6, S7, and S8 could establish the fourth digit of the SID by representing values that are offset from the first three digits of the SID by an offset value equal to the fourth digit of the SID. With this example, given the example values in the table of
With this or other examples, when the meter determines the SID represented by a watermark, the meter may determine and assign a quality level to the SID determination for that watermark, which could represent the meter's level of confidence in the SID determination. Further, the meter may include this determined quality level with its report of the determined SID value for that watermark.
This determined quality level per SID determination could optimally be used as a basis to weigh the significance of the meter's reported media-measurement data, which could help govern the extent to which a device or system would rely on the data as a basis to take actions such as controlling ad placement or the like. For instance, if the meter indicates that it is very confident in its SID determinations, then a media-measurement company, an advertiser, and/or one or more other entities may give more weight to those SID determinations in relying on the media-measurement data as a basis to control ad placement or to take other actions. Whereas, if the meter indicates that it has low confidence in its SID determinations, then such an entity may give less weight to those determinations, possibly not relying on them at all as a basis to control ad placement or to take other actions.
In an example implementation, for each watermark, the meter could gauge the quality level of the meter's SID determination based on an evaluation of apparent errors in the symbol values that the meter determined for the watermark. The meter may carry out this evaluation with respect to just the first eight symbol time segments of the watermark, among other possibilities. With those first eight symbol time segments, for instance, the meter may deem the SID determination to have a highest quality level of 8 if the meter identifies no apparent errors in the determined symbol values, and the meter may deduct a point from that quality level for each apparent error that the meter finds.
Errors that the meter may find in the determined symbol values for the watermark could take various forms. For instance, the errors could relate to apparent violations of predefined rules regarding expected watermark content or structure, among other possibilities.
By way of example, given that symbol time segment S5 of the example watermark should be a synchronization symbol having value X, if the meter finds that symbol time segment S5 has a value other than X, then the meter may deem that to be an error and therefore deduct a point from the quality level for the SID determination. As another example, if there is a rule that the first three digits of the SID need to be unique, and if the meter finds that two of those digits are the same as each other, then the meter may deem that to be one or two errors and therefore deduct one or two points from the quality level for the SID determination.
As still another example, given that the error-correction symbols (of segments S6, S7, or S8) in the example watermark should have values that are offset from first three SID digits by the same offset value as each other (representing the fourth SID digit), if the meter finds that a given one of the error-correction symbol values is offset from a corresponding one of the first three SID digits by a different offset value than the other two error-correction symbol values, the meter may likewise deem that to be one or two errors and therefore deduct one or two points from the quality level for the SID determination.
And as yet another example, if there is a rule that the offset of each error-correction symbol value from a corresponding SID digit needs to be non-zero and if the meter finds that one of the error-correction symbol values is offset by zero from a corresponding SID digit (i.e., is the same as the corresponding SID digit), then the meter may likewise deem that to be one or two errors and therefore detect one or two points from the quality level for the SID determination.
Through this process in the non-limiting example, if the meter finds no such errors in its decoding of a watermark, then the meter may assign and report a quality value of 8 to its SID determination for that watermark. Whereas, if the meter finds one or more errors in its decoding of a watermark, then the meter may deduct a corresponding number of points from the quality value of its SID determination for that watermark. In an example implementation, if the resulting quality level for a watermark is less than or equal to 6, then the meter's SID determination for that watermark may be deemed inconclusive; in other words, in that situation, the meter's decoding of that watermark may be deemed inconclusive. In that situation, the meter may forgo reporting such inconclusive results.
Various reasons may exist for these errors in watermark decoding. For instance, noise in the meter's environment may cause the meter to incorrectly read one of the SID digits as a value different than what was encoded in the watermark. This could happen if the noise reduces the SNR of one or more of the frequency components underlying the associated symbol, or if the noise adds one or more strong frequency components to the symbol time segment, among other possibilities.
In some media-measurement systems, this or other such degradation in watermark decoding from time to time may not be a concern, since the meter may roll up and report media exposure data with a temporal granularity that is much lower than the duration of any individual watermark or even of several watermarks in a row. For instance, some systems may call for reports of media exposure on a per minute basis. So in such a system, if the bulk of SID readings within a given minute are all the same SID, a meter could disregard aberrant readings within that minute.
Unfortunately, however, other (e.g.,. more recent) media-measurement systems may not be so tolerant, as they may require reporting of media exposure with much higher temporal granularity, on a sub-minute basis. For instance, some media-measurement systems may require that the meter log and report media-exposure data for every 6 seconds or every 10 seconds. Given that requirement, it may be unacceptable or commercially impractical for the meter to omit one or more SID readings in a row or even for the meter to provide one or more low-quality SID readings in a row.
The present disclosure provides a mechanism to help address this problem. In accordance with the disclosure, a meter that operates to decode and read SIDs from successive watermarks in audio media could engage in a predictive-error-correction (PEC) process to help address one or more instances of degraded watermark decoding, such as watermark decoding that produces low quality SID readings. The meter may carry out this PEC process through analysis of the results of its watermark-decoding efforts over time.
In an example implementation, the meter may carry out the PEC process to help improve the meter's SID determination for a given watermark, based on a comparison with the meter's SID determination for a relatively recent “verified watermark”.
To facilitate this in a non-limiting example, the meter may regularly evaluate its SID determinations to determine for each successive watermark whether to classify the watermark as a “verified watermark.” Characterizing a watermark as a “verified watermark” may establish that the SID that the meter decoded from that watermark is a very reliable identifier of media content in the panelist's environment. (This may therefore also amount to characterizing the SID represented by that watermark as a “verified SID”). Without limitation, the meter may deem a particular watermark to be a verified watermark responsive to (i) the meter having assigned to its SID determination for that particular watermark at least a predefined threshold high quality level, perhaps the highest possible quality level, e.g., 8, and (ii) the meter also having decoded the same SID value, also with at least the threshold high quality level, from another watermark within a first predefined threshold short timeframe, e.g., 12 hours or 24 hours, before the particular watermark.
When the meter determines that a particular watermark is a verified watermark, the meter may then proceed to apply the PEC process to each succeeding low-quality watermark that occurs with expected watermark timing within a second predefined short timeframe, such as 5, 7, 10, or 15 minutes, after the verified watermark, i.e., relatively soon after the verified watermark.
For instance, with the example watermark structure discussed above, once the meter determines that a particular watermark is a verified watermark, the meter may then identify as a “watermark at issue” each watermark that meets at least the following three criteria: (i) the meter assigned to the SID determination for the watermark at issue a threshold low quality level, perhaps anything lower than the highest possible quality level, among other possibilities, (ii) the watermark at issue occurs an integer multiple of the watermark period of 4.8 seconds after the verified watermark, and (iii) the watermark at issue occurs within the second predefined threshold short time frame after the verified watermark. The meter may thus carry out the PEC process as to each such watermark at issue, using the verified watermark as a benchmark for comparison.
(Note that, while the verified watermark in the example implementation would be present in the audio media threshold recently before the watermark at issue, it may also be possible to carry out this process with respect to a verified watermark that is present in the audio media threshold soon after the watermark at issue. This may work well as an alternative implementation, for instance, in a post-processing implementation, among other possibilities.)
The PEC process could involve programmatically evaluating each of one or more symbol time segments of the watermark at issue to determine whether that symbol time segment of the watermark at issue contains a threshold strong tone combination that matches the tone combination that the verified watermark has in the corresponding symbol time segment.
For instance, in carrying this process out as to symbol time segment S3 of the watermark at issue, the meter may (i) determine what the winning tone combination is in symbol time segment S3 of the verified watermark and then (ii) determine if that particular tone combination is threshold strong in the symbol time segment S3 of the watermark at issue. Likewise, in carrying out this process as to symbol time segment S5 of the watermark at issue, the meter may (i) determine what the winning tone combination is in symbol time segment S5 of the verified watermark and then (ii) determine if that particular tone combination is threshold strong in the symbol time segment S5 of the watermark at issue.
As to a given symbol time segment of the watermark at issue, the act of determining whether the winning tone combination of the corresponding symbol time segment of the verified watermark is threshold strong in the given symbol time segment of the watermark at issue could take various forms.
Without limitation, the question could be whether that tone combination is threshold strong compared with all of the other candidate tone combinations in the given symbol time segment of the watermark at issue. The meter could determine this, for instance, by (i) computing a ratio of the determined signal strength of that tone combination to an average of the signal strength of all of the other possible tone combinations in the given symbol time segment of the watermark at issue and (ii) determining whether that computed ratio is at least as strong as a predefined threshold level. This is effectively a SNR analysis, evaluating whether the tone combination at issue is strong enough compared with other tone combinations in the given symbol time segment of the watermark at issue. If the meter thereby determines that the winning tone combination of the corresponding symbol time segment of the verified watermark is threshold strong compared with all of the other candidate tone combinations in the given time segment of the watermark at issue, then the meter may conclude that that tone combination is threshold strong in the given symbol time segment of the watermark at issue.
The meter could carry out this analysis for each symbol time segment of a set of one or more symbol time segments of the watermark at issue. For instance, the meter could carry out this analysis for each of the symbol time segments that the meter deemed to have erroneous or potentially erroneous symbol-value determinations in the watermark at issue, responsive to those error determinations. Alternatively, the meter could carry out this analysis for all of the symbol time segments of watermark at issue. Other examples may be possible as well.
Based on this analysis, the meter may determine whether the answer is affirmative for all of the analyzed symbol time segments of the watermark at issue, i.e., whether it is the case that, in each analyzed symbol time segment of the watermark at issue, the winning tone combination of the corresponding symbol time segment of the verified watermark is threshold strong.
If so, then, based at least on that determination, the meter may reasonably conclude that the SID represented by the watermark at issue is probably the same as the SID determined for the verified watermark. In that case, the meter may correct (or, in some cases, confirm) its SID determination for the watermark at issue to be the SID that it determined for the verified watermark. Further, in that case, the meter may also set the quality level for that SID determination to be a predefined quality level, such as 7 (i.e., in the example, one less than the highest level), among other possibilities.
Whereas, if not (i.e., if the meter does not determine that the answer is affirmative for all of the analyzed symbol time segments of the watermark wat issue), then the meter may maintain whatever quality level it had set for that SID determination and proceed as it would otherwise in view of on that SID determination. Alternatively, if the meter can partially correct the SID determination through this process, the meter may increase the quality level of the SID determination to some lesser extent.
The identifier in this method could be a SID or any other type of identifier, possibly one that is useable as a basis to identify the media content in a media-measurement system, i.e., a media-ratings system, or for that matter could be any data that may be usefully encoded in a watermark and that represents and thus indicates or identifies information. Further, this method could be carried out by a computing system which, as noted above, may be provided in a meter configured to receive and analyze the media content and/or may be provided in a separate device or system such as a cloud server or other entity that my interwork with such a meter.
As shown in
Further, at block 302, the method includes, the computing system assigning a quality level to the given watermark, including (i) classifying as erroneous, based on one or more rules regarding watermark structure, one or more of the determined candidate symbol values and (ii) assigning the quality level based on how many of the determined candidate symbol values are classified as erroneous.
In addition, at block 304, the method includes, responsive to at least the assigned quality level being threshold low, the computing system engaging in an error-correction process including (i) making a determination that at least each symbol time segment in the given watermark whose candidate symbol value is classified as erroneous has a threshold strong tone combination matching the tone combination that represents the symbol value of a corresponding symbol time segment in a verified watermark and (ii) responsive to the determination, correcting the determined identifier value of the given watermark to be the identifier value of the verified watermark. To facilitate this operation, the computing system may determine that the quality level assigned to the given watermark is threshold low, e.g., that it is at least as low as a predefined threshold level.
In line with the discussion above, the act of making the determination that a given symbol time segment in the given watermark has a threshold strong tone combination matching the tone combination that represents the symbol value of the corresponding symbol time segment in the verified watermark could involve (a) determining the tone combination of the corresponding symbol time segment in the verified watermark and (b) determining that the same determined tone combination in the given symbol time segment of the given watermark is threshold strong.
Further, as discussed above, the act of determining that the same determined tone combination in the symbol time segment of the given watermark is threshold strong could involve (i) computing a ratio of strength of the determined tone combination in the given symbol to an average strength of symbol-value time segment of the given watermark and (ii) deeming the determined tone combination to be threshold strong responsive to the computed ratio being at least as high as a predefined threshold level.
As additionally discussed above, the act of making the determination that at least each symbol time segment in the given watermark whose candidate symbol value is classified as erroneous has a threshold strong tone combination matching the tone combination that represents the symbol value of a corresponding symbol time segment in the verified watermark could involve making the determination as to each of the symbol time segments in the sequence of symbol time segments of the given watermark, including but not limited to each symbol time segment whose candidate symbol value is classified as erroneous.
Still further, as discussed above, the method could additionally involve the computing system identifying a particular watermark as the verified watermark, as a basis to engage in the error-correction process. Here, for instance, if the particular watermark encodes a particular identifier value, the act of identifying the particular watermark as the verified watermark could be based on (i) the particular watermark being assigned a quality level at least as high as a threshold high quality level and (ii) another threshold-recent watermark in the sequence of watermarks encoding the same particular identifier value and also being assigned a quality level at least as high as the threshold high quality level. The threshold-recent watermark in this arrangement could be a watermark that was present within the media content within a predefined period before the given watermark, wherein the predefined period is between 12 and 24 hours.
As further discussed above, the method could additionally involve the computing system determining that the verified watermark is useable as a basis for the error-correction process as to the given watermark. For instance, this could include determining that the given watermark is present in the media content both (i) within a predefined threshold short time frame after the verified watermark is present in the media content, such as a time frame that is no more than 15 minutes, and (ii) an integer multiple of the watermark duration after the verified watermark in the media content. The act of engaging in the error-correction process could then be additionally responsive to the determining that the verified watermark is useable as a basis for the error-correction process as to the given watermark.
As shown in
The one or more audio-input modules 400 may comprise one or more microphones and/or other audio input mechanisms configured to receive acoustic audio from a panelist's environment and to provide associated one or more associated audio feeds, e.g., digitized audio feeds, for processing.
The at least one processor 402 may comprise one or more general purpose processing units (e.g., microprocessors) and/or one or more specialized processing units (e.g., digital signal processors, dedicated audio processors, dedicated watermark processors, etc.) Further, the at least one non-transitory data storage 404 may comprise one or more volatile and/or non-volatile storage components (e.g., flash, optical, magnetic, ROM, RAM, EPROM, EEPROM, etc.), which may be integrated in whole or in part with the one or more processors 402. As further shown, the at least one non-transitory data storage may store program instructions 410, which may be executable by the at least one processor 402 to carry out various computing-system operations described herein, such as those illustrated in
The at least one network interface 208 may comprise one or more interfaces to facilitate wired and/or wireless communication between possibly one or more components of the computing system 100 and/or between the computing system 100 and one or more other entities, such as to facilitate internet communication, among other possibilities. For example, a network interface in a meter may enable the meter to report to a cloud server or other entity various information that the meter extracts from received audio signals, among other possibilities.
The present disclosure also contemplates at least one non-transitory computer readable medium that is encoded with, stores, or otherwise embodies program instructions executable by at least one processor to carry out various operations as described above.
Exemplary embodiments have been described above. Those skilled in the art will understand, however, that changes and modifications may be made to these embodiments without departing from the true scope and spirit of the invention.
