Quantization index modulation (qim) digital watermarking of multimedia signals

Abstract

The invention addresses the problem of scale degradations that may occur in watermarking schemes based on quantization index modulation (QIM). In accordance with the invention, the quantization step size (D) being employed by the embedder (11) and detector (21) is derived (12, 22) from a measurable characteristic parameter which has the property that, when the applied signal is scaled by a factor (a), it is scaled by substantially the same factor. In a preferred embodiment, said parameter is the square root of the energy of the signal (), and the quantization step is a predetermined fraction (a) thereof.

Description

FIELD OF THE INVENTION

[0001] The invention relates to a method and arrangement for embedding auxiliary information into a media signal by subjecting signal components of said media signal to quantization index modulation. The invention also relates to a method and. arrangement for retrieving thus embedded data from a watermarked media signal.

BACKGROUND OF THE INVENTION

[0002] Digital watermarking is the art of embedding auxiliary information in audio-visual objects. Digital watermarking has a large number of applications among which copy(right) protection, royalty tracking, commercial verification, added value content, interactive toys and many more. The classical approach to digital watermarking is basically noise addition, whereby adding a known noise-like signal w modifies an original signal s. Watermark detection is essentially correlation, where the resulting correlation value consists of two components, viz. the welted term <s,s> and an interference term <s,w>. This latter interference term is the main reason that noise addition is, at least theoretically, a less than optimal method for watermarking.

[0003] Recent publications have shown that, assuming certain attack models, optimal watermarking can be achieved by quantization. In essence, quantization watermarking amounts to the following. In the space S of host signals s, N sets of code points Cn are chosen, where N is equal to the number of messages to be embedded (the payload of the watermark). A message m is embedded in a host signal s by modifying the host signal into a signal s, such that s and s are (perceptually) close and such that s is closer to a point in Cm than to any other point in any of the other code sets Cn, n different from m. This type of watermarking is usually referred to as Quantization Index Modulation or QIM. The distance between the points of the code sets is referred to as grid parameter or quantization step.

[0004] Decoding a watermark amounts to finding the closest points c in the union of code point sets, and deciding upon the message m if and only if the point c is member of the code set Cm.

[0005] A problem of the QIM watermarking scheme is that the grid parameter needs to be known at the detector side. Knowledge of the quantization step is however not guaranteed in many practical examples. In theoretical publications relating to QIM, it is generally assumed that the detector uses the same quantization step as the embedder. However, this leads to incorrect results if the watermarked signal has been subjected to degradations such as scaling.

OBJECT AND SUMMARY OF THE INVENTION

[0006] It is an object of the invention to provide a method and arrangement for embedding data in a media host signal which renders it possible to correctly retrieve the embedded data from a scaled watermarked signal. It is a further object of the invention to provide a corresponding method and arrangement for retrieving the embedded data.

[0007] To this end, the embedding method is characterized in that it comprises the step of deriving the quantization step from a measurable characteristic parameter of the host signal. The corresponding detecting method comprises the step of deriving the quantization step from the same measurable characteristic parameter of the watermarked signal. It is thereby achieved that substantially the same relative quantization step is used at both ends.

[0008] Preferably, scaling of the signal commutes with scaling of the quantization step. That is, if the signal is scaled by a certain scaling factor, then the characteristic parameter is scaled by the same scaling factor. An advantageous example of such characteristic parameter is the square root of the energy of the signal. The quantization step is preferably controlled to be proportional thereto.

[0009] Further advantageous embodiments of the embedding and detection methods and arrangements are defined in the sub claims and will be described hereafter by way of example.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG. 1 shows a general schematic diagram of a system comprising a watermark embedder and a detector according to the invention.

[0011] FIGS. 2 and 3 show diagrams to illustrate the operation of the system which is shown in FIG. 1.

[0012] FIGS. 4-7 show diagrams to illustrate the operation of a preferred embodiment of the system according to the invention.

[0013] FIG. 8 shows a schematic diagram of a preferred embodiment of a watermark embedder according to the invention.

[0014] FIG. 9 shows a schematic diagram of a preferred embodiment of a watermark detector according to the invention.

DESCRIPTION OF EMBODIMENTS

[0015] FIG. 1 shows a general schematic diagram of a system comprising a watermark embedder (or encoder) 1 and a detector (or decoder) 2 according to the invention. The watermark encoder embeds a watermark message b in a host signal s such that the distortion between the host signal s and the watermarked signal s is negligible. The decoder 2 must be able to detect the watermark message from the received signal s. FIG. 1 shoves a “blind” watermarking scheme. This means that the host signal s is not available to the decoder 2.

[0016] In practice, the watermarked signal has undergone signal processing, passed through a communication channel, and/or has been subject of an attack. This is shown in FIG. 1 as a channel 3 between embedder 1 and detector 2. The channel scales the amplitude of the watermarked signal s with a factor a (usually a<1). The channel may also add noise, and/or introduce an additional offset (not shown).

[0017] The watermark encoder 1 and decoder 2 involve a “codebook” that is available at both ends. In the encoder 1, the code book maps an input sample sj onto an output sample sj, the output sample value being dependent on the message symbol bj. The decoder 2 uses the same codebook to reconstruct the message symbol bj from the sample sj.

[0018] The QIM encoding/decoding principle is mostly easily understood by considering scalar quantization of signal sample values. To this end we choose a quantization step D, and construct two code sets C0 and C1 as follows: the set C0 consists of all the even multiples of D, and the set C1 consists of all the odd multiples of D. In its simplest form, watermarking a length K signal s=(s1, . . . , sK) with a bit string b=(b1, . . . , bK) is achieved by for each j rounding sj to the nearest even multiple of D for bj=0, and rounding it to the nearest odd multiple of D for bj=1. The bit string b can be recovered by rounding the components of s to the grid spanned by D, and concluding a 0 bit for every even multiple of D and a 1 bit for every odd multiple of D.

[0019] The codebook used by the watermark encoder and decoder is preferably randomized, dependent on a secure key to achieve secrecy of watermark communication. To this end, the values of s are dithered by using for each sample index j a secret dither value vj. The dither values vj are preferably real numbers. This prevents the samples sj from always lying on the grid spanned by D, so that an observer cannot even “see” that the signal has been watermarked. This watermark embedding scheme is illustrated in FIG. 2. The signal media samples sj on the left vertical axis, an example of which is denoted X, are rounded to the nearest even (bj=0) or odd (bj=1) multiple of D, and provided with an offset vj. In FIG. 2, the quantization step size is D=1.5 and the dither value for the particular signal sample is vj=−0.5. Hereinafter, the dither value vj will be expressed as the product of a multiplication factor vj and the quantization step size D: vj=νj×D, where−1<νj<1.

[0020] A mathematical expression of this embedding process referred to as “dithered uniform scalar quantization” can be derived as follows. The discrete levels that an output sample sj can assume for a given offset vj is:

s j =(2m+bj)×D+vj×D, where m=. . . , −2,−1,0,1,2, . . .   (1)

[0021] The output value sj must be as close as possible to the input value sj. This can be expressed as:

s j ≈s j

s j ≈(2m+bj)×D+vj×D

[0022] 1 $$ m ≅ s j - ( 2 ⁢ m + b j ) × D 2 ⁢ D

[0023] The latter condition is fulfilled if 2$\begin{array}{cc}m=\mathrm{round}\left\{\frac{s_j-\left(2m+b_j\right)\times D}{2D}\right\}& \left(2\right)\end{array}$

[0024] Substitution of (2) in (1) yields: 3$\begin{array}{cc}{\underset{\_}{s}}_j=\left[2\times \mathrm{round}\left\{\frac{\frac{s_j}{D}-b_j-v_j}{2}\right\}+v_j+b_j\right]\times D& \left(3\right)\end{array}$

[0025] This formula has the following interpretation. Firstly, for the sample value sj, we compute the “quantization index” sj/D. Secondly, we round this quantization index to a shifted version of integers. It is easy to see that for bj=0 or bj=1 the modulated indices lie on two distinct subsets. Finally we multiply by D to restore the original scale of the sample value s. One easily sees that the maximal distortion for a sample value is equal to D.

[0026] The purpose of the dither sequence is twofold. Firstly, it provides security: estimating the step size D has become a difficult task, as a simple analysis of tie sample value histogram is no longer sufficient. And even if D is known, estimating bj from bj+vj is impossible without knowledge of vj. Secondly, it provides robustness: if the dither sequence is sufficiently random, the signal s can be modeled as s plus a noise term. For the topics of the next sections, this allows a better approximation of D from s than if s were given a more constant offset. Equation (3) makes immediately clear why scale degradations are a problem for QIM. A scaled version axs of s leads to a quantization index axs/D, i.e. a times the original quantization index s/D. Given the detection formula in Equation (4) below, it is obvious that it is no longer possible to reliably retrieve the embedded information.

[0027] For completeness of disclosure of this invention, FIG. 3 illustrates the operation of an even more general embodiment of the QIM watermark embedding process. In this embodiment, ternary embedding (bj=0, 1, or2) is employed. Moreover, embedded symbols are not represented by discrete points of the s-axis, but by distinct ranges of values sj. It can easily be derived from this Figure that the output signal sj can now be described as:

s j =s j +λ(zj−sj)

[0028] where zj denotes the discrete points as defined above by Equation (3).

[0029] Detection of embedded information is simply a matter of computing of the quantization index, compensating for the dither and checking the parity of the result. For the binary embedding scheme, this is concisely expressed as follows: 4$\begin{array}{cc}{\underset{\_}{b}}_j=\mathrm{mod}\left(\mathrm{round}\left\{\frac{{\underset{\_}{s}}_j}{D}\right\}+v_j,2\right),& \left(4\right)\end{array}$

[0030] where bj denotes the estimated bit value.

[0031] Equation (4) indicates that the watermark decoder needs a few essential parameter values before the watermark payload can lie retrieved. Firstly it needs the dither sequence v=(v1, . . . ,vK) in order to make the correct interpretation as a 0 or 1 bit. In any practical system, where alignment of dither string and signal is usually not guaranteed, this immediately implies a synchronization problem. Secondly, and more importantly, the grid parameter D needs to be known at the detector side. Knowledge of D is however not guaranteed in many practical examples. A typical example of gain degradations can be found in watermarking of audio, where s is chosen to represent waveform sample values. A gain degradation in this case amounts to turning up or down the volume. If the gain factor is within limits, usually this will not be experienced as perceptual degradation. In particular, detecting a watermark over the air (playing out from a playback device, receiving over the air with a recording device) will involve gain degradations.

[0032] The problem this invention addresses is the retrieval of the quantization step size D from a received signal s without explicit knowledge of the gain factor a This problem has received little attention so far in literature. The key idea is to male the step size D dependent on the host signal s, in such a way that if s is scaled in amplitude by a factor a, the estimated step size D(axs) is also scaled by a factor a to axD(s). In other words, scaling of s commutes with scaling of D.

[0033] The quantization step D is, for example, a chosen fraction of the Lp-norm of the signal s (or a chosen fraction of individual signal samples sj), where p=1 or p=2. 5${\&LeftDoubleBracketingBar;s\&RightDoubleBracketingBar;}_p={\left\{\sum _j^{}{\&LeftBracketingBar;s_j\&RightBracketingBar;}^p\right\}}^{\frac{1}{p}}$

[0034] In a preferred embodiment, the quantization step size is proportional to the square root of the energy of the signal (i.e. p=2).

[0035] Returning to FIG. 1, it is shown that the watermark embedder 1 comprises a quantization index modulation (QIM) circuit 11 operating in accordance with Equation (3). The embedder receives the quantization step size D to be used for the QIM process from a quantization step controlling circuit 12 operating in accordance with:

D=α{square root}{square root over (E(s) )}

[0036] where E(s) is the energy of the host signal s and α is a predetermined factor.

[0037] Similarly, the watermark detector 2 comprises a demodulation (QIM−1) circuit 31 operating in accordance with Equation (4). The detector receives the quantization step size D′ to be used from a quantization step controlling circuit 12 operating in accordance with:

D′=α{square root}{square root over (E(s)}′)

[0038] where E(s′) is the energy of the received signal s and α is the same factor as used by the embedder 1.

[0039] The quantization index modulation may be applied to all signal samples in the original signal domain (audio waveform samples, video pixels) or selected ones of said signal samples. The quantization index modulation may also be applied to components of the signal in some transform domain, for example, DCT coefficients of video images or spectral frequency components of an audio signal.

[0040] A preferred embodiment will now be described in more details. In this example, the signal s is an audio clip. FIG. 4 shows a waveform of such an audio signal in the time domain. As is well recognized, audio is best represented in the frequency domain, and we therefore apply to a frequency representation using a Fast Fourier Transform (FFT). FIG. 5 shows the frequency spectrum of the audio signal.

[0041] In this preferred embodiment, the watermark embedding is done by modulating the amplitude of the power spectral components. The spectral components are modified by quantization. Determining the quantization step size on the basis of the total spectral energy has a few disadvantages. Firstly, it implies that every spectral component is quantized with the same step size D. This is perceptually not an optimal strategy, because the allowed distortion per spectral component is linear with the magnitude of the component. Basing the step size on the total energy implies that large components (typically the lower frequencies) will be too finely quantized, and the small components (typically the higher frequencies) will be too coarsely quantized. Secondly, gain degradation is very often not uniform over the frequency range, leading to a mismatch per component between the estimated gain factor aid the real gain factor. To overcome these problems, the spectrum is subdivided into several bands and the factor α is determined per band on the basis of the energy per band. In accordance with psycho-acoustical models, these bands are chosen to grow logarithmically with frequency. For each band, a fixed fraction of the square root of the mean power (RMSE) is chosen as the threshold (i.e. quantization step) for that band.

[0042] FIG. 6 shows an example of a logarithmic division in bands (dotted vertical grid 61). Reference numeral 63 denotes the host power spectral density. Reference numeral 62 denotes the resulting RMSE from which the quantization step size D (see FIG. 1) in the respective sub-bands is being derived, Note that the RMSE is a reasonable approximation of the host power spectral density for higher frequencies, but that for lower frequencies errors are larger.

[0043] FIG. 7 shows an enlarged portion of FIG. 6. The dashed line 73 in this Figure denotes the power spectral density of the embedded signal after embedding with a fractional factor α=0.1. It can be seen that the difference between original signal 63 and watermarked signal 73 is minimal. The dashed tine 72 denotes the resulting RMSE from which the quantization step size D′ (see FIG. 1) in the sub-bands is derived. It is evident that the original step sizes 62 and re-estimated step sizes 72 only differ marginally. Note that for an appropriate comparison, no gain degradation (a=1 in FIG. 1) is assumed in this example.

[0044] FIG. 8 shows a block diagram of the preferred embodiment of the embedding arrangement, which operates as described above. The arrangement includes a circuit 81 for segmenting the audio signal in time frames, a, fast Fourier transform circuit 82, and a circuit 83 for separating each Fourier coefficient into its phase and magnitude. The magnitudes constitute the host signal s for the embedding circuit 1 (cf. FIG. 1). The modified magnitudes s and the corresponding phases φ are subsequently merged (84) and inverse Fourier transformed (85). The time frames are finally concatenated (86) to form the watermarked audio signal.

[0045] FIG. 9 shows a block diagram of the corresponding detection arrangement. The arrangement comprises the same segmentation circuit 81, Fourier transform circuit 82, and separating circuit 83 as shown in FIG. 8. The magnitudes of the Fourier coefficients constitute the watermarked signal component s for the detector 2 (cf. FIG. 1).

[0046] It is to be noted that the proposed band based computation allows many variations. One that needs to be mentioned is the sliding & expanding average, which is a ‘continuous’ version of the band based average: for every frequency component fn, the quantization step size is computed on the basis of the energy in the frequency interval [n/factor, . . . ,factor×n]. This formula says that the averaging interval grows linearly with the frequency index n, which is in accordance with the idea of logarithmic frequency bands.

[0047] It is also to be noted that similar techniques can be used for image and video watermarking. In case of image watermarking, a natural option would be to quantize spatial sample values, where the quantization step sizes are based on some local statistical moments.

[0048] The invention can be summarized as follows. The problem of scale degradations is addressed that may occur in watermarking schemes based on quantization index modulation (QIM). In accordance with the invention, the quantization step size (D) being employed by the embedder (11) and detector (21) is derived (12, 22) from a measurable characteristic parameter which has the property that, when the applied signal is scaled by a factor (a), it is scaled by substantially the same factor. In a preferred embodiment, said parameter is the square root of the energy of the signal (·{square root over ((E(s))}), and the quantization step is a predetermined fraction (α) thereof.

Claims

1. A method of embedding auxiliary information into a media signal by subjecting signal components of said media signal to quantization index modulation employing a quantization step, characterized in that the method comprises the step of deriving said quantization step from a measurable characteristic parameter of said signal components.

2. A method as claimed in claim 1, wherein said deriving step includes controlling the quantization step to be proportional to said characteristic parameter, said characteristic parameter having the property of being scaled by a scaling factor when said media signal components are scaled by the same scaling factor.

3. A method as claimed in claim 1, wherein said characteristic parameter is the Lp-norm of the signal components.

4. A method as claimed in claim 1, wherein said characteristic parameter is the square root of the energy of said signal components.

5. A method as claimed in claim 1, wherein said signal components are individual samples of the media signal.

6. A method as claimed in claim 1, wherein said signal components are spectral frequency components of said media signal.

7. A method as claimed in claim 6, wherein said characteristic parameter is the square root of the energy of said spectral frequency components in respective frequency sub-bands.

8. A method as claimed in claim 7, wherein said sub-bands are logarithmically spaced sub-bands.

9. An arrangement for embedding auxiliary information into a media signal by subjecting signal components of said media signal to quantization index modulation employing a quantization step, characterized in that the arrangement comprises means for deriving said quantization step from a measurable characteristic parameter of said signal components.

10. A method of retrieving data being embedded in a watermarked media signal by subjecting signal components of a media host signal to quantization index modulation, the method comprising the step of employing a quantization step to retrieve the embedded data, characterized in that the method comprises the step of deriving said quantization step from a measurable characteristic parameter of said watermarked signal components.

11. A method as claimed in claim 10, wherein said deriving step includes controlling the quantization step to be proportional to said characteristic parameter, said characteristic parameter having the property of being scaled by a scaling factor when said media signal components are scaled by the same scaling factor.

12. A method as claimed in claim 10, wherein said characteristic parameter is the Lp-norm of the signal components.

13. A method as claimed in claim 10, wherein said characteristic parameter is the square root of the energy of said signal components.

14. A method as claimed in claim 10, wherein said signal components are individual samples of the media signal.

15. A method as claimed in claim 10, wherein said signal components are spectral frequency components of said media signal.

16. A method as claimed in claim 15, wherein said characteristic parameter is the square root of the energy of said spectral frequency components in respective frequency sub-bands.

17. A method as claimed in claim 16, wherein said sub-bands are logarithmically spaced sub-bands.

18. An arrangement for retrieving data being embedded in a watermarked media signal by subjecting signal components of a media host signal to quantization index modulation, the arrangement being arranged to retrieve the embedded data by employing a quantization step, characterized in that the arrangement comprises means for deriving said quantization step from a measurable characteristic parameter of said watermarked signal components.