Patent Application
20150370875
The present principles relate to video registration in a context of video investigation. The proposed technique is more specifically linked to a system for automatic registration of video copies. According to the present document, it is reminded that video registration is a process of establishing a frame-by-frame correspondence between two videos, known as temporal registration, possibly complemented by establishing a point-by-point correspondence between the corresponding frames, known as spatial registration. Image registration is the process of establishing point-by-point correspondence between two images. Video alignment and image alignment are synonyms to, respectively, video and image registration.
Video registration is important because it allows matching an original content with an unauthorized content in order to extract, from the unauthorized content, one or several marks (watermarks). These marks have usually been inserted in the original content. When copying the original content to create the unauthorized content, the marks are preserved in the copied content.
The goal of video investigation when searching for proofs of counterfeit is to analyze unauthorized video contents to determine the origin of the leakage and more generally to extract any information useful for characterizing the “piracy workflow”. An example is the decoding of watermarks identifying a set-top-box in a VOD scenario, but there exists many other problems to solve. Most of these problems, however, share the pre-requisite of compensating for temporal and geometrical distortions of pirate copies of content, prior to any further analysis.
Indeed, one major problem faced by investigators is to synchronize a copy with an original content. This synchronization is needed in order to be able to extract, from this copy, a various number of marks or identification information which could be useful for identifying an origin of the copy.
Since a direct access to a master content is not always feasible (or even allowed by the content owners) at investigation side, a classical “semi-blind” solution is to compute and transmit condensed “fingerprints” (FP) of the content (a fingerprint is, in this context, a representative feature or characteristic extracted from the content, and not a watermark inserted in this content). This condensed “fingerprints” of the content comprises specific properties of temporal evolution and local key-points on key-frames of the content. The condensed “fingerprints” (FP) of the content is constructed by the original content owner.
The investigators then extract the same fingerprints from a copy and a registration module is able to estimate temporal and geometric distortion parameters from input master and copy fingerprints. Those distortion parameters can then be used by further applications (such as watermark decoding) to compensate for distortions prior to analysis.
An investigation framework with distributed master FP for registration purpose implicitly assumes that an original content is distributed in a unique spatial and temporal format, so that a pirate copy has to be aligned with this unique spatial and temporal grid. For a system dedicated to a single task, such as decoding the payload of a given watermark for VOD, this approach is valid as the watermark is embedded in content distributed with a unique given frame rate and resolution. Such a framework will only be able, however, to investigate pirate copies leaking from this VOD distribution. Such framework is described in “A framework for video forensics based on local and temporal fingerprints” (S. Baudry, B. Chupeau, F. Lefèbvre, Int. Conf. on Image Processing 2009).
But a more generic investigation framework has to be able to analyze pirate copies originating from multiple sources, such as DVD, Blu-ray, legacy cinema, digital cinema, etc. Each distribution source thus have different master temporal and spatial resolutions (e.g. Blu-ray: 1920×1080@24 Hz, 1280×720@59.94 Hz . . . , DVD: 720×576@25 Hz, 720×480@29.97 Hz . . . , Digital cinema: 4096×2160@48 Hz . . . ). Furthermore several versions of the same movie usually coexist, with different editing, for example scenes suppressed for censorship purposes or localization purposes.
A framework supporting the investigation (e.g. by means of watermark decoding) of pirate copies originating from different distribution sources (e.g. digital cinema, Blu-ray or the like) thus requires as many different registration processes as there are different “master” resolutions and versioning. And a straightforward application of the previously described framework would then need to generate and distribute as many different fingerprints to feed these registration processes! This is not realistic either in terms of computational load/transmission bandwidth or difficulty to access to different distribution sources (Blu-ray, iPhone, digital cinema, VOD . . . ).
Thus, the framework proposed in “A framework for video forensics based on local and temporal fingerprints” is not adapted to multiple distribution formats of the content. The problem to solve is therefore to find a solution to the plurality of distribution format while enabling investigating on a plurality of pirate copies coming from this plurality of distribution format and without needing several sets of master fingerprints.
The present principles overcome the drawbacks of the prior art.
The present principles relate, in a first aspect, to a method for creating at least one distribution content from at least one original master content. The comprises the following ordered steps:
Thus, the proposed technique allows obtaining a master fingerprints before obtaining the distribution contents. The distribution contents have different formats and the master fingerprint is independent of these formats.
According to a specific embodiment said step of calculating said master fingerprints comprises:
According to a specific embodiment said step of creating said at least one distribution content comprises, for a given distribution content, at least one step of recording, in one database, of one set of transformations of said at least one original master content for obtaining said given distribution content.
According to a specific embodiment, said step of recording one set of transformations of said at least one original master content for obtaining said given distribution content comprises:
The present principles also relates to a device for creating at least one distribution content from at least one original master content. The device comprises:
In a second aspect, the invention relates to a method for registering content. The method comprises:
Thus, the proposed technique allows registering content only once: it is not needed to register one content several times (i.e. one time per distribution format). The registration is done once and the derived distortion model is applied on the content. If it is then noticed that the derived distortion model is not correct, then it is only necessary to calculated a new derived distortion model with a correct set of transformations.
The present principles also relates to a device for registering one content. The device comprises:
In a preferred embodiment, the various steps of the methods according to the invention are implemented by one or more software or computer programs comprising software instructions for execution by a data processor or a module, according to the invention and adapted for controlling the execution of the different process steps.
Accordingly, the present principles also provides a non-transitory program storage device, readable by a computer, tangibly embodying a program of instructions, executable by a computer or a data processor, the program including instructions for controlling the execution of the steps of a method as mentioned above.
This program can use any programming language, and be in the form of source code, object code or intermediate code between source code and object code, such as a partially compiled form, or in any other desirable form.
The invention also provides a medium readable by a data processor, and containing instructions of a program as mentioned above.
The information carrier may be any entity or device capable of storing the program. For example, the medium may comprise a storage medium, such as a ROM, for example a CD ROM or a microelectronic circuit, or a magnetic recording medium, such as a diskette (floppy disk) or a hard drive.
On the other hand, the information carrier may be a transmissible carrier such as an electrical or optical signal which may be conveyed via electrical or optical cable, by radio or by other means. The program according to the invention may in particular be downloaded over a network such as the Internet.
Alternatively, the information carrier may be an integrated circuit in which the program is incorporated, the circuit being adapted to perform or to be used in carrying out the process in question.
According to one embodiment, the invention is implemented using software and/or hardware. In this context, the term “module” can correspond in this document as well as a software component to a hardware component or a set of hardware and software components.
A software component is one or more computer programs, one or more sub-programs of a program, or more generally to any element of a program or software capable to implement a function or set of functions, according to what is described below for the module. Such software component is executed by a processor of a physical entity (TV, projector, terminal, server, gateway, router, etc.) and is likely to access the hardware resources of the physical entity (memory, storage media, bus communication, e-cards I/O, user interfaces, etc.).
Similarly, a hardware component is any component of a hardware (or hardware) that can implement a function or set of functions, according to what is described below for the module. It may be a component integrated with programmable hardware or software for the execution processor, for example an integrated circuit, a smart card, a memory card, an electronic card for the execution of a firmware, etc.
Each component of the system described above provides its own software modules. The various embodiments described above can be combined together for the implementation of the invention.
The proposed method is described in the following by way of examples in connection with the accompanying figures without limiting the scope of the protection as defined by the claim. In the figures:
As explained, when investigating a copy of content (called the “copy” in the following), the problem is to identify a distribution content from which the copy is made. The distribution content is identified by searching watermarks which are integrated in the copy. These watermarks are for example inserted in specific sets of frames in the distribution content or in specific location in frames in the distribution content. For retrieving these watermarks in the copy, it is thus first needed to identify, in the copy, this specific frames or set of frames in which the watermarks have to be found. This identification is done by registering the copy with the distribution content. The registration is done by comparing the fingerprints of the distribution content with the fingerprints of the copy.
It is recalled that, in the meaning of the present document, a major difference has to be done between a “fingerprint” which is an inherent feature or characteristic of the content and the watermark which is a specific information introduced in the content for the purpose of the investigation. With prior art solution, the problem is that since there is a plurality of distribution contents (4 k, Blu-ray, DVD, etc.), it is currently needed to have multiple set of fingerprints (one set of fingerprints per distribution content). The proposed technique does not comprise these drawbacks.
Basically, the invention relates to a method for creating one distribution content from one original master content where the method comprises first obtaining a set of fingerprint for the original master content and then creating the distribution content, with saving, in a database, a set of transformation of the original master content. Then, when a copy content has to be registered, the invention proposes a method for registering the content where a distortion model is obtained by comparing the set of fingerprint of the original master content with a set of fingerprint of the copy content. The set of transformation is then combined with the distortion model for registering the copy content.
In its first aspect, the solution proposed by the inventors comprises the following steps, described in relation with
Thus, the solution found by the inventors is to use a unique source of fingerprints for a given content called the master fingerprints. The master fingerprints are obtained from an original content (which is called the original master content). The original master content is for example the content with the highest definition/resolution (4K or digital cinema content). For example, the choice of the master fingerprints which fits to the purpose of the proposed technique is done according to distortion considerations. This means that the fingerprints which are extracted from the original master content have to be robust in order to be identifiable into any copy (even of bad or very bad quality) of a distribution content (it is assumed that the distribution content from which a copy is done is itself derived from the original master content).
Following this initial phase, distribution contents are created. These distribution contents are created for fitting various distribution formats. These formats can vary from the original master content format by various features: resolution, length, quality of the sound, number of scene or the like.
According to the proposed technique, rather than simply creating the distribution content, a conversion parameter database is populated along with the creation of the distribution content. This database comprises, for a given original master content, all the necessary transformations made for obtaining the various distribution contents (e.g for obtaining a DVD, a Blu-ray, etc.). Typically, for one original master content and one specific distribution content, the database comprises the transformations applied: on the resolution, the format (i.e. 2.85 to 16/9, up-scalling or down-scalling), the length, the number of images per second (oversampling or downsampling), etc.
Thus, instead of having a plurality of set of fingerprints (one per content distribution format), the investigator have a single set of fingerprints (master fingerprints) and a plurality of set of transformations (one set of transformations per content distribution format).
Then, according to the proposed technique, there is no need to make several registrations for a given copy: a single registration is enough. This is an important advantage because the registration of content consumes lots of resources. Thus, the fact of processing only one real registration (i.e. the matching of the fingerprints of the fraudulent copy with the matching of the master fingerprints) highly reduces the resources necessary for obtaining a correct registered content. The decoding technique, i.e. the technique for retrieving the watermarks in a copy of one content is then widely facilitated because the registration of the copy is made on the basis of the master fingerprints. The registration is made in three steps, described in relation with
The derived distortion model (DTM′) is then applied to the copy for delivering a registered copy. The registered copy is then investigated by an investigation module. The investigation module is used for retrieving one or several marks which can be included in the copy.
A generic block scheme of an embodiment of the proposed technique is depicted in
DTM′=DTMoSTF
−1 (2)
In a preferred embodiment:
The structure of a device for the execution of the method for creating distribution contents is illustrated schematically in
For processing the proposed techniques, the device comprises:
These modules or circuits can be implemented in the form of specific modules, such as software modules or hardware modules. In case of hardware modules, an FPGA or the like can be implemented.
The structure of a device for the execution of the method for registering content is illustrated schematically in
For processing the proposed method, the device comprises:
These modules or circuits can be implemented in the form of specific modules, such as software modules or hardware modules. In case of hardware modules, an FPGA or the like can be implemented.
| Number | Date | Country | Kind |
|---|---|---|---|
| 14305974.9 | Jun 2014 | EP | regional |
