The present disclosure generally pertains to the field of video recording and video playback, in particular to apparatus, systems and methods for encoding and decoding a watermark signal within a video image.
The content of a traditional billboard is determined by the content shown at recording time. The more the playback time is away from the recording time, the more outdated the billboard content will appear to a viewer. Additionally, traditional billboards offer no real personalization which make them inefficient.
Methods and systems are known which analyze video content to identify objects within the video content, and which replace selected identified objects by other objects. For example, it is known to identify blank segments or other objects within video scenes and replace the blank segments by advertisements.
However, identifying objects suitable for placing advertisements within a video scene may require image recognition means and may be thus be a technical complex task.
Thus, although there exist techniques for determining areas in a video signal for inserting new content within the video signal, it is generally desirable to provide alternative apparatus, systems and methods for determining areas in a video signal for inserting new content within the video signal.
According to a first aspect the disclosure provides an apparatus comprising a watermark decoder arranged for decoding a watermark signal from a transmission video signal, the watermark signal defining a content area within the transmission video signal for inserting new content within the transmission video signal.
According to a second aspect the disclosure provides an apparatus comprising a watermark encoder arranged for encoding a watermark signal into a content video signal to obtain a display video signal, the watermark signal defining a content area within the content video signal for inserting new content into the content video signal
According to a third aspect the disclosure provides a method comprising decoding a watermark signal from a transmission video signal, the watermark signal defining a content area within the transmission video signal for inserting new content within the transmission video signal.
According to a fourth aspect the disclosure provides a method comprising encoding a watermark signal into a content video signal to obtain a display video signal, the watermark signal defining a content area within the content video signal for inserting new content into the content video signal.
Further aspects are set forth in the dependent claims, the following description and the drawings.
Embodiments are explained by way of example with respect to the accompanying drawings, in which:
In the following embodiments it is described an apparatus comprising a watermark decoder arranged for decoding a watermark signal from a transmission video signal, the watermark signal defining a content area within the transmission video signal for inserting new content within the transmission video signal.
The apparatus may be any device capable of playing video, such as a television device, a DVD player, a recording device, a multimedia station, a personal computer, a smartphone, a tablet computer, a smart watch, a camera device, augmented reality products such as head mounted displays, or the like.
The watermark decoder may be implemented as software designed for running on a processor. Alternatively, the watermark decoder may be directly implemented as hardware.
The transmission video signal may be transmitted by a broadcasting station, e.g. via a digital television broadcast station, or via the Internet. Alternatively, the transmission video signal may be a video signal that is read from a video storage medium such as DVD or a harddisk device.
A transmission video signal may comprise a series of video frames, each video frame being one of many still images which compose a complete moving picture.
The content area defined by the watermark signal may be any part or section of a video frame.
The content area defined by the watermark signal may for example indicate an object within the transmission video signal. If an object is moving, the content area may change with every video frame according to the position of the moving object.
The content area defined by the watermark signal may for example indicate the area in a video frame that is occupied by a billboard represented in the video content.
The content area defined by the watermark signal may be a marked area inside a larger unmarked video where the marked area can be arbitrarily oriented in 3D. The watermark may thus be arranged in such a way that it allows orientation of the content area.
In cases where the content area can be defined by the boundary of the area, the watermark signal may define the content area by defining the boundary of the content area.
The watermark signal may be invisible or substantially invisible for a user watching the transmitted video signal.
The watermark signal may be arranged in such a way that it cannot be removed from the video signal by accident.
The watermark signal may also comprise steganographic watermarks. This may prevent also intentional removal of the watermark signal from the video signal.
The apparatus may further comprise a content inserter arranged for inserting the new content at the content area defined by the watermark signal.
The new content may be obtained from a content provider which hosts a content database. To this end, the apparatus may be connected to the content provider via a network connection. The content database may for example be located in the Internet.
The new content may for example be images with a better quality (e.g. resolution). In particular, the new content may replace original content within the transmitted video signal.
The new content may for example be an advertisement. In this regard, the content area defined by the watermark signal can be considered as a virtual billboard.
The new content may be still images or animated images. Inserting animated images may for example be helpful in situations where (in the original location) of the original content animation is not possible or not adequate.
The new content may also be current advertisements which replace old advertisements.
The new content may also be advertisements which are localized to the location-of-view, e.g. using the local language at the location-of-view or the language setting of a display device.
Still further, the new content may be advertisements of a different company (e.g. for sub-letting adspace e.g. after a campaign was finished or as the technical platform for dynamic adspace selling).
The content inserter may be arranged in such a way that new content is only inserted if the owner of the original content and the owner of the new content have agreed to do so.
The content inserter may further be arranged in such a way that it allows orientation of the new content so that the new content fits to the content area indicated by the watermark signal.
The watermark signal may comprise content area meta information.
The watermark decoder may for example apply a visual light communication (VLC) demodulation scheme. Different modulation/demodulation schemes can be used to overlay the content video with a data signal layer. Such modulation/demodulation schemes are described in more detail below with regard to a watermark encoder which produces the watermark signal.
The content area meta information may for example be encoded according to a visual light communication (VLC) demodulation scheme as described above.
According to an embodiment, the content area meta information may be encoded as a QR code. The QR code can be read from the transmission video signal by demodulating the transmission video signal according to one of the demodulation schemes mentioned above.
The content area meta information may comprise information that allows to identify the owner of an advertisement or ad space.
Based on the content area meta information, the watermark encoder may thus allow to recognize billboards of an ad owner in a video image. A space of a billboard in the content video signal may thus be replaced by some other content of the ad owner.
Further, the content area meta information may bear static data.
Still further, the content area meta information may bear dynamic data. For example, the watermark may not be the same for every frame, but can also change in order to achieve a higher watermark data rate in time. This may be achieved by using a 3D barcode instead of a 2D barcode.
The apparatus may further comprise a content manager arranged for selecting the new content based on the content area meta information.
The content area meta information may be encoded by means of a 2D barcode scheme, for example as a QR code. QR encoding/decoding schemes are well known to the skilled person. For example, the QR coding scheme according ISO/IEC standard 18004:2006 (“QR-Code 2005”) may be used. Alternatively, 3D codes may be used in the same manner. A 3D barcode may be a series of 2D barcodes (for example a series of QR codes) which change in time. That is, a 3D barcode can for example be seen as a kind of “barcode movie” where every n-th frame of the QR-code is different. As already described above, the use of such 3D barcodes may increase the codes ability to bear dynamic data.
The content manager may further be arranged for selecting the new content based on user context information. The new content may thus be personalized advertisements using actual knowledge about the viewers stored in the user context information. The user context information may for example be stored in a user context database.
The user context database may be located within the playback device.
Alternatively, the user context database may be located at a central location, for example on a server of a content provider to which the apparatus is connected.
In the following embodiments it is also described an apparatus comprising a watermark encoder arranged for encoding a watermark signal into a content video signal to obtain a display video signal, the watermark signal defining a content area within the content video signal for inserting new content into the content video signal.
The display video signal may for example be displayed on a signage display. The signage display may for example be a billboard which is designed to display advertisements.
The display video signal, when displayed on a signage display, may produce a displayed video that can be captured by a camera device. For example, if during a sports event, advertisements are shown on a signage display, the watermark encoder can be arranged to insert a watermark signal in the content video signal transmitted to the signage display. The watermark signal may be invisible to the spectators watching the signage display. The spectators thus only see the original content video carried by the content video signal before insertion of the watermark signal. However, if a camera is pointed at the signage display, the camera may capture the displayed video which comprises both the original content video and the watermark signal. That is, the watermark signal, although not being visible (or being substantially invisible) to a user may nevertheless be captured by a camera device and may be retrieved by a watermark decoder as described above with regard to the apparatus comprising a watermark decoder.
The watermark encoder may for example apply a visual light communication (VLC) modulation scheme to generate the watermark signal and insert the watermark signal into a content video signal. Different modulation schemes can be used to overlay the content video with a data signal layer. A modulation may for example be done directly on the single video signal per pixel without the need of having a special LED hardware inside the signage display. Therefore, no special signage display is needed (although the apparatus disclosed here can be built inside such a display).
The modulation scheme can for example use blocks of pixels together to transport a single bit of data. As, according to some embodiments, the data rate is not very high, the distance of a camera to the signage display can be quite large.
The watermark signal may comprise content area meta information. The content area meta information may for example be encoded according to the modulation schemes described above.
According to an embodiment, the content area meta information is encoded by means of a 2D barcode scheme, for example as a QR code. The 2D barcode scheme code can then be modulated on the content video signal by one of the modulation schemes described above.
For example, the 2D barcode area can be mapped in its entirety to the area of the content video. 2D barcodes on top of the content video signal, although being only one implementation example, may result in good robustness characteristics.
In the following embodiments it is also described a method comprising decoding a watermark signal from a display video signal, the watermark signal defining a content area within the display video signal for inserting new content within the display video signal.
Still further, in the following embodiments it is also described a method comprising encoding a watermark signal into a content video signal to obtain a display video signal, the watermark signal defining a content area within the content video signal for inserting new content into the content video signal.
In these methods, the watermark signal may be used for marking areas which can be used as virtual billboards.
Alternatively or in addition, the watermark signal may be used for marking the areas in a picture where text or other media is contained that needs to be replaced e.g. for localization purposes.
Still alternatively or in addition, the watermark signal may be used for marking the boundaries of certain areas in a picture.
Still alternatively or in addition, the watermark signal may be used as a calibration signal.
Embodiments are now explained in more detail with regard to the Figures.
In such a video recording as described with regard to
In the embodiments described above, a use of the watermark encoding and the watermark decoding was described which relates to signage displays and virtual billboards. However, the watermark encoding and the watermark decoding may also be applied in other use cases:
The watermark signal may for example be used for marking the areas in a picture where text or other media is contained that needs to be replaced e.g. for localization purposes. This may be helpful in situations where “language versions” of a movie are to be produced by replacing the text appearing in the movie (e.g. the movie title) by different language versions. By marking the areas where such media occurs that needs to be replaced for a certain localized version of the movie, this process can be moved from manual editing that creates a new video signal for the entire movie to an automatic process done on the display device. The replacement content pieces may be kept in a content database as described above so that they are available electronically. These pieces even could be rendered on the display device given all the resources (in the text case, this would be the text in the target language, a font, and the background to use).
Still further, the watermark signal may for example be used for marking the boundaries of certain areas in a picture. In modern movie creating processes, sometimes object and other area boundaries are known from the production process, e.g. areas that have been recorded in reality vs. areas that have been computer-generated, moving vs. static areas, all the object boundaries for computer-generated movies, known distances of objects to the camera, etc. These boundaries can also be coded as data in the movie using the watermark encoding described in the embodiments above. If necessary, a higher data rate may be achieved using 3D codes. The encoded data can then be used e.g. by compression algorithms in order to compress a movie better or by picture enhancing algorithms in a display device to mitigate interlace effects.
Still further, the watermark signal may for example be used as a calibration signal. The watermark encoding can for example be used to enhance the overall picture by having the original ad displaying some pre-defined pattern that the image enhancement algorithm knows. In order to allow this, the data that is encoded in the picture may contain the content video signal (or at least one frame). A picture enhancement system now knows both a) the original content video signal, and b) the recorded display signal (minus the modulation). This information can then be used to enhance the entire recorded signal.
It should further be noted that this disclosure also envisages a system that is comprised of a watermark encoder such as disclosed with regard to
The methods as described herein are also implemented in some embodiments as a computer program causing a computer and/or a processor to perform the method, when being carried out on the computer and/or processor. In some embodiments, also a non-transitory computer-readable recording medium is provided that stores therein a computer program product, which, when executed by a processor, such as the processor described above, causes the methods described herein to be performed.
In so far as the embodiments of the disclosure described above are implemented, at least in part, using software-controlled data processing apparatus, it will be appreciated that a computer program providing such software control and a transmission, storage or other medium by which such a computer program is provided are envisaged as aspects of the present disclosure.
All units and entities described in this specification and claimed in the appended claims can, if not stated otherwise, be implemented as integrated circuit logic, for example on a chip, and functionality provided by such units and entities can, if not stated otherwise, be implemented by software.
It should he recognized that the embodiments describe methods with an exemplary ordering of method steps. The specific ordering of method steps is however given for illustrative purposes only and should not be construed as binding. For example the ordering of 111 and 112 in the embodiment of
It should also be recognized that the division of the user display into units 17, 19, 21, and 29 according to the embodiment of
It should also be noted that the location of the units given in the embodiments is not meant as binding. For example the user context database 29 is shown in
Likewise the content database 27 described in the embodiment of
Other modifications as such described above are with the scope of the appended claims.
Note that the present technology can also be configured as described below:
(1) An apparatus comprising a watermark decoder arranged for decoding a watermark signal from a transmission video signal, the watermark signal defining a content area within the transmission video signal for inserting new content within the transmission video signal.
(2) The apparatus of (1), further comprising a content inserter arranged for inserting the new content at the content area defined by the watermark signal.
(3) The apparatus of (1) or (2) wherein the watermark signal further comprises content area meta information.
(4) The apparatus of anyone of (1) to (3) in which the watermark decoder applies a visual light communication demodulation scheme.
(5) The apparatus of anyone of (1) to (4) further comprising a content manager arranged for selecting the new content based on the content area meta information.
(6) The apparatus of anyone of (3) to (5) wherein the content area meta information is encoded as a 2D barcode.
(7) The apparatus of anyone of (5) to (6) wherein the content manager is further arranged for selecting the new content based on user context information.
(8) An apparatus comprising a watermark encoder arranged for encoding a watermark signal into a content video signal to obtain a display video signal, the watermark signal defining a content area within the content video signal for inserting new content into the content video signal.
(9) The apparatus of (8) in which the watermark decoder applies a visual light communication demodulation scheme.
(10) The apparatus of anyone of (8) or (9) wherein the watermark signal comprises content area meta information.
(11) The apparatus of (10) wherein the content area meta information is encoded as a 2D barcode.
(12) The apparatus of anyone of (8) to (11) wherein the new content is an advertisement.
(13) A method comprising decoding a watermark signal from a transmission video signal, the watermark signal defining a content area within the transmission video signal for inserting new content within the transmission video signal.
(14) A method comprising encoding a watermark signal into a content video signal to obtain a display video signal, the watermark signal defining a content area within the content video signal for inserting new content into the content video signal.
(15) The method of (14), wherein the watermark signal is used:
The present application claims priority to European Patent Application 15155994.5 filed by the European Patent Office on 20 Feb. 2015, the entire contents of which being incorporated herein by reference.
Number | Date | Country | Kind |
---|---|---|---|
15155994.5 | Feb 2015 | EP | regional |