System for transferring and filtering video content data

Abstract

Mobile devices are becoming multimedia terminals which can send and receive video as well as audio and text mesges. A sender 11 can transfer content data 13 to a receiver 12. It is described how somebody could edit and adapt a video clip and send it by attaching a pre-defined filter signal 14 which adds suitable effects to the clip (colour adaptation, overlaying of icons, fade between en scenes, etc.). Instead of using the filter to edit the video clip before transmission, the filter 14 is transmitted with the video content 13, so allowing the receiver to view the clip with or without the filter, or use a different filter.

Description

[0001] The invention relates to a method of transferring content data, in particular video, in which a sender provides the content data, the sender generates a transmission signal representing the content data, the transmission signal is transferred to a receiver via a transmission channel, a receiver receives the transmission signal and retrieves the content data, and the receiver renders the content data.

[0002] The invention further relates to a device for sending content data, in particular video, which device comprises input means for providing the content data, and means for generating a transmission signal representing the content data.

[0003] The invention further relates to a device for receiving content data, in particular video, which device comprises means for receiving a transmission signal, means for retrieving the content data from the transmission signal, and means for rendering the content data.

[0004] The invention further relates to a signal and to a record carrier on which the signal is recorded.

[0005] Such methods, such devices, and such a signal are known from U.S. Pat. No. 5,557,320. The document relates to a method of transferring content data in the form of a transmission signal from a sender to a receiver via a transmission channel in a sender-subscriber based, transmission traffic control system for video mail. The electronic data is transmitted primarily in the form of still or motion picture images, with or without audio track. A delivery network system provider offers a service in which the network has control over the actual time of video mail transmission. The path, bandwidth allocation, protocol format etc. can be set at the discretion of the network provider, and also compression/decompression or encryption processors can be provided. However for the user there is no flexibility for controlling the transmission of such content data.

[0006] Therefore it is an object of the invention to provide more flexible means for transferring content data. According to a first aspect of the invention this object is achieved with a method as in the opening paragraph, characterized in that the sender provides a filter for adapting the content data, the sender generates a filter signal representing the filter, the filter signal is transferred to a receiver via the transmission channel, the receiver receives the filter transmission signal and retrieves the filter, and the receiver processes the content data by the filter.

[0007] According to a second aspect of the invention this object is achieved with a sending device as in the opening paragraph, characterized in that the device comprises filter input means for providing a filter for adapting the content data, and means for generating a filter signal representing the filter.

[0008] According to a third aspect of the invention this object is achieved with a filter signal representing content data and a filter to be used in the first method, characterized in that the filter is for setting a remote processing unit for filtering the content data, in particular video.

[0009] According to a fourth aspect of the invention this object is achieved with a rendering device as in the opening paragraph, characterized in that the device comprises means for receiving a filter signal representing a filter, and processing means for processing the content data as set by the filter.

[0010] The invention is based on the following recognition. Devices, e.g. mobile phones, are becoming multimedia terminals which can send and receive video as well as audio and text messages. This invention addresses how somebody could edit and adapt a video clip they were sending to friends by attaching a pre-defined ‘filter’ which adds suitable effects to the clip (e.g. color adaptation, overlaying of icons, fade between en scenes, etc.). For example, if sending a video message back from a romantic holiday to your friends, you may like to edit the clip to give a more romantic atmosphere (music in the background, ‘rose-tinted ’ filter, gentle fade in & out of video). However, if sending a message from an ‘adventure holiday’, you might like to adapt the clip with a lively filter (like an MTV music video). Instead of using the filter to edit the video clip before transmission, the filter is transmitted with the video, so allowing the receivers to view the clip with or without the filter, or use their own filters.

[0011] According to a fifth aspect of the invention this object is achieved with a method of providing a filter to be used in the first method, characterized in that the filter is for setting a processing unit for filtering content data, in particular video, in which method a number of filters is made available for selection, a client request is received indicative of a selection of a filter, a filter signal is generated representing the filter, and the filter signal is transferred to the client via a transmission channel.

[0012] The effect is that a suitable filter can be selected and filters can be downloaded onto a sender or a receiver, without the filter itself being actually transferred between sender and receiver.

[0013] Further preferred embodiments of the method, devices and signal according to the invention are given in the further claims.

[0014] These and other aspects of the invention will be apparent from and elucidated further with reference to the embodiments described by way of example in the following description and with reference to the accompanying drawings, in which

[0015] FIG. 1 shows a sender and receiver,

[0016] FIG. 2 shows a sender, receiver and filter provider,

[0017] FIG. 3 shows video processing,

[0018] FIG. 4 shows a record carrier, and

[0019] FIG. 5 shows a filter.

[0020] In the Figures, elements which correspond to elements already described have the same reference numerals.

[0021] FIG. 1 shows a sender and receiver. The sender 11 has a content input unit 15 for providing content data. The content data may for example be a video clip or a still image, retrieved from an external source or created within the sender by a camera module. The format of the input video can be any suitable, preferably standard, encoding (e.g. MPEG video). Further the content data may include text or audio, provided internally or from an external source under the control of the user at the sending side. The sender is provided with a filter input unit 16 for defining a filter to be used for adapting the content data to the liking of the user. The sender further comprises a content signal unit 17 for generating a transmission signal 13 representing the content data, and a filter signal unit 18 for generating a filter signal representing the filter as selected by the user for sending. In an embodiment the device has a filter creation unit for defining a specific filter by input provided by the user, whereby the user can define the parameters of the filter. The content data is encoded to a transmission signal 13 by the content signal unit 17 and the filter data is encoded to a filter signal 14 by the filter signal unit 18. The signals are transferred to a receiver 12 via a transmission channel, e.g. a mobile phone network or internet, or a record carrier like a CD-R. The receiver 12 has a receiving signal unit 19 for receiving the transmission signal 13, and a filter input unit 20 for receiving the filter signal 14. The receiver 12 further comprises a processing unit for performing the filter operation as defined by the filter signal as described below with reference to FIG. 3. The content data is filtered and can be rendered, e.g. on a video display, or can be provided to an output. The filter as transferred by the filter signal 14 can be provided with a range of effects, for example as follows.

[0022] Colour filters: Alters the colour of each frame in a consistent manner, e.g.:

[0023] Changes the whole clip to black and white

[0024] Creates a graded rose-tinted filter (top of image very red, bottom of screen normal colour)

[0025] Fades certain colours in and out in different frames.

[0026] Image filters: Alters the structure of each frame, e.g.:

[0027] Blurs edges.

[0028] ‘Heat haze’ effect, which wobbles parts of the image from side to side as the frames progress.

[0029] ‘Hall of mirrors’ effects—stretch some parts of image, shrink others.

[0030] Overlay video/images. The operation is to draw icons on top of the video, or to have animated images moving across the screen, scrolling text at the bottom of the video, etc.

[0031] Scene fades. Fade from one scene to another.

[0032] The scene fade filter would require automatic scene change detection, and this implies that the filter must be of the ‘non-streaming’ type.

[0033] For adapting audio or replacing an audio track the filter may contain audio filter operations, for example ‘Add audio track’ to add background music to the video clip. The format of the filter input file can be selected. An example format is given in the embodiment described with FIG. 5 below. The filter operation is performed by a processing unit, which is set to the type of filtering as prescribed by the filter. The processing unit may take encoded video data as it is available from the content data input. In an embodiment the filter works on the decoded video, so it is not dependent on the video encoding format. In this way the filter processing can be used for different types of video data. There are many well-understood methods for filtering video to adapt the content (e.g. color/contrast/brightness as well as transitions between shots).

[0034] FIG. 2 shows a sender, receiver and filter provider. In addition to the system described with reference to FIG. 1, the sender 11 now has a filter input unit for retrieving a filter from an external filter provider 22 via a network 21, e.g. the intemet. Also the receiver has a filter input unit 25 for retrieving a filter from an external filter provider 22 via a network 21. The sender may select one of the candidate filters available at the provider and download the filter to his device. The filter may then be transferred to the receiver via the filter signal 24. Alternatively the sender my include just a reference to the filter in the filter signal 24, and the receiver then can retrieve the filter itself from the provider 22 or from a different external source. In an embodiment the receiver has a local storage for filter data for storing filters to be used more often. Now the filter indicated by reference may already be available in the receiver storage. Mobile phones currently allow some minor features to be downloaded over the network (often at a cost), e.g. new ringtones. Also downloading screensavers or applications (e.g. java applets) is well understood, e.g. for a PC or a Personal Digital Assistants PDA which can run a virtual machine. The downloading of a filter can be done using similar transmission techniques.

[0035] FIG. 3 shows video processing. The video input is received as an encoded video signal in the video input unit 31 and transferred to a decoder 32. After decoding the decoded video is transferred to a processing unit 33. The processing unit is set to perform filter operations by a filter from a filter input unit 35, e.g. a filter input file received from an external source. The video after filtering is transferred to an output unit 34. In an embodiment the output unit includes a display for rendering the processed video. In a further embodiment the device has a selector 36 for selecting a video output signal after filtering or a video signal without filtering. A further selector may be included for using a user filter locally provided from a memory 37 in the device. It is noted that the video filter contains a general set of rules which can adapt any video clip with a suitable style. The filter is independent of any particular video clip. In particular the filter processing is applied to the decoded video, and is not dependent on the type of encoding. However, in a different embodiment the processing unit may be connected to the encoded video, and the filter may be applied to the encoded video. For the filter two main variants of processing can be used: streaming or non-streaming. In the streaming model, the filter engine adapts each video frame, and then outputs it before receiving the next frame. In other words, it does not require any information about future frames of video to perform the filtering process; this has the advantage that it adds minimal delay to the video display process. In the Non-Streaming model, the filtering process for a particular frame is dependent on future frames in the video stream. A simple example of this case would be where the filter engine receives the whole decoded video clip, processes this (e.g. to find out where scene changes occur), and then outputs the filtered video clip. This allows more flexibility in the processing, but implies a delay in viewing the video.

[0036] In an embodiment the video input unit 35 is able to provide a filter from an external source. The external source may be a record carrier to be read by a reading unit, e.g. a CD reader. Alternatively video filters could be downloaded by many different means to a device, e.g. a mobile handset. The invention assumes that the mobile device has the ability to connect to a network using standard (internet-based) technology. If this is the case, each video filter could be stored on the network with its own URL (Universal Resource Locator) the handset would then request this URL for download. The handset would be able to identify the downloaded file as a ‘video filter file’ as these files would be assigned their own unique data type identifying them as video filters (e.g. if the download is via HTTP, the MIME-type is transmitted automatically with the data). It is noted that the video input unit 31 and the filter input unit 35 may be configured to receive a combined video & filter message from a sender. Transmission of data messages between two mobile devices is well understood (e.g. text messages like SMS). Further the email transmission between a sender and a receiver is well known, and could also be arranged between 2 internet-enabled mobile phones. For a combined video & filter message to be transmitted, it would be necessary to transmit the two components of the message together, along with their respective data types. For example, if using email as the underlying transport protocol, the message could be transmitted as a multipart MIME message; the receiving device would then receive the video and the filter at the same time, and display the filtered video accordingly. In an embodiment the sender and receiver can access data via URLs on the network by the well known HTTP protocol, e.g. using a browser, and receive & transmit email messages, e.g. via POP3/SMTP, and decode and display MPEG4 video. For example 2 WAP-enabled mobile phones (Wireless Access Protocol) can access data using the WTP protocol. Further the sender may include a video content generator, e.g. a camera for capture and an encoder for encoding video as MPEG4. Assuming this, the invention could be implemented using the following steps. Filters are downloaded via a browser over HTTP. They have a well-defined MIME-type (e.g. ‘text-video-filter’). The filter file format is XML-based; an example filter is shown below. On download, the filter is stored in the mobile phone memory. When transmitting the combined video & filter, the message is sent as a multipart-MIME email message (containing MPEG4 clip & ‘text-video-filter’ filter). Using these concepts, the development of a filter would then only be dependent on how the software for creating and displaying the video message is defined, and how the ‘filter engine’ is defined.

[0037] FIG. 4 shows a record carrier 41 provided with a track 42 intended for recording and a central hole 40. The track 42 is arranged in accordance with a spiral pattern of windings. A first part of the track in the middle area of the record carrier constitutes a Lead-In 43 for accommodating control data. The record carrier comprises a transparent substrate with a recording layer and a protective layer. The recording layer may be optically writable, for example via phase change, or magneto-optically writable by a device for writing information such as the known CD-Rewritable or CD-Recordable. The recording layer may also be provided with information via a production process, in which first a master disc is made which is subsequently multiplied through pressing. The information is organized in information blocks and is represented by optically readable marks in the form of a succession of areas reflecting much radiation and little radiation such as, for example a succession of pits of different lengths in a CD. The information blocks comprise a filter signal representing the filter data structure and parameters as described above. An embodiment of the record carrier comprises a large collection of filters, so that the user may select a suitable filter. In an embodiment the record carrier comprises video processing software for programming a standard processing unit, e.g. a PC, for performing the video encoding or decoding and filter processing.

[0038] FIG. 5 shows a filter. The filter 51 is defined in the XML type language (eXtended Markup Language). The first instruction indicates a colour filter, which is to be retrieved from an external source; in this case an internet URL is indicated. This part of the filter will change the colour to black and white. The next section may contain image operations, but non are included in this example. The last section contains an overlay operation, in this case an icon named smiley is to be overlaid by the processing unit.

[0039] Although the invention has been explained by embodiments using mobile handsets, any rendering device or other suitable device may be used. Also for the record carrier an optical disc has been described, but other media, such as a magnetic disc or tape, may be used. It is noted, that in this document the word ‘comprising’ does not exclude the presence of other elements or steps than those listed and the word ‘a’ or ‘an’ preceding an element does not exclude the presence of a plurality of such elements, that any reference signs do not limit the scope of the claims, that the invention may be implemented by means of both hardware and software, and that several ‘means’ may be represented by the same item of hardware. Further, the scope of the invention is not limited to the embodiments, and the invention lies in each and every novel feature or combination of features described above.

Claims

1. Method of transferring content data, in particular video, in which a sender (11) provides the content data, the sender generates a transmission signal (13) representing the content data, the transmission signal is transferred to a receiver (12) via a transmission channel, the receiver receives the transmission signal and retrieves the content data, and the receiver renders the content data, characterized in that the sender (11) provides a filter for adapting the content data, the sender generates a filter signal (14) representing the filter, the filter signal is transferred to a receiver via the transmission channel, the receiver (12) receives the filter transmission signal and retrieves the filter, and the receiver processes the content data by the filter.

2. Method of providing a filter to be used in the method of claim 1, the filter being for setting a processing unit for filtering content data, in particular video, in which method a number of filters is made available for selection, a client request is received indicative of a selection of a filter, a filter signal is generated representing the filter, and the filter signal is transferred to the client via a transmission channel (21).

3. Device for sending content data, in particular video, which device comprises input means (15) for providing the content data, and means (17) for generating a transmission signal representing the content data, characterized in that the device comprises filter input means (16) for providing a filter for adapting the content data, and means (18) for generating a filter signal representing the filter.

4. Device as claimed in claim 3, wherein the device comprises processing means (33) for processing the content data as set by the filter.

5. Device as claimed in claim 3, wherein the device comprises means (34) for rendering said processed content data.

6. Device as claimed in claim 3, wherein the device comprises filter input means (25) for retrieving a filter from an external source, in particular via a network or from a storage medium.

7. Device as claimed in claim 3, wherein the device comprises means for transmitting the transmission signal and the filter signal as a combined message.

8. Filter signal representing content data and a filter for use in the method of claim 1, the filter (51) being for setting a remote processing unit for filtering the content data, in particular video.

9. Filter signal as claimed in claim 8, wherein the filter (51) comprises parameters for filtering streaming video, in which filtering process is only dependent on frames of the video up to a frame to be displayed next.

10. Filter signal as claimed in claim 8, wherein the filter (51) comprises parameters for filtering non-streaming video, in which the filtering process is dependent on multiple frames in the video including frames to be displayed beyond the next frame.

11. Filter signal as claimed in claim 8, wherein the filter (51) comprises a color filter, and/or image structure filter, and/or an overlay filter, and/or a scene fade filter.

12. Filter signal as claimed in claim 9, wherein the filter (51) comprises an audio filter and/or additional audio

13. (currently amended) Record carrier (41) on which a filter signal as claimed in claim 8 is provided in a track (42) in which information patterns represent the filter signal, the filter being for setting a remote processing unit for filtering content data, in particular video.

14. Record carrier as claimed in claim 13, wherein the record carrier comprises a computer program product for processing content information according to the filter.

15. Device for receiving content data, in particular video, which device comprises means for receiving a transmission signal, means for retrieving the content data from the transmission signal, and means for rendering the content data, characterized in that the device comprises means (20) for receiving a filter signal representing a filter, and processing means (33) for processing the content data as set by the filter.

16. Device as claimed in claim 15, wherein the device comprises rendering means (34) for rendering said processed content data.

17. Device as claimed in claim 15, wherein the device comprises a selector (36) for rendering the content data with or without being processed by the filter.

18. Device as claimed in claim 15, wherein the device comprises filter input means (35) for retrieving a filter from an external source, in particular via a network or from a storage medium, as indicated by a reference in the filter signal.