This application claims the benefit under 35 U.S.C. §119(a) to an Indian Provisional Patent Application filed in the India Patent Office on Mar. 30, 2012 which was assigned Serial No. 1352/CHE/2012, an Indian Non-Provisional Patent Application Number, 1352/CHE/2012 filed on Apr. 3, 2012 and a Korean Patent Application No. 10-2013-0032028 filed on Mar. 26, 2013, the disclosure of which is hereby incorporated by reference herein.
The present invention relates to multimedia content tagging and more particularly to updating tags after editing the multimedia content based on editing operations.
With the advancement of communication technology, Internet service providers offer a variety of web-services that allow users to upload a variety of multimedia content, which may include images, audio, video and so on.
The content may be further shared with other users based on user's preference. Before sharing the content, the user may be interested in editing the content. For example, a user can combine several images into a single image, cutting off a part of the images, applying editing effects, and transform effects such as rotating, resizing, flipping and so on, then share it with other users using these web services. The edited content, however, does not contain all the original information. In particular, tagged information and the like may be lost.
A tag in a multimedia content refers to any additional information added to the multimedia content. A tag may be associated with a position in the multimedia content. This can be a spatial position (for example expressed in two or three dimensional coordinates, such as pixel coordinates) or a temporal position (for example, expressed in time lapse or frame number), or a combination of the two.
In case of image, the tag can be name of the person(s) in the image, location where the image was captured and so on. In conventional method, tags in the image will be lost when user performs any editing operations in the image. To this end, the user can manually input tag information at the desired content as content once edited can become untagged. However, these conventional content editing methods are disadvantageous as the users must manually input tag information to the edited content.
Moreover, the tag location are not updated at the proper field in the edited content whenever the user performs (1) transform editing operations on image such as cropping, resizing, flipping and (2) rotation operations or distortion operations such as warping, or (3) combining of two images such as stitching, blending etc.
Further, there are no effective solutions to update the tags for each editing operations performed on the original content and reapply the updated tag on the edited content for each format supported by destination platform.
Due to above mentioned drawbacks, existing methods fails to update the tags in the content when the content is edited. Hence, there is a need for a method in which the tags are updated at proper place automatically when the content is edited.
The present invention provides a method and system to update the tags at proper place in the edited multimedia content after the original multimedia content has been edited.
Another aspect of the invention is to provide a method to format the tags based on the format supported by destination platform in cross platform environment.
Accordingly the invention provides a method for reapplying at least one tag in an edited content based on at least one editing operation, wherein the method includes retrieving a first position of the at least one tag in an original content, applying at least one editing operation to the original content which in turn yields an edited content on a display, then calculating a second position of the at least one tag in the edited content, and reapplying at least one tag of the original content in the calculated second position for display.
The method allows a user to acquire the content with tags from a source platform, then the method formats the tags based on the format supported by different destination platforms.
In other words, the invention provides a method for automatically updating at least one tag of multimedia content, the method comprising: retrieving a first position of said at least one tag in the multimedia content; applying at least one editing operation to said multimedia content for providing edited multimedia content calculating a second position of said at least one tag in the edited multimedia content corresponding to the at least one editing operation; and updating the position of said at least one tag in the edited multimedia content using the calculated second position.
In an embodiment according the invention, the first and second position of the tag is expressed in first and second tag coordinates respectively, the at least one editing operation comprises a transformation of pixel positions, and the second tag coordinates are calculated by applying the transformation to the first tag coordinates.
Further, the method updates at least one tag after formatting at least one tag in the at least one of: first position, second position based on the formats supported by different platforms.
Accordingly the invention provides an apparatus for automatically updating at least one tag of an original content during an edit mode, the apparatus includes: a processor and a memory coupled to the processor. The memory storing one or more programs comprising instructions which, when executed by the processor cause the apparatus to: apply at least one editing operation to said original content for providing an edited content; calculate a second position of said at least one tag in the edited content according to predetermined criteria; and reapply said at least one tag of said original content at said calculated second position for display.
In other words, the invention provides an apparatus comprising a processor configured to implement a method as described above.
These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.
This invention is illustrated in the accompanying drawings, throughout which like reference letters indicate corresponding parts in the various figures. The embodiments herein will be better understood from the following description with reference to the drawings, in which:
The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein can be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
Throughout the document the term content and multimedia content are used interchangeably.
The embodiments herein achieve a method and system to update and reapply tags at proper place automatically even after the content is edited. The position of a tag can be a spatial position (for example expressed in two or three dimensional coordinates, such as pixel coordinates) or a temporal position (for example, expressed in time lapse or frame number), or a combination of the two. The method retrieves the first position of tags from original content and allows user to perform editing operations on the original content to generate edited content. After applying editing operations, the method herein calculates the second positions of the tags in the edited content as per editing operation applied. In an embodiment, the method calculates the second position of the tags one by one after each editing operation.
In an embodiment, at the end of all editing operations the method calculates the second position of tags by applying each editing operations one by one to tag. In other words, the second position is calculated by evaluating a cascade of transformations corresponding to the cascade of editing operations.
In an embodiment, the method calculates the second position of tags after at least one editing operation.
In an embodiment, the content can comprise of image, video, audio or the like.
In an embodiment, the tag may refer to the tags created by the user such as tags of people, face tags, location, and brands and so on.
Referring now to the drawings, and more particularly to
In an embodiment, the original image is obtained from a device such as Mobile phone, Camera, Computer and like, information and data servers (Social Networking Sites, multimedia content hosting sites and like) which provide means to upload/download multimedia content.
After receiving the image, the method checks (102) whether the user wants to apply any editing operation.
If the method finds that the user wants to apply editing operation, then the method applies the editing operation to the multimedia content and calculates (103) second position of the tags in edited image based on editing operation applied.
After calculating the second position(s) of tag, the method loops back to the previous step 102.
In an embodiment, the editing operation can include rotating, cropping, resizing, image stitching, image merging, image warping, image blending, cut copy paste, scribbling or the like.
In an embodiment, the editing operation can include changing at least one pixel value of the image.
In an embodiment, the editing operation can include changing at least one pixel position of the image.
In an embodiment, the editing operation can include acquiring content from source platform with tags as per source platform and upload same content on destination with tags as per format supported by destination platform.
In an embodiment, the editing operation can include acquiring content from source platform with tags as per source platform and editing the content and upload edited content on destination platform with edited tags as per format supported by destination platform.
In an embodiment, the editing operation can include changing at least one frame position in a sequence of image frames such as in video.
In an embodiment, the editing operation can include adding at least one frame at any position in video frame sequence.
In an embodiment, the editing operation can include removing at least one frame from any position in video frame sequence.
If the editing operation is distortion, which comprises of a nonlinear movement of pixels, then pixels of the original image can get distributed in the edited image based on distortion operation. The position of a tag before distortion operation on an image and after the distortion will get distributed to multiple positions. The method calculates second position of the tag from these multiple positions.
In an embodiment, the method calculates the second position of tag by performing mathematical operation in at least one position among these multiple positions. For example, the method uses the mathematical operation as either centroid or mean or median to calculate second position of the tag from at least one position among these multiple positions.
Further, the method updates (104) the first position of the tag in the original image to the second position of the tag of the edited image based on editing operation(s) performed.
The method of updating tags with example editing operations scenario is as follows:
The method first retrieves the first position of tag in the original image. For example, the tag is retrieved from
x=50
y=60
and assigns tag=“my_friend_tag”
where x and y are percentages with respect to width and height of the original image.
So, if the width of the image is 400 units and height of the image is 300 units, then 50% of width is 200 units and 60% of height is 180 units, and then position of the tag in terms of absolute co-ordinates would be 200 units in x direction and 180 units in y direction.
If editing operation performed is 90 degree clockwise rotation:
Then, the method rotates the edited image to 90 degree clockwise and second position of the tag will be calculated as follows:
X′=100−y (where 100 is the percentage)
Y′=x
and assign Tag=“my_friend_tag”
Considering the above example, the value of X′ and Y′ will be:
X′=100−60=40
Y′=50
Finally, the method updates the position of tag X′, Y′ in edited image.
It should be noted that various actions in flow diagram 100 may be performed in the order presented, in a different order or simultaneously. Further, in some embodiments, some actions listed in
More specifically,
The process of calculating second position of the tag is detailed below when either rotation or flip operation is performed.
‘W’ refers to the width of the image and ‘H’ refers to the height of the image in
X′=W−X (1)
Y′=Y (2)
If editing operation performed is 90 degree clockwise rotation, the new width W′ will be equal to the original height H of the image and the new height H′ will be equal to the original width W of the image in
X′=H−Y (3)
Y′=X (4)
The above mentioned method is an example and is not limited to only vertical flip and 90 degree clockwise rotation. However, the method calculates the second position of the tags when any type of flip operation and rotation to any degree is performed.
In an embodiment, the user performs the warping (nonlinear) operation and the flip operation in the original image.
The method calculates the second position of the tag and updates the tag in the flipped image as explained in the
Warping results in nonlinear pixel movements and the method provides the applying of tags at proper places in warped image. The position of the tag before applying warping operation on image and after warping operation can get distributed to multiple positions. Warping generally results in expanding or shrinking the image pixels. In the
For example, the method calculates the second position of the tag in the warped image as follows:
Xnew=xold*sin(xold*xold+yold*yold) (5)
Ynew=yold*sin(xold*xold+yold*yold) (6),
where xold and yold are the co-ordinates of first position of the tag in original image. Equations (5) and (6) are exemplary way of calculating second position of the tag. The proposed method can use any other mathematical equation to calculate second position of the tag. Xnew and Ynew are the second position of the tag in the edited image.
More specifically,
As shown, the first input image comprises a woman tagged 1, a man tagged 2 and a light house tagged 3. The second input image comprises a boy tagged 4, a girl tagged 5 and a tower tagged 6. When user wants to merge or combine these two input images, then the inventive method combines the images and updates the second position of the tags in combined image based on the first position of the tags in input images and combined image dimensions and input image dimensions.
The process of calculating the second position of the tags when two images are combined together as follows:
Assume that the initial position of tag in image 1 of width ‘W1’ and height ‘H’ is (X1, Y1) and initial position of tag in image 2 of width ‘W2’ and height ‘H’ is (X2, Y2). When these two images are combined, then the second position of the tag in the image 1 is (X1′, Y1′) and second position of tag in image 2 is (X2′, Y2′) respectively. After combining image 1 and image 2 in which image 1 combines to left and image 2 to right in combined image as shown in
X1′=X1 (7)
Y1′=Y1 (8)
and
X2′=W1+X2 (9)
Y2′=Y2 (10)
width of the combined image becomes W1+W2.
On the other hand, after combining image 1 and image 2 in which image 2 combines to left and image 1 to right in combined image as shown in
X1′=W2+X1 (11)
Y1′=Y1 (12)
and
X2′=X2 (13)
Y2′=Y2 (14)
width of the combined image becomes W2+W1.
It should be noted that the images to be combined can have different width and heights and may be combined in any direction with overlapping width or height. Based on the ratio between final edited image width, height and input image width, height the method determines the final positions of the tags.
The method calculates the second position of the tags in the cropped image as follows.
Assume the original image has width ‘W’ and height “H’. The first position of the tags in the original image is represented as (Xi, Yi), where ‘i’ may be from 0 to n, and n are the total number of tags present. As a result of cropping, the image width and height have changed to Wc and Hc, respectively. The inventive method calculates the second position of the tags (Xi′, Yi′) in the cropped image and updates the tags in the cropped image as shown in
Xi′=Xi−Xc (15)
Yi′=Yi−Yc (16)
Assume image 1 with width ‘W1’ and height ‘H’ has a tag ‘A’ in the tag area. Let image 2 with ‘W2’ and height ‘H’ has tag ‘B’ to blend with image 1. Blending results in adding additional tag(s) in the blending area and retaining the existing tag(s) in non-blending area and second positions of the tags are determined based on the blending position.
When the position of top-left corner of tag area for the image 2 is Xc=0, Yc=0 and a first position of tag information “B” included in a tag area of the image 2 is x2=0, y2=0, an offset between the position of the top-left corner (Xc,Yc) of the tag area and the first position (x2, y2) of the tag area (a2−1) can be calculated by the following equations.
X
0
=x2−Xc (15)
Y
0
=y2−Yc (16)
And, when blended position of the tag area including the tag information “B” in an image edited by the blending operation, is Xb, Yb, the second position of the tag information “B” can be calculated by the following equations.
x2′=X0+Xb (17)
y2=Y0+Yb (18)
After the copy/cut area is pasted over the image 1, and then pasted area occupies tag ‘A’ area completely as shown in
Otherwise, pasted area occupies tag ‘A’ area partially as shown in
The position of the tag ‘A’ after the pasted operation depends on the paste position in edited image. In an image edited by the pasted operation, the second position of the tag information “B” can be calculated by the same way as that of the equations 15˜18.
If the tag area is scribbled by using brush like functionalities, the tag will still remain when the tag area is partially covered and tag will be removed when tag area is fully covered.
If in image there is one tag then there must be some area associated with that tag. For example face is single tag but the area associated may be a face bound in the rectangle or face itself.
Now in brush related functionalities there is provision to paint on image or scribble over image. If user scribble or paint the tag area fully then that tag is removed otherwise if tag area is scribbled or painted partially that tag would remain.
As shown, the original image is initially flipped by the user and the person images with their respective tags are oriented differently according to the flip operation as previously explained with reference to
The overall computing environment 1001 can be composed of multiple homogeneous and/or heterogeneous cores, multiple CPUs of different kinds, special media and other accelerators. The processing unit 1004 is responsible for processing the instructions of the algorithm. Further, the plurality of processing units 1004 may be located on a single chip or over multiple chips.
The algorithm comprising of instructions and codes required for the implementation are stored in either the memory unit 1005 or the storage 1006 or both. At the time of execution, the instructions may be fetched from the corresponding memory 1005 and/or storage 1006, and executed by the processing unit 1004.
In case of any hardware implementations various networking devices 1008 or external I/O devices 1007 may be connected to the computing environment to support the implementation through the networking unit and the I/O device unit.
The embodiments disclosed herein can be implemented through at least one software program running on at least one hardware device and performing network management functions to control the elements. The elements shown in
The above-described methods according to the present invention can be implemented in hardware, firmware or as software or computer code that can be stored in a recording medium such as a CD ROM, an RAM, a floppy disk, a hard disk, or a magneto-optical disk or computer code downloaded over a network originally stored on a remote recording medium or a non-transitory machine readable medium and to be stored on a local recording medium, so that the methods described herein can be rendered in such software that is stored on the recording medium using a general purpose computer, or a special processor or in programmable or dedicated hardware, such as an ASIC or FPGA. As would be understood in the art, the computer, the processor, microprocessor controller or the programmable hardware include memory components, e.g., RAM, ROM, Flash, etc. that may store or receive software or computer code that when accessed and executed by the computer, processor or hardware implement the processing methods described herein. In addition, it would be recognized that when a general purpose computer accesses code for implementing the processing shown herein, the execution of the code transforms the general purpose computer into a special purpose computer for executing the processing shown herein.
The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the embodiments as described herein.
Number | Date | Country | Kind |
---|---|---|---|
1352/CHE/2012 | Mar 2012 | IN | national |
10-2013-0032028 | Mar 2013 | KR | national |