The present technology relates to video and motion pictures. More specifically, it relates to methods and systems for implementing user deformation of character images of video or movie.
Video and motion pictures are a popular form of entertainment. Video and movies can be distributed to viewers on recordable medium such as optical disks (e.g., DVD) or they may be downloaded as a video data file from a network. These may then by utilized for personal viewing on home entertainment equipment such as televisions, DVD players and computers. However, besides the act of viewing of the images of the scenes and characters of the videos, there is little more that a viewer can do with the video or movie. In fact, there is little or no opportunity for a viewer to interact with the character images of the movie or video.
The present technology is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, in which like reference numerals refer to similar elements including:
One aspect of the present technology, involves methods for displaying a video. Frames of a video are displayed on a display. The displayed frames of the video include a character image having a first anatomical feature. An input is received with a user interface associated with the display. Then, in response to the input, a second anatomical feature is displayed corresponding to the first anatomical feature. The second anatomical feature comprises a deformation of the first anatomical feature. The second anatomical feature is then displayed in place of the first anatomical feature during the video.
In some embodiments, the generating of the second anatomical feature involves detecting pixels of the first anatomical feature of the video by scanning pixel data of a frame of the video. In some embodiments, the generating of the second anatomical feature involves accessing metadata associated with the first anatomical feature of the video. The metadata may be frame identifier data to identify a frame containing the first anatomical feature and position data to identify positioning of the first anatomical feature. The metadata may also include action data indicative of a deformation procedure for the first anatomical feature. In some embodiments, displaying of the second anatomical feature involves overlaying at least in part the second anatomical feature with the first anatomical feature. This may be accomplished without modifying any frames of the video.
In some embodiments, the displaying of frames of the video includes displaying a deformation area indicator to indicate an anatomical feature of the video that can be subjected to viewer deformation. Moreover, the input with the user interface may take the form of a command to generate the second anatomical feature by a change in size of the first anatomical feature. The input may also be a command to generate the second anatomical feature with a change in orientation of the first anatomical feature. Moreover, deformation data corresponding to the generated deformations of the first anatomical feature of the video may be stored in a file separate from the video. The stored file can be transmitted in a format to permit a viewer of another copy of the video to display the second anatomical feature in place of the first anatomical feature during the viewer's display of the copy of the video. In some embodiments, modifications of the second anatomical feature for display with additional frames of the video may be automatically generated. These modifications can correspond to orientation and position changes of the first anatomical feature in the additional frames of the video with respect to a first frame of the video. In some embodiments the video may comprise a motion picture.
Example embodiments can permit the first anatomical feature to be a nose of the character and the second anatomical feature to be a deformed version of the nose. In addition, the first anatomical feature may be an eye of the character and the second anatomical feature may be a deformed version of the eye. In some examples, the displaying of the second anatomical feature in place of the first anatomical feature can comprise a viewer induced jiggling of an anatomy of the character.
In some embodiments, some or all of the features of these methods may be embodied in a machine readable medium having processor control instructions. Thus, the processor control instructions can control a processor to display a video as previously discussed. The processor control instructions can also include instructions to display frames of a video on a display, the displayed frames of the video comprising a character image having a first anatomical feature. The processor control instructions may also control receiving an input with a user interface associated with the display. Moreover, the processor control instructions may control, in response to the input, generation of a second anatomical feature corresponding to the first anatomical feature, the second anatomical feature comprising a deformation of the first anatomical feature. The processor control instructions may then control displaying the second anatomical feature in place of the first anatomical feature during the video.
In some embodiments, some or all of the features of these methods may be embodied in a video player apparatus. The apparatus may typically include an output port to send signals to a video display. The apparatus may also include a user interface to receive an input with respect to an anatomical feature of a character image on the display. The apparatus may also have a processing means for controlling a display of frames of a video on a display where the video frames include a character image having a first anatomical feature. The processing means may be configured for generating a second anatomical feature corresponding to the first anatomical feature in response to the input of the user interface. The second anatomical feature may be a deformation of the first anatomical feature. In addition, the processing means may also be configured for displaying the second anatomical feature in place of the first anatomical feature during the video.
Further embodiments and features of the technology will be apparent from the following detailed disclosure, abstract, drawings and the claims.
An example implementation of the present video or movie character image deformation technology is illustrated in
Typically, the character image 102 of the frames of the video or movie will also include anatomical features such as the anatomical feature 106 shown in
In accordance with an embodiment of the present technology, during a presentation of the film or movie frames 102 one or more deformed anatomical features of the character images may be displayed by a video player apparatus in place of the original anatomical feature of a character image of the video or film. For example, as illustrated in
Example steps of the methodology of a video player apparatus of the present technology are in the flow chart of
In an example embodiment of the methodology, a viewer of a video played on a video player apparatus may operate a user interface or other input device, such as a mouse, keyboard, remote control etc., to identify an anatomical feature of a character image of the video frame. Such identification may optionally involve the viewer manipulating the user interface to control a graphical selector on a display controlled by the video display apparatus. Such a graphical selector (illustrated as selector 555 in
For example, the image pixel data of a frame of the video may be automatically scanned by the video player apparatus within the selected area of the graphic selector to identify pixel data associated with an anatomical feature in the selected area. For example, facial features may be identified by implementing a recognition algorithm such as a face recognition algorithm or an iris recognition algorithm. Similarly, a nose anatomical feature may be identified by its typical positional relationship with respect to eye pixel data determined with the eye or iris recognition algorithm.
Alternatively or in addition thereto, metadata concerning the video pixel data for one or more anatomical features may be accessed based on the position or area of the frame of the video selected by the viewer with the graphic selector. In this regard, the metadata may contain information concerning the anatomical features of characters of the frames of the video to permit user deformation of particular anatomical features of the video. Thus, metadata may be provided for one or more frames of the video or movie or for each frame that includes one or more anatomical features for user deformation. For example, the metadata may contain position information for the pixels of a frame that depicts an anatomical feature. The position information may be considered a bounding box or active area for a deformable anatomical feature. The metadata for a frame may have more than one such bounding box depending on the number of deformable anatomical features. The metadata may also optionally contain data to represent action procedures that may be taken with respect to the anatomical feature such as enlarge, stretch, shrink, skew, rotate, pitch, roll, etc. Optionally, the metadata may contain three dimensional object data as discussed in more detail herein. For example, the metadata may include z-axis data for each bounding box of each frame of a deformable anatomical feature to assist with deformation or adjustment of the anatomical feature in accordance with relative camera angle adjustments that may exist across several frames of the video that include the particular anatomical feature. In this way, certain anatomical features may be tagged for viewer deformation by providing metadata for one or more particular anatomical features and their association with the video or the frames of the video. This metadata may be stored together with or separate from the file containing the pixel data of the video.
Thus, in response to the selection, a person watching the video or video viewer may then control the user interface to change an appearance of the selected anatomical feature. For example, the video viewer might drag a portion of the selected anatomical feature of the frame to stretch, skew, rotate, etc. or otherwise deform the selected anatomical feature. Thus, the anatomical feature may appear different from the original version of the anatomical feature. Moreover, while the anatomical feature may be so deformed, the other features of the scene of the frame of the video and the remainder of the character's unselected anatomical features would remain unchanged.
In one embodiment, the video player apparatus may then display the deformed anatomical feature by generating a deformed overlay image with pixel data of the anatomical feature. The apparatus may then overlay the generated pixel data at a position associated with the pixel data of the original anatomical feature when it is displayed by the video player apparatus with the original frame data that depicts the original anatomical feature. Moreover, while the original video frame data may be modified, this display may optionally be accomplished without changing any image data of the frames of the original video that has the original anatomical feature(s). Thus, the deformation data associated with the deformation image may be stored separately from the data of the video and may simply be displayed at the appropriate time in conjunction with the original frames of the video. Thus, the deformation data created by a user may optionally include frame number(s) of the video for which the deformed image should be displayed, position information for where it should be displayed in each frame, and pixel data for generating the deformed anatomical feature. In some embodiments, this may optionally be accompanied by the metadata of the video as previously discussed.
In some embodiments, after a viewer-initiated deformation of a particular anatomical feature of a particular frame, the video display apparatus may automatically generate further changes or transformations to the deformed anatomical feature for subsequent frames of the video so that the selected deformation may appropriately transform in correspondence with appearance changes of the original character or anatomical feature in the subsequent frames of the original video. These adjustments may be accomplished automatically in the sense that no further user modification would need to be made after a deformation was made by the user with respect to an image of a prior frame.
For example, as an unmodified video is displayed in subsequent frames, a camera (view) angle, zoom, field of view etc. may change or even the character itself may move in the field of view of the video. To match these changes so that the deformed feature may continue to correspond to the original feature, the deformed anatomical image feature may be modified to match the camera angle change, zoom change, etc. or even position change of the character of the original frames of the video. As illustrated in
These deformation methodologies may be implemented as hardware and/or software in a video player apparatus. For example,
As illustrated in the embodiment of
Accordingly, as illustrated in
Such a video player apparatus can provide movie and video viewers with an even more enjoyable viewing experience from what has been previously available to movie or video viewers. For example, as illustrated in
Moreover, with such a system, people can share their video anatomical deformations with others. For example, while viewing a video of a debate with a video player apparatus, a first user could deform some anatomical features of a character of the debate and then share those deformations with a friend by transferring the deformation data with or without the video to a friend who also has a video player apparatus. To this end, the video player apparatus may store or record the deformation data (with or without the metadata) or deformation images in a file that is separate from the video data of the debate. This separate storage can promote the efficient sharing or communication of the deformation data. Thus, when the friend views her own copy of the debate video, the friend's video player apparatus may be controlled by the user so that it utilizes the separate file with the deformation data to overlay or re-enact the deformation modifications generated by the first viewer with her video player apparatus. Such a stored file with deformation data and/or metadata may thus be in a format to permit a viewer of a different copy of the video to display the second anatomical feature in place of the first anatomical feature during the friend viewer's display of a different copy of the video. Typically, this may be accomplished with deformation data that associates the deformation images with frame identifications of the original video and/or positioning data within each frame.
With such a video player apparatus that permits anatomical deformations of character images, many deformations may be created. For example, a user can inflate or expand a head of a character image of the video as illustrated with the modification of the original character frame 628A in
In the foregoing description and in the accompanying drawings, specific terminology and drawing symbols are set forth to provide a thorough understanding of the present technology. In some instances, the terminology and symbols may imply specific details that are not required to practice the technology. For example, although the terms “first” and “second” have been used herein, unless otherwise specified, the language is not intended to provide any specified order or count but merely to assist in explaining elements of the technology.
Moreover, although the technology herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the technology. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the technology. For example, in some embodiments, preset deformation image data may also be provided for selection by a user, which may then be overlaid with the anatomical images of the original video. This present deformation image data may then be associated with the metadata for the video. For example, pre-set deformation image data may represent alternative eyes, noses, ears, hair, accessories, etc. that may be overlaid with the original anatomical features of the characters of the video according to the metadata and the user selections.