1. Field of the invention
The present invention generally relates to methods and systems for sharing multimedia content during a video communication session.
2. Description of the Related Art
As consumers and businesses depend more and more on remote collaboration capabilities, demand for making these collaborative processes is growing. Needs for collaborative processes include business collaboration, entertainment, social networking and mobile communication. Instances may arise when making a video call that the callers may wish to share content such as documents, pictures, music, etc. Thus, the information may be in a variety of multimedia formats. In some cases it may be beneficial to allow both the callers and the multimedia content to be viewed on the same screen. Rather than simply sending the files to the callers, it would be advantageous to share the multimedia content in the form of a background or wallpaper during the video call. Moreover, it would be advantageous to enable interaction with the background during the video call. This virtual interaction with the content would allow for a richer call experience.
Well known methods are available to provide “green screen” backgrounds in video display. Green screens or Chroma key compositing, has been used heavily in many fields to remove a background from the subject of a photo or video—particularly the newscasting, motion picture and videogame industries. A color range in the top layer is made transparent, revealing another image behind. Some of these methods have been used in video communication applications.
However, these “green screen” methods as applied to video communication technologies do not provide for the background to be shared among callers of a video call and they do not enable shared interactivity of the background among the callers. Moreover, “green screen” methods are not suited to video communication because the background in a video communication session is not fixed.
Accordingly, there is a need for sharing multimedia information in the form of an interactive a background to a video call and allowing the participants the ability to interact with the background with detected motions and gestures.
A method and system for simultaneous display of a multimedia background is disclosed. In some embodiments the system and method includes conducting a video call from a first caller on a first terminal to a second caller on a second terminal and substituting a multimedia content for at least a portion of an original background of the first terminal to create a combined multimedia background. The combined multimedia background is transmitted to the second terminal. Moreover, the combined multimedia background includes an image superimposed over the multimedia content. The combined multimedia background is enabled to be interactively controlled by a movement detection device.
Other and further embodiments of the present invention are described below.
So that the manner in which the above recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments.
To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. The figures are not drawn to scale and may be simplified for clarity. It is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further recitation.
In embodiments consistent with the present invention, a video call may be initiated by methods well known in the art.
In some embodiments, the invention is described as practiced on a local user device such as a handheld mobile device. However, the invention may also be practiced on a desktop computer, a server based environment or other devices capable of video communications.
Some portions of the detailed description which follow are presented in terms of operations on binary digital signals stored within a memory of a specific apparatus or special purpose computing device or platform. In the context of this particular specification, the term specific apparatus or the like includes a general purpose computer once it is programmed to perform particular functions pursuant to instructions from program software. In this context, operations or processing involve physical manipulation of physical quantities. Typically, although not necessarily, such quantities may take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared or otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to such signals as bits, data, values, elements, symbols, characters, terms, numbers, numerals or the like. It should be understood, however, that all of these or similar terms are to be associated with appropriate physical quantities and are merely convenient labels. Unless specifically stated otherwise, as apparent from the following discussion, it is appreciated that throughout this specification discussions utilizing terms such as “processing,” “computing,” “calculating,” “determining” or the like refer to actions or processes of a specific apparatus, such as a special purpose computer or a similar special purpose electronic computing device. In the context of this specification, therefore, a special purpose computer or a similar special purpose electronic computing device is capable of manipulating or transforming signals, typically represented as physical electronic or magnetic quantities within memories, registers, or other information storage devices, transmission devices, or display devices of the special purpose computer or similar special purpose electronic computing device.
The memory 210 includes an operating system 240, a media processing module 225, a voice and video communication module 235, a data communication module 255 and a user interface module 245. The operating system 240 controls the interoperability of the support circuits 230, CPU 220, memory 210 and the I/O interface 250. The user interface module 245 contains instructions for the I/O interface 250 to interact with the operating system 240 such as for operation of a Graphical User Interface (GUI).
The media processing module 225 provides for media capture, processing, and encoding operations. In some aspects, the media processing module 225 is configured to combine the video stream of a video call with another set of video data or still image data, such as to produce a single output video. In some aspects, the media processing module 225 applies various post-processing effects to the output video in order to make the output video appear as if it was generated from a single video stream, In some aspects, the media processing module 225 operates to eliminate or process out some or all of an existing background in order to support the insertion or layering of documents, video/music controls, pictures and other media and to assemble the inserted background and video call in a single output. In some aspects, the media processing module 225 may interface with a template associated with a particular video call to facilitate the combination of a video call and a background to create a single output containing both the video call and the particular multimedia content.
In some embodiments, the image of the caller 330 is separated from the original background of the video call through the use of face recognition methods well known in the art. In this embodiment, the image of the caller 330 is preserved and the original background is replaced by a selected multimedia background 320.
In other embodiments, the image of caller 330 is separated from the original background by detecting static or near static pixels. The image of the caller 330 is preserved in the foreground while the detected static or near static pixels that constitute the original background are replaced by the selected multimedia background 320.
At step S440, an invitation is sent from first terminal 105 to the second caller on the second terminal 115 to share the multimedia background 320. If the invitation is accepted at step S450, then at step S460 first terminal 105 will transmit the multimedia background 320 to second terminal 105 such that multimedia background 320 also becomes the multimedia background 320 of second terminal 115. The first terminal 105 enables the second terminal 115 to interact with multimedia background 320. The video call proceeds at step S470 with each terminal sharing and interacting with the multimedia background 320 simultaneously. The first terminal 105 transmits the multimedia background 320 to the second terminal 115 with an image of the first caller 330 superimposed over multimedia background 320. The first terminal 105 displays multimedia background 320 with an image of the second caller 330 superimposed over the multimedia background 320. Alternatively, both callers can be displayed superimposed over the multimedia background 320.
If the invitation is rejected at step S450, then the video call proceeds at step S470 with only first terminal 105 interacting with the multimedia background 320. First terminal 105 will transmit an image of the multimedia background 320 with first caller 330 superimposed over multimedia background 320 to the second terminal 115. First terminal 105 will display an image of the second caller as in a normal video call.
In some embodiments of the invention, multiple parties may be included in the video call to share and interact with the simultaneous display of multimedia background content. In this embodiment the multiparty call can serve as a conference call with each party interacting with the multimedia content.
In some embodiments of the present invention, a document is selected in step S430. Multimedia background 320 will be displayed as the chosen document and may be interactively controlled. The document may be annotated, highlighted, or resized, for instance, as it would be in a generally available word processing program. Controls 340 will be appropriately displayed to allow for the word processing functionality. Camera 260 detects motions over controls 340 to execute the word processing. Control of the document is not limited, however, to the functions of the controls 340. Camera 260 may also detect gestures and motions over multimedia background 320 for dragging and dropping, moving the document and other functions.
In another embodiment of the present invention, a picture is selected in step S430 as multimedia background 320. The appropriate controls 340 will be displayed on GUI 310 to enable the picture to be annotated, resized, or include other editing features which are controlled by gestures and motions over the controls 340. Additionally, dragging and dropping, moving the image and other functions may be performed through gestures and motions over the multimedia background 320.
In other embodiments of the present invention, a video is selected in step S430 as multimedia background 320. GUI 310 displays appropriate controls 340 for playing, rewind, fast forward and other functions available on a video player. The camera 260 detects motions such that one of the controls 340 may be selected to operate the video accordingly. Annotation or other functionality may also be available on controls 340. Additionally, other functions to manipulate or edit the video directly through gestures and motions over multimedia background 320.
In other embodiments of the present invention, a music file is selected in step S430 as multimedia background 320. GUI 310 displays appropriate controls 340 for playing, rewind, fast forward and other functions available on a music player. The camera 260 detects gestures and motions such that the controls 340 may be selected to manipulate the music accordingly. Additionally, a playlist may be displayed. The list of songs may be scrolled and selected, for example. Scrolling through the list, selections made and other functions controlling the list or the music may also be accomplished by gestures and motions detected over the multimedia background 320 rather than through controls 340.
In other embodiments of the present invention, a map is selected in step S430 as multimedia background 320. GUI 310 displays appropriate controls 340 to size, highlight directions, annotate or perform other functions to control the display of the map. Scrolling, moving and sizing may also be accomplished by gestures and motions detected over the multimedia background 320.
In other embodiments of the present invention, an animation is selected in step S430 as multimedia background 320. Appropriate controls 340 may be made available to manipulate, draw, move, or allow other control functions for the animation, for example. Animation characters and objects may be clipped and dragged onto the multimedia background 320 by motions detected over the background 320.
In other embodiments of the present invention, a game is selected in step S430. GUI 310 may display an appropriate game controller 340. Other callers may share multimedia background 320 to participate in the game together, each manipulating separate controllers but viewing and manipulating the same multimedia background 320.
In another embodiment of the methods of the present invention, a web page is selected in step S430 as multimedia background 320. The appropriate controls 340 will be displayed on GUI 310 to enable the selection of items on the web page, resize the page or include other features. Moving the web page, highlighting areas and other functions may be performed through gestures and motions over the multimedia background.
In another embodiment of the methods of the present invention, on a mobile device a different application is selected in step S430 and displayed as the multimedia background 320. The appropriate controls 340 will be displayed on GUI 310 to enable operation of the application. Some functions may be performed through gestures and motions over the multimedia background.
In other embodiments of the methods of the present invention, multiple callers may participate and be shown in the simultaneous display with each interacting with the selected multimedia background 320. The callers may appear individually or all together as multiple callers 330.
In the embodiments of the present invention the callers 330 may in addition be able to expand or shrink his/her size displayed in GUI 310 by detected gestures and motions over the image of the callers 330.
The embodiments discussed and figures presented are meant only as examples to illustrate the embodiments of the invention and are not intended as a limitation of the invention. Many variations may be employed in embodiments of the invention.
The methods described herein may be implemented in software, hardware, or a combination thereof, in different embodiments. In addition, the order of methods may be changed, and various elements may be added, reordered, combined, omitted or otherwise modified. All examples described herein are presented in a non-limiting manner. Various modifications and changes may be made as would be obvious to a person skilled in the art having benefit of this disclosure. Realizations in accordance with embodiments have been described in the context of particular embodiments. These embodiments are meant to be illustrative and not limiting. Many variations, modifications, additions, and improvements are possible. Accordingly, plural instances may be provided for components described herein as a single instance. Boundaries between various components, operations and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of claims that follow. Finally, structures and functionality presented as discrete components in the example configurations may be implemented as a combined structure or component. These and other variations, modifications, additions, and improvements may fall within the scope of embodiments as defined in the claims that follow.
While the foregoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.