Patent Application
20180020168
The present invention relates to a system and method for transmitting a chroma-keyed video and in particular, to a system and method transmitting a chroma-keyed video between at least two mobile communication devices.
Chroma key technology is commonly used in the arts of television and film production to replace the background of pictures, videos, and motion pictures. Replacing a background of a picture allows a person filmed in a studio to appear to be in a different environment. The film industry uses chroma key processing to produce visual special effects. For example, a background screen can be replaced with cityscapes, or with any other scene that can be captured or else digitally rendered.
Background replacement using chroma key technology is achieved by first filming a live subject, such as a performer or a presenter, against a monochromatic background screen. Typically, the subject is filmed in front of a blue screen or a green screen. The background screen is then captured separately, using the same lighting as is used to capture the presenter. The background screen captured separately serves as a background reference image. During the chroma keying process, background pixels in the original picture are identified by their monochromatic color. Each pixel in the background reference image is identified individually during the keying process. A replacement background is then substituted, pixel by pixel, for he background blue screen. Each background pixel is substituted with a pixel of the new, replacement background.
According to an embodiment of the present invention, there is disclosed a system for transmitting a chroma-keyed video between at least two mobile communication devices. The system includes a first mobile communication device for creating the chroma-keyed video through a software Application and transmitting the chroma-keyed video. A Cloud receives the chroma-keyed video from the first mobile communication device. A second mobile communication device receives the chroma-keyed video from the Cloud. The software Application within the second mobile communication device manipulates the chroma-keyed video received from the Cloud.
According to another embodiment of the present invention, there is disclosed a method for transmitting a chroma-keyed video between at least two mobile communication devices. The method includes creating the chroma-keyed video on a first mobile communication device through a software Application and transmitting the chroma-keyed video. Then, receiving the chroma-keyed video transmitted from the first mobile communication device on a Cloud, followed by receiving the chroma-keyed video from the Cloud on a second mobile communication device. Finally, manipulating the chroma-keyed video received from the Cloud using the software Application within the second mobile communication device.
The structure, operation, and advantages of the present invention will become further apparent upon consideration of the following description taken in conjunction with the accompanying figures (FIGS.). The figures are intended to be illustrative, not limiting. Certain elements in some of the figures may be omitted, or illustrated not-to-scale, for illustrative clarity. The cross-sectional views may be in the form of “slices”, or “near-sighted” cross-sectional views, omitting certain background lines which would otherwise be visible in a “true” cross-sectional view, for illustrative clarity.
In the drawings accompanying the description that follows, both reference numerals and legends (labels, text descriptions) may be used to identify elements. If legends are provided, they are intended merely as an aid to the reader, and should not in any way be interpreted as limiting.
In the description that follows, numerous details are set forth in order to provide a thorough understanding of the present invention. It will be appreciated by those skilled in the art that variations of these specific details are possible while still achieving the results of the present invention. Well-known processing steps are generally not described in detail in order to avoid unnecessarily obfuscating the description of the present invention.
In the description that follows, exemplary dimensions may be presented for an illustrative embodiment of the invention. The dimensions should not be interpreted as limiting. They are included to provide a sense of proportion. Generally speaking, it is the relationship between various elements, where they are located, their contrasting compositions, and sometimes their relative sizes that is of significance.
In the drawings accompanying the description that follows, often both reference numerals and legends (labels, text descriptions) will be used to identify elements. If legends are provided, they are intended merely as an aid to the reader, and should not in any way be interpreted as limiting.
A system 10 and method 12 for transmitting chroma-keyed videos between at least two mobile communication devices, i.e. cell phones of the present invention. As shown in
Referring again to
As further seen in
In general terms, the system 10 for transmitting chroma-keyed videos to mobile communication devices (hereafter “method”) effectively removes the background of a video on a mobile communication in real time (when the first and second mobile communication devices 14 and 16 allows) or in the cloud 20. Once the background of the video is removed, and only the foreground remains, a user is able to choose video or image backgrounds to replace the original background. The original background can be anything, including those with motion, as long as the user does not blend in. The user is also able to remove the background, add a new background and then stream the video live on their mobile device. While on a call, the user can continuously change out the backgrounds, in essence give a presentation from the airport on their mobile device 14 or 16.
First and second users have the ability to add virtual streams of videos and effects during the videos. Further, users can send both SMS messages and messages via the App 18 and 20, as well as video call. First and second users can add multiple messaging via the App 18 and 20. Further, first and second users can share and post videos across social messaging sites.
In step 42, the user may create an account with a personal profile within the Apps 18 and 22. The profile may be created with the Apps 18 and 22. Alternatively, it is within the terms of the embodiment that the profile be created from other social media venues, such as Facebook. The Apps 18 and 22 may have the option of culling provided social media, such as a Facebook page, to create a profile within the Apps.
Further, in step 44, the Apps 18 and 22 determines if the user has affiliated itself with an alternative social network, such as Twitter, Facebook, and Instagram. Assuming all listed social networks are enabled, the content created within the Apps 18 and 22 may be shared to these other forms of social media.
In step 46, the Apps 18 and 22 may search the user's mobile device 14 and 16 to locate any relevant existing media already stored on the device. A library may be created that contains relevant media that the user may access within the Apps 18 and 22. As content is recorded and captured, this media will be added to the library. The library may be stored within the memory of the mobile devices 14 and 16, or alternatively stored within the memory of the servers 32 of the Cloud 20.
In step 48, the Apps 18 and 22 may search amongst the user's contacts, both within the user's mobile device 14 and 16, and within any of the social networks within which the user allows access. A list of contacts is created by the Apps 18 and 22. Further, the user may manually add any additional contacts that he desires with whom he may trade content within the Apps 18 and 22.
In step 54, the user is able to remove the background of the selected video from their mobile device within the Apps 18 and 22. It should be noted that the background can consist of any amount of noise, as long as the user does not blend into the background from which they will need to be segmented. The background is eliminated in real time, such that the user can immediately see the removed portion of the video. However, the user may taps on themselves or the person(s) they want to remain in the video.
In step 56, the user may add visuals and virtual graphics and effects to the video, as well as a stream of product placement. In practice, the user can immediately see the image(s) or video(s) that they have placed behind them. The processing will either take place in real time or the video will render in the cloud, depending on the capability of the mobile device. An algorithm implemented into the Apps 18 and 22 determines whether the mobile device is capable of rendering on the device. Alternatively, the user can override and have the video rendered on cloud. In step 56a, if rendered on cloud, a text message will be sent to user when rendering is complete. If rendered in real time, the video is available immediately.
In step 58, the video that has been modified by the user is sent to a second user. The video may be sent through a variety of channels: In step 58a, the video is sent through the Apps 18 and 22; In step 58b, the video is sent through Email; In step 58c, the video is sent through SMS; In step 58d, the video is sent through any other social network.
In step 60, the user may share the video on any other of his social media site or network.
In general terms, the video being shared by the first user is created utilizing chroma key technology, also known as chroma key compositing. This is a visual effects/post-production technique for compositing (layering) two images or video streams together based on color hues (chroma range). The technique has been used heavily in many fields to remove a background from the subject of a photo or video—particularly the news casting, motion picture and videogame industries. A color range in the foreground footage is made transparent, allowing separately filmed background footage or a static image to be inserted into the scene. The chroma keying technique is commonly used in video production and post-production. This technique is also referred to as color keying or by various terms for specific color-related variants such as green screen, and blue screen. Chroma keying can be done with backgrounds of any color that are uniform and distinct, but green and blue backgrounds are more commonly used because they differ most distinctly in hue from most human skin colors.
A typical usage s the process of separating a person in a video from the background. Using a chroma-keying software application, the picture of the person is separated into its component parts-the live person and the background screen. The chroma-key based processing is a technique of processing video objects, utilizing the technique for layering two images or videos together based on color hues. Typically, this technique is to be utilized to remove a background from the subject of a photo or video.
Chroma-keying is often used in the news casting, movie, and video game fields. A commonly known application for key-based processing is for weather forecasts, in which a background blue screen is replaced with a large weather map. The final picture, including the meteorologist standing in front of a national weather map, is then broadcast for viewing. The meteorologist appears to be standing in front of the map when in fact the meteorologist is being filmed in a studio against a plain blue screen.
The chroma-keying in of the disclosed embodiment does not need a plain blue or green screen as a background. The chroma-keying is able to be done with multiple colors via the touch screen on a mobile communication device. The user is able to select the background colors to be removed from the video screen in real time.
In step 64, the video captured by a video camera is displayed on the screen of the mobile device 14 and 16 in real time.
In step 66, the video captured by the video camera is placed in an intermediary frame buffer of the mobile communication device using the graphics processing unit (GPU) of the mobile communication device 14 and 16.
In step 68, when the user taps on the screen of the communication device 14 and 16, coordinate locations from the screen are translated to the frame buffer.
In step 70, the red, green, and blue pixels are picked up from the frame buffer during the coordinate location translation mentioned herein above. This step is a significant aspect of the preferred embodiment because in the prior art devices, the color space of the image needs to be calibrated to the color space of the chosen color to be extracted. By contrast, in the processing of the present invention, the color is being picked up directly from the frame buffer so no calibration is necessary since the color is coming straight from the image.
In step 72, the color picked from the frame buffer is saved in memory and stored as an array of colors in the software of the present invention.
If desired, the user is able to overlay several other layers of videos above the chroma-keyed video through a sequence of events that occur during a user interaction for chroma keying.
In step 76, the colors are saved in memory and stored as an array are retrieved.
In step 78, a red, green, and blue, Alpha color table is created programmatically to cover all 256×256×256 colors. Note that currently, mobile devices 14 and 16 are incapable of putting the full 256×256×256 color table into memory. Therefore, a trilinear interpolation will be used to calculate interstitial color and alpha values to fill in the areas where memory is sparse. This is a significant portion of the smoothness of the alpha channel translation.
In step 80, as each color is created in the color lookup table, the Euclidean distance is measured from the colors saved in memory to the color in the color lookup table. Note that the Euclidean distance is important in that it determines where and how accurate the replacement occurs. This enables the user to increase or decrease the distance of how much or how little of the values of the adjacent pixels are used in the 3D CLUT.
In step 82, if the color falls inside a predetermined distance, i.e. the tolerance, the color row gets assigned a 0 alpha value.
In step 86, a video is streamed into a GPU buffer for manipulation and creation of an alpha-assigned buffer.
In step 88, a point-wise translation of the pixels into the alpha-assigned buffer is executed using the color table created above. Note that the translation of one color to another is a straight lookup of one value's row to the row in our custom 3D lookup table.
In step 90, the newly alpha-assigned buffer is combined with a separate video or image buffer.
In step 92, the resulting buffer is sent to the frame for viewing.
In step 96, the second user follows the steps of
In step 98, the second user has the ability to modify the video in a similar manner of the first user. The second user can film a chroma-key video of themselves to add themselves into the received video. The received video will become the background video for the second user's new video.
In step 98a, the second user may eliminate the background of the original video.
In step 98b, the second user may tap on any elements within the original video to remain therein.
In step 98c, the second user may add themselves or any other desired elements into the video. For example, the second user may add an animated video stream or text to the original video.
In step 100, the second user may send the modified video back to the original user. The video may be sent through a variety of channels: In step 100a, the video is sent through the Apps 18 and 22. In step 100b, the video is sent through Email. In step 100c, the video is sent through SMS. In step 100d, the video is sent through any other social network.
In step 102, the second user may send the modified video to any other desired parties, either through the Apps 18 and 22 or any other method, such as Email. Any of the additional parties have the ability to modify the contents of the video.
In step 104, the second user may share the video on any other of his social media site or network.
It should be noted that any user may create a live chroma-keyed video stream in the Apps 18 and 22. For example, the first user can live video call a video to the second user, and both users may participate on live video stream calls with chroma-keyed backgrounds and inserted backgrounds behind them. Then, either user has the ability to switch out the backgrounds of their live chroma-keyed video stream video while on the live video call.
Although the invention has been shown and described with respect to a certain preferred embodiment or embodiments, certain equivalent alterations and modifications will occur to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. In particular regard to the various functions performed by the above described components (assemblies, devices, etc.) the terms (including a reference to a “means”) used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component (i.e., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the herein illustrated exemplary embodiments of the invention. In addition, while a particular feature of the invention may have been disclosed with respect to only one of several embodiments, such feature may be combined with one or more features of the other embodiments as may be desired and advantageous for any given or particular application.
This application claims the benefit of U.S. Provisional Patent Application No. 62/362,645 filed on Jul. 15, 2016 which is incorporated in its entirety herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 62362645 | Jul 2016 | US |
