The present invention relates generally to video calling and in particular, to a method and apparatus for morphing a user during a video call.
With the increasing availability of on-line communication applications and broadband internet access, communicating with friends and family using real-time audio and video is becoming more and more popular. Unlike audio-only communication, where the appearance of the participant, or the participant's environment, is not at all important, the participant in a video conference often makes a great effort to look his or her best prior to commencing it. For example, women may feel compelled to apply make-up before appearing on camera, just as they would before going out for any other social activities.
Many techniques exist for morphing a user during a video call. For example, US Pub. No. 2006/0268101, S
A problem exists, however, in providing a user a simple technique to choose how to morph their image. With an infinite number of possible changes to a user's image, it may be very difficult for a user to choose how they appear in a video call. Additionally, in order to enhance the user experience, a simple technique for changing appearances may be desired. For example, a user may wish to appear differently, depending upon who they are talking to. More particularly, it would be beneficial to a user if they had a simple technique to choose how to morph their image. Therefore a need exists for a method and apparatus for morphing a user during a video call that allows a user to easily choose how they are morphed, and to easily change from one type of morphing to another type.
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions and/or relative positioning of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of various embodiments of the present invention. Also, common but well-understood elements that are useful or necessary in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various embodiments of the present invention. It will further be appreciated that certain actions and/or steps may be described or depicted in a particular order of occurrence while those skilled in the art will understand that such specificity with respect to sequence is not actually required. Those skilled in the art will further recognize that references to specific implementation embodiments such as “circuitry” may equally be accomplished via replacement with software instructions executing either on general purpose computing apparatus (e.g., CPU) or specialized processing apparatus (e.g., DSP). It will also be understood that the terms and expressions used herein have the ordinary technical meaning as is accorded to such terms and expressions by persons skilled in the technical field as set forth above except where different specific meanings have otherwise been set forth herein.
In order to alleviate the above-mentioned need, a method and apparatus for changing an appearance of a user during a video call is provided herein. Prior to making a call, a user identifies several images or techniques that can be used as morphing templates and also identifies a context where each template/technique will be used. During a video call, a morphing template/technique is then chosen based on a context of the call (e.g., time, place, caller identification, etc.). During the video call, the user's image will be morphed based on the chosen template or technique. Because templates or techniques used for morphing will be easily changed based on a context of the call, the user will be provided with a simple method and apparatus to for morphing their image.
The present invention encompasses a method for morphing a user during a video call. The method comprises the steps of determining a context, determining a morphing technique based on the context, and morphing the user based on the context and the morphing technique.
The present invention additionally encompasses a method comprising the steps of receiving an image from a user, receiving from the user, a context for the image, and storing the image and the associated context.
The present invention additionally encompasses an apparatus comprising context aware circuitry determining a context, a processor determining a morphing technique based on the context and morphing the user based on the context and the morphing technique.
Turning now to the drawings, where like numerals designate like components,
As shown system/device 100 includes camera 101, which may be a still camera or a video camera. Processor 102 preferably comprises an Nvidia Tegra 2 dual-core processor, and is configured to perform the necessary steps to appropriately morph a users image as described below. In this embodiment, processor 102 communicates with display 104 and communication circuit 106. Display 104 preferably comprises a 1280×800, 10.1 inch touch screen display. Display unit 104 also serves as a graphical user interface (GUI) used to receive input from a user. Communication circuit 106 comprises one or more of a network interface card, cellular modem, or Wi-Fi modem incorporating 802.11a/b/g/n. Communication circuitry 106 communicates to other network devices (e.g., device 110) through network 115. Finally, database 108 preferably comprises random access memory such as a Micro SD or on board flash storage space.
During operation of device 100, a user positions him or herself in front of camera 101. Camera 101 captures an image or video of the user and passes this image to processor 102. Processor 102 properly encodes the image/video and passes the image/video through network 115 to network device 110. Additionally, processor 102 may receive an image/video from network device 110 via circuitry 106. This image/video is sent to display unit 104 along with the user's captured image so that both the user and the image/video from device 110 can be displayed for viewing.
During a normal video call, a user of device 100 will have an ability to morph their image, sending the morphed image to device 110. Many techniques exist for morphing a user during a video call, and any technique may be utilized without varying from the scope of the present invention. However, a preferred technique for morphing an image of a user is described in U.S. Pat. No. 7,742,624, PERSPECTIVE IMPROVEMENT FOR IMAGE AND VIDEO APPLICATIONS, incorporated by reference herein. As described in the '624 patent, a user's image is morphed such that points in the image match corresponding points in one or more stored templates.
More particularly, processor 102 uses templates stored in database 108. The templates comprise several views of the user. An image is captured by camera 101 and a number of control points or features are identified in the captured image. The captured image is then warped or morphed to match each template. While the '624 patent only described the warping of a face, the present invention goes beyond simply warping the face to also modify the color of the skin, hairstyle, expression, and possibly the clothes worn based on a comparison with the template. In a particular embodiment, the clothes morphed onto the user may have a color that is chosen randomly so that the user does not look like they are wearing similar clothes every time they participate in a video call.
As discussed above, a problem exists in providing a user a simple technique to choose how to morph their image. With an infinite number of possible changes to a user's image, it may be very difficult for a user to choose how they appear in a video call. Additionally, in order to enhance the user experience, a simple method and apparatus for changing appearances may be desired. For example, a user may wish to appear differently, depending upon who they are talking to. Therefore, it would be beneficial to a user if they had a simple technique to choose how to morph their image.
In order to address this issue, context aware circuitry 109 is provided. Context-aware circuitry 109 may comprise any device capable of generating a current context. For example, context-aware circuitry 109 may comprise a GPS receiver capable of determining a location of the user device. Alternatively, circuitry 105 may comprise such things as an electronic clock, an electronic calendar, environmental sensors (e.g. a thermometer capable of determining an ambient temperature, humidity, . . . , etc.), an accelerometer, a barometer, speech recognition circuitry, a user's electronic calendar, short-range communication circuitry (e.g., Bluetooth™ circuitry, zigbee, wifi, etc) to determine what other electronic devices are near, biosensors such as a heart-rate monitor, galvanic skin response, pulse, oxygen saturation, an activity and motion detector (accelerometer), activity impact (pressure sensors), a blood-pressure monitor, respiratory sensors, blood sugar monitor, or a hydration sensor.
During operation a user will identify several images that will be used as templates. For example, a user may identify an image of them in formal attire, casual attire, pajamas, with makeup, without makeup, winter clothing, summer clothing, images that they feel make them look professional, sultry, fun loving, serious, happy, tired, etc. The user will also determine a context for each image to be used as a morphing template. (As mentioned above, in an alternate embodiment, images of the user are not provided. The user simply identifies how they are to be morphed based on an associated context). For example, a user may wish for them to be morphed into formal attire when speaking to their boss. In a similar manner, a user may wish to be morphed into gym attire between certain hours of the day.
Processor 102 will retrieve a current context, and automatically choose an appropriate picture for the user. Some use cases are identified below:
a clock;
a calendar;
location circuitry
GPS circuitry;
a thermometer;
a radio;
caller identification circuitry;
speech recognition circuitry; and
short-range communication circuitry.
Using the above context-aware circuitry, the current context may then comprise one or more of the following:
a time of day;
a time of year;
a location;
a temperature;
localized weather information;
country of origin of caller;
who is speaking;
what devices are near; and
a caller identification.
Using the context, processor 102 then determines a morphing technique based on the context. In this particular embodiment, the morphing technique is determined by accesses database 108 and determining an appropriate image template associated with the context (e.g., an appropriate picture or the user) for morphing (step 403). As discussed above, the image template chosen is based on the current context. The logic flow then continues to step 405 where image morphing takes place based on the context and the morphing technique. In this particular embodiment the image or video received from camera 101 and the appropriate image template are utilized for morphing as described above. Processor 102 performs the morphing operation. In a second embodiment of the present invention the image template or morphing technique is sent to remote processor 201 along with the current image or video. The remote processor 201 then performs the morphing operation.
The morphing technique may comprise one or more of the following to change the appearance of the user or the user's background:
morphing clothes onto the user;
morphing a particular color of clothes onto the user;
morphing makeup onto the user;
morphing clothes for a particular season onto the user;
morphing a particular emotional state onto the user;
morphing the user into a particular background;
morphing a particular skin color onto the user; and
morphing a head cover onto the user.
While the invention has been particularly shown and described with reference to a particular embodiment, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention. For example, while images of the user were used for morphing purposes, one of ordinary skill in the art will recognize that other morphing techniques may be used without varying from the scope of the invention as long as the morphing is associated with a context. For example, system 100 may simply determine a context, receiving an image or video of a user to be used in a video call. The context then can be associated with a morphing technique, and the image of the user then can be morphed based on the context and associated morphing technique. No image template may be necessary when morphing (See for example, US Pub. No. 2006/0268101). For example, a user may simply wish to be morphed into pajamas after a certain time of evening, or to have makeup placed on them self when a certain individual calls without providing an image of them in pajamas or makeup. It does not matter what techniques are used for morphing.
It should also be noted that multiple images and/or morphing techniques may be utilized simultaneously. For example, if the current context is “winter time” and “talking to boss”, a user may be morphed into formal winter attire.
Finally, while the above described morphing dealt particularly with a user's image, the same technique may be utilized to change the background of an image. For example, a user may wish to have their background appear as a workplace environment when a placing a work-related call. It is intended that such changes come within the scope of the following claims: