Patent Application
20140327730
The field of the invention relates generally to methods, media, and devices for providing visual resources of video conference participants.
Video conferencing systems are in wide-spread use and enable participants who may be in vastly different geographical locations to have face-to-face meetings without the need to travel great distances. When participating in such conferences, it may be useful to view the video of the participants throughout the conference, and to be able to determine who is on the conference at any time during the conference.
An embodiment of the invention may therefore comprise a method for presenting an aesthetic image, the method comprising analyzing an audio track of a user in a conference to determine when the user is an active speaker, when the user is an active speaker, analyzing a speech signal of the audio track to identify aesthetic phonemes for when the user is an active speaker, and determining at least one image of the user corresponding to the aesthetic phonemes.
An embodiment of the invention may further comprise A system for presenting an aesthetic image, the system comprising an audio analysis tool enabled to analyze an audio track of a conference to determine when a participant to a conference is an active speaker, analyze a speech signal of the audio track to identify aesthetic phonemes of the active speaker, and determine at least one image of the active speaker corresponding to the aesthetic phonemes.
Methods, media, and devices for providing visual resources of video conference participants are provided. An optimized image snapshot from a captured sequence of persons participating in a conference may be generated. It is understood that an optimized image may be a video frame in which a captured object, such as a person or persons, is fully visible in the frame. A fully visible capture may be one in which there is no cut out, or that is not occluded. The person of interest so understood that the perfect capture may not be attainable but that more preferred captures may be obtained.
High efficiency, low complexity audio analysis tools may be utilized to capture an audio track. The extraction of an image from a video sequence of a person may be target based on when it is known that the person is speaking. The propensity for an active speaker, who is engaged in the conference and aware that attention is focused on them, may be to attempt to optimize their appearance by looking at the camera and positioning themselves in front of the camera so as to not be obscured, is leveraged by the current invention.
Phonemes can also be analyzed to obtain a preferable capture. The audio track can be utilized in order to identify aesthetic facial expressions. The system of the invention may identify phonemes where the person actively speaking has lips and face positioned in an aesthetic manner. It is understood that a phoneme analysis may be less computationally complex than image analytics methodologies.
A captured image may be associated with a particular participant for which it was taken. There are many uses for which a captured image may be employed. A captured image may act as a representative image of a conference participant in replacement of a video stream. This may be such were the participant has muted his video or network conditions do not, at a particular time, provide optimal video quality. A captured image may act as a representative image of a conference participant in offline archiving systems. The captured image may identify a speaker in for those segments of the archived conference when that participant was actively speaking. A captured image may act as a representation of a conference participant in a system participant roster. A captured image may act as a visual queue for different events which may occur during a conference. Examples of these events are when a participant begins presenting, or when a participant sends a chat message, or “raises their hand”, etc. It is understood that the above are examples of uses of captured images and those skilled in the art may use such captured images in many other circumstance.
A system of the invention is enabled to receive a set of synchronized audio and video streams. The system will extract aesthetic video snapshots, or images, of participants to a conference. The audio and video streams of a particular user may be time-stamped by the system, such as by a set of synchronized capture clocks.
The system analyzes the audio track of a participant to the conference. When the participant is an active speaker, the system will analyze the speech signal and segment the audio by phonemes. It is understood that such an analysis can be performed by a VAD (Voice Activity Detection) algorithm.
In an embodiment of the invention, the system may classify the phonemes into a number of different sets. A first set may be phonemes that are associated with aesthetically pleasing face expressions. These phonemes are generally referred to as “aesthetic phonemes”. Other phonemes may be categorized into one or more other sets. When analyzing phonemes, the system may select periods just before or after the phoneme, such as a period of silence. When analyzing phonemes, the system may classify by a sub component of a particular phoneme. This may be such as a consonant or a vowel. The system may classify by a set or sequence of phonemes. The system may further refine an image selection by alternative audio parameters. These alternative audio parameters may be, for example, an estimation of the audio direction which may be indicated by the use of multiple microphones. The audio level, background noise estimation, cross talk detection, or other audio parameters, may be used for refinement.
The process at step 250, of determining an optimal image, may be performed by a highly computational intensive algorithm. While use of audio and phonemes reduces the computational intensity of real time analysis, these highly computational intensive algorithms, which may be utilized for step 25, do not need to be run in real time.
At step 260 of
The foregoing description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed, and other modifications and variations may be possible in light of the above teachings. The embodiment was chosen and described in order to best explain the principles of the invention and its practical application to thereby enable others skilled in the art to best utilize the invention in various embodiments and various modifications as are suited to the particular use contemplated. It is intended that the appended claims be construed to include other alternative embodiments of the invention except insofar as limited by the prior art.
