Patent Application
20120200659
The present invention relates generally to conferencing and, more specifically, to displaying unseen participants in a videoconference.
Videoconferencing may be used to allow two or more participants at remote locations to communicate using both video and audio. Each participant location may include a videoconferencing system for video/audio communication with other participants. Each videoconferencing system may include a camera and microphone to collect video and audio from a first or local participant to send to one or more other (remote) participants. Each videoconferencing system may also include a display and speaker to reproduce video and audio received from remote participant(s).
During a videoconference each participant may view one or more other participants in a videoconference. However, in some videoconferencing layouts, the participant may not be able to view every participant in the videoconference. Accordingly, improvements in videoconferences are desired.
Various embodiments are presented of a system and method for displaying unseen participants in a videoconference.
Initially, a videoconference may be established among a plurality of videoconferencing endpoints. Each videoconferencing endpoint may provide videoconferencing capabilities for one or more participants of the videoconference. Thus, the videoconference may be established for a plurality of participants.
During the videoconference, each videoconferencing endpoint may provide a videoconference layout on a display of the videoconferencing endpoint. The videoconference layout may include a plurality of different portions. Each portion of the videoconference layout may include one or more participants of the videoconference (e.g., at a respective other videoconference location). Thus, the videoconference layout may display a first subset of the plurality of participants. However, the videoconference layout may not include a second subset of the plurality of participants. Thus, a first participant at a first location may only view the first subset of participants in the videoconference layout, but may not be able to view the second subset of participants in the videoconference layout. As an example, the number of participants in the videoconference may be too large to all fit in the layout, and hence only a subset (first subset) of the participants may be displayed.
Accordingly, user input to view the second subset of the plurality of participants may be received from the first participant. In response to the user input, the second subset may be displayed in a first portion (e.g., a predetermined portion) of the videoconference layout. For example, different groups of one or more participants of the second subset may be temporarily displayed in the first portion of the layout over a plurality of iterations. Thus, at a first time (e.g., in a first iteration), a first group (e.g., one or more) of the second subset of participants may be displayed in the first portion. At a second time, a second group of the second subset of participants may be displayed in the first portion, and so on.
Separate user input may be received for displaying each of the different groups. For example, the separate user input may be provided at each iteration to change from one group to the next. Alternatively, each different group may be displayed automatically without user input to change each displayed group. For example, the groups may be displayed periodically, e.g., on a timed interval (e.g., every 1 second, 2 seconds, 5 seconds, 10 seconds, etc.). Thus, in this latter embodiment, the user may select an option to display unseen participants, and the videoconference system may automatically “cycle through” display of the unseen participants at a timed interval on a predetermined portion of the display.
The first portion of the videoconference layout may be a least prominent portion of the videoconference layout. For example, the least prominent portion may be a smaller portion of the videoconference layout. As another example, the least prominent portion may be in a corner of the videoconference layout (e.g., a top left or right corner). In some embodiments, the videoconference layout may have an “active speaker portion” and a “continuous presence portion”. The active speaker portion may be used to display a current active speaker or prominent participant in the videoconference. The continuous presence portion may include one or more subportions where other participants are shown. In layouts including these two types of portions, the least prominent portion may be in the continuous presence portion (e.g., in one of the subportions of the continuous presence portion). In one embodiment, a participant in a subportion of the continuous presence portion may be promoted to the active speaker portion. Accordingly, in embodiments where that subportion is no longer used, it may be used for displaying the unseen participants of the videoconference. Thus, that vacated subportion may be the “least prominent portion” of the videoconference layout. In another embodiment, the “least prominent portion” of the videoconference layout may be the portion of the videoconference that is used for a participant who is currently being displayed but has spoken the least in the videoconference (e.g., over the whole of the videoconference or over a recent time frame, such as 10 seconds, 30 seconds, 1 minute, 2 minutes, 5 minutes, etc.). Thus, the “least prominent portion” of the videoconference layout may refer to the portion of the videoconference layout that is least important to the participant viewing the videoconference layout, which may be based on portion position, whether the portion is in use, based on the participant (e.g., the participant's activity level) in the portion, etc.
After the second subset of participants has been displayed, the videoconference layout may then revert to display the first subset of the plurality of participants without displaying the second subset of the plurality of participants. Note that this may occur after all of the second subset of the participants has been displayed or when a subset of the second subset of participants have been displayed. For example, the method may return to displaying only the first subset of the plurality of participants in response to not receiving input for a predetermined period of time. As another example, the method may return to displaying the first subset of the plurality of participants after the user has seen all of the second subset of participants, e.g., after all the groups have been seen. Further, the method may return to displaying the first subset of the plurality of participants when the user provides input to return to that display, as desired.
A better understanding of the present invention may be obtained when the following detailed description is considered in conjunction with the following drawings, in which:
While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present invention as defined by the appended claims. Note the headings are for organizational purposes only and are not meant to be used to limit or interpret the description or claims. Furthermore, note that the word “may” is used throughout this application in a permissive sense (i.e., having the potential to, being able to), not a mandatory sense (i.e., must). The term “include”, and derivations thereof, mean “including, but not limited to”. The term “coupled” means “directly or indirectly connected”.
The following is a glossary of terms used in the present application:
Memory Medium≧Any of various types of memory devices or storage devices. The term “memory medium” is intended to include an installation medium, e.g., a CD-ROM, floppy disks, or tape device; a computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; or a non-volatile memory such as a magnetic media, e.g., a hard drive, or optical storage. The memory medium may comprise other types of memory as well, or combinations thereof. In addition, the memory medium may be located in a first computer in which the programs are executed, or may be located in a second different computer which connects to the first computer over a network, such as the Internet. In the latter instance, the second computer may provide program instructions to the first computer for execution. The term “memory medium” may include two or more memory mediums which may reside in different locations, e.g., in different computers that are connected over a network.
Carrier Medium—a memory medium as described above, as well as a physical transmission medium, such as a bus, network, and/or other physical transmission medium that conveys signals such as electrical, electromagnetic, or digital signals.
Computer System—any of various types of computing or processing systems, including a personal computer system (PC), mainframe computer system, workstation, network appliance, Internet appliance, personal digital assistant (PDA), smart phone, television system, grid computing system, or other device or combinations of devices. In general, the term “computer system” can be broadly defined to encompass any device (or combination of devices) having at least one processor that executes instructions from a memory medium.
FIGS. 1 and 2—Exemplary Participant Locations
In some embodiments, the participant location may include camera 104 (e.g., an HD camera) for acquiring images (e.g., of participant 114) of the participant location. Other cameras are also contemplated. The participant location may also include display 101 (e.g., an HDTV display). Images acquired by the camera 104 may be displayed locally on the display 101 and/or may be encoded and transmitted to other participant locations in the videoconference. In some embodiments, images acquired by the camera 104 may be encoded and transmitted to a multipoint control unit (MCU), which then provides the encoded stream to other participant locations (or videoconferencing endpoints)
The participant location may further include one or more input devices, such as the computer keyboard 140. In some embodiments, the one or more input devices may be used for the videoconferencing system 103 and/or may be used for one or more other computer systems at the participant location, as desired.
The participant location may also include a sound system 161. The sound system 161 may include multiple speakers including left speakers 171, center speaker 173, and right speakers 175. Other numbers of speakers and other speaker configurations may also be used. The videoconferencing system 103 may also use one or more speakerphones 105/107 which may be daisy chained together.
In some embodiments, the videoconferencing system components (e.g., the camera 104, display 101, sound system 161, and speakerphones 105/107) may be coupled to a system codec 109. The system codec 109 may be placed on a desk or on the floor. Other placements are also contemplated. The system codec 109 may receive audio and/or video data from a network, such as a LAN (local area network) or the Internet. The system codec 109 may send the audio to the speakerphone 105/107 and/or sound system 161 and the video to the display 101. The received video may be HD video that is displayed on the HD display. The system codec 109 may also receive video data from the camera 104 and audio data from the speakerphones 105/107 and transmit the video and/or audio data over the network to another conferencing system, or to an MCU for provision to other conferencing systems. The conferencing system may be controlled by a participant or user through the user input components (e.g., buttons) on the speakerphones 105/107 and/or input devices such as the keyboard 140 and/or the remote control 150. Other system interfaces may also be used.
In various embodiments, the codec 109 may implement a real time transmission protocol. In some embodiments, the codec 109 (which may be short for “compressor/decompressor” or “coder/decoder”) may comprise any system and/or method for encoding and/or decoding (e.g., compressing and decompressing) data (e.g., audio and/or video data). For example, communication applications may use codecs for encoding video and audio for transmission across networks, including compression and packetization. Codecs may also be used to convert an analog signal to a digital signal for transmitting over various digital networks (e.g., network, PSTN, the Internet, etc.) and to convert a received digital signal to an analog signal. In various embodiments, codecs may be implemented in software, hardware, or a combination of both. Some codecs for computer video and/or audio may utilize MPEG, Indeo™, and Cinepak™, among others.
In some embodiments, the videoconferencing system 103 may be designed to operate with normal display or high definition (HD) display capabilities. The videoconferencing system 103 may operate with network infrastructures that support T1 capabilities or less, e.g., 1.5 mega-bits per second or less in one embodiment, and 2 mega-bits per second in other embodiments.
Note that the videoconferencing system(s) described herein may be dedicated videoconferencing systems (i.e., whose purpose is to provide videoconferencing) or general purpose computers (e.g., IBM-compatible PC, Mac, etc.) executing videoconferencing software (e.g., a general purpose computer for using user applications, one of which performs videoconferencing). A dedicated videoconferencing system may be designed specifically for videoconferencing, and is not used as a general purpose computing platform; for example, the dedicated videoconferencing system may execute an operating system which may be typically streamlined (or “locked down”) to run one or more applications to provide videoconferencing, e.g., for a conference room of a company. In other embodiments, the videoconferencing system may be a general use computer (e.g., a typical computer system which may be used by the general public or a high end computer system used by corporations) which can execute a plurality of third party applications, one of which provides videoconferencing capabilities. Videoconferencing systems may be complex (such as the videoconferencing system shown in
The videoconferencing system 103 may execute various videoconferencing application software that presents a graphical user interface (GUI) on the display 101. The GUI may be used to present an address book, contact list, list of previous callees (call list) and/or other information indicating other videoconferencing systems that the user may desire to call to conduct a videoconference.
Note that the videoconferencing system shown in
In 402, a videoconference may be initiated or performed between a plurality of participants at respective participant locations. More specifically, the conference may be initiated between a first participant using a first endpoint (e.g., at a first participant location) and a plurality of other participants using other endpoints (e.g., at other participant locations). Thus, endpoints may be similar to those described above regarding
During the videoconference, in 404, each videoconferencing endpoint may provide a videoconference layout on a display of the videoconferencing endpoint. The videoconference layout may be selected by the participant or may be selected automatically (e.g., based on the number of participants or endpoints in the videoconference). For example, the participant(s) at a location may select a desired layout from a plurality of available layouts and/or the endpoint may automatically select a layout that is appropriate the number of participants.
The videoconference layout may include a plurality of different portions. Each portion of the videoconference layout may include one or more participants of the videoconference (e.g., at a respective other videoconference location). Thus, the videoconference layout may display a first subset of the plurality of participants. However, the videoconference layout may not include a second subset of the plurality of participants. Thus, a first participant at a first location may only view the first subset in the videoconference layout, but may not be able to view the second subset in the videoconference layout.
In some embodiments, the videoconference layout may include a first portion and a second portion. The first portion may be used to display images of a participant who is the current “active speaker” (e.g., the participant who is doing the majority of the current talking in the videoconference). Accordingly, the first portion may be referred to as the “active speaker” or “prominent” portion of the videoconference layout. However, it should be noted that the first portion may be used for other purposes than for the active speaker, as desired (in these cases, the method may still operate as described). The second portion may be used to display images of other participants in the videoconference (e.g., all or a subset of the remaining participants). The second portion may be referred to as the “continuous presence” portion of the videoconference layout. The “continuous presence” portion may allow the participant to view participants who are not the active speaker. This second portion may include a plurality of subportions, e.g., one for each other participant of the videoconference. Note that in the descriptions herein, the images of the participants being displayed in the layout may be referred to simply as ‘the participants being displayed in the layout’, i.e., the participants which the images are intended to represent; it should accordingly be understood that this usage does not refer to literal participants. Additionally, other types of layouts are envisioned which do not include active speaker portion(s).
In 406, user input to view the second subset of the plurality of participants may be received from the first participant. For example, the user input may be received to a user input device such as a remote control device, a keyboard, a mouse, an attached computer, etc. The user input may indicate that the participant wishes to view the unseen participants of the second subset. In one embodiment, the user input may be selected via a menu displayed on the videoconference display. Accordingly, the participant may select a “Who's in the Room” feature or menu item in order to view the second subset. In some embodiments, this selection may only be available while the participant is in certain mode(s), e.g., when a speaker mode is enabled.
In response to the user input, in 408, the second subset may be displayed in a first portion of the videoconference layout. For example, different groups of one or more participants of the second subset may be temporarily displayed in the first portion of the layout over a plurality of iterations. Thus, at a first time (e.g., in a first iteration), a first group of the second subset of participants may be displayed in the first portion. At a second time, a second group of the second subset of participants may be displayed in the first portion. As used herein “temporary display” of each of the groups of participants may refer to displaying the group for a long enough period of time that the viewing participant is able to identify or view each participant in the currently displayed group. It may be a length of time to allow for the viewing participant to judge the other participants' reaction to what is being discussed (or what was just discussed). This length of time may be as short as 1 or 2 seconds, but may also be longer, e.g., 5 seconds, 10 seconds, 30 seconds, etc.
Separate user input may be received for displaying each of the different groups. For example, the separate user input may be provided at each iteration to change from one group to another. In one embodiment, the participant may utilize a user input device, such as a remote, to provide the user input to change the currently displayed group of the second subset of participants. For example, the participant may press an up or down arrow (e.g., FECC buttons or 2 and 8 for DTMF buttons) to cycle to a previously displayed group or to the next group of unseen participants. Other buttons and user input methods are envisioned.
Alternatively, each different group may be displayed automatically without user input to change each displayed group. For example, the groups may be displayed periodically, e.g., on a timed interval, such as every 1 second, 2 seconds, 3 seconds, 5 seconds, 10 seconds, etc. In further embodiments, the time interval may correspond to the number of participants in the group. For example, a single participant may only be shown for 2 seconds while two participants may be shown for 3 or 4 seconds. As used herein “groups of participants” may refer to one or more participants, e.g., depending on how many are present at each different participant location.
The first portion of the videoconference layout may be a least prominent portion of the videoconference layout. For example, the least prominent portion may be a smaller portion of the videoconference layout. As another example, the least prominent portion may be in a corner of the videoconference layout (e.g., a top left or right corner). As indicated above, in some embodiments, the videoconference layout may have an “active speaker portion” and a “continuous presence portion”. In layouts including these two types of portions, the least prominent portion may be in the continuous presence portion (e.g., in one of the subportions of the continuous presence portion). In one embodiment, a participant in a subportion of the continuous presence portion may be promoted to the active speaker portion. Accordingly, in embodiments where that subportion is no longer used, it may be used for displaying the unseen participants of the videoconference. Thus, that subportion may be the “least prominent portion” of the videoconference layout. In another embodiment, the “least prominent portion” of the videoconference layout may be the portion of the videoconference that is used for a participant who is currently being displayed but has spoken the least in the videoconference (e.g., over the whole of the videoconference or over a recent time frame, such as 10 seconds, 30 seconds, 1 minute, 2 minutes, 5 minutes, etc.). Thus, the “least prominent portion” of the videoconference layout may refer to the portion of the videoconference layout that is least important to the participant viewing the videoconference layout, which may be based on portion position, whether the portion is in use, or based on the participant (e.g., the participant's activity level) in the portion.
In 410, after the second subset of participants has been displayed, the videoconference layout may continue to display the first subset of the plurality of participants without displaying the second subset of the plurality of participants. Note that this may occur after all of the second subset of the participants have been displayed or when a subset of the second subset of participants have been displayed. For example, the method may return to displaying only the first subset of the plurality of participants in response to not receiving input for a predetermined period of time. As another example, the method may return to displaying the first subset of the plurality of participants after the user has seen all of the second subset of participants, e.g., after all the groups have been seen. Further, the method may return to displaying the first subset of the plurality of participants when the user provides input to return to that display, as desired.
Embodiments of a subset or all (and portions or all) of the above may be implemented by program instructions stored in a memory medium or carrier medium and executed by a processor.
In some embodiments, a computer system at a respective participant location may include a memory medium(s) on which one or more computer programs or software components according to one embodiment of the present invention may be stored. For example, the memory medium may store one or more programs that are executable to perform the methods described herein. The memory medium may also store operating system software, as well as other software for operation of the computer system.
Further modifications and alternative embodiments of various aspects of the invention may be apparent to those skilled in the art in view of this description. Accordingly, this description is to be construed as illustrative only and is for the purpose of teaching those skilled in the art the general manner of carrying out the invention. It is to be understood that the forms of the invention shown and described herein are to be taken as embodiments. Elements and materials may be substituted for those illustrated and described herein, parts and processes may be reversed, and certain features of the invention may be utilized independently, all as would be apparent to one skilled in the art after having the benefit of this description of the invention. Changes may be made in the elements described herein without departing from the spirit and scope of the invention as described in the following claims.
