The present invention relates to a conferencing system and, more particularly, to a computer-based conferencing system enabling managed communication between the conference participants.
Many business activities are performed by teams of individuals that may be widely dispersed geographically. For example, product design and manufacturing are commonly performed by teams having members who are often located in facilities spread around the globe and/or who may be in transit between locations. If a decision is to be made concerning the project it may be necessary to quickly gather input and consensus from the members of the team regardless of their physical remoteness. Modern communication technology enables individuals to communicate over long distances and from remote locations. Conferencing systems facilitate communication between a plurality of remotely located users or conferees by allowing multiple users to communicatively interconnect with each other either directly as peers or by interconnecting with a central server that is interconnected to the other participants in the conference. Computer-based conferencing systems commonly provide for audio and video input from each of the conferees. In addition, a conferencing system may provide file sharing enabling conferees to view and edit files, including engineering drawings and spreadsheets, that are part of the team's project.
One goal of a conferencing system is to connect a plurality of remotely located conferees and enable communication between the conferees as if the conferees were sitting at the same conference table. However, as the number of conference locations, sources of video, audio or other data input to the conference, increases, the ability of a group to communicate effectively in a conference decreases. For example, a separate transport stream, commonly comprising audio, video and textual data streams, is required for each conference location. The bandwidth required for a conference is a function of the number of conference locations and the bandwidth required for each location. Each user may require around 150 Kbs or more for a video stream. Hundreds of mega-bits per second may be required to transmit the data streams for a conference with a large number of conference locations, effectively precluding a video conference if there are many conference locations or if a conference location is served by limited bandwidth.
In addition, in a face-to-face conference, the conferees can assimilate a number of sensory inputs from fellow conferees and can selectively focus attention on one or more of the inputs. However, the typical computer-based conferencing interface device, for example a personal computer, is incapable of rendering more than a few transport data streams at a time. Unless a server merges the data streams originating at multiple conference locations into a single stream or switches between streams, remote participants in a conference may be unaware of inputs by other conferees. There is generally no control over communications from the several conference locations and a participant has difficulty directing his/her conference interface device to focus attention on a selected input. As a result, the conference participant's experience is often less than optimal when there are many conferees.
What is desired, therefore, is a conferencing system that enables the individual members of a large group of participants in a conference to effectively communicate and interact with other remotely located conferees.
Referring in detail to the drawings where similar parts are identified by like reference numerals, and, more particularly to
A user's interface with the conferencing system typically comprises a personal computer but may comprise other types of computing devices, such as, personal digital assistants (PDA) or mobile telephones, that are capable of connecting to and communicating over a computer network as a node or client in a peer-to-peer, server mediated or client mediated network. Referring to
The conferencing interface device may also include a plurality of attached input/output (I/O) devices and other peripheral devices. Input devices may include an audio capture device 60, such as a microphone, a video capture device 62, such as a digital video camera, a keyboard 64 for textual input and a pointing device 66, such as a mouse, a trackball or a touch screen display. Output devices commonly include a display 68 for rendering video, text and other visual information, headphones 70 or speakers for audio output and a printer 72 or plotter to render hard copies of documents and images. Under the control of the CPU, data is transmitted to and received from each of the attached devices over a communication channel connected to the system bus 58. Typically, each device is attached to the system bus by way of an adapter, such as an interface adapter 74 providing an interface between the keyboard 64 and the system bus. Likewise, a display adapter 76 commonly provides an interface between the display 68 and a video card 78 that processes video data under the control of the CPU and is communicatively connected to the system bus. The printer 72 and similar peripheral devices are typically connected to the system bus by one or more input-output (I/O) adapters 80 commonly including an analog to digital converter (ADC) 82 and a digital to analog converter (DAC) 84.
The conferencing interface device 22 also includes communication facilities for communicatively interconnecting with other data processing devices including the conferencing server and other conferencing interface devices. These facilities may include a network interface card 86 or circuitry, and/or one or more modems 90 including ports 92 for connection to a telephone system or a wired network. In addition, the conferencing interface may be equipped with a wireless data transceiver 88 for wireless connection to the communication network. The communications facilities provide communication links 26 enabling connection to and communication with one or more computer networks such as a wide area network (WAN), commonly including the Internet, or a local area network (LAN), as appropriate at the user's location. The interface device 22 is preferably constructed in a manner that routes the packets that are received to the other parties in a conference. In this manner there is little processing that is required for the packets. The server may permit and/or blocking of users based upon any criteria.
The conferencing interface device 22 also comprises a conferencing application program 30 that is typically stored in the device's mass storage. In a preferred embodiment, the conferencing application 30 is a Java® language-based program which runs in the operating system 32 of the conferencing interface device. The Java Runtime Environment 34, the software required to run a Java language application, initiates the conferencing application program and passes application specific arguments between the application program and the operating system. The conferencing application 30 initiates and configures a media framework 36, a program or programs that enable capture, playback, streaming and transcoding of data streams for audio, video and other time-based media. The media framework captures audio and video data from the audio 60 and the video capture devices 62 and encodes the data in formatted data packets for transmission over the computer network. In the preferred embodiment the media framework comprises the Java Media Framework (JMF) but other methods and protocols could be used to capture audio and video, encode, decode and render the audio and video content of a conference.
Referring to
The media framework preferably formats the data packets of the captured audio and video data streams according to the Real-Time Transport Protocol (RTP). RTP is an Internet protocol for transporting real-time data, including audio and video. RTP and RTCP provides end-to-end network transport functions; such as timing, reconstruction, loss detection, security and content identification, suitable for applications, such as audio and video, comprising real-time data transmitted over unicast or multicast communication networks. The data transport function is augmented by a control protocol (RTCP), that allows monitoring of data delivery in a manner scalable to large multicast networks. Unicast refers to a transmission from a single source to a single destination and multicast refers to a transmission from a single source to a subset of all destinations capable of receiving the transmission. The formatted data packets are forwarded to an RTP connector 38 for transmission by the conferencing interface device's communication facility. The RTP connector 38 also receives RTP formatted data from the network through the conferencing interface device's communication facility and forwards the data to the media framework where the stream is depacketized and decoded for rendering. The RTP formatted data received from the communications facility includes data generated by other users or conference participants, represented by conferencing interface devices 22b and 22c. Other transmission protocols for transmitting audio and/or video may likewise be used.
Referring to
The conferencing interface device 22 also preferably includes a messaging framework 40 for sending and receiving messages, defined by the conferencing application program 30, to and from other participants in a conference. The messaging framework, preferably, delivers messages to participants who are present in the conference but, because a conference is a dynamic, real-time activity, does not typically store messages. In the event of a late entry to the conference, the messaging framework of the late-arriving conferee's conferencing interface device transmits a message to the other participants in the conference notifying them of the new attending participant. A conference participant's receipt of a message indicating the entry of a new participant in the conference triggers a response identifying the recipient and describing the recipient's current status.
A user can access the conferencing system 20 by entering a user name and password on a portal web page that is displayable with a web browser program 42 on the user's conferencing interface device 22. Referring to
A user authorized to access the conferencing system can schedule a conference from the conferencing system graphical interface 200 displayed on the user's interface device. Referring to
In addition, the add conference interface 250 enables the user scheduling a conference to select a mode for the conference. The conferencing system enables a team mode 268 in which all conference participants can simultaneously transmit and receive audio, video and data, to the extent that audio, video and data transmission is enabled for the conference. Since the team mode require transport streams containing audio, video and, in many cases, data from each of the conference locations, the number of locations from which participants can transmit is often limited by the bandwidth required for transmission of the multiple data streams. In the presentation mode 270, audio and video is transmitted primarily by a single individual while the remaining participants watch and listen. If permission is granted by the conference moderator, one or more of the other conferees may transmit audio and/or video. In addition, the user's may request permission from the moderator. The presentation mode limits the number of locations that are simultaneously transmitting and, commonly, limits transmission, from users other than the moderator, to audio. The bandwidth required for a conference in the presentation mode is typically less than the bandwidth required for the team mode allowing larger numbers of conference locations. When the presentation mode 270 is selected, the conferencing system designates the user scheduling the conference as the conference moderator. From the add-conference interface, the scheduler of the conference can designate who will be invited to the conference and send notification to the invitees with a message 272, if desired. When the conference's configuration has been specified by the scheduler of the conference, a conference configuration, including name, description, time and names of invitees, is transmitted to the application server and stored in the conferencing system database 152. The conferencing server transmits an invitation and any associated message to each of the conference invitees.
Referring to
In response to receiving the conference configuration data, the conferencing application initiates the media framework 314 which attempts to enable the respective devices and systems of the interface device, as required by the conference configuration, to capture, transmit, receive and render conference video, audio, and data. For example, a participant should be able to receive and render audio and the moderator of a conference should be, at least, able to capture and transmit audio. If the media framework cannot be initiated or any required medium capture or rendering device enabled, the conferencing application exits 304.
If the required data capture and rendering systems are successfully enabled, the conferencing application attempts to configure any required network resources 315. For example, the conferencing application attempts to contact the messaging server 156, join conference multicast groups or establish a socket connection to any video conferencing servers required by an RTP connector. To participate in a conference (such as Unicast) comprising audio and video, the conferencing interface device preferably creates four TCP (Transmission Control Protocol) sockets for communication between the user interface device and the video conferencing server 156. Also, the system could use a single socket for each stream, two sockets for each stream, UDP, or any other arrangement. The TCP sockets, comprising an Internet Protocol address and a port number, are endpoints for communications between the server and the user interface device. A TCP socket is established for each of video RTP communication, video RTCP communication, audio RTP communication and audio RTCP communication. The RTCP sockets enable communication of control information associated with transmission and reception of RTP data packets. RTCP provides for counting lost packets, measuring jitter and other housekeeping duties defined by the RTP protocol. If the conferencing application cannot configure the required network resources, the conferencing application exits 304.
If the required network resources are successfully configured a PRESENT message is transmitted to all conference participants 316 and the user, represented by the conferencing interface device 20a, enters the conference. The PRESENT message is transmitted to the messaging server 154 by the messaging framework 40 of conferencing interface device. Preferably, the messages transmitted and received by the messaging framework are propagated using Jabber communication protocols. Jabber is an open XML (Extensible Markup Language) protocol for instant messaging. In response to receipt of the PRESENT message from the newly entering conferencing interface device, the messaging server 154 transmits the PRESENT message to all of the previously entered participants in the conference and transmits presence status for the list of current participants to the newly entering conferencing interface device. The messaging framework of the newly entering conferencing interface device transmits a message to all of the current participants in the conference requesting that the participants acknowledge the newly entering participant by transmitting current status data for the respective participant. The system may likewise provide application specific status, such as for example, transmitting, requesting, receiving, and sleeping (idle) for audio, video, and/or text.
Preferably, each participating conferencing interface device maintains its own state data, specifying; for examples, without limiting possible state data, the current conference identification, whether the conferencing interface device is the moderator of a presentation conference, whether the conferencing interface device has permission to transmit audio, and whether the conferencing interface device is transmitting video. When a participant receives a message from the messaging server, the respective user interface device responds by transmitting its current state data to the newly entered participant and, if required by the conference mode, configures itself to receive an additional data stream from the new conference participant. During the conference, the state data of the respective participants may change, for example, permission to transmit audio may be granted to a conferencing interface device participating in a presentation type of conference or a conference participant may leave the conference. As required, the messaging server 154 transmits messages to the messaging framework of each current participant advising the participant of a change in the status of another participant.
Once the information regarding the conference participants has been received 318, the conferencing interface device begins rendering the conference's audio and video streams 320. Referring to
In the presentation mode, the conference host and moderator is the presenter and the other participants are generally passive listeners and viewers of the host's audio or video and audio presentation. Referring to
Video of the moderator is displayed in a viewing pane 504 during a presentation mode conference. Conference participants can toggle the video display of the moderator on or off by selecting and actuating a video toggle button 506 below the viewing pane. If the conference does not include video, no image is displayed in the viewing pane or no pane is displayed. The name of the presentation mode conference moderator 508 is displayed below the viewing pane with an audio volume slider 604 permitting the user to adjust the volume of the moderator's audio transmission. The moderator may initiate recording of the conference from a drop down menu in the menu bar and, in some embodiments of the conferencing system, other participants may also record the conference. A recording indicator 606 on the conferee conference console changes state to indicate when the conference is being recorded. In addition, all users may be provided with a recording indicator.
The names of the invitees to the conference are displayed in a participant panel 510 below the viewing pane. An invitee's presence in the conference is indicated by an icon 524 which is displayed proximate the invitee's name when the participant enters the conference.
Referring also to
Audio transmission is enabled 708 for the conferee having permission to transmit until the permission is either terminated by the moderator 710 or by the conferee having permission 714 to terminate or by the moderator exiting 716. Audio transmission by a participant having permission to transmit can be toggled on or off by a transmission control button 622 on the participant's conference console 600. In addition, the moderator can revoke the permission to transmit by selecting a permission revocation control 522 displayed on the control panel proximate the name of a conferee having permission to transmit. If permission is revoked by the moderator 710, audio transmission by the conferee is terminated 712 and the conferencing application resets the permission request control 707 and returns the participant to the passive listening and viewing state. Likewise, if the conferee with permission terminates audio transmission 714 by toggling the transmission control 622 or otherwise, transmission is terminated 714 the user may transmit again when desired. An icon 620 proximate the name of the conferee indicates that the conferee has permission to transmit even though the conferee may not be speaking.
The conferencing system enables management of communication between a large number of remotely located conference participants and improves the participant's satisfaction with computer based conferencing.
The conferencing system may also use the same techniques, alone or in combination, to provide a moderated desktop of a computer (desktop or server). In this manner, items on the desktop may be shared in a controlled manner.
The detailed description, above, sets forth numerous specific details to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention may be practiced without these specific details. In other instances, well known methods, procedures, components, and circuitry have not been described in detail to avoid obscuring the present invention.
All the references cited herein are incorporated by reference.
The terms and expressions that have been employed in the foregoing specification are used as terms of description and not of limitation, and there is no intention, in the use of such terms and expressions, of excluding equivalents of the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims that follow.
This application is a continuation of U.S. patent application Ser. No. 11/219,502, filed Sep. 1, 2005 which claims the benefit of U.S. Provisional App. No. 60/676,089, filed Apr. 28, 2005.
Number | Date | Country | |
---|---|---|---|
60676089 | Apr 2005 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 11219502 | Sep 2005 | US |
Child | 11407428 | Apr 2006 | US |