The present disclosure relates to scheduling and managing meetings.
Inviting participants to a meeting (in person or audio/video conference) can be tedious and not particularly intuitive. For example, a user that schedules a meeting uses a software application or web-based interface in which he/she types the names of the users or email identifiers into fields of a graphical user interface, and selects a day and time for the meeting. Some user interfaces have become so cumbersome that many people avoid using them altogether and delegate the meeting schedule function to another person.
Accordingly, there is a significant opportunity to streamline the meeting scheduling/creation process.
Overview
Techniques and configurations for an apparatus are provided for creating (scheduling) and managing meetings using gestures. Movements of a user's hand in a three-dimensional space are detected. The hand movements in the three-dimensional space are interpreted to identify a gesture intended by the user to set up or manage a meeting among a plurality of persons. An electronic command is generated from the detected gesture to set up or manage the meeting.
Example Embodiments
Referring first to
The gesture interface enabled endpoint 20 is designed to allow a user to create and manage meetings between participants through the use of hand gestures with or without associated audio (e.g., spoken voice or hand-clapping). The meeting that a user sets up may be an in-person meeting or an audio or video conference meeting that is supported by the conference system server 60, for example. The other endpoints 50(1)-50(N) may be any endpoint device capable of allowing a user to participate in an audio and/or video conference meeting. Moreover, one or more of the other endpoints 50(1)-50(N) may also be configured for a gesture interface like endpoint 20.
The conference system server 60 is a computing apparatus and related supported devices and resources capable of scheduling and/or hosting (audio and/or video) conferences for a plurality of participants. Examples of a conference system server 60 include servers associated with the WebEx® meeting service, Microsoft Live® meeting service, GoToMeeting® service and DimDim® meeting service.
The gesture interface enabled endpoint 20 comprises a network interface unit 22, a keyboard, a display 26, a microphone 28, a sensor array 30, a sensor interface 34, a processor 36 and a memory 40. The network interface unit 22 enables communication over network 45 for purposes of sending and receiving controls and data (e.g., email messages) associated with the setup of a meeting, as well as data associated with an audio and/or video conference meeting. For example, the network interface unit 22 is an Ethernet network interface card. The keyboard 24 is a physical keyboard that receives user keystroke input. The display 26 is a display device (e.g., flat panel display) that presents images to a user associated with a graphical user interface for the gesture interface techniques described herein, as well as a graphical user interface for other operations supported by the endpoint 20, including participating in audio and/or video conferences, and other more standard user computing operations.
The microphone 28 is configured to detect audio from a user. The microphone 28 may be incorporated into the keyboard 24 or display 26, or it may be a separate standalone device. More than one microphone 28 may be used in some deployments. For example, multiple microphones may be incorporated into the sensor array 30. The microphone 28 may comprise a high-fidelity microphone. A suitable interface card (not shown) is provided to convert analog audio detected by the microphone 28 into digital audio data that can be interpreted by the processor 36.
The sensor array 30 is an array of sensors configured to detect orientation and movements of a hand of a user for purposes of generating a command associated with the creation of or management of a meeting. The sensor interface 34 is coupled to the sensor array 30 and serves as an interface of the sensor data produced by the sensor array to processor 36. The sensor interface 34 converts the sensor output to appropriate digital data that is interpreted by the processor 34. Examples of the sensor array 30 are described in more detail hereinafter in connection with
The processor 36 serves as a controller for the endpoint 20. The processor 36 comprises one or more microprocessors or microcontrollers, for example, The processor 36 executes software instructions store in memory 40 to perform various operations associated with the gesture interface enabled endpoint 20 described herein. The processor 26 connects to the various other components in the endpoint 20 via a bus (not shown).
The memory 40 may comprise read only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical/tangible memory storage devices. The memory 40 stores software instructions for gesture-oriented rendering process logic 100, gesture and audio interpretation process logic 200, and a calendar application 300. In addition, the memory 40 stores data in a gesture database 400. Thus, in general, the memory 40 may comprise one or more computer readable storage media (e.g., a memory device) encoded with software comprising computer executable instructions and when the software is executed (by the processor 36) it is operable to perform the operations described for the gesture and audio interpretation process logic 100 and gesture-oriented rendering process logic 200.
Operations of the gesture and audio interpretation process logic 100 and gesture-oriented rendering process logic 200 are described in detail hereinafter in connection with
The gesture-oriented rendering process logic 200 receives commands from the gesture and audio interpretation process logic 100 and interacts with the calendar application 300 and the conference system 60 (possibly through the calendar application 300) to schedule and start meetings. The gesture-oriented rendering process logic 200 also renders a “gesture-friendly” graphical user interface on the display 26 to allow a user to select meeting participants, schedule meetings and start meetings. The gesture-oriented rendering process logic 200 presents the user with actions that can be performed by selecting objects presented on the display 26 through various schemes described herein.
The calendar application 300 is a software application, such as Microsoft Outlook®, that is used to schedule meeting, and through suitable plug-in software, can interface with the conference system server 60. Thus, a meeting related command generated by the gesture and audio interpretation process logic 100 is supplied to the calendar application 300, via the gesture-oriented rendering process logic 200. When the meeting is to be hosted by the conference system server 60, then the meeting related command is forward to the conference system server 60, which in turn sends out email invitations to the meeting participants, and at the appropriate time of the meeting, hosts the audio or video conference. When the meeting is an in-person meeting, then the conference system server 60 is not necessarily needed and the email invitations can be sent out to the meeting participants by the calendar application 300.
The gesture database 400 stores data that associates or maps gestures to commands to be supplied by the gesture-oriented rendering system to the conference system server 60 and/or calendar application 300. Examples of data stored in the gesture database 400 are described hereinafter.
In one form, the endpoint 20 may be a computing device (e.g., desktop computer, laptop computer, tablet computer, etc.) that is augmented with the sensor array 30 and supporting hardware and software for the gesture recognition operations described herein. In another form, the endpoint 20 may be a dedicated endpoint used in a high-fidelity video conference system, such as Cisco System, Inc.'s Telepresence video conference system.
Reference is now made to
As shown in
Reference is now made to
The detection zone may take on a variety of shapes. In one example, the sensors in the sensor array 30 are configured and arranged to detect close-range hand movements in a 2 inch deep by 24 inch wide detection zone.
In some implementations, a user may use Quick Response (QR) barcode stickers (or gloves with QR barcodes) on his/her finger tips that are used to facilitate the detection of the location of a user's fingers in the detection zone. To this end, the sensor interface 34 and/or the gesture and audio interpretation process logic 100 are configured to decode the QR barcodes on the user's fingertips. Similarly, the user may wear color-coded stickers (or a glove with different colored finger tips) to facilitate the location detection of a user's fingertips in the detection zone. While the user's hands are in the detection zone, he/she may extend one finger to make gesture commands, such as selecting a meeting participant. The sensor array 30 detects the extended finger and sends coordinates of the finger and the direction that the finger is pointing to the gesture and audio interpretation process logic 100. Finger pointing detection is described in more detail below in connection with
The gesture database 400 is now described. The table below depicts examples of the data stored in the gesture database 400. Specifically, the gesture database 400 stores data describing each of a plurality of gestures and a corresponding command. The gesture database 400 may also store commands for audio/voice as well. As shown in the table, the hand gesture of an outstretched palm corresponds to a command to initiate a meeting setup sequence, the hand gesture of a finger pointing with a forward motion corresponds to a command to select a displayed object at the location to which the finger points, a hand clap corresponds to a command to immediately start a meeting, and so on. The data stored in the gesture database 400 may be adapted or changed for different users. For example, one user may configure the endpoint 20 to interpret an outstretched palm gesture to correspond to one command and another user may configure the endpoint 20 to interpret that same gesture to correspond to another command.
In
Turning now to
Thus, the operations depicted in
Turning now to
To select a displayed object (for a participant or a meeting related command object), the user needs to continue making the pointing gesture and add to it a forward motion, such that the user's finger tip is moved closer to the display in the detection zone. At 114, the gesture and audio interpretation process logic 100 interprets the output data from the sensor array 30 (which tracks the location and forward motion of the user's fingertip) and detects the forward motion of the user's finger as corresponding to a selection command. At 116, the gesture and audio interpretation process logic 100 sends a selection command message with the calculated coordinates and trajectory vector to the gesture-oriented rendering process logic 200. Another way to generate a finger-based selection command is to monitor a finger retraction and extension motion. At 222, the gesture-oriented rendering process logic 200 presents an indication on the display to the user that the selection has been made, and also an indication that the user interface is in a “selection mode.” This allows the user to select more displayed objects.
Reference is now made to
Another way to perform an action with selected objects is for the user to clap his/her hands together within the detection zone. The microphone 28 will detect the clapping sound and the sensor array 30 will detect the hand movements, and the gesture and audio interpretation process logic 100 determines whether the user has clapped his/her hands together. As explained above, the gesture database is configurable and the clapping gesture may be associated with a command to schedule and start a meeting immediately. Thus, a particular gesture may be assigned to a command to schedule and start a meeting immediately.
Reference is now made to
Thus, in one example, the gesture-oriented rendering process logic 200 stores data to visually present a user's “contacts” in a format other than a list or simple icons. Moreover, the objects representing the user's contacts may be grouped by teams or organizational structure, such as a group of objects associated with the user's “management team” and another group of objects associated with the user's “sales team,” as well as “frequent contacts,” etc. The gesture-oriented rendering process logic 200 retrieves the data for and displays the objects for these groups in response to the user selecting an appropriate one of the buttons 524(1)-524(3). Frequent contacts may be displayed “in front” of less frequent contacts.
At 530, the gesture and audio interpretation process logic 100 interprets hand movements indicating a pointing and selection gesture associated with one of the objects (with or without a voice command) to generate a selection command (supplied to the gesture-oriented rendering process logic 200) of a displayed object for a person, and the gesture-oriented rendering process logic 200 supplies a command (to the calendar application and/or conference system server) to add a person corresponding to the selected object as a participant to be invited to the meeting. As shown in
Referring back to
At 550, a particular gesture is detected that is associated with completing the meeting setup sequence. For example, as shown in
At 560, the gesture-oriented rendering process logic 200 presents a visual object representation on the display representing the meeting. The user can select this object at any time to check the attendance status, add or remove invitees, change the time or venue of the meeting, etc., using the operations described above.
At 570, the gesture-oriented rendering process logic 200 can present a user interface environment on the display to allow the user to use certain expanding or compressing gestures to zoom in or out of the graphical user interface. An expand gesture may be made by the user moving his two hands apart and the compress gesture by moving their hands together.
The techniques described herein provide an intuitive way to create and manage meetings. A photo or video realistic representation of contacts is presented to a user, and the user makes selections for invitees with pointing or other suitable gestures that are detected an recognized by gesture recognition hardware and software. Meeting participants are selected by a user pointing at them in a three-dimensional space. Gestures are also used to select a time for the meeting, and initiate and complete the meeting creation/scheduling sequence. A visual object representation of the meeting is displayed to the user and the user may access the data associated with the meeting by selecting the object.
The above description is intended by way of example only.
Number | Name | Date | Kind |
---|---|---|---|
5168531 | Sigel | Dec 1992 | A |
5347306 | Nitta | Sep 1994 | A |
5398310 | Tchao et al. | Mar 1995 | A |
5454043 | Freeman | Sep 1995 | A |
5583946 | Gourdol | Dec 1996 | A |
5612719 | Beernink et al. | Mar 1997 | A |
5617312 | Iura et al. | Apr 1997 | A |
6359612 | Peter et al. | Mar 2002 | B1 |
7469381 | Ording | Dec 2008 | B2 |
7479949 | Jobs et al. | Jan 2009 | B2 |
7653883 | Hotelling et al. | Jan 2010 | B2 |
7665041 | Wilson et al. | Feb 2010 | B2 |
7668340 | Cohen et al. | Feb 2010 | B2 |
7705830 | Westerman et al. | Apr 2010 | B2 |
20020140667 | Horiki | Oct 2002 | A1 |
20040193413 | Wilson et al. | Sep 2004 | A1 |
20060044265 | Min | Mar 2006 | A1 |
20080036743 | Westerman et al. | Feb 2008 | A1 |
20080059578 | Albertson et al. | Mar 2008 | A1 |
20080165140 | Christie et al. | Jul 2008 | A1 |
20080309632 | Westerman et al. | Dec 2008 | A1 |
20080309634 | Hotelling et al. | Dec 2008 | A1 |
20090033737 | Goose et al. | Feb 2009 | A1 |
20090274339 | Cohen et al. | Nov 2009 | A9 |
20100095206 | Kim | Apr 2010 | A1 |
20100257526 | Zimmet et al. | Oct 2010 | A1 |
20110296353 | Ahmed et al. | Dec 2011 | A1 |
20120056800 | Williams et al. | Mar 2012 | A1 |
20120092304 | Katz | Apr 2012 | A1 |
20120204120 | Lefar et al. | Aug 2012 | A1 |
20120319945 | McCarthy et al. | Dec 2012 | A1 |
20130067343 | Groves et al. | Mar 2013 | A1 |
20130103446 | Bragdon et al. | Apr 2013 | A1 |
Number | Date | Country |
---|---|---|
2005091125 | Sep 2005 | WO |
2010086866 | Aug 2010 | WO |
Entry |
---|
Sato et al.; Real-Time Input of 3D Pose and Gestures of a User's Hand and Its Applications for HCI; © IEEE; 8 pages. |
Segen et al.; Shadow Gestures: 3D Hand Pose Estimation using a Single Camera; © 1999; IEEE; 7 pages. |
Oka et al; Real-Time Fingertip Tracking and Gesture Recognition; © 2002; 8 pages. |
Rahmat et al.; Removing Shadow for Hand segmentation based on Background Subtraction; © 2012; 5 pages. |
Mark Spoonauer, Video: PointGrab's Gesture Tech Comes to Fujitsu Laptops, More to Come, http://blog.laptopmag.com/video-pointgrab-brings-gesture-recognition-to-f . . . , Jul. 6, 2010, pp. 1-9. |
Number | Date | Country | |
---|---|---|---|
20120216151 A1 | Aug 2012 | US |