Patent Grant
9706169
This application is a Section 371 National Stage Application of International Application No. PCT/CN2015/081041, filed Jun. 9, 2015, entitled “Remote Conference System and Method of Performing Remote Conference”, which claims priority to Chinese Application No. 201510013171.2, filed on Jan. 9, 2015, incorporated herein by reference in their entirety.
1. Technical Field
Embodiments of the present invention relate to a remote conference system and a method of performing the remote conference.
2. Description of the Related Art
Internet-based remote conference is becoming an option for common businesses. However, an audience in a remote conference can never know whether a person who is talking is talking to him or not.
It is an object of embodiments of the present invention to provide a remote conference system and a method of performing a remote conference, thereby an audience in a remote conference can know whether a person who is talking is talking to him or not.
According to an embodiment of the present invention, there is provided a remote conference system, comprising: a first image-displaying system at a first conference location, configured to display images of a plurality of audiences; a detection system configured to detect motions or poses of at least part of heads of persons in the first conference location; a judgment system configured to judge whether a person who is talking is talking to a certain audience or some certain audiences of the plurality of audiences or to all the audiences based on the motions or poses detected by the detection system; a second image-displaying system, at a second conference location, configured to display special images to the certain audience or some certain audiences of the plurality of audiences when the judgment system has judged that the person who is talking is talking to the certain audience or some certain audiences of the plurality of audiences, such that the certain audience or some certain audiences of the plurality of audiences is/are aware that the person who is talking is talking to him or them.
According to an embodiment of the present invention, the remote conference system further comprise a voice tracking system, at the first conference location, configured to determine which person from the persons in the first conference location is talking.
According to an embodiment of the present invention, the second image-displaying system is further configured to display common images to all the audiences when the judgment system had judged that the person who is talking is talking to all the audiences.
According to an embodiment of the present invention, the second image-displaying system is further configured to display the common images to the rest of the audiences while displaying the special images to the certain audience or some certain audiences of the plurality of audiences.
According to an embodiment of the present invention, in addition to the person who is talking to the certain audience or some certain audiences of the plurality of audiences, other person(s) is/are further included in the first conference location, and the special images only include images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences or only include images of face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences.
According to an embodiment of the present invention, in addition to the person who is talking to the certain audience or some certain audiences of the plurality of audiences, other person(s) is/are further included in the first conference location, and the special images may only include images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences or may only include images of face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences while the common images include images of the person who is talking and at least one of other person(s), or images of faces of the person who is talking and at least one of other person(s).
According to an embodiment of the present invention, in addition to the images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences, the common images include more images of scene than the special images.
According to an embodiment of the present invention, the images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences have a larger size in the special images than in the common images.
According to an embodiment of the present invention, the images of the face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences have a larger size in the special images than in the common images.
According to an embodiment of the present invention, the detection system is an eyeball tracking system configured to detect motions of eyeballs of persons in the first conference location, and the judgment system judges whether the person who is talking is talking to the certain audience or some certain audiences of the plurality of audiences, or is talking to all the audiences, based on the motions of eyeballs detected by the eyeball tracking system.
According to an embodiment of the present invention, there is provided a method of performing a remote conference, comprising: displaying images of a plurality of audiences by means of a first image-displaying system at a first conference location; detecting, by means of a detection system, motions or poses of at least part of heads of persons in the first conference location; judging, by means of a judgment system, whether a person who is talking is directed to a certain audience or some certain audiences of the plurality of audiences or is directed to all the audiences based on the motions or poses detected by the detection system; displaying, by means of a second image-displaying system at a second conference location, special images to the certain audience or some certain audiences of the plurality of audiences when the judgment system has judged the person who is talking is talking to the certain audience or some certain audiences of the plurality of audiences, such that the certain audience or some certain audiences of the plurality of audiences is/are aware that the person who is talking is talking to him or them.
According to an embodiment of the present invention, the method of performing a remote conference further comprises: determining, by means of a voice tracking system, which person from the persons in the first conference location is talking, before the judgment of whether the person who is talking is directed to the certain audience or some certain audiences of the plurality of audiences or is directed to all the audiences.
According to an embodiment of the present invention, the method of performing a remote conference further comprises: displaying, by means of the second image-displaying system, common images to all the audiences when the judgment system had judged that the person who is talking is directed to all the audiences.
According to an embodiment of the present invention, the method of performing a remote conference further comprises: displaying, by means of the second image-displaying system, common images to the rest of the audiences while displaying the special images to the certain audience or some certain audiences of the plurality of audiences.
According to an embodiment of the present invention, in addition to the person who is talking to the certain audience or some certain audiences of the plurality of audiences, other person(s) is/are further included in the first conference location, and the special images only include images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences or only include images of face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences.
According to an embodiment of the present invention, in addition to the person who is talking to the certain audience or some certain audiences of the plurality of audiences, other person(s) is/are further included in the first conference location, and the special images only include images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences or only include images of face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences while the common images include the images of the person who is talking and at least one of other person(s), or the images of faces of the person who is talking and at least one of other person(s).
According to an embodiment of the present invention, in addition to the images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences, the common images include more images of scene than the special images.
With the abovementioned technical solution, for example, in the situation where the first conference location contains one person or a plurality of persons, sizes of the images of scene in these images are changed such that the audience(s) may know whether the talking is directed to him or them or not.
According to an embodiment of the present invention, the images of the person who is talking to the certain audience or some of the plurality of audiences have a larger size in the special images than in the common images.
According to an embodiment of the present invention, the images of the face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences have a larger size in the special images than in the common images.
The detection system is an eyeball tracking system, which is configured to detect motions of eyeballs of persons in the first conference location, and the judgment system judges whether the person who is talking is talking to the certain audience or some certain audiences of the plurality of audiences, or is talking to all the audiences, based on the motions of eyeballs detected by the eyeball tracking system.
Embodiments of the present invention are described in detail in conjunction with the description and the attached drawings.
Embodiments of the present invention are described in detail by reference to the attached drawings. In addition, for easy interpretation, lots of details are described in the following detailed description to provide comprehensive understanding on the embodiments of the present invention. However, it is obvious that one or more embodiments can be implemented without the above details. In other situations, known structures and devices are illustrated in the manner of diagram in order to simplify the drawings.
As shown in
It is noted that the conference location used herein means that, for example, persons A1, A2, A3 in the same conference location may achieve direct intercommunion by eye contact, while persons in different conferences, such as A and B, cannot achieve direct intercommunion by eye contact. In other words, the intercommunion or conference between A and B in different conferences is remote conference or remote intercommunion. However, an interactive system of the present invention may provide an indirect manner such that A and B in different conferences may launch a remote conference, including achievement of an intercommunion by eye contact. In embodiments of the present invention below, terms of “conference” and “remote” have the same meanings as those in the above embodiments.
The remote conference system according to the embodiment of the present invention further includes a voice tracking system V, at the first conference location, configured to determine which person from the persons A1, A2, A3 in the first conference location is talking. In this instance, it is automatically determined which person is talking in the situation where a plurality of persons A1, A2, A3 are contained in the first conference location 10. In addition, the remote conference system according to the embodiment of the present invention further includes a controller or the like, configured to control operations of various components included in the remote conference system.
According to the embodiment of the present invention, the detection system may be an eyeball tracking system E configured to detect motions of eyeballs of persons A1, A2, A3 in the first conference location 10, and the judgment system J judges whether the person who is talking is talking to the certain audience or some certain audiences of the plurality of audiences, or is talking to all the audiences, based on the motions of the eyeballs detected by the eyeball tracking system. By this way, it can be determined much more exactly whether the person who is talking is talking to the certain audience or some certain audiences of the plurality of audiences, or is talking to all the audiences.
In addition, alternatively, it is possible to use a face orientation detection system to detect orientation of a face of the person who is talking or any other suitable system, and thus the judgment system J will judge whether the person who is talking is talking to a certain audience or some certain audiences of the plurality of audiences or is talking to all the audiences.
For example, as shown in
As shown in
According to some embodiments of the present invention, in addition to the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3, other persons are further included in the first conference location 10. The special images only include images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3, or only include images of face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences. From this, in the situation where there are a plurality of persons in either of the two conference locations, images including the person who is talking may be provided to the certain audience or some certain audiences such that he or they know the person who is talking is talking to him or them.
According to some other embodiments of the present invention, in addition to the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3, the first conference location 10 further includes other persons. The special images only include images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3 or only include images of face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences, while the common images include images of the person who is talking and at least one of the other persons or images of faces of the person who is talking and at least one of the other persons. In the situation where the two conference locations both include a plurality of persons, only images including the person who is talking or images including all the persons in the first conference location are provided to the audiences, such that the audience(s) know whether the person who is talking is talking to him or them.
In an embodiment relating to the special images and the common images, in addition to the images including the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3, the common images are images including more scenes relative to the special images. From this, when the first conference location includes one person or a plurality of persons, sizes of the images of the scene in these images may be changed such that the audience(s) know whether the talking is directed to him or them.
As another embodiment relating to the special images and the common images, size of the images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3 is larger in the special images than in the common images. From this, in the situation where the first conference location includes one person or a plurality of person, the size of the images of the person who is talking in the images may be changed such that the audience(s) knows whether the talking aims to him or them. As a still another embodiment relating to the special images and the common images, the size of the images of face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3 in the special images is larger than size of that in the common images. From this, in the situation where the first conference location includes one person or a plurality of person, the size of the images of the face of the person who is talking in the images may be changed such that the audience(s) knows whether the talking aims to him or them.
For example, the voice tracking system V determines which person from the persons A1, A2, A3 in the first conference location 10 is talking. The judgment system J judges whether the person who is talking is talking to the certain audience or some certain audiences in the plurality of audiences B1, B2, B3, or is talking to all the audiences. According to result of the judgment from the judgment system J, the image-displaying system D2 of the conference system 100 provides difference images or identical images to the plurality of audiences B1, B2, B3. For example, the second image-displaying system D2 may be a multiple-viewing-field display system or may be a plurality of displays. By this way, the plurality of audiences B1, B2, B3 may be provided with different images, and thus a remote conference scene that is closer to the real situation can be provided, in accordance with different requirements.
For example, in the first conference location 10, as shown in
For example, in a video conference, the eyeball tracking system E may capture images in which the person A2 who is talking is gazing at the images of the audience B3 in the first image-displaying system D1 and the voice tracking system V may detect the voice from the person A2 who is talking, and thus the judgment system J may judge that the person A2 who is talking is talking to the audience B3 mostly. With the help of the camera system C, the images captured are transferred to the second image-displaying system D2, and the second image-displaying system D2 may display the special images to the audience B3, such as, the images of the single person A2 who is talking. Meanwhile, the second image-displaying system D2 displays the common images, such as, the video images of overall the first conference location 10 or the images of all persons A1, A2, A3 in the first conference location 10, to the rest audiences B1 and B2. In this instance, the directionality or targeted characteristic of the remote video conference may be improved, and communication block is avoided due to target indeterminacy between a speaker and a hearer.
As an another example, in a video conference, the eyeball tracking system E may detect that eyes of persons A1 and A2 are respectively on images of persons B1 and B2 in the first image-displaying system D1 by capturing motions of eyeballs of the persons A1, A2, A3, meanwhile, the voice tracking system V may detect voices from the persons A1 and A2, and thus the judgment system J may judge that the person A1 who is talking is talking to the audience B1 one to one and the person A2 who is talking is talking to the audience B3 one to one. The second image-displaying system D2 may display special images, such as images of the single person A1 who is talking, to the audience B1, and may display special images, such as images of the single person A2 who is talking, to the audience B3, while the second image-displaying system D2 may still display common images, such as the video images of overall the first conference location 10 or the images of all persons A1, A2, A3 in the first conference location 10, to the rest audience B2.
A method of performing a remote conference according to an embodiment of the present invention will be described as below.
As shown in
step S100 of: displaying, by means of a first image-displaying system D1, images of a plurality of audiences B1, B2, B3 in a first conference location 10;
step S200 of: detecting, by means of a detection system, motions or poses of at least part of heads of persons A1, A2, A3 in the first conference location 10;
step S300 of: judging, by means of a judgment system J, whether a person who is talking is talking to a certain audience or some certain audiences of the plurality of audiences B1, B2, B3 or to all the audiences based on the motions or poses detected by the detection system;
step S400 of: displaying, by means of a second image-displaying system D2 at a second conference location 20, special images to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3 when the judgment system J has judged the person who is talking is talking is to the certain audience or some certain audiences of the plurality of audiences, such that the certain audience or some certain audiences of the plurality of audiences B1, B2, B3 is/are aware that the person who is talking is talking to him or them. By this way, targeted audience(s) in the remote conference can be aware that a person who is talking is talking to him or them.
Referring to
step S500 of: displaying, by means of the second image-displaying system D2, common images to all the audiences when the judgment system J has judged that the person who is talking is talking to all the audiences. By this way, the audiences may know the talking is not directed to them.
According to some embodiments of the present invention, the method of performing a remote conference further includes:
before the judgment of whether the person who is talking is talking the certain audience or some certain audiences of the plurality of audiences or is talking to all the audiences, determining, by means of a voice tracking system, which person from the persons in the first conference location is talking. By this way, it is determined which person is talking in the situation where a plurality of persons are included in the first conference location.
According to an example of the present invention, the detection system may be an eyeball tracking system configured to detect motions of eyeballs of the persons in the first conference location. The judgment system judges whether the person who is talking is talking to the certain audience or some certain audiences of the plurality of audiences, or is talking to all the audiences, based on the motions of the eyeballs detected by the eyeball tracking system. By this way, it can be determined much more exactly whether the person who is talking is talking to the certain audience or some certain audiences of the plurality of audiences, or is talking to all the audiences.
According to some embodiments of the present invention, the method of performing a remote conference further includes: displaying, by means of the second image-displaying system D2, the common images to the rest of the audiences while displaying the special images to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3. In this instance, the rest audiences may know the talking does not aim to them.
According to some embodiments of the present invention, in addition to the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3, other persons are further included in the first conference location 10. The special images only include images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3, or only include images of face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences. From this, in the situation where a plurality of persons are included in either of the two conference locations, the images including the person who is talking may be provided to the certain audience or some certain audiences such that he or they know the person who is talking is talking to him or them.
According to some embodiments of the present invention, in addition to the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3, the first conference location 10 further includes other persons. The special images only include images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3 or only include images of face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences, while the common images include images of the person who is talking and at least one of the other persons or images of faces of the person who is talking and at least one of the other persons. In the situation where a plurality of persons are included in either of the two conference locations, the images including the person who is talking or the images including all persons in the first conference location are provided to the audiences, such that the audience(s) know whether the person who is talking is talking to him or them.
As an embodiment relating to the special images and the common images, in addition to the images including the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3, the common images include more images of scene than the special images. By this way, in the situation where the first conference location contains one person or a plurality of persons, the size of the images of the scene in the images may be changed such that the audience(s) may know whether the talking is directed to him or them.
As another embodiment relating to the special images and the common images, the images of the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3 have larger size in the special images than in the common images. By this way, when the first conference location includes one or more persons, the size of the images of the person who is talking in the images is changed such that the audience(s) may know whether the talking is directed to him or them.
As a still another embodiment relating to the special images and the common images, the images of face of the person who is talking to the certain audience or some certain audiences of the plurality of audiences B1, B2, B3 have larger size in the special images than in the common images. By this way, when the first conference location includes one or more persons, the size of the images of the face of the person who is talking in the images is changed such that audience(s) may know whether the talking is directed to him or them.
In addition, the person in the first conference location may be interpreted as one person or a plurality of persons.
The above embodiments are introduced to describe the present invention, rather than limit the present invention. Various modifications and variants may be made by those skilled in the art without departing from inspirit and scope of the present invention, and thus the equivalents of the technical solution of the present invention are fallen into the scope of the present invention. The scope of the present invention should be defined by the pending claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2015 1 0013171 | Jan 2015 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2015/081041 | 6/9/2015 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2016/110047 | 7/14/2016 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8560315 | Yasoshima | Oct 2013 | B2 |
| 9386270 | Griffin | Jul 2016 | B2 |
| 20040130614 | Valliath et al. | Jul 2004 | A1 |
| 20080298571 | Kurtz et al. | Dec 2008 | A1 |
| 20100250252 | Yasohima | Sep 2010 | A1 |
| 20130120522 | Lian | May 2013 | A1 |
| 20130188007 | Duong | Jul 2013 | A1 |
| 20130215214 | Dhopte | Aug 2013 | A1 |
| Number | Date | Country |
|---|---|---|
| 1732687 | Feb 2006 | CN |
| 101764990 | Jun 2010 | CN |
| 102025970 | Apr 2011 | CN |
| 102377975 | Mar 2012 | CN |
| 104539873 | Apr 2015 | CN |
| 2007005752 | Jan 2007 | WO |
| Entry |
|---|
| International Search Report and Written Opinion for International Patent Application No. PCT/CN2015/081041, dated Oct. 13, 2015, 8 pages. |
| English translation of Box No. V from the Written Opinion of the International Search Report for International Patent Application No. PCT/CN2015/081041, 2 pages. |
| First Office Action, including Search Report, for Chinese Patent Application No. 201510013171.2, dated Mar. 30, 2017, 9 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20160353059 A1 | Dec 2016 | US |
