This application is related to applications entitled, “VIDEO COMMUNICATION DEVICE AND METHOD FOR VIDEO COMMUNICATION”, filed Mar. 28, 2019 Ser. No. 16/367,674, “VIDEO COMMUNICATION DEVICE AND METHOD FOR VIDEO COMMUNICATION”, filed Mar. 28, 2019 Ser. No. 16/367,682, “VIDEO COMMUNICATION DEVICE AND METHOD FOR VIDEO COMMUNICATION”, filed Mar. 28, 2019 Ser. No. 16/367,693, and “WINDOW SYSTEM BASED ON VIDEO COMMUNICATION”, filed Mar. 28, 2019 Ser. No. 16/367,732.
The subject matter herein generally relates to a video communication device.
With the development of the communication technology, video communication has been widely used in work places and people's lives in general, such as remote video conference of a company, video chat with friends from different places. However, users of a remote video communication system cannot look directly into each other's eyes. The reason is that the eyes displayed on the display are spaced apart from the camera. As a result, users cannot make direct eye contact with each other when using the remote video communication system, thereby undermining the communication experience. At the same time, a video player can only be used to display video and cannot display different videos or views according to users' different visual directions.
Therefore, there is room for improvement within the art.
Many aspects of the embodiments can be better understood with reference to the following drawings. The components in the drawings are not necessarily drawn to scale, the emphasis instead being placed upon clearly illustrating the principles of the embodiments. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views. Implementations of the present technology will be described, by way of example only, with reference to the attached figures
It will be appreciated that for simplicity and clarity of illustration, where appropriate, reference numerals have been repeated among the different figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein can be practiced without these specific details. In other instances, methods, procedures, and components have not been described in detail so as not to obscure the related relevant feature being described. The drawings are not necessarily to scale, and the proportions of certain parts may be exaggerated to better illustrate details and features. The description is not to be considered as limiting the scope of the embodiments described herein.
Several definitions that apply throughout this disclosure will now be presented.
The connection can be such that the objects are permanently connected or releasably connected. The term “substantially” is defined to be essentially conforming to the particular dimension, shape, or other feature that the term modifies, such that the component need not be exact. The term “comprising” means “including, but not necessarily limited to”; it specifically indicates open-ended inclusion or membership in a so-described combination, group, series, and the like. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean “at least one.”
In
The local translucent display device 11 is used to display remote video information. The remote video information is captured by the remote camera 14 and transmitted to the local translucent display device 11. The local translucent display device 11 is translucent, so that objects in front of the translucent display device 11 may be captured from the back of the translucent display device 11. The local translucent display device 11 may include a translucent display. The local translucent display device 11 may also include a display and a pellicle mirror. The local translucent display device 11 may also include a projector and a translucent projection screen. In
In
In
The local video communication device 10 further comprises a micro processing unit 100. In
The local camera 12 may be used to capture local video information. And the local video information can be transmitted to the remote video communication device 10′. The local camera 12 is placed on the back of the local translucent display device 11. Since the local translucent display device 11 is translucent, the local camera 12 can capture the scene of the local user in front of the local translucent display device 11. The local camera 12 may include a movable camera. The local camera 12 may move and rotate under the control of the position control module 106. The local camera 12 may move freely in a plane parallel to the local translucent display device 11. A distance between the local camera 12 and the local translucent display device 11 can be adjusted as needed, as long as the local camera 12 does not touch the local translucent display device 11 and can be used to capture the scene in front of the local translucent display device 11. The position of the local camera 12 may be changed as the movement of the remote user's image displayed on the local translucent display device 11, so that the position of the local camera 12 always corresponds to the position of the remote user's image. In one embodiment, the position of the local camera 12 always corresponds to the position of the remote user's face image. In one embodiment, the position of the local camera 12 always corresponds to the position of the remote user's eyes image, so that the local user and the remote user can look directly at each other. The local camera 12 is equivalent to the eyes of the remote user. When the remote user moves, the local camera 12 also moves accordingly, and video captured by the local camera 12 also changes accordingly. Therefore, the video seen by the remote user is also constantly changed while the remote user moves.
Furthermore, the position of the local camera 12 can also be directly transmitted to the local video communication device 10 via the remote video communication device 10′. When the remote user moves, the remote video communication device 10′ transmits the remote user's location information to the position control module 106 of the local video communication device 10. The remote user's location information can be calculated according to the remote user's image captured by the remote camera 14 and the position of the remote camera 14. The local camera 12 may move to a position of the remote user's image according to the remote user's position to capture the local video, and transmit the local video to the remote translucent display device 13. Thus, the remote user can have a true immersive experience.
The local camera 12 may move in the following ways. In
In
The local camera 12 may include a binocular camera. The binocular camera may simulate the binocular vision of human eyes to capture scene information, and calculate the scene information in three dimensions to obtain the depth information of the scene. For example, the depth information between a local user and the local translucent display device 11 may be obtained by the local camera 12, then the depth information may be transmitted to the remote video communication device 10′, and the remote video communication device 10′ may select the video information according to the depth information. In
Furthermore, the depth information can may be obtained by the location acquisition module 102, or by calculating data using cloud computing, and the data is send to the cloud by the location acquisition module 102. The transmission of the depth information may be performed by the communication module 104. The local camera 12 may further include four camera units arranged in a crossover manner, for example, to calculate the occlusion relationship in the vertical direction. The local video communication device 10 may further include a depth sensor 120 to specifically sense the depth information.
In
The advantages of the local video communication device 10 may include, but are not limited to, the following. A camera is placed behind a translucent screen, and the camera may move following the position of the remote user's face image displayed on the translucent screen. Thus, when the local user looks at the position of the remote user's eyes image displayed on the translucent screen, the camera can capture the local user's face, and the local user and the remote user can look directly into each other's eyes. The local video communication device 10 provides users with an experience as if only a piece of transparent glass is placed between the local user and the remote user, which greatly increases the immersiveness. The local user and the remote user may feel like communicating with each other as if they are talking to each other on opposite sides of a glass.
In
The video communication system 2 is similar to the video communication system 1 in
In
In action 1401, the local video information is captured by the local camera 12, and the remote video information is captured by the remote camera 14. The local camera 12 is placed on the back of the local translucent display device 11, and the local camera 12 may capture the local video information through the local translucent display device 11. The remote camera 14 is placed on the back of the remote translucent display device 13, and the remote camera 14 may capture the remote video information through the remote translucent display device 13. When a camera captures video information through a translucent display device, light emitted by the pixel does not enter the back of the translucent display device due to the obstruction of a light shielding layer under the pixel, and the camera cannot be disturbed. Therefore, during the operation of the translucent display device, the camera on the back of the translucent display device may also work in real time. In one embodiment, both the remote camera 14 and the local camera 12 may include binocular wide-angle cameras.
Furthermore, in order to make the camera work without interference of light, an image display frame of the translucent display device and an image acquisition frame of the camera can be staggered. In
In action 1402, the local video information is captured by the local camera 12 and transmitted to the remote translucent display device 13, and the local user's location is also transmitted to the remote camera 14. The remote video information is captured by the remote camera 14 and transmitted to the local translucent display device 11, and the remote user's location is also transmitted to the local camera 12.
The user's location can be obtained by the following methods:
Method 1, a relative position of the user in video captured by the camera is obtained, and the position of the user according to the position of the camera and the relative position of the user is calculated. The “relative position of the user” means that the position of user in the video. In
Method 2, the position of the user by an identification device is obtained. The identification device can be placed in front of the translucent display device to identify the user's face.
In action 1403, the local camera 12 moves from the initial position to a corresponding location according to the remote user's location, so the position of the remote user's image on the local translucent display device 11 coincides with the corresponding location where the local camera 12 moves. When the local user looks at the position of the remote user's eyes image displayed on the local translucent display device 11, the local camera 12 may capture the local user's face, and the local user and the remote user can look directly into each other's eyes. Similarly, the remote camera 14 may also move to a corresponding location, and the remote user and the local user can look directly into each other's eyes.
Certainly, in a video communication process between two users, if one user leaves the field view of the camera, the camera corresponding to the user is no longer moving. For example, when the remote user leaves the room where the video communication system is located, which exceeds the field of view of the remote camera 14, the local camera 12 will not move. At the same time, the field of the video played by the remote translucent display device 13 is no longer changing. However, since the local user is still in front of the local translucent display device 11, the remote camera 14 still moves according to the location of the local user, and the local user can still see remote video information with different view fields.
In one embodiment, a method for video communication suitable for local video communication device 10 is provided. In
In action 1702, the local translucent display device 11 receives the remote video information through the communication module 104. And the remote video information can be processed by the video capture and processing module 101 and displayed on the local translucent display device 11 via the display module 105.
In action 1703, the position of remote user's image on the local translucent display device 11 can be calculated by the remote video communication device 10′ and then be transmitted to the local video communication device 10. The remote user's image can also be first transmitted to the local video communication device 10, and then the position of remote user's image is calculated by the local video communication device 10.
In action 1704, the local camera 12 is moved to the position corresponding to the position of remote user's image on the local translucent display device 11 via the position control module 106.
When the local video communication device 10 includes the plurality of local cameras 12 arranged in a two-dimensional array, the plurality of local cameras 12 don't move. But only one or more of the plurality of local cameras 12 are selected for next step. The position of the one or more of the plurality of local cameras 12 corresponds to the position of remote user's image on the local translucent display device 11.
In action 1705, the local video information are transmitted to the remote video communication device 10′ via the communication module 104.
The advantages of the method for video communication may include, but are not limited to, the following. The local user and the remote user can look at each other eye to eye while video communication by moving the cameras in real time. Thus, the method can increase the interactive experience of users. When the user moves in front of the translucent display device, the video information will be changed, and the user's immersion is increased.
In
The local display 31 may be used to display the remote video information. The local display 31 may include a normal display such as LED display, LCD display. The local identification device 32 is placed on the local display 31. The local identification device 32 may include one or more optical cameras, infrared cameras, and/or position detection units for use with position sensor worn by the user. In one embodiment, the local display 31 may include an LED display, and the local identification device 32 may include a Kinect device.
The remote camera 33 is placed at a remote location and capable of capturing the remote video information at the remote location. The remote camera 33 may be a movable camera, an array of cameras in a plurality of different orientations, or a wide angle camera. Thus, the remote camera 33 may capture a wide range of scenes. If the remote camera 33 is a movable camera, the remote camera 33 may move in a similar manner to the local camera 12.
The window system 30 may further comprise an information processor 34. The information processor 34 may be connected to the local terminal and the remote camera 33. The information processor 34 may calculate a range of the local user's visual field according to the face location information of the local user, select video information captured by the remote camera 33 according to the local user's visual field, and transfer the video information back to the local display 31. Thus, the local user can see remote real-time scenes. Furthermore, the information processor 34 can calculate a range of the local user's visual field according to the eyes' location information of the local user. As the local user moves, the local identification device 32 can identify changes of the local user's position, and the video information transferred back to the local display 31 changes according to the local user's position. When the remote camera 33 includes a plurality of cameras arranged in a two-dimensional array, the information processor 34 may select one or more of the plurality of cameras corresponding to the face location of the local user, calculate the video information captured by the one or more of the plurality of cameras, and transmit the video information back to the local display 31. Thus, when the local user moves in front of the local display 31, the video scene seen by the local user may be different.
The window system 30 may further comprise a local speaker, a local lighting device and a local spray device. The local speaker, the local lighting device and the local spray device can be used in conjunction with the local display 31 to simulate sound, light, and air humidity in the remote scene. The microphone, photosensitive device, air humidity detector can also be set in the remote scene accordingly.
When the remote scene is a beach scene, after the window system 30 is turn on, the local user feels like on the beach. The local user can see the blue sea, hear the sound of the waves, and feel the sun and the humid air. Thus, the local user can have an immersive experience. When the local user moves, the remote scene seen by the local user is also changed, and the local user feels like walking on the beach.
In
The window system 40 is substantially similar to the window system 30. Among the differences, the window system 40 includes the plurality of local displays 41. The information processor 44 may transmit video information to the plurality of local displays 41 according to each local user's visual field so that a plurality of local users can watch the video information at the same time. The plurality of cameras in the remote camera array 43 may respectively select different visual fields. Thus, the remote camera array 43 may simultaneously capture video information of different visual fields. Since positions of local users in front of each local display 41 may be different, the position information transmitted to the remote camera array 43 by the local identification devices 42 may also be different. The information processor 44 may calculate local user's visual field according to the position information and transmit video information to the plurality of local displays 41 according to each local user's visual field. The window system 40 may be simultaneously utilized by a plurality of local users at different places, and the plurality of local users can see different visual fields based on their different position information.
The advantages of the window system may include, but are not limited to, the following. The local user can watch different remote scenes according to different position information. The local user can interact with remote scenes in real time and have an immersive feeling.
The embodiments shown and described above are only examples. Even though numerous characteristics and advantages of the present technology have been set forth in the foregoing description, together with details of the structure and function of the present disclosure, the disclosure is illustrative only, and changes may be made in the detail, including in matters of shape, size, and arrangement of the parts within the principles of the present disclosure, up to and including the full extent established by the broad general meaning of the terms used in the claims.
Depending on the embodiment, certain of the actions of methods described may be removed, others may be added, and the sequence of actions may be altered. The description and the claims drawn to a method may comprise some indication in reference to certain action. However, the indication used is only to be viewed for identification purposes and not as a suggestion for ordering the action.
Number | Date | Country | Kind |
---|---|---|---|
2018 1 0264376 | Mar 2018 | CN | national |
Number | Name | Date | Kind |
---|---|---|---|
3794421 | Kano | Feb 1974 | A |
4928301 | Smoot | May 1990 | A |
8970663 | Gagneraud | Mar 2015 | B2 |
9330589 | Cope | May 2016 | B2 |
9462223 | White | Oct 2016 | B2 |
20050007445 | Foote | Jan 2005 | A1 |
20080043100 | Sobel et al. | Feb 2008 | A1 |
20090102763 | Border et al. | Apr 2009 | A1 |
20090278913 | Rosenfeld et al. | Nov 2009 | A1 |
20130106983 | Fritsch | May 2013 | A1 |
20170264865 | Huangfu | Sep 2017 | A1 |
20190019023 | Konttori | Jan 2019 | A1 |
Number | Date | Country |
---|---|---|
203966475 | Nov 2014 | CN |
2509309 | Oct 2012 | EP |
I248021 | Jan 2006 | TW |
200923495 | Jun 2009 | TW |
200948070 | Nov 2009 | TW |
Number | Date | Country | |
---|---|---|---|
20190306461 A1 | Oct 2019 | US |