This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2022-166128, filed Oct. 17, 2022, the entire contents of which are incorporated herein by reference.
Embodiments described herein relate generally to a display device and a system.
In recent years, web conference and video call using a display device with an in-camera have become prevalent. In such web conference and video call, the line of sight of a person watching an image (video) displayed on the screen may not correspond to that of a person displayed on the screen.
For this reason, it is desired to realize a technology capable of making the line of sight of a person watching an image displayed on the screen correspond to that of a person displayed in the image.
In general, according to one embodiment, a display device comprises a display panel having a display area where an image is displayed, a plurality of cameras provided at positions overlapping with the display area in plan view to capture a user opposed to the display device as a subject, a controller selecting one of the plurality of cameras as a camera to capture the subject, based on positions of eyes of a person included in the image displayed in the display area.
According to another embodiment, a display device comprises a display panel having a display area where an image is displayed, a plurality of cameras provided at positions overlapping with the display area in plan view, and a controller selecting one of the plurality of cameras as a camera to capture a subject, based on a predetermined position included in the image displayed in the display area.
According to yet another embodiment, a system includes a first display device used by a first user and a second display device used by a second user, and allows the first display device and the second display device to be communicatively connected to each other. The first display device comprises a first display panel having a first display area where an image is displayed, a plurality of first cameras provided at positions overlapping with the first display area in plan view to capture the first user, and a first controller selecting one of the plurality of first cameras as a camera to capture the first user, based on positions of eyes of the second user included in the image displayed in the first display area, and transmitting the image including the first user captured by the selected camera to the second display device. The second display device comprises a second display panel having a second display area where an image is displayed, a plurality of second cameras provided at positions overlapping with the second display area in plan view to capture the second user, and a second controller selecting one of the plurality of second cameras as a camera to capture the second user, based on positions of eyes of the first user included in the image displayed in the second display area, and transmitting the image including the second user captured by the selected camera to the first display device.
Embodiments will be described hereinafter with reference to the accompanying drawings.
The disclosure is merely an example, and the invention is not limited by contents described in the embodiments described below. Modification which is easily conceivable by a person of ordinary skill in the art comes within the scope of the disclosure as a matter of course. In order to make the description clearer, the sizes, shapes and the like of the respective parts may be changed and illustrated schematically in the drawings as compared with those in an accurate representation. Constituent elements corresponding to each other in a plurality of drawings are denoted by like reference numerals and their detailed descriptions may be omitted.
The display device 1 comprises a housing 2, a display panel 3, and a cover glass 4. The display device 1 is a display device comprising self-luminous display elements such as organic light emitting diodes (OLEDs) and micro-LEDs, and can be converted into, for example, a variety of different electronic devices such as smartphones, tablet terminals, monitor devices, PCs, and TVs.
The display panel 3 is a flat panel type display panel in which a first surface F1 shown in
In the example of
In the display device 1, a plurality of cameras 5 are arranged in positions overlapping with the display area DA in plan view. The cameras 5 are arranged in a space between the housing 2 and the display panel 3. The cameras 5 capture images on the first surface F1 side and may be referred to as in-cameras, front-facing cameras, or the like.
The cover glass 4 is supported by, for example, the side portion 2b and covers the first surface F1 of the display panel 3. The cover glass 4 may be adhered to the first surface F1. The camera 5 is arranged between the second surface F2 of the display panel 3 and the bottom portion 2a of the housing 2. For example, the camera 5 is fixed to the bottom portion 2a of the housing 2.
As shown in
Incidentally, in recent years, web conference and video call using a display device with an in-camera have become prevalent. In such web conference and video call, the line of sight of a person watching an image (video) displayed on the screen may not correspond to that of a person displayed on the screen.
In
In
For convenience of description, it is first assumed that the user U2 who is located below the front of the camera 50B and looks at the camera, is captured by the camera 50B provided in the display device 100B. The captured image is transmitted from the display device 100B to the display device 100A via the network.
The display device 100A receives the above image transmitted from the display device 100B and displays the received image on its screen. In this case, as shown on the left side of
On the other hand, as described above, since the line of sight SL1 of the user U1 is oriented in the direction forming the angle θ with the line of sight on camera SLC, the line of sight SL1 of the user U1 does not correspond to the line of sight on camera SLC and the camera 50A cannot capture the user U1 looking at the camera. Therefore, the camera 50A captures the face of the user U1 located in front of the camera 50A, i.e., the face of the user U1 who does not look at the camera. The captured image is transmitted from the display device 100A to the display device 100B via the network.
The display device 100B receives the above image transmitted from the display device 100A and displays the received image on its screen. In this case, as shown on the right side of
As described above, when web conference or video call is performed using the display device 100 comprising one camera 50 arranged in the center of the screen, if the face of one user (user U2) is displaced from the front of the camera, the other user (user U1) stops looking at the camera to make the line of sight correspond to one user (or since the camera does not catch the user's line of sight) and, as a result, one user becomes unable to make the line of sight correspond to the other user.
Therefore, the display device 1 of the embodiment proposes a configuration that can solve such a problem. More specifically, the display device 1 according to the embodiment comprises a plurality of cameras 5 and has a configuration of switching (selecting) the camera to be used for capturing according to a position of a communication partner displayed on the screen.
The controller 10 is a processor which controls the operation of each unit in the display device 1. The controller 10 includes a display control unit 11, an image recognition unit 12, and a camera selection unit 13.
The display control unit 11 controls the display of various images such as images received from an external device (for example, the display device 1 held by the communication partner in web conference or video call) on the display panel 3 via the communication unit 20. The image recognition unit 12 performs an image recognition process for an image displayed on the display panel 3 to specify predetermined positions, for example, positions of a face of a person in included in the image, and the right and left eyes of the person on the display panel 3. The camera selection unit 13 selects the camera 5 closest to the predetermined position, for example, the camera 5 located between the right and left eyes specified by the image recognition unit 12, more specifically, the camera 5 closest to the midpoint of a straight line connecting the right and left eyes, as a camera to capture the subject.
The communication unit 20 is a communication interface for performing wireless communication with an external device (for example, the display device 1 held by the communication partner in web conference or video call) via a network. The image captured by the camera 5 during the web conference or video call is transmitted to the display device 1 held by the communication partner, via the communication unit 20.
Next, the operation of the display device 1 in a case where web conference or video call is performed using the display device 1 of the embodiment will be described with reference to
In
In
For convenience of description, it is first assumed that the user U2 who is located below the front of the camera 5B and looks at the camera, is captured by one of a plurality of cameras 5B provided in the display unit 1B. The captured image is transmitted from the display device 1B to the display device 1A via the network.
The display device 1A receives the above image transmitted from the display device 1B and displays the received image on its screen. In this case, as shown on the left side of
The user U1 moves its line of sight to the lower area of the screen so as to watch the face of the user U2 displayed in the lower area of the screen of the display unit 1A from the front of the camera 5A arranged in the center of the screen, and can thereby make its line of sight correspond to the user U2 looking at the camera. In other words, the user U1 can make the own line of sight SL1 correspond to the line of sight SL2 of the user U2 by looking at the lower area of the screen of the display device 1A.
The display device 1A captures a situation of the user U1 at this time with the selected camera 5A. According to this, unlike the case described with reference to
The display device 1B receives the above image transmitted from the display device 1A and displays the received image on its screen. In this case, as shown on the right side of
As described above, according to the selected camera 5A of the display device 1A, since the user U1 looking at the camera is captured, the user U2 can make its line of sight correspond to the user U1 looking at the camera, by moving its line of sight to the upper area of the screen so as to watch the face of the user U1 displayed in the upper area of the screen of the display device 1B. In other words, the user U2 can make the own line of sight SL2 correspond to the line of sight SL1 of the user U1 by looking at the upper area of the screen.
Even if the position of the communication partner's face is displaced during the series of operations shown in
As described above, since the display device 1 of the embodiment selects the camera 5 provided at the position to which the user's line of sight is moved (more specifically, the camera 5 located between the right and left eyes of the person displayed on the screen) as the camera to capture the user, the user looking at the camera can be captured at any time. For this reason, when web conference or video call is performed using the display device 1 of the embodiment, even if the position of the face of one user (user U2) is displaced and the other user (user U1) moves its line of sight to make its line of side correspond to that of one user during the web conference or video call, the display device 1 can capture the other user looking at the camera. According to this, it is possible to make the line of sight of one user correspond to that of the other user at any time.
When the display device 1 of the embodiment is, for example, a 7-inch smartphone, 8×18 cameras 5 may be provided to be laid over the entire positions overlapping with the display area DA, as shown in
In addition, when the display device 1 of the embodiment is, for example, a 13-inch monitor device, 20×12 cameras 5 may be provided to be laid over the entire positions overlapping with the display area DA, as shown in
Furthermore, as shown in
Incidentally, the display device 1 of the embodiment may perform an initializing operation to be described below before the series of operations shown in
The display device 1A performs so-called face tracking using one of a plurality of cameras 5A provided in the display device 1A to specify the position of the face of the user U1 which is the subject, and the positions of the right and left eyes of the user U1. The display device 1A generates a virtual line VL extending perpendicularly from a midpoint of a straight line connecting right and left eyes of a specified user U1 toward the display device 1A (display panel 3), and selects the camera 5A arranged at a position intersecting with the virtual line VL as a first camera 5A to capture the user U1.
In addition, when receiving an image transmitted from the display device 1B, the display device 1A adjusts the display position of the image such that the above-described first camera 5A is positioned between the right and left eyes of the user U2 included in the image. In other words, the display device 1A adjusts the display position of the image including the user U2 such that the face of the user U2 is displayed in front of the user U1.
By performing the initializing operation described above, the display device 1A can start web conference or video call in a state of displaying the face of the user U2 in front of the user U1 and selecting the camera 5A located in front of the user U1 as the camera to capture the user U1. After the initializing operation is completed, the display device 1A is set to perform the series of operations shown in
Incidentally, the display device LA side has been described as an example, but the same initializing operation is also performed on the display device 1B side. In addition, for face tracking in the initializing operation, the cameras 5 arranged in the upper area above the center of the screen (for example, the cameras 5A and 5B with hatch lines in
It has been described that the display device 1 is used for web conference or video call and the lines of sight of the plurality of users using the display device 1 are made to correspond to one another in the above-described embodiment, but the display device 1 according to the embodiment can also be applied to a case of capturing the own figure using an in-camera.
For example, as shown in
In contrast, when the user U captures the own figure using the display device 1 of the embodiment, as shown in
The display device 1 according to the embodiment described above comprises a display panel 3 having a display area DA where images are displayed, a plurality of cameras 5 provided at positions overlapping with the display area DA in plan view and capturing a user of the display device 1 as a subject, and a controller 10 selecting one of the plurality of cameras 5 as a camera to capture the subject, based on positions of eyes of the person included in the image displayed in the display area DA. According to this, the line of sight of the person looking at the image displayed on the screen can be made to correspond to that of the person displayed in the image.
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel embodiments described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the embodiments described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
Number | Date | Country | Kind |
---|---|---|---|
2022-166128 | Oct 2022 | JP | national |