Patent Application
20240348977
This application claims priority based on Japanese Patent Application No. 2021-116005 filed Jul. 13, 2021, the entire disclosure of which is hereby incorporated by reference.
The present disclosure relates to an electronic device, a program, and a system.
Recently, the technology referred to as remote conferencing, such as web conferencing or video conferencing, has become increasingly common. Remote conferencing involves the use of electronic devices (or systems that include electronic devices) to achieve communication among participants located in multiple places. As an example, suppose a scenario in which a meeting is to be held in an office, and at least one of the meeting participants uses remote conferencing to join the meeting remotely from home. In this situation, audio and/or video of the meeting in the office is acquired by an electronic device installed in the office, for example, and transmitted to an electronic device installed in the home of the participant, for example. Audio and/or video in the home of the participant is acquired by an electronic device installed in the home of the participant, for example, and transmitted to an electronic device installed in the office, for example. Such electronic devices allow the meeting to take place without having all participants gather at the same location.
The related art has proposed a variety of technologies that could be applied to remote conferencing as described above. For example, Patent Literature 1 discloses a device that displays a graphic superimposed on an image captured by a camera. The graphic represents the output range of directional sound outputted by a speaker. This device enables a user to visually understand the output range of directional sound.
According to one embodiment, an electronic device includes a communicator, a sound outputter, and a controller. The communicator communicates with a terminal of a human speaker and communicates with another electronic device that outputs at least one of a predetermined auditory effect or a predetermined visual effect. The sound outputter outputs an audio signal of the human speaker that the communicator receives from the terminal as sound of the human speaker. The controller sets a volume of the sound that the sound outputter outputs. The controller controls the other electronic device to output at least one of the auditory effect or the visual effect when a level of the sound at a position of a candidate that is to be a recipient of the sound is changed at the terminal.
According to one embodiment, a program causes an electronic device to execute the following:
According to one embodiment, an electronic device includes a communicator, an outputter, and a controller. The communicator communicates with another electronic device that outputs sound of a human speaker. The outputter outputs at least one of a predetermined auditory effect or a predetermined visual effect. The controller controls the outputter to output at least one of the auditory effect or the visual effect on a basis of a request from the other electronic device.
According to one embodiment, a program causes an electronic device to execute the following:
According to one embodiment, a system includes first and second electronic devices capable of communicating with one another.
The first electronic device includes a sound outputter and a controller. The sound outputter outputs an audio signal of a human speaker received from a terminal of the human speaker as sound of the human speaker. The controller sets a volume of the sound that the sound outputter outputs. The controller controls the second electronic device to output at least one of a predetermined auditory effect or a predetermined visual effect when a level of the sound at a position of a candidate that is to be a recipient of the sound is changed at the terminal.
The second electronic device includes an outputter and a controller. The outputter outputs at least one of a predetermined auditory effect or a predetermined visual effect. The controller controls the outputter to output at least one of the auditory effect or the visual effect on a basis of a request from the first electronic device.
According to one embodiment, a system includes a first electronic device, a second electronic device, and a terminal of a human speaker.
The terminal and the first electronic device are configured to communicate with one another. The first electronic device and the second electronic device are configured to communicate with one another.
The first electronic device includes a sound outputter and a controller. The sound outputter outputs an audio signal of a human speaker received from the terminal as sound of the human speaker. The controller sets a volume of the sound that the sound outputter outputs. The controller controls the second electronic device to output at least one of a predetermined auditory effect or a predetermined visual effect when a level of the sound at a position of a candidate that is to be a recipient of the sound is changed at the terminal.
The second electronic device includes an outputter and a controller. The outputter outputs at least one of a predetermined auditory effect or a predetermined visual effect. The controller controls the outputter to output at least one of the auditory effect or the visual effect on a basis of a request from the first electronic device.
The terminal includes a sound pickup and a controller. The sound pickup picks up sound of the human speaker. The controller performs control to transmit an audio signal of the human speaker to the first electronic device and to transmit, to the first electronic device, input for changing a level of the sound at a position of a candidate that is to be a recipient of the sound.
According to one embodiment, an electronic device is capable of communicating with each of a terminal of a human speaker, a first electronic device that outputs an audio signal of the human speaker as sound of the human speaker, and a second electronic device that outputs at least one of a predetermined auditory effect or a predetermined visual effect.
The electronic device includes a controller that sets a volume of the sound that the first electronic device outputs.
When a level of the sound at a position of a candidate that is to be a recipient of the sound is changed at the terminal, the controller changes, or leaves unchanged, the volume of the sound, controls the first electronic device to output the sound, and controls the second electronic device to output at least one of the auditory effect or the visual effect.
According to one embodiment, a program causes an electronic device to execute the following:
According to one embodiment, a system includes a terminal of a human speaker, a first electronic device, a second electronic device, and a third electronic device.
The third electronic device is configured to communicate with the terminal, the first electronic device, and the second electronic device.
The terminal includes a sound pickup and a controller. The sound pickup picks up sound of the human speaker. The controller performs control to transmit an audio signal of the human speaker to the third electronic device and to transmit, to the third electronic device, input for changing a level of the sound at a position of a candidate that is to be a recipient of the sound.
The first electronic device includes a sound outputter that outputs an audio signal of a human speaker received from the terminal as sound of the human speaker.
The second electronic device includes an outputter and a controller. The outputter outputs at least one of a predetermined auditory effect or a predetermined visual effect. The controller controls the outputter to output at least one of the auditory effect or the visual effect on a basis of a request from the third electronic device.
The third electronic device includes a controller that sets a volume of the sound that the first electronic device outputs. When a level of the sound at a position of a candidate that is to be a recipient of the sound is changed at the terminal, the controller changes, or leaves unchanged, the volume of the sound, controls the first electronic device to output the sound, and controls the second electronic device to output at least one of a predetermined auditory effect or a predetermined visual effect.
In the present disclosure, an “electronic device” may be a device driven by electricity supplied from a power system or a battery, for example. In the present disclosure, a “user” may be an entity (typically a person) that uses, or could use, an electronic device according to one embodiment. A “user” may also be an entity that uses, or could use, a system including an electronic device according to one embodiment. In the present disclosure, “remote conferencing” is a general term for conferencing such as web conferencing or video conferencing, in which at least one participant joins by communication from a different location from the other participant(s).
Further improvement in functionality is desirable for an electronic device that enables communication between multiple locations in remote conferencing and the like. An objective of the present disclosure is to improve the functionality of an electronic device, a program, and a system that enable communication between multiple locations. According to one embodiment, improvement in functionality is possible for an electronic device, a program, and a system that enable communication between multiple locations. The following describes an electronic device according to one embodiment in detail, with reference to the drawings.
As illustrated in
The present disclosure is described mainly from the perspective of the first electronic device 1, referring to the first electronic device 1 as the “electronic device” and the second electronic device as the “other electronic device”. However, the present disclosure may also be described mainly from the perspective of the second electronic device 2, referring to the second electronic device as the “electronic device” and the first electronic device 1 as the “other electronic device”.
As illustrated in
As illustrated in
In one embodiment, a remote conferencing system may include the first electronic device 1, the second electronic device 2, and the terminal 100.
In the present disclosure, the network N as illustrated in
Expressions like the above may have the same and/or similar intention not only when the first electronic device 1 and/or the second electronic device 2 “communicates” with the terminal 100, but also when one “transmits” information to the other, and/or when the other “receives” information transmitted by the one. Expressions like the above may have the same and/or similar intention not only when the first electronic device 1 and/or the second electronic device 2 “communicates” with the terminal 100, but also when any electronic device communicates with any other electronic device.
The first electronic device 1 according to one embodiment may be placed in the meeting room MR as illustrated in
The second electronic device 2 according to one embodiment may be placed in the meeting room MR as illustrated in
The terminal 100 according to one embodiment may be placed in the home RL of the participant Mg in the manner as illustrated in
As described later, the terminal 100 outputs audio and/or video of at least one among the meeting participants Ma, Mb, Mc, Md, Me, and Mf in the meeting room MR. Therefore, the terminal 100 may be placed so that audio and/or video outputted from the terminal 100 reaches the participant Mg. The terminal 100 may be placed so that sound outputted from the terminal 100 reaches the cars of the participant Mg via a speaker, headphones, carphones, or a headset, for example.
Through the remote conferencing system including the first electronic device 1, the second electronic device 2, and the terminal 100 illustrated in
The following describes a functional configuration of the first electronic device 1, the second electronic device 2, and the terminal 100 according to one embodiment.
The first electronic device 1 according to one embodiment may be assumed to be any of a variety of devices. For example, the first electronic device 1 may be a device designed for a specific purpose. As another example, the first electronic device 1 according to one embodiment may be a general-purpose device such as a smartphone, tablet, phablet, notebook computer (notebook PC or laptop), or computer (desktop) connected to a device designed for a specific purpose.
As illustrated in
The controller 10 controls and/or manages the first electronic device 1 as a whole, including the functional units that form the first electronic device 1. To provide control and processing power for executing various functions, the controller 10 may include at least one processor, such as a central processing unit (CPU) or a digital signal processor (DSP), for example. The controller 10 may be achieved entirely with a single processor, with several processors, or with respectively separate processors. The processor may be achieved as a single integrated circuit (IC). The processor may be achieved as a plurality of communicatively connected integrated circuits and discrete circuits. The processor may be achieved on the basis of various other known technologies.
The controller 10 may include at least one processor and a memory. The processor may include a general-purpose processor that loads a specific program to execute a specific function, and a special-purpose processor dedicated to a specific process. The special-purpose processor may include an application-specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include a field-programmable gate array (FPGA). The controller 10 may also be a system-on-a-chip (SoC) or a system in a package (SiP) in which one or more processors cooperate. The controller 10 controls the operations of each component of the first electronic device 1.
The controller 10 may include one or both of software and hardware resources, for example. In the first electronic device 1 according to one embodiment, the controller 10 may be configured by specific means in which software and hardware resources work together. The amplifier 60 described below likewise may include one or both of software and hardware resources. In the first electronic device 1 according to one embodiment, at least one other functional unit may be configured by specific means in which software and hardware resources work together.
Control and other operations by the controller 10 in the first electronic device 1 according to one embodiment is further described later.
The storage 20 may function as a memory to store various information. The storage 20 may store, for example, a program to be executed in the controller 10 and the result of a process executed in the controller 10. The storage 20 may function as a working memory of the controller 10. As illustrated in
The communicator 30 functions as an interface for communicating with an external device or the like in a wired and/or wireless way, for example. The communication scheme carried out by the communicator 30 according to one embodiment may be a wireless communication standard. For example, wireless communication standards include cellular phone communication standards such as 2G, 3G, 4G, and 5G. For example, cellular phone communication standards include Long Term Evolution (LTE), Wideband Code Division Multiple Access (W-CDMA), CDMA2000, Personal Digital Cellular (PDC), the Global System for Mobile Communications (GSM®), and the Personal Handy-phone System (PHS). For example, wireless communication standards include Worldwide Interoperability for Microwave Access (WiMAX), IEEE 802.11, Wi-Fi, Bluetooth®, the Infrared Data Association (IrDA), and near field communication (NFC). The communicator 30 may include a modem that supports a communication scheme standardized by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T), for example. The communicator 30 can support one or more of the above communication standards.
The communicator 30 may include an antenna for transmitting and receiving radio waves and a suitable RF unit, for example. The communicator 30 may use an antenna for instance to communicate wirelessly with a communicator of another electronic device, for example. For example, the communicator 30 may communicate wirelessly with the terminal 100 illustrated in
As illustrated in
The image acquirer 40 may include an image sensor that captures images electronically, such as a digital camera, for example. The image acquirer 40 may include an image sensor that performs photoelectric conversion, such as a charge-coupled device (CCD) image sensor or a complementary metal-oxide-semiconductor (CMOS) sensor. The image acquirer 40 can capture an image of the surroundings of the first electronic device 1, for example. The image acquirer 40 may capture the situation inside the meeting room MR illustrated in
The image acquirer 40 may convert a captured image into a signal and transmit the signal to the controller 10. Accordingly, the image acquirer 40 may be connected to the controller 10 in a wired and/or wireless way. A signal based on an image captured by the image acquirer 40 may also be supplied to a functional unit of the first electronic device 1, such as the storage 20 or the display 90. The image acquirer 40 is not limited to an image capture device such as a digital camera, and may be any device capable of capturing the situation inside the meeting room MR illustrated in
In one embodiment, the image acquirer 40 may, for example, capture the situation inside the meeting room MR as still images at intervals of a predetermined time (such as 15 frames per second, for example). In one embodiment, the image acquirer 40 may, for example, capture the situation inside the meeting room MR as a continuous moving image. The image acquirer 40 may include a fixed-point camera or a movable camera.
The sound pickup 50 detects sound or speech, including human vocalizations, around the first electronic device 1. For example, the sound pickup 50 may detect sound or speech as air vibrations via a diaphragm, for example, and convert the detected air vibrations into an electrical signal. Typically, the sound pickup 50 may include any acoustic device to convert sound into an electrical signal, such as a mike (microphone). In one embodiment, the sound pickup 50 may detect the sound of at least one among the participants Ma, Mb, Mc, and Md in the meeting room MR illustrated in
The sound pickup 50 may convert collected sound or speech into an electrical signal and supply the electrical signal to the controller 10. The sound pickup 50 may also supply an electrical signal (audio signal) converted from sound or speech to a functional unit of the first electronic device 1, such as the storage 20. The sound pickup 50 may be any device capable of detecting sound or speech inside the meeting room MR illustrated in
The amplifier 60 appropriately amplifies the electrical signal (audio signal) of sound or speech supplied from the controller 10, and supplies an amplified signal to the sound outputter 70. The amplifier 60 may include any device that functions to amplify an electrical signal, such as an amp. The amplifier 60 may amplify an electrical signal (audio signal) of sound or speech according to an amplification factor set by the controller 10. The amplifier 60 may be connected to the controller 10 in a wired and/or wireless way.
In one embodiment, the amplifier 60 may amplify an audio signal that the communicator 30 receives from the terminal 100. The audio signal received from the terminal 100 may be an audio signal of a human speaker (for example, the participant Mg illustrated in
The sound outputter 70 outputs sound or speech by converting into sound an electrical signal (audio signal) appropriately amplified by the amplifier 60. The sound outputter 70 may be connected to the amplifier 60 in a wired and/or wireless way. The sound outputter 70 may include any device that functions to output sound, such as a speaker (loudspeaker). In one embodiment, the sound outputter 70 may include a directional speaker that conveys sound in a specific direction. The sound outputter 70 may also be capable of changing the directivity of sound.
In one embodiment, the sound outputter 70 may output an audio signal of a human speaker (for example, the participant Mg illustrated in
The direction adjuster 80 functions to adjust the direction of sound or speech outputted by the sound outputter 70. The direction adjuster 80 may be controlled by the controller 10 to adjust the direction of sound or speech outputted by the sound outputter 70. Accordingly, the direction adjuster 80 may be connected to the controller 10 in a wired and/or wireless way. In one embodiment, the direction adjuster 80 may include a power source and/or power mechanism such as a servo motor that can change the direction of the sound outputter 70, for example.
However, the direction adjuster 80 is not limited to a device that functions to change the direction of the sound outputter 70. For example, the direction adjuster 80 may also function to change the orientation of the entire enclosure of the first electronic device 1. In this case, the direction adjuster 80 may include a power source and/or power mechanism such as a servo motor that can change the orientation of the enclosure of the first electronic device 1. Changing the orientation of the enclosure of the first electronic device 1 allows for changing the direction of the sound outputter 70 provided in the first electronic device 1 as a result. The direction adjuster 80 can also change the orientation of the enclosure of the first electronic device 1 and thereby change the direction of the video (image) captured by the image acquirer 40.
The direction adjuster 80 may also be provided to a device such as a pedestal or trestle on which to place the enclosure of the first electronic device 1. In this case, the direction adjuster 80 likewise may include a power source and/or power mechanism such as a servo motor that can change the orientation of the enclosure of the first electronic device 1. As above, the direction adjuster 80 may function to change the direction or orientation of at least one of the sound outputter 70 or the first electronic device 1.
When the sound outputter 70 includes a directional speaker, for example, the direction adjuster 80 may also function to adjust (change) the directivity of sound or speech outputted by the sound outputter 70.
In one embodiment, the direction adjuster 80 may adjust the direction of sound of a human speaker (for example, the participant Mg illustrated in
The display 90 may be a display device of any type, such as a liquid crystal display (LCD), an organic electro-luminescence (EL) panel, or an inorganic electro-luminescence (EL) panel, for example. The display 90 may display various types of information such as characters, figures, or symbols. The display 90 may also display objects, icon images, and the like forming various GUI elements for prompting a user to operate the first electronic device 1, for example.
Various data necessary for displaying information on the display 90 may be supplied from the controller 10 or the storage 20, for example. Accordingly, the display 90 may be connected to the controller 10 or the like in a wired and/or wireless way. When the display 90 includes an LCD, for example, the display 90 may also include a backlight or the like where appropriate.
In one embodiment, the display 90 may display video based on a video signal transmitted from the terminal 100. The display 90 may display video of, for example, the participant Mg captured by the terminal 100 as the video based on a video signal transmitted from the terminal 100. Displaying video of the participant Mg on the display 90 of the first electronic device 1 enables persons such as the participants Ma, Mb, Mc, Md, Me, and Mf illustrated in
The display 90 may display unmodified video of the participant Mg captured by the terminal 100, for example. On the other hand, the display 90 may display an image (for example, an avatar) showing the participant Mg as a character, for example.
In one embodiment, the first electronic device 1 may be a device designed for a specific purpose, as described above. On the other hand, in one embodiment, the first electronic device 1 may include the sound outputter 70 and the direction adjuster 80 from among the functional units illustrated in
The second electronic device 2 illustrated in
When giving the second electronic device 2 the same and/or similar configuration as the first electronic device 1, the configuration of each functional unit is already explained in
In another example, the communicator 30 of the second electronic device 2 may communicate wirelessly with the terminal 100 illustrated in
On the other hand, the second electronic device 2 illustrated in
When giving the second electronic device 2 a different configuration from the first electronic device 1, the second electronic device 2 may be, for example, an electronic device including the controller 10, communicator 30, amplifier 60, and sound outputter 70 illustrated in
As illustrated in
The controller 110 controls and/or manages the terminal 100 as a whole, including the functional units that form the terminal 100. Basically, the controller 110 may have a configuration based on the same and/or similar concept as the controller 10 illustrated in
The storage 120 may function as a memory to store various information. The storage 120 may store, for example, a program to be executed in the controller 110 and the result of a process executed in the controller 110. The storage 120 may function as a working memory of the controller 110. As illustrated in
The communicator 130 functions as an interface for communicating in a wired and/or wireless way. The communicator 130 may use an antenna for instance to communicate wirelessly with a communicator of another electronic device, for example. For example, the communicator 130 may communicate wirelessly with the first electronic device 1 illustrated in
The image acquirer 140 may include an image sensor that captures images electronically, such as a digital camera, for example. The image acquirer 140 may capture the situation inside the home RL illustrated in
The sound pickup 150 detects sound or speech, including human vocalizations, around the terminal 100. For example, the sound pickup 150 may detect sound or speech as air vibrations via a diaphragm, for example, and convert the detected air vibrations into an electrical signal. Typically, the sound pickup 150 may include any acoustic device to convert sound into an electrical signal, such as a mike (microphone). In one embodiment, the sound pickup 150 may detect sound of the participant Mg in the home RL illustrated in
The sound outputter 170 outputs sound or speech by converting into sound an electrical signal (audio signal) outputted from the controller 110. The sound outputter 170 may be connected to the controller 110 in a wired and/or wireless way. The sound outputter 170 may include any device that functions to output sound, such as a speaker (loudspeaker). In one embodiment, the sound outputter 170 may output sound detected by the sound pickup 50 of the first electronic device 1. In this case, the sound detected by the sound pickup 50 of the first electronic device 1 may be the sound of at least one among the participants Ma, Mb, Mc, and Md in the meeting room MR illustrated in
The display 190 may be a display device of any type, such as a liquid crystal display (LCD), an organic electro-luminescence (EL) panel, or an inorganic electro-luminescence (EL) panel, for example. Basically, the display 190 may have a configuration based on the same and/or similar concept as the display 90 illustrated in
The display 190 may also be a touchscreen display that functions as a touch panel to detect input by a finger of the participant Mg or a stylus, for example.
In one embodiment, the display 190 may display video based on a video signal transmitted from the first electronic device 1. The display 190 may display video of, for example, the participants Ma, Mb, Mc, Md, and the like captured by (the image acquirer 40 of) the first electronic device 1 as the video based on a video signal transmitted from the first electronic device 1. Displaying video of the participants Ma, Mb, Mc, Md, and the like on the display 190 of the terminal 100 enables the participant Mg illustrated in
The display 190 may display unmodified video of the participants Ma, Mb, Mc, Md, and the like captured by the first electronic device 1, for example. On the other hand, the display 190 may display images (for example, avatars) showing the participants Ma, Mb, Mc, Md, and the like as characters, for example.
In one embodiment, the terminal 100 may be a device designed for a specific purpose, as described above. On the other hand, in one embodiment, the terminal 100 may include some of the functional units illustrated in
In particular, a device such as a smartphone or a notebook computer often has most if not all of the functional units illustrated in
The following describes operations by the first electronic device 1, the second electronic device 2, and the terminal 100 according to one embodiment. As illustrated in
In other words, the first electronic device 1 according to one embodiment is installed in the meeting room MR and detects the sound of at least one among the participants Ma, Mb, Mc, and Md. The sound detected by the first electronic device 1 is transmitted to the terminal 100 installed in the home RL of the participant Mg. The terminal 100 outputs the sound of at least one among the participants Ma, Mb, Mc, and Md received from the first electronic device 1. Thus, the participant Mg can listen to the sound of at least one among the participants Ma, Mb, Mc, and Md.
The second electronic device 2 according to one embodiment is installed in the meeting room MR and can output at least one of a predetermined auditory effect or a predetermined visual effect, for example, to at least one among the participants Ma, Mb, Mc, and Md. In one embodiment, the second electronic device 2 may output a masking sound including an operating sound or an environmental sound, for example, as the predetermined auditory effect to at least one among the participants Ma, Mb, Mc, and Md. In one embodiment, the second electronic device 2 may output a light drawing the attention of the participant, for example, as the predetermined visual effect to at least one among the participants Ma, Mb, Mc, and Md.
On the other hand, the terminal 100 according to one embodiment is installed in the home RL of the participant Mg and detects the sound of the participant Mg. The sound detected by the terminal 100 is transmitted to the first electronic device 1 installed in the meeting room MR. The first electronic device 1 outputs the sound of the participant Mg received from the terminal 100. Thus, at least one among the participants Ma, Mb, Mc, and Md can listen to the sound of the participant Mg.
The following describes operations by the first electronic device 1, the second electronic device 2, and the terminal 100 according to one embodiment, mainly from the perspective of the first electronic device 1 and second electronic device 2. Operations by the first electronic device 1, the second electronic device 2, and the terminal 100 according to one embodiment may include roughly three phases. In other words, operations by the first electronic device 1, the second electronic device 2, and the terminal 100 according to one embodiment may include the following:
The first electronic device 1, the second electronic device 2, and the terminal 100 are assumed to be connected in a wired and/or wireless way at the time when the operations illustrated in
When the operations illustrated in
After step S11, the controller 10 of the first electronic device 1 may acquire the position, in the meeting room MR, of a candidate (hereinafter also referred to as the “recipient candidate”) to which to convey sound (the sound of the participant Mg) received from the terminal 100 (step S12). In the situation illustrated in
In step S12, the position of the recipient candidate may be stored as a predetermined position, for example, in the storage 20, for example. In one example, when chairs or the like in the meeting room MR are arranged in predetermined positions, the controller 10 of the first electronic device 1 can acquire the position of the recipient candidate in advance. When chairs or the like in the meeting room MR are arranged in predetermined positions but the position of the recipient candidate has not been acquired in advance, the controller 10 of the first electronic device 1 may also acquire the position via the communicator 30, for example. If the position of the recipient candidate has not been acquired in advance, the controller 10 of the first electronic device 1 may also detect user input of the position via a keyboard or other input device, for example.
In step S12, the controller 10 of the first electronic device 1 may also detect the position of the recipient candidate with the image acquirer 40, for example. The image acquirer 40 is capable of capturing an image of the surroundings of the first electronic device 1. Accordingly, the image acquirer 40 can capture video of the meeting participants Ma, Mb, Mc, and Md in the meeting room MR. When the image acquirer 40 includes a movable camera, the controller 10 of the first electronic device 1 may also capture an image of the surroundings of the first electronic device 1 while changing the direction of the image acquirer 40. When the direction adjuster 80 is capable of changing the orientation of the enclosure of the first electronic device 1, the controller 10 may also capture an image of the surroundings of the first electronic device 1 while changing the direction of the first electronic device 1.
In step S12, the controller 10 of the first electronic device 1 may acquire the positions of the meeting participants Ma, Mb, Mc, and the like (recipient candidates) from an image like the one illustrated in
In step S12, as illustrated in
After acquiring the position of the recipient candidate in step S12, the controller 10 of the first electronic device 1 calculates or acquires the sound level at the position of each recipient candidate (step S13). In other words, the controller 10 of the first electronic device 1 calculates or acquires information indicating how high the sound level is at the position of each recipient candidate in the meeting room MR. The sound level in this case is the level of the sound of the participant Mg that is received from the terminal 100 and outputted from the first electronic device 1.
In step S13, the controller 10 of the first electronic device 1 may calculate or acquire the sound level at each position on the basis of, for example, the position of the first electronic device 1, the direction of the sound outputter 70, and the position of each recipient candidate. The direction of the sound outputter 70 may be the direction of the directivity of sound outputted from the sound outputter 70. The “sound level” may be any of various types of indicators that the recipient candidate can recognize aurally. For example, the “sound level” may be the level of sound pressure or the like.
In step S13, the controller 10 of the first electronic device 1 may acquire the sound level at the position of each recipient candidate from data obtained in advance through a demonstration experiment or the like. Such data may be pre-stored in the storage 20, for example, or may be acquired as needed from outside the first electronic device 1 via the communicator 30.
On the other hand, in step S13, the controller 10 of the first electronic device 1 may calculate or estimate the sound level at the position of each recipient candidate from various data. For example, the controller 10 may calculate or estimate the sound level at the position of each recipient candidate on the basis of the position and direction of the first electronic device 1 (or the sound outputter 70), the sound pressure of sound outputted by the sound outputter 70, the position of each recipient candidate, and the like. In step S13, the controller 10 of the first electronic device 1 may also calculate or estimate the sound level at positions in a predetermined range surrounding each recipient candidate from various data.
After calculating or acquiring the sound level in step S13, the controller 10 of the first electronic device 1 transmits information visually indicating the sound level to the terminal 100 (step S14). In step S14, the controller 10 of the first electronic device 1 may generate information visually indicating the sound level and transmit the information from the communicator 30 of the first electronic device 1 to the communicator 130 of the terminal 100.
The information visually indicating the sound level may be, for example, information that visually suggests the degree to which the sound of the participant Mg outputted by the first electronic device 1 is aurally recognizable at the position of each recipient candidate.
The participants Ma and Mb are displayed with emphasis inside an area A12 illustrated in
As one example of an aspect for visually indicating whether each recipient candidate can recognize the sound of the participant Mg, the controller 10 of the first electronic device 1 may make distinctions through shades of color used to display the images of the participants Ma, Mb, and Mc. As another example of an aspect for visually indicating whether each recipient candidate can recognize the sound of the participant Mg, the controller 10 of the first electronic device 1 may make distinctions through transparency levels used when displaying the images of the participants Ma, Mb, and Mc. As another example of an aspect for visually indicating whether each recipient candidate can recognize the sound of the participant Mg, the controller 10 of the first electronic device 1 may make distinctions through sizes used when displaying the images of the participants Ma, Mb, and Mc. As other aspects for visually indicating whether each recipient candidate can recognize the sound of the participant Mg, the controller 10 of the first electronic device 1 may make distinctions through any of various aspects to be used when displaying the images of the participants Ma, Mb, and Mc.
In this way, the controller 10 of the first electronic device 1 transmits, to the terminal 100, information visually indicating the level of sound of a human speaker (the participant Mg) at the position of a candidate that is to be the recipient of the sound of the human speaker. Thus, the terminal 100 can display, on the display 190, the level of sound of the human speaker at the position of a candidate that is to be the recipient of the sound of the human speaker.
The participant Mg can look at the display on the display 190 of the terminal 100 as illustrated in
As described above, in the (1) UI display phase, the participant Mg can understand via the UI whether their own sound can be heard by the other meeting participants.
When the operations illustrated in
After receiving the information indicating the position and orientation of the first electronic device 1 in step S21, the controller 10 of the second electronic device 2 acquires the position and orientation of the second electronic device 2 (step S22). In step S21, the controller 10 of the second electronic device 2 may acquire or receive the input of information on the position of the second electronic device 2 in advance. For example, the controller 10 of the second electronic device 2 may acquire the information by position information acquiring means such as GPS, or acquire the information by any other means. The controller 10 of the second electronic device 2 may acquire information on the orientation of the second electronic device 2 from the direction adjuster 80, for example, or acquire the information by any other means.
After acquiring the position and orientation of the second electronic device 2 in step S22, the controller 10 of the second electronic device 2 generates a conversion table for converting between a position with respect to the first electronic device 1 and a position with respect to the second electronic device 2 (step S23). For example, in step S23, the controller 10 of the second electronic device 2 may generate a conversion table for converting between coordinates in a coordinate system centered on the position of the first electronic device 1 and coordinates in a coordinate system centered on the position of the second electronic device 2. In this case, the controller 10 of the second electronic device 2 may align the orientation of the first electronic device 1 and the orientation of the second electronic device 2 before generating a conversion table for converting between coordinates in coordinate systems centered on the position of each.
The conversion table generated in step S23 allows the first electronic device 1 and the second electronic device 2 to each recognize the same recipient candidate as the same one. For example, the generated conversion table allows for converting the position of the participant Ma with respect to the first electronic device 1 to a position with respect to the second electronic device 2. As another example, the generated conversion table allows for converting the position of the participant Mb with respect to the second electronic device 2 to a position with respect to the first electronic device 1.
In step S23 illustrated in
Remote conferencing involving the first electronic device 1 and the terminal 100 may or may not have started at the time when the operations illustrated in
When the operations illustrated in
Upon receiving input for changing the sound level from the terminal 100 in step S31, the controller 10 of the first electronic device 1 may transmit, to the terminal 100, information for displaying a screen like the one illustrated in
For example, the participant Mg can change the sound level at the position of the participant Ma by operating a slider Sa corresponding to the participant Ma displayed on the display 190. The participant Mg can change the sound level at the position of the participant Mb by operating a slider Sb corresponding to the participant Mb displayed on the display 190. The participant Mg can change the sound level at the position of the participant Mc by operating a slider Sc corresponding to the participant Mc displayed on the display 190.
In
If the controller 10 of the first electronic device 1 does not receive input for changing the sound level from the terminal 100 in step S31, the controller 10 of the first electronic device 1 may stand by until receiving input for changing the sound level. If the controller 10 of the first electronic device 1 does not receive input for changing the sound level from the terminal 100 in step S31, the controller 10 of the first electronic device 1 may also end the operations illustrated in
In step S32, the controller 10 of the first electronic device 1 calculates or acquires the amplification factor and direction of sound for achieving the changed sound level. The “amplification factor” may be an amplification factor that the amplifier 60 uses to amplify an audio signal of a human speaker. The “direction of sound” may be the direction of sound of a human speaker, adjusted by the direction adjuster 80. The controller 10 of the first electronic device 1 may acquire the amplification factor and direction of sound for achieving the changed sound level from data obtained in advance through a demonstration experiment or the like. Such data may be pre-stored in the storage 20, for example, or may be acquired as needed from outside the electronic device 1 via the communicator 30. The controller 10 of the first electronic device 1 may also calculate the amplification factor and direction of sound for achieving the changed sound level from various data. For example, the controller 10 of the first electronic device 1 may calculate the amplification factor and direction of sound for achieving the changed sound level on the basis of the position and direction of the first electronic device 1 (or the sound outputter 70), the sound pressure of sound outputted by the sound outputter 70, the position of each recipient candidate, and the like.
After calculating or acquiring the amplification factor and direction of sound in step S32, the controller 10 of the first electronic device 1 controls at least one of the amplifier 60 or the direction adjuster 80 such that the amplification factor and direction of sound are achieved (step S33). In step S33, the controller 10 does not necessarily control at least one of the amplifier 60 or the direction adjuster 80 in cases where at least one of the amplification factor or the direction of sound is already achieved.
After controlling the amplifier 60 and/or the direction adjuster 80 in step S33, the controller 10 of the first electronic device 1 transmits information visually indicating the changed sound level to the terminal 100 (step S34). In step S34, the controller 10 may generate information visually indicating the changed sound level and transmit the information from the communicator 30 of the first electronic device 1 to the communicator 130 of the terminal 100. In the same and/or similar manner as the information illustrated in step S13 of
In step S34, the controller 10 of the first electronic device 1 may calculate or estimate the sound level at the position of each recipient candidate from various data. For example, the controller 10 of the first electronic device 1 may calculate or estimate the sound level at the position of each recipient candidate on the basis of the position and direction of the first electronic device 1 (or the sound outputter 70), the sound pressure of sound outputted by the sound outputter 70, the position of each recipient candidate, and the like. In step S34, the controller 10 of the first electronic device 1 may also calculate or estimate the sound level at positions in a predetermined range surrounding each recipient candidate from various data.
The participant Ma is displayed with emphasis inside an area A22 illustrated in
As illustrated in
Some plausible situations may not allow for optionally differentiating whether each recipient candidate can aurally recognize (in other words, hear or not hear) sound outputted from the sound outputter 70, depending on the sound pressure and/or direction of the outputted sound, for example. In such cases, the controller 10 of the first electronic device 1 may, for example, disallow movement of the slider Sb corresponding to the participant Mb illustrated in
At the time when the operations illustrated in
When the operations illustrated in
When sound input is received from the terminal 100 in step S41, the controller 10 of the first electronic device 1 controls the amplifier 60 to amplify the sound input according to the amplification factor calculated or acquired in step S32 of
After outputting sound in step S42, the controller 10 of the first electronic device 1 determines whether the sound level at the position of the recipient candidate is achievable using only the sound output of the first electronic device 1 (step S43). The sound level to be determined as achievable or not in step S43 may be, for example, the sound level changed in step S31 of
In step S43, if the sound level is determined to be achievable using the sound output of the first electronic device 1, the controller 10 of the first electronic device 1 transmits, to the terminal 100, information visually indicating the level of sound to be outputted (step S44). In step S44, the controller 10 of the first electronic device 1 may generate information visually indicating the level of sound to be outputted from the first electronic device 1 and transmit the information from the communicator 30 of the first electronic device 1 to the communicator 130 of the terminal 100. In the same and/or similar manner as the information illustrated in step S34 of
In step S44, the controller 10 of the first electronic device 1 may calculate or estimate the sound level at the position of each recipient candidate from various data. For example, the controller 10 of the first electronic device 1 may calculate or estimate the sound level at the position of each recipient candidate on the basis of the position and direction of the first electronic device 1 (or the sound outputter 70), the sound pressure of sound outputted by the sound outputter 70, the position of each recipient candidate, and the like. In step S44, the controller 10 of the first electronic device 1 may also calculate or estimate the sound level at positions in a predetermined range surrounding each recipient candidate from various data.
On the other hand, in step S43, only a sound level that is recognizable at the position of the participant Mb may be achievable in some cases, even if the controller 10 of the first electronic device 1 adjusts the level (for example, the amplification factor) and/or the direction of sound to be outputted from the sound outputter 70. In such cases, the controller 10 of the first electronic device 1 transmits to the second electronic device 2 a request to output predetermined sound or speech (step S45). Hereinafter, the request to output predetermined sound or speech that is transmitted to the second electronic device 2 is referred to as the “output request” where appropriate. In step S45, the controller 10 of the first electronic device 1 may transmit the output request from the communicator 30 of the first electronic device 1 to the communicator 30 of the second electronic device 2.
In step S45, the output request transmitted to the second electronic device 2 may contain information including, for example, the position of a person who is undesirable as a recipient (hereinafter also referred to as a “non-recipient”) from among the recipient candidates, for example. The output request may also contain information indicating a target sound level at the position of the non-recipient, for example. The output request may also contain information indicating the level of sound to be outputted from the first electronic device 1, for example. The output request may also contain information indicating the level, at the position of the non-recipient, of sound to be outputted from the first electronic device 1, for example. The output request may also contain information indicating a target level, at the position of the non-recipient, of sound or speech to be outputted from the second electronic device 2, for example.
After transmitting the output request to the second electronic device 2 in step S45, the controller 10 of the first electronic device 1 may execute the process illustrated in step S44.
When the operations illustrated in
Upon receiving the output request in step S51, the controller 10 of the second electronic device 2 controls the sound outputter 70 of the second electronic device 2 to output predetermined sound or speech according to the output request (step S52). The predetermined sound or speech outputted in step S52 may be, for example, a masking sound such as noise outputted to the non-recipient. Causing the second electronic device 2 to output sound or speech according to the output request can reduce the risk of the non-recipient recognizing the sound outputted by the first electronic device 1, even in cases where the non-recipient could recognize the sound outputted by the first electronic device 1 alone, for example.
The participant Ma is displayed with emphasis inside the area A22 illustrated in
In this way, in the first electronic device 1 according to one embodiment, the controller 10 sets an amplification factor that the amplifier 60 uses to amplify an audio signal of a human speaker. The “amplification factor” may be an amplification factor that the amplifier 60 uses to amplify an audio signal of a human speaker. The controller 10 of the first electronic device 1 controls the second electronic device 2 to output at least one of an auditory effect or a visual effect on the basis of input at the terminal 100 for changing the level of sound at the position of a candidate that is to be a recipient of the sound. The second electronic device 2 may output at least one of a predetermined auditory effect or a predetermined visual effect. The visual effect that the second electronic device 2 outputs is described later.
The controller 10 of the first electronic device 1 may set and also change the direction of sound adjusted by the direction adjuster 80. The “direction of sound” may be the direction of sound of a human speaker that the direction adjuster 80 adjusts.
The controller 10 of the first electronic device 1 may transmit a request (output request) for the second electronic device 2 to output at least one of an auditory effect or a visual effect. The controller 10 may transmit the output request from the communicator 30 of the first electronic device 1 to the second electronic device 2. In this case, the controller 10 of the first electronic device 1 may transmit the output request to the second electronic device 2 when the sound of a human speaker to be outputted from the sound outputter 70 is not at or below a predetermined level at the position of a predetermined candidate (for example, the non-recipient) among the recipient candidates. The controller 10 of the first electronic device 1 may also transmit the output request to the second electronic device 2 when the sound of a human speaker to be outputted from the sound outputter 70 exceeds a predetermined level at the position of a predetermined candidate (for example, the non-recipient) among the recipient candidates.
In the first electronic device 1 according to one embodiment, the sound outputter 70 may output an audio signal as the sound of a human speaker. The audio signal in this case may be the audio signal of a human speaker that the communicator 30 receives from the terminal 100. In one embodiment, the controller 10 of the first electronic device 1 may set the volume of sound of a human speaker that the sound outputter 70 is to output. In this case, when the level of sound of the human speaker is changed at the terminal 100, the controller 10 of the first electronic device 1 changes, or leaves unchanged, the volume of sound of the human speaker that the sound outputter 70 is to output. In one embodiment, the first electronic device 1 may, for example, change the amplification factor with the amplifier 60 to thereby change the volume of sound of the human speaker that the sound outputter 70 is to output.
As described above, in the meeting room MR, the first electronic device 1 can output the sound of the participant Mg present in the home RL. In the meeting room MR, the first electronic device 1 can convey sound to only the recipient candidate(s) to whom the participant Mg wants to convey sound. In other words, according to the first electronic device 1 according to one embodiment, a human speaker, namely the participant Mg, can choose the participant(s) who will be able to aurally recognize (in other words, hear) the sound of the participant Mg. Consequently, the first electronic device 1 according to one embodiment can have improved functionality for enabling communication between multiple locations.
Compared to the second electronic device 2 with the same configuration as the first electronic device 1 illustrated in
The second outputter 72 may be provided inside or outside the enclosure of the second electronic device 2. When the second outputter 72 is provided inside the enclosure of the second electronic device 2, a mechanism may be provided to change the direction of the second outputter 72 and/or the directivity of the auditory effect outputted from the second outputter 72. When the second outputter 72 is provided outside the enclosure of the second electronic device 2, a mechanism likewise may be provided to change the direction of the second outputter 72 and/or the directivity of the auditory effect outputted from the second outputter 72. When the second outputter 72 is provided outside the enclosure of the second electronic device 2, (more than one of) the second outputter 72 may be readied to correspond with each of the recipient candidates, such as the participants Ma, Mb, Mc, and Md, for example. The second outputter 72 may also be placed in a position different from the sound outputter 70, so that the second outputter 72 can produce a different auditory effect different from the sound outputter 70.
When the sound of the participant Mg outputted in step S42 of
In this case, the controller 10 of the second electronic device 2 may control the second outputter 72 to output a predetermined auditory effect in step S52 of
In yet another embodiment, the second outputter 72 provided to the second electronic device 2 may also output an ultrasonic wave as the predetermined auditory effect. The second outputter 72 may output an ultrasonic wave to a predetermined part of the non-recipient or to a predetermined part in the vicinity of the non-recipient, thereby drawing the attention of the non-recipient to reflections of the ultrasonic wave. As a result, the non-recipient may direct less attention to the sound of the participant Mg.
In cases like the above, a predetermined visual effect may also be outputted instead of, or together with, a predetermined auditory effect.
Compared to the second electronic device 2 illustrated in
The third outputter 93 may be provided inside or outside the enclosure of the second electronic device 3. When the third outputter 93 is provided inside the enclosure of the second electronic device 3, a mechanism may be provided to change the direction of the third outputter 93 and/or the directivity of the visual effect outputted from the third outputter 93. When the third outputter 93 is provided outside the enclosure of the second electronic device 3, a mechanism likewise may be provided to change the direction of the third outputter 93 and/or the directivity of the visual effect outputted from the third outputter 93. When the third outputter 93 is provided outside the enclosure of the second electronic device 3, (more than one of) the third outputter 93 may be readied to correspond with each of the recipient candidates, such as the participants Ma, Mb, Mc, Md, Me, and Mf, for example.
When the sound of the participant Mg outputted in step S42 of
In this case, the controller 10 of the second electronic device 3 may control the third outputter 93 to output a predetermined auditory effect in step S52 of
As described above, the second electronic device 2 (or 3) according to one embodiment may include a controller 10, a communicator 30, and an outputter (70, 72, 93). The communicator 30 communicates with a first electronic device 1 (other electronic device) that outputs sound of a human speaker. The outputter (70, 72, 93) outputs at least one of a predetermined auditory effect or a predetermined visual effect. The controller 10 controls the outputter (70, 72, 93) to output at least one of an auditory effect or a visual effect on the basis of a request from the first electronic device 1. The outputter (70, 72) may output a predetermined sound wave. The outputter 72 may output a predetermined ultrasonic wave. The outputter 93 may output predetermined light.
The foregoing describes embodiments according to the present disclosure on the basis of the drawings and examples, but note that a person skilled in the art could easily make various alternatives or revisions on the basis of the present disclosure. Consequently, it should be understood that the scope of the present disclosure includes these alternatives or revisions. For example, the functions and the like included in each component, each step, and the like may be rearranged in logically non-contradictory ways. A plurality of components, steps, or the like may be combined into one or subdivided. The foregoing describes embodiments of the present disclosure mainly in terms of a device, but an embodiment of the present disclosure may also be implemented as a method including steps to be executed by each component of a device. An embodiment of the present disclosure may also be implemented as a method or program to be executed by a processor provided in a device, or as a storage medium on which the program is recorded. It should be understood that the scope of the present disclosure includes these embodiments.
The embodiments described above are not limited solely to embodiments of the first electronic device 1 and/or the second electronic device. For example, the embodiments described above may also be carried out as a method of controlling a device like the first electronic device 1 and/or the second electronic device. As a further example, the embodiments described above may also be carried out as a program to be executed by a device like the first electronic device 1 and/or the second electronic device. Such a program is not necessarily limited to being executed only in the first electronic device 1 and/or the second electronic device. For example, such a program may also be executed in a smartphone or other electronic device that works together with the first electronic device 1 and/or the second electronic device.
The embodiments described above can be carried out from any of various perspectives. For example, one embodiment may be carried out as a system that includes the first electronic device 1, the second electronic device, and the terminal 100. As another example, one embodiment may be carried out as another electronic device (such as a server or a control device, for example) capable of communicating with each of the first electronic device 1, the second electronic device, and the terminal 100. In this case, the other electronic device such as a server, for example, may execute at least some of the functions and/or operations of the first electronic device 1 described in the foregoing embodiments. More specifically, the other electronic device such as a server, for example, may set (instead of the first electronic device 1) the amplification factor to be used when the first electronic device 1 amplifies an audio signal to output as sound. In this case, the first electronic device 1 can output the audio signal as sound by amplifying the audio signal according to the amplification factor set by the other electronic device such as a server, for example. One embodiment may be carried out as a program to be executed by the other electronic device such as a server, for example. One embodiment may be carried out as a system that includes the first electronic device 1, the second electronic device 2, the terminal 100, and the other electronic device (third electronic device) such as a server described above.
For example, the third electronic device (other electronic device such as a server, for example) described above may be a component like the electronic device 200 illustrated in
| Number | Date | Country | Kind |
|---|---|---|---|
| 2021-116005 | Jul 2021 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2022/026874 | 7/6/2022 | WO |
