The present disclosure relates to far field microphones provided in remote controls used for controlling external devices such as televisions, and more specifically, to methods of selectively switching power supplies used for such microphones to enhance both far and near field capabilities.
Consumer electronic products are increasingly utilizing speech recognition. There are two approaches to providing this service with respect to remote controls. According to the first approach, a near-field microphone is installed in the remote control, and the user presses a push to talk button to power the microphone when issuing a command or otherwise transmitting speech in the form of audio signals to a device operated by the remote control. According to the second approach, far-field microphones are embedded in the device (e.g., a TV), and the user can speak at any time (usually starting with a “wake-up command”) to trigger the speech recognition process, for example as with Apple's “Siri.”
When used in far field applications, the microphone must stay on and powered at all times in order to be ready respond to user commands issued from a distance. Thus, a power source that continuously supplies power to the microphone is required. In the case of some television remote controls, a power source of AC or DC, typically from a common power adaptor, also known as the ubiquitous “wall wart,” plugged into a power outlet, may be supplied by a base station or dock into which the remote control is inserted. Unfortunately, this means that the microphone cannot be used when the remote is removed from the dock (and hence disconnected from the power source) and used in near field applications. Conversely, if the remote is configured for battery operation, it must either remain continuously on or it must require the user to depress a button to supply power from the battery source within the remote control to the microphone. In the former case, the battery power supply would quickly become drained. In the latter case, far field functionality would be compromised because of the necessity for physically engaging the controller to issue commands. Thus, a need has arisen for a remote control with a dual-function microphone system that addresses the foregoing issues.
In accordance with a first aspect of the present disclosure, a remote control is provided. The remote control comprises a microphone subsystem operatively connected to a transceiver, and the microphone subsystem comprises at least one microphone. The remote control also comprises remote control electrical contacts for connecting the remote control to an external power source and an internal power source. When the remote-control electrical contacts are connected to the external power source, the external power source continuously supplies power to the microphone subsystem, and when the remote-control electrical contacts are not connected to the external power source, the internal power source is selectively connectable to the microphone subsystem. In certain examples, the external power source is provided through a physical connection between the remote control and a remote-control base station or base stand. In the same or other examples in which the remote control is handheld, the internal power source is an internal battery. In certain examples, the at least one microphone is a first microphone and a second microphone. In additional examples, when the remote-control electrical contacts are connected to the external power source, the microphone subsystem processes speech received by the at least two microphones as a far-field microphone array with beam forming. In the same or other examples, when the remote-control electrical contacts are not connected to the external power source, speech received by the at least one microphone is processed as a monaural signal in a near-field modality.
In additional examples, a remote-control system comprising the remote control and a remote-control base station is provided, and the remote-control base station comprises base electrical contacts that are selectively engageable with the remote-control electrical contacts to electrically connect the external power source to the microphone subsystem.
In accordance with a second aspect of the present disclosure, a method of providing speech audio signals to a television is provided. The method comprises providing a remote control having a microphone subsystem comprising at least one microphone, wherein the microphone subsystem is connected to an external power source. The method further comprises disconnecting the microphone subsystem from the external power source, thereby causing the microphone subsystem to be selectively connectable to an internal power source such as a battery. In certain examples, the method further comprises operating a control on the remote to connect the microphone subsystem to the internal power source and speaking into the microphone.
In accordance with a third aspect of the present disclosure, a remote control comprising a video camera is provided. The video camera produces continuous video when the remote control is docked in its base, and the base is connected to an external power source. In certain examples, the external power source is a power adapter plugged into a wall outlet. When the remote control is disconnected from the external power source, it remains in very-low-power mode until a user control is actuated. Upon actuation of the user control, the remote will exit the standby state and activate its circuitry, allowing the camera to capture one or a few images to send via Bluetooth to the TV, typically for facial recognition.
In accordance with a fourth aspect of the present disclosure, a non-transitory computer readable medium is provided which has a set of computer executable instructions stored thereon, wherein when executed by a processor, the instructions perform a method comprising detecting whether a remote control comprising a processor, a user control, and a microphone subsystem operatively connected to a transceiver is electrically connected to an external power source and configuring the microphone subsystem for selective connection to an internal power source in response to a selected user control manipulation if the remote control is not electrically connected to an external power source.
Described below are examples of remote controls having a microphone subsystem comprising at least one microphone. The microphone subsystem is operable in two modes. In an external power mode, the microphone subsystem is electrically connected to an external power source, for example, one provided by a base stand upon or in which the remote control sits. In this mode, the at least one microphone can detect speech at any time because no user engagement with the remote control is required in order for the at least one microphone to detect speech. In an internal power mode, the at least one microphone is disconnected from the external power source and is selectively connectable (by the user) to an internal power source, such as batteries contained within the remote control. In the internal power mode, the user must manipulate a user control, such as a push to talk button, on the remote control in order for the microphone subsystem to be energized by the internal power source, which prevents the batteries from draining quickly. Also, in the external power mode, the microphone subsystem is disconnected from the internal power source and is not selectively connectable to it by the user. In certain examples, the at least one microphone comprises two microphones, and in the external mode, the microphone subsystem processes detected speech as a far-field microphone array audio signal with beam forming, while in the internal power mode, the detected speech is processed as monaural signals.
Turning to
Referring to
Remote control 201 is generally rectangular and configured for hand-held operation to transmit television operational commands to smart TV 100. Remote control 201 includes a keypad 204 comprising a number of user controls each of which is actuatable to transmit a corresponding command to smart TV 100. The commands are transmitted via a near-infrared transmitter (not shown) to a corresponding receiver on smart TV 100 (not shown). Keypad 204 includes a set of content provider buttons 209 (e.g., VUDU, Netflix, Prime Video, etc.), each of which is manipulable (e.g., by depressing the buttons) to cause content to be transmitted to smart TV 100 from a server corresponding to the content provider.
A ring navigation controller 206 is provided and allows the user to navigate a cursor on the television display 102 to select graphics corresponding to desired commands. Enter button (“OK”) sits in the center of the ring controller 206 and is actuatable to enter a command corresponding to whatever on-screen graphic the user has selected with ring controller 206. Keypad 204 also includes a home button 205, volume up 210a, volume down 210b, and mute 212 buttons.
Remote control 201 also includes a microphone subsystem 715 (
Referring to
Contact pairs 216a and 216b and 304a and 304b may take a variety of forms. However, in the example of
Referring to
In the example of
Microphone subsystem 715 may also include an analog to digital converter 710 and/or a digital signal processor 711. In the example of
Microphones 707 and 708 receive acoustic soundwaves generated by a user's speech and convert them into analog audio signals that are amplified by pre-amplifiers 709a and 709b. The analog to digital converter 710 is connected to the first and second pre-amplifiers 709a and 709b, and the output of analog to digital converter 710 is a digital audio signal supplied to Digital Signal Processor 711 (DSP). Digital signal processor 711 in turn provides a processed digital audio signal to the Bluetooth transceiver 712 which transmits the signal to smart TV 100 from antenna 716. The transmitted signal is received by smart TV Wi-Fi antenna 1424a and network processor 1424 (
When the microphone subsystem 715 is connected to a source of external power, such as when the remote control 201 is docked in base station 300, signals from the microphone pair 707, 708 are processed by DSP 711 to provide a far-field mic array with beam forming, wherein the received audio provided by microphones 707 and 708 is focused to a narrow angle and adjusted in direction to find the highest volume of audio upon which it ceases scanning. When the user speaks into microphones 707 and 708, the resulting digital audio signal provided by the digital signal processor 711 is received via Bluetooth transceiver 712 and antenna 716 at the television wi-fi antenna 1424a, and speech recognition is performed by the processor system 1400 of the TV 100 or, in other embodiments, the speech signal is passed on to a cloud-based processing system for recognition. These external power mode signal processing functions may be carried out by CPU 720 executing corresponding computer executable instructions stored on a non-transitory computer readable medium such as flash memory 726.
Remote control 201 also includes an internal DC power source 702. One example of an internal DC power source is a pair of AA or AAA batteries. Referring to
When remote control 201 is removed from base station 300, internal DC power source 702 is selectively connectable to the microphone subsystem 715 by depressing microphone power button 202. As described further below, in this exemplary implementation of an internal power supply mode, the entire remote-control circuit of
As indicated in
Referring again to
When the remote control 201 is removed from the base station 300, the external voltage loss from contact 216a is detected by voltage monitor 721, causing processor 720 to return a power enable signal to power distribution module 713 via power enable signal line 740 to return to a standby state. In certain examples, the sensed voltage arising from the disconnection of the external power source (e.g., base station 300) is compared to a threshold voltage or threshold change in voltage in determining whether to return to a stand-by state.
The DSP 711 is signaled via I2C bus 718 to reconfigure the microphone subsystem for near-field microphone mode. In the near-field microphone mode, microphone 708 is disabled, and audio from microphone 707 is processed as a monaural signal prior to sending audio via Bluetooth 712 to the TV 100. Furthermore, in the off-base, hand-held, near-field mode, the entire remote control 201 electronics operate in a power-saving mode and do not activate to full-power until a user engages a control (e.g., by depressing a keyboard button), and until the engagement is detected on the keyboard matrix 703. The user depressing the microphone power button 202 on remote control 201 is one such event to cause the system to power up.
When operating remote control 201 in a hand-held mode, it is preferable to hold microphone 707 close to the mouth for accurate speech recognition as the voice to background noise will be at a minimum. With the remote 201 held in front of the face, the microphone subsystem 715 is most sensitive when in a near-field configuration, that is without beam forming. In the near-field configuration usually only one microphone is needed for speech transmission duties. Hence, when the remote control 201 is removed from base station 300, the processor 720 of the remote control 201 sends a data configuration signal via the I2C bus to the DSP 711 to cause the DSP 711 to process the digital audio from microphone 707 as a simple monaural signal with bandpass filtering that is optimal for speech recognition (300 Hz to 3000 Hz). These DC power supply mode signal processing functions may be carried out by CPU 720 executing corresponding computer executable instructions stored on a non-transitory computer readable medium such as flash memory 726.
However, it is not always convenient, nor might the user wish, to have to hold the remote control in front of the mouth. Thus, as mentioned previously, remote control 201 is operable in a far-field mode when the remote control 201 is docked in base station 300 as shown in
In accordance with one method of use of remote control 201, a user docks remote control 201 in base station 300 so that remote control electrical contacts 216a and 216b are in electrical contact with base contacts 304a and 304b. Voltage monitor 721 senses a voltage on remote control contact 216a, and power is enabled to power distribution module 713, causing power to be continuously supplied to microphone subsystem 715.
While at a distance from remote 201 (possibly at a distance at which the remote is beyond the user's reach), the user issues an oral “wake-up command” to begin using the microphones 707 and 708. The user then issues selected television operational commands, such as “go to Netflix”, “increase volume,” etc. The commands are amplified by the microphone preamplifiers 709a and 709b, converted to digital signals in analog to digital converter 710, and processed into far-field signals via a beam forming algorithm in digital signal processor 711. The far-field signals are transmitted to the television 100 via Bluetooth transceiver 712 and antenna 716. The transmitted commands are received by television at Wi-Fi antenna 1424a and are processed by network processor 1424 for use by any one of a number of apps 1406.
The user then decides to watch a TV show and removes remote control 201 from remote control base station 300, thereby disconnecting contacts 216a and 216b from the base stand source. The remote circuitry of
The speech amplified by microphone pre-amplifier 709a is converted to a digital signal in analog to digital converter 710 and is processed by digital signal processor 711 as a simple monaural signal with a bandpass filtering optimal for speech recognition (300 Hz to 3000 Hz) which is then transmitted to the TV 100 via Bluetooth transceiver 712 and antenna 716. The digital signal is received by TV Wi-Fi antenna 1424a and is processed by network processor 1424 for use by app <n> 1406.
In certain examples, it is convenient to provide a video camera 719 on remote control 201. As one example, such a video camera 719 may be useful for capturing user images processed by facial recognition software. In one implementation, video camera 719 captures an image or images of the user upon depressing the microphone power button 202 while holding the remote control 201. In this environment, the image or images are conveyed to the TV100 of the invention that provides, among other uses, facial recognition for many useful applications such as security or online purchases.
In another embodiment, the video camera 719 remains energized whenever remote control 201 is docked in base station 300, and base station 300 is connected to a source of alternating current. In this embodiment, video camera 719 provides continuous video image data of the room or other area in which it is located. In this embodiment, the continuous video from the remote-control is transmitted via Bluetooth 712 to the TV 100 for applications such as motion detection for home security, video games, video watch parties, and video conferencing applications, to name but a few. In all of the continuous video use cases either the user is not present, or the user wishes not to hold the remote 201. In another example, the user may speak a reaction to a particular piece of content which is then transmitted as an audio or text message to another viewer watching the same content on another TV and/or stored in a database record keyed to the user and the content for future use in presenting content recommendations to the user. Video camera 719 may also capture facial expressions, which App <n> 1406 transmits to other viewers watching the same content or otherwise may use to identify future content recommendations.
The processing system supporting the functionality of the smart TV 100 as disclosed herein, is summarized in
In addition to providing a video camera 719 on remote 201, TV 100 may also have one or more video cameras 101. Additional information regarding the environment directly in front of the TV 100 may be collected by one or more video camera systems 101 integrated into or associated with the TV 100. In
The one or more instances of video camera 101 in combination with the camera processor 1402 associated with the smart TV system provides digital picture information to the processing system 1400. The processing system 1400 is typically implemented as a system-on-a-chip (SOC) 1403 consisting of a CPU 1407, a Graphical Processing Unit (GPU) 1406, RAM 1408, permanent storage (e.g.—flash memory) 1408, a video frame buffer 1405, a specialized Artificial Intelligence (AI) processor 1423 and other necessary elements for use in a processor system of a smart TV. The camera information 1402a (video stream) of the disclosure may be processed by the Video Frame Processor 1407 under the control of App Manager 1410 running in the memory of the SOC 1403 which processes the incoming camera video stream to act on the video information under the control of the particular application running in TV App <n> 1406, where “n” refers to a n integer corresponding to a particular application.
The TV App <n> 1406 may also be executing a video calling or conferencing application or executing an entertainment application such as a video “watch party” or otherwise processing video, both processing incoming video from the other end or ends of a video conference call as well as providing the network streaming to send the processed video of the Camera Processor 1402 through the Internet to the other parties of a multi-way video application. The App Manager 1410 may assist the one or more TV Apps <n> 1406 in processing the video broadcasts received by the TV Tuner 1425 or the HDMI Input 1420 received from a set-top box, or video received over the Internet by IP Network Interface 1422. In all examples, the App Manager 1410 does, among other things, the processing of the composite video output of any TV Apps <n> 1406 that are currently active in memory so that the composite video picture involving local and remote video sources and whatever other elements such as graphic overlays generated by TV Apps <n> 1406 are scaled and positioned appropriate to the executing application or service.
This application claims the benefit of U.S. Provisional Patent Application No. 63/172,392, filed on Apr. 8, 2021, the entirety of which is hereby incorporated by reference.
Number | Name | Date | Kind |
---|---|---|---|
5267323 | Kimura | Nov 1993 | A |
6043626 | Snyder | Mar 2000 | A |
6584439 | Geilhufe et al. | Jun 2003 | B1 |
20050046751 | Simmons | Mar 2005 | A1 |
20130335196 | Zhang et al. | Dec 2013 | A1 |
20170070066 | Ng | Mar 2017 | A1 |
Entry |
---|
Roettgers, Janko, CES Trend: “Your Next Smart Speaker May Be a TV”, Jan. 9, 2019, Variety.com https://variety.com/2019/digital/news/smart-tvs-far-field-voice-control-1203103141/. |
Number | Date | Country | |
---|---|---|---|
20220329938 A1 | Oct 2022 | US |
Number | Date | Country | |
---|---|---|---|
63172392 | Apr 2021 | US |