An example embodiment of the invention relates generally to audio management technology and, more particularly, relates to a method, apparatus, and computer program product for capturing one or more directional sound fields in communication devices.
The modern communications era has brought about a tremendous expansion of wireline and wireless networks. Computer networks, television networks, and telephony networks are experiencing an unprecedented technological expansion, fueled by consumer demand. Wireless and mobile networking technologies have addressed related consumer demands, while providing more flexibility and immediacy of information transfer.
Current and future networking technologies continue to facilitate ease of information transfer and convenience to users. Due to the now ubiquitous nature of electronic communication devices, people of all ages and education levels are utilizing electronic devices to communicate with other individuals or contacts, receive services and/or share information, media and other content. One area in which there is a demand to increase ease of information transfer relates to the delivery of services to communication devices. The services may be in the form of applications that provide audio features. Some of the audio features of the applications may be provided by microphones of a communication device.
At present, the positions of the microphones in a communication device such as a mobile device may be limited which may create problems in achieving optimal audio output. Currently, some existing solutions address these problems by utilizing beamforming technology to produce beams to facilitate directional audio capture.
The directional beam quality may be determined by the number and locations of the microphones of a communication device used to construct the beams. However, the possible microphone positions may be limited, for example, in a mobile device. As such, the microphones may not necessarily be placed to achieve optimal beamforming. As one example, in a mobile device such as a mobile phone or a tablet computer, one side of the mobile device may be mostly covered by a screen, where microphones may be unable to be placed.
Furthermore, the microphones are usually placed to optimize the functioning of other applications. For example, in a mobile phone there may be a microphone for telephony usage, another microphone for active noise cancellation, and another microphone for audio capture related to video recording. The distance between these microphones may be too large for the conventional beamforming approach since the aliasing effect may take place in an instance in which the distance of the microphones is larger than half the wavelength of sound. This may limit the frequency band of operation for a beamformer. For example, in an instance in which there are two microphones that are located in the opposite ends of the mobile phone, their mutual distance may be several centimeters. This may limit the beamformer usage to low frequencies (for example, for a microphone distance of 10 centimeters (cm), the theoretical limit of the beamformer usage is less than 1.7 kilo hertz (kHz) in the frequency domain). As such, at present, the positions of microphones in communication devices may be too far apart which may cause problems in forming beams to achieve optimal audio.
A method, apparatus and computer program product are therefore provided for capturing a directional sound field(s) in one or more communication devices. For instance, an example embodiment may utilize a beamforming technology with array signal processing for capturing a directional sound field(s). By utilizing array signal processing, an example embodiment may capture sound field(s) in a desired direction while suppressing sound from other directions.
In an example embodiment, a communication device may include several microphones. These microphones may be placed concerning applications including, but not limited to, telephony, active noise cancellation, video sound capture (e.g., mono), etc. The positions of the microphones may also be influenced by the communication device form factor and design. In one example embodiment, the microphones that are already available or included in the communication device (e.g., a mobile device) may be utilized for directional sound capture using array processing. As such, it may not be necessary to add more microphones specifically for a directional sound capture application(s), and still, good directional sound quality may be attained. As described above, there may be several microphones available in a communication device. An example embodiment may optimize the directional audio capture using these microphones in a novel beamforming configuration.
As such, an example embodiment may utilize microphones that may not be optimally placed regarding array processing. As a consequence, there are three main issues taken into account by some example embodiments. Firstly, the distance between microphones may not be optimal for beamforming. Secondly, the assumption of propagation in a lossless medium may not be valid. The mechanics of a communication device such as, for example, a smartphone may shadow the audio signal differently for different microphones which may depend on the propagation direction. Thirdly, as described above, using existing microphones, it may be challenging to design a beamformer that would have an acceptable directional response for all the required frequencies.
As such, in the design of the directional recording a new approach is adopted by an example embodiment. Firstly, in an example embodiment, the microphone signals may be divided into subbands (for example, to produce subband signals). Secondly, an example embodiment may optimize the beamformer parameters separately and independently for each frequency subband and each directional sound field. Thirdly, in an example embodiment, the optimization may be done in an iterative manner using measurement data.
An example embodiment may solve the issues that are caused by the unoptimal microphone placement. For instance, a first issue may be that the distance between the microphones limits the applicable frequency range for the beamformer. In this regard, for each frequency subband, an example embodiment may choose the best possible set of microphones. For example, microphones positioned in the ends of a communication device (e.g., a mobile device) may be used in a low frequency domain taking into account a restriction posed by the aliasing effect. In an example embodiment, the microphones with a smaller mutual distance (for example, on front and back covers of the mobile device) may be used in the higher frequency subbands.
The second issue, causing problems with existing solutions, concerns the assumption of sound propagation in a lossless medium. In an example embodiment, the shadowing effect of a communication device (e.g., a mobile device) mechanics may be taken into account during the iterative optimization of the beamformer coefficients hj(k) since the optimization may be based on measurement data.
As described above, the third issue, causing problems with existing solutions, deals with the frequency band of operation of the beamformer. In an example embodiment, the beamformer parameters may be optimized separately for each frequency subband. The different parameter values for each subband may allow an example embodiment to generate directional audio fields throughout the needed frequency range.
Also, in an instance in which some of the microphone signals are blocked or deteriorated, for example, by user interference or wind noise, etc. an example embodiment may switch and utilize secondary microphones in the affected frequency subbands. Information of the microphones being blocked may be detected from an algorithm(s), for example, based on an example embodiment analyzing the microhone signal levels. In addition, the beam parameters for the set of microphones including the secondary microphones may be predetermined in order to produce the desired directional output.
In one example embodiment, a method for providing directional audio capture is provided. The method may include assigning at least one beam direction, among a plurality of beam directions, in which to direct directionality of an output signal of one or more microphones. The method may further include dividing microphone signals of each of the one or more microphones into selected frequency subbands wherein an analysis is performed. The method may further include selecting at least one set of microphones of a communication device for the selected frequency subbands. The method may further include optimizing the assigned beam direction by adjusting at least one beamformer parameter based on the selected set of microphones and at least one of the selected frequency subbands.
In another example embodiment, an apparatus for providing directional audio capture is provided. The apparatus may include a processor and a memory including computer program code. The memory and computer program code are configured to, with the processor, cause the apparatus to at least perform operations including assigning at least one beam direction, among a plurality of beam directions, in which to direct directionality of an output signal of one or more microphones. The at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to divide microphone signals of each of the one or more microphones into selected frequency subbands wherein an analysis is performed. The at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to select at least one set of microphones of a communication device for the selected frequency subbands. The at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to optimize the assigned beam direction by adjusting at least one beamformer parameter based on the selected set of microphones and at least one of the selected frequency subbands.
In another example embodiment, a computer program product for providing directional audio capture is provided. The computer program product includes at least one computer-readable storage medium having computer-readable program code portions stored therein. The computer-executable program code instructions may include program code instructions configured to assign at least one beam direction, among a plurality of beam directions, in which to direct directionality of an output signal of one or more microphones. The program code instructions may also divide microphone signals of each of the one or more microphones into selected frequency subbands wherein an analysis is performed. The program code instructions may also select at least one set of microphones of a communication device for the selected frequency subbands. The program code instructions may also optimize the assigned beam direction by adjusting at least one beamformer parameter based on the selected set of microphones and at least one of the selected frequency subbands.
In another example embodiment, an apparatus for providing directional audio capture is provided. The apparatus may include a processor and a memory including computer program code. The memory and computer program code are configured to, with the processor, cause the apparatus to at least perform operations including enabling one or more microphones to detect at least one acoustic signal from one or more sound sources. The at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to communicate with a beamformer wherein at least one beam direction is assigned based on a recording event. The at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to analyze one or more microphone signals to select at least one set of microphones for the recording event, wherein the beamformer optimizes at least one parameter of the assigned beam direction based on the selected set of microphones.
Having thus described some example embodiments of the invention in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:
Some embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the invention are shown. Indeed, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Like reference numerals refer to like elements throughout. As used herein, the terms “data,” “content,” “information” and similar terms may be used interchangeably to refer to data capable of being transmitted, received and/or stored in accordance with embodiments of the invention. Moreover, the term “exemplary”, as used herein, is not provided to convey any qualitative assessment, but instead merely to convey an illustration of an example. Thus, use of any such terms should not be taken to limit the spirit and scope of embodiments of the invention.
Additionally, as used herein, the term ‘circuitry’ refers to (a) hardware-only circuit implementations (for example, implementations in analog circuitry and/or digital circuitry); (b) combinations of circuits and computer program product(s) comprising software and/or firmware instructions stored on one or more computer readable memories that work together to cause an apparatus to perform one or more functions described herein; and (c) circuits, such as, for example, a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation even if the software or firmware is not physically present. This definition of ‘circuitry’ applies to all uses of this term herein, including in any claims. As a further example, as used herein, the term ‘circuitry’ also includes an implementation comprising one or more processors and/or portion(s) thereof and accompanying software and/or firmware. As another example, the term ‘circuitry’ as used herein also includes, for example, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, other network device, and/or other computing device.
As defined herein a “computer-readable storage medium,” which refers to a non-transitory, physical or tangible storage medium (for example, volatile or non-volatile memory device), may be differentiated from a “computer-readable transmission medium,” which refers to an electromagnetic signal.
Additionally, as referred to herein a “recording event” may include, but is not limited to, a capture of audio (e.g., an audio capture event) which may be associated with telephony (e.g., hands-free or hands-portable telephony), stereo recording, directional mono recording, surround sound recording (e.g., surround sound 5.1 recording, surround sound 7.1 recording, etc.) directional stereo recording, front end for audio processing, speech recognition and any other suitable cellular or non-cellular captures of audio. For example, a recording event may include a capture of audio associated with corresponding video data (e.g., a live video recording), etc.
The network 30 may include a collection of various different nodes (of which the second and third communication devices 20 and 25 may be examples), devices or functions that may be in communication with each other via corresponding wired and/or wireless interfaces. As such, the illustration of
One or more communication terminals such as the mobile terminal 10 and the second and third communication devices 20 and 25 may be in communication with each other via the network 30 and each may include an antenna or antennas for transmitting signals to and for receiving signals from a base site, which could be, for example a base station that is a part of one or more cellular or mobile networks or an access point that may be coupled to a data network, such as a Local Area Network (LAN), a Metropolitan Area Network (MAN), and/or a Wide Area Network (WAN), such as the Internet. In turn, other devices such as processing elements (for example, personal computers, server computers or the like) may be coupled to the mobile terminal 10 and the second and third communication devices 20 and 25 via the network 30. By directly or indirectly connecting the mobile terminal 10 and the second and third communication devices 20 and 25 (and/or other devices) to the network 30, the mobile terminal 10 and the second and third communication devices 20 and 25 may be enabled to communicate with the other devices or each other, for example, according to numerous communication protocols including Hypertext Transfer Protocol (HTTP) and/or the like, to thereby carry out various communication or other functions of the mobile terminal 10 and the second and third communication devices 20 and 25, respectively.
Furthermore, the mobile terminal 10 and the second and third communication devices 20 and 25 may communicate in accordance with, for example, radio frequency (RF), near field communication (NFC), Bluetooth (BT), Infrared (IR) or any of a number of different wireline or wireless communication techniques, including Local Area Network (LAN), Wireless LAN (WLAN), Worldwide Interoperability for Microwave Access (WiMAX), Wireless Fidelity (WiFi), Ultra-Wide Band (UWB), Wibree techniques and/or the like. As such, the mobile terminal 10 and the second and third communication devices 20 and 25 may be enabled to communicate with the network 30 and each other by any of numerous different access mechanisms. For example, mobile access mechanisms such as LTE, Wideband Code Division Multiple Access (W-CDMA), CDMA2000, Global System for Mobile communications (GSM), General Packet Radio Service (GPRS) and/or the like may be supported as well as wireless access mechanisms such as WLAN, WiMAX, and/or the like and fixed access mechanisms such as Digital Subscriber Line (DSL), cable modems, Ethernet and/or the like.
In an example embodiment, the first communication device (for example, the mobile terminal 10) may be a mobile communication device such as, for example, a wireless telephone or other devices such as a personal digital assistant (PDA), mobile computing device, tablet computing device, camera, video recorder, audio/video player, positioning device, game device, television device, radio device, or various other like devices or combinations thereof. The second communication device 20 and the third communication device 25 may be mobile or fixed communication devices. However, in one example, the second communication device 20 and the third communication device 25 may be servers, remote computers or terminals such as, for example, personal computers (PCs) or laptop computers.
In an example embodiment, the network 30 may be an ad hoc or distributed network arranged to be a smart space. Thus, devices may enter and/or leave the network 30 and the devices of the network 30 may be capable of adjusting operations based on the entrance and/or exit of other devices to account for the addition or subtraction of respective devices or nodes and their corresponding capabilities.
In an example embodiment, the mobile terminal as well as the second and third communication devices 20 and 25 may employ an apparatus (for example, apparatus of
Referring now to
The processor 70 may be embodied in a number of different ways. For example, the processor 70 may be embodied as one or more of various processing means such as a coprocessor, microprocessor, a controller, a digital signal processor (DSP), processing circuitry with or without an accompanying DSP, or various other processing devices including integrated circuits such as, for example, an ASIC (application specific integrated circuit), an FPGA (field programmable gate array), a microcontroller unit (MCU), a hardware accelerator, a special-purpose computer chip, or the like. In an example embodiment, the processor 70 may be configured to execute instructions stored in the memory device 76 or otherwise accessible to the processor 70. As such, whether configured by hardware or software methods, or by a combination thereof, the processor 70 may represent an entity (for example, physically embodied in circuitry) capable of performing operations according to an embodiment of the invention while configured accordingly. Thus, for example, when the processor 70 is embodied as an ASIC, FPGA or the like, the processor 70 may be specifically configured hardware for conducting the operations described herein. Alternatively, as another example, when the processor 70 is embodied as an executor of software instructions, the instructions may specifically configure the processor 70 to perform the algorithms and operations described herein when the instructions are executed. However, in some cases, the processor 70 may be a processor of a specific device (for example, a mobile terminal or network device) adapted for employing an embodiment of the invention by further configuration of the processor 70 by instructions for performing the algorithms and operations described herein. The processor 70 may include, among other things, a clock, an arithmetic logic unit (ALU) and logic gates configured to support operation of the processor 70.
In an example embodiment, the processor 70 may be configured to operate a connectivity program, such as a browser, Web browser or the like. In this regard, the connectivity program may enable the apparatus 50 to transmit and receive Web content, such as for example location-based content or any other suitable content, according to a Wireless Application Protocol (WAP), for example.
Meanwhile, the communication interface 74 may be any means such as a device or circuitry embodied in either hardware, a computer program product, or a combination of hardware and software that is configured to receive and/or transmit data from/to a network and/or any other device or module in communication with the apparatus 50. In this regard, the communication interface 74 may include, for example, an antenna (or multiple antennas) and supporting hardware and/or software for enabling communications with a wireless communication network (for example, network 30). In fixed environments, the communication interface 74 may alternatively or also support wired communication. As such, the communication interface 74 may include a communication modem and/or other hardware/software for supporting communication via cable, digital subscriber line (DSL), universal serial bus (USB), Ethernet or other mechanisms.
The microphones 71 may include a sensor that converts sound into an audio signal(s). The microphones 71 may be utilized for various applications including, but not limited to, stereo recording, directional mono recording, surround sound, front end for audio processing such as for telephony (e.g., hands-portable or hands free) or speech recognition and any other suitable applications.
The user interface 67 may be in communication with the processor 70 to receive an indication of a user input at the user interface 67 and/or to provide an audible, visual, mechanical or other output to the user. As such, the user interface 67 may include, for example, a keyboard, a mouse, a joystick, a display, a touch screen, a microphone, a speaker, or other input/output mechanisms. In an example embodiment in which the apparatus is embodied as a server or some other network devices, the user interface 67 may be limited, remotely located, or eliminated. The processor 70 may comprise user interface circuitry configured to control at least some functions of one or more elements of the user interface, such as, for example, a speaker, ringer, microphone, display, and/or the like. The processor 70 and/or user interface circuitry comprising the processor 70 may be configured to control one or more functions of one or more elements of the user interface through computer program instructions (for example, software and/or firmware) stored on a memory accessible to the processor 70 (for example, memory device 76, and/or the like).
The apparatus 50 includes a media capturing element, such as camera module 36. The camera module 36 may include a camera, video and/or audio module, in communication with the processor 70 and the display 85. The camera module 36 may be any means for capturing an image, video and/or audio for storage, display or transmission. For example, the camera module 36 may include a digital camera capable of forming a digital image file from a captured image. As such, the camera module 36 may include all hardware, such as a lens or other optical component(s), and software necessary for creating a digital image file from a captured image. Alternatively, the camera module 36 may include only the hardware needed to view an image, while a memory device (e.g., memory device 76) of the apparatus 50 stores instructions for execution by the processor 70 in the form of software necessary to create a digital image file from a captured image. In an example embodiment, the camera module 36 may further include a processing element such as a co-processor which assists the processor 70 in processing image data and an encoder and/or decoder for compressing and/or decompressing image data. The encoder and/or decoder may encode and/or decode according to a Joint Photographic Experts Group, (JPEG) standard format or another like format. In some cases, the camera module 36 may provide live image data to the display 85. In this regard, the camera module 36 may facilitate or provide a camera view to the display 85 to show or capture live image data, still image data, video data (e.g., a video recording and associated audio data), or any other suitable data. Moreover, in an example embodiment, the display 85 may be located on one side of the apparatus 50 and the camera module 36 may include a lens positioned on the opposite side of the apparatus 50 with respect to the display 85 to enable the camera module 36 to capture images on one side of the apparatus 50 and present a view of such images to the user positioned on the other side of the apparatus 50.
In an example embodiment, the processor 70 may be embodied as, include or otherwise control the directional audio capture module. The directional audio capture module 78 may be any means such as a device or circuitry operating in accordance with software or otherwise embodied in hardware or a combination of hardware and software (for example, processor 70 operating under software control, the processor 70 embodied as an ASIC or FPGA specifically configured to perform the operations described herein, or a combination thereof) thereby configuring the device or circuitry to perform the corresponding functions of the directional audio capture module 78 as described below. Thus, in an example in which software is employed, a device or circuitry (for example, the processor 70 in one example) executing the software forms the structure associated with such means.
In an example embodiment, the directional audio capture module 78 may capture a directional sound field(s). For example, the directional audio capture module 78 may utilize beamforming technology with array signal processing to capture one or more directional sound fields. By utilizing array signal processing the directional audio capture module 78 may capture a sound field(s) in a desired direction(s) while suppressing sound from other directions.
As examples, the directional audio capture module 78 may capture directional sound fields related to stereo, surround sound, directional mono recording associated with a video, telephony processing in a hand-portable or hands-free mode and any other suitable directional sound fields. For instance, the directional sound field captured by the directional audio capture module 78 may be used as a front end for sound processing such as speech recognition as one example or used in audio or videoconferencing applications, as another example.
Referring now to
The processor 104 may also be connected to at least one communication interface 107 or other means for displaying, transmitting and/or receiving data, content, and/or the like. The user input interface 105 may comprise any of a number of devices allowing the network device 100 to receive data from a user, such as a keypad, a touch display, a joystick or other input device. In this regard, the processor 104 may comprise user interface circuitry configured to control at least some functions of one or more elements of the user input interface. The processor 104 and/or user interface circuitry of the processor 104 may be configured to control one or more functions of one or more elements of the user interface through computer program instructions (e.g., software and/or firmware) stored on a memory accessible to the processor 104 (e.g., volatile memory, non-volatile memory, and/or the like).
In one example embodiment, the processor 104 may optimize filter coefficients and may provide the optimized filter coefficients as parameters to the directional audio capture module 78 of apparatus 50. The processor 104 may optimize the filter coefficients based in part on performing a frequency subband division and microphone(s) selection, as described more fully below. The directional audio capture module 78 may utilize the received optimized filter coefficients as parameters to perform beamformer processing of corresponding microphone signals, as described more fully below. In some example embodiments, the processor 70 of the apparatus 50 may perform the optimization of the filter coefficients and may provide the optimized filter coefficients as parameters to the directional audio capture module 78 to perform the beamformer processing.
The directional audio capture module 78 may utilize a filter-and-sum beamforming technique for noise reduction in communication devices. In the filter-and-sum beamforming technique the recorded data may be processed by the directional audio capture module 78 by implementing Equation (1) below
where M is the number of microphones (e.g., microphones 71) and L is the filter length. The filter coefficients are denoted by hj(k) and the microphone signal is denoted by xj. In the filter-and-sum beamforming, the filter coefficients hj(k) are optimized regarding the microphone positions. In an example embodiment, a processor (e.g., processor 70, processor 104) may optimize the filter coefficients for the filter-and-sum beamforming technique given the microphone (e.g., microphone(s) 71) positions. In an example embodiment, the optimization of the filter coefficients may be performed by a processor (e.g., processor 70, processor 104) and the filter coefficients may then be provided as parameters to the directional audio capture module 78 which may perform beamformer processing of corresponding microphone signals. Additionally, the directional audio capture module 78 may utilize multiple independent beam designs for different frequency subbands, as described more fully below. In an example embodiment, the directional audio capture module 78 may also utilize predefined beams and/or predefined beamformer parameters. The beams may be designed based in part on using measurement data.
Referring now to
The directional audio capture module (e.g., directional audio capture module 78) of the communication device 90 may utilize a designed beamformer for low frequencies which may enhance the directional capture and utilize the natural directionality of the microphones in the higher frequency subbands. One example application in which some of the microphone pairs may be utilized is enhanced stereo capture. Some of the microphone pairs may also be utilized for applications enhancing the audio quality of a hands-free call or in a hand-portable mode or any other suitable audio applications.
In the example embodiment of
In an instance in which there are three microphones available (such as, for example, microphones 1, 3, and 4, or 1, 3, and, 7, or 1, 3, and 9) the directional audio capture module may be utilized to design a beamformer that utilizes the microphone pair 1 and 4 in low frequency subbands and microphone pair 1 and 3 in higher frequency subbands to generate a directional capture utilized in the hands-free or hands-portable telephony applications or as a front end for other audio processing applications. In this manner, the directional audio capture module may block low frequency disturbance in a null direction of the beam.
In one example embodiment, by utilizing 4 microphones (such as, for example, microphones 1, 2, 3, and 4) the directional capture module of the communication device 90 may generate a directional capture utilized in the hands-free or hand-portable telephony applications, as a front end for other audio processing applications, as an enhanced surround sound capture or as a directional stereo capture, as described more fully below by utilizing four microphones (such as, for example, microphones 1, 2, 3, and 4).
In another example embodiment, in an instance in which the directional audio capture module utilizes more than 4 microphones in the communication device 90, the directional audio capture may enable choosing of an optimal set of microphones regarding an application. By utilizing the directional audio capture module an independent set of microphones for each frequency subband may be chosen. For low frequency subbands a set of microphones with large mutual distance may be chosen. For the higher frequency subbands a set of microphones that are close to each other may be chosen. For each subband the distance between the microphones may be less than half of the shortest wavelength of that subband. Some examples of the applications supported by at least a subset of the microphones of the communication device of
Microphones 8 and 9—stereo recording,
Microphones 1 and 3 or 2 and 4—directional mono recording,
Microphones 1-4—surround sound 5.1 recording or directional stereo recording,
Microphones 1-4, 8-9—surround sound 7.1 recording,
Microphones 1-11—surround sound recording including the height channels (microphones 5-7 may be utilized in one example embodiment), and
Microphone 1 and any of the microphones 3-7—front end for audio processing such as, for example, for telephony (e.g., hand-portable or hands-free) or speech recognition.
The directional audio capture module may utilize microphones of the apparatus for any other suitable applications (e.g., audio applications).
In an instance in which some of the microphone signals of a subset of the microphones of the communication device 90 of
For purposes of illustration and not of limitation, consider an instance in which a user of the communication device 90 (e.g., apparatus 50) is recording video and the user accidentally blocks microphone 10 which is providing the output of the audio for the video. In this regard, the directional audio capture module 78 may switch to microphone 11 instead of microphone 10 in an instance in which the directional audio capture module 78 determines that the signal (e.g., the audio signal) output from microphone 10 is weak or deteriorated denoting that the microphone 10 may be partially or completely blocked. In this example embodiment, the directional audio capture module 78 may switch to microphone 10 in response to determining that the microphone signal level output from microphone 10 is unacceptable.
Referring now to
In 5.1 surround sound there are five different directions for audio capture: (1) front left (−30°), (2) front right (30°), (3) front (0°), (4) surround left (−110°), and (5) surround right (110°), as shown in
Referring now to
For each subband, the set of microphones that provides the best directional output may be chosen by a processor (e.g. processor 70, processor 104). In the lower frequency subbands (e.g., below 1.5 kHz) microphones located in different ends of the communication device 150 may be used as shown in
In an example embodiment, the directional audio capture module 78 may perform the beamformer processing in each of the seven frequency subbands of
For purposes of illustration and not of limitation, the three lowest frequency subbands (e.g., frequency subbands 12, 14, 16) of the seven frequency subbands may be used for microphone pair 1 and 4 and microphone pair 2 and 3. On the other hand, the four highest frequency subbands (e.g., frequency subbands 18, 22, 24, 26) of the seven frequency subbands may be used for microphone pair 1 and 3 and microphone pair 2 and 4.
In response to performing the beamforming processing in each of the frequency subbands for the different pairs or sets of microphones the directional audio capture module 78 may combine the microphone output signals to produce directional output signals as described more fully below.
Referring now to
Referring now to
At operation 1105, directional measurement data may be utilized (for example, by a processor (e.g., processor 70, processor 104)) in part, to optimize the beamformer parameters. For instance, the directional measurement may be performed in an anechoic chamber, in which the communication device is rotated 360 degrees in 10 degree steps. At each step (e.g., each 10 degree step), white noise is played from a loudspeaker at 1 m distance from the communication device, as shown in
Referring now to
The filter coefficients or beam parameters hj(k) may then be iteratively altered for example by a processor (e.g., processor 70, processor 104) to maximize the power ratio for the direction and subband being processed. For example, in an instance in which the desired or selected beam direction is front left, a processor (e.g., processor 70, processor 104) may calculate the power in this direction from 0° to −60° versus the power in all other directions (e.g., the front right beam, the surround right beam, the surround left beam) to determine the power ratio (R=power in the desired direction/power in all other directions) for the front left beam. In an instance in which the power ratio is selected for the desired direction, a processor (e.g., processor 70, processor 104) may optimize the beam parameters so that the beam is directed in the desired direction which is the front left direction in this example. In an instance in which another beam direction is selected such as, for example, the front right direction, a processor (e.g., processor 70) may calculate the power in the desired direction of 0° to 60° versus power in all other directions (e.g., the front left direction, the surround left direction, the surround right direction).
In this example, the beam parameters hj(k) may be optimized in order to maximize the power ratio R. However, in an alternative example embodiment any other optimization criterion may be utilized taking into account the particular application where the directional sound capture is needed. For example, in some instances a good attenuation of sound may be desired from a certain direction.
Referring now to
In the example embodiment of
The directional signals generated by the beamformer processing modules 93 may be provided to the synthesis filter banks 95. Each of the synthesis filter banks 95 may combine the directional signals for each of the subbands for the corresponding directions to produce directional output signals y1, y2, . . . yZ. For purposes of illustration and not of limitation, in the example in which there are four beam directions for 5.1 surround sound, y1 may correspond to the directional output signal for front left, y2 may correspond to the directional output signal for front right, y3 may correspond to the directional output signal for surround left and y4 may correspond to the directional output signal for surround right.
Referring now to
In the example embodiments of
Referring now to
In the example embodiments of
Referring now to
At operation 1710, the communication device (e.g., communication device 150) may include means, such as the processor 70 and/or the like, for selecting at least one set of microphones (e.g., microphone pair 1 and 4 and microphone pair 2 and 3, etc.) of a communication device for selected frequency subbands. At operation 1715, the communication device may include means, such as the directional audio capture module 78, the processor 70 and/or the like, for optimizing the assigned beam direction by adjusting at least one beamformer parameter based on the selected set of microphones and at least one of the selected frequency subbands. In some alternative example embodiments, the assigning of the beam direction, the dividing of the microphone signals into selected frequency subbands and the selection of the set of microphones for selected frequency subbands may be performed by a processor such as, for example, processor 104 of network device 100 to optimize filter coefficients. The processor 104 of the network device 100 may provide the optimized filter coefficients as parameters to the directional audio capture module 78 to enable the directional audio capture module 78 to optimize the assigned beam direction by adjusting at least one beamformer parameter based on the selected set of microphones and at least one of the selected frequency subbands.
Referring now to
It should be pointed out that
Accordingly, blocks of the flowcharts support combinations of means for performing the specified functions. It will also be understood that one or more blocks of the flowcharts, and combinations of blocks in the flowcharts, can be implemented by special purpose hardware-based computer systems which perform the specified functions, or combinations of special purpose hardware and computer instructions.
In an example embodiment, an apparatus for performing the methods of
Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, although the foregoing descriptions and the associated drawings describe exemplary embodiments in the context of certain exemplary combinations of elements and/or functions, it should be appreciated that different combinations of elements and/or functions may be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, different combinations of elements and/or functions than those explicitly described above are also contemplated as may be set forth in some of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.
This application is a continuation of U.S. application Ser. No. 14/956,005, filed Dec. 1, 2015, which is a continuation of U.S. application Ser. No. 13/652,167, filed Oct. 15, 2012, which issued as U.S. Pat. No. 9,232,310, the contents of each of which are hereby incorporated herein in their entirety by reference.
Number | Date | Country | |
---|---|---|---|
Parent | 14956005 | Dec 2015 | US |
Child | 15928257 | US | |
Parent | 13652167 | Oct 2012 | US |
Child | 14956005 | US |