This application generally relates to an array microphone. In particular, this application relates to a multi-dimensional array microphone configured to provide improved frequency-dependent directivity.
Audio environments, such as conference rooms, boardrooms, and other meeting rooms, video conferencing settings, and the like, can involve the use of one or more microphones to capture sound from various audio sources active in the environment. The audio sources may include in-room human speakers, for example. The captured sound may be disseminated to a local audience in the environment through speakers (for sound reinforcement) and/or to others located remotely (such as via a telecast, webcast, or the like). For example, persons in a conference room may be conducting a conference call with persons at a remote location.
In general, microphones and other audio capturing devices, such as, e.g., conferencing devices, are available in a variety of sizes, form factors, mounting options, and wiring options to suit the needs of particular environments. Moreover, the microphones can be designed to produce different polar response patterns, including, for example, omnidirectional, cardioid, subcardioid, supercardioid, hypercardioid, and bidirectional. The types of conferencing devices, their operational characteristics (e.g., lobe direction, gain, polar pattern, etc.), and their placement in a particular audio environment may depend on a number of factors, including, for example, the locations of the audio sources, locations of listeners, the desire to exclude unwanted noises, physical space requirements, aesthetics, room layout, and/or other considerations. As an example, in some environments, a conferencing device may be placed on a table or lectern to be near the audio sources and/or listeners. In other environments, a conferencing device may be mounted overhead or on a wall to capture the sound from, or project sound towards, the entire room, for example.
Micro-Electrical-Mechanical-System (“MEMS”) microphones, or microphones that have a MEMS element as the core transducer, have become increasingly popular due to their smaller package size, thus enabling the audio device to have a smaller, thinner profile overall (e.g., compared to traditional microphones like dynamic, crystal, condenser/capacitor, boundary, button, etc.); high performance characteristics (e.g., high signal-to-noise ratio (“SNR”), low power consumption, good sensitivity, etc.); lower cost (e.g., compared to electret or condenser microphone cartridges); and general ease of assembly. While a conventional MEMS microphone, by itself, has an inherently omnidirectional polar pattern (i.e. the microphone is equally sensitive to sounds coming from any and all directions, regardless of the microphone's orientation), placing MEMS microphones in an array configuration, and applying appropriate beamforming techniques (e.g., signal processing), can produce a directional response, or a beam pattern that is more sensitive to sound coming from one or more specific directions than sound coming from other directions.
The techniques of this disclosure provide systems and methods designed to, among other things, (1) provide an array microphone with a three-dimensional structure and microphone layout that has improved directivity transverse to the axis of the array, (2) steerable control of the array's direction of greatest sensitivity to any point at any azimuth and elevation around a sphere surrounding the center of the array, using appropriate beamforming techniques, thus providing increased ability to direct the array to where it is most effective at collecting sound, as well as generation of nulls, and (3) high performance acoustic characteristics suitable for conferencing environments, stage, sports, and other entertainment environments, including selective coverage of the stage or audience and/or isolation of ambience from game audio, and live sound reinforcement.
For example, one embodiment includes an array microphone comprising a plurality of microphone boards arranged in a linear pattern along a first axis and comprising a plurality of microphone elements configured to cover a plurality of frequency bands, each microphone board comprising: a first linear array comprising a first microphone element of the plurality of microphone elements and one or more second microphone elements of the plurality of microphone elements, the first microphone element located on the first axis and the one or more second microphone elements located on a second axis orthogonal to the first axis, and a second linear array comprising the first microphone element and one or more third microphone elements of the plurality of microphone elements, the one or more third microphone elements located on a third axis orthogonal to the first axis and to the second axis. According to various aspects, the linear pattern can be configured to place the microphone boards in a harmonically-nested configuration for covering the plurality of frequency bands.
Another example embodiment provides an array microphone comprising a plurality of microphone boards arranged in a first linear pattern along a first axis of the array microphone, the plurality of microphone boards comprising a plurality of microphone elements configured to cover a plurality of frequency bands, each microphone board comprising: a first microphone element of the plurality of microphone elements located on the first axis; one or more second microphone elements of the plurality of microphone elements located on a second axis of the microphone board, the second axis being orthogonal to the first axis; and one or more third microphone elements of the plurality of microphone elements located on a third axis of the microphone board, the third axis being orthogonal to the first axis and to the second axis, wherein the first microphone elements of the plurality of microphone boards are configured to form the first linear pattern along the first axis, and for each microphone board, the first microphone element and the one or more second microphone elements are configured to form a second linear pattern along the second axis of the corresponding microphone board, and the first microphone element and the one or more third microphone elements are configured to form the second linear pattern along the third axis of the corresponding microphone board.
Another example embodiment provides a microphone system, comprising: an array microphone comprising a plurality of microphone elements arranged on a plurality of microphone boards, the array microphone configured to provide audio coverage for a plurality of frequency bands; and one or more audio processors in communication with the array microphone and comprising one or more beamformers, the one or more audio processors being configured to, using the one or more beamformers: steer an audio pick-up lobe of the array microphone towards an audio source detected at a first point; and generate an audio output based on audio signals generated by the audio source and captured using the audio pick-up lobe, wherein the one or more beamformers are configured for steering the audio pick-up lobe towards any one of a plurality of points located on a sphere surrounding a center of the array microphone, the plurality of points comprising the first point. According to some aspects, the plurality of points are located at different azimuths and elevations around the sphere. According to some aspects, the plurality of points cover a solid angle of four times pi steradians. According to some aspects, the audio pick-up lobe comprises a direction of highest sensitivity for the array microphone.
These and other embodiments, and various permutations and aspects, will become apparent and be more fully understood from the following detailed description and accompanying drawings, which set forth illustrative embodiments that are indicative of the various ways in which the principles of the invention may be employed.
In general, array microphones are comprised of multiple microphone elements aligned in a specific pattern or geometry (e.g., linear, circular, etc.) and configured to operate as a single microphone device. For example, a linear array microphone is comprised of microphone elements situated relatively close together along a single axis. Array microphones can have different configurations and frequency responses depending on the placement of the microphone elements relative to each other and the direction of arrival for sound waves. The directionality of array microphones can also provide steerable coverage, or pick up patterns that focus on desired audio sources and reject unwanted sounds, such as room noise, and this steerability can enable pick up of multiple audio sources with a single array or device. For example, a broadside linear array comprised of a line of MEMS microphones arranged perpendicular to the preferred direction of sound arrival can achieve a desired pickup pattern using a delay and sum beamformer to combine the signals from the various microphone elements accordingly. In some broadside arrays, the microphone elements are placed in nested pairs about a central point and may be spaced apart from each by certain predetermined distances in order to cover a variety of frequencies.
Systems and methods are provided herein for a multi-dimensional array microphone that has added directivity for most, if not all, frequencies in all dimensions. In particular, the array microphone comprises a first plurality of microphone elements arranged in a linear pattern along a first axis, a second plurality of differential and complementary microphone elements arranged along a second perpendicular axis, and a third plurality of differential and complementary microphone elements arranged along a third, mutually perpendicular axis. The linear pattern can be configured to place the first plurality of microphone elements in a harmonically-nested configuration for covering a plurality of frequency bands. The second and third pluralities of microphone elements can be configured to form second and third linear patterns, respectively, that are mirror images of each other. This geometry provides not only array control perpendicular to the first axis (or the axis of the array microphone) but also, control around the axis of the array, such that the direction of maximum or greatest microphone sensitivity can be pointed towards any location at any azimuth and elevation within a sphere surrounding the center of the array, or covering a solid angle of 4*pi steradians, using appropriate beamforming techniques. For example, in embodiments, microphone elements along two or more axes can be combined to generate any first-order pattern (e.g., cardioid, toroidal, etc.) steered to any direction about the defined sphere in order to capture a desired sound source, or generate a null for rejecting an unwanted sound source.
Moreover, the multi-dimensional form factor of the array microphone is provided by arranging the microphone elements on a plurality of microphone boards (e.g., made of printed circuit board (PCB) substrate or the like) that are structurally independent but electrically linked together and configured to minimize the impacts of the array structure on the incident acoustic field. For example, in embodiments, each board is aligned with the first axis at a central point for supporting a respective one of the first plurality of microphones and has a predefined shape that extends out from that point and is configured to support the second plurality of microphones along the second axis and support the third plurality of microphones along the third axis. The multi-dimensional structure is further configured to minimize the total number of microphone elements used and the amount of PCB substrate used, so as to reduce the diffraction and geometric shading that would otherwise be present in a three-dimensional array, all while improving the directivity and flexibility of the array.
The array microphone 100 comprises a plurality of microphone elements 102 (also referred to herein as “transducers” and “cartridges”) capable of forming multiple pickup patterns in order to optimally or consistently detect and capture sound from the audio sources. The polar patterns that can be formed by the array microphone 100 may depend on the placement of the microphone elements 102 within the array 100, as well as the type of beamformer(s) used to process the audio signals generated by the microphone elements 102. For example, a sum and differential beamformer may be used to form a cardioid, subcardioid, supercardioid, hypercardioid, bidirectional, and/or toroidal polar pattern directed to a desired sound source. Additional polar patterns may be created by combining the original polar patterns, and the combined pattern can be steered to any angle along the plane of, for example, the table on which the array microphone 100 rests, the wall on which the array microphone 100 is mounted, or other plane that is perpendicular to the axis of the array microphone 100. In some embodiments, a combination of beamforming techniques can be used to steer the direction of highest sensitivity to any direction at any azimuth and elevation around a sphere that surrounds the center of the array, or to cover a solid angle of 4*pi steradians. Other beamforming techniques may be utilized to combine the outputs of the microphone elements 102, so that the overall array microphone 100 achieves a desired frequency response, including, for example, lower noise characteristics, higher microphone sensitivity, and coverage of discrete frequency bands, as described in more detail herein. More description of the beamforming techniques used with the array microphone 100 is provided below with reference to
In preferred embodiments, each of the microphone elements 102 may be a MEMS (micro-electrical mechanical system) transducer with an inherent omnidirectional polar pattern. In some embodiments, the microphone elements 102 may be any type of omnidirectional microphone. In other embodiments, the microphone elements 102 may have other polar patterns and/or may be condenser microphones, dynamic microphones, piezoelectric microphones, or other types of traditional microphones. In still other embodiments, the arrangement and/or processing techniques described herein can be applied to other types of arrays comprised of omnidirectional transducers or sensors where directionality is desired (such as, e.g., sonar arrays, radio frequency applications, seismic devices, etc.). Also, although
Each of the microphone elements 102 can detect sound and convert the sound into an audio signal. In some cases, the audio signal can be a digital audio output (e.g., MEMS transducers). For other types of microphones, the audio signal may be an analog audio output, and components of the array microphone 100, such as analog to digital converters, processors, and/or other components, may process the analog audio signals to ultimately generate one or more digital audio output signals. The digital audio output signals may conform to the Dante standard for transmitting audio over Ethernet, in some embodiments, or may conform to another standard. In certain embodiments, one or more pickup patterns may be formed by a processor of the array microphone 100 from the audio signals of the microphone elements 102, and the processor may generate a digital audio output signal corresponding to each of the pickup patterns. In other embodiments, the microphone elements 102 may output analog audio signals and other components and devices (e.g., processors, mixers, recorders, amplifiers, etc.) external to the array microphone 100 that may process the analog audio signals.
As shown in
The plurality of microphone boards 104 can be configured to position the microphone elements 102 in three-dimensional space so that they collectively form a multi-dimensional array configured to cover a plurality of frequency bands (e.g., 20 hertz (Hz)≤f≤20 kilohertz (kHz)). In particular, each microphone board 104 may have a matching or uniform shape comprised of multiple sides or surfaces for positioning the microphone elements 102 in different dimensions, as well as a bottom or back side for positioning the overall board 104 in an upright manner. For example, in the illustrated embodiment, the microphone boards 104 have a generally triangular shape with three sides meeting to form three corners. A first side 108 of each board 104 is attached to a mounting surface 106a of the housing 106, while the other two sides 110 and 112 are freestanding and substantially perpendicular to the mounting surface 106a. As shown, the microphone elements 102 may be included on the freestanding sides 110 and 112 of the microphone boards 104, while the first side 108 (also referred to herein as a “plain side”) may have no microphone elements 102 thereon. This arrangement ensures that the microphone elements 102 are positioned in free space and, for example, away from any circuitry near the mounting surface 106a or other part of the housing 106. In other embodiments, the microphone boards 104 may be configured to form any other shape capable of implementing the techniques described herein.
According to embodiments, the array microphone 100 can produce a substantially consistent frequency response across a variety of settings or orientations, including, for example, whether mounted on a wall or other vertical surface, placed on a table, desktop, lectern, or other horizontal surface, or attached to a ceiling. That is, regardless of the array orientation, the audio pick-up lobes of the array microphone 100 can be directed towards a desired sound source with increased rear rejection and steering control, or isolated forward acceptance, thus improving the array's ability to reject unwanted sound sources and reflections in the room and provide a high signal to noise ratio (SNR) in any dimension.
To illustrate,
According to embodiments, dividing the microphone elements 102 across multiple microphone boards 104 that are structurally and physically separate allows for a more efficient use of the copper lines or wires, and minimizes or reduces an impact of the overall array structure on an incident acoustic field. Minimizing the structural footprint of the array microphone 100 also reduces the diffraction, resonance, and geometric shading, or shadowing effects, that would otherwise be present in a three-dimensional array structure, as will be appreciated. In embodiments, the microphone boards 104, themselves, are also configured to have minimal interactions with the incident acoustic field by reducing the structural footprint of each board 104. For example, as shown, each microphone board 104 has a cut-out or open center 117 to minimize the amount of PCB substrate, or other material, used to create the structure of the board 104. In some embodiments, the open center 117 may be configured (e.g., sized and shaped) to be as large as possible while still leaving enough room on the board 104 for placement and/or attachment of the microphone elements 102 and to maintain a structural integrity of the board 104. Similarly, a thickness of each microphone board 104 may be selected to further minimize interactions with the incident acoustic field while also maintaining the structural integrity and stability of the board 104.
In embodiments, the microphone boards 104 can be linearly arranged along a length of the array microphone 100, or its housing 106, and perpendicular to a preferred or expected direction of arrival for incoming sound waves, so as to form a linear pattern along the first axis 115 (or common axis) of the array microphone 100. For example, as shown in
According to embodiments, the linear pattern formed by the microphone boards 104 along the first axis 115 can be configured to place the microphone boards 104 in a harmonically-nested configuration for covering a desired plurality of frequency bands, using one or more beamformers or other audio processing techniques. For example, the linear pattern formed by the microphone boards 104 can be configured to operate in different octaves (e.g., 600-1200 Hertz (Hz), 1200-2400 Hz, 2400-4800 Hz, etc.) within the covered plurality of frequency bands, so that the overall beam pattern for the array microphone 100 remains essentially constant from octave to octave. In some embodiments, the linear pattern may be implemented using a sub-band-based scaled aperture (SSA) approach that uses a different array aperture for each octave, so that progressively lower frequency octaves are processed by progressively wider linear arrays. In order to enhance spatial resolution, the aperture of the linear array formed by the microphone boards 104 may be doubled when moving from a higher octave to the next lower one (e.g., as shown in
As best seen in
As also shown in
Referring additionally to
According to embodiments, each microphone board 104 comprises a first linear array 120 disposed along a second axis 121 (e.g., the Y axis) that is orthogonal to the first axis 115 (e.g., the X axis) and a second linear array 122 disposed along a third axis 123 (e.g., the Z axis) that is orthogonal to both the first axis 115 and the second axis 121, as shown in
The secondary arrays 120 and 122 can be configured to form differential arrays along the second axis 121 and the third axis 123, respectively, based on harmonic nesting techniques, as described in more detail with respect to
In embodiments, using one or more beamforming techniques described herein, each of the differential arrays formed on the microphone boards 104 may be configured to generate a first-order polar pattern that is oriented along a line defined by the microphone elements 102 in that array (e.g., along the second axis 121 or the third axis 123), and is directable in either the positive or negative direction. In addition, the resulting first-order polar patterns, from all or many of the microphone boards 104, can be combined to steer an overall audio pick-up lobe of the array microphone 100 towards any direction within the plane 116 (or the YZ plane in
In embodiments, the microphone elements 102 are arranged across the microphone boards 104 to collectively form a harmonically nested linear array, using one or more beamforming techniques. For example, the plurality of microphone elements 102 may be configured to form, across the plurality of microphone boards 104, a first sub-array for covering a first octave of the covered plurality of frequency bands and a second sub-array for covering a second octave of the plurality of frequency bands, the first sub-array being nested within the second sub-array, for example, as shown in
Referring back to
Referring additionally to
In the illustrated example, the second plane 103 of the array microphone 100, though not shown, may be situated on a XZ plane of the XYZ axes because the second plane 103 is comprised of the board sides 110 that are aligned with the Z axis of the XZ plane (e.g., the third axis 123 shown in
The harmonically-nested arrangement of the microphone elements 102a and 102b on the first plane 101 of the array microphone 100 will now be described with reference to
In a preferred embodiment, harmonic nesting techniques are used to select the distances between adjacent first microphone elements 102a in the primary array 118, such that the linear pattern formed by the first, second, third, and fourth sub-arrays are harmonically nested, as shown in
Various considerations may be used to determine the smallest distance value, D1, between the first microphone elements 102a along the first axis 115 and thus, the distance between adjacent microphone boards 104. In some embodiments, the distance between adjacent microphone boards 104 along the first axis 115 may be configured based on a frequency value included in the covered plurality of frequency bands. In such cases, the distance, D1, between the first microphone elements 102a may be selected based on the frequency bands that are to be spatially sampled in the array microphone 100. In some embodiments, the distance between adjacent microphone boards 104 along the first axis 115 may be configured based on a linear aperture size of the array microphone 100. In such cases, the distance, D1, between the first microphone elements 102a may be selected based on the desired linear array aperture size, as well as a total number of the first microphone elements 102a being used to form the linear array pattern (or primary array 118), thus enabling the array microphone 100 to have a scalable geometry that can be reconfigured to fit the design needs of a particular application. Other design considerations may also determine the D1 value, including, for example, desired locations for the frequency nulls, a desired amount of electrical delay, spatial sampling density, criteria for avoiding spatial aliasing, and processing overhead. In one example embodiment, the D1 distance is approximately 150 millimeters (mm) along the axis of the array.
As depicted in
In embodiments, on each microphone board 104, each of the second microphone elements 102b and the first microphone element 102a being duplicated thereby jointly form a microphone set, or differential pair, that is configured to operate in a frequency octave covered by the duplicated first microphone element 102a. For example, in each microphone set, a spacing or distance between the first microphone element 102a and the corresponding second microphone element 102b along the orthogonal axis 121 may be selected based on the frequency octave covered by that set. Moreover, the first and second microphone elements 102a and 102b of each microphone set may be treated or handled as a single microphone unit of the array microphone 100 by acoustically combining the two microphone elements 102a and 102b to create a new pickup pattern for that microphone set (e.g., using appropriate beamforming techniques).
Also in embodiments, microphone sets from different microphone boards 104 may be grouped together as sub-arrays to produce one or more combined outputs for the array microphone 100. As an example, all of the microphone sets configured to cover the first octave (e.g., N) can be combined or aggregated, across multiple microphone boards 104, to create a sub-array for operating in that octave (e.g., using appropriate beamforming techniques), for example, as shown by Sub-array 1 in
More specifically, the first group of microphone sets 134 comprises the elements 124 from the first microphone elements 102a that are disposed along the first axis 115 to form the first sub-array for covering the first, or Nth, octave. The first group of microphone sets 134 further comprises the second microphone elements 102b that are added to duplicate the first sub-array along the second axes of the corresponding microphone boards 104. In each microphone set 134, the second microphone element 102b is disposed a first distance, D2, from the corresponding first microphone element 102a. Similarly, the second group of microphone sets 136 comprises the elements 126 from the first microphone elements 102a that are disposed along the first axis 115 to form the second sub-array for covering the second, or (N-1)th, octave. The second group of microphone sets 136 further comprises the second microphone elements 102b that are added to duplicate the second sub-array along the second axes 121 of the corresponding microphone boards 104. In each microphone set 136, the second microphone element 102b is disposed a second distance that is twice the first distance, D2, from the corresponding first microphone element 102a. Likewise, the third group of microphone sets 138 comprises the elements 128 from the first microphone elements 102a that are disposed along the first axis 115 to form the third sub-array for covering the third, or (N-2)th, octave. The third group of microphone sets 138 further comprises the second microphone elements 102b that are added to duplicate the third sub-array along the second axes 121 of the corresponding microphone boards 104. In each microphone set 138, the second microphone element 102b is disposed a third distance that is four times the first distance, D2, from the corresponding first microphone element 102a. And lastly, the fourth group of microphone sets 140 comprises the elements 130 from the first microphone elements 102a that are disposed along the first axis 115 to form the fourth sub-array for covering the fourth, or (N-3)th, octave. The fourth group of microphone sets 140 further comprises the second microphone elements 102b that are added to duplicate the fourth sub-array along the second axes 121 of the corresponding microphone boards 104. In each microphone set 140, the second microphone element 102b is disposed a fourth distance that is about 6.67 times the first distance, D2, from the corresponding first microphone element 102a.
Thus, like the distances between adjacent first microphone elements 102a along the first axis 115, the distance between the microphone elements in a given differential pair are halved with each octave's worth of frequencies, or increased by double multiples (i.e. a factor of 2) with each decreasing octave, except for the very highest frequencies. In embodiments, the distance D2 between the first microphone element 102a and the second microphone element 102b in each of the first plurality of microphone sets 134 may be equal to a half wavelength of a desired frequency from the octave covered by the sets 134 (i.e. the Nth octave), for example, to create nulls at the desired frequency. The distance D2 may also be selected to optimize cardioid formation when combining the first microphone element 102a and the second microphone element 102b of a given microphone set to produce a combined output, as described below. In one example embodiment, the D2 distance is approximately 7.5 millimeters (mm) along the corresponding second axis 121.
As shown in
In embodiments, the plurality of microphone sets formed by the microphone elements 102 on each of the first plane 101 and the second plane 103 are arranged orthogonal to the first axis 115 (or along the second orthogonal axis 121 and the third orthogonal axis 123, respectively, as shown in
Other dimensions of the array microphone 100 (e.g., an overall depth or a height/width along the axis 119) may be determined by the number of differential pairings included on the orthogonal axes 121 and 123 of the microphone boards 104 and the spacing between adjacent microphone elements along each orthogonal axis (e.g., as shown in
To illustrate,
Various components of the audio system 300 may be implemented using software executable by one or more computers, such as a computing device with a processor and memory, and/or by hardware (e.g., discrete logic circuits, application specific integrated circuits (ASIC), programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.). For example, some or all components of the beamformer 304 may be implemented using discrete circuitry devices and/or using one or more processors (e.g., audio processor and/or digital signal processor) (not shown) executing program code stored in a memory (not shown), the program code being configured to carry out one or more processes or operations described herein. Thus, in embodiments, the audio system 300 may include one or more processors, memory devices, computing devices, and/or other hardware components not shown in
The beamformer 304 may be used to apply appropriate beamforming techniques to the audio signals captured by the microphone elements of the array microphone 302 to create a desired pickup pattern, such as, e.g., a first-order polar-pattern (e.g., cardioid, super-cardioid, hypercardioid, etc.), and/or to steer the pattern to a desired angle, relative to the axis of the array, to obtain additional directionality. For example, in some embodiments, the beamformer 304 may be configured to combine audio signals captured by each microphone element disposed on a first or primary axis of the array microphone 302 to form a primary array output; combine the audio signals captured by the microphone elements one the first axis with those captured by the differential microphone elements correspondingly disposed on different orthogonal axes of the array microphone 302 to form outputs for each differential pair; for each orthogonal plane of the array microphone 302, combine the differential pair outputs for that plane to form octave-specific sub-array outputs; combine the sub-array outputs for each orthogonal plane to create a linear array output with a first-order polar pattern (such as, e.g., a cardioid pickup pattern) directed along an axis in that plane; combine the linear array outputs for the two orthogonal planes to generate a directional output steered to any direction or angle within a plane that is perpendicular to the first axis of the array; and combine one or more of the linear array outputs with the primary array output to generate a directional output steered to any direction or angle about a sphere centered around a center of the array. The beamformer 304 may be configured to use any appropriate beamforming algorithm to combine the received audio signals and generate the steered output, including, for example, delay and sum techniques, weight and sum techniques, sum and difference techniques, filter and sum techniques, minimum variance distortionless response (“MVDR”) techniques, first-order differential techniques, or any combination thereof.
As shown, the array microphone 302 comprises a plurality of first microphone elements 302a disposed along a primary or first axis that is aligned with the X axis, such that the coordinates for each first microphone element 302a have a zero value along both the Y and Z axes (e.g., similar to the first microphone elements 102a disposed on the first axis 115 shown in
According to embodiments, the beamformer 304 may comprise a plurality of individual beamformers, or beamforming components, configured to combine the audio signals received from the array microphone 302 and generate a steered output that is directed towards a desired audio source (e.g., sound source 113 in
As shown, the beamformer 304 comprises a primary beamforming component 308 configured to receive the audio signals captured by the first microphone elements 302a and combine those signals to generate primary sub-array outputs for the primary array formed along the X axis (e.g., primary array 118 of
The beamformer 304 also comprises two differential pair beamforming components 310 and 312, one for each orthogonal axis of the microphone boards included in the array microphone 302. More specifically, a first differential pair beamforming component 310 (or “first differential beamformer”) can be configured to combine the audio signals captured by each differential pair included on the Y axis of each microphone board. Similarly, a second differential pair beamforming component 312 (or “first differential beamformer”) may be configured to combine the audio signals captured by each differential pair included on the Z axis of each microphone board. Each differential beamformer 310 and 312 may generate one or more differential pair outputs for each microphone board depending on the number of differential pairs included thereon. For example, for the central microphone board 104a shown in
According to embodiments, the first differential beamformer 310 can be further configured to combine the Y axis differential pair outputs generated across the plurality of microphone boards to create an octave-specific output for each of the harmonically-nested sub-arrays formed by the boards within the first orthogonal plane. Similarly, the second differential beamformer 312 can be further configured to combine the Z axis differential pair outputs generated across the plurality of microphone boards to create an octave-specific output for each of the harmonically-nested sub-arrays formed by the boards within the second orthogonal plane. That is, each sub-array output generated by the differential beamformers 310 and 312 may be configured to cover a specific octave of the covered frequency bands. In embodiments, the differential beamformers 310 and 312 may be configured to generate these differential sub-array outputs using one or more beamforming algorithms, such as, e.g., delay and sum techniques, filter and sum techniques, delay and difference techniques, magnitude shading techniques, and minimum variance distortionless response (“MVDR”) techniques.
The beamformer 304 further comprises two directional beamformers 314 and 316 configured to combine the primary sub-array outputs generated by the primary beamformer 308 with corresponding differential sub-array outputs generated by each of the differential beamformers 310 and 312 to create a linear array output for each orthogonal plane of the array microphone 302. In particular, a first directional beamformer 314 can be configured to combine the primary sub-array outputs with corresponding Y axis differential sub-array outputs provided by the first differential beamformer 310. For example, each primary sub-array output may be combined with the Y axis sub-array output that corresponds to the same octave. Similarly, a second directional beamformer 316 can be configured to combine the primary sub-array outputs with corresponding Z axis differential sub-array outputs provided by the second differential beamformer 312. For example, each primary sub-array output may be combined with the Z axis sub-array output that corresponds to the same octave. The linear array output generated by the first directional beamformer 314 may have a first-order polar pattern directed along a Y axis of the first orthogonal plane (e.g., the XY plane), while the linear array output generated by the second directional beamformer 316 may have a first-order polar pattern directed along a Z axis of the second orthogonal plane (e.g., the XZ plane). In embodiments, the directional beamformers 314 and 316 can be configured to use any appropriate beamforming algorithm for pattern formation and/or to create the various sub-array outputs, including, for example, sum and difference techniques (e.g., as shown in
As shown, the beamformer 304 also comprises a steering beamforming component 318 configured to combine the linear array outputs received from the first and second directional beamformers 314 and 316 to generate a steered output. In embodiments, the steering beamforming component 318 (or “steering beamformer”) can be configured to provide additional control of the array output around the axis of the array, or transverse to the array axis. For example, due to the multi-dimensional placement and harmonic nesting of the microphone elements 302a, 302b, and 302c, the steering beamformer 318 can be configured to create a directional output that can have any first-order pattern (e.g., cardioid, toroid, etc.) and can be steered to any direction within a plane (e.g., the YZ plane) that is perpendicular to the first axis. The steering beamformer 318 also receives the primary array output from the primary beamformer 308 and can be configured to combine the primary array output with the directional outputs from the beamformers 314 and 316 to generate a directional output steered to any angle relative to the axis of the array (e.g., the X axis), or to any direction at any azimuth or elevation about a sphere centered around the center of the array. The steering beamformer 318 may include a combiner or the like for combining together the various outputs to generate a steered directional output for the overall array microphone 302. In embodiments, the steering beamformer 318 may be configured to use any appropriate beamforming algorithm to combine the two directional outputs, including, for example, weight and sum (or magnitude and summation) techniques, filter and sum techniques, and differential techniques.
Thus, the beamformer 304 can be configured to apply array processing techniques (e.g., beamformers 308, 310, and 312), coupled with secondary beamforming processing techniques (e.g., beamformers 314 and 316), and a tertiary beamforming step of magnitude and summation techniques to provide greater control of the array, or the audio pick-up lobe with highest sensitivity. In embodiments, the beamformer 304 may be agnostic to the order in which its components operate due to the linear independence of the three beamforming steps. For example, because the microphone elements within the array microphone 302 are disposed on different axes, the audio signals captured by those elements can be processed along any one of the axes in any order. Thus, while
In some embodiments, the audio inputs 402 may be the audio signals captured by the differential pairs located on each microphone board of the array microphone 302, and the beamformer 400 may be used to combine each differential pair to form cardioid elements with narrowed lobes (or sound pick-up patterns), for example, as compared to the full omnidirectional polar pattern of the individual microphone elements of the array microphone 302. For example, the first audio input 402 may include the audio signals captured by the first microphone element 302a on each microphone board and the second audio input 402 may include the audio signals captured by one of the second microphone elements 302b arranged on the first orthogonal axis of the same microphone board, or one of the third microphone elements 302c arranged on the second orthogonal axis of the same microphone board.
In other embodiments, the audio inputs 402 may include sub-array outputs received from the primary beamformer 308 and one of the differential beamformers 310 and 312 to cover the same covered octave. In such cases, the beamformer 400 may be configured to use sum and difference beamforming techniques to combine the sub-array outputs to form a directional output that is specific to the frequency octave covered by the inputs 402 and constitutes a first-order pattern within the corresponding orthogonal plane of the array microphone 302. For example, the first audio input 402 may include one of the primary sub-array outputs received from the primary beamformer 308, and the second audio input 402 may include a differential sub-array output corresponding to the same octave as the first audio input 402 and received from one of the differential beamformers 310 and 312. The exact beamforming technique used to combine the sub-array signals may vary depending on how the corresponding sub-array is formed, or how the microphone elements are arranged within that sub-array (e.g., linear array, orthogonal array, broadside array, endfire array, etc.). For example, audio signals received from microphone elements arranged in a linear or broadside array may be summed together to generate the sub-array signal. Other techniques may be used for other types of sub-arrays.
As shown in
As also shown, beamformer 400 further includes a correction component 408 for correcting the differential output generated by the difference component 406. The correction component 408 may be configured to correct the differential output for a gradient response caused by the difference calculation. For example, the gradient response may give a 6 dB per octave slope to the frequency response of the audio inputs 402. In order to generate a first-order polar pattern (e.g., cardioid) for the pair of audio inputs 402 over a broad frequency range, the differential output must be corrected so that it has the same magnitude as the summation output. In a preferred embodiment, the correction component 408 applies a correction value of (c*d)/(j*ω) to the difference output to obtain a corrected difference output for the two audio inputs 402 (e.g., (Mic 1−Mic 2)*((c*d)/(j*ω))), where c equals the speed of sound in air at 20 degrees Celsius, d equals the distance between the first and second microphones (e.g., D2 or a multiple thereof), and ω equals the angular frequency. In some cases, a second magnitude correction may be performed to match the sensitivity of the difference component to that of the summation component.
The beamformer 400 also includes a combiner 410 configured to combine, or sum, together the summed output generated by the summation component 404 and the corrected difference output generated by the correction component 408. The combiner 410 thus generates a combined output signal with directional polar pattern (e.g., cardioid) for the pair of audio inputs 402, as shown in
In embodiments, the beamformer 500 can be configured to use cross-over filtering techniques to create the first-order polar patterns. As shown, the beamformer 500 comprises a plurality of filters 502, 504, 506, and 508 and each filter is configured to receive audio signals from a different set of microphone elements. In particular, a very high pass filter 502 is configured to receive audio signals from microphone elements, or sub-arrays, that are configured to cover very high frequencies (e.g., greater than about 4 kHz). A high frequency bandpass filter 504 is configured to receive audio signals from microphone elements, or sub-arrays, that are configured to cover high frequencies (e.g., about 2 kHz to about 4 kHz). A mid frequency bandpass filter 506 is configured to receive audio signals from microphone elements, or sub-arrays, that are configured to cover middle or mid-range frequencies (e.g., about 1 kHz to about 2 kHz). And a low pass filter 508 is configured to receive audio signals from microphone elements, or sub-arrays, that are configured to cover low frequencies (e.g., less than about 1 kHz). The cut-off frequencies for the filters 502, 504, 506, and 508 may be selected based on the specific frequency response characteristics of the corresponding set of microphone elements, including, for example, location of frequency nulls, a desired frequency response for the microphone array, etc.
In various embodiments, the filters 502, 504, 506, and 508 may be analog or digital filters (e.g., digital finite impulse response (FIR) filters on a digital signal processor (DSP), or the like). While
As shown, the filtered outputs of the filters 502, 504, 506, and 508 may be provided to a summation element 510 of the beamformer 500. The summation element 510 may be configured to combine or sum the filtered outputs to generate a combined output signal that may represent a cardioid output for the input microphone elements or other first-order polar pattern.
Thus, the techniques described herein provide an array microphone with improved directivity transverse to the axis of the array, thus providing a larger level of control with regards to where the array is most effective at collecting sound, and a geometry that is designed to allow for minimal diffraction and resonance. For example, the structure of the array microphone is broken up into separate unit cells (or microphone boards) in order to use a minimum amount of interfering material (e.g., PCB substrate) while still placing the microphone elements in a multi-dimensional formation. Moreover, due to the microphone arrangement and overall geometry of the array microphone, the main lobe, or beam with greatest sensitivity, can be placed in any direction within a plane that is perpendicular to the axis of the array. In some cases, the array microphone described herein can be used to simultaneously generate a plurality of individual audio channels, each tailored to capture a particular talker or audio source, while removing room noise, other talker noise, and other unwanted sounds. For example, the array microphone can be used for stage or live music reinforcement to provide selective coverage of the stage or audience, or in live sports environments to isolate ambience from game audio. Thus, the array microphone can provide not only improved directivity with wideband audio application (e.g., 20 Hz≤f≤20 kHz), but also improved signal to noise ratio (SNR) and acoustic echo cancellation (AEC) properties. The techniques described herein also provide an array microphone with greater flexibility, e.g., compared to existing linear arrays, that allows for expanded functionality. For example, the pattern formation techniques described herein may enable the array microphone to be used as a mid-side stereo array, or for mid-side stereo generation that is highly directional and could improve spatialization in simple two-channel recordings (e.g., with the addition of elevation information). As another example, the array geometry described herein may allow for first-order ambisonic collection, with only minor alteration to the signal flow. In some cases, the techniques described herein can be used for integrated or UC devices to minimize coupling between transmit and receive acoustic systems.
Referring back to
One or more components of the audio system 300 may be in wired or wireless communication with one or more other components of the system 300. For example, the array microphone 302 may transmit the plurality of audio signals to the beamformer 304, the output generation unit 306, a separate audio processor (not shown), or a computing device comprising one or more of the same, using a wired or wireless connection. In some embodiments, one or more components of the audio system 300 may communicate with one or more other components of the system 300 via a suitable application programming interface (API). For example, one or more APIs may enable components of the audio processor to transmit audio and/or data signals between themselves.
In some embodiments, one or more components of the audio system 300 may be combined into, or reside in, a single unit or device. For example, all of the components of the audio system 300 may be included in the same device, such as the microphone 302, or a computing device that includes the microphone 302. As another example, the output generation unit 306 may be included in, or combined with, the beamformer 304 and/or with the microphone 302. In some embodiments, the audio system 300 may take the form of a cloud based system or other distributed system, such that the components of the system 300 may or may not be physically located in proximity to each other.
The components of the audio system 300 may be implemented in hardware (e.g., discrete logic circuits, application specific integrated circuits (ASIC), programmable gate arrays (PGA), field programmable gate arrays (FPGA), digital signal processors (DSP), microprocessor, etc.), using software executable by one or more servers or computers, or other computing device having a processor and memory (e.g., a personal computer (PC), a laptop, a tablet, a mobile device, a smart device, thin client, etc.), or through a combination of both hardware and software. For example, some or all components of the microphone 302, the beamformer 304, and/or the output generation unit 306 may be implemented using discrete circuitry devices and/or using one or more processors (e.g., audio processor and/or digital signal processor) executing program code stored in a memory (not shown), the program code being configured to carry out one or more processes or operations described herein. Thus, in embodiments, one or more of the components of the audio system 300 may include one or more processors, memory devices, computing devices, and/or other hardware components not shown in the figures.
All or portions of the processes described herein may be performed by one or more processing devices or processors (e.g., analog to digital converters, encryption chips, etc.) that are within or external to the audio system 300 of
Any of the processors described herein may include a general purpose processor (e.g., a microprocessor) and/or a special purpose processor (e.g., an audio processor, a digital signal processor, etc.). In some examples, the processor(s) described herein may be any suitable processing device or set of processing devices such as, but not limited to, a microprocessor, a microcontroller-based platform, an integrated circuit, one or more field programmable gate arrays (FPGAs), and/or one or more application-specific integrated circuits (ASICs).
Any of the memories or memory devices described herein may be volatile memory (e.g., RAM including non-volatile RAM, magnetic RAM, ferroelectric RAM, etc.), non-volatile memory (e.g., disk memory, FLASH memory, EPROMs, EEPROMs, memristor-based non-volatile solid-state memory, etc.), unalterable memory (e.g., EPROMs), read-only memory, and/or high-capacity storage devices (e.g., hard drives, solid state drives, etc.). In some examples, the memory described herein includes multiple kinds of memory, particularly volatile memory and non-volatile memory.
Moreover, any of the memories described herein may be computer readable media on which one or more sets of instructions can be embedded. The instructions may reside completely, or at least partially, within any one or more of the memory, the computer readable medium, and/or within one or more processors during execution of the instructions. In some embodiments, the memory described herein may include one or more data storage devices configured for implementation of a persistent storage for data that needs to be stored and recalled by the end user. In such cases, the data storage device(s) may save data in flash memory or other memory devices. In some embodiments, the data storage device(s) can be implemented using, for example, SQLite data base, UnQLite, Berkeley DB, BangDB, or the like.
Any of the computing devices described herein can be any generic computing device comprising at least one processor and a memory device. In some embodiments, the computing device may be a standalone computing device included in the audio system 300, or may reside in another component of the audio system 300, such as, e.g., the microphone 302, the beamformer 304, or the output generation unit 306. In such embodiments, the computing device may be physically located in and/or dedicated to the given environment or room, such as, e.g., the same environment in which the microphone 302 is located. In other embodiments, the computing device may not be physically located in proximity to the microphone 302 but may reside in an external network, such as a cloud computing network, or may be otherwise distributed in a cloud-based environment. Moreover, in some embodiments, the computing device may be implemented with firmware or completely software-based as part of a network, which may be accessed or otherwise communicated with via another device, including other computing devices, such as, e.g., desktops, laptops, mobile devices, tablets, smart devices, etc. Thus, the term “computing device” should be understood to include distributed systems and devices (such as those based on the cloud), as well as software, firmware, and other components configured to carry out one or more of the functions described herein. Further, one or more features of the computing device may be physically remote and may be communicatively coupled to the computing device.
In some embodiments, any of the computing devices described herein may include one or more components configured to facilitate a conference call, meeting, classroom, or other event and/or process audio signals associated therewith to improve an audio quality of the event. For example, in various embodiments, any computing device described herein may comprise a digital signal processor (“DSP”) configured to process the audio signals received from the various microphones or other audio sources using, for example, automatic mixing, matrix mixing, delay, compressor, parametric equalizer (“PEQ”) functionalities, acoustic echo cancellation, and more. In other embodiments, the DSP may be a standalone device operatively coupled or connected to the computing device using a wired or wireless connection. One exemplary embodiment of the DSP, when implemented in hardware, is the P300 IntelliMix Audio Conferencing Processor from SHURE, the user manual for which is incorporated by reference in its entirety herein. As further explained in the P300 manual, this audio conferencing processor includes algorithms optimized for audio/video conferencing applications and for providing a high quality audio experience, including eight channels of acoustic echo cancellation, noise reduction and automatic gain control. Another exemplary embodiment of the DSP, when implemented in software, is the IntelliMix Room from SHURE, the user guide for which is incorporated by reference in its entirety herein. As further explained in the IntelliMix Room user guide, this DSP software is configured to optimize the performance of networked microphones with audio and video conferencing software and is designed to run on the same computer as the conferencing software. In other embodiments, other types of audio processors, digital signal processors, and/or DSP software components may be used to carry out one or more of audio processing techniques described herein, as will be appreciated.
Moreover, any of the computing devices described herein may also comprise various other software modules or applications (not shown) configured to facilitate and/or control the conferencing event, such as, for example, internal or proprietary conferencing software and/or third-party conferencing software (e.g., Microsoft Skype, Microsoft Teams, Bluejeans, Cisco WebEx, GoToMeeting, Zoom, Join.me, etc.). Such software applications may be stored in the memory of the computing device and/or may be stored on a remote server (e.g., on premises or as part of a cloud computing network) and accessed by the computing device via a network connection. Some software applications may be configured as a distributed cloud-based software with one or more portions of the application residing in the computing device and one or more other portions residing in a cloud computing network. One or more of the software applications may reside in an external network, such as a cloud computing network. In some embodiments, access to one or more of the software applications may be via a web-portal architecture, or otherwise provided as Software as a Service (SaaS).
In general, a computer program product in accordance with embodiments described herein includes a computer usable storage medium (e.g., standard random access memory (RAM), an optical disc, a universal serial bus (USB) drive, or the like) having computer-readable program code embodied therein, wherein the computer-readable program code is adapted to be executed by a processor (e.g., working in connection with an operating system) to implement the methods described herein. In this regard, the program code may be implemented in any desired language, and may be implemented as machine code, assembly code, byte code, interpretable source code or the like (e.g., via C, C++, Java, ActionScript, Python, Objective-C, JavaScript, CSS, XML, and/or others). In some embodiments, the program code may be a computer program stored on a non-transitory computer readable medium that is executable by a processor of the relevant device.
The terms “non-transitory computer-readable medium” and “computer-readable medium” include a single medium or multiple media, such as a centralized or distributed database, and/or associated caches and servers that store one or more sets of instructions. Further, the terms “non-transitory computer-readable medium” and “computer-readable medium” include any tangible medium that is capable of storing, encoding, or carrying a set of instructions for execution by a processor or that cause a system to perform any one or more of the methods or operations disclosed herein. As used herein, the term “computer readable medium” is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals.
Any process descriptions or blocks in the figures should be understood as representing modules, segments, or portions of code that include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the embodiments described herein, in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those having ordinary skill in the art.
It should be noted that in the description and drawings, like or substantially similar elements may be labeled with the same reference numerals. However, sometimes these elements may be labeled with differing numbers, such as, for example, in cases where such labeling facilitates a more clear description. In addition, system components can be variously arranged, as is known in the art. Also, the drawings set forth herein are not necessarily drawn to scale, and in some instances, proportions may be exaggerated to more clearly depict certain features and/or related elements may be omitted to emphasize and clearly illustrate the novel features described herein. Such labeling and drawing practices do not necessarily implicate an underlying substantive purpose. The above description is intended to be taken as a whole and interpreted in accordance with the principles taught herein and understood to one of ordinary skill in the art.
In this disclosure, the use of the disjunctive is intended to include the conjunctive. The use of definite or indefinite articles is not intended to indicate cardinality. In particular, a reference to “the” object or “a” and “an” object is intended to also denote one of a possible plurality of such objects.
This disclosure describes, illustrates, and exemplifies one or more particular embodiments of the invention in accordance with its principles. The disclosure is intended to explain how to fashion and use various embodiments in accordance with the technology rather than to limit the true, intended, and fair scope and spirit thereof. That is, the foregoing description is not intended to be exhaustive or to be limited to the precise forms disclosed herein, but rather to explain and teach the principles of the invention in such a way as to enable one of ordinary skill in the art to understand these principles and, with that understanding, be able to apply them to practice not only the embodiments described herein, but also other embodiments that may come to mind in accordance with these principles. The embodiment(s) provided herein were chosen and described to provide the best illustration of the principle of the described technology and its practical application, and to enable one of ordinary skill in the art to utilize the technology in various embodiments and with various modifications as are suited to the particular use contemplated. All such modifications and variations are within the scope of the embodiments as determined by the appended claims, as may be amended during the pendency of this application for patent, and all equivalents thereof, when interpreted in accordance with the breadth to which they are fairly, legally, and equitably entitled.
This application claims priority to U.S. Provisional Pat. App. No. 63/501,998, filed on May 12, 2023, the contents of which are incorporated herein in their entirety.
Number | Date | Country | |
---|---|---|---|
63501998 | May 2023 | US |