Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition functionality

Description

TECHNICAL FIELD

This application generally relates to an array microphone having automatic focus and placement of beamformed microphone lobes. In particular, this application relates to an array microphone that adjusts the focus and placement of beamformed microphone lobes based on the detection of sound activity after the lobes have been initially placed, and allows inhibition of the adjustment of the focus and placement of the beamformed microphone lobes based on a remote far end audio signal.

BACKGROUND

Conferencing environments, such as conference rooms, boardrooms, video conferencing applications, and the like, can involve the use of microphones for capturing sound from various audio sources active in such environments. Such audio sources may include humans speaking, for example. The captured sound may be disseminated to a local audience in the environment through amplified speakers (for sound reinforcement), and/or to others remote from the environment (such as via a telecast and/or a webcast). The types of microphones and their placement in a particular environment may depend on the locations of the audio sources, physical space requirements, aesthetics, room layout, and/or other considerations. For example, in some environments, the microphones may be placed on a table or lectern near the audio sources. In other environments, the microphones may be mounted overhead to capture the sound from the entire room, for example. Accordingly, microphones are available in a variety of sizes, form factors, mounting options, and wiring options to suit the needs of particular environments.

Traditional microphones typically have fixed polar patterns and few manually selectable settings. To capture sound in a conferencing environment, many traditional microphones can be used at once to capture the audio sources within the environment. However, traditional microphones tend to capture unwanted audio as well, such as room noise, echoes, and other undesirable audio elements. The capturing of these unwanted noises is exacerbated by the use of many microphones.

Array microphones having multiple microphone elements can provide benefits such as steerable coverage or pick up patterns (having one or more lobes), which allow the microphones to focus on the desired audio sources and reject unwanted sounds such as room noise. The ability to steer audio pick up patterns provides the benefit of being able to be less precise in microphone placement, and in this way, array microphones are more forgiving. Moreover, array microphones provide the ability to pick up multiple audio sources with one array microphone or unit, again due to the ability to steer the pickup patterns.

However, the position of lobes of a pickup pattern of an array microphone may not be optimal in certain environments and situations. For example, an audio source that is initially detected by a lobe may move and change locations. In this situation, the lobe may not optimally pick up the audio source at the its new location.

Accordingly, there is an opportunity for an array microphone that addresses these concerns. More particularly, there is an opportunity for an array microphone that automatically focuses and/or places beamformed microphone lobes based on the detection of sound activity after the lobes have been initially placed, while also being able to inhibit the focus and/or placement of the beamformed microphone lobes based on a remote far end audio signal, which can result in higher quality sound capture and more optimal coverage of environments.

SUMMARY

The invention is intended to solve the above-noted problems by providing array microphone systems and methods that are designed to, among other things: (1) enable automatic focusing of beamformed lobes of an array microphone in response to the detection of sound activity, after the lobes have been initially placed; (2) enable automatic placement of beamformed lobes of an array microphone in response to the detection of sound activity; (3) enable automatic focusing of beamformed lobes of an array microphone within lobe regions in response to the detection of sound activity, after the lobes have been initially placed; and (4) inhibit or restrict the automatic focusing or automatic placement of beamformed lobes of an array microphone, based on activity of a remote far end audio signal.

In an embodiment, beamformed lobes that have been positioned at initial coordinates may be focused by moving the lobes to new coordinates in the general vicinity of the initial coordinates, when new sound activity is detected at the new coordinates.

In another embodiment, beamformed lobes may be placed or moved to new coordinates, when new sound activity is detected at the new coordinates.

In a further embodiment, beamformed lobes that have been positioned at initial coordinates may be focused by moving the lobes, but confined within lobe regions, when new sound activity is detected at the new coordinates.

In another embodiment, the movement or placement of beamformed lobes may be inhibited or restricted, when the activity of a remote far end audio signal exceeds a predetermined threshold.

These and other embodiments, and various permutations and aspects, will become apparent and be more fully understood from the following detailed description and accompanying drawings, which set forth illustrative embodiments that are indicative of the various ways in which the principles of the invention may be employed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an array microphone with automatic focusing of beamformed lobes in response to the detection of sound activity, in accordance with some embodiments.

FIG. 2 is a flowchart illustrating operations for automatic focusing of beamformed lobes, in accordance with some embodiments.

FIG. 3 is a flowchart illustrating operations for automatic focusing of beamformed lobes that utilizes a cost functional, in accordance with some embodiments.

FIG. 4 is a schematic diagram of an array microphone with automatic placement of beamformed lobes of an array microphone in response to the detection of sound activity, in accordance with some embodiments.

FIG. 5 is a flowchart illustrating operations for automatic placement of beamformed lobes, in accordance with some embodiments.

FIG. 6 is a flowchart illustrating operations for finding lobes near detected sound activity, in accordance with some embodiments.

FIG. 7 is an exemplary depiction of an array microphone with beamformed lobes within lobe regions, in accordance with some embodiments.

FIG. 8 is a flowchart illustrating operations for automatic focusing of beamformed lobes within lobe regions, in accordance with some embodiments.

FIG. 9 is a flowchart illustrating operations for determining whether detected sound activity is within a look radius of a lobe, in accordance with some embodiments.

FIG. 10 is an exemplary depiction of an array microphone with beamformed lobes within lobe regions and showing a look radius of a lobe, in accordance with some embodiments.

FIG. 11 is a flowchart illustrating operations for determining movement of a lobe within a move radius of a lobe, in accordance with some embodiments.

FIG. 12 is an exemplary depiction of an array microphone with beamformed lobes within lobe regions and showing a move radius of a lobe, in accordance with some embodiments.

FIG. 13 is an exemplary depiction of an array microphone with beamformed lobes within lobe regions and showing boundary cushions between lobe regions, in accordance with some embodiments.

FIG. 14 is a flowchart illustrating operations for limiting movement of a lobe based on boundary cushions between lobe regions, in accordance with some embodiments.

FIG. 15 is an exemplary depiction of an array microphone with beamformed lobes within regions and showing the movement of a lobe based on boundary cushions between regions, in accordance with some embodiments.

FIG. 16 is a schematic diagram of an array microphone with automatic focusing of beamformed lobes in response to the detection of sound activity and inhibition of the automatic focusing based on a remote far end audio signal, in accordance with some embodiments.

FIG. 17 is a schematic diagram of an array microphone with automatic placement of beamformed lobes of an array microphone in response to the detection of sound activity and inhibition of the automatic placement based on a remote far end audio signal, in accordance with some embodiments.

FIG. 18 is a flowchart illustrating operations for inhibiting automatic adjustment of beamformed lobes of an array microphone based on a remote far end audio signal, in accordance with some embodiments.

FIG. 19 is a schematic diagram of an array microphone with automatic placement of beamformed lobes of an array microphone in response to the detection of sound activity and activity detection of the sound activity, in accordance with some embodiments.

FIG. 20 is a flowchart illustrating operations for automatic placement of beamformed lobes including activity detection of sound activity, in accordance with some embodiments.

DETAILED DESCRIPTION

The description that follows describes, illustrates and exemplifies one or more particular embodiments of the invention in accordance with its principles. This description is not provided to limit the invention to the embodiments described herein, but rather to explain and teach the principles of the invention in such a way to enable one of ordinary skill in the art to understand these principles and, with that understanding, be able to apply them to practice not only the embodiments described herein, but also other embodiments that may come to mind in accordance with these principles. The scope of the invention is intended to cover all such embodiments that may fall within the scope of the appended claims, either literally or under the doctrine of equivalents.

It should be noted that in the description and drawings, like or substantially similar elements may be labeled with the same reference numerals. However, sometimes these elements may be labeled with differing numbers, such as, for example, in cases where such labeling facilitates a more clear description. Additionally, the drawings set forth herein are not necessarily drawn to scale, and in some instances proportions may have been exaggerated to more clearly depict certain features. Such labeling and drawing practices do not necessarily implicate an underlying substantive purpose. As stated above, the specification is intended to be taken as a whole and interpreted in accordance with the principles of the invention as taught herein and understood to one of ordinary skill in the art.

The array microphone systems and methods described herein can enable the automatic focusing and placement of beamformed lobes in response to the detection of sound activity, as well as allow the focus and placement of the beamformed lobes to be inhibited based on a remote far end audio signal. In embodiments, the array microphone may include a plurality of microphone elements, an audio activity localizer, a lobe auto-focuser, a database, and a beamformer. The audio activity localizer may detect the coordinates and confidence score of new sound activity, and the lobe auto-focuser may determine whether there is a previously placed lobe nearby the new sound activity. If there is such a lobe and the confidence score of the new sound activity is greater than a confidence score of the lobe, then the lobe auto-focuser may transmit the new coordinates to the beamformer so that the lobe is moved to the new coordinates. In these embodiments, the location of a lobe may be improved and automatically focused on the latest location of audio sources inside and near the lobe, while also preventing the lobe from overlapping, pointing in an undesirable direction (e.g., towards unwanted noise), and/or moving too suddenly.

In other embodiments, the array microphone may include a plurality of microphone elements, an audio activity localizer, a lobe auto-placer, a database, and a beamformer. The audio activity localizer may detect the coordinates of new sound activity, and the lobe auto-placer may determine whether there is a lobe nearby the new sound activity. If there is not such a lobe, then the lobe auto-placer may transmit the new coordinates to the beamformer so that an inactive lobe is placed at the new coordinates or so that an existing lobe is moved to the new coordinates. In these embodiments, the set of active lobes of the array microphone may point to the most recent sound activity in the coverage area of the array microphone.

In other embodiments, the audio activity localizer may detect the coordinates and confidence score of new sound activity, and if the confidence score of the new sound activity is greater than a threshold, the lobe auto-focuser may identify a lobe region that the new sound activity belongs to. In the identified lobe region, a previously placed lobe may be moved if the coordinates are within a look radius of the current coordinates of the lobe, i.e., a three-dimensional region of space around the current coordinates of the lobe where new sound activity can be considered. The movement of the lobe in the lobe region may be limited to within a move radius of the current coordinates of the lobe, i.e., a maximum distance in three-dimensional space that the lobe is allowed to move, and/or limited to outside a boundary cushion between lobe regions, i.e., how close a lobe can move to the boundaries between lobe regions. In these embodiments, the location of a lobe may be improved and automatically focused on the latest location of audio sources inside the lobe region associated with the lobe, while also preventing the lobes from overlapping, pointing in an undesirable direction (e.g., towards unwanted noise), and/or moving too suddenly.

In further embodiments, an activity detector may receive a remote audio signal, such as from a far end. The sound of the remote audio signal may be played in the local environment, such as on a loudspeaker within a conference room. If the activity of the remote audio signal exceeds a predetermined threshold, then the automatic adjustment (i.e., focus and/or placement) of beamformed lobes may be inhibited from occurring. For example, the activity of the remote audio signal could be measured by the energy level of the remote audio signal. In this example, the energy level of the remote audio signal may exceed the predetermined threshold when there is a certain level of speech or voice contained in the remote audio signal. In this situation, it may be desirable to prevent automatic adjustment of the beamformed lobes so that lobes are not directed to pick up the sound from the remote audio signal, e.g., that is being played in local environment. However, if the energy level of the remote audio signal does not exceed the predetermined threshold, then the automatic adjustment of beamformed lobes may be performed. The automatic adjustment of the beamformed lobes may include, for example, the automatic focus and/or placement of the lobes as described herein. In these embodiments, the location of a lobe may be improved and automatically focused and/or placed when the activity of the remote audio signal does not exceed a predetermined threshold, and inhibited or restricted from being automatically focused and/or placed when the activity of the remote audio signal exceeds the predetermined threshold.

Through the use of the systems and methods herein, the quality of the coverage of audio sources in an environment may be improved by, for example, ensuring that beamformed lobes are optimally picking up the audio sources even if the audio sources have moved and changed locations from an initial position. The quality of the coverage of audio source in an environment may also be improved by, for example, reducing the likelihood that beamformed lobes are deployed (e.g., focused or placed) to pick up unwanted sounds like voice, speech, or other noise from the far end.

FIGS. 1 and 4 are schematic diagrams of array microphones 100, 400 that can detect sounds from audio sources at various frequencies. The array microphone 100, 400 may be utilized in a conference room or boardroom, for example, where the audio sources may be one or more human speakers. Other sounds may be present in the environment which may be undesirable, such as noise from ventilation, other persons, audio/visual equipment, electronic devices, etc. In a typical situation, the audio sources may be seated in chairs at a table, although other configurations and placements of the audio sources are contemplated and possible.

The array microphone 100, 400 may be placed on or in a table, lectern, desktop, wall, ceiling, etc. so that the sound from the audio sources can be detected and captured, such as speech spoken by human speakers. The array microphone 100, 400 may include any number of microphone elements 102a,b, . . . , zz, 402a,b, . . . , zz, for example, and be able to form multiple pickup patterns with lobes so that the sound from the audio sources can be detected and captured. Any appropriate number of microphone elements 102, 402 are possible and contemplated.

Each of the microphone elements 102, 402 in the array microphone 100, 400 may detect sound and convert the sound to an analog audio signal. Components in the array microphone 100, 400, such as analog to digital converters, processors, and/or other components, may process the analog audio signals and ultimately generate one or more digital audio output signals. The digital audio output signals may conform to the Dante standard for transmitting audio over Ethernet, in some embodiments, or may conform to another standard and/or transmission protocol. In embodiments, each of the microphone elements 102, 402 in the array microphone 100, 400 may detect sound and convert the sound to a digital audio signal.

One or more pickup patterns may be formed by a beamformer 170, 470 in the array microphone 100, 400 from the audio signals of the microphone elements 102, 402. The beamformer 170, 470 may generate digital output signals 190a,b, c, . . . z, 490a,b, c, . . . , z corresponding to each of the pickup patterns. The pickup patterns may be composed of one or more lobes, e.g., main, side, and back lobes. In other embodiments, the microphone elements 102, 402 in the array microphone 100, 400 may output analog audio signals so that other components and devices (e.g., processors, mixers, recorders, amplifiers, etc.) external to the array microphone 100, 400 may process the analog audio signals.

The array microphone 100 of FIG. 1 that automatically focuses beamformed lobes in response to the detection of sound activity may include the microphone elements 102; an audio activity localizer 150 in wired or wireless communication with the microphone elements 102; a lobe auto-focuser 160 in wired or wireless communication with the audio activity localizer 150; a beamformer 170 in wired or wireless communication with the microphone elements 102 and the lobe auto-focuser 160; and a database 180 in wired or wireless communication with the lobe auto-focuser 160. These components are described in more detail below.

The array microphone 400 of FIG. 4 that automatically places beamformed lobes in response to the detection of sound activity may include the microphone elements 402; an audio activity localizer 450 in wired or wireless communication with the microphone elements 402; a lobe auto-placer 460 in wired or wireless communication with the audio activity localizer 450; a beamformer 470 in wired or wireless communication with the microphone elements 402 and the lobe auto-placer 460; and a database 480 in wired or wireless communication with the lobe auto-placer 460. These components are described in more detail below.

In embodiments, the array microphone 100, 400 may include other components, such as an acoustic echo canceller or an automixer, that works with the audio activity localizer 150, 450 and/or the beamformer 170, 470. For example, when a lobe is moved to new coordinates in response to detecting new sound activity, as described herein, information from the movement of the lobe may be utilized by an acoustic echo canceller to minimize echo during the movement and/or by an automixer to improve its decision making capability. As another example, the movement of a lobe may be influenced by the decision of an automixer, such as allowing a lobe to be moved that the automixer has identified as having pertinent voice activity. The beamformer 170, 470 may be any suitable beamformer, such as a delay and sum beamformer or a minimum variance distortionless response (MVDR) beamformer.

The various components included in the array microphone 100, 400 may be implemented using software executable by one or more servers or computers, such as a computing device with a processor and memory, graphics processing units (GPUs), and/or by hardware (e.g., discrete logic circuits, application specific integrated circuits (ASIC), programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

In some embodiments, the microphone elements 102, 402 may be arranged in concentric rings and/or harmonically nested. The microphone elements 102, 402 may be arranged to be generally symmetric, in some embodiments. In other embodiments, the microphone elements 102, 402 may be arranged asymmetrically or in another arrangement. In further embodiments, the microphone elements 102, 402 may be arranged on a substrate, placed in a frame, or individually suspended, for example. An embodiment of an array microphone is described in commonly assigned U.S. Pat. No. 9,565,493, which is hereby incorporated by reference in its entirety herein. In embodiments, the microphone elements 102, 402 may be unidirectional microphones that are primarily sensitive in one direction. In other embodiments, the microphone elements 102, 402 may have other directionalities or polar patterns, such as cardioid, subcardioid, or omnidirectional, as desired. The microphone elements 102, 402 may be any suitable type of transducer that can detect the sound from an audio source and convert the sound to an electrical audio signal. In an embodiment, the microphone elements 102, 402 may be micro-electrical mechanical system (MEMS) microphones. In other embodiments, the microphone elements 102, 402 may be condenser microphones, balanced armature microphones, electret microphones, dynamic microphones, and/or other types of microphones. In embodiments, the microphone elements 102, 402 may be arrayed in one dimension or two dimensions. The array microphone 100, 400 may be placed or mounted on a table, a wall, a ceiling, etc., and may be next to, under, or above a video monitor, for example.

An embodiment of a process 200 for automatic focusing of previously placed beamformed lobes of the array microphone 100 is shown in FIG. 2. The process 200 may be performed by the lobe auto-focuser 160 so that the array microphone 100 can output one or more audio signals 180 from the array microphone 100, where the audio signals 180 may include sound picked up by the beamformed lobes that are focused on new sound activity of an audio source. One or more processors and/or other processing components (e.g., analog to digital converters, encryption chips, etc.) within or external to the array microphone 100 may perform any, some, or all of the steps of the process 200. One or more other types of components (e.g., memory, input and/or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be utilized in conjunction with the processors and/or other processing components to perform any, some, or all of the steps of the process 200.

At step 202, the coordinates and a confidence score corresponding to new sound activity may be received at the lobe auto-focuser 160 from the audio activity localizer 150. The audio activity localizer 150 may continuously scan the environment of the array microphone 100 to find new sound activity. The new sound activity found by the audio activity localizer 150 may include suitable audio sources, e.g., human speakers, that are not stationary. The coordinates of the new sound activity may be a particular three dimensional coordinate relative to the location of the array microphone 100, such as in Cartesian coordinates (i.e., x, y, z), or in spherical coordinates (i.e., radial distance/magnitude r, elevation angle θ (theta), azimuthal angle φ (phi)). The confidence score of the new sound activity may denote the certainty of the coordinates and/or the quality of the sound activity, for example. In embodiments, other suitable metrics related to the new sound activity may be received and utilized at step 202. It should be noted that Cartesian coordinates may be readily converted to spherical coordinates, and vice versa, as needed.

The lobe auto-focuser 160 may determine whether the coordinates of the new sound activity are nearby (i.e., in the vicinity of) an existing lobe, at step 204. Whether the new sound activity is nearby an existing lobe may be based on the difference in azimuth and/or elevation angles of (1) the coordinates of the new sound activity and (2) the coordinates of the existing lobe, relative to a predetermined threshold. The distance of the new sound activity away from the microphone 100 may also influence the determination of whether the coordinates of the new sound activity are nearby an existing lobe. The lobe auto-focuser 160 may retrieve the coordinates of the existing lobe from the database 180 for use in step 204, in some embodiments. An embodiment of the determination of whether the coordinates of the new sound activity are nearby an existing lobe is described in more detail below with respect to FIG. 6.

If the lobe auto-focuser 160 determines that the coordinates of the new sound activity are not nearby an existing lobe at step 204, then the process 200 may end at step 210 and the locations of the lobes of the array microphone 100 are not updated. In this scenario, the coordinates of the new sound activity may be considered to be outside the coverage area of the array microphone 100 and the new sound activity may therefore be ignored. However, if at step 204 the lobe auto-focuser 160 determines that the coordinates of the new sound activity are nearby an existing lobe, then the process 200 continues to step 206. In this scenario, the coordinates of the new sound activity may be considered to be an improved (i.e., more focused) location of the existing lobe.

At step 206, the lobe auto-focuser 160 may compare the confidence score of the new sound activity to the confidence score of the existing lobe. The lobe auto-focuser 160 may retrieve the confidence score of the existing lobe from the database 180, in some embodiments. If the lobe auto-focuser 160 determines at step 206 that the confidence score of the new sound activity is less than (i.e., worse than) the confidence score of the existing lobe, then the process 200 may end at step 210 and the locations of the lobes of the array microphone 100 are not updated. However, if the lobe auto-focuser 160 determines at step 206 that the confidence score of the new sound activity is greater than or equal to (i.e., better than or more favorable than) the confidence score of the existing lobe, then the process 200 may continue to step 208. At step 208, the lobe auto-focuser 160 may transmit the coordinates of the new sound activity to the beamformer 170 so that the beamformer 170 can update the location of the existing lobe to the new coordinates. In addition, the lobe auto-focuser 160 may store the new coordinates of the lobe in the database 180.

In some embodiments, at step 208, the lobe auto-focuser 160 may limit the movement of an existing lobe to prevent and/or minimize sudden changes in the location of the lobe. For example, the lobe auto-focuser 160 may not move a particular lobe to new coordinates if that lobe has been recently moved within a certain recent time period. As another example, the lobe auto-focuser 160 may not move a particular lobe to new coordinates if those new coordinates are too close to the lobe's current coordinates, too close to another lobe, overlapping another lobe, and/or considered too far from the existing position of the lobe.

The process 200 may be continuously performed by the array microphone 100 as the audio activity localizer 150 finds new sound activity and provides the coordinates and confidence score of the new sound activity to the lobe auto-focuser 160. For example, the process 200 may be performed as audio sources, e.g., human speakers, are moving around a conference room so that one or more lobes can be focused on the audio sources to optimally pick up their sound.

An embodiment of a process 300 for automatic focusing of previously placed beamformed lobes of the array microphone 100 using a cost functional is shown in FIG. 3. The process 300 may be performed by the lobe auto-focuser 160 so that the array microphone 100 can output one or more audio signals 180, where the audio signals 180 may include sound picked up by the beamformed lobes that are focused on new sound activity of an audio source. One or more processors and/or other processing components (e.g., analog to digital converters, encryption chips, etc.) within or external to the microphone array 100 may perform any, some, or all of the steps of the process 300. One or more other types of components (e.g., memory, input and/or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be utilized in conjunction with the processors and/or other processing components to perform any, some, or all of the steps of the process 300.

Steps 302, 304, and 306 of the process 300 for the lobe auto-focuser 160 may be substantially the same as steps 202, 204, and 206 of the process 200 of FIG. 2 described above. In particular, the coordinates and a confidence score corresponding to new sound activity may be received at the lobe auto-focuser 160 from the audio activity localizer 150. The lobe auto-focuser 160 may determine whether the coordinates of the new sound activity are nearby (i.e., in the vicinity of) an existing lobe. If the coordinates of the new sound activity are not nearby an existing lobe (or if the confidence score of the new sound activity is less than the confidence score of the existing lobe), then the process 300 may proceed to step 324 and the locations of the lobes of the array microphone 100 are not updated. However, if at step 306, the lobe auto-focuser 160 determines that the confidence score of the new sound activity is more than (i.e., better than or more favorable than) the confidence score of the existing lobe, then the process 300 may continue to step 308. In this scenario, the coordinates of the new sound activity may be considered to be a candidate location to move the existing lobe to, and a cost functional of the existing lobe may be evaluated and maximized, as described below.

A cost functional for a lobe may take into account spatial aspects of the lobe and the audio quality of the new sound activity. As used herein, a cost functional and a cost function have the same meaning. In particular, the cost functional for a lobe i may be defined in some embodiments as a function of the coordinates of the new sound activity (LC_i), a signal-to-noise ratio for the lobe (SNR_i), a gain value for the lobe (Gain_i), voice activity detection information related to the new sound activity (VAD_i), and distances from the coordinates of the existing lobe (distance (LO_i)). In other embodiments, the cost functional for a lobe may be a function of other information. The cost functional for a lobe i can be written as J_i(x, y, z) with Cartesian coordinates or J_i(azimuth, elevation, magnitude) with spherical coordinates, for example. Using the cost functional with Cartesian coordinates as exemplary, the cost functional J_i(x, y, z)=f (LC_i, distance(LO_i), Gain_i, SNR_i, VAD_i). Accordingly, the lobe may be moved by evaluating and maximizing the cost functional J_iover a spatial grid of coordinates, such that the movement of the lobe is in the direction of the gradient (i.e., steepest ascent) of the cost functional. The maximum of the cost functional may be the same as the coordinates of the new sound activity received by the lobe auto-focuser 160 at step 302 (i.e., the candidate location), in some situations. In other situations, the maximum of the cost functional may move the lobe to a different position than the coordinates of the new sound activity, when taking into account the other parameters described above.

At step 308, the cost functional for the lobe may be evaluated by the lobe auto-focuser 160 at the coordinates of the new sound activity. The evaluated cost functional may be stored by the lobe auto-focuser 160 in the database 180, in some embodiments. At step 310, the lobe auto-focuser 160 may move the lobe by each of an amount Δx, Δy, Δz in the x, y, and z directions, respectively, from the coordinates of the new sound activity. After each movement, the cost functional may be evaluated by the lobe auto-focuser 160 at each of these locations. For example, the lobe may be moved to a location (x+Δx, y, z) and the cost functional may be evaluated at that location; then moved to a location (x, y+Δy, z) and the cost functional may be evaluated at that location; and then moved to a location (x, y, z+Δz) and the cost functional may be evaluated at that location. The lobe may be moved by the amounts Δx, Δy, Δz in any order at step 310. Each of the evaluated cost functionals at these locations may be stored by the lobe auto-focuser 160 in the database 180, in some embodiments. The evaluations of the cost functional are performed by the lobe auto-focuser 160 at step 310 in order to compute an estimate of partial derivatives and the gradient of the cost functional, as described below. It should be noted that while the description above is with relation to Cartesian coordinates, a similar operation may be performed with spherical coordinates (e.g., Δazimuth, Δelevation, Δmagnitude).

At step 312, the gradient of the cost functional may be calculated by the lobe auto-focuser 160 based on the set of estimates of the partial derivatives. The gradient ∇J may calculated as follows:

$\nabla J = (g x_{i}, {gy}_{i}, g z_{i}) \approx (\frac{\begin{matrix} J_{i} (x_{i} + Δ x, y_{i}, z_{i}) - \\ J_{i} (x_{i}, y_{i}, z_{i}) \end{matrix}}{Δ x}, \frac{\begin{matrix} J_{i} (x_{i}, y_{i} + Δ y, z_{i}) - \\ J_{i} (x_{i}, y_{i}, z_{i}) \end{matrix}}{Δ y}, \frac{\begin{matrix} J_{i} (x_{i}, y_{i}, z_{i} + Δ z) - \\ J_{i} (x_{i}, y_{i}, z_{i}) \end{matrix}}{Δ z})$

At step 314, the lobe auto-focuser 160 may move the lobe by a predetermined step size p in the direction of the gradient ∇J calculated at step 312. In particular, the lobe may be moved to a new location: (x_i+μgx_i,y_i+μgy_i,z_i+μgz_i) The cost functional of the lobe at this new location may also be evaluated by the lobe auto-focuser 160 at step 314. This cost functional may be stored by the lobe auto-focuser 160 in the database 180, in some embodiments.

At step 316, the lobe auto-focuser 160 may compare the cost functional of the lobe at the new location (evaluated at step 314) with the cost functional of the lobe at the coordinates of the new sound activity (evaluated at step 308). If the cost functional of the lobe at the new location is less than the cost functional of the lobe at the coordinates of the new sound activity at step 316, then the step size p at step 314 may be considered as too large, and the process 300 may continue to step 322. At step 322, the step size may be adjusted and the process may return to step 314.

However, if the cost functional of the lobe at the new location is not less than the cost functional of the lobe at the coordinates of the new sound activity at step 316, then the process 300 may continue to step 318. At step 318, the lobe auto-focuser 160 may determine whether the difference between (1) the cost functional of the lobe at the new location (evaluated at step 314) and (2) the cost functional of the lobe at the coordinates of the new sound activity (evaluated at step 308) is close, i.e., whether the absolute value of the difference is within a small quantity E. If the condition is not satisfied at step 318, then it may be considered that a local maximum of the cost functional has not been reached. The process 300 may proceed to step 324 and the locations of the lobes of the array microphone 100 are not updated.

However, if the condition is satisfied at step 318, then it may be considered that a local maximum of the cost functional has been reached and that the lobe has been auto focused, and the process 300 proceeds to step 320. At step 320, the lobe auto-focuser 160 may transmit the coordinates of the new sound activity to the beamformer 170 so that the beamformer 170 can update the location of the lobe to the new coordinates. In addition, the lobe auto-focuser 160 may store the new coordinates of the lobe in the database 180.

In some embodiments, annealing/dithering movements of the lobe may be applied by the lobe auto-focuser 160 at step 320. The annealing/dithering movements may be applied to nudge the lobe out of a local maximum of the cost functional to attempt to find a better local maximum (and therefore a better location for the lobe). The annealing/dithering locations may be defined by (x_i+rx_i,y_i+ry_i,z_i+rz_i), where (rx_i, ry_i, rz_i) are small random values.

The process 300 may be continuously performed by the array microphone 100 as the audio activity localizer 150 finds new sound activity and provides the coordinates and confidence score of the new sound activity to the lobe auto-focuser 160. For example, the process 300 may be performed as audio sources, e.g., human speakers, are moving around a conference room so that one or more lobes can be focused on the audio sources to optimally pick up their sound.

In embodiments, the cost functional may be re-evaluated and updated, e.g., steps 308-318 and 322, and the coordinates of the lobe may be adjusted without needing to receive a set of coordinates of new sound activity, e.g., at step 302. For example, an algorithm may detect which lobe of the array microphone 100 has the most sound activity without providing a set of coordinates of new sound activity. Based on the sound activity information from such an algorithm, the cost functional may be re-evaluated and updated.

An embodiment of a process 500 for automatic placement or deployment of beamformed lobes of the array microphone 400 is shown in FIG. 5. The process 500 may be performed by the lobe auto-placer 460 so that the array microphone 400 can output one or more audio signals 480 from the array microphone 400 shown in FIG. 4, where the audio signals 480 may include sound picked up by the placed beamformed lobes that are from new sound activity of an audio source. One or more processors and/or other processing components (e.g., analog to digital converters, encryption chips, etc.) within or external to the microphone array 400 may perform any, some, or all of the steps of the process 500. One or more other types of components (e.g., memory, input and/or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be utilized in conjunction with the processors and/or other processing components to perform any, some, or all of the steps of the process 500.

At step 502, the coordinates corresponding to new sound activity may be received at the lobe auto-placer 460 from the audio activity localizer 450. The audio activity localizer 450 may continuously scan the environment of the array microphone 400 to find new sound activity. The new sound activity found by the audio activity localizer 450 may include suitable audio sources, e.g., human speakers, that are not stationary. The coordinates of the new sound activity may be a particular three dimensional coordinate relative to the location of the array microphone 400, such as in Cartesian coordinates (i.e., x, y, z), or in spherical coordinates (i.e., radial distance/magnitude r, elevation angle θ (theta), azimuthal angle φ (phi)).

In embodiments, the placement of beamformed lobes may occur based on whether an amount of activity of the new sound activity exceeds a predetermined threshold. FIG. 19 is a schematic diagram of an array microphone 1900 that can detect sounds from audio sources at various frequencies, and automatically place beamformed lobes in response to the detection of sound activity while taking into account the amount of activity of the new sound activity. In embodiments, the array microphone 1900 may include some or all of the same components as the array microphone 400 described above, e.g., the microphones 402, the audio activity localizer 450, the lobe auto-placer 460, the beamformer 470, and/or the database 480. The array microphone 1900 may also include an activity detector 1904 in communication with the lobe auto-placer 460 and the beamformer 470.

The activity detector 1904 may detect an amount of activity in the new sound activity. In some embodiments, the amount of activity may be measured as the energy level of the new sound activity. In other embodiments, the amount of activity may be measured using methods in the time domain and/or frequency domain, such as by applying machine learning (e.g., using cepstrum coefficients), measuring signal non-stationarity in one or more frequency bands, and/or searching for features of desirable sound or speech.

In embodiments, the activity detector 1904 may be a voice activity detector (VAD) which can determine whether there is voice and/or noise present in the remote audio signal. A VAD may be implemented, for example, by analyzing the spectral variance of the remote audio signal, using linear predictive coding, applying machine learning or deep learning techniques to detect voice and/or noise, and/or using well-known techniques such as the ITU G.729 VAD, ETSI standards for VAD calculation included in the GSM specification, or long term pitch prediction.

Based on the detected amount of activity, automatic lobe placement may be performed or not performed. The automatic lobe placement may be performed when the detected activity of the new sound activity satisfies predetermined criteria. Conversely, the automatic lobe placement may not be performed when the detected activity of the new sound activity does not satisfy predetermined criteria. For example, satisfying the predetermined criteria may indicate that the new sound activity includes voice, speech, or other sound that is preferably to be picked up by a lobe. As another example, not satisfying the predetermined criteria may indicate that the new sound activity does not include voice, speech, or other sound that is preferably to be picked up by a lobe. By inhibiting automatic lobe placement in this latter scenario, a lobe will not be placed to avoid picking up sound from the new sound activity.

As seen in the process 2000 of FIG. 20, at step 2003 following step 502, it can be determined whether the amount of activity of the new sound activity satisfies the predetermined criteria. The new sound activity may be received by the activity detector 1904 from the beamformer 470, for example. The detected amount of activity may correspond to the amount of speech, voice, noise, etc. in the new sound activity. In embodiments, the amount of activity may be measured as the energy level of the new sound activity, or as the amount of voice in the new sound activity. In embodiments, the detected amount of activity may specifically indicate the amount of voice or speech in the new sound activity. In other embodiments, the detected amount of activity may be a voice-to-noise ratio, or indicate an amount of noise in the new sound activity.

If the amount of activity does not satisfy the predetermined criteria at step 2003, then the process 2000 may end at step 522 and the locations of the lobes of the array microphone 1900 are not updated. The detected amount of activity of the new sound activity may not satisfy the predetermined criteria when there is a relatively low amount of speech of voice in the new sound activity, and/or the voice-to-noise ratio is relatively low. Similarly, the detected amount of activity of the new sound activity may not satisfy the predetermined criteria when there is a relatively high amount of noise in the new sound activity. Accordingly, not automatically placing a lobe to detect the new sound activity may help to ensure that undesirable sound is not picked.

If the amount of activity satisfies the predetermined criteria at step 2003, then the process 2000 may continue to step 504 as described below. The detected amount of activity of the new sound activity may satisfy the predetermined criteria when there is a relatively high amount of speech or voice in the new sound activity, and/or the voice-to-noise ratio is relatively high. Similarly, the detected amount of activity of the new sound activity may satisfy the predetermined criteria when there is a relatively low amount of noise in the new sound activity. Accordingly, automatically placing a lobe to detect the new sound activity may be desirable in this scenario.

Returning to the process 500, at step 504, the lobe auto-placer 460 may update a timestamp, such as to the current value of a clock. The timestamp may be stored in the database 480, in some embodiments. In embodiments, the timestamp and/or the clock may be real time values, e.g., hour, minute, second, etc. In other embodiments, the timestamp and/or the clock may be based on increasing integer values that may enable tracking of the time ordering of events.

The lobe auto-placer 460 may determine at step 506 whether the coordinates of the new sound activity are nearby (i.e., in the vicinity of) an existing active lobe. Whether the new sound activity is nearby an existing lobe may be based on the difference in azimuth and/or elevation angles of (1) the coordinates of the new sound activity and (2) the coordinates of the existing lobe, relative to a predetermined threshold. The distance of the new sound activity away from the microphone 400 may also influence the determination of whether the coordinates of the new sound activity are nearby an existing lobe. The lobe auto-placer 460 may retrieve the coordinates of the existing lobe from the database 480 for use in step 506, in some embodiments. An embodiment of the determination of whether the coordinates of the new sound activity are nearby an existing lobe is described in more detail below with respect to FIG. 6.

If at step 506 the lobe auto-placer 460 determines that the coordinates of the new sound activity are nearby an existing lobe, then the process 500 continues to step 520. At step 520, the timestamp of the existing lobe is updated to the current timestamp from step 504. In this scenario, the existing lobe is considered able to cover (i.e., pick up) the new sound activity. The process 500 may end at step 522 and the locations of the lobes of the array microphone 400 are not updated.

However, if at step 506 the lobe auto-placer 460 determines that the coordinates of the new sound activity are not nearby an existing lobe, then the process 500 continues to step 508. In this scenario, the coordinates of the new sound activity may be considered to be outside the current coverage area of the array microphone 400, and therefore the new sound activity needs to be covered. At step 508, the lobe auto-placer 460 may determine whether an inactive lobe of the array microphone 400 is available. In some embodiments, a lobe may be considered inactive if the lobe is not pointed to a particular set of coordinates, or if the lobe is not deployed (i.e., does not exist). In other embodiments, a deployed lobe may be considered inactive based on whether a metric of the deployed lobe (e.g., time, age, etc.) satisfies certain criteria. If the lobe auto-placer 460 determines that there is an inactive lobe available at step 508, then the inactive lobe is selected at step 510 and the timestamp of the newly selected lobe is updated to the current timestamp (from step 504) at step 514.

However, if the lobe auto-placer 460 determines that there is not an inactive lobe available at step 508, then the process 500 may continue to step 512. At step 512, the lobe auto-placer 460 may select a currently active lobe to recycle to be pointed at the coordinates of the new sound activity. In some embodiments, the lobe selected for recycling may be an active lobe with the lowest confidence score and/or the oldest timestamp. The confidence score for a lobe may denote the certainty of the coordinates and/or the quality of the sound activity, for example. In embodiments, other suitable metrics related to the lobe may be utilized. The oldest timestamp for an active lobe may indicate that the lobe has not recently detected sound activity, and possibly that the audio source is no longer present in the lobe. The lobe selected for recycling at step 512 may have its timestamp updated to the current timestamp (from step 504) at step 514.

At step 516, a new confidence score may be assigned to the lobe, both when the lobe is a selected inactive lobe from step 510 or a selected recycled lobe from step 512. At step 518, the lobe auto-placer 460 may transmit the coordinates of the new sound activity to the beamformer 470 so that the beamformer 470 can update the location of the lobe to the new coordinates. In addition, the lobe auto-placer 460 may store the new coordinates of the lobe in the database 480.

The process 500 may be continuously performed by the array microphone 400 as the audio activity localizer 450 finds new sound activity and provides the coordinates of the new sound activity to the lobe auto-placer 460. For example, the process 500 may be performed as audio sources, e.g., human speakers, are moving around a conference room so that one or more lobes can be placed to optimally pick up the sound of the audio sources.

An embodiment of a process 600 for finding previously placed lobes near sound activity is shown in FIG. 6. The process 600 may be utilized by the lobe auto-focuser 160 at step 204 of the process 200, at step 304 of the process 300, and/or at step 806 of the process 800, and/or by the lobe auto-placer 460 at step 506 of the process 500. In particular, the process 600 may determine whether the coordinates of the new sound activity are nearby an existing lobe of an array microphone 100, 400. Whether the new sound activity is nearby an existing lobe may be based on the difference in azimuth and/or elevation angles of (1) the coordinates of the new sound activity and (2) the coordinates of the existing lobe, relative to a predetermined threshold. The distance of the new sound activity away from the array microphone 100, 400 may also influence the determination of whether the coordinates of the new sound activity are nearby an existing lobe.

At step 602, the coordinates corresponding to new sound activity may be received at the lobe auto-focuser 160 or the lobe auto-placer 460 from the audio activity localizer 150, 450, respectively. The coordinates of the new sound activity may be a particular three dimensional coordinate relative to the location of the array microphone 100, 400, such as in Cartesian coordinates (i.e., x, y, z), or in spherical coordinates (i.e., radial distance/magnitude r, elevation angle θ (theta), azimuthal angle φ (phi)). It should be noted that Cartesian coordinates may be readily converted to spherical coordinates, and vice versa, as needed.

At step 604, the lobe auto-focuser 160 or the lobe auto-placer 460 may determine whether the new sound activity is relatively far away from the array microphone 100, 400 by evaluating whether the distance of the new sound activity is greater than a determined threshold. The distance of the new sound activity may be determined by the magnitude of the vector representing the coordinates of the new sound activity. If the new sound activity is determined to be relatively far away from the array microphone 100, 400 at step 604 (i.e., greater than the threshold), then at step 606 a lower azimuth threshold may be set for later usage in the process 600. If the new sound activity is determined to not be relatively far away from the array microphone 100, 400 at step 604 (i.e., less than or equal to the threshold), then at step 608 a higher azimuth threshold may be set for later usage in the process 600.

Following the setting of the azimuth threshold at step 606 or step 608, the process 600 may continue to step 610. At step 610, the lobe auto-focuser 160 or the lobe auto-placer 460 may determine whether there are any lobes to check for their vicinity to the new sound activity. If there are no lobes of the array microphone 100, 400 to check at step 610, then the process 600 may end at step 616 and denote that there are no lobes in the vicinity of the array microphone 100, 400.

However, if there are lobes of the array microphone 100, 400 to check at step 610, then the process 600 may continue to step 612 and examine one of the existing lobes. At step 612, the lobe auto-focuser 160 or the lobe auto-placer 460 may determine whether the absolute value of the difference between (1) the azimuth of the existing lobe and (2) the azimuth of the new sound activity is greater than the azimuth threshold (that was set at step 606 or step 608). If the condition is satisfied at step 612, then it may be considered that the lobe under examination is not within the vicinity of the new sound activity. The process 600 may return to step 610 to determine whether there are further lobes to examine.

However, if the condition is not satisfied at step 612, then the process 600 may proceed to step 614. At step 614, the lobe auto-focuser 160 or the lobe auto-placer 460 may determine whether the absolute value of the difference between (1) the elevation of the existing lobe and (2) the elevation of the new sound activity is greater than a predetermined elevation threshold. If the condition is satisfied at step 614, then it may be considered that the lobe under examination is not within the vicinity of the new sound activity. The process 600 may return to step 610 to determine whether there are further lobes to examine. However, if the condition is not satisfied at step 614, then the process 600 may end at step 618 and denote that the lobe under examination is in the vicinity of the new sound activity.

FIG. 7 is an exemplary depiction of an array microphone 700 that can automatically focus previously placed beamformed lobes within associated lobe regions in response to the detection of new sound activity. In embodiments, the array microphone 700 may include some or all of the same components as the array microphone 100 described above, e.g., the audio activity localizer 150, the lobe auto-focuser 160, the beamformer 170, and/or the database 180. Each lobe of the array microphone 700 may be moveable within its associated lobe region, and a lobe may not cross the boundaries between the lobe regions. It should be noted that while FIG. 7 depicts eight lobes with eight associated lobe regions, any number of lobes and associated lobe regions is possible and contemplated, such as the four lobes with four associated lobe regions depicted in FIGS. 10, 12, 13, and 15. It should also be noted that FIGS. 7, 10, 12, 13, and 15 are depicted as two-dimensional representations of the three-dimensional space around an array microphone.

At least two sets of coordinates may be associated with each lobe of the array microphone 700: (1) original or initial coordinates LO_i(e.g., that are configured automatically or manually at the time of set up of the array microphone 700), and (2) current coordinates {right arrow over (LC_i)} where a lobe is currently pointing at a given time. The sets of coordinates may indicate the position of the center of a lobe, in some embodiments. The sets of coordinates may be stored in the database 180, in some embodiments.

In addition, each lobe of the array microphone 700 may be associated with a lobe region of three-dimensional space around it. In embodiments, a lobe region may be defined as a set of points in space that is closer to the initial coordinates LO_iof a lobe than to the coordinates of any other lobe of the array microphone. In other words, if p is defined as a point in space, then the point p may belong to a particular lobe region LR_i, if the distance D between the point p and the center of a lobe i (LO_i) is the smallest than for any other lobe, as in the following:

$p \in {LR}_{i} if fi = \underset{1 \leq i \leq N}{argmin} (D (p, L O_{i})) .$

Regions that are defined in this fashion are known as Voronoi regions or Voronoi cells. For example, it can be seen in FIG. 7 that there are eight lobes with associated lobe regions that have boundaries depicted between each of the lobe regions. The boundaries between the lobe regions are the sets of points in space that are equally distant from two or more adjacent lobes. It is also possible that some sides of a lobe region may be unbounded. In embodiments, the distance D may be the Euclidean distance between point p and LO_i, e.g., √{square root over ((x₁−x₂)²+(y₁−y₂)₂+(z₁−z₂)²)}. In some embodiments, the lobe regions may be recalculated as particular lobes are moved.

In embodiments, the lobe regions may be calculated and/or updated based on sensing the environment (e.g., objects, walls, persons, etc.) that the array microphone 700 is situated in using infrared sensors, visual sensors, and/or other suitable sensors. For example, information from a sensor may be used by the array microphone 700 to set the approximate boundaries for lobe regions, which in turn can be used to place the associated lobes. In further embodiments, the lobe regions may be calculated and/or updated based on a user defining the lobe regions, such as through a graphical user interface of the array microphone 700.

As further shown in FIG. 7, there may be various parameters associated with each lobe that can restrict its movement during the automatic focusing process, as described below. One parameter is a look radius of a lobe that is a three-dimensional region of space around the initial coordinates LO_iof the lobe where new sound activity can be considered. In other words, if new sound activity is detected in a lobe region but is outside the look radius of the lobe, then there would be no movement or automatic focusing of the lobe in response to the detection of the new sound activity. Points that are outside of the look radius of a lobe can therefore be considered as an ignore or “don't care” portion of the associated lobe region. For example, in FIG. 7, the point denoted as A is outside the look radius of lobe 5 and its associated lobe region 5, so any new sound activity at point A would not cause the lobe to be moved. Conversely, if new sound activity is detected in a particular lobe region and is inside the look radius of its lobe, then the lobe may be automatically moved and focused in response to the detection of the new sound activity.

Another parameter is a move radius of a lobe that is a maximum distance in space that the lobe is allowed to move. The move radius of a lobe is generally less than the look radius of the lobe, and may be set to prevent the lobe from moving too far away from the array microphone or too far away from the initial coordinates LO_iof the lobe. For example, in FIG. 7, the point denoted as B is both within the look radius and the move radius of lobe 5 and its associated lobe region 5. If new sound activity is detected at point B, then lobe 5 could be moved to point B. As another example, in FIG. 7, the point denoted as C is within the look radius of lobe 5 but outside the move radius of lobe 5 and its associated lobe region 5. If new sound activity is detected at point C, then the maximum distance that lobe 5 could be moved is limited to the move radius.

A further parameter is a boundary cushion of a lobe that is a maximum distance in space that the lobe is allowed to move towards a neighboring lobe region and toward the boundary between the lobe regions. For example, in FIG. 7, the point denoted as D is outside of the boundary cushion of lobe 8 and its associated lobe region 8 (that is adjacent to lobe region 7). The boundary cushions of the lobes may be set to minimize the overlap of adjacent lobes. In FIGS. 7, 10, 12, 13, and 15, the boundaries between lobe regions are denoted by a dashed line, and the boundary cushions for each lobe region are denoted by dash-dot lines that are parallel to the boundaries.

An embodiment of a process 800 for automatic focusing of previously placed beamformed lobes of the array microphone 700 within associated lobe regions is shown in FIG. 8. The process 800 may be performed by the lobe auto-focuser 160 so that the array microphone 700 can output one or more audio signals 180 from the array microphone 700, where the audio signals 180 may include sound picked up by the beamformed lobes that are focused on new sound activity of an audio source. One or more processors and/or other processing components (e.g., analog to digital converters, encryption chips, etc.) within or external to the array microphone 700 may perform any, some, or all of the steps of the process 800. One or more other types of components (e.g., memory, input and/or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be utilized in conjunction with the processors and/or other processing components to perform any, some, or all of the steps of the process 800.

Step 802 of the process 800 for the lobe auto-focuser 160 may be substantially the same as step 202 of the process 200 of FIG. 2 described above. In particular, the coordinates and a confidence score corresponding to new sound activity may be received at the lobe auto-focuser 160 from the audio activity localizer 150 at step 802. In embodiments, other suitable metrics related to the new sound activity may be received and utilized at step 802. At step 804, the lobe auto-focuser 160 may compare the confidence score of the new sound activity to a predetermined threshold to determine whether the new confidence score is satisfactory. If the lobe auto-focuser 160 determines at step 804 that the confidence score of the new sound activity is less than the predetermined threshold (i.e., that the confidence score is not satisfactory), then the process 800 may end at step 820 and the locations of the lobes of the array microphone 700 are not updated. However, if the lobe auto-focuser 160 determines at step 804 that the confidence score of the new sound activity is greater than or equal to the predetermined threshold (i.e., that the confidence score is satisfactory), then the process 800 may continue to step 806.

At step 806, the lobe auto-focuser 160 may identify the lobe region that the new sound activity is within, i.e., the lobe region which the new sound activity belongs to. In embodiments, the lobe auto-focuser 160 may find the lobe closest to the coordinates of the new sound activity in order to identify the lobe region at step 806. For example, the lobe region may be identified by finding the initial coordinates LO_iof a lobe that are closest to the new sound activity, such as by finding an index i of a lobe such that the distance between the coordinates of the new sound activity and the initial coordinates LO_iof a lobe is minimized:

$i = \underset{1 \leq i \leq N}{argmin} (D (s, L O_{i})) .$

The lobe and its associated lobe region that contain the new sound activity may be determined as the lobe and lobe region identified at step 806.

After the lobe region has been identified at step 806, the lobe auto-focuser 160 may determine whether the coordinates of the new sound activity are outside a look radius of the lobe at step 808. If the lobe auto-focuser 160 determines that the coordinates of the new sound activity are outside the look radius of the lobe at step 808, then the process 800 may end at step 820 and the locations of the lobes of the array microphone 700 are not updated. In other words, if the new sound activity is outside the look radius of the lobe, then the new sound activity can be ignored and it may be considered that the new sound activity is outside the coverage of the lobe. As an example, point A in FIG. 7 is within lobe region 5 that is associated with lobe 5, but is outside the look radius of lobe 5. Details of determining whether the coordinates of the new sound activity are outside the look radius of a lobe are described below with respect to FIGS. 9 and 10.

However, if at step 808 the lobe auto-focuser 160 determines that the coordinates of the new sound activity are not outside (i.e., are inside) the look radius of the lobe, then the process 800 may continue to step 810. In this scenario, the lobe may be moved towards the new sound activity contingent on assessing the coordinates of the new sound activity with respect to other parameters such as a move radius and a boundary cushion, as described below. At step 810, the lobe auto-focuser 160 may determine whether the coordinates of the new sound activity are outside a move radius of the lobe. If the lobe auto-focuser 160 determines that the coordinates of the new sound activity are outside the move radius of the lobe at step 810, then the process 800 may continue to step 816 where the movement of the lobe may be limited or restricted. In particular, at step 816, the new coordinates where the lobe may be provisionally moved to can be set to no more than the move radius. The new coordinates may be provisional because the movement of the lobe may still be assessed with respect to the boundary cushion parameter, as described below. In embodiments, the movement of the lobe at step 816 may be restricted based on a scaling factor α(where 0<α≤1), in order to prevent the lobe from moving too far from its initial coordinates LO_i. As an example, point C in FIG. 7 is outside the move radius of lobe 5 so the farthest distance that lobe 5 could be moved is the move radius. After step 816, the process 800 may continue to step 812. Details of limiting the movement of a lobe to within its move radius are described below with respect to FIGS. 11 and 12.

The process 800 may also continue to step 812 if at step 810 the lobe auto-focuser 160 determines that the coordinates of the new sound activity are not outside (i.e., are inside) the move radius of the lobe. As an example, point B in FIG. 7 is inside the move radius of lobe 5 so lobe 5 could be moved to point B. At step 812, the lobe auto-focuser 160 may determine whether the coordinates of the new sound activity are close to a boundary cushion and are therefore too close to an adjacent lobe. If the lobe auto-focuser 160 determines that the coordinates of the new sound activity are close to a boundary cushion at step 812, then the process 800 may continue to step 818 where the movement of the lobe may be limited or restricted. In particular, at step 818, the new coordinates where the lobe may be moved to may be set to just outside the boundary cushion. In embodiments, the movement of the lobe at step 818 may be restricted based on a scaling factor β(where 0<β≤1). As an example, point D in FIG. 7 is outside the boundary cushion between adjacent lobe region 8 and lobe region 7. The process 800 may continue to step 814 following step 818. Details regarding the boundary cushion are described below with respect to FIGS. 13-15.

The process 800 may also continue to step 814 if at step 812 the lobe auto-focuser 160 determines that the coordinates of the new sound activity are not close to a boundary cushion. At step 812, the lobe auto-focuser 160 may transmit the new coordinates of the lobe to the beamformer 170 so that the beamformer 170 can update the location of the existing lobe to the new coordinates. In embodiments, the new coordinates {right arrow over (LC_i)} of the lobe may be defined as {right arrow over (LC_i)}={right arrow over (LO_i)}+min(α,β){right arrow over (M)}={right arrow over (LO_i)}+{right arrow over (M_r)}, where {right arrow over (M)} is a motion vector and {right arrow over (M_r)} is a restricted motion vector, as described in more detail below. In embodiments, the lobe auto-focuser 160 may store the new coordinates of the lobe in the database 180.

Depending on the steps of the process 800 described above, when a lobe is moved due to the detection of new sound activity, the new coordinates of the lobe may be: (1) the coordinates of the new sound activity, if the coordinates of the new sound activity are within the look radius of the lobe, within the move radius of the lobe, and not close to the boundary cushion of the associated lobe region; (2) a point in the direction of the motion vector towards the new sound activity and limited to the range of the move radius, if the coordinates of the new sound activity are within the look radius of the lobe, outside the move radius of the lobe, and not close to the boundary cushion of the associated lobe region; or (3) just outside the boundary cushion, if the coordinates of the new sound activity are within the look radius of the lobe and close to the boundary cushion.

The process 800 may be continuously performed by the array microphone 700 as the audio activity localizer 150 finds new sound activity and provides the coordinates and confidence score of the new sound activity to the lobe auto-focuser 160. For example, the process 800 may be performed as audio sources, e.g., human speakers, are moving around a conference room so that one or more lobes can be focused on the audio sources to optimally pick up their sound.

An embodiment of a process 900 for determining whether the coordinates of new sound activity are outside the look radius of a lobe is shown in FIG. 9. The process 900 may be utilized by the lobe auto-focuser 160 at step 808 of the process 800, for example. In particular, the process 900 may begin at step 902 where a motion vector {right arrow over (M)} may be computed as {right arrow over (M)}={right arrow over (s)}−{right arrow over (LO_i)}. The motion vector may be the vector connecting the center of the original coordinates LO_iof the lobe to the coordinates {right arrow over (s)} of the new sound activity. For example, as shown in FIG. 10, new sound activity S is present in lobe region 3 and the motion vector {right arrow over (M)} is shown between the original coordinates LO₃of lobe 3 and the coordinates of the new sound activity S. The look radius for lobe 3 is also depicted in FIG. 10.

After computing the motion vector M at step 902, the process 900 may continue to step 904. At step 904, the lobe auto-focuser 160 may determine whether the magnitude of the motion vector is greater than the look radius for the lobe, as in the following: |{right arrow over (M)}|=√{square root over ((m_x)²+(m_y)²)}+(m_z)²>(LookRadius)_i. If the magnitude of the motion vector {right arrow over (M)} is greater than the look radius for the lobe at step 904, then at step 906, the coordinates of the new sound activity may be denoted as outside the look radius for the lobe. For example, as shown in FIG. 10, because the new sound activity S is outside the look radius of lobe 3, the new sound activity S would be ignored. However, if the magnitude of the motion vector {right arrow over (M)} is less than or equal to the look radius for the lobe at step 904, then at step 908, the coordinates of the new sound activity may be denoted as inside the look radius for the lobe.

An embodiment of a process 1100 for limiting the movement of a lobe to within its move radius is shown in FIG. 11. The process 1100 may be utilized by the lobe auto-focuser 160 at step 816 of the process 800, for example. In particular, the process 1100 may begin at step 1102 where a motion vector {right arrow over (M)} may be computed as {right arrow over (M)}={right arrow over (s)}−{right arrow over (LO_i)}, similar to as described above with respect to step 902 of the process 900 shown in FIG. 9. For example, as shown in FIG. 12, new sound activity S is present in lobe region 3 and the motion vector {right arrow over (M)} is shown between the original coordinates LO₃of lobe 3 and the coordinates of the new sound activity S. The move radius for lobe 3 is also depicted in FIG. 12.

After computing the motion vector {right arrow over (M)} at step 1102, the process 1100 may continue to step 1104. At step 1104, the lobe auto-focuser 160 may determine whether the magnitude of the motion vector {right arrow over (M)} is less than or equal to the move radius for the lobe, as in the following: |{right arrow over (M)}|≤(MoveRadius)_i. If the magnitude of the motion vector {right arrow over (M)} is less than or equal to the move radius at step 1104, then at step 1106, the new coordinates of the lobe may be provisionally moved to the coordinates of the new sound activity. For example, as shown in FIG. 12, because the new sound activity S is inside the move radius of lobe 3, the lobe would provisionally be moved to the coordinates of the new sound activity S.

However, if the magnitude of the motion vector {right arrow over (M)} is greater than the move radius at step 1104, then at step 1108, the magnitude of the motion vector {right arrow over (M)} may be scaled by a scaling factor α to the maximum value of the move radius while keeping the same direction, as in the following:

$\vec{M} = \frac{{(M o v e R a d i u s)}_{i}}{\langle \vec{M} \rangle} \vec{M} = α \vec{M},$

where the scaling factor α may be defined as:

$α = {\begin{matrix} \frac{{(M o v e R a d i u s)}_{i}}{\langle \vec{M} \rangle}, & \langle \vec{M} \rangle > {(M o v e R a d i u s)}_{i} \\ 1, & \langle \vec{M} \rangle \leq {(M o v e R a d i u s)}_{i} \end{matrix} .$

FIGS. 13-15 relate to the boundary cushion of a lobe region, which is the portion of the space next to the boundary or edge of the lobe region that is adjacent to another lobe region. In particular, the boundary cushion next to the boundary between two lobes i and j may be described indirectly using a vector {right arrow over (D_ij)} that connects the original coordinates of the two lobes (i.e., LO_iand LO_j). Accordingly, such a vector can be described as: {right arrow over (D_ij)}={right arrow over (LO_j)}−{right arrow over (LO_i)}. The midpoint of this vector {right arrow over (D_ij)} may be a point that is at the boundary between the two lobe regions. In particular, moving from the original coordinates LO_iof lobe i in the direction of the vector {right arrow over (D_ij)} is the shortest path towards the adjacent lobe j. Furthermore, moving from the original coordinates LO_iof lobe i in the direction of the vector {right arrow over (D_ij)} but keeping the amount of movement to half of the magnitude of the vector {right arrow over (D_ij)} will be the exact boundary between the two lobe regions.

Based on the above, moving from the original coordinates LO_iof lobe i in the direction of the vector {right arrow over (D_ij)} but restricting the amount of movement based on a value A (where 0<A<1)

$(i . e ., A \frac{\langle \vec{D_{ij}} \rangle}{2})$

will be within (100*A) % of the boundary between the lobe regions. For example, if A is 0.8 (i.e., 80%), then the new coordinates of a moved lobe would be within 80% of the boundary between lobe regions. Therefore, the value A can be utilized to create the boundary cushion between two adjacent lobe regions. In general, a larger boundary cushion can prevent a lobe from moving into another lobe region, while a smaller boundary cushion can allow a lobe to move closer to another lobe region.

In addition, it should be noted that if a lobe i is moved in a direction towards a lobe j due to the detection of new sound activity (e.g., in the direction of a motion vector {right arrow over (M)} as described above), there is a component of movement in the direction of the lobe j, i.e., in the direction of the vector {right arrow over (D_ij)}. In order to find the component of movement in the direction of the vector {right arrow over (D_ij)}, the motion vector {right arrow over (M)} can be projected onto the unit vector {right arrow over (Du_ij)}={right arrow over (D_ij)}/|{right arrow over (D_ij)}| (which has the same direction as the vector {right arrow over (D_ij)} with unity magnitude) to compute a projected vector {right arrow over (PM_ij)}. As an example, FIG. 13 shows a vector {right arrow over (D_ij)} that connects lobes 3 and 2, which is also the shortest path from the center of lobe 3 towards lobe region 2. The projected vector {right arrow over (PM₃₂)} shown in FIG. 13 is the projection of the motion vector {right arrow over (M)} onto the unit vector {right arrow over (D₃₂)}/{right arrow over (|D₂₃|)}.

An embodiment of a process 1400 for creating a boundary cushion of a lobe region using vector projections is shown in FIG. 14. The process 1400 may be utilized by the lobe auto-focuser 160 at step 818 of the process 800, for example. The process 1400 may result in restricting the magnitude of a motion vector M such that a lobe is not moved in the direction of any other lobe region by more than a certain percentage that characterizes the size of the boundary cushion.

Prior to performing the process 1400, a vector {right arrow over (D_ij)} and unit vectors {right arrow over (Du_ij)}={right arrow over (D_ij)}/{right arrow over (|D_ij|)} can be computed for all pairs of active lobes. As described previously, the vectors {right arrow over (D_ij)} may connect the original coordinates of lobes i and j. The parameter A_i(where 0<A<1) may be determined for all active lobes, which characterizes the size of the boundary cushion for each lobe region. As described previously, prior to the process 1400 being performed (i.e., prior to step 818 of the process 800), the lobe region of new sound activity may be identified (i.e., at step 806) and a motion vector may be computed (i.e., using the process 1100/step 810).

At step 1402 of the process 1400, the projected vector {right arrow over (PM_ij)} may be computed for all lobes that are not associated with the lobe region identified for the new sound activity. The magnitude of a projected vector {right arrow over (PM_ij)} (as described above with respect to FIG. 13) can determine the amount of movement of a lobe in the direction of a boundary between lobe regions. Such a magnitude of the projected vector {right arrow over (PM_ij)} can be computed as a scalar, such as by a dot product of the motion vector {right arrow over (M)} and the unit vector {right arrow over (Du_ij)}={right arrow over (D_ij)}/{right arrow over (|D_ij|)}, such that projection PM_ij=M_xDu_ij,z+M_yDu_ij,y+M_zDu_ij,z.

When PM_ij<0, the motion vector {right arrow over (M)} has a component in the opposite direction of the vector {right arrow over (D_ij)}. This means that movement of a lobe i would be in the direction opposite of the boundary with a lobe j. In this scenario, the boundary cushion between lobes i and j is not a concern because the movement of the lobe i would be away from the boundary with lobe j. However, when PM_ij>0, the motion vector M has a component in the same direction as the direction of the vector {right arrow over (D_ij)}. This means that movement of a lobe i would be in the same direction as the boundary with lobe j. In this scenario, movement of the lobe i can be limited to outside the boundary cushion so that

$P M_{rij} < A_{i} \frac{\langle \vec{D_{ij}} \rangle}{2},$

where A_i(with 0<A_i<1) is a parameter that characterizes the boundary cushion for a lobe region associated with lobe i.

A scaling factor β may be utilized to ensure that

$P M_{rij} < A_{i} \frac{\langle \vec{D_{ij}} \rangle}{2} .$

The scaling factor β may be used to scale the motion vector {right arrow over (M)} and be defined as

$β_{j} = {\begin{matrix} \frac{A_{i} \frac{\langle \vec{D_{ij}} \rangle}{2}}{P M_{ij}}, & P M_{ij} > A_{i} \frac{\langle \vec{D_{ij}} \rangle}{2} \\ 1, & P M_{ij} \leq A_{i} \frac{\langle \vec{D_{ij}} \rangle}{2} \end{matrix} .$

Accordingly, if new sound activity is detected that is outside the boundary cushion of a lobe region, then the scaling factor β may be equal to 1, which indicates that there is no scaling of the motion vector {right arrow over (M)}. At step 1404, the scaling factor β may be computed for all the lobes that are not associated with the lobe region identified for the new sound activity.

At step 1406, the minimum scaling factor β can be determined that corresponds to the boundary cushion of the nearest lobe regions, as in the following:

$β = \min_{j} β_{j} .$

After the minimum scaling factor β has been determined at step 1406, then at step 1408, the minimum scaling factor β may be applied to the motion vector {right arrow over (M)} to determine a restricted motion vector {right arrow over (M_r)}=min(α,β){right arrow over (M)}.

For example, FIG. 15 shows new sound activity S that is present in lobe region 3 as well as a motion vector {right arrow over (M)} between the initial coordinates LO₃of lobe 3 and the coordinates of the new sound activity S. Vectors {right arrow over (D₃₁)}, {right arrow over (D₃₂)}, {right arrow over (D₃₄)} and projected vectors {right arrow over (PM₃₁)}, {right arrow over (PM₃₂)}, {right arrow over (PM₃₄)} are depicted between lobe 3 and each of the other lobes that are not associated with lobe region 3 (i.e., lobes 1, 2, and 4). In particular, vectors {right arrow over (D₃₁)}, {right arrow over (D₃₂)}, {right arrow over (D₃₄)} may be computed for all pairs of active lobes (i.e., lobes 1, 2, 3, and 4), and projections PM₃₁, PM₃₂, PM₃₄are computed for all lobes that are not associated with lobe region 3 (that is identified for the new sound activity S). The magnitude of the projected vectors may be utilized to compute scaling factors β, and the minimum scaling factor β may be used to scale the motion vector {right arrow over (M)}. The motion vector may therefore be restricted to outside the boundary cushion of lobe region 3 because the new sound activity S is too close to the boundary between lobe 3 and lobe 2. Based on the restricted motion vector, the coordinates of lobe 3 may be moved to a coordinate S_rthat is outside the boundary cushion of lobe region 3.

The projected vector {right arrow over (PM₃₄)} depicted in FIG. 15 is negative and the corresponding scaling factor β₄(for lobe 4) is equal to 1. The scaling factor β₁(for lobe 1) is also equal to 1 because

$P M_{3 1} < A_{3} \frac{\vec{\langle D_{3 1} \rangle}}{2},$

while the scaling factor β₂(for lobe 2) is less than 1 because the new sound activity S is inside the boundary cushion between lobe region 2 and lobe region 3

$(i . e ., {PM}_{3 2} > A_{3} \vec{\frac{\langle D_{3 2} \rangle}{2}}) .$

Accordingly, the minimum scaling factor β₂may be utilized to ensure that lobe 3 moves to the coordinate S_r.

FIGS. 16 and 17 are schematic diagrams of array microphones 1600, 1700 that can detect sounds from audio sources at various frequencies. The array microphone 1600 of FIG. 16 can automatically focus beamformed lobes in response to the detection of sound activity, while enabling inhibition of the automatic focus of the beamformed lobes when the activity of a remote audio signal from a far end exceeds a predetermined threshold. In embodiments, the array microphone 1600 may include some or all of the same components as the array microphone 100 described above, e.g., the microphones 102, the audio activity localizer 150, the lobe auto-focuser 160, the beamformer 170, and/or the database 180. The array microphone 1600 may also include a transducer 1602, e.g., a loudspeaker, and an activity detector 1604 in communication with the lobe auto-focuser 160. The remote audio signal from the far end may be in communication with the transducer 1602 and the activity detector 1604.

The array microphone 1700 of FIG. 17 can automatically place beamformed lobes in response to the detection of sound activity, while enabling inhibition of the automatic placement of the beamformed lobes when the activity of a remote audio signal from a far end exceeds a predetermined threshold. In embodiments, the array microphone 1700 may include some or all of the same components as the array microphone 400 described above, e.g., the microphones 402, the audio activity localizer 450, the lobe auto-placer 460, the beamformer 470, and/or the database 480. The array microphone 1700 may also include a transducer 1702, e.g., a loudspeaker, and an activity detector 1704 in communication with the lobe auto-placer 460. The remote audio signal from the far end may be in communication with the transducer 1702 and the activity detector 1704.

The transducer 1602, 1702 may be utilized to play the sound of the remote audio signal in the local environment where the array microphone 1600, 1700 is located. The activity detector 1604, 1704 may detect an amount of activity in the remote audio signal. In some embodiments, the amount of activity may be measured as the energy level of the remote audio signal. In other embodiments, the amount of activity may be measured using methods in the time domain and/or frequency domain, such as by applying machine learning (e.g., using cepstrum coefficients), measuring signal non-stationarity in one or more frequency bands, and/or searching for features of desirable sound or speech.

In embodiments, the activity detector 1604, 1704 may be a voice activity detector (VAD) which can determine whether there is voice present in the remote audio signal. A VAD may be implemented, for example, by analyzing the spectral variance of the remote audio signal, using linear predictive coding, applying machine learning or deep learning techniques to detect voice, and/or using well-known techniques such as the ITU G.729 VAD, ETSI standards for VAD calculation included in the GSM specification, or long term pitch prediction.

Based on the detected amount of activity, automatic lobe adjustment may be performed or inhibited. Automatic lobe adjustment may include, for example, auto focusing of lobes, auto focusing of lobes within regions, and/or auto placement of lobes, as described herein. The automatic lobe adjustment may be performed when the detected activity of the remote audio signal does not exceed a predetermined threshold. Conversely, the automatic lobe adjustment may be inhibited (i.e., not be performed) when the detected activity of the remote audio signal exceeds the predetermined threshold. For example, exceeding the predetermined threshold may indicate that the remote audio signal includes voice, speech, or other sound that is preferably not to be picked up by a lobe. By inhibiting automatic lobe adjustment in this scenario, a lobe will not be focused or placed to avoid picking up sound from the remote audio signal.

In some embodiments, the activity detector 1604, 1704 may determine whether the detected amount of activity of the remote audio signal exceeds the predetermined threshold. When the detected amount of activity does not exceed the predetermined threshold, the activity detector 1604, 1704 may transmit an enable signal to the lobe auto-focuser 160 or the lobe auto-placer 460, respectively, to allow lobes to be adjusted. In addition to or alternatively, when the detected amount of activity of the remote audio signal exceeds the predetermined threshold, the activity detector 1604, 1704 may transmit a pause signal to the lobe auto-focuser 160 or the lobe auto-placer 460, respectively, to stop lobes from being adjusted.

In other embodiments, the activity detector 1604, 1704 may transmit the detected amount of activity of the remote audio signal to the lobe auto-focuser 160 or to the lobe auto-placer 460, respectively. The lobe auto-focuser 160 or the lobe auto-placer 460 may determine whether the detected amount of activity exceeds the predetermined threshold. Based on whether the detected amount of activity exceeds the predetermined threshold, the lobe auto-focuser 160 or lobe auto-placer 460 may execute or pause the adjustment of lobes.

The various components included in the array microphone 1600, 1700 may be implemented using software executable by one or more servers or computers, such as a computing device with a processor and memory, graphics processing units (GPUs), and/or by hardware (e.g., discrete logic circuits, application specific integrated circuits (ASIC), programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

An embodiment of a process 1800 for inhibiting automatic adjustment of beamformed lobes of an array microphone based on a remote far end audio signal is shown in FIG. 18. The process 1800 may be performed by the array microphones 1600, 1700 so that the automatic focus or the automatic placement of beamformed lobes can be performed or inhibited based on the amount of activity of a remote audio signal from a far end. One or more processors and/or other processing components (e.g., analog to digital converters, encryption chips, etc.) within or external to the array microphones 1600, 1700 may perform any, some, or all of the steps of the process 1800. One or more other types of components (e.g., memory, input and/or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be utilized in conjunction with the processors and/or other processing components to perform any, some, or all of the steps of the process 1800.

At step 1802, a remote audio signal may be received at the array microphone 1600, 1700. The remote audio signal may be from a far end (e.g., a remote location), and may include sound from the far end (e.g., speech, voice, noise, etc.). The remote audio signal may be output on a transducer 1602, 1702 at step 1804, such as a loudspeaker in the local environment. Accordingly, the sound from the far end may be played in the local environment, such as during a conference call so that the local participants can hear the remote participants.

The remote audio signal may be received by an activity detector 1604, 1704, which may detect an amount of activity of the remote audio signal at step 1806. The detected amount of activity may correspond to the amount of speech, voice, noise, etc. in the remote audio signal. In embodiments, the amount of activity may be measured as the energy level of the remote audio signal. At step 1808, if the detected amount of activity of the remote audio signal does not exceed a predetermined threshold, then the process 1800 may continue to step 1810. The detected amount of activity of the remote audio signal not exceeding the predetermined threshold may indicate that there is a relatively low amount of speech, voice, noise, etc. in the remote audio signal. In embodiments, the detected amount of activity may specifically indicate the amount of voice or speech in the remote audio signal. At step 1810, lobe adjustments may be performed. Step 1810 may include, for example, the processes 200 and 300 for automatic focusing of beamformed lobes, the process 400 for automatic placement of beamformed lobes, and/or the process 800 for automatic focusing of beamformed lobes within lobe regions, as described herein. Lobe adjustments may be performed in this scenario because even though lobes may be focused or placed, there is a lower likelihood that such a lobe will pick up undesirable sound from the remote audio signal that is being output in the local environment. After step 1810, the process 1800 may return to step 1802.

However, if at step 1808 the detected amount of activity of the remote audio signal exceeds the predetermined threshold, then the process 1800 may continue to step 1812. At step 1812, no lobe adjustment may be performed, i.e., lobe adjustment may be inhibited. The detected amount of activity of the remote audio signal exceeding the predetermined threshold may indicate that there is a relatively high amount of speech, voice, noise, etc. in the remote audio signal. Inhibiting lobe adjustments from occurring in this scenario may help to ensure that a lobe is not focused or placed to pick up sound from the remote audio signal that is being output in the local environment. In some embodiments, the process 1800 may return to step 1802 after step 1812. In other embodiments, the process 1800 may wait for a certain time duration at step 1812 before returning to step 1802. Waiting for a certain time duration may allow reverberations in the local environment (e.g., caused by playing the sound of the remote audio signal) to dissipate.

The process 1800 may be continuously performed by the array microphones 1600, 1700 as the remote audio signal from the far end is received. For example, the remote audio signal may include a low amount of activity (e.g., no speech or voice) that does not exceed the predetermined threshold. In this situation, lobe adjustments may be performed. As another example, the remote audio signal may include a high amount of activity (e.g., speech or voice) that exceeds the predetermined threshold. In this situation, the performance of lobe adjustments may be inhibited. Whether lobe adjustments are performed or inhibited may therefore change as the amount of activity of the remote audio signal changes. The process 1800 may result in more optimal pick up of sound in the local environment by reducing the likelihood that sound from the far end is undesirably picked up.

Any process descriptions or blocks in figures should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the embodiments of the invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those having ordinary skill in the art.

This disclosure is intended to explain how to fashion and use various embodiments in accordance with the technology rather than to limit the true, intended, and fair scope and spirit thereof. The foregoing description is not intended to be exhaustive or to be limited to the precise forms disclosed. Modifications or variations are possible in light of the above teachings. The embodiment(s) were chosen and described to provide the best illustration of the principle of the described technology and its practical application, and to enable one of ordinary skill in the art to utilize the technology in various embodiments and with various modifications as are suited to the particular use contemplated. All such modifications and variations are within the scope of the embodiments as determined by the appended claims, as may be amended during the pendency of this application for patent, and all equivalents thereof, when interpreted in accordance with the breadth to which they are fairly, legally and equitably entitled.

Claims

1. A method, comprising: deploying a plurality of lobes from an array microphone in an environment;selecting one of the plurality of lobes to move, based on location data of sound activity in the environment;determining whether a metric associated with the sound activity is greater than or equal to a metric associated with the selected lobe; andrelocating the selected lobe based on the location data of the sound activity, when it is determined that the metric associated with the sound activity is greater than or equal to the metric associated with the selected lobe.
2. The method of claim 1, wherein the location data of the sound activity comprises coordinates of the sound activity in the environment.
3. The method of claim 2, wherein selecting the one of the plurality of lobes is based on a proximity of the coordinates of the sound activity to the selected lobe.
4. The method of claim 1, wherein the metric associated with the sound activity comprises a confidence score denoting one or more of a certainty of the location data of the sound activity, or a quality of the sound activity.
5. The method of claim 1, further comprising storing the metric associated with the sound activity in a database as the metric associated with the selected lobe, when it is determined that the metric associated with the sound activity is greater than or equal to the metric associated with the selected lobe.
6. The method of claim 5, wherein determining whether the metric associated with the sound activity is greater than or equal to the metric associated with the selected lobe comprises: retrieving the metric associated with the selected lobe from the database; andcomparing the metric associated with the sound activity with the retrieved metric associated with the selected lobe.
7. The method of claim 2, wherein selecting the one of the plurality of lobes to move is based on one or more of: (1) a difference in an azimuth of the coordinates of the sound activity and an azimuth of the selected lobe, relative to an azimuth threshold, or (2) a difference in an elevation angle of the coordinates of the sound activity and an elevation angle of the selected lobe, relative to an elevation angle threshold.
8. The method of claim 7, wherein selecting the one of the plurality of lobes to move is based on a distance of the coordinates of the sound activity from the array microphone.
9. The method of claim 8, further comprising setting the azimuth threshold based on the distance of the coordinates of the sound activity from the array microphone.
10. The method of claim 7, wherein selecting the one of the plurality of lobes to move comprises selecting the selected lobe when (1) an absolute value of the difference in the azimuth of the coordinates of the sound activity and the azimuth of the selected lobe is not greater than the azimuth threshold; and (2) an absolute value of the difference in the elevation angle of the coordinates of the sound activity and the elevation angle of the selected lobe is greater than the elevation angle threshold.
11. The method of claim 1, further comprising storing the location data of the sound activity in a database as a new location of the selected lobe, when it is determined that the metric associated with the sound activity is greater than or equal to the metric associated with the selected lobe.
12. The method of claim 1: further comprising determining a time duration since a last move of the selected lobe;wherein when the metric associated with the sound activity is greater than or equal to the metric associated with the selected lobe and the time duration exceeds a time threshold, relocating the selected lobe based on the location data of the sound activity.
13. The method of claim 1, further comprising: evaluating and maximizing a cost functional associated with coordinates of the sound activity; andwhen the cost functional associated with the coordinates of the sound activity has been maximized, relocating the selected lobe based on adjusted location data, wherein the adjusted location data comprises the location data of the sound activity that is adjusted based on the evaluation and maximization of the cost functional associated with the coordinates of the sound activity.
14. The method of claim 13, wherein the cost functional is evaluated and maximized based on one or more of the coordinates of the sound activity, a signal to noise ratio associated with the selected lobe, a gain value associated with the selected lobe, voice activity detection information associated with the sound activity, or a distance between the selected lobe and the location data of the sound activity.
15. The method of claim 13, wherein the adjusted location data is adjusted in a direction of a gradient of the cost functional.
16. The method of claim 13, wherein evaluating and maximizing the cost functional comprises: (A) moving the selected lobe based on the location data of the sound activity;(B) evaluating the cost functional of the moved selected lobe;(C) moving the selected lobe by a predetermined amount in each three dimensional direction;(D) after each movement of the selected lobe at step (C), evaluating the cost functional of the selected lobe at each of the moved locations;(E) calculating a gradient of the cost functional based on estimates of partial derivatives that are calculated based on the evaluated cost functionals at the location data of the sound activity and each of the moved locations of step (C);(F) moving the selected lobe by a predetermined step size in a direction of the gradient;(G) evaluating the cost functional of the selected lobe at the moved location of step (F);(H) adjusting the predetermined step size when the cost functional of step (G) is less than the cost functional of step (B), and repeating step (F); and(I) when an absolute value of a difference between the cost functional of step (G) and the cost functional of step (B) is less than a predetermined amount, denoting the moved location of step (F) as the adjusted location data.
17. The method of claim 16, wherein evaluating and maximizing the cost functional further comprises: when the absolute value of the difference between the cost functional of step (G) and the cost functional of step (B) is less than the predetermined amount: dithering the selected lobe at the moved location of step (F) by random amounts; andevaluating the cost functional of the selected lobe at the dithered moved coordinates.
18. The method of claim 1, further comprising: determining limited location data for movement of the selected lobe, based on the location data of the sound activity and a parameter associated with the selected lobe; andrelocating the selected lobe based on the limited location data.
19. The method of claim 18: wherein each of the plurality of lobes is associated with one of a plurality of lobe regions;the method further comprising identifying the lobe region including the sound activity, based on the location data of the sound activity in the environment, wherein the identified lobe region is associated with the selected lobe;wherein the parameter is further associated with the identified lobe region.
20. The method of claim 18, wherein the parameter comprises a look radius around the selected lobe, the look radius comprising a space around the selected lobe where the sound activity can be considered; andwherein determining whether the selected lobe is near the sound activity comprises determining whether the sound activity is within the look radius, based on the location data of the sound activity.
21. The method of claim 18, wherein the parameter comprises a move radius, the move radius comprising a maximum distance from the selected lobe that the selected lobe is permitted to move; andwherein the limited location data comprises: the location data of the sound activity, when the location data of the sound activity denotes that the sound activity is within the move radius; orthe move radius, when the location data of the sound activity denotes that the sound activity is outside of the move radius.
22. The method of claim 18, wherein the parameter comprises a boundary cushion in the lobe region, the boundary cushion comprising a maximum distance from the selected lobe that the selected lobe is permitted to move towards a boundary of a neighboring lobe region; andwherein the limited location data comprises: the location data of the sound activity, when the location data of the sound activity denotes that the sound activity is outside of the boundary cushion; ora location outside of the boundary cushion, when the location data of the sound activity denotes that the sound activity is within the boundary cushion.
23. The method of claim 1, further comprising: receiving a remote audio signal from a far end;detecting an amount of activity of the remote audio signal; andwhen the amount of activity of the remote audio signal exceeds a predetermined threshold, inhibiting performance of the steps of selecting the one of the plurality of lobes to move and relocating the selected lobe.
24. An array microphone system, comprising: a plurality of microphone elements, each of the plurality of microphone elements configured to detect sound and output an audio signal;a beamformer in communication with the plurality of microphone elements, the beamformer configured to generate one or more beamformed signals based on the audio signals of the plurality of microphone elements, wherein the one or more beamformed signals correspond with one or more lobes each positioned at a location in an environment;an audio activity localizer in communication with the plurality of microphone elements, the audio activity localizer configured to determine (1) coordinates of new sound activity in the environment and (2) a metric associated with the new sound activity; anda lobe auto-focuser in communication with the audio activity localizer and the beamformer, the lobe auto-focuser configured to: receive the coordinates of the new sound activity and the metric associated with the new sound activity;determine whether the coordinates of the new sound activity are near an existing lobe, wherein the existing lobe comprises one of the one or more lobes;when the coordinates of the new sound activity are determined to be near the existing lobe, determine whether the metric associated with the new sound activity is greater than or equal to a metric associated with the existing lobe; andwhen it is determined that the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe, transmit the coordinates of the new sound activity to the beamformer to cause the beamformer to update the location of the existing lobe to the coordinates of the new sound activity.
25. The system of claim 24: wherein the metric associated with the new sound activity comprises a confidence score associated with the new sound activity; andwherein the lobe auto-focuser is configured to determine whether the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe by determining whether the confidence score associated with the new sound activity is greater than or equal to a confidence score associated with the existing lobe.
26. The system of claim 25, wherein the confidence score associated with the new sound activity denotes one or more of a certainty of the coordinates of the new sound activity or a quality of the new sound activity.
27. The system of claim 25, further comprising a database in communication with the lobe auto-focuser, wherein the lobe auto-focuser is further configured to store the confidence score associated with the new sound activity in the database as a new confidence score associated with the existing lobe, when it is determined that the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe.
28. The system of claim 27, wherein the lobe auto-focuser is configured to determine whether the confidence score associated with the new sound activity is greater than or equal to the confidence score associated with the existing lobe by: retrieving the confidence score associated with the existing lobe from the database; andcomparing the confidence score associated with the new sound activity with the retrieved confidence score associated with the existing lobe.
29. The system of claim 24, wherein the lobe auto-focuser is configured to determine whether the coordinates of the new sound activity are near the existing lobe, based on one or more of: (1) a difference in an azimuth of the coordinates of the new sound activity and an azimuth of the location of the existing lobe, relative to an azimuth threshold, or (2) a difference in an elevation angle of the coordinates of the new sound activity and an elevation angle of the location of the existing lobe, relative to an elevation angle threshold.
30. The system of claim 29, wherein the lobe auto-focuser is configured to determine whether the coordinates of the new sound activity are near the existing lobe, based on a distance of the coordinates of the new sound activity from the system.
31. The system of claim 30, wherein the lobe auto-focuser is further configured to set the azimuth threshold based on the distance of the coordinates of the new sound activity from the system.
32. The system of claim 29, wherein the lobe auto-focuser is configured to determine that the coordinates of the new sound activity are near the existing lobe when (1) an absolute value of the difference in the azimuth of the coordinates of the new sound activity and the azimuth of the location of the existing lobe is not greater than the azimuth threshold; and (2) an absolute value of the difference in the elevation angle of the coordinates of the new sound activity and the elevation angle of the location of the existing lobe is greater than the elevation angle threshold.
33. The system of claim 24, further comprising a database in communication with the lobe auto-focuser, wherein the lobe auto-focuser is further configured to store the coordinates of the new sound activity in the database as a new location of the existing lobe, when it is determined that the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe.
34. The system of claim 24, wherein the lobe auto-focuser is further configured to: when the coordinates of the new sound activity are determined to be near the existing lobe, determine whether the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe, and based on a time duration since a last move of the existing lobe; andwhen it is determined that the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe and the time duration since the last move of the existing lobe exceeds a time threshold, transmit the coordinates of the new sound activity to the beamformer to cause the beamformer to update the location of the existing lobe to the coordinates of the new sound activity.
35. The system of claim 24, wherein the lobe auto-focuser is further configured to: when it is determined that the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe, evaluating and maximizing a cost functional associated with the coordinates of the new sound activity; andwhen the cost functional associated with the coordinates of the new sound activity has been maximized, transmit adjusted coordinates of the new sound activity to the beamformer to cause the beamformer to update the location of the existing lobe to the adjusted coordinates;wherein the adjusted coordinates comprise the coordinates of the new sound activity that are adjusted based on the evaluation and maximization of the cost functional associated with the coordinates of the new sound activity.
36. The system of claim 35, wherein the cost functional is evaluated and maximized based on one or more of the coordinates of the new sound activity, a signal to noise ratio associated with the existing lobe, a gain value associated with the existing lobe, voice activity detection information associated with the new sound activity, or a distance between the location of the existing lobe and the coordinates of the new sound activity.
37. The system of claim 24, wherein the lobe auto-focuser is further configured to: when it is determined that the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe: determine a lobe region that the coordinates of the new sound activity is within;determine whether the coordinates of the new sound activity are near the existing lobe, based on the coordinates of the new sound activity and a parameter associated with the existing lobe and the lobe region; andwhen it is determined that the coordinates of the new sound activity are near the existing lobe: restrict the update of the location of the existing lobe to limited coordinates within the lobe region around the existing lobe, wherein the limited coordinates are based on the coordinates of the new sound activity and the parameter associated with the existing lobe and the lobe region; andtransmit the limited coordinates to the beamformer to cause the beamformer to update the location of the existing lobe to the limited coordinates.
38. The system of claim 37: wherein the parameter comprises a look radius in the lobe region around the existing lobe, the look radius comprising a space around the location of the existing lobe where the new sound activity can be considered; andwherein the lobe auto-focuser is further configured to determine whether the coordinates of the new sound activity are near the existing lobe by determining whether the coordinates of the new sound activity are within the look radius.
39. The system of claim 37: wherein the parameter comprises a move radius in the lobe region, the move radius comprising a maximum distance from the location of the existing lobe that the existing lobe is permitted to move; andwherein the limited coordinates comprise: the coordinates of the new sound activity, when the coordinates of the new sound activity are within the move radius; orthe move radius, when the coordinates of the new sound activity are outside the move radius.
40. The system of claim 37: wherein the parameter comprises a boundary cushion in the lobe region, the boundary cushion comprising a maximum distance from the location of the existing lobe that the existing lobe is permitted to move towards a boundary of a neighboring lobe region; andwherein the limited coordinates comprise: the coordinates of the new sound activity, when the coordinates of the new sound activity are outside of the boundary cushion; ora location outside of the boundary cushion, when the coordinates of the new sound activity are within the boundary cushion.
41. The system of claim 24: further comprising an activity detector in communication with a far end and the lobe auto-focuser, the activity detector configured to: receive a remote audio signal from the far end;detect an amount of activity of the remote audio signal; andtransmit the detected amount of activity to the lobe auto-focuser; andwherein the lobe auto-focuser is further configured to: when the amount of activity of the remote audio signal exceeds a predetermined threshold, inhibit the lobe auto-focuser from performing the steps of determining whether the coordinates of the new sound activity are near the existing lobe, determining whether the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe, and transmitting the coordinates of the new sound activity to the beamformer.
42. The system of claim 24: further comprising an activity detector in communication with a far end and the lobe auto-focuser, the activity detector configured to: receive a remote audio signal from the far end;detect an amount of activity of the remote audio signal; andwhen the amount of activity of the remote audio signal exceeds a predetermined threshold, transmit a signal to the lobe auto-focuser to cause the lobe auto-focuser to stop performing the steps of determining whether the coordinates of the new sound activity are near the existing lobe, determining whether the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe, and transmitting the coordinates of the new sound activity to the beamformer.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Patent Application No. 62/821,800, filed Mar. 21, 2019, U.S. Provisional Patent Application No. 62/855,187, filed May 31, 2019, and U.S. Provisional Patent Application No. 62/971,648, filed Feb. 7, 2020. The contents of each application are fully incorporated by reference in their entirety herein.

US Referenced Citations (981)

Number	Name	Date	Kind
1535408	Fricke	Apr 1925	A
1540788	Mcclure	Jun 1925	A
1965830	Hammer	Jul 1934	A
2075588	Meyers	Mar 1937	A
2113219	Olson	Apr 1938	A
2164655	Kleerup	Jul 1939	A
D122771	Doner	Oct 1940	S
2233412	Hill	Mar 1941	A
2268529	Stiles	Dec 1941	A
2343037	Adelman	Feb 1944	A
2377449	Prevette	Jun 1945	A
2481250	Schneider	Sep 1949	A
2521603	Prew	Sep 1950	A
2533565	Eichelman	Dec 1950	A
2539671	Olson	Jan 1951	A
2777232	Kulicke	Jan 1957	A
2828508	Labarre	Apr 1958	A
2840181	Wildman	Jun 1958	A
2882633	Howell	Apr 1959	A
2912605	Tibbetts	Nov 1959	A
2938113	Schnell	May 1960	A
2950556	Larios	Aug 1960	A
3019854	Obryant	Feb 1962	A
3132713	Seeler	May 1964	A
3143182	Sears	Aug 1964	A
3160225	Sechrist	Dec 1964	A
3161975	Mcmillan	Dec 1964	A
3205601	Gawne	Sep 1965	A
3239973	Hannes	Mar 1966	A
3240883	Seeler	Mar 1966	A
3310901	Sarkisian	Mar 1967	A
3321170	Vye	May 1967	A
3509290	Mochida	Apr 1970	A
3573399	Schroeder	Apr 1971	A
3657490	Scheiber	Apr 1972	A
3696885	Grieg	Oct 1972	A
3755625	Maston	Aug 1973	A
3828508	Moeller	Aug 1974	A
3857191	Sadorus	Dec 1974	A
3895194	Fraim	Jul 1975	A
3906431	Clearwaters	Sep 1975	A
D237103	Fisher	Oct 1975	S
3936606	Wanke	Feb 1976	A
3938617	Forbes	Feb 1976	A
3941638	Horky	Mar 1976	A
3992584	Dugan	Nov 1976	A
4007461	Luedtke	Feb 1977	A
4008408	Kodama	Feb 1977	A
4029170	Phillips	Jun 1977	A
4032725	Mcgee	Jun 1977	A
4070547	Dellar	Jan 1978	A
4072821	Bauer	Feb 1978	A
4096353	Bauer	Jun 1978	A
4127156	Brandt	Nov 1978	A
4131760	Christensen	Dec 1978	A
4169219	Beard	Sep 1979	A
4184048	Alcaide	Jan 1980	A
4198705	Massa	Apr 1980	A
D255234	Wellward	Jun 1980	S
D256015	Doherty	Jul 1980	S
4212133	Lufkin	Jul 1980	A
4237339	Bunting	Dec 1980	A
4244096	Kashichi	Jan 1981	A
4244906	Heinemann	Jan 1981	A
4254417	Speiser	Mar 1981	A
4275694	Nagaishi	Jun 1981	A
4296280	Richie	Oct 1981	A
4305141	Massa	Dec 1981	A
4308425	Momose	Dec 1981	A
4311874	Wallace, Jr.	Jan 1982	A
4330691	Gordon	May 1982	A
4334740	Wray	Jun 1982	A
4365449	Liautaud	Dec 1982	A
4373191	Fette	Feb 1983	A
4393631	Krent	Jul 1983	A
4414433	Horie	Nov 1983	A
4429850	Weber	Feb 1984	A
4436966	Botros	Mar 1984	A
4449238	Lee	May 1984	A
4466117	Rudolf	Aug 1984	A
4485484	Flanagan	Nov 1984	A
4489442	Anderson	Dec 1984	A
4518826	Caudill	May 1985	A
4521908	Miyaji	Jun 1985	A
4566557	Lemaitre	Jan 1986	A
4593404	Bolin	Jun 1986	A
4594478	Gumb	Jun 1986	A
D285067	Delbuck	Aug 1986	S
4625827	Bartlett	Dec 1986	A
4653102	Hansen	Mar 1987	A
4658425	Julstrom	Apr 1987	A
4669108	Deinzer	May 1987	A
4675906	Sessler	Jun 1987	A
4693174	Anderson	Sep 1987	A
4696043	Iwahara	Sep 1987	A
4712231	Julstrom	Dec 1987	A
4741038	Elko	Apr 1988	A
4752961	Kahn	Jun 1988	A
4805730	O'Neill	Feb 1989	A
4815132	Minami	Mar 1989	A
4860366	Fukushi	Aug 1989	A
4862507	Woodard	Aug 1989	A
4866868	Kass	Sep 1989	A
4881135	Heilweil	Nov 1989	A
4888807	Reichel	Dec 1989	A
4903247	Van Gerwen	Feb 1990	A
4923032	Nuernberger	May 1990	A
4928312	Hill	May 1990	A
4969197	Takaya	Nov 1990	A
5000286	Crawford	Mar 1991	A
5038935	Wenkman	Aug 1991	A
5058170	Kanamori	Oct 1991	A
5088574	Kertesz, III	Feb 1992	A
D324780	Sebesta	Mar 1992	S
5121426	Baumhauer	Jun 1992	A
D329239	Hahn	Sep 1992	S
5189701	Jain	Feb 1993	A
5204907	Staple	Apr 1993	A
5214709	Ribic	May 1993	A
D340718	Leger	Oct 1993	S
5289544	Franklin	Feb 1994	A
D345346	Alfonso	Mar 1994	S
D345379	Chan	Mar 1994	S
5297210	Julstrom	Mar 1994	A
5322979	Cassity	Jun 1994	A
5323459	Hirano	Jun 1994	A
5329593	Lazzeroni	Jul 1994	A
5335011	Addeo	Aug 1994	A
5353279	Koyama	Oct 1994	A
5359374	Schwartz	Oct 1994	A
5371789	Hirano	Dec 1994	A
5383293	Royal	Jan 1995	A
5384843	Masuda	Jan 1995	A
5396554	Hirano	Mar 1995	A
5400413	Kindel	Mar 1995	A
D363045	Phillips	Oct 1995	S
5473701	Juergen	Dec 1995	A
5509634	Gebka	Apr 1996	A
5513265	Hirano	Apr 1996	A
5525765	Freiheit	Jun 1996	A
5550924	Helf	Aug 1996	A
5550925	Hori	Aug 1996	A
5555447	Kotzin	Sep 1996	A
5574793	Hirschhorn	Nov 1996	A
5602962	Kellermann	Feb 1997	A
5633936	Oh	May 1997	A
5645257	Ward	Jul 1997	A
D382118	Ferrero	Aug 1997	S
5657393	Crow	Aug 1997	A
5661813	Shimauchi	Aug 1997	A
5673327	Julstrom	Sep 1997	A
5687229	Sih	Nov 1997	A
5706344	Finn	Jan 1998	A
5715319	Chu	Feb 1998	A
5717171	Miller	Feb 1998	A
D392977	Kim	Mar 1998	S
D394061	Fink	May 1998	S
5761318	Shimauchi	Jun 1998	A
5766702	Lin	Jun 1998	A
5787183	Chu	Jul 1998	A
5796819	Romesburg	Aug 1998	A
5848146	Slattery	Dec 1998	A
5870482	Loeppert	Feb 1999	A
5878147	Killion	Mar 1999	A
5888412	Sooriakumar	Mar 1999	A
5888439	Miller	Mar 1999	A
D416315	Nanjo	Nov 1999	S
5978211	Hong	Nov 1999	A
5991277	Maeng	Nov 1999	A
6035962	Lin	Mar 2000	A
6039457	O'Neal	Mar 2000	A
6041127	Elko	Mar 2000	A
6049607	Marash	Apr 2000	A
D424538	Hayashi	May 2000	S
6069961	Nakazawa	May 2000	A
6125179	Wu	Sep 2000	A
D432518	Muto	Oct 2000	S
6128395	De Vries	Oct 2000	A
6137887	Anderson	Oct 2000	A
6144746	Azima	Nov 2000	A
6151399	Killion	Nov 2000	A
6173059	Huang	Jan 2001	B1
6198831	Azima	Mar 2001	B1
6205224	Underbrink	Mar 2001	B1
6215881	Azima	Apr 2001	B1
6266427	Mathur	Jul 2001	B1
6285770	Azima	Sep 2001	B1
6301357	Romesburg	Oct 2001	B1
6329908	Frecska	Dec 2001	B1
6332029	Azima	Dec 2001	B1
D453016	Nevill	Jan 2002	S
6386315	Roy	May 2002	B1
6393129	Conrad	May 2002	B1
6424635	Song	Jul 2002	B1
6442272	Osovets	Aug 2002	B1
6449593	Valve	Sep 2002	B1
6481173	Roy	Nov 2002	B1
6488367	Debesis	Dec 2002	B1
D469090	Tsuji	Jan 2003	S
6505057	Finn	Jan 2003	B1
6507659	Iredale	Jan 2003	B1
6510919	Roy	Jan 2003	B1
6526147	Rung	Feb 2003	B1
6556682	Gilloire	Apr 2003	B1
6592237	Pledger	Jul 2003	B1
6622030	Romesburg	Sep 2003	B1
D480923	Neubourg	Oct 2003	S
6633647	Markow	Oct 2003	B1
6665971	Lowry	Dec 2003	B2
6694028	Matsuo	Feb 2004	B1
6704422	Jensen	Mar 2004	B1
D489707	Kobayashi	May 2004	S
6731334	Maeng	May 2004	B1
6741720	Myatt	May 2004	B1
6757393	Spitzer	Jun 2004	B1
6768795	Jimen	Jul 2004	B2
6868377	Laroche	Mar 2005	B1
6885750	Egelmeers	Apr 2005	B2
6885986	Gigi	Apr 2005	B1
D504889	Andre	May 2005	S
6889183	Gunduzhan	May 2005	B1
6895093	Ali	May 2005	B1
6931123	Hughes	Aug 2005	B1
6944312	Mason	Sep 2005	B2
D510729	Chen	Oct 2005	S
6968064	Ning	Nov 2005	B1
6990193	Beaucoup	Jan 2006	B2
6993126	Kyrylenko	Jan 2006	B1
6993145	Combest	Jan 2006	B2
7003099	Zhang	Feb 2006	B1
7013267	Huart	Mar 2006	B1
7031269	Lee	Apr 2006	B2
7035398	Matsuo	Apr 2006	B2
7035415	Belt	Apr 2006	B2
7050576	Zhang	May 2006	B2
7054451	Janse	May 2006	B2
D526643	Ishizaki	Aug 2006	S
D527372	Allen	Aug 2006	S
7092516	Furuta	Aug 2006	B2
7092882	Arrowood	Aug 2006	B2
7098865	Christensen	Aug 2006	B2
7106876	Santiago	Sep 2006	B2
7120269	Lowell	Oct 2006	B2
7130309	Boaz	Oct 2006	B2
D533177	Andre	Dec 2006	S
7149320	Haykin	Dec 2006	B2
7161534	Tsai	Jan 2007	B2
7187765	Popovic	Mar 2007	B2
7203308	Kubota	Apr 2007	B2
D542543	Bruce	May 2007	S
7212628	Mirjana	May 2007	B2
D546318	Yoon	Jul 2007	S
D546814	Takita	Jul 2007	S
D547748	Tsuge	Jul 2007	S
7239714	De Blok	Jul 2007	B2
D549673	Niitsu	Aug 2007	S
7269263	Dedieu	Sep 2007	B2
D552570	Niitsu	Oct 2007	S
D559553	James	Jan 2008	S
7333476	LeBlanc	Feb 2008	B2
D566685	Koller	Apr 2008	S
7359504	Reuss	Apr 2008	B1
7366310	Stinson	Apr 2008	B2
7387151	Payne	Jun 2008	B1
7412376	Florencio	Aug 2008	B2
7415117	Tashev	Aug 2008	B2
D578509	Thomas	Oct 2008	S
D581510	Albano	Nov 2008	S
D582391	Morimoto	Dec 2008	S
D587709	Niitsu	Mar 2009	S
D589605	Reedy	Mar 2009	S
7503616	Linhard	Mar 2009	B2
7515719	Hooley	Apr 2009	B2
7536769	Pedersen	May 2009	B2
D595402	Miyake	Jun 2009	S
D595736	Son	Jul 2009	S
7558381	Ali	Jul 2009	B1
7565949	Tojo	Jul 2009	B2
D601585	Andre	Oct 2009	S
7651390	Profeta	Jan 2010	B1
7660428	Rodman	Feb 2010	B2
7667728	Kenoyer	Feb 2010	B2
7672445	Zhang	Mar 2010	B1
D613338	Marukos	Apr 2010	S
7701110	Fukuda	Apr 2010	B2
7702116	Stone	Apr 2010	B2
D614871	Tang	May 2010	S
7724891	Beaucoup	May 2010	B2
D617441	Koury	Jun 2010	S
7747001	Kellermann	Jun 2010	B2
7756278	Moorer	Jul 2010	B2
7783063	Pocino	Aug 2010	B2
7787328	Chu	Aug 2010	B2
7830862	James	Nov 2010	B2
7831035	Stokes	Nov 2010	B2
7831036	Beaucoup	Nov 2010	B2
7856097	Tokuda	Dec 2010	B2
7881486	Killion	Feb 2011	B1
7894421	Kwan	Feb 2011	B2
D636188	Kim	Apr 2011	S
7925006	Hirai	Apr 2011	B2
7925007	Stokes	Apr 2011	B2
7936886	Kim	May 2011	B2
7970123	Beaucoup	Jun 2011	B2
7970151	Oxford	Jun 2011	B2
D642385	Lee	Aug 2011	S
D643015	Kim	Aug 2011	S
7991167	Oxford	Aug 2011	B2
7995768	Miki	Aug 2011	B2
8000481	Nishikawa	Aug 2011	B2
8005238	Tashev	Aug 2011	B2
8019091	Burnett	Sep 2011	B2
8041054	Yeldener	Oct 2011	B2
8059843	Hung	Nov 2011	B2
8064629	Jiang	Nov 2011	B2
8085947	Haulick	Dec 2011	B2
8085949	Kim	Dec 2011	B2
8095120	Blair	Jan 2012	B1
8098842	Florencio	Jan 2012	B2
8098844	Elko	Jan 2012	B2
8103030	Barthel	Jan 2012	B2
8109360	Stewart, Jr.	Feb 2012	B2
8112272	Nagahama	Feb 2012	B2
8116500	Oxford	Feb 2012	B2
8121834	Rosec	Feb 2012	B2
D655271	Park	Mar 2012	S
D656473	Laube	Mar 2012	S
8130969	Buck	Mar 2012	B2
8130977	Chu	Mar 2012	B2
8135143	Ishibashi	Mar 2012	B2
8144886	Ishibashi	Mar 2012	B2
D658153	Woo	Apr 2012	S
8155331	Nakadai	Apr 2012	B2
8170882	Davis	May 2012	B2
8175291	Chan	May 2012	B2
8175871	Wang	May 2012	B2
8184801	Hamalainen	May 2012	B1
8189765	Nishikawa	May 2012	B2
8189810	Wolff	May 2012	B2
8194863	Takumai	Jun 2012	B2
8199927	Raftery	Jun 2012	B1
8204198	Adeney	Jun 2012	B2
8204248	Haulick	Jun 2012	B2
8208664	Iwasaki	Jun 2012	B2
8213596	Beaucoup	Jul 2012	B2
8213634	Daniel	Jul 2012	B1
8219387	Cutler	Jul 2012	B2
8229134	Duraiswami	Jul 2012	B2
8233352	Beaucoup	Jul 2012	B2
8243951	Ishibashi	Aug 2012	B2
8244536	Arun	Aug 2012	B2
8249273	Inoda	Aug 2012	B2
8259959	Marton	Sep 2012	B2
8275120	Stokes, III	Sep 2012	B2
8280728	Chen	Oct 2012	B2
8284949	Farhang	Oct 2012	B2
8284952	Reining	Oct 2012	B2
8286749	Stewart	Oct 2012	B2
8290142	Lambert	Oct 2012	B1
8291670	Gard	Oct 2012	B2
8297402	Stewart	Oct 2012	B2
8315380	Liu	Nov 2012	B2
8331582	Steele	Dec 2012	B2
8345898	Reining	Jan 2013	B2
8355521	Larson	Jan 2013	B2
8370140	Vitte	Feb 2013	B2
8379823	Ratmanski	Feb 2013	B2
8385557	Tashev	Feb 2013	B2
D678329	Lee	Mar 2013	S
8395653	Feng	Mar 2013	B2
8403107	Stewart	Mar 2013	B2
8406436	Craven	Mar 2013	B2
8428661	Chen	Apr 2013	B2
8433061	Cutler	Apr 2013	B2
D682266	Wu	May 2013	S
8437490	Marton	May 2013	B2
8443930	Stewart, Jr.	May 2013	B2
8447590	Ishibashi	May 2013	B2
8472639	Reining	Jun 2013	B2
8472640	Marton	Jun 2013	B2
D685346	Szymanski	Jul 2013	S
D686182	Ashiwa	Jul 2013	S
8479871	Stewart	Jul 2013	B2
8483398	Fozunbal	Jul 2013	B2
8498423	Thaden	Jul 2013	B2
D687432	Duan	Aug 2013	S
8503653	Ahuja	Aug 2013	B2
8515089	Nicholson	Aug 2013	B2
8515109	Dittberner	Aug 2013	B2
8526633	Ukai	Sep 2013	B2
8553904	Said	Oct 2013	B2
8559611	Ratmanski	Oct 2013	B2
D693328	Goetzen	Nov 2013	S
8583481	Viveiros	Nov 2013	B2
8599194	Lewis	Dec 2013	B2
8600443	Kawaguchi	Dec 2013	B2
8605890	Zhang	Dec 2013	B2
8620650	Walters	Dec 2013	B2
8631897	Stewart	Jan 2014	B2
8634569	Lu	Jan 2014	B2
8638951	Zurek	Jan 2014	B2
D699712	Bourne	Feb 2014	S
8644477	Gilbert	Feb 2014	B2
8654955	Lambert	Feb 2014	B1
8654990	Faller	Feb 2014	B2
8660274	Wolff	Feb 2014	B2
8660275	Buck	Feb 2014	B2
8670581	Harman	Mar 2014	B2
8672087	Stewart	Mar 2014	B2
8675890	Schmidt	Mar 2014	B2
8675899	Jung	Mar 2014	B2
8676728	Velusamy	Mar 2014	B1
8682675	Togami	Mar 2014	B2
8724829	Visser	May 2014	B2
8730156	Weising	May 2014	B2
8744069	Cutler	Jun 2014	B2
8744101	Burns	Jun 2014	B1
8755536	Chen	Jun 2014	B2
8811601	Mohammad	Aug 2014	B2
8818002	Tashev	Aug 2014	B2
8824693	Åhgren	Sep 2014	B2
8842851	Beaucoup	Sep 2014	B2
8855326	Derkx	Oct 2014	B2
8855327	Tanaka	Oct 2014	B2
8861713	Xu	Oct 2014	B2
8861756	Zhu	Oct 2014	B2
8873789	Bigeh	Oct 2014	B2
D717272	Kim	Nov 2014	S
8886343	Ishibashi	Nov 2014	B2
8893849	Hudson	Nov 2014	B2
8898633	Bryant	Nov 2014	B2
D718731	Lee	Dec 2014	S
8903106	Meyer	Dec 2014	B2
8923529	Mccowan	Dec 2014	B2
8929564	Kikkeri	Jan 2015	B2
8942382	Elko	Jan 2015	B2
8965546	Visser	Feb 2015	B2
D725059	Kim	Mar 2015	S
D725631	Mcnamara	Mar 2015	S
8976977	De	Mar 2015	B2
8983089	Chu	Mar 2015	B1
8983834	Davis	Mar 2015	B2
D726144	Kang	Apr 2015	S
D727968	Onoue	Apr 2015	S
9002028	Haulick	Apr 2015	B2
D729767	Lee	May 2015	S
9038301	Zelbacher	May 2015	B2
9088336	Mani	Jul 2015	B2
9094496	Teutsch	Jul 2015	B2
D735717	Lam	Aug 2015	S
D737245	Fan	Aug 2015	S
9099094	Burnett	Aug 2015	B2
9107001	Diethorn	Aug 2015	B2
9111543	Åhgren	Aug 2015	B2
9113242	Hyun	Aug 2015	B2
9113247	Chatlani	Aug 2015	B2
9126827	Hsieh	Sep 2015	B2
9129223	Velusamy	Sep 2015	B1
9140054	Oberbroeckling	Sep 2015	B2
D740279	Wu	Oct 2015	S
9172345	Kok	Oct 2015	B2
D743376	Kim	Nov 2015	S
D743939	Seong	Nov 2015	S
9196261	Burnett	Nov 2015	B2
9197974	Clark	Nov 2015	B1
9203494	Tarighat Mehrabani	Dec 2015	B2
9215327	Bathurst	Dec 2015	B2
9215543	Sun	Dec 2015	B2
9226062	Sun	Dec 2015	B2
9226070	Hyun	Dec 2015	B2
9226088	Pandey	Dec 2015	B2
9232185	Graham	Jan 2016	B2
9237391	Benesty	Jan 2016	B2
9247367	Nobile	Jan 2016	B2
9253567	Morcelli	Feb 2016	B2
9257132	Gowreesunker	Feb 2016	B2
9264553	Pandey	Feb 2016	B2
9264805	Buck	Feb 2016	B2
9280985	Tawada	Mar 2016	B2
9286908	Zhang	Mar 2016	B2
9294839	Lambert	Mar 2016	B2
9301049	Elko	Mar 2016	B2
D754103	Fischer	Apr 2016	S
9307326	Elko	Apr 2016	B2
9319532	Bao	Apr 2016	B2
9319799	Salmon	Apr 2016	B2
9326060	Nicholson	Apr 2016	B2
D756502	Lee	May 2016	S
9330673	Cho	May 2016	B2
9338301	Pocino	May 2016	B2
9338549	Haulick	May 2016	B2
9354310	Visser	May 2016	B2
9357080	Beaucoup	May 2016	B2
9403670	Schelling	Aug 2016	B2
9426598	Walsh	Aug 2016	B2
D767748	Nakai	Sep 2016	S
9451078	Yang	Sep 2016	B2
D769239	Li	Oct 2016	S
9462378	Kuech	Oct 2016	B2
9473868	Huang	Oct 2016	B2
9479627	Rung	Oct 2016	B1
9479885	Ivanov	Oct 2016	B1
9489948	Chu	Nov 2016	B1
9510090	Lissek	Nov 2016	B2
9514723	Silfvast	Dec 2016	B2
9516412	Shigenaga	Dec 2016	B2
9521057	Klingbeil	Dec 2016	B2
9549245	Frater	Jan 2017	B2
9560446	Chang	Jan 2017	B1
9560451	Eichfeld	Jan 2017	B2
9565493	Abraham	Feb 2017	B2
9578413	Sawa	Feb 2017	B2
9578440	Otto	Feb 2017	B2
9589556	Gao	Mar 2017	B2
9591123	Sorensen	Mar 2017	B2
9591404	Chhetri	Mar 2017	B1
D784299	Cho	Apr 2017	S
9615173	Sako	Apr 2017	B2
9628596	Bullough	Apr 2017	B1
9635186	Pandey	Apr 2017	B2
9635474	Kuster	Apr 2017	B2
D787481	Jorunn	May 2017	S
D788073	Silvera	May 2017	S
9640187	Niemisto	May 2017	B2
9641688	Pandey	May 2017	B2
9641929	Li	May 2017	B2
9641935	Ivanov	May 2017	B1
9653091	Matsuo	May 2017	B2
9653092	Sun	May 2017	B2
9655001	Metzger	May 2017	B2
9659576	Kotvis	May 2017	B1
D789323	Mackiewicz	Jun 2017	S
9674604	Deroo	Jun 2017	B2
9692882	Mani	Jun 2017	B2
9706057	Mani	Jul 2017	B2
9716944	Yliaho	Jul 2017	B2
9721582	Huang	Aug 2017	B1
9734835	Fujieda	Aug 2017	B2
9754572	Salazar	Sep 2017	B2
9761243	Taenzer	Sep 2017	B2
D801285	Timmins	Oct 2017	S
9788119	Vilermo	Oct 2017	B2
9813806	Graham	Nov 2017	B2
9818426	Kotera	Nov 2017	B2
9826211	Sawa	Nov 2017	B2
9854101	Pandey	Dec 2017	B2
9854363	Sladeczek	Dec 2017	B2
9860439	Sawa	Jan 2018	B2
9866952	Pandey	Jan 2018	B2
D811393	Ahn	Feb 2018	S
9894434	Rollow, IV	Feb 2018	B2
9930448	Chen	Mar 2018	B1
9936290	Mohammad	Apr 2018	B2
9966059	Ayrapetian	May 2018	B1
9973848	Chhetri	May 2018	B2
9980042	Benattar	May 2018	B1
D819607	Chui	Jun 2018	S
D819631	Matsumiya	Jun 2018	S
10015589	Ebenezer	Jul 2018	B1
10021506	Johnson	Jul 2018	B2
10021515	Mallya	Jul 2018	B1
10034116	Kadri	Jul 2018	B2
10054320	Choi	Aug 2018	B2
10061009	Family	Aug 2018	B1
10062379	Katuri	Aug 2018	B2
10153744	Every	Dec 2018	B1
10165386	Lehtiniemi	Dec 2018	B2
D841589	Andreas	Feb 2019	S
10206030	Matsumoto	Feb 2019	B2
10210882	Mccowan	Feb 2019	B1
10231062	Pedersen	Mar 2019	B2
10244121	Mani	Mar 2019	B2
10244219	Sawa	Mar 2019	B2
10269343	Wingate	Apr 2019	B2
10366702	Morton	Jul 2019	B2
10367948	Wells-Rutherford	Jul 2019	B2
D857873	Shimada	Aug 2019	S
10389861	Mani	Aug 2019	B2
10389885	Sun	Aug 2019	B2
D860319	Beruto	Sep 2019	S
D860997	Sae	Sep 2019	S
D864136	Kim	Oct 2019	S
10440469	Barnett	Oct 2019	B2
D865723	Cho	Nov 2019	S
10566008	Thorpe	Feb 2020	B2
10602267	Grosche	Mar 2020	B2
D883952	Lucas	May 2020	S
10650797	Kumar	May 2020	B2
D888020	Lyu	Jun 2020	S
10728653	Graham	Jul 2020	B2
D900070	Lantz	Oct 2020	S
D900071	Lantz	Oct 2020	S
D900072	Lantz	Oct 2020	S
D900073	Lantz	Oct 2020	S
D900074	Lantz	Oct 2020	S
10827263	Christoph	Nov 2020	B2
10863270	Cornelius	Dec 2020	B1
10930297	Christoph	Feb 2021	B2
10959018	Shi	Mar 2021	B1
10979805	Chowdhary	Apr 2021	B2
D924189	Park	Jul 2021	S
11109133	Lantz	Aug 2021	B2
D940116	Cho	Jan 2022	S
20010031058	Anderson	Oct 2001	A1
20020015500	Belt	Feb 2002	A1
20020041679	Beaucoup	Apr 2002	A1
20020048377	Vaudrey	Apr 2002	A1
20020064158	Yokoyama	May 2002	A1
20020064287	Kawamura	May 2002	A1
20020069054	Arrowood	Jun 2002	A1
20020110255	Killion	Aug 2002	A1
20020126861	Colby	Sep 2002	A1
20020131580	Smith	Sep 2002	A1
20020140633	Rafii	Oct 2002	A1
20020146282	Wilkes	Oct 2002	A1
20020149070	Sheplak	Oct 2002	A1
20020159603	Hirai	Oct 2002	A1
20030026437	Janse	Feb 2003	A1
20030053639	Beaucoup	Mar 2003	A1
20030059061	Tsuji	Mar 2003	A1
20030063762	Tajima	Apr 2003	A1
20030063768	Cornelius	Apr 2003	A1
20030072461	Moorer	Apr 2003	A1
20030107478	Hendricks	Jun 2003	A1
20030118200	Beaucoup	Jun 2003	A1
20030122777	Grover	Jul 2003	A1
20030138119	Pocino	Jul 2003	A1
20030156725	Boone	Aug 2003	A1
20030161485	Smith	Aug 2003	A1
20030163326	Maase	Aug 2003	A1
20030169888	Subotic	Sep 2003	A1
20030185404	Milsap	Oct 2003	A1
20030198339	Roy	Oct 2003	A1
20030198359	Killion	Oct 2003	A1
20030202107	Slattery	Oct 2003	A1
20040013038	Kajala	Jan 2004	A1
20040013252	Craner	Jan 2004	A1
20040076305	Santiago	Apr 2004	A1
20040105557	Matsuo	Jun 2004	A1
20040125942	Beaucoup	Jul 2004	A1
20040175006	Kim	Sep 2004	A1
20040202345	Stenberg	Oct 2004	A1
20040240664	Freed	Dec 2004	A1
20050005494	Way	Jan 2005	A1
20050041530	Goudie	Feb 2005	A1
20050069156	Haapapuro	Mar 2005	A1
20050094580	Kumar	May 2005	A1
20050094795	Rambo	May 2005	A1
20050149320	Kajala	Jul 2005	A1
20050157897	Saltykov	Jul 2005	A1
20050175189	Lee	Aug 2005	A1
20050175190	Tashev	Aug 2005	A1
20050213747	Popovich	Sep 2005	A1
20050221867	Zurek	Oct 2005	A1
20050238196	Furuno	Oct 2005	A1
20050270906	Ramenzoni	Dec 2005	A1
20050271221	Cerwin	Dec 2005	A1
20050286698	Bathurst	Dec 2005	A1
20050286729	Harwood	Dec 2005	A1
20060083390	Kaderavek	Apr 2006	A1
20060088173	Rodman	Apr 2006	A1
20060093128	Oxford	May 2006	A1
20060098403	Smith	May 2006	A1
20060104458	Kenoyer	May 2006	A1
20060109983	Young	May 2006	A1
20060151256	Lee	Jul 2006	A1
20060159293	Azima	Jul 2006	A1
20060161430	Schweng	Jul 2006	A1
20060165242	Miki	Jul 2006	A1
20060192976	Hall	Aug 2006	A1
20060198541	Henry	Sep 2006	A1
20060204022	Hooley	Sep 2006	A1
20060215866	Francisco	Sep 2006	A1
20060222187	Jarrett	Oct 2006	A1
20060233353	Beaucoup	Oct 2006	A1
20060239471	Mao	Oct 2006	A1
20060262942	Oxford	Nov 2006	A1
20060269080	Oxford	Nov 2006	A1
20060269086	Page	Nov 2006	A1
20070006474	Taniguchi	Jan 2007	A1
20070009116	Reining	Jan 2007	A1
20070019828	Hughes	Jan 2007	A1
20070053524	Haulick	Mar 2007	A1
20070093714	Beaucoup	Apr 2007	A1
20070116255	Derkx	May 2007	A1
20070120029	Keung	May 2007	A1
20070165871	Roovers	Jul 2007	A1
20070230712	Belt	Oct 2007	A1
20070253561	Williams	Nov 2007	A1
20070269066	Derleth	Nov 2007	A1
20080008339	Ryan	Jan 2008	A1
20080033723	Jang	Feb 2008	A1
20080046235	Chen	Feb 2008	A1
20080056517	Algazi	Mar 2008	A1
20080101622	Sugiyama	May 2008	A1
20080130907	Sudo	Jun 2008	A1
20080144848	Buck	Jun 2008	A1
20080168283	Penning	Jul 2008	A1
20080188965	Bruey	Aug 2008	A1
20080212805	Fincham	Sep 2008	A1
20080232607	Tashev	Sep 2008	A1
20080247567	Kjolerbakken	Oct 2008	A1
20080253553	Li	Oct 2008	A1
20080253589	Trahms	Oct 2008	A1
20080259731	Happonen	Oct 2008	A1
20080260175	Elko	Oct 2008	A1
20080279400	Knoll	Nov 2008	A1
20080285772	Haulick	Nov 2008	A1
20090003586	Lai	Jan 2009	A1
20090030536	Gur	Jan 2009	A1
20090052684	Ishibashi	Feb 2009	A1
20090086998	Jeong	Apr 2009	A1
20090087000	Ko	Apr 2009	A1
20090087001	Jiang	Apr 2009	A1
20090094817	Killion	Apr 2009	A1
20090129609	Oh	May 2009	A1
20090147967	Ishibashi	Jun 2009	A1
20090150149	Cutter	Jun 2009	A1
20090161880	Hooley	Jun 2009	A1
20090169027	Ura	Jul 2009	A1
20090173030	Gulbrandsen	Jul 2009	A1
20090173570	Levit	Jul 2009	A1
20090226004	Moeller	Sep 2009	A1
20090233545	Sutskover	Sep 2009	A1
20090237561	Kobayashi	Sep 2009	A1
20090254340	Sun	Oct 2009	A1
20090274318	Ishibashi	Nov 2009	A1
20090310794	Ishibashi	Dec 2009	A1
20100011644	Kramer	Jan 2010	A1
20100034397	Nakadai	Feb 2010	A1
20100074433	Zhang	Mar 2010	A1
20100111323	Marton	May 2010	A1
20100111324	Yeldener	May 2010	A1
20100119097	Ohtsuka	May 2010	A1
20100123785	Chen	May 2010	A1
20100128892	Chen	May 2010	A1
20100128901	Herman	May 2010	A1
20100131749	Kim	May 2010	A1
20100142721	Wada	Jun 2010	A1
20100150364	Buck	Jun 2010	A1
20100158268	Marton	Jun 2010	A1
20100165071	Ishibashi	Jul 2010	A1
20100166219	Marton	Jul 2010	A1
20100189275	Christoph	Jul 2010	A1
20100189299	Grant	Jul 2010	A1
20100202628	Meyer	Aug 2010	A1
20100208605	Wang	Aug 2010	A1
20100215184	Buck	Aug 2010	A1
20100215189	Marton	Aug 2010	A1
20100217590	Nemer	Aug 2010	A1
20100245624	Beaucoup	Sep 2010	A1
20100246873	Chen	Sep 2010	A1
20100284185	Ngai	Nov 2010	A1
20100305728	Aiso	Dec 2010	A1
20100314513	Evans	Dec 2010	A1
20110002469	Ojala	Jan 2011	A1
20110007921	Stewart	Jan 2011	A1
20110033063	Mcgrath	Feb 2011	A1
20110038229	Beaucoup	Feb 2011	A1
20110096136	Liu	Apr 2011	A1
20110096631	Kondo	Apr 2011	A1
20110096915	Nemer	Apr 2011	A1
20110164761	Mccowan	Jul 2011	A1
20110194719	Frater	Aug 2011	A1
20110211706	Tanaka	Sep 2011	A1
20110235821	Okita	Sep 2011	A1
20110268287	Ishibashi	Nov 2011	A1
20110311064	Teutsch	Dec 2011	A1
20110311085	Stewart	Dec 2011	A1
20110317862	Hosoe	Dec 2011	A1
20120002835	Stewart	Jan 2012	A1
20120014049	Ogle	Jan 2012	A1
20120027227	Kok	Feb 2012	A1
20120076316	Zhu	Mar 2012	A1
20120080260	Stewart	Apr 2012	A1
20120093344	Sun	Apr 2012	A1
20120117474	Miki	May 2012	A1
20120128160	Kim	May 2012	A1
20120128175	Visser	May 2012	A1
20120155688	Wilson	Jun 2012	A1
20120155703	Hernandez-Abrego	Jun 2012	A1
20120163625	Siotis	Jun 2012	A1
20120169826	Jeong	Jul 2012	A1
20120177219	Mullen	Jul 2012	A1
20120182429	Forutanpour	Jul 2012	A1
20120207335	Spaanderman	Aug 2012	A1
20120224709	Keddem	Sep 2012	A1
20120243698	Elko	Sep 2012	A1
20120262536	Chen	Oct 2012	A1
20120288079	Burnett	Nov 2012	A1
20120288114	Duraiswami	Nov 2012	A1
20120294472	Hudson	Nov 2012	A1
20120327115	Chhetri	Dec 2012	A1
20120328142	Horibe	Dec 2012	A1
20130002797	Thapa	Jan 2013	A1
20130004013	Stewart	Jan 2013	A1
20130015014	Stewart	Jan 2013	A1
20130016847	Steiner	Jan 2013	A1
20130028451	De Roo	Jan 2013	A1
20130029684	Kawaguchi	Jan 2013	A1
20130034241	Pandey	Feb 2013	A1
20130039504	Pandey	Feb 2013	A1
20130083911	Bathurst	Apr 2013	A1
20130094689	Tanaka	Apr 2013	A1
20130101141	Mcelveen	Apr 2013	A1
20130136274	Per	May 2013	A1
20130142343	Matsui	Jun 2013	A1
20130147835	Lee	Jun 2013	A1
20130156198	Kim	Jun 2013	A1
20130182190	Mccartney	Jul 2013	A1
20130206501	Yu	Aug 2013	A1
20130216066	Yerrace	Aug 2013	A1
20130226593	Magnusson	Aug 2013	A1
20130251181	Stewart	Sep 2013	A1
20130264144	Hudson	Oct 2013	A1
20130271559	Feng	Oct 2013	A1
20130294616	Mulder	Nov 2013	A1
20130297302	Pan	Nov 2013	A1
20130304476	Kim	Nov 2013	A1
20130304479	Teller	Nov 2013	A1
20130329908	Lindahl	Dec 2013	A1
20130332156	Tackin	Dec 2013	A1
20130336516	Stewart	Dec 2013	A1
20130343549	Vemireddy	Dec 2013	A1
20140003635	Mohammad	Jan 2014	A1
20140010383	Mackey	Jan 2014	A1
20140016794	Lu	Jan 2014	A1
20140029761	Maenpaa	Jan 2014	A1
20140037097	Mark	Feb 2014	A1
20140050332	Nielsen	Feb 2014	A1
20140072151	Ochs	Mar 2014	A1
20140098233	Martin	Apr 2014	A1
20140098964	Rosca	Apr 2014	A1
20140122060	Kaszczuk	May 2014	A1
20140177857	Kuster	Jun 2014	A1
20140233777	Tseng	Aug 2014	A1
20140233778	Hardiman	Aug 2014	A1
20140264654	Salmon	Sep 2014	A1
20140265774	Stewart	Sep 2014	A1
20140270271	Dehe	Sep 2014	A1
20140286518	Stewart	Sep 2014	A1
20140295768	Wu	Oct 2014	A1
20140301586	Stewart	Oct 2014	A1
20140307882	Leblanc	Oct 2014	A1
20140314251	Rosca	Oct 2014	A1
20140341392	Lambert	Nov 2014	A1
20140357177	Stewart	Dec 2014	A1
20140363008	Chen	Dec 2014	A1
20150003638	Kasai	Jan 2015	A1
20150025878	Gowreesunker	Jan 2015	A1
20150030172	Gaensler	Jan 2015	A1
20150033042	Iwamoto	Jan 2015	A1
20150050967	Bao	Feb 2015	A1
20150055796	Nugent	Feb 2015	A1
20150055797	Nguyen	Feb 2015	A1
20150063579	Bao	Mar 2015	A1
20150070188	Aramburu	Mar 2015	A1
20150078581	Etter	Mar 2015	A1
20150078582	Graham	Mar 2015	A1
20150097719	Balachandreswaran	Apr 2015	A1
20150104023	Bilobrov	Apr 2015	A1
20150117672	Christoph	Apr 2015	A1
20150118960	Petit	Apr 2015	A1
20150126255	Yang	May 2015	A1
20150156578	Alexandridis	Jun 2015	A1
20150163577	Benesty	Jun 2015	A1
20150185825	Mullins	Jul 2015	A1
20150189423	Giannuzzi	Jul 2015	A1
20150208171	Funakoshi	Jul 2015	A1
20150237424	Wilker	Aug 2015	A1
20150281832	Kishimoto	Oct 2015	A1
20150281833	Shigenaga	Oct 2015	A1
20150281834	Takano	Oct 2015	A1
20150312662	Kishimoto	Oct 2015	A1
20150312691	Virolainen	Oct 2015	A1
20150326968	Shigenaga	Nov 2015	A1
20150341734	Sherman	Nov 2015	A1
20150350621	Sawa	Dec 2015	A1
20150358734	Butler	Dec 2015	A1
20160011851	Zhang	Jan 2016	A1
20160021478	Katagiri	Jan 2016	A1
20160029120	Nesta	Jan 2016	A1
20160031700	Sparks	Feb 2016	A1
20160037277	Matsumoto	Feb 2016	A1
20160055859	Finlow-Bates	Feb 2016	A1
20160080867	Nugent	Mar 2016	A1
20160088392	Huttunen	Mar 2016	A1
20160100092	Bohac	Apr 2016	A1
20160105473	Klingbeil	Apr 2016	A1
20160111109	Tsujikawa	Apr 2016	A1
20160127527	Mani	May 2016	A1
20160134928	Ogle	May 2016	A1
20160142548	Pandey	May 2016	A1
20160142814	Deroo	May 2016	A1
20160142815	Norris	May 2016	A1
20160148057	Oh	May 2016	A1
20160150315	Tzirkel-Hancock	May 2016	A1
20160150316	Kubota	May 2016	A1
20160155455	Petteri	Jun 2016	A1
20160165340	Benattar	Jun 2016	A1
20160173976	Podhradsky	Jun 2016	A1
20160173978	Li	Jun 2016	A1
20160189727	Wu	Jun 2016	A1
20160192068	Ng	Jun 2016	A1
20160196836	Yu	Jul 2016	A1
20160234593	Matsumoto	Aug 2016	A1
20160249132	Oliaei	Aug 2016	A1
20160275961	Yu	Sep 2016	A1
20160295279	Srinivasan	Oct 2016	A1
20160300584	Pandey	Oct 2016	A1
20160302002	Lambert	Oct 2016	A1
20160302006	Pandey	Oct 2016	A1
20160323667	Shumard	Nov 2016	A1
20160323668	Abraham	Nov 2016	A1
20160330545	Mcelveen	Nov 2016	A1
20160337523	Pandey	Nov 2016	A1
20160353200	Bigeh	Dec 2016	A1
20160357508	Moore	Dec 2016	A1
20170019744	Matsumoto	Jan 2017	A1
20170064451	Park	Mar 2017	A1
20170105066	Mclaughlin	Apr 2017	A1
20170134849	Pandey	May 2017	A1
20170134850	Graham	May 2017	A1
20170164101	Rollow, IV	Jun 2017	A1
20170180861	Chen	Jun 2017	A1
20170206064	Breazeal	Jul 2017	A1
20170230748	Shumard	Aug 2017	A1
20170264999	Fukuda	Sep 2017	A1
20170303887	Richmond	Oct 2017	A1
20170308352	Kessler	Oct 2017	A1
20170374454	Bernardini	Dec 2017	A1
20180083848	Siddiqi	Mar 2018	A1
20180102136	Varma	Apr 2018	A1
20180109873	Xiang	Apr 2018	A1
20180115799	Thiele	Apr 2018	A1
20180160224	Graham	Jun 2018	A1
20180196585	Densham	Jul 2018	A1
20180219922	Bryans	Aug 2018	A1
20180227666	Barnett	Aug 2018	A1
20180292079	Branham	Oct 2018	A1
20180310096	Shumard	Oct 2018	A1
20180313558	Byers	Nov 2018	A1
20180338205	Abraham	Nov 2018	A1
20180359565	Kim	Dec 2018	A1
20190042187	Truong	Feb 2019	A1
20190166424	Harney	May 2019	A1
20190215540	Nicol	Jul 2019	A1
20190230436	Tsingos	Jul 2019	A1
20190259408	Freeman	Aug 2019	A1
20190268683	Miyahara	Aug 2019	A1
20190295540	Grima	Sep 2019	A1
20190295569	Wang	Sep 2019	A1
20190319677	Hansen	Oct 2019	A1
20190371354	Lester	Dec 2019	A1
20190373362	Ansai	Dec 2019	A1
20190385629	Moravy	Dec 2019	A1
20190387311	Schultz	Dec 2019	A1
20200015021	Leppanen	Jan 2020	A1
20200021910	Rollow, IV	Jan 2020	A1
20200037068	Barnett	Jan 2020	A1
20200068297	Rollow, IV	Feb 2020	A1
20200100009	Lantz	Mar 2020	A1
20200100025	Shumard	Mar 2020	A1
20200137485	Yamakawa	Apr 2020	A1
20200145753	Rollow, IV	May 2020	A1
20200152218	Kikuhara	May 2020	A1
20200162618	Enteshari	May 2020	A1
20200228663	Wells-Rutherford	Jul 2020	A1
20200251119	Yang	Aug 2020	A1
20200275204	Labosco	Aug 2020	A1
20200278043	Cao	Sep 2020	A1
20200288237	Abraham	Sep 2020	A1
20210012789	Husain	Jan 2021	A1
20210021940	Petersen	Jan 2021	A1
20210044881	Lantz	Feb 2021	A1
20210051397	Veselinovic	Feb 2021	A1
20210098014	Tanaka	Apr 2021	A1
20210098015	Pandey	Apr 2021	A1
20210120335	Veselinovic	Apr 2021	A1
20210200504	Park	Jul 2021	A1
20210375298	Zhang	Dec 2021	A1

Foreign Referenced Citations (149)

Number	Date	Country
2359771	Apr 2003	CA
2475283	Jan 2005	CA
2505496	Oct 2006	CA
2838856	Dec 2012	CA
2846323	Sep 2014	CA
1780495	May 2006	CN
101217830	Jul 2008	CN
101833954	Sep 2010	CN
101860776	Oct 2010	CN
101894558	Nov 2010	CN
102646418	Aug 2012	CN
102821336	Dec 2012	CN
102833664	Dec 2012	CN
102860039	Jan 2013	CN
104036784	Sep 2014	CN
104053088	Sep 2014	CN
104080289	Oct 2014	CN
104347076	Feb 2015	CN
104581463	Apr 2015	CN
105355210	Feb 2016	CN
105548998	May 2016	CN
106162427	Nov 2016	CN
106251857	Dec 2016	CN
106851036	Jun 2017	CN
107221336	Sep 2017	CN
107534725	Jan 2018	CN
108172235	Jun 2018	CN
109087664	Dec 2018	CN
208190895	Dec 2018	CN
109727604	May 2019	CN
110010147	Jul 2019	CN
306391029	Mar 2021	CN
2941485	Apr 1981	DE
0077546430001	Mar 2020	EM
0381498	Aug 1990	EP
0594098	Apr 1994	EP
0869697	Oct 1998	EP
1180914	Feb 2002	EP
1184676	Mar 2002	EP
0944228	Jun 2003	EP
1439526	Jul 2004	EP
1651001	Apr 2006	EP
1727344	Nov 2006	EP
1906707	Apr 2008	EP
1952393	Aug 2008	EP
1962547	Aug 2008	EP
2133867	Dec 2009	EP
2159789	Mar 2010	EP
2197219	Jun 2010	EP
2360940	Aug 2011	EP
2710788	Mar 2014	EP
2721837	Apr 2014	EP
2772910	Sep 2014	EP
2778310	Sep 2014	EP
2942975	Nov 2015	EP
2988527	Feb 2016	EP
3131311	Feb 2017	EP
2393601	Mar 2004	GB
2446620	Aug 2008	GB
S63144699	Jun 1988	JP
H01260967	Oct 1989	JP
H0241099	Feb 1990	JP
H05260589	Oct 1993	JP
H07336790	Dec 1995	JP
3175622	Jun 2001	JP
2003060530	Feb 2003	JP
2003087890	Mar 2003	JP
2004349806	Dec 2004	JP
2004537232	Dec 2004	JP
2005323084	Nov 2005	JP
2006094389	Apr 2006	JP
2006101499	Apr 2006	JP
4120646	Aug 2006	JP
4258472	Aug 2006	JP
4196956	Sep 2006	JP
2006340151	Dec 2006	JP
4760160	Jan 2007	JP
4752403	Mar 2007	JP
2007089058	Apr 2007	JP
4867579	Jun 2007	JP
2007208503	Aug 2007	JP
2007228069	Sep 2007	JP
2007228070	Sep 2007	JP
2007274131	Oct 2007	JP
2007274463	Oct 2007	JP
2007288679	Nov 2007	JP
2008005347	Jan 2008	JP
2008042754	Feb 2008	JP
2008154056	Jul 2008	JP
2008259022	Oct 2008	JP
2008263336	Oct 2008	JP
2008312002	Dec 2008	JP
2009206671	Sep 2009	JP
2010028653	Feb 2010	JP
2010114554	May 2010	JP
2010268129	Nov 2010	JP
2011015018	Jan 2011	JP
4779748	Sep 2011	JP
2012165189	Aug 2012	JP
5028944	Sep 2012	JP
5139111	Feb 2013	JP
5306565	Oct 2013	JP
5685173	Mar 2015	JP
2016051038	Apr 2016	JP
100298300	May 2001	KR
100901464	Jun 2009	KR
100960781	Jun 2010	KR
1020130033723	Apr 2013	KR
300856915	May 2016	KR
201331932	Aug 2013	TW
I484478	May 2015	TW
1997008896	Mar 1997	WO
1998047291	Oct 1998	WO
2000030402	May 2000	WO
2003073786	Sep 2003	WO
2003088429	Oct 2003	WO
2004027754	Apr 2004	WO
2004090865	Oct 2004	WO
2006049260	May 2006	WO
2006071119	Jul 2006	WO
2006114015	Nov 2006	WO
2006121896	Nov 2006	WO
2007045971	Apr 2007	WO
2008074249	Jun 2008	WO
2008125523	Oct 2008	WO
2009039783	Apr 2009	WO
2009109069	Sep 2009	WO
2010001508	Jan 2010	WO
2010091999	Aug 2010	WO
2010140084	Dec 2010	WO
2010144148	Dec 2010	WO
2011104501	Sep 2011	WO
2012122132	Sep 2012	WO
2012140435	Oct 2012	WO
2012160459	Nov 2012	WO
2012174159	Dec 2012	WO
2013016986	Feb 2013	WO
2013182118	Dec 2013	WO
2014156292	Oct 2014	WO
2016176429	Nov 2016	WO
2016179211	Nov 2016	WO
2017208022	Dec 2017	WO
2018140444	Aug 2018	WO
2018140618	Aug 2018	WO
2018211806	Nov 2018	WO
2019231630	Dec 2019	WO
2020168873	Aug 2020	WO
2020191354	Sep 2020	WO
211843001	Nov 2020	WO

Non-Patent Literature Citations (276)

Entry
International Search Report and Written Opinion for PCT/US2020/024063 dated Aug. 31, 2020, 18 pp.
“Philips Hue Bulbs and Wireless Connected Lighting System,” Web page https://www.philips-hue.com/en-in, 8 pp, Sep. 23, 2020, retrieved from Internet Archive Wayback Machine, <https://web.archive.org/web/20200923171037/https://www.philips-hue.com/en-in> on Sep. 27, 2021.
“Vsa 2050 II Digitally Steerable Column Speaker,” Web page https://www.rcf.it/en_US/products/product-detail/vsa-2050-ii/972389, 15 pages, Dec. 24, 2018.
Advanced Network Devices, IPSCM Ceiling Tile IP Speaker, Feb. 2011, 2 pgs.
Advanced Network Devices, IPSCM Standard 2′ by 2′ Ceiling Tile Speaker, 2 pgs.
Affes, et al., “A Signal Subspace Tracking Algorithm for Microphone Array Processing of Speech,” IEEE Trans. on Speech and Audio Processing, vol. 5, No. 5, Sep. 1997, pp. 425-437.
Affes, et al., “A Source Subspace Tracking Array of Microphones for Double Talk Situations,” 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings, May 1996, pp. 909-912.
Affes, et al., “An Algorithm for Multisource Beamforming and Multitarget Tracking,” IEEE Trans. on Signal Processing, vol. 44, No. 6, Jun. 1996, pp. 1512-1522.
Affes, et al., “Robust Adaptive Beamforming via LMS-Like Target Tracking,” Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Apr. 1994, pp. IV-269-IV-272.
Ahonen, et al., “Directional Analysis of Sound Field with Linear Microphone Array and Applications in Sound Reproduction,” Audio Engineering Socity, Convention Paper 7329, May 2008, 11 pp.
Alarifi, et al., “Ultra Wideband Indoor Positioning Technologies: Analysis and Recent Advances,” Sensors 2016, vol. 16, No. 707, 36 pp.
Amazon webpage for Metalfab MFLCRFG (last visited Apr. 22, 2020) available at <https://www.amazon.com/RETURN-FILTERGRILLE-Drop-Ceiling/dp/B0064Q9A7l/ref=sr 12?dchild=1&keywords=drop+ceiling+return+air+grille&qid=1585862723&s=hi&sr=1-2>, 11 pp.
Armstrong “Walls” Catalog available at <https://www.armstrongceilings.com/content/dam/armstrongceilings/commercial/north-america/catalogs/armstrong-ceilings-wallsspecifiers-reference.pdf>, 2019, 30 pp.
Armstrong Tectum Ceiling & Wall Panels Catalog available at <https://www.armstrongceilings.com/content/dam/armstrongceilings/commercial/north-america/brochures/tectum-brochure.pdf>, 2019, 16 pp.
Armstrong Woodworks Concealed Catalog available at <https://sweets.construction.com/swts_content_files/3824/442581.pdf>, 2014, 6 pp.
Armstrong Woodworks Walls Catalog available at <https://www.armstrongceilings.com/pdbupimagesclg/220600.pdf/download/data-sheet-woodworks-walls.pdf>, 2019, 2 pp.
Armstrong World Industries, Inc., I-Ceilings Sound Systems Speaker Panels, 2002, 4 pgs.
Armstrong, Acoustical Design: Exposed Structure, available at <https://www.armstrongceilings.com/pdbupimagesclg/217142.pdf/download/acoustical-design-exposed-structurespaces-brochure.pdf>, 2018, 19 pp.
Armstrong, Ceiling Systems, Brochure page for Armstrong Softlook, 1995, 2 pp.
Armstrong, Excerpts from Armstrong 2011-2012 Ceiling Wall Systems Catalog, available at <https://web.archive.org/web/20121116034120/http://www.armst rong.com/commceilingsna/en_us/pdf/ceilings_catalog_screen-2011.pdf>, as early as 2012, 162 pp.
Armstrong, i-Ceilings, Brochure, 2009, 12 pp.
Arnold, et al., “A Directional Acoustic Array Using Silicon Micromachined Piezoresistive Microphones,” Journal of the Acoustical Society of America, 113(1), Jan. 2003, 10 pp.
Atlas Sound, I128SYSM IP Compliant Loudspeaker System with Microphone Data Sheet, 2009, 2 pgs.
Atlas Sound, 1′X2′ IP Speaker with Micophone for Suspended Ceiling Systems, https://www.atlasied.com/i128sysm, retrieved Oct. 25, 2017, 5 pgs.
Audio Technica, ES945 Omnidirectional Condenser Boundary Microphones, https://eu.audio-technica.com/resources/ES945%20Specifications.pdf, 2007, 1 pg.
Audix Microphones, Audix Introduces Innovative Ceiling Mics, http://audixusa.com/docs_12/latest_news/EFplFkAAklOtSdolke.shtml, Jun. 2011, 6 pgs.
Audix Microphones, M70 Flush Mount Ceiling Mic, May 2016, 2 pgs.
Automixer Gated, Information Sheet, MIT, Nov. 2019, 9 pp.
AVNetwork, “Top Five Conference Room Mic Myths,” Feb. 25, 2015, 14 pp.
Beh, et al., “Combining Acoustic Echo Cancellation and Adaptive Beamforming for Achieving Robust Speech Interface in Mobile Robot,” 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep. 2008, pp. 1693-1698.
Benesty, et al., “A New Class of Doubletalk Detectors Based on Cross-Correlation,” IEEE Transactions on Speech and Audio Processing, vol. 8, No. 2, Mar. 2000, pp. 168-172.
Benesty, et al., “Adaptive Algorithms for Mimo Acoustic Echo Cancellation,” AI2 Allen Institute for Artifical Intelligence, 2003.
Benesty, et al., “Differential Beamforming,” Fundamentals of Signal Enhancement and Array Signal Processing, First Edition, 2017, 39 pp.
Benesty, et al., “Frequency-Domain Adaptive Filtering Revisited, Generalization to the Multi-Channel Case, and Application to Acoustic Echo Cancellation,” 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing Proceedings, Jun. 2000, pp. 789-792.
Benesty, et. al., “Microphone Array Signal Processing,” Springer, 2010, 20 pp.
Berkun, et al., “Combined Beamformers for Robust Broadband Regularized Superdirective Beamforming,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 23, No. 5, May 2015, 10 pp.
Beyer Dynamic, Classis BM 32-33-34 DE-EN-FR 2016, 1 pg.
Beyer Dynamic, Classis-BM-33-PZ A1, 1 pg.
BNO055, Intelligent 9-axis absolute orientation sensor, Data sheet, Bosch, Nov. 2020, 118 pp.
Boyd, et al., Convex Optimization, Mar. 15, 1999, 216 pgs.
Brandstein, et al., “Microphone Arrays: Signal Processing Techniques and Applications,” Digital Signal Processing, Springer-Verlag Berlin Heidelberg, 2001, 401 pgs.
Brooks, et al., “A Quantitative Assessment of Group Delay Methods for Identifying Glottal Closures in Voiced Speech,” IEEE Transaction on Audio, Speech, and Language Processing, vol. 14, No. 2, Mar. 2006, 11 pp.
Bruel & Kjaer, by J.J. Christensen and J. Hald, Technical Review: Beamforming, No. 1, 2004, 54 pgs.
BSS Audio, Soundweb London Application Guides, 2010, 120 pgs.
Buchner, et al., “An Acoustic Human-Machine Interface with Multi-Channel Sound Reproduction,” IEEE Fourth Workshop on Multimedia Signal Processing, Oct. 2001, pp. 359-364.
Buchner, et al., “An Efficient Combination of Multi-Channel Acoustic Echo Cancellation with a Beamforming Microphone Array,” International Workshop on Hands-Free Speech Communication (HSC2001), Apr. 2001, pp. 55-58.
Buchner, et al., “Full-Duplex Communication Systems Using Loudspeaker Arrays and Microphone Arrays,” IEEE International Conference on Multimedia and Expo, Aug. 2002, pp. 509-512.
Buchner, et al., “Generalized Multichannel Frequency-Domain Adaptive Filtering: Efficient Realization and Application to Hands-Free Speech Communication,” Signal Processing 85, 2005, pp. 549-570.
Buchner, et al., “Multichannel Frequency-Domain Adaptive Filtering with Application to Multichannel Acoustic Echo Cancellation,” Adaptive Signal Processing, 2003, pp. 95-128.
Buck, “Aspects of First-Order Differential Microphone Arrays in the Presence of Sensor Imperfections,” Transactions on Emerging Telecommunications Technologies, 13.2, 2002, 8 pp.
Buck, et al., “First Order Differential Microphone Arrays for Automotive Applications,” 7th International Workshop on Acoustic Echo and Noise Control, Darmstadt University of Technology, Sep. 10-13, 2001, 4 pp.
Buck, et al., “Self-Calibrating Microphone Arrays for Speech Signal Acquisition: A Systematic Approach,” Signal Processing, vol. 86, 2006, pp. 1230-1238.
Burton, et al., “A New Structure for Combining Echo Cancellation and Beamforming in Changing Acoustical Environments,” IEEE International Conference on Acoustics, Speech and Signal Processing, 2007, pp. 1-77-1-80.
BZ-3a Installation Instructions, XEDIT Corporation, Available at <chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/viewer.html?pdfurl=https%3A%2F%2Fwww.servoreelers.com%2Fmt-content%2Fuploads%2F2017%2F05%2Fbz-a-3universal-2017c.pdf&clen=189067&chunk=true>, 1 p.
Cabral, et al., Glottal Spectral Separation for Speech Synthesis, IEEE Journal of Selected Topics in Signal Processing, 2013, 15 pp.
Campbell, “Adaptive Beamforming Using a Microphone Array for Hands-Free Telephony,” Virginia Polytechnic Institute and State University, Feb. 1999, 154 pgs.
Canetto, et al., “Speech Enhancement Systems Based on Microphone Arrays,” VI Conference of the Italian Society for Applied and Industrial Mathematics, May 27, 2002, 9 pp.
Cao, “Survey on Acoustic Vector Sensor and its Applications in Signal Processing” Proceedings of the 33rd Chinese Control Conference, Jul. 2014, 17 pp.
Cech, et al., “Active-Speaker Detection and Localization with Microphones and Cameras Embedded into a Robotic Head,” IEEE-RAS International Conference on Humanoid Robots, Oct. 2013, pp. 203-210.
Chan, et al., “Uniform Concentric Circular Arrays with Frequency-Invariant Characteristics-Theory, Design, Adaptive Beamforming and DOA Estimation,” IEEE Transactions on Signal Processing, vol. 55, No. 1, Jan. 2007, pp. 165-177.
Chau, et al., “A Subband Beamformer on an Ultra Low-Power Miniature DSP Platform,” 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, 4 pp.
Chen, et al., “A General Approach to the Design and Implementation of Linear Differential Microphone Arrays,” Signal and Information Processing Association Annual Summit and Conference, 2013 Asia-Pacific, IEEE, 7 pp.
Chen, et al., “Design and Implementation of Small Microphone Arrays,” PowerPoint Presentation, Northwestern Polytechnical University and Institut national de la recherche scientifique, Jan. 1, 2014, 56 pp.
Chen, et al., “Design of Robust Broadband Beamformers with Passband Shaping Characteristics using Tikhonov Regularization,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 17, No. 4, May 2009, pp. 565-681.
Chou, “Frequency-Independent Beamformer with Low Response Error,” 1995 International Conference on Acoustics, Speech, and Signal Processing, pp. 2995-2998, May 9, 1995, 4 pp.
Chu, “Desktop Mic Array for Teleconferencing,” 1995 International Conference on Acoustics, Speech, and Signal Processing, May 1995, pp. 2999-3002.
Circuit Specialists webpage for an aluminum enclosure, available at <https://www.circuitspecialists.com/metal-instrument-enclosure-la7.html?otaid=gpl&gclid=EAlalQobChMl2JTw-Ynm6AlVgbblCh3F4QKuEAkYBiABEgJZMPD_BwE>, 3 pp.
ClearOne Introduces Ceiling Microphone Array With Built-In Dante Interface, Press Release; GlobeNewswire, Jan. 8, 2019, 2 pp.
ClearOne Launches Second Generation of its Groundbreaking Beamforming Microphone Array, Press Release, Acquire Media, Jun. 1, 2016, 2 pp.
ClearOne to Unveil Beamforming Microphone Array with Adaptive Steering and Next Generation Acoustic Echo Cancellation Technology, Press Release, InfoComm, Jun. 4, 2012, 1 p.
Clearone, Clearly Speaking Blog, “Advanced Beamforming Microphone Array Technology for Corporate Conferencing Systems,” Nov. 11, 2013, 5 pp., http://www.clearone.com/blog/advanced-beamforming-microphone-array-technology-for-corporate-conferencing-systems/.
Clearone, Beamforming Microphone Array, Mar. 2012, 6 pgs.
Clearone, Ceiling Microphone Array Installation Manual, Jan. 9, 2012, 20 pgs.
Clearone, Converge/Converge Pro, Manual, 2008, 51 pp.
Clearone, Professional Conferencing Microphones, Brochure, Mar. 2015, 3 pp.
Coleman, “Loudspeaker Array Processing for Personal Sound Zone Reproduction,” Centre for Vision, Speech and Signal Processing, 2014, 239 pp.
Cook, et al., An Alternative Approach to Interpolated Array Processing for Uniform Circular Arrays, Asia-Pacific Conference on Circuits and Systems, 2002, pp. 411-414.
Cox, et al., “Robust Adaptive Beamforming,” IEEE Trans. Acoust., Speech, and Signal Processing, vol. ASSP-35, No. 10, Oct. 1987, pp. 1365-1376.
CTG Audio, Ceiling Microphone CTG CM-01, Jun. 5, 2008, 2 pgs.
CTG Audio, CM-01 & CM-02 Ceiling Microphones Specifications, 2 pgs.
CTG Audio, CM-01 & CM-02 Ceiling Microphones, 2017, 4 pgs.
CTG Audio, CTG FS-400 and RS-800 with “Beamforming” Technology, Datasheet, As early as 2009, 2 pp.
CTG Audio, CTG User Manual for the FS-400/800 Beamforming Mixers, Nov. 2008, 26 pp.
CTG Audio, Expand Your IP Teleconferencing to Full Room Audio, Obtained from website htt.)://www ct audio com/ex and-, our-i - teleconforencino-to-ful-room-audio-while-conquennc.1-echo-cancelation-issues Mull, 2014.
CTG Audio, Frequently Asked Questions, as early as 2009, 2 pp.
CTG Audio, Installation Manual and User Guidelines for the Soundman SM 02 System, May 2001, 29 pp.
CTG Audio, Installation Manual, Nov. 21, 2008, 25 pgs.
CTG Audio, Introducing the CTG FS-400 and FS-800 with Beamforming Technology, as early as 2008, 2 pp.
CTG Audio, Meeting the Demand for Ceiling Mics in the Enterprise 5 Best Practices, Brochure, 2012, 9 pp.
CTG Audio, White on White—Introducing the CM-02 Ceiling Microphone, https://ctgaudio.com/white-on-white-introducing-the-cm-02-ceiling-microphone/, Feb. 20, 2014, 3 pgs.
Dahl et al., Acoustic Echo Cancelling with Microphone Arrays, Research Report 3/95, Univ. of Karlskrona/Ronneby, Apr. 1995, 64 pgs.
Decawave, Application Note: APR001, UWB Regulations, A Summary of Worldwide Telecommunications Regulations governing the use of Ultra-Wideband radio, Version 1.2, 2015, 63 pp.
Desiraju, et al., “Efficient Multi-Channel Acoustic Echo Cancellation Using Constrained Sparse Filter Updates in the Subband Domain,” Acoustic Speech Enhancement Research, Sep. 2014, 4 pp.
DiBiase et al., Robust Localization in Reverberent Rooms, in Brandstein, ed., Microphone Arrays: Techniques and Applications, 2001, Springer-Verlag Berlin Heidelberg, pp. 157-180.
Diethorn, “Audio Signal Processing For Next-Generation Multimedia Communication Systems,” Chapter 4, 2004, 9 pp.
Digikey webpage for Converta box (last visited Apr. 22, 2020) <https://www.digikey.com/product-detail/en/bud-industries/CU-452-A/377-1969-ND/439257? utm_adgroup=Boxes&utm_source=google&utm_medium=cpc&utm_campaign=Shopping_Boxes%2C%20Enclosures%2C%20Racks_NEW&utm term=&utm_content=Boxes&gclid=EAlalQobChMI2JTw-Ynm6AIVgbblCh3F4QKuEAkYCSABEgKybPD_BwE>, 3 pp.
Digikey webpage for Pomona Box (last visited Apr. 22, 2020) available at <https://www.digikey.com/product-detail/en/pomonaelectronics/3306/501-2054-ND/736489>, 2 pp.
Digital Wireless Conference System, MCW-D 50, Beyerdynamic Inc., 2009, 18 pp.
Do et al., A Real-Time SRP-PHAT Source Location Implementation using Stochastic Region Contraction (SRC) on a Large-Aperture Microphone Array, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing—ICASSP '07, , Apr. 2007, pp. I-121-I-124.
Dominguez, et al., “Towards an Environmental Measurement Cloud: Delivering Pollution Awareness to the Public,” International Journal of Distributed Sensor Networks, vol. 10, Issue 3, Mar. 31, 2014, 17 pp.
Dormehl, “HoloLens concept lets you control your smart home via augmented reality,” digitaltrends, Jul. 26, 2016, 12 pp.
Double Condenser Microphone SM 69, Datasheet, Georg Neumann GmbH, available at <https://ende.neumann.com/product_files/7453/download>, 8 pp.
Eargle, “The Microphone Handbook,” Elar Publ. Co., 1st ed., 1981, 4 pp.
Enright, Notes From Logan, June edition of Scanlines, Jun. 2009, 9 pp.
Fan, et al., “Localization Estimation of Sound Source by Microphones Array,” Procedia Engineering 7, 2010, pp. 312-317.
Firoozabadi, et al., “Combination of Nested Microphone Array and Subband Processing for Multiple Simultaneous Speaker Localization,” 6th International Symposium on Telecommunications, Nov. 2012, pp. 907-912.
Flanagan et al., Autodirective Microphone Systems, Acustica, vol. 73, 1991, pp. 58-71.
Flanagan, et al., “Computer-Steered Microphone Arrays for Sound Transduction in Large Rooms,” J. Acoust. Soc. Am. 78 (5), Nov. 1985, pp. 1508-1518.
Fohhn Audio New Generation of Beam Steering Systems Available Now, audioXpress Staff, May 10, 2017, 8 pp.
Fox, et al., “A Subband Hybrid Beamforming for In-Car Speech Enhancement,” 20th European Signal rocessing Conference, Aug. 2012, 5 pp.
Frost, III, An Algorithm for Linearly Constrained Adaptive Array Processing, Proc. IEEE, vol. 60, No. 8, Aug. 1972, pp. 926-935.
Gannot et al., Signal Enhancement using Beamforming and Nonstationarity with Applications to Speech, IEEE Trans. on Signal Processing, vol. 49, No. 8, Aug. 2001, pp. 1614-1626.
Gansler et al., A Double-Talk Detector Based on Coherence, IEEE Transactions on Communications, vol. 44, No. 11, Nov. 1996, pp. 1421-1427.
Gazor et al., Robust Adaptive Beamforming via Target Tracking, IEEE Transactions on Signal Processing, vol. 44, No. 6, Jun. 1996, pp. 1589-1593.
Gazor et al., Wideband Multi-Source Beamforming with Adaptive Array Location Calibration and Direction Finding, 1995 International Conference on Acoustics, Speech, and Signal Processing, May 1995, pp. 1904-1907.
Gentner Communications Corp., AP400 Audio Perfect 400 Audioconferencing System Installation & Operation Manual, Nov. 1998, 80 pgs.
Gentner Communications Corp., XAP 800 Audio Conferencing System Installation & Operation Manual, Oct. 2001, 152 pgs.
Gil-Cacho et al., Multi-Microphone Acoustic Echo Cancellation Using Multi-Channel Warped Linear Prediction of Common Acoustical Poles, 18th European Signal Processing Conference, Aug. 2010, pp. 2121-2125.
Giuliani, et al., “Use of Different Microphone Array Configurations for Hands-Free Speech Recognition in Noisy and Reverberant Environment,” IRST-Istituto per la Ricerca Scientifica e Tecnologica, Sep. 22, 1997, 4 pp.
Gritton et al., Echo Cancellation Algorithms, IEEE ASSP Magazine, vol. 1, issue 2, Apr. 1984, pp. 30-38.
Hald, et al., “A class of optimal broadband phased array geometries designed for easy construction,” 2002 Int'l Congress & Expo. on Noise Control Engineering, Aug. 2002, 6 pp.
Hamalainen, et al., “Acoustic Echo Cancellation for Dynamically Steered Microphone Array Systems,” 2007 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, Oct. 2007, pp. 58-61.
Hayo, Virtual Controls for Real Life, Web page downloaded from https://hayo.io/ on Sep. 18, 2019, 19 pp.
Herbordt et al., A Real-time Acoustic Human-Machine Front-End for Multimedia Applications Integrating Robust Adaptive Beamforrning and Stereophonic Acoustic Echo Cancellation, 7th International Conference on Spoken Language Processing, Sep. 2002, 4 pgs.
Herbordt et al., GSAEC—Acoustic Echo Cancellation embedded into the Generalized Sidelobe Canceller, 10th European Signal Processing Conference, Sep. 2000, 5 pgs.
Herbordt et al., Multichannel Bin-Wise Robust Freguency-Domain Adaptive Filtering and Its Application to Adaptive Beamforming, IEEE Transactions on Audio, Speech, and Language Processing, vol. 15, No. 4, May 2007, pp. 1340-1351.
Herbordt, “Combination of Robust Adaptive Beamforming with Acoustic Echo Cancellation for Acoustic Human/Machine Interfaces,” Friedrich-Alexander University, 2003, 293 pgs.
Herbordt, et al., Joint Optimization of LCMV Beamforming and Acoustic Echo Cancellation for Automatic Speech Recognition, IEEE International Conference on Acoustics, Speech, and Signal Processing, Mar. 2005, pp. III-77-III-80.
Holm, “Optimizing Microphone Arrays for use in Conference Halls,” Norwegian University of Science and Technology, Jun. 2009, 101 pp.
Huang et al., Immersive Audio Schemes: The Evolution of Multiparty Teleconferencing, IEEE Signal Processing Magazine, Jan. 2011, pp. 20-32.
ICONYX Gen5, Product Overview; Renkus-Heinz, Dec. 24, 2018, 2 pp.
International Search Report and Written Opinion for PCT/US2016/022773 dated Jun. 10, 2016.
International Search Report and Written Opinion for PCT/US2016/029751 dated Nov. 28, 2016, 21 pp.
International Search Report and Written Opinion for PCT/US2018/013155 dated Jun. 8, 2018.
International Search Report and Written Opinion for PCT/US2019/031833 dated Jul. 24, 2019, 16 pp.
International Search Report and Written Opinion for PCT/US2019/033470 dated Jul. 31, 2019, 12 pp.
International Search Report and Written Opinion for PCT/US2019/051989 dated Jan. 10, 2020, 15 pp.
International Search Report and Written Opinion for PCT/US2020/035185 dated Sep. 15, 2020, 11 pp.
International Search Report and Written Opinion for PCT/US2020/058385 dated Mar. 31, 2021, 20 pp.
International Search Report and Written Opinion for PCT/US2021/070625 dated Sep. 17, 2021, 17 pp.
International Search Report for PCT/US2020/024005 dated Jun. 12, 2020, 12 pp.
Invensense, “Microphone Array Beamforming,” Application Note AN-1140, Dec. 31, 2013, 12 pp.
Invensense, Recommendations for Mounting and Connecting InvenSense MEMS Microphones, Application Note AN-1003, 2013, 11 pp.
Ishii et al., Investigation on Sound Localization using Multiple Microphone Arrays, Reflection and Spatial Information, Japanese Society for Artificial Intelligence, JSAI Technical Report, SIG-Challenge-B202-11, 2012, pp. 64-69.
Ito et al., Aerodynamic/Aeroacoustic Testing in Anechoic Closed Test Sections of Low-speed Wind Tunnels, 16th AIAA/CEAS Aeroacoustics Conference, 2010, 11 pgs.
Johansson et al., Robust Acoustic Direction of Arrival Estimation using Root-SRP-PHAT, a Realtime Implementation, IEEE International Conference on Acoustics, Speech, and Signal Processing, Mar. 2005, 4 pgs.
Johansson, et al., Speaker Localisation using the Far-Field SRP-PHAT in Conference Telephony, 2002 International Symposium on Intelligent Signal Processing and Communication Systems, 5 pgs.
Johnson, et al., “Array Signal Processing: Concepts and Techniques,” p. 59, Prentice Hall, 1993, 3 pp.
Julstrom et al., Direction-Sensitive Gating: A New Approach to Automatic Mixing, J. Audio Eng. Soc., vol. 32, No. 7/8, Jul./Aug. 1984, pp. 490-506.
Kahrs, Ed., The Past, Present, and Future of Audio Signal Processing, IEEE Signal Processing Magazine, Sep. 1997, pp. 30-57.
Kallinger et al., Multi-Microphone Residual Echo Estimation, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, Apr. 2003, 4 pgs.
Kammeyer, et al., New Aspects of Combining Echo Cancellers with Beamformers, IEEE International Conference on Acoustics, Speech, and Signal Processing, Mar. 2005, pp. III-137-III-140.
Kellermann, A Self-Steering Digital Microphone Array, 1991 International Conference on Acoustics, Speech, and Signal Processing, Apr. 1991, pp. 3581-3584.
Kellermann, Acoustic Echo Cancellation for Beamforming Microphone Arrays, in Brandstein, ed., Microphone Arrays: Techniques and Applications, 2001, Springer-Verlag Berlin Heidelberg, pp. 281-306.
Kellermann, Integrating Acoustic Echo Cancellation with Adaptive Beamforming Microphone Arrays, Forum Acusticum, Berlin, Mar. 1999, pp. 1-4.
Kellermann, Strategies for Combining Acoustic Echo Cancellation and Adaptive Beamforming Microphone Arrays, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing, Apr. 1997, 4 pgs.
Klegon, “Achieve Invisible Audio with the MXA910 Ceiling Array Microphone,” Jun. 27, 2016, 10 pp.
Knapp, et al., The Generalized Correlation Method for Estimation of Time Delay, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. ASSP-24, No. 4, Aug. 1976, pp. 320-327.
Kobayashi et al., A Hands-Free Unit with Noise Reduction by Using Adaptive Beamformer, IEEE Transactions on Consumer Electronics, vol. 54, No. 1, Feb. 2008, pp. 116-122.
Kobayashi et al., A Microphone Array System with Echo Canceller, Electronics and Communications in Japan, Part 3, vol. 89, No. 10, Feb. 2, 2006, pp. 23-32.
Kolund{hacek over (z)}ija, et al., “Baffled circular loudspeaker array with broadband high directivity,” 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, Dallas, TX, 2010, pp. 73-76.
Lai, et al., “Design of Robust Steerable Broadband Beamformers with Spiral Arrays and the Farrow Filter Structure,” Proc. Intl. Workshop Acoustic Echo Noise Control, 2010, 4 pp.
Lebret, et al., Antenna Array Pattern Synthesis via Convex Cptimization, IEEE Trans. on Signal Processing, vol. 45, No. 3, Mar. 1997, pp. 526-532.
LecNet2 Sound System Design Guide, Lectrosonics, Jun. 2, 2006.
Lectrosonics, LecNet2 Sound System Design Guide, Jun. 2006, 28 pgs.
Lee et al., Multichannel Teleconferencing System with Multispatial Region Acoustic Echo Cancellation, International Workshop on Acoustic Echo and Noise Control (IWAENC2003), Sep. 2003, pp. 51-54.
Li, “Broadband Beamforming and Direction Finding Using Concentric Ring Array,” Ph.D. Dissertation, University of Missouri-Columbia, Jul. 2005, 163 pp.
Lindstrom et al., An Improvement of the Two-Path Algorithm Transfer Logic for Acoustic Echo Cancellation, IEEE Transactions on Audio, Speech, and Language Processing, vol. 15, No. 4, May 2007, pp. 1320-1326.
Liu et al., Adaptive Beamforming with Sidelobe Control: A Second-Order Cone Programming Approach, IEEE Signal Proc. Letters, vol. 10, No. 11, Nov. 2003, pp. 331-334.
Liu, et al., “Frequency Invariant Beamforming in Subbands,” IEEE Conference on Signals, Systems and Computers, 2004, 5 pp.
Liu, et al., “Wideband Beamforming,” Wiley Series on Wireless Communications and Mobile Computing, pp. 143-198, 2010, 297 pp.
Lobo, et al., Applications of Second-Order Cone Programming, Linear Algebra and its Applications 284, 1998, pp. 193-228.
Luo et al., Wideband Beamforming with Broad Nulls of Nested Array, Third Int'l Conf. on Info. Science and Tech., Mar. 23-25, 2013, pp. 1645-1648.
Marquardt et al., A Natural Acoustic Front-End for Interactive TV in the EU-Project DICIT, IEEE Pacific Rim Conference on Communications, Computers and Signal Processing, Aug. 2009, pp. 894-899.
Martin, Small Microphone Arrays with Postfilters for Noise and Acoustic Echo Reduction, in Brandstein, ed., Microphone Arrays: Techniques and Applications, 2001, Springer-Verlag Berlin Heidelberg, pp. 255-279.
Maruo et al., On the Optimal Solutions of Beamformer Assisted Acoustic Echo Cancellers, IEEE Statistical Signal Processing Workshop, 2011, pp. 641-644.
McCowan, Microphone Arrays: A Tutorial, Apr. 2001, 36 pgs.
MFLCRFG Datasheet, Metal_Fab Inc., Sep. 7, 2007, 1 p.
Microphone Array Primer, Shure Question and Answer Page, <https://service.shure.com/s/article/microphone-array-primer?language=en_US>, Jan. 2019, 5 pp.
Milanovic, et al., “Design and Realization of FPGA Platform for Real Time Acoustic Signal Acquisition and Data Processing” 22nd Telecommunications Forum TELFOR, 2014, 6 pp.
Mohammed, A New Adaptive Beamformer for Optimal Acoustic Echo and Noise Cancellation with Less Computational Load, Canadian Conference on Electrical and Computer Engineering, May 2008, pp. 000123-000128.
Mohammed, A New Robust Adaptive Beamformer for Enhancing Speech Corrupted with Colored Noise, AICCSA, Apr. 2008, pp. 508-515.
Mohammed, Real-time Implementation of an efficient RLS Algorithm based on IIR Filter for Acoustic Echo Cancellation, AICCSA, Apr. 2008, pp. 489-494.
Mohan, et al., “Localization of multiple acoustic sources with small arrays using a coherence test,” Journal Acoustic Soc Am., 123(4), Apr. 2008, 12 pp.
Moulines, et al., “Pitch-Synchronous Waveform Processing Techniques for Text-to-Speech Synthesis Using Diphones,” Speech Communication 9, 1990, 15 pp.
Multichannel Acoustic Echo Cancellation, Obtained from website http://www.buchner-net.com/mcaec.html, Jun. 2011.
Myllyla et al., Adaptive Beamforming Methods for Dynamically Steered Microphone Array Systems, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, Mar.-Apr. 2008, pp. 305-308.
New Shure Microflex Advance MXA910 Microphone With Intellimix Audio Processing Provides Greater Simplicity, Flexibility, Clarity, Press Release, Jun. 12, 2019, 4 pp.
Nguyen-Ky, et al., “An Improved Error Estimation Algorithm for Stereophonic Acoustic Echo Cancellation Systems,” 1st International Conference on Signal Processing and Communication Systems, Dec. 17-19, 2007, 5 pp.
Office Action for Taiwan Patent Application No. 105109900 dated May 5, 2017.
Office Action issued for Japanese Patent Application No. 2015-023781 dated Jun. 20, 2016, 4 pp.
Oh, et al., “Hands-Free Voice Communication in an Automobile With a Microphone Array,” 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing, Mar. 1992, pp. I-281-I-284.
Olszewski, et al., “Steerable Highly Directional Audio Beam Loudspeaker,” Interspeech 2005, 4 pp.
Omologo, Multi-Microphone Signal Processing for Distant-Speech Interaction, Human Activity and Vision Summer School (HAVSS), INRIA Sophia Antipolis, Oct. 3, 2012, 79 pgs.
Order, Conduct of the Proceeding, Clearone, Inc. v. Shure Acquisition Holdings, Inc., Nov. 2, 2020, 10 pp.
Pados et al., An Iterative Algorithm for the Computation of the MVDR Filter, IEEE Trans. on Signal Processing, vol. 49, No. 2, Feb. 2001, pp. 290-300.
Palladino, “This App Lets You Control Your Smarthome Lights via Augmented Reality,” Next Reality Mobile AR News, Jul. 2, 2018, 5 pp.
Parikh, et al., “Methods for Mitigating IP Network Packet Loss in Real Time Audio Streaming Applications,” GatesAir, 2014, 6 pp.
Pasha, et al., “Clustered Multi-channel Dereverberation for Ad-hoc Microphone Arrays,” Proceedings of APSIPA Annual Summit and Conference, Dec. 2015, pp. 274-278.
Petitioner's Motion for Sanctions, Clearone, Inc. v. Shure Acquisition Holdings, Inc., Aug. 24, 2020, 20 pp.
Pettersen, “Broadcast Applications for Voice-Activated Microphones,” db, Jul./Aug. 1985, 6 pgs.
Pfeifenberger, et al., “Nonlinear Residual Echo Suppression using a Recurrent Neural Network,” Interspeech 2020, 5 pp.
Phoenix Audio Technologies, “Beamforming and Microphone Arrays—Common Myths”, Apr. 2016, http://info.phnxaudio.com/blog/microphone-arrays-beamforming-myths-1, 19 pp.
Plascore, PCGA-XR1 3003 Aluminum Honeycomb Data Sheet, 2008, 2 pgs.
Polycom Inc., Vortex EF2211/EF2210 Reference Manual, 2003, 66 pgs.
Polycom, Inc., Polycom Soundstructure C16, C12, C8, and SR12 Design Guide, Nov. 2013, 743 pgs.
Polycom, Inc., Setting Up the Polycom HDX Ceiling Microphone Array Series, https://support.polycom.com/content/dam/polycom-support/products/Telepresence-and-Video/HDX%20Series/setup-maintenance/en/hdx_ceiling_microphone_array_setting_up.pdf, 2010, 16 pgs.
Polycom, Inc., Vortex EF2241 Reference Manual, 2002, 68 pgs.
Polycom, Inc., Vortex EF2280 Reference Manual, 2001, 60 pp.
Pomona, Model 3306, Datasheet, Jun. 9, 1999, 1 p.
Powers, et al., “Proving Adaptive Directional Technology Works: A Review of Studies,” The Hearing Review, Apr. 6, 2004, 5 pp.
Prime, et al., “Beamforming Array Optimisation Averaged Sound Source Mapping on a Model Wind Turbine,” ResearchGate, Nov. 2014, 10 pp.
Rabinkin et al., Estimation of Wavefront Arrival Delay Using the Cross-Power Spectrum Phase Technique, 132nd Meeting of the Acoustical Society of America, Dec. 1996, pp. 1-10.
Rane Corp., Halogen Acoustic Echo Cancellation Guide, AEC Guide Version 2, Nov. 2013, 16 pgs.
Rao, et al., “Fast LMS/Newton Algorithms for Stereophonic Acoustic Echo Cancelation,” IEEE Transactions on Signal Processing, vol. 57, No. 8, Aug. 2009.
Reuven et al., Joint Acoustic Echo Cancellation and Transfer Function GSC in the Frequency Domain, 23rd IEEE Convention of Electrical and Electronics Engineers in Israel, Sep. 2004, pp. 412-415.
Reuven et al., Joint Noise Reduction and Acoustic Echo Cancellation Using the Transfer-Function Generalized Sidelobe Canceller, Speech Communication, vol. 49, 2007, pp. 623-635.
Reuven, et al., “Multichannel Acoustic Echo Cancellation and Noise Reduction in Reverberant Environments Using the Transfer-Function GSC,” 2007 IEEE International Conference on Acoustics, Speech and Signal Processing, Apr. 2007, 4 pp.
Ristimaki, Distributed Microphone Array System for Two-Way Audio Communication, Helsinki Univ. of Technology, Master's Thesis, Jun. 15, 2009, 73 pgs.
Rombouts et al., An Integrated Approach to Acoustic Noise and Echo Cancellation, Signal Processing 85, 2005, pp. 849-871.
Sällberg, “Faster Subband Signal Processing,” IEEE Signal Processing Magazine, vol. 30, No. 5, Sep. 2013, 6 pp.
Sasaki et al., A Predefined Command Recognition System Using a Ceiling Microphone Array in Noisy Housing Environments, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep. 2008, pp. 2178-2184.
Sennheiser, New microphone solutions for ceiling and desk installation, https://en-us.sennheiser.com/news-new-microphone-solutions-for-ceiling-and-desk-installation, Feb. 2011, 2 pgs.
Sennheiser, TeamConnect Ceiling, https://en-us.sennheiser.com/conference-meeting-rooms-teamconnect-ceiling, 7 pgs.
SerDes, Wikipedia article, last edited on Jun. 25, 2018: retrieved on Jun. 27, 2018, 3 pp., https://en.wikipedia.org/wiki/SerDes.
Sessler, et al., “Directional Transducers,” IEEE Transactions on Audio and Electroacoustics, vol. AU-19, No. 1, Mar. 1971, pp. 19-23.
Sessler, et al., “Toroidal Microphones,” Journal of Acoustical Society of America, vol. 46, No. 1, 1969, 10 pp.
Shure AMS Update, vol. 1, No. 1, 1983, 2 pgs.
Shure AMS Update, vol. 1, No. 2, 1983, 2 pgs.
Shure AMS Update, vol. 4, No. 4, 1997, 8 pgs.
Shure Debuts Microflex Advance Ceiling and Table Array Microphones, Press Release, Feb. 9, 2016, 4 pp.
Shure Inc., A910-HCM Hard Ceiling Mount, retrieved from website <http://www.shure.com/en-US/products/accessories/a910hcm> on Jan. 16, 2020, 3 pp.
Shure Inc., Microflex Advance, http://www.shure.com/americas/microflex-advance, 12 pgs.
Shure Inc., MX395 Low Profile Boundary Microphones, 2007, 2 pgs.
Shure Inc., MXA910 Ceiling Array Microphone, http://www.shure.com/americas/products/microphones/microflex-advance/mxa910-ceiling-array-microphone, 7 pgs.
Shure, MXA910 With IntelliMix, Ceiling Array Microphone, available at <https://www.shure.com/en-US/products/microphones/mxa910>, as early as 2020, 12 pp.
Shure, New MXA910 Variant Now Available, Press Release, Dec. 13, 2019, 5 pp.
Shure, Q&A in Response to Recent US Court Ruling on Shure MXA910, Available at <https://www.shure.com/en-US/meta/legal/q-and-a-inresponse-to-recent-us-court-ruling-on-shure-mxa910-response>, As early as 2020, 5 pp.
Shure, RK244G Replacement Screen and Grille, Datasheet, 2013, 1 p.
Shure, The Microflex Advance MXA310 Table Array Microphone, Available at <https://www.shure.com/en-US/products/microphones/mxa310>, as early as 2020, 12 pp.
Signal Processor MRX7-D Product Specifications, Yamaha Corporation, 2016.
Silverman et al., Performance of Real-Time Source-Location Estimators for a Large-Aperture Microphone Array, IEEE Transactions on Speech and Audio Processing, vol. 13, No. 4, Jul. 2005, pp. 593-606.
Sinha, Ch. 9: Noise and Echo Cancellation, in Speech Processing in Embedded Systems, Springer, 2010, pp. 127-142.
SM 69 Stereo Microphone, Datasheet, Georg Neumann GmbH, Available at <https://ende.neumann.com/product_files/6552/download>, 1 p.
Soda et al., Introducing Multiple Microphone Arrays for Enhancing Smart Home Voice Control, The Institute of Electronics, Information and Communication Engineers, Technical Report of IEICE, Jan. 2013, 6 pgs.
Soundweb London Application Guides, BSS Audio, 2010.
Symetrix, Inc., SymNet Network Audio Solutions Brochure, 2008, 32 pgs.
SymNet Network Audio Solutions Brochure, Symetrix, Inc., 2008.
Tan, et al., “Pitch Detection Algorithm: Autocorrelation Method and AMDF,” Department of Computer Engineering, Prince of Songkhla University, Jan. 2003, 6 pp.
Tandon, et al., “An Efficient, Low-Complexity, Normalized LMS Algorithm for Echo Cancellation,” 2nd Annual IEEE Northeast Workshop on Circuits and Systems, Jun. 2004, pp. 161-164.
Tetelbaum et al., Design and Implementation of a Conference Phone Based on Microphone Array Technology, Proc. Global Signal Processing Conference and Expo (GSPx), Sep. 2004, 6 pgs.
Tiete et al., SoundCompass: A Distributed MEMS Microphone Array-Based Sensor for Sound Source Localization, SENSORS, Jan. 23, 2014, pp. 1918-1949.
TOA Corp., Ceiling Mount Microphone AN-9001 Operating Instructions, http://www.toaelectronics.com/media/an9001_mt1e.pdf, 1 pg.
Togami, et al., “Subband Beamformer Combined with Time-Frequency ICA for Extraction of Target Source Under Reverberant Environments,” 17th European Signal Processing Conference, Aug. 2009, 5 pp.
U.S. Appl. No. 16/598,918, filed Oct. 10, 2019, 50 pp.
Van Compernolle, Switching Adaptive Filters for Enhancing Noisy and Reverberant Speech from Microphone Array Recordings, Proc. IEEE Inf. Conf. on Acoustics, Speech, and Signal Processing, Apr. 1990, pp. 833-836.
Van Trees, Optimum Array Processing: Part IV of Detection, Estimation, and Modulation Theory, 2002, 54 pgs., pp. i-xxv, 90-95, 201-230.
Van Veen et al., Beamforming: A Versatile Approach to Spatial Filtering, IEEE ASSP Magazine, vol. 5, issue 2, Apr. 1988, pp. 4-24.
Vicente, “Adaptive Array Signal Processing Using the Concentric Ring Array and the Spherical Array,” Ph.D. Dissertation, University of Missouri, May 2009, 226 pp.
Wang et al., Combining Superdirective Beamforming and Frequency-Domain Blind Source Separation for Highly Reverberant Signals, EURASIP Journal on Audio, Speech, and Music Processing, vol. 2010, pp. 1-13.
Warsitz, et al., “Blind Acoustic Beamforming Based on Generalized Eigenvalue Decomposition,” IEEE Transactions on Audio, Speech and Language Processing, vol. 15, No. 5, 2007, 11 pp.
Weinstein, et al., “Loud: A 1020-Node Microphone Array and Acoustic Beamformer,” 14th International Congress on Sound & Vibration, Jul. 2007, 8 pgs.
Weinstein, et al., “LOUD: A 1020-Node Modular Microphone Array and Beamformer for Intelligent Computing Spaces,” MIT Computer Science and Artifical Intelligence Laboratory, 2004, 18 pp.
Wung, “A System Approach to Multi-Channel Acoustic Echo Cancellation and Residual Echo Suppression for Robust Hands-Free Teleconferencing,” Georgia Institute of Technology, May 2015, 167 pp.
XAP Audio Conferencing Brochure, ClearOne Communications, Inc., 2002.
Yamaha Corp., MRX7-D Signal Processor Product Specifications, 2016, 12 pgs.
Yamaha Corp., PJP-100H IP Audio Conference System Owner's Manual, Sep. 2006, 59 pgs.
Yamaha Corp., PJP-EC200 Conference Echo Canceller Brochure, Oct. 2009, 2 pgs.
Yan et al., Convex Optimization Based Time-Domain Broadband Beamforming with Sidelobe Control, Journal of the Acoustical Society of America, vol. 121, No. 1, Jan. 2007, pp. 46-49.
Yensen et al., Synthetic Stereo Acoustic Echo Cancellation Structure with Microphone Array Beamforming for VOIP Conferences, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing, Jun. 2000, pp. 817-820.
Yermeche, et al., “Real-Time DSP Implementation of a Subband Beamforming Algorithm for Dual Microphone Speech Enhancement,” 2007 IEEE International Symposium on Circuits and Systems, 4 pp.
Zavarehei, et al., “Interpolation of Lost Speech Segments Using LP-HNM Model with Codebook Post-Processing,” IEEE Transactions on Multimedia, vol. 10, No. 3, Apr. 2008, 10 pp.
Zhang, et al., “F-T-LSTM based Complex Network for Joint Acoustic Echo Cancellation and Speech Enhancement,” Audio, Speech and Language Processing Group, Jun. 2021, 5 pp.
Zhang, et al., “Multichannel Acoustic Echo Cancelation in Multiparty Spatial Audio Conferencing with Constrained Kalman Filtering,” 11th International Workshop on Acoustic Echo and Noise Control, Sep. 14, 2008, 4 pp.
Zhang, et al., “Selective Frequency Invariant Uniform Circular Broadband Beamformer,” EURASIP Journal on Advances in Signal Processing, vol. 2010, pp. 1-11.
Zheng, et al., “Experimental Evaluation of a Nested Microphone Array With Adaptive Noise Cancellers,” IEEE Transactions on Instrumentation and Measurement, vol. 53, No. 3, Jun. 2004, 10 pp.

Related Publications (1)

	Number	Date	Country
	20210051397 A1	Feb 2021	US

Provisional Applications (3)

Number	Date	Country
62971648	Feb 2020	US
62855187	May 2019	US
62821800	Mar 2019	US

Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition functionality

Information

Patent Number

Date Filed

Date Issued

Inventors

Original Assignees

Examiners

Agents

CPC

Field of Search

CPC

International Classifications

Abstract