Patent Grant
7742609
The present invention relates to audio mixing systems. More particularly, the present invention pertains to audio mixing consoles and mixing systems for use in live performance applications.
Audio mixing consoles are used to control and adjust the audio characteristics and sound mix of audio signals generated by musical instruments, microphones, and like, as perceived by listeners at live audio performances. In recent years, analog mixing consoles (sometimes referred to simply as “mixers”) used in live performance applications have been supplanted by digital mixers. However, one of the single biggest flaws with conventional digital mixers is that their user interfaces resemble their older analog predecessors. For example, analog mixers use large arrays of mechanical and electromechanical knobs and faders to allow the console operators to individually adjust the audio characteristics associated with multiple audio sources and channels. Such arrays are simply not necessary for a digital mixing product but their use has not been entirely abandoned. With conventional digital mixer user interfaces, an experienced audio professional is required to page through multiple layers of on-screen menus to locate the desired feature on the mixer. This experience can create even more frustration than operating a product containing dedicated adjustment hardware. In addition, conventional digital mixer interfaces are confusing and not intuitive such that to operate them efficiently one must have extensive training in interpreting the displayed menus.
As an example of the inefficiencies caused by extensive menu layering and confusing digital mixer nomenclature, a sound engineer at a live performance venue may notice that an on stage guitar monitor has excessive audible “boom” on the bass drum and that the vocal is buried in the audio mix. Using a conventional mixing system and user interface, the sound engineer has to understand and recall which sub-mix the guitar player is on (assuming the guitar player has the luxury of his own sub-mix). Further, the engineer has to recall from memory which mixer input is associated the bass drum. The engineer then has to find the low frequency EQ knob and turn it down, assuming this is possible without affecting the overall house mix. Also, the sound engineer has to remember where the vocals come in, how they are mixed into the sub-mix, and then turn them up but not so much as to cause feedback.
What is needed, then, is a digital audio mixing system for use in live performance applications that provides a more efficient and understandable user interface.
The audio mixing system of the present invention provides an elegant answer to the need for an efficient and user-friendly digital mixer and user interface for controlling audio associated with a live amplified performance. It provides a cost-effective solution to a problem mixing console designers have attempted to solve for years. The heart of the system is a powerful interface providing the most powerful digital mixer features controlled by a simple to use software front end.
In accordance with one embodiment of the invention, a system in accordance with the invention will include a software user interface, system host PC running on a WINDOWS-based operating system and with an internal digital signal processor (DSP) card to perform digital mixing functions. In accordance with another aspect of the invention, the system includes a system console having an array of multiple LCD touch screen displays and a fader board (tactile) control surface operatively connected to the host PC, and an audio patch bay unit. In a further embodiment of the system, one or more stage boxes are linked to each other and to the system host PC by wired or wireless connections. Each stage box and studio box contains a multi-channel analog audio interface, analog-to-digital converters, and a wired or wireless digital links to each other and to the system host PC. The stage boxes and studio boxes are functionally the same as the system fader board control surface and are used as interfaces to stage instruments, speakers, microphones, and the like (sometimes collectively referred to as stage elements).
The system provides an improved control interface by visually and functionally (in multiple functional views) abstracting the channel strips found in prior art mixing consoles. Accordingly, changing a variable in a mix is as simple as selecting the stage element audio source (instrument, microphone, or speaker) that the sound engineer wants to change, and then selecting the audio parameter associated with that stage element that needs adjustment. For example, using the example summarized above for conventional systems, the same problem can be handled by a sound engineer at a system console as follows: The engineer taps the icon of the guitar player's monitor speakers on the touch screen. He then selects “Select Bass Drum Mix List” and taps “Too Boomy”. Finally, the engineer selects “Vocal1” from the Mix List and taps “Buried”. This causes the software in the mixing system to implement the adjustments electronically, without the engineer having to scroll or page through layers of cryptic menus.
The stage portion of the system 10 will include one or more stage boxes 22 which are functionally equivalent to the console patch bay unit 20. In a preferred embodiment of the system 10, the system components are interconnected using a universal digital media communications link (hereinafter referred to as a “universal digital audio link”) such as that defined in the system and protocol introduced by Gibson Guitar Corporation and disclosed in U.S. Pat. No. 6,353,169 for a “Universal Audio Communications and Control System and Method”, the disclosure of which is fully incorporated herein by reference. Accordingly, the system 10 will include: a 64×32 channel mixer with full metering on all inputs and outputs; 64 compressors; 64 parametric equalizer (“EQ's”); plug-in insert effects; real-time total live-in to live-out latency of <3 ms with a single board configuration; and streaming audio to/from a hard disk on host PC 12.
As shown in more detail in
Positioned below, or otherwise visually and operatively associated with, each display 16 is a fader board tactile control surface 18 containing an array of motorized faders that reflect information shown on the displays 16. The individual faders electromechanically “snap to” the current settings reflected on the corresponding display 16. Manipulating the “real” faders on control surfaces 18 and touching the virtual controls on touch screen displays 16 causes console 14 to send mixing control signals to the host PC 12. The host PC and internal DSP use these mixing control signals to electronically interact, through patch bay unit 20, with the stage elements, i.e., the audio source and destination components, thereby affecting the “mix” or perceived sound coming from the audio components on stage (stage elements). The stage boxes 22 can provide operational connections to the stage elements as needed.
The system 10 of the invention can support 64 simultaneous inputs and 32 simultaneous outputs. Each output can have a custom mix of any or all of the inputs. Additionally, there may be “soft” inputs. A soft input can be an auxiliary return or track from the hard drive on host PC 12.
The host PC 12 and internal DSP are provided with software, including device drivers and Application Program Interface (API) modules to seamlessly integrate all needed mixing, recording, and DSP functions into the system 10. The actual writing of the software to implement these functions is conventional, as is the programming necessary to implement the novel user interface described herein.
The stage boxes 22 (and patch bay unit 20) are each a 16-channel in, 16-channel out, professional quality analog interface for the system 10. In addition to being able to function in a stand-alone mode, the stage box 22 uses a universal digital audio link to send audio up to 100 meters between units without signal loss. The stage box 22 includes advanced preamplifiers (not shown) that operate over a gain range of −60 dB to +15 dB. The analog trim can be remotely controlled via a universal digital audio link control link.
In addition to analog performance, the stage boxes 22 include analog-to-digital (A/D) converters that are capable of up to 24 bit, 96 kHz samples. Phantom power and hard pad can also be controlled remotely using a universal digital audio link. The system 10 can also be adapted for use with SPDIF and AES/EBU, and MIDI protocols and interfaces.
The system user interface is presented to a system user primarily as a series or combination of graphical interfaces presented on one or more touch screen displays 16. The user interface includes multiple functional “views” presented to the user in two modes—setup and real-time—including initial setup windows and dialogs, and real time operational interfaces, referred to herein as “stage view”, “virtual console view”, “mixer view”, and “cute view”. In addition, the user interface can optionally include a “drum editor view” for configuring an on-stage drum set.
First-time Setup
The setup mode of system 10 includes a setup process in which system input and output connections are made in the DSP architecture. This greatly simplifies the process of making connections and configuring the system DSP mixer. The result of this setup process will be a table of inputs and outputs with specific properties. User “friendly” names are assigned by the system user to each input, representing different stage elements. The table below reflects one example of a “virtual patch bay” table of inputs, friendly names, and input properties that is developed during system setup.
The user interface presented during system setup is similar but not identical to a conventional “wizard” type setup window so as to provide a familiar visual environment to the system user. A series of pop-up menus allows the user to configure connections in the patch bay unit 20.
The first set of system setup presets will toggle through basic stage setups. The system software is configured to generate and store input and output assignments as part of standard system stage configuration “presets.” Sample system setups and presets include “club”, “amphitheatre”, “church”, “lecture hall”, “multi-room” and “custom” as follows:
Club—This preset is defined by the basic configuration with the default setup being:
Amphitheatre—This preset is the same as Club, but with one additional musician, microphone, and monitor and with a larger stage.
Church
Lecture Hall
Multi-Room:—The multi-room stage view interface includes multiple visual boxes representing different rooms.
If there are two stage boxes 22 on a port, the stage box 22 that is farthest from the host PC 12 is called unit 1, and the one located between the host PC 12 and stage box unit 1 is referred as unit 2.
Show Setup
During system setup, the default settings are modified and initial input labels are assigned and placed. The user interface includes two types of “show” setups: Venue and Performance. The difference between Venue type and Performance type is that Venue type is designed to be setup once while a Performance setup is changed before each show. Also, custom configurations can be stored in this environment.
The following Venue and Performance types can be setup:
Band—This can be broken down to a group of presets, for example:
Theater—This is a setup for a play or similar presentation, and should include wireless microphone rigs, PZM microphones, and optional Pit Orchestra as stage elements.
Service
A church venue can be defined as a preset without having to be overly specific. Stage element inputs can include a wireless microphone, speakers 1 and 2, chorus and a several keyboard inputs.
Drums
Another novel feature of the system user interface and software is the drum editor. The drum editor is a hierarchical part of the information displayed on touch screen display 16. Because drums require many different configurations and inputs, the drum editor is loaded as a simple alternative to labeling generic inputs on individual drums. The default drum configuration is a 5 piece drum set. An example of a drum editor user interface display is shown in
The overhead drum set can be arranged to suit the type of set that is being used. Often a microphone is used to amplify several cymbals or drums. In the drum editor, only drums and cymbals with their own microphone are provided with a specific icon. Microphones used for multiple inputs use the Overhead (OH) icon.
Bass drum, tom-tom drum, snare drum, hats and OH each have different audio gains and equalization settings. Each icon should have displayed the gain and EQ associated with it.
Once the basic configuration of the stage is established, the user can see the selections made reflected on the stage view portion of the user interface, as shown in
System Software and User Interface Definition
As indicated above, the system 10 supports two modes: setup and real time. The setup mode requires use of only one of the touch screen displays 16 and a conventional mouse. The setup screen occupies all of one screen in a display 16. A standard menu bar is displayed at the top of the screen. The setup mode user interface is functionality organized by the following selections in the menu bar:
“Cute View” refers to a non-conventional view of a system configuration. The conventional view is implemented via “channel strips” as described under the real time section. The Cute View is always visible on one of the displays 16 (display/monitor #1) both in setup mode and in real time mode. (See
Icons in the Cute View can be dragged to any location with persistence. Double-clicking an icon in the Cute View brings up the source edit dialog (if the icon represents an audio source, such as a keyboard), or the destination edit dialog if the icon represents a destination, such as a monitor speaker.
As seen on
The configuration dialog allows editing of the following parameters:
The source edit dialog allows editing of the following parameters pertaining to audio source components as stage elements:
The destination edit dialog allows editing of the following parameters pertaining to destination audio components as stage elements:
The aux edit dialog allows editing of the following parameters:
Real time mode uses from one to four touch-screens 16. All screens can be operated by touch or mouse. Monitor #1 contains the Cute View, the Master Fader, and the Info Bar. All other displays/monitors contain conventional channel strips.
Cute View
In real time mode, the Cute View is available on display 16 #1. Referring to the setup mode definition, the following differences are noted:
The master fader 34 is a high-resolution fader that controls scaling of all output levels for all destinations. Beneath the fader is a toggle. Switching the toggle “on” enables stream to disk for all destination objects in which the stream to disk option is enabled.
Info Bar
The info bar 36 displays information about the currently selected object. If no object is selected, all of the objects are paged. The following information is shown:
Windows that open in real time are non-modal, though normally restricted to only one window that is associated with a particular object. Real-time windows have a toolbar in the top left corner. Some real-time windows have custom tools in the toolbar, but all of them share the following tools:
Source real time windows have the following components:
The discrete level window has a fader that controls the mix level for each output to which this source is connected. Each fader is labeled with the instance name of the output, (or aux A, B, or C). Above each fader is an animated VU and margin for the connection. If the output mix levels for the associated source were determined using the Pan Control Window, and any of the faders are moved, the pan control icon reverts to displaying the word “Discrete”.
Pan Control Windows
The pan control window 38 contains a grid with meaningless tick spacing. It graphically illustrates the location of all destinations of type “house”, as represented in the Cute View. The grid also illustrates a virtual location for the associated audio source that can be dragged to any position by the user. The mix level for the source to any house destination is determined by the distance from the virtual source icon to the associated house destination icon.
Levels that are changed using the pan control window 38 cause the fader controls in the discrete level window to be updated. Moving one of those faders to adjust a level discretely invalidates the settings of the pan control window and closes it.
EQ Control Window
The EQ control window 40 (
When a point is touched on the grid, a level fader is enabled and associated with that point. Finer gain adjustments can be made with it. When a point is touched on the grid, if it is a band filter, a Q fader is also enabled and associated with that point. Adjustments to the width of the band filter, expressed in relative Q, can be made with that fader. When a point is touched on the grid, a horizontal fader is enabled and associated with that point. Fine adjustments in a two-octave range can be made with that fader. The grid also displays a calculated response curve for the EQ effect.
Compressor Control Window
The compressor control window 42 (
The grid has ticks indicating dB levels for input level (horizontal), and output level (vertical). Two points can be dragged inside the grid. One point controls the threshold and can only be dragged vertically. The other point controls the compression ratio. It can only be dragged vertically, and not below the threshold point. A line is plotted which represents the dynamic response. The line is animated with the VU for the input of the associated source.
Channel Strips
Channel strips 44 (
A channel strip 44 has the following components:
External faders control the trim levels corresponding to the channel strips, except the first fader. It is reassigned by the system any time a software fader is moved (unless that fader is a trim that is already assigned to a hardware fader). Any fader being controlled by the assignable fader is highlighted.
Simplified User Interface
The following changes can be made to the system user interface in order to simplify it:
The following additions can be made to the system user interface in order to simplify it:
Input Type Functionality
During setup, the user can select an input type. For example, a microphone could be not only of type “vocal”, but even more specific subcategories such as “announcer”, “lecturer”, or “singer”. The types would control some effects. For example, “vocal” type applies a band pass between 80 Hz and 14000 Hz in order to filter 60 Hz hum and hiss.
The “Announcer” type will automatically have an (optional) control that works like a chain compressor. When the microphone input is active, all other levels are brought down.
“Lecturer” type is a solo speaker giving a speech or lecture, and could have some compression useful for making the speech clear.
“Singer” type would apply a tighter band pass, and some default compression useful for vocals.
If all simplification options are implemented, along with aesthetic and labeling changes, the system user interface would then be very simple. Unsophisticated users can rely on the “stage” view. The user would then touch the icon corresponding to the input they want to adjust, and then be presented with a simple panel with labels like “volume”, “bass”, “mid”, “treble”, etc.
Enhanced Setup:
The setup mode already has the potential to be very simple if a large database of predefined objects is created. Users can simply pick objects from a tree of categories. They are added to the stage, and can be dragged to a virtual position.
Optionally, the system 10 can support using a microphone with a known frequency response for calibration. This microphone must be able to send input to the system 10 which is analyzed with a Fast Fourier Transform, using the host PC 12 processor. A sound “sweet spot” is chosen in the venue, and the microphone is placed in that position. Through an interactive process of playing noise through the speakers, analyzing the sampled input (with the microphone's known response subtracted), the speaker levels can be automatically calibrated, and final EQ could be determined in order to remove resonant frequencies, and flatten the character of the speakers. Other calibrations could be done using this calibration technique, such as virtual positioning of speakers and instruments.
I/O Port Definitions
For all I/O Ports (source or destination), the following parameters can be selected to create the port definition:
If the port is for a source, the following definitional information is needed:
If the port is for a destination, the following definitional information is needed:
Custom audio parameters can be defined in a variety of ways. For example, a custom parameter may be defined that tightens the EQ and raises volume at the same time. A custom parameter is described as a list of things a parameter changes, with an offset and multiplier for each.
Thus, using the system 10 of this invention, the sound mix at a live performance venue can be setup and then controlled in real time using a digital mixing console with a highly efficient and easy to comprehend and operate user interface. The user is provided with one or more preset stage and venue configurations, with defined audio sources and destinations. The sources and destinations (stage elements) are visually displayed as graphical icons with “friendly” names and icons and are assigned to various mixer inputs and outputs. The icons are moved to different positions on the display to reflect the physical arrangement on the stage. Audio characteristic associated with each stage element (e.g., gain and EQ) are displayed in connection with each icon. To adjust an audio parameter, the icon is touched on the display and then appropriate adjustments are made using virtual console and mixer function views on the system display. Standard adjustments can be selected by simply touching “friendly” names on the display.
Thus, although there have been described particular embodiments of the present invention of a new and useful Live Performance Audio Mixing System with Simplified User Interface, it is not intended that such references be construed as limitations upon the scope of this invention except as set forth in the following claims.
This application claims benefit of co-pending U.S. Patent Provisional Patent Application Serial No. 60/370,872, filed Apr. 8, 2002, entitled “Live Performance Audio Mixing System with Simplified User Interface”, the disclosure of which is hereby incorporated by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4792974 | Chace | Dec 1988 | A |
| 5027689 | Fujimori | Jul 1991 | A |
| 5153829 | Furuya et al. | Oct 1992 | A |
| 5212733 | DeVitt et al. | May 1993 | A |
| 5390295 | Bates et al. | Feb 1995 | A |
| 5524060 | Silfvast et al. | Jun 1996 | A |
| 5526456 | Heinz | Jun 1996 | A |
| 5559301 | Bryan, Jr. et al. | Sep 1996 | A |
| 5608807 | Brunelle | Mar 1997 | A |
| 5739454 | Kunimoto | Apr 1998 | A |
| 5740436 | Davis et al. | Apr 1998 | A |
| 5778417 | Kentish et al. | Jul 1998 | A |
| 5812688 | Gibson | Sep 1998 | A |
| 6031529 | Migos et al. | Feb 2000 | A |
| 6067072 | Fujishita et al. | May 2000 | A |
| 6118883 | Rocha | Sep 2000 | A |
| 6140565 | Yamauchi et al. | Oct 2000 | A |
| 6169540 | Rosenberg et al. | Jan 2001 | B1 |
| 6281420 | Suzuki et al. | Aug 2001 | B1 |
| 6353169 | Juszkiewicz et al. | Mar 2002 | B1 |
| 6359632 | Eastty et al. | Mar 2002 | B1 |
| 6490359 | Gibson | Dec 2002 | B1 |
| Number | Date | Country | |
|---|---|---|---|
| 20040030425 A1 | Feb 2004 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60370872 | Apr 2002 | US |
