Patent Application
20210042006
An Application Data Sheet is filed concurrently with this specification as part of this application. Each application to which this application claims benefit or priority as identified in the concurrently filed Application Data Sheet is incorporated by reference herein in its entirety and for all purposes.
There are a number of controllers used by musicians for triggering drum sounds, manipulating rhythmic beats, adjusting parameters and launching sample-based clips. Minimal user feedback is provided requiring the musician to look at a computer screen for information of the state of the sound producing software. Available controller devices have limited utility and are usually targeted at a specific type of data manipulation or work only with specific software. Typically these devices have used simple switches, variable knobs or slide potentiometers and trigger pads. These limitations require users to have multiple pieces of gear that produce a limited amount of control.
According to a first class of implementations, a controller is provided that includes N control pads and one or more processors, where N is an integer greater than zero. Each control pad has sensor circuitry associated therewith configured to generate one or more sensor signals representing corresponding touch events on a corresponding surface of the control pad. The one or more sensor signals associated with each control pad also represent corresponding locations of the touch events on the surface of the control pad. The one or more processors are configured to generate control information from the sensor signals. In a first operational mode the one or more processors are configured to map the control information to N control functions. In a second operational mode the one or more processors are configured to map the control information to N×M control functions corresponding to the locations of the touch events on the surfaces of the N control pads, where M is an integer greater than one.
According to one such implementation, N is 16 and M is 4. The N control pads are arranged in a 4×4 array, and the locations of the touch events on the surfaces of the N control pads corresponding to the N×M control functions form an 8×8 control array. According to another such implementation, the first mode of operation corresponds to a drum pad controller, and the second mode of operation corresponds to a grid controller. According to another such implementation, the one or more sensor signals associated with each control pad also represent pressure of the touch events on the surface of the control pad, and the pressure of the touch events is reflected in the control information.
According to yet another such implementation, the sensor circuitry associated with each control pad includes four sensor quadrants. Each sensor quadrant includes first conductive elements connected to a first voltage reference and second conductive elements connected to a second voltage reference. At least some of the first and second conductive elements are connected to the corresponding voltage reference via a resistive element. The sensor circuitry also includes a conductive material configured to make contact with at least some of the first and second conductive elements in at least some of the sensor quadrants in response to the touch events, thereby forming one or more voltage dividers corresponding to the sensor quadrants with which contact is made. According to a specific implementation, the conductive material or at least some of the first and second conductive elements are or include a piezoresistive material. According to another specific implementation, the first and second conductive elements are arranged as concentric conductive traces across the four sensor quadrants, at least some of the concentric conductive traces being discontinuous, thereby defining the four sensor quadrants.
According to another class of implementations, a sensor is configured to generate one or more sensor signals representing corresponding touch events on a corresponding surface associated with the sensor. The sensor includes sensor circuitry that includes four sensor quadrants. Each sensor quadrant includes first conductive elements connected to a first voltage reference and second conductive elements connected to a second voltage reference. At least some of the first and second conductive elements are connected to the corresponding voltage reference via a resistive element. The sensor circuitry also includes a conductive material configured to make contact with at least some of the first and second conductive elements in at least some of the sensor quadrants in response to the touch events, thereby forming one or more voltage dividers corresponding to the sensor quadrants with which contact is made.
According to one such implementation, the conductive material or at least some of the first and second conductive elements are or include a piezoresistive material. According to another such implementation, the first and second conductive elements are arranged as concentric conductive traces across the four sensor quadrants. At least some of the concentric conductive traces are discontinuous, thereby defining the four sensor quadrants.
According to another such implementation, the one or more sensor signals represent pressure of the touch events on the surface of the sensor.
According to yet another such implementation, the one or more sensor signals represent corresponding locations of the touch events on the surface of the sensor. According to a more specific implementation, the touch events correspond to an object in contact with the surface of the sensor, and the one or more sensor signals also represent movement of the object across the surface of the sensor. According to an even more specific implementation, the one or more sensor signals also represent pressure of the touch events on the surface of the sensor, and the one or more sensor signals also represent wiggling of the object on the surface of the sensor.
According to another class of implementations, a sensor is configured to generate one or more sensor signals representing corresponding touch events on a corresponding surface associated with the sensor. The sensor comprising sensor circuitry that includes an arrangement of conductive elements. First one of the conductive elements are connected to a voltage reference, and second ones of the conductive elements are configured to receive sequential drive signals. At least some of the first and second conductive elements are connected to a resistive element. The sensor circuitry also includes a conductive material configured to make contact with at least some of the first and second conductive elements at locations associated with the touch events, thereby forming one or more voltage dividers when the second conductive elements with which contact by the conductive material is made are driven by a corresponding one of the sequential drive signals.
According to one such implementation, the conductive material or at least some of the first and second conductive elements are or include a piezoresistive material.
According to another such implementation, the one or more sensor signals represent corresponding locations of the touch events on the surface of the sensor. According to a more specific implementation, the one or more sensor signals represent the corresponding locations of multiple and simultaneous ones of the touch events on the surface of the sensor. According to another specific implementation, the touch events correspond to an object in contact with the surface of the sensor, and the one or more sensor signals also represent movement of the object across the surface of the sensor. According to an even more specific implementation, the one or more sensor signals also represent pressure of the touch events on the surface of the sensor, and wherein the one or more sensor signals also represent wiggling of the object on the surface of the sensor.
According to another such implementation, the one or more sensor signals represent pressure of the touch events on the surface of the sensor. According to a more specific implementation, the one or more sensor signals also represent corresponding locations of multiple and simultaneous ones of the touch events on the surface of the sensor.
According to yet another such implementation, the arrangement of the first and second conductive elements is an alternating arrangement of the first and second conductive elements. According to a more specific implementation, the alternating arrangement of the first and second conductive elements is either linear, circular, or rectilinear.
According to yet another such implementation, a control system includes at least one instance of the sensor, and one or more processors configured drive the second conductive elements with the sequential drive signals, to generate control information from the one or more sensor signals, and to map the control information to one or more control functions. According to a more specific implementation, the one or more sensor signals represent pressure of the touch events on the surface of the sensor, and a first one of the touch events corresponds to the conductive material making contact with more than one pair of the first and second conductive elements. The one or more processors are configured to generate the control information to represent a first location of the first touch event as corresponding to one of the pairs of the first and second conductive element with reference to the representation of the pressure of the first touch event in the one or more sensor signals. According to another specific implementation, the one or more sensor signals represent pressure of the touch events on the surface of the sensor, and the one or more sensor signals also represent corresponding locations of multiple and simultaneous ones of the touch events on the surface of the sensor. The one or more processors are configured to generate the control information to represent the pressure associated with each of the simultaneous touch events. According to another specific implementation, the one or more sensor signals represent corresponding locations and pressure of the touch events on the surface of the sensor, and the one or more processors are configured to map the control information to multiple control functions corresponding to different gestures. According to yet another specific implementation, the first and second conductive elements are each wedge-shaped and the arrangement of the first and second conductive elements is an alternating circular arrangement of the first and second conductive elements. The one or more processors are configured to generate the control information to represent a distance from a center of the circular arrangement based on a number of the conductive elements represented in the one or more sensor signals.
According to yet another class of implementations, computer program products and computer-implemented methods are provided for configuring a music controller to operate with one or more of a plurality of music applications from a plurality of different vendors. The plurality of music applications are presented as selectable options in a user interface of a computing device. One or more selections of the music applications are received. A location in memory of the computing device is identified for each selected music application. A plurality of script code files is stored in each location. Each script code file maps control information generated by the music controller to functions of the selected music application for the corresponding location. Each selected music application is configured to perform the functions in response to the control information from the controller using the script code files stored in the corresponding location in the memory of the computing device.
According to one such implementation, copies of the script code files are stored in one or more directories in the memory of the computing device separate from the locations for each of the selected music applications.
According to another such implementation, an example file for each selected music application is stored in the memory of the computing device. Each example file including code representing one or more sounds that may be generated using the corresponding selected music application in conjunction with the controller.
A further understanding of the nature and advantages of various implementations may be realized by reference to the remaining portions of the specification and the drawings.
Reference will now be made in detail to specific embodiments of the invention including the best modes contemplated by the inventors for carrying out the invention. Examples of these specific embodiments are illustrated in the accompanying drawings. While the invention is described in conjunction with these specific embodiments, it will be understood that it is not intended to limit the invention to the described embodiments. On the contrary, it is intended to cover alternatives, modifications, and equivalents as may be included within the spirit and scope of the invention as defined by the appended claims. In the following description, specific details are set forth in order to provide a thorough understanding of the present invention. The present invention may be practiced without some or all of these specific details. In addition, well known features may not have been described in detail to avoid unnecessarily obscuring the invention.
Various implementations provide a highly configurable controller that includes a number of different types of control mechanisms that emulate a wide variety of conventional control mechanisms (e.g., buttons, sliders, pads, rotary controls, etc.) using pressure and location sensitive sensors that generate high-density control information which may be mapped to the controls of a wide variety of devices and software.
A particular implementation shown in
According to some implementations, pressure and position sensitive pads 102 can be used to elicit percussive sounds that vary in timbre based on where the user strikes the pad which is analogous to real acoustic percussion instruments. Faders 108 are analogous to volume level controls on traditional mixing boards. Buttons 104 can be used to make selections from lists presented by a computer menu and can scroll or navigate more rapidly through the list based on how firmly the button is pushed. A wide variety of other applications of the sensor capabilities and high density control information generated by controllers as implemented herein will be apparent to those of skill in the art.
Sliders 108 and 110, the operation of which is described below, are scanned or driven by microprocessor 208 via decoders, e.g., decoder 210 (only one of which is shown for clarity). The outputs of sliders 108 and 110 are then converted to digital form (not shown) for input to microprocessor 208. Other sensors, e.g., buttons 106 shown in
According to a particular class of implementations, a subset of the sensors (e.g., the sliders and rotary sensors) are implemented using an array of driven or scanned conductive printed circuit board (PCB) elements alternating with conductive elements connected to a voltage reference, e.g., ground, through a resistor. The array overlaid with a conductive pressure sensitive compressible material that may be a piezoresistive material such as, for example, germanium, polycrystalline silicon, amorphous silicon, non-woven impregnated fabric and single crystal silicon.
The common elements 306 are tied to ground through a resistor. When pressure is applied by the user (e.g., using a finger or stylus) the driven element(s) 302 at the point of contact are connected to the surrounding common elements 306 by the overlying conductive material (not shown). The voltage at the junction of the common elements and the driven element(s) is thereby raised and transmitted via a multiplexer to an ADC (not shown). The microprocessor driving the driven elements (not shown) also sequentially measures the corresponding signal levels, e.g., the voltage at the junction of the driven element(s) and nearby common element(s), to determine whether and where a touch event occurs. The ADC output can be represented as shown in the bar graph of
According to implementations in which the overlying material that connects the sensor resistive elements is a piezoresistive material, processing of this data by the microprocessor yields both pressure information (e.g., the sum of all raised data points) and location information (e.g., the centroid of the data points). According to a particular class of implementations, the positional resolution derived from the array may be much greater than the number of elements, i.e., interpolation between the data values using a centroid calculation or similar method yields greater positional accuracy than simply using the locations of the array elements. For example, if the ADC converter that measures the voltages of 16 driven elements is 8 bits the positional resolution is theoretically 1 part in 256 for each of the 16 element locations, or 1 part in 4096 for the array using a simple interpolation approach.
According to some implementations, driven/scanned sensors can be configured to recognize multiple touch points. The graph of
Various configurations of resistive elements are contemplated that yield information about different styles of human gesture. For example, if the elements are arranged in a linear array as shown in
Analyzing the spread of data values using known statistical methods may provide information related to the location of the touch event with respect to the center of circular array 500. This can be expressed as the radius of the gesturing finger. According to a particular implementation, the radial “slices” converge in the center of the array to a radius approximating the size of the activating source, e.g., such as 4-5 mm for a human finger. In the bar graph of
Another sensor that may be used with various implementations of a controller as described herein is illustrated in
Each of four quadrants of 90 degrees includes discontinuous conductive traces 602 spanning about 90 degrees of arc that are connected to each other and tied to a reference voltage through a resistor (e.g., 10K ohm) and to an ADC through a multiplexer (e.g., as shown in
As the user presses on a silicone pad that covers the piezo-resistive material, the conductive trace of the four quadrants are connected to the reference voltage (via the piezoresistive material) to varying degrees. The location chosen by the user and the pressure exerted in that position can be determined by a microprocessor (e.g., 208) through a comparison and evaluation of the signal levels present in each of the quadrants. As will be understood, this arrangement can provide a very high density of control information that may be mapped to a wide variety of applications. For example, in addition to the control of musical effects, the control information generated using such a sensor may be used to control a cursor or pointer, or the depicted point of view, in the user interface of a computing device. In addition, as discussed below, this arrangement may also be flexibly configured to operate as a single sensor or as multiple sensors.
According to various implementations, the geometry, density, opacity and coating of an silicone layer overlying the sensors may be controlled to accomplish any subset of the following: (1) hold the piezoresistive material at a specified distance from the trace patterns; (2) deform evenly over the length, diameter or area of the specific sensor providing a linear and even response; (3) transfer the luminance from an LED indicator over the top of the sensor area; (4) diffuse the LED indicator light so it does not appear as a point; (5) protect the piezoresistive material, LED, trace and/or electronic components underneath; and/or (6) provide an appropriate tactile surface so the finger can slide or grip in a manner appropriate for how the specific sensor is activated.
An example of the relationship of a silicone layer to an underlying sensor configuration is shown in
According to a particular class of implementations, position and pressure information generated using appropriately configured sensors (such as those described herein and suitable alternatives) may be used to provide sophisticated types of control. For example, changes in position and/or pressure on a particular sensor may be used to modulate control information around one or more quantized states associated with the sensor. A particular example in the context of a musical effects controller is when a sensor is configured as a piano or synthesizer key. In such a case, changes in position and/or pressure, e.g., when the user wiggles or slides his finger back and forth, can be used to distort or modulate the pitch of the primary note associated with the key, e.g., to effect note bending or vibrato.
According to one such implementation illustrated in
When the key is activated, e.g., by a touch event, one or both of the piezoresistive discs contact the underlying conductive elements of the sensor(s), and the control information is mapped to the primary quantized state associated with the key, i.e., the corresponding note is reproduced. An initial absolute value is generated representing the difference between the signal(s) generated by the top and bottom sensors. An offset is added to the initial absolute value to define an activation threshold. After a period of delay from when the key is first declared active, real-time absolute values calculated like the initial value are streamed from the key and compared to the activation threshold. If the activation threshold is exceeded, e.g., because the user is rocking the key back and forth by wiggling his finger, the microprocessor maps the correspondingly modulated control information received from the key (i.e., the pressure and/or position information) to the desired effect, e.g., note bending, vibrato, etc.
For more information on materials and sensor configurations that may be used with various implementations, as well as software and devices that may be controlled, please refer to U.S. patent application Ser. No. 12/904,657 entitled Foot-Operated Controller filed on Oct. 14, 2010 (Attorney Docket No. KSMOP004), the entire disclosure of which is incorporated herein by reference for all purposes.
At least some implementations include an array of LEDs to illuminate the sensor array and employ a scheme for driving the array in a way that allows hundreds of LEDs to be driven off limited current, e.g., the current available from a USB connection. A particular implementation of a controller (e.g., QuNeo 100 of
A practical way to control 251 LEDs is to create a 16 by 16 matrix that sources and sinks current for groups of 16 LEDs at a time. This reduces the best case duty cycle or time that the LED is on to 1/16th. Regular efficiency LEDs are ⅓rd the cost of high efficiency LEDs and the LED cost is the dominant component of the product's overall component cost. One problem is how to gain maximum brightness without exceeding the 2.5 watts of available USB power using standard efficiency LEDs; particularly given the fact that not all of this 2.5 watts can be dedicated to LED illumination as the QuNeo must perform other functions. Providing the 16 LEDs with a maximum of 30 mA of current at the 1/16th duty cycle may result in a display that is insufficiently bright for many applications.
While 30 mA is the rated steady state current maximum of regular commercially available LEDs, most can accept a current spike of up to 150 mA for ˜100 microseconds. Heat is the limiting factor for steady state current drive, but the LED can be driven brighter for a brief period of time. Therefore, according to a particular class of implementations, a drive circuit is provided for accomplishing the drive current shown in
The parallel R and C associated with each LED (e.g., R1 and C1 associated with LED1) provides a brief near DC path to ground creating a limited current spike for each of the LEDs when switched on from an off state. As the duration of the spike is limited by the charge time of the capacitor, no other supervisory or switching circuitry is required, no increase in processor overhead is created, and component cost is minimal. This design not only increases the brightness of the LEDs, it also allows LED refresh cycles to be much shorter than they would otherwise have to be in order to create the desired level of brightness. This possibility of fast refresh rates allows opportunities for more sophisticated display techniques (such as dimming and color-mixing) that are normally only possible with a costly and component-heavy design.
According to a particular implementation, the default refresh rate for a 16×16 LED matrix is ˜1000 Hz. By skipping refresh opportunities, the LEDs may be dimmed in a controllable manner. 1000 Hz also allows 16 levels of brightness as well as Off, with no discernible flickering perceived even at low brightness levels. This feature allows the device to respond to a wide range of inputs with a complex and nuanced visual display without exceeding current limit requirements or adding significantly to the cost of the controller.
As is well known, there are two dominant switch or pad layouts used in the conventional musical controller world. One is a 4×4 array of drum pads in which sixteen different drum sounds are mapped 1:1 to the sixteen pads. Hitting a pad causes the corresponding drum sound to be produced. Another popular controller uses an array of 8×8 switches that can be configured, for example, to trigger corresponding audio files, or to determine which one of 8 beats of a rhythm is generated (horizontal pads) for each of 8 different instruments (vertical pads).
According to a particular class of implementations and as illustrated in
According to some implementations, QuNeo may be configured to power up with presets that emulate the functionality of popular drum pad and grid controllers. For example, in the traditional drum pad mode, QuNeo may behave like an Akai MPD/MPC pad controller configured for dynamic electronic drumming and pad based sequencing. For example, QuNeo may be set up similar to a standard drum pad controller, with four sets of notes from C2 to E7. An appropriate velocity value may be associated with each note. 16 notes may be playable (1 per pad), e.g., starting from the bottom left and ascending chromatically to the right and toward the top.
Alternatively, as a grid controller, the QuNeo may emulate popular 8×8 grid based controllers like the Monome and Novation's Launchpad. Monome style sequencing and sample manipulation is readily emulated, with the added bonus of pressure output for each of the 64 steps. In another example, clip launching in Ableton Live is also easily supported while operating as a grid controller, with 64 possible clips per bank. Multi color LEDs, also in an 8×8 grid, may act as indicators for beats and playing/recording.
According to a particular class of implementations, QuNeo may be configured to work with a wide variety of third-party software and/or devices. This versatility is enabled through the installation of ancillary code into the third-party software to enable the mapping of QuNeo's high-density control information to existing music applications, e.g., synthesizers, DJ applications, recording applications, etc.
Such implementations of QuNeo have a variety of associated templates that allow QuNeo to integrate and work with the various third-party software. These templates facilitate installation of Python script code into the third-party software that maps the QuNeo control information to the inputs recognized by the interface stack of the third-party software. When the QuNeo software is activated (e.g., on a user's PC as downloaded from the web), the user is presented with a list of third-party software. When the user selects one or more options with which he wants the QuNeo to work, the installer finds the selected software and installs the necessary scripts. Corresponding manuals may also be loaded which explain how to use the QuNeo to control the selected software.
QuNeo has the potential to interface with many different software applications since it is a class-compliant MIDI device with control information mappings to complement many of the most popular pieces of software. When the user installs QuNeo software (e.g., downloaded to his PC from the web), the user is prompted to select from among the available software templates as shown in screenshot 1200 of
For example, if the user selects the Ableton Live template, an Ableton Live example file is put into the user's install directory along with an automatically installed QuNeo Editor and associated documentation. The example file includes sounds the user can try out to illustrate the way in which QuNeo is initially set up to control the Ableton Live software. Selection of the Ableton Live template also results in installation of a group of remote mapping files. These mapping files control which of the application's functions are triggered when data are received from QuNeo. For example, when a certain button on QuNeo is pressed, the application's play button is triggered). In the case of Ableton Live, the mapping files are Python scripts. However, it should be noted that the mapping file format may vary from application to application.
The installer places the remote mapping files in a location on the user's computer where the corresponding application can find them and interact with them accordingly. The location may be a default, in the user's Applications folder on a Mac. Alternatively, this may be accomplished by asking the user to browse to a particular directory—usually to the location of the selected application itself—as shown in screenshot 1300 of
There is some error checking in place to make sure the user doesn't accidentally select the wrong location, and then the remote mapping files are copied into the selected location. In the case of Ableton Live, the Python scripts go into a folder within the application where other supported controller scripts typically exist (e.g., as shown in screenshot 1400 of
A copy of the remote mapping files may also be placed in the install directory in case there is an error during installation, if the user wants to manually add the files to a different version of Ableton Live, or in case they are just curious to look at them. The install directory might end up looking like screenshot 1500 of
As is well known, Ableton Live is for electronic music performance and composition. Another example of third-party software that may be controlled using QuNeo is Mixxx—a free DJ program. The Mixxx template allows users to play the rotary sensors like turntables, create loops using the pads, and, in turn Mixxx control information is used to control the QuNeo LEDs. So, with a single purchase, i.e., the QuNeo, the user has everything he needs to start DJ-ing right away. Mixxx uses a different file format to keep track of its commands. To recognize QuNeo as a controller Mixxx needs two files with very particular names placed into its MIDI folder called “KMI QuNeo.midi.xml” and “KMI-QuNeo-scripts.js.” When Mixxx sees these two files it creates an option in its menu of MIDI controllers called “KMI QuNeo.” When the user selects this option, QuNeo is configured to control the Mixxx application in the way defined by these two files.
If the user decides to install the Mixxx template in screenshot 1200, he is again prompted to navigate to the location of his Mixxx application (as illustrated in screenshot 1600 of
It should be noted that, despite references to particular computing paradigms and software tools herein, the computer program instructions with which embodiments of the invention may be implemented may correspond to any of a wide variety of programming languages, software tools and data formats, and be stored in any type of volatile or nonvolatile, non-transitory computer-readable storage medium or memory device, and may be executed according to a variety of computing models including, for example, a client/server model, a peer-to-peer model, on a stand-alone computing device, or according to a distributed computing model in which various of the functionalities may be effected or employed at different locations. In addition, reference to particular protocols herein are merely by way of example. Suitable alternatives known to those of skill in the art may be employed without departing from the scope of the invention.
While the invention has been particularly shown and described with reference to specific embodiments thereof, it will be understood by those skilled in the art that changes in the form and details of the disclosed embodiments may be made without departing from the spirit or scope of the invention. For example, implementation have been described herein in which sensors are implemented in which piezoresistive material is brought into contact with conductive elements on an underlying PCB. However, it should be noted that implementations are contemplated in which the elements on the PCB themselves include or are constructed with piezoresistive material, and the overlying material that contacts the PCB elements may be conductive and/or piezoresistive.
Finally, although various advantages, aspects, and objects of the present invention have been discussed herein with reference to various embodiments, it will be understood that the scope of the invention should not be limited by reference to such advantages, aspects, and objects. Rather, the scope of the invention should be determined with reference to the appended claims.
| Number | Date | Country | |
|---|---|---|---|
| 61610924 | Mar 2012 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16148570 | Oct 2018 | US |
| Child | 16948131 | US | |
| Parent | 14728873 | Jun 2015 | US |
| Child | 16148570 | US | |
| Parent | 13799304 | Mar 2013 | US |
| Child | 14728873 | US |
