Patent Grant
7145070
This application is the U.S. National Phase of International Application No. PCT/IE2003/000102 filed on 14 Jul. 2003, which is incorporated herein by reference.
The present invention relates to an interactive multimedia apparatus.
The present invention is an improvement to invention disclosed in International publication No. WO 01/95052 by the same inventor, the contents of which are incorporated herein by reference thereto.
It is an object of the present invention to provide an interactive multimedia apparatus which will provide users with a digital instrument platform allowing a wider range of more challenging experiences. Additionally, it presents users with a sophisticated digital musical instrument which will enable them to create musical and other multimedia content as if they were accomplished musicians/artists. Additionally, it is an object of the invention to allow non-musical trained users to compose, accompany, solo, gig and have fun as if they were an accomplished instrument player. Finally, it is an object of the present invention to assist users in developing knowledge of chord structures, complex chord structures, voicing, scales and the fingering of notes on the fret of stringed instruments.
Accordingly, the present invention provides an interactive multimedia apparatus comprising:—
Advantageously, the provision of an independent transducer to monitor each dynamic range state control member dramatically increases the functionality of the apparatus and further increases the interaction between an operator and the apparatus allowing an operator greater control over the output generated by the apparatus.
Preferably, the independent transducers are selected from a group of electrical, optical, pressure, movement, magnetic and piezo-electric transducers.
All of these types of transducers provide acceptable solutions for the sensing of vibrations, and the amplitude of the vibrations.
Ideally, the piezo-electric transducer is selected from a group consisting of piezo-electric crystal transducers, piezo-electric ceramic transducers and piezo-electric film transducers.
In a particularly preferred embodiment, the piezo-electric transducer is a piezo-electric crystal transducer.
Preferably, the digital musical instrument is a guitar/guitar type device.
Ideally, a number of the dynamic range state control members are strings of a stringed instrument.
Preferably, each independent transducer monitors one individual string and converts movement of the string into electrical signals.
Ideally, at least one dynamic range state control member is a foot pedal.
Preferably, one or each foot pedal is monitored by a potentiometer.
Ideally, activation of the foot pedal dynamically modifies a variable control such as volume, pan or special effect parameter controls.
Preferably, the central control unit has a visual display unit (V.D.U.) and the suite of software has a graphical user interface (G.U.I) displayable on the V.D.U.
Ideally, the control unit of the digital musical instrument has a control panel comprising a plurality of limited state control members, namely switches for navigating through the G.U.I. of the software suite and also modifying the parameters of the software suite.
Preferably, the central control unit has memory for storing user assignments of digital media against specific control members using the control panel, the memory being accessible by a software module of the software suite in response to activation of a control member or by activation of a combination of one or more limited state or dynamic range state control members to open, render, modify, adjust, add effects, change parameters and change controls of the rendered digital media.
Ideally, indicators are provided on the digital musical instrument and are controllable by software on the central control unit in response to an assignment of digital media against specific control members by a user, the indicators being provided to show the user which of the dynamic range state control members have been assigned to produce an audio/visual output if activated.
Preferably, the indicators are a plurality of light emitting diodes disposed on the digital musical instrument below each string, each L.E.D. being associated with one string.
Ideally, at least one limited state (on/off) control member and at least one dynamic range state control member are provided.
Preferably, between one and sixteen limited state (on/off) control members and between one and twelve dynamic range state control members are provided.
Ideally, each control member is associatable with any file stored in the store of the central control unit.
Preferably, at least some of the digital audio files are recorded instrument notes from a plurality of stringed instrument types.
Ideally, the store further comprises a chord and scale store of digital media containing the note associations for a wide range of defined chord and scale sequences, the store also defining the correct note associated with each string position for the selected chord or limited scale sequence so that the notes are played in the correct sequence when the strings are strummed up or down.
Ideally, the central control unit has input/output connections for the Internet.
Preferably, the software in the central control unit detects the length, and amplitude and the frequency of the electrical signals received from the digital musical instrument in response to a control member's excitation and alters its rendering of the digital media mapped to that control member or any other control member in sympathy with that excitation.
The software always generates an output even if the user operates the control members at the wrong time or incorrectly and in which the outputs provided are sympathetic to the main track's structure and melody.
Ideally, the digital media are rendered simultaneously with a CD, DVD or other primary source of music playing on the audio-visual equipment. Sound is generated in direct response to an operator's action, which enhances the overall musical experience for the operator.
Preferably, the apparatus is provided with a series of special effect controls which when operated by a user send signals to the software to produce a variety of special effects on the digital media output. There are a variety of different ways of manipulating sounds using effects. Frequencies of samples can be raised or lowered, resonance can be distorted (overdrive) or echo can be added through delay and feedback.
Conveniently, the apparatus includes means for storing the newly generated output to any desired storage device such as a hard disk, a compact disc, a DVD device or the like.
Preferably, the apparatus includes a series of visual display screens which are operable to enable a user to launch a game experience, select different operating modes, choose a source music device, select a backing track and/or assign digital sound effects files to the special effects controls on the digital musical instrument.
Optionally, the digital media may contain a riff, a sample, a loop or a track. A riff is a series of notes that form a section of a musical track. A song might contain a guitar riff of eight notes followed by a series of guitar riffs to form a lead solo. Some software music studios have riff generators that allow the creation of unique riffs instead of using pre-recorded riffs. There are also different digital storage formats for riffs such as .wav and .mp3. A sample is a pre-recorded piece of music that is usually not very long such as a five-second bass riff, or a two-second drum loop. Many CD's are available that offer thousands of royalty-free sound samples. A loop is a riff that when repeated over and over again forms a seamless track of music. A bass loop may contain a six-note riff that can be repeated a number of times to form some of the bass-line of a song.
Preferably, the apparatus includes an interface for sending the electrical signals generated by the transducers to the central control unit, the interface being provided between the transducers and the input port of the central control unit, the interface unit enabling the user to generate a plurality of different control signals to the central control unit.
Ideally, the interface includes an amplifier and level detectors to detect the force with which the user strums or strikes the strings. The software includes means to decipher the electrical signals from the instrument and alter one or more parameters of the rendered media in sympathy with the force of the string excitation.
Ideally, the interface unit is provided with a potentiometer which varies the duration of the sound of the digital media file.
Ideally, customised driver software is provided with the instrument avoiding the necessity for calibration.
Preferably, the communication means deployed between the digital musical instrument and the central control unit is a Universal Serial Bus (USB).
Preferably, the central control unit comprises a personal computer, a cable or satellite television decoder or a games console and the audio/audio-visual means comprises a mono or stereo high fidelity audio apparatus, a television, a monitor or a like audio/audio-visual output means.
In a still further aspect of the invention, an operator can use the digital musical instrument and the software interface as a sixteen or twenty-four track-recording studio. The studio allows an operator to save their compositions in a format for future rendering and also in a format for writing their own CD's. Other export formats are MP3 and wav.
Ideally, an operator can drop samples of riffs and loops onto individual tracks to compose their own music/songs. Bass loops, drum loops, rhythm guitar and lead guitar riffs and loops in different musical instruments are provided. Samples are available on CDs and can be downloaded from the Internet.
Preferably, the user will be able to set beats per minute BPM, create his own riffs, loops, and effects and change the pitch of selected sections. BPM stands for beats per minute and is also known as the tempo of a song, or in other words the speed at which a song is played. Different songs will have different BPM e.g. a lot of Techno/Dance/Hip-Hop will have 130–180 BPM. It is important when creating a song made up of sample riffs and loops that all the samples have the same BPM. Some software programs allow the transposition of samples from one tempo to another without changing the pitch of the sample.
Ideally, a number of digital musical instruments can be connected to the central control unit at one time allowing multi-user operation of the apparatus. One guitarist could control the lead guitar, another the bass, and another the rhythm guitar and roles could be switched while playing. In Jam mode, players could improvise by playing over specially composed songs or by playing their own tracks/songs or by playing in random selection mode. In this mode a number of operators could have a ‘battle of the bands’ competition against each other.
In another aspect of the invention, an operator uses the apparatus as a learning aid and has to strum to the correct tempo of the music as well as making different track selections and adding the proper effects at the right time.
In a further aspect of the invention, the apparatus is used as a controller to bring the operator through different levels of a custom designed computer game. The game plot could go through different levels of becoming a rock star such as going to music school, learning to play, forming a band, writing songs, playing gigs, getting a manager, recording in a studio, getting a record deal, releasing an album, designing CD sleeves, making a pop/rock video, animations/clips etc, competing in the charts and all the various stages could be conducted as a competition over the Internet.
Most guitar players write songs initially as a sequence of Chords. There are numerous music books available to give the guitar tablature (Chords) for different music albums/styles. These could also be provided from a website.
The present invention is a combination of digital hardware and computer software program. It operates on mass market computer-based multimedia platforms, i.e. personal computers or games consoles such as Sony Playstation, Microsoft X-Box or Nintendo Dreamcast (APS Registered Trade Marks).
The digital media content for use with the invention can be any third-party generated media that can be rendered on a personal computer or games console.
The content also includes discrete soundbites (effects sounds, i.e. riffs, beats, loops etc. selectable by the switches on the peripheral) that can be triggered during the playback experience.
The invention will now be described with reference to the accompanying drawings, which show, by way of example only, an interactive multimedia apparatus in accordance with the invention in which:—
Referring to the drawings and initially to
Referring to the drawings and now to
Referring to the drawings and now to
Referring to the drawings and now to
In use, a user navigates through a G.U.I. of the software suite on a VDU (15, 25) using the control panel 48 and assigns specific files stored on the store to one or more control members 42, 44, 4546. Additionally, the user may apply various effects to either or both of the limited state control members 44 and the dynamic range state control members 42, 4546 which is more fully explained with reference to
If a user wishes to output sound from a different file, another limited state control member 44 must be pressed on the guitar 41. This in turn signals the software to open a different associated file. If a user wishes to apply a special effect they can activate any control member 42, 44, 45, 46 such as the arm 45 and volume control is achievable by twisting volume control buttons 46. When a user becomes tired of the various files that they have downloaded onto the central control unit (2, 12, 22), they may select a different collection of files using the control panel 48 in conjunction with a graphical user interface displayed on the visual display. A user can directly access the internet using the guitar 41 as a means for navigation.
Referring to the drawings and now to
This invention allows non-musical trained users to compose, accompany, solo, gig and have fin as if they were an accomplished instrument player.
Additionally the invention will assist users in developing a knowledge of chord structures, complex chord structures, scales and the location of notes on the fret of a stringed instrument (3, 14, 24, 41).
This invention allows users to not only dynamically apply global effect parameters to a selected note or chord, but will allow users to select, control and adjust any individual or group of parameters which make-up the component parameters of the selected special effect generator.
The guitar (3, 14, 24, 41) product is a USB low speed (1.5 Mhz) bus powered device. It has 12 pushbutton switches SW1 to SW12, 1 optoswitch OPT1/red LED 204SRC pair. It is connected to the PC via a 3 m 4 core screened cable.
Schematic Description
Power
The guitar (3, 14, 24, 41) product receives its +5 volts power from the PC via USB connector CN1. The maximum current drawn will be approximately 50 mA.
Hardware Reset
When power is first applied, the CPU will be reset by 2 off 0.1 uF capacitors and 10 k resistor combination. Suspend Mode. All USB devices must support suspend mode. Suspend mode enables the device to enter a low power mode if no activity is detected for more than 3 mS. Any bus activity will keep the device out of the suspend state. When the device is in suspend mode it must draw less than 500 uA. CPU Ports A and C are configured as outputs, and set to high, when entering suspend mode because as inputs each pin of ports A and C will draw 50 uA due to the internal pull-up resistors on these ports. CPU Port B does not contain any internal pull-up resistors but external pull-up resistors are implemented in hardware at the optocoupler phototransistor output. Thus all port B CPU pins should be configured as outputs and set to high applied before entering Suspend mode. Disabling the Analogue to Digital converter will save some current in suspend mode. In suspend mode the current drawn by the guitar (3, 14, 24, 41) must be less than 500 uA.
Entering Suspend Mode
The product can be woken up from suspend mode by switching the bus state to the resume state, by normal bus activity, by signalling a reset or by an external interrupt. During suspend mode the internal CPU oscillator is turned off. In this state the CPU will not be able to detect key presses.
Leaving Suspend Mode
The CPU (U1) is a ST7263 manufactured by ST Microelectronics. The CPU version is a surface mounted type called ST72T631K4M1. The CPU clock is set by a 24 MHz crystal (A)
Switches SW1 to SW12
There are a total of 12 push button switches (normally open) connected to the CPU. Each switch is monitored by 1 separate input. Each input is joined to +5 volts via a pull-up resistor. When a switch is pressed the input will drop from +5 volts to 0 volts.
Opto Switch OPT1 and LED1
OPT1 is a phototransistor which is switched on when LED1 (red LED) is on. LED1 shines light on OPT1. OPT1 is switched off by cutting the light beam with your thumb.
U2
This IC protects the CPU from any spurious signals picked up by the external wire. USB004 is a Chinese recommended part.
U3
This IC is a low power Op amp TS931 ILT. (ST Microelectronics). It is used to amplify the signal from the magnetic pickup.
J1 and J2 (3.5 mm Jack Sockets)
J1 or J2 allow for an input from a standard variable resistor foot pedal. The middle connection is the wiper of the potentiometer. A typical value for this potentiometer is approx. 22 k ohms
Bill of Materials
The above schematics and explanation are shown for example purposes only and the invention is not limited in its scope by the operation of the schematic design, the components used or the specification or capabilities of the components or the range of additional peripheral devices which could be added to the design. The design is not limited to the number of control members (switches) or magnetic pick-ups, lights etc. shown in these schematics.
Particular attention should be drawn to
Each transducer output is associated with a single string and is connected to a separate pin on the CPU, in a similar fashion to that described in the schematic
The two foot-pedal jack sockets J1 and J2 shown in
Preferably the Interactive multimedia apparatus, see
In use, the invention may be as follows;
Usually chords are played on a stringed instrument by the placement of the user's fingers on the selected strings 42 in the appropriate section of the fret on the neck piece and then the user activates the strings 42 in a manner that produces the desired sound output with the corresponding vibration amplitude and duration. The user may strum, pick, strum up, strum down, strum and pick, pick between selected strings 42 etc. to achieve their desired results. The correct fingering position on the fret for each string 42 is a critical component in the generation of each note in the overall chord structure.
Asian Pacific, European and USA music compositions use 3 note chords extensively, with the more accomplished, creative and dynamic instrumentalists using more complex chord combinations of 4, 5 and 6 notes to provide a more complete and colourful signature to their playing. 3 note chords can also have many combinations of individual notes from within its own scale, which provide the user with a rich palate of chords from which to select.
If we take as an example only, the ‘C Chord’ and referring to
Db is the same note as C# and the other b(flat) notes are the equivalent of the # (sharp) notes.
The three note chords or triads for the C scale are C, E, G; for Cm they are C, Eb, G, for CSus4 they are C, F, G; for Cm65 they are C, Eb, Gb; for C+ they are C, E, G#. It is obvious that for each note there is a multiplicity of chord associations for the user to create or select from. It is nearly impossible for the non-accomplished user to create the more complex chords, particularly those using 4, 5 or 6 note combinations.
Many users with limited experience and training have been able to create simple compositions using 3 chords variations. They can use the chords of the root note of that key and the chords of the fourth and fifth notes of the scale of the root note. Limiting the creative and entertaining experience to 3 or a small number of chord combinations is most frustrating and irritating to users. To progress beyond the simple “3 chord trick” combination requires a lot of learning and practice with fret fingering etc. This invention eliminates the complexity of correct fret fingering, simple chord creation, complex chord creation, learning note combinations associated for simple and complex chords etc. Additionally this invention will quickly teach the user if they wish to learn, the correct notes associated with each chord they select and moreover the user is shown which string 42 has triggered each note of the selected chord.
This invention allows users to quickly create and render an unlimited number of combinations of simple and complex chords, where each note of the chord is rendered at its correct interval in the chord structure as it would be played by an accomplished user of a stringed instrument.
Additionally, this invention will allow the user to play complex arpeggios as the notes associated with each arpeggio are stored on file and are assigned by the software to the correct string position.
The chord generation and note rendering methodology for string-based instruments and that of keyboard-based instruments differ greatly. With keyboard-based instruments, the user usually plays a chord by simultaneously depressing the keys of the associated notes of the chord. It is not normally possible for the user of a stringed instrument to strum or to pick each note of a chord simultaneously whilst fingering the notes on the fret. This is because there is a time difference interval between the triggering of each note of the chord. This invention allows for this time difference interval, as the chord store contains the correct location of each note of the selected chord and then assigns each note to the correct string position on the interactive multimedia apparatus. Therefore, as each individual transducer's energy level is detected for each string activation, the individual notes of the chords are rendered as the corresponding string 42 is activated. In this way the user is provided with a true representation of an accomplished player's experience.
The LED visual displays under each string 42 will be energised by the application software to indicate to the user the active strings 42 for the selected notes of the chord. The provision of the LED display is to allow users to visibly see the different note assignments to each string for each chord combination. The software can apply appropriate notes to the unassigned strings 42 to simulate the sounds that these unassigned strings 42 would make if the user wishes to strike across the whole of the string area in a violent action similar to the action of their hard-rock idols. The user can chose from a set-up menu, whether they wish to configure the device for violent actions and have the software automatically trigger the correct sounds for the unassigned strings 42.
The activation of the strings 42 resulting from the user actions will be interpreted by the software to produce a sound output that truly reflects the user's striking actions.
The user experiences described above are achieved using the techniques described as follows:
In this embodiment, the Interactive Multimedia apparatus (1, 11, 21) has 12 control members, refer to
The transducer specified in this application will comprise five in number (as example only and not limited to this number) individual, uncoupled and electrically isolated transducers, which provides for five separate electrical outputs as a response to the user's activation of the individual string 42.
For example purposes only we will describe how the user will interact with the software to simulate similar results to those achieved by an accomplished instrumentalist.
Accordingly there is provided a store of recorded instrument notes from a plurality of stringed instrument types. A chord store is provided which contains the note associations for a very wide range of defined chord structures. The users, if they so wish, can additionally create their own bespoke chords or note associations, by selecting an association of notes and defining their string associations.
This chord store will provide the note associations for a very wide selection of known chord and scale definitions and structures. To simulate the playing of a stringed instrument using the Interactive multimedia apparatus (1, 11, 21) and its associated software application program, demands the faithful reproduction of the selected instrument sound in complete and total harmony and sympathy with the user's actions. It is imperative that the chord store contains the exact string assignment for each note in the chord so that the action of strumming or picking the chord will trigger the notes in the correct order that they were played by the user. In this invention, the store which has been created contains the note associations and the string assignments for each chord contained in the store. Additionally, by user selection, the chord store may assign complimentary and sympathetic notes to any or all of the unassigned strings 42. The assignment of sympathetic notes to unassigned strings 42, will provide a more colourful and verbose chord rendering.
Accordingly, if the user wishes to render a chord, for example the chord ‘C’, the user would firstly select the instrument file as shown in
String assignments for the Simple D major chord would be String 1, Note F#; String 2, Note D and String 3, Note A. If the user has pre-selected the option to apply an additional note, then the software will add a further note ‘D’ to the 4th String.
The user can assign chords and special effects to the control members 44, which for example purposes only will be illustrated as follows; the user will select from a window, see
In this example we show in
When using an adjustable control member (42, 45, 46), the user will select and assign the appropriate control members,
When the user activates the selected control member (42, 44, 45, 46), which in this example is a foot-pedal, beyond the minimum threshold level of 5%, then the software will adjust the frequency parameter of the effect in sympathy with the movement of the dynamic range state control member (42, 45, 46).
Additionally there is provided further embodiments of this invention, which provides an entertaining, but less challenging and less educational experience for the user. In this further embodiment, the chords would be rendered by the software in response to the user striking any of the strings 42 of the interactive multimedia apparatus (1, 11, 21), in any sequence of strumming, picking etc. In this further embodiment, the notes of the chords are not rendered as a response to the individual string 42 activation as described in the earlier embodiment. This embodiment may be a starting position of choice for beginners, who would quickly progress to the more advanced embodiment as described earlier in this application. In this embodiment, the user must select the preferred tempo they wish to use. The software will trigger the rendering of the notes of the selected chord at time intervals appropriate to the selected tempo.
Additionally, the chord store is not restricted to the notes of a variety of string instrument types. The chord store could include notes, chords or sounds from any instrument type or from any percussion type instrument or from any wind-based instrument or from any reed-based instrument or from any instrument that is activated by a bow movement or from any device that comes within the classification of a musical instrument, which would be rendered as either individual notes, chords or sounds or combination of sounds by user selection. This invention is not limited in any way by the examples provided in this application or to the instrument types or to the method of application or stimulation.
It is to be understood that the invention is not limited to the specific details described above which are given by way of example only and that various modifications and alterations are possible without departing from the invention as defined in the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| S2002/0580 | Jul 2002 | IE | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IE03/00102 | 7/14/2003 | WO | 00 | 1/4/2005 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2004/008430 | 1/22/2004 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 2986953 | De Armond et al. | Jun 1961 | A |
| 4468999 | Bonanno | Sep 1984 | A |
| 4630520 | Bonanno | Dec 1986 | A |
| 4791848 | Blum, Jr. | Dec 1988 | A |
| 5033353 | Fala et al. | Jul 1991 | A |
| 5455381 | Juszkiewicz et al. | Oct 1995 | A |
| 5723802 | Johnson et al. | Mar 1998 | A |
| Number | Date | Country |
|---|---|---|
| 1081680 | Mar 2001 | EP |
| WO 0195052 | Dec 2001 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20050235813 A1 | Oct 2005 | US |
