Patent Application
20130157761
The present invention relates to a system and a method for a simplified “song specific” keyboard, and in particular to an electronic game that utilizes an electronic board with a limited number of buttons/keys for providing game input as prompted by a game display and usually in coordination with a recorded recognized song or song segment.
The idea of teaching the playing of an actual keyboard instrument through a computer has been discussed in prior art documents, such as US2006/0252503. There are also several prior art systems in the market that try to teach piano through a computer, notably the Miracle Piano Systems, published by Software Toolworks, Nintendo, Synthesia published byHDPiano.com, among others. However, these devices are primarily intended for education, and in general are not effective in having fun with music or learning to play actual phrases and parts of music.
There are also several electronic games that receive game input from musical instruments, such as Guitar Hero, published by RedOctane, Rock Band and Dance Dance Revolution. However, these systems are rhythm action games that utilize mock-type (i.e., not “real”) musical instruments and do not provide significant music education value. A “real” musical instrument, as defined here, has means for selecting the “pitches” of a song and includes both actual instruments and musical instrument simulators that have places on the instrument that can be associated with a “pitch”. “Pitch” represents the perceived fundamental frequency of a sound.
Accordingly, there is a need for a musical instrument teaching method and system that is fun to play as a game and more effective than presently available music teaching systems and methods.
The invention provides an electronic game that utilizes an electronic board, or a controller, or a real musical instrument for providing game input as prompted by a game display and usually in coordination with a recorded recognized song or song segment.
In general, in one aspect, the invention features a melody based game system configured to receive game input by playing one or more song segments including a controller, a computing unit, and a display. The controller has a limited number of programmable real keys configured to be manipulated by a player for playing one or more song segments and thereby providing game input. Each of the song segments includes one or more pitches and one or more of the programmable real keys are programmed to generate the one or more pitches comprised within each of the song segments, respectively. The computing unit communicates with the controller and includes a music application. The display is connected to the computing unit and is configured to display prompts for manipulating the one or more programmed real keys in coordination with playing of the one or more pitches of each of the song segments. The music application simultaneously causes a song segment to be played and provides prompts indicating a manipulation sequence of the one or more programmed real keys corresponding to the playing of the one or more pitches of the song segment. Manipulation of the one or more programmed real keys by the player coinciding with the displayed prompts in pitch, intensity, and timing during the playing of the one or more pitches of the song segment causes a positive scoring event.
Implementations of this aspect of the invention may include one or more of the following features. The computing unit may be a computer, a tablet computer, an iPad™, an iPhone™, a Smartphone a Playstation™, an Xbox™, a Wii™, a PlayStation™, a Nintendo DS™, a game controlling device or a handheld game controlling device. The game system may further includes a database comprising musical recordings and data specific to each musical recording that associate pitches of each musical recording with a specific manipulation sequence of the real keys. The musical recordings may be song segments, song fragments, musical arrangements, instrumental musical pieces, or vocal musical pieces. No manipulation of the one or more programmed real keys by the player coinciding with the displayed prompts or manipulation of the programmed real keys by the player not coinciding in pitch, intensity, and timing with the simultaneously displayed prompts during the playing of the one or more pitches of the song segment causes a negative scoring event. The number of the programmable real keys is in the range between 3 and 11. The number of the programmable real keys is seven. The programmable real keys are touch areas defined on a screen of a touch enabled display. The computing unit may be a tablet computer having a touch enabled display screen and the controller and the programmable real keys are touch areas defined in the touch enabled display screen. The one or more programmable real keys that are programmed to generate the one or more pitches comprised within each of the song segments are designated during the playing of each of the song segments with a designation comprising one of light, colors, shapes, numbers, letters, textures, font type, font size, or key relative position. A melody of any length including a plurality of song segments is played by sequential reprogramming of the programmable real keys. The game system further includes a bidirectional communication mechanism between the computing unit and the controller.
In general, in another aspect, the invention features a melody based game system configured to receive game input by playing one or more song segments including a tablet computer and a controller. The tablet computer has a touch enabled display screen and a music application. The controller includes a limited number of programmable real keys configured to be manipulated by a player for playing one or more song segments and thereby providing game input. Each of the song segments comprises one or more pitches and the one or more of the programmable real keys are programmed to generate the one or more pitches comprised within each of the song segments, respectively. The music application simultaneously causes a song segment to be played and provides prompts indicating a manipulation sequence of the one or more programmed real keys corresponding to the playing of the one or more pitches of the song segment. The controller and the programmable real keys comprise touch areas defined in the touch enabled display screen and the prompts are displayed on the touch enabled display screen. Manipulation of the one or more programmed real keys by the player coinciding with the prompts in pitch, intensity, and timing during the playing of the one or more pitches of the song segment causes a positive scoring event.
In general, in another aspect, the invention features a method for a melody based game system configured to receive game input by playing one or more song segments including the following. First, providing a controller including a limited number of programmable real keys configured to be manipulated by a player for playing one or more song segments and thereby providing game input. Each of the song segments comprises one or more pitches and one or more of the programmable real keys are programmed to generate the one or more pitches comprised within each of the song segments, respectively. Next, providing a computing unit communicating with the controller and comprising a music application. Next, providing a display connected to the computing unit and configured to display prompts for manipulating the one or more programmed real keys in coordination with playing of the one or more pitches of each of the song segments. The music application simultaneously causes a song segment to be played and provides prompts indicating a manipulation sequence of the one or more programmed real keys corresponding to the playing of the one or more pitches of the song segment. Manipulation of the one or more programmed real keys by the player coinciding with the displayed prompts in pitch, intensity, and timing during the playing of the one or more pitches of the song segment causes a positive scoring event.
Among the advantages of the invention may be one or more of the following. The present invention combines melody and rhythm action into a new generation of instrument games, expanding the class of musical genre offered, and the demographics to be reached. Real melody and its corresponding notes are associated with colors/patterns.
The corresponding colors/patterns are then associated with specific “keys”/positions on a musical instrument or a “controller”, so that when a “key”/position is pressed a recognizable frequency (i.e., a note) is generated which is associated with the desired note. When the right combinations of these notes are pressed at the right time or rhythm, then the melody is “played” correctly. The “played’ melody may be audibly heard by the player and/or scored for its correctness. In cases where the “controller” is not an actual musical instrument, but is able to select or generate different musical pitches (frequencies) from its various “keys”/buttons that are programmed by the game console, then in effect, the game has turned the controller into a musical instrument. The colors/patterns may be displayed on a screen/display and the screen/display may be integrated with the “controller’/musical instrument or may be separate from the “controller’/musical instrument.
Melody based action games are different from rhythm based games because they have different pitches (frequencies) that are actually being played, whereas rhythm only based games are associated only with the timing of the actions. Melody based action games have the advantage that they can be used for teaching music while allowing the players to have fun with the game. This is especially true, when the controller is a real musical instrument, such as a piano, keyboard, guitar, or wind instrument, among others. Melody based action games are also challenging and fun to play because they allow the game players to hear the fruit of their success in matching notes to colors/patterns to keys/positions and thereby to produce real melodies. Appendix C lists some further differences between this melody/rhythm game and existing games.
Referring to
Electronic board/controller 720 includes only as many buttons 722a-722d as are needed to play any of the included songs or a song segments. Some songs or song segments may not use all the buttons. In some embodiments there is an indication that only some of the buttons are needed for a particular song. In other embodiments the buttons that are not used are implied by not using buttons of certain color or shape. As was explained above, a song segment is defined as a partial segment of a song in time, a partial segment in terms of which instrumental or vocal parts were included and/or a partial segment in that it is a simplification of the actual musical notes played. In one example, an electronic board/controller 720 configured to play “Mary Had A Little Lamb” has only four buttons 722a-722d, as shown in
One can play a melody of any length by using a sequence of simplified “song-specific” electronic boards. A sequence of “song-specific” electronic boards is actually a sequence of the same electronic board 720 shown in
Since exactly the same set of physical buttons/keys is used for the different sets of pitches of each song segment, it is possible to always have these buttons/keys coded the same way for identification purposes, regardless of the actual song or pitches, provided that we always order the pitches used in the song in an accommodating and corresponding way. In one example, within a song segment, we always associate “blue” with the lowest pitch actually used, “green” with the second lowest, “red” with the third lowest, and light blue with the fourth lowest. New colors are added as necessary. We then map those color/pitches to a linear set of buttons in which the first is colored “blue”, the second “green”, the third “red”, and the fourth “light blue”. In other embodiments, the white keys of a music keyboard, or the adjacent strings or the adjacent frets of an appropriately equipped stringed instrument or the fingerings of a wind instrument, are tinted in a similar way as “blue”, “green”, “red” and “light blue”. In yet other embodiments, footpads, button of a game controller, areas of a touch sensitive surface, areas pointed at (with appropriate detection) or even facial gestures detected are associated with the colors described above. In some embodiments, a series of tinted or otherwise differentiated light beams are used instead of buttons/keys and one plays melodies by entering fingers, hands, legs, another object or the shadow of any of the mentioned objects into the light beam. These various implementations could lead to some novel and dramatic ways to play melodies and other melodic parts on stage or accommodate the limited physically expressive possibilities of a handicapped person.
Library 160 includes, in addition to the recorded songs, data specific to each song which associate the pitches of the song with specific buttons in the electronic board, as will be described below. The display 706 may be a computer display, a TV, a video game console, an arcade machine display or any other display device. Display 706 displays the image of a virtual electronic board 708 corresponding to the actual electronic board 720, colored lines 711a-711c, prompts 714a-714c and a target bar (or landing area) 707. The virtual electronic board 708 is designed to have the same number of buttons as the actual electronic board/controller 720. The displayed image is controlled by the RKM software 750. The RKM software 750 causes specific buttons 712a-712c of the virtual electronic board 708 to become highlighted with different colors depending on the pitches of the song that is played. The programming of the electronic board buttons 722a-722d is also controlled by the RKM software 750. Lines 711a-711d diverge from a common starting point and end at a point directly over the buttons 712a-712d, that are highlighted. They are spaced apart by non-equal spaces and the common starting point may be off the screen. As different buttons are highlighted during the playing of a song, the end points of path lines 711a-711d move to point toward the buttons that are highlighted while their starting point remains fixed. In this way they act as visual pointers toward the buttons that are hit on the virtual electronic board 708 and therefore need to be hit on the actual electronic board 720. Prompts 714a-714c run along path lines 711a-711c, respectively, and extinguish after they arrive at the target bar/landing area 707. The time prompts 714a-714d hit the target bar 707 corresponds to the time the player needs to hit the corresponding buttons on the actual electronic board/controller 720 in order to have a successful score. Buttons 712a-712d, path lines 711a-711d and prompts 714a-714d have the same color, respectively. In one example buttons 712a, path line 711a and prompt 714a are blue, button 712b, path line 711b, and prompt 714b are green and button 712c, path line 711c, and prompt 714c are red. Display 706 also includes a list of the available songs and song segments 118, as shown in
Referring to
In some embodiments, the system also includes a bidirectional communication mechanism 704 between the computer 702 and the electronic board/controller 720, similar to the one shown in
A reverse MIDI keyboard system may also be used to light up buttons/keys 722a-722d in the electronic board/controller 720 from a game controller. MIDI keyboards normally transmit pitch and velocity (among other things) to a processor. The pitch is an integer between 0 and 127 and corresponds to a chromatic pitch. Similarly the velocity is also in that range with ‘0’ being used to indicate off. If we reverse this and consider the velocity number to be a color instead (for instance a color selection from a list of up to 127 colors with 0 representing off), then the processor controlling the button lighting is analogous to a keyboard sending data to the processor, except that it is in the reverse direction. Both include pitch. Instead of 127 possible degrees of velocity we send up to 127 variations of color, using the same bits in the message format that would have been used to send velocity. It is almost as if the computer is pressing a note in return and the harder it presses the note, the more different colors it is able to achieve.
In other embodiments, keys wired in a way similar to a computer keyboard are used. In particular, most Windows PC and Mac keyboards do an adequate job of relaying both key presses and releases to the processor in a timely manner. In these cases, the key-caps of keyboard are replaced with colored key caps. In one example, a row of keys on a computer keyboard is programmed to correspond to the pitches used in a song. In this example, the row A-S-D-F-G-H-J-K-L-;- is used. The keys are marked with colored stickers or are replaced with colored key-caps. One or more players can use the programmed computer keyboard to play alone or in cooperation with each other or against each other using the songs stored in the library 760.
As was mentioned above, the computing unit 702 may be an Xbox™, a Wii™, a Play station™, an iPad™, an iPhone™, a Smartphone, a tablet computer, or any other computing or game controlling device. These type of controllers have very limited digital output bits, which makes it difficult to integrate them with other gaming systems including the systems of this invention. This integration problem is addressed by utilizing the motor output signals of the controlling device. In particular, an Xbox™ controller has two motor outputs. However, the output to these motors is not a steady DC signal that could be considered to be a zero or one. Instead, if a motor is on, the output to it is a steady pulse wave with controllable duty cycle (pulse width). The software specifies a value between 0.0 and 1.0 to the controller and it delivers a wave to the motor with a duty cycle from about 5% to 50%. A table shows what output values produce what duty cycle. By measuring the width of the resulting pulses on the controller, using digital or analog logic or a microprocessor, it is possible to retrieve digital data. Approximately 40 pulses occur per second. By timing the pulses, each pulse can be categorized into several length categories. If there are, for example, eight distinguishable length categories, that yields three bits of information, meaning that we could use each of the two motor signals to transfer reliably up to 120 bits/second, even if we distinguish only eight different lengths of pulses. Combining the two motor signals gives us 240 bits/second. For the lighted keyboard system 700, only one bit of data need be transmitted for each key position since we need only distinguish between on and off. If colors are used in a predetermined successive order (as they can be by the game), then we simply give each “on” key the next successive color. This means that for a 25-key keyboard, we can relay the necessary information for a change of lighting in about 1/10 second, which is timely enough for these purposes.
Another scheme that allows us to control the keyboard lighting with about 1/10 second response, even if the output connection to the keyboard was limited to 10 bits/second includes the following. Since the one or more successive lighting changes for a particular song are known in advance of playing the song, they could each be transmitted before the playing of the song began, even if it took several seconds to do so. In this case, the keyboard has a small amount of memory that remembers each of the lighting configuration for each time a change in lighting occurs, and then, by simply transmitting one bit to start the song and a bit each time a lighting change is to occur the processor can control the lighting of the keyboard even under extremely low data communication conditions. These considerations are relevant for most gaming platforms because they frequently only support a very limited number of controllers. The easiest way to add a new piece of peripheral hardware to such a system is to have it emulate a controller that the gaming platform already supports.
The advantages of the simplified interactive melody and rhythm action musical game includes the following. An electronic board, or a controller or a musical instrument with a small number of buttons/keys is less intimidating than one with many more buttons/keys. An electronic board or a controller or a musical instrument with a small number of buttons/keys is similar to a simple video game control in complexity and therefore easy to use. A player with relatively little experience can play a melody of any length by using a succession of programmed simplified “song-specific electronic keyboard pitch arrangement”. In some embodiments, the electronic board is designed to be easily used by players with many types of physical disabilities. The electronic board/controller/musical instrument is portable and is also adaptable to other interesting interpretations (for instance, waving arms or feet to get different pitches).
Referring to
It is assumed that a “song segment” is a subset of the note pitches of a piece of music, delimited optionally by time section (such as “verse”, “chorus”, “intro” . . . ) within the performance, instrument being used or some other criteria (such as “right hand notes of the piano part”). A “song segment” from the well known keyboard synthesizer “riff” of Tainted Love (made famous by Soft Cell and others) is shown in
In another example, if we were using the same physical or touchscreen keyboard to play Mary Had A Little Lamb, shown in
As was mentioned above, these “buttons” need not be actual buttons or be colored or follow a regular pattern. For example, they could be arranged in other ways in order to be more appealing or more dramatic to a particular audience, to fit into a theme or to coordinate with other activities or needs. Examples of the various button shapes and arrangements are shown in
In
In
For some songs, the simplification is even more drastic, eading to positive player experiences even more quickly.
The coloring need not be different for each pitch and may follow a color scheme based on some other performance or aesthetic need. For instance, the implementation of
The song-specific concept can be used for song-segments with any number of pitches. For example,
People who have limited range of motions, such as having only eye movement, moving only a few fingers or other limitations often have limited chance for creative and/or playful activities. The pitches of a song-specific keyboard are mapped by sensors or other means to the few motions that a person can make, and this allows the person with the limited range of motion to play musical games or experiment with creating music using a set of chosen pitches or using the same pitches as in a preferred song-segment. The motion detection need not be through a physical device. A mechanism such as Microsoft's Kinect™ may be used to detect the motions and interpret them as different keys of a song-specific keyboard. In one example, a patient's leg motions are converted to the notes of a scale, similar to a slide trombone. However, adding an implementation of song-specific technology, means that instead of being limited to ‘n’ notes of the range of a particular modal or chromatic scale, the patient can now play any song segment with up to ‘n’ pitches, thereby greatly increasing the amount of music he can perform.
A song-specific implementation can dramatically reduce the number of keys needed for a particular song-passage, thereby reducing the number of keys from what might be found on a professional music keyboard to what might be inexpensively found on a cereal box toy.
Referring to
As used in an iPad™ implementation, the song-specific keyboard allows non-musician players to more easily play song segments they might be required to play as part of a game. It also makes it possible to do far more in the limited screen size of an iPad™ as compared to the relative ease in positioning pitches on a full-sized music keyboard (about 9″ wide for the iPad™ compared to about 19″ wide for a conventional 25-key keyboard). This is especially important because in a game the limited space of the iPad™ screen must be shared with other elements of game play.
The game display of Rock Band 3™, Guitar Hero™ or Tap Tap™ may at first appear to be similar to the song-specific keyboard. However, there is a fundamental and important difference. The song-specific keyboard defined here has a one-to-one correspondence between the buttons (or other mechanisms or implementations) of the keyboard and the pitches being used in the song segment being performed. That is not true of the three other game displays mentioned above. This is very important because it means that once someone has learned to play a song segment on the song specific keyboard, that same song segment can be played on a piano or similar instrument simply by putting one's fingers on top of the keys for the pitches used in the song segment and repeating the finger motions used on the song-specific keyboard. In other words, once one's fingers are in place, that song-segment can be played on a real piano keyboard in the exact same way it was performed on the song-specific keyboard. That is not true of the other products mentioned because one button on those games results in the playing of different pitches at different times. If the same motions from one of these games were attempted on a real piano keyboard, the fingers would have to move from place to place within the performance in order to get the multiple pitches corresponding with the button. Not only is this more difficult, but the performer would generally not have any idea of which piano key corresponded to a particular button at any particular time.
To be specific, in the song game around “Sweet Child O′ Mine” in Expert mode Guitar Hero 2, within the first five seconds of the performance sometimes the first button from the left results in a musical ‘Db’ pitch and sometimes it results in a musical ‘Gb pitch. A video performance of this song in the game was available at http://www.youtube.com/watch?v=VY_DOeCCbCg. The first two seconds, approximately, of the performance described is shown in
Similarly, a Tap Tap Revenge™ 3 performance of Fireflies now exists at http://www.youtube.com/watch?v=1A9-sgX7qMg. In the first part of the intro, note that the number of pitches used is much more than the three buttons used by the player.
Similarly, in Rock Band 3™ a performance of Bohemian Rhapsody, at about 2:50 a guitar lead line starts as shown below. It has many more pitches than the five buttons available to the player, but we will just consider the first five notes, shown in
The numbers indicate the buttons numbered from the left, or respectively green, red and yellow in color.
Note that button 2 plays an F at one point and then perhaps a second later plays a Bb. Also note that the pitch of ‘G’ is played by button 3 at one point and by button 1 at another. Clearly there is no one-to-one correspondence between buttons in the game and pitches in the music.
In any of the last three existing products shown, it is not possible to play the passage on a piano by merely placing your fingers on the appropriate keys and making the same finger motions in rhythm that you did in the game. Compare that to the two examples of the same two passages illustrated above as they might be performed on a song specific keyboard, each with an illustration of a song-specific keyboard that could be used and the corresponding finger positions to play the same passage on a piano, as shown in
Referring to
Referring to
Referring to
The system also includes a bidirectional communication mechanism 104 between the computer 102 and the instrument 120. Mechanism 104 causes specific keys 122a-122c on the actual keyboard 120 to get highlighted based on signals controlled by the RKM software 150 and received from the computer 102 during the playing of a song. Mechanism 104 also transmits signals from the keyboard keys to the computer 102. The highlighted specific keys 122a-122c on the keyboard 120 correspond to the pitches of the song that is played. In one example, mechanism 104 is implement via bidirectional musical instrument digital interface (MIDI) communications. MIDI is an industry standard protocol that enables electronic musical instruments and computers to communicate, control, synchronize and exchange data with each other. A MIDI instrument or controller transmits “event messages” such as the pitch and intensity of the musical notes that are played, control signals for parameters such a volume and vibrato and clock signals to set the tempo. When a musical performance is played on a MIDI instrument (or controller) it transmits MIDI channel messages from its MIDI Out connector.
A typical MIDI channel message sequence corresponding to a key being struck and released on a keyboard includes the following:
The user presses the middle C key with a specific velocity (which is usually translated into the volume of the note) and the instrument sends one Note-On message.
The user changes the pressure applied on the key while holding it down—a technique called Aftertouch and the instrument sends one or more Aftertouch messages.
The user releases the middle C key, again with the possibility of velocity of release controlling some parameters and the instrument sends one Note-Off message.
Note-On, Aftertouch, and Note-Off are all channel messages. For the Note-On and Note-Off messages, the MIDI specification defines a number (from 0-127) for every possible note pitch (C, C #, D etc.), and this number is included in the message.
In one example, a MIDI channel message includes the following four parts:
(1) the “command” - - - in this case NoteOn (0x80) or NoteOff(0x90)
(2) the “channel” - - - any channel could be consistently used, for example channel 1.
(3) data1 - - - this is the MIDI pitch
(4) data2 - - - for NoteOn, this is the “velocity”, which in the present invention is used to indicate the type of highlighting. NoteOff is then used to turn off the highlighting for that note.
Many keyboards do not generate “Aftertouch” messages and sometimes a NoteDown message with a velocity of zero actually indicates a Note release, making it a substitute for the NoteUp messages. In this implementation the velocity is used as the note highlighting for messages that the processor sends to the keyboard. If the velocity were non-zero, the velocity could be interpreted as an indication of how a note should be highlighted, selecting one of up to 127 possible colors, for example. Alternately, the seven bits of the velocity could be interpreted as a color, using for instance two bits for red intensity, three bits for green intensity and two bits for blue intensity. This would result in three possible red illumination levels (besides off), seven possible green intensity levels besides off, and three possible blue levels besides off, which can then be combined in a red-green-blue (RGB) fashion into one of 127 possible colors. Since the colors need to be easily distinguishable from one another in the described application, it is hard to imagine a situation in which more than 127 color shades might be needed. But if that should become necessary, additional bits could be conveyed. For instance, if two notes on messages for the key were sent in rapid succession, the velocity data of the second message could be interpreted as additional bits, giving a total of 14 bits instead of 7, and 17,367 shades besides completely off.
In implementations where independent signals are used for the three RGB components, one MIDI channel is used for red intensity, one for green intensity and one for blue. Similarly, the highlighting may utilize a subtractive color technology rather than an additive one, and the intensities of concern will be cyan, yellow and magenta instead of red, green and blue.
In alternate implementations, the “highlighting” method may not use color at all and it may have very few or even just one level (or color) other than off. For instance, keys may be physically moved slightly to show that they were highlighted or made to vibrate or physically “hum” or be of a different temperature or conduct heat differently, which one could easily discern through touch.
Another way to convey the control information through MIDI is to convey it as “controller” information rather than as “note channel” information. “SysX” technology may be used in this implementation. The exact use of channels and message content is defined almost arbitrarily and combined in various ways with other messages, but the meaning attached to the messages includes the highlighting message info above.
In any of these systems, there may be “panic” messages which causes all highlighting to reset to off. In addition to or in place of the messages above instructing single keys to highlight, “bulk” messages may be used which would give the keyboard a new highlighting scheme to use in place of the current one. These messages may be implemented in MIDI, XML, JSON or some other technology.
In one particular implementation, the music keys are considered to be numbered from 0 successively left to right, regardless of their MIDI pitch. In this implementation the f instruction: “2,0;3,1;4,2;7,3;10,4;11,5” tells the keyboard to discard any previous lighting and to
Highlight key number 2 in color 0
Highlight key number 3 in color 1
Highlight key number 4 in color 2
Highlight key number 7 in color 3
Highlight key number 10 in color 4
Highlight key number 11 in color 5
The remainder of the keys are not to be highlighted. In this system, the actual colors are taken from a predefined, fixed list of colors. For example, 0:blue, 1:green, 2:red, 3:yellow, 4:cyan, 5:orange. If the keyboard starts with a ‘C’, the corresponding pitches will be D, D#, E, G, A# and B, respectively. However, it would also be possible to have a system which downloaded the list of colors at the beginning or periodically as necessary or desirable.
In the embodiment of
Referring to
The division of a song and its vocal and instrumental parts into song segments is arbitrary. A person or process breaking a song into song segments can make choices to please the needs and abilities of an intended audience, opting to try to keep the number of pitches used near or below a particular number. In one example, a simple automated process, could decide to use 16-bar segments of tracks in a Musical Instrument Digital Interface (MIDI) file for the song segments. That would work, but better results can be achieved by considering the aspects of particular songs in a more detailed manner, whether manually or in an automated process. In one embodiment, represented by the examples above, the number of keys used within a song segment is kept around seven. In other song segments between three and eleven keys are used. This requires skill and discretion in choosing the song segments. Once, the song segments are defined, each can be seen to include certain pitches (or “perceived fundamental frequencies” or keys) and not others and that defines how it will appear and be used on the keyboard.
A “song segment” may contain a certain number of pitches, some of which may be played more than once. In one example, the melody to “Mary Had A Little Lamb”, as commonly played, has four different pitches, but all are used more than once, as shown in
It is not necessary that this target song segment be performed in the original work by the same type of instrument used by the player. For instance, a keyboard could be used to play a vocal melody line, a synthesizer used to play a brass line or a guitar lead, a guitar used to play the notes of a string section line, among others This sort of substitution often happens in real musical ensembles, so it is not even unnatural to do it here.
The song segment may contain more than one pitch played at a time. For instance, it may be a left or right hand part of a piano selection. Alternatively, the pitches played by both hands maybe considered together as the pitches of a song segment. Although, as we shall see, there are advantages to defining the song segment in such a way as to include a limited number of pitches, that restriction is not included in how we are defining a song segment.
It might even be played by more than one instrument or player if, in the mind of the listener, it might meld together into a musical concept. For example, a set of tympani is a set of individual tuned drums that are physically independent from one another, but the melodic line produced by using all of these drums together might be considered to be a single song segment. In another example, classical composers at times have written melodic lines that run from the violins through the violas ending, for instance, on the cello. The composer may conceive and the listener may hear this as a single melodic line, even though it is played serially by different players on different instruments. The part played by the player may be transposed from the original. It might even be simplified from the original.
As was mentioned above, the keyboard 120 has some means to highlight a set of keys on the keyboard (or fret positions on the fretboard). Though it is of some use to highlight all of them in the same way, greater benefit can be obtained by varying each highlighted key in a different way. For example, on a keyboard with lighted keys, the non-highlighted pitches might be unlit and the highlighted pitches lit. Though it is of some use for them to be all one color, it would be a functional improvement if more than one color were used and better still if each color used was unique and easily distinguishable.
For instance, using the “Mary Had a Little Lamb” example in the key of C, the pitches would be C, D, E and G from lowest pitch to highest pitch. We might light them respectively blue, green, red and yellow 122a 122b, 122c, 122d, respectively, as shown in
Using the four color representation, one could think of “Mary Had a Little Lamb” as red-green-blue-green-red. However, on keyboard 120, not every key is colored. In fact, all of the keys which are not selected are not colored or colored the same or identified by some unique changeable characteristic. The keys that are used are colored, as was described above, and remain colored during the entire duration of the song segment. Therefore at a glance, the player can tell which keys do not have to be considered for a song segment and which ones will be used. The keyboard 120 ideally illustrates this for all the keys even before the playing of the song segment begins. This allows the player to quickly consider how to place his hand or hands, or feet for a pedal clavier, in such a way as to be able to easily and quickly access those keys, ideally even before the playing of the song passage begins. This described the concept of the “focused” keyboard.
This is very different from approaches that indicate one or sometimes more notes on a keyboard or on a keyboard representation just before they are used, such as Piano Wizard or Synthesia. In these prior art cases, the keys have all the same or different colors and there is not any form of constant highlighting of the keys used in the song and, more importantly, constant non-highlighting of all the keys not used in the song, even when they occur physically between keys that are used, such as the F in “Mary Has A Little Lamb”.
In the focused keyboard 120 of this invention, it would be considered functionally better if the same colors were used. The present approach does not use one color for C's, another for D's, and so on. Instead we use one color for the lowest note actually used in a particular song segment, another for the second lowest note actually used, and so on. Therefore, if we use a different song segment, or a song segment played on a different key, we may use the same colors, but they will generally be on different keys since the keys used by the new song segment or the song segment played on a different key would be different. In the example of
It is possible that a song segment might encompass an entire song from start to finish, such as in “Mary Had A Little Lamb” above. However, in many cases, more than one song segment might be used in succession. For instance, there might be one song segment for the intro, one for each verse, one for the first part of the chorus, one for the remainder of the chorus and one for the ending. Since it is desirable to have not more than a few pitches in each song segment (for example, less than twelve and, when reasonable, no more than six), in complicated musical passages or those involving chords, it might be desirable to change to a new song segment quite often, sometimes even within a measure.
Referring to
Many alternative coloring strategies can be accommodated by the focused keyboard 120. Referring to
A focused keyboard's 120 color (or other designation) selection could in some cases be set up specially for something highly dependent on the particular song segment, for instance making things easier to grasp by capitalizing on some symmetry in the music. Referring to
Referring to
There may be circumstances that make one scheme preferential over another, such as cost considerations or a principle in a music lesson being taught. For example, the scheme of
The controlling software or hardware 150 used with the focused keyboard 120 may always consistently use the same approach to key coloring. Alternately, it may use different strategies at different times, either chosen by the user's preference or chosen within the usage context, for instance, keeping pitch colors the same when pitch-oriented lessons were given and keeping the color for the thumb key the same when teaching lessons about fingering.
Ideally the game or other mechanism allows the player to first work with each song segment individually, possibly even allowing it to be slowed down or advanced a note at a time. That way, when the song segments are used together and the highlighting changes (ideally as the last note of the previous segment is being played), the user is ready and expecting the “focus” change since they have already played the song segments individually. In another possible embodiment, a focused keyboard 120 or fretboard could be used without playing with recorded sounds or even trying to play in rhythm. It would still help the player to only focus on the keys used for the particular passage.
The present system doesn't lead the user through, note by note. Instead, in system 100 the notes actually played during a song segment are highlighted, whether or not they are a part of a particular chord or scale. Rarely this might coincide to come up with the same set of keys, but usually it would be quite different. That's because many of the song segments are melody or counterpoint lines and these usually do not simply follow the notes of a chord or scale up and down. In one example, a song segment covers a chordal accompaniment pattern that includes playing a C and F chord, possibly in “broken fashion”. The notes of the C-chord are C-E-G, the notes of the F-chord are C-F-A and the notes for the entire C-Major scale are C-D-E-F-G-A-B-C. The highlighting of the notes on the keyboard is based on the notes that are actually played in the song segment, i.e., C-E-F-G-A, as shown in
Expanding further with a musical example, a simplified right-hand accompaniment pattern to John Lennon's “Imagine” is shown in
In contrast, a non-focused keyboard which highlights chords, may display the left group of notes 572 for the odd-numbered measures C-E-G and the right group of notes 574 for the even measures C-F-A, shown in
The scoring of the game is described with reference to
If a note is matched pretty well, it results in an explosion 117a, as shown in
Referring to
Game 100 is able to accurately match and score notes in real-time, as fast as they are played. Prior art games may appear to do a similar thing in their scoring, but there is an important difference. While prior art games may allow graduated scoring for notes (or steps) that are slightly off in time, they do nothing that compares to graduated scoring for notes that are different but close in pitch. In the present game, any of the twenty-five keys on the standard two-octave keyboard might be pressed at any time, whether lit or not and partial score and matching will be considered for even unlit keys that are played at nearly the right time as the lighted note. This is doing matching in two dimensions (time and pitch) rather than just in time, i.e., the problem space is geometrically larger.
Since this is a music game, what the player hears when playing is of paramount importance. There is actually some flexibility in what sound might best be offered. Because the audio sounds live and involving and closely matches what the player is playing on the game keyboard, the player feels as if they are part of making the sound.
This can be the case even when the sounds heard do not reflect or include what the player is doing at all, particularly if the player is doing well. A similar situation can happen with real musicians in a group. If one is playing clarinet in a concert band with a lot of clarinets and the nearby other clarinets are playing the same part, it can be hard to discern one's own playing and even if one stops playing, it may not sound that much different. All the parts playing the same notes blend in together and it is difficult to discern the instruments individually. Comparably, if the sound played from the player's keyboard is very similar to the sound on the recording and they are doing well at playing the same part with the recording, they may feel as if they are hearing themselves play, even when only the recording is being heard. This effect is strengthened by the fact that the ear hears sound intensity in a logarithmic way. When two instruments play instead of one, the combined sound is three decibels louder, but that is not heard as a doubling of the sound volume.
There are situations in which the player might really want to blend in well and want the sound of their own “instrument” to closely match the sound qualities of the associated instrument on the recording and want to hear their part only quietly or not at all. For instance, when showing off for friends, one might want to sound just like the player in the recording, be heard distinctively a little, but not be too obvious when wrong or mistimed notes were played. On the other hand, when the player is alone and seriously wanting to do better, they might want to be able to hear their part clearly and distinctly by having it loud compared to the recording and possibly of a somewhat different tone quality. For example, the recording might be using an electric piano sound but the player chooses to hear a different and distinguishable model of electric piano (say Wurlitzer-like instead of Fender Rhodes-like) or even chooses to hear their playing with an acoustic piano or perhaps a marimba sound. This can be very helpful when learning, since the player can relate to both the similarities between their playing and the audio track as well as the differences. Another effect that can be controlled by the computer and used separately or in combination with this is to play the players part and the recorded part in two different places spatially, for instance the audio track in the left stereo channel and the player in the right stereo channel.
In cases where the game is using an audio recording in which it can separate the instrument that corresponds with the player from the other instruments in the recording, there are even more useful possibilities. For instance, the corresponding instrument in the audio track may be dropped altogether, leaving only the player playing that part in the result.
If the player can control the mix, they could also turn the audio recording off altogether, only hearing their own playing. A player might also find it fun, interesting and useful to remix the levels of the instruments and voices in the recording, to the extent the audio recording made it possible to do so. In the current implementation, the player can change the mix between game audio and their own playing at any time. On the screen, they can see the current mix represented by the brightness of the keyboard icon compared to the brightness of the band icon, as shown in
In one embodiment of game 100, there is an audio track (stereo pair of tracks, actually) that contains what the player is to hear during the game. Most often, this is the original audio of the song, as performed in a well known version by the creating artists without modification. This is one reason why players and bystanders seem to love the sound of the game. In some cases, this audio is slowed down or even slightly speeded up for the benefit of game ease or difficulty. In other cases, this audio may be transposed (pitch changed) so that the player's part on the often two-octave keyboard might correspond properly to the notes. The only sound that is added in this case are a few “lead-in” beats that alert the player that the audio is about to begin. The sound of these metronome-like lead-in beats is generally percussive in nature, though it could be a voice (“one, two, three, four”) like a band leader. Currently the game uses one hi-hat cymbal sound for the downbeat and another for the other beats in the lead-in, though other prototypes used bongo sounds or other instruments.
An alternative to supplying an audio track in the above manner is to supply MIDI-tracks (played through some MIDI-compatible synthesis method) or a combination of the two. One advantage to the audio is that it sounds more natural to the player and avoids the need to supply software and/or hardware to play the described MIDI-tracks. In one embodiment, the above-mentioned audio and the generated lead-in beats is all that the player hears during the actual playing of the song during the game. Between songs, the keyboard does other things, as described elsewhere. When the audio needs to be slowed down excessively, it may sound unrealistic and even grotesque. In these cases when a usable and pleasant slowed down audio track is not available, it may be preferable to substitute with a MIDI equivalent. That does require the presence of MIDI software or hardware within the system or external to it which might not be necessary otherwise, since the simple lead-in beats can easily be done by other means. Another possibility to support playing very slow (typically less than 50%) of the original tempo is to use alternate audio tracks specifically designed to be slower performances of the same songs. These might be recorded by the original artists or by others.
There are other aural things that can be played along with the playing of the song. The existence, strength and mix of these might be determined as a player preference or fixed values might be tested and built into the game. These possibilities include the following:
a) Hearing what the player is actually playing on a similar or dissimilar instrument. A similar instrument blends in better. A dissimilar instrument makes it easier to hear the differences between what the player is playing and the actual part in the song. Either may be a good choice depending on circumstances. In some embodiments, we have played the player's part in a different octave to make it similar, but distinct that way.
b) Hearing the desired part played on a synthesized similar instrument. In this case, audio may be used that deliberately does not include the part of the desired instrument, which might require audio specially produced for the game.
c) Hearing a deliberately somewhat dissonant version of the desired track played if the player's performance has errors which trigger this. This “error” track could be accomplished in either audio or MIDI. After the player has met some criteria and/or after some delay has passed and/or after some new part of the song is reached, the playing of this “error” track would stop and the sound would revert to what it had been.
d) Hearing some rhythm parts made louder or with more emphasis in some or the entire song excerpt to make it easier to follow the beat, though with current music this seldom seems necessary.
e) Reinforcing or otherwise making the featured (often keyboard) part of the song more emphatic and easily heard, possibly by changing its strength or tone color or quieting or silencing less relevant instruments or voices.
There are a lot of possible choices here. They could be dictated by player preferences, considered preset choices, circumstances within the game or some combination of these.
In the embodiment of
As was mentioned above, in the embodiment of
Referring to
Some keyboards have small speakers built-in, as shown n
A minimum of four colors or other distinguishing mechanism would be desired for a reasonable game. If only a small number of colors were used, they could be repeated again in order on higher notes without adding an intolerable amount of confusion. However, the game works significantly better, when each key that is lit has its own color. In some embodiments, between one and nine colors per song excerpt have been used, but there is nothing magical about the number “nine” and more colors could be available and used on some songs.
From an electronic point of view, keyboard 120 (or any other musical instrument) signals the pressing of notes electronically to the game processor. It is highly desirable that it signals their release as well, but one could still have a good game without that feature. Keyboard 120 also accepts electronic signals telling it to make a particular key a particular color.
Other desirable features of game 100 include the following among others:
Keyboard 120 may have a small amount of memory that would allow it to store the sounds of instruments recently used in the game and/or instruments in the game specified by the player. This will be useful in cases keyboard 120 is temporarily removed from the game. It may also have other built-in sounds as well. With enough memory and appropriate onboard intelligence, the keyboard 120 may even be able to download and store audio song samples from the game so that the player could play along with them even without the game. In some embodiments, the keyboard 120 incorporates the computer 102 and/or display 106.
Other embodiments of the invention may include one or more of the following. The keys in the focused keyboard 120 may be designated with shapes, numbers, letters, textures or type font size, instead of color. Another possibility is to reposition keys to show their status in the song segment. For instance, the keys not used could be slightly lowered. This may be done with or without another designation system, such as colored keys. The “prompting” display of the keys, as was described above, shows the keys used by colored dots arranged in order of pitch in one dimension and in order of times in the perpendicular or another dimension. These dots move towards a target position 107, as shown in
Furthermore, in other embodiments, two (or more) song segments for the same song are shown and used simultaneously together. For example, something with both a piano part and synthesizer part, as is common in Elton John recordings. Alternatively, it may be two or more synthesizers, or a piano and organ, or even two or more string, horn or vocal parts. Both parts are shown on the computer display or on two different displays or even on two displays in two or more different locations connected by wiring or some sort of network. Games/learning exercises that can be played in this setup include the following among others:
One player plays two parts on a larger keyboard, one with each hand. A third or even a fourth part could be played with pedals in the style of a classical organist or some synthesizer artists. One player plays two parts with a hand on each of two keyboards. Again, adding pedal parts is possible. The players may use the game to work up to this complexity one step at a time. Two or more players may each play one part on a larger keyboard or instruments. Two or more players may each play one or more parts on separate keyboards or instruments.
Combinations of different instrument may be used, for instance, keyboard and fretboard. The fact that each part is presented in a focused manner to the player involved would help keep the complexity manageable, and this would, of course, very much be like real musicians playing together in ensemble.
In other variations, two or more players are involved with the same song segment or segments. Some variations include the following. The players alternately play the song segment in a competitive manner. They might use the same keyboard (or instrument) or they might each use their own, particularly if they were in different locations. It is also possible to set up a competitive situation if they are not playing the same song segment scenario and/or level, but to be fair one would have to handicap if the difficulties involved were different. Similarly, a handicap system could give a weaker player a chance to compete meaningfully against a stronger player. It would also be possible for a player on a keyboard to compete with one on a fretboard given a handicap system or other means to compensate, even though the ease of playing the part might favor one type of instrument over another.
In other embodiments, instruments other than keyboards are used. They include guitar with lighted fretboard, bass guitar lighted fretboard, soprano sax lighted keys, alto sax lighted keys and EWI lighted keys, among others, as shown in
The present invention also relates to a method and a system that associates musical notes to colors and to “colored” keys/buttons on an electronic board. The notes are associated with the melody at specific frequencies (pitches) of a song or a song segment. The notes are then associated with specific colors displayed on a screen and the key/buttons are associated with the specific colors. In one implementation, the system is used in a Melody Rhythm Action Game application. The game is played by hitting the keys/buttons corresponding to a specific color at the same time the specific color is displayed on the screen, thereby playing (matching) the notes that are associated with the melody of the song or song segment. The key/buttons on the electronic board are colored by light of a specific color (either reflected or transmitted), or by paint.
The invention also relates to a method and a system that converts any game controller/electronic board with a few buttons into a musical instrument (“simplified interactive musical instrument”). By definition a musical instrument is simply something that generates notes/frequencies when struck or played at different positions/keys. Thus when specific frequencies are assigned to specific positions/keys/buttons, and sound comes out when the keys/buttons are hit (i.e. played), then by definition a musical instrument is created. In one example, a game controller with 5 keys/buttons is used to play any pattern of melodies or “music”.
Several embodiments of the present invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. Accordingly, other embodiments are within the scope of the following claims.
What is claimed is:
The notion of using an ordinary appearing piano (or other) keyboard (or fretboard, etc.) as a song-specific keyboard (or fretboard, etc.) might seem alien at first. However, considering it from another point of view, it can actually be quite simple and be viewed as a way of “tuning” a keyboard to a song, much like how guitarists sometimes retune their instruments for different songs or how a pedal steel guitar, orchestral harp or tympani are all dynamically tuned by means of pedals.
I'm going to talk about one and two octave keyboards here though the principles can be extended to keyboards larger and smaller other variants.
Here's a conventioanl two-octave keyboard. On it, I have written how the keys are ordinarily “tuned”.
Note that as commonly used, the black keys can be called by two different names, depending on whether they are referenced to the white key to the left (in which case they are “sharps” or to the white key to the right (in which case they are “flats”.) The notation G# is pronounced G-sharp. The notation Gb is pronounced G-flat. Nothing new here so far. This is just how keys are usuallly named on a piano keyboard. A keyboard with sharps and flats is sometimes called “chromatic”. Fretboard and wind instrument fingering is similar in some ways and different in others but accomplishes the same purpose.
Not all instruments can produce all of the chromatic scale tones shown above. For instance, most harmonicas, bugles, and tympani cannot without going to extraordinary measures within a song. So it is possible for an instrument to be melodic without being fully chromatic. In fact, to play a melody only really requires being able to play the notes that are used within that particular melody and that will be our point of departure from typical keyboard usage, as we will see below.
On most musical instruments, the same key usually produces the same pitch all the time, with the exceptions for some instruments noted above. For some wind instruments, the wind pressure and mouth position may also affect which pitch is produced.
It is common for instruments to go from lower to higher pitch from left to right and we will generally follow that convention (though some artists such as Joe Zawinul arenoted for setting up their keyboard from right to left, partly to encourage experimentation).
Let's consider the notes in what I have defined elsewhere as a song segment. It could be a whole melody, a part of a melody or some other significant and recognizable part using notes with specific pitches.
We can consider all of the notes that are used anywhere within a particular song segment. We can order them in pitch from low to high and number the lowest one “1”, the second lowest one “2”, etc. There could be dozens of pitches involved within a song segment, but that would be unusual. If we limit how we define the song segment in time, complexity or other means, we can scope it small enough so that the number of pitches involved is thirteen or less. In fact, very recognizable song segments often only have about seven pitches used and some have as few as three or four. (One well known samba has only two used in the entire chorus - - - that's interesting because it means that we can produce recozniable music sometimes with as few as two keys - - - so using up to thirteen will actually cover a large amount of ground! Even seven will allow us to do many interesting and recognizable things.)
Frequently we'd used colors instead of numbers, such as blue for the lowest note, red for the second lowest, etc. following some arbitrary scheme, but here I will continue using numbers because it will make it clearer to illustrate the principles. But keep in mind that if we used a coloring scheme such as the one just mentioned, the coloring and key position for the lowest note, the second lowest note, etc. would remain the same for each song, so keys could simply be painted particular colors and then used for song after song provided something like a computer “retuned” what pitches the keys actually played for each song or song segment.
I'm going to use conventional piano-style keyboard to show some of the possibilities for laying out the pitches in a logical and playable manner but we could use the same principles on a fretboard, wind instrument or anything at all that could be logically divided into designated “pieces” that could be “played” in some fashion.
Keyboards often run from “C to C”, meaning that if pitches were assigned in a conventional manner to the keyboard, it would start and end with a C. For instance, here is a two octave C to C keyboard, though I'm not going to label the keys because I want to emphasize their pattern.
Note that in many of these examples, the final key on the right is retangular, even though if the same pitch were in the middle of the keyboard, it would have a chunk taken out of an upper corner (or two) to accommodate the positioning of a black key.
There can also be a one octave C to C keyboard.
A one octave F to F keyboard is also a common possibility and looks almost the same except for the positioning of the black keys.
In fact, you can logically see a one octave F to F keyboard as a subset of a two octave C to C keyboard.
To make things easier for the player/beginner musician, we can logically pick out which keys to use. Usually we'll only use about seven, so even on a one octave keyboard there will be unused keys. In fact, even though I will usually show about a dozen keys numbered, for a particular song we would only use as many as were necessary. For instance, for “Mary Had A Little Lamb”, a four-pitch song, we would only use the keys numbered 1, 2, 3 and 4.
That said, here is one simple way to use a two-octave keyboard. In fact, we will only use white keys:
I'll call this a two-octave linear mapping.
Since we generally only use seven or so notes, one might comment that when we played a song-segment it would usually look as if we were always playing down on the left side of the keyboard. Since we have an excess or notes, let's do a mapping starting nearer the center. If we run out of notes, we can always add a few black keys or even go back and use some of the white keys that we skipped:
If we only use about seven notes, it will seem as if we are playing near the center of the keyboard most of the time. Ill call this “Linear in G” since I started on a G with key number 1. I would have started anywhere, but I liked how this was centered and how I started with a group of two black keys between the first thrree notes.
We could also have started one key to the left, for example. When I do that, I call it Linear in F.
With two octave, we can usually get by just using the white keys. If we were trying to save money on the design or construction or coloring issues of such an instrument, the black keys would not necessarily have to move or function at all! They might be merely decorative or help the player find a particular key position by pattern easier. (Of course, we don't neccessariy have to use the two black key, three black key pattern of a piano. The keys could be any fashionable pattern or might even be arranged like the 144-bass buttons of an accoridion or some entirely new arrangement.
For size, economy or other reasons, it might be desirable to scale down to a one octave or possibly even smaller keyboard. In that case, merely using the white keys might still be enough since there are eight of them, but sometimes we might need a few more pitches or perhaps some players might enjoy using black keys as well as white keys. We could use any arbitrary system to assign pitches, but here are a few I have tried:
One octave C to C Chromatic order
This is what I call “Downstairs Upstairs in C” order, first using the white keys, then moving up to use the black keys.
One octave F to F chromatic. (This can also be another way to use the center section of a two-octave C to C keyboard.)
One Octave Downstairs Upstairs In F
I've also done it with typewriter keyboards and even with an Xbox controller. If these are done with colored keycaps or buttons instead of numbers they are much easier for a player to match, but I am showing numbers here to be clear about order.
Note that there are so many keys on a Qwerty keyboard or on a multi-octave music keyboard, that multiple players could play at once on the same keyboard competitively or cooperatively, with the same sound or different sounds. Or multiple players could each have their own keyboard (music, qwerty or even Xbox).
Here's an Xbox controller mapping that I've used to play melodies. (It felt pretty strange and reminded me of playing an ocarina, but I think I could get used to it with a little practive.)
Many other variations are possible. One might ask, why might want to have a two-octave C to C keyboard when one is only using the center F to F section. A number of reasons are possible. For one, the player/musician, espcially “wannabe's”, might like the appearance of the larger keyboard. Another reason is that the game or instruction might later advance to also using the keyboard in a conventional way, in which case having the full two octaves or even more might be a very handy thing. Or perhaps, at a later time, possibly as an upgrade, the full two octave keyboard could be used with added lighting in the way described in the Focused Keyboard, teaching fingering skills for melodies that diectly transfer to most keyboard instruments.
How this Melody/Rhythm Game Differs From Existing Music Games
This is a summary of the major differences in our approach to a music game compared to existing technology and games. Note that in this discussion “key” refers to a key on a keyboard (or a fret position on a fretboard or other physical position on other instrument) where the key produces a musical “note” or a particular “pitch” (roughly, frequency). Therefore, in this discussion, the terms “key”, “note” and “pitch” are closely associated and are often interchangeable. (Note that “key” here does not refer to key signature, i.e. C Major or F minor.)
There have been a lot of keyboard games of various types in which result in the player playing a melody or melodic part. Most notably, there has been The Miracle Piano System (known by several other names) and more recently Piano Wizard. Some of these games, such as Piano Wizard, use a color coding that is related to the pitch of the note (i.e. the keys with a C pitch are one color, the keys with a C# pitch another, the keys with D still another, etc.). Note that in a game like this the colors are not related to the particular song played. Other methods, such as Piano Wizard and Casio lighted keyboards, use a single color for all the keys or use no color at all. Note that none of these systems distinguish between the keys actually used for a particular song or song part and those not used within that song or song part.
The games described here, on the other hand, do distinguish between the keys actually used in a song or song segment and those which are not. Accordingly, the designations on the keyboard and/or designations referring to the keyboard change depending on the song or song part played. In some cases, they may change within a song from one part of the song to another. This differentiates this game from others in this category.
Also note that in embodiments of these games that support it (by the hardware used), the colors used are not related to the key pitch. For instance, in one embodiment described, the lowest key is always one particular color (say “Blue”) regardless of whether the lowest pitch is a C, a D, or an E-flat. This also differentiates this game from others.
Melody/Rhythm Games compared to Rhythm Games
There are some games currently available, such as Guitar Hero and Rock Band, in which the player taps buttons or performs some similar action in time with and response to the rhythm of the game in most cases also prompted with a visual display. While in these game the player might tap out a melody of, say, a guitar solo in rhythm with a recording of the guitar solo, there is not a one-to-one correspondence between the pitches of the melody and the buttons on the instrument, i.e. the same button might be used to respond to a F pitch at one point and shortly thereafter respond to a F-sharp, G, or some other pitch, generally without the player being aware that anything had changed. In some cases, the same pitch may be represented by a different button in another part of the song or one button might correspond to more than one pitch being played. In the games described here, the same key or button always corresponds to the same pitch within the given song or song segment. Within a song segment, one key means one particular pitch and that particular pitch is only mapped to that one key.
Melody/Rhythm Games compared to Melody Games
There are some games, in particular on some Casio keyboards, where a key is lit and when the player plays that key the next key used in the song is lit up. Thus, the player is led note by note through a song. (In some cases, more than one light might go on indicating that two keys are to be played simultaneously). The only lights that are on are, at most, the key or keys being played and the key or keys to be played next. The game described here differs from that in two significant ways: all the keys used in a song or segment are designated at once for the length of the song or song segment, not just immediately before they are needed - - - and - - - in many of these other games, the player sets the rhythm by the speed that they play speeding up or slowing down at their will whereas in our game the rhythm of the song is steady and the player must play at the same speed to get the best score—the speed of the song does not vary with the speed of the player.
The distinction between game controllers and actual musical instruments can get quite fuzzy these days, particularly where computers are involved enhancing and transforming either. Most electronic games these days now involve a computer processor and most electronic musical instruments also involve a computer processor.
Therefore, what the player actually “plays” in the game described can range from a game controller to a computer keyboard to a toy musical keyboard to an actual musical keyboard to a fretboard to the keys of a wind instrument or to the fingerboard of a bowed instrument to moving ones hands in the air to other embodiments. The game could be played using any distinguishable movements that could be associated with pitches. The associated instrument need not even be electronic, since its sound could be detected, for instance, with a microphone and/or the player's movements by a camera or other motion sensing mechanism.
Most of the descriptions here will concentrate on a game being played using the keys of an electronic musical keyboard, but in each case, it could be a fretboard, a toy instrument, a computer keyboard, a game controller, a touchscreen or any of the embodiments mentioned above. In the process of testing, this game has already been played on a music keyboard, a computer keyboard, a game controller and toy musical instruments models. This represents only a few of the possible ways to implement this game.
It is also worth noting as a summary point that in some implementations that keyboard (or fretboard, etc.) is used only as a passive input device to the game, but in other embodiments, the game controls features of the keyboard in a way to correspond with game usage, most notably to illuminate or otherwise highlight (or further highlight) the keys that are actually used within the current song or song segment. Depending on the particular embodiment, all the keys so used might highlight in the same way or, alternatively, several means of highlight (such as several colors) might be used to further distinguish one highlighted key (or button, etc.) from another. Further, in embodiments where highlighted keys are further distinguished from one another, the highlight for each individual key might be unique (such as a unique color) or it may be one of several types of highlighting (for instance, when the hardware is incapable of supplying a unique color for every highlighted key). In these latter cases, the highlighting system may or may not correspond with the pitch “letter” of the key, i.e. the same color of highlighting may or may not be used for all pitch “C's” and if it is necessary to use the same type of color or highlighting more than once, the keys highlighted with one particular color may not necessarily be the same pitch.
Appendix C illustrates several types of highlight schemes possible for the song segment consisting of the first two lines of the popular song “Take Me Out To The Ball Game”.
The following examples use the melody of the first two lines of the popular song Take Me Out To The Ball Game as a song segment. Though the actual music notation and pitch letter names generally do not appear directly in the game, here they are for reference:
If the keyboard used for the game embodiment is one only capable of highlight keys one way, the highlighting of the keyboard for the playing of this song segment might look like this:
In another embodiment of the game, a keyboard might be used that could light each key any one of three colors. For that embodiment, this song segment might look like this:
Another embodiment of the game might have a rotating pattern of, say, five colors repeated in a chromatic pattern, e.g.
On such a keyboard, this song segment would look like:
On still another embodiment, there might be six colors repeated in a chromatic pattern:
The song segment might then appear like this:
It might also appear that same way on an envisioned seven color scheme that repeats on each octave yet gives each white key pitch a distinct color:
The most important thing to note is that the keys actually used in the song or song segment are highlighted in some way to easily distinguish them from those that are not used. Also note that the highlighting remains present throughout the entire song or segment.
The most important effect of this method of highlighting is to eliminate all of the keys that are not used in the song or song segment from the player's consideration. Since the number of pitches actually used in the song or song segment is generally much smaller than the total number of pitches available, the player is able to focus only on this smaller number of keys and therefore success at playing the melody and rhythm involved is greatly improved and the player experiences more satisfaction.
The keys or buttons need not be arranged in a way corresponding to an actual musical instrument, but if they are, the player will find it easy to transfer the patterns learned in playing the game to performance on an actual conventional musical instrument.
For cost considerations or other reasons, it may be desirable to use a keyboard (or buttons, fretboard, etc.) on which keys are differentiated by color or some other means but on which this highlighting is static and does not change.
For example, here is a one-octave “painted” keyboard on which only the “white” keys are able to be used (we would not use this as a “focused” keyboard for any songs or song segments that required the use of any “black” keys if we were trying to use a layout similar to an actual musical instrument).
We could still use this as a focused keyboard if we are able to indicate which keys are actually used by some other distinctive means. Following the “Take Me Out to the Ball Game” example, we might for instance use this keyboard with a display that indicates all the notes actually used in the song or song segment.
For instance, here is the above keyboard associated with a screen or other display for the game:
Note that there are six keys used within this song segment (the first two lines of Take Me Out To The Ball Game) and we can easily see which ones they are by both the fact that the line colors match those keys and by the fact that the lines are arranged in the same order as the keys and physically approach them if the keyboard is aligned below the display. Only six of the eight possible “white” keys are used and none of the possible black keys are used, thereby limited the player's consideration to only 6 of 13 possible pitches in the range of the keyboard. If the keyboard were larger than an octave, this consideration would be even more dramatic, being 6 of 25 for a two-octave keyboard or 6 of 88 for a full sized piano. Even someone with no musical experience generally has no trouble keeping track of only six possibilities, especially when they are already familiar with the melody and rhythm of the song. This level of concentration is similar to conventional video games which might require the player to operate six different buttons or parts of buttons within a short time period.
With some reduction in perceptual ease, this setup could be reduced to even simpler hardware or logical embodiments. For instance, we could eliminate the use of slanted lines, perhaps making it possible to use a simpler display:
Note that the display (or panel) might be attached directly to the keyboard. Also note that since the use of each color on this display is restricted to a unique area, this display could, for example, be achieved by a panel that selectively illuminate lines and dots in white with a multi-color tinted filter placed over it. (The display of the previous example could also be produced that way, though the tinted areas would be trapezoids instead of rectangles.)
In a more primitive form, the only important distinction between notes might be whether they are included in the song or not. This might lead to a very basic display like this:
Note that this form of display still makes it possible for the player to easily tell which keys are included in the song or song segment and which are not.
Also note that the examples used so far in this appendix used white keys only and showed how a keyboard could be used realistically even if the “black keys” did not operate. But the examples could have used instead a painted keyboard with colored black keys of unique colors or repeated colors, for instance this one:
Please note that all of the examples in this appendix operate the keyboard as a “focused keyboard”, rather than a “song specific” keyboard. Recall that on a “focused keyboard” the keys (pitches) that are not used are not highlighted and the configuration of pitches found on the keyboard resembles that of a conventional musical keyboard. On the “song specific” keyboard, pitches that are not used do not appear as keys at all on the keyboard, the remaining keys are adjacent to one another and the arrangement of pitches on the keys changes with each song or song segment and does not resemble the arrangement of pitches on a conventional music keyboard. The “song specific” keyboard is fully described elsewhere in this document.
This application claims the benefit of U.S. provisional application Ser. No. 61/543,393 filed on Oct. 5, 2011 and entitled SYSTEM AMD METHOD FOR A SONG SPECIFIC KEYBOARD which is commonly assigned and the contents of which are expressly incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61543393 | Oct 2011 | US |
