This invention relates to a computer input system, more particularly, to a digital music input system, method of processing the input signal, and keyboard input apparatus.
With the growing functionality in multimedia computers, especially with the advent of the MIDI (Musical Instrument Digital Interface) interface, computers are playing an increasingly important role in the area of digital music. MIDI is a protocol standard that defines how musical events and note properties are communicated between musical devices, such as synthesizers and sequencers.
As a standard input device, computer keyboard is a critical component of a computer system. A computer keyboard generally consists of a set of keys and the control circuit, which comprises a matrix of switches. A circuit switch is closed when the corresponding key is pressed. By scanning the switches and detecting the on/off state, the microcontroller inside the keyboard can generate key codes and transfer them to the computer following a standard protocol, such as the AT interface, PS/2, USB, or a wireless mechanism using infra-red (IR) or radio. The motherboard on the computer contains a keyboard controller that communicates with the keyboard. It converts a received key code from the microcontroller inside the keyboard into a system code and stores it into a buffer, and then generates a keyboard interrupt signal. A keyboard interrupt service program retrieves the system code after receiving the interrupt signal, and passes the code to the operating system for further processing with other application.
In contrast to the tremendous advances in other hardware components, computer keyboards still follow their original design, which has not experienced fundamental improvements over the past several decades. Standard keyboards contain 101, 104, or 107 keys. The layout of the keys, adapted from typewriters, typically uses the QWERTY (named after the arrangement of the first six keys in the first row of a standard keyboard) format. Ergonomic keyboards and multimedia keyboards have been developed. The former is to adjust the keyboard shape according to human posture, thereby alleviating the stress on human bodies. The latter adds certain special keys, called short-cut keys, for controlling commonly used computer programs. Since existing computer keyboards are designed for character input, they are not suitable for music input.
In summary, there exist the following three methods for inputting computer music. Unfortunately, each approach has its drawback.
1) The most common approach in today's digital music system is relying on a special-purpose MIDI keyboard to input music to the computer, and using a MIDI-capable sound card to play the music. Although MIDI hardware components provide high-quality sound effects and broad control capabilities, they are very expensive. Because a MIDI keyboard needs to contain the special-purpose hardware for generating MIDI signals, it is often ten times more costly than a standard computer keyboard. In addition, assembling a MIDI system is a complex process for novices. Furthermore, incompatibility may exist among various hardware devices. Therefore, this method is only ideal for a small group of specialists, but not desirable for a broad range of consumers. The high cost and complexity are the primary reasons that have hindered the mass adoption of digital music.
2) Another method is to use a conventional computer keyboard for inputting music notes. Because the alpha-numerical keyboard layout, designed for character input, is dramatically different from that of a musical instrument, a standard computer keyboard is extremely inconvenient to use for music input. Although several efforts have been made to improve this, they all prioritize character input over music input. Consequently, they are still very cumbersome to apply because they do not solve the fundamental problem resulting from the incompatible form factors between computer and musical keyboards. Among these approaches, U.S. Pat. Nos. 5,646,648, 6,066,795, and 6,351,225 propose to associate the music input functionality with a subset of keys on the standard computer keyboard. For example, U.S. Pat. Nos. 5,646,648 and 6,351,225 designated the top row of the standard keyboard to represent one octave of music notes. Limited by the standard layout of a computer keyboard, however, the range of notes offered by such approaches is very limited. U.S. Pat. No. 6,444,888 describes a method to define the relationship between key code and music notes such that it maps the most frequently used music notes to the most convenient keys. The drawback of this approach is twofold: (a) there is a big learning curve for using such a mapping and (b) it still relies on a standard computer keyboard, which has the inconvenient layout for music input.
3) The third method is to use a music keyboard adaptor on top of the computer keyboard. The music keyboard adaptor is a frame with piano-like keys. By pressing the keys on the frame, the keys underneath are instrumented. U.S. Pat. Nos. 4,352,313, 4,704,940, and 5,971,635 follow such a method. The disadvantage of this approach is that the two level of instrumentation adds annoying delays for music play back. Many standard keyboards also have clicking noise, affecting the sound effect. In addition, it is impossible to design a generic frame that fits the shape and size of an arbitrary computer keyboard.
It is an object of the present invention to provide a computer music input system, and keyboard input apparatus with a plurality of functions.
It is another object of the present invention to provide a method of processing the input signals from a keyboard, which converts the keyboard signals into music signals to alleviate the hardware dependence on special-purpose MIDI devices in a computer music system, through the use of computer software.
According to one aspect of the present invention, it provides a computer music input system, comprising an input means having a music-input device composed of music-input keys and a character-input device composed of character-input keys for generating key codes corresponding to the pressed music-input or character-input keys; a music processing means for establishing the mapping table between key codes and music notes, receiving the key codes generated by said input means and converting the key codes into special notes, and generating digital sound signals for the intended note; and an output means having a computer sound card and the associated output device (such as a speaker or a headphone) for converting the digital sound signals generated by said music processing means into analog signals and playing back the input music.
According to another aspect of the present invention, it provides a keyboard input apparatus for a computer music input system. The apparatus comprises a music-input device having a plurality of music-input keys; a character-input device having a plurality of character-input keys; and a digital circuit comprising a matrix of switches for generating key codes in response to the press of external keys on the keyboard input apparatus, and outputting the key codes to the computer.
According to yet another aspect of the present invention, it provides a method of processing music signals in computer music input system, comprising the steps of: setting up the mapping between key codes and music notes to establish a key-to-note mapping table and storing said mapping table in the computer; monitoring key interrupt events generated by a keyboard input apparatus, identifying the corresponding note according to said key-to-note mapping table; synthesizing the digital signal of the sound that emulates the tone of the note being played; and converting digital signal to analog signal for play back.
The present invention provides an inexpensive solution that solves the music input inconvenience caused by traditional character-oriented keyboard layout. This is achieved through the combination of a low cost and effective input apparatus, and the power of computer software. In comparison with special-purpose MIDI keyboards, the present invention applies the music processing program to facilitate music input. Therefore, it eliminates the dependence on expensive MIDI hardware and avoids the complex setup process for MIDI systems. Comparing to standard computer keyboards or music keyboard adaptors, this invention dramatically enhances the convenience for music input. With our approach, the music-input component is relatively independent of the character-input component. As a result, the layout of music keys is not limited by the standard keyboard format, and music input and character input do not interfere with each other. Because the music-input component resembles musical keyboard instruments, music input can be performed in a natural and convenient way. Combined with suitable software applications, the music input system of the present invention can be applied for music education, music editing, and family entertainment.
Above and other objects and advantages of the present invention will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompany drawings. It should be noted that the invention is not limited to these embodiments.
Referring to
If the music processing program 23 is not started, any input from the input apparatus is treated as standard character input. If music input is needed, the music processing program 23 needs to be started—it “knows” how to interpret the key codes received from the input apparatus. When a key is pressed, the corresponding key code is sent to the computer. Upon receiving the key code, the controller (not shown in the figure) in the computer 22 generates an interrupt request that is handled by the interrupt service program. The interrupt service program passes the keyboard interrupt event and the corresponding key code to the music processing program 23. The music processing program 23 can exist either as part of the operating system or as an application program. Program 23 monitors the keyboard interrupt events generated by the input apparatus. In particular, it handles the “key-pressed” and “key-released” events. It transforms the key events into musical events, emulates the sound effect through sound synthesis, and passes the sampling data to the device driver of the sound card 24. The sound card 24 then transforms the digital data to analog data and plays it back through the output device 25.
Now the detailed method of processing music signals is explained. Because the signals generated by the input apparatus 21 are only standard key codes, they need to be interpreted by the music processing program 23. It should be noted that the music processing unit 23 can be implemented by hardware or software independently, or by the combination of hardware or software.
In step S313, the processing program monitors the keyboard event in real-time. After retrieving the key code from an interrupt event, the program consults the mapping table and determines whether that key represents a music note. If the key does not represent a music note, the flow proceeds to step S320 to determine whether the key has any special functionality. If the key has a special functionality, the processing program performs the intended function in step S322 and repeats the monitoring loop starting at S313.
If the key is detected to represent a music note (step S314 is positive), the flow proceeds to step S315 to determine if the event is “key-pressed”. If so, step S316 is taken for loading sampling waveform data (details will be elaborated later). If the result in step S315 is false, step S317 is taken to determine whether the event is “key-released”. If so, the process proceeds to step S318 to determine if the sustain mode is on. If so, the process repeats the monitoring loop starting at S313. Otherwise, step S319 is taken and the processing program stops feeding sound data to the sound card.
In the above process, if the received event is “key-pressed” and the key represents a music note, the music processing program generates a “Music Start” event. If the received event is “key-released”, the processing program generates a “Music Stop” event. If the key represents a special function key, the processing program 23 performs the function accordingly. The processing program 23 may perform additional tasks according to the intended need, e.g., editing, displaying notes on the screen, or playing back the music.
The process of generating the digital signal of a music note (step S316 in
In addition to the sampling method, waveform table can also be calculated following certain algorithms. This can be done either in advance or in real-time. Common algorithms include FM Frequency Modulation, Subtractive Simulation Synthesis, or Linear Arithmetic Synthesis. In comparison with the sampling method, the real-time synthesis approach can save storage space and can generate flexible tones but it imposes a higher computational demand.
Another method of the present invention is to integrate standard computer keyboard input techniques with the MIDI technology as follows: The music processing program handles the keyboard interrupt event, and generates the corresponding MIDI message when receiving a musical event. When “key-pressed” is detected for a key representing a music note, the music processing program immediately creates a “Note On” MIDI message. When “key-released” is detected for a key representing a music note and the sustain mode is off, the music processing program immediately generates a “Note On” MIDI message with the velocity value set as 0. The MIDI message is passed to the software or hardware components that support the MIDI protocol to complete the sound play back. The advantage of this method is that the processing program can be compatible with other application software that supports MIDI. The disadvantage is that going through various layers of the MIDI protocol might increase the delay time for play back.
Multifunction Input Apparatus
A novel input apparatus is applied in the present invention to make music input more convenient. In order to lower the cost, the input apparatus is designed to complete music input without resorting to special MIDI hardware.
Similar to the technology used in a standard computer keyboard, the input apparatus of the invention uses the underlying control circuit to monitor the status of the keys. In contrast to a standard computer keyboard, however, the input apparatus comprises two components, one for music input and the other for character input. The music-input component consists of a set of piano-like keys; it may further comprise special function keys to configure preferable settings. The character-input component has a layout similar to that of a traditional computer keyboard. One characteristic of the input apparatus is that music keys and character keys can share key codes. Not only does this simplifies the design of the underlying circuit, it also eliminates the constraint on the total number of notes, which is imposed by some existing solutions in which key codes cannot be shared between music and character keys.
The two components of the input apparatus can be connected using different configurations. While applying the same technology and providing the same functionality, the configuration flexibility allows users to choose the most desirable way to assemble the input device according to their specific needs.
One configuration embodiment of the input apparatus is that the two components are separate modules; each has its individual keys and control circuit. They are connected in serial through cables or sockets. This configuration is space-saving since the music-input module only needs to be hooked up as needed.
Another embodiment of the input apparatus is that the two components are separate modules; each has its individual keys and control circuit. Instead of being connected in serial, the two components are connected in parallel, i.e., they are independently connected to the computer.
Yet another embodiment of the input apparatus is that the two components are built into a single unit, sharing the same control circuit and communication interface with the computer. To save space, the music-input component and the character-input component are located on the opposite sides of the keyboard. When a different input mode is needed, the user simply needs to turn the keyboard up side down.
Further another embodiment of the input apparatus is that the two components are built into a single unit, sharing the same control circuit and communication interface with the computer. In contrary to the double-sided configuration, all keys are located on the same side of the keyboard.
In order to allow the input apparatus to be used even without a computer, an embodiment of the input apparatus may further comprise a sound synthesizer, which is connected to the control circuit of the input device so that sound can be produced directly when a music key is pressed. An embodiment of the input apparatus may further comprise a pedal module to emulate the sustain effect or mute effect of a piano. An embodiment of the input apparatus may also comprise a velocity-detection device to make the keys force-sensitive.
This particular embodiment samples the sound of a grand piano using 16 Bit data and a sampling rate of 44.1 KHz. It then captures a 0.5 second fragment of the waveform and stores it in a waveform table in hard disk. The music processing program is implemented in a programming language, e.g., C++. When started, it initializes the key-to-note mapping table, and enters a loop for handling the “key-pressed” and “key-released” interrupt events. For any key corresponding to a music note, the processing program converts the keyboard interrupt event to a “Music Start” event and starts to feed the sampling signal to the sound card for play back. In the mean time, it displays the musical event on screen through a graphical user interface (GUI). This embodiment also applies some function keys to control user settings. For example, F1 is used to exit the program; F2 is used to toggle the sustain mode; F3 and F4 are respectively used to increase and to decrease the volume; F5 and F6 are respectively used to increase and to decrease the octave range. If the “key-released” signal is received for a key representing a music note while the sustain mode is off, a “Music Off” event is generated to stop the play back. By default, most computer keyboards have a feature to repeatedly generate key codes if a key is held down. Although useful for character input, this would be an annoying side effect for music input. To solve this problem, the status of key releases is tracked in the music processing program. The program checks whether the key has been released when the “key-pressed” event is received. The event is caused by automatic key code generation and can be ignored if the key has not been recently released.
Although the present invention has been described by way of exemplary embodiments, it should be understood that many changes and substitutions may further be made by those skilled in the art without departing from the spirit and scope of the present invention which is defined by the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
03150136.2 | Jul 2003 | CN | national |