Not applicable.
Not applicable.
The present invention relates to a method for playing and processing audio data by at least two computers over a packet switching network.
From DE 697 10 569 T2, the entire contents of which is incorporated herein by reference, a method is known for the real time playing of music with a client server structure (multiple node structure). For so-called MIDI data, the method proposes to provide control data for the creation of a musical tone, to break the control data into data blocks, to generate a recovery data block for recovering the control data, to transmit the data block over a communication network, and likewise, to transmit the recovery data block over the communication network. Thus with this client-server structure, the control data for a musical instrument is distributed using a server, which enables an audience with a plurality of listeners to follow a concert, by generating a music from the MIDI data at each listener from the control data. Further, it is proposed to assign a consecutive sequence number to the individual packets of the MIDI data that retains the sequence of the packets, and makes it possible to reorder them after the transmission. This MIDI data also contains in its header the time data, which indicates the musical play time of the subsequent MIDI data. The play time of the music together with the information about the size of the MIDI data permits the music to be played at the intended speed.
From DE 101 46 887 A1, the entire contents of which is incorporated herein by reference, a method is known for synchronizing digital data streams with audio data on two or more data processing devices. For this, one of the data processing devices generates a control signal that describes an absolute time position in the data stream. With the known method, the data processing units are directly connected with each other over an ASIO interface.
From U.S. Pat. No. 6,175,872 B1, the entire contents of which is incorporated herein by reference, a system is known for the managing and synchronizing of MIDI data. The computers, which play the MIDI data exchanged through the network, are synchronized relative to a standard clock. For synchronizing the MIDI data, a timestamp with the absolute time plus a relative time delay is appended to a packet. The relative time delay arises from the position of the computer on which the data are intended to be played.
U.S. Pat. No. 6,067,566, the entire contents of which is incorporated herein by reference, relates to a method for playing of MIDI data streams while these are still being received. For this, it is proposed to provide a parser 207 and a time converter 209. The parser 207 reads event messages 117 and event data, which contain, in each case, details for the elapsed time (elapsed time descriptor 119). Here, the elapsed time refers to the beginning of a track (see column 5, lines 40-43). During play of files with several MIDI tracks, these are read in sequentially after one another. During playing of n-tracks, first, n−1 tracks are all received and saved. The saved tracks are played together with the not yet completely received track, when the track being played has reached the current position (SongPos 217) in the already saved tracks.
It is the technical object of the invention to provide a method with which the audio data from remote computers can be combined with precise timing.
The method relates to the playing and processing of audio data by at least two computers over a packet switching network. In the process, a peer-to-peer connection is created between the computers. With the method according to the invention, a first computer receives audio data, for example, from an instrument or a microphone via an audio input. The audio data of the first computer is assigned timestamps. A second computer, which is connected only over the data network with the first computer, is initialized for playing further audio data. The further audio data is similarly provided with timestamps. The audio data of the at least two computers is buffered in a storage, and using their timestamps, arranged such that it is possible to synchronously play the audio data. The method according to the invention permits audio data to be sent over a packet switched data network to a singer or musician, and for this to be played synchronized with other audio data. Through this, for example, during the recording and during the processing of the audio data, the participants can be located at separate locations, where despite the delay over the data network, the audio data can be played together synchronously. Consecutive sample numbers are provided as timestamps, and correspond to a starting time. The exact sample synchronization of the audio data creates a correlation in the range of 10 to 20 microseconds depending on the sampling rate in the audio data. The starting time is determined by the first computer. For this, the starting time of the audio data received from the computer is defined relative to the starting time in the further audio data. To be able to set the starting time exactly, a copy of the further audio data is located on the first computer. Possibly, it can also be provided that only a copy of the beginning of further data is present such that the audio data can be aligned sample exact with the further audio data. Preferably, the further audio data is located on the second computer, where it is then combined with the receipt of the audio data.
It has proven to be especially helpful, to record together with the audio data also information about the computer. This information can be used to help better coordinate the computers with each other.
The method according to the invention is not limited to one additional data stream, rather, according to the method according to the invention, multiple audio data can also be combined, for example, the instruments of a band or an orchestra.
In particular with singing and/or instruments, the microphone or the associated instruments are connected with the first computer, and the received audio data is recorded there after it has been supplied with timestamps. For this, it is especially advantageous, when the further data is also played in the first computer, while, at the same time the new audio data is being recorded. The audio data, which is transmitted with the method, can be present as audio, video, and/or MIDI data.
The method according to the invention is explained in more detail in the following using an exemplary embodiment:
The present invention concerns a method for synchronization of audio data such that musicians using the method can contact each other over the Internet and can play music together other using a direct data connection. The collaboration occurs using a peer-to-peer connection with which the multiple musicians can collaborate, precisely timed.
The central point of the collaboration is that the audio data of the participants are synchronized with each other. With the method, participant A puts his system into the play mode, this state is then transmitted to the second participant B. From this time hence, the data received by participant B is not further transferred directly for play, rather, it is buffered until participant B has also placed his system into the play state.
For a possible further development, the receiver B can also generate a control signal, from the received data, which it sends to a sequencer of system A to automatically start it. Then, system B is automatically started after A was started, and the two additional idle time steps 16 in
For coordination of the play in the system, it has proven helpful along with the audio data, and possibly video data, to also transfer data regarding the operating state of the system. As an example of this, whether a transport has started, or if currently the stop mode prevails. Further, additional information can be exchanged periodically between the participants, to be able to compensate possible differences in their systems.
Because the audio plug-in instances 24 and 26 are in general, inserted in the channels by a higher-level application, for example, a sequencer or a DAW, the example represented in
The preceding sequence offers the possibility to suppress (mute on play) the sending of the data by a corresponding adjustment in 36. Through this, a type of “talkback” function is achieved, so that the producer can not be heard by the singer or musician during the recording, which due to the time delay can be disruptive. Using the selection 48 (THRU), the user can similarly adjust whether a sending channel itself can be heard. Alternatively, the input samples of the channel can be replaced by the received samples of the connected partners.
Thus, using the selection switch 48, it can be selected whether the originally recorded data 32 is to be directly played back unchanged, or whether this data is to be play back synchronized with the data of the second participant 40. If for example, it is selected using the selection switch 36 that the incoming audio data 32 is not to be sent, in stage 38 signals for synchronizing the play with, for example, video data, can still be created.
The concept represented in
The video data are basically synchronized like the audio data. That means, when both participants have started the transport system (see
The method according to the invention is explained in the following using an example:
Computer A is used by a producer, and computer B is used by a singer. Both have an instance of the plug-in connected into their microphone input channel. Both send and receive (talkback), the producer has activated “mute on play” 36. In the idle state, A and B can talk to each other. Additionally, both already have an identical or a similar playback in their time line project of the higher-level application.
The singer starts to form the connection on his computer, and begins sing to his playback. On the side of the producer (computer A), the following takes place:
Now, the producer starts his sequencer on his side, as previously mentioned, this can also occur automatically. The sequencer of the producer now records, wherein the following holds true for the producer:
His microphone samples continue to be suppressed, because the singer in the meantime has advanced further. Only when the producer also removes “mute on play”, can he request, for example, to stop the recording. The producer hears the singer synchronized to the playback stored on his computer. Further, the video data is played back synchronized with the playback stored at the producer.
If, for example, an instrument takes the place of the singer, a second instance of the plug-in can be connected for this into the guitar channel. Then, a microphone channel would be provided for speech and talkback, which during the recording is likewise switched to “mute on play”, such that the producer hears only digitally during the recording. The guitar channel is defined using TRANSMIT.
In the implementation, the method according to the invention provides that, for example, a VMNAaudioPacket is defined. In the AudioPacket, the samplePosition is defined as a counter. The samplePosition indicates the current position of the time scale, when the method is not running. If the project is running, the samplePosition indicates the position of the packet relative to a running (continuously) counter. This running counter is defined using a specific start signal, wherein the counter is set to 0, when the packet counter is set to 0. Depending on the operating mode of the method, the position of the packet is calculated accordingly.
Including the data exchange for the synchronization of the data stream, the method is represented as follows:
In
This completes the description of the preferred and alternate embodiments of the invention. Those skilled in the art may recognize other equivalents to the specific embodiment described herein which equivalents are intended to be encompassed by the claims attached hereto.
Number | Date | Country | Kind |
---|---|---|---|
102005006487.6 | Feb 2005 | DE | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/EP2006/001252 | 2/10/2006 | WO | 00 | 10/9/2007 |