Patent Grant
7711443
A portion of the disclosure of this patent document contains material, which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright or mask work rights whatsoever.
Embodiments of the present invention generally relate to systems and methods for recording and processing audio received from one or more wireless devices. More specifically, the present invention relates to systems and methods for recording and processing audio having one or more tracks received from one or more wireless devices operating in either an asynchronous or synchronous mode.
Many systems and methods have been created to record performance audio. Some such systems include a multi-track audio recorder wired to one or more microphones. Typically, one or more performers performing on a sound stage are recorded by one or more microphones that are directly wired to the multi-track recorder. The multi-track recorder combines the single track of audio received from each microphone to create one multi-track audio file. In many such systems, the received audio and/or the multi-track audio is timestamped with a time reference signal such as a Society of Motion Picture and Television Engineers (“SMPTE”) timecode signal containing information regarding the hour, minute, second, frame, type of timecode (i.e., nondrop or drop frame), and user-definable information. Such information allows audio to be more easily matched and/or combined with simultaneously recorded video.
Other such systems include a multi-track audio recorder and an associated audio receiver that receive audio wirelessly from one or more wireless transmitters. Such wireless transmitters may take the form of body packs that are worn by each performer. Typically, the audio receiver receives each performer's audio from the performer's respective body pack via an analog or digital wireless transmission and transmits it to the audio recorder. The audio recorder then combines the wireless transmissions received from all body packs to create one multi-track audio file.
Due to the occurrence of wireless transmission errors such as dropouts, some existing wireless systems include audio receivers having two or more redundant receiver circuits. The incorporation of additional, redundant receiver circuits provides a better opportunity to avoid missed audio transmissions. For example, the use of two receiver circuits may allow a second receiver to receive audio that may have not been received by a first receiver circuit and vice versa. However, although such redundancy accounts may correct wireless transmission errors, such redundancy does not prevent loss of data due to interference (i.e., a distortion of the received audio signal due to receipt of multiple wireless signals). Upon the occurrence of interfering signals, audio created during a performance (e.g., a live performance) may simply be lost due to the inability of the receiver to receive a clean audio signal.
Briefly stated, in one aspect of the present invention, a system for recording locally generated audio is provided. This system includes: at least one master timecode generator for generating a plurality of master timecodes; and at least one local audio device wearable by a creator of said locally generated audio including: at least one local audio device receiver for wirelessly receiving said master timecodes; at least one audio input port for receiving locally generated audio from an audio input device; at least one memory; at least one control unit in communication with said local audio device receiver, said audio input device, and said memory for creating local audio data from said locally generated audio and storing said local audio data in said memory; and at least one local audio device wireless transmitter for wirelessly transmitting said local audio data in real time, said at least one local audio device wireless transmitter in communication with said at least one control unit; wherein said local audio data includes stamped local audio data and unstamped local audio data; wherein said stamped local audio data includes at least one timestamp to reference at least a portion of said local audio data to at least one of said master timecodes; and wherein said unstamped local audio data does not include a reference to said master timecodes.
A further understanding of the present invention can be obtained by reference to the embodiments set forth in the illustrations of the accompanying drawings. Although the illustrated embodiments are exemplary of systems for carrying out the present invention, both the organization and method of operation of the invention, in general, together with further objectives and advantages thereof, may be more easily understood by reference to the drawings and the following description. The drawings are not intended to limit the scope of this invention, which is set forth with particularity in the claims as appended or as subsequently amended, but merely to clarify and exemplify the invention.
For a more complete understanding of the present invention, reference is now made to the accompanying drawings in which:
As required, a detailed illustrative embodiment of the present invention is disclosed herein. However, techniques, systems and operating structures in accordance with the present invention may be embodied in a wide variety of forms and modes, some of which may be quite different from those in the disclosed embodiment. Consequently, the specific structural and functional details disclosed herein are merely representative, yet in that regard, they are deemed to afford the best embodiment for purposes of disclosure and to provide a basis for the claims herein, which define the scope of the present invention. The following presents a detailed description of one embodiment (as well as some alternative embodiments) of the present invention.
Referring first to
In the embodiment of the present invention depicted in
Both live and replayed audio transmitted by local audio devices 102 may be received at receiver 106 and recorded by audio recorder 108. Receiver 106 and recorder 108 may be virtually any commercially available receiver and recorder. Receiver 106 receives the wireless RF signals (e.g., modulated RF carrier signals) generated by all active local audio devices 102 and converts the signals to a format capable of being recorded by a commercially available recording device including, but not limited to, Zaxcom, Inc.'s DEVA® multi-track recorder. In some embodiments, such commercially available recording devices record audio with a locally generated SMPTE-compatible timecode signal.
The ability to synchronize the local timestamps at each local audio device 102 and recorder 108 using the methods of the present invention as discussed in greater detail below allows any audio that is not recorded by recorder 108 during an event due to transmission errors to be recovered by replaying the missed audio and recording the replayed audio in the correct time sequence with respect to the other audio samples. In other words, since the audio samples are stored locally in each local audio device 102 with timestamps that are synchronized with the timestamps of recorder 108, whenever audio is not recorded at recorder 108, it may simply be replayed at local audio devices 102 starting at the timecode of the missed audio. Since the local audio device and recorder timestamps are synchronized, the replayed audio may be inserted in the proper time sequence with respect to the other recorded audio samples based upon the synchronized timestamp data. Synchronization is essential to ensure that each performer's audio is synchronized with all other performers' audio and to ensure that the newly recorded replayed audio is in the correct sequence with respect to the previously recorded live audio. Such synchronization must maintain a high accuracy for each performer's timestamps with respect to all other performers' timestamps to prevent the occurrence of phasing artifacts when the multiple audio recordings are combined to create one single recording.
In some embodiments of the present invention, receiver 106 automatically senses an error in transmission caused by, for example, a communication loss, interference, etc. In some embodiments of the present invention, the error in transmission is sensed by comparing a calculated checksum to the transmitted checksum to determine if data was lost during transmission. An error is determined if the calculated and transmitted checksums do not match. Upon sensing a transmission error, receiver 106 may transmit a request to RCU 104 requesting playback of the audio recorded locally on local audio devices 102 beginning at a timecode prior to the occurrence of the transmission error. In response, RCU 104 transmits a digital command to all local audio devices 102 to playback the audio stored in the respective memory 332 (
Alternatively, playback may be requested manually by a user of a recording system such as recording system 100. In this scenario, upon hearing that a transmission error (i.e., a loss of audio data) has occurred, the user manually prompts RCU 104 to transmit a digital command to all local audio devices 102 to playback the audio stored in memory 332 (
RCU transmitter 208 allows RCU 104 to transmit a master time reference signal, digital commands, audio, and the like to other devices such as local audio devices 102, receiver 106, and recorder 108. In one aspect of the present invention, the time reference signal is a SMPTE timecode signal containing information regarding the hour, minute, second, frame, type of timecode (i.e., nondrop or drop frame), and user-definable information (e.g., the transport status of recorder 108, the name of a scene, the name of a take, etc.). This master time reference signal provides a time reference for all local audio devices 102, which may use this information for a variety of purposes such as jam synchronizing their respective local timecode generators 304 (
When recording system 100 is operating in a synchronous mode, transmission of the master time reference signal ensures that all of the components of recording system 100 store all locally recorded audio with timestamps that are highly accurate as compared to the timestamps of all other local audio devices 102 and/or all other components of recording system 100. The timestamps are then used during playback and recording to ensure that the replayed audio from all local audio devices 102 is synchronized with previously recorded audio and with the audio replayed by all other local audio devices 102. In contrast, when recording system 100 is operating in an asynchronous mode, transmission of the master time reference signal allows the files containing recorded audio to be timestamped with the master time reference information to allow the recorded audio to be accurately synchronized post-recording.
RCU transmitter 208 also allows audio generated locally at RCU 104 to be transmitted to the other components of recording system 100. Such audio may be received from an audio input device such as RCU audio input device 212 via audio input device port 214. RCU audio input device 212 may be any type of commercially available audio input device such as a microphone and audio input device port 214 may be any commercially available audio input device port that is compatible with RCU audio input device 212 and the internal components of RCU 104. The received audio as well as any digital signals (e.g., microphone input level, line input level, etc.) are then buffered and/or amplified by RCU preamp 216 and are converted from analog to digital by RCU ADC 222 such that the audio may be read in digital form by RCU local control unit 210. This audio may then be processed and sent via RCU transmitter 208 in either analog or digital form. If the audio is to be sent in analog form, RCU local control unit 210 may be equipped with an on-board DAC or an independent DAC may be incorporated in RCU 104 without departing from the scope of the present invention. Or, alternatively, analog audio received from RCU audio input device 212 may be passed directly to RCU transmitter 208 for transmission in analog form to the other components of the recording system. In such embodiments, RCU transmitter 208 may be equipped with a frequency modulation (“FM”) modulator or the like. Furthermore, in such embodiments, although the analog audio is passed through to RCU transmitter 208, the audio signal may be additionally converted to digital form for local recording of the received audio. In yet another alternate embodiment, audio may be transmitted and recorded in analog form thereby eliminating RCU ADC 222.
In some embodiments of the present invention, RCU local control unit 210 may be a digital signal processor such as Texas Instruments part number TMS320C5509A. However, the present invention is not so limited. Any combination of hardware and software may be substituted for any component described herein without departing from the scope of the present invention.
RCUs 104 may be handheld units such as RCU 104 depicted in
In addition to allowing a user to modify local RCU settings, RCU keypad 220 and display 218 also allow the RCU to remotely control individual local audio devices. The user may perform a variety of functions for the local audio device including, but not limited to, transmitter and receiver frequencies, transmitter enable, microphone gain, high pass filter, record mode select, time code entry, playback control, audio bank storage, and status request.
For example, local audio device transmitter and receiver frequencies may be adjustable in predetermined frequency steps. Alternatively, the local audio device transmitter may be remotely enabled and disabled. Microphone gain may be adjusted, which in turn adjusts the current setting of a preamp such as local preamp 316. Adjustment of the high pass filter may be incorporated to enable and disable, or otherwise adjust, the high pass audio filter of the audio input device such as audio input device 312.
In addition, record mode select allows recording modes such as endless loop record mode or timed record mode to be remotely adjusted. Timecodes may also be set remotely for each local audio device. Playback control allows one or more local audio devices to be commanded remotely to playback audio starting at a specific timecode. Completion of playback may be automatically or manually determined. Functions such as audio bank storage allow a remote user to manually store chunks of audio data in safe locations of the local audio device memory (i.e., in locations in which the audio data will not be overwritten). Finally, status of the local audio device may be requested. The status may be provided via display 218 or via spoken language generated by local audio device 102 and transmitted to a receiver or receiver/recorder combination for recording with the recorded audio.
Although many specific features and functions for the RCU have been delineated herein, other features and functions may be added or eliminated without departing from the scope of the present invention.
Additionally, handheld embodiments may include any one of a variety of commercially available batteries to function with the power supply 206 without departing from the scope of the present invention. Power supply 206 may be virtually any power component or combination thereof that is compatible with the other components of RCU 104 including, but not limited to, a Texas Instruments TPS62000DGS Power Module alone or in combination with a Linear Technology LTC3402 Synchronous Boost Converter.
However, non-handheld embodiments of RCU 104 are also envisioned such as tabletop models, personal computer (“PC”) models, etc. Also, RCU 104 may be optionally equipped with external interface 252 (
Turning next to
Local transmitter 308 also allows audio generated locally at local audio device 102 to be transmitted to the other components of recording system 100. Such audio may be received from an audio input device such as local audio input device 312 via local audio input device port 314. Local audio input device 312 may be any type of commercially available audio input device such as a microphone and local audio input device port 314 may be any commercially available audio input device port that is compatible with local audio input device 312 and the internal components of local audio device 102. The received audio as well as any digital signals (e.g., microphone input level, line input level, etc.) are then buffered and/or amplified by local preamp 316 and are converted from analog to digital by local ADC 322 such that the audio may be read in digital form by local control unit 310. This audio may then be processed and sent via local transmitter 308 in either analog or digital form. If the audio is to be sent in analog form, local control unit 310 may be equipped with an on-board DAC or an independent DAC may be incorporated in local audio device 102 without departing from the scope of the present invention. Or, alternatively, analog audio received from local audio input device 312 may be passed directly to local transmitter 308 for transmission in analog form to the other components of the recording system. In such embodiments, local transmitter 308 may be equipped with a frequency modulation (“FM”) modulator or the like. Furthermore, in such embodiments, although the analog audio is passed through to local transmitter 308, the audio signal may be additionally converted to digital form for local recording of the received audio. In yet another alternate embodiment, audio may be transmitted and recorded in analog form thereby eliminating local ADC 322.
In some embodiments of the present invention, local control unit 310 may be a digital signal processor such as Texas Instruments part number TMS320C5509A. However, the present invention is not so limited. Any combination of hardware and software may be substituted for any component described herein without departing from the scope of the present invention.
Similarly, local receiver 302 allows audio received from other components of recording system 100 to be played locally at local audio device 102. Such audio may be received in either analog or digital form at local receiver 302. However, if the audio is to be received in analog form, local control unit 310 may be equipped with an on-board ADC or an independent ADC may be incorporated in local audio device 102 without departing from the scope of the present invention to allow local control unit 310 to receive the audio in digital form. Thereafter, the audio may be processed or relayed directly to local DAC 324, which converts the audio data back to analog form. The analog audio may then be amplified by local amp 326 prior to transmission through local audio output device port 328 to local audio output device 330. Local audio output device 330 may be any type of commercially available audio output device such as headphones, speakers, and the like, and local audio output device port 328 may be any commercially available audio output device port that is compatible with local audio output device 330 and the internal components of local audio device 102. Local receiver 302 may be virtually any receiver compatible with the other components of local audio device 102 including, but not limited to, a Micrel Semiconductor MICRF505 RadioWire® transceiver.
Memory 332 of local audio device 102 locally stores audio processed by local control unit 310 in one or more audio files. In one aspect of the present invention, local control unit 310 receives recordable audio from local audio input device 312, which may be worn by the performer and connects to local audio device 102 at local audio input device port 314. However, in alternate embodiments, local control unit 310 may also receive audio from other components of recording system 100 via local receiver 302. The locally stored audio files include timestamps (e.g., timestamps may be stored in the header of the audio file) that indicate when, during the audio event, each segment of audio occurred. The timestamps may be generated based upon timecodes created by local timecode generator 304 or based upon master timecodes. Such master timecodes may be received using a plurality of methods or components including, but not limited to, wirelessly from a master timecode source through local receiver 302, from a timecode source connected to local audio input device port 314, and from local audio input device 312 wherein the master timecodes are received from an ultrasonic signal. Local timecode generator 304 may be synchronized with the master timecode generator during recording of the audio event as described in further detail below with respect to
Memory 332 may be virtually any type of commercially available removable or non-removable memory including, but not limited to, flash memory cards, compact flash memory cards, Universal Serial Bus (“USB”) thumbdisks, and the like. Use of removable memories 332 facilitates removal and insertion of these memories into a PC or the like for electronic combination or mixing of the recorded audio data. Such electronic mixing may be performed via commercially available software such as Pro Tools or the like and may be performed in addition to or in lieu of live wireless recording of the audio event.
Local audio devices 102 also receive non-audio information (e.g., time reference signals, digital commands, audio, etc.) from other components of recording system 100 via local receiver 302. During synchronous operation of recording system 100, a portion of the received data may be used to synchronize local timecode generator 304 to the master timecode generator integral to one of the components of recording system 100 (e.g., RCU 104, recorder 108, etc.) using a process such as that described below with respect to
As described in further detail below with respect to
When local audio devices 102 such as those depicted in
Whenever playback of locally recorded audio is required (e.g., to remedy recording errors caused by transmission losses), RCU 104 transmits a digital command to all local audio devices 102 to playback the audio data stored in the respective memories 332 starting with and subsequent to a specific time reference as indicated by a specific timecode. The digital command is received by local receivers 302, which transmit or relay the command to their respective local control unit 310. Thereafter, local control units 310 access the data stored in the respective memory 332 and cause this data to be played or transmitted sequentially via local transmitter 308 starting with the data associated with the requested timecode. The use of timecodes and synchronization of local and master timecode generators, as well as local and recorder audio sampling rates, as discussed herein allows multiple local audio devices 102 to replay audio with the exact timing that occurred during the audio event.
Local audio devices 102 may be bodypacks such as the local audio device 102 depicted in
When multiple local audio devices are incorporated in to a group, each local audio device in the group as well as other components of the recording system (e.g., an RCU) may be assigned a group ID. Similarly, the unit ID identifies each specific local audio device within the group of local audio devices.
For local audio devices transmitting encrypted audio and data, the transmitter encryption code is set to match the encryption code of all receiving devices (e.g., an RCU, recorder, or receiver). Correctly setting this code allows the receiving device to properly decrypt the received transmission, while preventing unauthorized users from recording the data.
The operating mode of each local audio device can encompass any one of a number of modes. For example, the operating modes may include USA or European modes, as well as stereo modes. Selection of a specific mode may alter settings such as transmitter bandwidth, audio sampling parameters, and the like.
Although many specific features and functions for the local audio devices have been delineated herein, other features and functions may be added or eliminated without departing from the scope of the present invention.
Additionally, handheld embodiments may include any one of a variety of commercially available batteries to function with the power supply 306 without departing from the scope of the present invention. Power supply 306 may be virtually any power component or combination thereof that is compatible with the other components of local audio device 102 including, but not limited to, a Texas Instruments TPS62000DGS Power Module alone or in combination with a Linear Technology LTC3402 Synchronous Boost Converter.
Alternate embodiments of local audio device 102 are envisioned in which local receiver 302 is eliminated. In one such embodiment, local transmitter 308 is enabled whenever an audio event requiring recording is occurring. Local timecode generator 304 may be designed to generate timecodes whenever local transmitter 308 is enabled. When local transmitter 308 is not operating, the current value of local timecode generator 304 is stored in non-volatile memory to allow local timecode generator 304 to continue counting from the last generated timecode when the local transmitter 308 is re-enabled. Such embodiments include a timecode generator capable of generating unique timecodes for several years without a repeated timecode.
During recording, each local audio device 102 transmits data to one or more receivers and/or recorders. During recording, the receivers and/or recorders automatically detect corrupted audio data received from local audio devices 102 and maintain a list of same. The list of corrupted audio data contains references to the respective local audio device 102 from which the corrupted audio data was received to allow such data to be recovered post-recording.
Post-recording, memories 332 may be removed from each local audio device 102 such that locally recorded data may be retrieved and used to repair the corruption of the audio file generated by the receiver/recorders that occurred due to the receipt of corrupted audio data. Such data recovery may be performed using the multi-memory unit of the present invention or an equivalent. In one embodiment, the multi-memory unit may connect directly to the receivers and/or recorders to allow this equipment to directly retrieve the required audio data. In another embodiment, memories 332 may be connected directly to the receivers/recorders for retrieval of the audio data, thereby eliminating the need for any extraneous equipment such as a personal computer.
Since the timecodes generated locally by each local audio device 102 may vary with respect to each other, the receivers, and/or the recorders, the present invention provides a method for ensuring that audio data retrieved from memories 332 is inserted in the proper time sequence with respect to the audio file(s) generated by the receiver/recorders. To achieve this, during recording, the receiver(s) and/or recorders generate or populate a cross-reference table, database, or the like that correlates the timecodes of the audio files generated by the receiver/recorders, as well as the timecodes of all audio data received from all local audio devices 102. That is, the cross-reference mechanism correlates each timecode generated by a receiver or recorder to each timecode generated by each local audio device. In this manner, the timecodes of audio retrieved from memories 332 may be cross-referenced to determine the correlating timecode of the audio file generated by the receiver/recorders. Thereafter, the retrieved audio may optionally be re-stamped with the timecode of the receiver/recorder and inserted in its proper place within the receiver/recorder audio file. In this manner, audio may be wirelessly recorded with zero data loss.
Referring now to
At 404, initialization occurs. During initialization, the local control unit such as local control unit 310 or other form of central processing unit is reset. Thereafter, the local transmitter, local receiver, ADC, DAC, and local timecode generator clock are initialized. The process then optionally proceeds to 406, at which the sampling rate of the ADC is set. Alternatively, the sampling rate may be set via hardware or via software executed as part of a separate algorithm. In some embodiments of the present invention, a sample rate of 48 kHz is incorporated.
Next, at 408, wireless receive channels are established between the local audio device and one or more wireless devices such as RCUs (e.g., RCU 104), receivers, and audio recorders. To establish the channel, the local receiver of the audio device receives one or more data packets from the remote wireless device and stores the packets in a designated buffer. For example, when establishing wireless communication with a RCU, the local audio device may receive one or more data packets containing information such as a master timecodes, transport status (i.e., transport mode of an audio recorder), and the like. These packet(s) are then stored in an RX buffer (i.e., a reserved segment of memory used to hold data while it is being processed). Process 400 then proceeds to 410.
At 410, the local control unit reads the master timecode contained in the RX buffer and jam synchronizes the local timecode generator with the master timecode. The jam sync synchronizes the local audio device with the RCU while allowing the local audio device to supply its own timecode. Local supply of synchronized timecodes ensures proper timing during periods in which the master timecodes cannot be read (e.g., the RCU is temporarily unstable, wireless communication dropouts, etc.).
Next, at 412, process 400 queries the transport status stored in the RX buffer. If at 412, the transport status is stop, process 400 returns to 410. However, if at 412, the transport status is record, process 400 proceeds to 414. At 414, a new audio file is created in memory (e.g., on a flash card) and the newly created file is timestamped. In one aspect of the present invention, timestamping includes storing the timecode in the file header. Process 400 then proceeds to 416.
At 416, the local control unit waits for an audio sample interrupt from the ADC. Once an audio sample interrupt occurs, process 400 proceeds to 418. At 418, the audio sample is retrieved from the ADC and stored in the local memory. In one aspect of the present invention, the audio sample is stored in the next available address of the local memory. Next, at 420, the timecode generator counter is incremented, thereby indicating that the time period for one sample of audio has elapsed.
Process 400 then proceeds to 422, at which the local control unit transmits the audio sample through the local transmitter to the other wireless devices such as RCUs, receivers, audio recorders, and the like. For example, audio from multiple local audio devices may be transmitted to a multi-track recorder for recording of the audio event while each local audio device locally records its performer's audio. At 424, process 400 queries the RF buffer of the local receiver to determine the availability of a new master timecode packet. If at 424, a new master timecode packet has not been received from the RF receiver, process 400 returns to 416. However, if at 424, a new master timecode packet has been received, process 400 proceeds to 426 as depicted in
At 426, process 400 executes a feedback loop algorithm, which modifies the speed of the local timecode generator as necessary to maintain its synchronization with the master timecode generator (e.g., a timecode generator contained within the RCU or master recorder). This algorithm may be implemented using any one of a variety of methods. In one embodiment of the present invention, a feedback loop algorithm, such as process 500 depicted in
Referring now to
Alternatively, if at 510 TCdiff is not less than zero, process 500 proceeds to 514, at which TCdiff is analyzed to determine if it is greater than zero. If yes, process 500 proceeds to 516 and the feedback error voltage supplied to the local oscillator's DAC by the local control unit is decreased below the previously supplied feedback error voltage. The local oscillator's DAC then supplies the new feedback error voltage to the local oscillator, which, in turn, supplies a new clock input voltage to the local timecode generator and a new sample rate input to the ADC. In this manner, the speed of the local timecode generator and the sample rate of the ADC are decreased to maintain synchronization with the master timecode generator. However, alternate embodiments of the present invention are envisioned in which only one of either the speed of the local timecode generator or the sample rate of the ADC is modified. Furthermore, alternate embodiments are envisioned in which an inverse relationship occurs (e.g., DAC voltage is increased when TCDiff is greater than zero and it is decreased when TCDiff is less than zero).
If TCdiff is neither less than zero as determined at 510 or greater than zero as determined at 514, then TCdiff is equal to zero. In this scenario, the local and master timecode generators are synchronized and, therefore, no adjustment is made to the speed of the local timecode generator. At this point, process 500 ends at 518.
Although
Referring back to
Turning next to
At 604, a master unit, such as RCU 104, receiver 106, or recorder 108 transmits master timecodes to each local audio device, and process 600 proceeds to 606. At 606, each local audio device synchronizes (e.g., jam syncs) its respective on board local timecode generator with the master timecodes received from the master unit, thereby synchronizing all local audio device timecode generators with the master timecode generator contained within the master unit. Process 600 then proceeds to 608. At 608, local audio devices begin locally recording audio received from an audio input device. This audio is stored in the memory of the respective local audio device with timestamps generated by the local timecode generator. Each local audio device also simultaneously transmits its received audio to recorders or receiver/recorder combinations such as receivers 106 and recorders 108 in real time. The audio received from each of the local audio devices (e.g., the local audio device of each performer) may be combined to create one or more multi-track audio files that are stored with master timestamps generated by the receiver/recorder's internal master timecode generator.
Process 600 then proceeds to 610. At 610, process 600 queries the initiation of audio replay. The initiation of audio replay may be manual or automatic. For example, if a user detects a loss of audio, the user may manually initiate audio replay beginning at the specific timecode reference at which the transmission error occurred. Alternatively, if a loss of audio is automatically detected by the receiving equipment, a playback request may be sent from the receiving equipment to the controlling unit such as a remote control unit. In response, such controlling unit may command the local audio devices to replay or retransmit the missed audio to the receiving equipment beginning at the timecode at which the loss of data occurred or at a conveniently close time thereto (e.g., zero to ten seconds prior to the loss of data).
If, at 610, audio replay is not initiated either manually or automatically, process 600 returns to 608. However, if, at 610, audio replay is initiated, process 600 proceeds to 612. At 612, a controlling unit, such as RCU 104, sends a signal to the local audio devices requesting playback of the stored audio starting at a specific timecode.
Next, at 614, each local audio device processes the playback command and synchronizes playback to the timecode contained in the playback command. In addition, at least one local audio device transmits the synchronization data to the receiving equipment (e.g., receiver 106, recorder 108, etc.) to synchronize recording of the replayed audio. Process 600 then proceeds to 616. However, in alternate embodiments of the present invention, the receiving equipment and the local audio devices may simultaneously receive the synchronization and time reference data from the transmitting equipment (e.g., the controlling unit).
At 616, one or more local audio devices transmit, or replay, its respective stored audio starting with the audio that corresponds to the time specified by the timecode. The receiving equipment simultaneously records the replayed audio from each of the local audio devices and stores it within the previously recorded audio according to its timecode data. That is, due to the highly accurate synchronization of all of the components of the recording system, the receiving equipment may insert the replayed audio data that was not recorded during the audio event due to wireless transmission errors into the original recording at the nearly the exact time at which the missed audio originally occurred, thereby compensating for any transmission losses. Process 600 then proceeds to 618. At 618, one or more local audio devices continue to replay audio while the receiving equipment records the audio.
At 620, process 600 queries the status of audio replay. If, at 620, the audio has been fully replayed, process 600 proceeds to 608. At 608, the local audio devices may record a new audio event or may replay a different segment of recorded data. Otherwise, if, at 620, all requested audio has not been replayed or re-recorded, process 600 returns to 618.
Referring now to
At 704, initialization occurs. During initialization, the local control unit such as local control unit 310 or other form of central processing unit is reset. Thereafter, the local transmitter, local receiver, ADC, DAC, and clock are initialized. The process then proceeds to 706, at which the sampling rate of the ADC is set. In some embodiments of the present invention, a sample rate of 48 kHz is incorporated.
Next, at 708, wireless receive channels are established between the local audio device and one or more wireless devices such as RCUs (e.g., RCU 104), receivers, and audio recorders. To establish the channel, the local receiver of the audio device receives one or more data packets from the remote wireless device and stores the packets in a designated buffer. For example, when establishing wireless communication with a RCU, the local audio device may receive one or more data packets containing information such as a timecode, transport status (i.e., transport mode of an audio recorder), and the like. These packet(s) are then stored in an RX buffer. Process 700 then proceeds to 710.
At 710, the local control unit reads the transport status and the master timecode contained in the RX buffer. Next, at 712, process 700 queries the transport status. If at 712, the transport status is stop, process 700 returns to 710. However, if at 712, the transport status is record, process 700 proceeds to 714. At 714, a new audio file is created in memory (e.g., on a flash card) and the timecode is stored in the header of the newly created file. Process 700 then proceeds to 716.
At 716, the local control unit waits for an audio sample interrupt from the ADC. Once an audio sample interrupt occurs, process 700 proceeds to 718. At 718, the audio sample is retrieved from the ADC and stored in the local memory. In one aspect of the present invention, the audio sample is stored in the next available address of the local memory. Process 700 then proceeds to 720, at which the local control unit transmits the audio sample through the local transmitter to the other wireless devices such as receivers, audio recorders, and the like.
At 722, process 700 queries the RF buffer of the local receiver to determine the availability of a new master timecode packet. If at 722, a new master timecode packet has not been received from the RF receiver, process 700 returns to 716. However, if at 722, a new master timecode packet has been received, process 700 optionally proceeds to 724. At 724, the timecode is stored as an escape sequence in the next available address of the local memory. Process 700 then proceeds to 726. At 726, process 700 queries the continuous loop record mode. If at 726 the continuous loop record mode is enabled, process 700 returns to 716 to wait for an audio sample interrupt from the ADC as discussed above. However, if at 726, the continuous loop record mode has not been enabled, process 700 proceeds to 728. At 728, process 700 queries the transport status. If at 728 the transport status is record, process 700 returns to 716 to wait for an audio sample interrupt from the ADC as discussed above. However, if at 728, the transport status is stop, process 700 returns to 710, at which process 700 continuously reads the transport status and master timecodes from the RX buffer until the transport status changes from stop to record at 712.
Operation of the present invention in asynchronous mode allows one or more components of local audio devices such as local audio devices 102 (e.g., local timecode generator, comparator, counter, etc.) to be eliminated in embodiments in which the local audio devices utilize master timecodes generated by the master timecode generator rather than locally generated timecodes.
Deferring next to
In one aspect of the present invention, the memory of each local audio device such as local audio device 102 may be removed after completion of a performance, videotaping, etc. Each memory may then be inserted into a corresponding one of memory ports 802. Thereafter, all of the individual audio files may be combined to provide one or more comprehensive audio files. Or, alternatively, each audio file may be individually reformatted or otherwise manipulated prior to creation of one or more comprehensive audio files.
In embodiments of the present invention in which the recording system recorded the audio event in asynchronous mode, or in which long periods (e.g., 8 hours) of recording occurred, multi-memory unit 800 may be used to resample the audio samples to ensure that each audio file's timestamps are properly synchronized. One example of such as process is illustrated in the flowchart of
Referring now to
The resampling process depicted in
If all of the audio from all local audio devices is resampled in this manner, each resulting resampled audio file appears as if it was originally sampled with an accurate audio sample clock derived from the master timecode source. This resampling allows each audio file to include a single timestamp that marks the master timecode of the first audio sample of the audio file. Furthermore, since the audio files now appear as if they have been sampled by an extremely accurate audio sample clock, each audio sample's timestamp may be accurately calculated based solely on the audio sample rate and the timestamp of the first audio sample of the audio file. This condition allows the audio files to be formatted and/or stored as a standard timecoded broadcast .WAV file, thereby allowing them to be read, edited, etc. using standard, commercially-available editing systems. That is, the files may be processed in the same manner as if the audio file had been generated by a standard multi-track audio recorder. Such condition allows the present invention to be easily integrated with other industry standard recording equipment.
One such resampling process is illustrated in
At 904, process 900 determines the desired starting and ending timecodes and stores this data in the variables TimeCodeStart and TimeCodeEnd, respectively. The desired starting and ending timecodes may be input by a user or may be suggested or automatically determined by the algorithm. Process 900 then proceeds to 906. At 906, a variable, i, is initialized to a value of zero. The variable i corresponds to the position of audio samples or data points in a data array represented by the variable AudioSample[i]. Process 900 then proceeds to 908.
At 908, process 900 begins an iterative search for the audio file that matches the desired starting timecode of the output file by comparing the value of TimeCodeStart with the value of the timecode of AudioSample[i]. If, at 908, the value of TimeCodeStart is equal to the value of the AudioSample[i] timecode, process 900 proceeds to 912. However, if at 908 the value of TimeCodeStart is not equal to the value of the AudioSample[i] timecode, process 900 proceeds to 910. At 910, the variable i is increased by a value of one thereby allowing the value located in the next position of the audio sample array to be compared to the value of TimeCodeStart when process 900 returns to 908.
If the value of TimeCodeStart is equal to the value of the AudioSample[i] timecode, process 900 proceeds to 912. At 912, a variable, n, is initialized to a value of one. The variable n is added to the variable i to allow process 900 to continue to traverse the audio sample array while maintaining the location of the audio sample at the starting timecode, which is represented by the variable AudioSample[i]. Process 900 then proceeds to 914. At 914, the value of the AudioSample[i+n] timecode is compared to the value of TimeCodeEnd. If at 914, the value of the AudioSample[i+n] timecode is greater than or equal to the value of TimeCodeEnd, process 900 proceeds to 916. At 916, the value of the AudioSample[i+n] timecode is again compared to the value of TimeCodeEnd. If at 914, the value of the AudioSample[i+n] timecode is greater than the value of TimeCodeEnd, process 900 proceeds to 928, at which process 900 terminates. However, if at 916, the value of the AudioSample[i+n] timecode is equal to the value of TimeCodeEnd, process 900 proceeds to 922.
Conversely, if at 914, the value of the AudioSample[i+n] timecode is less than the value of TimeCodeEnd, process 900 proceeds to 918. At 918, the value of the AudioSample[i+n] timecode is compared to the value of CurrentTimeCodeEscapeSequence. If, at 918, the value of the AudioSample[i+n] timecode is not equal to the value of TimeCodeEscapeSequence, process 900 proceeds to 920 where the variable n is increased by one and process 900 returns to 914. However, if at 918, the value of the AudioSample[i+n] timecode is equal to the value of TimeCodeEscapeSequence, process 900 proceeds to 922.
At 922, the average time period “T” that elapsed between the audio samples that occurred between AudioSample[i] and AudioSample[i+n] may be calculated by subtracting the value of the timecode of AudioSample[i] from the value of the timecode of AudioSample[i+n] and dividing by n, wherein n is now equivalent to the number of audio samples that occurred between the current timestamped audio sample and the previous timestamped audio sample. Process 900 then proceeds to 924. At 924, AudioSamples[i] through AudioSamples[i+n] are re-sampled at any desired sample rate based upon the value of T as calculated in 922, or any other desired sample rate, using an audio resampling algorithm (e.g., linear interpolation). Process 900 then proceeds to 926, at which the variable i is set to a value equal to the current value of i plus the current value of n and process 900 returns to 912. The iterative process continues until the value of the AudioSample[i+n] timecode is greater than the value of TimeCodeEnd, whereby process 900 proceeds to 928, at which process 900 terminates.
A similar interpolation algorithm, such as the algorithm depicted in
In one use of an embodiment of the present invention, multiple local audio devices store audio samples with wirelessly-received timecode and transport status samples continuously for the entire duration of the work day (e.g., an 8 hour period). In a typical scenario, while the local audio devices are recording continuously, a technician intermittently records segments of the eight-hour audio event. For example, in a film setting, each segment would typically represent a movie ‘take’ and might range from one to five minutes in duration. Consequently, the master recorder generates individual audio files (i.e., at least one audio file for each recorded segment such as a movie take), whereas each local audio device generates one massive audio file. Therefore, there is a need for a method of segmenting each large local audio file into smaller audio files that correspond to the segments recorded by the master recorder.
The segmentation method (i.e., the method of segmenting the large local audio devices' files to match the multiple, smaller master recorder's audio file) requires knowledge of which portions of the single local audio device audio file are important and which portions can be discarded. This information can be inferred from the transport status of the master recorder since it is typically operated by someone with this knowledge. Therefore, when the transport status of the master recorder changes from stop to record, it can be inferred that a new master recorder audio file begins, and, subsequently, when the transport status of the master recorder changes from record to stop, it can be inferred that the same master recorder audio file has ended. In addition, when the transport status of the master recorder remains in the stop mode, it can be inferred that the audio recorded by the local audio device during this time period may be discarded. This audio may be discarded post-processing as per algorithms such as that depicted in
In embodiments of the present invention in which such data is discarded during live recording, the transport status and master timecode of the master recorder are wirelessly transmitted to the local audio devices. This information may be processed by the local audio devices to allow them to create a new audio file with the current master timecode of the master recorder whenever the received transport status and master timecode indicate that the transport status has changed from stop to record. Similarly, the local audio devices may end the newly created audio file when the received transport status indicates that it has changed from record to stop. In this scenario, the resulting local audio device files will automatically be segmented and will each be marked with a master timestamp at the beginning of each file.
However, in embodiments of the present invention in which unimportant audio is not discarded during live recording and, therefore, one or more large audio files are created, the large audio files may be segmented as per a process such as process 1000 as illustrated in
At 1004, a copy of the audio file directory containing the segmented audio files that correspond to the same time period as the local audio device's single large audio file is obtained from the master recorder. Process 1000 then proceeds to 1006. At 1006, a variable y is initialized to a value of zero. The variable y corresponds to the number of each file contained in the audio file directory copied from the master recorder. Process 1000 then proceeds to 1008, at which the variable y is increased by one and a variable x is initialized to a value of one. The variable x corresponds to the position of each audio sample within a particular file. Process 1000 then proceeds to 1010, at which the copied audio file directory is queried to determine if a file[y] (i.e., the file named with the number that corresponds to the value of y) exists in the audio file directory. If no, process 1000 proceeds to 1028 and terminates.
If file[y] does exist, process 1000 proceeds to 1012, at which process 1000 determines the starting and ending timecodes for file[y] and stores them in the variables TimeCodeStart and TimeCodeEnd, respectively. Process 1000 then proceeds to 1014, at which process 1000 compares the value of TimeCodeStart to the value of the timecode associated with AudioSample[x] stored in the memory of the local audio device. If at 1014 the value of TimeCodeStart is not equal to the value of the timecode associated with AudioSample[x], process 1000 proceeds to 1016. At 1016, the variable x is increased by one and process 1000 returns to 1014. In this manner, TimeCodeStart is compared to each consecutive AudioSample[x] until the AudioSample timestamped with a value equal to TimeCodeStart is found. In some embodiments of the present invention, process 1000, or an equivalent thereof, is performed after process 900, or an equivalent thereof, to ensure that each of the audio samples has a timestamp (e.g., an interpolated timestamp).
When the AudioSample[x] having a timecode equivalent to TimeCodeStart is found at 1014, process 1000 proceeds to 1018. At 1018, AudioSample[x] is extracted and process 1000 proceeds to 1020, at which the variable x is increased by one and process 1000 proceeds to 1022. At 1022, process 1000 compares the value of TimeCodeEnd to the value of the timecode associated with AudioSample[x]. If at 1022, the value of TimeCodeEnd is not equal to the value of the AudioSample[x] timecode, process 1000 returns to 1018, whereupon audio samples are consecutively extracted until the timecode of the current AudioSample[x] equals TimeCodeEnd. If, at 1022, the value of TimeCodeEnd is equal to the value of the timecode of AudioSample[x], process 1000 proceeds to 1024, at which the final AudioSample[x] of the segmented audio file is extracted and the audio file is saved at 1026.
Process 1000 then proceeds to 1008, at which the variable y is increased by one and process 1000 proceeds to 1010 at which the audio file directory is queried to determine the existence of file [y]. If file[y] exists, process 1000 proceeds to 1012 and it continues thereafter as described above. However, if at 1010, it is determined that file[y] does not exist, process 1000 proceeds to 1028, at which it terminates.
Although several processes have been disclosed herein as software, it is appreciated by one of skill in the art that the same processes, functions, etc. may be performed via hardware or a combination of hardware and software. Similarly, although the present invention has been disclosed with respect to wireless systems, these concepts may be applied to hardwired systems and hybrid hardwired and wireless systems without departing from the scope of the present invention.
While the present invention has been described with reference to one or more embodiments, which embodiments have been set forth in considerable detail for the purposes of making a complete disclosure of the invention, such embodiments are merely exemplary and are not intended to be limiting or represent an exhaustive enumeration of all aspects of the invention. The scope of the invention, therefore, shall be defined solely by the following claims. Further, it will be apparent to those of skill in the art that numerous changes may be made in such details without departing from the spirit and the principles of the invention.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5668884 | Clair et al. | Sep 1997 | A |
| 20010034214 | Koike | Oct 2001 | A1 |
| 20020026256 | Hilton | Feb 2002 | A1 |
