Patent Application
20210266113
The present invention relates to the field of wireless transmission, especially the field of wireless radio transmission of digital audio data with low latency (delay) and still with a high immunity against audio data loss due to radio interference and limited radio coverage. Specifically, the invention relates to methods for buffering of audio data blocks to provide a low latency and still minimizing the risk for audio artifacts due to wireless packet loss. Thus, the invention is advantageous for microphones, musical instruments, and gaming equipment such as keyboard, mouse, game controller, microphone, headphone and headset, or for video applications e.g. as part of Virtual Reality devices.
For wireless transmission of audio data, radio interference problems are especially pronounced for radio transmission in the license free ISM spectrum, i.e. in the 2.4 GHz and 5.8 GHz bands. In these bands many different wireless technologies operate in uncoordinated manners. It is unavoidable that wireless transmitted data can get interfered and lost, and thus also audio data can be lost and consequently can lead to audible artifacts. A high immunity against audible artifacts can be traded for an increased latency and vice versa.
Current practice for handling interference and wireless packet loss are different depending of technology and application. For example, in Bluetooth music streaming applications audio data packets are buffered in large buffers containing several hundreds of milliseconds (ms) of audio. In case of wireless packet loss audio data packets may be retransmitted when possible. For applications like music streaming it is not a problem to introduce latency of the size of +100 ms, since the streaming is one-way. E.g. solutions can be a long audio buffer on the receiver side, thus allowing for on-demand retransmission.
However, for wireless communication applications such as cordless and cellular telephony, a rather low latency is required, and here the effect of packet losses is often masqued by different packet loss concealment (PLC) methods depending on the used voice encoding algorithms. Other methods are based on redundant transmission of the same audio packet on different frequencies (dual slot diversity), where the same audio packet is always transmitted twice (or more).
For applications such as live stage performance involving wireless microphones and loudspeakers, a very low audio data latency and robustness against interference (to avoid audio drop-outs or other audible artifacts) are crucial requirements. The same applies to real-time bidirectional audio applications like headsets which typically require audio data communications with minimum latency for optimal function.
Thus, according to the above description, it is an object of the present invention to provide a robust transmission method that minimizes the risk for audio artifacts, but still keeps the latency low enough for demanding applications such as live performances etc.
In a first aspect, the invention provides a method for transmission of a digital audio signal divided into digital audio data blocks of a fixed length, e.g. 0.5-5.0 ms, between a fixed part (FP) being a timing master and one or more portable parts (PP) being timing slaves on a wireless Radio Frequency (RF) transmission channel according to a time division multiplexing or time division duplexing scheme with a fixed transmission time interval, e.g. 1 ms, wherein a raw payload bandwidth of the wireless RF transmission channel is a fixed factor larger, e.g. 1.5-3.0 times larger, than a bandwidth required to transmit the audio data blocks, wherein the wireless RF transmission comprises a fixed wireless frame length, each wireless frame containing one or more time slots, the method comprising:
Such method is advantageous to stream an audio signal over a wireless RF channel with limited capacity, e.g. in the ISM band, and where latency and robustness requirements are high. E.g. the method is suitable for stage microphones or loudspeakers, gaming headsets etc., where a fixed and low latency is important and using an RF channel with a limited capacity. Especially, using a short RQ and an equally short TQ of a length of such as 2-8 blocks in combination with short audio data blocks of such as 0.5-2 ms, the latency can be very low. Due to the timing synchronization in the handling of audio data blocks from input to the FP to output from the PP, via the RF transmission channel, the delay has a fixed value determined by the length of the audio data blocks and the lengths of the RQ and TQ. Still, the transmission is robust to packet losses due to the use of acknowledgement messages and retransmission of lost audio data blocks from the TQ, and even with a very limited raw payload capacity of the RF transmission channel. The higher capacity, the higher robustness can be obtained.
However, with a factor of such as 1.5-3 times the capacity required for transmission of the audio data blocks, a high robustness can be achieved and still with a latency lower than e.g. 10 ms.
The high immunity against interference allows application within high quality or ultra high quality audio.
The invention is based on the insight of the inventors that the application of a time division multiplexing or duplexing scheme in combination with short audio data blocks allow for a low (and fixed) latency transmission and at the same time, even with a short length TQ and RQ, a high robustness can be achieved even in combination with a limited extra transmission capacity of the RF channel available, i.e. even less than 3 times the payload capacity required to merely transmit the audio data blocks, preferably 2 times.
The practical implementation of a TQ and an RQ are known by the skilled person from other wireless transmission systems.
The method is further advantageous for applications where both audio and Human Interface Device (HID) data are preferably transmitted along with audio data. This is the case e.g. in wireless gaming equipment involving headsets as well as various gaming controllers. Such HID data must be transferred with the lowest possible latency, e.g. joystick position data and the like from a gaming controller, e.g. for on-line gaming. By nature, the subjective effect of lost HID data is not as significant as loss of audio data, and thus in some applications it may be preferred to transfer HID data with lowest possible latency and allow erroneous HID data to be discarded. According to the method, a transmitted wireless data packet can contain audio data blocks as well as HID data blocks. Preferably, in contrast to the audio data blocks, the HID data blocks are un-queued, and thus not possible to retransmit in case of transmission errors. Rather, the lack of queuing provides an ultra low latency and a lost data block is typically not critical for HID data, in contrast to audio data.
Further, the method is applicable also for full duplex audio data transfer between FP and PP(s), namely where the roles of the FP and PP are reversed compared to the definition of the first aspect.
The below phrases will be used and are therefore explained:
In the following preferred embodiments and features will be described.
The invention is applicable with wireless RF transmitter and receiver hardware including electronics and antennas etc. as known in the art. Thus, in the description of the present invention, it is to be understood to be applicable with any such known hardware to implement the FP and PP(s). The invention is applicable in general independent of the selected RF bearer. Especially, the RF bearer frequency can be selected to be within 1 GHz to 200 GHz, e.g. within 2-5 GHz. E.g. a license free ISM band may be selected.
The audio data blocks may be formed by any digital representation of an audio signal. E.g. digital data indicative of compressed or non-compressed audio sampled at a rate of such as 8, 16, 24, 32, 44.1, 48, 96 or 192 kHz and with any amplitude resolution, e.g. of such as 8, 10, 12, 16, 20 or 24 bit. Especially, the audio data blocks may represent an ADPCM encoded audio signal.
The fixed length of the TQ is preferably 2-8, such as 2-4, e.g. 3, 4 or 5 may be chosen. The fixed length of the RQ is also preferably, 2-8, and especially, the TQ and the RQ may have equal lengths, specifically both TQ and RQ may have a length of 3, 4 or 5. In general, the TQ and RQ length adds to the total delay, but the possibility of avoiding audible artifacts in case of packet loss increases with increased TQ and RQ length.
The fixed length of the audio data blocks may preferably be selected to be within 0.2 ms to 8 ms, especially such as 0.5 ms to 4.0 ms. Such short blocks allow for a very low total latency.
The fixed length of the wireless frame is preferably selected to be within 0.2 ms to 20 ms, such as 0.2 ms to 10 ms, such as a length of 0.2 ms to 5 ms, such as 0.5 ms to 3 ms.
In a specific set of embodiments,
In preferred embodiments, a ratio between required audio data bandwidth and wireless payload transmission bandwidth is 1:N, where N is an integer number larger than 1. In such configuration the size of audio data blocks entered to the transmission queue and the size of transmitted audio data blocks will be the same. A preferred value is N=2, which enables a specific an error handling method in the transmitter side. For N=2, and when the TQ runs full on the FP side, the oldest audio data block in the queue will effectively be transmitted twice, which exactly corresponds to a traditional dual slot transmission method. For configurations where N is larger than 1, but is not an integer value, the transmitted audio data blocks have a different size than audio data blocks entered in to the queue. In such configurations the partly content of two audio data blocks have to be concatenated to form data packet of proper size for transmission. On the PP side, reverse processing must be performed.
The method may comprise
Especially, said HID data block is preferably transmitted un-queued at the PP and output un-queued at the FP. Thus, in spite the same RF link is used, the audio data and the HID data can be treated separately, where audio data blocks are queued for possible retransmission, while HID data blocks are un-queued.
The method may comprise retransmitting one queued audio data block from the FP a plurality of times. Possible strategies or algorithms for handling retransmission in response to e.g. the available transmission capacity in relation to the number of queued audio data blocks in the TQ and RQ will be described in more details later.
Preferably, a transmission sequence number is attached to the audio data blocks transmitted over the wireless interface. This transmission sequence number is incremented for each transmission of new audio data block. In case of retransmission of audio data blocks, the transmission sequence number remains unchanged. This sequence number enables the receiver to distinguish between reception of new audio data blocks and retransmission of already received audio data blocks. Received retransmitted audio data blocks are discarded in the receiver. The transmission sequence number is usually transmitted in a wireless protocol control data field.
The receiver side preferably acknowledges the received audio data blocks by transmitting an acknowledgement reply containing the sequence number of the last successfully received audio data block. This information is sent from the receiver side in the wireless protocol control data field and usually named receiver sequence number. The value of the receiver sequence number may alternatively be coded as the value of the next sequence number to be received. At the transmitter side, when receiving the receiver sequence number, the transmitter may compare the transmitter sequence number and the receiver sequence number and determine if the audio block transmission has been acknowledged by the receiver. Because the transmitter side does only transmit one new audio data block, and potentially retransmit the same audio data block until acknowledgement is received from the receiver side, the size of the transmitter sequence number and receiver sequence number can be reduced to a single bit.
The method preferably comprises duplex audio data transmission between the fixed part (FP) and the portable part (PP). Especially, the PP may be arranged to transmit audio data blocks to the FP on said RF transmission channel. Especially, this may be interesting for a PP in the form of a wireless headset applications, which have both microphone and speaker. Preferably, there are then separate queues for the two transmission directions and the audio data block queuing method operates independently for the two directions. In such configuration, the wireless protocol control field contains both transmitter sequence number and receiver sequence number. It is to be understood that the same audio data block transmission scheme can be used for PP to FP audio transmission as described for the FP to PP audio transmission.
Especially, the wireless frame may comprise at least one time slot for downlink data, and one or more time slots for uplink data.
Each time slot may comprise a Cyclic Redundancy Check (CRC) error check field. Especially, seeding of the CRC derived from system identity can be used for discriminating packets between multiple systems.
In advantageous embodiments of the method, dualslot transmission is provided. Especially, this may be in the form of RF transmission of one audio data block on two RF bearers with different frequencies in one wireless frame, i.e. providing a payload capacity of a factor of 2 times what is required. If retransmission of an audio data block is needed, one wireless frame may comprise data from a new audio data block as well as the audio data block to be retransmitted. In case there is no need for retransmission, the new audio data block can simply be transmitted on both RF bearers, i.e. 100% redundant transmission. This method allows retransmission of data block several times. Especially, the two RF bearers may be selected based on measures of actual interference levels, thus it is possible to select two frequencies among a set of supported RF channels, with the minimal RF interference. Thus, preferably the two duplex bearer frequencies dynamically allocated and reallocated in order avoid or minimize interference and to cause minimum interference to other users of the same frequency band. Thus, the method involves both time diversity and frequency diversity by the transmission of the same data in two different timeslots in each wireless frame and on two different channels. A selection algorithm preferably serves to determine first and second frequencies F1, F2 which are indicated as the RF bearers having the lowest measure of interference level, among all supported channels. To further increase communication range and transmission reliability, antenna diversity can be used, i.e. using two or more antennas on both FP and PP for transmitting and receiving, which adds space diversity additionally.
In a second aspect, the invention provides a wireless RF transmitter comprising at least one RF transmitter circuit connected to an antenna, and being arranged to operate as the FP according to the method of the first aspect.
In a third aspect, the invention provides a wireless RF receiver comprising at least one RF receiver circuit connected to an antenna, and being arranged to operate as the portable part PP according to the method according to the first aspect.
It is to be understood that the required RF circuit, antenna and programming required to implement a RF transmitter and RF receiver, as well as handling of time synchronization between FP and PP, will be known by the skilled person based on the present description of the inventive method.
In a fourth aspect, the invention provides a system comprising at least one of: the wireless RF transmitter according to the second aspect, and one or more wireless RF receivers according to the third aspect. Especially, the system may be one of: a wireless headset, a wireless mouse, a wireless gaming controller, a wireless keyboard, a wireless microphone, a wireless loudspeaker, and a wireless intercom system, a video system, and a Virtual Reality system.
Audio devices may require fixed bandwidth and fixed latency, whereas some HID devices may have different requirements to bandwidth and latency. The requirements to downlink and uplink may be different. However, it is to be understood that all can be complied with in embodiments of the invention.
The method, and the RF transmitter and receiver devices arranged to operate according to the method may form separate dedicated devices, or these components may be integrated partly or fully into audio devices.
In a further aspect, the invention provides a computer readable program product comprising an executable code part for an FP device and an executable code part for a PP device, which when executed on respective processors in the FP device and the PP device, cause the FP device and the PP device in cooperation to perform the method according to the first aspect.
It is appreciated that the same advantages and embodiments described for the first aspect apply as well the further mentioned aspects. Further, it is appreciated that the described embodiments can be intermixed in any way between all the mentioned aspects.
The invention will now be described in more detail with regard to the accompanying figures of which
The figures illustrate specific ways of implementing the present invention and are not to be construed as being limiting to other possible embodiments falling within the scope of the attached claim set.
The FP receives an input audio signal A_I, and transmits wireless data packets WP each with an audio data block on the RF bearer, and the wireless data packets WP are received by the PP which unpacks the audio data blocks to arrive at the original audio signal A_O. The PP transmit in response to a successfully received wireless data packet an acknowledgement message ACK back to the FP, which can then retransmit a lost audio data block in the next wireless data packet.
In some embodiments, the FP performs a scanning and preferably performs own measurement as well as collects Received Signal Strength Indication data from the PP for the supported channels, so as to dynamically select the two frequencies F1, F2 according to a predetermined algorithm for finding the frequencies F1, F2 with the minimal possible RF interference from other RF traffic in the RF band covering the supported channels, e.g. an ISM band.
By using short audio data blocks, and short buffers or queues at the FP and the PP side, a low latency can be obtained, and with a small extra capacity of the RF transmission channel, a high reliability can be obtained, since it is possible to retransmit lost wireless data packets WP. Using an RF channel with a fixed capacity of a factor of 2 of the payload capacity required for transmission of the audio data blocks, and with audio data block lengths of such as 2 ms, and with audio data block buffer or queue length at the FP and the PP side with a length of 4, a good compromise between low latency and high immunity against packet losses has been found. The low latency and high reliability may e.g. be appreciated for products such as gaming devices with audio capabilities.
In
In
The following method steps are understood to be continuously performed in each time interval in order to stream of audio. First step is receiving R_AB a digital audio data block at the FP, e.g. directly form of an output of an analog-to-digital converter from an audio device, e.g. a gaming headset, or from an output from a processor having performed a pre-processing, e.g. involving a compression, encryption or the like, on the audio signal. The audio data blocks have a fixed predetermined length, such as 2 ms.
Next step is queuing TQ_AB the audio data block at an empty space in a fixed length FIFO type TQ having 4 spaces for audio data blocks.
Next step is transmitting T_WP a wireless data packet containing an audio data block from an output end of the TQ from the fixed part FP to the portable part PP on the wireless RF transmission channel.
At the PP end, the wireless data packet is received R_WP, and the audio data block therein is queued RQ_AB at an empty space in an input end of a fixed length FIFO type RQ having 4 spaces for audio data blocks. In response to the received audio data block, the PP transmits T_ACK an acknowledgement message to the FP on the RF transmission channel. Further, the PP outputs O_AB an audio data block from an output end of the receiver queue RQ, and thus the resulting audio data block has been successfully transmitted, and e.g. the audio data therein can be applied to a digital-to-analog converter, e.g. after being decoded or decrypted, if required, e.g. to produce an audio signal to be played by a loudspeaker or a headphone or the like.
Some embodiment involve duplex transmission between the FP and the PP. This may be in the form of 2-way audio data transfer, and in addition it is possible as well to provide 1- or 2-way HID data transfer.
In a special embodiment, dualslot diversity is provided in the form of transmission of two data packet in one wireless frame, on two different RF transmission bearer frequencies. Such embodiment may especially involve sequentially scanning such as 10-100 supported channels within a limited frequency band, such as a limited band around 2.4 GHz, wherein the channels closely spaced in frequency, e.g. space by such as spaced by 1-2 MHz. The scanning serves to determine two bearer frequencies for the RF transmission, where the scanning reveals the least RF interference level. A control algorithm can be provided to determine which two data packets to transmit in a time window. Such control algorithm may have inputs regarding number of audio data blocks in the TQ and RQ, as well as other parameters.
In the following, a further explanation of preferred embodiments will be described, i.e. related to the queue or buffer of audio data blocks on the transmitter side, the transmitter queue TQ, and the queue or buffer of audio data blocks on the receiver side, the receiver queue RQ. The TQ and RQ are both FIFO types.
On the transmitter side, audio data blocks are entered in to the TQ at a fixed rate. On the receiver side the audio data blocks are taken out of the RQ at a fixed rate. In a transmitting device, such as for example a wireless microphone, the analog signal from a microphone transducer is analog-to-digital converted. The digital audio data, e.g. with a resolution of 8 to 32 bit, is compressed by an audio compression algorithm, typically implemented in an Digital Signal Processor. (DSP). The output data of the audio compression are collected to fixed size audio data blocks. The audio data blocks are entered to the TQ. At next occurrence of timeslot for wireless transmission, the oldest audio data block from the TQ is transmitted. If the TQ is empty when wireless transmission timeslot occurs, the wireless transmitter can either omit transmission or it can transmit an empty data packet. Usually, the wireless transmitter also transmits a wireless protocol control field, which may indicate empty data packet. When the transmitter side receives acknowledgement from the receiver side, the oldest audio data block is deleted from the TQ. If no acknowledgement is received, the audio data block will stay on the TQ and be retransmitted at next occurrence of timeslot for wireless transmission.
At the receiving device, such as for example a loudspeaker, when the wireless receiver receives an audio data block, it is entered to RQ. The receiver side replies with an acknowledgement to the transmitter side. If the RQ is already full, no acknowledgement is sent. At fixed time interval the oldest audio data block is removed from the RQ, and each data element in the block is feed to an audio de-compression algorithm, typically implemented in an DSP. The digital output data, which may have a resolution of 8 to 32 bit, may be digital-to-analog converted and outputted to a loudspeaker transducer.
The state of the TQ and RQ when no wireless transmission errors occur is a full RQ and an empty TQ. When wireless transmission errors occur, the RQ is not full, and the TQ is not empty. As long as the TQ does not run full and RQ does not run empty, there is a fixed latency from audio data input in the transmitter side to audio data output in the receiver side, even when retransmissions of wireless packets occur. Usually, the time for entry of an audio data block to TQ and the time for removal of audio data block from the RQ are aligned to the wireless transmission frames for achieving minimum overall system latency.
As the preferred state of the queues are a full RQ and an empty TQ, special handling is required during initialization of an audio transfer session. During such initialization phase, no audio data blocks are removed from the RQ until the RQ is completely full.
In the scenarios of server transmission error rate, which result in the empty RQ and full TQ, specific error handling procedures may be performed. When the TQ is full and new audio data blocks are ready to be entered to the TQ, the oldest audio data block is removed from the TQ, even though it has not been acknowledged from the receiver side. At the receiver side, when the RQ is empty, and an audio block is requested, the de-compression algorithm is informed about missing audio data and can perform packet loss concealment (PLC) to eliminate or at least reduce audible artifacts.
Following the above description of the preferred function of the TQ and RQ,
The solid arrows of the wireless interface WI represent successful transmission of wireless data packets, and white arrows represent empty wireless data packet transmissions. An empty wireless packet transmission means one without audio data block or one with redundant transmission of same audio data block, but with unchanged transmission sequence number. In frames Fr3, Fr4 and Fr5, the shaded arrows represent wireless data packet transmission errors. In frame Fr5 the TQ is nearly full and the RQ is nearly empty. In the frames Fr6, Fr7 and Fr8, both wireless transmissions are successful, and the preferred status of TQ and RQ are regained. In frames Fr9, Fr10 and Fr11 the wireless transmissions are error free and the TQ and RQ have their preferred status.
In a specific embodiment, the MAC/PHY is designed based on the following features:
The following are preferred features:
To sum up, the invention provides a method for wireless RF transmission of short audio data blocks, e.g. 0.5 ms to 2 ms blocks, with low latency. The method involves a fixed part (FP) serving as synchronization master, and one or more portable parts (PP) being synchronization slaves in a time division scheme with fixed transmission intervals, and with a fixed and limited payload capacity of the RF transmission channel, such as 1.5-3 times the capacity required to transmit the audio data blocks in real-time. Short length transmission and receiving queues (TQ, RQ), e.g. having each 2-8 spaces for audio data blocks, for the audio data blocks are used to allow retransmission of blocks in response to a missing acknowledge response from the portable part (PP). The queuing is operated so as to result in a fixed latency determined e.g. by the transmission and receiving queue (TQ, RQ) lengths. A two-way audio scheme can be implemented following the same principle and utilizing the same RF transmission principles. The method provides a roboust and low latency wireless audio interface suitable for dedicated audio devices and/or combined audio and Human Interface Devices (HIDs), e.g. for gaming equipment.
Although the present invention has been described in connection with the specified embodiments, it should not be construed as being in any way limited to the presented examples. The scope of the present invention is to be interpreted in the light of the accompanying claim set. In the context of the claims, the terms “including” or “includes” do not exclude other possible elements or steps. Also, the mentioning of references such as “a” or “an” etc. should not be construed as excluding a plurality. The use of reference signs in the claims with respect to elements indicated in the figures shall also not be construed as limiting the scope of the invention. Furthermore, individual features mentioned in different claims, may possibly be advantageously combined, and the mentioning of these features in different claims does not exclude that a combination of features is not possible and advantageous.
| Number | Date | Country | Kind |
|---|---|---|---|
| 18182173.7 | Jul 2018 | EP | regional |
| 18197482.5 | Sep 2018 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2019/067859 | 7/3/2019 | WO | 00 |
