Network traffic prioritization

Information

  • Patent Grant
  • 8005957
  • Patent Number
    8,005,957
  • Date Filed
    Tuesday, December 21, 2010
    14 years ago
  • Date Issued
    Tuesday, August 23, 2011
    13 years ago
Abstract
Prioritizing network traffic among two or more distinct channels of communication within a single application in a node configured to communicate with one or more other nodes over a network is disclosed. For a particular time quantum, a bandwidth quantum may be distributed amongst two or more communication channels according to priorities associated with those channels.
Description
FIELD OF THE INVENTION

The present invention relates to computer network communication, and more specifically to prioritizing network traffic among distinct channels of communication within a single application.


BACKGROUND OF THE INVENTION

Computing systems are becoming increasing interconnected through network environments. Such network environments may be centralized or decentralized. A decentralized computing environment may be defined by a number of computing systems interconnected to communicate with one another, wherein each computing system can perform both client and server functions. A peer-to-peer (P2P) network represents an example of a decentralized computing environment in which each computing system within the P2P network is defined as a peer of every other computing system within the network. For discussion purposes, each peer computing system within the P2P network is referred to as a node. Additionally, each node within the P2P network may be configured to execute software having substantially equivalent functionality. Therefore, each node may act as both a provider and a user of data and services across the P2P network. Peer to peer networks are distributed data networks without any centralized hierarchy or organization. Peer to peer data networks provide a robust and flexible means of communicating information between large numbers of computers or other information devices, referred to in general as nodes.


A P2P network relies primarily on the computing power and bandwidth of the participants in the network rather than concentrating it in a relatively low number of servers. P2P networks are typically used for connecting nodes via largely ad hoc connections. Such networks are useful for many purposes. P2P networks may be used, e.g., for sharing content files containing audio, video, data or anything in digital format is very common, and real-time data, such as telephony traffic, may also be transmitted using P2P technology.


P2P applications often involve a significant amount of communication between nodes over different communication channels. By way of example, such channels may include an audio channel, a video channel, and a file transfer channel. A given application, e.g., audio-video (A/V) chat may communicate using all three channels. Typically, an application has a limited amount of network bandwidth available for communication. The application distributes the available bandwidth among the communication channels.


Prior art network implementations involving multiple communication channels typically adopt an “all or nothing” approach that can lead to starvation. For example, consider a very low bandwidth situation where a user is attempting to engage in A/V chat involving transmission of captured audio and video frames. If the user does not have enough bandwidth available to transmit all of the captured audio and video frames, prior art techniques typically gives complete priority to the audio frames and not transmit any video frames. This may reduce quality of service for the A/V chat.


It is within this context that embodiments of the present invention arise.





BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention may be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which



FIG. 1 is a block diagram of networked devices illustrating prioritization of network traffic according to an embodiment of the present invention.



FIG. 2 is a flow diagram illustrating a method of for prioritizing network traffic according to an embodiment of the present invention.



FIG. 3 is a block diagram of a node configured to implement network traffic prioritization according to an embodiment of the present invention.



FIG. 4 is a flow diagram illustrating a particular example of prioritizing network traffic amongst audio, video and file transfer channels in a node of the type shown in FIG. 3 according to an embodiment of the present invention.





DESCRIPTION OF THE SPECIFIC EMBODIMENTS

Although the following detailed description contains many specific details for the purposes of illustration, anyone of ordinary skill in the art will appreciate that many variations and alterations to the following details are within the scope of the invention. Accordingly, the exemplary embodiments of the invention described below are set forth without any loss of generality to, and without imposing limitations upon, the claimed invention.


According to embodiments of the present invention, bandwidth starvation issues may be avoided by prioritizing network traffic among distinct channels of communication within a single application. FIG. 1 depicts a flow diagram of a method 100 for prioritizing network traffic among two or more distinct channels of communication within a single application in a node configured to communicate with one or more other nodes over a network. Embodiments of the present invention may be understood by referring to FIG. 1 and FIG. 2.


As seen from FIG. 1, several nodes, e.g., Node A 110, Node B 120, Node C, 130 and Node D 140 may be configured to communicate over a network 101. Each node may include a network interface 126 to facilitate communication over the network 101. Two or more nodes, e.g., Node A 110 and Node B 120 may be behind a router 150. Two or more nodes may run an application 102 that allows the two nodes two communicate over two or more distinct channels C1 . . . CN, where N is greater than or equal to 2. Each channel represents a different mode of communication of data traffic. The data for each mode communication may be formatted differently from that of the others. Examples of “channels” include, but are not limited to, audio streams, video streams, file transfer, drawing commands on a shared whiteboard, or any other bulk classification of data traffic. By way of example, and without loss of generality, the application 102 may be an audio-video (A/V) chat application involving audio, video and file transfer channels.


Each node 110, 120, 130, 140 may include a bandwidth module 104, which may be implemented in software or hardware. The bandwidth module 104 is responsible for estimating the available bandwidth for communication with the other nodes. For example, if Node A 110 is participating in audio video chat with Node C 130 and Node D 140, the bandwidth module estimates the available bandwidth Node A 110 has for communication with Node C 130 and Node D 140. Similarly, the bandwidth module 104 at Node C 130 may estimate the available bandwidth node C 130 has for communication with Node A 110 and Node D 140. In embodiments of the present invention, it is desirable to have accurate bandwidth estimates. A specific example of a bandwidth detection module is described in commonly assigned U.S. Provisional Patent Application 60/992,282, to Yutaka Takeda et al, entitled NETWORK BANDWIDTH DETECTION AND DISTRIBUTION. Network traffic prioritization may be implemented by a scheduler 106 running on one of the nodes, e.g., Node A 110. The scheduler 106 may be implemented in software, hardware, or some combination of software and hardware. By way of example, and without loss of generality, the scheduler 106 may be programmed with instructions that implement a method 200 for prioritizing network traffic illustrated in FIG. 2.


As seen from FIG. 2, at 202 a bandwidth quantum BQ may be determined for a given time interval referred to herein as a time quantum T. Although the choice of time quantum T is somewhat arbitrary, certain considerations are worth bearing in mind when choosing the size of the time quantum T. For example, a smaller time quantum will be more sensitive to isolated bandwidth changes. A longer time quantum will smooth sudden changes, but will result in the system taking longer to adjust to sudden, large changes. The bandwidth quantum BQ is a total amount of data (e.g., in kilobits (Kb)) that may be transmitted during the time quantum T. The bandwidth quantum BQ may be determined from a known or estimated bandwidth available for communication with a given node, e.g., as obtained from the bandwidth module 104. The bandwidth quantum BQ may be determined from an estimated bandwidth, e.g., in kilobits per second (Kbps) and the duration of the time quantum T. By way of example, the bandwidth quantum BQ may be determined from a product of a bandwidth estimate and the duration of the time quantum T. As a specific numerical example, suppose that the duration of the time quantum T is 50 milliseconds and the bandwidth module 104 in Node A 110 determines that 500 Kbps of bandwidth are available for communication between Node A 110 and Node C 130 and that 200 Kbps of bandwidth are available for communication between Node A 110 and Node D 140. The bandwidth quantum BQ for communication between Node A 110 and Node C 130 may be approximately (500 Kbps)(50 ms)=25 kilobits (Kb). Similarly, the bandwidth quantum BQ for communication between Node A 110 and Node D 140 may be approximately (200 Kbps)(50 ms)=10 kilobits (Kb).


The scheduler 106 may be configured to implement the distribution of the bandwidth quantum BQ amongst the available channels C1 . . . CN. Referring again to FIG. 2, at 204 the bandwidth quantum BQ is distributed amongst the communication channels based on priorities P1 . . . PN correspondingly assigned to the channels The channels are assigned corresponding reserved portions of the bandwidth quantum BQ. The size Ri of the reserved portion assigned to a given channel Ci of the N channels is greater than zero. The size Ri may be determined based on the corresponding priority Pi assigned to the given channel Ci. For example, the size Ri of a given reserved portion may be determined by:








R
i

=

BQ
·


P
i




j







P
j





,





where








j







P
j








is a sum of the priorities P1 . . . PN for all of the channels C1 . . . CN.


The quantity







P
i




j



P
j







is sometimes referred to herein as the relative priority Preli for channel Ci. The scheduler 106 may distribute the bandwidth quantum BQ according to priorities assigned by one or more policy modules 108. By way of example, the policy modules 108 may include individual policy modules PM1 . . . PMN for each of the communication channels C1 . . . CN. The policy modules 108 may generate policy parameters 109 that the scheduler 106 uses to determine the distribution of the bandwidth quantum BQ amongst the channels C1 . . . C5. The parameters 109 may include priorities P1 . . . PN. The policy modules 108 may assign priorities P1 . . . PN based on the needs of the application 102. The priorities P1 . . . PN determine how the bandwidth quantum BQ is shared between the communication channels C1 . . . CN. Such “bandwidth distribution” may be implemented at some fixed frequency (e.g., 50 Hz in some implementations) by the scheduler 106. At regular intervals, the scheduler 106 may obtain a bandwidth estimation from the bandwidth module 104, determine the bandwidth quantum BQ and divide the bandwidth quantum BQ between the communication channels C1 . . . CN based on their priorities P1 . . . PN.


As a numerical example, assume the bandwidth module 104 estimates a bandwidth BW of 200 kbps for a certain connection. Further suppose that a software designer has configured the scheduler 106 to assign a priority P1=100 to an audio channel C1, a priority P2=20 to a video channel C2 and a priority P3=5 to a file transfer channel C3. If scheduler 106 is configured to trigger at 50 Hz, the time quantum T is approximately 20 milliseconds. By multiplying the estimated bandwidth BW by the time quantum T one obtains a bandwidth quantum BQ=200 Kbps×0.02 s=4 kb. From the priorities P1, P2 and P2 of 100, 20 and 5 one may calculate relative priorities of Prel1=100/125=0.8, Prel2=20/125=0.16 and Prel3=10/125=0.04. Thus, for this distribution cycle, audio is assigned a portion Rt=3.2 Kb, video is assigned a portion R2=0.64 Kb and file transfer is assigned a portion R3=0.16 Kb.


The priorities P1 . . . PN may be assigned to the communication channels C1 . . . CN based on the nature and type of data being transmitted over the channels. For example, in the context of A/V chat, nature of streaming audio data may dictate that an audio channel be assigned a relatively high priority than streaming video data. Specifically, streaming audio tends to be relatively tolerant to packet loss. Streaming audio also tends to use a relatively constant amount of bandwidth (e.g., about 12 Kbps) compared to either video or file transfer.


Streaming video data tends to use a relatively wide range of bandwidth, e.g., about 100-300 Kbps depending on compression, and is tolerant to packet loss. Streaming video tends to be less constant than audio. As a result, a streaming video channel may be assigned a lower priority than a streaming audio channel. File transfer data tends to be unpredictable and intermittent and uses as much bandwidth as is available. Unlike audio and video channels, a file transfer channel could theoretically use an infinite amount of bandwidth if it were available. File transfer data is intolerant to packet loss. File transfer data may be significantly less time insensitive than either video or audio data. Consequently, file transfer data channels may be assigned a significantly lower priority than streaming video channels.


Furthermore, priorities may be assigned based on other considerations, e.g., economic considerations. For example, the data channels C1 . . . CN may include one or more channels devoted to advertising data. The priorities assigned to such advertising channels may be based partly on rates that advertisers are willing to pay to have their advertisements distributed by the nodes. Specifically, within the scheduler 106, selected ones of the priority modules PM1 . . . PMN associated with advertising channels may be configured to assign a higher priority if a higher advertising rate has been paid.


In addition to priorities, each policy module PM1 . . . PMN may determine maximum values M1 . . . MN and minimum values m1 . . . mN for the portions of the bandwidth quantum BQ that are corresponding assigned to channels C1 . . . CN. Such maximum and minimum values may be regarded as a subset of the policy parameters 109. Generally, for a given channel Ci, m1<MN Maximum and minimum values may be used to tailor the distribution of the bandwidth quantum BQ at 204 to the needs of the application. The maximum and minimum values may have any suitable value. A given minimum value may be as low as zero. A given maximum may be infinite. In some cases, the scheduler 106 may initially assign a give channel Ci a portion that has a size Ri greater than its corresponding maximum value Mi. In such a case, the scheduler 106 may be configured, e.g., by suitable programming, to distribute the difference between Ri and Mi amongst the remaining channels. For example, if a channel C1 has a maximum bandwidth portion M1 of 5 Kb and its portion R1 is 8.5 Kb. The difference, 8.5 Kb−5 Kb=3.5 Kb, may be distributed amongst the remaining channels, e.g., on a pro-rata basis dependent on relative priority. For example, suppose there are only two other channels C2 and C3 with relative priorities P2rel=0.4 and P3rel=0.1. The distributor 106 may split up the remaining 3.5 Kb as follows. Channel C2 could be allocated an amount equal to







3.5






Kb
·

0.4

0.4
+
0.1




=

2.8





Kb






in addition to its regular portion R2 of the bandwidth quantum BQ. Similarly, channel C3 could be allocated







3.5






Kb
·

0.1

0.4
+
0.1




=

0.7





Kb






in addition to its regular portion R3 of the bandwidth quantum BQ.


If a given communication channel Ci has a minimum bandwidth quantum portion mi all channels of a lower priority that C, may be excluded from bandwidth distribution at 204 during the time quantum T until a minimum amount of data mi has been transmitted by the channel Ci. If more than one channel has a minimum, the bandwidth quantum is initially distributed to the highest priority channel with a minimum and any other channels having a higher priority. It is important to select the minimum values m1 . . . mN carefully in order to avoid bandwidth starvation. The minimum amount of data mi represents a minimum amount of data to be sent by the channel C, during the time quantum T. The minimum value may also be thought of as an expected minimum size of data to send. It is possible that the provided minimum value may be a best guess, and more or less data could actually be available to send. For example, a channel for audio data may always use close to 12 Kbps, so 12 Kbps may be set as the minimum value mi for that channel. However, depending on the compression used, the actual amount of data may be slightly below 12 kbps at times or slightly over 12 kbps.


Furthermore the policy modules PM1 . . . PMN may generate parameters associated with queuing policies QP1 . . . QPN to the channels C1 . . . CN. Each queuing policy determines what happens when multiple sends (e.g., data units such as packets or frames) are made on a single channel, but cannot all be sent immediately. A given queuing policy QPi may be matched by a software designer to the type of data being sent through the corresponding channel Ci. The nature of the queuing policy QPi may depend on the nature of the data being sent over the channel Ci. For example, real time streaming data, e.g., streaming audio or video, may have different requirements than non-streaming data. In the case of video channel C2, two or more video frames may be captured from a camera at different times. Due to the real-time nature of streaming video data, an older frame may be useless and could be discarded in favor of a newer frame. Thus the video channel C2 may use a queuing policy that selectively discards older images in favor of the newer images. A similar policy may be implemented for the audio channel C1. For the file transfer channel C3, by contrast, it may be more desirable to transmit every packet regardless of age. Consequently, the filer transfer channel C3 may have a different queuing policy attached it. By way of example, the file transfer channel may have a queuing policy that buffers all packets that cannot be sent during the time quantum T.


Each channel Ci may utilize the bandwidth portion Ri that was assigned to it during the time quantum T. Referring again to FIG. 2, at 206 ready data may be transmitted for each channel Ci that has any ready data to transmit. The amount of ready data transmitted at 206 for a given channel Ci is generally greater than zero and less than or equal to the size Ri of the reserved portion for the channel C1. As used herein the term “ready data” refers to data that is available for transmission. Ready data may be available for transmission by virtue of being generated by the application. In some cases, such ready data may be transmitted during the time quantum T as soon as it is generated. In addition, ready data may be available by virtue of being stored in one or more buffers 111. Generally, the buffers 111 may include N communication channel buffers B1 . . . BN correspondingly associated with communication channels C1 . . . CN. In some embodiments, the router 150 may include buffers 111. As used herein, the term buffer generally refers to a region of computer memory used to temporarily hold data while it is being moved from one place to another. Buffers are used, e.g., when there is a difference between the rate at which data is received and the rate at which it can be processed, or in the case that these rates are variable. Typically, the data is stored in a buffer as it is retrieved from a source of the data (e.g., an input device or program generating the data) or just before it is sent to an output device (e.g., a network interface). In addition, a buffer may be used when moving data between processes within a computer. By way of example, each communication channel buffer may be implemented in hardware, software or some combination of both hardware and software. By way of example, and without limitation, the communication buffers B1 . . . BN, may be implemented in software.


If a given channel Ci does not use its entire reserved portion Ri during the time quantum T, any unused part of the reserved portion Ri may be redistributed to the other channels. By way of example, if not all of the ready data has been sent at 208 the scheduler 106 may determine at 210 whether there is any unused reserved portion Ri to redistribute. If there is both unsent data and one or more unused reserved portions of the bandwidth quantum, then at 212 any unused part of any reserved portion Ri may be redistributed amongst any of the N communication channels Ci . . . CN having un-transmitted ready data. Such redistribution may take place on a pro-rata basis based on relative priority. This sequence may repeat until all ready data has been sent or there is no unused portion of the bandwidth quantum BQ. Otherwise, the cycle may begin again, e.g., at normal distribution at 204 or (optionally) determination of a new bandwidth quantum BQ at 202. The cycle may then return repeat continually.


It is noted that the beginning and end of the cycle are somewhat arbitrary. For example, although from FIG. 2 it would appear that redistribution 212 takes place at the end of the time quantum T, redistribution may alternatively take place at the beginning of the time quantum T. Generally, redistribution may take place at some time after normal distribution and transmission in one time quantum and before normal distribution in the next time quantum.



FIG. 3 is a block diagram illustrating the components of a node 300 suitable for implementing network traffic prioritization according to an embodiment of the present invention. By way of example, and without loss of generality, the node 300 may be implemented as a computer system, such as a personal computer, video game console, personal digital assistant, or other digital device, suitable for practicing an embodiment of the invention. The node 300 may include a central processing unit (CPU) 301 configured to run software applications and optionally an operating system. The CPU 301 may include one or more processing cores. By way of example and without limitation, the CPU 301 may be a parallel processor module, such as a Cell Processor. An example of a Cell Processor architecture is described in detail, e.g., in Cell Broadband Engine Architecture, copyright International Business Machines Corporation, Sony Computer Entertainment Incorporated, Toshiba Corporation Aug. 8, 2005 a copy of which may be downloaded at http://cell.scei.co.jp/, the entire contents of which are incorporated herein by reference.


In the node 300 a memory 302 may be coupled to the CPU 301. The memory 302 may store applications and data for use by the CPU 301. The memory 302 may be in the form of an integrated circuit, e.g., RAM, DRAM, ROM, and the like). A computer program 303 may be stored in the memory 302 in the form of instructions that can be executed on the processor 301. The instructions of the program 303 may be configured to implement, amongst other things, one or more applications, such as the application 102 described above with respect to FIG. 1. By way of example, and without loss of generality, the program 303 may include an application, such as an A/V chat application involving two or more channels of communication. Such channels may include, but are not limited to an audio channel C1, a video channel C2 and a file transfer channel C3. The memory 302 may also contain instructions, that, when executed by the CPU 301 implement a bandwidth module 304 having features in common with the bandwidth module 104 described above. The memory 302 may also contain instructions configured to implement a scheduler 306 having features in common with the scheduler 106 described above. The memory 302 may also contain instructions configured to implement one or more policy modules 308 having features in common with the policy modules 108 described above. By way of example and without loss of generality, the policy modules 308 may include an audio policy module PM1 for the audio channel C1, a video policy module PM2 for the video channel C2, and a file transfer policy PM3 for the file transfer channel C3.


The memory 302 may contain data that is generated by or usable by the program 303, bandwidth module 304, scheduler 306, and policy modules 308. Specifically, such data may include, but is not limited to policy module parameters 309, a bandwidth quantum BQ and a time quantum T. The policy module parameters 309 may include priorities P1, P2, and P3 respectively associated with the audio channel C1, video channel C2, and file transfer channel C3. The policy module parameters 309 may further include minimum values m1, m2 and m3 respectively associated with the audio channel Ci, video channel C2 and file transfer channel C3 as well as maximum values Mi, M2 and M3 respectively associated with the audio channel C1, video channel C2 and file transfer channel C3.


In addition, the memory 302 may be configured to include one or more buffers 310 for data generated by the program 303 for transmission via the communication channels. By way of example and without loss of generality, the buffers 310 may include an audio buffer B1 configured to buffer audio channel data 311, a video buffer B2 configured to buffer video channel data 312 and a file transfer buffer B3 configured to buffer file transfer channel data 313. The scheduler 306 may be configured, e.g., by appropriate programming, to implement buffering of data in accordance with one or more queuing policies 314. By way of example, and without loss of generality, the queuing policies 314 may include an audio data queuing policy QP1, a video data queuing policy QP2 and a file transfer queuing policy QP3. The queuing policies 314 may be configured, to determine what happens when multiple sends are made on a single channel, but cannot all be sent immediately, e.g., as described above.


The node 300 may further include a storage device 315 that provides non-volatile storage for applications and data. By way of example, the storage device 315 may be a fixed disk drive, removable disk drive, flash memory device, tape drive, CD-ROM, DVD-ROM, Blu-ray, HD-DVD, UMD, or other optical storage devices. The node 300 may also include well-known support functions 320 commonly used in computing systems. Such support functions may include such features as input/output (I/O) elements 321, power supplies (P/S) 322, a clock (CLK) 323 and cache 324.


One or more user input devices 325 may be used to communicate user inputs from one or more users to the node 300. By way of example, one or more of the user input devices 325 may be coupled to the node 300 via the I/O elements 321. Examples of suitable input devices 325 include keyboards, mice, joysticks, touch pads, touch screens, light pens, still or video cameras, and/or microphones. In the particular case of A/V chat, it is desirable for the user interface devices 325 to include both a camera and a microphone. A network interface 326 allows the node 300 to communicate with other computer systems via an electronic communications network 327. The network interface 326 may include wired or wireless communication over local area networks and wide area networks such as the Internet. The node 300 may send and receive data and/or requests for files via one or more message packets 328 over the network 327.


The node 300 may further comprise a graphics subsystem 330, which may include a graphics processing unit (GPU) 335 and graphics memory 340. The graphics memory 340 may include a display memory (e.g., a frame buffer) used for storing pixel data for each pixel of an output image. The graphics memory 340 may be integrated in the same device as the GPU 335, connected as a separate device with GPU 335, and/or implemented within the memory 302. Pixel data may be provided to the graphics memory 340 directly from the CPU 301. Alternatively, the CPU 301 may provide the GPU 335 with data and/or instructions defining the desired output images, from which the GPU 335 may generate the pixel data of one or more output images. The data and/or instructions defining the desired output images may be stored in buffers 310 and/or graphics memory 340. In an embodiment, the GPU 335 may be configured (e.g., by suitable programming or hardware configuration) with 3D rendering capabilities for generating pixel data for output images from instructions and data defining the geometry, lighting, shading, texturing, motion, and/or camera parameters for a scene. The GPU 335 may further include one or more programmable execution units capable of executing shader programs.


The graphics subsystem 330 may periodically output pixel data for an image from graphics memory 340 to be displayed on a display device 350. The display device 350 may be any device capable of displaying visual information in response to a signal from the computer system 300, including CRT, LCD, plasma, and OLED displays. The node 300 may provide the display device 350 with an analog or digital signal. By way of example, the display 350 may include a cathode ray tube (CRT) or flat panel screen that displays text, numerals, graphical symbols, or images. In addition, the node 300 may include one or more audio speakers 352 that produce audible or otherwise detectable sounds. To facilitate generation of such sounds, the node 300 may further include an audio processor 355 adapted to generate analog or digital audio output from instructions and/or data provided by the CPU 301, memory 302, and/or storage 315. In the particular case of A/V chat, it is desirable for the node 300 to include a graphical display device 350 and an audio speaker 352.


The components of the node 300, including the CPU 301, memory 302, support functions 320, data storage 315, user input devices 325, network interface 326, graphics subsystem 330 speaker 352 and audio processor 355 may be operably connected to each other via one or more data buses 360. These components may be implemented in hardware, software, firmware or some combination of two or more of these.


By way of example, and without loss of generality, software designers may implement embodiments of the present invention in software applications by creating a plurality of communication channels, and assigning a priority and a queuing policy to each one. Data could then be sent through these configured communication channels and the scheduler 306, policy modules 308, and queuing policies 314 may control the actual transmission of the data over the network 327.



FIG. 4 illustrates an example of operation of the operation of a node of the type shown in FIG. 3. In the example illustrated in FIG. 4 it is assumed, for the purpose of example, that the audio buffer B1 contains no buffered audio data 311, the video buffer B2 contains 4 Kb of buffered video data 312 and the file transfer buffer B3 contains 10 Kb of buffered data 313 for file transfer. It is also assumed, for the sake of simplicity, that there is no unreserved bandwidth quantum at the beginning of a time quantum T1. In this example, the audio channel C1 has been assigned a priority value P1=85, the video channel C2 has been assigned a priority value P2=10 and the file transfer channel C3 has been assigned a priority value P3=5. It is assumed in FIG. 4, that the size of the available bandwidth for one or more 20 millisecond time quanta has been determined by the bandwidth module 304 to be 500 Kbps. The scheduler 306 therefore determines that the corresponding bandwidth quantum is 10 Kb. Using the formula described above, the scheduler 306 may calculate relative priorities Prel1=0.85, Prel2=0.10 and Prel3=0.05 for the audio channel C1, video channel C2 and file transfer channel C3, respectively. In accordance with the formula Ri=(Preli)(BQ), the scheduler 306 may then reserve portions of the bandwidth quantum BQ of R2=1 Kb for the video channel C2, R1=8.5 Kb for the audio channel C1 and R3=500 b for the file transfer channel C2 as indicated at 402, 404 and 406 respectively. The scheduler 306 may then cause the node 300 to send 1 Kb of the buffered video data 312 and buffer the remaining 3 Kb as indicated at 408. Similarly, the scheduler 306 may then cause the node 300 to send 500 b of the buffered file transfer data 313 and buffer the remaining 9.5 Kb as indicated at 410. If no audio data is available, the scheduler 306 may continue to reserve 8.5 Kb for the audio channel C3 as indicated at 412. At some later point during the time quantum T1 the program 303 may generate 1.5 Kb of audio data for the audio channel C1 as indicated at 414. This data may be sent over the audio channel C1 while reserving the remaining (8.5 Kb−1.5 Kb)=7 Kb as indicated at 416. At some further point in the time quantum T1 an additional 1 Kb of audio data may be generated by the program 303 as indicated at 418. This data may be sent over the audio channel C1 while reserving (7 Kb−1 Kb)=7 Kb as indicated at 420.


At some point the time quantum T1 ends and a new time quantum T2 begins. In this example, 6 Kb remains from the reserved portions R1, R2, R3 of the bandwidth quantum BQ. This remaining portion may be returned for redistribution at the beginning of the new time quantum T2 as indicated at 422. The 6 Kb of returned reserved portion provides a redistribution quantum RD that may be distributed on a pro-rata basis amongst any of the channels C1, C2 C3 having un-transmitted ready data, e.g., buffered data.


In this example, the video channel C2 has 3 Kb of buffered data and the file transfer channel C3 has 9.5 Kb of buffered data at the beginning of the new time quantum T2. Since the audio channel C1 has no buffered data at this point, the 6 Kb of unused reserved portion is distributed between the video channel C2 and the file transfer channel C3. In this example, the priority P2 for the video channel C2 is twice as large as the priority for the file transfer channel C3. The scheduler 306 may therefore reserve 4 Kb of the redistribution quantum RD for the video channel C2 as indicated at 424 and 2 Kb of the redistribution quantum RD for the file transfer channel C3 as indicated at 426. The scheduler 306 may then cause the node 300 to send 2 Kb of buffered file transfer data 313 as indicated at 428. The remaining 3 Kb of buffered video data 312 may be sent and the leftover 1 Kb reserved as indicated at 430. The leftover 1 Kb may be returned to the redistribution quantum RD as indicated at 432. Since, at this point, only the file transfer channel C3 has buffered data, the entire 1 Kb of the redistribution quantum RD may be reserved for the file transfer channel C3 as indicated at 434. The scheduler 306 may then cause the node 300 to send 1 Kb of buffered file transfer data 313 while buffering the remaining 6.5 Kb as indicated at 436.


At this point in this example, the entire bandwidth quantum BQ has been used and normal distribution may take place for the new time quantum T2. For example, if the sizes of the time quantum and available bandwidth are the same as before, 1 Kb may be reserved for the video channel C2 at 438, 8.5 Kb may be reserved for the audio channel C1 at 440 and 500 b may be reserved for the file transfer channel C3 at 442. The reserved portions R1, R2, and R3 may be used during the remainder of the new time quantum T2 in a manner similar to that described above with respect to the previous time quantum T2. Any unused reserved portions may be redistributed at the beginning of a subsequent time quantum.


Although the discussion of FIG. 4 is directed to an example involving three specific communication channels, those of skill in the art will recognize that the concept illustrated in that example may be applied to two channels or more than three channels in any similarly configured node.


While the above is a complete description of the preferred embodiment of the present invention, it is possible to use various alternatives, modifications, and equivalents. Therefore, the scope of the present invention should be determined not with reference to the above description but should, instead, be determined with reference to the appended claims, along with their full scope of equivalents. Any feature described herein, whether preferred or not, may be combined with any other feature described herein, whether preferred or not. In the claims that follow, the indefinite article “A”, or “An” refers to a quantity of one or more of the item following the article, except where expressly stated otherwise. In the claims that follow, the expressions first and second are used to distinguish between different elements and do not imply any particular order or sequence. The appended claims are not to be interpreted as including means-plus-function limitations, unless such a limitation is explicitly recited in a given claim using the phrase “means for.”

Claims
  • 1. In a node configured to communicate with one or more other nodes over a network, a method for prioritizing network traffic among two or more distinct channels of communication within a single application, the method comprising: a) distributing a bandwidth quantum BQ for a time quantum T amongst N≧2 communication channels C1 . . . CN, based on priorities P1 . . . PN correspondingly assigned to channels C1 . . . CN, wherein each channel is assigned a reserved portion of the bandwidth quantum BQ, wherein a size Ri of the reserved portion assigned to a given channel Ci of the N channels C1 . . . CN, is greater than zero and wherein Ri is determined based on the corresponding priority Pi assigned to the given channel Ci; andb) transmitting an amount of ready data, if any, for each channel Ci that is greater than zero and less than or equal to the size Ri of the reserved portion for that channel.
  • 2. The method of claim 1 wherein the size Ri of a given reserved portion is determined by:
  • 3. The method of claim 1, further comprising determining the bandwidth quantum BQ prior to a).
  • 4. The method of claim 1 wherein the two or more communications channels include an audio channel, a video channel, and a file transfer channel.
  • 5. The method of claim 1, further comprising, if the size Ri assigned to a given channel Ci is greater than a predetermined maximum size Mi for the channel Ci, allocating a difference Ri−Mi to one or more other channels.
  • 6. The method of claim 1, further comprising, if the size Ri for a given channel Ci is less than a predetermined minimum size mi for the channel Ci excluding from a) any channels with lower priority than Ci until the minimum size mi has been met.
  • 7. The method of claim 6 wherein if more than one of the channels has a predetermined minimum size, a) includes initially distributing the bandwidth quantum BQ to a highest priority channel with a minimum size and any other channels having a higher priority.
  • 8. The method of claim 1 wherein a) comprises determining each priority Pi according to a predetermined policy module for the corresponding channel Ci.
  • 9. The method of claim 8, wherein a) further comprises assigning a maximum and/or minimum value to the size Ri according to the predetermined policy module.
  • 10. The method of claim 1, wherein a) further comprises assigning a maximum and/or minimum value to the size Ri according to a predetermined policy module for the corresponding channel Ci.
  • 11. The method of claim 1 wherein the single application is an audio-video chat application.
  • 12. The method of claim 1, wherein b) further comprises implementing a queuing policy for one or more of the communication channels C1 . . . CN.
  • 13. The method of claim 12 wherein implementing the queuing policy includes buffering one or more data units by a given communication channel Ci that cannot be transmitted during the time quantum T.
  • 14. The method of claim 13 wherein implementing the queuing policy includes discarding an older data unit in favor of a new data unit.
  • 15. The method of claim 14 wherein the older data unit and new data unit comprise audio frames or video frames.
  • 16. The method of claim 1, wherein the distinct channels of communication are used within a single peer-to-peer application.
  • 17. A node configured to communicate with one or more other nodes over a network, the node comprising: a processor; anda memory coupled to the processor, the memory having therein a set of instructions executable by the processor, the instructions being configured to implement a method for prioritizing network traffic among two or more distinct channels of communication within a single application, the method comprising:a) distributing a bandwidth quantum BQ for a time quantum T amongst N≧2 communication channels C1 . . . CN, based on priorities P1 . . . PN correspondingly assigned to channels C1 . . . CN, wherein each channel is assigned a reserved portion of the bandwidth quantum BQ, wherein a size Ri of the reserved portion assigned to a given channel Ci of the N channels C1 . . . CN, is greater than zero and wherein Ri is determined based on the corresponding priority Pi assigned to the given channel Ci; andb) transmitting an amount of ready data, if any, for each channel Ci that is greater than zero and less than or equal to the size Ri of the reserved portion for that channel.
  • 18. The node of claim 17, further comprising one or more instructions in memory configured to implement one or more policy modules configured to determine the priorities P1 . . . PN for the corresponding channels C1 . . . CN.
  • 19. The node of claim 18, wherein one or more of the policy modules is configured to assign a maximum and/or minimum size to an amount of data that can be transmitted during the time quantum T on one or more of the channels C1 . . . CN.
  • 20. The node of claim 18, wherein one or more of the policy modules is configured to assign a maximum and size Mi to an amount of data that can be transmitted during the time quantum T over a given channel Ci wherein the set of instructions is configured to allocate a difference Ri−Mi from a given channel Ci to one or more other channels if the size Ri assigned to the given channel Ci is greater than the maximum size Mi.
  • 21. The node of claim 18, wherein one or more of the policy modules is configured to assign a minimum size mi to an amount of data that can be transmitted during the time quantum T over a given channel Ci, wherein the set of instructions is configured to exclude from a) any channels with lower priority than a channel Ci until an amount of data greater than or equal to the minimum size mi has been sent over the channel Ci.
  • 22. The node of claim 21, wherein the set of instructions is configured such that a) includes initially distributing the bandwidth quantum BQ to a highest priority channel with a minimum size and any other channels having a higher priority if more than one of the channels is assigned a minimum size to an amount of data that can be transmitted during the time quantum T.
  • 23. The node of claim 17, wherein the set of instructions is configured such that b) further comprises implementing a queuing policy for one or more of the communication channels C1 . . . CN.
  • 24. The node of claim 23, wherein the set of instructions is configured such that implementing the queuing policy includes buffering one or more data units by a given communication channel Ci, that cannot be transmitted during the time quantum T.
  • 25. The node of claim 23, wherein the set of instructions is configured such that implementing the queuing policy includes discarding an older data unit in favor of a newer data unit.
  • 26. The node of claim 25 wherein the older data unit and newer data unit comprise one or more audio frames or video frames.
  • 27. The node of claim 17 wherein the single application is an audio-video chat application.
  • 28. The node of claim 17 wherein the two or more communications channels include an audio channel, a video channel, and a file transfer channel.
  • 29. The node of claim 17, further comprising a bandwidth module configured to determine the bandwidth quantum BQ.
  • 30. The node of claim 17, wherein the distinct channels of communication are used within a single peer-to-peer application.
  • 31. A tangible, non-transitory, computer-accessible storage medium comprising program instructions, wherein the program instructions are computer-executable on a node to implement a method for prioritizing network traffic among two or more distinct channels of communication within a single application, the method comprising: a) distributing a bandwidth quantum BQ for a time quantum T amongst N≧2 communication channels C1 . . . CN, based on priorities P1 . . . PN correspondingly assigned to channels C1 . . . CN, wherein each channel is assigned a reserved portion of the bandwidth quantum BQ, wherein a size Ri of the reserved portion assigned to a given channel Ci of the N channels C1 . . . CN, is greater than zero and wherein Ri is determined based on the corresponding priority Pi assigned to the given channel Ci; andb) transmitting an amount of ready data, if any, for each channel Ci that is greater than zero and less than or equal to the size Ri of the reserved portion for that channel.
CLAIM OF PRIORITY

This application is a continuation of commonly-assigned, U.S. patent application Ser. No. 12/267,233 to James E. Marr, Yutaka Takeda, Attila Vass, Payton White and Stephen C. Detwiler entitled “NETWORK TRAFFIC PRIORITIZATION”filed Nov. 7, 2008 now U.S. Pat. No. 7,856,501, the entire disclosures of which are incorporated herein by reference. This application claims the priority benefit of commonly-assigned, co-pending U.S. Provisional Patent application No. 60/992,295 to James E. Marr, Yutaka Takeda, Attila Vass, Payton White and Stephen C. Detwiler entitled “NETWORK TRAFFIC PRIORITIZATION” filed Dec. 4, 2007, the entire disclosures of which are incorporated herein by reference. This application claims the priority benefit of commonly-assigned, U.S. Provisional Patent application No. 60/992,282 to Yutaka Takeda, James E. Marr, Stephen C. Detwiler, Attila Vass, and Payton White entitled “NETWORK BANDWIDTH DETECTION AND DISTRIBUTION”, filed Dec. 4, 2007, the entire disclosures of which are incorporated herein by reference. This application claims the priority benefit of co-pending U.S. patent application Ser. No. 12/267,269 to James E. Marr, Yutaka Takeda, Attila Vass, Payton White and Stephen C. Detwiler entitled “NETWORK BANDWIDTH DETECTION, DISTRIBUTION AND TRAFFIC PRIORITIZATION” filed Nov. 7, 2008, the entire disclosures of which are incorporated herein by reference. This application claims the priority benefit of co-pending U.S. patent application Ser. No. 12/267,254 to Yutaka Takeda, James E. Marr, Stephen C. Detwiler, Attila Vass, and Payton White entitled “NETWORK BANDWIDTH DETECTION AND DISTRIBUTION” filed Nov. 7, 2008, the entire disclosures of which are incorporated herein by reference.

US Referenced Citations (212)
Number Name Date Kind
4764928 Akerberg Aug 1988 A
4787051 Olson Nov 1988 A
4843568 Krueger Jun 1989 A
5128671 Thomas, Jr. Jul 1992 A
5313454 Bustini et al. May 1994 A
5528265 Harrison Jun 1996 A
5544325 Denny et al. Aug 1996 A
5596720 Hamada et al. Jan 1997 A
5630184 Roper et al. May 1997 A
5673252 Johnson et al. Sep 1997 A
5701427 Lathrop Dec 1997 A
5768382 Schneier et al. Jun 1998 A
5768531 Lin Jun 1998 A
5809106 Kreitzer et al. Sep 1998 A
5812531 Cheung et al. Sep 1998 A
5835726 Shwed et al. Nov 1998 A
5856972 Riley et al. Jan 1999 A
5898679 Brederveld et al. Apr 1999 A
6012096 Link et al. Jan 2000 A
6152824 Rothschild et al. Nov 2000 A
6157368 Faeger Dec 2000 A
6212633 Levy et al. Apr 2001 B1
6247061 Douceur et al. Jun 2001 B1
6327630 Carroll et al. Dec 2001 B1
6349210 Li Feb 2002 B1
6375572 Masuyama Apr 2002 B1
6405104 Dougherty Jun 2002 B1
6421347 Borgstahl et al. Jul 2002 B1
6480748 Gerszberg et al. Nov 2002 B1
6487583 Harvey et al. Nov 2002 B1
6487600 Lynch Nov 2002 B1
6549786 Cheung et al. Apr 2003 B2
6553515 Gross et al. Apr 2003 B1
6590865 Ibaraki et al. Jul 2003 B1
6616531 Mullins Sep 2003 B1
6640241 Ozzie et al. Oct 2003 B1
6641241 Ozzie et al. Oct 2003 B2
6641481 Mai et al. Nov 2003 B1
6661799 Molitor Dec 2003 B1
6667972 Foltan et al. Dec 2003 B1
6668283 Sitaraman et al. Dec 2003 B1
6701344 Holt et al. Mar 2004 B1
6704574 Lin Mar 2004 B2
6712697 Acres Mar 2004 B2
6760775 Anerousis et al. Jul 2004 B1
6772219 Shobatake Aug 2004 B1
6779017 Lamberton et al. Aug 2004 B1
6799255 Blumenau et al. Sep 2004 B1
6807575 Emaru et al. Oct 2004 B1
6816703 Wood et al. Nov 2004 B1
6829634 Holt et al. Dec 2004 B1
6848997 Hashimoto et al. Feb 2005 B1
6891801 Herzog May 2005 B1
6899628 Leen et al. May 2005 B2
6920501 Chu et al. Jul 2005 B2
6934745 Krautkremer Aug 2005 B2
6978294 Adams et al. Dec 2005 B1
7016942 Odom Mar 2006 B1
7017138 Zirojevic et al. Mar 2006 B2
7035911 Lowery et al. Apr 2006 B2
7043641 Martinek et al. May 2006 B1
7065579 Traversat et al. Jun 2006 B2
7082316 Eiden et al. Jul 2006 B2
7096006 Lai et al. Aug 2006 B2
7120429 Minear et al. Oct 2006 B2
7123608 Scott et al. Oct 2006 B1
7127613 Pabla et al. Oct 2006 B2
7130921 Goodman et al. Oct 2006 B2
7134961 Hora Nov 2006 B2
7155515 Brown et al. Dec 2006 B1
7168089 Nguyen et al. Jan 2007 B2
7174382 Ramanathan et al. Feb 2007 B2
7177950 Narayan et al. Feb 2007 B2
7177951 Dykeman et al. Feb 2007 B1
7194654 Wray et al. Mar 2007 B2
7197565 Abdelaziz et al. Mar 2007 B2
7203841 Jackson et al. Apr 2007 B2
7240093 Danieli et al. Jul 2007 B1
7263070 Delker et al. Aug 2007 B1
7272636 Pabla Sep 2007 B2
7321928 Feltin et al. Jan 2008 B2
7340500 Traversat et al. Mar 2008 B2
7346015 Shipman Mar 2008 B2
7392375 Bartram et al. Jun 2008 B2
7398388 Xu et al. Jul 2008 B2
7407434 Thomas et al. Aug 2008 B2
7426185 Musacchio et al. Sep 2008 B1
7429215 Rozkin et al. Sep 2008 B2
7451490 Pirich et al. Nov 2008 B2
7457279 Scott et al. Nov 2008 B1
7529193 Zimmerman et al. May 2009 B2
7533172 Traversat et al. May 2009 B2
7803052 Multerer et al. Sep 2010 B2
7856501 Marr et al. Dec 2010 B2
7908393 Marr et al. Mar 2011 B2
20010005368 Rune Jun 2001 A1
20010017856 Asokan et al. Aug 2001 A1
20010021188 Fujimori et al. Sep 2001 A1
20010044339 Cordero et al. Nov 2001 A1
20010046213 Sakoda Nov 2001 A1
20020002074 White et al. Jan 2002 A1
20020006114 Bjelland et al. Jan 2002 A1
20020013838 Kushida et al. Jan 2002 A1
20020035604 Cohen et al. Mar 2002 A1
20020049841 Johnson et al. Apr 2002 A1
20020055989 Stringer-Calvert et al. May 2002 A1
20020075844 Hagen Jun 2002 A1
20020085097 Colmenarez et al. Jul 2002 A1
20020097732 Worster et al. Jul 2002 A1
20020107786 Lehmann-Haupt et al. Aug 2002 A1
20020107935 Lowery et al. Aug 2002 A1
20020119821 Sen et al. Aug 2002 A1
20020138471 Dutta et al. Sep 2002 A1
20020143855 Traversat et al. Oct 2002 A1
20020147810 Traversat et al. Oct 2002 A1
20020161821 Narayan et al. Oct 2002 A1
20020183004 Fulton et al. Dec 2002 A1
20020184310 Traversat et al. Dec 2002 A1
20020184311 Traversat et al. Dec 2002 A1
20030027634 Matthews, III Feb 2003 A1
20030028585 Yeager et al. Feb 2003 A1
20030045359 Leen et al. Mar 2003 A1
20030046292 Subramanian et al. Mar 2003 A1
20030055892 Huitema et al. Mar 2003 A1
20030079003 Burr Apr 2003 A1
20030084282 Taruguchi May 2003 A1
20030097408 Kageyama et al. May 2003 A1
20030104829 Alzoubi et al. Jun 2003 A1
20030115258 Baumeister et al. Jun 2003 A1
20030126229 Kantor et al. Jul 2003 A1
20030126245 Feltin et al. Jul 2003 A1
20030152034 Zhang et al. Aug 2003 A1
20030158961 Nomura et al. Aug 2003 A1
20030162556 Libes Aug 2003 A1
20030177187 Levine et al. Sep 2003 A1
20030182421 Faybishenko et al. Sep 2003 A1
20030182428 Li et al. Sep 2003 A1
20030191828 Ramanathan et al. Oct 2003 A1
20030217096 McKelvie et al. Nov 2003 A1
20030217135 Chatani et al. Nov 2003 A1
20030223430 Lodha Dec 2003 A1
20030227939 Yukie et al. Dec 2003 A1
20030229779 Morais et al. Dec 2003 A1
20030229789 Morais et al. Dec 2003 A1
20030233281 Takeuchi et al. Dec 2003 A1
20040007618 Oram et al. Jan 2004 A1
20040015548 Lee Jan 2004 A1
20040018839 Andric et al. Jan 2004 A1
20040024879 Dingman et al. Feb 2004 A1
20040063497 Gould Apr 2004 A1
20040085947 Ekberg et al. May 2004 A1
20040087369 Tanaka May 2004 A1
20040088369 Yeager et al. May 2004 A1
20040103179 Damm et al. May 2004 A1
20040110563 Tanaka Jun 2004 A1
20040133631 Hagen et al. Jul 2004 A1
20040139228 Takeda et al. Jul 2004 A1
20040162871 Pabla et al. Aug 2004 A1
20040181463 Goldthwaite et al. Sep 2004 A1
20040207880 Thakur Oct 2004 A1
20040212589 Hall et al. Oct 2004 A1
20040236863 Shen et al. Nov 2004 A1
20040236945 Risan et al. Nov 2004 A1
20040243665 Markki et al. Dec 2004 A1
20040254977 Zhang Dec 2004 A1
20040267876 Kakivaya et al. Dec 2004 A1
20050007964 Falco et al. Jan 2005 A1
20050015626 Chasin Jan 2005 A1
20050020354 Nguyen et al. Jan 2005 A1
20050026698 Pirich et al. Feb 2005 A1
20050063409 Oommen Mar 2005 A1
20050064939 McSheffrey et al. Mar 2005 A1
20050065632 Douglis et al. Mar 2005 A1
20050080858 Pessach Apr 2005 A1
20050086287 Datta Apr 2005 A1
20050086288 Datta et al. Apr 2005 A1
20050086329 Datta et al. Apr 2005 A1
20050086350 Mai Apr 2005 A1
20050086369 Mai et al. Apr 2005 A1
20050105526 Stiemerling et al. May 2005 A1
20050141522 Kadar et al. Jun 2005 A1
20050221858 Hoddie Oct 2005 A1
20050250487 Miwa Nov 2005 A1
20050251577 Guo et al. Nov 2005 A1
20050259637 Chu et al. Nov 2005 A1
20050262411 Vertes Nov 2005 A1
20060063587 Manzo Mar 2006 A1
20060067290 Miwa Mar 2006 A1
20060068702 Miwa Mar 2006 A1
20060075127 Juncker et al. Apr 2006 A1
20060084504 Chan et al. Apr 2006 A1
20060111979 Chu May 2006 A1
20060209822 Hamamoto Sep 2006 A1
20060288103 Gobara et al. Dec 2006 A1
20070058792 Chaudhari et al. Mar 2007 A1
20070061460 Khan et al. Mar 2007 A1
20070077981 Hungate et al. Apr 2007 A1
20070081459 Segel et al. Apr 2007 A1
20070198418 MacDonald et al. Aug 2007 A1
20070213124 Walker et al. Sep 2007 A1
20070217436 Markley et al. Sep 2007 A1
20070237153 Slaughter et al. Oct 2007 A1
20080298376 Takeda et al. Dec 2008 A1
20090013175 Elliott Jan 2009 A1
20090077245 Smelyansky et al. Mar 2009 A1
20090094370 Jacob et al. Apr 2009 A1
20090111532 Salokannel et al. Apr 2009 A1
20090138610 Gobara et al. May 2009 A1
20090144423 Marr Jun 2009 A1
20090144424 Takeda Jun 2009 A1
20090144425 Marr et al. Jun 2009 A1
20090240821 Juncker et al. Sep 2009 A1
Foreign Referenced Citations (18)
Number Date Country
0 913 965 May 1999 EP
1 107 508 Jun 2001 EP
1 374 959 May 2003 EP
2829648 Mar 2003 FR
2001 53901 Feb 2001 JP
2002 10321 Jan 2002 JP
2004 135778 May 2004 JP
2004 136009 May 2004 JP
2004 141225 May 2004 JP
2005 319047 Nov 2005 JP
2005 323116 Nov 2005 JP
2005 323117 Nov 2005 JP
0211366 Feb 2002 WO
03069495 Aug 2003 WO
2004038541 May 2004 WO
WO2004063843 Jul 2004 WO
WO2005088466 Sep 2005 WO
WO 2009073312 Jun 2009 WO
Related Publications (1)
Number Date Country
20110099278 A1 Apr 2011 US
Provisional Applications (2)
Number Date Country
60992295 Dec 2007 US
60992282 Dec 2007 US
Continuations (1)
Number Date Country
Parent 12267233 Nov 2008 US
Child 12975183 US