Claims
- 1. A video-conferencing system, comprising:
a decoder; an encoder connected to the decoder by a network; a packet loss detection mechanism co-located with the decoder, the packet loss detection mechanism configured to detect a lost packet and to transmit one of a packet loss notification message and a lost packet identification message; and one of
a sustained packet loss detection mechanism co-located with one of the encoder and the packet loss detection mechanism, the sustained packet loss detection mechanism configured to receive the packet loss notification message and to inform the encoder that the packet loss notification message indicates a presence of one of a first and a second predetermined packet loss scenario, and an error concealment device co-located with the decoder and connected to the packet loss mechanism, the error concealment device configured to replace motion vector data lost from a current frame with motion vector data from a previous frame in response to the lost packet identification message.
- 2. The video-conferencing system of claim 1, wherein
the encoder is configured to insert an INTER or INTRA block according to a first and a second insertion method if the packet loss notification message indicates the presence of the first and the second predetermined packet loss scenarios, respectively.
- 3. The video-conferencing system of claim 1, wherein the first predetermined packet loss scenario comprises:
a periodicity in a last N packet loss notifications that exceeds a predetermined packet loss periodicity value.
- 4. The video-conferencing system of claim 3, wherein the first predetermined packet loss scenario comprises:
at least K of N×N ratios which are less than a predetermined threshold t, where T0, T1, . . . and TN are a set of time instances corresponding to a set of N most recent packet loss notifications; di is a plurality of time differences d0=T1−T0, d1=T2−T1, . . . dN−1=TN−TN−1 between packet loss notifications; and the N×N ratios equal ri,j=|di|/|dj| corresponding to every possible combination of two time differences.
- 5. The video-conferencing system of claim 1, wherein
the encoder is configured to insert a plurality of INTRA blocks separated by a predetermined distance corresponding to a packet loss occurrence rate.
- 6. The video-conferencing system of claim 5, wherein
the predetermined distance is configured to vary such that a higher packet loss occurrence rate results in a closer spacing of inserted INTRA blocks.
- 7. The video-conferencing system of claim 5, wherein
the encoder is configured to update an entire picture using INTRA blocks during a predetermined number of frames.
- 8. The video-conferencing system of claim 7, wherein
the encoder is configured to prevent a motion vector from pointing into an area of the entire picture that has not yet been updated by an INTRA block.
- 9. The video-conferencing system of claim 1, wherein
the encoder is configured to take anticipated packet losses into account when making a decision between inserting an INTRA block and inserting an INTER block.
- 10. The video-conferencing system of claim 1, wherein
the encoder is further configured to determine one of a reliability of a reference block and a change in a reference block mean value.
- 11. The video-conferencing system of claim 10, wherein
the encoder is configured to determine a reliability of a reference block on the basis of a period since the reference block was last updated as INTRA.
- 12. The video-conferencing system of claim 11, wherein
the encoder is configured to determine a reliability Rn of a reference block in a frame n is determined as follows: Rn=1−p for INTRA blocks; Rn=(1−p)Rn−1 for INTER blocks with non-zero difference signal; and Rn=Rn−1 for INTER blocks with zero difference signals, where p is a packet loss probability and Rn−1 is one of a reliability of a same block in a previous frame and an expression dependent on a reliability of neighbor blocks that includes non-zero motion vectors effects.
- 13. The video-conferencing system of claim 12, wherein
the encoder is configured to determine a change in mean value, MDn for a block in a frame n as follows: MDn=|Mn−Mn−1 where Mn and Mn−1 are mean values of a block in frame n and n−1, respectively.
- 14. The video-conferencing system of claim 13, wherein
the encoder is configured to determine a sum of absolute differences between original block pixels, c(i,j) and pixels pointed to by a chosen motion vector, p(i,j) as follows 4SAD1=[∑i,j&LeftBracketingBar;c(i,j)-p(i,j)&RightBracketingBar;][(ar+b)(c(1-Rn)+dpMDn)+1],where r is a bit rate of a compressed video stream measured in kbps, p is a packet loss probability, and a, b, c, and d are empirically determined constants, corresponding to a communication system, determine a sum of absolute differences between the original blocks of pixels and a mean value as follows 5SAD2=∑i,j&LeftBracketingBar;c(i,j)-Mn&RightBracketingBar;+K,where K is an empirically determined constant; and insert the INTRA block if SAD2<SAD1 or insert the INTER block if SAD2≧SAD1.
- 15. The video-conferencing system of claim 14, wherein
the encoder is configured to choose constants a, b, c, and d such that a penalty term increases with a change in mean value MDn, the packet loss rate p, and the bit rate r, and decreases with reliability Rn.
- 16. The video-conferencing system of claim 1, wherein
the error concealment device is configured to estimate a motion vector V′n(k, 1) of a lost frame n in block position (k,l) with an actual motion vector Vn−1(k,l) of a previous frame n−1 at block position (k,l).
- 17. The video-conferencing system of claim 16, wherein
the error concealment device is configured to set a residual of the lost frame n to a predetermined residual value.
- 18. The video-conferencing system of claim 17, wherein
the error concealment device is configured to set the residual of the lost frame n to zero.
- 19. A method for video-conferencing, comprising:
collecting one of packet loss notifications and a lost packet identifier; and one of
determining if the collected packet loss notifications correspond to one of a first and a second predetermined packet loss scenario, and replacing video data lost in a current frame using motion vector information from a previous frame in response to the lost packet identifier.
- 20. The video-conferencing method of claim 19, further comprising:
inserting one of an INTRA and an INTER block according to a first and a second method if the collected packet loss notifications correspond to the first and the second predetermined packet loss scenario, respectively.
- 21. The video-conferencing method of claim 19, further comprising:
detecting the first packet loss scenario if a periodicity in last n packet loss notifications exceeds a predetermined packet loss periodicity value.
- 22. The video-conferencing method of claim 21, further comprising:
detecting the first packet loss scenario if at least K of N×N ratios are less than a predetermined threshold t, where T0, T1, . . . and TN are a set of time instances corresponding to a set of N most recent notifications; di is a plurality of time differences d0=T1−T0, d1=T2−T1, . . . dN−1=TN−TN−1; and the N×N ratios equal ri,j=|di|/|dj| corresponding to every possible combination of two time differences.
- 23. The video-conferencing method of claim 19, further comprising:
inserting a plurality of INTRA blocks separated by a predetermined distance upon receipt of the packet loss notification, wherein the predetermined distance corresponds to a packet loss occurrence rate.
- 24. The video-conferencing method of claim 19, further comprising:
varying the predetermined distance such that a higher packet loss occurrence rate results in a closer spacing of inserted INTRA blocks.
- 25. The video-conferencing method of claim 19, further comprising:
updating an entire picture using INTRA blocks during a predetermined number of frames.
- 26. The video-conferencing method of claim 19, further comprising:
preventing a motion vector from pointing into areas of the entire picture that has not yet been updated by an INTRA block.
- 27. The video-conferencing method of claim 19, further comprising:
estimating anticipated packet losses when making a decision between inserting an INTRA and inserting an INTER blocks.
- 28. The video-conferencing method of claim 19, further comprising:
determining one of a reliability of a reference block and a change in mean value when a packet loss scenario is detected.
- 29. The video-conferencing method of claim 28, further comprising:
determining a reliability Rn of a reference block in a frame n on the basis of a period since the reference block was last updated as INTRA.
- 30. The video-conferencing method of claim 29, wherein the step of determining a reliability Rn comprises:
determining Rn=1−p for INTRA blocks; Rn=(1−p)Rn−1 for INTER blocks with non-zero difference signal; and Rn=Rn−1 for INTER blocks with zero difference signals, where p is a packet loss probability and Rn−1 is one of a reliability of a same block in a previous frame and an expression dependent on a reliability of neighbor blocks that includes non-zero motion vectors effects.
- 31. The video-conferencing method of claim 30, further comprising:
determining a change in mean value, MDn for a block in a frame n as follows: MDn=|Mn−Mn−1|where Mn and Mn−1 are mean values of a block in frames n and n-1, respectively.
- 32. The video-conferencing method of claim 31, further comprising:
determining, in a presence of the predetermined packet loss scenario, SAD1 as follows: 6SAD1=[∑i,j&LeftBracketingBar;c(i,j)-p(i,j)&RightBracketingBar;][(ar+b)(c(1-Rn)+dpMDn)+1],where r is a bit rate of a compressed video stream measured in kbps, p is a packet loss probability, and a, b, c, and d are empirically determined constants, corresponding to a communication method; determining a sum of absolute differences between the original blocks of pixels and a mean value as follows: determine a sum of absolute differences between the original blocks of pixels and a mean value as follows 7SAD2=∑i,j&LeftBracketingBar;c(i,j)-Mn&RightBracketingBar;+K,where K is an empirically determined constant; and inserting the INTRA block if SAD2<SAD1 or inserting the INTER block if SAD2≧SAD1.
- 33. The video-conferencing method of claim 32, further comprising:
choosing constants a, b, c, and d such that a penalty term increases with a change in mean value MDn, the packet loss rate p, and the bit rate r, and decreases with reliability Rn.
- 34. The video-conferencing method of claim 19, wherein the step of replacing video data lost comprises:
setting an estimated motion vector V′n(k,l) for frame n in block position (k,l) equal to an actual motion vector Vn−1(k,l) of a previous frame n−1 at block position (k,l).
- 35. The video-conferencing method of claim 34, wherein the step of replacing video data lost further comprises:
setting a residual corresponding to the actual motion vector to a predetermined residual value.
- 36. The video-conferencing method of claim 35, wherein the step of setting a residual comprises:
setting the residual to zero.
- 37. A video-conferencing interface device connecting two networks, comprising:
a decoder connected to an upstream encoder by a first of the two networks; an encoder connecting the decoder to a downstream decoder by a second of the two networks; a packet loss detection mechanism connected to the decoder configured to detect a lost packet and to transmit one of a packet loss notification message and a lost packet identifier; and one of
a sustained packet loss detection mechanism connected to the packet loss detection mechanism and the upstream encoder, the sustained packet loss detection mechanism configured to receive the packet loss notification message and to inform the upstream encoder that the packet loss notification message indicates a presence of one of a first and a second predetermined upstream packet loss scenario, and an error concealment device connected to the decoder and the packet loss detection mechanism, the error concealment device configured to replace a lost motion vector with a replacement motion vector from a previous frame.
- 38. The video-conferencing gateway device of claim 37, wherein
the encoder is connected to a second sustained packet loss notification device configured to identify a presence of a first and a second predetermined downstream packet loss scenario, the encoder further configured to insert an INTER or INTRA block according to one of a first and a second method corresponding to the presence of the first and the second predetermined downstream packet loss scenario.
- 39. A video-conferencing interface device connecting two networks, comprising:
a packet loss detection mechanism configured to detect a lost packet and to transmit a lost packet identifier; and an error concealment device connected to the packet loss detection mechanism, the error concealment device including a decoder and an encoder, the error concealment device configured to replace a lost motion vector with a replacement motion vector from a previous frame.
- 40. The video-conferencing interface device of claim 39, wherein the error concealment device is further configured to
decompress incoming packets of compressed video data, detect a lost packet of decompressed video data, estimate a lost packet motion vector from a corresponding previous frame motion vector, compress the estimated lost packet motion vector, reset a residual corresponding to the lost packet motion vector to a predetermined value, compress the reset residual, combine the compressed estimated motion vector and compressed reset residual to form an estimated packet, and output the estimated packet to the second network.
- 41. The video-conferencing interface device of claim 40, wherein:
the packet loss detection device is further configured to transmit a packet loss notification message.
- 42. The video-conferencing interface device of claim 41, further comprising:
a sustained packet loss detection mechanism connected to the packet loss detection mechanism and to an upstream encoder, the sustained packet loss detection mechanism configured to receive the packet loss notification message from the packet loss detection device and to inform the upstream encoder that the packet loss notification message indicates a presence of one of a first and a second predetermined upstream packet loss scenario.
- 43. A video-conferencing encoder, comprising:
a connection to a downstream decoder; wherein the encoder is configured to insert an INTER or INTRA block according to one of a first and a second method, said first and second methods selected in response to a presence of a first and a second predetermined downstream decoder packet loss scenario, respectively.
- 44. The video-conferencing encoder of claim 43, further comprising:
a sustained packet loss detection mechanism configured to receive a downstream packet loss notification message from a packet loss detection mechanism co-located with the downstream decoder, the downstream packet loss notification message configured to indicate a presence of one of the first and the second predetermined downstream packet loss scenario.
- 45. A video-conferencing decoder, comprising:
a connection to an upstream encoder; a packet loss detection mechanism configured to detect a lost packet and to transmit one of a packet loss notification message and a lost packet identifier; and one of
a sustained packet loss detection mechanism connected to the packet loss detection mechanism and the upstream encoder, the sustained packet loss detection mechanism configured to receive the packet loss notification message and to inform the encoder that the packet loss notification message indicates a presence of one of a first and a second predetermined upstream packet loss scenario, and an error concealment device connected to the packet loss detection mechanism and configured to replace a lost motion vector with a motion vector from a previous frame.
- 46. A video-conferencing system, comprising:
an upstream encoder on an upstream network; a downstream decoder on a downstream network; and a video conferencing interface device connecting the upstream and the downstream networks, including
a decoder connected to the upstream encoder; an encoder connecting the decoder to the downstream decoder; a packet loss detection mechanism connected to the decoder and configured to detect a lost packet and to transmit one of a packet loss notification message and a lost packet identifier; and one of
a sustained packet loss detection mechanism connected to the packet loss detection mechanism and the upstream encoder, the sustained packet loss detection mechanism configured to receive the packet loss notification message and to inform the upstream encoder that the packet loss notification message indicates a presence of one of a first and a second predetermined upstream packet loss scenario, and an error concealment device connected to the decoder and the packet loss detection mechanism, the error concealment device configured to replace a lost motion vector with a motion vector from a previous frame.
- 47. The video-conferencing system of claim 46, wherein
the encoder is connected to a second sustained packet loss notification device configured to identify a presence of a first and a second predetermined downstream packet loss scenario, the encoder further configured to insert an INTER or INTRA block according to one of a first and a second method corresponding to the presence of the first and the second predetermined downstream packet loss scenario.
- 48. A video-conferencing system, comprising:
an upstream encoder on an upstream network; a downstream decoder on a downstream network; and a video conferencing interface device connecting the upstream and the downstream networks, including
a packet loss detection mechanism configured to detect a lost packet and to transmit a lost packet identifier; and an error concealment device connected to the packet loss detection mechanism, the error concealment device including a decoder and an encoder, the error concealment device configured to replace a lost motion vector with a replacement motion vector from a previous frame.
- 49. The video-conferencing interface system of claim 48, wherein the error concealment device is further configured to
decompress incoming packets of compressed video data, detect a lost packet of decompressed video data, estimate a lost packet motion vector from a corresponding previous frame motion vector, compress the estimated lost packet motion vector, reset a residual corresponding to the lost packet motion vector to a predetermined value, compress the reset residual, combine the compressed estimated motion vector and compressed reset residual to form an estimated packet, and output the estimated packet to the downstream network.
- 50. The video-conferencing system of claim 48, further comprising:
a sustained packet loss detection mechanism connected to the packet loss detection mechanism and to an upstream encoder, the sustained packet loss detection mechanism configured to receive a packet loss notification message from the packet loss detection device and to inform the upstream encoder that the packet loss notification message indicates a presence of one of a first and a second predetermined upstream packet loss scenario.
- 51. The video-conferencing system of claim 48, wherein
the error concealment device is further configured to
decompress incoming packets of compressed video data, detect a lost packet of decompressed video data, estimate a lost packet motion vector from a corresponding previous frame motion vector, compress the estimated lost packet motion vector, reset a residual corresponding to the lost packet motion vector to a predetermined value, compress the reset residual, combine the compressed estimated motion vector and compressed reset residual to form an estimated packet, and output the estimated packet for to the downstream network.
- 52. A video-conferencing system, comprising:
means for collecting one of packet loss notifications and a lost packet identifier; and one of
means for determining if the collected packet loss notifications correspond to one of a first and a second predetermined packet loss scenario, and means for replacing video data lost in a current frame using motion vector information from a previous frame in response to the lost packet identifier.
- 53. The video-conferencing method of claim 52, further comprising:
means for inserting one of an INTRA and an INTER block according to a first and a second method if the collected packet loss notifications correspond to the first and the second predetermined packet loss scenario, respectively.
- 54. A video-conferencing error concealment method, comprising:
decompressing incoming packets of compressed video data; detecting a lost packet of decompressed video data; estimating a lost packet motion vector from a corresponding previous frame motion vector; compressing the estimated lost packet motion vector; resetting a residual corresponding to the lost packet motion vector to a predetermined value; compressing the reset residual; combining the compressed estimated motion vector and compressed reset residual to form an estimated packet; and outputting the estimated packet.
- 55. A video-conferencing error concealment device, comprising:
means for decompressing incoming packets of compressed video data; means for detecting a lost packet of decompressed video data; means for estimating a lost packet motion vector from a corresponding previous frame motion vector; means for compressing the estimated lost packet motion vector; means for resetting a residual corresponding to the lost packet motion vector to a predetermined value; means for compressing the reset residual; means for combining the compressed estimated motion vector and compressed reset residual to form an estimated packet; and means for outputting the estimated packet.
- 56. A computer program product configured to host and provide instructions corresponding to anyone of the method steps of claims 19-36.
- 57. A computer program product configured to host and provide instructions corresponding to anyone of the method steps of claim 54.
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] The present patent application is related and claims priority to provisional U.S. application 60/423,393 filed on Nov. 4, 2002. The present application is also related to co-pending U.S. application attorney docket number 243613US, filed on the same day as the present application. The entire contents of both provisional U.S. application 60/423,393 and co-pending U.S. application attorney docket number 243610US are incorporated herein by reference.
Provisional Applications (1)
|
Number |
Date |
Country |
|
60423393 |
Nov 2002 |
US |