The present inventions relate to devices and/or methods of digital video and/or audio reception and/or output, having and/or implementing error detection and/or concealment circuitry and techniques to detect, locate and conceal errors in the received signals in video and/or audio decoding systems. More particularly, in one aspect, to a satellite, terrestrial and/or cable receiver (for example digital broadcasting TV receiver (for example, a mobile-type TV receiver)) which implements transport stream de-multiplexer circuitry, having error detection, identification, and/or concealment circuitry therein, to detect one or more errors in the transport stream and, under certain conditions, in response thereto, to conceal such one or more errors to, for example, a user, operator, listener and/or viewer.
Briefly, a digital broadcast TV receiver may generally consist of a TV tuner for (i) tuning the receiver to, for example, a user selected channel of the frequency band and (ii) converting the received RF signal to a baseband signal. The digital broadcast TV receiver also includes baseband processor circuitry that responsively acquires one or more channels (associated with one or more of the user selected channels) by demodulating and decoding the baseband signal into a transport data stream. The digital broadcast TV receiver further includes transport stream de-multiplexer circuitry to identify the selected program stream and extract and separate audio and video data streams from the transport data stream.
The digital broadcasting TV receiver also includes video and audio decoder circuitry which decompresses or decodes the corresponding audio and video data streams. Video and audio output circuitry provides video and audio rendering functions using the decompressed or decoded audio and video data streams. Finally, the digital broadcasting TV receiver generally includes a user interface (for example, a display and/or a speaker(s)) for corresponding video display and/or audio play-back.
When the digital broadcast TV reception is interrupted, insufficient, erroneous, inadequate and/or incompatible due to, for example, interference (for example, weather interference), transport stream packets, having errors contained therein, are often received by the digital broadcasting TV receiver. In conventional systems, the baseband processing circuitry (including, for example, channel decoder circuitry) responsively demodulates and decodes the baseband signal into transport stream packets having one or more error bits and/or flags enabled or asserted (for example, the transport error indicator (“TEI”) bit in an MPEG-2 environment) for those packets having errors contained therein. In the MPEG-2 environment, the de-multiplexer circuitry typically discards a transport stream packet having an asserted TEI bit or flag. When video elementary stream and/or audio elementary stream are/is corrupted, the video decoder and/or audio decoder often generate erroneous video and audio data. Audio and video artifacts may present when the erroneous audio and video data are played back and/or displayed to, for example, a user, operator, listener and/or viewer.
There are many inventions described and illustrated herein. The present inventions are neither limited to any single aspect nor embodiment thereof, nor to any combinations and/or permutations of such aspects and/or embodiments. Moreover, each of the aspects of the present inventions, and/or embodiments thereof, may be employed alone or in combination with one or more of the other aspects of the present inventions and/or embodiments thereof. For the sake of brevity, many of those permutations and combinations will not be discussed separately herein.
Importantly, the present inventions are neither limited to any single aspect nor embodiment, nor to any combinations and/or permutations of such aspects and/or embodiments. Moreover, each of the aspects of the present inventions, and/or embodiments thereof, may be employed alone or in combination with one or more of the other aspects and/or embodiments thereof. For the sake of brevity, certain permutations and combinations are not discussed and/or illustrated separately herein.
In a first principle aspect, certain of the present inventions are directed to a device to output video and/or audio data (for example, corresponding to a selected channel which is one of a plurality of channels of a broadcast spectrum), the device comprising (i) baseband processor circuitry to demodulate a baseband signal into a data stream (for example, MPEG type data stream, such as an MPEG-2 transport or program data stream) having a plurality of packets including a plurality of video and/or audio packets wherein each video and/or audio packet includes video and/or audio payload, (ii) de-multiplexer circuitry, coupled to the baseband processor circuitry, to: (a) de-multiplex the data stream to obtain the video and/or audio payload of the plurality of video and/or audio packets, (b) detect and locate one or more errors in one or more of the video and/or audio packets, and (c) generate error characterization data (for example, information which is representative of the type of error and/or the location of the error in the video and/or audio payload) which is representative of or characterizes one or more errors in the one or more of the video and/or audio packets; and (iii) decoder circuitry, coupled to the de-multiplexer circuitry, to: (a) receive the video and/or audio payload and the error characterization data, and (b) conceal the one or more errors in the video and/or audio payload using the error characterization data and/or output video data using the video and/or audio payload and the video error characterization data.
In another principle aspect, certain of the present inventions are directed to a device to output video data (for example, corresponding to a selected channel which is one of a plurality of channels of a broadcast spectrum), the device comprising baseband processor circuitry to demodulate a baseband signal into a data stream (for example, MPEG type data stream, such as an MPEG-2 transport or program data stream) having a plurality of packets including a plurality of video packets wherein each video packet includes video payload, and de-multiplexer circuitry, coupled to the baseband processor circuitry, to: (i) de-multiplex the data stream to obtain the video payload associated with each video packet, (ii) detect and locate one or more errors in a video payload, and (iii) generate video error characterization data (for example, data which is representative of the type of error and/or the location of the error in the video payload) in response to the detection of an error in a video payload. The device further includes video decoder circuitry, coupled to the de-multiplexer circuitry, to: (i) receive the plurality of video payloads and the video error characterization data, (ii) decompress the video payloads, and (iii) conceal one or more errors detected in a video payload using video error characterization data associated therewith.
The video decoder circuitry may generate output video data using the video payloads and the video error characterization data. Indeed, the device may further include a user interface to display video which is representative of the output video data.
In another embodiment, the de-multiplexer circuitry further de-multiplexes the data stream into a plurality of audio packets and de-multiplexer circuitry is configured to (i) detect and locate one or more errors in an audio payload of an audio packet, and (ii) generate audio error characterization data (which may be representative of the type of error and/or the location of the error in the audio payload) which is representative of or characterizes one or more errors detected in the audio payload. The device, in this embodiment, may include audio decoder circuitry, coupled to the de-multiplexer circuitry, to (i) receive the audio payload and the audio error characterization data, (ii) decompress the audio payload, and (iii) conceal one or more errors detected in the audio payload using the audio error characterization data.
In one embodiment, the video decoder circuitry may generate output video data using the video payload and video error characterization data, the audio decoder circuitry may generate output audio data using the audio payload and audio error characterization data, and the device may further include a user interface to (i) display video which is representative of the output video data and (ii) output audio which is representative of the output audio data.
In another principal aspect, the present inventions are directed to a device to output video data corresponding to a selected program which is associated with a data stream (for example, transport data stream or a program data stream) which includes a plurality of video packets, the device comprises de-multiplexer circuitry (i) to de-multiplex the data stream into a plurality of video packets wherein each video packet includes video payload, and (ii) configured to detect and locate one or more errors in one or more of the video packets. The de-multiplexer circuitry of this embodiment includes error data generation circuitry to generate a plurality of descriptors, wherein each descriptor is associated with a video payload. The descriptor includes a video error flag, wherein the video error flag is enabled when an error is detected in the video packet of the associated video payload, and video error characterization data which is representative of or characterizes one or more errors in the video packet when an error is detected in the video packet of the associated video payload.
The device of this aspect includes video decoder circuitry, coupled to the de-multiplexer circuitry, to: receive a plurality of video payloads and the descriptors associated therewith, and generate output video data using (i) the plurality of received video payloads and (ii) if a video error flag of a descriptor associated with a video payload of the received video payloads is enabled, the descriptor associated with the video payload having the enabled video error flag.
The video error characterization data may include information which is representative of the type of error and/or the location of the error in the video packet of the associated video payload. The data stream may be is an MPEG type data stream. Moreover, each video packet further includes a video header. Further, the de-multiplexer circuitry may output the descriptor and the associated video payload substantially simultaneously to the video decoder circuitry.
In one embodiment, the plurality of received video payloads includes a first video payload and the first video payload is associated with a first descriptor, wherein, in response to an enabled video error flag of the first descriptor, the video decoder circuitry generates the output video data using the first video payload and the video error characterization data of the first descriptor by the concealing one or more errors in the first video payload based on the video error characterization data of the first descriptor.
The device may include a user interface to display video which is representative of the output video data. The device of this embodiment may also include baseband processor circuitry to demodulate a baseband signal into the data stream having a plurality of the video packets, and wherein the baseband processor circuitry outputs the data stream corresponding to a selected channel to the de-multiplexer circuitry.
In another embodiment of this aspect of the inventions, the de-multiplexer circuitry (i) further de-multiplexes the data stream into a plurality of audio packets, wherein each audio packet includes an audio payload, and (ii) is configured to detect and locate one or more errors in one or more of the audio packets. In addition, the error data generation circuitry further generates a plurality of audio descriptors, wherein each audio descriptor is associated with an audio payload, and wherein the audio descriptor includes (i) an audio error flag, wherein the audio error flag is enabled when an error is detected in the audio packet of the associated audio payload, and (ii) audio error characterization data which is representative of or characterizes one or more errors in the audio packet when an error is detected in the audio packet of the associated audio payload. Further, the device further includes audio decoder circuitry, coupled to the de-multiplexer circuitry, to (i) receive a plurality of audio payloads and the descriptors associated therewith, and (ii) generate output audio data using the audio payload and, if the audio error flag of the associated audio descriptor is enabled, the audio descriptor associated therewith.
The device may include a user interface to (i) display video which is representative of the output video data and (ii) output audio which is representative of the output audio data. The audio error characterization data may include information which is representative of the type of error and/or the location of the error in the associated audio payload.
Further, the device may include baseband processor circuitry to demodulate a baseband signal into the data stream having a plurality of the video packets and a plurality of the audio packets, and wherein the baseband processor circuitry outputs the data stream corresponding to a selected channel to the de-multiplexer circuitry.
Notably, although not discussed in detail, the present inventions are also directed to methods and techniques of digital video and/or audio reception and/or output, having and/or implementing error detection and/or concealment techniques to detect, locate and conceal errors in the received signals in video and/or audio decoding systems and outputting such concealed video and/or audio data. Indeed, as stated above, there are many inventions, and aspects of the inventions, described and illustrated herein. This Summary is not exhaustive of the scope of the present inventions. Indeed, this Summary may not be reflective of or correlate to the inventions protected by the claims in this or in continuation/divisional applications hereof.
Moreover, this Summary is not intended to be limiting of the inventions or the claims (whether the currently presented claims or claims of a divisional/continuation application) and should not be interpreted in that manner. While certain embodiments have been described and/or outlined in this Summary, it should be understood that the present inventions are not limited to such embodiments, description and/or outline, nor are the claims limited in such a manner (which should also not be interpreted as being limited by this Summary).
Indeed, many other aspects, inventions and embodiments, which may be different from and/or similar to, the aspects, inventions and embodiments presented in this Summary, will be apparent from the description, illustrations and claims, which follow. In addition, although various features, attributes and advantages have been described in this Summary and/or are apparent in light thereof, it should be understood that such features, attributes and advantages are not required whether in one, some or all of the embodiments of the present inventions and, indeed, need not be present in any of the embodiments of the present inventions.
In the course of the detailed description to follow, reference will be made to the attached drawings. These drawings show different aspects of the present inventions and, where appropriate, reference numerals illustrating like structures, components, materials and/or elements in different figures are labeled similarly. It is understood that various combinations of the structures, components, and/or elements, other than those specifically shown, are contemplated and are within the scope of the present inventions.
Moreover, there are many inventions described and illustrated herein. The present inventions are neither limited to any single aspect nor embodiment thereof, nor to any combinations and/or permutations of such aspects and/or embodiments. Moreover, each of the aspects of the present inventions, and/or embodiments thereof, may be employed alone or in combination with one or more of the other aspects of the present inventions and/or embodiments thereof. For the sake of brevity, certain permutations and combinations are not discussed and/or illustrated separately herein.
Again, there are many inventions described and illustrated herein. The present inventions are neither limited to any single aspect nor embodiment thereof, nor to any combinations and/or permutations of such aspects and/or embodiments. Each of the aspects of the present inventions, and/or embodiments thereof, may be employed alone or in combination with one or more of the other aspects of the present inventions and/or embodiments thereof. For the sake of brevity, many of those combinations and permutations are not discussed separately herein.
There are many inventions described and illustrated herein. In one aspect, the present inventions are directed to de-multiplexer circuitry for use in video and/or audio decoding systems that, in addition to de-multiplexing of transport data streams (for example, a MPEG-2 type data streams), includes (i) circuitry to detect and/or identify one or more errors in one or more packets of a transport data stream, (ii) circuitry to locate one or more errors in one or more packets of a transport data stream, and/or (iii) circuitry to conceal or mask the one or more errors in one or more packets of the transport data stream prior to providing and/or outputting the de-multiplexed transport data stream to video and/or audio decoder circuitry. The present inventions may be employed in a satellite, terrestrial and/or cable digital television environment (including, for example, digital television receiver (for example, digital broadcasting TV receiver, for example, mobile-type TV receiver)) and/or digital data (video and/or audio) playback devices (for example, Compact Disc (CD) or Digital Versatile Disc (DVD) player). The concealed errors in one or more packets of a transport data stream may be, under certain conditions, less-noticeable or unnoticeable to, for example, a user, operator, listener and/or viewer. Indeed, the concealment circuitry may reduce, minimize, and/or eliminate any adverse impact of the errors in one or more packets of a transport data stream to, for example, a user, operator, listener and/or viewer.
In another aspect, the present inventions are directed to methods of detecting and/or identifying one or more errors in one or more packets of a transport data stream (for example, a MPEG-2 type data stream), locating one or more errors in one or more packets of a transport data stream and/or, under certain circumstances, concealing or masking the one or more errors in one or more packets of the transport data stream prior to providing the de-multiplexed transport data stream to video and/or audio decoder circuitry. Again, the present inventions may be employed in a satellite, terrestrial and/or cable digital television environment (among others) and/or receiver (for example digital broadcasting TV receiver) which implements coded transport data stream communications. The methods according to certain aspects of the present inventions may, under certain conditions, conceal errors in one or more packets of a transport data stream such that any adverse impact of such errors are reduced, minimized, and/or eliminated in one or more packets of a transport data stream.
In yet another aspect, the present inventions are directed to circuitry and methods of detecting and/or identifying one or more errors in one or more packets of a transport data stream (for example, a MPEG-2 type data stream), locating one or more errors in one or more packets of a transport data stream, and/or, in response, generating and outputting, in conjunction with the de-multiplexed transport data stream, or separately therefrom, signals or data which are representative of or characterize error information (for example, characteristics of the error including, for example, the type of error, the location of the error, and/or the extent of the error), to video and/or audio decoder circuitry. In these embodiments, the video and/or audio decoder circuitry may interpret, analyze and/or employ the signals or data which are representative of error information (or characterize errors) and conceal or mask the one or more errors in one or more packets of the transport data stream.
Thus, in addition to or in lieu of implementing error concealment circuitry and techniques in the de-multiplexer circuitry, error concealment circuitry and techniques may be implemented in video and/or audio decoder circuitry, wherein such error concealment circuitry and techniques employ the data which are representative of error information and, under certain circumstances, conceal or mask errors in one or more packets of the transport data stream. In this regard, the video and/or audio decoder circuitry may receive the error information from or generated by the de-multiplexer circuitry (separately from, substantially coincident with and/or in conjunction or together with the de-multiplexed transport payload or data stream or packet), and in response thereto, using and/or based thereon, perform and/or implement error concealment or additional error concealment (relative to any error concealment implemented in the de-multiplexer circuitry).
As such, the de-multiplexer circuitry, in addition to de-multiplexing program packets from the transport data stream, implements error detection or identification, error location and/or concealment, prior to providing such de-multiplexed transport data stream to the video and/or audio decoder circuitry. In addition thereto or in lieu thereof, the de-multiplexer circuitry may provide signals or data that are representative of or characterize the error(s) to the video and/or audio decoder circuitry (or other circuitry) which responsively performs and/or implements error concealment based on or using such error information.
Notably, the present inventions, in certain aspects, may provide a more efficient and effective error detection technique in that certain error detection may be performed based on one or more error enabled/asserted bits and/or flags (for example, the transport error indicator (“TEI”) bit asserted in an MPEG-2 environment) set, enabled and/or asserted by baseband processor circuitry in the receiver. In this regard, the one or more error enabled/asserted bits and/or flags may be identified, located and/or determined by parsing the data stream based on, for example, the bit-stream syntax (which may be well defined or understood, standardized and/or proprietary).
Moreover, it should be further noted that implementing the inventions of the present inventions, in certain aspects, may provide for a more efficient approach in that error detection and identification may be performed on a more compressed data stream domain (as compared to the data stream provided to the video and/or audio decoder circuitry). As such, the amount of data to be processed may be less than that of the de-compressed data stream (received by the video and/or audio decoder circuitry). Indeed, the inventions may be implemented using hardware and/or software techniques (and/or combinations thereof) thereby providing the opportunity for a low-cost and flexible design.
With reference to
In one embodiment, transport stream de-multiplexer circuitry 12 receives a transport data stream 14 having a header (comprising, for example, a plurality of bytes) and a payload or data load (comprising, for example, a plurality of bytes). The transport data stream 14 typically includes a defined format or data hierarchy of a predefined header and a predefined payload or data load (for example, a MPEG-2 type data stream which is described in detail in/at ISO/IEC 13818, which is attached to the Provisional Application as Attachment 1). Using or based on the definition or characteristics of the transport data stream, transport stream de-multiplexer circuitry 12 identifies the selected program stream, and extracts and separates audio and/or video data streams. In addition, transport stream de-multiplexer circuitry 12 analyzes transport data stream 14 to detect and/or identify one or more errors in one or more packets of a transport data stream (for example, a MPEG-2 type data stream). In certain embodiments, transport stream de-multiplexer circuitry 12 conceals or masks the one or more errors prior to providing and/or outputting the de-multiplexed transport data stream to video and/or audio decoder circuitry. In addition thereto or in lieu thereof, transport stream de-multiplexer circuitry 12 outputs signals or data which are representative of or characterize error(s) (for example, characteristics of the error(s)) to, for example, video and/or audio decoder circuitry. In these embodiments, downstream circuitry may interpret the signals or data which are representative of or characterize the error(s) and, under certain circumstances, conceal or mask one or more errors detected or identified in one or more packets of the transport data stream.
Thus, in operation, transport stream de-multiplexer circuitry 12 may output audio data streams to audio decoder circuitry and video data streams to video decoder circuitry. The transport stream de-multiplexer circuitry 12 may also output information which is representative of or characterize error(s) (for example, characteristics of the error(s)) identified in such data streams to the video and/or audio decoder circuitry (or other circuitry). The video and/or audio decoder circuitry (or other circuitry) may perform and/or implement error concealment based on or using the error information output by the transport stream de-multiplexer circuitry 12. Notably, in a digital broadcasting TV receiver environment, video and audio decoder circuitry decodes and decompresses the corresponding audio and video data streams and video and audio output circuitry provide video and audio rendering functions (using the decoded and decompressed audio and video data streams) to, for example, a user interface (for example, a display and/or a speaker(s)) for corresponding video display and/or audio play-back.
With reference to
Notably, the present inventions may be implemented in conjunction with any type of tuner circuitry 16 and baseband processor circuitry 18 (including discrete devices or integrated devices), whether now known or later developed. All tuner circuitry 16 and baseband processor circuitry 18, consistent with digital communications outlined herein, are intended to fall within the scope of the present inventions.
With continued reference to
As mentioned above, the present inventions may be employed in a satellite, terrestrial and/or cable communications environments (among others) which implements transport stream de-multiplexer circuitry. (See, for example,
With reference to
The data filter circuitry 28 receives the transport data stream and filters the data stream to identify certain data contained therein. For example, in the context of MPEG-2, data filter circuitry 28 may analyze (for example, parse) the data stream to identify the packet identifier (PID) as well as determine or identify the transport error indicator (“TEI”). Here, the transport stream de-multiplexer circuitry 12 determines whether the PID value matches at least one of the specified values in a PID filter table, wherein in those situations where the PID value does not match at least one of the specified values in a PID filter table, the packet may be discarded because it is not pertinent to the user/operator and/or the selected channel(s). Notably, in the event that the TEI is asserted/enabled, in one embodiment, transport stream de-multiplexer circuitry 12 may determine the location(s) of the “corrupted” or erroneous slice(s) based on, for example, the slice numbers before and after the transport stream packets that have asserted/enabled TEI bits. As such, the analysis (for example, a parsing operation) of the transport data stream by data filter circuitry 28 may interpret and/or identify the sequence header, the group of picture (GOP) header, the picture header, the slice header, PID values and/or TEI values of the stream.
As discussed in more detail below, data filter circuitry 28 of transport stream de-multiplexer circuitry 12 may monitor or count the number of consecutive packets having an asserted/enabled TEI bit or flag. In one embodiment, where data filter circuitry 28 detects a predetermined number of consecutive packets having an asserted/enabled TEI bit or flag (for example, 200 consecutive packets having an asserted TEI bit or flag), transport stream de-multiplexer circuitry 12 may determine, for example, there is a moderate level of channel interference, in which baseband processor circuitry 18 remains locked and TP sync is maintained (i.e., the packet(s) is/are properly synchronized for a given packet of the transport data stream—that is, no loss of TP sync), error data (for example, data which represents or characterizes the error) may be provided to circuitry in transport stream de-multiplexer circuitry 12 (for example, error data generation circuitry 40). In response, error data generation circuitry 40 (of transport stream de-multiplexer circuitry 12—see,
Notably, a PID filter table may be created, established, maintained and/or defined to specify the PID values that the PID filtering operation employs to compare against the PID value of a transport data stream packet. In one embodiment, thirty-two programmable PID values may be used to filter the transport data stream. A PID index may be associated with each of the thirty-two entries in the PID filter table. As noted above, when enabled, the PID filtering operation discards packets whose PID values do not match any of the PID values specified in the PID filter table.
The data filtering circuitry 28 may be implemented using a plurality of discrete or integrated logic, a state machine, a special or general purpose processor (suitably programmed) and/or a field programmable gate array (or combinations thereof). Indeed, it may be advantageous to implement PID and TEI filtering operations using primarily hardwired logic (for example, hardware acceleration circuitry—a plurality of logic (for example, EX-OR gates) arranged in predetermined configuration) to enhance performance/speed of the filtering operations. All permutations and/or combinations of hardwired and programmable circuitry (which is programmed, for example, via software) for implementing the data filter circuitry are intended to fall within the scope of the present inventions.
The detection and extraction circuitry 30 detects a start code prefix of a packet of the transport data stream (for example, in the context of the MPEG-2 environment or communication, a twenty-four bit binary string “00000000 00000000 00000001”). In one embodiment, upon detecting the start code prefix of a given packet of the transport data stream, detection and extraction circuitry 30 extracts the start code value that follows the start code prefix in the packet. The detection and extraction operations may continue until the entire packet is processed.
Notably, the start code extraction operation may be performed in conjunction with only transport data stream packets that are associated with video related data streams.
As such, in this embodiment, in the event the PID value of a given packet of the transport data stream matches or corresponds to that of the video stream, transport stream de-multiplexer circuitry 12 may invoke the start code extraction process for that packet. In the event that the PID value of the given packet does not match or correspond to that of the video stream, the start code extraction operation may not be performed on the packet by detection and extraction circuitry 30.
Similar to data filtering circuitry 28, detection and extraction circuitry 30 may be implemented using a plurality of discrete logic, a state machine, a special or general purpose processor (suitably programmed) and/or a field programmable gate array (or combinations thereof). Indeed, it may be advantageous to implement detection and extraction circuitry 30 using primarily hardwired logic (for example, hardware acceleration circuitry—a plurality of logic (for example, EX-OR gates) arranged in predetermined configuration) to enhance performance/speed of the filtering operations. All permutations and/or combinations of hardwired and programmable circuitry (which is programmed, for example, via software) for implementing the detection and extraction circuitry are intended to fall within the scope of the present inventions.
The memory 32 (including buffer(s) 34) stores the results of the start code detection and extraction and the demultiplexed packet(s) of the transport data stream. The demultiplexed packet(s) may include a modified format relative to the original format of the transport data stream. For example, in the context of MPEG-2, the original format (see
The memory access circuitry 36 facilitates that data transfer of the coded data. Notably, memory 32 may have a capacity to simultaneously and/or concurrently store and/or maintain a plurality of packets, for example, 100 packets. The memory (including buffer(s) 34) may be integrated or discrete memory of any kind or type, including SRAM, DRAM, latches, and/or registers. All memory types and forms, and permutations and/or combinations thereof, whether now known or later developed, are intended to fall within the scope of the present inventions. Moreover, in those instances where transport stream de-multiplexer circuitry 12 includes memory access circuitry 36, such circuitry 36 may be DMA type circuitry to provide a desired, enhanced and/or appropriate data transfer rate or bandwidth to, for example, decoder circuitry 20 (see, for example,
With reference to
The error data generation circuitry 40 may be implemented via a plurality of discrete or integrated logic, and/or one or more state machines, special or general purpose processors (suitably programmed) and/or programmable gate arrays (or combinations thereof). Such circuitry may be integrated into other circuitry of transport stream de-multiplexer circuitry 12 or separate therefrom. All circuitry (for example, discrete or integrated logic, state machine(s), special or general purpose processor(s) (suitably programmed) and/or programmable gate array(s) (or combinations thereof)) to generate signals or data which are representative of error information or characterize one or more errors, consistent with inventions described and/or illustrated herein, are intended to fall within the scope of the present inventions.
With continued reference to
In one embodiment, descriptors 44 include one or more error flags or indictors that signify the existence of an error in the associated payload or data load 42. (See, for example,
The descriptors 44, in another embodiment, may include error information/data in addition to, or in lieu of, one or more error flags. (See, for example,
Notably, descriptors 44 may also include management information/data that may be employed by the decoder circuitry to, for example, configure the decoder circuitry or circuitry related thereto. (See, for example, FIGS. 5F and 5H-5J). In addition thereto, or in lieu thereof, the management information/data may control, manage and/or modify the detection, extraction, data storing and/or decoding operations.
The descriptors 44 may be provided, transmitted and/or available to the decoder circuitry simultaneously or concurrently with the associated payload or data load 42, or before or after the associated payload or data load 42 is provided, transmitted and/or available to the decoder circuitry. Moreover, the payload or data load 42 and associated descriptor 44 may be provided to the decoder circuitry in a parallel or serial manner. Notably, all types, forms and/or manners of transmission, and circuitry or configurations therefor, are intended to fall within the scope of the present inventions.
As noted above, error data generation circuitry 40 may be implemented using a plurality of discrete logic, a state machine, a special or general purpose processor (suitably programmed) and/or a field programmable gate array (or combinations thereof). Indeed, it may be advantageous to implement error data generation circuitry 40 using a special or general purpose processor (or controller) to provide flexibility in the event that one or more operations of transport stream de-multiplexer circuitry 12 are changed, updated, enhanced, modified and/or eliminated. All permutations and/or combinations of hardwired and programmable circuitry (which is programmed, for example, via software) for implementing the error data generation circuitry are intended to fall within the scope of the present inventions.
Notably, error data generation circuitry 40 may include or share circuitry with other elements of a system (or components thereof) and/or perform one or more other operations, which may be separate and distinct from the extraction of information from the transport data stream and generation of information that is used by the decoder circuitry. For example, where the error data generation circuitry 40 is implemented via a special or general purpose processor (or controller), such processor or controller may implement or perform the error data generation operations as described herein as well as other operations or functions which may be related to, or separate and distinct from those of transport stream de-multiplexer circuitry 12. For example, where the error data generation circuitry 40 is implemented via a special or general purpose processor (or controller), such special or general purpose processor (or controller) may also be the decoder circuitry and thereby perform the decoding operations, such as the audio decoding operations.
In another embodiment, with reference to
Notably, in another embodiment, transport stream de-multiplexer circuitry 12 includes clock generation circuitry to output clocking information for the decoder circuitry (among other things) to provide or enhance synchronization of operations of transport stream de-multiplexer circuitry 12 and decoder circuitry. (See, for example,
As mentioned above, errors detected by transport stream de-multiplexer circuitry 12 may be addressed, repaired, concealed and/or masked by transport stream de-multiplexer circuitry 12 and/or by decoder circuitry 20 (or circuitry which is supervisory, attendant or concomitant thereto). Indeed, as mentioned above, decoder circuitry 20 may be a portion of a suitably programmed processor. In one exemplary embodiment, when implemented in the environment of MPEG-2 communication, transport stream de-multiplexer circuitry 12 may monitor the picture type (i.e., B-type, I-type or P-type) for each picture as it de-multiplexes the transport data stream. In one embodiment, where the picture is a B-type and the transport data stream or packet includes an erroneous or corrupted slice, transport stream de-multiplexer circuitry 12 may drop or discard the picture rather than attempting to address, repair, conceal and/or mask the error.
However, in the event that an erroneous or corrupted slice is detected in an I-type or P-type picture, transport stream de-multiplexer 12 may generate, provide and/or output data which is representative of the error (for example, characteristics of the error) to decoder circuitry 20. In response to such data, decoder circuitry 20 may address, repair, conceal and/or mask the corrupted slice by, for example, duplicating the slice of the same picture location in a previous picture or frame.
With reference to
With the aforementioned in mind, the flowchart of
Again, the technique indicated in
In the event transport stream de-multiplexer circuitry 12 determines that there is an error in the transport data stream and/or packet thereof, transport stream de-multiplexer circuitry 12 may locate and/or discard the audio frame and substitute an interpolated version of the audio frame which may be based on previous and/or subsequent decoded audio frame(s). In addition thereto, or in lieu thereof, transport stream de-multiplexer circuitry 12 may provide data which is representative of or characterize the error to decoder circuitry 20. Such data may facilitate concealment of the error by decoder circuitry 20—for example, by discarding the audio frame and substituting audio data which is an interpolated version determined from previous and/or subsequent decoded audio frame(s).
With reference to
The descriptor 44 generated by error generation circuitry 40 may include error information/data which are representative of, for example, characteristics of the error(s) including, for example, the type of error(s), the location of the error(s) (for example, in the context of MPEG-2, locate where the error is in terms of, for example, the macro-block position, slice position and/or picture position in the encoded data stream), and/or the extent of the error(s). In addition thereto, or in lieu thereof, the error information/data may include concealment information or instructions which the decoder circuitry may employ to address, conceal and/or mask erroneous information in the payload or data load.
In one embodiment, descriptors 44 may be stored in one or more queues 46 of memory 32b. (See, for example,
With reference to
Notably, in this particular exemplary embodiment, video descriptor 44a also includes the access unit start address (“AU Start Address”) which is representative of the starting address of the associated decoded video payload or data load in memory 32a. The video descriptor 44a of this embodiment also includes access unit byte length (“AU Byte Length”) which is representative of the length of the associated decoded video payload or data load in memory 32a.
With reference to
Notably, fields “DTS and “PTS” may be employed by the decoder circuitry for, among other things, synchronization of operations in the decoder circuitry and/or between the decoder circuitry and transport stream de-multiplexer circuitry. Further, “Descriptor Terminating Word” is a data sequence or word that indicates the end of a descriptor or descriptor packet.
The exemplary descriptors 44a and 44b illustrated in
Notably, with reference to
The error data generation circuitry 40 of the transport stream de-multiplexer circuitry 12 of the present inventions (see, for example,
In the event that baseband processor circuitry 18 (for example, the channel decoder circuitry) detects a loss of TP Sync for a given packet of the transport data stream (Situation 1), with reference to
Notably, in an alternative embodiment, baseband processor circuitry 18 (for example, the channel decoder circuitry) may create the special NULL packet and output the special NULL packet, rather than the decoded data from the received transport data stream, to transport stream de-multiplexer circuitry 12. As such, in this embodiment, during the period of time in which there is a loss of synchronization, baseband processor circuitry 18 (for example, the channel decoder circuitry) does not write the transport data stream into buffer 26 of transport stream de-multiplexer circuitry 12, but resumes after synchronization is established or reestablished.
The baseband processor circuitry 18 may “count” the number of consecutive packets having a loss of synchronization. In one embodiment, where the number of consecutive packets have lost synchronization exceeds a predetermined number or value, baseband processor circuitry 18 may enable or generate a flag and provide such flag to transport stream de-multiplexer circuitry 12, for example, on a dedicated signal line. (See, for example, “error data line(s)” in
The transport stream de-multiplexer circuitry 12 analyzes the input and detects the special NULL packet from transport stream buffer 34. In response, error data generation circuitry 40 generates a descriptor that reflects, indicates and/or characterizes an error in the packet and that concealment is to be performed on the associated video payload or data load of the associated packet of the transport data stream 14. In one exemplary embodiment, error data generation circuitry 40 generates a descriptor for the associated video and/or audio payload or data load, having the access unit concealment flag (“AUCFlag”) enabled (set to 1). (See, for example, the exemplary video descriptor of
The video and/or audio decoder circuitry reads the descriptor (for example, the descriptor having an enabled access unit concealment flag) and implements one or more error concealment techniques for the associated video and/or audio payload or data load. For example, in one exemplary embodiment in the context of MPEG-2 where the payload or data load is video data/information and current picture is an I-type or P-type picture, and the previous picture was an I-type or P-type picture which was decoded without error or with slice concealment, the video decoder “copies” the previous I-type or P-type picture into a reference frame buffer/memory that stores the reconstructed picture for current I-type or P-type picture. Accordingly, when it is time to display the “current” I-type or P-type picture, (i) the previously displayed B-type picture is “repeated” or redisplayed in place of or as a substitute for the I-type or P-type picture where the previously displayed B-type picture has been decoded “normally”, or (ii) the previous I-type or P-type picture is “repeated” or re-displayed in place of or as a substitute for the current I-type or P-type picture.
In the event that the current picture is a B-type, and the previous picture was a B-type picture, the decoder circuitry “repeats” or re-displays the previous B-type picture in place of or as a substitute for the current B-type picture via access of a backward reference picture buffer/memory. Where, however, the current picture is a B-type, and the previous picture was an I-type or a P-type picture, the decoder circuitry displays the data stored in a forward reference picture buffer/memory, which contains a complete picture, in place of or as a substitute for the current B-type picture.
Notably, in each instance, in this embodiment, the video decoder circuitry may implement a predetermined concealment technique(s) or operation(s) in response to the error information (for example, contained in the descriptor or descriptor packet) which is generated and/or provided by transport stream de-multiplexer circuitry 12.
In one embodiment, in the event that baseband processor circuitry 18 (for example, the channel decoder circuitry) detects a loss of lock of the signal reception (for example, the RF input signal is too weak or channel interference is too strong), baseband processor circuitry 18 may enable or generate a loss of lock flag and provide such flag to transport stream de-multiplexer circuitry 12, for example, on a dedicated signal line. (See, for example, “error data line(s)” in
With continued reference to
As mentioned above, transport stream de-multiplexer circuitry 12 may monitor and/or count the number of consecutive packets having an asserted/enabled TEI bit or flag. For example, in one embodiment, transport stream de-multiplexer circuitry 12 may detect a predetermined number of consecutive packets having an asserted/enabled TEI bit or flag (for example, 200 consecutive packets having an asserted TEI bit or flag) and determine, for example, there is a moderate level of channel interference, in which baseband processor circuitry 18 remains lock (i.e., no loss of reception or signal lock) and TP sync is maintained (i.e., no loss of TP Sync), data may be generated which represents or characterizes the error (for example, the number of asserted consecutively asserted/enabled TEI bits or flags). Such data may be provided to other circuitry in transport stream de-multiplexer circuitry 12 (for example, error data generation circuitry 40) which may responsively implement one or more error handling mechanisms to address, mask and/or conceal the errors.
The data may be provided via an additional transport packet or a descriptor packet which may be inserted into transport data stream 14 representing or characterizing the error (for example, the number of asserted consecutively asserted/enabled TEI bits or flags) or communicated separately therefrom. In one embodiment, where data filter circuitry 28 detects or determines 200 consecutive packets having an asserted TEI bit or flag and transport stream de-multiplexer circuitry 12 determines baseband processor circuitry 18 remains locked (i.e., no loss of reception or signal lock) and TP sync is maintained (i.e., no loss of TP sync), data may be generated which represents or characterizes the error (for example, the number of asserted consecutively asserted/enabled TEI bits or flags) may be as follows:
Thus, in this embodiment, data filter circuitry 28 of transport stream de-multiplexer circuitry 12 monitors or detects the TEI value for each in-coming packet of the transport data stream. Where transport stream de-multiplexer circuitry 12 detects TEI bit or flag is enabled (1′b1) for a specific packet, transport stream de-multiplexer circuitry 12 increments a counter numTeiAsserted, discards the corresponding packet; otherwise, transport stream de-multiplexer circuitry 12 processes the packet in the normal manner. The data filter circuitry 28 may implement such functions using, for example, the pseudo-code of:
Notably, where error data generation circuitry 40 detects that one or more slices of the associated coded video includes errors (in this exemplary video descriptor, Slices one, two and four), error data generation circuitry 40 may generate the exemplary video descriptor of
In response to receiving and parsing the exemplary video descriptor of
In one exemplary embodiment in the context of MPEG-2 where the current picture is a B-type picture, the video decoder determines whether the previous picture was (i) an I-type or P-type picture, and “duplicates” or re-displays the corresponding co-located slice(s) in a forward reference picture buffer in place of or as a substitute for the to-be-concealed slice(s) or (ii) a B-type picture, and “duplicates” or re-displays the corresponding co-located slice(s) in the backward reference picture buffer that will be duplicated for the to-be-concealed slice(s). Accordingly, in this exemplary embodiment, when it is time to display the I-type or P-type picture, the corresponding co-located slice(s) of the previous picture or in the backward reference picture buffer is “duplicated” or redisplayed in place of or as a substitute for the to-be-concealed slice(s). However, where the current picture is a B-type picture, and (i) the previous picture is an I-type or P-type picture, the corresponding co-located slice(s) of the previous picture or in the forward reference picture buffer are “duplicated” or redisplayed in place of or as a substitute for the to-be-concealed slice(s), or (ii) the previous picture is a B-type picture, the corresponding co-located slice(s) in the backward reference picture buffer are “duplicated” or redisplayed in place of or as a substitute for the to-be-concealed slice(s).
Notably, in each instance, in this embodiment, the video decoder circuitry implements the predetermined concealment technique(s) or operation(s) in response to the error information (for example, contained in the descriptor or descriptor packet) which is generated and/or provided by transport stream de-multiplexer circuitry 12.
In the event that error data generation circuitry 40 detects or determines a discontinuity of the continuity_counter of a coded video data packet (Situation 3), with continued reference to
In another embodiment, where receiver circuitry or receiver device is implemented in a mobile DTV reception environment, it may be advantageous to detect certain errors or potential errors in the baseband processor circuitry (for example, the channel decoder), which may be provided to the transport stream de-multiplexer circuitry. Thereafter, the transport stream de-multiplexer circuitry, as discussed herein, may address and/or implement concealment or masking techniques and/or generate descriptors or descriptor packets to be provided to decoder circuitry (for example, MPEG compliant decoder). For example, it may be advantageous to detect a loss of TP sync (i.e., the synchronization status of the packet(s) of the transport data stream) and input signal lock condition (i.e., whether there is a loss of lock of the signal reception—for example, the RF input signal is too weak or channel interference is too strong) in the baseband processor circuitry and output signals representative thereof to the transport stream de-multiplexer circuitry.
With reference to, for example,
The following table summarizes error cases and responsive error handling techniques implemented by transport stream de-multiplexer circuitry 12 in connection with loss of lock, loss of TP sync and TEI error signals or data.
Notably, where loss of lock is not indicated, detected, declared and/or enabled, and loss of TP sync is indicated, detected, and/or enabled but memory 32a does not contain video data or frames and/or has been empty for a predetermined period of time, based on the error cases and responsive error handling techniques of the table above, transport stream de-multiplexer circuitry 12 may implement Error Handling Mechanism (“EHM”) 0 at slice level concealment, which may present an issue in the event that multiple pictures are “corrupted”. In that situation, error data generation circuitry 40 may implement EHM1 (or switch from EHM0 to EHM1) for picture level concealment. To facilitate detection of this situation, transport stream de-multiplexer circuitry 12 may include circuitry (for example, a count-down or count-up counter) to monitor and/or detect the “fill” state of memory 32a in the context of video data or frames. Such circuitry may be implemented in memory 32a.
Upon receipt of loss of lock and/or loss of TP sync signals or data, transport stream de-multiplexer circuitry 12 may:
Notably, where data generation circuitry 40 detects or determines an error in one or more coded audio packets or frames, for each corrupted or erroneous coded audio packet or frame, data generation circuitry 40 generates an audio descriptor indicating an error in the coded audio pack or frame. In response, the audio decoder circuitry implements concealment techniques with respect to such one or more coded audio packets or frames.
In one exemplary embodiment, error data generation circuitry 40 generates an audio descriptor for the associated coded audio payload or data load having an enabled access unit concealment flag (“AUCFlag”). (See, for example,
In sum, the exemplary audio concealment techniques described above may provide a “soft mute in” effect in the event of an error in the audio transport data stream.
Notably, the three situations discussed above with respect to errors in the video related data may be characterized as typical errors that may be experienced in the mobile TV reception environment, among others. Indeed, in the mobile TV reception environment, one or more of these three situations (including various combinations thereof) may be expected in a typical reception environment—in between periods of acceptable, strong or good reception.
As noted above, the error data generation circuitry of the transport stream de-multiplexer circuitry of the present inventions may detect and/or identify many different types of errors in the transport data stream. Indeed, all such errors are intended to fall within the scope of the present inventions.
Under those circumstances where transport stream de-multiplexer circuitry 12, in addition to outputting a demultiplexed transport data stream, outputs signals or data which are representative of error information (for example, characteristics of the error(s) including, for example, the type of error(s), the location of the error(s), and/or the extent of the error(s)) (for example, the number of consecutive packets having an asserted/enabled TEI bit or flag and/or the number of consecutive packets having a loss of synchronization (TP sync)) to decoder circuitry 20, including video decoder circuitry 20a and/or audio decoder circuitry 20b. (See, for example,
The video and/or audio decoder circuitry 20a and 20b, respectively, may be a plurality of discrete or integrated logic, a state machine, a special or general purpose processor (suitably programmed) and/or a field programmable gate array (or combinations thereof). Further, in those embodiments where transport stream de-multiplexer circuitry 12 does not provide error information to video and/or audio decoder circuitry 20a and 20b, respectively and/or where such decoder circuitry does not employ such error information, the present inventions may be implemented in conjunction with any type of video and/or audio decoder circuitry, whether now known or later developed. Under those circumstances, all circuitry (for example, discrete or integrated logic, a state machine, a special or general purpose processor (suitably programmed) and/or a field programmable gate array (or combinations thereof)) to decode the coded data streams, consistent with inventions described and/or illustrated herein, are intended to fall within the scope of the present inventions.
Notably, transport stream de-multiplexer circuitry 12 according to the present inventions may include additional circuitry and implement additional operations/processes. For example, with reference to
There are many inventions described and illustrated herein. While certain embodiments, features, attributes and advantages of the inventions have been described and illustrated, it should be understood that many others, as well as different and/or similar embodiments, features, attributes and advantages of the present inventions, are apparent from the description and illustrations. As such, the embodiments, features, attributes and advantages of the inventions described and illustrated herein are not exhaustive and it should be understood that such other, similar, as well as different, embodiments, features, attributes and advantages of the present inventions are within the scope of the present inventions.
For example, transport stream de-multiplexer circuitry 12, according to the present inventions, may electrically couple to decoder circuitry 20 and/or processor circuitry 24 which is electrically coupled to user interface 22 (for example, display and/or speaker) in a point to point manner (see
A system implementing the transport stream de-multiplexer circuitry 12 according to the present inventions may include additional circuitry and implement additional operations/processes. For example, with reference to
Importantly, the present inventions are neither limited to any single aspect nor embodiment thereof, nor to any combinations and/or permutations of such aspects and/or embodiments. Moreover, each of the aspects of the present inventions, and/or embodiments thereof, may be employed alone or in combination with one or more of the other aspects of the present inventions and/or embodiments thereof. For the sake of brevity, many of those permutations and combinations are not discussed separately herein.
As such, the above embodiments of the present inventions are merely exemplary embodiments. They are not intended to be exhaustive or to limit the inventions to the precise circuitry, techniques, and/or configurations disclosed. Many modifications and variations are possible in light of the above teaching. It is to be understood that other embodiments may be utilized and operational changes may be made without departing from the scope of the present inventions. As such, the foregoing description of the exemplary embodiments of the inventions has been presented for the purposes of illustration and description. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the inventions not be limited solely to the description above.
Further, although exemplary embodiments and/or processes have been described above, at times, in the context of MPEG-2, the inventions described and/or illustrated herein may also be implemented in conjunction with other coded communications. As such, the discussions in the context of MPEG-2 are merely exemplary.
Moreover, although exemplary embodiments and/or processes have been described above, at times, in the context of transport data streams, the inventions described and/or illustrated herein may also be implemented in conjunction with other data streams including, for example, program data streams associated with digital data (video and/or audio) playback devices (for example, CD DVD player) implementing MPEG-2 or the like formats. For the sake of brevity, the discussions above will not be repeated in connection with other data streams including, for example, program data streams; however, the inventions and embodiments thereof are fully applicable to other data streams including, for example, program data streams, which are intended to fall within the scope of the present inventions.
It should be noted that the term “circuit” may mean, among other things, a single component (for example, electrical/electronic and/or microelectromechanical) or a multiplicity of components (whether in integrated circuit form or otherwise), which are active and/or passive, and which are coupled together to provide or perform a desired function. The term “circuitry” may mean, among other things, a circuit (whether integrated or otherwise), a group of such circuits, one or more processors, one or more state machines, one or more processors implementing software, one or more gate arrays, programmable gate arrays and/or field programmable gate arrays, or a combination of one or more circuits (whether integrated or otherwise), one or more state machines, one or more processors, one or more processors implementing software, one or more gate arrays, programmable gate arrays and/or field programmable gate arrays. The term “data” may mean, among other things, a current or voltage signal(s) whether in an analog or a digital form, which may be a single bit (or the like) or multiple bits (or the like).
It should be further noted that the various circuits and circuitry disclosed herein may be described using computer aided design tools and expressed (or represented), as data and/or instructions embodied in various computer-readable media, in terms of their behavioral, register transfer, logic component, transistor, layout geometries, and/or other characteristics. Formats of files and other objects in which such circuit expressions may be implemented include, but are not limited to, formats supporting behavioral languages such as C, Verilog, and HLDL, formats supporting register level description languages like RTL, and formats supporting geometry description languages such as GDSII, GDSIII, GDSIV, CIF, MEBES and any other suitable formats and languages. Computer-readable media in which such formatted data and/or instructions may be embodied include, but are not limited to, non-volatile storage media in various forms (e.g., optical, magnetic or semiconductor storage media) and carrier waves that may be used to transfer such formatted data and/or instructions through wireless, optical, or wired signaling media or any combination thereof. Examples of transfers of such formatted data and/or instructions by carrier waves include, but are not limited to, transfers (uploads, downloads, e-mail, etc.) over the Internet and/or other computer networks via one or more data transfer protocols (e.g., HTTP, FTP, SMTP, etc.).
Indeed, when received within a computer system via one or more computer-readable media, such data and/or instruction-based expressions of the above described circuits may be processed by a processing entity (e.g., one or more processors) within the computer system in conjunction with execution of one or more other computer programs including, without limitation, net-list generation programs, place and route programs and the like, to generate a representation or image of a physical manifestation of such circuits. Such representation or image may thereafter be used in device fabrication, for example, by enabling generation of one or more masks that are used to form various components of the circuits in a device fabrication process.
This non-provisional application claims priority to U.S. Provisional Application Ser. No. 61/194,315, entitled “Devices and Methods of Digital Video and/or Audio Reception and/or Output having Error Detection and/or Concealment Circuitry and Techniques”, filed Sep. 26, 2008 (hereinafter “the Provisional Application”); the contents of the Provisional Application are incorporated by reference herein, in their entirety.
Number | Date | Country | |
---|---|---|---|
61194315 | Sep 2008 | US |