This application claims the priority benefit of Indian patent application number 1667/Del/2008, filed on Jul. 11, 2008, entitled “SYNCHRONIZATION OF SECONDARY DECODED MEDIA STREAMS WITH A PRIMARY MEDIA STREAM,” which is hereby incorporated by reference to the maximum extent allowable by law.
1. Field of the Invention
The present invention discloses a system and method for synchronization of one or more decoded media streams with a primary media stream, where the secondary media streams can be altered to delay or drop samples to effect synchronization.
2. Discussion of the Related Art
Multimedia based systems are the most common form of communication and entertainment systems in use today. The appreciation of such systems is difficult for people suffering from vision/aural impairment. To enable such people to understand the media an “audio description/video description” that provides audible description of the scene/subtitles for the deaf and hearing impaired are transmitted along with the normal audio and video signals.
An “Audio Description” channel is an auxiliary component associated with TV services which delivers a verbal description of a visual as an aid to understanding. The Audio Description channel is offered on a separate channel known as the SAP i.e. “Secondary Audio Program” channel. However, it is noted that the audio description stream is not able to keep pace with the main audio stream of a transport channel. It therefore needs to be adjusted periodically in order to maintain synchronization with the main channel.
Similarly subtitle or subpicture tracks in various languages, including those made especially for the deaf and hearing impaired are transmitted stored as bitmap images with transparent background and are shown over the video during playback.
Many systems and methods have been disclosed which keep the audio description channel in sync with the main multimedia stream. U.S. Pat. No. 5,661,665 discloses a category of systems and methods for multimedia synchronization wherein the individual media streams samples are routed through different sets of processing components.
US 2004/0128702 describes another category of such systems and methods which are used for outputting a main media stream and a secondary media stream in sync with each other.
The methods and systems as described above provide a solution to synchronize two similar kinds of media. However the systems will require the knowledge of the mixing delays at the output which can be variable and unpredictable. This can lead to wrong alignment of the main and secondary media channels and which is more important. This synchronization mechanism is not only limited to media type as PCM audio but also to can be useful for video and subtext, to BTSC encoded main audio and SAP etc.
Therefore, a system and method is required that achieves the afore-mentioned objectives.
To achieve at least the desired objective, as well as others, one embodiment of the present disclosure describes a system comprising a media stream synchronizer for synchronizing one or more secondary decoded media streams to a primary decoded media stream, said media stream synchronizer comprising a Presentation Time Stamp (PTS) extractor for the decoded primary media stream, individual PTS extractors for each secondary media stream,
a PTS comparator for each secondary media stream, said PTS comparator receiving the output of said PTS extractor for the primary media stream and the output of the PTS extractor of its secondary media stream an output enabler for each secondary media stream, controlled by the output of its PTS comparator for providing the synchronized secondary media stream, a PTS Generator for each synchronized secondary media stream providing the final synchronized secondary media stream with corrected PTS, and a mixer combining the decoded primary media stream and all the decoded synchronized secondary media streams to produce the final output.
The present disclosure also describes a set-top box comprising a media stream synchronizer for synchronizing one or more secondary decoded media streams to a primary decoded media stream, said media stream synchronizer comprising a Presentation Time Stamp (PTS) extractor for the decoded primary media stream, individual PTS extractors for each secondary media stream, a PTS comparator for each secondary media stream, said PTS comparator receiving the output of said PTS extractor for the primary media stream and the output of the PTS extractor of its secondary media stream an output enabler for each secondary media stream, controlled by the output of its PTS comparator for providing the synchronized secondary media stream, a PTS Generator for each synchronized secondary media stream providing the final synchronized secondary media stream with corrected PTS, and a mixer combining the decoded primary media stream and all the decoded synchronized secondary media streams to produce the final output.
The present disclosure further describes a Video Cassette Recorder (VCR) comprising a media stream synchronizer for synchronizing one or more secondary decoded media streams to a primary decoded media stream, said media stream synchronizer comprising a Presentation Time Stamp (PTS) extractor for the decoded primary media stream, individual PTS extractors for each secondary media stream, a PTS comparator for each secondary media stream, said PTS comparator receiving the output of said PTS extractor for the primary media stream and the output of the PTS extractor of its secondary media stream an output enabler for each secondary media stream, controlled by the output of its PTS comparator for providing the synchronized secondary media stream, a PTS Generator for each synchronized secondary media stream providing the final synchronized secondary media stream with corrected PTS, and a mixer combining the decoded primary media stream and all the decoded synchronized secondary media streams to produce the final output.
The present disclosure further describes a media stream synchronizer for synchronizing one or more secondary decoded media streams to a primary decoded media stream, said media stream synchronizer comprising a Presentation Time Stamp (PTS) extractor for the decoded primary media stream, individual PTS extractors for each secondary media stream, a PTS comparator for each secondary media stream, said PTS comparator receiving the output of said PTS extractor for the primary media stream and the output of the PTS extractor of its secondary media stream an output enabler for each secondary media stream, controlled by the output of its PTS comparator for providing the synchronized secondary media stream, a PTS Generator for each synchronized secondary media stream providing the final synchronized secondary media stream with corrected PTS, and a mixer combining the decoded primary media stream and all the decoded synchronized secondary media streams to produce the final output.
This disclosure also teaches a method for synchronizing one or more secondary decoded media streams to a primary decoded media stream comprising extracting the Presentation Time Stamp (PTS) of the decoded primary media stream, extracting the individual PTS for each secondary media stream, comparing the PTS of each secondary media stream, with the PTS of the primary media stream, enabling each secondary media stream based on the result of said comparison, generating an updated PTS for each synchronized secondary media stream, and mixing the decoded primary media stream and all the decoded synchronized secondary media streams to produce the final output.
These and other features and aspects of the various embodiments of the invention will be better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings:
The embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to these embodiments. The present invention can be modified in various forms. The embodiments of the present invention described herein are only provided to explain more clearly the present invention to the ordinarily skilled in the art. In the accompanying drawings, like reference numerals are used to indicate like components.
The term PTS(n) meaning the nth input of a secondary decoded media stream has been used interchangeably with PTS(i) i.e. input of the stream and PTS, and the term PTS(M) is taken to be same as term PTSM, where M denotes the master input
The methods as described in
The number of samples required i.e. the sufficient data from a particular input is
N
i
=┌T
out
*F
i┐
Where, Tout=Output Frame Duration (Sec)
Fi=Input Sampling Frequency of ith Input (Hz)
Sufficient data ensures the proper processing and synchronization of the media streams.
The mathematical representation of conditions used in above steps is:
THRESHOLD is the allowable limit of the PTS difference which will not cause any observable synchronization problem. Ideally, the value of THRESHOLD must be as small as possible.
On the basis of the comparison, the output of the gating unit skips some amount of data, then that data is assumed to have been consumed. The obtained sufficient data is then sent for processing in the mixer after dropping. The data sent is calculated on basis of the following:—
If the data is required to be paused, the system shall output data on the basis of the following:
The PTS value on the secondary decoded media stream is then extrapolated 757 linearly. After, the synchronization and mixing has been achieved on said decoded media streams, PTS(M) is incremented as per the produced samples i.e.
PTS
M
=PTS
M+(Tout/(90*1000)); PTS is in 90 KHz ticks
If the primary decoded media stream has an PTS associated with the first sample then PTSM is updated to that value with similar updates being done on the secondary decoded media streams as well.
Thus, after mixing the output of the mixer has a single PTS associated with it and since the secondary decoded media streams are aligned within the accepted threshold of the primary decoded media stream, they are in sync.
As a further application, a presentation module will synchronize the mixed media output with a video stream of the channel in TV services wherein according to said application Fading value (FADE) is applied on the primary decoded media stream to ensure standard signal levels on the secondary decoded media streams. Panning value (PAN) is applied to place the “describer” at any preferred horizontal within the sound field. For stereo the PAN value is restricted to ±30° of the center front.
The application of FADE and PAN values are as encoded in the stream. But in the absence of the values due to corruption or loss of signal, they need to ramp up to the default value (0x00) and maintain a smooth restoration of the values from (0x00) wherein the ramp up/down should be over a period of at least 1 sec.
The ramp up/down will be done in steps as follows
But if the system gets valid values even before reaching the default values value when the valid value of the parameter is received is determined Vinitial=Vinitial+Sn*n; where n is the number of steps already being incremented.
With this new initial value as the base value and we repeat afore mentioned steps 1 and 2. But if the final value is changing within 1 sec then the following steps are taken:—
The step size on receipt of a new valid parameter is calculated as below with the assumption that a 1 sec window is applied:
S′
n(Vnew−Vcur)/Ns−n);
where
The disclosure shows and describes embodiments of the invention; however the invention is capable of use in various other combinations, modifications, and environments and is capable of changes or modifications within the scope of the inventive concept as expressed herein, commensurate with the above teachings and/or the skill or knowledge of the relevant art. The embodiments described hereinabove are further intended to explain best modes known of practicing the invention and to enable others skilled in the art to utilize the invention in such, or other, embodiments and with the various modifications required by the particular applications or uses of the invention. Accordingly, the description is not intended to limit the invention as disclosed herein. Also, it is intended that the appended claims be construed to include alternative embodiments.
Number | Date | Country | Kind |
---|---|---|---|
1667/DEL/2008 | Jul 2008 | IN | national |