Patent Grant
11805293
This application is a U.S. National Stage application under 35 U.S.C. § 371 of an International application number PCT/KR2018/003667, filed on Mar. 28, 2018, which is based on and claimed priority of a Korean patent application number 10-10-2017-0040338, filed on Mar. 29, 2017, in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference herein in its entirety.
The disclosure relates to a method and device for transmitting and receiving a signal in a multimedia system, and particularly to a method and device for transmitting and receiving a complex multimedia signal in a multimedia system.
As the demand for multimedia content increases, various multimedia schemes have been proposed. A representative multimedia scheme among them is a moving picture experts group (MPEG) media transport (MMT) scheme.
The MMT scheme is a technology for transporting and delivering coded media data associated with multimedia services via a heterogeneous packet-switched network including Internet protocol (IP) networks and digital broadcasting networks. Here, the coded media data includes timed audiovisual media data and non-timed data.
Also, the MMT scheme has been proposed to efficiently and effectively deliver media data. Particularly, regarding the MMT scheme, various schemes for transmitting and receiving complex multimedia data including various types of multimedia data and multimedia services have been proposed.
Meanwhile, the aforementioned information is disclosed only as background information for helping in understanding the disclosure. No decision and no opinion on whether any of the above content can be applied as the prior art of the disclosure have been made yet.
According to an aspect of the disclosure, a method and device for transmitting and receiving a signal in a multimedia system are provided.
According to an aspect of the disclosure, a method and device for transmitting and receiving a complex multimedia signal in a multimedia system are provided.
According to an aspect of the disclosure, when a multimedia system supports different types of data for the same content, a method and device for supporting an operation associated with a case where the type of data being reproduced is changed are provided.
According to an aspect of the disclosure, a method and device for transmitting and receiving information associated with a change of subtitles in a multimedia system are provided.
According to an aspect of the disclosure, a method and device for supporting subtitles based on multiple languages in a multimedia system are provided.
According to an aspect of the disclosure, a method and device for supporting an operation associated with a change of subtitles in a multimedia system are provided.
According to an aspect of the disclosure, a method and device for improving a user experience associated with a change of subtitles in a multimedia system are provided.
In accordance with an aspect of the disclosure, a method of a transmission device in a multimedia system is provided. The method includes: detecting a request to change from first-type data to second-type data while the first-type data is reproduced at a given point in time; and transmitting information required for replaying at least one of data reproduced before the given point in time among the first-type data and data reproduced before the given point in time among the second-type data, before the second-type data is reproduced.
In accordance with another aspect of the disclosure, a method of a reception device in a multimedia system is provided. The method includes: detecting a request to change from first-type data to second-type data while the first-type data is reproduced at a given point in time; and receiving information required for replaying at least one of data reproduced before the given point in time among the first-type data and data reproduced before the given point in time among the second-type data, before the second-type data is reproduced.
In accordance with an aspect of the disclosure, a transmission device in a multimedia system is provided. The transmission device includes: a controller configured to detect a request to change from first-type data to second-type data while the first-type data is reproduced at a given point in time; and a transmission unit configured to transmit information required for replaying at least one of data reproduced before the given point in time among the first-type data and data reproduced before the given point in time among the second-type data, before the second-type data is reproduced.
In accordance with another aspect of the disclosure, a reception device in a multimedia system is provided. The reception device includes: a controller configured to detect a request to change from first-type data to second-type data while the first-type data is reproduced at a given point in time; and a reception unit configured to receive information required for replaying at least one of data reproduced before the given point in time among the first-type data and data reproduced before the given point in time among the second-type data, before the second-type data is reproduced.
According to an embodiment, a signal can be transmitted and received in a multimedia system.
According to an embodiment, a complex multimedia signal can be transmitted and received in a multimedia system.
According to an embodiment, in the case in which an MMT system provides a multimedia streaming service including subtitle data of one or more languages, deterioration in a user experience, which may occur when a subtitle language is changed and the lengths of subtitles are different from each other, can be alleviated.
According to an embodiment, in the case in which different types of data for the same content are supported in a multimedia system, an operation performed when the type of data being reproduced is changed can be supported.
According to an embodiment, information associated with a change of subtitles can be transmitted and received in a multimedia system.
According to an embodiment, subtitles based on multiple languages can be supported in a multimedia system.
According to an embodiment, an operation associated with a change of subtitles in a multimedia system can be supported.
According to an embodiment, a user experience associated with a change of subtitles in a multimedia system can be improved.
The above and other aspects, features, and advantages, according to specific exemplary embodiments of the disclosure, will be more apparent from the following detailed descriptions taken in conjunction with the accompanying drawings, in which:
It should be noted that similar reference numerals are used to indicate identical or similar elements, features, and structures through the above figures.
The following detailed described that refers to the accompanying drawings help in comprehensively understanding various embodiments of the disclosure defined by the claims and the equivalents thereof. Although the following detailed description includes various specific concrete explanations to assist with understanding, they are considered to be only examples. Accordingly, those skilled in the art may recognize that various modifications and changes of the various embodiments described herein can be made without departing from the range and scope of the disclosure. Further, descriptions of the known functions and elements can be omitted for clarity and brevity.
The terms and words used in the following detailed description and the claims are not limited to literal meanings, and are simply used for helping obtain a clear and consistent understanding of the disclosure. Therefore, it should be apparent to those skilled in the art that the following description of various embodiments of the disclosure is provided for illustrative purposes only and is not intended to limit the disclosure that is defined by the appended claims and equivalents thereof.
Further, it will be appreciated that singular expressions such as “an” and “the” include plural expressions as well, unless the context clearly indicates otherwise. Accordingly, as an example, a “component surface” includes one or more component surfaces.
Although the terms including an ordinal number such as first, second, etc. can be used for describing various elements, the structural elements are not restricted by the terms. The terms are used merely for the purpose to distinguish an element from the other elements. For example, a first element could be termed a second element, and similarly, a second element could be also termed a first element without departing from the scope of the disclosure. As used herein, the term “and/or” includes any and all combinations of one or more associated items.
The terms used herein are used only to describe particular embodiments, and are not intended to limit the disclosure. As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise. In the disclosure, the terms such as “include” and/or “have” may be construed to denote a certain characteristic, number, step, operation, constituent element, component or a combination thereof, but may not be construed to exclude the existence of or a possibility of addition of one or more other characteristics, numbers, steps, operations, constituent elements, components or combinations thereof.
Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by those of skill in the art to which the disclosure pertains. Such terms as those defined in a generally used dictionary are to be interpreted to have the meaning equal to the contextual meaning in the relevant field of art.
According to various embodiments of the disclosure, an electronic device may include a communication functionality. The terminal may, for example, be a smart phone, a tablet Personal Computer (PC), a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook PC, a personal digital assistant (PDA), a Portable Multimedia Player (PMP), an MP3 player, a mobile medical device, a camera, or a wearable device (e.g., Head-Mounted Device (HMD), electronic clothes, an electronic bracelet, an electronic necklace, an electronic appcessory, an electronic tattoo, or a smart watch).
According to various embodiments of the disclosure, the electronic device may be a smart home appliance with a communication functionality. The smart home appliance may, for example, be a television, a Digital Video Disk (DVD) player, an audio player, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave, a washer, a drier, an air purifier, a set-top box, a TV box (e.g., Samsung HomeSync™, Apple TV™, or Google TV™), a gaming console, an electronic dictionary, a camcorder, or an electronic photo frame.
According to various embodiments of the disclosure, the terminal may be a medical appliance (e.g., Magnetic Resonance Angiography (MRA) device, Magnetic Resonance Imaging (MRI) device, Computed Tomography (CT) device, and ultrasonic device), a navigation device, a Global Positioning System (GPS) receiver, an Event Data Recorder (EDR), a Flight Data Recorder (FDR), an automotive infotainment device, a marine electronic device (e.g., ship navigation device and a gyrocompass), avionics, security equipment, or an industrial or home robot.
According to various embodiments of the disclosure, the electronic device may be a part of furniture or a building/structure, an electronic board, an electronic signature receiving device, a projector, and various kinds of measuring instruments (e.g., water meter, electric meter, gas meter, and electromagnetic wave meter), each of which has a communication functionality.
According to various embodiments of the disclosure, the electronic device may be a combination of the above-mentioned devices. Further, it will be apparent to those skilled in the art that the terminal according to various embodiments of the disclosure is not limited to the above-mentioned devices.
According to various embodiments, each of a transmission device and a reception device may be, for example, an electronic device. According to various embodiments, the transmission device may be referred to as a transmission entity, and the reception device may be referred to as a reception entity.
Also, according to various embodiments, the transmission device may be a service provider or a terminal.
Also, according to various embodiments, the reception device may be a terminal or a service provider.
Also, according to various embodiments, the term “terminal” may be interchangeable with terms, such as a mobile station (MS, hereinafter referred to as a MS), a wireless terminal, a mobile device, a user equipment (UE, hereinafter referred to as a UE), and the like. Also, according to various embodiments, the term “service provider” may be interchangeable with terms such as “broadcasting server” and the like.
According to an aspect of the disclosure, a method and device for transmitting and receiving a signal in a multimedia system are provided.
According to an aspect of the disclosure, a method and device for transmitting and receiving a complex multimedia signal in a multimedia system are provided.
According to an aspect of the disclosure, when a multimedia system supports different types of data for the same content, a method and device for supporting an operation associated with the case in which the type of data being reproduced is changed are provided.
According to an aspect of the disclosure, a method and device for transmitting and receiving information associated with a change of subtitles in a multimedia system are provided.
According to an aspect of the disclosure, a method and device for supporting subtitles based on multiple languages in a multimedia system are provided.
According to an aspect of the disclosure, a method and device for supporting an operation associated with a change of subtitles in a multimedia system are provided.
According to an aspect of the disclosure, a method and device for improving a user experience associated with a change of subtitles in a multimedia system are provided.
A method and device proposed in an embodiment may be applicable to various communication systems, for example, a digital video broadcasting system such as an Internet protocol television (IPTV, hereinafter referred to as “IPTV”) service and a mobile broadcasting service such as a digital multimedia broadcasting (DMB, hereinafter referred to as “DMB”) service, a digital video broadcasting-handheld (DVB-H, hereinafter referred to as “DVB-H”) service, an advanced television systems committee-mobile/handheld (ATSC-M/H, hereinafter referred to as “ATSC-M/H”) service, a moving picture experts group (MPEG) media transport (MMT, hereinafter referred to as “MMT”) system, an evolved packet system (EPS, hereinafter referred to as “EPS”), a long-term evolution (LTE, hereinafter referred to as “LTE”) mobile communication system, a long-term evolution-advanced (LTE-A, hereinafter referred to as “LTE-A”) mobile communication system, a high-speed downlink packet access (HSDPA, hereinafter referred to as “HSDPA”) mobile communication system, a high-speed uplink packet access (HSUPA, hereinafter referred to as “HSUPA”) mobile communication system, a high rate packet data (HRPD, referred to as “HRPD”) mobile communication system of a 3rd-generation partnership project 2 (3GPP2, hereinafter referred to as “3GPP2”), a wideband code division multiple access (WCDMA, hereinafter referred to as “WCDMA”) mobile communication system of 3GPP2, a code division multiple access (CDMA, hereinafter referred to as “CDMA”) mobile communication system of 3GPP2, an institute of electrical and electronics engineers (IEEE, hereinafter referred to as “IEEE”) 802.16m communication system, a mobile internet protocol (Mobile IP, hereinafter referred to as “Mobile IP”) system, and the like.
Also, a method and device proposed in an embodiment may be applicable to an advanced television systems committee (ATSC, hereinafter referred to as “ATSC”) system based on an MMT scheme.
Hereinafter, for ease of description, in various embodiments, it is assumed that the multimedia system is a multimedia system based on an MMT scheme.
First, terms used in various embodiments will be described as follows.
First, multimedia data will be described.
The multimedia data may include one or more media components. The media components may be, for example, video files, audio files, subtitles, and the like. Here, the subtitles may include captions, which are text data obtained by converting the content of audio data into text data to help a user who has difficulty hearing, subtitles which express the corresponding content, for example, a dialogue between persons in a movie or the like, as text of a language, which may be the same as or different from the language of the corresponding content, and the like. For example, for a movie produced in English, various types of subtitle data may be generated, such as English subtitles, Korean subtitles, Japanese subtitles, English captions, and the like.
Second, data streaming will be described.
Multimedia data may be transmitted from a transmission device to a reception device via a network such as a mobile communication network or the like, and upon the transmission, the multimedia data may be reproduced by the reception device via a series of processing processes, which is multimedia streaming. The multimedia data is fragmented in units defined according to a transport protocol and a transport format used for transmission, and the fragmented units are transmitted. When a single piece of multimedia data is transmitted, media components included in the multimedia data and metadata which describes the relationship among the media components are transmitted together according to regulations defined in the transport protocol and the transport format. Therefore, even though the reception device does not yet receive the entirety of the multimedia data, the reception device may reproduce a part of the multimedia data, of which transmission is completed, on the basis of the metadata.
For example, when the multimedia data includes subtitle data produced in one or more languages, for example, multiple languages, the length of subtitle data may differ depending on the linguistic characteristics of the corresponding language even though the language describes the same content. In this instance, the time at which the corresponding part, that is, the corresponding subtitle data, is to be reproduced is also different. For example, when a subtitle language for a scene is Korean, the length of subtitle data may be 30 words. When the subtitle language is English, the length of subtitle data may be 20 words. Accordingly, the length of subtitle data is different for each subtitle language, and the presentation start time, presentation end time, and presentation duration of the subtitle data are also different. Here, the presentation start time may be also expressed as “presentation begin time” or “presentation start time”.
Therefore, while multimedia data including subtitle data produced in one or more languages, for example, multiple languages, is reproduced, when a predetermined event, such as selection by a user of a terminal or another cause, occurs, and the subtitle language, that is, the language of the subtitle data, is changed from a first language to a second language, the subtitle data may be abnormally reproduced since the length of subtitle data is different. Hereinafter, for ease of description, a subtitle written in a first language is referred to as a “first subtitle”, and a subtitle written in a second language is referred to as a “second subtitle”.
In the state in which a first subtitle, which is presented for a scene, for example, a first scene, is changed to a second subtitle, if the presentation of the second subtitle for the first scene is finished earlier than the presentation of the first subtitle, a user of a terminal may not completely read any of the first subtitle and the second subtitle, but may wait until a second subtitle for a second scene subsequent to the first scene is presented.
This may be a factor that deteriorates a user experience in a multimedia data streaming service including subtitle data produced in multiple languages.
Therefore, in the case in which a multimedia streaming service including subtitle data produced in one or more languages, for example, multiple languages, is provided in a multimedia system, an embodiment of the disclosure provides a method and device for reducing the deterioration in user experience caused by a change in the length of subtitles when the language used for generating subtitle data, that is, the subtitle language, is changed.
First, terms used in an embodiment will be described as follows.
First, an MMT data model in an MMT system according to an embodiment will be described with reference to
Referring to
Also, an asset is a set of media data included in a single multimedia presentation, and an asset may include one or more media-processing units (MPU) having the same asset identifier (ID). Each of the MPU included in a single asset may include fragments of the asset, which do not overlap each other. That is, two successive and different MPUs do not include the same media sample.
Also, an MPU indicates a unit of media data that may be consumed independently in a presentation engine of an MMT reception device.
The process in which a single MMT device processes a single MPU may include encapsulation or de-capsulation of media data in units of MPUs and packetization or de-packetization.
Also, a single MPU may be fragmented in units of MPU fragments. Accordingly, one MPU may be fragmented into one or more MPU fragments. Here, an MPU fragment may be, for example, a media fragment unit (MFU).
Also, a single MPU may be fragmented in units of data units for packetization. Therefore, a single MPU may be fragmented into one or more data units.
A single MPU may be specified by a single asset ID corresponding to the data included in the MPU and a single MPU sequence number. For example, two different MPUs included in a single asset may have the same asset ID, but different MPU sequence numbers from each other.
Also, media data included in an MPU may include timed media data which is related to time and non-timed media data which is irrelevant to time. When the media data includes timed media data, presentation information that explains the relationship among different assets of one package on the time axis needs to be provided together with the timed media data. For all MPUs, presentation time information may be provided as a part of the presentation information, which will be described with reference to
Referring to
Second, an MPU based on an ISO Base Media File Format of an MMT system according to an embodiment will be described.
First, ISO Base Media File Format (ISO BMFF) indicates the ISO/IEC 14496-12 standard. A single MPU may need to be compatible with an ISOBMFF file. The MPU sequence number and asset ID of an MPU is provided from an ‘mmpu’ box, and the MPU sequence number and the asset ID may be elements to specify a corresponding MPU, among MPUs encapsulated in a file, as a unique MPU.
Also, in the case of timed media, a single “sidx” box may exist, which is an index of movie fragments of an MPU. A “moov” box needs to contain all codex configuration information required for decoding and reproducing media data.
Also, timed media is stored as a track of ISOBMFF. Here, a single media track is allowed for the timed media data. Also, non-timed media is stored as a part of metadata of ISOBMFF.
An example of MMT encapsulation in an MMT system according to an embodiment will be described with reference to
Referring to
Third, the configuration of multimedia data in an MMT system according to an embodiment will be described.
First, when a piece of multimedia content is provided to a terminal, the multimedia content may include video data, audio data, and subtitle data. Particularly, subtitles may be provided as a plurality of pieces of subtitle data based on multiple languages. For example, media components included in a piece of multimedia data may include a single piece of video data, a single piece of audio data, and two pieces of subtitle data. The two pieces of subtitle data may include English subtitle data and Korean subtitle data.
Each media component may be stored in the form of a track defined in ISO Base Media File Format (ISO/IEC 14496-12), and the track may be defined as an asset in an MMT system.
Therefore, the multimedia data including the video data, audio data, English subtitle data, and Korean subtitle data may be produced as a video asset, an audio asset, an English subtitle asset, and a Korean subtitle asset in the MMT system, and each asset may be fragmented into one or more MPUs.
In the case of a subtitle asset, on the assumption that a set of subtitle data expressed in one scene is a subtitle paragraph (hereinafter referred to as a “paragraph” for ease of description), data corresponding to a single paragraph may be included in one MPU.
Fourth, signaling information in an MMT system according to an embodiment will be described.
In the MMT system, a transmission device and a reception device may perform transmission and reception of signaling information required for MMT packet delivery and MMT packet consumption.
The MMT signaling information is transmitted in the form of an MMT signaling message, and the common format of MMT signaling messages may be expressed as shown in Table 1 below.
Table 1 indicates the syntax of the normal format of MMT signaling messages, and the semantics of parameters included in Table 1 may be expressed as shown in Table 2.
In Table 2, Table 61 indicates Table 34 of ISO/IEC 23008-1:2014(E) 1st Edition.
In Table 2, the message_id field indicates the identifier of the signaling message. Valid message identifier values are listed in Table 34 of ISO/IEC 23008-1:2014(E) 1st Edition, and the field values of the message_id field may have the following meaning.
If the field value of the message_id field is, for example, 0x0000, this indicates a package access (PA) message. If the field value of the message_id field is, for example, any one value in the range of 0x0001 to 0x000F, this indicates a media presentation information (MPI) message. For a package, 16 adjacent values are allocated to MPI messages. If (the field value of the message_id field) % 16 is equal to 15, the MPI message delivers complete presentation information (PI). If (the field value of the message_id field) % 16 is equal to N, the MPI message delivers a subset (subset-N) MPI. Here, N is a value in the range of 0 to 14.
Also, if the field value of the message_id field is, for example, any one value in the range of 0x0010 to 0x001F, this indicates an MMT package table (MPT) message. For a package, 16 adjacent values are allocated to MPT messages. If (the field value of the message_id field) % 16 is equal to 15, the MPT message delivers a complete MPT. If (the field value of the message_id field) % 16 is equal to N, the MPT message delivers a subset (subset-N) MPT. Here, N is a value in the range of 0 to 14.
Also, if the field value of the message_id field is, for example, 0x0200, this indicates clock relation information (CRI) messages. If the field value of the message_id field is, for example, 0x0201, this indicates device capability information (DCI) messages. If the field value of the message_id field is, for example, 0x0202, this indicates application layer forward error correction (AL-FEC) messages. If the field value of the message_id field is, for example, 0x0203, this indicates hypothetical receiver buffer model (HRBM) messages. If the field value of the message_id field is, for example, any one value in the range of 0x0204 to 0x7FFF, this indicates reservation for the purpose of the International Organization for Standardization (ISO). If the field value of the message_id field is, for example, any one value in the range of 0x8000 to 0xFFFF, this indicates reservation for private purposes.
In Table 2, the version field indicates the version of the signaling message. Both an MMT transmission entity and an MMT reception entity may verify whether or not a received message has a new version.
In Table 2, the length field indicates the length of the signaling message. The lengths of signaling messages, excluding a PA message and an MPI message, may be, for example, 2 bytes. The lengths of the PA message and the MPI message may be 4 bytes, since it is occasionally impossible to express the length of an MPI table in a length field of 2 bytes. Here, it should be understood that the PA message includes at least one MPI table.
In Table 2, the extension field includes extension information for signaling messages that require extension. The content and length of the extension field may be specified for each corresponding signaling message.
In Table 2, the message_payload field includes the payload of the signaling message. The format of the message_payload field may be identified by the value of the message_id field. Fifth, an MMT package (MP) table in an MMT system according to an embodiment will be described.
Information required for transmitting and consuming an MMT package may be stored in the form of a table for each role, and the required tables may be transmitted from an MMT transmission device to an MMT reception device via an MMT signaling message.
The list of all assets included in a single MMT package is stored in an MP table, and may be transmitted via a package access (PA) message. The MP table may include time-related information of all assets included in the MMT package, the location of data, and related descriptors required for consuming assets.
Here, the MP table may be expressed as shown in Table 3. The MP table may be implemented as a complete MP table or a subset MP table. The complete MP table may include information related to the package including all assets, and the subset MP table may include a part of information stored in the complete MP table. Also, MP table subset-0 may include the minimum information required for consuming the package.
Table 3 indicates the syntax of the MP table, and the semantics of parameters included in Table 3 may be expressed as shown in Table 4.
In Table 4, the table_id field indicates the identifier of the MP table. The complete MP table and each subset MP table may use different table identifiers. An MP table subset number may be implicitly represented by the table_id field. Since the table_id values are assigned contiguously, the MP table subset number may be deduced from the table_id field. That is, the MP table subset number is equal to a value obtained by subtracting the field value of the table_id field of a base MP table from the field value of the table_id field. The MP table subset number provides the subset number of a corresponding MP table. The number “0” indicates a base MP table. The numbers “1” to “14” indicate MP table subsets. The number “15” has a special meaning since it indicates a complete MP table.
In Table 4, the version field indicates the version of the MP table. Upon reception of the version field, a newer version immediately overrides an old version. If the table_id field indicates a complete MP table, if a subset-0 MP table has the same version value as the version field (when the field value of the MP_table_mode field is “1”), if all MP table subsets having lower subset numbers have the same version value as the version field (when the field value of the MP_table_mode field is “0”), or if processing of the MP table subsets is independent (when the field value of the MP_table_mode field is “2”), if the subset-0 MP table has a newer version, all MP table subsets with higher subset numbers up to 14, which are stored in advance in the MMT reception entity, are considered outdated, except for the case in which the MP_table_mode is an independent mode. When the MP table subset number is not 0 and the MP_table_mode is “1”, the content of the MP table subset with a version different from that of the subset-0 MP table stored in the MMT reception entity may be disregarded. When the MP table subset number is not 0 and the MP_table_mode is “0”, the content of the MP table subset with a version different from those of lower-subset MP table subsets stored in the MMT reception entity may be disregarded. The field value of the version field may be increased by modulo-256 every time that a version is changed.
In Table 4, the length field indicates the length of the MP table in bytes, and bytes from the start of a next field to the last byte of the MP table are counted. The value “0” is invalid for the length field.
In Table 4, the MP_table_mode field indicates the mode of an MP table processing when an MP table subset mechanism is used. In a “sequential_order_processing_mode”, using a nonzero subset number of the corresponding MP table, an MMT reception entity may receive all MP table subsets that have lower subset numbers and have the same version as that of the corresponding MP table subset before the MMT reception entity processes the corresponding MP table subset. For example, if the MMT reception entity does not receive a subset-2 MP table having the same version, the MMT reception entity is incapable of processing a subset-3 MP table. In “order_irrelevant_processing_mode”, using the subset number of the corresponding MP table subset, which is set to a nonzero value, the MMT reception entity may need to process the MP table subset immediately after receiving the MP table subset, as far as the subset-0 MP table stored in the MMT reception entity has the same version as that of the corresponding MP table subset. In “independent_processing_mode”, the version of each MP table subset is managed individually. A fragmented MP table in which each MP table subset is delivered by one of a plurality of MMT transmission entities is adapted in the “independent_processing_mode”. The independent mode of subsets of the MP table may be used for multi-channel instantiation. That is, the MP table subsets from subset-0 MP table to subset-N MP table are assigned as logical channels from channel-0 (Ch-0) to channel N (Ch-N). When an MPI message delivers both an MPI table subset and an associated MP table subset, the PI mode in the MPI table and the MP_table_mode in the MP table may have the same value.
Also, the MP_table_mode of Table 3 has the meanings according to the values shown in Table 5.
Also, each parameter included in Table 3 has the meaning shown in Table 6.
Also, in Table 6, sections, for example, sections 9.5.1, 9.6.1, and 9.6.2 indicate sections of the ISO/IEC 23008-1:2014(E) 1st Edition standard.
In Table 6, the MMT_package_id field indicates the unique identifier of the corresponding package. The MMT_package_id_length field indicates the length of an MMT_package_id string, excluding a terminating null character, in bytes. The MMT_package_id_byte field indicates a byte of the MMT_package_id. When MMT_package_id_byte field is a string, the terminating null character is not included in the string. In Table 6, asset_id indicates an asset identifier as defined in the section 9.6.2 of the ISO/IEC 23008-1:2014(E) 1st Edition standard, and a detailed description thereof will be omitted. In Table 6, the asset_type field provides the type of the asset, and the asset_type field indicates a four character code (4CC) type registered with MP4REG (http://www.mp4ra.org).
In Table 6, the MP_table_descriptors field provides descriptors for the MP table, and the MP_table_descriptors_length field includes the length of a descriptor syntax loop. The length is counted from the next field to the last of the descriptor syntax loop. Some descriptors may be inserted into the corresponding syntax loop. For example, an additional_package information URL descriptor may be used, which may provide the URL of a package information webpage for the corresponding package.
In Table 6, the MP_table_descriptors_byte field may indicate one byte in the descriptors loop. The number_of_assets field provides the number of assets of which the information is provided by the corresponding MP table. The packet_id field provides the identifier of an MMTP session in an MMTP packet header. The asset_clock_relation_flag field indicates whether an asset uses a network time protocol (NTP) clock or another clock system as a clock reference. If the field value of the asset_clock_relation_flag field is “1”, an asset_clock_relation_id field is included. If the field value of the asset_clock_relation_flag is “0”, the NTP clock is used for the asset.
In Table 6, the asset_clock_relation_id field provides the clock relation identifier of the asset. The asset_clock_relation_id field is used to make reference to the clock relationship delivered by a CRI_descriptor( ) for the asset. The field value of the asset_clock_relation_id field is one of the clock_relation_id values provided by the CRI descriptors. Here, the CRI descriptor is described in section 9.5.1 of the ISO/IEC 23008-1:2014(E) 1st Edition standard, and a detailed description thereof will be omitted.
In Table 6, the asset_timescale_flag field indicates whether or not “asset_timescale” information is provided. If the field value of the asset_timescale_flag field is set to “1”, asset_timescale information is included. If the field value of the asset_timescale_flag field is set to “0”, asset_timescale information is 90,000 (90 kHz).
In Table 6, the location_count field provides the number of pieces of location information associated with an asset. When an asset is delivered via one location, the field value of the location_count field is set to “1”. When bulk delivery is performed, in which MPUs included in an asset are delivered via multiple channels, the field value of the location_count field is not set to “1”. When one asset is delivered via multiple locations, the MMT reception entity may receive all MPUs of the asset from all indicated locations.
In Table 6, the asset_timescale field provides time unit information for all timestamps used for the asset expressed as the number of units within one second.
In Table 6, the MMT_general_location_info_for_asset_location field provides location information of an asset. Here, general location reference information for an asset may be used, which is described in section 9.6.1 of the ISO/IEC 23008-1:2014(E) 1st Edition standard. The value of the location_type field, which is between ‘0x00’ and ‘0x06’, may be used for an asset location.
In Table 6, the asset_descriptors_length field indicates the number of bytes counted from the start of the next field to the end of the Asset descriptors syntax loop. The asset_descriptors_byte field indicates a byte in asset descriptors. The default_asset_flag field indicates whether an asset is marked as a default asset. If an asset is marked as a default asset, an MPU timestamp descriptor may exist for a corresponding timed asset. If the field value of the default_asset_flag field is “0”, the asset may be marked as a default asset.
Also, asset_id will be described as follows.
First, an asset_id syntax element group may be used for providing an asset ID. If an “mmpu” box exists, the values in the asset_id syntax element group may be the same as the asset ID of the “mmpu” box. A description of allocation of an asset ID when the “mmpu” box does not exist will be omitted.
Here, the asset_id syntax may be as shown in Table 7.
The meaning of each parameter included in Table 7 may be as shown in Table 8.
Also, in Table 8, a section, for example, section 6.3.3, indicates a section of the ISO/IEC 23008-1:2014 1st Edition standard.
In Table 8, the asset_id_scheme field provides an asset ID scheme as defined in section 6.3.3 of the ISO/IEC 23008-1; 2014 1st Edition standard. The asset_id_length field indicates the length of the asset_id field in bytes, and the asset_id_byte field indicates a byte in the asset_id.
Also, identifier mapping will be described as follows.
First, an identifier mapping syntax group element provides mapping between a content identifier and an MMTP packet sub-flow, and the MMTP packet sub-flow is a subset of MMTP packets that share the same packet identifier packet_id. The content identifier may be provided in other forms, such as an asset ID, a URL, or a pattern.
Here, the syntax of Identifier_mapping( ) may be as shown in Table 9.
The meaning of each parameter included in Table 9 may be as shown in Table 10.
In Table 10, “Table 6” indicates Table 11 provided below, and the identifier_type field provides the type of identifier used for the mapping to the packet_id field. The list of available identifier types is provided in Table 11 below.
In Table 10, the identifier types for identifier mapping may be as shown in Table 11 below.
Also, a section in Table 11, for example, section 9.6.2, indicates a section of the ISO/IEC 23008-1:2014(E) 1st Edition standard. Therefore, detailed descriptions thereof will be omitted. Also, the meaning of each parameter included in Table 9 may be as shown in Table 12 below.
In Table 12, the URL is the list of URLs. The field value of the URL_count field indicates the number of URLs provided in the list of URLs. In Table 12, the URL_length field indicates the length of the following URL in bytes. The URL byte field indicates a byte of the URL, which is formatted as a UTF-8 character string. In the regex_length field, the identifier is provided as a regular expression that matches a set of URLs, and the regex_length field indicates the length of the RegEx string.
In Table 12, the regex_byte field indicates a byte of the RegEx string provided as a UTF-8 string. In the representation_id_length field, the identifier is provided as a DASH Representation@id. The field value of the representation_id_length field indicates the length of the DASH Representation@id string.
In Table 12, the representation_id_byte field indicates a byte of the Representation@id string. In the private_length field, the identifier is provided as private data. The private_length field provides the length of the private identifier in bytes, and the private_byte field indicates a byte of the private identifier.
Sixth, an MPU timestamp descriptor of an MMT system according to an embodiment will be described.
Presentation time information is transmitted for each MPU of each asset. The presentation time information may be included in an MP table that describes the characteristics of each asset. In the MMT system, the presentation start time of each MPU is described in an MPU timestamp descriptor, and the MPU timestamp descriptor is included in the MP table. Therefore, MPUs in each asset may be synchronized with each other on the presentation time axis.
The MPU timestamp descriptor may be as shown in Table 13 below. (This descriptor provides presentation time of the first AU of MPU in presentation order after application of any offset such as the one provided by ‘elst’ box. When presentation information presents this descriptor shall be ignored.)
Here, the meaning of each parameter included in Table 13 may be as shown in Table 14 below.
In Table 14, the descriptor_tag field indicates a tag value indicating the type of a descriptor. The descriptor_length field indicates the length counted from a byte subsequent to the descriptor_length field to the last byte of the descriptor, in bytes. In Table 14, the mpu_sequence_number field indicates the sequence number of an MPU reproduced at a subsequent mpu_presentation_time. The mpu_presentation_time field indicates the presentation time of a first access unit (AU) in a designated MPU, on the basis of a 64-bit NTP timestamp format.
An example of the internal structure of an MMT transmission device in an MMT system according to an embodiment will be described with reference to
Referring to
When a request to change from a currently transmitted first subtitle to a second subtitle is received from an MMT reception device or another device that communicates with the MMT transmission device 100, the subtitle change indication unit 101 may correct signaling information that describes MMT package and MMT data that are currently transmitted and may deliver information required for performing configuration to change subtitles (subtitle change configuration) to the MMT data transmission unit 102.
When the first subtitle and the second subtitle are included in different assets, an asset ID of a subtitle asset needs to be changed in an MP table, and an MPU timestamp descriptor including the presentation time information of each MPU, which is included in the asset to which the second subtitle belongs, needs to be transmitted.
In the case of changing the first subtitle to the second subtitle, a method of determining a transmission termination part of the first subtitle data and a transmission start part of the second subtitle data, and correcting related information to be suitable for a determined time, will be described in detail with reference to an operation of performing configuration to change subtitles. Detailed descriptions thereof will be omitted.
The information required for changing subtitles may be recorded in a subtitle change descriptor, and the subtitle change descriptor may be transmitted via an MMT signaling message. This will be described in subsequent descriptions related to a subtitle change descriptor, and a detailed descriptions thereof will be omitted here.
Also, the MMT data transmission unit 102 may transmit, to the MMT reception device, a subtitle change descriptor in which the information required for performing configuration to change subtitles is recorded, corrected signaling information, and MMT data, which are received from the subtle change indication unit 101.
In
Referring to
The media data output unit 203 may physically exist inside the MMT reception device 200 or may exist outside the MMT reception device 200.
The MMT data reception unit 201 may receive MMT data transmitted from the MMT transmission device. The MMT data may include media data included in an MMT package, a signaling message, related data, and the like.
The subtitle change configuration unit 202 may generate information required for reproducing subtitle data and may transmit the same to the media data output unit 203 using a subtitle change descriptor transmitted from the MMT transmission device so that a first subtitle is presented until a determined time and a second subtitle is presented from the determined time. Here, the method of generating the information required for reproducing the subtitle data will be described with reference to an operation for performing configuration to change subtitles and detailed descriptions thereof will be omitted here.
The media data output unit 203 may combine the information required for reproducing subtitle data transferred from the subtitle change configuration unit 202 with the media data and may output multimedia content.
Although
In
Also, as illustrated in
In this instance, until the start of Paragraph 2 of the second subtitle, Paragraph 1 of the first subtitle and/or Paragraph 1 of the second subtitle may be replayed, so that the user of the terminal sufficiently experiences the context provided by the subtitle data. That is, previous subtitle data which has already passed is replayed during the empty period between the last paragraph of the first subtitle and the first paragraph of the second subtitle so that the user of the terminal may sufficiently experience the context provided by the subtitle data.
In order to replay previous subtitle data during the empty period between the first subtitle and the second subtitle, that is, during a period of time during which no subtitles are provided due to changing from the first subtitle to the second subtitle, (i) a duration (Duration reset) during which subtitle data is to be replayed and (ii) subtitle data to be replayed need to be determined.
Therefore, an MMT transmission device needs to transmit a subtitle change descriptor to an MMT reception device so that the MMT reception device determines the duration during which the subtitle data is to be replayed and the subtitle data to be replayed.
The subtitle change descriptor may include (1) a presentation start time (MPUPT_new_passed) of Paragraph 1 of the second subtitle, (2) a presentation start time (MPUPT_new_next) of Paragraph 2 of the second subtitle, (3) a presentation duration (duration_original_passed) of Paragraph 1 of the first subtitle, and (4) a presentation duration (duration_new_passed) of Paragraph 1 of the second subtitle.
Here, the subtitle change descriptor may be expressed as shown in Tables 15 and 16. Tables 15 and 16 are described based on the case in which a single paragraph is stored in a single MPU. However, as described above, a single paragraph may be included in a single MPU, or multiple paragraphs may be included in a single MPU.
First, according to the MMT standard, a single package may correspond to a single program, for example, a news show, a soccer game, or the like. The single package may include subtitles in multiple languages, for example, English subtitles, Korean subtitles, and the like, together with video and audio data. In this instance, subtitle data of each language may be mapped to a single asset according to the MMT standard. Subtitles of each language may include presentation start time information and presentation end time information. Although subtitles of each language are produced for the same content, the length of subtitles of each language may differ depending on the characteristics of the corresponding language. Also, changing from subtitles in a predetermined language to subtitles in another language may be performed even during a transmission or rendering process. No information may be displayed on a screen for a predetermined duration because the time at which to perform changing has not reached the presentation start time of the new language subtitles. In this instance, the subtitles before changing, that is, the original subtitles, may be displayed instead, so that the user experience may be improved.
To this end, a server needs to describe related information in a subtitle change descriptor, and may transmit the subtitle change descriptor to the terminal. The subtitle change descriptor may include information associated with an MPU presentation time and duration for each subtitle MPU. The terminal may determine a subtitle asset to be displayed and the period of time during which the subtitle asset is to be displayed during an empty period (space period) before the presentation start time of subsequent subtitles on the basis of the information associated with the MPU presentation time and duration.
Table 16 represents a subtitle change descriptor, that is, information that a server, that is, a subtitle server, transmits to the terminal so that the terminal determines the subtitles to be displayed, and the period of time during which the subtitles are to be displayed, according to an embodiment. Here, the meaning of each parameter included in Table 16 may be as shown in Table 17.
Also, in Table 17, a section, for example, section 9.5.2, indicates a section of the ISO/IEC 23008-1:2014(E) 1st Edition standard.
In Table 17, the descriptor_tag field indicates a tag value indicating the type of a descriptor. The descriptor_length field indicates the length counted from a byte subsequent to the descriptor_length field to the last byte of the descriptor in bytes. The MPUPT_new_passed field indicates the mpu_presentation_time of a passed MPU which is included in a changed subtitle asset. The mpu_presentation_time is defined in section 9.5.2 of the ISO/IEC 23008-1:2014(E) 1st Edition standard.
In Table 17, the MPUPT_new_next field indicates the mpu_presentation_time of a subsequent MPU in the changed subtitle asset. The mpu_presentation_time is defined in section 9.5.2 of the ISO/IEC 23008-1:2014(E) 1st Edition standard.
In Table 17, the duration_original_passed field indicates the presentation time duration of a passed MPU in the original subtitle asset, and the duration_original_passed field is expressed in milliseconds.
In Table 17, the duration_new_passed field indicates the presentation time duration of a passed MPU in the changed subtitle asset, and the duration_new_passed field is expressed in milliseconds.
A subtitle descriptor is written as a subtitle change descriptor and may be included in an MP table as one of the asset descriptors of a subtitle asset. The MP table may be included in a PA message and may be transmitted from the MMT transmission device to the MMT reception device every time that information associated with assets included in the MMT package is changed.
Also, the subtitle descriptor may be transmitted independently from the MMT transmission device to the MMT reception device via a separate MMT signaling message, as opposed to the PA message. The signaling message including the subtitle descriptor may be, for example, a subtitle change message. The format of the subtitle change message may comply with the above-described MMT signaling information format.
Subsequently, a method of performing configuration to change subtitles in an MMT system according to an embodiment will be described with reference to
Referring to
Configuration to change subtitles is performed as follows.
1. An MMT reception device receives a subtitle change descriptor transmitted from a server.
2. The MMT reception device calculates the empty period between Paragraph 1 of the second subtitle and Paragraph 2 of the second subtitle on the basis of (1), (2), and (4).
3. The MMT reception device determines a paragraph to be replayed until presentation of Paragraph 2 of the second subtitle begins, and a duration (Duration_rest) during which the paragraph is to be replayed, by taking into consideration the period calculated in #2 and the following items, that is, (1), (2), (3), (4), and (5).
Here, Duration_rest is the duration (=MPUPT_new_next) from the point in time at which changing subtitles is attempted to the point in time at which presentation of Paragraph 2 of the second subtitle begins.
In the existing MMT system, subtitles are not displayed on a screen during Duration_rest. However, according to an embodiment, subtitles may be displayed during Duration_rest in order to improve the user experience.
Therefore, a paragraph to be replayed during Duration_rest may be Paragraph 1 of the first subtitle, Paragraph 1 of the second subtitle, or both Paragraph 1 of the first subtitle and Paragraph 1 of the second subtitle. If both Paragraph 1 of the first subtitle and Paragraph 1 of the second subtitle are replayed, a duration for replaying Paragraph 1 of the first subtitle and a duration for replaying Paragraph 1 of the second subtitle may be determined to be the same as or different from each other. Here, a duration including both the duration for replaying Paragraph 1 of the first subtitle and the duration for replaying Paragraph 1 of the second subtitle may correspond to Duration_rest.
4. The MMT reception device determines a replay mode of a paragraph to be replayed during Duration_rest determined in #3.
5. The MMT reception device may adjust the MPU presentation time (MPU_presentation_time) of the paragraph to be replayed on the basis of the replay mode determined in #4.
6. In the case of the trick replay mode, the replay time of data included in the paragraph to be replayed is adjusted on the basis of the replay mode determined in 4. For example, when the determined paragraph includes N media samples, the replay duration of each media sample may be adjusted to be the replay duration of the determined paragraph χDuration_restχN. Here, the replay duration of the media sample may be included in MPU metadata.
7. Information and data determined in #2 to #6, along with related media data, are delivered to the renderer.
As described above, two or more paragraphs may be included in a single MPU. In this instance, instead of an MPU presentation time, the presentation start time of a first data unit of data included in a paragraph may be used.
Also, in the process of determining Duration_rest and the replay mode, a predetermined empty period may be provided before the start of Paragraph 2 of the second subtitle in consideration of the ratio of the empty period between Paragraph 1 and Paragraph 2 of the second subtitle to Duration_rest. In this instance, Duration_rest and the MPU_presentation_time of the determined paragraph additionally need to be adjusted.
An embodiment is based on the case of using an empty period, that is, a period of time until Paragraph 2 of a second subtitle is presented when a subtitle language is changed from a first language to the second language while Paragraph 1 of the first subtitle is being presented. However, video data is changed to a subsequent scene during the empty period. If previous subtitles for a previous scene are replayed, the subtitles may be quite different from the intention of the content and the user experience may deteriorate. Accordingly, depending on the decision made by the MMT reception device, the method according to an embodiment of the disclosure may not be performed for some portions of the entire content. That is, the MMT reception device may selectively apply the method according to an embodiment of the disclosure as the occasion demands. The MMT reception device makes decisions in consideration of context recognition technology for recognizing the context of content, user preference information, the age of a user, the speed at which a user reads words, the size of a display, the resolution of the display, the field of view of the display, spatial information of the display, a region of interest, or the like.
All the above-described operations for changing subtitles may be performed by the MMT reception device or may be performed by the MMT transmission device. If all operations for changing subtitles are performed in the MMT transmission device, the MMT reception device may perform normal operations for receiving and reproducing MMT data.
In
Referring to
First, the controller 813 may control the overall operation of the MMT transmission device 800. The controller 813 may control the MMT transmission device 800 to perform overall operation related to an operation of changing subtitles according to an embodiment. Here, the overall operation related to the operation of changing subtitles is the same as described with reference to
The transmission unit 811 may transmit various types of data, various signals, and various messages to an MMT reception device or the like under the control of the controller 813. Here, the various types of data, various signals, and various messages that the transmission unit 811 transmits are the same as those of
Also, the reception unit 815 may receive various types of data, various signals, and various messages from the MMT reception device or the like under the control of the controller 813. Here, the various types of data, various signals, and various messages that the reception unit 815 receives are the same as those of
The storage unit 817 may store programs and various types of data required for the operation of the MMT transmission device 800, particularly, information related to an operation of changing subtitles according to an embodiment. Also, the storage unit 817 may store various types of data, various signals, and various messages that the reception unit 815 receives from the MMT reception device or the like.
Although
In
Referring to
First, the controller 913 may control the overall operation of the MMT reception device 900. The controller 913 may control the MMT reception device 900 to perform overall operation related to an operation for changing subtitles according to an embodiment. Here, the overall operation related to the operation of changing subtitles is the same as that described with reference to
The transmission unit 911 may transmit various types of data, various signals, and various messages to an MMT transmission device or the like under the control of the controller 913. Here, the various types of data, various signals, and various messages that the transmission unit 911 transmits are the same as those of
Also, the reception unit 915 may receive various types of data, various signals, and various messages from the MMT transmission device or the like under the control of the controller 913. Here, the various types of data, various signals, and various messages that the reception unit 915 receives are the same as those of
The storage unit 917 may store programs and various types of data required for the operation of the MMT reception device 900, particularly, information related to an operation of changing subtitles according to an embodiment. Also, the storage unit 917 may store various types of data, various signals, and various messages that the reception unit 915 receives from the MMT transmission device or the like.
Although
According to an embodiment, a transmission device in a multimedia system is provided. The transmission device may include a controller configured to detect a request to change from first-type data to second-type data while the first-type data is reproduced at a given point in time and a transmission unit configured to transmit information required for replaying at least one of data reproduced before the given point in time among the first-type data and data reproduced before the given point in time among the second-type data, before the second-type data is reproduced.
Here, when the first-type data includes a first-type first data unit and the second-type data includes a second-type first data unit, the information includes information required for replaying at least one of data reproduced before the given point in time among the first-type first data unit and data reproduced before the given point in time among the second-type first data unit, before the second-type first data unit is reproduced.
Here, the information includes: at least one selected from among a presentation time of a first third data unit included in a passed second data unit of the second-type first data unit, a presentation time of a first third data unit included in a subsequent second data unit of the second-type first data unit, a presentation duration of a passed second data unit of the first-type first data unit, and a presentation duration of the passed second data unit of the second-type first data unit.
The first data unit includes an asset, the second data unit includes a media-processing unit (MPU), and the third data unit includes an access unit.
The first-type data and the second-type data are associated with the same content.
According to an embodiment, a reception device in a multimedia system is provided. The reception device includes a controller, configured to detect a request to change from first-type data to second-type data while the first-type data is being reproduced at a given point in time, and a reception unit, configured to receive information required for replaying at least one of data reproduced before the given point in time among the first-type data and data reproduced before the given point in time among the second-type data, before the second-type data is reproduced.
Here, when the first-type data includes a first-type first data unit and the second-type data includes a second-type first data unit, the information includes information required for replaying at least one of data reproduced before the given point in time among the first-type first data unit and data reproduced before the given point in time among the second-type first data unit, before the second-type first data is reproduced.
Here, the information includes: at least one selected from among a presentation time of a first third data unit included in a passed second data unit of the second-type first data unit, a presentation time of a first third data unit included in a subsequent second data unit of the second-type first data unit, a presentation duration of a passed second data unit of the first-type first data unit, and a presentation duration of the passed second data unit of the second-type first data unit.
Here, the controller replays at least one of data reproduced before the given point in time among the first-type first data unit and data reproduced before the given point in time among the second-type first data unit, before the subsequent second data unit of the second-type first data unit is reproduced.
Here, the controllers identify an empty period between the passed second data unit of the second-type first data unit and the subsequent second data unit of the second-type first data unit on the basis of the presentation time of the first third data unit included in the passed second data unit of the second-type first data unit, the presentation time of the first third data unit included in the subsequent second data unit of the second-type first data unit, and the presentation duration of the passed second data unit of the second-type first data unit, determines at least one of data reproduced before the given point in time among the first-type first data unit and data reproduced before the given point in time among the second-type first data unit, to be an object to be replayed, and determining a time duration for replaying the object to be replayed, on the basis of the empty period, metadata of the passed second data unit of the first-type first data unit, metadata of the passed second data unit of the second-type first data unit, the capability of a decoder, and the capability of a renderer, and replays the object to be replayed at the time duration for replaying.
Here, the controller replays the object to be replayed at the time duration for replaying on the basis of one mode determined from among a first mode, a second mode, and a third mode, wherein the first mode includes a mode for replaying data corresponding to the time duration for replaying from a presentation end time of the object to be replayed, the second mode includes a mode for replaying an entirety of the object to be replayed within the time duration for replaying, and the third mode includes a mode for replaying the object to be replayed in the time duration for replaying on the basis of a scheme determined by the reception device.
Here, the controller adjusts a third data unit presentation time of the object to be replayed on the basis of the determined mode.
The first data unit includes an asset, the second data unit includes a media-processing unit (MPU), and the third data unit includes an access unit.
The first-type data and the second-type data are associated with the same content.
Particular aspects of the disclosure may be implemented as a computer-readable code in a computer-readable recording medium. The computer-readable recording medium is a predetermined data storage device which can store data which can be read by a computer system. The computer-readable recording medium may include a Read-Only Memory (ROM), a Random-Access Memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optimal data storage devices, and carrier waves (such as data transmission through the Internet). The computer-readable recording medium may be distributed through computer systems connected to the network, and accordingly the computer-readable code is stored and executed in a distributed manner. Further, functional programs, codes, and code segments for achieving the disclosure may be easily interpreted by programmers skilled in the art which the disclosure pertains to.
It will be understood that a method and apparatus according to an embodiment of the disclosure may be implemented in the form of hardware, software, or a combination of hardware and software. Any such software may be stored, for example, in a volatile or non-volatile storage device such as a ROM, a memory such as a RAM, a memory chip, a memory device, or a memory IC, or a recordable optical or magnetic medium such as a CD, a DVD, a magnetic disk, or a magnetic tape, regardless of its ability to be erased or its ability to be re-recorded. It can be also appreciated that the software may be stored in a machine (for example, a computer)-readable storage medium. It can be also appreciated that the memory included in the mobile terminal is one example of machine-readable devices suitable for storing a program including instructions that are executed by a processor device to thereby implement embodiments of the disclosure.
Accordingly, the disclosure includes a program for a code implementing the apparatus and method described in the appended claims of the specification and a machine (a computer or the like)-readable storage medium for storing the program. Further, the program may be electronically transferred by a predetermined medium such as a communication signal transferred through a wired or wireless connection, and the disclosure appropriately includes equivalents of the program.
Further, an apparatus according to an embodiment of the disclosure may receive the program from a program providing device that is wiredly or wirelessly connected thereto, and may store the program. The program providing device may include a program including instructions through which a program processing device performs a preset content protecting method, a memory for storing information and the like required for the content protecting method, a communication unit for performing wired or wireless communication with the program processing device, and a controller for transmitting the corresponding program to a transceiver at the request of the program processing device or automatically.
Meanwhile, although the concrete embodiments of the disclosure have been described in the detailed description of the disclosure, various modifications can be made without departing from the scope of the disclosure. Therefore, the scope of the disclosure should not be defined as being limited to the embodiments, but should be defined by the appended claims and equivalents thereof
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2017-0040338 | Mar 2017 | KR | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/KR2018/003667 | 3/28/2018 | WO |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2018/182313 | 10/4/2018 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 10244203 | Enigma | Mar 2019 | B1 |
| 10642574 | Fleizach | May 2020 | B2 |
| 20040081434 | Jung et al. | Apr 2004 | A1 |
| 20050123267 | Tsumagari et al. | Jun 2005 | A1 |
| 20060078301 | Ikeda et al. | Apr 2006 | A1 |
| 20070172199 | Kobayashi et al. | Jul 2007 | A1 |
| 20090300700 | Yeh et al. | Dec 2009 | A1 |
| 20100098389 | Shimada | Apr 2010 | A1 |
| 20100232762 | Kendall et al. | Sep 2010 | A1 |
| 20110142415 | Rhyu | Jun 2011 | A1 |
| 20130169762 | Kanemaru et al. | Jul 2013 | A1 |
| 20130209058 | Cho et al. | Aug 2013 | A1 |
| 20130219444 | Kitazato et al. | Aug 2013 | A1 |
| 20170048485 | Dewa et al. | Feb 2017 | A1 |
| 20170064371 | Kitazato et al. | Mar 2017 | A1 |
| 20170347134 | Bae | Nov 2017 | A1 |
| 20170353770 | Jiang | Dec 2017 | A1 |
| 20190090018 | Yoshizawa | Mar 2019 | A1 |
| 20190141361 | Deshpande | May 2019 | A1 |
| Number | Date | Country |
|---|---|---|
| 101278348 | Oct 2008 | CN |
| 101600087 | Dec 2009 | CN |
| 102883172 | Jan 2013 | CN |
| 103257821 | Aug 2013 | CN |
| 106416283 | Feb 2017 | CN |
| 3145210 | Mar 2017 | EP |
| 2008-072586 | Mar 2008 | JP |
| 2009-177720 | Aug 2009 | JP |
| 2015-216621 | Dec 2015 | JP |
| 10-2008-0056028 | Jun 2008 | KR |
| 10-0970727 | Jul 2010 | KR |
| 10-2011-0066578 | Jun 2011 | KR |
| 10-2011-0110338 | Oct 2011 | KR |
| 2007142648 | Dec 2007 | WO |
| 2017-155371 | Sep 2017 | WO |
| Entry |
|---|
| Chinese Office Action dated Mar. 29, 2021, issued in Chinese Patent Application No. 201880021673.1. |
| Korean Office Action dated Apr. 26, 2022, issued in Korean Patent Application No. 10-2019-7017763. |
| Zhang et al., Towards Video Quality of Experience and Selective Attention: A Subtitle-Based Measurement Study, Sep. 1, 2016. |
| The Research of Scene Change Detection Based on MPEG Soccer Video, Jun. 15, 2010. |
| Chinese Office Action dated Oct. 27, 2021, issued in Chinese Patent Application No. 201880021673.1. |
| Extended European Search Report dated Oct. 22, 2019, issued in a counterpart European application No. 18775519.4-1208. |
| Number | Date | Country | |
|---|---|---|---|
| 20230179826 A1 | Jun 2023 | US |
