Patent Application
20220279030
The present invention relates to a data processing apparatus, data receiving apparatus and a method of controlling them, and a storage medium.
In recent years, MPEG-DASH (Dynamic Adaptive Streaming over Http) standardized by ISO has proliferated as a technique of performing streaming distribution of video data by HTTP (Hypertext Transfer Protocol). More specifically, a transmitting apparatus divides video data into segments of a predetermined time length, and describes a URL (Uniform Resource Locator) used to acquire a segment in a file called a playlist. A receiving apparatus first acquires the playlist, requests a desired segment from the transmitting apparatus using information described in the playlist, and acquires the segment. Also, PTL 1 discloses a configuration for acquiring video data using MPEG-DASH in which a receiving apparatus selects and receives a desired segment in a plurality of segments generated by temporally and spatially dividing an image included in video data.
PTL 1: Japanese Patent Laid-Open No. 2016-9925
In the MPEG-DASH, however, only distributing a moving image (video data) as encoded data is assumed, and distributing a still image is not assumed. In PTL 1, segments obtained by temporally and spatially dividing one moving image file are distributed. However, when distributing a file, for example, an HEIF file that can include a plurality of images in one file, it is impossible to distribute individual images. Note that HEIF is short for High Efficiency Image File Format. Also, in this specification, “image” is a generic term for a moving image and a still image. Hence, for example, even if a plurality of still images are stored in an HEIF file held in a content server, and a receiving apparatus needs only some of the still images in the HEIF file, the receiving apparatus needs to download the whole HEIF file.
One aspect of the present invention provides a technique for enabling distribution of a desired image of a plurality of images stored in one file.
According to another aspect of the present invention, there is provided a data processing apparatus comprising: an acquiring unit configured to analyze a file of a predetermined format including a plurality of contents and acquire the plurality of contents and relation information representing a relationship between the plurality of contents; a storage unit configured to store the plurality of contents acquired by the acquiring unit; a generating unit configured to generate a playlist that describes acquisition information used to individually acquire the plurality of contents from the storage unit and the relation information; and a communicating unit configured to transmit the playlist to an external apparatus and transmit, to the external apparatus, content that the external apparatus requests using the acquisition information included in the playlist.
According to another aspect of the present invention, there is provided a data receiving apparatus comprising: a receiving unit configured to receive the playlist transmitted by the communicating unit of the data processing apparatus described above; a requesting unit configured to request transmission of a necessary content from the data processing apparatus using the acquisition information and the relation information described in the playlist; and a processing unit configured to receive the content transmitted from the data processing apparatus in response to the request of the requesting unit and process the content.
According to another aspect of the present invention, there is provided a method of controlling a data processing apparatus, comprising: analyzing a file of a predetermined format including a plurality of contents and acquiring the plurality of contents and relation information representing a relationship between the plurality of contents; storing, in a storage, the plurality of contents acquired in the acquiring; generating a playlist that describes acquisition information used to individually acquire the plurality of contents from the storage and the relation information; and transmitting the playlist to an external apparatus and transmitting, to the external apparatus, content that the external apparatus requests using the acquisition information included in the playlist.
According to another aspect of the present invention, there is provided a method of controlling a data receiving apparatus, comprising: receiving the playlist transmitted from a data processing apparatus executing the method described above; requesting transmission of a necessary content from the data processing apparatus using the acquisition information and the relation information described in the playlist; and receiving the content transmitted from the data processing apparatus in response to the requesting, and processing the content.
According to another aspect of the present invention, there is provided a non-transitory computer readable storage medium storing a program for causing a computer to execute a method of controlling a data processing apparatus, the method comprising: analyzing a file of a predetermined format including a plurality of contents and acquiring the plurality of contents and relation information representing a relationship between the plurality of contents; storing, in a storage, the plurality of contents acquired in the acquiring; generating a playlist that describes acquisition information used to individually acquire the plurality of contents from the storage and the relation information; and transmitting the playlist to an external apparatus and transmitting, to the external apparatus, content that the external apparatus requests using the acquisition information included in the playlist.
According to another aspect of the present invention, there is provided a non-transitory computer readable storage medium storing a program for causing a computer to execute a method of controlling a data receiving apparatus, the method comprising: receiving the playlist transmitted from a data processing apparatus executing the above-described method of controlling a data processing apparatus; requesting transmission of a necessary content from the data processing apparatus using the acquisition information and the relation information described in the playlist; and receiving the content transmitted from the data processing apparatus in response to the requesting, and processing the content.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Hereinafter, embodiments will be described in detail with reference to the attached drawings. Note, the following embodiments are not intended to limit the scope of the claimed invention. Multiple features are described in the embodiments, but limitation is not made to an invention that requires all such features, and multiple such features may be combined as appropriate. Furthermore, in the attached drawings, the same reference numerals are given to the same or similar configurations, and redundant description thereof is omitted.
The network 150 communicably connects the transmitting apparatus 100 and the receiving apparatus 200. Examples of the network 150 are a wired LAN (Local Area Network) and a wireless LAN (Wireless LAN). However, the network 150 is not limited to these, and may be, for example, the Internet, a WAN (Wide Area Network) such as 3G/4G/LTE/5G, an ad hoc network, or Bluetooth.
The functional configuration of the transmitting apparatus 100 according to the first embodiment will be described next with reference to
In a file analysis unit 101 and an encoded data extraction unit 102, a file of a predetermined format including a plurality of contents is analyzed to acquire the plurality of contents and relation information representing the relationship between the plurality of contents. In this embodiment, the predetermined format is, for example, an HEIF (High Efficiency Image File Format) file. The relation information is the relationship between a still image and a thumbnail image, the relationship between a derived image and an image constituting that, or the like. The file analysis unit 101 acquires an HEIF file and analyzes its structure. The encoded data extraction unit 102 extracts the encoded data of the contents stored in the HEIF file based on the analysis result of the HEIF file. A segment generation unit 103 converts the encoded data extracted by the encoded data extraction unit 102 into a time length or a bit rate suitable for communication as needed, thereby generating a segment that stores encoded data. An encoded data conversion unit 104 converts the extracted encoded data into a different encoding format as needed. Note that the segment generation unit 103 also has a function of generating a segment that stores the encoded data generated by the encoded data conversion unit 104.
A distribution data storage unit 105 stores the plurality of contents acquired in the above-described way such that an external apparatus (the receiving apparatus 200 in the example of this embodiment) can individually acquire these. More specifically, the distribution data storage unit 105 stores the data generated by the segment generation unit 103 and the encoded data conversion unit 104. A playlist generation unit 106 generates a playlist in which acquisition information (a URL (Uniform Resource Locator) in this embodiment) used to individually acquire the plurality of contents from the distribution data storage unit 105 and the above-described relation information are described. More specifically, based on the analysis result of the HEIF file, the playlist generation unit 106 generates a playlist in which a URL that enables access to the data stored in the distribution data storage unit 105 is described. Details of the relation information will be described later. In response to a request from the receiving apparatus 200 that is an external apparatus, a communication unit 107 transmits the generated playlist, the distribution data, and the like to the receiving apparatus 200 via the network 150.
The functional units as described above can be implemented by one or a plurality of CPUs executing predetermined programs stored in one or a plurality of memories and controlling. Note that some or all of the above-described functional units may be implemented by dedicated hardware.
The hardware configuration of the transmitting apparatus 100 according to the embodiment will be described here with reference to the block diagram of
The CPU 121 controls the whole transmitting apparatus 100 using computer programs and data stored in the ROM 122 or the RAM 123, thereby implementing the functions of the transmitting apparatus 100 shown in
The display unit 125 is formed by, for example, a liquid crystal display, an LED, or the like, and displays a GUI (Graphical User Interface) or the like used by the user to operate the transmitting apparatus 100. The operation unit 126 is formed by, for example, a keyboard, a mouse, a joystick, a touch panel, or the like, and inputs various kinds of instructions to the CPU 121 when operated by the user. The CPU 121 operates as a display control unit configured to control the display unit 125 and an operation control unit configured to control the operation unit 126.
The communication IN 127 is used for communication between the transmitting apparatus 100 and an external apparatus (for example, the receiving apparatus 200). In this embodiment, the communication EF 127 is an interface connected to the network 150. If the transmitting apparatus 100 is connected to the network 150 by wire, a cable for communication is connected to the communication EF 127. If the transmitting apparatus 100 is wirelessly connected to the network 150, the communication OF 127 includes an antenna. The bus 128 connects the units of the transmitting apparatus 100 and transmits information.
Note that in this embodiment, the auxiliary storage device 124, the display unit 125, and the operation unit 126 exist inside the transmitting apparatus 100. However, at least one of these may exist as another device outside the transmitting apparatus 100 or may be omitted.
The procedure of processing for analyzing an HEIF file in the transmitting apparatus 100 according to this embodiment will be described next with reference to
First, in step S301, the file analysis unit 101 acquires an HEIF file as a distribution target. Next, in step S302, the file analysis unit 101 confirms whether handler type pict exists in the metaheader of the HEIF file. Upon determining that pict handler does not exist (NO in step S302), the file analysis unit 101 ends the processing without performing analysis any more. On the other hand, upon determining that pict handler exists (YES in step S302), in step S303, the file analysis unit 101 acquires the number and types of items stored in the HEIF file. Note that in the HEIF file, contents such as stored images and metadata associated with the images are called items.
Next, in step S304, the file analysis unit 101 acquires reference information between the items. For example, the HEIF file stores reference information representing the relationship between a still image and a thumbnail image thereof The HEIF file can also store images used to generate one image by combining a plurality of images. The one image generated by combining the plurality of images is called a derived image. In the HEIF, if a derived image is included as an item, reference information representing the relationship between the derived image and a plurality of images that form the derived image is stored in the HEIF file. In addition, to generate the derived image, pieces of information of positions to arrange the plurality of images are needed. Hence, if a derived image is included in the items of the HEIF file (YES in step S305), in step S306, the file analysis unit 101 acquires coordinate information used to generate the derived image. Note that if no derived image is included in the items of the HEIF file (NO in step S305), step S306 is skipped. Handling of a derived image will be described in more detail in the second embodiment.
In step S307, the file analysis unit 101 acquires the alternative information of an item from the HEIF file. As a detailed case of setting of alternative information, for example, if a derived image cannot be displayed, an image to be displayed in place of that is designated. An alternative image will be described in more detail in the second embodiment. Next, in step S308, the file analysis unit 101 specifies a property linked with each item. Detailed examples of the property are encoding information of an item and rectangular size information. Next, in step S309, the file analysis unit 101 acquires encoding information and the rectangular size information of the image from the specified property. Next, in step S310, the file analysis unit 101 acquires the position and the size information of the item in the HEIF file.
Based on the position and size information of the item acquired by the file analysis unit 101 in the above-described way, the encoded data extraction unit 102 extracts the item from the HEIF file. The encoded data conversion unit 104 converts the extracted encoded data into a different encoding format as needed. For example, if the encoding format before data conversion is HEVC, the encoded data conversion unit 104 converts this into JPEG as needed. Acquisition information can include a URL capable of acquiring the data converted into the different encoding format. The segment generation unit 103 generates a segment from the encoded data extracted by the encoded data extraction unit 102 and the encoded data converted by the encoded data conversion unit 104, and stores the segment in the distribution data storage unit 105. Using the reference information, encoding information, image size information, and the like acquired by the file analysis unit 101, the playlist generation unit 106 generates a playlist used by the receiving apparatus 200 to access the segment stored in the distribution data storage unit 105.
Examples of a playlist generated by the transmitting apparatus 100 according to the first embodiment will be described next with reference to
The playlist in
Also, in
Also, in
Note that in the HEIF file, since one item identifier is designated as information representing a primary item, expression by an MPD can be implemented by setting the value of one primary attribute to “1”, as described above. Note that instead of using two levels of “0” and “1”, when “0” is set to the highest priority, and the priority is lowered as the value becomes large to “1”, “2”, . . . , priorities of three or more levels can easily be expressed.
Another example of the playlist will be described next with reference to
Referring to
Note that a mime type 505 shown in
In
Referring to
A description example of a playlist for combinations of a plurality of images and thumbnail images will be described with reference to
Referring to
In the first embodiment, all the above description examples of an MPD describe a plurality of images and thumbnail images thereof. A case in which the description amount is decreased by collectively describing identical attribute values will be described with reference to
Referring to
A case in which as an item other than an image, metadata such as EXIF is described in an MPD will be described next with reference to
Referring to
As described above, according to the first embodiment, it is possible to distribute a desired image of a plurality of images stored in one HEIF file.
In the first embodiment, a playlist that enables distribution of an HEIF file storing a plurality of images has been described. In the second embodiment, a case in which a derived image or an alternative image of HEIF is described in an MPD will be described. Note that the configuration and operation of a transmitting apparatus 100 serving as a communication system and a data processing apparatus according to the second embodiment are the same as in the first embodiment (
As shown in
Also, as segments below SubRepresentation, four pieces of segment information 1004 forming the derived image (grid image) are described. Each segment of the segment information 1004 describes, as a reference identifier, the identifier of Representation including the constituent image (=segment), thereby explicitly indicating which image forms the derived image. Also, as information representing the position of each segment, coordinate information indicating the position to display the segment is described in the segment information 1004. Note that in
Concerning the derived image of HEIF, types other than grid also exist. In particular, an overlay image can be expressed by the same description method as the grid image. To generate and display an overlay image as a derived image, not only the coordinate information of the images forming the derived image but also layer information representing the order of overlaying these is necessary. The layer information can be described as, for example, one of the pieces of attribute information of the segments described in the segment information 1004. Alternatively, the order of the segments described in the segment information 1004 may indicate the order of layers.
The playlist described with reference to
Referring to
A case in which the alternative information of an item is acquired in step S307 of
Referring to
As a method of indicating an alternative image, a case in which not priority but an alternative group is set will be described next with reference to
Referring to
A case in which an already generated derived image is prepared in consideration of a case in which the derived image cannot be displayed on the receiving apparatus 200 will be described next with reference to
In the playlist shown in
As described above, in the second embodiment, relation information concerning a derived image or an alternative image has been exemplified. Note that in the second embodiment, the derived image describing method of displaying one image by combining a plurality of still images has mainly been explained. When the method described in this embodiment is used, it is possible to easily describe a derived image that combines a still image and a moving image, as a matter of course.
As the third embodiment, a case in which an image to be displayed is changed in accordance with a reproduction time, like a slideshow, will be described next. Note that the configurations and processes of a communication system and a transmitting apparatus according to the third embodiment are the same as in the first embodiment (
A method of decreasing the description amount of an MPD in correspondence with the case described with reference to
If a distributable image is common to a plurality of periods, it is redundant to describe information (attribute information, a URL for acquisition, and the like) concerning the same image for each period. Hence, in
As described above, according to the first to third embodiments, since an HEIF file can be distributed by MPEG-DASH, it is possible to, for example, acquire only a desired still image from an HEIF file or set an acquirable still image for each predetermined time.
In the fourth embodiment, a receiving apparatus 200 that receives a playlist from a transmitting apparatus 100 described in the first to third embodiments and acquires content (image) from the transmitting apparatus 100 using the received playlist will be described.
Next, in step S1803, the CPU 1703 selects a desired segment from segments in which the storage destination of the playlist is described. The segment selection may be done based on, for example, a user operation for the receiving apparatus 200, or may be done based on an instruction from an application under execution. Next, in step S1804, the CPU 1703 acquires the data of the segment selected in step S1803 from the transmitting apparatus 100. That is, the CPU 1703 requests, via the communication unit 1701, the data of the segment selected in step S1803 from the transmitting apparatus 100, and receives the data (for example, a still image, a thumbnail image, or the like) of the segment from the transmitting apparatus 100 via the communication unit 1701. Note that the CPU 1703 can acquire the segment from the transmitting apparatus 100 using a URL described in the segment in the playlist. In step S1805, the CPU 1703 reproduces the data acquired in step S1804. For example, the CPU 1703 displays the received still image on the display 1704.
According to the receiving apparatus 200 as described above, for example, the following processing is implemented. Upon receiving the playlist shown in
In addition, when the CPU 1703 receives the playlist shown in
As described above, the receiving apparatus that has received a playlist according to the first to third embodiments can selectively receive a necessary item in a plurality of items included in the HEIF file without receiving the whole HEIF file including the plurality of items.
The embodiments have been described above in detail. The present invention can take an embodiment of, for example, a system, an apparatus, a method, a program, or a recording medium (storage medium). More specifically, the present invention may be applied to a system formed by a plurality of devices (for example, a host computer, an interface device, an image capturing apparatus, a web application, and the like), or may be applied to an apparatus formed by a single device.
According to the above described embodiments, there is provided a technique for enabling distribution of a desired still image of a plurality of still images stored in one file.
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2019-211706 | Nov 2019 | JP | national |
This application is a Continuation of International Patent Application No. PCT/JP2020/041753, filed Nov. 9, 2020, which claims the benefit of Japanese Patent Application No. 2019-211706, filed Nov. 22, 2019, both of which are hereby incorporated by reference herein in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2020/041753 | Nov 2020 | US |
| Child | 17747756 | US |
