The present invention relates to software and hardware architectures for processing multimedia data.
In recent years, moving pictures, speech and the like have been put to various uses as multimedia data handled in computers. For example, apparatuses called hard disk recorders in which a moving picture in a television broadcast or the like is converted in real time by a moving picture CODEC to be recorded on a hard disk, players which display, on video displays, moving picture contents data recorded as digital data on optical disks including DVDs, etc., have come into widespread use. It is ordinarily required that processing of such contents be performed in a certain time period by reading contents data at a designated bit rate. For use of such contents data, it is important to enable a program module for executing such complicated stream processing to be easily developed. Systems therefore exist in which a program for performing contents processing is executed in the kernel of an operating system (OS) on which the desired performance can be easily ensured.
In ordinary OSs, programs executed in the kernel can be executed with priority over programs executed on user level. For this reason, a contents processing program which needs to be executed so that processing is completed in a certain time period is executed in the kernel. For example, according to an art disclosed in JP-A-10-283195, a plurality of data-processing device drivers each called a filter are connected to enable content processing. Each filter is a module specialized in correspondence with a device and a data format and having an interface for input/output and control. Data processing using the filters is performed in such a manner that data processed by a particular one of the filters is delivered to the other filters. This system is characterized in that filters operate on a kernel portion of an OS, the desired performance can therefore be ensured easily and a program can be formed by only connecting filters.
However, when a stream processing program is provided in the kernel of an OS, a configuration dependent on a particular execution environment, i.e., on the OS in particular, is formed.
On the other hand, an arrangement such as one disclosed in JP-A-8-279963 also exists in which a stream processing device in hardware form is formed in a layered configuration to improve portability. In this device, however, a separate piece of hardware is used for each of different kinds of processing and there is, therefore, a problem that the processor is increased in size and price.
As described above, almost all the conventional methods use, as implementation enabling handling of streams on drivers, implementation on an OS of a kernel portion on which the desired performance can be easily ensured. In such a case, the dependence on an OS and a hardware configuration (which configuration will be referred to as “platform”) is high and it is difficult for a program for using contents to be adapted to different platforms.
Also, each platform has contents data processing means specific to it. Therefore, it is necessary to develop a program for using contents with respect to each of different OSs or hardware configurations, and an increase in the number of development steps results.
To solve the above-described problems, the present invention provides contents processing means adaptable to various platforms, and a contents processing architecture for the contents processing means.
In the present invention, processing of contents is realized on the basis of a combination of layers having functions necessary for contents processing.
According to the present invention, when a contents processing program on one platform is transferred to a different platform, it can operate without being substantially changed. The number of program development steps can therefore be reduced and a program for using contents data can be developed with improved facility.
Other objects, features and advantages of the invention will become apparent from the following description of the embodiments of the invention taken in conjunction with the accompanying drawings.
An embodiment of the present invention will be described. The system architecture is constituted by an OS including a driver section which provides an interface with hardware, a system interface section which is a layer provided as an interface in which functions varying from OS to OS (e.g., functions such as system call and a method of access to device drivers varying from platform to platform) are combined to provide common functions, a contents processing section which is located above the system interface section and which performs processings including processing of contents data and processing for obtaining data from a data source, a contents management section which provides data processing for contents to be handled, and an application program using functions provided by these layers.
In this structure, an interface for exchanging data and events between a pair of upper and lower layers in the above-described layers is prescribed. For example, the system interface section combines OS functions necessary for processing of contents, e.g., system calls for memory management, access to a disk and generation of a task into common functions and provides the common functions to the upper layers, thereby enabling the layers above the system interface section to be implemented independently of the platform. “Combing functions into common functions” refers to a process in which on the basis of an instruction from an upper program layer, the instruction is executed in a lower program layer by a processing procedure defined in advance.
The contents processing section performs processing of contents data by utilizing functions provided by the system interface section. Patterns of data processing in this contents processing are managed by the contents management section. The contents management section provides an interface with the application program. From the application program, one of the data processing patterns is designated via the interface to enable the contents processing section to construct a data path for contents processing. The data path for contents processing is a path which is formed by designating the functional modules necessary for processing contents to be processed and through which processed contents data flows. Thereafter, an instruction is issued from the application program via the contents management section to start and stop contents processing according to timing designated by a user, thus enabling control of processing.
Even on a different platform, the application program thus formed can be made executable by making a necessary minimum change therein, if a contents processing system in accordance with a contents processing architecture provided in the present invention is implemented.
The contents processing architecture can also be applied even in a case where a portion or the whole of the hardware is formed in an LSI. Also in such a case, the application program can be used without being substantially changed.
The system interface section 20 provides an interface in which functions provided by the OS 5 and device access functions for accessing devices through various device drivers (HDD driver, NIC driver, etc.) are combined into common functions to enable access to the devices by a method independent of the platform. That is, even if the platform is changed, the method of accessing a hard disk drive (HDD) device or a network is basically the same. For example, for file read, the functions of designating the name of a file to be read, a read mode, a read position and a read length are provided. The procedure for this designation and the interface for access vary from platform to platform. Therefore there is a need to change the program using these functions when the program is transferred to a different platform. Interfaces and procedures varying from platform to platform are absorbed in the system interface section 20 and a common interface and procedure are provided to the upper layers. Further, for example, in a case where data exchange is performed between a plurality of application programs 80, functions provided by the system interface section 20 may be used to form a program independent of the platform. The function provided by the system interface 20 is to combine functions provided by the platform into common functions. Data is changed in the data access section 12 in the contents processing section 10. Details of data change processing will be described with reference to
The contents processing section 10 includes a data access section 12 which perform data exchange with the driver section 50 through the system interface section 20, and a data processing section 11 which processes data.
The data processing section 11 processes data received from the data access section 12 in accordance with a predetermined processing procedure and returns the processed data as its output to the data access section 12. For example, when an instruction to transfer data stored in a HDD to an external appliance connected to a network is provided from the application program, the data processing section 11 adds suitable headers to data read from the hard disk by the data access section 12, and the data is output to the network through the data access section 12. For example, in a case where data is transmitted on the basis of UDP (User Datagram Protocol), there is a possibility of packet loss or the transmission sequence and the arrival sequence differing from each other. A method of adding headers having a sequence of numbers to the data to enable detection of such an error is conceivable. Thus, “Suitable headers” are data added to contents data to enable the data to be correctly processed by a device or to be correctly transmitted. It is also conceivable that, with respect to input/output of a device other than the network, headers necessary for the device are added and deleted in the same manner. For example, checksum or the like for checking whether or not transfer is performed is conceivable. Contents data handled in the data access section 12 and the data processing section 11 are not output to the layers above the contents management section 30.
The contents management section 30 controls the contents processing section 10 by using a control interface provided in the contents processing section 10. The contents management section 30 also manages, on a resource management table 300, resources which can be handled by appliances to which this architecture is applied. “Resources” denote hardware resources necessary when contents processing is performed, e.g., a CPU, memories, usable devices and bandwidths usable by the devices. The resource management table will be described below in detail with reference to
The function of providing a list of contents which can be handled by the platform to the application program 80 and refusing an instruction from the application program 80 for processing exceeding the performance of the hardware is also provided. Details of this function will be described below with reference to
A process from the provision of an instruction from the application program 80 to make the contents processing section 10 perform contents processing to the completion of contents processing will be described with reference to
The contents management section 30 determines whether or not contents processing in accordance with the instruction is executable (step 1000). The method of this determination will be described with reference to
For example, processing in a case where instruction is transmitted from the application program 80 to the contents management section 10 to send out data read out from a hard disk 103 to a network adaptor 104 is as described below. First, determination is made as to whether processing according to the instruction from the application program is executable. In this case, the hard disk and the network adaptor are provided and determination is made as to whether or not they are available (as described below in detail). If it is determined that they are available, a data path for causing contents to flow from the hard disk to the network adaptor is set. When the application program issues an instruction to start transfer, transfer of data through the data path is performed. Processing is continued until transmission of the contents file to the end of the contents file is completed or until the application program issues an instruction to stop processing.
In this data transfer, the contents data is not delivered to the application program 80 and processing is performed in the contents processing section 10 and the data access section 11. Other operations for stop of contents data, pause, return, fast forward, reverse, etc., are also performed in the same manner.
A method of determining whether or not processing according to an instruction can be performed by using the resource management table 300 will be described with reference to the resource management table shown in
A hardware configuration capable of implementing this embodiment will next be described with reference to
As an example of the serial interface 108, an interface such as RS-232C, USB (universal serial bus) or IEEE1394 for serial transfer of data is conceivable. The CODEC processor 109 is a means for performing processing for converting contents data into a particular moving picture format, e.g., MPEG2 or MPEG4 or obtaining video information and audio information from a particular moving picture format. The functions of this device can be provided by means of software. The decryption processor 110 performs processing for decrypting contents data if the contents data is provided in an encrypted state, and processing for encrypting contents data. The functions of this processor can also be provided by means of software. As an example of the bus 111, a general-purpose bus or a local bus specialized for the CPU or the peripheral chip is conceivable. As an example of the general-purpose bus, a PCI (Peripheral Components Interconnect) bus is conceivable. Use of different buses coupled by a bus bridge is also conceivable. The operating unit 112 is a means for enabling a user using a remote controller or a keyboard to control the devices. A plurality of devices or units may be provided as each of the devices and units constituting this embodiment as required. The devices other than the CPU, the RAM and the bus are implemented according to one's need. The application program and the software required in each layer are implemented by being loaded in the memory and executed by the CPU.
A process from receiving a broadcast from the network adaptor 104 by using software having the architecture shown in
First, the application program 80 issues an instruction to the content management section 30 to “show on the display a broadcast from the network” (step 140). According to this instruction, the content management section 30 determines necessary processing in accordance with the instruction, extracts necessary devices by referring to the resource management table and determines whether or not the devices are available (step 1410). The method of this determination is as described above with reference to
The devices extracted in this processing are the network adaptor 104, the CODEC processor 109 and the display 107. If it is determined that these devices are available, the contents management section 30 instructs the contents processing section 10 to input to the CODEC processor 109 input data from the network adaptor 104 and to supply the output from the CODEC processor 109 to the display 107 (step 1420). In the contents processing section 10, the data access section 12 is set so that input and output of data to and from the devices according to the instruction can be performed (step 1430) and the data processing section is set to that the data thereby obtained is suitably processed (step 1440). Details of this processing will be described below with reference to
The procedure for initialization of the data processing section 11 and the data access section 12 in the contents processing section 10 and data conversion processing (steps 1430 and 1440) will be described with reference to
Processing in step 1430 shown in
In the data processing section 11, program modules such as functions for data processing 1 (111), data processing 2 (112) and data processing 3 (113) exist. These modules perform data conversion processing. As an example of data conversion processing, data format conversion, addition and deletion of headers, compression and decompression of data or encryption is conceivable. In this embodiment, if input data is data MPEG encoded, DES encrypted and input in the form of TCP/IP packets, the headers are deleted by data processing 1, the encrypted data is decrypted by data processing 2 and the data is MPEG decoded by data processing 3. Determination of the concatenated order of the data processing modules, etc., is made by the contents management section. The contents management section determines which data processing modules are used for contents processing to be performed according to an instruction and how the data processing modules are connected, and makes corresponding settings in the contents processing section. Details of this processing will be described below with reference to
Each data processing module has an interface for data input/output and can perform data exchange with other data processing modules and data access processing modules. Data to be processed may be input or output through the data access modules. For example, if input data is not encrypted data in this embodiment, processed data after the completion of data processing 2 is output to data access processing B. Thus, this processing corresponds to acquisition of data from a device or output of data to a device. Input/output between the data processing modules is not limited to one-to-one data exchange, and plurality-to-plurality, one-to-plurality and plurality-to-one patterns are conceivable. For example, a one-to-plurality pattern corresponds to a case where reproduction of obtained contents data is performed simultaneously with recording of the data, and a plurality-to-one pattern corresponds to a case where an audio and a video exist as separate contents data and processing for recording the separate groups data as one content in a file. In processing in step 1440, the data processing modules for performing data processing necessary for the desired contents processing are initialized and concatenated.
This, in the contents management section, the data access module and data processing modules suitable for contents processing according to an instruction from the application program are selected to set a necessary processing path.
Also, determination as to how contents data should be decoded or encoded is made according to the contents provision method or at the time of recording. For example, with respect to digital broadcasting, the method of encoding contents and the kind of encryption are made public in advance. The instruction pattern 210 shown in
Concatenation of modules refers to delivery of processed contents data between modules. For example, in a case where a data access module delivers to a data processing module contents data read out from a device and the data processing modules the delivered data, transmission/reception of the read data between the two modules may suffice. As this method, function designation may be performed by using as a parameter the address of the buffer memory in which the data is recorded or a method may be used in which the modules have an input queue and the data is put in the queue. Use of a shared memory and data exchange using a socket or the like are also conceivable.
A channel through which data flows as described above, i.e., a data path, is set to prepare for contents processing. When data access processing for the data entrance of this data path, i.e., the device to which data is input is started, the data is obtained from the device is delivered to the data processing module to be processed along the data path. Finally, the data is delivered to the exit for the data, i.e., data access processing for outputting the data. The data is thereby output to the device. It is conceivable that device access processing may vary partially depending on the platform. For example, even in ordinary devices, input/output data changes when the platform is changed. In such a case, processing for removing such variation to provide common data may be performed in data access processing. In this manner, the data handled in the platform above the data access section 12 is made completely independent of the platform. For example, such an effect is achieved in a case where data such as a character code differing in used form depending on the platform is converted into a more standard character code such as Unicode in the data access section for example.
While in this example the same devices as those provided in the LSI 120 are not provided outside the LSI 120, two or more equivalent devices may be provided. For example, while the tuner is provided in the LSI 120, a tuner is also provided outside the LSI 120. The data exchanger 129 is used for communication between the CPU 100 and the Sub-CPU 121. Various methods for such implementation depending on the configurations of the LSI 120 and the system are conceivable. For example, as a simple method, a method of converting data by using a FIFO (first in first out) type of buffer. Exchange of a large amount of data at a high speed using a shared memory and DMA (direct memory access) transfer is also conceivable. Further, a method of using a register in an LSI and a method of using functions provided by an OS are conceivable. As this means, one or a plurality of means may exist.
The contents processing architecture for processing using devices outside the LSI 120 is also constituted by a driver section 50, a system function provision section 40, a system interface section 20, a contents processing section 10, a contents management section 30 and an application program 80. The configuration of this architecture is the same as that shown in
Thereafter, an instruction is provided to the application program 81 on the LSI 120 to start contents processing, and the application program 81 instructs the contents processing section 11 to start processing (step 1150) and also instructs the contents processing section 10 to start contents processing (step 1160). The data obtained by the tuner 125 is thereby output to the data sharing unit 129 (step 1170). The contents processing section 10 outputs this data to the network adaptor (step 1180). This processing is repeatedly performed until the end of the data is reached or until the user inputs a stop instruction (step 1190). Communication between the application program 81 on the LSI 120 and the LSI driver 51 is performed by using the data sharing unit 129, as in the case of contents of data exchange.
According to this embodiment, the application program can execute processing independently of the platform. Also, contents processing section and a relating contents processing architecture adaptable to various platforms can be provided. When the contents processing program is transferred to a different platform, the contents processing program can operate without being substantially changed.
It should be further understood by those skilled in the art that although the foregoing description has been made on embodiments of the invention, the invention is not limited thereto and various changes and modifications may be made without departing from the spirit of the invention and the scope of the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
2004-085958 | Mar 2004 | JP | national |