Patent Application
20070097223
1. Field of the Invention
The present invention relates to a system, apparatus, method and computer program for externally configuring an image processing parameter for use in an image input device.
2. Description of the Related Art
Conventionally, among image processing applications such as photo retouching software operating on a PC (personal computer), an application which displays both an original image and its processed image obtained by applying desired image processing for comparison is known.
Using such image processing application for color adjustment of an original image enables a user to tune a color adjustment parameter to obtain a desired result while comparing the original image and a processed image to which a designated color adjustment has been applied.
Further, in digital cameras, setting of a layout of obtained still images for displaying the images arranged like a photo album has been proposed (Japanese Patent Application Laid-Open No. 2005-176216).
In this manner, image processing on image data obtained by image capturing using photo retouching software or the like on a PC and setting a display layout in a digital camera and outputting the laid out images have conventionally been proposed. However, these processings are independently performed on the PC and the camera and the processings do not cooperate with each other.
At this point, when a device to handle a moving picture such as a video camera is used, as a moving picture changes in real time differently from a still image, timing control is difficult in a single device, or excessive processing load is caused. Further, even a similar image processing is applied to an input image, the obtained result may differ when the processing is applied. For example, there often occurs a difference between the result of image adjustment such as image quality adjustment or color adjustment by tuning a parameter of a video camera upon image capturing and the result of a similar image adjustment by PC software after image capturing.
Further, even image input devices of the same model may have slightly different characteristics. From these points, it can be considered that image quality adjustment or the like by a collaborative operation among devices easily produces a better result.
The image quality adjustment by a collaborative operation between a PC and a video camera may be performed as follows. First, software on the PC transmits an image processing request signal with a desired configured parameter to the video camera. The video camera performs image processing corresponding to the image processing request on an image obtained by image capturing, and transmits a video image resulted from the image processing to the PC. The software on the PC obtains the video image resulted from the image processing and displays the image on a screen. This operation is repeated a plurality of times while the image processing parameter is changed, then the software on the PC simultaneously displays the images obtained from the video cameras for comparison. Then a user selects an image having preferred color tone from the displayed images. Then the software on the PC sends an image processing request with an image processing parameter configured in correspondence with the selected image to the video camera. Thereafter, image obtained by image capturing have the user's desired color tone.
The image processing by the collaborative operation between a software on the PC and the video camera can be performed in this manner. However, it is necessary to repeat the image processing request and the image capturing and output processing a plurality of times between these devices. The time required from the image processing request issued from the software on the PC to the acquisition of processed image differs according to the performance of the PC or video camera model. From the viewpoint of software processing, it is simple to perform the image processing request and the image display at constant intervals. However, to smoothly perform the collaborative operation in various combinations of PCs and video cameras, it is necessary to set sufficient time intervals, which increases processing time. Further, generally, a decoding processing required for displaying compressed video data transmitted from the video camera is a heavy burden imposed on the PC.
Particularly, assuming that the digital video camera and the PC are set in an actual image capturing site and used in a collaborative operation, it is desirable that the PC is a portable type machine. In this case, as the PC is battery-driven, it is desirable to reduce the processing load on the PC for reduction of electric consumption.
The present invention is directed to overcome the above-described drawbacks and disadvantages. For example, the present invention enables to connect an image input device, such as a video camera, with an external device to easily configure an image processing parameter in the image input device from the external device.
According to an aspect of the present invention, there is provided a parameter configuration apparatus for externally configuring a value of an image processing parameter used in an image input device which outputs image data resulted from image processing according to the image processing parameter, comprising: a change request unit which transmits a change request to change the value of the image processing parameter to a designated value to the image input device; a reception unit which receives the image data from the image input device; a display unit which displays the received image data; and a control unit which controls the change request unit to sequentially transmit a plurality of change requests to change the value of the image processing parameter to values respectively different from the designated value, control the reception unit to sequentially receive a plurality of image data reflecting the plurality of change requests, control display unit to display the plurality of image data, and control the change request unit to transmit the change request to change the value of the image processing parameter to a value corresponding to one of the plurality of image data selected by a user, as the designated value.
According to another aspect of the present invention, there is provided a parameter configuration method for externally configuring a value of an image processing parameter used in an image input device which outputs image data resulted from image processing according to the image processing parameter, comprising the steps of: transmitting a change request to change the value of the image processing parameter to a designated value to the image input device; receiving the image data from the image input device; displaying the received image data; and controlling the transmitting step to sequentially transmit a plurality of change requests to change the value of the image processing parameter to values respectively different from the designated value, controlling the reception step to sequentially receive a plurality of image data reflecting the plurality of change requests, controlling the display step to display the plurality of image data, and controlling the transmitting step to transmit the change request to change the value of the image processing parameter to a value corresponding to one of the plurality of image data selected by a user, as the designated value.
Further features and aspects of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features and aspects of the present invention and, together with the description, serve to explain the principles of the present invention.
Exemplary embodiments, features and aspects of the present invention will now be described in detail below with reference to the attached drawings.
Note that the image input device of the present embodiment is not limited to the digital video camera but applicable to any other device capable of performing operations described as those of the DVC 1. For example, a digital camera, a cellular phone and a PDA having a moving picture capturing function may be employed. Further, the parameter configuration apparatus of the present embodiment is not limited to the personal computer but applicable to any other device capable of performing operations described as those of the PC 2.
In
In the DVC 1, a lens 10 brings a subject image into focus on an image sensing device 11. The image sensing device 11 converts the subject image focused through the lens 10 into an electric signal. A driver 12 drives the image sensing device 11. As the image sensing device 11, an image sensor such as a CCD sensor or a CMOS sensor may be used. Further, an A/D converter 13 converts an analog image signal outputted from the image sensing device 11 into a digital image signal. An image processor 14 performs image processing such as color tone adjustment on the digital image signal outputted from the A/D converter 13. A recorder 15 records the digital image signal processed by the image processor 14 onto a storage medium such as a tape medium, a disk medium, a semiconductor memory or a hard disk. A system controller 16 controls the entire digital video camera. A storage 17 holds an operation program of the system controller 16, an in-process image signal, various image processing parameters and the like. The storage 17 is also used as a work area for the system controller 16. Further, a network interface (IF) 18 is an interface for the DVC 1 to perform communication with the PC 2. In the present embodiment, the network interface 18 is compliant with the IEEE 1394 standards (IEEE Std 1394-1995). Note that the network interface 18 is not limited to that compliant with the IEEE 1934 standards, but a network interface compliant with the USB standards (USB 2.0) may be used.
Further, in
Further, first image processor 25 and second image processor 26 perform processing to convert image data inputted from the network interface 21 into data in a format displayable on the display unit 20 by respectively different methods. Note that in
In the image processing system of the present embodiment, software on the PC 2 controls the DVC 1 utilizing AV/C commands or the like.
Data transfer using an IEEE 1394 interface is made by isochronous transaction and asynchronous transaction. The isochronous transaction is used upon real-time transfer of moving picture, while the asynchronous transaction is used upon transfer of control signal or the like. The DVC 1 of the present embodiment always outputs a video image being obtained by image capturing using the isochronous transaction from the network interface 18.
In the present embodiment, the video data outputted from the digital video camera is stream data (MPEG-TS) compression-coded in the MPEG 2 format. Accordingly, to display the video data on the display unit 20 in the PC 2 as the receiving side, it is necessary to perform decompression processing.
Further, the PC 2 can adjust a color gain (color depth) which is one of the image processing parameters of the DVC 1 by transmitting a command or control signal in the asynchronous transaction.
Next, the operation of the image processing system having the above configuration will be described.
In the present embodiment, the PC 2 which functions as a parameter configuration apparatus has two display modes.
The first display mode is a preview mode to display image data transmitted from the DVC 1 without any change.
Note that the area displayed in the focus check area 202 may be movable. In this case, for example, a frame 204 to display an enlarged display area is superpose-displayed on the main preview area 201, and the frame 204 is moved by an operation of the operation unit 24. Then an enlarged image in an area corresponding to the position of the frame is displayed in the focus check area 202. Note that as described later, it is desirable that, not a resized image data for display in the main preview area 201 but image data before resizing (i.e., before reduction) is used for a display in the focus check area 202. When the enlarged display area is fixed, the frame 204 may be omitted.
The second display mode is the variation mode to simultaneously display images resulted from image processing with respectively-changed different image processing parameters.
In a variation screen 301, images showing the results of changes of image processing parameter are displayed. In
Next, the operation of the PC 2 in the preview mode will be described using the flowchart of
In
The first to third memory areas (905 to 907) are set in the storage 23 of the PC 2. The image processing modules (902 to 904) are modules to perform processing from reception of image data from the DVC 1 to display on the display 20. The image processing modules 902 to 904 correspond to the first and second image processors 25 and 26 in
The decompression processing module 902 performs decompression on compression-coded image data outputted from the DVC 1. The resize processing module 903 performs resize processing on decompressed image data to a display size, i.e., a display size in the main preview area shown in
The main module 910 detects an operation using the operation unit 24, performs communication with the DVC 1, and performs controls other than display processing. The processing method setting module 901, the image processing modules 902 to 904 and the main module 910 are control programs executed by the CPU 22. The functions of the respective modules are realized by controlling the respective blocks shown in
In step S401, the main module 910 checks the presence/absence of display switch request between the preview mode/variation mode. That is, it is determined whether or not the mode switching button 203/303 in the respective display screen has been operated using the operation unit 24 in the display screen. If it is determined that the switching of display mode to the variation mode has been instructed, the process moves to a processing procedure in the variation mode (
In step S402, image reception preparation processing is performed.
In the image reception preparation processing performed in step S402, the processing method setting module 901 performs initial setting of the respective modules 902 to 904 and ensures memory areas necessary for image processing.
In step S403, image data reception is performed. An image stream transmitted from the DVC 1 is stored into the first memory area 905 as a part of the storage 23 through the network interface 18.
As described above, in the present embodiment, the image data outputted from the DVC 1 is compression coded in the MPEG 2 format. As the detailed specification of the MPEG 2 format are well known, detailed the explanation of the specification will be omitted. Generally, in the MPEG 2 format, compression is performed by GOP (Group Of Picture) which is a plurality of picture frames. The a plurality of picture frames constructing the GOP includes an I frame which is intra-frame compressed frame, a P frame compressed using a difference between a current picture and a previous picture in time sequence, and a B frame compressed using a difference between a current picture and a previous picture and a difference between the current picture and a subsequent picture in time sequence. In the present embodiment, one GOP has fifteen image frames including one I frame, four P frames and ten B frames.
Accordingly, in step S403, I frames, P frames and B frames are stored in accordance with the GOP structure into the first memory area 905 as shown in
In step S404, processing as the first image processor 25 (first image processing) is performed. In this case, the decompression processing module 902 reads image data stored in the first memory area 905 by GOP, and performs decompression processing on all the frames. Then, the decompressed image data is stored into the second memory area 906 (corresponding to a second memory area 112 in
The decompressed image data stored in the second memory area 906 is resized to a display size by the resize processing module 903, and stored into a third memory area 907.
In step S405, the resized image data stored in the third memory area 907 is displayed by the display module 904 in the main preview area 201 of the preview screen 200. In this arrangement, an image, currently obtained by image capturing by the digital video camera, is displayed in real time in the main preview area 201 as in the case of a so-called electronic view finder. Further, the display module 904 displays e.g. a predetermined area of the image data before resizing stored in the second memory area 906 in the focus check area 202. As the number of pixels of the image data before resizing is greater than that of the resized image data, the image data before resizing can be displayed as e.g. an image enlarged from a part of the main preview area.
In step S406, the main module 910 checks the presence/absence of mode switch request using the mode switching button 203, and if it is determined that a mode change request exists, the process moves to a processing procedure in the variation mode (
In step S407, the main module 910 checks whether or not the image processing parameter configuration program is to be terminated. The checking is realized by detecting the presence/absence of so-called application termination instruction. If it is determined that a termination instruction exists, all the modules are terminated, while if no termination instruction exists, the image reception and display processing is continued.
In this manner, in the preview mode, as shown in
Next, the operation of the PC 2 in the variation mode will be described using the flowchart of
First, in step S501, processing reflection time is measured. The main module 910 of the parameter configuration program issues an image processing request to the DVC 1 using the asynchronous transaction, and measures processing reflection time from the issuance of the request to detection of a processed image in correspondence with the image processing request. The details of the processing will be described later.
In step S502, the main module 910 obtains a current value of a parameter to be changed (color gain in the present embodiment) among the image processing parameters of the DVC 1. In the present embodiment, the color gain as the parameter to be changed has 0 to 10 integral values, and a color becomes deeper in accordance with increment of the integral value. The acquisition of color gain value is performed using the asynchronous transaction.
The main module 910 transmits an asynchronous packet to obtain a parameter value to the DVC 1, and the DVC 1 receives the packet then transmits an asynchronous packet where a current color gain value is set to the PC 2.
In step S503, the main module 910 reads the color gain value from the asynchronous packet received from the DVC 1, and sets the value in “Current” as an internal variable. The variable “Current” holds the current color gain value.
In step S504, preparation for acquisition and display of image outputted from the DVC 1 is made. More particularly, as in the case of step S402, the processing method setting module 901 performs initial setting of the respective image processing modules 902 to 904 and ensures the first to third memory areas.
In step S505, the respective image processing modules 902 to 904 perform image acquisition processing. The details of the processing will be described later.
In step S506, the main module 910 checks the presence/absence of mode change request. If the mode change request exists, the current mode is changed to the preview mode and preview mode processing in
In step S507, it is checked whether or not the parameter setting configuration program is to be terminated, and if not, the process moves to processing in
In
Note that as described above, in the present embodiment, the type of parameter to be changed can be switched by switching of the tab 305 in the variation display screen 300. Accordingly, the user, who has changed the color gain, may switch the tab 305 thereby select another parameter. The display on the variation screen in
Next, the processing reflection time measurement processing in step S501 will be described using the flowchart of
In step S601, a timer managed by the main module 910 is reset, thereby an internal variable PTime is set to “0”. In step S602, the main module 910 transmits an asynchronous packet, including a command requesting output of a color bar image as an example of an image having a known pattern, to the DVC 1, and starts the timer. Note that when a time difference from the timer reset to the command transmission is negligible, the timer may be started at the same time of the timer reset. In step S603, the main module 910 decompresses the received image using the decompression processing module 902, and detects whether or not a color bar image has been received.
That is, the DVC 1 outputs an image obtained by image capturing before the transmission of the color bar output request command at S602. On the other hand, a color bar image is a still image data having predetermined colors in predetermined positions. Accordingly, the main module 910 checks in the decompressed image whether or not pixel values in particular positions are predetermined pixel values, thereby easily determines whether or not the received image is an image obtained by image capturing or a color bar image.
In step S603, if it is determined that a color bar image has not been received, step S603 is repeated until a color bar image is detected. When a color bar image has been detected, then in step S604, the variable PTime is set to the timer value upon detection of the color bar image. Thus, the variable PTime holds processing reflection time from the image processing request from the PC 2 to the acquisition of the result.
In the present embodiment, the time from the transmission of the color bar output request command to the detection of color bar image as a result is measured as the processing reflection time. However, output request of another image may be transmitted as long as the image can be distinguished from ordinary image obtained by image capturing. For example, in a case where an image obtained by the DVC 1 can be outputted as a color image or monochrome image and output of color/monochrome image is selectable in accordance with a command transmitted from the outside, the time from transmission of a switch request command to detection of switched image is measured as the processing reflection time.
That is, when the digital video camera outputs a color image, the time from transmission of a monochrome video output request command to detection of a monochrome image is measured. Further, when the digital video camera outputs a monochrome image, the time from transmission of a color video output request command to detection of a color image is measured. The detection of monochrome/color image is easily performed by checking color difference information of particular pixel on the image.
Next, the image acquisition processing in step S505 will be described using the flowchart of
In step S701, the main module 910 configures the color gain value of an image obtained next as Pvalue. For example, when the value of the variable Current holding the current value obtained in step S503 is “5”, the Pvalue is set to “4” decremented by “1” from the current value.
In step S702, the main module 910 transmits an asynchronous packet, including a command to configure the value of parameter to be changed to the value of the PValue, to the DVC 1. In step S703, the process stops (sleeps) during the Ptime, waiting for determination of image changed using the Pvalue.
Note that “the process stops” means transition to a state where electric consumption is less than that in a normal processing state (power saving mode). Accordingly, in addition to the stoppage of the processing, arbitrary operations performed in a so-called sleep mode or power saving mode such as slow down of the clock of the CPU, the power off of the display, and stoppage of hard disk drive may be performed.
After the elapse of the PTime, the process restarts, and in step S704, the received image data is stored into the first memory area (905) as shown in
Further, the resize processing module 903 changes the I frame image data stored in the second memory area 906 to the display size (size corresponding to the area 304), and stores the resized data into the third memory area 907 as shown in
In the variation mode, decompression, resizing and display processings are performed on a part of image frames constructing image data, particularly I frame. Accordingly, the processing load is light. Further, after the transmission of parameter change request command in step S702, the process stops during the PTime in step S703. Accordingly, in the variation mode, the electric consumption can be saved.
In step S706, an image recorded on the third memory area 907 is displayed in one of the areas 304 of the variation display screen 300 by the display module 904. Further, when the parameter value is set to the Current value, it is displayed in the central area 302.
Thus, the image processed with the current parameter value arranged in the central position can be compared with the images resulted from processing with respectively-changed different the parameter values at once. The user can set a parameter to obtain a desired result.
In the present embodiment, the above processing is repeatedly performed on a color gain value which can be set to “0” to “10”. That is, in step S707, the main module 910 determines whether or not images corresponding to all the configurable parameter values have been obtained. If it is determined that the images have not been obtained, the main module 910 configures the Pvalue to one of the remaining values, then the image acquisition is performed. As the configuration of the Pvalue is not the subject matter of the present invention, the Pvalue may be sequentially configured from “0” to “10”, or may be configured in accordance with other algorithms.
Further, it is not necessary to perform the image acquisition processing on all the configurable values of the parameter to be changed. Especially, regarding a parameter having many configurable values or minutely-configurable parameter, it may be arranged such that the image acquisition processing is performed on only discrete values corresponding to the number of the areas 304 in the variation display screen 300.
Otherwise, it may be arranged such that regarding the same parameter, different parameter value ranges are allocated to respective screens, and the screen is switched using the tab 305. For example, when the number of areas 304 is five per one screen and a parameter having configurable ten values other than the Current value is handled, images corresponding to value 1 to value 5 are displayed in the areas 304 of screen 1, and images corresponding to value 6 to 10 are displayed in the areas 304 of screen 2. The screen 1 and screen 2 are switched in accordance with operation of the tab 305.
Further, it may be arranged such that upon change from the screen 1 to the screen 2, the image acquisition processing to obtain the images corresponding to the value 6 to value 10 is performed.
As described above, according to the present embodiment, it is possible to configure an image processing parameter in an image input device from an external device while checking image data obtained by the image input device on a display of the external device. Especially, when the external device is an image processing apparatus, as the parameter value can be configured so as to obtain a desired result on the image processing apparatus side, the problem that the result differs in accordance with stage of image processing can be solved.
Further, the parameter value configuration can be more easily performed by displaying images obtained with a plurality of parameter values such that the images can be compared with an image obtained with a current parameter value.
Further, when a parameter value is configured, as the processing load can be reduced, the electric consumption can be reduced, and the parameter value configuration can be performed in an image capturing site.
Further, the above exemplary embodiment may be realized as software using a computer (or a CPU, an MPU or the like) of a system or an apparatus.
Accordingly, a computer program installed into the computer to realize the functional processings of the present invention also realizes the present invention. That is, the computer program to realize the functional processings of the present invention is included in the present invention.
In such case, the computer program may be executed in any form, such as an object code, a program executed by an interpreter, or script data supplied to an OS as long as it has the functions of the computer program.
In this case, the computer program to realize the functional processings of the present invention is supplied to the computer via a storage medium or cable/radio communication. Examples of storage media that can be used for supplying the computer program are magnetic storage media such as a flexible disk and a hard disk, optical/magneto-optical disks such as an MO, a CD and a DVD, and nonvolatile semiconductor memories.
As for the method of supplying the computer program using cable/radio communication, a server on a computer network may be utilized. In this case, a computer program file as a file including the computer program constituting the present invention is stored in a server. As the computer program file, an executable file or source code may be used.
Then, the computer program file is supplied to a client computer accessed the server by downloading. In this case, the computer program file may be divided into a plurality of segment files and stored in different servers.
That is, the server to supply the computer program file to realize the functional processings of the present invention to the client computer is also included in the present invention.
Further, it is also possible to encrypt and store the computer program of the present invention on a storage medium, distribute the storage medium to users, supply key information to users who meet certain requirements, and allow these users to decrypt the encrypted program by using the key information and install the computer program in the user computers. The key information is supplied by e.g. downloading from a website via the Internet.
Further, the computer program that causes the computer to implement the functions described in the above exemplary embodiment may use an operating system already running on the computer.
Further, it may be arranged such that at least a part of the computer program constituting the present invention is provided as firmware such as an expansion board inserted into a computer, and the functions described in the above exemplary embodiment are realized utilizing a CPU of the expansion board or the like.
While the present invention has been described with reference to the exemplary embodiments, it is to be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all modifications, equivalent structures and functions.
This application claims the benefit of Japanese Patent application No. 2005-310501, filed on Oct. 25, 2005, which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2005-310501 (PAT. | Oct 2005 | JP | national |
