This application is based on application No. 2004-334464 filed in Japan, the contents of which are hereby incorporated by reference.
1. Field of the Invention
The present invention relates to processing on image data captured by a digital camera.
2. Description of the Background Art
As file formats for image data captured by digital cameras, JPEG format and TIFF format are known. However, file generation in the JPEG format or TIFF format causes a problem of deterioration in image data. Hereinbelow, a procedure for file generation and reasons for such deterioration will be discussed.
In digital cameras, image data captured by a CCD is converted into digital form in an AD converter. Then, shading correction, white balance control, pixel interpolation, color space conversion and gamma conversion are conducted as necessary, and thereafter, YCrCb conversion is performed to generate a luminance image and a color difference image. The luminance image is subjected to sharpness adjustment (edge enhancement), and the color difference image is subjected to smoothing.
In the JPEG format, the luminance image and color difference image are subjected to JPEG compression, and then generated into an image file (JPEG file). In the TIFF format, the luminance image and color difference image are subjected to RGB conversion, and then generated into an image file (TIFF file).
Data having undergone AD conversion generally has a bit length greater than 8 bit, however, the bit length gradually decreases in the above-described image processing. (For instance, data originally having a 12-bit length is brought into a 10-bit length after color space conversion, and an 8-bit length after gamma conversion.) Finally, image data has an 8-bit length in each of R (red), G (green) and B (blue) in the JPEG format or TIFF format.
The decrease in bit length and the occurrence of cancellation of significant digits means deterioration in data. Moreover, since JPEG compression is lossy compression, image deterioration is caused by compression.
RAW format is used as a file format for avoiding such image deterioration. In the RAW format, image data converted into digital form in the AD converter is subjected to shading correction only, and then generated into an image file (RAW file). Therefore, compression is not conducted, and cancellation of significant digits can be reduced to a minimum, allowing image deterioration to be reduced.
An image file generated in a digital camera is transmitted to an electronic processing terminal such as a PC (personal computer). In the JPEG or TIFF format, such image file can be viewed using an image processing application called a viewer, and image processing can further be performed. A RAW file can be viewed after conversion processing is performed thereon, and it is possible to perform image processing with high accuracy and high flexibility since various types of processing as described above are not performed previously on the RAW file.
As described, in the RAW format, image data having undergone AD conversion is subjected to shading correction only, and then generated into an image file. Therefore, in the case of performing conversion processing on the RAW file in the PC, image processing on the RAW file such as white balance control, pixel interpolation, color space conversion and gamma conversion needs to be performed in the PC. In this case, the RAW file has a great bit length and hence, a large volume of data, resulting in a problem of extended processing time.
It is an object of the present invention to provide an image processing system capable of shortening processing time.
According to the present invention, an image processing system includes an image capturing apparatus and an electronic processing terminal connectable to the image capturing apparatus. The electronic processing terminal includes an instruction part for giving an instruction to perform predetermined processing on image data and an unprocessed data transmitter for transmitting the image data to be subjected to the predetermined processing to the image capturing apparatus. The image capturing apparatus includes a receiver for receiving the image data transmitted from the electronic processing terminal, a processor for performing the predetermined processing on the image data as received to obtain processed image data, the processor having a specific processor specifically designed to a certain image processing function including the predetermined processing, and a processed data transmitter for transmitting the processed image data to the electronic processing terminal.
The predetermined processing is performed in the specific processor included in the image capturing apparatus, not in the electronic processing terminal. Since this specific processor is specifically designed to a certain image processing function including the predetermined processing, it is possible to shorten processing time as compared to the case of using a general-purpose functional processor in the electronic processing terminal.
These and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.
Configuration
The digital camera 1 has a lens device 2 and a viewfinder 6 on its front face; a main switch 4 and a shutter-release button 5 on its top face; and a media insertion slot 3 removably accepting a media 7 serving as a memory (recording medium) including a memory card and the like, on its side face.
The PC 10 has a monitor 14 for displaying information in the form of an image to a user and a keyboard 15 for a user to input information. The PC 10 further has a media insertion slot 12 removably accepting the media 7 on the front face of its body.
The viewfinder 6 allows a user to view a subject therethrough. The mode-switching dial 22 is a dial for switching among modes such as a still image capturing mode for capturing a still image, a motion image capturing mode for capturing a motion image and a playback mode for playing back a captured image. The four-way cross key 23 is a key for selecting among various operations such as changing of zoom magnification. The LCD 24 is a screen for displaying image data. The decision button 25 is a button for deciding an operation.
In
The above-mentioned lens device 2, CCD 32, AD converter 37, image data processor 33, LCD 24, media 7 and memory 34 are controlled by a CPU 31. The image data processor 33 is capable of processing not only image data transmitted from the AD converter 37 but also image data stored in the media 7 or memory 34.
The CPU 31 controls the entire digital camera 1 including an external I/F (interface) 35 for transmitting image data or the like to/from the outside by communications under USB or the like via the cable 11, a power controller 36 for performing power control on the entire digital camera 1 and a flash 38 serving as a light emitter. The power controller 36 supplies power using any one of the external I/F 35, an AC power port 39 and a built-in battery 40.
Next, referring to
The CCD 32 generates image data by image capturing through the lens device 2.
The generated image data is converted into digital form in the AD converter 37.
The image data having converted into digital form is subjected to compensation for insufficient illumination in an area corresponding to the periphery of the lens device 2 in a shading corrector 51.
The image data having undergone compensation for insufficient illumination is generated directly into a RAW file in the RAW format. This RAW file contains image data of RAW format, tag information including image capturing information and the like and thumbnail image data, which will be described later referring to
The image data having undergone white balance control is subjected to pixel interpolation in a pixel interpolator 53. Specifically, as shown in
The image data having undergone pixel interpolation is subjected to color space conversion in a color space converter 54 as necessary into an sRGB color space, for example.
The image data having undergone color space conversion is subjected to gamma value correction in a gamma corrector 55.
The image data having undergone gamma correction is converted from RGB form into YcrCb form in a YCrCb converter 56. Accordingly, a luminance signal Y (luminance image data) and color difference signals (color difference image data) Cr, Cb are generated.
The generated luminance image data is subjected to sharpness adjustment in an edge enhancer 57. The generated color difference image data is subjected to smoothing in an LPF part 58.
In the JPEG format, the luminance image data having undergone sharpness adjustment and color difference image data having undergone smoothing are subjected to JPEG compression in a JPEG compressor 59. In the TIFF format, the luminance image data having undergone sharpness adjustment and color difference image data having undergone smoothing are converted into RGB form in an RGB converter 60.
The image data having undergone JPEG compression or image data having converted into RGB form is generated into a JPEG file or TIFF file, each containing tag information and thumbnail image data, and is stored in the media 7.
In
The controller 41 includes a white balance controller 52a, a pixel interpolator 53a, a color space converter 54a, a gamma corrector 55a, a YCrCb converter 56a, an edge enhancer 57a, an LPF part 58a, a JPEG compressor 59a and an RGB converter 60a.
These processing parts of the controller 41 have similar functions as those having similar names provided in the image data processor 33 shown in
Overall Operation
Hereinbelow, the overall operation of the image processing system shown in
First, referring to
As shown in
Next, as shown in
Next, the user sets desired computing parameters for processing the image 66a by operating the keyboard 15 or the like. As such computing parameters, a gain in white balance control, a gamma value in gamma correction, the degree of edge enhancement, the range of smoothing and a weighting coefficient in LPF and the like can be set.
Next, as shown in
Next, the operation of the image processing system will be discussed in detail referring to a flow chart shown in
First, in step S1, the thumbnail images 63 to 67 (and the folder tree) are displayed on the monitor 14.
Next, the process proceeds into step S2, in which the user selects the thumbnail image 66.
Then, the process proceeds into step S3, in which the PC 10 judges whether the image 66a corresponding to the image 66 is in the RAW format (RAW image). When the image 66a is a RAW image, the process proceeds into step S4. When the image 66a is not a RAW image, the process proceeds into step S5.
In step S4, the image 66a which is a RAW image is subjected to conversion processing (here, conversion into a displayable image representing format). At this time, image data of the image 66a and computing parameters are transmitted from the PC 10 to the digital camera 1 through the cable 11. As computing parameters, default settings of the PC 10 may be used. Since the image 66a only needs to be displayable at this stage on the precondition that image processing is performed in a later step, computing parameters for conversion may be provided in the digital camera 1.
The image data received by the digital camera 1 is sequentially subjected to processing based on computing parameters in the white balance controller 52, pixel interpolator 53, color space converter 54, gamma converter 55, YCrCb converter 56, edge enhancer 57, LPF part 58 and RGB converter 60. The image data having undergone the above processing is transmitted from the digital camera 1 to the PC 10 through the cable 11. Then, the process proceeds into step S7.
In step S5, the PC 10 judges whether the image 66a is in the JPEG format (JPEG image). When the image 66a is a JPEG image, the process proceeds into step S6, in which JPEG decompression is performed, and the process proceeds into step S7. When the image 66a is not a JPEG image, it is in the TIFF format (TIFF image), which does not require either conversion or JPEG decompression, and the process proceeds directly into step S7.
In step S7, the image 66a is displayed on the monitor 14. More specifically, in the case of displaying the image 66a from an image file generated in the digital camera 1, conversion processing including white balance control and subsequent steps is not necessary for a JPEG or TIFF image, but is necessary for a RAW image. Therefore, conversion processing is performed in the image data processor 33 having a higher processing speed, not in the controller 41. This allows processing time for a RAW image to be shortened.
Next, the process proceeds into step S8, in which the PC 10 asks the user whether to perform predetermined image processing on the image 66a through the monitor 14. The user replies whether to perform processing on the image 66a by operating the keyboard 15 or the like. In the case where processing is to be performed, the process proceeds into step S9. In the case where processing is not to be performed, the process returns to step S2, in which another desired thumbnail image is selected among the thumbnail images 63 to 67.
In step S9, the user sets desired computing parameters for processing the image 66a by operating the keyboard 15 or the like.
Next, the process proceeds into step S10, in which the PC 10 judges whether the image 66a is a RAW image. When the image 66a is a RAW image, the process proceeds into step S11. When the image 66a is not a RAW image, the process proceeds into step S12.
In step S11, the image 66a which is a RAW image is processed. In this case, image processing is performed on RAW image data yet to undergo conversion processing in step S4. The RAW image data corresponding to the image 66a and the computing parameters as set in step S9 are transmitted from the PC 10 to the digital camera 1 through the cable 11.
The RAW image received by the digital camera 1 is sequentially subjected to processing based on the computing parameters in the white balance controller 52, pixel interpolator 53, color space converter 54, gamma converter 55, YCrCb converter 56, edge enhancer 57, LPF part 58 and RGB converter 60. This means conversion and image processing are performed. The image data having undergone the above processing is transmitted from the digital camera 1 to the PC 10 through the cable 11. The process proceeds into step S15.
In step S12, the PC 10 judges whether the image 66a is a JPEG image. When the image 66a is a JPEG image, the process proceeds into step S13, in which JPEG decompression is performed, and then the process proceeds into step S14. When the image 66a is not a JPEG image, it is a TIFF image, which does not require JPEG decompression, and the process proceeds directly into step S14.
In step S14, the PC 10 performs image processing on image data corresponding to the image 66a which is a JPEG or TIFF image.
Since a JPEG or TIFF image does not require either white balance control or pixel interpolation and only those among subsequent steps for which parameters are determined need to be performed, processing is performed in the PC 10. More specifically, steps for which parameters are determined among steps performed in the color space converter 54, gamma converter 55, YCrCb converter 56, edge enhancer 57, LPF part 58 are performed sequentially. Then, the process proceeds into step S15.
In step S15, the result of image processing is displayed. The user can thereby check whether or not a desired result has been achieved. Then, the process proceeds into step S16.
In step S16, the PC 10 asks the user whether to store the image. When the image is to be stored, the process proceeds into step S17. When the image is not to be stored, the process is finished.
In step S17, a file is generated in a format corresponding to the RAW image processed in step S11 or the JPEG or TIFF image processed in step S14, and the PC 10 stores the file in the memory 42 or media 7. Through the above steps S1 to S17, a series of processes by the viewer is completed.
Conversion Processing
As described, in the image processing system according to the present embodiment, the digital camera 1, not the PC 10, performs conversion processing on a RAW image. Hereinbelow, steps for the conversion processing will be discussed referring to
First, the operation of the PC 10 will be discussed referring to the flow chart shown in
First, in step S21, the PC 10 judges whether the digital camera 1 is connected thereto. When the digital camera 1 is connected, the process proceeds into step S22. When the digital camera 1 is not connected, the process proceeds into step S25.
Next, in step S22, the PC 10 judges whether the digital camera 1 is busy (executing another process). When it is busy, the process proceeds into step S25. When it is not busy, image processing is executable in the digital camera 1, and thus, the process proceeds into step S23.
In step S23, the PC 10 transmits image data and computing parameters to the digital camera 1, and the process proceeds into step S24. More specifically, in steps S21 to S23, the PC 10 judges whether image processing is executable in the digital camera 1 referring to information about the digital camera 1, to thereby perform transmission control.
In step S24, the PC 10 gives an instruction to cause the digital camera 1 to perform image processing, and then, receives image data having been processed from the digital camera 1.
In step S25, the PC 10 itself performs image processing using the controller 41 without transmitting image data to the digital camera 1. In this case, image processing is performed by software in the PC 10 although it takes longer time than in the digital camera 1. Through the above steps S21 to S25, the operation of the PC 10 is completed.
Next, the operation of the digital camera 1 will be discussed referring to the flow chart shown in
First, in step S31, the digital camera 1 judges whether image data and computing parameters have been received from the PC 10. Only when they have been received, the process proceeds into step S32.
In step S32, the digital camera 1 stores the received image data and computing parameters in the memory 34 or media 7. The volume of data increases by processing image data. In the case where either of the memories lacks enough free space for storing processed image data, the digital camera 1 informs the PC 10 that the processed image data cannot be stored due to the lack of enough space. The judgment as to whether or not storage is possible may be made by the PC 10, not by the digital camera 1. More specifically, in the flow chart shown in
Next, the process proceeds into step S33, the digital camera 1 judges whether a computation start command (processing start signal) has been received from the PC 10. Only when the command has been received, the process proceeds into step S34.
In step S34, the digital camera 1 processes the received image data based on the received computing parameters.
Next, the process proceeds into step S35, in which the digital camera 1 stores the result of computation (processed image data) in the memory 34 or media 7.
Next, the process proceeds into step S36, in which the digital camera 1 transmits a computation completion command to the PC 10.
Next, the process proceeds into step S37, in which the digital camera 1 judges whether a computation result transmission command (transmission request signal) has been received from the PC 10. Only when the command has been received, the process proceeds into step S38.
In step S38, the digital camera 1 transmits the result of computation (processed image data) to the PC 10.
Through the above steps S31 to S38, the operation of the digital camera 1 is completed. The above description is directed to the case of processing one image (66a), however, in the case of processing a plurality of images, steps S31 to S38 may be repeated by the number of cycles equal to that of images. Alternatively, a plurality of pieces of image data in a batch may be transmitted to the digital camera 1, subjected to a computation and transmitted back to the PC 10, which means steps S31 to S38 are performed only by one cycle.
As described, the image processing system according to the present embodiment performs conversion processing in the specific processor (image data processor 33) in the digital camera 1, not in the general-purpose functional processor (controller 41) in the PC 10. This can achieve the effect of shortening processing time.
In the first preferred embodiment, only sections of the digital camera 1 related to computing and communications (transmission of image data) are used in the operation shown in the flow chart of
In step S31-1, the digital camera 1 uses the power controller 36 to stop power supply to the sections not related to computing and communications as well as to switch the power source to an optimum one. In step S38-1, the digital camera 1 uses the power controller 36 to restart power supply to the sections to which power supply has been stopped as well as to switch the power source to the original one.
First, in step S311, the power controller 36 stops power supply to the CCD 32, flash 38 and the like not required for computing and communications. In other words, the power controller 36 serving as a power supply controller according to the present invention selectively supplies power to the receiver, processor and processed data transmitter.
Next, in step S312, the power controller 36 judges whether an AC power is connected to the AC power port 39. When the AC power is connected, the process proceeds into step S313, in which the AC power is selected as a power source. When the AC power is not connected, the process proceeds into step S314.
Next, in step S314, the power controller 36 judges whether the PC 10 is connected to the external I/F 35 of the digital camera 1 with the cable 11. When the PC 10 is connected, the process proceeds into step S315, in which the PC 10 is selected as a power source. In this case, power is supplied from the PC 10 through the external I/F 35. When the PC 10 is not connected, the process proceeds into step S316, in which the built-in battery 40 (electricity storage unit) is selected as a power source.
As described, in the image processing system according to the present embodiment, the digital camera 1 stops power supply to the sections not related to computation and communications during a computation to thereby reduce power consumption, and selects a power source from the AC power, PC 10 and built-in battery 40 giving priority in this order. Therefore, supplying power from outside the digital camera 1 to the extent possible can achieve the effect of stabilizing operation as compared to the case of using the built-in battery 40 only.
In the case where abnormality occurs in a power source during a computation, the power controller 36 may exercise control such that a defective power source is switched to another one having no defect through the procedure shown in steps S312 to 316 in
In the operation shown in the flow chart of
In step S21-1, the PC 10 judges which of the digital camera 1 and PC 10 performs conversion processing faster. This judgment is made based on the computing power of the digital camera 1, the computing power of the PC 10, communications time required for data transmission between the digital camera 1 and PC 10 and the volume of computation. More specifically, computing time predicted by the following equations (1) and (2), that is, first predicted time T1 and second predicted time T2 are calculated and compared.
First predicted time T1=volume of computation/computing power of digital camera 1+communications time (1)
Second predicted time T2=volume of computation/computing power of PC 10 (2)
In the equations (1) and (2), the computing power of each of the digital camera 1 and PC 10 is obtained based on the number of clock cycles and storage space of a processor of each of the digital camera 1 and PC 10, and the like. The volume of computation is determined based on the volume of data and details of computation, and the volume of data is obtained from the number of pixels of an image. The communications time is obtained by dividing the volume of data by the transmission speed of the communications line (cable 11) between the digital camera 1 and PC 10.
When T1<T2 holds, the PC 10 judges that the digital camera 1 can perform conversion processing faster, and the process proceeds into step S21. When T1>T2 holds, the PC 10 judges itself as being able to perform conversion processing faster, and the process proceeds into step S25, in which conversion processing is performed by the PC 10.
As described, in the image processing system according to the present embodiment, the PC 10 itself performs conversion processing without transmitting image data to the digital camera 1 in the case where the PC 10 judges itself as being able to perform conversion processing faster. This can achieve the effect of shortening processing time in addition to the effect achieved by the first preferred embodiment.
In the operation shown in the flow chart of
In step S33-1, the digital camera 1 analyzes whether the computing parameters stored in step S32 fall within a range that can be processed by the image data processor 33. Such range is determined based on the configuration of the image data processor 33 such that, for instance, a gain of white balance control ranges from 0.5 to 2.0, a gamma value from 1.0 to 2.0, a gain of edge enhancement from 1.0 to 2.0 and smoothing size of LPF is ±3.
Next, the process proceeds into step S33-2, in which, when these computing parameters fall within the above ranges, it is judged that computing is executable, and the process proceeds into step S34, in which a computation is performed in the digital camera 1. When these computing parameters do not fall within the above ranges, it is judged that computing is unexecutable, and the process proceeds into step S33-3, in which the digital camera 1 transmits a computation unexecutable command to the PC 10. Then, the process proceeds into step S33-4, in which the PC 10 performs a computation.
As described, in the image processing system according to the present embodiment, the PC 10 performs a computation when it is judged that the digital camera 1 cannot perform a computation. This can achieve the effect of reducing errors associated with computations.
Although it is described above that the PC 10 performs a computation when the computing parameters do not fall within the executable ranges in step S33-2, an approximate computation may be performed in the digital camera 1 using optimum parameters within executable ranges. For instance, when a gamma value is 2.5 which exceeds the upper limit, the digital camera 1 performs processing using a gamma value of 2.0 which is the upper limit. Therefore, it is possible to shorten processing time while reducing errors associated with computations.
Further, it is described above that the analysis and judgment in steps S33-1 and S33-2 are performed by the digital camera 1, however, this judgment may be performed by the PC 10 before transmitting image data and computing parameters to the digital camera 1. In the case where the digital camera 1 cannot perform a computation, it is possible to cause the PC 10 not to transmit image data and computing parameters to the digital camera 1 because the PC 10 makes the judgment. This can achieve improved processing efficiency of the image processing system.
In the conversion processing shown in the flow chart of
As described, in the image processing system according to the present embodiment, since computing parameters are described in the tag information D2, a RAW image data and computing parameters can be handled as one piece of data. Therefore, it is possible to handle the RAW image data and computing parameters easily.
In the flow chart shown in
In the operation shown in the flow chart of
First, in step S51, the digital camera 1 judges whether image data is present in the media 7. When image data is present, the process proceeds into step S52, and when no image data is present, the process is finished.
In step S52, the digital camera 1 judges whether computing parameters are present in the media 7. When computing parameters are present, the process proceeds into step S53, and when no computing parameters are present, the process is finished.
In step S53, the digital camera 1 performs a computation based on detected image data and computing parameters.
Next, the process proceeds into step S54, in which the digital camera 1 records the result of computation (image data having been processed) on the media 7. Through the above steps S51 to S54, the operation of the digital camera 1 is completed.
As described, in the image processing system according to the present embodiment, the digital camera 1 receives data employing the media 7 instead of the cable 11 used in the first preferred embodiment. Therefore, the PC 10 can execute another job without the need to carry out communications with the digital camera 1 while the digital camera 1 is performing a computation. This can achieve more improved processing efficiency.
In the operation shown in the flow chart of
In step S22-1, the PC 10 searches for image data to be computed in the digital camera 1. This search is made based on attribute information such as the name of a data file, the date of file generation, capacity and the like. More specifically, the PC 10 makes a request of the digital camera 1 to provide file attribute information. The digital camera 1 searches the contents of the media 7 inserted into the media insertion slot 12 to obtain file attribute information and provide it for the PC 10. The PC 10 checks whether a file identical to image data to be computed is stored in the digital camera 1. When a file identical to image data is stored, the process proceeds into step S22-3, and when a file identical to image data is not stored, the process proceeds into step S23. In step S22-3, the PC 10 transmits only computing parameters (and computation start command) omitting transmission of image data to the digital camera 1.
Generally, in image capturing in the digital camera 1 and image processing (conversion by the viewer) in the PC 10, a user follows a procedure for capturing an image by the digital camera 1, then connecting the digital camera 1 and PC 10, and capturing image data in the digital camera 1 into the PC 10. Therefore, image data to be computed may already be present in the digital camera 1 in the case where the digital camera 1 having captured an image is connected to the PC 10 for image processing Just after image capturing.
As described, in the image processing system according to the present embodiment, transmission of image data from the PC 10 is omitted in the case where image data to be computed is already present in the digital camera 1. This can achieve more improved processing efficiency.
While the invention has been shown and described in detail, the foregoing description is in all aspects illustrative and not restrictive. It is therefore understood that numerous modifications and variations can be devised without departing from the scope of the invention.
Number | Date | Country | Kind |
---|---|---|---|
JP2004-334464 | Nov 2004 | JP | national |