Patent Application
20080068625
The present description will be directed in particular to elements forming part of, or cooperating more directly with, apparatus and methods in accordance with the present invention. It is to be understood that elements not specifically shown or described may take various forms well known to those skilled in the art.
The image control system 18 includes a controller or logic and control unit (LCU) 20, preferably a digital computer or microprocessor operating according to a stored program for sequentially actuating the workstations within printer system 10, effecting overall control of printer 12 and its various subsystems, including the image capturing device 14, such as a scanner and or camera or other device with scanning capability and a colorimetric measurement device 16, and related devices and sensors. These plus other possible components and software make up the image control system 18, which can be described as a serial combination of digital workflow process and one or more color reproduction devices, such as various physical printing processes. The Logic and Control Unit (LCU) 20 including one or more computers acting in response to signals from various sensors associated with the apparatus provides timing and control signals to the respective components to control the various components and process control parameters of the apparatus in accordance with methods well known by those skilled in the art. The image control system 18 includes the controller or logic and control unit (LCU) 20, preferably a digital computer or microprocessor operating according to a stored program for sequentially actuating the workstations within printer 12, effecting overall control of printer 10 and its various subsystems. Aspects of process control are described in U.S. Pat. No. 6,121,986 incorporated herein by this reference.
The LCU 20 includes a microprocessor and suitable tables and control software which is executable by the LCU 20. The control software is preferably stored in memory associated with the LCU 20. Sensors associated with the fusing and glossing assemblies, as well as other image quality features, provide appropriate signals to the LCU 20. In any event, the glosser and other image control elements can also have separate controls providing control over items such as the temperature of the glossing roller and the downstream cooling of the fusing belt and control of glossing nip pressure. In response to one or more sensors, the LCU 20 issues command and control signals that adjust all aspects of the image that determine image quality, such as the heat and/or pressure within fusing nip (not shown) so as to reduce image artifacts which are attributable to and/or are the result of release fluid disposed upon and/or impregnating a receiver member that is subsequently processed by/through a finishing device such as a glossing assembly (not shown). Additional elements provided for control may be assembled about the various module elements, such as for example a meter for measuring the uniform electrostatic charge and a meter for measuring the post-exposure color within a patch area of an image area on the printed surface.
The color printing device and system needs to be calibrated and characterized for accurate color reproduction that incorporate a graininess correction. This includes setting the printer to a standard specification for each color separation as well as printing and measuring large numbers of test patches to construct an accurate color transformation. The color transformation that characterizes the printing system maps color, bi-directionally, between device-dependent color (used in printing a document) and device-independent color (e.g. in the document to be printed). For example, it transforms the device independent CIELAB color system data, defined above, into CMYK data. The color transforms may be stored in a look-up table (LUT) format, in general, for easy access in processing color data. The International Color Consortium (ICC) color profile, which characterizes the printing system, contains multiple color transformation tables in LUT form.
Corrections can be made for graininess, which is a subjective response, as well as for granularity, which is an objective measurement. One way to correct for granularity (an objective measurement) is by using a fixed look up table (LUT) with or without a graininess conversion, such as described below. One difference between these measurements and related corrections is that granularity does not put an importance (visual weighting function) on the visual sensitivity to grain based on color, but graininess does. When only monitoring a few fixed color which are already pre-weighted, then granularity measurement may be sufficient from a monitoring and control viewpoint. In addition it is possible to use granularity tracking and corrective action in conjunction with the use of ROI and accumulation discussed below to take corrective action. This is valuable especially if somebody does not plan to go across colors. For example when only monitoring cyan, magenta, yellow etc on process control patches, then granularity measurement will do. Even for specific color such as a certain skin tone. Graininess is useful when we go across in diagnostic, such as yellow is less important than magenta, for example, so when it comes to corrective action, the problem has to be a lot bigger in yellow granularity before the fix needs to be done compared to magenta granularity. Graininess values even the field, hence corrective action is taken if a certain visual grain is above a threshold. Granularity will need different thresholds for different colors. For more complicated image content, graininess metric is easier to use. However, if only a few colors need to be tracked, granularity metric can be used effectively. Of course, a limited set of threshold values can also be used for graininess values measurement and control (for example, personal preference of one color over the others, etc.).
The LCU 20 also is programmed to provide closed-loop control of printer machine 10 in response to signals from various sensors and encoders. Aspects of process control are described in U.S. Pat. No. 6,121,986 incorporated herein by this reference. The printing device prints one or more images in one or more colors, including black and clear. The image may be included in a set of one or more images, such as in images of the pages of a document. An image may be divided into segments, objects, or structures each of which is in itself an image. A segment, object or structure of an image may be of any size up to and including the whole image.
The LCU 20 will typically include temporary data storage memory, a central processing unit, timing and cycle control unit, and stored program control. Data input and output is performed sequentially through or under program control. Input data can be applied through input signal buffers to an input data processor, or through an interrupt signal processor, and include input signals from various switches, sensors, and analog-to-digital converters internal to printing system 10, or received from sources external to printing system 10, such from as a human user or a network control. The output data and control signals from LCU 20 are applied directly or through storage latches to suitable output drivers and in turn to the appropriate subsystems within printing system 10. The look-up table based on a complete color representation in the relevant color dimension from three color dimensions, e.g. {L*,a*,b*}, in the CIELAB color space. This can include a nominal graininess value for each color that provides a nominal expected graininess value and/or granularity for that printer, paper, color, screen and other distinguishable criteria that affect the output, including the quality of the print. Other methods could be used and include graphical or mathematical representations of the corrective measures and could also include graphical or mathematical representations of the nominal values in a like way.
Before color control and calibration was supplied using accurate macroscopic color measurement and relied on the use of a colorimetric measurement device 16, such as a colorimeter or a spectrophotometer, to provide a surrogate for human visual response. The colorimetric measurement using a spectrophotometer first estimates the entire spectral response of the reflected light, and then, using standardized response characteristics, converts the spectral response to provide measurements according to color standards, such as {CIE XYZ, {CIELab} and {Reflection Densities} and achieves highly accurate color estimation with small between-instrument and inter-instrument variations.
The traditional scanner calibration utilizes a test target, printed with a fixed set of colorants that samples the entire printer device color space, for example CMYK or RGB. A scan of this well-defined test target provides a representation of the target in Scanner RGB values. A colorimetric measurement device, such as spectrophotometer, is also utilized to provide colorimetric measurements of this same printed test target. A multidimensional color mapping function from the scanner device color space, Scanner RGB, to the colorimetric color space, such as CIELAB, can then be created. Because the scanner is not a colorimetric measurement device, the color mapping function has to be re-created when one or more colorants are changed. Moreover, it is well known that this straightforward approach is not accurate enough for applications demanding high color stability and accuracy.
The image control system 18, including an image capture system adapted to capture a digital image of the receiver after an image has been printed, and to generate captured image data reflecting the appearance of the image on the receiver is shown in
The image capturing device 14, here shown as the color scanner in the image control system 18, is different from the colorimetric measurement device 16 because of differences in the sensor spectral responsitivities from the responsivities of the CIE standard observer. Thus, observer color metamerism imposes a lower boundary on the measurement error when the image capturing device 14 is used to replace the macroscopic colorimetric measurement device as is done in the image control system 18 and the related method. The image control device is able to benefit from the attributes that the image capturing device 14 has, such as its capability of capturing large areas in fine detail allowing a scanner to quantify microscopic image artifacts and its convenience, efficiency and lower cost. The image control system 18 is able to replace the macroscopic colorimetric measurement device with the image capturing device 14 in the image control system 18 by allowing only one colorant to be present in a test target at a time. As a result, only one-dimensional color information is available to describe the color appearing on the page in a restricted manner.
Reflection densities are a natural choice in this controlled and highly restricted environment. However this constraint cannot be utilized when using the flatbed scanner to measure color with at least two colorants present. A complete color representation requires all three color dimensions, e.g. {L*,a*,b*}, in the CIELAB color space. To reach a compromise between a method, including an algorithm, applicability and transformation accuracy, assume that the printing workflow is transparent to all users. Although this assumption is rather restrictive, for example, when we are asked to evaluate image quality attributes objectively on any given printed target, the printing workflow is usually under control when a set of test targets is sent to a printing system for quality assessment and process diagnosis. As a result, it is reasonable to assume that this prior information with respect to the current printing workflow can be propagated to the following scanning process so as to facilitate the scanner calibration.
Basic subtractive color principles indicates that {cyan, magenta, yellow} can be considered as the complement colors of {red, green, blue} in terms of their spectral responsitivities. Assuming that a flatbed scanner with {Red}, {Green} and {Blue} sensors can reliably quantify the {cyan, magenta and yellow} colors on a reflective print, we can deduce that a fairly accurate color transformation can be constructed when only {cyan, magenta, yellow} colorants are present on a reflective print. However, since CMYK four-color printing (or even more than five colorants) is prevalent in the commercial printing industry, researchers have shown that the presence of the extra black colorant and all supplemental colorants will noticeably degrade the scanner calibration performance. It is possible to extract the supplemental color channel information and utilize this supplemental information in the scanner calibration and color transformation process that is applied to scanned documents in order to evaluate color information. In this particular embodiment, the scanner calibration method, including an algorithm, includes two portions: a quadratic global regression and neural network based residual approximation. The embodiment the data capture device for this method and algorithm can be a flatbed scanner with an automatic document feeder or a high-speed digital camera.
The system, shown here as a digital printing system but which could include other color reproduction systems such as color monitors or cell phone and camera monitors, includes an image control method 114 that begins by calibrating the color reproduction device, in this case in conjunction with a printing process 116, with high-accuracy using a graininess metric including the steps of identifying one or more ROIs (regions of interest) 118. Then determining a current graininess value for each selected color component in the ROI 120, and calculating a graininess difference between the current graininess value and a nominal expected graininess value 122 based on nominal granularity value(s) 124 that are from a LUT or other sources as will be discussed below in more detail. Then determining when the calculated graininess difference falls outside an expected range 126 based on a tolerable or expected range 127 and taking corrective action 128 based on a corrective list or similar information. Although the invention is being described as part of a printing process, it would be understood by those skilled in the art that this system and the corrective method could be applied to any color reproduction device and that the document represents other types of images, such as a color monitor or screen in a camera or cell phone for example.
Since the corrective action is for image enhancement, this method can be applied in the spatial, frequency, or spatial-frequency domains in conjunction with or without image enhancement filters. The objective is to create an accurate translation process between the image-capturing device 14 and the colorimetric measurement device 16 because the colorimetric measurement device closely relates to human visual response while a digital capturing device might not. It is important that the image-capturing device 14 and the colorimetric measurement device 16 are measuring the same location to achieve a valid and accurate translation process.
The scanner 14, such as a flatbed scanner is adopted for its capability to capture a wide frequency response range. Color calibration is conducted with respect to every set of colorants on printed samples (4c and 5c) to reduce possible metamerism error with calibration method discussed in ‘Perceptual Color Graininess of Printed Pages via flatbed Scanner, by Yee S. Ng, Chunghui Kuo and Di Lai in the Final Programs and Proceedings, ICIS '06 International Congress of Imaging Science, pp. 122-125’. As shown in
A color grain metric, also referred to as a graininess metric, can then be built to link the objective color granularity measurement using the flatbed scanner to subjective graininess value as discussed in the Ng ICIS '06 paper. It is also possible to create the graininess metric using other area sensors including use of one of an in-line scanner with linear sensors, flatbed scanner, and camera, such as a 2-D camera. As shown in
The de-screened images (patches) are transformed into the L*a*b* color space via the RGB→L*a*b* transformation process outlined in the scanner calibration process mentioned above. Then the de-screened images (in L*a*b* space) are used to obtain a color granularity estimate 222 (based on CIEDE2000 color difference space) and a psychophysical experiment is done to correlate with subjective color graininess. The color granularity estimate 222 can be obtained in other ways such as with the use of accumulated data as will be discussed below. For the color granularity estimate 222, a sampling area of 12.72 mm2 and sampling block size of 1.27 mm×1.27 mm according to ISO13660 Standard is used for granularity (represented in color variation in the CIEDE2000 color difference space as discussed in the Kuo NIP21 paper) measurement. Let σi represents the standard deviation in color (in L*a*b* space) of each block, we can derive the color granularity
of the sampled area. After the psychometric color grain experiment one can build a color grain metric 225 that links the color granularity GΔE 226 to color visual grain, VGc 228, so that the visual importance of graininess can be linked to the granularity measurement. In the case of the Ng ICIS '06 paper, for example, one finds that VGc can be reasonably represented by the granularity GL due to L* variation 230 alone (namely σi from L* variation alone) with the following equation, and as shown in
VG
c=23.16×GL−3.6
Therefore, if color process control patches can be used in conjunction with a calibrated flatbed scanner or an in-line scanner for the printer, graininess (visually important) feedback can be obtained for the printing process, and corrective action can be taken when process graininess changes over time.
If the ROI (region of interest) identification can be made by customers (for areas of images that are of significance to the customer) on actual images (say viewable via a monitor), region growing method (with constraints, such as color difference of within 9 ΔE from the position that customer is pointing at, should be included) can be used to acquire color variation data in real-time printing from actual customer images (not test patches which are inefficient). Graininess reading can be obtained, monitored and feedback for control. Region growing method has been described in Kuo's NIP21 paper. The image control system can further incorporating pre-selected colors, such as skin-tone and blue sky, in actual customer images to automatically determine the ROI.
Graininess of the printing process for the printer color gamut can be characterized by printing out test patches (based on a combination of C,M,Y,K coverage combinations) covering regions of the color gamut. Then color granularity can be measured via the calibrated scanner (as discussed above), and the normal color graininess of the printing system including processes such as screening, toning, transfer and fusing are characterized using the graininess/granularity metric. A color graininess map for a 4-color printing process (plotted in the L*, a*, b* color space) is shown in
The * marks in
To simplify the discussion, let's look at the graininess map for a few process control patches (30%, 50%, 70% and 100% dot) in
However, using prints from process control patches can also mean waste for the system. Given the scanner's capability to acquire color information at high resolution, one can actually acquire calibrated color information (therefore the color variation information that shows up as color granularity) from customer prints 250. For example, one has shown before that with segmentation and region growing methods (Kuo NIP21 paper), if we have asked for operator input as to the color region to monitor as shown in the process in
Graininess can be a subjective response, but granularity is an objective measurement. Granularity can use a fixed LUT with or without the graininess conversion discussed below. The difference is that granularity does not put importance on the visual sensitivity to grain based on color, but graininess does. In many cases the correction for granularity is sufficient to achieve the desired results, such as when only monitoring a few fixed colors which are already pre-weighted, then granularity measurement will do from a monitoring and control viewpoint. For example, if one is only monitoring cyan, magenta, yellow etc on process control patches, then granularity measurement will be sufficient even for specific color such as a certain skin tone. Graininess can be useful when used as a diagnostic since colors can have different importance, such as when yellow is less important than magenta which would require paying attention to the magenta color performance first when it comes to corrective action for graininess. Graininess evens the field, so if a certain visual grain is above a threshold, fix it. Granularity will need a different threshold for different colors.
The length of bolder line in
Similar methodology can be applied for other popular colors such as skin tone. For example if operator select a picture location (Pj) with extended region of Caucasian skin tone of ˜L*a*b* values of (L*=64, a* of 27 and b* of 38), the equivalent color separations % coverage after color mapping in one implementation is {cyan=3%, magenta=46%, yellow=63%, black=7%}, namely the predominant separation is magenta and yellow. One can monitor the sky tone graininess with this method.
Now, if we are monitoring the blue sky and skin tone, and if there is a graininess increase of blue sky, but not skin tone, due to printing process problem (eg., caused by the cyan transfer alone), the graininess difference 272 of the blue sky (˜15.4) from the nominal graininess value can be tracked as shown in
On the other hand, if the graininess increase of the blue sky is coming from the magenta transfer, then the skin tone graininess will be affected as well as shown in
The interactive way of monitoring specific areas of interest by operator can be further extended into automatic monitoring. Since some of the most important memory color region of significant interest in graininess is known (for example the blue sky and skin tone area mentioned above). One can extend the monitoring concept to include colors of blue-sky tone of various types (blue sky with desert, blue sky with snow, blue sky with grass) and lightness, skin tones of various shades (Caucasian, Asian, Indian, African, etc). So one can acquire in-line (via calibrated scanner) graininess information of extended regions (size) of color within a certain limit of the monitoring colors from real prints. So even if the prints are changing, the accumulative information of graininess of the monitored colors can be used for new process baseline, diagnostic and corrective action purpose without printing wasteful process control patches.
Automating the image control system includes both accumulating graininess data over time so that it can be used by the system and/or operator and customer to determine what corrections should be made. This could be done by modifying the reset nominal values or changing the other criteria that effect print quality. It is suitable for variable images, versus the customer marked ROI or process controlled patches (both are fixed images). This automatic control system could also include separately or in combination with the automatic accumulation of graininess data, the automatic determination of ROI, such as incorporating pre-selected colors, such as skin-tone and blue sky, in actual customer images to automatically determine the ROI. The assigned colors are used to monitor and each color has a nominal value. Which can be represented in a LUT or other means. The system can accumulate data to adjust this nominal value or to compare to operating conditions and other input parameters as discussed above in conjunction to
When used in correcting graininess or in granularity tracking in conjunction with corrective action or ROI determination, as accumulation data, the objective measurement can often be all that is needed as far as a corrective action to achieve the desired results. This is especially valuable if somebody does not plan to go across colors. For example if one is only monitoring cyan, magenta, yellow etc on process control patches, then granularity measurement will do. Even for specific color such as a certain skin tone. Graininess is useful when we go across colors in a diagnostic, such as yellow is less important than magenta, for example, so when it comes to corrective action, one has be a lot bigger in granularity in yellow before the fix needs to be done compased to magenta. Graininess evens the field, so if a certain visual grain is above a threshold, fix it. Granularity will need different thresholds for different colors.
The preselected colors (such as memory colors of importance, skin-tone, blue sky and such) to monitor and accumulate from real images. The difference between customer selected ROI and process control patches to this mode is that in the preselected color monitoring case, the images are variable, while process control patches and customer selected ROIs are fixed images.
The invention has been described in detail with particular reference to certain preferred embodiments thereof, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.
