This Application claims priority of Taiwan Application No. 098142839, filed on Dec. 15, 2009, the entirety of which is incorporated by reference herein.
1. Field of the Invention
The invention relates to image generation methods and related image processing systems, and more particularly, to halftone image generation method and related image processing system for converting a grayscale image into a halftone image using an improved dot diffusion method.
2. Description of the Related Art
Digital halftoning is a process to display grayscale images with a two-tone texture pattern. Halftoning is mainly used as printouts for materials such as magazines, newspapers, and books, generating a black-and-white format. Halftoning mainly takes advantage of the fact that the human visual system is not highly sensitive, so that black and white pixels of a dense uniform grid may be used to represent a desired grayscale effect. The halftoning methods include ordered dithering, dot diffusion, error diffusion, and direct binary search (DBS). Of these methods, the dot diffusion uses a class matrix and a diffused weighting and through parallel processing provides an acceptable image quality and faster processing efficiency. The dot diffusion generally used is the dot diffusion halftoning algorithm proposed by Knuth and Mese. The dot diffusion halftoning algorithm proposed by Knuth is a kind of algorithm that attempts to retain the advantages of error diffusion and simultaneously provide parallel processing. The dot diffusion has only one design parameter, namely the class matrix, which determines the order of the pixels to be halftone processed.
Based on the concept that in a class matrix the processing order will significantly affect the reconstruction image quality, the optimization proposed by Knuth aims to reduce the baron (no members of higher numerical value currently exist around the member being processed) and near-baron (only one member of higher numerical value currently exists around the member being processed) in the class matrix. Although this concept is simple and direct, the method does not consider the human visual characteristics, resulting in the generation of images that strain the human eyes. Based on an idea for improvement, Mese took into account the human visual characteristics in the optimization process of his class matrix.
However, Mese only used a single grayscale value 16 in the training set to obtain the final class matrix in his optimization process. The trained class matrix may not generate the best results when set in the natural images containing other grayscale values. Moreover, Mese's class matrix optimization process failed to take into account the diffused weighting value and the diffused area, thus limiting the space for the growth of the trained class matrix for the reconstruction of image quality.
It is therefore an objective to provide halftone image generation methods and image processing systems to convert the original grayscale image into a halftone image using the dot diffusion halftoning technology, which is implemented at high efficiency and used to improve the quality of the halftone image output.
In one embodiment, a halftone image generation method used in a system for generating a halftone image is provided. The system at least includes an image input module and a halftoning processing module. First, an original image is received by the image input module. Thereafter, a dot diffusion process is performed to the original image to generate the halftone image by using a first class matrix with a first size and a corresponding first diffused weighting matrix with a first diffused area size, wherein the first class matrix indicates a processing order of the dot diffusion process and the first class matrix with the first size, the first diffused area size and the corresponding first diffused weighting matrix with the first diffused area size are optimized results determined in advance by the halftoning processing module performing a weighting calculation and a quality evaluation to at least one reference image according to class matrixes of different sizes and diffused areas of different sizes.
In another embodiment, an image processing system for generating a halftone image is provided. The system at least includes an image input module and a halftoning processing module. The image input module receives an original image. The halftoning processing module is coupled to the image input module and performs a dot diffusion process to the original image to generate the halftone image by using a first class matrix with a first size and a corresponding first diffused weighting matrix with a first diffused area size, wherein the first class matrix indicates a processing order of the dot diffusion process and the first class matrix with the first size, the first diffused area size and the corresponding first diffused weighting matrix with the first diffused area size are optimized results determined in advance by the halftoning processing module performing a weighting calculation and a quality evaluation to at least one reference image according to class matrixes of different sizes and diffused areas of different sizes.
Halftone image generation methods and systems may take the form of a program code embodied in a tangible media. When the program code is loaded into and executed by a machine, the machine becomes an apparatus for practicing the disclosed method.
The invention can be more fully understood by reading the subsequent detailed description and examples with reference to the accompanying drawings, wherein:
The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.
The invention relates to the image processing method, in which dot diffusion halftoning technology of higher implementation efficiency is used to convert the original image of a grayscale into a halftone image. The diffused area from a range of sizes is used to cope with the class matrix of different sizes (e.g., 8*8 and 16*16) when performing the optimization process, in order to obtain the best diffused weighting and the diffused area that can be used to improve the output halftone image quality. Based on the reason that halftone image quality obtained by the dot diffusion of this invention is the best dot diffusion by this stage, and for being able to retain inherent characteristics of parallel processing of dot diffusion, it is much higher than the already known diffusion method in the implementation efficiency, such as the direct binary search and error diffusion method.
An image processing system for generating halftone images according to the invention is illustrated in the following.
The image output module 130 is coupled to the halftoning processing module 120 for receiving the halftone images 150 generated by the halftoning processing module 120 and outputting the halftone image 150. For example, the image output module 130 may print out the halftone image 150.
In the embodiments of the invention, a grayscale or multi-scale image is converted to a halftone image by dot diffusion processes. Methods for calculation of converting a grayscale or multi-scale image to a halftone image by dot diffusion processes are well-known in the art, and thus detailed descriptions are omitted here for brevity while only results are listed in the following for reference.
Please refer to
where the variable hm,n denotes diffused weighting. Please refer to
The formula w=Σm=−11Σn=−1hm,n indicates the summation of the diffused weights corresponding to those unprocessed pixels.
Furthermore, in the following embodiments, the Peak Signal-to-Noise Ratio (hereinafter referred to as PSNR) will be used as an assessment criterion for the halftone image quality. The PSNR value may be an image quality difference measurement for images before and after the halftoning process, i.e. the original image 140 and the halftone image 150. Generally, the larger the PSNR value of the two images (i.e. the images before and after the halftoning process) is, the smaller the difference there between is. If the PSNR value of the two images exceeds 30 dB, the difference between the two images may not be visible to the human eye. Therefore, a PSNR value may be used as a reference value of a predicted result. For example, suppose a halftone image of size is P×Q, then the quality assessment PSNR may be defined as below:
where xi,j denotes the original grayscale image, bi,j denotes the halftone image, wm,n denotes the Least-Mean-Square (LMS) trained human visual filter at position (m,n), and R denotes the support region of the human visual filter (e.g. the size of the filter may be fixed at 7×7). Note that the value of wm,n can be obtained by the following formula:
In which wi,j,opt stands for optimized LMS coefficient, ei,j2 is the Mean-Square-Error (MSE) of xi,j and {circumflex over (x)}i,j, and μ is the convergence speed (in this embodiment: 10−5) used to adjust the optimization process of the LMS. If wm,n>wmm,n,opt, then reduce the value of wm,n, and when wm,n<wmm,n,opt then increase the value of wm,n.
In addition, LMS filter describes in the following the human visual characteristics: (1) The sensitivity in the diagonal direction is lower than that in the orthogonal direction; and (2) the center of the filter has the highest sensitivity, and the sensitivity degree decreases with the increase of distance. This PSNR will be used to assist the assessment of halftone image quality in the following.
First, in step S310, the image input module 110 receives an original image 140.
Then, in step S320, the halftoning processing module 120 performs a dot diffusion process to the original image 140 to generate a halftone image 150 by using a first class matrix with a first size and a corresponding first diffused weighting matrix with a first diffused area size. Among them, the first class matrix includes a processing order of the dot diffusion process, wherein the halftoning processing module 120 may perform a class matrix and diffused weighting co-optimization procedure in advance based on the various types of matrixes of different sizes (e.g. 8×8 and 16×16), as well as the multiple diffused areas of different sizes (e.g. 3×3, 5×5, 7×7, and 9×9) to perform weight calculation and quality assessment on one or more of the reference images in order to obtain the first class matrix with the first size in the class matrix, the size of the first diffused area in the first diffused area, and the first diffused weighting matrix that corresponds to the first class matrix. It is noted that the halftoning processing module 120 may directly utilize the obtained optimization result in subsequent halftoning processes without further optimization. In this step, the halftoning processing module 120 may, based on the original image 140, divide the original image 140 into several blocks. Then, the blocks in each area will carry out the dot diffusion procedure based on the first class matrix, diffused area size, and diffused weighting of the first diffused weighting matrix, generating the halftone image of 150.
In this invention, the optimization process of the class matrix co-optimizes the diffused weighting and class matrix, which include the swapping operations undertaken by the class matrix members and the adjustment operations of the diffused weighting in the diffused weighting matrix. The details are explained as follows.
Class Matrix and Diffused Weighting Co-Optimization
In this invention, the diffused weighting and class matrix are co-optimized, but this will result in combinations in access. In this part, the three limitations below shall serve as legal assumptions for three simplified combinations. The limitations are as follows:
(1) Out of all the diffused weightings in the diffused area, the closer a diffused weighting is to the central location, the higher its importance will be.
(2) The diffused weightings that are in the same Euclidean distance from the center are given the same weighting.
(3) The four horizontal and vertical positions that are the closest to the center have a fixed weighting of 1.
The first limitation takes into account the human visual characteristics while the second and third limitations are intended to reduce the coefficient of the number of possible combinations. For example, supposing the diffused weighting is of size 3×3, according to the above limitations, the diffused weighting can be simplified so that only one value in the corner needs optimization. Please refer to
Supposing the class matrix is of 8×8, in the optimization process, any member of the class matrix will engage in swapping with the rest of the 63 members while the different reference images in
First, in step S510, the initial class matrix C and the initial diffused weighting matrix H are provided. In this embodiment, supposing the class matrix of
In step S520, one of the possible diffused area sizes is selected respectively to test the initial class matrix C. Four diffused areas in the sizes of 3×3, 5×5, 7×7, and 9×9 are respectively used to test the initial class matrix C to obtain the diffused weightings in these areas. The initial class matrix C may be divided into blocks based on the sizes of the diffused areas (e.g. 3×3) selected. Then, with the blocks as the unit, the following optimization procedures may be performed.
In step S530, in sequence order, a member is successively selected from the selected size of the diffusion area of the initial class matrix C being tested and is swapped with each member in the class matrix with one of the other 63 members to generate a swapped class matrix. For example, supposing the member in the class matrix is a one-dimensional sequence, and the class matrix is of size 8×8, any member C(i) of the class matrix will be successively swapped with one of the remaining 63 members C(j), where i≠j.
In step S540, under each swapping operation, a diffusion weighting adjustment is performed for the initial diffused weighting matrix H to generate an adjusted diffused weighting matrix based on a specific rule. In this step, the diffused weightings may be adjusted by adjusting 10−5 in all its coefficient values. During the diffused weighting generation, the specific rules are the three limiting conditions mentioned above, that is, coefficients of the horizontal and vertical locations closest to the center in the diffused areas are fixed to 1. Moreover, the locations with the same Euclidean distance from the center will set the same weighting values (See
In step S550, the class matrix after every swapping and the diffused weighting matrix after every adjustment are used to perform the dot diffusion procedure to generate the corresponding second halftone images and calculate the second average PSNR of the generated second halftone images. In this step, the previous PSNR calculation equation can be used to obtain the second average PSNR for each of the second halftone images.
In step S560, the formula max(PSNR(swapped_C switched_H)) is applied to keep the swapped class matrix and the switched diffused weighting that lead to the highest reconstructed image quality as the new class matrix and diffused weighting. The highest PSNR member swapping location and the corresponding optimized diffused weighting obtained in the swapping process are retained to serve as the new class matrix and diffused weighting for the next round of optimization. The class matrix and adjusted diffused weighting combination after swapping will ensure better image quality. In this step, a quality assessment of the images before and after the swapping is performed to determine whether the image quality has been improved. If no change that improves the image quality has taken place, the class matrix and diffused weighting before the change shall be retained. After completing a round of optimization, in step S570, whether all the class matrix members have been optimized will further be determined. If so, step S580 will be performed. If some other members have not been processed (No in step S570), the flow returns to step S530 and another class matrix member C(i) is chosen to allow the member to swapping with every remaining member in the diffused area and generate a class matrix. Steps S540 to S560 will be repeated.
In step S580, it is determined whether any swapping can improve the image quality obtained from the dot diffusion process. If so, it means optimization has not been completed and repeated operations of the steps from S530 to S570 is needed. If not (No in step S580), the optimization procedure will be terminated. Meanwhile, the final class matrix and diffused weighting matrix are the previously mentioned optimized first class matrix with the first size and the corresponding first diffused weighting matrix with the first diffused area size.
For example, in one embodiment, supposing the sizes of the considered class matrix are 8×8 and 16×16 and the diffused area sizes are 5×5, 7×7 and 9×9, the previous optimization process can be used to obtain the 8 final convergent class matrix and diffused weighting combinations, taking into account the class matrixes of different sizes and the diffused weightings and areas. Then, the 8 class matrixes are used to test the different reference images as shown in
Therefore, according to the halftone images generation methods and related imaging processing systems of the invention, the improved dot diffusion processing method that targets the original grayscale or color scale image applications is used to carry out parallel processing to obtain halftone images. Of which, the diffused weightings and class matrixes take into account the human visual characteristics in the process of optimization in order to carry out image quality assessments. Therefore, when people look at the generated halftone images, they will have a more comfortable feel. Moreover, in the co-optimization process, images of multiple-color scales instead of single-color scales are used in actual applications of training. Hence, the class matrix trained can better meet the actual application needs. Furthermore, the halftone image generation method of the invention, similar to error diffusion, can be used to integrate the halftone image quality, which have the ordered dithering and parallel processing features needed to provide a high quality and high efficiency halftone imaging generating method, which is especially suitable when used on printers and for newspaper and magazine outputs.
It is understood that although each of the aforementioned modules or units of the invention has been illustrated as a single component of the system, two or more such components can be integrated together, thereby decreasing the number of the components within the system. Similarly, one or a multiple of the above components can be separately used, thereby increasing the number of the components within the system. In addition, the modules or the unit components of the invention can be implemented by any hardware, firmware, or software methods or combination thereof.
Systems and methods thereof, or certain aspects or portions thereof, may take the form of a program code (i.e., executable instructions) embodied in tangible media, such as floppy diskettes, CD-ROMS, hard drives, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine thereby becomes an apparatus for practicing the methods. The methods may also be embodied in the form of a program code transmitted over some transmission medium, such as electrical wiring or cabling, through fiber optics, or via any other form of transmission, wherein, when the program code is received and loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the disclosed methods. When implemented on a general-purpose processor, the program code combines with the processor to provide a unique apparatus that operates analogously to application specific logic circuits.
While the invention has been described by way of example and in terms of preferred embodiment, it is to be understood that the invention is not limited thereto. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to the skilled in the art). Therefore, the scope of the appended claims should be accorded to the broadest interpretation so as to encompass all such modifications and similar arrangements.
Number | Date | Country | Kind |
---|---|---|---|
098142839 | Dec 2009 | TW | national |