The present invention is related to electronic image analysis and processing and more specifically to image harmonization.
A portion of the disclosure of this patent document may contain material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. The following notice applies to the software and data as described below and in the drawings hereto: Copyright © 1999, Sony Electronics, Inc., All Rights Reserved.
Often when two or more images are placed in close proximity such that the images are visible to the human eye simultaneously, one or more of the images may appear to not “fit in” or in some manner are incompatible with the other images. This incompatibility can often occur regardless of subject matter or viewing angle or perspective of the image. This incompatibility is often due to differences in various characteristics of each one of the images. Some examples of the characteristics include color balance, sharpness, histograms, white balance, resolution, size, color noise, brightness, contrast, luminosity, digital artifacts and analog artifacts. If any one or more of these characteristics are sufficiently different from one image to the next, the one different image can be detected by the human eye as being incompatible with the other images.
A system and method of harmonizing a plurality of images is disclosed. First a plurality of images are selected. Next, a plurality of characteristics of each one of the plurality-of-images are evaluated. Then; at least one of the plurality of characteristics is selected to harmonize and the selected characteristic is harmonized in at least one of the plurality of images. Finally, the harmonized image or images are output.
The present invention describes systems, clients, servers, methods, and computer-readable media of varying scope. In addition to the aspects and advantages of the present invention described in this summary, further aspects and advantages of the invention will become apparent by reference to the drawings and by reading the detailed description that follows.
The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements;
a and 1b-illustrate embodiments of an environment which can perform the image harmonizing process of the present invention.
c-illustrates one embodiment of a system which can be used in the system of
FIG. 2-illustrates a flow diagram of a harmonizing process of the present invention.
A system and method of harmonizing a plurality of images is disclosed. First a plurality of images are selected. Next, a plurality of characteristics of each one of the plurality of images are evaluated. Then, at least one of the plurality of characteristics is selected to harmonize and the selected characteristic is harmonized in at least one of the plurality of images. Finally, the harmonized image or images are output.
When displaying two or more images together i.e. such as on a web page, in printed photo albums, hanging on walls, etc., the images often appear, to the human eye, to appear as though they do not belong together or are in some manner incompatible. This appearance is often because of differences in one or more of a plurality of characteristics between the images. Examples of the plurality of characteristics include color balance, sharpness, histogram differences, white balance, resolution, size, color noise, brightness, digital artifacts and analog artifacts, contrast, luminosity and other characteristics well known to those skilled in the art. One intended advantage of the present invention is to improve the compatibility of a plurality of images when the images are displayed in close proximity.
For an alternative embodiment, the harmonizer 110 could be a image harmonizer service accessible through a website accessible via the internet where the customer 115 provides the image data to the harmonizer 110 and the consumer 120 uses the harmonized image data. For one embodiment, the customer 115 pays the harmonizer 110 for the image harmonizing service. For another embodiment, the consumer 120 pays the harmonizer for the image harmonization service. For yet another embodiment, the consumer 120 pays the customer 115 for the harmonized image data It will be appreciated that various compensation schemes can be implemented by the server owner including as subscription a service, one-time charges, or session time without departing from the scope of the invention. In addition, the owner of the server may obtain revenue from banner advertisements that are displayed to the customer 115 and/or the consumer 120 during the processing of the input images when the image harmonizer 110 is harmonizing the image data Such banner ads can be targeted to the user based on the problems identified in the images, e.g. an image that is a low resolution or coarse image could cause an advertisement for higher resolution camera or scanner systems to appear.
For another embodiment, the images can be input via an electronic imaging device such as a video camera or an electronic or digital, still image camera or a scanner or selected from a database of image data such as a website or other computer readable media.
In one embodiment, as shown in
One embodiment of an image harmonizer 140, is illustrated in FIG. 1C. The harmonizer 140, in one embodiment, includes a processor 150, memory 155, a system bus 165 and input/output capability 160. The memory 155 is configured to store instructions and data which when executed by the processor 150 perform the method described herein. The memory may also store the content and revenue information used in the process described herein. Input/output and related content components 160 include a delivery of content/related content, display of content/related content portion of content/related content or a representation of the content.
The description of
One embodiment of the present invention is a method of harmonizing a plurality of images as illustrated in
For an alternative embodiment, a level of harmonization can be selected or limited so that a particular characteristic of an image will not be changed more than a selected amount or level. For another embodiment, one or more of the input images are selected as not being modified. The other images are then modified or adjusted so as to harmonize more with the characteristics of the selected, non-modified images. For still another embodiment, the selected images are harmonized and the non-selected images are not harmonized or otherwise modified.
One embodiment includes a method of harmonizing color data of a plurality of images by adjusting the color characteristics of the plurality of images to make the images more compatible. First, two or more images are selected. Next, the color data of each image is analyzed. Color data includes brightness, contrast, luminosity, and other characteristics that effect the appearance of color in an image. The color data is then compared. Next, harmonized color data is determined by selecting one characteristic of color data and the best compromise such as an average or a minimum or maximum value of that characteristic is determined. Next, a color transform is created for each image. The color transform then adjusts the original color data of each image to be consistent with the harmonized color data. Next the color transforms are applied to each image. The resulting images are adjusted to be more alike in their color characteristics.
For one example the brightness value of each image is determined, and then an average brightness value is calculated as the harmonized color data. Then, the brightness value of each color transform for each image is adjusted to the harmonized brightness value. Next, the color transforms are applied to each image to adjust the brightness of each of the images and the adjusted images are output.
For alternative embodiments, the level of a selected characteristic can be selected or the level of the characteristic of one image can be selected. For example, a plurality of images are input into the harmonizer, one image is selected, then the brightness of each one of the plurality of images is determined. Next, the brightness of each one of the non-selected images is adjusted to the brightness of the selected image.
In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will be evident that various modifications may be made thereto without departing from the broader spirit and scope of the invention as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.
This application claims the benefit of U.S. Provisional Application No. 60/181,777, entitled: MULTIPLE IMAGE HARMONIZATION SYSTEM, filed Feb. 11, 2000.
Number | Name | Date | Kind |
---|---|---|---|
4485409 | Schumacher | Nov 1984 | A |
4902128 | Siebecker et al. | Feb 1990 | A |
5193000 | Lipton et al. | Mar 1993 | A |
5561745 | Jackson et al. | Oct 1996 | A |
5611028 | Shibasaki et al. | Mar 1997 | A |
5694331 | Yamamoto et al. | Dec 1997 | A |
6083158 | Bearman et al. | Jul 2000 | A |
6097389 | Morris et al. | Aug 2000 | A |
6112226 | Weaver et al. | Aug 2000 | A |
6123362 | Squilla et al. | Sep 2000 | A |
6169544 | Onoda | Jan 2001 | B1 |
6222947 | Koba | Apr 2001 | B1 |
6324545 | Morag | Nov 2001 | B1 |
6396963 | Shaffer et al. | May 2002 | B2 |
6434579 | Shaffer et al. | Aug 2002 | B1 |
6437306 | Melen | Aug 2002 | B1 |
6522351 | Park | Feb 2003 | B1 |
6577760 | Ohta et al. | Jun 2003 | B1 |
6590586 | Swenton-Wall et al. | Jul 2003 | B1 |
Number | Date | Country | |
---|---|---|---|
60181777 | Feb 2000 | US |