The present invention generally relates to colorimetric adjustment of defined areas within an image frame and more particularly relates to a color imaging system for selectively adjusting flesh tones within an image frame of a digitized motion picture in accordance with viewer preference.
The difficulty of capturing and reproducing accurate and pleasing flesh tones is widely recognized in the imaging arts. For example, color negative/positive photographic systems in use today are designed to produce pleasing prints for most of the people in a target population. The print appearance includes both pleasing tone and color reproduction to produce colorful prints with good contrast, and particularly excellent flesh tone reproduction. Typically, existing photo systems are designed to be optimized for a particular skin type and preference. For example film colorimetric parameters can be optimized for capture and reproduction of some segment of the full range of flesh tones of world populations, such as Caucasian, Oriental, Asian, Indian, or Black flesh tones. Necessarily, photographic film, paper, and printer sets-ups are designed for providing pleasing color for a particular market segment, having a subset of the full range of flesh tones. As a result, a system optimized for producing photographs of people who have a lighter flesh tone may yield disappointing results for producing photographs of people who have a darker flesh tone. This can be especially true for the same film type and techniques used to photograph people whose flesh tones lie on the extremes of the range. This can result in loss of overall image quality, particularly with respect to facial features.
Digital imaging now allows the capability to make colorimetric adjustments that can compensate for local differences in flesh tone reproduction. With digital images, such as those obtained from a digital camera or from a scanned print, colorimetric response of an imaging or display system can be adapted to allow an operator or consumer to modify flesh tone reproduction in order to provide a more pleasing output. For example, commonly assigned U.S. Pat. No. 6,396,599 (Patton et al.) entitled “Method and Apparatus for Modifying a Portion of an Image in Accordance with Colorimetric Parameters” discloses a system that locates a flesh tone area in an image and allows an operator to adjust calorimetric attributes of that area in order to provide a more satisfactory output image. Colorimetric analysis and various supporting tools such as face recognition algorithms can be employed with such a technique in order to provide efficient identification of flesh tones in a scanned print or digital camera image.
While methods such as those of the Patton et al. '599 disclosure present capable solutions for handling individual photographic images, the problem of flesh tone adaptation in motion picture imaging is more difficult. In conventional motion picture film processing, various editing procedures may be executed on a digitized image, scanned from the original film shot at the studio. After editing, the final image can then be rewritten onto a print film for distribution to local theaters and exhibitors. Thus, for conventional motion picture film production, editing personnel can perform colorimetric operations on a frame-by-frame basis, using the scanned digital image data. Using a sequence of frame image digitization, object and outline detection and masking, selective calorimetric modification, and printing to film, it is possible for editing personnel to adjust or to add color in successive frames of a conventional film motion picture. This overall sequence is used, for example, in colorization of black-and-white motion picture film, using techniques such as those disclosed in U.S. Pat. Nos. 5,912,994 (Norton et al.); 5,050,984 (Geshwind); and 3,784,736 (Novak).
The introduction of digital cinema is expected to bring about numerous changes in how motion pictures are produced and distributed. Referring to
As the data path of
Thus, what is needed is an apparatus and method for selectively adjusting the calorimetric attributes of flesh tone or changing other local color characteristics within an image frame of a digitized motion picture.
It is an object of the present invention to allow the modification of flesh tones, or of other selectable color areas, of a digital motion picture. With this object in mind, the present invention provides a method for modifying at least one colorimetric attribute of a predetermined region of a motion picture frame comprising:
It is a feature of the present invention that it employs colorimetric transforms to modify selected portions of successive motion picture image frames in order to adapt the color of objects in the motion picture to specified preferences.
It is an advantage of the present invention that it allows color modification of image data at more than one point in the motion picture production chain.
It is a further advantage of the present invention that it allows a local exhibitor to adapt colorimetric characteristics of flesh tones to suit local preferences. This capability can also be extended to other identified objects in the motion picture frame.
These and other objects, features, and advantages of the present invention will become apparent to those skilled in the art upon a reading of the following detailed description when taken in conjunction with the drawings wherein there is shown and described an illustrative embodiment of the invention.
The present description is directed in particular to elements forming part of, or cooperating more directly with, apparatus in accordance with the invention. It is to be understood that elements not specifically shown or described may take various forms well known to those skilled in the art.
Referring to
With digital cinema, it is possible for studio 110 (
With respect to the digital image data stream, metadata could be supplied using the same communications channel used for the digital cinema content. Metadata might be interleaved with image data or provided before or after image content. Alternately, image data and metadata could be provided using separate data channels.
Referring to
Control logic processor 48 may provide an operator with the option to select attributes of the color transform used as LUT 46. Alternately, LUT 46 itself could be provided or specified along with the metadata for use by server 40. For example, an advertising distributor may decide to automatically download as metadata or specify a LUT 46 suitable for flesh tone modification when the advertising content is shown in any region of the world. In this way, a different LUT 46 can be applied to the same image data, depending on where content is shown, allowing suitable flesh tone transforms to be applied for the same content whether it is displayed in India, Brazil, or Finland, for example.
The listing of Table 1 is intended to be illustrative only; alternate and/or additional metadata fields could be provided for identifying an area of frame 20 that can be modified using alternate calorimetric transforms. The data format of the metadata could be a simple, open data format, such as employing familiar comma-separated fields, for example. More complex encoding schemes could be used, such as those employing compression, security algorithms wherein a key is required for decoding, or other known mechanisms for data transfer.
Methods for applying LUT 46 as a colorimetric transform are well known in the art. It must be emphasized that other types of calorimetric transforms and algorithms based on, e.g., multi-dimensional (e.g., 3D) look up tables, matrices, and so on, are available for modifying color characteristics of an area of an image, well known to those skilled in the color imaging arts. Furthermore, a transform or algorithm may be selected for application to the whole image, if the transform or algorithm would primarily change skintones within the frame, and only slightly affect other tones and colors. In effect, in that case, the calorimetric transform is applied to pixels within the whole frame.
In an alternate embodiment, studio 110 does not provide metadata defining flesh tone regions 22, 24 or other areas of frame 20 that may be subject to colorimetric transform. Instead, local exhibitor at exhibition system 140 may apply various types of imaging algorithms for detecting flesh tone regions 22, 24 in the received image data. Utilities such as skin tone and facial feature recognition algorithms, well known in the imaging arts, could be used to identify flesh tone regions 22, 24 in order to allow LUTs 46 or other suitable transforms to be applied.
It must be re-emphasized that the specification of the present disclosure describes modification of flesh tone regions 22 and 24, as were shown in the examples of
Metadata provided for flesh tone regions 22, 24 or for other areas of image frame 20 may alternately specify limits or boundary values for LUT application to these regions. In this way, for example, the range of flesh tone selections for one or more actors could be restricted in order to prevent unrealistic treatment. This would allow some measure of control of the skin tone range by studio 110.
Using the method of the present invention, studio 110 could provide exhibition system 140 with image data that is intended to be used as a type of master motion picture. LUTs or other suitable transforms could then be provided along with the metadata from studio 110 or may be provided to exhibition system 140 by some other transmittal means. It is then the function of the exhibitor to ensure that the master motion picture is suitably treated at the exhibition site before projection.
Also using the method of the present invention, a single frame can be modified or the modification can be applied to a set of consecutively displayed motion picture frames, as would often be the case in a motion picture modification. In the latter case, it is important that the calorimetric adjustments from one frame to the next are done in a consistent manner for objects persisting from one frame to the next. As an example, for the best results for skintone reproduction of different races, different transforms might be used for humans in each frame having different skintones (e.g., asian, Caucasian, or african-american skintones, and so on). It would be important for each human in successive frames to be adjusted in the same way or a person's reproduction could shift from frame to frame. This is an additional desirable constraint, since it could be difficult if a facial recognition algorithm, or a skintone recognition algorithm, were to be used unless these algorithms were capable of identifying specific individual skintone types.
The transforms used to modify the imagery in selected areas need not be limited to color transformations. Other modifications to the imagery can be supported, e.g., transformations that would affect sharpness, tonescale, color balance, and the like. For example, if an undesirable object, such as an advertisement, is present in the scene and there is a desire to obscure the object, a transform may be applied which blurs the selected pixels representing the object. This technique may also be used to obscure, i.e., unsharpen or throw out of focus, distracting objects and faces. For example, this may be especially useful for providing a more enjoyable viewing experience of an image of an emphasized person or couple, such as a bridal couple, if other faces in the image are deemphasized. In general, such a method modifies at least one attribute of an object by the steps of (a) providing metadata defining a predetermined region containing the object; and (b) applying a transform to pixels within the predetermined region, thereby modifying the attribute of the object whereby the object is treated differently relative to other objects in the frame.
The invention has been described in detail with particular reference to certain preferred embodiments thereof, but it will be understood that variations and modifications can be effected within the scope of the invention as described above, and as noted in the appended claims, by a person of ordinary skill in the art without departing from the scope of the invention. For example, various types of imaging transforms could be applied at different stages of production for digital motion picture content, in addition to the LUTs 46 described. The method of the present invention could be applied in preparation of conventional film-based motion pictures, but would be applied at an editing facility, allowing distribution of the same image content having different colorimetric transforms applied to suit local/regional preferences.
Thus, what is provided is an apparatus and method for selectively adjusting flesh tones or other areas within an image frame, or to an entire image frame, of a digitized motion picture in accordance with viewer preference.
Reference is made to commonly assigned copending application Ser. No. 10/401,923, entitled “Method and System for Modifying Digital Cinema Frame Content” and filed on 28 Mar. 2003 in the names of David L. Patton, Dale McIntyre, John R. Fredlund, Michael E. McCrackan, Carlo V. Hume, and Arthur J. Cosgrove.