Television – Image signal processing circuitry specific to television – Special effects
Reexamination Certificate
2001-06-29
2004-03-30
Lee, Michael H. (Department: 2614)
Television
Image signal processing circuitry specific to television
Special effects
C348S591000, C348S592000
Reexamination Certificate
active
06714257
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention is related to the field of image processing, and in particular to the processing of image data that contains a key that identifies regions for inserting other image data.
2. Description of Related Art
Color keying, or chroma keying, is used when certain regions of an image or graphics window are to be replaced by different image or graphic information in a later processing step. For example, a “bluescreen” is often used as a background for a television announcer when the announcer is being recorded. A background image, corresponding, for example to “file footage” from a related story, is subsequently inserted into the image containing the announcer, to form a composite image containing the announcer in the foreground and the file footage in the background. Corresponding regions of the file footage background are inserted wherever the bluescreen color is detected in the image of the announcer. The particular color that is chosen as the replaced background color is termed a color key, and is generally selected to be easily distinguishable from colors that would be expected in the foreground image.
Luminance keying operates similarly, wherein the brightness level of regions of an image define whether to insert a replacement image in each region. Chapter 9 of the text “VIDEO DEMYSTIFIED”, by Keith Jack, ©HighText Interactive, Inc., San Diego, Calif., presents various techniques for luminance and color keying, and is incorporated by reference herein.
Because color keying is more commonly used for forming a composite image of individual background and foreground image data, and for each of reference, the term color keying is used herein to refer to the process of distinguishing between regions of an image based on a value associated with one or more components of the data elements that form the image. In like manner, the term color-keyed is used herein to identify the scheme used to provide a means for identifying the insertion region of an image, even though any of a variety of techniques may be employed to provide this identification. The term color-keyed image is used to define the image that contains regions that are to be replaced, and the term background image is used to define the image that contains the replacement data, even though the inserted data may not form a ‘background’, per se. That is, for example, the color-keyed image may include a non-key colored background and a foreground object that contains the color key value. The insertion of the data from the “background image” in this example, effects an insertion of foreground image information into the color-keyed image. The term composite image is used herein to define the image that contains the insertion of the background image into the color-keyed regions of the color-keyed image.
As the field of image processing and image manipulation continues to expand, the need for flexible color keying schemes continues to grow. In particular, a need exists for allowing images from different data sources, at potentially different image scales, to be combined. A straightforward solution is to modify the scale of the background image to correspond to the scale of the color-keyed image, and then effect the color keyed merging. If the scale, or available resolution, of the desired output image is different from that of the color-keyed image, the composite image is appropriately scaled. Note, however that this scheme limits the effective resolution of the background image to the resolution of the color-keyed image, regardless of whether the available output resolution is finer than that of the color-keyed image. Preferably, each of the color-keyed image and the background image would be scaled to the desired output image scale before being merged, so that image resolution is not lost by constraining the resolution of the background image to match the resolution of the color-keyed image, or vice versa.
Changing the scale of a color-keyed image, however, may have an adverse effect on the identification of color-keyed regions. Most image scalers, commonly termed sample rate converters, include filters that provide each output sample as a weighted average of multiple input samples in the vicinity of the output sample. For example, if the image is being up-sampled, wherein more output samples are produced than input samples, each output sample value is based on an interpolation of the input sample values in the vicinity of the location of the output sample. If the image is being down-sampled, each output sample is an average of the input samples in the vicinity of the location of the output sample. This interpolation or averaging, which is required to prevent discontinuities and other anomalies in the scaled image data, provides an output value that is a blending of multiple neighboring input values. As such, values along the edges of color-keyed regions will be blended with values in the non-color-keyed regions, and the blended value will likely be a non-color-keyed value that is not present in the original image. If a composite image is filtered, there will be some color blending at the edges of the objects, but because the blended color is based on the colors on each side of the edge, it merely appears as a less-sharp edge. In the example of a blue-colored screen behind an announcer who is purposely not wearing this shade of blue, however, the blended value at the edge of the announcer will be a somewhat-blue color that differs from the color key value, and differs from the colors of the announcer's clothing or skin. The effect of this non-color-key bluish color at the edges of the color-keyed region is a visually apparent ‘outline’ of the announcer when the background image replaces the regions that match the color key value, but does not replace the non-color-key bluish color at the edges.
BRIEF SUMMARY OF THE INVENTION
It is an object of this invention to provide an apparatus and method that allows for the scaling and/or filtering of a color-keyed image that allows the replacement of the color-keyed regions with background image information, without introducing visible artifacts. It is a further object of this invention to provide an apparatus and method that allows for the scaling and/or filtering of a color-keyed image without introducing visible artifacts using existing video processing memory and structures.
These objects and others are achieved by extracting the color-keyed regions from a color-keyed image, and independently scaling the color-keyed regions and the non-color-keyed regions. The independently scaled regions are subsequently merged to form a scaled color-key image with clearly distinguished color-keyed regions and non-color-keyed regions. To minimize the blurring of edges in the non-color-key regions, the non-color-key colors are extended into color-keyed regions after the color-keyed information is extracted from the color-keyed image. To minimize the encroachment of the scaled color-keyed regions into the scaled non-color-keyed regions of the scaled color-key image, the edges of the scaled color-key regions are sharpened by defining the color-keyed region as the region wherein each scaled/filtered color-key value exceeds a non-zero threshold value. To facilitate the use of existing memory and structure in images that are encoded using three components per pixel, such as RGB, YUV, etc., the extracted color-keyed regions are stored in the memory structure that is typically used to store an optional fourth component, such as a transparency or texture parameter.
REFERENCES:
patent: 4602286 (1986-07-01), Kellar et al.
patent: 4875097 (1989-10-01), Jackson
patent: 5608464 (1997-03-01), Woodham
patent: 6525741 (2003-02-01), Klassen et al.
patent: 11088892 (1999-03-01), None
Keith, Jack, Video Demystified, 1996, pp. 387-425.
Koninklijke Philips Electronics , N.V.
Lee Michael H.
Ure Michael J.
LandOfFree
Color key preservation during sample rate conversion does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Color key preservation during sample rate conversion, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Color key preservation during sample rate conversion will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3251203