Image analysis – Color image processing
Reexamination Certificate
1999-04-26
2004-04-13
Patel, Jayanti K. (Department: 2625)
Image analysis
Color image processing
C345S613000, C358S531000
Reexamination Certificate
active
06721446
ABSTRACT:
FIELD OF THE INVENTION
This invention relates to identifying intrinsic pixel colors and pixel opacities in a region of uncertain pixels.
BACKGROUND OF THE INVENTION
A common task in the manipulation of digital images is the removal of one or more foreground objects from a scene and the composition of this object with a new background image. This is typically a difficult task for several reasons:
1) blending of an object with the background scene: a pixel at an edge of an object may have contributions from both the foreground and the background, its color is consequently a blend of the two regions;
2) object complexity: even for objects with hard edges, the object border often contains detail that requires tedious effort to define manually; and
3) combinations of 1) and 2): an example is hair or fur, the shapes are complex and regions with thin fibers lead to color blending.
In general, the problem does not have a simple unambiguous solution. The movie industry has handled this by simplifying the scene, by filming objects or people against a simple background (blue screen) having as uniform a color as possible. Techniques have been developed to produce approximate solutions in this situation. Software products that can be used to mask an object, require a great deal of manual effort for complex objects such as subjects with hair. Existing products also enable a degree of color extraction from simplified background scenes by applying operations to the color channels.
SUMMARY OF THE INVENTION
In general, in one aspect, the invention features processing a digital image that includes first and second regions by estimating an intrinsic color of a given pixel located in an area of interest that is adjacent to at least one of the first and second regions. The estimating includes extrapolating from colors of multiple pixels in one of the first and second regions and multiple pixels in the other of the two regions.
Implementations of the invention may include one or more of the following features. The original color of the given pixel relates to the original colors of pixels in both the first and second regions. The estimated intrinsic color of the given pixel relates to original colors in only one or the other of the first and second regions. The area of interest includes one of the first and second regions; or is adjacent to both of the first and second regions. The first region is a foreground object and the second region is a background.
The first and second regions have any arbitrary degree of color variation in the visible spectrum over a spatial scale that is on the same order of magnitude or smaller than the minimum span of the area of interest. The estimating includes analyzing both the color and spatial proximity of pixels in the first and second regions.
The estimating includes extrapolating from the closest pixels in the first and second regions; or flowing colors into the area of interest from one or both of the first and second regions. The flowing of colors includes averaging of color values for each of a set of pixels in the first region and a set of pixels in the second region. The digital image includes layers of pixel information and the estimating is based on pixel information in only one of the layers; or in other implementations on pixel information in a composition of all the layers.
An opacity value is determined for the given pixel, indicative of the extent to which the intrinsic color of the given pixel relates to original colors in the first and second regions, based on a result of the estimating of the intrinsic color. The given pixel includes original opacity information, and the opacity value is also based on the original opacity information. In some implementations the opacity determination includes use of a neural network trained on the image original colors and estimated intrinsic colors. The opacity values are used to composite one of the first and second regions with another digital image.
The estimating includes extrapolating estimates of intrinsic colors of the first and second regions using searches in color space and image coordinate space. The estimating assumes a linear blending model. The estimating includes flowing colors from edges of the area of interest to fill the area of interest with estimates of the colors of the first and second regions.
Estimating the intrinsic color includes determining two color sample sets for the given pixel, each of the color sample sets being associated with one of the first and second regions, and estimating the intrinsic color based on the two color sample sets. The original color of the given pixel is compared with colors in the color sample sets. A single color is selected from each of the color sample sets based on an error minimization technique.
In general, in another aspect, the invention features enabling a user to paint an area of the digital image to identify at least an area of interest adjacent to at least one of a first region and a second region. After the user has defined the area of interest, the intrinsic colors of pixels in the area of interest are estimated based on color information for pixels in the first region and the second region.
Implementations of the invention may include one or more of the following features. The painting is done with a brush tool that can be configured by the user. The painted area can be built up by repeated painting steps and portions of the painted area can be erased by the user interactively. The user paints the area of interest and separately identifies a location that is in one of the first and second regions. Or the user paints at least one of the first and second regions and the area of interest and separately identifies a color associated with one of the first and second regions. The user designates one of the first and second regions by identifying a pixel location in that region. The user identifies the color by applying an eyedropper tool to one pixel or a set of pixels in the one region. One of the regions is flood filled based on the identified pixel location to designate that region as a foreground. The painted area may be modified by a user interactively and repeatedly. The user is enabled to paint additional areas of interest between other pairs of first and second regions.
In general, in another aspect, the invention features receiving a mask associated with an area of interest in a digital image, the mask including values representing opacities of pixels in the region of interest with respect to an adjacent region of interest. Intrinsic colors for the pixels are estimated based on the mask.
In general, in another aspect, the invention features enabling a user to control an original extraction by manipulating a brush on a display of the image, enabling the user to control a touch up extraction following the original extraction, and considering a pixel identified for touch up extraction only if the pixel was of uncertain color in the original extraction.
Implementations of the invention may include one or more of the following features. An intrinsic color is determined for each of the pixels that were of uncertain color based on a forced foreground or background color. The forced color is selected by the user or is determined automatically from the original colors within the foreground region.
In general, in another aspect, the invention features determining, for each pixel in an area of interest in a digital image, the nearest pixel in a first region of the image that is adjacent to the area of interest and the nearest pixel in a second region of the image that is adjacent to the area of interest. A processing area is defined that is smaller than the image. A pixel window is defined that is smaller than the defined processing area. The processing area is scanned at a succession of overlapping positions that together span the image. At each overlapping position of the processing area, the pixel window is scanned across the processing area. At each position of scanning of the pixel window, stored information for pixels in the window is updated, the
Newell Martin E.
Wilensky Gregg D.
Adobe Systems Incorporated
Fish & Richardson P.C.
Patel Jayanti K.
LandOfFree
Identifying intrinsic pixel colors in a region of uncertain... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Identifying intrinsic pixel colors in a region of uncertain..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Identifying intrinsic pixel colors in a region of uncertain... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3250964