Image analysis – Image enhancement or restoration
Reexamination Certificate
2001-11-16
2003-07-01
Rogers, Scott (Department: 2624)
Image analysis
Image enhancement or restoration
C382S268000, C382S275000, C382S282000, C358S531000, C358S452000, C358S453000
Reexamination Certificate
active
06587592
ABSTRACT:
BACKGROUND
The present application describes systems and techniques relating to generating new data values for an image region, for example, generating replacement data values to heal defective pixels using boundary conditions and/or using a sample of textured data values.
Traditional techniques for removing unwanted features from images include wire removal techniques such as used in movies, dust and scratches filters used in software applications such as PHOTOSHOP®, provided by Adobe Systems, Inc. of San Jose, Calif., inpainting and other algorithms. In typical replacement pixel data generation techniques, selected pixels in an image are regenerated based on values of the pixels bordering the selected pixels, and first and second order partial differential equations (e.g., the Laplace equation). For example, traditional inpainting techniques generally are based on second order the Laplace equation and/or anisotropic diffusion. These techniques typically result in noticeable discontinuities at the edges of the inpainted region.
Other techniques of generating replacement data values in an image region include applying area operations such as blurring or performing median calculations (e.g., using Gaussian filters and median filters) at each pixel in the selected region. The image area, or neighborhood, used for the area operation generally will include one or more selected pixels with undesirable data values. Thus, the neighborhood needs to be large enough to swamp the contributions of the undesirable pixel data. Oftentimes, a user must specify how large to make the neighborhood to minimize the effects of the undesirable data values. For example, techniques that are based on frequency domain separations generally require that the user specify a neighborhood size that will be used by the filter that separates gross details from fine details.
Some conventional techniques also apply a high-frequency component from another image to a healed region after it has been modified to replace defective pixel values. But the results of such traditional techniques for removing unwanted features of an image often do not reflect the true properties of most images. Areas of images that are filled-in using conventional techniques frequently have discontinuities at the boundary of the filled-in region and/or look blurred or otherwise appear to lack detail. These filled-in areas are often easily noticed and do not look like a natural part of the image, either because the surrounding areas are textured, or because pixel intensity changes sharply at the boundary of each filled-in area.
SUMMARY
In one aspect, a selected image region is healed by propagating values in a boundary region into replacement data values for the selected image region by iteratively applying a procedure or algorithm (e.g., iteratively applying one or more kernels), such that values in the boundary region are continuously put back into the iterative calculations. An example of such a function uses kernels to generate replacement data values having multiple orders of continuity at the boundary. Optionally, replacement data values for an image region to be healed are generated based on a difference between existing data values and texture data values, thereby introducing texture (e.g., pattern, noise) components to a resulting healed region in the image. Mutiresolution processing and tiling can also be used to enhance performance.
In another aspect, a method of processing an image includes determining boundary pixels bounding a modification region in an image, the boundary pixels having values and being outside the modification region, and generating new pixel values for the modification region using the boundary pixel values such that a rate of a rate of change in pixel values is minimized from the boundary pixel values to new modification region pixel values. The generation of new pixel values can be performed for multiple channels to create similar overall gradations of color and illumination from the boundary pixel values to new modification region pixel values. The generation of new pixel values also can be performed by iteratively applying multiple kernels.
Multiresolution processing can be used to improve performance. If the image can be an image saved using a lossy compression technique (e.g., a JPEG image), this can be detected and the modification region can automatically set equal to one or more compression artifact regions between compression blocks.
In another aspect, a method of processing an image includes subtracting texture pixel values from pixel values corresponding to at least a portion of an image, generating new pixel values for a modification region of the image using the texture-subtracted pixel values, adding texture pixel values to the new pixel values, and writing the texture-added new pixel values to the image. The texture pixel values can be pixel values from a texture image having high frequency components or a repeating or non-repeating pattern. The new pixel value generation can be performed by iteratively applying a kernel.
The new pixel value generation can be performed by iteratively applying multiple kernels to create a continuous rate of change in pixel values from boundary pixel values to new modification region pixel values. The new pixel value generation can be performed by iteratively applying the multiple kernels at multiple resolutions of the image, starting with a lowest resolution version of the image. Each kernel can be applied separately to the multiple resolutions of the image, starting with the smallest kernel. Image tiling can be used, including padding between tiles to accommodate a largest kernel from the multiple kernels.
Conversion to a higher precision value representation (e.g., a fixed point value representation) can be performed before applying the multiple kernels. Each image can be blurred before applying the multiple kernels at a level. These techniques can be used to process multiple channel images.
Implementations of the systems and techniques described here may occur in hardware, software or a combination of both, and may include machine instructions for causing a machine to perform the operations described.
One or more of the following advantages may be provided. The systems and techniques described may result in significantly improved healing of image regions having unwanted features. Images healed using these systems and techniques may have a smooth transition from inside a healed region to outside the healed region. When applied to multiple channels in a color image, the resulting image may have similar overall gradations of color and illumination across the boundary of the selected region. Texture components may be added to a healed region allowing it to blend in with its surroundings, thereby making the healed region visually unnoticeable.
In addition, these systems and techniques may result in reduced processing time, and may reduce memory usage. For instance, generating replacement data values based on the difference between existing data values and texture data values may reduce processing time by half. Other than designating the pixels to be healed, and optionally a corresponding texture image, no filter parameters are required, because the shape of the healing region can be used to determine all remaining parameters needed to heal the image.
REFERENCES:
patent: 6141459 (2000-10-01), Gendel
patent: 6266054 (2001-07-01), Lawton et al.
Marcelo Bertalmio et al, “Image Painting”, Proceedings of SIGGRAPH 2000, New Orleans, USA, Jul. 2000 (2 versions).
William L. Briggs, “A Multigrid Tutorial”, copyright 1987; chapter 1-3.
Anil N. Hirani et al, “Combining Frequency and Spatial Domain Information for Fast Interactive Image Noise Removal”, Proceedings of SIGGRAPH 96 (New Orleans, LA, Aug. 4-9, 1996). InComputer GraphicsProceedings, Annual Conference Series, 1996,ACM SIGGRAPH, pp. 269-276.
Homan Igehy et al, “Image Replacement Through Texture Sythesis”,Proceedings of the 1997 IEEE International Conference on Image Processing.
Manuel M. Oliveira, “Fast Digital I
Chien Jen-Chan
Georgiev Todor G.
Hamburg Mark
Adobe Systems Incorporated
Fish & Richardson P.C.
Rogers Scott
LandOfFree
Generating replacement data values for an image region does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Generating replacement data values for an image region, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Generating replacement data values for an image region will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3063944