Television – Camera – system and detail – With single image scanning device supplying plural color...
Reexamination Certificate
1998-09-29
2001-05-22
Eisenzopf, Reinhard J. (Department: 2614)
Television
Camera, system and detail
With single image scanning device supplying plural color...
C382S167000, C358S525000, C358S512000
Reexamination Certificate
active
06236433
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The invention relates generally to image processing. More specifically, the invention relates to image scaling and color interpolation/recovery.
2. Description of the Related Art
A digital image of a scene/environment has a particular size which is defined by the number of rows and columns of pixels (individual color/intensity points) that it contains. The image size or “resolution” is thus expressed as the number of columns multiplied by the number of rows. For instance, an image with a resolution of 768×576 has 768 columns and 576 rows of pixels for a total of 442,368 pixels.
Often, the original size of an image as captured by an imaging device such as a camera or as later represented is too large for a particular application. While a larger resolution image contains more image information (more pixels per area) and is likely of a more desirable visual quality than a lower resolution image, bandwidth, memory and other constraints may dictate that a lower resolution image be used. For certain devices, such as digital cameras, it may be desirable to reduce its overall cost by utilizing a smaller resolution image so that the required storage component in the device is also smaller. In the context of videoconferencing, for instance, certain standardized image formats such as QCIF (Quarter Common Intermediate Format) have been defined so that receiving and transmitting nodes do not have to be concerned with converting discordant image sizes. In videoconferencing, it is often desirable to maintain a certain “frame” rate (the rate at which individual image frames are received and/or rendered for output). To maintain this frame rate, formats such as QCIF have been defined which are typically smaller than most captured digital image sizes, particularly those captured from certain digital cameras. Since an image may not be originally the same resolution as that desired by a particular application, a process known as image scaling is employed. When an image is scaled “up,” its size is increased and when it is scaled “down” its size is reduced. Hereinafter, when referring to “scaling” or “scaled image”, down scaling or reduction in image size is the intended meaning and usage of those terms.
The scaling of an image should be distinguished from image cropping, where the resolution is reduced by cutting out a portion of the image. Scaling implies that while the size of the image is reduced, the entire scene/environment in the unscaled image (hereinafter variously referred to as “original” or “unscaled” image) is maintained in great majority. The scene from the original image remains complete but is represented in a lower resolution after scaling.
Image scaling has been achieved in the art in several ways. The most common scaling technique averages pixels in particular image region in equal weighting and then “decimates” or throws away entire pixels in the region, thereby generating a pixel in the scaled image. The averaged pixel replaces an entire region of pixels, with the replaced region not necessarily the same size as the averaging region. For instance, consider a 2:1 scaling procedure where each two by two region of pixels in the original image is to be replaced by a single pixel in the scaled image. When determining the value of the scaled image pixel, it may be desirable to average together a larger region than the 2 by 2 region of replacement, such as a 3 by 3 neighborhood. In such an instance, the “sampling” region (3×3) is said to be larger than the “scaling” region (2×2) and may be useful in ensuring that more of the image is considered so that features that start in the scaling region and bleed over past the scaling region are given the proper consideration. An averaging method where each pixel in the sampling region is given equal weighting however is deficient in several regards. Primarily, the equal averaging of pixels has the effect of losing much of the original image information. Equal weight averaging does little to identify image features, since it treats all parts of the image region identically and then decimates all pixels.
In addition to image scaling, another, typically independent, image processing technique called color interpolation is employed to recover the missing colors in a pixel location generated by an image sensor as explained below. In digital still and video cameras and certain other imaging devices, raw images are first represented as rectangular row and column of pixels with each pixel having the intensity value of a particular color only. In the case of RGB (Red, Green and Blue) sub-sampling imaging devices, images are obtained and stored in a “Bayer” pattern. The Bayer pattern, when three color planes are sub-sampled using a color filter array (CFA), is characterized by including on one row of an image, alternating pixels of Red and Green, and then on a next row, alternating pixels of Blue and Green. For instance, the Bayer pattern for the first four rows of pixels (with each pixel typically an 8-bit value) is as follows (with the rows thereafter repeating the same pattern):
GRGRGR . . .
BGBGBG . . .
GRGRGR . . .
BGBGBG . . .
•
•
•
As a result, each pixel location contains an intensity value for a single color component only. Assuming, as is the case in some imaging devices, that each pixel of the Bayer pattern has 8 bits of resolution (i.e., the pixel is a value representing the intensity of the color ranging from 0-255), then a “full color” pixel, one having all three R, G and B components would be a 24-bit value. Color interpolation is the recovery of the two missing color components for each pixel color interpolation.
Often, scaling and color interpolation are performed independently and by separate processes. If scaling is performed prior to color interpolation, original sensor information regarding color content will be unknown to the interpolation process yielding a poorer quality image. If, however, where it is known as priori that both scaling and color interpolation are desired for the end image, there is a need for a combined technique that performs both. Further, given the shortcomings of conventional scaling and color interpolation techniques, the combined technique should be designed so as to yield acceptable image quality.
Furthermore, with regard to implementation, if scaling is to be implemented in hardware such as a CMOS (Complementary Metal-Oxide Semiconductor) imaging device, it is important to reduce the computational complexity of the scaling procedure, especially when many other functions must also be carried out by the device. When an imaging device is used to transmit image frames (a sequence of individual still images) for the purpose of videoconferencing, the transmission must be fast enough to maintain the frame rate and be compatible with the bandwidth capability of the interface between the imaging device and the processing device (computer system) that is used to package and transmit the captured image frames to the destination node. In devices that are dual-moded, which may provide both motion and still imaging, there is also desired methods and apparatus that can readily provide different levels of scaling interchangeably.
SUMMARY OF THE INVENTION
What is disclosed is a method comprising defining a scaling region by indicating in a CFA (Color Filter Array) a starting location, and generating a super-pixel which is a downscaled version of the scaling region, the super-pixel fully color interpolated, the downscaling and the color interpolation achieved in an integrated manner.
REFERENCES:
patent: 5053861 (1991-10-01), Tsai et al.
patent: 5065229 (1991-11-01), Tsai et al.
patent: 5418565 (1995-05-01), Smith
patent: 5778106 (1998-07-01), Juenger et al.
patent: 5914749 (1999-06-01), Bwolek et al.
patent: 6046772 (2000-04-01), Howell
Acharya Tinku
Metz Werner
Blakely , Sokoloff, Taylor & Zafman LLP
Eisenzopf Reinhard J.
Intel Corporation
Tran Trang U.
LandOfFree
Scaling algorithm for efficient color... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Scaling algorithm for efficient color..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Scaling algorithm for efficient color... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2548041