Television – Camera – system and detail – Combined image signal generator and general image signal...
Reexamination Certificate
1999-06-08
2004-10-19
Moe, Aung (Department: 2612)
Television
Camera, system and detail
Combined image signal generator and general image signal...
C348S272000
Reexamination Certificate
active
06806902
ABSTRACT:
The present invention relates generally to digital photography and particularly to correcting bad pixel data in a digital camera.
BACKGROUND OF THE INVENTION
One of the significant factors affecting the performance and yield of color image sensor devices is bad pixels. These are considered to be individual pixels locations that, due to a processing defect, produce an output signal that is either higher or lower than neighboring pixels when exposed to a similar light source. This defect can be in the form of an excess leakage current adding or subtracting from the photo generated current, or a short or open circuit located within the photocell. CMOS image sensors are particularly susceptible to these failures due to their relatively high amount of circuitry per photocell. It is desirable to have a method to detect and remove these bad pixels when the light level is such that the defect becomes noticeable.
Before describing prior art and the invention, the steps used to create a color image from a single color image sensor are briefly described. In a monochrome sensor, an array of photocells is exposed to light for a period of time and the state of the individual photocells is then serially output through readout circuitry. This type of readout circuitry can vary depending upon the technology being used (CMOS versus CCD), but the result is the same, a stream of data corresponding to the light landing on the first column and first row, followed by the light hitting the second column and first row, continuing until the first row has been completely read out. At this point, the second row is read out, first column followed by second column until all columns have been read out. This continues until the entire array has been read out. This sequences the data in a format compatible with the way most display devices accept their input.
FIG. 1
shows a diagram of a typical system used in a CMOS image sensor to achieve this. The size of the array is arbitrary to this algorithm. In a single color image sensor, each of the photocells is covered with a color filter material that passes light of different wavelengths. Frequently, for progressively scanned image sensors, three types of filter material are chosen to be red (R), green (G) and blue (B), and are arranged in a ‘Bayer’ pattern as shown in FIG.
2
. In this case, when a given row of the array is read, the output alternates between photocells responsive to two different spectrums of light. For every other row, the output alternates between the blue and green spectrums, and during the opposite rows alternates between the green and red spectrums. After readout it is necessary to form a complete RGB value for each pixel of the array. This is typically performed by storing one or more lines of data from the photo array, and interpolating from the surrounding pixels a complete RGB value for each pixel. One method to perform this function is shown in FIG.
3
.
FIG. 3
shows the potential pixel values
110
that can be used to form the complete RGB value for the four pixels
114
at locations Col (column)
1
and
2
, Row
1
and
2
. As can be seen from this figure, a single pixel error will corrupt multiple pixels if not corrected prior to the RGB interpolation for each pixel.
It has previously been shown (B. Dierickx, G. Meynants, “Missing pixels correction algorithm for image sensors,” AFPAEC Europto/SPIE, Zurich, 18-21 May 1998; proc. SPIE vol. 3410, pp. 200-203, 1998. http://www.imec.be/bo/publications/wpc98.ps.) that it is possible to detect and correct bad pixels in a monochrome sensor based on an evaluation of the surrounding pixel elements. This method is based on establishing an upper and lower bound on an acceptable value of a given pixel and limiting the pixel's value to that range. However, the proposed method by Dierickx and Meynants for dealing with color image sensors was impractical for two reasons. First, they proposed that the correction be performed on the reconstructed RGB (or YUV) signals, provided that the reconstruction results in single pixel defects when the original defect was single pixel in nature. This, however, is not the case for most reconstruction algorithms in use today. As is shown in
FIG. 3
, a single pixel defect will affect more than one pixel in the output video. Second, the technique would need to be performed on three components if implemented in the reconstructed RGB (or YUV) domain, increasing the circuitry required three-fold.
Therefore, there is a need for a system that detects and corrects bad pixels in a color sensor without the additional circuitry that is required by Direickx and Meynants' method.
SUMMARY
In summary, the present invention is a system and method for detecting and correcting bad pixel data from a digital color sensor. Embodiments of the present invention are configured to develop two estimates for the value of a given pixel based on the values of surrounding pixels. One estimate is used to limit the maximum value that a given pixel is allowed to take and the other estimate is used to limit the minimum value that a given pixel is allowed to take. These values are used to detect bad pixels (pixels whose value is not between the minimum and maximum values) and to correct the bad pixels (by setting the value of the bad pixel based on the minimum and maximum values).
One embodiment of the present invention includes memory and computing circuitry. The memory includes four line memories to store four rows worth of graphics data read out of a photocell array and a set of pixel delay elements, each storing a pixel datum used to test and correct a pixel under test. For a given pixel under test, the computing circuitry uses only the values output from the pixel delay elements to determine the minimum and maximum limits for that pixel. In one embodiment, the values output by the pixel delay elements and used by the computing circuitry define a 5 pixel ×5 pixel plus symbol (+) centered at the pixel under test.
In one embodiment, the computing circuitry determines the minimum and maximum limits for a given pixel under test by first determining the derivatives of the pixels above and below and to the left and right of the given pixel. The computing circuitry then computes two vertical and two horizontal estimates for the pixel under test by respectively subtracting and adding the derivatives from/to the values of the nearest photocells that are responsive to the same color of light as the pixel under test. The computing circuitry then sets the minimum limit for the pixel under test to the smallest of the four estimates and the maximum limit to the largest of the four estimates. Alternatively, to prevent the erroneous correction of good pixel data, the computing circuitry considers, in addition to the four estimates, pixel data of the same color as the pixel under test when setting the Max and Min limits. If the value of the pixel under test is less then the minimum limit, the computing circuitry resets it to the minimum limit. If the value of the pixel under test is greater than the maximum limit, the computing circuitry resets it to the maximum limit. The corrected pixel is then written back to the appropriate line memory so that subsequent computations using that pixel will not be affected by bad pixel data.
In yet another embodiment, the computing circuitry determines the correct value for the given pixel by interpolating the minimum and maximum limits. This form of correcting bad pixels, which does not require low pass filtering techniques, maintains good image detail.
The present invention is equally applicable to monochrome or color pixel data.
REFERENCES:
patent: 4602291 (1986-07-01), Temes
patent: 5086343 (1992-02-01), Cook et al.
patent: 5327246 (1994-07-01), Suzuki
patent: 5805216 (1998-09-01), Tabei et al.
patent: 6002433 (1999-12-01), Watanabe et al.
patent: 6028628 (2000-02-01), Van Der Valk
patent: 6104839 (2000-08-01), Cok et al.
patent: 6381357 (2002-04-01), Tan et al.
patent: 6509927 (2003-01-01), Prater et al.
patent: 6529238 (2003-03-01), Mahant-
Chrontel, Inc.
Dorsey & Whitney LLP
Moe Aung
LandOfFree
System and method for correcting bad pixel data in a digital... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for correcting bad pixel data in a digital..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for correcting bad pixel data in a digital... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3276849