Television – Camera – system and detail – Combined image signal generator and general image signal...
Reexamination Certificate
1998-07-08
2002-12-10
Vu, Ngoc-Yen (Department: 2612)
Television
Camera, system and detail
Combined image signal generator and general image signal...
C348S237000, C348S253000, C348S280000, C358S520000
Reexamination Certificate
active
06493029
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to image restoration techniques for use with image sensors and in particular to signal processing methods for luminance estimation for use with matrix array image sensors of the type commonly incorporated in video cameras.
2. Description of Related Art
The development of low-cost video cameras is driven by the existing Closed Circuit Television (CCTV) camera marketplace and by expected new markets, such as, for example, video conferencing. A major cost of these cameras is the image sensor chip whose cost is a monotonic function of sensor area. Two major goals of sensor designers are to improve sensitivity by increasing pixel area and improve resolution by increasing pixel count. Both of these steps have the disadvantage of increasing sensor area and hence sensor cost. If the sensor area is to remain constant then, until now, image resolution could only be increased at the cost of lowered sensitivity of the sensor and sensitivity could only be increased at the cost of lowered resolution.
In one known type of video camera a colour image is formed on a single image sensor device which is spatially divided into many (typically 100,000 or more) pixels and each pixel is covered with a colour filter corresponding to one of a number of desired spectral components. These may typically be red, green and blue components or, for example, cyan, magenta and yellow. Further colour sets are also sometimes used.
Many different types of colour matrix patterns are possible. The signal outputs of all the pixels are combined by means of a reading operation at the sensor device output. Many image restoration techniques have been developed for combining the signals from individual pixels in the matrix array into a composite image representation which presents a “real” -looking image to the human eye. One known technique involves superimposing a highpass-filtered or “edge-extracted” luminance signal onto lowpass-filtered or “spatially smoothed” component colour signals (e.g. red, green, blue) obtained from the pixel matrix. Such image restoration techniques can enhance the perceived resolution or “sharpness” of the video camera image.
One problem associated with image restoration is obtaining an accurate luminance signal from the sensor. It is known that the green spectral component is closely related to the luminance response of the human eye. It has therefore been known to use the signal outputs of green pixels, or alternatively another colour of pixel whose colour is representative of luminance, to obtain the necessary luminance signal. As the chosen colour of pixels (e.g. green) only account for some of the pixels in a colour image sensor array, however, it is desirable to estimate luminance at the remaining pixel sites.
It is an object of the present invention to provide a signal processing method for accurately estimating luminance for use in matrix array colour image sensors.
U.S. Pat. No. 4,630,307 describes a signal processing method for estimating luminance for. use in matrix array sensors having luminance-sensing pixels arranged in a chequerboard pattern. The method uses signals obtained from luminance-sensing pixels to estimate luminance values at pixels in the matrix array which are not capable of sensing luminance.
SUMMARY OF THE INVENTION
In the following text the term “row” is defined as a horizontal row and the term “column” is defined as a vertical row.
According to a first aspect, the present invention comprises a signal processing method for estimating luminance, for use with matrix patterned image sensors where elected sensor pixels from which luminance-representative signals are taken only occupy alternate horizontal and vertical positions in the matrix pattern, the method comprising identifying each pixel site which is at the centre of both a row and a column of five pixels and which is not the site of a said elected sensor pixel, and for each such identified site processing the signals output from the immediately adjacent four pixels in said row and column to establish both signal amplitude and pixel position and evaluating a luminance value for the pixel of the identified site from the amplitude values of the median pair of said signal amplitudes in accordance with an algorithm where decision taking is based upon the location of the pixels from which the two signals of highest amplitude emanate, the algorithm being such that when the two signals of highest amplitude emanate from pixels in the same row or column a further signal processing step is undertaken to identify, in said row and column, the signal outputs from the two respective next adjacent pixels to the pixels having the highest and lowest signal amplitudes, and the signal output from the pixel at the identified site, to enable an amplitude comparison to be made according to the result of which one of the said median pair is selected.
One advantage of the signal processing method according to the first aspect of the invention is that it utilizes signal outputs from pixels in said row and column which pixels are-next adjacent to the highest and lowest amplitude ones of said four pixels immediately adjacent the identified site.
Preferably, the selected one of said median pair is the higher amplitude one where the signal output from the next adjacent pixel to said pixel having the highest signal amplitude is closer in amplitude to the signal output from the pixel at the identified site than the signal output from the next adjacent pixel to said pixel having the lowest signal amplitude. Preferably, also, the selected one is the lower amplitude one of said median pair where the signal output from the next adjacent pixel to said pixel having the lowest signal amplitude is closer in amplitude to the signal output from the pixel at the identified site than the signal output from the next adjacent pixel to said pixel having the highest signal amplitude.
According to a second aspect the present invention comprises a signal processing method for estimating luminance for use with matrix array colour image sensors in which elected luminance-sensing pixels capable of producing signals representative of luminance are arranged in a chequerboard pattern with colour sensing pixels which are capable of producing signals representative of colour, the method comprising the steps of: ranking the signal values representative of luminance obtained from a cluster of four luminance-sensing pixels surrounding a first colour-sensing pixel, analyzing the spatial distribution of said ranked signal values so as to detect the presence of any diagonal luminance contour edge, horizontal luminance contour stripe or vertical luminance contour stripe at said first colour-sensing pixel, and determining an estimated discrete signal value representative of luminance for said first colour-sensing pixel based on one or more middle ranked signal values in a colour-enhancing fashion dependent upon, firstly, whether any diagonal luminance contour edge is detected and, secondly, whether any horizontal or vertical luminance contour stripe which is detected is interpreted as a dark stripe on a light background or a light stripe on a dark background, wherein when a horizontal or vertical luminance contour stripe is detected, the signal values representative of colour obtained from a plurality of said colour-sensing pixels in the vicinity of said cluster of four luminance-sensing pixels are compared so as to determine whether said detected horizontal or vertical luminance contour stripe is interpreted as a dark stripe on a light background or a light stripe on a dark background.
One advantage of the signal processing method according to the second aspect of the invention is that it utilizes colour signal values from pixels other than said elected luminance-sensing pixels to influence the estimated luminance signal value for said first colour-sensing pixel.
Preferably, said plurality of colour-sensing pixels in the vicinity of said cluster of four luminance-sensing pixels includes s
Denyer Peter Brian
Smith Stewart Gresty
VLSI Vision Limited
Vu Ngoc-Yen
LandOfFree
Image restoration method and associated apparatus does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image restoration method and associated apparatus, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image restoration method and associated apparatus will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2924003