Data processing: vehicles – navigation – and relative location – Vehicle control – guidance – operation – or indication – Vehicle subsystem or accessory control
Reexamination Certificate
2001-03-05
2003-10-07
Beaulieu, Yonel (Department: 3661)
Data processing: vehicles, navigation, and relative location
Vehicle control, guidance, operation, or indication
Vehicle subsystem or accessory control
C250S208100
Reexamination Certificate
active
06631316
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention generally relates to image processing systems, and more particularly, to image processing systems used to control vehicle equipment, such as vehicle headlamps, windshield wipers, etc.
Recently, many vehicular control accessories utilizing image processing systems have been proposed. For example, U.S. Pat. No. 5,837,994 entitled “Control System to Automatically Dim Vehicle Head Lamps,” commonly assigned with the present invention and hereby incorporated by reference, discloses a control system which utilizes an image sensor and a processor to detect the headlamps of other vehicles at night and automatically control the state of the high beam headlamps of a vehicle. Another example of an image processing system to control vehicle equipment is given in U.S. Pat. No. 5,923,027 entitled “Moisture Sensor and Windshield Fog Detector,” also commonly assigned with the present invention and hereby incorporated by reference. This patent discloses an image processing system, which acquires images of a vehicle windshield in order to detect the presence of rain or fog.
In each of these systems, as well as several other disclosed automotive image processing systems (see, for example, U.S. Pat. Nos. 5,765,116, 5,675,489, and 5,660,454 and PCT Published Patent Application No. WO 00/53465), images are acquired by an image sensor and stored into a memory in their entirety for subsequent processing. While technically very convenient, the use of this amount of image memory presents problems when adapting the system for commercial use, especially in the highly cost-sensitive automotive market. Most low cost microcontrollers or digital signal processors (DSPs) suitable for these types of applications are equipped with only a few hundred bytes to a few kilobytes of random access memory (RAM) into which the images can be stored. The processing core of many microcontrollers is typically available with a variety of RAM configurations, with the price increasing as the amount of memory increases. Therefore, it is advantageous to use a microcontroller with the least amount of RAM possible. The use of a small amount of RAM in prior art systems limits the size of the image, which can be stored and thus greatly limits the resolution.
A common off-the-shelf image sensor may have 352×288 pixels, known as the CIF format. Storing an entire image from this sensor requires approximately 100 kilobytes of RAM—far more than is typically available on a low cost microcontroller. Some microcontrollers have the ability to increase the amount of RAM available by the addition of an external memory chip. These microcontrollers must have an external memory bus, increasing the pin count and thus the cost and complexity of the microcontroller and the circuit board to which it is attached. The cost of the external memory itself must also be considered, and despite rapid memory price declines, this cost is anticipated to remain significant for some time to come. Finally, if an image must be transferred to memory before it can be processed, the total time required to acquire and analyze an image will be greater than if the analysis could occur simultaneously with the acquisition.
What is needed is a low cost image processing system to control automotive equipment which is capable of analyzing images without first storing them to memory, thus reducing the total amount of memory required in the system.
SUMMARY OF THE PRESENT INVENTION
The present invention solves the problem of the prior art by providing an image processing system, which does not require a memory with enough storage locations to store the digital grayscale value of every pixel in the image. The system contains an image sensor array containing a plurality of pixels, each of which is configured to provide an output indicative of the amount of light received by the pixel over the exposure time. The system also provides an analog-to-digital (A/D) converter to quantize the signal from the pixel into a digital grayscale value. The system also provides a processor in communication with the image sensor and A/D converter to analyze the images acquired by the image processor and control vehicle equipment based upon the results of processing the images. Finally, the system contains a memory for storing the grayscale value of some of the pixels and for storing other data used by the processor. By providing an image processing system with less available memory than is needed to store all pixels, the cost and the complexity of the image processing system can be reduced.
According to one aspect of the present invention, the amount of memory required is reduced by acquiring and analyzing only a portion of the image at a given time. This may be accomplished by acquiring a single row of pixels at a time or by acquiring a subwindow of pixels, which is a subset of the window containing all the pixels.
In another aspect of the present invention, a digital image processing filter is implemented by only storing a most recent group of pixels and performing the filter algorithm on this group of pixels. The most recent group of pixels may be, for example, the last few rows of the image received. The system performs the filter algorithm on the last few acquired rows and discards the oldest row as a new row is acquired.
In another aspect of the invention, some of the pixel values received by the processor are discarded as they are received and only a subset of the values is stored in the memory. Pixels may be discarded at a uniform rate throughout the image or at a non-uniform rate. For example, pixels may be discarded at a high rate near the periphery of the image and at a low or zero rate near the center of the image to provide a greater resolution near the center of the image.
In another aspect of the invention, the values of adjacent pixels may be averaged with one another and a single value stored in memory representing the average of several adjacent pixels. The number of pixels averaged with each other may be uniform throughout the image or may be non-uniform. Many pixels may be averaged with each other near the periphery of the image and few or zero pixels may be averaged with each other near the center of the image to provide a greater resolution near the center of the image.
In another aspect of the invention, objects identified in the image may be extracted from the image and stored as the image is being acquired. For example, the present invention may be used to identify headlamps of oncoming vehicles in order to determine the high beam state of the headlamps of the controlled vehicle. The present invention can identify the presence of an oncoming headlamp in the image as it is being acquired and store various information about the headlamp in an object list.
In another aspect of the present invention, the image data may be compressed as it is being acquired. This compression is accomplished by many ways including reducing the number of quantization levels (bit depth) of a pixel value as it is being received. Reducing bit depth of the image may be performed either uniformly or non-uniformly across the image and by requantizing either linearly or non-linearly. This aspect of the present invention is particularly useful for reducing data rate when the camera is positioned away from the processor.
In another aspect of the present invention, image data is reduced by storing segments of connected pixels, which together make up an object of interest. For example, a string of connected pixels in one row of an image, all with grayscale values above a threshold, may be stored as one object. Rather than storing the grayscale values of each pixel, only the starting and ending pixels and the cumulative grayscale value of all the pixels are stored. In a color implementation, the average color of the segment may be stored. Finally, two dimensional groups of objects may also be stored. This aspect is particularly useful for greatly reducing the memory requirements and transmitting image data with the least possible overhead. The processing
Ockerse Harold C.
Pierce Mark W.
Stam Joseph S.
Beaulieu Yonel
Gentex Corporation
Price Heneveld Cooper DeWitt & Litton
LandOfFree
Image processing system to control vehicle headlamps or... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image processing system to control vehicle headlamps or..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image processing system to control vehicle headlamps or... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3156326