Communications: electrical – Land vehicle alarms or indicators – Of collision or contact with external object
Reexamination Certificate
2001-12-14
2004-04-27
Hofsass, Jeffrey (Department: 2632)
Communications: electrical
Land vehicle alarms or indicators
Of collision or contact with external object
C340S435000, C340S438000, C340S995130, C340S426140, C340S426150, C340S576000, C340S903000
Reexamination Certificate
active
06727807
ABSTRACT:
FIELD OF THE INVENTION
The invention relates to automobiles and, in particular, to a system and method for processing various images and detecting conditions in the driving environment based thereon and notifying the driver of certain conditions, when appropriate.
BACKGROUND OF THE INVENTION
Much of today's driving occurs in a demanding environment. The proliferation of automobiles and resulting traffic density has increased the amount of external stimulii that a driver must react to while driving. In addition, today's driver must often perceive, process and react to a driving condition in a lesser amount of time. For example, speeding and/or aggressive drivers give themselves little time to react to a changing condition (e.g., a pothole in the road, a sudden change of lane of a nearby car, etc.) and also give nearby drivers little time to react to them.
Despite advancements in digital signal processing technologies, including computer vision, pattern recognition, image processing and artificial intelligence (AI), little has been done to assist drivers with the highly demanding decision-making involved in normal driving. In one system implemented in the Cadillac DeVille, military “Night Vision” is adapted to detect objects in front of the automobile. Heat in the form of high emission of infrared radiation from humans, other animals and cars in front of the car is captured using cameras (focusing optics) and focused on an infrared detector. The detected infrared radiation data is transferred to processing electronics and used to form a monochromatic image of the object. The image of the object is projected by a head-up display near the front edge of the hood in the driver's peripheral vision. At night, objects that may be outside the range of the automobiles headlights may thus be detected in advance and projected via the heads-up display. The system is described in more detail in the document “DeVille Becomes First Car To Offer Safety Benefits Of Night Vision” at http://www.gm.com/company/gmability/safety/crash_avoidance
ewfeatures
ight_vision.html.
Among other deficiencies of the DeVille Night Vision system, the display only provides the thermal image of the object, and the driver is left to identify what the object is by the contour of the thermal image. The driver may not be able to identify the object. For example, the thermal contour of a person riding a bicycle (which has a relatively low thermal signature) may be too alien for a driver to readily discern. The mere presence of such an unidentifiable object may also be distracting. Finally, it is difficult for the driver to judge the relative position of the object in the actual environment, since the thermal image of the object is displayed near the front edge of the hood.
U.S. Pat. No. 5,414,439 to Groves et al. also relates to a Night Vision type system that outputs a video signal of a thermal image to a head up display (HUD). In one case, virtual images of pedestrians are projected below the actual image seen by the driver, thus forewarning the driver of pedestrians that may not be visible in the actual image. Alternatively, the virtual images are superimposed on the real scene. In U.S. Pat. No. 5,001,558 to Burley et al., light captured by a color camera is superimposed on thermal signals captured by an infrared imaging device. Thus, for example, a red signal light emitted by a traffic signal is captured by the color camera and superimposed on the thermal image created by the warm traffic signal and displayed.
A method of detecting pedestrians and traffic signs is described in “Real-Time Object Detection For “Smart” Vehicles” by D. M. Gavrila and V. Philomin, Proceedings of IEEE International Conference On Computer Vision, Kerkyra, Greece 1999 (available at www.gavrila.net), the contents of which are hereby incorporated by reference herein. A template hierarchy captures a variety of object shapes, and matching is achieved using a variant of Distance Transform based-matching, that uses a simultaneous coarse-to-fine approach over the shape hierarchy and over the transformation parameters. The Introduction section refers to informing the driver (or taking other measures) regarding certain potential hazards, such as a collision with a pedestrian, speeding, or turning the wrong way down a one-way street. It is noted, however, that the focus of the document is on detection of the above-mentioned objects and does not describe how or the particular circumstances under which the driver is alerted of a potentially hazardous situation.
A method of detecting pedestrians on-board a moving vehicle is also described in “Pedestrian Detection From A Moving Vehicle” by D. M. Gavrila, Proceedings Of The European Conference On Computer Vision, Dublin, Ireland, 2000, the contents of which are hereby incorporated by reference herein. The method builds on the template hierarchy and matching using the coarse-to-fine approach described above, and then utilizes Radial Basis Functions (RBFs) to attempt to verify whether the shapes and objects are pedestrians. The focus of the document is also on detection of pedestrians and does not describe alerting of the driver.
“Autonomous Driving Approaches Downtown” by U. Franke et al., IEEE Intelligent Systems, vol. 13, no. 6, 1998 (available at www.gavrila.net) describes an image recognition that focuses on application to autonomous vehicle guidance. The Introduction section refers to numerous prior art vision systems for lateral and longitudinal vehicle guidance, lane departure warning and collision avoidance. The document describes image recognition of pedestrians, obstacles (such as other automobiles), road boundaries and markings, traffic signals and traffic signs. While the focus of the document is on vehicle guidance, among other things, the document also refers to rear-end collision avoidance or red traffic light recognition and warning, tracking road contours and providing a lane departure warning, and warning the driver when driving faster than the speed limit. In particular, FIG. 4.9 shows a display of a detected red light, where an enlarged and prominent image of the red light is displayed adjacent the actual image of the red light in a monitor-style display. (See, also, “Smart Vehicles” at www.gavrila.net/Computer_Vision/Smart_Vehicles/smart_vehicles.html, which indicates that a monitor-style display is used.)
SUMMARY OF THE INVENTION
The prior art fails to provide a comprehensive system that detects potentially hazardous traffic situations via image recognition, and then present an image of the hazard to the driver in a manner that is designed to alert the driver without unnecessarily distracting him or her and effectively provide the location of the potential hazard.
It is thus an objective of the invention to provide a system and method for alerting a driver of an automobile to a traffic condition. The system comprises at least one camera having a field of view and facing in the forward direction of the automobile. The camera captures images of the field of view in front of the automobile. A control unit receives images of the field of view from the camera and identifies objects therein of at least one predetermined type. The control unit analyzes the object images of the at least one predetermined type to determine whether one or more of the identified objects of the at least one predetermined type present a condition that requires the driver's response. A display receives control signals from the control unit regarding those objects of the at least one predetermined type that present a condition that requires the driver's response. The display displays an image of those objects that require a response to the driver that is positioned and scaled so that it overlays (is superimposed with respect to) the actual object as seen by the driver, the displayed image of the object enhancing a feature of the actual object to alert the driver.
The method comprises the steps of first capturing images of the field of view in front of the automobile. Objects in the image o
Colmenarez Antonio
Gutta Srinivas
Trajkovic Miroslav
Hofsass Jeffrey
Koninklijke Philips Electronics , N.V.
Previl Daniel
LandOfFree
Driver's aid using image processing does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Driver's aid using image processing, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Driver's aid using image processing will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3259677