Image analysis – Applications – Vehicle or traffic control
Reexamination Certificate
2001-03-20
2002-09-17
Jones, Andrew W. (Department: 2621)
Image analysis
Applications
Vehicle or traffic control
C340S933000
Reexamination Certificate
active
06453056
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates generally to the field of automated image analysis and identification. More specifically, the present invention relates to a method and apparatus for automatically generating a database of road sign images and positions where the road signs are identified from video images depicting roadside scenes that are recorded from a vehicle navigating a road and having a system that stores location metrics for those video images.
BACKGROUND OF THE INVENTION
The recognition of road sign images as objects of interest in a set of video images has been developed primarily for use in connection with automated vehicle navigation systems. By recognizing road sign images from the real time output of a forward facing video camera on board a vehicle, instructions and navigational assistance can be provided to a driver or an entirely automated vehicle navigation system can be developed. Unlike object recognition systems that are deployed in a controlled environment with known lighting and background conditions, road sign recognition systems must be able to perform under a wide range of environmental and lighting conditions. In addition, the road sign images must be recognized quickly and in real time so that the information can be immediately available for use. Fortunately, most navigational road sign images are made of regular shapes and have colors that are known and conform to certain standards and combinations. All of these factors have resulted in the use of template matching and color pair matching as the most common image analysis techniques for quickly isolating road sign images from the real time video signal.
While existing techniques that identify road sign images by using template matching can identify a limited subset of road signs that are particularly helpful in driving instruction and navigational information, the need to perform these operations in real time limits the overall number and types of road signs that can be identified by such a template matching technique. The more template patterns and color pairs that are added to a template matching technique, the longer it takes to process the video image. Consequently, many types of road signs that convey non-navigational information, such as parking information, are excluded from the template matching process in order to reduce the total possible number of combinations that must be evaluated.
U.S. Pat. No. 5,633,946 describes a method and apparatus for collecting and processing visual and spatial information from a moving platform. This patent describes a vehicle with multiple video cameras oriented in different directions that record road scenes as the vehicle is driven along a road. Positional information from a Global Positioning System (GPS) receiver and an inertial navigation system (INS) in the vehicle is simultaneously recorded with the video images. Frames of the video signal from multiple cameras are interleaved together and each frame is recorded on a video tape with a time code along with the current spatial position information provided by the GPS receiver and the INS. The video images are then analyzed with a centerline offset process to create street segments that represent a sequence of video images associated with a given segment of a street or road. The patent describes a number of applications for using the street segment information which can include the creation and update of address ranges, the integration of address attribute information, the creation and maintenance of street network topologies, the collection of vehicle routing information, the creation and maintenance of map boundary polygon topologies and attributes and the accurate location of point features and their attributes. In each of these cases, however, the video images in a given street segment must be examined visually by an operator in order to extract relevant attribute information.
One problem that has yet to be addressed concerning road signs has been the creation and maintenance of comprehensive and accurate information as to the identity and location of all of the road signs in a given municipality or jurisdiction. Contrary to what might be expected, most municipal street departments or county highway departments do not have accurate records of the number, type and location of all road signs within their jurisdiction. For a typical larger municipality, it is not uncommon to have more than 100,000 road signs in use. This large number of road signs effectively precludes using manual techniques to create and maintain accurate sources of information as to the location and condition of each road sign. As a result, most road sign maintenance and repair is performed either in connection with the building or upgrading of a road, or on an ad hoc basis as a result of complaints from the public or visual observations by road maintenance crews.
While existing techniques for road sign recognition have been adapted for use with vehicle navigation systems, it would be desirable to provide a system for automatically generating a database of road sign images and positions where the road signs are identified from video images depicting roadside scenes that are recorded from a vehicle navigating a road and having a system that stores location information.
SUMMARY OF THE INVENTION
The exemplary embodiment described, enabled, and taught herein is directed to the task of building a database of road signs by type, location, orientation, and condition by processing vast amounts of video image frame data. The image frame data depict roadside scenes as recorded from a vehicle navigating said road. By utilizing differentiable characteristics the portions of the image frame that depict a road sign are stored as highly compressed bitmapped files each linked to a discrete data structure containing one or more of the following memory fields: sign type, relative or absolute location of each sign, reference value for the recording camera, reference value for original recorded frame number for the bitmap of each recognized sign. The location data is derived from at least two depictions of a single sign using techniques of triangulation, correlation, or estimation. Thus, output signal sets resulting from application of the present method to a segment of image frames can include a compendium of data about each sign and bitmap records of each sign as recorded by a camera. Thus, records are created for image-portions that possess (and exhibit) detectable unique differentiable characteristics versus the majority of other image-portions of a digitized image frame. In the exemplary sign-finding embodiment herein these differentiable characteristics are coined “sign-ness.” Thus, based on said differentiable characteristics, or sign-ness, information regarding the type, classification, condition (linked bitmap image portion) and/or location of road signs (and image-portions depicting said road signs) are rapidly extracted from image frames. Those image frames that do not contain an appreciable level of sign-ness are immediately discarded.
Differentiable characteristics of said objects include convexity/symmetry, lack of 3D volume, number of sides, angles formed at corners of signs, luminescence or lumina values, which represent illumination tolerant response in the L*u*v* or LCH color spaces (typically following a transforming step from a first color space like RGB); relationship of edges extracted from portions of image frames, shape, texture, and/or other differentiable characteristics of one or more objects of interest versus background objects. The differentiable characteristics are preferably tuned with respect to the recording device and actual or anticipated recording conditions are taught more fully hereinbelow.
The method and apparatus of the present invention rapidly identifies, locates, and stores images of objects depicted in digitized image frames based upon one or more differentiable characteristic of the objects (e.g., versus non-objects and other detected background noise). The present invention may be impleme
Laumeyer Robert Anthony
Retterath James Eugene
Facet Technology Corporation
Jones Andrew W.
Patterson Thuente Skaar & Christensen P.A.
LandOfFree
Method and apparatus for generating a database of road sign... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for generating a database of road sign..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for generating a database of road sign... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2901084