Image analysis – Applications – Manufacturing or product inspection
Reexamination Certificate
1997-12-18
2001-12-11
Au, Amelia M. (Department: 2623)
Image analysis
Applications
Manufacturing or product inspection
C382S209000, C382S220000
Reexamination Certificate
active
06330353
ABSTRACT:
FIELD OF THE INVENTION
This disclosure relates to a method for optical inspection and, more particularly, to a method of refinement of localization of patterns based on optical flow constraints.
DESCRIPTION OF THE RELATED ART
Automatic visual inspection of parts for defect detection in manufacturing processes is one of the most important applications in machine vision. The performance of an automatic visual inspection system can be determined by its reliability, efficiency and generality. The inspection system needs to be reliable under different illumination conditions and in noisy environments. The reliability of the inspection system, which is usually characterized by its false alarm rate, is very crucial to the quality control in a manufacturing process.
The efficiency of an automatic inspection system is directly related to the throughput of the product. In addition, automatic inspection systems may perform different kinds of inspection tasks.
A number of methods have been proposed for automatic visual inspection. They can be roughly categorized into two approaches namely, the image reference approach and the design-rule verification approach. The image reference or image subtraction approach compares every pixel in the inspection image with the corresponding pixel in the reference image, which is a sensed defect-free image or a synthetically generated image from a CAD model. The design-rule verification approach checks for the violation of a set of generic rules, such as design width and spacing standards, in the image. The image reference approach is very popular in automatic visual inspection due to its general applicability to a variety of inspection tasks. However, it requires very precise alignment of the inspection pattern in the image. Although the design-rule verification approach does not need very accurate alignment, it usually requires complicated algorithm design for each individual inspection task. In addition, the design-rule verification approach is in general less reliable than the image reference approach.
Visual inspection processes are often used to provide a check on the quality of products. A faster and more reliable method is advantageous for automated visual inspection processes. Therefore, a fast and precise pattern alignment algorithm, which can be used in the image reference approach for automated visual inspection is desirable. To achieve very precise pattern alignment, exhaustive template search is extremely time consuming when the size of the pattern is large. Some methods have been proposed to resolve this alignment problem.
In one proposal, an image registration technique is performed by fitting feature points in the zero-crossings extracted from the image to be inspected to the corresponding points extracted from the CAD model via an affine transformation. Unfortunately, the correspondence between the two set of feature usually cannot be reliably obtained. Another proposal employed a sum-of -squared-differences (SSD) method to determine the shift between the two images. In addition to its restriction on the recovery of shift alignment only, this method could not handle illumination changes between the image to be inspected and the reference image.
Localization refinement includes the ability to determine between defects in an inspection pattern or the misalignment of the pattern. In an inferior system misaligned patterns are rejected by the inspection system resulting in undue costs.
Therefore, a need exists for an accurate, efficient and robust method for determining precise 2D localization of an inspection pattern for applications in automated visual inspection.
SUMMARY OF THE INVENTION
A method for localization refinement of inspection patterns includes the steps of providing a template image comprising pixels in a pattern, each pixel having an intensity, providing an input image having a same pattern of pixels as the template image and minimizing an energy function formed by weighting a sum of modified optical flow constraints at locations of the pixels of both the template image and the input image to determine a shift and rotation between the pattern of the template image and the input image.
In other methods, the steps may include partitioning the template image into blocks of pixels, determining a reliability measure for each pixel in each block and identifying the pixel location for each block having a largest reliability measure as the feature point for each block.
The steps of providing an input image with an initial shift and rotation to the template image and minimizing the energy function formed by weighting the sum of modified optical flow constraints at locations of the feature points of the template image to determine the shift and rotation between the template image and the input image may also be included. The step of minimizing an energy function formed by weighting a sum of modified optical flow constraints may further include the steps of calculating a Hessian matrix and a gradient vector of the energy function based on an initial guess of a shift and a rotation, updating the initial guess based on the calculating of the Hessian matrix and the gradient vector of the energy function and iteratively recalculating the Hessian matrix and the gradient vector of the energy function until an updated guess is within an acceptable increment from a previous updated guess. The step of smoothing the template image to reduce noise effects may also be included. The step of minimizing an energy function formed by weighting a sum of modified optical flow constraints may further include the step of
incorporating an illumination change factor into the optical flow constraints for accounting for pixel intensity changes due to illumination effects.
A method for localization refinement of inspection patterns the steps of determining features points on a template image from among blocks of pixels, providing an input image with an initial shift and rotation to the template image and minimizing an energy function formed by weighting a sum of modified optical flow constraints at locations of the feature points of the template image to determine a shift and rotation between the template image and the input image.
In other methods, the step of determining feature points may include partitioning the template image into blocks of pixels, determining a reliability measure for each pixel in each block and identifying the pixel location for each block having a largest reliability measure as the feature point for each block. The step of minimizing an energy function formed by weighting a sum of modified optical flow constraints at locations of the feature points further includes the steps of calculating a Hessian matrix and a gradient vector of the energy function based on an initial guess of a shift and a rotation, updating the initial guess based on the calculating the Hessian matrix and the gradient vector of the energy function and iteratively recalculating the Hessian matrix and the gradient vector of the energy function until an updated guess is within an acceptable increment from a previous updated guess. The step of smoothing the template image to reduce noise effects may also be included. The step of minimizing an energy function formed by weighting a sum of modified optical flow constraints at locations of the feature points may further include the step of incorporating an illumination change factor into the optical flow constraints for accounting for pixel intensity changes due to illumination effects.
A method for computer inspection for determining misalignment between inspection patterns includes the steps of providing a template image comprising blocks of pixels, determining features points on the template image from among blocks of pixels by selecting a pixel location in each block having a largest reliability measure, averaging pixels in an area surrounding each feature point to reduce noise in the template image, providing an input image with an initial shift and rotation guess to the template image, minimizing an energy function f
Fang Ming
Lai Shang-Hong
Au Amelia M.
Miller Martin
Paschburg Donald B.
Siemens Corporate Research Inc.
LandOfFree
Method of localization refinement of pattern images using... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method of localization refinement of pattern images using..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method of localization refinement of pattern images using... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2573592