Image analysis – Image enhancement or restoration – Focus measuring or adjusting
Reexamination Certificate
1999-01-22
2002-10-22
Lee, Thomas D. (Department: 2624)
Image analysis
Image enhancement or restoration
Focus measuring or adjusting
C382S236000
Reexamination Certificate
active
06470097
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to image restoration and more particularly to applying total variational regulation to a blind image.
2. Description of the Prior Art
Digital image restoration has received considerable attention in the field of image processing and computer vision during the past two decades. It contains applications in astronomial imaging, X-ray and MRI medical imaging, degraded media (film or videotape) restoration, artifact removal in image coding, law enforcement, etc. The problem of image restoration with known blur functions has been well studied in the past. Inverse filtering and regularized least squares are two of the most popular approaches for this problem. These are further described by M. R. Banham and A. K. Katsaggelos, in “Digital Image Restoration”, IEEE Signal Processing Magazine, Vol. 14, No. 2, pp. 24-41, 1997; and by A. K. Katsaggelos in Digital Image Restoration, Springer-Verlag, New York, 1991. Recently, researchers have focused on the more challenging problem, which is to recover the true image together with the blur function from the degraded image. This problem is called blind image restoration or blind image deconvolution. This problem is more practical since the blur function is usually unknown for most cases. D. Kundur and D. Hatzinakos in “Blind Image deconvolution”, IEEE Signal Processing Magazine, Vol. 13, No.3, pp. 43-64, 1996, gave a review of previous methods, which include methods based on the frequency domain zeros in the PSF (point spread function) and the methods based on representing the blurred image as an ARMA (autoregressive moving average) process. The point spread function is further described by T. G. Stockham Jr., T. M. Cannon, and R. B. Ingebresten in “Blind Deconvolution Through Digital Signal Processing”, Proc. IEEE, Vol. 63, pp. 678-692, 1975; and by R. L. Lagendijk, A. M. Tekalp, and J. Biemond in “Maximum Likelihood Image And Blur Identification”, Optical Engineering, Vol. 29, No.5, pp. 422-435, 1990. The autoregressive moving average is further described by A. M. Tekalp, H. Kaufman, and J. Woods in “Identification Of Image And Blur Parameters For The Restoration Of Noncausal Blurs”, IEEE Trans. Acoust., Speech, Signal Processing, Vol. 34, pp. 963-972, 1986; and by K. T. Lay and A. K. Katsaggelos in “Image Identification And Restoration Based On Expectation-maximization Algorithm”, Optical Engineering, Vol. 29, No. 5, pp. 436-445, 1990. The methods based on the frequency domain zeros are restricted to some specific types of blurs, such as motion blurs or defocused blurs with a circular aperture. In addition, these methods are sensitive to the additive noises. The methods based on the ARMA model are suspensible to some artifact in the restored image due to the artificial assumption that the true image is a two-dimensional autoregressive (AR) process.
More recently, Y.-L. You and M. Kaveh in “A Regularization Approach To Joint Blur Identification And Image Restoration”, IEEE Trans. Image Processing, Vol. 5, No. 3, pp. 416-428, 1996 developed a regularization approach for the blind image restoration problem, which is an ill-posed inverse problem due to the fact that the solution is not unique. Their regularization approach imposes additional smoothness constraint on the true image as well as the blur function, thereby making the problem well posed. This leads to an image-blur coupled function optimization problem. An alternating minimization algorithm with the conjugate gradient iterations was proposed to solve this coupled optimization problem.
You and Kaveh's regularization formulation used an H
1
norm for the smoothness constraint on the image and the blur. This smoothness constraint prohibits discontinuities in the solution. Unfortunately, there usually exist sharp discontinuities in the true images. The desirable details of the true image can be lost due to the smoothed discontinuities. Although You and Kaveh proposed a weighted regularization to alleviate this problem, this weighting scheme is somehow ad hoc and the parameters involved in this scheme may need to be tuned in a case by case basis.
A new regularization approach that employed the total variation (TV) norm in stead of the standard H
1
norm for the image constraint was proposed by L. Rudin, S. Osher, and E. Fatemi in “Nonlinear Total Variation Based Noise Removal Algorithms”, Physica D., Vol. 60, pp. 259-268, 1992 for the image denoising problem. The TV regularization has been proved to be capable of preserving discontinuities while imposing smoothness constraints and it is effective for recovering blocky images. T. F. Chan and C. K. Wong in “Total Variation Blind Deconvolution”, IEEE Trans. Image Processing, Vol. 7, No. 3, pp. 370-375, 1998 modified You and Kaveh's regularization formulation by using the TV norm for the smoothness constraint on the image as well as the blur function instead of the H
1
norm, thus preserving the discontinuities in the recovered image function.
SUMMARY OF THE INVENTION
The present invention provides a new formulation for blind image restoration from an image sequence. Total variational (TV) regularization is employed to allow discontinuities in a true image function. An iterative alternating algorithm using quasi-Newton iterations is provided to solve an image-blur coupled nonlinear optimization problem. This formulation is then extended to the blind image restoration from an image sequence by introducing motion parameters into a multi-frame data constraint.
The input to the blind image restoration system of the present invention contains an image sequence and initial values for image blur and motion parameters. Within an image blur parameters updater, the system first updates the image blur parameters by using Quasi-Newton iterations to minimize the energy function with the motion parameters and restored image, fixed with their current values. After that, the signal flows to a motion parameters updater where the motion parameters between subsequent frames in the image sequence are updated by using Newton iterations to minimize the energy function with the blur parameters and restored image, fixed with their current values. The restored image is then updated in a restored image updater by using a preconditioned conjugate gradient algorithm to minimize the energy function derived from the total variational (TV) regularization formulation. The TV-based energy function is then computed in a TV-based energy function computer by using the currently updated parameter values. Within a converged decider, if the relative difference between the current energy function value and the energy value computed in the previous iteration is within a threshold, then it is converged and the restored image is outputted. If it has not converged, the signal flows back to the image blur parameters updater to update the parameters.
REFERENCES:
patent: 5440647 (1995-08-01), Floyd, Jr. et al.
patent: 5576548 (1996-11-01), Clarke et al.
patent: 5978505 (1999-11-01), Sutton et al.
patent: 6169817 (2001-01-01), Parker et al.
patent: 6317519 (2001-11-01), Dusseux
Cui Yuntao
Lai Shang-Hong
Brinich Stephen
Lee Thomas D.
Paschburg Donald B.
Siemens Corporation Research, Inc.
LandOfFree
Total variational blind image restoration from image sequences does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Total variational blind image restoration from image sequences, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Total variational blind image restoration from image sequences will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2952059