Television – Image signal processing circuitry specific to television – Noise or undesired signal reduction
Reexamination Certificate
1998-07-29
2003-06-24
Kostak, Victor R. (Department: 2611)
Television
Image signal processing circuitry specific to television
Noise or undesired signal reduction
C348S607000
Reexamination Certificate
active
06583823
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a method and an apparatus for removing a scratch, an unnecessary element, and noise from a digitized motion picture to repair the motion picture.
2. Description of the Related Art
An old movie film usually has much dust and a lot of scratches. When such a film is electronically read through a scanner and converted to a digital picture, it is necessary to electronically remove dust and scratches. Since what the original image should be cannot be found, pixels at the dust and scratches have no information. It is necessary to repair the image according to the information of surrounding pixels.
In a recent movie, a special effect is employed in many cases in which a scene is taken with an actor or a vehicle flying in the air by the use of wire and the wire is electronically deleted afterwards. Also in such cases, since portions hidden by the wire, which should have been seen, cannot be repaired from the taken film, it is necessary to reproduce them. This can be handled as a case for repairing a pixel which lacks its information.
In addition, for an image in which an error occurs during radio transfer or computer-network transfer, it is required to use a repair technique to make a defect caused by the error inconspicuous.
To retouch an image, the most traditional and the most widely used method is retouching by hands. Software (called paint software) which implements on a computer an operation for drawing a picture on paper with a pen or an air brush is used to remove a scratch in an image by drawing a figure on the image by the use of an electronic pen or an electronic brush.
It is of course a hard job to repair one frame completely. It is further difficult to repair a motion picture. In other words, even the same pixels in frames are not uniformly retouched since hand retouching is used, and as a result, if each frame is acceptably retouched when seen independently, a human eye notices retouching discontinuity in the time domain as noise when the motion picture formed of the frames is continuously seen. Removing this noise is extremely difficult. It is needless to say that this work requires time and skill. It is not rare that retouching a motion picture formed of about 100 frames takes about one week, and this is a large factor to reduce time efficiency in video operations in a movie or a video product.
An attempt has been made to automate image repair with the use of computer image processing in order to make the work efficiently.
A typical case is the use of an image processing filter. With the use of a low-pass filter (low-frequency passing filter), for example, portions which lack information are blurred to fit the surroundings. Repair is possible to some extent. Just with a low-pass filter, the entire image become blurred. To improve this point, many filters have been examined, including a median filter, which uses statistical operations. An image-repair technique using such a filter is described in detail, for example, in “Fundamentals of digital image processing” written by A. K. Jain, Prentice hall, which is one of main books in image processing. These filters are mainly useful for a lack of information at a small, discrete pixel, but are not suited to repair a large area formed of continuous pixels.
Therefore, these filters are not sufficient for removing a scratch in an image, for repairing the background after an unnecessary element such as wire is removed, or for repair of burst errors during image transfer.
A scratch in a digital image means unnecessary pixels in the image. Scratch noise is formed of a number of adjacent pixels rather than a small independent group of pixels. In an image, scratch noise is formed of, for example, a pixel area of 10 pixels by 100 pixels, and is positioned in an image area having details such as a texture or an edge cut by the scratch. To remove the scratch from the image, the pixels constituting the scratch area need to represent the original data or to be replaced with data equivalent to the original data. To retouch the scratch area effectively, the pixel data needs to generate at the scratch area an image as sharp as those at the surroundings. It is required that a clear edge be maintained to be continuous and a texture generated to replace the wire or the scratch pixels be harmonized with the surrounding texture.
The effect of a conventional technique in digital processing changes according to an image feature generated regularly or at random and the relative size of the feature in the image. A feature generated regularly includes a texture in a brick wall or a cloth. A feature generated at random includes an asphalt road, a concrete road or sandy beach. The effect of the conventional technique also depends on the size and the type of a scratch area. A small, isolated group of noise pixels in a smooth or blurred area in an image can be relatively easily removed by the use of conventional techniques, such as filtering, cloning, and painting. The conventional noise-removing techniques, however, cannot be applied to a scratch formed of a number of adjacent pixels, or a scratch at a texture area or at an area having conspicuous edge or line in an image. The conventional techniques in image noise removal can be classified into two types, (1) intra-frame technique and (2) inter-frame technique. The difference between these two general techniques is mainly where data which substitutes for a noise pixel is obtained. In the inter-frame technique, a pixel required for substituting for a noise pixel is generally copied from a previous frame or continuous frames. On the other hand, in the intra-frame technique, data in an image frame to be repaired is generally used for substituting for a noise pixel.
The inter-frame technique cannot achieve a good result in a picture in which a camera moves extremely or in a scene having a movement. These techniques cannot be applied either to a case in which a scratch extends to several image frames, or to a case in which a damaged image frame is the only one image frame for providing data used for repairing the image. Unfortunately, a scratch generally extends to a number of image frames due to the movement of a movie film in a general projector. In such a case, the corresponding image data obtained from the previous image frame or continuous image frames cannot be used for substituting for a scratch area or for copying. To remove scratch noise, the conventional techniques provide some methods: (a) low-pass filtering and other linear filtering, (b) median filtering and other non-linear filtering, (c) statistical combination of textures, (d) cloning, including copying other parts in an image, (e) painting by hand, (f) method based on projection, and (g) method based on solving a simultaneous equation. With these methods, the best result cannot be obtained in removing various scratches or in removing a scratch in various image conditions.
It is considered that all image signals are formed of a combination at a certain ratio of many sine-wave signals having different frequencies. To analyze such an image as image components, fast Fourier transform (FFT) is used. Processing for the information generated by FFT is performed in the frequency domain. Direct processing for an image signal is performed in the spatial domain.
The methods (a) to (e) are used in either one of the domains. In methods used in the frequency domain, the structure of the entire image can be obtained, but local processing control (line continuity and clearness) is lost. As a result, a line and other details become unclear.
On the other hand, in methods used only in the spatial domain, local processing control and local information are obtained, but information on the structure of the entire image cannot be obtained. A limitation in a local vicinity is caused by a restriction in actual computer processing in some cases. Some methods similar to intermediate filtering are not capable of using the entire information effectively.
In a conventional image repai
Shimada Shigehiro
Totsuka Takashi
Bell Boyd & Lloyd LLC
Kostak Victor R.
Sony Corporation
LandOfFree
Methods, apparatuses, and mediums for repairing a pixel... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Methods, apparatuses, and mediums for repairing a pixel..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Methods, apparatuses, and mediums for repairing a pixel... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3157573