Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
1999-11-05
2002-08-27
Le, Vu (Department: 2613)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
C375S240170, C348S699000
Reexamination Certificate
active
06442203
ABSTRACT:
TECHNICAL FIELD
This invention relates to processing of digitally encoded video and movie signals, and more particularly to a system and method for de-interlacing, motion compensation and/or frame rate conversion of digitally encoded video and movie signals.
BACKGROUND
Frame-Rate and Interlace
At the present time, the world's “standard definition” and “high definition” television systems have parameters which are relatively incompatible internationally. Even within the U.S., the Advanced Television Systems Committee (“ATSC”) is proposing a variety of formats which are relatively incompatible with each other, as well as incompatible with other international standards. Of all of the parameters of television systems, the most problematic and incompatible are frame-rate and interlace.
Most video camera and film images are captured with a single picture output rate. The common output rates are 24 frames-per-second (fps) for film, 25 fps (film in Europe for TV), 50 Hz interlaced, and 60 Hz interlaced. It would also be desirable to have 72 Hz and/or 75 Hz display rates in order to eliminate flicker on CRT's and other flicker-type display devices. (Computer displays most commonly use 75 Hz display to eliminate flicker). A 60 Hz display rate (U.S. and Japan NTSC TV) and 50 Hz display (European PAL and SECAM TV) have substantial flicker, which becomes intolerable on large bright screens.
One approach to resolving the problem of multiple incompatible frame rates is disclosed in U.S. Pat. No. 5,737,027, entitled PIXEL INTERLACING APPARATUS AND METHOD, assigned to the assignee of the present invention (hereby incorporated by reference). That system used a special camera pixel pattern to generate multiple frame rates, which are otherwise incompatible, from a common signal using a “Pixelace” technique. The “Pixelace” technique uses sub-groups at a high Least Common Multiple (LCM) frame rate of all desired output frame rates in order to allow output at all of the otherwise incompatible frame rates. However, high frame rates cameras are not yet available which can perform at 1800 fps, which is the LCM of the rates of 24, 25, 30, 50, 60, 72, and 75 fps. Thus, while this system is indeed a solution to the frame rate problem, it requires custom cameras which generate pixels in the “Pixelace” format.
Interlace—the sequential display of a field of even raster lines and a field of odd raster lines to make a single frame—makes any form of video conversion difficult. Thus, re-sizing, speed adjustment, frame rate conversion, or resolution change all become very difficult, and the converted results are usually poor in quality.
For a decade or two, “standards converters” have been offered to convert between 50 Hz interlaced PAL and 60 Hz/59.94 Hz interlaced NTSC. These standards converters have been used for some live international sports events coverage such as the Olympics. Such converters often provide poor results, such as both a soft blurry image as well as peculiar artifacts (such as gymnasts with three legs and four arms during their transient acrobatics).
Some of the artifacts from frame rate conversion are theoretically incapable of being properly detected or repaired. Both interlace and standard image frame capture leave “holes” (i.e., no video information) in their observation of a subject image over time. In particular, interlaced fields have holes between the odd or even scanlines. Thus, for example, small horizontal objects can actually be present in a scene but fall between the unobserved gaps between the scanlines, and thus not appear as part of the video information of any field or frame.
Frame capture on film and video has a duration when a scene is being observed, but also has a time, when the shutter is closed, when there is no observation. This occurs in film because of the need to close the shutter in order to advance to the next frame of film. This occurs in video cameras in order to allow time for the sensor (usually a CCD circuit) to pass the image electrons to the readout electronics. A “short shutter” is also sometimes used to reduce blur in some types of scene, where the amount of time the shutter is closed is manually increased during the capture of a particular scene. For film, the largest duty cycle of open shutter is usually 205 degrees out of 360 degrees for a rotary shutter (57% duty cycle). For CCD sensors, the largest duty cycle is about 80%, depending upon the particular sensor and electronic shutter.
FIG. 1
shows an example of a temporal (time) sampling filter for film and CCD cameras. When the shutter is closed (e.g., between the end of Frame n-1 and the beginning of Frame n), no image information is being recorded.
A correct temporal filter cannot be achieved even by a 100% duty cycle (which is a box filter, still subject to some types of aliasing), but would require a time sample for each frame which extended well into the time of previous and subsequent frames. The problem of “not looking” during some of the frame (this is known as “temporal undersampling”), as well as the “box filter” shape of a “constant look” during the shutter-open time, results in theoretically incorrect time filters. This leads to unavoidable “temporal aliasing”.
In particular, during the time a shutter is closed, crucial information may occur which is not observed. For example, if at frame “n” a football is to the right of a goalpost, and at frame “n+1” the football is to the left of the goalpost, the crucial information about whether the field-goal was good or not is missing because the shutter was closed during the time the football was passing by the goalpost.
A more optimal temporal sampling pattern would modulate the sensor's sensitivity over time using a function which extends well into neighboring frames. This is not possible with existing 3-CCD sensor cameras (or inexpensive single CCD cameras). Overlap in time implies that multiple CCD's for each color must be used. In addition, current CCD's and their on-off shutters do not allow modulation of sensitivity over time, and would need to be modified to support such sensing patterns.
FIG. 2
shows an example of the theoretical shutter characteristics that would result in such a more optimal temporal sampling filter.
It is worthy of note that the Pixelace technique cited above allows such modulated temporal sampling filters to be simulated, by applying scale factors to pixel values within pixel plates based upon their temporal relationship to the filter center time. Further, pixel plates can be applied to construct multiple frames, thereby supporting the overlap necessary for more optimal filters. However, care must be taken to “normalize” the pixel values based upon pixel plate overlap and temporal filter function position. Longer frame times (such as with 24 fps) allow more accurate construction of the filter shape using Pixelace, since more pixel groups are available at the LCM rate to support more data points within the filter shape.
In the absence of new sensor structures, high speed CCDs, or Pixelace compatible cameras, conventional CCD cameras and motion picture film cameras will produce frame (or interlaced field) samples which have inherent temporal undersampling and aliasing. The aliasing will result in artifacts, such as backward-rotating wagon wheels. Aliasing due to undersampling and use of a box filter also make it difficult to de-interlace or make frame rate conversions. Artifacts which occur from such aliasing are harmonically related to the frame rate conversion relationships. For example, a factor of two or three increase or decrease in frame rate (such as 48 Hz or 72 Hz display of 24 fps movies) is better than non integral relationships (such as 3-2 pulldown for 60 Hz display of 24 fps movies).
U.S. Pat. No. 5,852,565, entitled TEMPORAL AND RESOLUTION LAYERING IN ADVANCED TELEVISION (assigned to the assignee of the present invention and hereby incorporated by reference), teaches that some of the frame-rate and resolution incompatibilities may be handled by restricting frame rate capt
DemoGraFX
Fish & Richardson P.C.
Le Vu
LandOfFree
System and method for motion compensation and frame rate... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for motion compensation and frame rate..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for motion compensation and frame rate... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2936813