Television – Format conversion – Involving both line number and field rate conversion
Reexamination Certificate
1999-04-16
2002-06-18
Lee, Michael H. (Department: 2614)
Television
Format conversion
Involving both line number and field rate conversion
C348S445000, C348S459000
Reexamination Certificate
active
06407775
ABSTRACT:
BACKGROUND
Images captured on film commonly are optically compressed in the horizontal direction through the use of anamorphic lenses. When the film is played back using a film projector, the compression is optically reversed to restore the proper aspect ratio to the captured images. This process of optical horizontal compression and decompression allows the capture of a wide screen image on standard 35 millimeter or 70 millimeter film.
Sometimes film is transformed into images in a video signal or a sequence of digital still images for display on electronic displays, such as television monitors or computer monitors. Such images generally have one of two formats: progressive or interlaced. Progressive images are defined by a series of scan lines that generally are displayed at approximately the same time. In interlaced images, these scan lines are grouped into sets of odd lines and even lines that are displayed in an alternating manner. Television signals commonly are captured in an interlaced format. Images that are electronically displayed also typically have a different image rate from film. Rate conversion is accomplished through a process called pulldown and/or the use of an interlaced image format. If an anamorphic image is displayed on an electronic display, if no processing steps are taken the image remains horizontally compressed. To display anamorphic images in a proper aspect ratio on an electronic display, several processes generally are used to resize images. Example resizing operations include “pan and scan,” “zoom,” “tilt,” “squeeze,” or “letterboxing.”
Pan and scan involves selecting an area of each image. The size of the selected area is chosen to account for the horizontal compression of the original image and the size of the image to be displayed. The selected area generally moves horizontally from image to image. The selected area is resized horizontally, and sometimes vertically, to fill the display. Pan and scan may involve either expansion or reduction of the size of the image. Tilt, sometimes referred to as pan and tilt, also involves selecting an area of an image, but the area generally moves vertically from image to image. Zoom generally involves selecting any area of an image and expanding or reducing it in size. Letterboxing involves resizing an anamorphic image, i.e., by reducing its size vertically, to restore the proper aspect ratio to the image, resulting in blank, e.g., black, areas in the display above and below the displayed image. Squeeze is a similar to letterboxing, but is a horizontal reduction or expansion in which blank areas may appear to the left and/or right of the image.
Reduction or expansion of an image vertically generally involves averaging or filtering two or more adjacent lines in order to produce a different number of lines. Similarly, reduction or expansion of an image horizontally generally involves averaging or filtering two or more adjacent pixels in a line to produce a different number of pixels.
If a field of an interlaced image is resized, because each field has only either even lines or odd lines, the lines next to each other in the field are not next to each other in the image frame. Therefore, resizing causes information from non-adjacent lines to be combined, resulting in errors in the resized image. If two fields of an image that was originally captured in an interlaced format are combined to form a full frame before resizing, an object in the image may be moving between two fields, resulting in a different kind of error in the resized image.
Images on film may be transferred to an interlaced signal format, digitized and recombined to create a progressive image without errors. Resizing of the progressive image avoids errors otherwise associated with resizing interlaced images. However, pan and scan or pan and tilt operations tend to have errors similar to twitter or judder if the resized progressive images are subsequently converted to interlaced images with pulldown. If resizing and pulldown are both performed on fields of interlaced images, in the time domain of the interlaced signal, pan and scan motion is optimized, but resizing of the fields introduces artifacts.
SUMMARY
To minimize errors, pan and scan, letterboxing or other resizing operations are performed on progressive scan images or temporally coherent fields, with the resizing parameters defined at the image rate of the output images. In particular, changes in the area of an image that is selected are made in increments that match the temporal resolution and ordering of the output images. Thus, if a resizing operation outputs images according to a specified sequence, errors in the output images are reduced, particularly if the output images are interlaced and result from application of a pulldown sequence. Although the resize and pulldown operations may be considered to occur in separate time domains, the resizing and pull-down operations may be performed simultaneously.
In one embodiment, temporally coherent interlaced images, e.g., created by scanning film, are stored in a memory at a first rate, e.g., 48 fields per second. Output images are generated at the rate of the electronic display, e.g., 59.97 or 50 fields per second, following any pull-down sequence or other specified image ordering. For each output image to be generated, lines from both fields of a frame are read from memory. Therefore, all of the data from a frame is available to produce a high quality resized image from which the output image may be generated. Because all of the lines of the input image are available to the resizer, a simple interpolation filter can be used to perform the resizing operation and produce high quality output images.
The technique may be used in many applications. Example applications include converting images digitized from film to a format suitable for display on either NTSC or PAL standard monitors. Additional applications include, but are not limited to, conversion of PAL video to NTSC video, conversion of non-interlaced digital television formats to interlaced formats, and conversion between high definition television and other digital television formats.
Accordingly, an apparatus, or method or computer program product for resizing images receives pixel data of a sequence of input images having a first temporal resolution. Each of the received input images is resized to produce an output image using both odd and even lines from the input image. A sequence of the output images is provided at a second temporal resolution. The input images may be defined by temporally coherent interlaced fields or progressive scan images. The second temporal resolution may be different from the first temporal resolution. Resizing may use an area within the input image to produce the output image. The position of the area within the input image may vary over time. The position of the area within the input image may be determined at the second temporal resolution.
REFERENCES:
patent: 5500678 (1996-03-01), Puri
patent: 5754248 (1998-05-01), Faroudja
patent: 5796442 (1998-08-01), Gove et al.
patent: 5999220 (1999-12-01), Washino
patent: 6072531 (2000-06-01), Shibano
patent: 6151074 (2000-11-01), Werner
patent: 6151075 (2000-11-01), Shin et al.
patent: 6256069 (2001-07-01), Rosen et al.
patent: WO9515659 (1995-06-01), None
patent: WO9739586 (1997-10-01), None
patent: WO9846016 (1998-10-01), None
John Amanatides and Don P. Mitchell “Antialiasing of Interlaced Video Animation” AT&T Bell Labs, Computer Graphics vol. 24, No. 4, Aug. 1990.
Cacciatore Raymond D.
Frink Craig R.
Avid Technology Inc.
Gordon Peter J.
Lee Michael H.
LandOfFree
Image resizer and frame rate converter with pulldown controller does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image resizer and frame rate converter with pulldown controller, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image resizer and frame rate converter with pulldown controller will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2945807