Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
1999-10-13
2002-10-15
Rao, Andy (Department: 2613)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
C375S240290, C375S240120
Reexamination Certificate
active
06466624
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates generally to digital video and, more particularly, to digital video reconstruction.
2. Discussion of Prior Art
Video processing has evolved with the economic phases of video formats. In early analog video, filtering and delay line manipulation of continuous signals, s
c
(t), typically provided only one-dimensional (“1-D”) processing of a small neighborhood of data along a single scan line. Compression-primarily signal limiting and added component interleaving-included band-limiting, interlace scanning, RGB-to-YUV color-space conversion, subcarrier insertion and vestigial side-band modulation. Enhancement processing stages, such as comb filtering, YUV-to-RGB color-space restoration, were also added for correction of compression effects.
Two-dimensional (“2D”) digital video enabled more precise multiple scan-line processing of discrete signals, s(t). Among other advantages, single-image compression techniques, such as the Joint Photographic Experts Group standard (“JPEG”), could now be used to provide digital video image reproduction without perceivable artifacts or “transparent” digital coding. Newer enhancement processing stages, such as time base correction (“TBC”), 2-D comb filtering, edge enhancement and noise reduction, were also enabled.
Recently, 3-dimensional or “3-D” video processing (i.e. of horizontal, vertical and temporal image aspects) has emerged, most notably, Moving Pictures Experts Group or “MPEG” standards. MPEG-1, for example, introduced block-based motion compensated prediction (“MCPn”), which describes the interframe movement of blocks displaced from arbitrary locations. Using MCPn, rudimentary groups of pictures or “GOPs” are formed in which a higher-bitrate “intra-coded” or “I” frame/macroblock can be followed by lower-bitrate differentially-coded predictive and bi-directional or “P” & “B” frames/macroblocks (e.g. IBBPBB). Advantageously, differential-coding typically provides a three-fold compression improvement over still-frame digital image coding. Further, such 3-D coding techniques as synthetic coding (e.g. MPEG-4) are expected to provide even greater compression through more advanced motion models than those used according to current block-based coding.
Despite such advances, however, traditional processing approaches continue to be utilized. For example, while coding, decoding and enhancement processing are typically included within matched encoder-decoder pairs or “codecs,” such processing continues to be conducted as separate and distinct processing stages. One likely reason is that the predominant video codec standards MPEG and its progeny define the generic standard-compliant decoder as one that uses proscribed rules and algorithms or “semantics” that react to coded bitstream elements to provide a one-to-mapping from the input bitstream into an expected output sequence of samples; using such standards, the resulting uncompressed video signals resemble analog signals closely enough that traditional post processing enhancement methods can be readily applied. Another possible reason, among others, is that the conversion of the intermediate decoder output stream into a display format is usually defined by a separate application specification, such as ATSC, DVD or DVB and their progeny.
As shown in
FIG. 1
, for example, a conventional MPEG encoder
101
typically comprises separate processing stages for pre-processing
111
, coding
113
and (optionally) multiplexing
115
a received video source; complimentarily, an MPEG decoder
103
includes stages for de-multiplexing
131
received standard-coded data, decoding
133
the de-multiplexed data, and then post-processing
135
the resulting decoded data samples. Preprocessing stages typically provide for artifact reduction (e.g. noise filtering, time-base correction, etc.) and codec accommodation (e.g. anti-alias low-pass filtering; entropy minimization filtering, downsampling, etc.). Post-processing stages, which conventionally typically provide for codec accommodation (e.g. de-interlacing) and display format conversion, but can also enable image improvement.
Unfortunately, such traditional approaches are capable of only limited image improvement. To make matters worse, conventional approaches require substantial estimation, iteration and computation, which need is exacerbated by real-time operation required for continuous video display. De-interlacing, for example, aims to convert an interlaced signal for progressive display. However, while an interlace signal might contain some progressive content (e.g. 3:2 pulldown film) or interlace coding (e.g. MPEG interlace DCT and field prediction tools), the challenge of de-interlacing remains that of using decoded samples to estimate what the decoded image content would have been if it had been progressively scanned. To make matters even more difficult, the most effective estimation technique potentially useable by conventional codecs, for tracking decoded objects across several frame periods and then filtering along those points, is very computationally expensive.
Other feature enhancements are similarly limited by traditional processing approaches. For example, conventional frame rate conversion uses repeated frames to increase display rate, and frame interpolation to improve object motion smoothness; however, conventional frame interpolation suffers from object tracking requirements as with de-interlacing. Motion blur reduction can also be used to recover some detail lost to object motion during camera integration; however, detail needed by the “inverse blurring algorithm” is likely lost through compression and decoding, and only minor improvement can be achieved by fusing information across frame periods of decoded sample data. Feature enhancement can further be used to emphasize detail that is otherwise below the human visible threshold. However, sub-threshold emphasis is often at odds with conventional encoder filtering-out of imperceptible image attributes, and conventional high-pass filtering of decoded data samples is capable of providing only limited feature enhancement and can actually increase the visibility of compression artifacts.
In emerging “superresolution” techniques, an attempt is made to provide for image restoration and enhancement using “enhancement-facilitating” information found to exist within decoded data samples. For example, bitstream vectors are used in an attempt to link areas in the original reference picture (i.e. prior to quantization) which most closely resemble the current picture. Typically, each vector is refined to half- pixel accuracy by comparing the original current macroblock against the decoded reference picture. Each final selected vector then forms a prediction address from the decoded reference picture. Further accuracy for the current macroblock is also attempted by adding the DCT-coded prediction error to the prediction formed in an earlier motion compensated prediction or “MCP” stage. One model of video restoration theory, for example, describes the observed signal, g, as the original signal, s, convoluted by the point spread function distortion (“PSF-distortion”) D plus the noise, v, as given by the following equation 1:
g=Ds+v
[Equation 1]:
Unfortunately, the existence of enhancement information in decoded image samples is only a fortuitous byproduct of pre-encoder processing, preprocessing, coding and decoding, and conventional superresolution has not yet been proven viable using real-world encoded (i.e. and then decoded) video. Thus, while some enhancement capability has been demonstrated in controlled contexts, conventional superresolution, as with other conventional techniques, is found to be computationally expensive and unreliable. Worse yet, the inconsistent intra-frame and temporal enhancement produced by such methods are often obvious and distracting to a viewer, such that the results produced might be even more detrimental than without such enhancement.
Accordingly, there is a
Pixonics LLC
Rao Andy
Wells St. John P.S.
LandOfFree
Video decoder with bit stream based enhancements does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Video decoder with bit stream based enhancements, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Video decoder with bit stream based enhancements will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2999321