Television – Format conversion
Reexamination Certificate
1998-12-22
2001-11-27
Lee, Michael (Department: 2614)
Television
Format conversion
C348S458000, C348S581000, C375S240020, C382S224000
Reexamination Certificate
active
06323905
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to a picture conversion apparatus and a picture conversion method. In particular, the invention relates to a picture conversion apparatus and a picture conversion method which makes it possible to obtain a picture having better picture quality.
In converting a standard-resolution or low-resolution picture (hereinafter referred to as an SD (standard definition) picture where appropriate) into a high-resolution picture (hereinafter-referred to as an HD (high definition) picture where appropriate), or in enlarging a picture, pixel values of absent pixels are interpolated (compensated for) by using what is called an interpolation filter or the like.
However, even if pixels are interpolated by using an interpolation filter, it is difficult to obtain a high-resolution picture because HD picture components (high-frequency components) that are not included in an SD picture cannot be restored.
In view of the above, the present applicant previously proposed a picture conversion apparatus which converts an SD picture into an HD picture including high-frequency components that are not included in the SD picture.
In this picture conversion apparatus, high-frequency components that are not included in an SD picture are restored by executing an adaptive process for determining a prediction value of a pixel of an HD picture by a linear combination of the SD picture and predetermined prediction coefficients.
Specifically, for instance, consider the case of determining a prediction value E[y] of a pixel value y of a pixel (hereinafter referred to as an HD pixel where appropriate) constituting an HD picture by using a linear first-order combination model that is prescribed by linear combinations of pixel values (hereinafter referred to as learning data where appropriate) x
1
, x
2
, . . . of a certain number of SD pixels (pixels constituting an SD picture) and predetermined prediction coefficients w
1
, w
2
, . . . . In this case, a prediction value E[y] can be expressed by the following formula.
E[y]=w
1
x
1
+w
2
x
2
+. . . (1)
For generalization, a matrix W that is a set of prediction coefficients w, a matrix X that is a set of learning data, and a matrix Y′ that is a set of prediction values E[y] are defined as follows:
x
=
(
x
11
x
12
⋯
x
1
n
x
21
x
11
⋯
x
2
n
⋯
⋯
⋯
⋯
x
m1
x
m2
⋯
x
mn
)
W
=
(
W
1
W
2
⋯
W
N
)
,
Y
′
=
(
E
[
y
1
]
E
[
y
2
]
⋯
E
[
y
m
]
)
(2)
The following observation equation holds:
XW=Y′
(3)
Consider the case of determining prediction values E[y] that are close to pixel values y of HD pixels by applying a least squared method to this observation equation. In this case, a matrix Y that is a set of true pixel values y of HD pixels as teacher data and a matrix E that is a set of residuals e of prediction values E[y] with respect to the pixel values y of the HD pixels are defined as follows:
E
=
(
e
1
e
2
⋯
e
m
)
,
Y
′
=
(
y
1
y
2
⋯
y
m
)
(4)
From Formula (3), the following residual equation holds:
XW=Y+E
(5)
In this case, prediction coefficients w
i
for determining prediction values E[y] that are close to the pixel values y of the HD pixels are determined by minimizing the following squared error:
∑
i
=
1
m
e
i
2
(6)
Therefore, prediction coefficients w
i
that satisfy the following equations (derivatives of the above squared error with respect to the prediction coefficients w
i
are 0) are optimum values for determining prediction values E[y] close to the pixel values y of the HD pixels.
e
1
∂
e
1
∂
w
i
+
e
2
∂
e
2
∂
w
i
+
…
+
e
m
∂
e
m
∂
w
i
=
0
(
i
=
1
,
2
,
…
,
n
)
(7)
In view of the above, first, the following equations are obtained by differentiating Formula (5) with respect to prediction coefficients w
i
.
∂
e
i
∂
w
1
=
x
i1
,
∂
e
i
∂
w
2
=
x
i2
,
…
,
∂
e
i
∂
w
n
=
x
m
,
(
i
=
1
,
2
,
…
,
m
)
(8)
Formula (9) is obtained from Formula (7) and (8).
∑
i
=
1
m
e
i
x
i1
=
0
,
∑
i
=
1
m
e
i
x
i2
=
0
,
…
∑
i
=
1
m
e
i
x
in
=
0
(9)
By considering the relationship between the learning data x, the prediction coefficients w, the teacher data y, and the residuals e in the residual equation of Formula (5), the following normal equations can be obtained from Formula (9):
{
(
∑
i
=
1
m
x
i1
x
i1
)
w
1
+
(
∑
i
=
1
m
x
i1
x
i2
)
w
2
+
…
+
(
∑
i
=
1
m
x
i1
x
in
)
w
n
=
∑
i
=
1
m
x
i1
y
i
)
(
∑
i
=
1
m
x
i2
x
i1
)
w
1
+
(
∑
i
=
1
m
x
i2
x
i2
)
w
2
+
…
+
(
∑
i
=
1
m
x
i2
x
in
)
w
n
=
∑
i
=
1
m
x
i2
y
i
)
(
∑
i
=
1
m
x
in
x
i1
)
w
1
+
(
∑
i
=
1
m
xnx
i2
)
w
2
+
…
+
(
∑
i
=
1
m
x
in
x
in
)
w
n
=
∑
i
=
1
m
x
in
y
i
)
}
(10)
The normal equations of Formula (10) can be obtained in the same number as the number of prediction coefficients w to be determined. Therefore, optimum prediction coefficients w can be determined by solving Formula (10) (for Formula (10) to be soluble, the matrix of the coefficients of the prediction coefficients w need to be regular). To solve Formula (10), it is possible to use a sweep-out method (Gauss-Jordan elimination method) or the like.
The adaptive process is a process for determining optimum prediction coefficients w in the above manner and then determining prediction values E[y] that are close to the component signals y according to Formula (1) by using the optimum prediction values w (the adaptive process includes a case of determining prediction coefficients w in advance and determining prediction values by using the prediction coefficients w).
The adaptive process is different from the interpolation process in that components not included in an SD picture but included in an HD picture are reproduced. That is, the adaptive process appears the same as the interpolation process using an interpolation filter, for instance, as long as only Formula (1) is concerned. However, the adaptive process can reproduce components in an HD picture because prediction coefficients w corresponding to tap coefficients of the interpolation filter are determined by what is called learning by using teacher-data y. That is, a high-resolution picture can be obtained easily. From this fact, it can be said that the adaptive process is a process having a function of creating resolution of a picture.
FIG. 9
is an example of a configuration of a picture conversion apparatus which converts an SD picture as a digital signal into an HD picture.
An SD picture is supplied to a delay line
107
, and blocking circuits
1
and
2
. The SD picture is delayed by, for instance, one frame by the delay line
107
and then supplied to the blocking circuits
1
and
2
. Therefore, the blocking circuits
1
and
2
are supplied with an SD picture of a current frame (hereinafter referred to as a subject frame where appropriate) as a subject of conversion into an HD picture and an SD picture of a 1-frame preceding frame (hereinafter referred to as a preceding frame where appropriate).
In the blocking circuit
1
or
2
, HD pixels which constitute an HD picture of the subject frame are sequentially employed as the subject pixel and prediction taps or class taps for the subject pixel are formed from the SD pictures of the subject frame and the preceding frame.
It is assumed here that, for example, HD pixels and SD pixels have a relationship as shown in FIG.
10
. That is, in
Fujiwara Takayoshi
Kondo Tetsujiro
Node Yasunobu
Okumura Yuuji
Desir Jean W.
Frommer William S.
Frommer & Lawrence & Haug LLP
Lee Michael
Shallenburger Joe H.
LandOfFree
Picture conversion apparatus picture conversion method... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Picture conversion apparatus picture conversion method..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Picture conversion apparatus picture conversion method... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2574122