Moving picture encoder and moving picture decoder

Details Moving picture encoder and moving picture decoder Moving picture encoder and moving picture decoder

1999-02-17

2004-02-17

Le, Vu (Department: 2613)

Pulse or digital communications

Bandwidth reduction or expansion

Television or motion video signal

C375S240100, C375S240160

Reexamination Certificate

active

06693960

ABSTRACT:

TECHNICAL FIELD
The present invention relates to technical field of a digital image processing and especially relates to a moving image encoding apparatus for efficiently encoding image data, and a moving image decoding apparatus for decoding the encoded data produced by the moving image encoding apparatus.
BACKGROUND ART
In the image encoding, a method of synthesizing different moving image sequences has been studied.
In a literature, “Image Encoding Using a Hierarchical Expression and Multiple Templates” (Shingaku Giho IE94-159, pp. 99-106 (1995)), there is described a method in which a moving image sequence as a background and a moving image sequence of a partial moving image as a foreground (for example, human image, picture of fish and the like cut down by a Chroma key technique) are superimposed to prepare a new sequence.
In addition, in a literature, “Temporal Scalability based on Image Content”, (ISO/IEC/JTC1/SC29/WG11 MPEG95/211 (1995)), there is described a method in which a moving image sequence of a partial moving image having a high frame rate is superimposed on a moving image sequence having a low frame rate to prepare a new sequence.
With this method, as shown in
FIG. 15
, prediction-encoding is performed at a low frame rate in a lower layer, and prediction-encoding is performed at a high frame rate only for a selected area (dotted portion) in an upper layer. In this case, an image frame decoded up to the upper layer is obtained by superimposing an image frame decoded by the lower layer and an area decoded by the upper layer. Moreover, a frame encoded by the lower layer is not encoded in the upper layer, and the decoded image of the lower layer is directly copied. In addition, it is assumed that a portion which attracts the audience's attention, such as a personality portion is selected as the selected area.
FIG. 11
shows a block diagram of the conventional art. On the encoding side in the conventional art, the input moving image layer is thinned out between frames by a first thinning-out section
1101
and a second thinning-out section
1102
, and made to be the frame rate or less of the input image, then input to an upper layer encoding section and a lower layer encoding section. Here, the frame rate in the upper layer is assumed to be higher than the frame rate of the lower layer.
In the lower layer encoding section
1104
, the entire input moving image is encoded. As the encoding method, an international standard method for encoding moving images, for example, MPEG or H. 261 is used. In the lower layer encoding section
1104
, decoded image of the lower layer is prepared, and input to a superimposing section
1105
upon being utilized for prediction-encoding.
In the upper layer encoding section
1103
, only a selected area of the input moving image is encoded. Here, the international standard method for encoding moving images such as MPEG and H.261 is again used, but only the selected area is encoded based on the area information. However, the frame encoded in the lower layer is not encoded in the upper layer. The area information is the information showing the selected area of, for example, a personality portion, and is a binarized image which takes value 1 at a position in the selected area and takes value 0 at a position other than the selected area. Also in the upper layer encoding section
1103
, only the selected area of the moving image is decoded, and input to the superimposing section
1105
.
In an area information encoding section
1106
, the area information is encoded by utilizing a chain code or the like.
The superimposing section
1105
outputs a decoded image of the lower layer, when the lower layer frame has been encoded in the frame to be superimposed. When the lower layer frame has not been encoded in the frame to be superimposed, the superimposing section
1105
outputs a moving image by using two decoded image of the lower layer before and behind the frame to be superimposed and one decoded image of the upper layer. The two image frames of the lower layer are before and behind the upper layer frame. The moving image prepared here is input to the lower layer encoding section
1103
and utilized for the prediction-encoding. The image forming method in the superimposing section
1105
is as described below.
First, two interpolated images of the lower layer are prepared. If it is assumed that the decoded image of the lower layer at time “t” is B(x, y, t)(provided that x and y are coordinates representing a position of a pixel in the space), and that the time of the two frames of the lower layer are t1 and t2, respectively, and the time of the upper layer is t3 (provided that t1<t3<t2), the interpolated image at time t3 (x, y, t3) can be calculated by the following expression (1):
I
(
x, y, t
3)=[(
t
2−
t
3)B(
x, y, t
1)+(
t
3−
t
1)B(
x, y, t
2)]/(
t
2−
t
1)  (1)
Then, a decoded image E of the upper layer is superimposed on the interpolated image I determined by the above expression (1). For this purpose, weight information W (x, y, t) for interpolation is prepared from the area information M (x, y, t), to obtain a superimposed image S by the following expression (2):
S
(
x, y, t
)=[1
−W
(
x, y, t
)]
I
(
x, y, t
)+
E
(
x, y, t
)
W
(
x, y, t
)  (2)
Here, the area information M (x, y, t) is a binarized image which takes 1 within the selected area and takes 0 outside the selected area, and by applying a low-pass filter to this image for plural times, the weight information W (x, y, t) can be obtained. That is to say, the weight information W (x, y, t) takes 1 within the selected area, takes 0 outside the selected area, and takes 0 to 1 in the boundary of the selected area. The above-mentioned description is for the image forming method in the superimposing section
1105
. The encoded data encoded in the lower layer encoding section, the upper layer encoding section, and the area information encoding section is integrated in an encoded data-integrating section (not shown) and transmitted or accumulated.
Then, on the decoding side in the conventional art, the encoded data is disintegrated into an encoded data of the lower layer, an encoded data of the upper layer and an encoded data of the area information by an encoded data-disintegrating section (not shown). These encoded data is decoded by a lower layer decoding section
1108
, an upper layer decoding section
1107
and an area information decoding section
1109
, as shown in FIG.
11
.
A superimposing section
1110
on the decoding side comprises the same apparatus as that of the superimposing section
1105
on the encoding side, and an image is superimposed in the same method as described in the description on the encoding side, using a lower-layer decoded image and an upper-layer decoded image. The moving image superimposed here displayed on a display, as well as being input to the upper layer decoding section
1107
, and utilized for the prediction of the upper layer. Though a decoding apparatus for decoding both the lower layer and the upper layer has been described here, if it is a decoding apparatus having only a decoding section of the lower layer, the upper layer encoding section
1107
and the superimposing section
1110
are not required, hence a part of the encoded data can be reproduced with a small hardware scale.
At this time, since the frame rates of the lower layer and the upper layer are different, it is necessary to synthesize the lower layer corresponding to the upper layer from the lower layer frames temporally before and after of the lower layer frame. However, when an output image is obtained from two lower-layer decoded images and one upper-layer decoded image, the output image is synthesized by the interpolation of two lower layer frames, therefore when the position of the selected area changes with time, a big distortion is caused in the periphery of the selected area, resulting in a big deterioration of the image quality.
This problem can be solved by using a method su

Affiliated with

Also associated with

Rating

Moving picture encoder and moving picture decoder does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Moving picture encoder and moving picture decoder, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Moving picture encoder and moving picture decoder will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3329288