Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
1999-09-17
2003-03-25
Kelley, Chris (Department: 2613)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
C348S575000, C382S232000, C386S349000
Reexamination Certificate
active
06539054
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to an image output apparatus and an image reproduction method. More particularly, the present invention relates to a reproduction process for decoding and compositing encoded video object data corresponding to a plurality of objects composing a predetermined image (scene), to reproduce video data corresponding to the predetermined scene.
The present invention also relates to an object composition apparatus and an object composition method and, more particularly to an object composition process for compositing object data corresponding to video data of respective objects according to auxiliary information relating to a composite image and the respective objects.
Moreover, the present invention relates to a data storage medium which contains a program for implementing the reproduction process by software and a program for implementing the object composition process by software.
BACKGROUND OF THE INVENTION
In recent years, we have greeted the age of “multimedia” which handles audio, video, and other data integrally. Conventional information media such as newspapers, magazines, televisions, telephones, radios, and telephones, have been adopted as the subjects of the multimedia. In general, the multimedia represents graphics, speeches, and especially images, as well as characters in relation with each other. In order to handle the conventional information media as the subject of the multimedia, it is essential that information of the conventional information media be represented in a digital format.
Let's give information of each information medium in terms of the quantity of digital information. For example, characters require information of 1-2 bytes per character, while audio requires information of 64 kbits per second (quality for telecommunication), and a moving image requires information of 100 Mbits or more per second (quality for current television broadcasting). Hence, it is not practical to handle such enormous amount of data as it is in a digital format. For example, although visual telephones have already been put to practical use by means of an ISDN (Integrated Services Digital Network) which accommodates a transmission rate ranging from 64 kbps to 1.5 Mbps, video data of a television camera cannot be directly sent over the ISDN.
Accordingly, there is a demand for an information compression technique. In case of the visual telephones, a moving image compression technique according to H.261 or H.263 standard which is internationally standardized by an ITU-T (International Telecommunication Union Telecommunication Standardization Sector) is employed. Also, according to an information compression technique conforming to MPEG (Moving image Experts Group) 1 standard, audio and video information can be recorded in a normal CD (compact disc) for music.
MPEG is an international standard for compression of moving image data (image signal corresponding to the moving image). According to the e MPEG1 standard, moving image data is compressed into 1.5 Mbps, that is, a TV signal is compressed into about 1/100. While a transmission rate according to the MPEG1 standard is restricted to about 1.5 Mbps, according to MPEG2 standardized to meet demands of higher image quality, the moving image data is compressed into 2-15 Mbps.
Under the existing circumstances, MPEG4 is being standardized by a group (ISO/IEC JTC1/SC29/WG11) which has also standardized MPEG1 and MPEG2. The compression technique (object coding scheme) according to MPEG4 enables encoding and signal operation for each of objects composing a scene (one frame image), and new functions required for multimedia. As references for MPEG4, there is “ISO/IEC14496-1 MPEG-4 Systems, Final Committee Draft, May 15, 1998”.
Commonly, in a coding scheme for moving pictures, a moving picture is handled as a series of still pictures (frames), and video data is compressively encoded frame by frame. On the other hand, in the object coding scheme according to MPEG4, an image having a specific shape (foreground), a background and the like included in a frame are respectively assumed to be one object and video data corresponding to the frame (one frame image)) is handled for each video object corresponding to the object. This object coding scheme enables appropriate compressive-coding process object by object, and thereby improves a data compression rate of video data in one frame. In addition, in this object coding scheme, information indicating placement of respective objects on one frame and the like, is handled independently of the object data, thereby improving convenience with which the object data is processed and edited.
In the object coding scheme according to MPEG4 as international standard, encoded video data corresponding to a plurality of objects is decoded and composited to provide reproduced data corresponding to a composite image (reproduced scene), which is to be displayed.
Encoded video data corresponding to the respective objects is packetized and transmitted. Specifically, the encoded video object data is divided into code sequences of appropriate lengths, to which additional information such as headers are added, resulting in packets to be transmitted.
According to MPEG4, encoded video object data corresponding to a plurality of objects composing a scene is packetized and multiplexed, and transmitted as a bit stream.
FIG.
16
(
a
) shows a data structure of this multiplexed bit stream. A multiplexed bit stream Bs includes, for example, packets P(n), P(n+1), and P(n+2). The packet P(n) comprises a header H(n) and a data part D(n). The packet P(n+1) comprises a header H(n+1) and a data part D(n+1). The packet P(n+2) comprises a header H(n+2) and a data part D(n+2).
A data part of each of the packets contains a code sequence constituting corresponding encoded video object data, and a header thereof contains identification information for identifying a content of data stored in a corresponding data part, or time management information used for decoding and reproducing the data.
The time management information is added to each access unit as a unit to-be-decoded. The time management information is called a “time stamp”, which includes a DTS (Decoding Time Stamp) as time management information for decoding, and a CTS (Composition Time Stamp) as time management information for composition. One time stamp (CTS) for one processing will do, because time for another processing is found by calculation. It should be noted that the DTS needs to be added as a time stamp for each frame when the order of a reproduction process for each frame including composition and display is different from that of a decoding process for each frame. The access unit is equivalent to one frame in terms of video data and is equivalent to one audio frame in terms of audio data.
Whenever a data part of a packet contains head data of an access unit, a corresponding packet header contains a time stamp for the access unit.
FIG.
16
(
b
) shows a portion (frame data) Fd
1
corresponding to one frame as the access unit of the encoded video object data corresponding to the first object, and FIG.
16
(
c
) shows a portion (frame data) Fd
2
corresponding to one frame as the access unit of the encoded video object data corresponding to the second object. These frame data Fd
1
and Fd
2
is called VOP (Video Object Plane) and to respective headers thereof, time stamps Ts
1
and Ts
2
are added, respectively.
Conventionally, as an image composition method for compositing plural object data to display one scene, there is a technique termed “CGD” (Computational Graceful Degradation) (hereinafter referred to as a CGD method).
An example of this CGD method is to estimate decoding ability of a decoder for decoding encoded object data and outputting decoded data and reduce the number of steps in the decoding process so that the decoding process is completed by the time when decoded data should be output. Another example is to add priority information to respective frames and thereby
Hagai Makoto
Ishida Takanori
Matsui Yoshinori
Kelley Chris
Matsushita Electric - Industrial Co., Ltd.
Tu Tung Vo
Wenderoth , Lind & Ponack, L.L.P.
LandOfFree
Image output apparatus, image reproduction method, object... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image output apparatus, image reproduction method, object..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image output apparatus, image reproduction method, object... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3069208