Computer graphics processing and selective visual display system – Computer graphics processing – Graphic manipulation
Reexamination Certificate
2000-03-06
2003-12-02
Razavi, Michael (Department: 2672)
Computer graphics processing and selective visual display system
Computer graphics processing
Graphic manipulation
C345S630000, C345S632000, C345S633000, C345S634000, C345S418000, C345S419000, C348S586000, C348S589000, C348S600000, C348S584000
Reexamination Certificate
active
06657637
ABSTRACT:
TECHNICAL FIELD
The present invention relates to a moving image combining apparatus combining computer graphics with video images.
RELATED ART
The following processing is conventionally performed when combining and displaying computer graphics and video images in a virtual space displayed by a computer using three-dimensional (hereafter 3D) graphics. A computer graphics (hereafter CG) image is generated by performing graphics-generating calculations using coordinate values showing locations and outlines for objects in a virtual 3D space. A video frame is extracted from a video sequence, and pasted onto the generated CG image using a method known as texture mapping. The resulting image is then displayed. High-speed repetition of this processing sequence for generating of a CG image, extracting of a video frame, pasting the video frame onto the CG image and displaying the resulting image enables CG images on which video frames have been pasted to be displayed sequentially, giving the appearance of a moving image.
However, the respective display rates for computer graphics and video images prior to combining are not necessarily identical. In a video sequence, a fixed number of frames can be displayed during a fixed time (this is hereafter to as the display rate). One standard for the video image display rate is 30 frames per second. In contrast, for computer graphics, the calculation time required to generate a CG image from coordinate values for object locations and outlines varies according to the number of objects to be displayed. As a result, achieving a uniform display rate is normally difficult.
Suppose the video image and computer graphics display rates are respectively 30 and 10 frames per second and moving images are combined at the computer graphics display rate. This means that, of the 30 video frames that can potentially be displayed in one second, only the 10 frames coinciding with display times of CG images can be displayed. Consequently, the remaining 20 frames cannot be displayed, so that the movement of the video sequence is jerky.
If moving images are combined at the video image display rate, however, the calculation required to generate a CG image cannot be completed in the interval between the display of consecutive video frames, meaning that it may not be possible to generate a CG image on every occasion.
DISCLOSURE OF THE INVENTION
In order to overcome the above problems, an object of the present invention is to provide a moving image combining apparatus combining computer graphics and video images at their respective display rates, a moving image combining method, and a recording medium recording a program for combining moving images.
An invention achieving the above object is a moving image combining apparatus combining computer graphics images (hereafter referred to as CG images) and at least one video sequence composed of a plurality of video frames, the moving image combining apparatus including the following. An information storage unit storing object information showing an outline and location for at least one object in three-dimensional (3D) space, a video obtaining unit obtaining from an external source at least one video sequence composed of a plurality of video frames generated at a fixed video display rate, an image storage unit, a receiving unit for receiving position information showing a position of a moving viewpoint, a graphics generating unit for generating CG images one at a time at a graphics display rate and, on completing the generation of a CG image, writing the CG image into the image storage unit, the CG image obtained by projecting each object whose outline and location is shown by the object information onto a projection surface, as seen from a current position of the moving viewpoint shown by the position information, and a video frame generating unit for fetching at least one video frame from the at least one video sequence at the video display rate and writing the fetched at least one video frame over a CG image, the CG image being stored in the image storage unit immediately prior to the time that the at least one video frame was fetched.
This construction enables generating of CG images and decoding of video frames to be performed in parallel using separate processes, and the generated CG image and still video to be combined in the storage unit. As a result, computer graphics and video images can be combined at their respective display rates.
The graphics generating unit may further perform rendering on each generated CG image, and write the rendered CG images into the image storage unit.
This construction enables rendering to be performed on a CG image, so that realistic graphics can be obtained.
Here, the following construction may also be used. Each object includes at least one video display area. The moving image combining apparatus combines, on at least one video screen located on the projection surface, at least one video sequence and a CG image, each video screen corresponding to a video display area. The object information includes information showing an outline and location for each video display area. The graphics generating unit further calculates screen information showing an outline and location for each video screen, each video screen obtained by projecting a video display area shown by an outline and location in the object information onto the projection surface. The video frame generating unit overwrites fetched video frames at each location shown by the screen information, so that each fetched video frame fits an outline shown in the screen information.
This construction enables a video sequence to be combined on a video screen of an object.
The video frame generating unit may also be constructed so that it generates transformed video frames by transforming the fetched video frames to fit an outline shown in the screen information; and overwrites the transformed video frames into the image storage unit.
This construction enables a video frame to be transformed to fit the video screen of the object, enabling the video sequence to be combined more realistically.
The following construction may also be used. Each object has a plurality of video display areas. The video obtaining unit obtains a plurality of video sequences from an external source. The moving image combining apparatus combines, on each of a plurality of video screens on a projection surface, one of the video sequences with a CG image, each video screen corresponding to one of the plurality of video display areas. The object information includes information showing outlines and locations for a plurality of video display areas. The graphics generating unit calculates screen information for each piece of information showing the outline and location for one of the plurality of video display areas. The video frame generating unit fetches video frames from each of the plurality of video sequences, and overwrites fetched video frames from the different video sequences at the different locations shown by the plurality of pieces of screen information, so that the fetched video frames fit the outlines shown in the screen information.
This construction enables video sequences to be combined on each of a plurality of video screens, when an object has a plurality of video display areas.
The video frame generating unit may also include the following. A priority ranking determining unit for determining a priority ranking for each video screen based on the plurality of pieces of calculated screen information. A video decoding unit for obtaining video frames from each of the plurality of video sequences, based on the determined priority ranking. A masking location calculating unit for calculating locations to be masked on each video screen, based on the plurality of pieces of calculated screen information and the priority ranking determined for each video screen. A masking unit for masking the transformed video frames at the calculated locations. Here, the video frame generating unit overwrites the transformed video frames which have been masked into the image stor
Inagaki Satoru
Nishigori Yoshihisa
Tsuda Kenjirou
Chung Daniel J
Razavi Michael
LandOfFree
Moving image combining apparatus combining computer graphic... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Moving image combining apparatus combining computer graphic..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Moving image combining apparatus combining computer graphic... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3107536