Computer graphics processing and selective visual display system – Computer graphics processing – Three-dimension
Reexamination Certificate
1998-03-09
2002-06-04
Zimmerman, Mark (Department: 2671)
Computer graphics processing and selective visual display system
Computer graphics processing
Three-dimension
C345S426000, C345S423000, C345S629000, C345S632000, C345S427000
Reexamination Certificate
active
06400362
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to an image processing method and apparatus for generating and displaying a virtual environment for virtual reality.
2. Related Background Art
As one of conventional techniques of expressing three-dimensional objects or spaces and presenting observation images from arbitrary positions and directions, the following method is known:
(1) Three-dimensional objects or spaces are reconstructed by expressing three-dimensional objects or spaces using shape model data such as polygon data, curved surface data, and the like, texture data representing their surface attributes and patterns, light source data, and the like, and drawing the space images viewed from arbitrary positions and directions by computer graphics rendering techniques.
Upon building a three-dimensional virtual environment with this method, a method of expressing elements (coordinate transformation data, shape data, surface attribute data, illumination, and the like) that construct the virtual images using a tree structure is known. This is because a space, ground, building, room, furniture, illuminator, ornament, and the like that construct a three-dimensional space originally have a hierarchically nesting relationship among them. For example, an ornament on a table depends on the layout of the table so that it moves together with the layout of the table, and it is often convenient to arrange such ornament relative to the coordinate system of the table. For this reason, a data structure which has hierarchical dependence on the layout is used. As an example of this method, the virtual environment is expressed by an n-ary tree structure.
For example,
FIG. 4
shows an image of an example of a simple virtual environment. In case of
FIG. 4
, paying attention to a room, table, and sofa, the room is described on a coordinate system C
2
, which is transformed from a world coordinate system C
0
by a coordinate transformation T
2
, and the table and sofa in that room are respectively described on coordinate systems C
3
and C
4
which are transformed from the coordinate system C
2
by coordinate transformations T
3
and T
4
. The pot on the table is described on a coordinate system C
5
which is transformed from the coordinate system C
3
by a coordinate transformation T
5
. Furthermore, light space data is set on the table. This data is described on a coordinate system C
6
which is transformed from the coordinate system C
3
by a coordinate transformation T
6
as in the pot. When this environment is expressed by a schematic tree structure, a tree shown in
FIG. 5
is obtained.
By changing the tree structure by adding or deleting construction elements of the tree with this tree structure, or replacing the elements with other ones, objects, spaces, and the like in the virtual environment can be added, deleted, or replaced.
(2) A method of reconstructing three-dimensional objects or spaces by generating light space data on the basis of a group of images obtained by actually sensing three-dimensional objects or spaces, and generating and displaying images that can be seen from arbitrary positions and directions on the basis of the light space data is known.
With this method, each object is reconstructed as a set of light rays coming from the object without calculating its shape.
However, conventional methods (1) and (2) suffer the following problems.
It is hard for conventional method (1) to generate or reconstruct shape data of an object with a very complicated shape. Also, it is hard to acquire shape data of an object with a complicated shape from a real object using a three-dimensional measurement device. Especially, it is more difficult to reconstruct an image of a real object having an existing complicated shape or complicated surface pattern or reflection characteristics (absorption/transmission characteristics). Furthermore, in general, an artificial object is relatively easily expressed, but a natural object is hard to express. However, the merit of this method is to be able to express an artificial, simple three-dimensional space such as a room, a row of houses and stores on a street, or the like mainly constructed by planes with a small data volume. In addition, the method of expressing a virtual environment using a tree structure is an excellent one.
In conventional method (2), after light space data is generated from a predetermined number of sensed images, an observation image can be generated from any arbitrary view point position (there are, in fact, some constraints to be exact). Since this method does not reconstruct any object shape but describes an object on the basis of sensed images of the object itself, it is an excellent description technique that can express an object with high reality. However, a large data volume is required, and image sensing is difficult to attain when the object is in a broad range. This technique is rather suitable for describing a three-dimensional object, but is not suitable for expressing a broad three-dimensional space such as a room, a row of houses and stores on a street, and the like.
Furthermore, conventional methods (1) and (2) are fundamentally different techniques, and no method that can combine these methods while effectively utilizing their characteristics and merits alone is available.
SUMMARY OF THE INVENTION
It is an object of the present invention to provide an image processing method and apparatus which can more efficiently change a virtual environment constructed by various kinds of data.
In order to achieve the above object, according to the present invention, there is provided an image processing method of generating and displaying a virtual environment, comprising:
the tree structure generation step of generating a tree structure which includes, as construction elements, shape data that geometrically describes a three-dimensional object and three-dimensional space, and light space data that describes a three-dimensional object and three-dimensional space on the basis of a sensed image;
the virtual environment generation step of generating a virtual environment including both the shape data and light space data on the basis of the tree structure generated in the tree structure generation step;
the instruction step of instructing desired processing for a construction element of the tree structure as a basis of the virtual environment generated in the virtual environment generation step;
the change step of changing the tree structure by the processing for the construction element instructed in the instruction step; and
the regeneration step of regenerating the virtual environment on the basis of change contents in the change step.
Also, in order to achieve the above object, according to the present invention, there is provided an image processing apparatus for generating and displaying a virtual environment, comprising:
tree structure generation means for generating a tree structure which includes, as construction elements, shape data that geometrically describes a three-dimensional object and three-dimensional space, and light space data that describes a three-dimensional object and three-dimensional space on the basis of a sensed image;
virtual environment generation means for generating a virtual environment including both the shape data and light space data on the basis of the tree structure generated by the tree structure generation means;
instruction means for instructing desired processing for a construction element of the tree structure as a basis of the virtual environment generated by the virtual environment generation means;
change means for changing the tree structure by the processing for the construction element instructed by the instruction means; and
regeneration means for regenerating the virtual environment on the basis of change contents by the change means.
REFERENCES:
patent: 5261041 (1993-11-01), Susman
patent: 5379371 (1995-01-01), Usami et al.
patent: 5894310 (1999-04-01), Arsenault et al.
patent: 6151028 (2000-11-01), Kumagai et al.
Katayama Akihiro
Kumagai Atsushi
Uchiyama Shinji
Nguyen Kimbinh T.
Zimmerman Mark
LandOfFree
Image processing method and apparatus does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Image processing method and apparatus, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Image processing method and apparatus will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2920658