Computer graphics processing and selective visual display system – Computer graphics processing – Three-dimension
Reexamination Certificate
2000-01-06
2002-11-05
Vo, Cliff N. (Department: 2671)
Computer graphics processing and selective visual display system
Computer graphics processing
Three-dimension
C345S420000, C345S582000
Reexamination Certificate
active
06476803
ABSTRACT:
BACKGROUND
1. Technical Field
The invention is related to a computer-implemented object modeling system and process, and more particularly to such a system and process that employs noise elimination and robust surface extraction techniques.
2. Background Art
Computer modeling of natural or manmade objects from images has found a wide range of applications. For example, an electronic catalogue of products for Internet advertisement, or the visualization of museum objects on the Internet. The object could even be a human face, and the applications include talking heads and games, and very low bandwidth video conferencing on Internet.
There are several considerations in deciding what system to employ to accomplish the aforementioned object modeling. First, there are the image collection costs. Many object modeling applications do not lend themselves to elaborated camera setups or customized fixtures for holding the object, as is needed with many of today's object modeling systems for capturing the needed images of the object being modeled. The cost and difficulty in positioning the camera and/or the object are simply to great in many cases to make the modeling practical. It would be much more desirable that so-called casual images be used and that the object modeling system be capable of using such images. Casual images of an object are those that can be captured without having to control the camera's position or orientation, and which do not require costly object-holding fixtures. One particularly attractive way of capturing casual images of an object is with a desktop digital camera. Desktop-type digital cameras are becoming cheap and ubiquitous, and so would be an excellent choice for capturing the images needed to model an object at minimum cost.
Another consideration in selecting an object modeling system is the data storage requirements of the system. If the storage requirements are extensive, the cost of the system's memory could become prohibitive. In addition, if large amounts of data must be handled, the object modeling will not be suitable for remote presentation, such as on the web.
Lastly, the complexity of the object modeling system should be considered. For example, if the processing requirements of the system are overly complex, then the time required to model an object may become excessive. In addition, special computing hardware may be needed, thereby driving up the cost of the system. Further, if the system requires a great deal of user interaction in the modeling process, it may be too daunting for many too learn and use the system.
The current object modeling techniques can be roughly divided into two categories. The first category uses a 3D model-based representation. In this category, a CAD-like model is built, which is very concise. However, it is extremely difficult to obtain an accurate model because of imperfections in system calibration, uncertainty in feature extraction, and errors in image matching. In addition, these techniques often need to acquire and store a large number of images, and the collection cost is high because the camera's pose must be memorized for each image taken. The other category involves an image-based representations. In this category, one needs to acquire and store nearly all images necessary for subsequent visualization. Therefore, visualization of an object model is essentially a redisplay of images, yielding photorealistic results. However, a major drawback of this technique is that it requires all images be stored for future rendering, thus requiring a great deal of system memory.
Given the foregoing considerations, it is clear there is a need for an improved object modeling system and process.
It is noted that in the remainder of this specification, the description refers to various individual publications identified by a numeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, “reference [
1
]” or simply “[
1
]”. A listing of the publications corresponding to each designator can be found at the end of the Detailed Description section.
SUMMARY
The present invention involves a new object modeling system and process that allows easy image capture, requires only a small amount of data storage, and constructs the desired model efficiently. The system and process first acquires images of an object that is to be modeled. These images can be captured in a variety of ways, but regardless of the procedure employed, the images should collectively depict every surface of the object that it is desired to model. One possible procedure for capturing the needed images of the object would be to use a stereo camera rig and obtain stereo images of the object (e.g., using a trinocular stereo system). Another, more standard, method would be to capture a series of 2D images of the object being modeled. One particularly attractive procedure for obtaining the 2D images is to capture the aforementioned casual images of the object. As explained previously, casual images are those that can be captured without having to control the camera's position or orientation, and which do not require costly camera equipment or object-holding fixtures.
The images of the object, regardless of how they were captured, are employed in the next phase of the object modeling process to compute a series of 3D reconstructions of the object. The reason for initially computing multiple 3D reconstructions from the images is to ensure every surface of the object being modeled is represented by some part of a reconstruction and to allow for significant overlap in the reconstructions to increase the accuracy of the modeling process. It is also preferred that a multiframe stereo reconstruction process be employed to increase the accuracy of the reconstructions. To this end, a small group of consecutive images (e.g., 5 frames) could be used to produce each reconstruction. There are many existing methods that can be used to compute the 3D reconstructions, any of which could be employed as desired. For example, if the acquired images are individual 2D views of the object, a standard feature point tracking and structure-from-motion approach could be used to produce the desired 3D reconstructions. It is noted that these standard reconstruction approaches using 2D images typically require that the intrinsic parameters associated with the camera used to capture the images be computed. Alternately, if the acquired images are captured using a stereo rig, such as for example a trinocular stereo system, the 3D reconstructions could be generated via a conventional 3D registration procedure.
Regardless of how the 3D reconstructions are obtained, it is preferred that the reconstruction data be processed to eliminate noise effects before proceeding to the next phase of the object modeling process. This is especially desirable where stereo matching techniques have been used to generate the reconstructions as they tend to be noisy. One preferred noise elimination procedure is “automatic clustering”. This method begins by calculating the mean and then the variance of the points in the point cloud in each orthogonal direction. A threshold is then applied in each direction to define a cuboid-shaped bounding box. Any point lying outside the bounding box is eliminated.
The procedure is repeated until there is no longer any significant change in the mean and variance between iterations. While the automatic clustering method is good at eliminating extraneous reconstruction points outside the defined bounding boxes, it does nothing to eliminate extraneous points within the box, such as might exist in voids or holes associated with the object. A “3D spatial filtering” procedure can be used to remove such points from the reconstruction data. The 3D spatial filtering begins by dividing a 3D space containing all the reconstruction points into voxels. To minimize processing, an octree scheme [
3
] is employed resulting in only voxels containing reconstruction points being considered. For eac
Loop Charles
Shan Ying
Zhang Zhengyou
Lyon Richard T.
Lyon & Harr LLP
Microsoft Corporation
LandOfFree
Object modeling system and process employing noise... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Object modeling system and process employing noise..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Object modeling system and process employing noise... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2970696