Television – Panoramic – With continuously rotating element
Reexamination Certificate
1998-06-17
2002-01-01
Rao, Andy (Department: 2713)
Television
Panoramic
With continuously rotating element
C348S036000
Reexamination Certificate
active
06335754
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to an image recording apparatus and method for describing a virtual space on the basis of sensed images and, more particularly, to a method of efficiently adding location information associated with image sensing location to a sensed image sequence obtained by sensing an object using cameras. The present invention also relates to an image database system built to describe a virtual space on the basis of sensed images. Furthermore, the present invention relates to a recording medium that stores a computer program for implementing the above-mentioned recording control.
In recent years, attempts have been made to build civic environments where many people socially live in cyber spaces formed by computer networks. Normally such virtual spaces are described and displayed using conventional CG techniques. However, since CG expressions based on geographic models have limitations, Iamge-Based Rendering (IBR) based on sensed images has come to the forefront of the technology.
Reproducing sensed image data as they are amounts to merely experiencing what a photographer has gone through. For this reason, a technique for generating and presenting an arbitrary scene in real time using the IBR technique has been proposed. More specifically, when sensed images are processed as independent ones, and are re-arranged in accordance with a viewer's request, the viewer can walk through his or her desired moving route at a remote place, and can feel a three-dimensional virtual space there.
It is effective for searching for and re-constructing a desired image in accordance with the viewer's request to use the location information of points where the individual images were taken. That is, an image closest to the viewers request is selected from a database, and undergoes proper image interpolation so as to generate and display an optimal image.
FIG. 1
shows the principle of wide-area walkthrough using sensed images.
More specifically, sensed images are prepared for narrow areas
1
,
2
,
3
, and
4
. In order to implement wide-area walkthrough (e.g., along a route
10
or
11
) that allows the viewer to walk across these narrow areas, an image in a space between adjacent narrow areas must be obtained by interpolation. When the viewer is currently located at a position between the narrow areas
2
and
3
, and the space between these areas is obtained by interpolation, specific sensed images for the areas
2
and
3
must be obtained by a search on the basis of information associated with the current location of the viewer between the areas
2
and
3
. In other words, in order to obtain required images by a search on the basis of the location information of the user, a database of sensed images must be prepared in advance in accordance with location data upon image sensing.
In order to attain precise interpolation and to smoothly connect the interpolated images and sensed images, as the viewer may walk through in the 360° range around him or her, sensed images of the environment must be taken by a large number of cameras disposed to point in various directions, and an image database must be built using these sensed images.
In order to attain precise interpolation on the basis of images obtained using a plurality of cameras, the image sensing centers of many cameras must agree with each other. However, it is not easy to arrange many cameras in such way.
To solve this problem, conventionally, a plurality of mirrors are set symmetrically about a given point, and the mirror surfaces of the individual mirrors are set so that light beams coming from the surrounding portions are reflected upward, thereby setting the image sensing centers of the cameras at one point, as shown in FIG.
2
.
However, in the prior art, in order to build a database of sensed images, location information is added in units of image frames of the individual sensed images.
The location information is added to each image in such a manner that image sensing is done while moving a video camera at a constant speed so as to successively record images on a video tape, and the position of the portion of interest is calculated on the basis of the distance from the beginning of that video tape to the position of that tape portion which stores the image to which the location information is to be added. Then, a sensed image database is formed by combining the images on the tape and the image sensing location information of these images.
Hence, suchworks are nearly manually done, and requires much time as the number of sensed images becomes larger.
In order to implement wide-area walkthrough, each sensed image must have a broad range (wide angle). When a wide-angle image is obtained by a single camera, if image interpolation is done using neighboring pixels of the wide-angle image, many errors are produced in an image obtained by interpolation, and continuity is substantially lost between the interpolated image and sensed image. When a plurality of cameras (n cameras) are set at wide-angle positions to take images, the addition work of the location information increases n times.
SUMMARY OF THE INVENTION
The present invention has been made in consideration of the above situation and has as its object to provide an image recording method and apparatus, which can efficiently add location information associated with each image sensing location to a sensed image sequence obtained by a camera.
It is another object of the present invention to provide a database apparatus which builds an image database to which time information is added so that location information associated with each image sensing location can be efficiently added to a sensed image sequence obtained by a camera.
In order to achieve the above objects, according to the present invention, a method of recording images obtained by sensing an object using a plurality of cameras which point in a plurality of discrete azimuth directions so as to obtain a basis for forming a sequence of three dimensional image spaces, is characterized by comprising the steps of:
storing, in a first memory, data frames of images generated by sequentially sensing an object using a camera of the plurality of cameras together with specifying information that specifies each data frame;
acquiring location information indicating an image sensing location of each data frame, and storing, in a second memory, the location information together with acquisition time information indicating an acquisition time when the location information is acquired; and
storing, in a third memory, a pair of specifying information that specifies each data frame, and acquisition time information corresponding to the location information acquired at the time when the data frame is generated.
Image data require a large memory capacity. When no high-speed processing is required, a sequential memory has a large capacity. Hence, according to a preferred aspect of the present invention, the first memory comprises a sequential memory, and the specifying information is a frame number indicating a recording order of the data frames in the first memory.
According to a preferred aspect of the present invention, the specifying information is information indicating a relative time upon recording the data frame in the first memory.
According to a preferred aspect of the present invention, the first memory comprises video tape memories arranged in units of cameras, and the second memory comprises a magnetic disk.
According to a preferred aspect of the present invention, the specifying information includes time codes generated by each camera. The time codes have different default values among the cameras. Hence, in this aspect, calibration is attained by calculating the difference between the time code, as a reference, of a specific one of the plurality of cameras, and that of another camera.
According to a preferred aspect of the present invention, the plurality of cameras are mounted on a vehicle.
According to a preferred aspect of the present invention, posture information indicating
Endo Takaaki
Katayama Akihiro
Mixed Reality Systems Laboratory Inc.
Rao Andy
LandOfFree
Synchronization between image data and location information... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Synchronization between image data and location information..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Synchronization between image data and location information... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2846531