Data processing: presentation processing of document – operator i – Presentation processing of document – Layout
Reexamination Certificate
1999-10-20
2004-05-11
Shah, Sanjiv (Department: 2176)
Data processing: presentation processing of document, operator i
Presentation processing of document
Layout
C345S012000, C704S503000
Reexamination Certificate
active
06735738
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a technique for compressing or expanding data for sounds combined with animation while synchronizing the sounds and the animation so as to reproduce the data.
2. Description of the Prior Art
With recent rapid development of computer processing techniques regarding sounds and animation, multimedia equipment handling sounds and animation such as digital video has been spread rapidly. In particular, in the field of story-oriented animated cartoons that handle sounds and animation at the same time, computer processing is widely used for the edition or other purposes, and a highly advanced technique is in demand for the edition function for sounds and animation. For example, in the case where video data is to be created by combining animation data representing motions of a character with speech data that has been edited separately, it is important to produce natural video images by synchronizing the two data. Herein, “video images” refers to multimedia information including sounds and animation, and includes animated cartoons and video images having a story that are created by actual shooting.
Furthermore, in the case where a portion of the video data that is completed as one scenario is desired to be used in another piece of work, it may be necessary to compress or expand the video data to be used. In this case, it is important to compress or expand the data so that the synchronization of the acoustic data and the animation data is not destroyed.
However, in conventional video data, the speech data and the animation data are generated in time series with respect to a character independently in many cases, and they may differ in the recording time length. Therefore, it is necessary to compress or expand one of the data or both of the data and to synchronize the two data.
In the case where the video data are compressed or expanded, various methods are used to retain the synchronization of the acoustic data and the animation data. For example, Japanese Laid-Open Patent Publication (Tokkai-Hei) No. 5-3561 discloses a method for editing or reproducing a story including speech and animation, in which the speech data and the animation data are edited separately, and then reconstructed in synchronization to create one story.
However, in such a method, since the speech data and the animation data are compressed or expanded separately and then synchronized later, it is not guaranteed that the content of the speech and the animation are precisely in synchronization except at the start point and the end point of the synchronization. Since particularly definite video images tend to be required nowadays, a fine adjustment, for example, to reproduce a speech that agrees with the motion of the mouth of the character, is required. Nevertheless, there are many problematic cases, for example, where the speech does not agree with the motion of the mouth of the character, or where the speech is output when the mouth of the character is not moving.
SUMMARY OF THE INVENTION
Therefore, with the foregoing in mind, it is an object of the present invention to provide a simple method for precise synchronous processing, and method and device for reconstructing acoustic data and animation data in synchronization that can reproduce natural video images, in the case where video data including the acoustic data and the animation data that are in synchronization are compressed or expanded.
A device for reconstructing acoustic data and animation data in synchronization of the present invention for compressing or expanding a pair of acoustic data and animation data that are in synchronization includes a data segment dividing part for dividing a series of acoustic data and a series of animation data into a plurality of segments; a parameter time series instructing part for instructing a parameter time series that determines a degree of compression or expansion of the acoustic data and the animation data of each of the segments divided by the data segment dividing part; an input frame dividing part for further dividing the acoustic data divided by the data segment dividing part at equal intervals into input frames; an output frame length determining part for determining an output frame length of the acoustic data based on the parameter time series determined by the parameter time series instructing part; an acoustic animation synchronous data output part for outputting a correspondence table for synchronous points between the acoustic data before compression or expansion and the acoustic data after compression or expansion with respect to the acoustic data to be compressed or expanded based on the parameter time series determined by the parameter time series instructing part; an acoustic data reconstructing part for compressing or expanding the acoustic data based on the output frame length; and an animation data reconstructing part for compressing or expanding the animation data based on the correspondence table.
In this embodiment, the data previously is divided into segments at every point where synchronization is required. Therefore, when the acoustic data alone is compressed or expanded, the animation data accordingly can be compressed or expanded in synchronization with the acoustic data. Thus, the video data can be compressed or expanded simply and reliably.
In the device for reconstructing animation data and acoustic data in synchronization of the present invention, it is preferable that the parameter time series is extracted based on only the acoustic data in the parameter time series instructing part, because the scene with sounds is often an important scene that requires synchronization in the video images, such a scene where the character is speaking.
Furthermore, in the device for reconstructing animation data and acoustic data in synchronization of the present invention, it is preferable that the parameter time series is extracted based on only the animation data in the parameter time series instructing part. The synchronous point can be extracted easily based on the change in the motion of the character.
Furthermore, in the device for reconstructing animation data and acoustic data in synchronization of the present invention, it is preferable that the parameter time series is extracted based on both of the acoustic data and the animation data in the parameter time series instructing part. The weighting of both of the data combined allows synchronous points to be set more precisely.
Furthermore, in the device for reconstructing animation data and acoustic data in synchronization of the present invention, it is preferable that the parameter time series is input manually by a user in the parameter time series instructing part. This embodiment allows fine adjustment to minimize the sense of incongruity that a viewer feels.
Next, a method for reconstructing acoustic data and animation data in synchronization of the present invention for compressing or expanding a pair of acoustic data and animation data that are in synchronization includes dividing the acoustic data and the animation data into a plurality of segments; instructing a parameter time series that determines a degree of compression or expansion of the acoustic data and the animation data of each divided segment; further dividing the divided acoustic data at equal intervals into input frames; determining an output frame length based on the determined parameter time series; outputting a correspondence table for synchronous points between the acoustic data before compression or expansion and the acoustic data after compression or expansion with respect to the acoustic data to be compressed or expanded based on the determined parameter time series; compressing or expanding the acoustic data based on the correspondence table; and compressing or expanding the animation data based on the correspondence table.
In this embodiment, the data previously is divided into segments at every point where synchronization is required. Therefore, when the acoustic data alone is com
Nguyen Maikhanh
Shah Sanjiv
LandOfFree
Method and device for reconstructing acoustic data and... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and device for reconstructing acoustic data and..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and device for reconstructing acoustic data and... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3211203