Television – Synchronization – Locking of video or audio to reference timebase
Reexamination Certificate
1998-10-29
2001-02-06
Lee, Michael (Department: 2714)
Television
Synchronization
Locking of video or audio to reference timebase
C348S512000, C348S578000
Reexamination Certificate
active
06184937
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates to the improved performance of devices for inserting realistic indicia into video sequences, and particularly, to the enhancement by the addition of audio effects related to the insertions.
2. Description of the Related Art
Electronic devices for inserting electronic images into live video signals, such as described in U.S. Pat. No. 5,264,933 to Rosser, et al. and U.S. Pat. No. 5,491,517 to Kreitman et al., have been developed and used commercially for the purpose of inserting advertising and other indicia into video sequences, including live broadcasts of sporting events. To varying degrees of success, these devices seamlessly and realistically incorporate indicia into the original video in real time. Realism is maintained even as the original scene is zoomed, panned, or otherwise altered in size or perspective.
U.S. Pat. No. 5,264,933 to Rosser, et al. discusses having the video insert respond to sound in the event, i.e. having the video insert pulse or change color in response to a rise in crowd noise. It does not, however, disclose the reverse possibility of adding a sound effect to the audio to coincide with a video insert, i.e. adding a beat to the program sound to coincide with the pulsing of the insertion, or altering the program audio in response to audio or visual cues in the program or in response to some operator command.
Other patents concerning video insertion technology, such as U.S. Pat. No. 5,491,517 to Kreitman et al., U.S. Pat. No. 5,353,392 to Luquet et al., or U.S. Pat. No. 5,488,675 to Hanna or PCT applications PCT/US94/01679 and PCT/US94/11527 of Sharir and Tamir confine themselves strictly to the video portion of a broadcast. None of the aforementioned patents or applications disclose methods for making an inserted indicia look more realistic by adding synchronized audio effects.
SUMMARY OF THE INVENTION
The invention comprises both a method and an apparatus to enhance real time insertion of indicia into video by altering the audio portion of a broadcast as well as the video portion of the broadcast. The invention applies equally well to real time insertion of video indicia accomplished by means of pattern recognition technology; by means of camera motion sensors attached to the cameras recording the event; or by a combination of pattern recognition and camera motion sensors.
In the present invention each still or animated video sequence intended for insertion into the live video has an associated audio sequence. When the still image, animated image sequence, or video sequence is inserted, the associated sound sequence is also activated. Sound activation may be triggered by the start of the insertion; some action in the video portion of the insertion; some action either in the video or audio channel of the broadcast; by some combination of action in the audio and video channel, or partially or wholly, by an operator. In addition to triggering, the playing, volume, modulation, termination, or any other attribute of the associated sound sequence, may be influenced by the inserted image, animation or video, audio or video channel of the event, some combination of the audio and video channel of the event or partly or wholly by an operator.
The associated audio sequence is stored either digitally in system memory in the same manner as the video sequences are stored, or separately on either an analogue or digital medium.
A live video insertion system is enhanced so that, in addition to channels for program video and video insertion, an enhanced audio processor is added within an audio channel. In a standard live video insertion system the audio channel is merely a delay line allowing the program audio to be delayed during video processing. The enhanced audio processor interacts with the pattern recognition and tracking part of the live video insertion system (LVIS™). If the audio mixing is done digitally, there is also means to convert the program audio from analogue to digital and back to analogue after the mixing is done.
The enhanced audio processor also includes means for audio pattern recognition for adding an audio sequence to the broadcast audio, or otherwise alter the broadcast audio. Audio pattern recognition can be used alone or in conjunction with commands from the video pattern recognition and tracking module of the LVIS™. It can also be used in conjunction with operator commands.
Making an inserted indicia appear as if it is actually part of the original video scene is an important aspect of the technology. Appropriate audio cues can considerably enhance the visual illusion that the inserted video indicia is part of the original scene. Audio enhancement of the illusion is particularly effective if the inserted indicia is an animated sequence and the added audio is timed to coincide with specific actions of the animation. For example, an inserted video indicia can be programmed to pulse on and off. To enhance this illusion, a sound effect can increase or decrease in volume in sync with the pulsing video insertion. Other examples include changes in pitch, tone, reverberation, added echo, spoken dialogue, or musical jingles of an audio insert that are synchronized with changes in the inserted video.
Alteration of the original program sound rather than addition of a separate audio insert can be done as well. For instance, crowd noise could be artificially modulated to coincide with a change in the inserted logo. Consider an animated version of a team mascot. As the artificially inserted team mascot raises and lowers its arms, the crowd volume could increase or decrease accordingly adding to the illusion that the mascot was actually in the stadium.
Further enhancements include synchronizing an audio addition or the actual broadcast audio with an audio or visual cue occurring in the action of the event being broadcast. In a baseball game, for instance, a cartoon character insertion can react to an audio event such as the crack of the bat with a suitable phrase in the distinctive voice of the character. Or, the reaction could be synchronized to a visual cue such as swinging the bat. A combination of visual and audio cues may be partially or entirely operator activated and synchronized to an event like the start or end of an inning. Additionally, if the insert in a baseball game appeared to be hit by the ball, a suitable sound appearing to come from the injured insert could be added to the program audio. Utilizing known speech recognition techniques, the audio cue could be a command, a well known phrase, or team name.
Added sound can follow the movement of a video insert. For instance, the volume associated with the insert could increase as the camera zooms in and the insertion grows in size. For stereo broadcasts, the ratio of the left and right channels can be altered as the insert panned off to the side such that the sound seems to follow the insert.
REFERENCES:
patent: 4605964 (1986-08-01), Chard
patent: 4692803 (1987-09-01), Tobita
patent: 5202761 (1993-04-01), Cooper
patent: 5264933 (1993-11-01), Rosser et al.
patent: 5353392 (1994-10-01), Luquet et al.
patent: 5387943 (1995-02-01), Silver
patent: 5430495 (1995-07-01), Ko
patent: 5436672 (1995-07-01), Medioni et al.
patent: 5467139 (1995-11-01), Lankford
patent: 5488675 (1996-01-01), Hanna
patent: 5491517 (1996-02-01), Kreitman et al.
patent: 5880788 (1999-03-01), Bregler
patent: PCT/US94/01679 (1994-02-01), None
patent: PCT/US94/11527 (1994-10-01), None
Rosser Roy J.
Williams Brown F.
Lee Michael
Princeton Video Image, Inc.
Woodbridge Richard C.
Woodbridge & Associates P.C.
LandOfFree
Audio enhanced electronic insertion of indicia into video does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Audio enhanced electronic insertion of indicia into video, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Audio enhanced electronic insertion of indicia into video will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2581185