Television – Camera – system and detail – Camera image stabilization
Reexamination Certificate
2000-07-13
2004-08-24
Christensen, Andrew (Department: 2615)
Television
Camera, system and detail
Camera image stabilization
C348S208990, C348S208100, C348S208500, C348S208600
Reexamination Certificate
active
06781623
ABSTRACT:
The invention relates to image processing, more especially to a method of and apparatus for processing motion picture images taken with a moving camera such as a hand-held camera, or a hand-held terminal device including a camera.
BACKGROUND OF THE INVENTION
In the future, there is likely to be considerable demand for telephones with a multi-media video and audio capability.
FIG. 1
of the accompanying drawings illustrates one possible design for a video telephone in the form of a hand-held terminal
14
. The hand-held terminal has a main housing
10
to which is mounted a video display
12
and an antenna
26
. The display is provided for showing moving picture images received by the terminal from a wireless transmission to the antenna
26
. A camera
16
and
18
for taking images is built into the housing
10
. The camera is provided to take a sequence of image frames and to supply them to the antenna
26
for wireless transmission to a base station. The camera will most likely be a digital camera based on a charged coupled device (CCD)
16
, or other array detector, and will have conventional lens optics
18
, possibly in conjunction with optical fibre components. The camera will have an optical axis “O”. In addition, there will be a notional vertical axis “V” of the terminal defined by vertical alignment of the display and camera. The alignment of the display
12
will be made to coincide with the alignment of the projection of the image viewed by the camera on the rectangular active area of the CCD chip
16
. The antenna
26
may be a broad-band transceiver antenna
26
, or some other antenna arrangement such as separate helical antennae for receiving and transmitting arranged within the housing
10
. The main housing
10
will also comprise various keys or buttons for dialling and other functions, and have an in-built loudspeaker and microphone for the audio part of the signal. These components are not shown.
Consider a video telephone communication between two users, Janet and John. During the call, Janet will hold her terminal by one or two hands for comfortable viewing of John on the display. For Janet, whether or not she is holding her terminal at the correct orientation will be of secondary importance. However, for John, any mis-orientation of Janet's terminal will be a problem, since it will result in Janet's image being mis-oriented on the display of John's hand-held terminal. For John, this will be a nuisance and detract from his subjective evaluation of picture quality.
FIGS. 2
to
4
of the accompanying drawings illustrate the orientation problem.
FIG. 2
of the accompanying drawings illustrates Janet's image displayed on John's hand-held terminal with proper alignment of Janet's hand-held terminal relative to herself. The image is shown as a number of shaded objects, as would result from use of a standard such as MPEG-4. Janet is object
3
, the remaining objects
1
,
2
and
4
being background objects.
FIG. 3
of the accompanying drawings shows Janet's image as superimposed on the CCD chip
16
of her hand-held terminal, which is now being held by her tilted at an angle. More particularly, the vertical axis “V” of the hand-held terminal now extends at an angle &thgr; to an axis “U” characteristic of Janet's image.
FIG. 4
of the accompanying drawings shows Janet's image as it appears on the display of John's hand-held terminal when Janet is holding her terminal as shown in FIG.
3
. Thus, if John holds his terminal straight, Janet appears to be leaning over. John could re-align Janet's image by rotating his terminal, but this would affect his image as displayed on Janet's terminal. Reaction times and transmission lag could result in an unstable picture orientation if communicating parties attempt hand correction of the vertical alignment in this way.
More generally, in any application where there is a possibility of rotating a camera about its optical axis, the image taken by the camera will appear distorted in the perception of a viewer when displayed on a remote terminal. For example, a door or building will appear to be leaning over at an angle, or the horizon of a landscape will appear tilted.
Appreciation of this problem leads to the following conclusions for hand-held terminals comprising a display and a built-in video camera:
(i) It will be inconvenient and difficult for a user to hold a hand-held terminal so that his/her image aligns vertically with a vertical axis defined by the hand-held terminal's camera and display.
(ii) The vast majority of images of interest will have a preferred vertical alignment axis that will need to be aligned with a vertical axis of the transmitting user's hand-held terminal for maximum perceived picture quality on the receiving user's display.
It is therefore an object of the invention to provide a method and apparatus by which vertical mis-alignment of images taken with a hand-held camera device can be automatically corrected for.
SUMMARY OF THE INVENTION
According to a first aspect of the invention there is provided a hand-held device comprising a display for displaying moving pictures on a frame-by-frame basis and a camera having an optical axis extending generally away from the display to image a person who is viewing the display. The hand-held device further comprises a sensor configured to determine a rotational angle between an alignment axis of the hand-held device and a reference alignment axis in real space, and a signal processing circuit arranged to associate image frames taken by the camera with respective rotational angles determined by the sensor.
By associating each frame with a rotational angle reflecting vertical mis-alignment of the data content of the image, vertical mis-alignment can be corrected for by applying a rotational transform to the image frames, either in the hand-held device itself or subsequently.
In one embodiment, a digital signal processor is operatively arranged between the camera's detector and an output stage of the hand-held device so as to apply a rotational transform to each image frame taken by the camera prior to supply of that frame to the output stage.
In another embodiment, a digital signal processor is operatively arranged between an input stage of a terminal device and its display so as to apply a rotational transform to each image frame received by the input stage prior to supply to the display, the transform being a rotation of the image frame through an angle derived from the rotational angle associated with that frame which is supplied to the hand-held device with the image data device. In this embodiment, the terminal device may in fact not be a hand-held device, but could be a larger device such as a bulky projector, home video player or personal computer.
In a further embodiment, a digital signal processor for applying the rotational transform is arranged in a wireless base station used for relaying data between transceiver parties. The transform angle is derived from the rotational angle associated with that frame which is supplied to the base station with the image data by the transmitting party.
According to a second aspect of the invention there is provided an image processing apparatus, comprising a digital signal processor for processing a sequence of image frames by: (a) determining a vertical alignment axis for each frame of the sequence from an analysis of the data content of that frame; (b) applying a rotational transform to each frame to map the vertical alignment axis determined by the analysis onto a fixed alignment axis of the frame; and (c) outputting the sequence of image frames. This approach differs from that of the first aspect of the invention in that the vertical mis-alignment is determined from image processing of the data content of the image frames themselves, rather than by an independent measurement of a physical parameter, such as gravity, with a sensor.
In one embodiment, the image processing apparatus of the second aspect of the invention is provided in a
Brady III W. James
Christensen Andrew
Genco Brian
Marshall, Jr. Robert D.
Telecky , Jr. Frederick J.
LandOfFree
Vertical compensation in a moving camera does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Vertical compensation in a moving camera, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Vertical compensation in a moving camera will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3305531