Image analysis – Image transformation or preprocessing – Combining image portions
Reexamination Certificate
1999-11-12
2004-02-03
Mehta, Bhavesh M. (Department: 2625)
Image analysis
Image transformation or preprocessing
Combining image portions
C382S282000, C382S291000, C382S294000, C382S298000, C358S540000, C358S450000, C358S538000
Reexamination Certificate
active
06687419
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to the automatic creation of a larger image of an object or scene from a series of smaller source views. Such an image is known as a ‘montage’ and, particularly when using a microscope and camera combination, enables a higher resolution image to be formed of a much larger region of the object or scene than is possible otherwise.
When recording a scene by means of a camera, three particular limitations may pose problems: firstly, there may be a limited field of view (ie. in the x and y coordinate directions); secondly, there may be a limited depth of field (in the z coordinate direction, ie. that of the camera axis); and, thirdly, there may be limited camera resolution. In particular, these can be severe in the case of an image being recorded (eg photographed) through a microscope.
Prior art solutions to one or more of these problems are as follows:
Accurately reposition the camera or the subject and record multiple images, then ‘stitch together’ afterwards. An example of a system using this technique is the MIA package from Soft Imaging System GmbH (SiS) which generally requires direct control of a microscope stage in order to position the object under the objective lens in a series of positions such that overlapping images can be ‘stitched’ together to form a larger image. Disadvantages are that the positions need to be known accurately.
Our Auto-Montage technique in which multiple digitally recorded images are taken at different focus settings of the camera or z-positions of the subject, and then formed into a single image where each pixel is selected from the source image which shows the most contrast (equivalently, is most in focus).
High resolution and hence more expensive cameras. For electronic cameras, cost becomes prohibitive beyond about 2 k pixels square.
Although the present invention is aimed primarily at dealing with the first and third of the three problems outlined above, it may also, in suitable embodiments, deal with second problem.
SUMMARY OF THE INVENTION
According to the present invention there is provided a system, for creating a larger montage image of an object from a series of smaller source images of different views of an object, which includes
imaging means for providing, continually in use, images of an object;
data storage means for storing data representative of the montage image of the object;
means for including the data representative of a first and subsequent source images in that representative of the montage image, in order to increase the data stored representative of the montage image; and
means for providing a measure of the extent of matching of the most recent source image provided by the imaging means and individual ones of a plurality of regions of the montage image which correspond in area;
the arrangement being such that data representative of subsequent source images is included in the data representative of the montage image when the measure of the extent of matching is at a maximum, whereby the source image is matched to the montage image at the correct position.
The present invention also includes a method of creating a larger montage image of an object from a series of smaller source images of different views of an object, which method includes
generating a series of source images of an object;
allocating memory within a data storage means for storing data representative of the montage image of the object;
storing data representative of a first and subsequent source images in the memory, in order to store data representative of the montage image; and
measuring the extent of matching of a most recent source image and individual ones of a plurality of regions of the montage image which correspond in area;
data representative of the subsequent source images being included in the data representative of the montage image, when the measure of the extent of matching is at a maximum, so that the source image is matched to the montage image at the correct position.
Preferably, a threshold value of the measure of the extent of matching is preselected so that source images which will not match with the montage image are discarded.
The montage image is thus built up gradually from the subsequent source image data being added to the first source image data which represents a first data set for the montage image. The source image data can be provided by a camera or other image capture device moved relative to the object.
By comparing the most recent source image with regions of the montage image in turn, the source image can be ‘located’ relative to the montage image and the additional data representative of the source image incorporated into the data set defining the montage image so as to expand the montage image. This process can be repeated until the montage image is complete. The comparison may be carried out in a number of different ways, some of which are detailed later, and may be carried out on a subset of the image data of both the montage image and the source images.
The size of the completed montage image may be predetermined, but may be allowed to increase by allocating more memory to the montage image.
Thus, new data is stored representative of a new image at a position at which the new source image closely matches a region of the montage image. If this position has not changed since the previous measurement, the source is stationary and thus there is determined an appropriate position at which to insert the source image into the montage image.
It is preferred to add source image data to the montage image data when the source image data is provided from a stationary image. This provides two benefits:
Any motion blur or interlace distortion of the source images is minimised, so the image pasted into the montage should be free of these artefacts;
If the system waits for the movement to stop before doing the more time-consuming elements of the process (fine position determination, 3D Auto-Montage), movement can be tracked rapidly and the time-consuming elements of the processing can be done when the delay is less significant to the operator.
Advantageously, the system indicates when it has determined the subject to be stationary (and is therefore updating the montage image), and when it is moving. The system may also indicate when it cannot determine the position confidently, either because of poor matching or low overlap. Furthermore, the system may indicate when it requires the user to pause for some reason (such as to save some data).
The system is particularly suitable for use with a microscope and camera combination for capturing highly magnified images of small objects. Preferably, the system includes a computer and software adapted to carry out the steps outlined above, the montage image and the current source image (or a rectangle representing its location relative to the montage image) being displayed on a monitor to provide feedback to a user operating the system (in the case of a microscope, the operator moving the microscope stage in order to re-position the object relative to the camera).
Because of the nature of the acquisition process, it is important that this feedback be given simply and directly. Colours, flashing and audio signals may all be employed so that the user has a ‘quiet cockpit’ when the system is operating normally, and can concentrate on exploring the subject.
The determination of the appropriate position at which a newly captured image should be incorporated into the montage image may be achieved by a variety of means, including the location and matching of specific or significant features in the image, or by finding the location of the source image at which the difference between the source and the montage images is at a minimum.
Preferably, a similarity measure is used, so that the source is, notionally, shifted across the montage image until a maximum is found in the similarity measure. This may conveniently be achieved by means of a type of cross-correlation statistic called ‘covariance’. Cross-correlation and covariance are known to be
Kassa Ysef
Mehta Bhavesh M.
Rosenthal & Osha L.L.P.
Synoptics Limited
LandOfFree
Automatic image montage system does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Automatic image montage system, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Automatic image montage system will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3320392