Image analysis – Pattern recognition – Template matching
Reexamination Certificate
1997-12-31
2001-05-01
Couso, Jose L. (Department: 2721)
Image analysis
Pattern recognition
Template matching
C382S181000, C382S219000, C348S402100, C348S408100
Reexamination Certificate
active
06226406
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of Invention
The present invention relates to the field of computer software and computer program testing. More specifically, it relates to verification of graphics images produced by a computer graphics program.
2. Discussion of Related Art
In the computer software industry, the amount of time needed to test software products, specifically computer graphics programs, has become a significant bottleneck in the product development timeline. The testing cycle for graphics programs, where images typically must undergo thousands of tests, presently takes several weeks. The test cycle is typically conducted after the software product has been completed by the developer and must be completed before the product is released.
With a typical computer graphics program, the test engineer must verify that the software produces output images, as expected, with no significant errors. The images generated by the computer program must be within an acceptable range of images described in the program's specification. Regardless of how confident application developers are with the integrity and performance of the software, before the software is commercially released, test data is needed to support the assertion that the product runs as expected. The bottleneck occurs in creating this test data. by the increasing complexity of graphics libraries, which contain routines to generate images and is required when running a graphics program.
Efforts to reduce the number of reference images have led to other drawbacks which can be explained by the fact that computer graphics use mathematical models to simulate scence designed by the user. Whenever mathematical approximations are used, there are going to be errors involved. With image comparison verification, because one of the goals is to reduce the number of reference images, test engineers tend to combine many test programs into one test frame. This increases the error margin in the test engineer's prediction of the expected image, since a higher number of mathematical models is used in one test frame. Consequently, the test engineer usually describes the expected output image using general terms, such as “a shaded red square with white shading in the lower right corner.” The problem with this type of description is that different people looking at the same image may focus on different areas and may focus on portions different from the portion focused on by the test engineer designing the test case.
Another time-consuming factor arises from a pixel being in error due to more than one bug in the graphics program. Cumulative errors may cause a pixel in an output image to be different from a pixel in a reference image. A senior engineer typically has to make a judgment call as to the significance of that pixel. Because some errors are more significant than others, it must be determined which errors can be ignored and which ones need to be addressed. This determination typically requires the experience of a senior test engineer, who also cannot always determine which errors are significant and which ones are not.
FIGS. 1
a-c
show the different images needed for image comparison verification in the prior art.
FIG. 1
a
shows a screen shot of a typical reference image of various geometric shapes and lines.
FIG. 1
b
is a screen shot of a typical corresponding output image which is to be tested against the reference image of
FIG. 1
a
.
FIG. 1
c
is a screen shot of an image showing the differences, pixel by pixel, between the output and reference images.
As is evident from the discussion above, there are several significant drawbacks from using traditional image comparison testing. First, each hardware/software configuration requires a separate set of reference images resulting in a proliferation of images. Second, the testing method does not provide qualitative information on failed cases. There is a need to generate a new set of reference images whenever there is a new hardware/software configuration for the program. Third, since there are many possibilities or reasons a test may fail it is necessary to have a senior test engineer supervise the error checking step. Finally, there is no room for error tolerance which would eliminate insignificant or “don't care” type rendering differences between the images.
Another method of testing computer graphics programs is referred to as sample point testing. With this type of testing, pixel values are hardcoded in the test programs and are given only a color tolerance. The test data determines what type of testing is going to be applied to that pixel and the area around that pixel (the location of the center of the area is hardcoded in the test program). However, the hardcoded values of the pixels apply only to a particular hardware/software configuration.
A major problem arises in that maintaining and updating the test programs is a very time consuming and error prone procedure. Essentially, the test program code has to be changed every time there is any type of update. Having the hard coded expected values of selected points contained within the test program itself makes the test platform dependent. Thus, the significant shortcomings of sample point verification are that it provides limited error checking and that the expected values of the points are bard coded into the test program. Even error tolerances which may be included in the test program cannot be modified for individual test cases without having to modify the test program itself.
Therefore, what is needed is a method of testing computer graphics programs that is efficient and eliminates insignificant rendering differences. It would also be desirable to have a testing methodology in which the errors can be identified and qualified with a high degree of confidence and, at the same time, not require a high level of maintenance of reference images. It is also desirable to have more modular type testing so that errors can be identified and corrected in less time.
SUMMARY OF THE INVENTION
To achieve the foregoing and in accordance with the purpose of the present invention, methods, systems, and software for testing the accuracy of images generated by computer graphics programs are disclosed. An output image is verified by being compared to a reference image, also known as the golden image. In one aspect of the present invention, the image to be tested, referred to as the output image, is partitioned into several checking apertures. A checking aperture can be of either a block type or a point type. In either case, each checking aperture is made up of several pixels. One or more attributes of the output image is compared against the reference image. For pixels in a point type checking aperture, a positional tolerance is determined. Expected values of pixels in the reference image are then derived by examining the specification of the computer graphics program. The positional tolerances and the expected values are stored in a reference file. A test program is executed in which the output image and the reference image are then compared. Any differences between the two images are then stored in an errors file.
In another embodiment, the reference image and the output image are compared in increments of checking apertures. A checking aperture from the output image is mapped onto the corresponding checking aperture in the reference image. In yet another embodiment, a color tolerance is determined for each pixel in a block type checking aperture.
In another embodiment, a reference file including positional tolerances, color tolerances, and expected values of pixels is created for each computer platform that the graphics program is being tested on, thereby resulting in a set of reference files. Upon execution of a test program to check the accuracy of the output image, the reference file is invoked and an errors file is created. The tolerances are set based on acceptable deviations of color and position of pixels in the output image from the corresponding pixels in the reference image. The errors file contains
Beyer Weaver & Thomas LLP
Couso Jose L.
Desire Gregory
Sun Microsystems Inc.
LandOfFree
Method and apparatus for hybrid sampling image verification does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for hybrid sampling image verification, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for hybrid sampling image verification will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2460437