Error detection/correction and fault detection/recovery – Data processing system error or fault handling – Reliability and availability
Reexamination Certificate
1998-12-10
2002-10-01
Gaffin, Jeffrey (Department: 2182)
Error detection/correction and fault detection/recovery
Data processing system error or fault handling
Reliability and availability
C714S025000, C703S022000, C707S793000
Reexamination Certificate
active
06460147
ABSTRACT:
TECHNICAL FIELD
A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever.
BACKGROUND OF RELATED ART
The present invention involves a method and apparatus for automated software testing. More specifically the present invention provides the Tester with a method and system to describe the software under test without requiring the Tester to know all the details of the software.
Automated software testing can be defined as a program that automatically enters a predetermined set of characters or user commands in order to test new or modified versions of software applications.
Historically it has been observed that there are some serious problems with software automation test case testing tools. One of the key issues is the use of parameters by automation tools. Parameters must be known in advance. As the number of parameters increase, the shear numbers limit what can reasonably be tested.
If test tools can support differing languages, each have a unique set of parameters. If parameters change from one version to another, each different set of parameters requires modification to the system under test and/or the test tool's parameter handling.
Another key issue is that automated tests are likely to be static (i.e. they are primarily derived from manual test scripts). Therefore, to test new or changed functions, the test case must be revised. This is also true if a new or changed parameter is added or revised, respectively. This results in high maintenance.
Yet another key issue is that the function to be tested must exist in some detail for automated test cases to be written. The implication of this is that it is difficult to debug an incomplete function. The coding of the test cases delay the ultimate execution and completion of the testing. In addition, when a new defect is accidentally found outside the testing space, a new test case is required.
The present invention will be better understood and its numerous objects and advantages will become more apparent to those skilled in the art by reference to the following drawings, in conjunction with the accompanying specification.
Currently, it is almost an impossible task to test all the various combinations of parameters within a reasonable time frame. There is no way you can test all parameters, any more than you can test all the paths in White Box Testing. It is NP-complete (from a mathematical point of view, it is believed to be a solvable problem; however, it would take an almost infinite length of time to solve), and even if one does not need to test parameters, one must test them because today's tools need them to work.
Current automated test cases are acceptable for regression testing, but are considered unacceptable for nonregression testing (e.g. development systems). The maintenance as noted is too high. One changed parameter and the test case must be revised. If the vendor changes the tool, it is likely that the test case must be revised. The maintenance of test cases on a developing system is very high. The user must know what all the parameters are and the function they support before they can begin coding the test case.
It would therefore be a distinct advantage to have a method and system that would eliminate many of the above recited problems and concerns. The present invention provides such a method and system.
SUMMARY OF THE PRESENT INVENTION
The present invention, Smart Testing Method For Automated Software Testing, resolves the problems outlined above. Essentially this involves Goal oriented testing that queries the environment and takes action based on facts and a model of the system under test. The action can be modified by rules and exceptions, including training testing to focus on certain behaviors. Regarding the key issues outlined above, the use of parameters can be eliminated. Regarding the issue of automated tests being static and primarily derived from manual static test scripts, the instant system is dynamic quasi-random and can be trained to perform ever changing tasks including classic static emulation. Where, as noted in the prior art, functions to be tested must exist in some detail for automated tests to be written, in the instant system, few functional details are necessary. The instant system can run on an incomplete model.
Test case automation efforts have essentially involved taking a manual script and automating it. To be successful, a Tester must have “up front” knowledge of the details of what they are going to automate. For example, if the Tester wants to automatically select the “Save” menu item, he has to know its in the first drop-down menu for “File” on the main menu bar, and, also, either know its drop-down position or its name. Applicants, in the present invention, have been successful only because their application efforts to date have involved testing existing functions, better known as regression testing.
Assume, for a moment that the “Save” menu item is a new function. All the Tester knows is that there is a requirement for a file save menu item function and if the function succeeds, a file that previously existed will be rewritten with a new date and/or time. If the function fails a pop up window with the word “error” will appear. In the manual testing paradigm this new function would likely have been tested using “gorilla testing”. It is, as implied, a random “hunt and pick” manual method of testing, but is not repeatable.
One solution to the test automation problem would be for the Tester to know in advance all the design details. But chances are rare that would happen with any degree of accuracy required to automate the test. Another solution is, thus, proposed in the present invention.
The present Smart Tester invention gives the Tester a method to describe the system under test without having to know all the details of the system.
The architecture of the Smart Test is described as follows. Instead of writing a script, the Tester designs a functional model of the system to be tested. The system could be anything from a single application to a LAN environment. Assume for a moment that the system is an application. The Tester would, for example, model a main window, a menu bar, drop-down menus, specialized windows, etc. The model can be incomplete; however, what is complete must be accurate. For example, if there is no menu bar, the model mentioned above is inaccurate and will not work. The more complete the model, the better the chance the testing will cover existing functions.
The Tester would also define any facts that the model might need; for example, the name of the file to be opened and saved. The Tester then defines any goals or subgoals to be attained. For example, a goal might be saving the file. Then the rules under which the model will operate are defined. One rule might be “if the file (named as a fact) has its date and/or time changed, then the goal of saving the file was reached and the test will end.
REFERENCES:
patent: 4744084 (1988-05-01), Beck et al.
patent: 5490249 (1996-02-01), Miller
patent: 5544308 (1996-08-01), Giordano et al.
patent: 5546321 (1996-08-01), Chang et al.
patent: 5548718 (1996-08-01), Siegel et al.
patent: 5586025 (1996-12-01), Tsuji et al.
patent: 5589765 (1996-12-01), Ohmart et al.
patent: 5686833 (1997-11-01), Spinner
patent: 5869961 (1999-02-01), Spinner
patent: 5892947 (1999-04-01), DeLong et al.
patent: 5913023 (1999-06-01), Szermer
patent: 6061643 (2000-05-01), Walker et al.
patent: 6148277 (2000-11-01), Asava et al.
patent: 6349393 (2002-02-01), Cox
U.S. application Ser No. 09/240,923, Cox, filed Jan. 29, 2000.
Gaffin Jeffrey
Leeuwen Leslie A. Van
Mai Rijue
LandOfFree
System and method for automated testing of software systems... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for automated testing of software systems..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for automated testing of software systems... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2942955