Data processing: software development – installation – and managem – Software program development tool – Testing or debugging
Reexamination Certificate
2001-05-18
2004-07-27
Zhen, Wei (Department: 2122)
Data processing: software development, installation, and managem
Software program development tool
Testing or debugging
Reexamination Certificate
active
06769114
ABSTRACT:
TECHNICAL FIELD
The present invention relates in general to automatic software testing and, in particular, to methods and apparatus for preventing software modifications from invalidating previously passed integration tests.
BACKGROUND
Most software programs are changed from one version to the next to implement new functionality, to improve performance, and to correct defects. Very often, a software change will lead to modifications in several different subroutines.
Once a software change is made, the modification may unintentionally affect previously tested functionality. For large and complex software, this inadvertent destruction happens quite often. As a result, during integration testing of the modified software, two sets of integration test cases are exercised. A set of regression test cases are performed to determine if previously tested functionality is still in tact after the new modifications. In addition, a set of functional test cases are performed to determine if the modification produces the new desired results.
If an error is found during the integration testing, changes made to the software to fix the error may in turn unintentionally destroy other tested functionality. A large software project typically goes through several cycles of “modify and test” before the number of errors found “converges” to a satisfactory low threshold. For many large projects, running the previously passed integration tests requires a substantial investment of time (e.g., weeks to months). As a project release date approaches, the development team often decides not to rerun all the previously passed tests in an effort to save time. To contain the risk of unintentionally destroying previously passed tests, they typically decide to accept only a portion of the program modifications for new functionality or bug fixes. These decisions compromise the integrity of the software product. After an extensive review of the development processes of two of the premiere software companies, Cusumano and Yoffie describe these pitfalls in their paper, “Software Development in Internet Time” which is incorporated herein by reference.
It is well known that the cost to find and fix a software error increases exponentially as the software development process progresses through coding, unit testing, subsystem integration testing, and system integration testing. The book, “Software Engineering Economics”, by Barry Boehm and published by Prentice-Hall in 1981, which is incorporated herein by reference, gives many such examples.
A typical software project may devote one third of the available human resources and time to integration testing. In the earlier stages of development, it is typically up to the individual programmer to develop his/her own test cases and unit test environment. Because of lack of resources and time, unit test cases are typically not as extensive as integration test cases. In particular, there is no automatic tool that can help the programmer to check whether what they are working on is in conflict with previously passed tests and with the work of other programmers until integration time. In other words, software projects typically are not effective in taking advantage of the cost and time savings in finding more errors earlier in the development process.
Some earlier attempts at solving software testing problems include methods to automate test execution such as by recording and playing back graphical user interface manipulations as disclosed in U.S. Pat. No. 5,511,185 and U.S. Pat. No. 5,600,789 (both incorporated herein by reference). Others attempt to reduce the number of test cases based on how the software was modified. The article, “Analyzing Regression Test Selection Techniques,” by Rothermel et al in IEEE Transaction on Software Engineering, August 1996, discusses many such methods, and U.S. Pat. No. 5,694,540 describes one such method (both incorporated herein by reference). Still other methods generate a test environment (stubs and drivers) for unit testing such as the AUT system for Cobol programs in early IBM systems as documented in the book “Software Testing and Evaluation”, by DeMillo et al., which is incorporated herein by reference. Still others consider generating test data automatically generated by analyzing source code. The article, “Techniques for Automated Test Data Generation,” by C. V. Ramamoorthy et al, published in the Conference Record of the Ninth Asilomar Conference on Circuits, Systems an Computers in November, 1975 described a number of these methods.
However, prior art software testing methods suffer form certain drawbacks. For example, prior art methods do not prevent new software modifications from unintentionally failing previously passed integration tests. Further, they do not take advantage of the fact that software testing is easier and less costly during unit testing than during integration testing.
REFERENCES:
patent: 5966541 (1999-10-01), Agarwal
patent: 6349393 (2002-02-01), Cox
patent: 6351826 (2002-02-01), Kato
patent: 6510402 (2003-01-01), Logan et al.
patent: 6601018 (2003-07-01), Logan
patent: 6647513 (2003-11-01), Hekmatpour
Rothermel et al. Selecting Regression Tests for Object-oriented Software. IEEE. 1994. pp. 14-25.*
Li et al. An Overview of Regression Testing. ACM. 1999. pp. 69-73.*
Korel et al. Automated Regression Test Generation. ISSTA. 1998. pp. 143-152.*
Rothermel et al. A Safe, Efficient Regression Test Selection Technique. ACM. 1997. pp. 173-210.*
International Search Report dated Jan. 18, 2002.
Boris, Beizer “Software Testing Techniques—2ndEditon”, Van Norstrand Reinhold, New York, NYpp. 452-459 (1990).
Demillo et al., “Software Testing and Evaluation”, The Benjamin/Cummings Publishing Company, Inc., Menlo Park, CA pp. 143-150 and 172-177 (1987).
Kaiser et al., “Infuse: Fusing Integration Test Management with Change Management”, Proceedings of the Annual International Computer Software and Applications Conference (COMPSAC), Orlando, FL pp. 552-558 (Sep. 20, 1989).
Marshall & Gerstein & Borun LLP
Zhen Wei
LandOfFree
Methods and apparatus for preventing software modifications... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Methods and apparatus for preventing software modifications..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Methods and apparatus for preventing software modifications... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3192060