Data processing: measuring – calibrating – or testing – Measurement system – Performance or efficiency evaluation
Reexamination Certificate
1999-02-12
2002-03-12
Hoff, Marc S. (Department: 2857)
Data processing: measuring, calibrating, or testing
Measurement system
Performance or efficiency evaluation
C716S030000, C716S030000, C703S013000
Reexamination Certificate
active
06356858
ABSTRACT:
TECHNICAL FIELD
The present invention relates to coverage tools which are used for both verification in the execution of programs and verification in the operation of a hardware system, and relates particularly to a coverage measurement tool enabling a user to create a solution for coverage models of his choice.
BACKGROUND
Verification is becoming one of the biggest problems of the software industry. The cost of verification as part of the development process is usually between 40-80%. The practice of letting the users find the bugs and fixing them is in the next release is becoming dangerous and costly for three main reasons. The first one is that the users expect the program to work correctly and if it does not, then, there is a cost to the reputation. The second one which is the actual cost of fixing the software could be very large when there is a large install base. The third one is that litigation can be used if the software error caused harm to the user. In software development, it will soon be required of a development to demonstrate that his product was tested adequately.
Functional verification comprises a large portion of the effort in designing a processor. The investment in expert time and computer resources is huge, and so is the cost of delivering faulty products. In current industrial practice, most of the verification is done by generating a massive amount of tests by random test generators. The use of advanced random test generators can increase the quality of the generated tests, but it cannot detect cases in which some areas of the design are not being tested, while other parts are tested repeatedly.
The main technique for checking and showing that the testing has been thorough is called test coverage analysis. Simply stated, the idea is to create, in some systematic fashion, a large and comprehensive list of tasks and check that in the testing phase, each task was covered. Coverage can help in monitoring the quality of testing, and assist in directing the test generators to create tests that hit areas that have not been tested before.
Many coverage tools exist in the market. Most coverage tools are program-based and check code coverage of the execution of programs. Examples include code coverage in C or VHDL and covering states and transitions of state machines. The main disadvantage of code coverage is that it does not <<understand>> the application domain. Therefore, it is very hard to tune the tools to areas which the user thinks are of significance. To overcome this problem, some domain dependent tools have been developed which measure specific functional events. In all these tools the coverage models are hard-coded.
Therefore, in order to make the tools applicable to many users, the models that are implemented in these tools are generic. This leaves a user that wants to cover models that are specific to his design with two options: to measure coverage manually, or to build his own coverage tool for these models. Both options require a large effort and are error prone. Therefore, specific models are rarely used.
SUMMARY OF THE INVENTION
Accordingly, the main object of the invention is to propose a new methodology for coverage of a design calling for separation of the coverage model from the coverage measurement tool.
Another object of the invention is to provide a single general purpose tool meeting all the coverage needs of a user both for generic and specific models.
Another object of the invention is to generate and implement a single general purpose tool which is oblivious to the coverage model and provide all the functionality of existing tools.
Yet another object of the invention is to provide a coverage measurement tool enabling a user to define its own coverage model, gather and process traces to measure the coverage of this model and generate reports on the results of the measurement.
The invention relates therefore to a coverage measurement tool enabling a user to create a specific coverage tool for a coverage model consisting of a set of coverage tasks specified by the user for checking a design such as a hardware or software system, the tool being associated with a database and with model definition means providing a model definition which defines the coverage model. Such a tool comprises an insertion engine for storing into the database a table containing the traces resulting from a plurality of tests, a processing engine for processing the traces stored in the database according to the model definition, and a coverage analyzer for analyzing the measurement results from the processing engine and preparing coverage analysis reports according to the model definition.
According to another aspect, the invention relates also to a method of generating a coverage measurement tool enabling a user to define its own coverage model consisting of a set of coverage tasks meeting certain conditions on the events that create the tasks for checking a design such as a hardware or software system, and comprising the steps of collecting event data from tests performed on the design and inserting the event data into an event trace, extracting from the event trace the tasks meeting the conditions in order to produce a trace task list and determining whether each task of this list is a legal task in order to update a list of the legal tasks when the task is a legal task or to put it into an illegal list when the task is an illegal task.
The advantages of such a coverage measurement tool are enormous. First, it allows the users to define their own models, according to their specific needs, and still enjoy all the functionality of a dedicated coverage tool without the need to develop such a tool. Therefore, it provides users with the means to measure coverage on models which were not available to them before. This tool also allows organizations to use a single tool for most of their coverage needs, and not be forced to buy and maintain a set of tools, one for each coverage model. Another advantage of using an external model definition is that it enables sharing of coverage models between projects. Finally, this tool enables the users to adapt their coverage models to their testing resources, and refine these models during testing. For example, users that can afford only quick and dirty testing can define coarse grain models with a small amount of tasks, while users that want to do comprehensive testing can define finer grain models.
REFERENCES:
patent: 5724504 (1998-03-01), Aharon et al.
patent: 6141630 (2000-10-01), McNamara et al.
Grinwald et al., User defined coverage—a tool supported methodology for design verification, Jun. 1998, IEEE, pp. 158-163.
Gluska Alon
Malka Yossi
Ur Schmuel
Ziv Avi
Fitch Even Tabin & Flannery
Hoff Marc S.
International Business Machines Corp.
Percello Louis J.
Vo Hien
LandOfFree
Coverage measurement tool for user defined coverage models does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Coverage measurement tool for user defined coverage models, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Coverage measurement tool for user defined coverage models will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2863765