Data processing: measuring – calibrating – or testing – Measurement system – Measured signal processing
Reexamination Certificate
1999-12-13
2003-09-02
Hoff, Marc S. (Department: 2857)
Data processing: measuring, calibrating, or testing
Measurement system
Measured signal processing
C341S050000, C345S097000
Reexamination Certificate
active
06615163
ABSTRACT:
TECHNICAL FIELD
The system and method described herein relates generally to the field of testing, and more particularly, to developing testing configurations for compatibility testing.
BACKGROUND
In today's world, intense competition in many industries places significant pressure on companies and their employees to both reduce product development and testing cycle times, while at the same time many products are increasing in complexity. Further, in many industries the trend has been a demand from customers for companies to provide wide ranges of choices for customers, such as to allow customers the ability to select one particular type of component from a choice of many, for each of several different components present in a product. For example, in the computer industry customers may wish to purchase a computer system, but specify, for example, the specific type of central processing unit (CPU) the system will include, the amount of memory, different types of software, and many other features.
When multiple variations of multiple components can be selected companies must, to the best of their ability, test each possible combination to ensure that the different components are compatible with one another, and that the system will function satisfactorily with that particular selection of components. When the number of selections is not too large, testing each combination may be possible. However, where the number of configurations is large, testing of each configuration of components is neither possible nor practical. Under these circumstances, an efficient system and method for selecting a reasonable number of configurations is desired such that the maximum number of combinations of components can be examined with minimum number of tests or trials.
In the context of testing compatibility, whether in the computer industry or otherwise, the result of any test is either pass or fail rather than a quantitative result. Under these circumstances confounding effect is of no interest, as is the case when there are multiple variables and a quantitative result is sought. Thus, when designing configuration tests, the objective is to select a minimum set of configurations that will examine the greatest number of combinations of components, thus avoiding redundancy and at the same time avoiding holes in the test space, thereby increasing test efficiency.
By maximizing the coverage there is a better chance to achieve a greater number of failures. For example, if three different selections are available for each of two different components (selection 1, 2 or 3 for the first component and selection A, B or C for the second component), there are a total of six different variables and nine possible combinations. In configuration testing, the components are referred to as “factors” while the selections are referred to as “levels.” It is most desirable to ensure that each of the six different levels appear in at least one test. If this occurs, there will be 100% coverage for each level taken individually, and after completing the testing it will be known with certainty whether each level, taken alone, will cause the system as a whole to fail.
The second level of coverage is every conceivable combination of two variables, i.e.,
1
A;
1
B;
1
C;
2
A;
2
B;
2
C,
3
A;
3
B; and
3
C. 100% coverage would ensure that any combination of two will be covered in the trials. Although with large numbers of configurations 100% coverage at the first, second, and all subsequent levels is not possible, the objective is most importantly to maximize coverage at the first level, second most importantly to maximize coverage at the second level, and so on.
In developing testing configurations, matrices are often used in which individual columns represent the possible factors, rows represent the number of trials, and the entries in the matrix represent the combination of levels (the “level combination”) for any given trial. For example, for a system that includes two factors (F
1
, F
2
) each of which have two levels (L
1
, L
2
), a testing matrix could be as follows:
F1
F2
1
L1
L1
2
L1
L2
3
L2
L1
4
L2
L2
For factors having two levels, entries in the matrix can be replaced by 1's or 0's to provide the following matrix:
F1
F2
1
0
0
2
0
1
3
1
0
4
1
1
where 0 represents L
1
and 1 represents L
2
. Note that all possible combinations can be covered (100% coverage) in four trials. In choosing how to populate a testing matrix the objective in configuration testing is to populate the matrix in such a way that coverage is maximized for the number of trials.
In achieving this objective of maximizing coverage, the principle of orthogonality is always important. An orthogonal array is an array of 1's and 0's that for strength n has the property that for any n number of columns selected all possible combinations of 1's and 0's will appear an equal number of times. For example, for an orthogonal array of strength 2, when selecting any two columns all possible combinations of 1's and 0's (00, 01, 10, and 11) will appear an equal number of times. Thus, an orthogonal array of strength n not only helps ensure that certain combinations of variables are not disproportionately tested, but also ensures 100% coverage for any combination of n different variables. Degradation of orthogonality for any testing matrix represents an imbalance in the variables represented in the trials and, therefore, can cause less than desirable coverage. Thus, it is desirable to maintain orthogonality of the highest strength in a testing matrix for the least number of trials.
A well known limitation of orthogonal arrays, however, is that orthogonal arrays of size 2
n
can have only n+1 independent orthogonal arrays of strength n. For example, to maintain a strength of 3 (n=3), an orthogonal array of vector size 8 (2
3
) can have only 4 (3+1) columns. In other words, for only 4 columns will the property hold true that in any 3 out of the 4 columns all possibilities of 1's and 0's (000, 001, 010, 100, 011, 101, 110, and 111) will appear an equal number of times. In testing configuration matrices, 4 columns represents only four bi-level factors. It would, of course, be desirable to maintain orthogonality across a larger number of factors for the same number of trials.
In developing configuration testing matrices, it has been discovered that Hadamard matrices are useful in populating the matrix, particularly for pass/fail testing applications. Hadamard matrices are square matrices of order n whose entries are (+)'s and (−)'s. If we replace (+) with 1 and (−) with 0 they will be orthogonal arrays of strength two, that have the property HH′=nI, where I is an identity matrix. Thus, for any one column or for any two columns in a Hadamard matrix, any given combination of 1's and 0's will appear an equal number of times. Hadamard matrices can easily be used for factors having only two levels since each level can be represented by either a 0 or a 1.
More recently, it has been shown that Hadamard matrices can also be used for variables having more than two levels by combining columns. For example, for a variable having four levels, each level can be represented by the combinations 00, 01, 10, and 11. By combining two columns in the Hadamard matrix each of these combination can be represented. See “Pass/Fail Functional Testing and Associated Test Coverage,” Breyfogle, R. W., Quality Engineering 4(2), 1991, pp. 227-234. Combining columns of the Hadamard matrix, particularly in a random fashion, results in degradation of the orthogonal properties of the Hadamard matrix particularly when, for any array size of 2
n
, there are more than n+1 columns (more than n+1 factors).
Another disadvantage of combining columns to account for variables having greater than two levels occurs when the number of levels is not a power of two. For example, for a variable having three levels, those three levels can be represented by 00, 01 and 10, leaving the
Osella Stephen A.
Rasoulian Hamid
Baker & Botts L.L.P.
Dell USA L.P.
Kim Paul
LandOfFree
System and method for developing testing configurations does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for developing testing configurations, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for developing testing configurations will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3092799