Education and demonstration – Question or problem eliciting response – Response of plural examinees communicated to monitor or...
Reexamination Certificate
1997-03-27
2001-07-10
Martin-Wallace, Valencia (Department: 3713)
Education and demonstration
Question or problem eliciting response
Response of plural examinees communicated to monitor or...
C434S118000, C434S322000
Reexamination Certificate
active
06259890
ABSTRACT:
FIELD OF THE INVENTION
This invention generally relates to the field of computer-based test creation systems, and more particularly, to the on-line creation of items for tests using a computer-based test creation system.
BACKGROUND OF THE INVENTION
For many years, standardized tests have been administered to examinees for various reasons such as for educational testing or for evaluating particular skills. For instance, academic skills tests, e.g., SATs, LSATs, GMATs, etc., are typically administered to a large number of students. Results of these tests are used by colleges, universities and other educational institutions as a factor in determining whether an examinee should be admitted to study at that particular institution. Other standardized testing is carried out to determine whether or not an individual has attained a specified level of knowledge, or mastery, of a given subject. Such testing is referred to as mastery testing, e.g., achievement tests offered to students in a variety of subjects and the results being used for college credit in such subjects.
FIG. 1
depicts a sample question and related directions which might be given on a standardized test. The stem
12
, the stimulus
14
, responses
16
, and directions
18
for responding to the stem
12
are collectively referred to as an item. The stimulus
14
is the text and/or graphical information, e.g., a map, scale, graph, or reading passage, to which a stem
12
may refer.
After all of the examinees' tests are graded, statistical and other processing may be provided for various reasons. For instance, to assess one examinee's score, it is necessary to compare his or her score to those of other examinees taking the same test. Another important reason to evaluate the test results for statistical purposes is to create and update an information bank containing the performance statistics of each item used or created for previous tests. This information may then be used for the creation of future tests.
A goal of standardized testing is to efficiently construct a test for the purpose of measuring a skill, ability, etc. Therefore, each test is constructed to conform to a test specification which defines the rules and/or constraints for selecting the items. In constructing a test, test developers select items from a pool of items so that the combination of selected items satisfy the test specification.
A test is typically divided into sections of questions. The test specification generally defines the number of items to be presented in the test, the number of test sections, the number of questions in each section, the time for taking the test, and the allotted time for responding to all the items in each test section. The test specification also specifies criteria for item selection. These are based on at least four item characteristics which include: (1) item content, e.g., mathematical questions relating to arithmetic, algebra, or geometry; (2) cross-information among items, e.g., more than one item testing the same point; (3) number of items/set, i.e., identification of a subset of items of a larger set; and (4) statistical properties of items derived from pretesting, e.g. difficulty of the selected items.
In recent years, the methods for creating, delivering, administering, and scoring tests have been determined to be inadequate. Due to the number of examinees taking standardized tests, the demand for developing new and more diverse tests and a need to provide more flexibility in scheduling tests without sacrificing administration costs and security have increased. One solution to these demands would be to automate the entire testing process. Only a few attempts have been made, however, to automate only portions of the testing process. Furthermore, these attempts are limited in their ability to generate a variety of item types.
For example, a prior art computerized testing system, “TD/DC,” is disclosed in U.S. Pat. No. 5,565,316, entitled “System and Method for Computer Based Testing” and assigned to Educational Testing Service, Princeton, N.J. The TD/DC system, however, has drawbacks in that the creation stage is not automated. During the item creation stage of the TD/DC system, all work has to be done manually through the use of a work folder. In particular an item author creates an item on paper and places the paper in a work folder. This work folder is then passed from person to person for reviewing, editing or commenting on the item. The work folder finally arrives back at the original author.
Since it is not automated, the entire test creation process with the TD/DC system involves many steps and “hand-offs” of the item/work folder. For example, it has been found to take 179 hand-offs and 197 steps to create one item for the verbal section of the SAT test! Because of the hand-offs, much time is wasted waiting for a user to pass the work folder off to the next user. On the math section of the SAT test, only 22% of the elapsed time to create an item is actually spent working on that item. Moreover, on the GRE Chemistry test, 30% of the total working time is spent making transcriptions between paper and computer. An automated item creation system is desired which will streamline the item creation process by making the process more efficient and reducing the required number of steps and hand-offs.
When using the TD/DC system, it is not feasible for people to review an item at the same time to collaborate their thoughts. For one reason, there is only one copy of the item contained in the one work folder. An item creation system is thus desired which will allow several users to collaborate on a particular item at the same time.
Another drawback of the TD/DC system is that throughout the item creation process, everything is done on paper. That is, the item is created on paper, edits may be made to the item on this same sheet of paper, art work may be drawn by hand on the back of this same sheet of paper, or edits and additions may simply be added to the work folder on other sheets of paper. This makes it very difficult for a user to view the current state of the item. Moreover, a user or reviewer never sees what the item will look like until it is viewed in a completed test, either in a computer based test or a test booklet. Even when an item is finally input into a computer in an electronic form in the TD/DC system, it is only held in an unformatted text string, unable to be viewed as it would appear in an actual test. It is desirable for the item creation system to allow users to view the item as it will appear in the test.
Another drawback to the TD/DC system is that it cannot handle multimedia item types. That is, because every step of the creation process is done on paper, multimedia items such as audio, video or animation items cannot be made with the TD/DC system. It is thus desired to provide a versatile system for test creation so that it will allow for the creation of multimedia item types.
Another drawback to the TD/DC system is that evaluation of the performance of items is not efficient. For example, the TD/DC system relies on users to identify problems in items based on reviewing statistical information on items after administrations of such items in tests. It is desired to improve the efficiency of the item performance evaluation stage of the test creation process, for example, by employing an automatic system of identifying items that have problems and initiating electronic evaluation reports requesting evaluation personnel to review such items and to resolve such problems.
The main object of the present invention is to address the above-noted problems with the prior art item creation systems by providing a more efficient item creation system. The prior art system, i.e., the TD/DC system, lacks efficiency in that much of it is still paper based, it involves numerous steps and hand-offs, it lacks automatic or automated steps, and the item is never viewed as it is intended until it is present in its final form, i.e., in the computer based test or in a test booklet. Because many currently
Burns Richard
Driscoll Gary
Keener Barbara
Sanchez-Lazer Teresa
Snieckus Alison
Educational Testing Service
Harris Chanda
Martin-Wallace Valencia
Woodcock Washburn Kurtz Mackiewicz & Norris LLP
LandOfFree
System and method for computer based test creation does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for computer based test creation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for computer based test creation will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2471583