Education and demonstration – Question or problem eliciting response – Response of plural examinees communicated to monitor or...
Reexamination Certificate
1997-07-31
2001-10-30
Martin-Wallace, Valencia (Department: 3713)
Education and demonstration
Question or problem eliciting response
Response of plural examinees communicated to monitor or...
C434S350000
Reexamination Certificate
active
06311040
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to systems and methods for scoring answers to open-ended questions and, more particularly, to automated systems and methods for distributing answers to qualified readers and facilitating the scoring thereof.
2. Description of Related Art
The automation of test scoring is a complex problem that has generated a great deal of interest, owing to a significant economic pressure to optimize efficiency and accuracy and to minimize human involvement. Optimal mark reading (OMR) systems are well known in the art, such as those for scanning forms having pencil marks within preprinted areas such as ovals. OMR systems generally sense data recorded within the preprinted areas by detecting light absorbed in the near infrared, which is referred to as NIR scanning. This method permits the differentiation of the pencil marks from the preprinted information, which is provided in a pigment that does not absorb in the NIR. OMR systems thus permit a gathering of data that is easily converted into digital form, scored against an answer database, and saved without consuming excessive storage space.
An additional level of complexity is added, however, with the inclusion of open-ended or essay-type questions. These questions must typically be scored by a human reader, and thus either the physical test form or a visible image thereof must be available for at least the time required for scoring. A digitally stored visible image can be obtained by an image processing apparatus, for example.
A multiplicity of systems and methods for addressing the scoring of test answer sheets have been disclosed in the art. For example, Poor (U.S. Pat. No. 5,452,379), Keogh et al. (U.S. Pat. No. 5,134,669), Clark and Clark et al. (U.S. Pat. Nos. 5,321,611; 5,433,615; 5,437,554; 5,458,493; 5,466,159; and 5,558,521) disclose systems and methods for combining OMR and image processing wherein only a predefined area of a document (an “area of interest”) is captured and stored.
Another aspect of the problem of processing test answer sheets having both multiple-choice and open-ended questions involves the scanning apparatus used to convert a written document into digital data. The use of combined OMR and image capture devices is disclosed by Poor '379, Keogh et al. '669, Clark et al. '554.
SUMMARY OF THE INVENTION
It is therefore an object of the present invention to provide a system and method for processing and scoring test answer sheets having both multiple-choice and open-ended questions.
It is another object to provide such a system and method that retains a full image of a test form so that it is retrievable by a scorer.
It is an additional object to provide such a system and method that captures OMR and image data in a unitary device.
It is a further object to provide such a system and method that obviates the need for trigger or timing marks on a test form.
It is yet another object to provide such a system and method that distributes answers for scoring to a qualified reader.
It is yet an additional object to provide a flexible system architecture for imaging test answer sheets, storing the images, and distributing the images to a qualified reader for scoring.
It is yet a further object to provide such a system and method that includes a tool for performing a geometric measurement upon a displayed image of an answer sheet.
These and other objects are provided by the imaging and scoring system and method of the present invention. The system includes integrated hardware elements and software processes for capturing optical mark and full visual images of an answer page, for storing the images, for retrieving the images, for distributing the visual images to a reader for scoring, for assisting the reader in scoring, and for monitoring the reader's performance.
The scanning system comprises means for sequentially advancing each page of a plurality of answer pages along a predetermined path. Positioned along the path are mark imaging means (OMR, optical mark recognition; OCR, optical character recognition) for capturing a location of an optical mark on each answer page and visual imaging means for capturing a full visual image of each answer page. A forms database in a server is provided that contains data on the physical location and type (e.g., multiple-choice or open-ended) of each answer on each page. Software means resident in the server operate with the forms database to determine whether the captured image contains an answer to an open-ended question. If such an open-ended answer is supposed to be found on the page being imaged, the full visual image of the page is stored.
In a particular embodiment the scanner further comprises means for aligning the page image without the use of timing or tracking marks. The aligning means comprises means for detecting a page edge, which is sufficient for pages having only open-ended answers.
The present invention further includes a system and method for distributing one of a batch of answer images to a reader for scoring. The answer images typically comprise open-ended answers such as are obtained from the scanning system and method as described above. Preferably each batch of answer images are from a common test, although this is not intended as a limitation.
The method comprises the steps of fetching a batch of answers to a test question from a storage device and placing them in a temporary cache. These fetching and temporary storing steps are preferably under the control of a server. This server contains a database associating each answer batch with a qualification required of a reader. Another database resident therein contains a list of qualifications possessed by each reader.
A reader who is in electronic communication with the cache indicates a readiness for scoring, and that reader's qualifications, which are resident in the server, permit the routing to the reader of one of an available batch of answers based upon predetermined criteria such as priority associated with a test to be scored. An answer image from an appropriate answer batch is electronically delivered to the reader's workstation for scoring. Once the scoring of that answer is complete, the server will distribute additional answer images to that reader until the batch is completely scored or the reader exits the system. Typically, a similarly qualified group of readers score answer images from the same batch.
The present invention additionally includes a system and method for displaying a test answer page to a reader for scoring. In this aspect, the page number for a particular test is used to access a forms layout database, which contains a location of the sector on which the open-ended question is expected to be found. The page image is then formatted to display that answer sector to the reader. Means are also provided for permitting access to the remainder of the page, such as by scrolling on a workstation screen, or to additional pages if the item answer covers multiple pages.
Formatting also comprises providing a scoring protocol for the answer and displaying commensurate indicia to the reader to assist in scoring. For example, a button bar can be displayed on a screen, an item of which can be selected for entering a score.
Another scoring facilitator available to the reader comprises a geometric measurement tool that can be superimposed on an answer and manipulated to provide an indication of how close to an “ideal” answer the student has come.
Scoring is also assisted by an electronic querying system and method, whereby a query is electronically transmitted to successively higher levels of supervisors until an answer can be obtained. The answer is then electronically relayed back through the same levels so that all intermediate personnel can benefit from the knowledge.
In order to monitor the scoring effectiveness of a reader, means are provided for transmitting a calibration answer for scoring. The reader is unaware that this is not another answer in the regular workflow queue. The score gra
Gonzalez Jose
Kucinski Bernard
Sonnenschein Joan
Allen Dyer Doppelt Milbrath & Gilchrist, P.A.
Harris Chanda
Martin-Wallace Valencia
The Psychological Corporation
LandOfFree
System and method for scoring test answer sheets having... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for scoring test answer sheets having..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for scoring test answer sheets having... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2603832