Education and demonstration – Question or problem eliciting response – Grading of response form
Reexamination Certificate
2000-07-24
2002-12-10
Cheng, Joe H. (Department: 3714)
Education and demonstration
Question or problem eliciting response
Grading of response form
C434S327000, C434S323000, C434S362000, C434S350000, C434S118000, C706S927000
Reexamination Certificate
active
06493536
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to systems and methods for teaching scoring and for assessing scoring effectiveness and, more particularly, to such systems and methods for teaching and assessing holistic scoring.
2. Description of Related Art
The automation of test scoring is a complex problem that has generated a great deal of interest, owing to a significant economic pressure to optimize efficiency and accuracy and to minimize human involvement. Open-ended or essay-type questions must typically be scored by a human reader, and thus either the physical test form or a visible image thereof must be available for at least the time required for scoring. In addition, scorers (also referred to as readers or resolvers) must be trained in order to become accomplished in analyzing and scoring the answers to open-ended questions effectively, accurately, and quickly.
Computerized systems for scoring open-ended questions are known in the art. In addition. such systems are known that provide feedback to a scorer on validity, reliability, and speed based upon a standard question and model answer. For example, Clark and Clark et al. (U.S. Pat. Nos. 5,321,611; 5,433,615; 5,437,554; 5,458,493; 5,466,159; and 5,558,521) disclose systems and methods for collaborative scoring, wherein scores of two or more resolvers are compared, and a record is kept of each of the resolver's scores. This group of patents also teach the collection of feedback on a resolver, which includes the monitoring of scoring validity, reliability, and speed. One of the criteria is a calculation of a deviation of the resolver's score and a model score by using “quality items.” Also discussed is an on line scoring guide for use by the resolver during scoring.
However, there are no systems and methods known in the art that are specifically directed to the teaching of scoring open-ended questions and to providing scoring rules; model answers. scores, and rationales therefor; and feedback to a resolver during the teaching process.
SUMMARY OF THE INVENTION
It is therefore an object of the present invention to provide a system and method for assessing a scorer's grading of an open-ended question.
It is an additional object to provide such a system and method for interactively assisting a scorer in learning a holistic scoring technique.
It is another object to provide such a system and method for tracking a scorer's progress during a practice session.
It is a further object to provide such a system and method for practicing holistic scoring in a variety of content domains such as, but not limited to, reading, writing, science, and mathematics in the same program.
It is yet another object to provide such a system and method for presenting a set of holistic scoring rules, or rubric, to the scorer.
These and other objects are achieved by the system and method of the present invention. One aspect of the method is for teaching a scorer holistically to score an answer to an open-ended question. Holistic scoring is a technique whereby a unitary, typically numerical, score is given for an answer to an open-ended question; for example, in an essay-type response, spelling and grammatical errors and content are all taken into account when granting the score.
The teaching method comprises the step of displaying a student response, which in a particular embodiment may be chosen by the scorer to be presented in handwritten or typed text form, to an open-ended question to a scorer. In a preferred embodiment the scorer is permitted to access for display a scoring rubric for the question, which comprises a set of rules on which the scoring for that question should be based. The scorer then assesses the response and enters a score for the response, which is received by the system. Finally, a model score is presented to the scorer. A comparison of the model score with the scorer's entered score permits him or her to assess his or her scoring efficacy, that is, how close the entered score came to the model score prescribed for the response.
The tutorial software program of the present invention, which may also be referred to simply as a tutorial, in a preferred embodiment comprises a plurality of databases, or. alternatively, a plurality of sectors in a unitary database, containing:
1. A plurality of student responses to an open-ended question. Preferably, each student response is present in an original handwritten image form and in a text form. The text form retains all original errors from the handwritten image.
2. A model score for each student response.
3. A scoring rubric for each question.
4. An analysis of each student response and a rationale for the model score for each student response.
The teaching system of the present invention comprises a computer, or data-processing system, such as, for example, a personal computer or workstation. The computer has resident therein, or has means for communicating with a storage device having resident thereon, a database as described above.
The system also comprises means for displaying a student response to a question to a scorer, means for permitting the scorer to access the scoring rubric for the question, means for receiving a score from the scorer. As described above, these means typically include a personal computer or networked workstation having a keyboard, screen, pointing device, and communication means for accessing a storage device.
Software means are also resident in the computer for presenting on the display means a model score to the scorer to permit the scorer to assess his or her scoring efficacy, that is, how close the assigned score is to the model score. The software means also comprises means for displaying an explanation or annotation of the model score assigned. In addition, means are provided within the processor for tracking the scorer's progress during a practice session with the tutorial. This tracking is preferably accomplished by calculating a running correlation between the model answer and the score entered for each response.
The invention contemplates a system and method for teaching a scorer within a chosen level and discipline. For example, a particular tutorial may comprise a set of questions keyed to a grade level in a particular subject area (e.g., grade 7, history) or in related areas (e.g., grade 8, reading and writing, wherein reading competency is assessed by a student's response to a question on a reading selection, and writing competency is assessed by the student's response to an essay-type question). Alternatively, a set of responses to questions may address the subject matter contained in a professional licensing or qualification examination (e.g., for a laboratory technician).
The features that characterize the invention, both as to organization and method of operation, together with further objects and advantages thereof, will be better understood from the following description used in conjunction with the accompanying drawing. It is to be expressly understood that the drawing is for the purpose of illustration and description and is not intended as a definition of the limits of the invention. These and other objects attained, and advantages offered. by the present invention will become more fully apparent as the description that now follows is read in conjunction with the accompanying drawing.
REFERENCES:
patent: 4958284 (1990-09-01), Bishop et al.
patent: 5035625 (1991-07-01), Munson et al.
patent: 5059127 (1991-10-01), Lewis et al.
patent: 5259766 (1993-11-01), Sack et al.
patent: 5321611 (1994-06-01), Clark et al.
patent: 5433615 (1995-07-01), Clark
patent: 5437554 (1995-08-01), Clark et al.
patent: 5458493 (1995-10-01), Clark et al.
patent: 5466159 (1995-11-01), Clark et al.
patent: 5558521 (1996-09-01), Clark et al.
patent: 5658161 (1997-08-01), Roberts et al.
patent: 5672060 (1997-09-01), Poor
patent: 5690497 (1997-11-01), Clark et al.
patent: RE36028 (1999-01-01), Deesen et al.
patent: 5978648 (1999-11-01), George et al.
patent: 6120299 (2000-09-01), Trenh
Jongsma Eugene A.
Mellott David K.
Allen Dyer Doppelt Milbrath & Gilchrist, P.A.
Cheng Joe H.
Harris Chanda
The Psychological Corporation
LandOfFree
Computerized system and method for teaching and assessing... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Computerized system and method for teaching and assessing..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Computerized system and method for teaching and assessing... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2937917