Education and demonstration – Question or problem eliciting response
Reexamination Certificate
2003-04-29
2004-06-15
Cheng, Joe H. (Department: 3713)
Education and demonstration
Question or problem eliciting response
C434S118000, C434S362000
Reexamination Certificate
active
06749435
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to a system for processing answers to test questions.
BACKGROUND OF THE INVENTION
The scoring of test answer sheets involves complex problems. These test answer sheets typically include a series of response positions such as, for example, “bubbles,” ovals, or rectangles. A person taking a test would, for example, darken in an appropriate oval with a pencil to answer a multiple choice question. These test answer sheets may also include handwritten answers, such as essay or short answer questions. Systems for scanning and scoring the bubbles on such answer sheets are known in the art. Increased difficulties are encountered, however, when such answer sheets either include other types of answers, such as handwritten answers, or cannot be machine graded. For example, if the student has failed to include his or her name on the test answer sheet, the system may be unable to machine score the test answer.
The goals in scoring test answers that cannot be machine scored include efficiency and consistency. These test answer sheets are typically scored by test resolvers either by manually scoring the physical test answer sheet or scoring an electronic representation of the test answer sheet on a computer. Ideally, the scores provided by the various test resolvers for a particular test question should be consistent, since the scores are used in comparing performance of the students against one another. In addition, a test resolver should ideally work efficiently so as to maintain consistently high scoring rates. The test resolver should not have such a high scoring rate that the consistency or quality of scoring significantly declines; likewise, the test resolver should not have such a low scoring rate that the too few answer sheets are being scored. This manual scoring of test answer sheets, however, makes it difficult to monitor the consistency of scoring among the various test resolvers.
In many situations, test resolvers actually travel to a particular location so that all test resolvers may simultaneously score test answer sheets. Requiring the test resolvers to travel to a given location is inconvenient for the resolvers and expensive for those who administer the tests. Furthermore, tracking the performance of test resolvers against both their own performance and the performance of other resolvers can be very difficult with a manual scoring environment.
The process of resolving test questions is currently done manually, and this presents problems. A resolver is manually presented with the actual test answer sheets for scoring. This process is relatively inefficient, since the resolvers must score the answer sheets one at a time and in the order in which they are presented. Also, manual scoring systems do not have the capability to efficiently gather and categorize the test answers for subsequent analysis. Therefore, with a manual system it is very difficult to determine how teaching methods should be changed to decrease, for example, the number of incorrect answers.
A need thus exists for a system that promotes and achieves consistency and efficiency in scoring or resolving of tests.
SUMMARY OF THE INVENTION
The present evaluator management system and method facilitates consistent, accurate, and high quality scoring of data items. A collaborative scoring system and method allows two evaluators to score the same data item and resolves potential discrepancies in the scores provided by the evaluators.
The system also has the capability to use quality items to monitor a resolver's scoring performance. The system transmits a quality-control test item, having a known score, to a test resolver. After receiving a score from the resolver, the system compares the known score with the resolver's score and can generate a quality profile for the resolver.
The system also provides performance feedback to test resolvers by monitoring the scoring rates of the resolvers. If a resolver's scoring rate, in comparison to another scoring rate, contains a discrepancy, the system can report the discrepancy. The system can also recommend breaks to a resolver, or provide a resolver with diversionary activities, in response to a discrepancy in scoring rates. The system also has the capability to route test answers to resolvers based on classifications of resolvers and persons who generated the test answers. Monitoring and selecting resolvers based on classifications promotes fair and consistent scoring of test answers.
Finally, the system provides for an on-line scoring guide to dynamically provide a test resolver with scoring rules for a test answer displayed to the resolver.
REFERENCES:
patent: 3405457 (1968-10-01), Bitzer
patent: 3538626 (1970-11-01), Frank
patent: 3762072 (1973-10-01), From
patent: 3932948 (1976-01-01), Goddard et al.
patent: 4004354 (1977-01-01), Yamauchi
patent: 4151659 (1979-05-01), Lien et al.
patent: 4205780 (1980-06-01), Burns et al.
patent: 4478584 (1984-10-01), Kaney
patent: 4518267 (1985-05-01), Hepp
patent: 4518361 (1985-05-01), Conway
patent: 4553261 (1985-11-01), Froessl
patent: 4627818 (1986-12-01), Von Fellenberg
patent: 4648062 (1987-03-01), Johnson et al.
patent: 4671772 (1987-06-01), Slade et al.
patent: 4694352 (1987-09-01), Ina et al.
patent: 4705479 (1987-11-01), Maron
patent: 4708503 (1987-11-01), Poor
patent: 4715818 (1987-12-01), Shapiro et al.
patent: 4741047 (1988-04-01), Sharpe, II
patent: 4760246 (1988-07-01), Shepard
patent: 4764120 (1988-08-01), Griffin et al.
patent: 4789543 (1988-12-01), Linder
patent: 4798543 (1989-01-01), Spiece
patent: 4845739 (1989-07-01), Katz
patent: 4867685 (1989-09-01), Brush et al.
patent: 4878175 (1989-10-01), Norden-Paul et al.
patent: 4895518 (1990-01-01), Arnold et al.
patent: 4908759 (1990-03-01), Alexander, Jr. et al.
patent: 4930077 (1990-05-01), Fan
patent: 4937439 (1990-06-01), Wanninger et al.
patent: 4958284 (1990-09-01), Bishop et al.
patent: 4978305 (1990-12-01), Kraft
patent: 4996642 (1991-02-01), Hey
patent: 5002491 (1991-03-01), Abrahamson et al.
patent: 5003613 (1991-03-01), Lovelady et al.
patent: 5011413 (1991-04-01), Ferris et al.
patent: 5023435 (1991-06-01), Deniger
patent: 5035625 (1991-07-01), Munson et al.
patent: 5038392 (1991-08-01), Morris et al.
patent: 5054096 (1991-10-01), Beizer
patent: 5058185 (1991-10-01), Morris et al.
patent: 5059127 (1991-10-01), Lewis et al.
patent: 5072383 (1991-12-01), Brimm et al.
patent: 5086385 (1992-02-01), Launey et al.
patent: 5100329 (1992-03-01), Deesen et al.
patent: 5101447 (1992-03-01), Sokoloff et al.
patent: 5103490 (1992-04-01), McMillin
patent: 5105354 (1992-04-01), Nishimura
patent: 5119433 (1992-06-01), Will
patent: 5134669 (1992-07-01), Keogh et al.
patent: 5140650 (1992-08-01), Casey et al.
patent: 5147205 (1992-09-01), Gross et al.
patent: 5151948 (1992-09-01), Lyke et al.
patent: 5176520 (1993-01-01), Hamilton
patent: 5180309 (1993-01-01), Egnor
patent: 5195033 (1993-03-01), Samph et al.
patent: 5204813 (1993-04-01), Samph et al.
patent: 5211564 (1993-05-01), Martinez et al.
patent: 5258855 (1993-11-01), Lech et al.
patent: 5259766 (1993-11-01), Sack et al.
patent: 5261823 (1993-11-01), Kurokawa
patent: RE34476 (1993-12-01), Norwood
patent: 5267865 (1993-12-01), Lee et al.
patent: 5294229 (1994-03-01), Hartzell et al.
patent: 5302132 (1994-04-01), Corder
patent: 5310349 (1994-05-01), Daniels et al.
patent: 5318450 (1994-06-01), Carver
patent: 5321611 (1994-06-01), Clark et al.
patent: 5344132 (1994-09-01), LeBrun et al.
patent: 5376007 (1994-12-01), Zirm
patent: 5379213 (1995-01-01), Derks
patent: 5387104 (1995-02-01), Corder
patent: 5418865 (1995-05-01), Bloomberg
patent: 5433615 (1995-07-01), Clark
patent: 5437554 (1995-08-01), Clark et al.
patent: 5437555 (1995-08-01), Ziv-El
patent: 5452379 (1995-09-01), Poor
patent: 5458493 (1995-10-01), Clark et al.
patent: 5466159 (1995-11-01), Clark et al.
patent: 5496175 (1996-03-01), Oyama et al.
patent: 5544255 (1996-08-01), Smithies et al.
patent: 5558521 (1996-09-01), Clark et al.
patent: 5565316 (1996-10-01), Kershaw et al.
patent: 5596698
Bramlett, Jr. William L.
Clark Jay V.
Moyer Judith
Cheng Joe H.
Merchant & Gould P.C.
NCS Pearson, Inc.
LandOfFree
Collaborative and quality control scoring system and method does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Collaborative and quality control scoring system and method, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Collaborative and quality control scoring system and method will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3356762