Education and demonstration – Question or problem eliciting response
Reexamination Certificate
2001-03-08
2004-02-10
Walberg, Teresa (Department: 3713)
Education and demonstration
Question or problem eliciting response
C434S30700R
Reexamination Certificate
active
06688889
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to a computerized, or other machine based test preparation system, and more particularly, to a method and apparatus for enhancing learning and improving examinee scores on standardized exams through the use of individually tailored diagnostics and remediation.
BACKGROUND OF THE INVENTION
1. The Proliferation of “High-stakes” Examinations and Conventional Test Preparation Methods
“High-stakes” examinations are very common today. Typically they are time-based exams testing a set of predetermined subject areas. A number of these examinations, such as the Scholastic Aptitude Test (“SAT”) series of examinations and others like it (e.g., MCAT, LSAT, ACT, GRE, GED, CLEP, BAR exam, DMV exams), have been labeled as “high-stakes” testing. In such “high-stakes” tests, the primary objective is the placement of an examinee on a latent trait or ability dimension, for a variety of purposes such as selection and placement (i.e., SAT, MCAT or LSAT), or certification (i.e., GED and DMV exams). Most of these tests include items from a variety of scholastic domains (e.g., SAT: verbal, mathematics; LSAT: logical reasoning, reading comprehension, verbal) that are arranged in a formal structure. The test items are chosen and developed by the test makers so as to “reliably” place examinees on the latent dimension of interest to the examiner and consumers of the standardized scores from such exams.
One factor which is thought to be capable of influencing examinee performance on these tests is coaching, or formal test preparation efforts. Because of the proliferation of these “high-stakes” examinations, an entire test preparation industry has arisen to help prepare examinees and improve their scores on these exams. Offerings include classroom-based tutoring, stand-alone printed publications, and computer-based materials (e.g., disk, CD-ROM, internet). All of these offerings claim to be able to increase an examinee's score on the particular standardized exams to which they are directed.
The most conventional test preparation offerings have been traditionally represented by such organizations as Kaplan Learning and The Princeton Review study centers, or self-study methods based on printed test preparation texts such as 10
Real SATS
, and
Gruber's Complete Preparation for the New SAT: Eighth Edition
. Through the use of such methods, examinee score increases have been modest, generally resulting in score increases on the order of ⅕th of a standard deviation.
More recently, computer based exam preparation materials have been developed and offered including The College Board's
One
-
on
-
One with the SAT
, The Princeton Review's
Inside the SAT, ACT
&
PSAT
, published by The Learning Company, and
The Crash Course for SAT, PSAT
&
ACT
, published by ARCO Publishing. Additionally, some of the testing centers mentioned above have begun offering computerized training materials generally corresponding to their traditional classroom based approach.
Common characteristics of these computerized offerings include: (i) presentation of timed “sample exams” and practice exams, (ii) scoring of responses from these exams, (iii) some question-specific feedback (e.g., response chosen, correct answer, brief explanation), and (iv) general test-taking tips (e.g., pacing, skipping questions). Features which differentiate these offerings include: (i) the use or non-use of audio and/or graphics, (ii) the ability to mark items to be skipped and returned to, (iii) feedback of a study plan based upon the results of a “sample exam”, and (iv) the provision of explanations for each of the response alternatives for each item.
Several of these computerized offerings have been distributed over the internet. Some of the web sites offering exam preparation and review include: (i) Score.Kaplan.com (based on materials offered by Kaplan Learning), (ii) Review.com, (iii) Testprep.com, (iv) ACTive Prep at Act.org, (v) powerprep.com, and (vi) Novanet.com (based on materials offered by The Princeton Review). A review of these web sites as they existed in November, 2000 revealed variations in complexity from “page-turners” to relatively complete implementations of the printed volumes on which some of them are based. In general though, they reflect the same range of complexity and operation as found in the other computerized and CD-ROM offerings discussed.
Several of the web-based offerings also provide “sample exams” which can be taken by the user. Information is generally fed back to the user of such offerings in the form of raw and scaled scores. In some cases, the feedback may also include a re-presentation of the exam items, the indication of the user's response and the correct choice, and an explanation of why the correct answer is correct and why each of the alternatives are wrong. While responses to the “sample exams” in some cases provide the basis for “diagnostic” feedback, the diagnosis in this context is defined from a conventional testing perspective and is determined merely by the number of incorrect answers rather than the types of incorrect answers. Thus, a study plan, or diagnosis, if provided, is usually based upon the user's distribution of scores across the various sections of the examination and results in a simplistic recommendation of remediation, such as the need to review geometric principles or increase vocabulary.
2. Recent Development in Cognitive Diagnostic Assessments
Educators and researchers, influenced by recent developments in cognitive psychology and societal concerns regarding the influence of testing on equality of education, have sought testing instruments that would reveal the mechanisms, structures and processes that are activated when an examinee takes a test, and thus, would inform the instructional process. Conventional tests, while adequately serving as selection and/or placement instruments, are not well suited for determining a course of instruction or for identifying the source of problem-solving errors.
A category of testing called cognitively diagnostic assessment (“CDA”) or dynamic testing has been developed which may provide a basis for individualized instruction for each examinee in a domain of interest. Such tests are based upon cognitive theories of learning, and as such, are not concerned with the representative sampling of items from a content domain (such as algebraic equations), but rather, with the examinee's knowledge and application of cognitive attributes which are thought to be required or not required to adequately solve a given problem. CDA testing provides information regarding the strategies that examinees use to attack problems, relationships they perceive among concepts, and principles they understand in a domain. The goal of these testing methods is to determine, on the basis of a simple test, what the strengths and weaknesses of an examinee are, relative to a specified list of cognitive attributes of interest to the teacher and the tester.
CDA-type tests are typically built around an attribute by item matrix (i.e., a Q-matrix). Thus, for an examinee to solve a given problem, it is assumed that they have knowledge of, and the ability to apply, one or more cognitive attributes related to the item or problem. The failure of an examinee to solve a problem is then attributed to the absence of a requisite cognitive attribute or to a lack of skill in its application.
The major difficulty experienced with most CDA tests is one of numerosity—the number of possible sources of error grows exponentially as the number of attributes and the number of items increase. For example, some attempts by researchers to form a Q-matrix for 60 items on the SAT math test yielded more than 3,000 prototypical error patterns. Other researchers developed models containing only 4 attributes—strategy, completeness, positivity and slips—which were proposed to be evocative of properties that could be uses in developing and interpreting diagnostic assessment tests. An evaluation of all of these models revealed t
Detwiler Susan
Flaherty John
Flaherty Michael
Huss Todd
Roy David
Boostmyscore.com
Christman Kathleen M
Orrick Herrington & Sutcliffe LLP
Walberg Teresa
LandOfFree
Computerized test preparation system employing individually... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Computerized test preparation system employing individually..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Computerized test preparation system employing individually... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3336413