Latent property diagnosing procedure

Details Latent property diagnosing procedure Latent property diagnosing procedure

2001-04-20

2004-12-14

Fernstrom, Kurt (Department: 3712)

Education and demonstration

Question or problem eliciting response

Grading of response form

C434S354000

Reexamination Certificate

active

06832069

ABSTRACT:

BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention provides a method of doing cognitive, medical and psychiatric, and diagnosis in general of latent properties of objects that are usually people using binary scored probing of the objects.
2. Description of the Prior Art
Part 1: Background Prerequisite to Description of Prior Art
Standardized Testing as Currently Practiced; Cognitive Diagnosis Defined
Before describing the prior art related to the invention, it is necessary to discuss needed background material. Both large scale standardized testing and classroom testing typically use test scores to rank and/or locate examinees on a single scale. This scale is usually interpreted as ability or achievement in a particular content area such as algebra or the physics of motion. Indeed, the two almost universally used approaches to “scoring” standardized tests, namely classical test theory (Lord, F. and Novick, M.,1968
, Statistical Theories of Mental Test Score
, Reading, Mass., Addison Wesley—although an ancient book, still the authority on classical test theory) and “unidimensional” item response theory (IRT), assign each examinee a single test score. An “item” is merely terminology for a test question. The standardized test score is usually the number correct on the test, but can include in its determination partial credit on some items, or the weighting of some items more than others. In classroom testing, teachers also typically assign a single score to a test.
The result of this single score approach to testing is that the test is only used either to rank examinees among themselves or, if mastery standards are set, to establish examinee levels of overall mastery of the content domain of the test. In particular, it is not used to produce a finely grained profile of examinee “cognitive attributes” within a single content domain. That is, an algebra test can be used to assess John's overall algebra skill level relative to others or relative to the standard for algebra mastery but it cannot determine cognitive attribute mastery, such as whether John factors polynomials well, understands the rules of exponents, understands the quadratic formula, etc., even though such fine grained analyses are clearly to be desired by instructor, student, parent, institution, and government agency, alike.
Herein, cognitive diagnosis refers to providing fine-grained profiles of examinee cognitive attribute mastery
on-mastery.
Statistical Method or Analysis
The cognitive diagnostic algorithm that forms the core of the invention is a particular statistical method. A statistical method or analysis combines collected data and an appropriate probability model of the real world setting producing the data to make inferences (draw conclusions). Such inferences often lead to actual decision-making. For instance, the cognitive diagnosis indicating that Tanya is deficient on her mastery of the quadratic formula can be followed up by providing remediation to improve her understanding of the quadratic formula.
To clarify what a statistical method is, an overly simple, non-cognitive example is illustrative. As background, it seems worth noting that a valuable aspect of statistical methods is that they explicitly state the inherent error or uncertainty in their inferences. In particular, a valid statistical analysis is careful not to draw inferences that go beyond what is reasonably certain based on the available information in the data, accomplishing this by including a measure of the uncertainty associated with the inference, such as providing the standard error, a fundamental statistical concept. As such, this makes any statistical method for doing cognitive diagnosis superior to any deterministic model based method (variously called rule-based, artificial intelligence, data-mining, etc., depending on the particular deterministic approach taken).
The difference between a deterministic inference and a statistical inference is illustrated in a simple setting. A coin is claimed to be loaded in favor of coming up heads. It is tossed 10 times and produces 7 heads. The non-statistical, deterministic approach with its inherent failure to address possible inference error or uncertainty simply reports that the inferred probability p of heads is 0.7 and hence concludes that the claim is true. The statistical approach reports that even though the most likely probability p of heads is indeed 0.7, nonetheless, because of the uncertainty of this inference due to the very limited amount of data available, all that can really be confidently predicted is that 0.348≦p≦0.933. Thus from the statistical inference perspective, there is not strong evidence that the coin is unfair. This statistical perspective of appropriate caution is the superior way to proceed.
Similarly, cognitive diagnoses using the Unified Model (UM) discussed hereafter will only assign attribute mastery or attribute non-mastery to an examinee for a particular attribute when the examinee test data provides strong evidence supporting the particular conclusion drawn, like Jack's mastery of the algebraic rules of exponents.
Now a non-cognitive example of a statistical method in more detail than the illustration above is given.
EXAMPLE 1
A drug with unknown cure probability p (a number between 0 and 1) is administered to 40 ill patients. The result is that 30 are cured. The standard binomial probability model is assumed (that is, it is assumed the patients respond independently from one another and there is the same probability of cure for each patient). Based on this model and the data, it is statistically inferred from the mathematical properties of the binomial probability model that the actual cure rate is p=0.75 with confidence that the error in this estimate is less than ±0.14. Thus, the inference to be drawn, based on this limited amount of data, is that p lies in the interval (0.60,0.89). By contrast, if there were 400 patients in the drug trial (much more data, that is) with 300 cures occurring, then it would be inferred p=0.75 as before, but now with much more precise confidence that the estimation error is less than ±0.04. More data provides more confidence that the inherent uncertainty in the inference is small.
Educational Measurement, Item Response Theory (IRT), and the Need for Educational Measurement/IRT-based Cognitive Diagnostic Models
The current paradigm that dominates probability modeling of educational test data is item response theory (Embretson, S. and Reise, S. (2000)
Item Response Theory for Psychologists
. Mahwah, N.J., Lawrence Erlbaum). This assigns a probability of getting an item right to be a function of a single postulated latent (unobservable) ability variable, always interpreted as a relatively broad and coarse-grained ability like algebra ability. Different examinees are postulated to possess different levels of this latent ability. Since the higher the level the greater the probability of getting the item right, it is justified to call this latent variable “ability”.
FIG. 1
shows the standard logistic item response function (IRF) of an item as a function of ability &thgr;. Each such function provides P(&thgr;)=probability of getting an item right for a typical examinee of ability &thgr;.
Typically, as herein, the scale for examinee ability is such that ability less than −2 indicates very low ability examinees (the lowest 2.5%), 0 indicates an average ability examinee and above 2 indicates very high ability examinees (the highest 2.5%). IRT based statistical methods are currently heavily used in educational measurement to statistically assess (infer from test data and the IRT model) examinee latent ability levels.
Educational measurement is the applied statistical science that uses probability models and statistical methods to analyze educational data (often test data) to provide information about learning processes and about various educational settings and to evaluate individual level and group level (state, school district, nation, etc.) in

Affiliated with

Also associated with

Rating

Latent property diagnosing procedure does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Latent property diagnosing procedure, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Latent property diagnosing procedure will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3317398