Education and demonstration – Psychology
Reexamination Certificate
2002-03-15
2004-07-27
Walberg, Teresa (Department: 3713)
Education and demonstration
Psychology
C434S322000, C705S014270
Reexamination Certificate
active
06767213
ABSTRACT:
BACKGROUND OF THE INVENTION
For many years, a primary method for assessing the cognitive abilities of individuals has been by means of paper and pencil tests. However, paper and pencil tests have limited flexibility, do not present realistic ways of responding, do not provide immediate feedback of an examinee's performance, and are costly with respect to scoring time and personnel requirements to administer them. Additionally, many situations exist in which such tests are inappropriate or have severe limitations for assessing higher level cognitive processes. There is a need, for example, for better methods of assessment of the cognitive and metacognitive skills required by managers and executives, especially those at the upper levels of organizations.
Such assessments are needed to identify executive leadership potential in assessment and promotion of managers and as a basis for executive development and training programs. Until recently, these skills were not fully identified or sufficiently validated. Also, measurement of such skills has been limited in the kinds and variety of responses appropriate to high level strategic decision making and problem solving. Recent research has identified the cognitive and metacognitive skills related to effective leadership in complex organizations, for example general problem solving, planning and implementation, solution construction, solution evaluation, social judgment and metacognitive processes. See, for example,
Development And Evaluation Of Cognitive And Metacognitive Measures For Predicting Leadership Potential,
Joanne C. Marshall-Mies and Edwin A. Fleishman et al., Leadership Quarterly, 11(1), 135-153. However, no testing methods and computerized means for measuring these cognitive and metacognitive skills related to effective leadership are available. What are needed are a computerized testing device for and method of assessing cognitive and metacognitive capabilities.
SUMMARY OF THE INVENTION
The objective of this invention is to provide a computer-based cognitive and metacognitive skill assessment and development tool suitable for implementation in supporting executive development. Current research on performance at mid and top levels of large scale organizations strongly suggests that cognitive and metacognitive skills are required for successful executive-level performance. Cognitive conceptual (integrative) skills are very important. However, the cognitive skill of self-awareness (metacognition) is critical, in that it involves the self-generated feedback and iterative evaluation process used in complex problem solving and decision-making.
The invention of the Metacognitive Leadership Exercises (MLE) assesses cognitive and metacognitive skills used in unstructured problem solving skills central to leader effectiveness. This is the primary reason for the existence of the invention. Use of this invention will provide critical information related to effective organizational management requirements.
The first task is to design the MLE measures to be appropriate for the proposed management population and to focus on strategic leadership and decision-making. Use of scenarios, prompts, responses, and a unique scoring methodology resulted in a computer-based assessment battery including six metacognitive measures (General Problem Solving, Planning & Implementation, Solution Construction, Solution Evaluation, Social Judgment, and Metacognitive Process) and a background questionnaire.
The second task was to demonstrate the reliability and construct- and criterion-related validity of the MLE measures. This involves an experimental study relating the MLE scores, a performance criterion measure, measures of related constructs (correlates), and collecting and analyzing the data. The study evaluated six MLE measures and a background questionnaire, a performance criterion, and three sets of correlates, i.e., the Modified Career Path Appreciation Scale or MCPA, seven MANSPEC variables, and four Myers-Briggs Type Indicator® (MBTI®) scores.
In using the MLE, the examiner is asked to participate via written materials and distribution of a disk which installs the MLE instructions and exercises, collects and scores the data, and installs the software. Participants complete the MLE using IBM-compatible computers with Microsoft Windows operating systems, in a job related validation study conducted at the National Defense University. Internal and external analyses are preferably conducted to determine the sample representativeness and the reliability and construct- and criterion-related validity of the MLE measures. The results are as follows.
Self-administration of the MLE on a variety of pcs and laptops running Windows and Unix was successful. Over 96 percent of the participants had no technical difficulties executing or completing the MLE measures, and over 94 percent made no comments or made positive comments about their experience with the MLE instructions and exercises.
The multiple-item MLE measures were reliable or internally consistent as evidenced by moderate to high alpha coefficients of 0.53 to 0.84 for Average Quality on the General Problem Solving, Social Judgment, and Metacognitive Process measures and 0.85 for Maximum Quality on the Metacognitive Process measure. The high level managerial population places severe time constraints on the data collection. Applying the Spearman-Brown correction formula, it is estimated that reliability would be increased significantly if the measures were doubled, i.e., to a range of 0.70 to 0.91 for General Problem Solving and Social Judgment and to a range of 0.89 to 96 for the highly reliable Metacognitive Process measure. These results reveal that individual differences in scores obtained by high level executives produce consistent results on these measures of metacognitive skill.
The MLE score distributions indicate that senior level individuals differ in performance on the MLE measures; the large standard deviations indicated a considerable range of scores for any given measure. These findings, especially given the unique sample, confirmed that the MLE scores reflect individual differences in the skills measured. Also, relatively low to moderate inter-correlations among component MLE measure scores confirmed partial independence of the measures and provided evidence that the measures tap different dimensions of problem solving associated with leadership skills.
MLE measures are slightly to moderately related to several correlates, showing a convergence between the constructs measured. Of particular interest is the significant but moderate correlation of General Problem Solving and Social Judgment with the MCPA, a work sample which taps work disposition as articulated by the subject and judged by the interviewer and which shows how discretion is exercised and how the person copes with complexity. It purports to measure fluid intelligence, concept formation, insight/intuition, comfort with ambiguity/uncertainty, and characteristic strategies and feelings about work. Also of interest is the correlation of General Problem Solving and Social Judgment with Analytical Skills, a Holland Vocational Preference Inventory score which indicates a preference for using one's analytical and problem solving abilities and for using intelligence to achieve ends.
The MLE General Problem Solving and Social Judgment measures provide positive correlations with MCPA and Analytical Skills. Two multiple regression analyses are conducted using the MCPA as the dependent variable. The first is a simultaneous multiple regression with General Problem Solving and Social Judgment as independent variables. Together, the MLE measures show a multiple correlation of 0.38 (p<0.01), with General Problem Solving representing unique variance in predicting the MCPA. The second regression enters Analytical Skills first, followed by the simultaneous addition of the MLE scores. This results in a multiple correlation of 0.60 (p<0.01), with Analytical Skills representing unique variance in predicting the MCPA and General P
Christman Kathleen M.
Dykema Gossett PLLC
Management Research Institute, Inc.
Walberg Teresa
LandOfFree
System and method for assessing organizational leadership... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for assessing organizational leadership..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for assessing organizational leadership... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3219296