Watson-Glaser™ II Critical Thinking Appraisal

FAQ

Overview

What is the Watson-Glaser II?

• Two 40-item short forms, Forms D and E
• Three reports, including:
• Profile Report
• Interview Report
• Development Report
• Technical Manual

How is the new Watson-Glaser II different from the previous edition?

Enhancements include:

• More contemporary and business-relevant items
• Better face validity and applicability of items for individuals from countries other than the United States
• Inclusion of a higher proportion of difficult items to better separate individuals along a continuum of critical thinking
• Development of two 40-item forms that can be administered in approximately the same time as the previous Short Form, while discriminating among candidates as effectively as the older 80-item forms.
• New reports, including an updated basic Profile Report, Interview Report, and Development Report
• Interpretable subtest scores that provide information about three critical thinking skill domains—the ability to Recognize Assumptions, Evaluate Arguments, and Draw Conclusions.

For a more complete analysis, see our comparison page.

What does the Watson-Glaser II assess and what is the RED Model?

The Watson-Glaser II assesses overall critical thinking, as well as three important components of critical thinking, known as the "RED Model". Components of the RED Model include the ability to Recognize assumptions, Evaluate the merit of arguments or information that is presented, and Draw appropriate conclusions based on the evidence available.

For what type or level of candidates can I use the Watson-Glaser II for?

The Watson-Glaser II is appropriate for use with individuals employed in professional positions (e.g., executives, individual contributors) and with college undergraduate and graduate students.

Purchasing

Can I get credit for unused online administrations for previous Watson-Glaser forms and put it toward the purchase of Watson-Glaser II online administrations?

Yes; please contact your Pearson Talent Assessment Account Manager (1.888.298.6227) for more information.

If I only want a Profile report now, can I go back later and generate additional types of reports (e.g., Interview, Development)?

Information for one report cannot be used to generate other types of reports later; the type of report needed must be specified in advance of assessment administration.

Are there paper versions available for the Watson-Glaser II?

Paper versions for Watson-Glaser II Form D and E are available by special request by calling Pearson (1.888.298.6227.)

Are there international forms available for the Watson-Glaser II?

• The planned release date for Pan-American Spanish and other forms is late 2009.
• A Pan-American Spanish version of the Watson-Glaser Short Form will be available through TalentLens.com beginning in June, 2009.

Administration

How long does the Watson-Glaser II take to administer?

The test is untimed, but most candidates finish in approximately 35 minutes.

What is the reading level of the Watson-Glaser II?

Watson-Glaser II items were written at or below the 9th-grade reading level.

Norms

What norms are available and where can I get more information about the types of individuals included in each norm group?

• 14 norm groups for different positions and occupations are currently available; these norms will be updated and added to regularly.

Can I use my existing Watson-Glaser norms with the new forms?

• Yes, equating tables and instructions for their use are provided in the Watson-Glaser II Technical Manual for all forms.

Scoring

Why do Watson-Glaser II subscale scores sometimes appear to be inconsistent with the overall score? For example, it appears possible that someone can have a relatively low overall percentile score (e.g., 12th percentile or lower) and yet obtain "Average" subscale scores.

The Watson-Glaser II total and subscale scores are reported on different scales, which may result in what appear to be inconsistencies among them. This reporting approach was adopted because the subscales contain fewer items and generally have lower reliability than the total score, and because the subscales are used for development rather than selection.

The Watson-Glaser II subscale scores are reported as "Low," "Average," or "High." "Low" subscale scores are set conservatively so that examinees must score in approximately the bottom 11% of scores on the subscale. As an example, this would equate to a score of 0-3 items correct on the 12-item Recognize Assumptions scale, using the Executive norm group.

The Watson-Glaser II total score is based on all 40 Watson-Glaser II items. Each possible total raw score (0–40 items correct) is converted to and reported as a total percentile score between 1 and 100.

The trade off of setting subscale scores conservatively, so that only the bottom 11% of scores are reported as "Low," is that some examinees may be reported as "Average" on each of the three subscales, even when their total percentile score (based on their performance across the scales) appears low (e.g., total percentile score of 12). When making hiring or other selection decisions, it is essential to focus on the total percentile score, as this score is based on more items and is generally more reliable than the subscale scores.

If a person misses all 12 Recognize Assumptions items, but appears to do well on the Evaluate Arguments and Draw Conclusions subscales, what interpretation can I make?

In less than 1% of all cases, an examinee will not get any of the 12 Recognize Assumptions subscale items correct. Because the chances of getting all 12 items incorrect on this scale are extremely low, a score of "0" most likely indicates that the candidate lacks abilities or skills in this area, or misunderstood the directions or answer choices. Carefully examine the candidate’s scores on the other two scales to get a better understanding of which is more likely. For example, for a candidate who scored "0" on Recognize Assumptions and "Low" on the other two subscales, the "0" is more likely to indicate a lack of the abilities or skills necessary to effectively recognize assumptions.

For a candidate who scored "0" on Recognize Assumptions and "High" on the other two subscales, the "0" more likely indicates a misunderstanding of the directions for Recognize Assumptions, or other potential issues unrelated to ability or skill level. One important reason for this interpretation is that effectiveness in recognizing assumptions is necessary for effectiveness in drawing conclusions.

Sample Items

Inference

The CEO of a large manufacturer of office equipment (ICO) gave a speech in a private meeting of wealthy investors: "In order for ICO to meet its annual revenue goals, our government must let other currencies strengthen against our own. The cost of our nation’s exports is reducing demand for our products overseas." Later that week, she gave a speech to all the employees at one of the ICO factories. She was asked more than once, "Do you think that my job is safe in the near future?" The CEO replied, "Yes, this year’s revenues are lower than expected, but I am confident that ICO will see stronger revenues next year."

Most of the jobs at ICO are dependent on selling products in ICO’s domestic market.

Answer: Insufficient Data

Recognizing Assumptions

Corporations collecting and selling data about individuals’ Internet activities will soon make it impossible for Internet users to keep personal information private.

Right now, not all major corporations collect and sell the personal information of Internet users.

Answer: Assumption Not Made

Deduction

Thirty years ago, there was no fluoridated public water anywhere in Javier Province. Most cities in the province had rates of tooth decay above the national average. Today most cities in Javier Province have rates of tooth decay below the national average, but all the cities without fluoridated public water have rates above the national average.

In Javier Province, most cities with tooth decay rates above the national average do not have fluoridated public water.

Answer: Conclusion Does Not Follow

Interpretation

A study found that corporate executives who make risky business decisions also tend to receive more speeding tickets. Based on this study, the Acme Corporation favors applicants with good driving records when it hires executives. Before this policy started, many executives at the Acme Corporation made risky decisions that harmed the corporation. Now executives at the corporation generally make cautious decisions.

In the past, most executives at the Acme Corporation had received several speeding tickets.

Answer: Conclusion Does Not Follow

Evaluation of Arguments

Should cigarette sales be against the law in this country?

Yes; if cigarette sales were illegal, many people would give up smoking, improving their health and the health of people exposed to their secondhand smoke.

Answer: Argument Strong

 

Reliability & Validity

Watson-Glaser II has been extensively validated to provide the most accurate picture available of critical thinkers.

What’s the reliability of the new Watson-Glaser II?

• Form D Coefficient Alpha = .83
• Form E Coefficient Alpha = .81

What validity evidence has been collected on the Watson-Glaser II?

• The Watson-Glaser II Technical Manual fully describes the studies conducted and their results. In summary, expected patterns of correlations were found with:
• Cognitive ability (e.g., r = .60 with WAIS-IV fluid reasoning composite; n = 49)
• Occupational and educational attainment (e.g., r = .28 with job level; n = 432; r = .33 with education level; n = 581)
• Job performance (e.g., r = .28 with supervisory ratings of core critical thinking behaviors; n = 68)
• Attitudes or personality preferences related to critical thinking performance (e.g., for the correlation between Watson-Glaser II Evaluate Arguments and Myers-Briggs Feeling, r = -.27, n = 60)

Sub-Factor Scores

What is the three factor interpretation and what research is there to support it?

• Pearson exploratory factor analyses of Watson-Glaser Form A, B, and Short consistently showed 3 factors as the most interpretable solution:

• Recognize Assumptions: Recognize Assumptions subtest
• Evaluate Arguments: Evaluate Arguments subtest
• Draw Conclusions: Infer, Deduce, and Interpret subtests

• CFA results on the new test form (Form D) supported the three factor interpretation (AGFI = .93; RMSEA = .03; n = 306)
• The Watson-Glaser II Technical Manual provides additional information supporting the three-factor solution