CRITIC: Learning to Evaluate Claims
Monday May 22, 2006
Critical thinking is very important - every day we are confronted with a host of claims which we need to be able to evaluate. Is there any way people can learn to do a better and more consistent job?
In the May/June, 2005 issue of Skeptical Inquirer, Brad Matthies offers a mnemonic method for evaluating claims which is based upon one developed by Wayne R. Bartz. CRITIC asks:
Claim?
Role of the claimant?
Information backing the claim?
Testing?
Independent verification?
Conclusion?
Matthies explains how each step can work:
Claim
What is your source saying? Is the source's claim both timely and relevant to your particular question or thesis? Has the source presented the claim in a clear and reasonable manner, or is there evidence of motivationally biased language?
Role of the Claimant
Is the author of the information clearly identifiable? If so, can his or her credibility be established? Also, based on your prior examination of the claim, is there any reason to suspect bias on the part of the author?
Information Backing the Claim
What information does the source present to back the claim? Is it information that can be verified, or does this source rely on testimony or anecdotal evidence? If this source presents original research, does the source explain how the author gathered the data? If the source is an article, does it cite references and are they credible? If the source is a journal article, is the journal peer-reviewed?
Testing
How might you test the claim your source is making? Conduct your own qualitative or quantitative research (e.g., marketing research, statistical analysis, design a research study, etc.).
Independent Verification
Has another reputable information source evaluated the claims the source is making? Does this source support or refute the original claim? After conducting a review of the literature, what do the experts have to say about the claim? Are the experts basing their opinions on detailed analysis and testing, or are they just presenting opinions with little or no evidence? Moreover, are the experts truly experts on the topic, or are they presenting opinions about a topic they are not qualified to discuss?
Conclusion
What is your conclusion about the source? Taking into account the first five steps of CRITIC which apply to your source, make a judgment: Should this source be used in a paper or report? Information evaluation can be very subjective, so it is important to consider all of the ascertainable facts.
Matthies makes a lot of important points above. Sometimes, it seems as though most people don't know or recognize the importance of any of them. Ideally such mnemonic devices wouldn't be necessary, but even so they do provide an interesting way for organizing and structuring how we can approach claims. They can certainly make good teaching tools and there is no question but that schools should invest more time into teaching skepticism and critical thinking.
Read More:
Comments
Austin, I have a question and a comment for you.
First the question: How much would you say that a person’s credibility should play a role in evaluating their claim? One should, for example, be wary of claims made by creationists about evolution, but in the other direction, at what point does relying on a person’s credibility on a subject turn into an “argument from authority” fallacy?
And, my comment:
there is no question but that schools should invest more time into teaching skepticism and critical thinking.
Absolutely. Speaking just from personal observation, it seems many freethinkers I’ve met developed their fine critical & analytical skills in spite of, rather than because of their (formal) education. While I don’t yet have kids of my own, if & when I do I’ll want to have an active role in their education, maybe not as extensive as full home-schooling, but at the same time not assuming that the public schools they attend will be teaching them much of the really important things in life (e.g. civil liberties, critical thinking, etc.). One of the jobs I have is scoring state educational exams for K-12 and let me tell you, I fear for the future of this country. My wife has been a teacher & college T.A., and she feels the same way.
An argument from authority isn’t always a fallacy. Relying on a person’s credibility becomes a fallacious appeal to authority at the same point any appeal to authority becomes fallacious: when the authority isn’t an authority in the subject at question, when the authority isn’t specifically trained in the subject at question, or when the authority is making claims that are heavily disputed by other experts (follow the above link for more).
Re: Teaching Skepticism; Ideally, skepticism and critical thinking would be incorporated into all subjects, not just a special class. This would help students learn how to apply skepticism, not just learn it in the abstract.
Evaluating the approximate correctness of the claim by evaluating the claimer is almost a logical fallacy, but I agree that it is very useful when evaluating the level of skepticism one should have against the claim and is really the only time it should be applied.