Student Overconfidence

People tend to be overconfident in their answers to a wide variety of general knowledge questions, and in particular when the questions are difficult (Plous, 1993). How do researchers study overconfidence? One approach is to ask participants to estimate the probability that their judgment is correct. These estimates are then used to calibrate between confidence and accuracy. A person is perfectly calibrated when his proportion of judgment at a given level of confidence is identical to his expected probability of being correct. Another approach is to ask participants to give a "confidence intervals" that have a specific probability (usually .9 or .98) of containing an unknown quantity. In one study, participants were 98% sure that an interval contained the correct answer but they were right only 68% of the time.

In one of the summer sessions of an introductory CS courses, 16 students out of a class of 68 students overestimated their final course grade after they have received feedback from their first midterm, and 3 students underestimated their final course grade.

Overconfidence can be relearned, just like any belief system. People who were initially overconfident could learn to make better judgments after 200 tries with intensive performance feedback (Lichtenstein and Fischhoff, 1980). Arkes et al. (1987) found that overconfidence could be eliminated by giving participants feedback after five "deceptively difficult problems". Yet another study by Lichtenstein and Fischhoff shows that by having the participants generate opposing reasons alone was sufficient to reduce accuracy overconfidence, but this has not been confirmed in subsequent studies.

References:

Arkes, H.R., Christensen, C., Lai, C., and Blumer, C. (1987). Two methods of reducing overconfidence. Organizational Behavior and Human Decision Processes. 39, 133-144.

Lichtenstein, S., Fischhoff, B., Phillips, L. 1980. Training for calibration. Organizational Behavior and Human Performance, 26, 149-171.

Lichtenstein, S., Fischhoff, B., Phillips, L. 1982. Calibration of Probabilities: The state of the art to 1980. In D. Kahneman, P. Slovic, and A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp 306-334). Cambridge, England: Cambridge University Press.

Plous, S. 1993. The Psychology of judgment and decision making. New York: McGraw-Hill.

Plous, S. 1995. A Comparison of Strategies for Reducing Interval Overconfidence in Group Judgments. The American Psychological Association Inc. 80:4 p 443-454

Some other resurces I found: papers, chapter on overconfidence, presentation on certainty