The Dr Fox Effect: Charisma, subject knowledge and the curse of expertise
Why effective teaching depends less on generic performance and more on knowledge organised for instruction.
The man who knew nothing
In 1970, psychologists and psychiatrists were invited to a lecture on “Mathematical Game Theory as Applied to Physician Education.” The lecture, supposedly given by Dr Myron L. Fox, a graduate of the Albert Einstein College of Medicine and a student of the great John von Neumann, was actually given by an actor who knew nothing about either game theory or physician education. The audience of MDs and PhDs were, in fact, unwitting subjects in a study conducted by Donald Naftulin, John Ware and Frank Donnelly on “educational seduction.”
The audience were divided into two groups; one group was given a lecture by an actual scientist about something relevant and interesting, the other group listened to Dr Fox pedal his nonsense. In the first experiment, Dr Fox was instructed to lecture in as boring a monotone as he could manage. The audience was then tested on how much they’d retained and, surprise surprise, it wasn’t much. In a second experiment, Dr Fox went to town, using the full range of his thespian skills: he had his audience laughing, concentrating and nodding along. Even though the content of the lecture was absolute pap, filled with what Deborah Merritt describes as “double talk, neologisms, non sequiturs, and contradictory statements,” students rated the lecture as interesting, stimulating and valid.1
This phenomenon, that a charismatic speaker could fool a knowledgeable audience into believing any old rubbish was meaningful and worthwhile, became known as the Dr Fox Effect. Dr Fox bamboozled three separate audiences of professional and graduate students. Merritt, in a critique of student surveys used to evaluate lecturers, put it like this:
Despite the emptiness of his lecture, fifty-five psychiatrists, psychologists, educators, graduate students, and other professionals produced evaluations of Dr. Fox that were overwhelmingly positive. … The disturbing feature of the Dr. Fox study, as the experimenters noted, is that Fox’s nonverbal behaviors so completely masked a meaningless, jargon-filled, and confused presentation.
The problem isn’t that audiences enjoy good speakers. Of course they do. The problem is that we’re surprisingly bad at distinguishing the feeling that we’re learning from the reality of learning. We may enjoy fluency, warmth, pace, confidence, humour and theatricality, and mistake these things for understanding. In schools, we do something similar whenever we use engagement, enjoyment, participation, confidence or lesson polish as proxies for learning. These things aren’t worthless, but they’re not the same as students knowing more, remembering more, or being able to do more later.
This may help explain the wild popularity of the late Sir Ken Robinson’s 2006 TED Talk on schools killing creativity. It’s had something like 76 million views, despite being, in my view, utterly and completely wrong-headed. Robinson was funny, charming, fluent and seductive. He told a story people wanted to believe: that schools crush children’s creativity and that the answer is to liberate them from the dead hand of traditional academic instruction. But creativity is not released by reducing the amount children know; it’s cultivated through knowledge, discipline, imitation, practice and the gradual acquisition of the very traditions Robinson was so breezily willing to caricature.
Now, although other researchers have confirmed the existence of the Dr Fox Effect, it appears that although people in the audience rate a good speaker positively regardless of what they say, little actual learning takes place unless the speaker possesses considerable subject knowledge. Psychologists Eyal Peer and Elisha Babad replicated the original study in 2012 but added an additional item to the questionnaire people who attended the lecture were given. The question asked whether audience members felt they’d actually learned anything. The results were interesting: even those students who had evaluated Dr Fox as a highly effective speaker were aware that they’d learned nothing from the lecture.
The problem with ‘engagement’
So, what does this tell us about effective teaching in schools? There’s still a commonly held belief in education that a good teacher can teach anything well and that subject knowledge, whilst not without value, is clearly less important than the pedagogic skills and personal charisma of the teacher. The argument suggests that it’s much better for teachers to focus on acquiring and practising generic teaching skills than on developing subject knowledge.
Although students may well enjoy engaging lessons and motivating speakers, these things don’t actually seem to make much difference to learning. Rob Coe has referred to engagement was a “poor proxy” for learning. This is one of those claims that manages to be both obviously true and weirdly provocative. Of course students need to attend to what they’re learning. Of course we’d rather they were interested than bored but engagement is, at best, merely a condition that may make learning more likely.
In The Hidden Lives of Learners, Graham Nutthall discusses his research which found that “students can be busiest and most involved with material they already know.” Students are far more likely to get stuck into tasks they’re comfortable with and already know how to do than into the more uncomfortable business of grappling with uncertainty. A classroom can look wonderfully engaged because students are busy rehearsing what’s already secure.
The same problem applies to motivation. Here’s Nuthall again: “Learning requires motivation, but motivation does not necessarily lead to learning.” Motivation and engagement may be vital elements in learning, but their impact depends on what they’re used in conjunction with. Motivated to do what? Engaged by what? If students are being motivated to perform superficial tasks, or to practise things they can already do, then the whole enterprise may be a waste of time.
The illusion of confidence
Tom Loveless made a related point in a report for the Brown Center, where he tanalysed PISA findings on student engagement and maths attainment. He found that the relationship between engagement and attainment was far from straightforward. Several high-performing countries reported lower levels of student engagement than the US, and, at national level, students’ enjoyment of maths and enthusiasm for maths lessons were negatively associated with achievement. Confidence was no more reliable: American students reported being more confident than Singaporean students, but their test performance much worse.2 The point isn’t that engagement causes poor performance, but that motivation, enjoyment and confidence are unreliable proxies for learning.
Of course, none of this proves that engagement and intrinsic motivation are actually bad for attainment, but it does cast serious doubt on policies and practices that try to boost engagement in the belief that results will follow. As Loveless put it:
PISA provides, at best, weak evidence that raising student motivation is associated with achievement gains. Boosting motivation may even produce declines in achievement.”
That isn’t a licence for dull lessons bit it is a warning against confusing the visible signs of student enthusiasm with the less visible business of durable learning. What students enjoy may not involve much learning. Working hard is, well, hard work. Grappling with uncertainty often feels uncomfortable. If learning depends on students having to think hard, then activities designed chiefly to maximise engagement may easily steer them away from the very struggle that produces retention and transfer.
There’s also compelling evidence that we’re pretty terrible at judging when we learn best. Most of us prefer blocked or massed practice: we review the basics, complete a few related exercises, reach a feeling of fluency and move on. It feels efficient because performance improves quickly during the practice itself. Interleaving, by contrast, often feels harder and produces more mistakes during instruction, even though it tends to produce better long-term retention. There are similar problems with retrieval practice. Students tend to prefer strategies that feel productive, such as rereading or restudying, while underestimating the value of effortful retrieval. Strategies that produce quick fluency make students feel as if they’ve learned more than they have whereas strategies that feel effortful, uncertain and less successful in the moment may feel worse, but often produce stronger retention later.3
Part of the problem is that blocked practice gives us the warm, fuzzy feeling of cognitive ease. It creates the illusion of knowing. We prefer the comfort of feeling successful now to the discomfort of learning something well enough to remember and use later. In that sense, engagement can become another version of the Dr Fox Effect: a seductive surface signal that makes us feel we’re making progress when the harder evidence is far less reassuring.
