Irony is still alive

It shouldn’t come as a surprise that psychological studies on “priming” may have overstated the effects. It sounds plausible that thinking about words associated with old age might make someone walk slower afterwards for example, but as has been shown for many effects like this, they are nearly impossible to replicate.

Now Ulrich Schimmack, Moritz Heene, and Kamini Kesavan have dug a bit deeper into this, in a post at Replicability-Index titled “Reconstruction of a Train Wreck: How Priming Research Went off the Rails”. They analysed all studies cited in Chapter 4 of Daniel Kahneman’s book “Thinking Fast and Slow”. I’m also a big fan of the book, so this was interesting to read.

I’d recommend everyone with even a passing interest on these things to go and read the whole fascinating post. I’ll just note the authors’ conclusion: “…priming research is a train wreck and readers […] should not consider the presented studies as scientific evidence that subtle cues in their environment can have strong effects on their behavior outside their awareness.”

The irony is pointed out by Kahneman himself in his response: “there is a special irony in my mistake because the first paper that Amos Tversky and I published was about the belief in the “law of small numbers,” which allows researchers to trust the results of underpowered studies with unreasonably small samples.”

So nobody, absolutely nobody, can avoid biases in their thinking.

A few bad apples or rotten to the core?

An article in Nature on the ethics of bankers has been widely reported. The researchers asked bankers and non-bankers to toss a coin in private, and then report the numbers of heads and tails – the higher the number of one of the two, the bigger the payout they received. This is a commonly used setting to study honesty. Even though the researchers will never know how honest each participant was individually, overall the amount of heads and tails should be 50/50 if everyone reports their results honestly. Any major deviation from the expected figures indicates some foul play. And on the whole, people tend to be remarkably honest in studies like this; definitely more honest (and therefore poorer as a result) than standard economic theory would predict.

An interesting twist in this particular study, however, was that half of the bankers were primed by asking them questions about their work before the coin tossing; in effect they were reminded what they did and where they worked. And that changed the results significantly. Without the priming, the bankers were as honest as everyone else. But with the nudge about their livelihood, they became much less honest. The effect was clearly significant (in all senses of the word) in the overall average cheating figures.

But what hasn’t been noted in the newspaper articles I’ve seen, was the change in the distribution of the results? Did all bankers become a bit more dishonest, or did some become a lot more crooked? Well, the original article shows the entire distributions for the control and treatment groups, and it seems to me that the answer is: a bit of both. The entire distribution of reported results shifts slightly, resulting in maybe 10% more in rewards than the participants should have received. But there is also a massive increase in the number of people who claim to have got nothing but heads (or tails, whichever gave them the reward).

In conclusion then, based on this study, most bankers in their natural habitat are probably a little bit naughty, but some of them are really quite seriously bad.