Ben Goldacre, The Guardian, Saturday 6 November 2010
Popular science is often triumphalist, presenting research as a set of completed answers, when in reality much of what gets published makes a glorious, necessary mess.
Here is an example. Solomon Asch’s legendary studies from the 1950s on conformity are among my favourite experiments of all time: some people in a room are asked to judge the length of a line; all but one are stooges, and they unanimously assert what is obviously an incorrect answer; but the one true unsuspecting experimental subject conforms to the majority view, despite knowing that it’s incorrect, about a third of the time.
It’s a chilling result that feels right, and over the past half a century researchers have replicated the study over 100 times in 17 countries, allowing hints of patterns to be spotted in the results. One analysis of US studies found that conformity has declined since the 1950s. Another found that “collectivist” countries tend to show higher levels of conformity than individualist ones.
This month the International Journal of Psychology published a new variant. Instead of one real subject in a room full of stranger stooges, they used polarising glasses – the same technology used to present a different image to the left and right eye for 3D films – to show participants different images on the same screen, at the same time, in the same room. This meant that friends could disagree, legitimately, and so exert social pressure, but without faking it.
The results were problematic. Overall, sometimes the minority people did conform to peer pressure, giving incorrect answers. But when the results were broken down, women did conform, a third of the time, but men did not. This poses a problem. Why were the results of this study different to the original study?
It could be that the subjects were different. The Asch experiments were only conducted in men, and they did conform. Perhaps modern Japanese undergraduates are different to 1950s US undergraduates (although cultural and generational differences have not previously been shown to be so large that they abolish the conformity effect completely).
It could be that the task, where you have to judge the length of a line, was slightly different. But if anything, the task in the new experiment was harder than the original, because the polarising glasses required that extra visual noise be added in, and if judgements were trickier, and therefore closer calls, then you might expect that conformity would increase, rather than decrease.
Or it could that the relationships were different. Perhaps conforming effects are less pronounced among people who know each other, rather than in a room full of stranger stooges: perhaps you feel more comfortable disagreeing with friends. This would be an important answer, if true, because when we extrapolate from the lab to the everyday, we’re probably more interested in conformity effects among acquaintances, because that’s what happens in a real community.
Maybe these questions will be resolved with a new experiment – you could probably design one yourself that would discriminate between the different possible explanations – but that will depend on whether someone is interested enough, and whether they can get the money and the time. Perhaps the paper will sink like a stone, and be ignored or overlooked, as sometimes happens with uncomfortable data.
But what you should know is this: alongside the triumphalism, and the answers, in reality, grey and conflicting results like these run deep in the research literature. They’re not an aberration, or a disappointment, in fact they are arguably the glorious norm, in the noise of over 20,000 academic journals, publishing well over a million articles every year. Alongside the giants, and the clean easy answers, challenging and ambiguous findings like these are what science is really made of.
MFunnell said,
November 6, 2010 at 6:22 am
allowing hints of patterns to be spotted in the results.
Ben’s link sends you to a paywall. Here’s a link to the same article free.
www.nd.edu/~wcarbona/Bond-Smith-Asch-meta-analysis.pdf
jamougha said,
November 6, 2010 at 9:52 am
I expect it’s because they correctly attributed the difference in perceptions to the glasses.
alicebell said,
November 6, 2010 at 1:30 pm
Would I just be being a cow to say this seems to ignore the “glorious mess” that is popular science publishing?
I went into researching pop sci back as an undergrad assuming it tended to present “research as a set of completed answers” and realised, as with most things, it’s a lot more complicated than that.
It entirely depends on what bits of pop sci you look at – where, when, on what and who by. It’s easy to pull examples either way. (I’m not entirely sure a systematic survey of the genre is desirable even if it’s possible, that’s a whole other issue though…) Still, I have looked and thought about this a fair bit, and in my experience, popular science can be as good at opening up questions as it is at answering them. Even in the kids books I specialise in – where you might imagine writers shy away from the complexity of grey areas – writers seem to relish sharing the excitement of a bit ambiguity with their readers.
alicebell said,
November 6, 2010 at 1:31 pm
p.s. as does this post, of course – if you’d include badscience blog as pop sci. Which I do.
Matthias said,
November 7, 2010 at 11:10 am
Perhaps, in the first experiment, the stooges were pusing their “truth” harder because it was their job.
Whereas in the current version, they probably were far less sure of “their truth” and could conceivably be swayed themselves.
andybak said,
November 7, 2010 at 1:19 pm
I would suggest that many areas of Psychology have more in common with the social sciences with all the problematic issues of interpretation, cultural bias and a weaker epistemological foundation than they do with the hard sciences which at least approximate the model that the ‘scientific method’ in all it’s idealised glory would require.
‘Conformity’ will never be a testable concept in the same way that ‘velocity’ or ‘mass’ are and it does neither discipling much service to pretend that it is.
Psychology might one day develop into a true science but it has to start at a much lower level and earn it’s place at the table.
muscleman said,
November 7, 2010 at 2:39 pm
If the result is about degrees of acquaintance then there is a good explanation as to why. When deciding whether or how much to disagree with those you know the future has to be taken into account (by most people anyway). You risk making yourself unpopular and/or appearing to be intransigent if you go against the majority opinion and those can have consequences in the community. Women tend to care more than men about things like this as well as having interests in consensus (as the weaker sex they are in more danger during real conflict) so will be more likely to agree for the sake of peace and group cohesion. Men also tend to be able to be in conflict and then friends again. What matters is not conflict but how you conduct yourself during it.
These are of course broad generalisations and plenty of both sexes vary from these broad norms. But they are also demonstrably real cannot be discounted.
Much of the history of civilisation is how to manage disagreement without it leading to conflict. It was one big reason why we invented contracts and civil courts. Adjudicating competing claims was a big function of community priests back when everyone belonged as well. Not to mention that if you say you didn’t see the sun shift in the sky during the outside mass then it either shows your faith is lacking (or absent) or that you are a heretic. So you agree with the deluded people even though deep down you know they are deluded. Groups are like that.
pjie2 said,
November 7, 2010 at 2:45 pm
If judgements were trickier, and therefore closer calls, then you might expect that conformity would increase, rather than decrease.
Really? I’d say the opposite. If the task is difficult, then it’s reasonably plausible that other people will get it wrong – i.e. there’s no need to conform to their view. In contrast, if the task is comparatively trivial, and yet everyone else’s opinion is against you, the more likely assumption is that you’ve misunderstood the task and that you’re wrong.
andrewwyld said,
November 8, 2010 at 9:21 am
Since popular science books are for enthusiasts, they are generally about telling the story of the science, and most stories involve mess in the middle but not at either end. I’d be interested in the proportion of pop sci books that tell it in a classic exposition-conflict-climax-resolution way (“In the nineeenth century Newton was king! Then James Clerk Maxwell found some problems with electromagnetism and Michelson and Morley found the velocity of light was constant. Oh no! Then Einstein came up with a brilliant solution based on trigonometry and completely disregarding the intuitive concepts of space and time as constant. He has been proven gloriously right by experiment. Roll credits.”)
Of course a lot of pop science will also tell about ongoing mess, because, again, it’s written for and by enthusiasts who like that about science and are excited about what might happen next. But in journalism the only time the mess is represented, as far as I can see, is in those “silly old scientists, can’t even tell that bumblebees can fly” type articles.
andrewwyld said,
November 8, 2010 at 9:39 am
By the way, surely the symmetry of this experiment accounts for some of the difference? Even if all but one person see one version of events, everyone in the room goes in equal: they might see one thing, they might see another. It’s also unlikely that strong social pressure would be exerted by friends over the length of a line during something they know to be an experiment (this may depend on the cover story). Ultimately it’s a line in a room. Real social pressure would be exerted over something like political allegiance.
I’m also interested in whether there is such a thing as conformity of dissent. For example, most people fit on a neat left-right spectrum and accept a raft of disparate issues as “right”, even though it’s unlikely a random person would randomly reach the same conclusions about all those moral questions. Why? I suspect “the enemy of my enemy is my friend” comes into play here. Could this be measured?
Daibhid C said,
November 8, 2010 at 12:59 pm
Muscleman – That doesn’t explain the original results, where the test subjects were 1) male, and 2) had no reason to believe they’d ever interact with these people again. If you’re right, there should have been no pressure of conformity there.
irishaxeman said,
November 8, 2010 at 7:16 pm
Remember that Asch’s original purpose was to measure the robust nature of dissent, which he demonstrated. We also vary considerably in our personal, rational or irrational, reactions to conformity.
irishaxeman said,
November 8, 2010 at 7:17 pm
Remember that Asch’s original purpose was to measure the robust nature of dissent, which he demonstrated. We also vary considerably in our personal, rational or irrational, reactions to conformity. There are personality variables in there as well as learnt attitudes to the sensation of conformity/dissent. Much more complex than presented?
ajohnson said,
November 8, 2010 at 11:27 pm
The subjects were wearing glasses that effectively concealed their eyes: this would be why the results were different. people behave very differently when others cannot see directly into their eyes.
GravyB said,
November 8, 2010 at 11:43 pm
I think the answer is in the bottom of the glasses. That’s where I find my answers.
outeast said,
November 9, 2010 at 9:29 am
It’s worth pointing out, perhaps, that some of the studies in the Bond and Smith paper Ben linked to looked at conformity among friends ansd among strangers – and with contrasting results:
‘Williams and Sogon (1984) found a much higher level of conformity when the majority were friends than when they were strangers. Matsuda (1985), however, did not find expected differences in conformity when three types of relation between the individual and majority were compared.’
Given this, speculative explanations of why conformity would be less among friends (or among strangers, for that matter) are pretty much of a distraction. The effect itself needs to be clearly shown before explanations of the effect are ventured.
outeast said,
November 9, 2010 at 9:37 am
PS Both Williams and Sogon (1984) and Matsuda (1985) were studies conducted with Japanese subjects; the former was certainly with students (like the new study), not sure about the latter. But I thought I’d add that note – it’s significant, obviously, as it means the research populations were at least somewhat comparable and that eliminates some potential variables.
homedesignfit said,
November 9, 2010 at 2:50 pm
its nice for me…thanks your good info…
Home Design Cozy
MedsVsTherapy said,
November 9, 2010 at 3:01 pm
I have lots of thoughts on this one. Especially regarding AndyBak’s suggestion that psychology apparently cannot be “scientific,” like physics.
But first:
This is a grave issue. Asch, and others, were driven to study conformity as the beginning to understanding how so many people could stand by and watch Hilter’s holocaust of “undesirables” (Jews, Poles, Sam-Sex-Orientation people, people with mental retardation, and so on) occur without even a peep.
A few researchers latched on to the idea of “conformity” being a possible explanation. So, Zimbardo, Milgram, and Asch carried out these classic studies with stunning findings: despite our best intentions or best evidence, we will deny reality of our own senses and conform to the prevailing judgment or decision.
So, this is not simply an academic topic.
On this side of the pond, here is what is in my morning news:
www.msnbc.msn.com/id/40069887
At a party of high school kids, one declares he will attack the next innocent person who happens by. A few people join in, and succeed at killing an innocent high schooler. In front of a party of 80 normal human beings.
This event cannot even be attributed to hate-crime, or whatever other story might be cooked up.
There will always be some nut who might want to kill someone. But will there always be the lack of will to inetrvene from bystanders?
MedsVsTherapy said,
November 9, 2010 at 3:22 pm
Andybak sez:
” ‘Conformity’ will never be a testable concept in the same way that ‘velocity’ or ‘mass’ are and it does neither discipling much service to pretend that it is.
Psychology might one day develop into a true science but it has to start at a much lower level and earn it’s place at the table.”
What place?
You have a hypothesis. You define it in a testable, disprovable way. You figure out how to measure your concepts, and you measure them.
You carry out your test. You evaluate the degree that the obtained data fit your hypothesis.
You accept the concept that others can criticize your methods, and you work with criticism to refine your concept, your measures, your strategy for testing your hypothesis.
you stick with your working hypothesis until, by this process, you figure out ways to revise it, or eventually replace it altogether.
You strive to minimize interference from various confounding factors. You manipulate independent variables in various ways, and watch how outcomes vary.
We psychologists are scientists. Sorry. Boo, hoo hoo. I did not mean to spoil yor day.
Pretty much anyone with a PhD in psychology from a decent program is well-versed in epistomology, theory and history of science, the philosophy and practice of measurement, research design, and statistical analysis of data.
If you do not know this, it is because you are un-informed.
Anytime I see actual experimental results from a study in the “hard” sciences, I see variable data. I see modest ability to predict outcomes. Same as in psychology.
If “scientists” know physics so well, then why do airplanes need mid-course corrections?
Why can’t anyone predict tomorrow’s weather accurately?
In a high school lab, you can develop a clear “physics” experiment where the answer will come out the way you want.
When I discuss the grad experience with people who have earned their PhD in the “hard sciences,” they all admit that things are not squeeky clean and exact; that their mentors had them repeat studies over and over “until the results came out right.”
The belief that “hard sciences” are really reliable and steady, or whatever, is simply a myth. when applied to the real world, as this conformity study strives, with real people and real interpersonal influence, it is a challenge to design and carry out an experiment.
We do not yet know how to do it perfectly. However, we are performing science, and we are guided, exactly, by the same philosophy and methods of any other scientific discipline.
This study was not a matter of gathering opinions, or asking whomever walked by to answer whatever question popped into the mind, in response to whatever stimuli someone decided to present.
Consider complexity: physics is largely dealing at the level of the basic physical forces, and inorganic matter.
There is yet another layer of complexity: organic matter. Tissues. Organs. Organisms.
And yet another layer: oranisms with willpower to go this way or that, to think this or that.
And yet another layer of complexity: social / group / crowd behavior. This is an influence in other animals, as well as humans.
The recent science news has touted the emerging finding that bacteria communicate with each other somehow.
This level of complexity has utterly gone beyond some simple physics high school classroom demonstration of force = mass x aceleration (whcih always works out, provided you can round off a bit at the decimals).
“Physics” may be “hard science.” Trying to understand things such as “bystander apathy,” and “conformity,” and “genocide,” is “harder science.”
iamjenny said,
November 11, 2010 at 8:04 pm
Well said. There are a lot of misconceptions when it comes to psychology, and there are areas of psychology which are less scientific than others, but this does not mean you can discredit it as not being a science.
To answer pjie2’s post, the reason that
‘If judgements were trickier, and therefore closer calls, then you might expect that conformity would increase, rather than decrease’ stands, is that in general, when answers are more ambiguous we are more likely to look to others for the answer as we are unsure ourselves. This adds informational influence to the conformity aspect.
Del said,
November 16, 2010 at 12:54 pm
There is something about this that reminds me of the time I worked for a multinational food company where random employees had to attend daily tasting sessions to confirm quality. Invariably the sensitivity of one’s taste buds seemed to improve directly in proportion to one’s seniority in the organisation.
MedsVsTherapy said,
December 22, 2010 at 3:41 pm
Doh! The periodic table gets updated with error band for atomic weights!
www.iupac.org/publications/ci/2011/3302/1_coplen.html
In psychology, we use scientific method, and we have been using error bands for quite a while.
Now, for how long have the data been sitting in front of chemists, and all the while millions of schoolkids have been mis-lead to believe that “the science is settled” regarding the periodic table?
Hard science. Right.