Saturday Edition

Get the Blog Feed
What is RSS?

Viagra Online In Florida

The 90% (Bullshit) Factor

A Tweet that showed up yesterday mused that about 90% of statistics are made up. I laughed, but it's probably about right. Well, not made up, exactly, but highly and selectively doctored.

Reading the Tweet coincided with a book purging project which led me to pick up the well received 1982 book, Betrayers of the Truth: Fraud and Deceit in the Halls of Science, by two prominent science journalists, William Broad and Nicholas Wade. It is at once entertaining and serious. Along the way, the work of the likes of Galileo, Newton, the chemist John Dalton and American physics Nobelist Robert Millikan are raked over the coals. Data that's too good to be true, experiments that after many efforts could not be replicated by even the best scientists, simple fudge factors applied with abandon, etc.

And then I tripped over my all-time favorite, which I used to use in seminars, when discussing the real (messy) world of science and innovation. The Austrian monk Gregor Mendel is widely acclaimed as the "father of modern genetics." But he is also a poster child for questionable data. Though he has his defenders, one detractor wrote a brief essay, "Peas on Earth," that appeared in a professional journal:

"In the beginning there was Mendel, thinking his lonely thoughts alone. And he said: 'Let there be peas,' and there were peas and that was good. And he put the peas in the garden saying unto them 'Increase and multiply, segregate and assort yourself independently,' and they did and it was good. And now it came to pass that when Mendel gathered up his peas, he divided them into round and wrinkled, and called the round 'dominant' and the wrinkled 'recessive,' and it was good. But now Mendel saw that there were 450 round peas and 102 wrinkled ones; this was not good. For the law stateth that there should be only 3 round for every wrinkled. And Mendel said unto himself 'Gott in Himmel, an enemy has done this, he has sown bad peas in my garden under the cover of night.' And Mendel smote the table in righteous wrath, saying 'Depart from me, you cursed and evil peas, into the outer darkness where you shalt be devoured by rats and mice,' and lo it was done and there remained 300 round peas and 100 wrinkled peas, and it was good. It was very, very good. And Mendel published."

Love it!

Maybe one of the good side effects of the Web is that the proliferation (tsunami) of absurd data (a/k/a utter bullshit) will lead to a general increase in skepticism. Very few things are what they seem, regardless of their imprimatur (think of Wall Street and its battalions of MIT-Stanford-Harvard-Chicago PhD mathematicians). The caution light should be permanently yellow.

(NB1: The book is also replete with instructive sagas like that of Ignaz Semmelweis. With childbed [puerperal] fever claiming up to 30 percent of mothers' lives in even the best European maternity hospitals, Semmelweis was able to virtually eliminate it in his own clinic simply by having doctors wash their hands in a chlorine solution before examinations and procedures. Alas, Semmelweis had an all-time low EQ, and was abrasive beyond measure; moreover, at a volatile time, his political views were on the fringe. Hence his work was ignored out of hand, and tens of thousands of lives were unnecessarily lost over the following three decades—Semmelweis died in restraints in a mental institution in 1865. Once more we observe that science in the real world strays from "just the facts, ma'am" more often than not—and personal style almost always matters more than one would imagine.)

(NB2: Another book I grabbed was The War of the World, by the renowned British historian Niall Ferguson. It recounts in all too vivid detail the unmatched human violence of the better forgotten 20th century. On the "true facts" dimension, Ferguson at one point calls into question the sacred notion that a few brave Spitfire pilots held off the German horde. There is no disputing or diminishing the pilots' remarkable bravery, yet Ferguson points out that at the beginning of the Battle of Britain the RAF had more fighter aircraft and many more trained pilots than the Germans, and was out-producing the Germans in terms of new aircraft by a ratio of about 3 to 1. Britain's estimates of German pilot strength were off by a factor of 7, Ferguson reports. Um, so much for statistical accuracy; and, hey, nobody ever accused my all-time hero Churchill of being less than a great actor.)

Tom Peters posted this on 05/18/10.

Comments

I think you will find that it is exactly 87.78% of statistics that are made up......

Always have decimal places in made up statistics it's the only way to be credible.

Posted by PaulH at May 18, 2010 10:14 AM

Russ Roberts and Ed Leamer have a thoughtful discussion on this, analyzing the data 100 ways and only reporting the one way that supports your conclusion. But they also talk about how to do things right. Roberts and Leamer are economists, but their discussion is relevant to any use of statistics.

See EconTalk, 10 May 2010.

Posted by John Cook at May 18, 2010 10:27 AM

Alas, I fear a side-effect of the Web will be that we create a generation who know little but are very good at looking up things on the Web. In which case, with under-developed thinking abilities, they'll simply believe whatever piece they happen on.

I was talking to a couple of trainees recently who didn't know the difference between scepticism and cynicism. Even when it was explained to them, one reacted: "Well, it's all the same thing, isn't it?" I'm British, I "do" irony and this wasn't irony. Oh dear...

Posted by Mark JF at May 18, 2010 10:42 AM

"exactly 87.78%"

Dead on!

Posted by tom peters at May 18, 2010 10:48 AM

I don't know about the numbers for the Battle of Britain. I think part of it is that although Germany undoubtably possessed more aircraft how many of them were available in that particular theatre is up for debate.

Plus Hurricane pilots may disagree with the Spitfire comment! - the Hurricane destroyed more aircraft than the Spit - although this is generally as it was a more stable gun platform and was sent after the bombers while the Spits were mixed up with the 109s

The Spit and the Hurricane are an interesting example of the evolution versus revolution points of view.

The Hurricane is a transition design - although it looked modern it still had a frame and some parts fabric covered. The Spit was a completly stressed skin construction and is the big leap forward (revolution) in design.

On paper the Spit is far superior but basic stats are sometimes not the whole story! the turn around time for a Spitfire was 25 minutes - the hurricane was back in the air in 9 minutes also the work required to repair the Spit was much higher making it harder to keep operational.

Sometimes lower spec technology is better especially if your organisation is used to it and can make it work well.

Posted by PaulH at May 18, 2010 10:53 AM

The first book looks very interesting. Alas it's out of print; I will search for it in the SFPL.

One of my colleagues (who teaches finance) once noted that a lot of research-driven policy with vast implications (financial and otherwise) doesn't have any due diligence to determine the validity of that underlying research. So changes to regulation that cost billions or endanger many lives are undertaken with less due diligence than is done for a $50k angel investment in a web startup.

(I already have Ferguson's; his books are always thought-provoking, though sometimes he gets under people's skin with the implications he derives.)

Posted by Jose Camoes Silva at May 18, 2010 11:02 AM

"There are three kinds of lies: lies, damned lies, and statistics." - Mark Twain, Benjamin Disraeli, ...

Posted by JPasquale at May 18, 2010 11:11 AM

"Sometimes lower spec technology is better especially if your organisation is used to it and can make it work well."

U.S. A-10, classic modern case. About three moving parts, but a peerless tank killer. (Didn't survive Pentagon cuts in favor of fewer sexier thingd-often horrendous mistake.)

Posted by tom peters at May 18, 2010 12:09 PM

Well said. And with the power of the internet and many people wishing, hoping to make money as experts, there is bound to be more and more bullshit. Thank God we also have the ability to retort and disprove with the click of the mouse.
cheers.
Michael

Posted by Michael Van Osch at May 19, 2010 10:07 AM

I'm excited by the advancements made possible by the internet and I’m certainly not worried about the effect of the internet regarding bullshit – rest assured the art of bullshit was perfected many years ago and way before the web. There are numerous experts in the field of bullshit who don’t need additional training from the internet.

I agree with Mark and Michael there are down sides but overall the internet is probably the biggest game-changing development in my lifetime. I say grasp its potential and use is it for our own personal and professional development.

All the worries about the internet are relative; for instance when I was a teenager I remember my late Dad bemoaning the passing of his beloved slide rule (who can remember them?) Dad could not understand why we youngsters were not taught how to use the slide rule. He was convinced standards will fall alarmingly because kids now have ‘these new fangled battery operated calculators’ that do all the work for them.

There is something in most (all?) of us that loves to look back with rose-tinted specs and not celebrate the wonderful time we live in now and indeed look to the future with optimism.

I trust our youngsters to get it right – more right than my generation in fact. And if they get it wrong, to who have they looked up to as their role models? I’ll put my hand up to plead guilty. We each have to do our best for our own kids - then just let go and hope.

Posted by Trevor Gay at May 19, 2010 10:40 AM

Ah - JPasquale beat me to my favorite lies comment. Good job.

So, it's one kind of bad when the lies are about history. Another kind when the lies influence public policy. The recent "climategate" revelation comes to mind. How many poor policy decisions have we made and how many more are we about to make based on the fabricated "global warming"?

How about Bill Gates in his recent TED talk calling for a global population reduction - to reduce carbon emissions - to reduce global temperature. Really Bill? You want all these people DEAD? News flash Bill - you got suckered by the 90% Bullshit factor.

How did the 'scientists' phrase it, "we need to hide the decline"? And the guy who made the famous "hockey stick" slide - can't find his data... Really?

Yeah Tom - good message. Be critical. Keep thinking for yourself. Ask questions.

Posted by ATeasdale at May 19, 2010 3:03 PM

Have you read "Cult of statistical significance:how the standard error costs us" by Stephen T. Ziliak and Deirdre N. McCloskey? They have made excellent arguments against the importance of statistical significance and prove that it is an abstraction commonly used to falsify data.
The book highlights danger and corruption that flows from the overwhelming importance placed upon statistical significance by using examples from various fields like pharma, law, forensics. Another interesting aspect of the book is a paean to Gosset at the beginning of the book.

Posted by Yogini at May 21, 2010 3:09 PM

"U.S. A-10, classic modern case. About three moving parts, but a peerless tank killer. (Didn't survive Pentagon cuts in favor of fewer sexier thingd-often horrendous mistake.)"

Not sure if Wiki has the complete story, but according to wiki, the A-10 is scheduled to be around until 2028.
Damned Internet.

http://en.wikipedia.org/wiki/A-10_Thunderbolt_II

Posted by zorro at May 23, 2010 5:31 PM