The Hobbit
J. R. R. Tolkien
4.8 on Amazon
102 HN comments
Animal Farm: 1984
George Orwell and Christopher Hitchens
4.9 on Amazon
101 HN comments
Good to Great: Why Some Companies Make the Leap and Others Don't
Jim Collins
4.5 on Amazon
100 HN comments
How to Lie with Statistics
Darrell Huff and Irving Geis
4.5 on Amazon
99 HN comments
A Brief History of Time
Stephen Hawking
4.7 on Amazon
98 HN comments
The Bell Curve: Intelligence and Class Structure in American Life (A Free Press Paperbacks Book)
Richard J. Herrnstein and Charles Murray
4.7 on Amazon
98 HN comments
The Mom Test: How to Talk to Customers & Learn If Your Business Is a Good Idea When Everyone Is Lying to You
Rob Fitzpatrick and Robfitz Ltd
4.7 on Amazon
96 HN comments
Influence: The Psychology of Persuasion, Revised Edition
Robert B. Cialdini
4.6 on Amazon
95 HN comments
Man's Search for Meaning
Viktor E. Frankl , William J. Winslade, et al.
4.7 on Amazon
94 HN comments
The Federalist Papers
Alexander Hamilton and James Madison
4.6 on Amazon
93 HN comments
Calculus Made Easy
Silvanus P. Thompson and Martin Gardner
4.5 on Amazon
92 HN comments
The Mind Illuminated: A Complete Meditation Guide Integrating Buddhist Wisdom and Brain Science for Greater Mindfulness
John Yates , Matthew Immergut , et al.
4.7 on Amazon
92 HN comments
Superintelligence: Paths, Dangers, Strategies
Nick Bostrom, Napoleon Ryan, et al.
4.4 on Amazon
90 HN comments
On Writing: A Memoir of the Craft
Stephen King, Joe Hill, et al.
4.8 on Amazon
90 HN comments
Rework
Jason Fried and David Heinemeier Hansson
4.5 on Amazon
90 HN comments
r4umonJan 30, 2019
https://web.ma.utexas.edu/users/mks/statmistakes/StatisticsM...
How to Lie with Statistics by Darrell Huff
https://en.wikipedia.org/wiki/How_to_Lie_with_Statistics
tester756onMay 7, 2021
why?
wsc981onApr 17, 2017
Ah, just discovered the book can be read for free online: https://archive.org/details/HowToLieWithStatistics
oriettaxxonMar 17, 2017
leephillipsonJuly 5, 2015
btillyonFeb 19, 2021
After all this exact form of misleading graph was one of the topics in the famous book How to Lie With Statistics written in 1954. And I don't think that the observation was new then.
DoreenMicheleonOct 1, 2018
Problems like this are part of why we have sayings like:
Measure twice, cut once.
GIGO (Garbage in, garbage out)
Some saying about measuring to extreme precision, then "cutting with an axe."
wscottonOct 6, 2020
Storks like roosting on chimney pots and larger houses have more chimneys and more children.
MzonOct 19, 2015
grumpyautistonMay 7, 2021
wyconOct 2, 2014
A very entertaining read: throughout the short book, Huff satirically likens abusers of statistics to criminals.
mrutsonMar 11, 2019
simion314onNov 27, 2019
DoreenMicheleonMay 25, 2021
"There are three kinds of lies: Lies, damn lies and statistics" -- Mark Twain
nathancahillonApr 14, 2014
[0] http://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/039...
DoreenMicheleonFeb 14, 2019
dandareonSep 18, 2018
Sorry, I will not continue reading this article.
MzonOct 23, 2016
How to lie with statistics (Good intro to concepts, light on actual math)
The cartoon guide to statistics (the first chapter or two covers what I learned in my college Intro to Stats class -- the rest is over my head)
therealdrag0onMay 27, 2020
smartmiconFeb 19, 2021
mkettnonFeb 6, 2020
D. Huff et al - How to Lie with Statistics
J. Ellenberg - How Not to Be Wrong: The Power of Mathematical Thinking
IIRC Bill Gates recommended them.
therealdrag0onDec 14, 2020
gumbyonFeb 3, 2018
billswiftonAug 2, 2009
scribuonJan 6, 2019
https://en.wikipedia.org/wiki/How_to_Lie_with_Statistics
sytelusonNov 27, 2012
partiallyproonOct 28, 2017
billswiftonMay 27, 2009
icantdrive55onJuly 2, 2017
2. How to lie with statistics, by Darrel Huff.
3. I don't know if there's a real problem. Articles like this should be illegial.
4. Maybe not illegial, but have all studies/advertisements include the raw numbers too.
MzonDec 12, 2012
(But have an upvote. I am pretty sure your intent was to be supportive.)
jjgreenonOct 6, 2016
MzonJune 23, 2017
Personally, I am more interested in finding some sort of meaningful info on how to make money on games or apps at all. Yes, I know it can be done. I am interested in figuring out how it is done.
mrslaveonJuly 24, 2020
(Edit: I was wrong about affiliate links. Not deleting to publically self-shame.)
The real data science would be producing articles like this automatically, and with good SEO, to drive revenue.
The list is OK. I've studied ISL (and some of ESL). A friend really enjoyed Think Stats. Charles Wheelan's book is in the same vein as How To Lie With Statistics (Darrell Huff?) but in greater depth.
I started with R because that's what my team was mostly using when I got into this area. Hadley Wickham's free books are good too.
mooredsonOct 9, 2019
Classic, short book about fundamental stats concepts everyone should know.
therealdrag0onJune 8, 2020
Perhaps consider reading "how to lie with statistics".
eitallyonMar 26, 2015
I would actively point teenagers away from Atlas Shrugged. While it might be enlightening, and they might enjoy the story, it has the potential to be quite destructive if they interpret her philosophy of objectivism & egoism in a way that leads them to apply it to their own lives. I just don't think this is a good idea. Her other novels, whether The Fountainhead or We The Living aren't nearly so overbearing.
Another one: How To Lie with Statistics
spikelsonJan 26, 2014
This is a rather extreme perspective and probably not effective in most situations where data visualization is used. The techniques used in the famous book "How to Lie with Statistics" are actually mostly about data visualization and are much more practical. This explains their popularity and that of tools that emphasize them like PowerPoint.
From Tufte's point of view we have a lemons market in data visualization: "bad" visualization drives out "good" visualization because it is more effective[1]. Editors, businessmen and politicians will always choose the visualization that empasises their point (i.e. distorts the underlying data). While they are happy with the results Tufte is dismayed.
[1] http://en.wikipedia.org/wiki/The_Market_for_Lemons
mrslaveonMar 22, 2021
[0] https://www.goodreads.com/book/show/51291.How_to_Lie_with_St...
[1] https://www.goodreads.com/book/show/43722897-the-art-of-stat...
yborisonOct 1, 2019
"How to Lie With Statistics"
https://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/03...
hancheonFeb 19, 2021
> While discussions of misleading graphs are not new (e.g., Huff, 1954), empirical research on their assumed consequences is scattered across fields.
So on the surface, at least, this looks like just another paper belabouring the obvious.
greenyodaonApr 21, 2019
majormajoronFeb 15, 2021
> Serious question: How do we know any of this reporting is true?
> There's no data sources provided, no way to check follow up health (i'e. what if these people all dropped dead a week later), no way to prove the data isn't a complete fabrication, etc. Also, they make sure to use dynamic terms so we can't be sure these numbers have any valuable meaning at all (e.g. were Covid tests with drastically different cycle counts used?). Not to mention all of this reporting is praising a single vaccine producer in a billion dollar industry.
> I just read "How to Lie with Statistics" by Darrell Huff and this whole thing is the epitome of a manipulatable situation.
Combine the two and you no longer look "hey lets make sure people aren't lying to us," you just look like you have your own agenda.
So if you can lie with statistics, you can certainly lie even more with claimed anecdotes...
(The point of a vaccine is to trigger and train the immune system. The immune system revving up can cause symptoms of its own. If you get the fever of the immune system responding, without the lung damage cause you don't have the original actual virus, that's a huge win!)
ubermanonJan 22, 2021
2. How trust in media can be regained
presents a chart labeled:
Percentage of Americans who trust traditional media
that has a Y-Axis from 45% to 60% rather than 0% to 100% in order to magnify their claims.
I'm not saying there is no media trust issue in America, but using clearly biased charting techniques taken right out of the book "How to Lie with Statistics" (by Darrel Huff) is not the way to instill trust.
PietertjeonDec 23, 2015
The Wright Brothers, David McCullough - wonderful book on the Wright Brothers, easy to read, shows that persistence and logic thinking pays out. 5/5
The autobiography of Malcom X, Malcolm X - Nice bio, a bit repetitive sometimes. 3/5
Atlas Shrugged, Ayn Rand - Although the book has more than 1200 pages it really kept me going. I read this book to get a better understanding of the ideology of some republicans. Fun read. 4.5/5
Freakonomics, Steven D. Levitt - Levitt studies all kind of different everyday questions using economics. 4/5, short, easy to read
Guantanamo Diary, Mohamedou Ould Slahi - Diary of a Guantanamo prisoner who has been imprisoned since 2002. The US has never charged him with a crime. Profound and disturbing. 5/5
No place to hide, Glenn Greenwald - Story on Edward Snowden, probably read by most of HN. Enjoyed it, that's it. 4/5
How to lie with statistics, Darrel Huff - Short book on statistics, easy to read and fun. 4/5
mooredsonOct 11, 2018
microtheriononMay 22, 2018
"A Pattern Language" by Christopher Alexander.
"The No Asshole Rule".
"How to Lie with Statistics".
wyconJune 23, 2017
https://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/03...
jacques_chesteronFeb 19, 2015
greenyodaonFeb 7, 2014
https://archive.org/details/HowToLieWithStatistics
mathattackonNov 5, 2013
But despite this, statistics done well are very powerful.
MzonMar 15, 2016
I think it is entirely fair to say that some oddball subgroup is skewing the results. It is no different from discussing mean, median and mode in real estate data.
Ever heard of the book "How to lie with statistics"?
xhedleyonMay 7, 2021
I own an inherited blue Pelican paperbook copy from my pharmacist grandfather.
https://en.wikipedia.org/wiki/How_to_Lie_with_Statistics
DoreenMicheleonDec 9, 2017
Don't get me wrong. I would be very interested in finding real solutions for the tendency to see Blacks gunned down by the police. My assumption is this known trend is the driving force behind this piece.
I just think this is the wrong way to approach the problem space.
How to lie with statistics is an excellent book. I highly recommend it. It is a large factor in my decision to not bother to read this piece too thoroughly.
HarkinsonMar 28, 2018
The authors of this report are former employees of the Illinois Policy Institute: http://www.wirepoints.com/mark-glennon/ https://www.illinoispolicy.org/author/ilpoliski/
The Illinois Policy Institute is, politely, a conservative think tank: https://en.wikipedia.org/wiki/Illinois_Policy_Institute
The full report plays a lot of games to produce these striking, misleading graphs, like comparing yearly numbers with total values of all future pension liabilities at face value instead of any attempt at NPV. If you're curious to spot more, the 1993 edition of How to Lie With Statistics chapter 7 ("The Semi-Attached Figure") and 9 ("How to Statisticulate") are a lighthearted read.
MzonDec 6, 2015
So it is possible to get past this, if you want.
I had my sons play math based games, read math books full of paragraphs and not problems, and so on. You might try engaging in math as recreation. Read books like "The Number Devil" and "Alice in Flatland" and "The cartoon guide to statistics" (or any cartoon guide book ever on any subject, because they are all basically awesome) and "How to lie with statistics."
Try to figure out what pieces make you stressed and look for a way to fix that piece of it. For example, if it is about dealing with people face to face, then email them an answer. Something like that can help break the pattern of stress and help you reboot your emotional relationship to the subject.
Best of luck.
DoreenMicheleonDec 2, 2019
It stuck with me because it's a nutshell version of the book "How to lie with statistics" in a humorous and highly memorable form. Someone with one breast, one testicle and a little penis would be a bizarre statistical outlier for the human race, but "mathematically" speaking, it's an accurate description.
It speaks to a lot of common errors of logic and it's pithy.
wodenokotoonSep 22, 2020
Particularly I recommend his visit on numberphile, where he gives a scathing critique of "How to Lie with Statistics"
[1] https://www.youtube.com/watch?v=-n-d1ApBTFw&t=3s
DoreenMicheleonSep 7, 2018
There's a great book called How to lie with statistics. It makes a lot of excellent points about how framing the same data differently can support different conclusions.
So, some thoughts:
Law makers who passed it may not have realized the average 65 year old would live another 12 years. They may have only known average life expectancy was 67.
Six years longer may not sound like much, but it's 50 percent longer. That can have significant repercussions.
MzonMar 8, 2017
A lot of so-called facts are just blurbs intended to support someone's manipulative agenda. Sadly, this is even true of this article. Towards the end, it begins making ugly political commentary instead of sticking to psychological phenomenon.
The people who claim to have science on their side are often merely claiming to be intellectual elites and implying that anyone who disagrees with them is an idiot. It isn't scientific at all. So, it is unsurprising that it does not go over well with people who are basically being called stupid by people more interested in winning the argument themselves than in trying to have a meaningful discussion of good information.
MzonApr 1, 2014
A lot of people do not understand the substance behind the numbers and this leads to garbage in, garbage out. That's what this article is about: Understanding what's behind the numbers and not being fooled by them. The previous piece was also about understanding the substance and knowing how to work the numbers to make other people happy with the proposed deal.
A good read on similar topics: How to Lie with Statistics.
* https://news.ycombinator.com/item?id=7451018
MzonJuly 12, 2017
When I homeschooled my sons, I knew this approach would not work. My oldest has trouble with numbers, but he got a solid education in the concepts. He has a better grasp of things like GIGO than most folks. We also pursued a stats track (at their choice) rather than an algebra-geometry-trig track.
Stats is much more relevant to life for most people most of the time and there are very user-friendly books on the topic, like "How to lie with statistics." If you are struggling with this stuff, I highly recommend pursuing something like that.
hnnewguyonMay 14, 2015
It isn't really disingenuous. It's become fashionable to claim that "all graphs should start at zero", but that's not necessarily how you display data. Here is Edward Tufte:
"In general, in a time-series, use a baseline that shows the data not the zero point. If the zero point reasonably occurs in plotting the data, fine. But don't spend a lot of empty vertical space trying to reach down to the zero point at the cost of hiding what is going on in the data line itself. (The book, How to Lie With Statistics, is wrong on this point.)"
He's talking about time-series in this case, but the point remains: you need to understand the data. Sometimes (as in this case, where the trend is important, not the magnitude of change) showing the "zero" point is irrelevant.
capnrefsmmatonJan 31, 2019
defrostonFeb 8, 2013
Matlab (and/or numpy / sage / any of the "math" packages) work best performing operations on vectors and matrices of data (long strings of values / grids & cubes (&hyper cubes) of values ).
Expanding on the "little mathematics background" is more important than the "no matlab" issue, one follows from the other.
If she's headed into epidemiology and other such areas then make sure she's read "How To Lie With Statistics" at the very least.
dmlorenzettionDec 30, 2013
If she's interested in biology, you might consider "Animals in Translation" by Temple Grandin. It's more about animals than about technology, but it touches on how understanding animals affects the low-tech systems that you use to handle them.
For interesting/easy intros to why some math skills are important, look at Darrell Huff's "How to Lie With Statistics" or John Allen Paulos' "Innumeracy" (or "A Mathematician Reads the Newspaper").
I haven't read any of them, but I hear Petroski's books recommended.
For a great short read that touches on architectural engineering, law, and ethics, look for Joe Morgenstern's "Fifty-Nine Story Crisis" which appeared in the New Yorker in May 1995. It's about how the structural engineer for the Citicorp tower realized its design was flawed, after the tower had been built and occupied. You can find it on the web.
Some of these are not overtly about "learn this technology", but to me they all sell the idea of technology and science as both central to our lives and interesting.
capoonOct 3, 2012
Any reaction to Bing could have been a result of people being exposed to it for the first time not necessarily it being the “better” option. It also could be that people are having a positive impression simply because it’s not as bad as they imagined it would be, as indicated by this claim: “64% of people were surprised by the quality of Bing’s web search results.”.
Also there is the matter of the ridiculous disclaimer about the features being omitted in the side-by-side as if they aren't integral to the search experience, not to mention that seemingly all queries to Google appear to be originating from Seattle which degrades the quality of local queries by user located elsewhere.
On a related note I recommend reading the How to lie with statistics book, it's required reading especially when outlets copy/paste press releases:
http://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/039...
valhallaonAug 14, 2015
DoreenMicheleonNov 9, 2020
jb775onFeb 15, 2021
There's no data sources provided, no way to check follow up health (i'e. what if these people all dropped dead a week later), no way to prove the data isn't a complete fabrication, etc. Also, they make sure to use dynamic terms so we can't be sure these numbers have any valuable meaning at all (e.g. were Covid tests with drastically different cycle counts used?). Not to mention all of this reporting is praising a single vaccine producer in a billion dollar industry.
I just read "How to Lie with Statistics" by Darrell Huff and this whole thing is the epitome of a manipulatable situation.
dangrsmindonFeb 26, 2007
It is clearly a biased sample, listing only "successful" companies but not the much larger set of unsuccessful ones. The "exception" of Excite thrown in with 6 founders, IMO makes the author's intentions (conscious or unconscious) clear since one would expect some argument about whether Excite was really a success in the end.
Then there is the question of what constitutes a founder. This isn't always so easy to define. How many people founded IBM?
Despite Thomas Watson being listed as the "founder" of IBM, it is hard to even answer this question since IBM was formed out of the merger of three companies before it was even called IBM. (http://www.answers.com/topic/history-of-ibm)
Several responders pointed out correctly that the number of potential communications channels rises as the number of founders increases. And while that's clearly true, it is also true that more founders means more mouths to feed early in the company's development. And to run any business you need to learn to manage communication amongst your team.
Finally, one has to be careful not to confuse correlation with causation. Just because Microsoft was founded by two people, doesn't mean that was the primary reason for their success. It might also have had something to do with who the two people were after all. ;)
DoreenMicheleonFeb 26, 2020
Social phenomenon are inherently hard to study in some kind of objective fashion. People will give the answers they know are expected of them in a world that is openly and unapologetically hostile to them.
"How to lie with statistics" is one of my favorite works. Two people with different points of view can look at the exact same data and reach completely different conclusions about what the data means.
Humans are incredibly prone to confirmation bias. Humans look for the data that supports their existing world view and hand wave off data that conflicts with it. This is a very well known phenomenon.
Edit in response to your edit:
We’ve had this discussion before months ago. We can just agree to disagree.
Sure. I don't recall discussing it with before and I didn't start this discussion. You did. You chose to reply to something I said here, knowing we've discussed it before and we don't agree.
So calls to "agree to disagree" sound a bit like I should just shut up and let you have the last word.
I'm not rebutting every single comment in the discussion of the article I wrote. If you didn't want to actually talk to me in specific, you absolutely didn't have to.
mcguireonSep 2, 2015
It's possible to love numbers and still not be interested in empirical data. One thing you may notice about much raw economic data is that it was selected because (a) it was easy to collect, or (b) it gives "reasonable results" according to the accepted theory.
See also How To Lie With Statistics.
Acrobatic_RoadonJuly 9, 2021
nathancahillonJan 24, 2014
I suggest reading "How to Lie with Statistics"[1]
[1] http://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/039...
Edit: I guess that sounded a little harsh. No doubt, Snapchat's growth has been explosive and if this was just another blog-spam post about that fact, I'd let it be. But you're selling a statistical software! I'd expect better from people who know what they're doing. Also, "How to Lie with Statistics" is a really good book, should be required reading for anyone dealing with numbers. Did not mean that as an attack on your product.
cromulentonOct 24, 2010
Baselines
In general, in a time-series, use a baseline that shows the data not the zero point. If the zero point reasonably occurs in plotting the data, fine. But don't spend a lot of empty vertical space trying to reach down to the zero point at the cost of hiding what is going on in the data line itself. (The book, How to Lie With Statistics, is wrong on this point.)
For examples, all over the place, of absent zero points in time-series, take a look at any major scientific research publication. The scientists want to show their data, not zero.
The urge to contextualize the data is a good one, but context does not come from empty vertical space reaching down to zero, a number which does not even occur in a good many data sets. Instead, for context, show more data horizontally! .
-- Edward Tufte, October 18, 2001
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0...
briantakitaonJuly 19, 2021
It's interesting that it takes an editorial to make people suspicious of statistics, how statistics can be abused, & the conflicts of interests that many people who utilize statistics have. Sample bias needs to be treated as deliberate dishonesty rather than a simple mistake. These people who make these mistakes are professionals and should know better. Their code of conduct should penalize them harshly for making these sort of mistakes.
A strict code of conduct with harsh professional penalties are necessary to remove bad actors who hide behind subtle lies that have a major impact on public policy & public opinion. A slap on the wrist means it's always worthwhile to lie with statistics. A removal of license & banishment from the profession on the 1st or 2nd offense would quickly remove the bad actors. This code of conduct should also extend to the peer review process. If the peers pass bad statistics, the peers need to be held accountable as well.
IvyMikeonFeb 23, 2014
https://archive.org/details/HowToLieWithStatistics
habermanonOct 24, 2010
Ugh, the more I read of Tufte the less I like him. He loves to make blanket statements (like calling a point "wrong") in an authoritative way, and he's amassed enough adoring followers that people repeat his words like gospel instead of calling him on it.
It's valid to say that a non-zero-based scale shows the data better. But it's also completely valid to note that non-zero-based scales can be used in alarmist ways to make data extremely misleading.
I thought "The Visual Display of Quantitative Information" was terrible, and I went in really wanting to like it: http://www.amazon.com/review/R11NYC3OE3LBE
hyperbovineonFeb 11, 2012
akg_67onAug 13, 2015
* Darrell Huff, Irving Geis How to Lie With Statistics
* Ian Ayres Super Crunchers Why Thinking-by-Numbers Is the New Way to Be Smart
* David Salsburg The Lady Tasting Tea How Statistics Revolutionized Science in the Twentieth Century
* Emanuel Deman, Models Behaving Badly
* Benoit Mandelbrot, Richard L. Hudson The Misbehavior of Markets A Fractal View of Financial Turbulence
* Nassim Taleb, Fooled by Randomness
MzonSep 28, 2015
B) However, I also lived in the bay area at one time, in Solano County, and was pursuing education with an eye towards going into some kind of urban planning related career. In fact, I founded and moderated a subforum for a time on the most successful urban planning forum around at that time. So I have some familiarity with how crazy prices were back then, before AirBnB was a gleam in anyone's eye. And also I have some familiarity with the various factors that go into forcing housing prices up. Saying AirBnB contributes to the problem is not crazy talk. But acting like they are the single most important factor meriting the passage of a bill intended to kill them off -- I want a tad more data than "But look at the crazy high local housing prices, man!" Because that falls far short of proving they are having that big of an effect.
C) Yeah, I am very familiar with the saying. I am well aware of how hard it is to be both profitable and ethical. So far, I have managed to be pretty ethical. I am also dirt poor. So I am a little tired of hearing that anyone making money is clearly The Devil. The fact that this is part of a pro AirBnB PR campaign does not ipso facto make it inherently evil. The other side is also engaging in a PR campaign, and they also have vested interest that you can put a dollar amount on. Sometimes, people are actually doing work they actually fucking believe in. Those people still need to EAT and put a roof over their head. I am so goddamn sick of the idea that all the good people are dead martyrs and, if you still draw breathe, you need to feel guilty about every single fucking thing you do to try to keep body and soul together.
lordnachoonNov 24, 2020
For a start, schools could teach people about how we actually unravel truths from statistics. It's a bit unfortunate that one of the most famous books in the area is called "How to Lie with Statistics". But all those things you mention, did you learn them in school? No, you tend to learn the conclusions but not the reasoning behind.
dredmorbiusonOct 10, 2013
DoreenMicheleonFeb 10, 2020
So why didn't pay for programmers plummet when "supply doubled" because we added men to the equation?
I'm happy to entertain explanations like "Well, whether it is PC or not, everyone knows women end up doing the housework. So the reality is that men typically have more time, energy and mental focus to offer, so it's not unreasonable to think they will bring more to the table."
But please don't give me mathematical BS explanations that don't fit the data. I've had a class in statistics and I've read plenty of statistics books, including a great classic called "How to lie with statistics." Math doesn't really explain the gender based outcomes that have been well documented for decades at a minimum.
mechanical_fishonJune 5, 2008
Essentially, all results from Google Trends are normalized.... More and more non-technical people are using google and thus the percentage of searches for programming declines.
Sounds reasonable to me. These graphs don't even have units on the Y axis. They're useless without some idea of what they're measuring: absolute or relative numbers?
Somebody needs to read How to Lie With Statistics.
DanBConMar 10, 2012
I agree with you that a poor metric will do harm, and that there are plenty of examples in "western" manufacturing. (I used to work in electronic sub contract manufacturing, and I saw the nonsense that was ISO900x documentation.)
Going further, there are a few things which need new names.
"Mexican Bridge" - there's a four lane road bridge and politicians want to improve traffic flow so they remove the lines and paint more, turning it into a 6 lane bridge. That's a 50% increase in lanes, but now they are narrower and so there are more accidents. The politicians decide to go back to four lanes. That's a 33% reduction. Because they are politicians they call this a 50% - 33% = 17% increase. (This is from "How to lie with statistics", a great book with occasional uncomfortable language.)
"Chinese Whispers" - the childhood game where one child starts by whispering a sentence to her neighbour, who whispers it to his neighbour, etc, and then at the end of the chain you compare the original with the final sentence. I've heard this called "Broken Telephone"; but no-one (in the UK) knows what I mean when I call it that.
"Indian Giver" Louie CK does an excellent routine on this.
And then there are other things which are fine, but which some people think aren't: "Nitty Gritty" (nothing to do with slave trading ships) "Chinese wall" (in business, no racist connotation) etc.
pokoleoonMar 27, 2016
Most of the usual tricks ('drop the axes', percentage-points, etc) are there, but there are many other, less obvious tricks.
One of the cooler arguments in the book is that it's easy to lean on someone's implicit assumption of volume to modify their understanding.
If you inflate a 15% increase in house spending to look larger than it is, drawing pictures of houses that are 15% wider will make people intuit a 50%[1] increase, despite reading 15%. The author suggests that even if you're incredibly clear with the text surrounding the charts, people still use the charts to understand the scales of change.
[0] http://amzn.to/1RmCVmL (disclaimer: affiliate link)
[1] 1.15^3 ~= 1.5
kvcc01onJune 4, 2015
That said, I think everyone would benefit by reading a bit about these subjects, whether or not they’re developers. We seem to have a built-in tendency to underestimate the effect of randomness in life (see Fooled by Randomness and How to Lie with Statistics). To counteract that, I find statistical methods (e.g., hypothesis testing, confidence intervals, etc.) to be very useful tools to keep in mind. Having basic familiarity with such tools may not necessarily make us better programmers but it'll probably make us more rational decision makers in the general sense.
Kurtz79onMar 11, 2019
From the linked article:
"There are many possible explanations for the sudden demise of Huff’s book.
The industry realized in 1968 that it had a credibility problem: the public
was unlikely to trust a book funded and promoted by the Tobacco Institute.
Perhaps the campaign was shelved after Tiderock closed up shop. Perhaps it
was too late—some tobacco regulations had already been passed, others were
inevitable, and new tactics were needed. Or perhaps the thought of legal trouble made them retract their offer to advertise the book, making Macmillan kill
the project.
But Macmillan was on the brink of signing a contract with Huff. Did
they have second thoughts, or was Huff told to kill the project? Or were the
Tiderock consultants correct in their assessment that “this mass of verbiage
needs drastic editing” and was unpublishable without serious revision?
The Documents Library offers no clues. Only documents produced or
received by the industry are included, so any correspondence between Huff
and Macmillan may not appear. Andrew Gelman, professor of statistics at
Columbia University, reviewed the ethics of Huff’s involvement with the industry and suggested Huff could have intentionally killed the project to save
his own reputation, which would have been destroyed by his association with
tobacco.2 But just a few months before the book’s demise he had been fighting for a prestigious hardcover and better royalties.
Whatever the reasons, How to Lie with Smoking Statistics went unpublished, protecting its better-known sibling How to Lie with Statistics from guilt
by association."
Would the original book have commanded the same respect, if associated with another which would bascially be an example of what the first was meaning to expose?
throwawayswedeonJuly 13, 2021
I didn't claim to know everything there is to know about statistics and research in general, but I think I know enough to tell that this is not a study, but a marketing campaign for whatever this usehaystack thing is.
> First, they should have tried to poll around 400 people (and perhaps they did!),
If you had read the article, you would have known that that's not what they did.
> By typical polling standards (95% confidence interval, random sample from filtered population), we expect that 77-89% of UK software developers are actually burnt out. That is still three out of every four people in the industry!
Do I understand correctly that you've worked on this? That certainly explains the hostility, but nevertheless.
> we expect that 77-89% of UK software developers are actually burnt out.
That's not what the link/study says.
> That is still three out of every four people in the industry!
This is absolutely wrong, without a shred of a doubt.
I recommend that you read: How to lie with Statistics by Darrell Huff.
mechanical_fishonSep 9, 2008
W.W. Sawyer, What is Calculus About? and Mathematician's Delight
Courant and Robbins, What is Mathematics?
Hogben, Mathematics for the Million
Steinhaus, Mathematical Snapshots
Ivars Peterson, The Mathematical Tourist
Davis and Hersh, The Mathematical Experience
Polya, How to Solve It
Huff, How to Lie With Statistics
McGervey, Probabilities in Everyday Life
Raymond Smullyan: The Lady or the Tiger, Alice in Puzzle-Land, others
Anything by Martin Gardner. I happen to have picked up Mathematical Magic Show and Mathematical Circus, but I'm sure there are many other collections.
I also recommend cryptography stuff. David Kahn's The Codebreakers is not really a math book, but it is awesome and it stars mathematicians, as does Simon Singh's The Code Book. You could read Schneier's Applied Cryptography.
This is HN, so I would be remiss if I didn't point out that you can learn a lot of fun and useful math by reading SICP, Knuth, or any good algorithms book.
If anybody out there knows a good, spirited statistics book addressed to someone who knows calculus, tell me. I keep planning to go through Fundamentals of Applied Probability Theory but I never get around to it; see "Related Resources" here:
http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Compute...
Having said all of that: I have a Ph.D. in physics/EE, so I've got to tell you, if you haven't tried calculus you haven't lived. ;) I'm not sure how to go about learning calculus in a fun way for a mathematician -- I took fairly standard first- and second-year college courses in calculus and physics and learned it that way. The folks on Amazon seem kind of enthusiastic about Spivak:
http://www.amazon.com/Calculus-Michael-Spivak/dp/0914098918/...
drcubeonJan 25, 2013
Not that I think you or the article is lying. But I think you are confused by the article's use of the word "average". To you, it means "mean". To the article, it means "median". If you had both used the word you meant, there wouldn't be an issue.
itcrowdonSep 11, 2019
For a popsci work, you could check out "how to lie with statistics", a classic.
https://en.m.wikipedia.org/wiki/How_to_Lie_with_Statistics
cpercivaonApr 5, 2009
DoreenMicheleonFeb 29, 2020
It was something I saw in a print magazine a long time ago. I can't cite it.
It's easy to pull out a single phrase, like "never celebrated," and nitpick it. In comparison to the handwringing in the news at the time about the imminent global catastrophe we were facing, the success of getting it out out and averting the level of global atmospheric catastrophe that was expected wasn't really celebrated.
We did the same with Y2K. The world expected a financial meltdown. People were stockpiling guns and give years of flour in their basements and then it became "Gosh, I can't program my VCR."
We take success for granted. We don't celebrate that we have 7 billion people with long life expectancy and high standard of living compared to 200 years ago. Instead, we complain about overpopulation and global warming and how we are hurtling towards our doom.
"How to lie with statistics" is a terrific book about how we can take the same data and say different things with it. We tend to frame it quite negatively.
In the end, none of us is getting out of here alive. We're all just dancing on this Earth for a short while.
There are people literally trying to find ways to solve death and wondering if we can be frozen and brought back and so forth. We're never satisfied and there's some evidence that it's partly because we choose to focus on the negative and with 7 billion people there no shortage of bad news, but not because things are necessarily actually worse in absolute terms.
mechanical_fishonOct 13, 2008
There's nothing wrong with hypothesizing that (a) modern medical practices are leading to deficiencies in Vitamin D among certain populations and (b) Vitamin D deficiency is linked to autism. But the responsible next step is to get some good data, or at least link to some. It's not to write stuff like this:
Since the 1970's, our country has witnessed a 3000% increase in the sales of sunscreen products, alongside a 30% increase in deaths from melanoma[17]. Isn't it time to admit that sunscreen is not working?
This is a telling paragraph. It tells me that the author is prone to making mistakes straight out of Darrell Huff's How to Lie With Statistics: Clock A strikes midnight a few seconds before clock B; isn't it time to admit that A caused B to strike?
Autism seems to be one of those issues that really inspires the quack epidemiologist. When you combine it with the opportunity to lecture new mothers -- our culture's favorite vice -- the temptation becomes irresistable. But it's worth remembering that actual epidemiology is really hard. Diagnosing autism without bias is hard. Randomly sampling the population for autism is even harder. Even something relatively simple, like measuring Vitamin D levels in the blood, is tedious and expensive and difficult to do without bias. But just because anecdotes and handwaving are much easier doesn't make them more believable.
kriroonFeb 2, 2017
drallisononMar 27, 2017
dmlorenzettionFeb 11, 2011
The exercise was to take a set of data and make two graphs, one demonstrating a point and the other demonstrating its opposite. I think the data had to do with student test scores over the years-- they had to prove that scores were going up, and that they were going down, by their choice of what to plot out of the dataset.
I really liked working with that book in the class. Its form factor makes it non-intimidating, not like a textbook. And I felt like the title added a frisson of transgression.