Hacker News Books

Great book. Also check out The Signal and the Noise by Nate Silver.

Nate Silver's The Signal and the Noise [1] is excellent.

[1] http://www.amazon.com/Signal-Noise-Many-Predictions-Fail--bu...

I liked Nate Silver's chapter on climate modeling predictions in his book The Signal and the Noise. He wasn't impressed. Maybe somebody needs to start a fantasy weather league.

while we're at it, Nate's book The Signal and the Noise is a pretty easy and fun read if you're into statistics, polling and politics

Nate Silver's "The signal and the noise" had an excellent chapter on it. Did a good job getting to the core of the argument, putting it into non-specialist terms, and evaluating the results skeptically, but with scientific rigor. And no agenda. Highly recommended.

You could do worse that reading "The Signal And The Noise". Very well written and entertaining while it gently teaches you The Way Of The Bayesian.

The Signal and the Noise by Nate Silver talks about this a lot. From predicting politics to economics to sports to climate change, group predictions are almost universally better than individual expert predictions.

"The Signal and the Noise" might qualify. How to reason about systems of messy data.... like the weather for example. Very well written and entertaining.

I know exactly what you mean. Most predictions are cast into the wind and only the ones that are correct are recognized later. Nate Silver talks about this in his book "The Signal and The Noise"; I highly recommend it.

Nate Silver's book The Signal and the Noise explained it to me really simply, and was a great read. Not as immediately helpful perhaps as some of the links in response to your question, but I thought it was worth sharing.

Just finished reading Hugh Howey's "Dust". The Wool, Shift, and Dust series from Hugh Howey are great dystopic sci-fi. Picked up The Signal and the Noise from Nate Silver as my next book.

+1 Thinking, Fast and Slow. I was going to post that as the most influential book of the year.

(I started reading Nate Silver's The Signal and the Noise when I got to a reference to Thinking, Fast and Slow. Switched and haven't gone back yet.)

Also reading The Signal and the Noise: Why So Many Predictions Fail — but Some Don't - Nate Silver

And also in progress:

Buried for Pleasure - Edmund Crispin

The Dying Trade - Peter Corris

Think Bayes - Allen B Downey

Coincidently, I just finished reading The Signal and the Noise by Nate Silver too. I found it interesting and not too technical. It covered lot of ground.

Now, I am reading Innumeracy by John Allen Paulos. It is more in depth and technical than Nate's.

It's interesting, but it's not news. Nate Silver published this information in his book The Signal and the Noise back in 2012.

You should read Silver's book; "The Signal and the Noise". There he reveals that he was a poker and baseball geek long before and much more than a political analyst. In the book he actually shows respects for both views.

A really good book to read on this is "the Signal and the Noise" by Nate Silver. Basically, the scientific method requires modeling the system correctly which only works for certain kinds of systems. Systems where the complexity exceeds the available data causes people to see patterns where they don't exist and to be overly confident in their interpretation of the data.

The entire area of nutrition and weight management is a joke. Medicine is problematic and Psychology is questionable.

Another problem is that the popular media publishes inconclusive studies because they are click bait. People read them because they are interested in living longer and losing weight and being happy.

Fear also turns studies into clickbait. The latest was the blizzard of articles on if bugs are dying off or not.

Then there is the fact that negative studies tend to be ignored and when the study failed or it introduces data that does not conform to general knowledge.

All this makes people question studies where the data is conclusive.

I read this morning in Nate Silver's new book (recommended by Fred Wilson and published September 2012) "The Signal and the Noise" that the average time a stock was held was 6 years a few decades ago, recently the average is closer to 6 months.

I don’t know since they aren’t accompanied with comments but my hunch is they may be conflating scientific consensus with model uncertainty. I.e., if I bring up model uncertainty, then I’m casting doubt on the scientific consensus (which isn’t the case). It’s more about our confidence in the models which is the point of the article.

Nate Silver’s “The Signal and the Noise” gives a much better description than I could do

In the 9 days since we still have models that are all over the place. Nate Silver covers the difficulty in predictive modeling of pandemics in his book "The Signal and the Noise" and in the years since it was written not much has changes to they're basically rehashing the points he made then.

I like Nate Silver's book, "The Signal and the Noise." If you read it, you shouldn't focus on the scenarios he describes specifically. You should try to take his mindset, walk away from the book, and try to decode the signal from the noise everywhere.

It's a helpful book, perhaps maybe even more so than your typical self-help book. I would have liked for such a book to have existed when I was a teenager.

"It's just my opinion" strikes me as the larger evasion of responsibility here. There's nothing irresponsible about reporting the facts of a story so that people can make informed decisions.

> More data can also mean more noise.

Nate Silver wrote a book titled "The Signal and the Noise." I'm sure he's aware of the danger. Data Analysis (Data Science, whatever you want to call it) is all about turning data into information. Reducing noise.

Ok, I finally watched the video and I don't think it was worth my time at all. It was nice that they were polite to each other but really we have understood the basics of how increased C02 causes global warming for about a century. Current detailed climate models are kind of crap, Nate Silver went into why in some detail in his book on modeling, The Signal and the Noise, which I recommend but that doesn't change the anything I said in the above post - climate models are as likely to undershoot the dangers of warming as they are to overshoot them. Christy basically seemed to be arguing that there's uncertainty so he was permitted to believe what he wanted to. That's not a rational way to approach problems, especially given how he exaggerated the nature of the uncertainty.

In the book "The Signal And The Noise" Nate Silver talked about political campaigns quite a lot. Definitely worth a look.

Both, "Consilience" by E.O. Wilson and...

"The signal and the noise" by Nate Silver

are worth look.

Hello, I studied economics and statistics in college and grad school, and worked as a teaching assistant for undergraduate statistics courses. Here is a short, annotated bibliography of my favorite statistics books.

1. Ayres, Ian (2007) Super Crunchers: Why Thinking by Numbers is the New Way to Be Smart

[Good introductory summary of the main concepts in statistics with many real-world examples]

2. Bernstein, Peter (1996) Against the Gods: Remarkable Story of Risk

[Intellectual history of statistics, accessible to beginning students.]

3. Healey, Joseph (2005) Statistics: A Tool for Social Research, 7E

[This is the text book that was used in the undergraduate statistics courses while I was working as a teaching assistant at UC Santa Cruz.]

4. Kahneman, Daniel (2011) Thinking, Fast and Slow

[Kahneman combines cognitive psychology with statistical concepts; highly recommended]

5. Silver, Nate (2012) Signal and the Noise: Why So Many Predictions Fail, but Some Don't

[Silver's book offers an excellent summary of major concepts in statistics and how they are applied to real-world problems]

6. Taleb, Nassim Nicholas (2005) Fooled by Randomness, 2E

_________ (2010) Black Swan: Impact of the Highly Improbable, 2E

[Important critique of statistics and how it is mis-used and mis-applied, particularly in econometrics]

Hope this helps. Shoot me an email if you have any questions. Good luck.
mitchelldeacon9@gmail.com

I think it's more about the way we communicate model results. In the media, it's communicated as absolutes like "The ice caps will be gone by 2100!"

In reality, the model results are communicated in uncertainty. More akin to "There's a 70% chance the ice caps will be gone by 2100 if the current level of CO2 emissions are not abated." That's a much less sexy headline to say nothing about the fact that most people wouldn't be shocked if a 3 in 10 chance event actually happens. Nate Silver does a much better job explaining this in his book "The Signal and the Noise"

Here's my whole list for the year in reverse chronological:

- Hillbilly Elegy by JD Vance

- Tools of Titan by Tim Ferriss

- Competing Against Luck by Clayton Christensen

- Scrum: A Breathtakingly Brief and Agile Introduction by Chris Sims

- Build Better Products by Laura Klein

- Capital in the Twenty-first Century by Thomas Picketty

- Shoe Dog by Phil Knight

- Lean Customer Development by Cindy Alvarez

- Impossible to Inevitable by Aaron Ross & Jason Lemkin

- Grit by Angela Duckworth

- Love Sense by Sue Johnson

- Thinking, Fast and Slow by Daniel Kahneman

- Working Effectively With Legacy Code by Michael Feathers

- Smarter Faster Better by Charles Duhigg

- Sprint by Jake Knapp

- Fooled by Randomness by Nassim Taleb

- Becoming a Supple Leopard by Kelly Starrett

- Superforecasting by Philip Tetlock

- The Inner Game Of Tennis by Timothy Gallwey

- Design Sprint by Richard Banfield

- The Structure of Scientific Revolutions by Thomas Kuhn

- The Signal and the Noise by Nate Silver

- Advanced Swift by Chris Eidoff

- Siddhartha by Hermann Hesse

Some of these books are older and had been on my list for awhile. Some were released this year. Most of these books are very good. I usually stop reading bad books by the end of the first chapter.

Poor Economics - Abhijit Banerjee & Esther Duflo

Thinking, Fast and Slow - Daniel Kahneman

The Signal and The Noise - Nate Silver

Predictably Irrational - Dan Ariely

This Time is Different - Carmen Reinhart & Kenneth Rogoff

Subliminal - Leonard Mlodinow

Fooled by Randomness - Nassim Taleb

For something more lighthearted, I also enjoyed:

An Economist Gets Lunch - Tyler Cowen

Inside Jokes - Matthew Hurley & others
(mostly skipped the dense parts)

50% means that they ran the simulation say 10 times and in 5 of those simulations there was rain in the area. How to translate that to real life - you should probably carry an umbrella if you don't like getting wet. Also interesting to note that weather presenters usually bump up the probability of rain by 10-20%, because people complain about false negatives (rained when it was predicted not to) rather than false positives (didn't rain when it was predicted to).

If you're interested in a good intro to this topic I recommend the chapter on weather prediction from The Signal and the Noise by Nate Silver. Its a fascinating read.

One interesting fact that I learned from Nate Silver's book, The Signal and the Noise, is that weather forecasting is a four-dimensional problem (space + time), so to produce a forecast that is twice as detailed requires approximately 16x the computing resources. Historically, the resolution of weather forecasts has doubled roughly every eight years, in line with with Moore's Law.

Fooled by Randomness by Nassim Nicholas Taleb

The Signal and the Noise by Nate Silver

Well, I think it's distinct, but I guess I'm biased. I was reminded of Richardson from the Nautilus link you reference, but I thought this article was a little more accessible. I got interested in Richardson from a chapter of Nate Silver's book "The Signal and the Noise", although it's focussed more on his attempts at predicting weather. Adaptation by Silver here: http://www.nytimes.com/2012/09/09/magazine/the-weatherman-is...

I also considered this shorter summary: http://world.std.com/~jlr/comment/statistics.htm

The Wikipedia article on Richarson:
http://en.wikipedia.org/wiki/Lewis_Fry_Richardson

This 1957 scholarly piece: http://deepblue.lib.umich.edu/bitstream/handle/2027.42/67679...

And this full UC Press book:
http://www.ucpress.edu/op.php?isbn=9780520038295

I only found them now, but a couple chapters by Richardson himself are also online:

Statistics of Deadly Quarrels: http://www.unz.org/Pub/NewmanJames-1957v02-01254?View=PDF

Mathematics of War and Foreign Politics: http://www.unz.org/Pub/NewmanJames-1957v02-01240?View=PDF

Out of those, I thought the link I submitted was most appropriate. But no problem is you want to bury it and keep the discussion focussed on the Nautilus article.

Nate Silvers wrote an entire book on this subject called "The Signal and the Noise" [1]. Humans are so often taken in by people claiming to be able to make predictions by combining new data points. The more unusual, or unrelated to the subject matter, the better. They make good headlines but (not surpsingly) almost always turn out to be heavily flawed in practice.

You can basically measure how much a pundit/expert is going to be wrong in their predictions by how ideological they are in their analysis. The best indicator is when they use only one or two metrics as a basis of a prediction of an otherwise very complex scenario.

One example from the book is how a researcher became famous before the 2000 US presidential elections by claiming to predict races with 90% accuracy [2]. He claimed that by measuring a) per-capita disposable income combined with b) # of military causalities you can determine whether democrat or republicans get elected. He said historical data backs up his theory. He then proceeded to fail to predict that years election and faded into obscurity.

Nate did his own historical analysis and demonstrated it was only 60% accurate instead of 90%. Plus that was only if you ignore 3rd party candidates as the model assumes a two-party system.

Plenty of other examples are provided in the book which makes me highly suspicious of the value of the predictions made in this article.

The general idea is that we need to stop looking for simple one-off solutions to complex problems. Instead we should adopt multi-factor approaches which suffer from fewer biases and are better grounded in reality. Otherwise these predictions are just another form of anti-intellectualism.

[1] http://www.amazon.com/Signal-Noise-Many-Predictions-Fail--bu...

[2] the "Bread and Peace" model by Douglas Hibbs of the University of Gothenberg http://query.nytimes.com/gst/fullpage.html?res=9803E5DD1F3DF...

This is very dangerous advice indeed. You got lucky, you got out in 2007: for every person like you, there are many more who thought they had the answer too (and knew when to get out) and did so too late. This is the irrational exuberance of markets, as documented in every cycle in history.

Don't read Mises.org, or at least not only. Read John Kenneth Galbraith's work: "A Short History of Financial Euphoria", "The Great Crash". Read Michael Lewis' work: "Boomerang", "The Big Short". Read even the pop economics works that expand on how people think, versus how they act: "Nudge", "Thinking: Fast and Slow".

Read Nate Silver's "The Signal and the Noise", and you'll see that predicting the market has improved no more than earthquakes has. There is no "physics" of the market, there are no silver bullets and predictors who understand everything will happen. If those people existed and acted on their bets, they would be wealthy beyond reason.

Skip "The Black Swan", and go straight for "Antifragile". It's Taleb's opus, according to him, but it describes the error in trying to make these predictions. They're very fragile. Your 2007 "pulling out of the market" could have been brilliance or folly, and you just happened to get lucky.

But don't take my word for it, take this quote by Keynes:

"The market can remain irrational longer than you can remain solvent."

No one can predict the future, least of all the irrational herd mentality of the market. Anyone who says they can, or they have a formula, or that they know "when to pull out of the market" is a charlatan or lucky, or both.

Silver's 2012 book "The Signal and the Noise" discusses our inability to rationally process probability, pointing out that commercial weather forecasts (e.g. Accuweather) never list a probably of rain under 20-25%. A 5% chance of rain is a mathematical possibility but people "feel" like 5% = "will never happen" and get angry if it rains.

Nobel Prize winner Daniel Kahneman's life's work is about this, what he calls "System 1" and "System 2" of our brain, where System 1 is a fast responder that provides insta-feedback but is largely incapable of processing mathematical inputs. His 2011 book "Thinking Fast and Slow" summarizes his work well.

I'm not sure popular media can be trained to frame statistical probabilities in a way that doesn't provide people with the certainty they crave. But who knows?

Exactly. And it's even more complicated: the author assumes that funding is distributed evenly over the year, when it's not. Very few deals close during the summer, and the period September-December is particularly active.

To the OP: you are jumping to conclusions too quickly without asking the right questions. You also seem too attached to your hypothesis. I recommend that you read Nate Silver's new book "The Signal and the Noise."

FWIW despite not having been in school for nearly 2 decades, I take a few short notes on all books I read in a personal wiki.

I don't do this as I read, but after reading the book. Or sometimes a couple times while reading the book if it's particularly interesting. If you do it as you read, you may fall into the trap of taking too many notes. And the forced recall after a few days improves retention, rather than a recall after 5 minutes.

Crucially, I connect the ideas to other books and Internet articles I've read. It's surprisingly easy IME to let keep a book in a knowledge silo; only by connecting the information will you actually learn something.

For example, I read both Nate Silver's Signal and the Noise, and Taleb's Black Swan / Antifragile, etc. Both of the books are roughly on drawing conclusions from data with statistics (or really the limitations of statistics in the latter case).

But if you don't actually compare the two books then it's pretty easy to read but not understand. I don't claim that I understood everything, but certainly taking a paragraph of notes is better than nothing.

----

Paul Graham has a nice essay about whether you actually remember what you read!

http://www.paulgraham.com/know.html

I also recall reading a NYTimes piece about this. It's good to wonder about your retention, like the author of this blog post is. Although like others, I don't agree with the title. I think it's entirely possible that many people are not good readers. That is a skill that has to be taught -- there's a reason that every child in the states is MANDATED TO DO for ~12 years. And it can be done poorly, or it can be done well.

I think if you don't actually check if you know something, then you don't know it. So I think I agree that many people could improve their retention of books with a few small adjustments to their process.

People are reading and writing more than ever with the Internet, and I generally view that as a good thing. But I suppose it wouldn't be surprising if the retention of long form books suffered due to that development over the last couple decades. People learn to read earlier and "informally" rather than in a more deliberate fashion.

Nate Silver discusses the Fox/Hedgehog division in his excellent "The Signal and the Noise" frequently, and he largely takes on the meaning proposed by Phillip Tetlock in his book on expert political judgement.

The idea that Tetlock puts forward is that pundits divide into two categories, Foxes and Hedgehogs. Hedgehogs explain the world and make predictions according to one large all encompassing principle. Foxes don't, and apply different principles to different circumstances. Tetlock concludes, and Silver endorses this, that Foxes are better at understanding the world than Hedgehogs.

An example might be the (proposed) principle that free markets lead to increased prosperity. A hedgehogy capitalist might state the above as a broadly applicable principle. A foxy capitalist might opt for something like: "yes, many times increased freedom makes us prosper, but sometimes it doesn't, it depends, let's talk about the specifics".

I take this to be an argument about complexity. The fox side of the argument is roughly that the world is more complex than most people allow for, and one principle or world view will usually not cover an area of knowledge well.

It is pretty boring. Big data doesn't necessarily mean big insights. So lots of times it can end up being a wild goose chase with management not realizing they are funding science experiments.

For any one thinking of it as a career, I highly recommend Nate Silver's - The Signal and the Noise: Why So Many Predictions Fail - But Some Don't

Signal and the noise by Nate Silver(Very nice read. Chapters on climate change and GDP forecasting were a bit slow, but everything else was a page turner)
http://www.amazon.com/dp/159420411X

Why I left Goldman Sachs by Greg Smith (Good insight into the 2008 financial breakdown and a look into the day to day operations of Goldman Sachs)
http://www.amazon.com/Why-Left-Goldman-Sachs-Street/dp/14555...

The Hobbit
http://www.amazon.com/Hobbit-There-Again-Illustrated-Author/...

Data Mining: Concepts and Techniques(Great intro into data mining)
http://www.amazon.com/Data-Mining-Concepts-Techniques-Manage...

Programming Collective Intelligence(You can play around with actual implementations of the concepts in the previous book)
http://www.amazon.com/Programming-Collective-Intelligence-Bu...

Ghost in the Wires by Kevin Mitnick (Was really nice to see the details behind Mitnick's adventures)
http://www.amazon.com/Ghost-Wires-Adventures-Worlds-Wanted/d...

On War By Clausewitz(Really enjoyed this book.)http://www.amazon.com/War-Carl-von-Clausewitz/dp/1448676290

In no particular order and probably not remembering all:

The signal and the noise - Nate Silver;

Black Swan - Nassim Nicholas Taleb;

Antifragile - Nassim Nicholas Taleb;

1984 - Orwell;

Man's search for meaning - Viktor Frankl;

Diplomacy - Henry Kissinger (not only international politics but also deep-thinking strategy that can be used anywhere);

Meditations - Marcus Aurelius;

Superforecasting - Philip Tetlock;

Propaganda - Edward Bernays;

Pitch anything - Oren Klaff;

Guns, Germs and Steel - Jared Diamond;

How to win friends and influence people& Stop worrying (both by Dale Carnegie);

The Selfish Gene - Richard Dawkins;

Trust - Francis Fukuyama;

The author asks,"Is one of these facts untrue?". Clearly the answer is that the second "fact" is not a fact it's just provocative ranting from the mouth of yet another journalist who's trying to get clicks by invoking Musk's name.

Most people's view of economics serve primarily as an intellectual Rorschach Test and in this case both Musk's and this author's perceptions seem wide of the bull's eye but in opposite directions.

Economics has almost no "first principles" of which Musk is fond and so he lacks a reference point. The author on the other hand seems to be one of those who makes up their reality out of whole cloth. In short a "hedgehog" according to the book "The Signal and the Noise".

Surely we can do much better than Nathan J. Robinson.

Currently reading The Signal and the Noise: Why So Many Predictions Fail — but Some Don't - Nate Silver

Before that, I had just finished The Idea Factory: Bell Labs and the Great Age of American Innovation by Jon Gertner.

Next, I may read Naked Statistics: Stripping the Dread from the Data by Charles Wheelan.

Funny, I just posted this: http://willsllc.github.com/blog/books-2012/

Most Thought-Provoking Books of 2012

The Idea Factory: Bell Labs and the Great Age of American Innovation
by Jon Gertner published March 15, 2012
Over the span of a few decades, a single research lab invented the transistor, the microprocessor, radar, the communication satellite, the CD, and more.

The Power of Habit: Why We Do What We Do in Life and Business
by Charles Duhigg published February 28, 2012
Why toothpaste tingles, how Febreeze was a flop, and hundreds of other tidbits that are perfect for cocktail parties and future Jeopardy episodes.

The Signal and the Noise: Why So Many Predictions Fail-but Some Don't
by Nate Silver published September 27, 2012
Weaves together baseball, earthquakes, the weather, poker, and terrorism. Chapter 7 is the best description of Bayes theoreom I've ever read.

The Honest Truth About Dishonesty: How We Lie to Everyone---Especially Ourselves
by Dan Ariely published June 5, 2012
The third Ariely book, and just as fun. Would be ranked #1 except it's essential the same formula as his prior two gems.

Automate This: How Algorithms Came to Rule Our World
by Christopher Steiner published August 30, 2012
Surprisingly good read from a first-time author (and YC alum). Expands on Andreessen's quip to cover trading, couter-terrorism, the Arab Spring and more.

Rather than waiting for data from future games, you should backtest on old data and see how it performs (http://en.wikipedia.org/wiki/Backtesting).

You might also be interested in checking out the book The Signal and The Noise by Nate Silver (He runs the FiveThirtyEight political/data blog that notably predicted last year's election results with great accuracy.)

Non-fiction:

On Intelligence - Jeff Hawkins

How To Create A Mind - Ray Kurzweil

The Language Instinct - Steven Pinker

The Origin of Wealth - Eric Beinhocker

The Signal and the Noise - Nate Silver

The Money Culture - Michael Lewis

Knowledge and the Wealth of Nations - David Warsh

Smart Machines: IBM's Watson and the Era of Cognitive Computing - John E. Kelly III and Steve Hamm

The Idea Factory - Jon Gertner

Winning The Knowledge Transfer Race - Michael J. English and William H. Baker

Wellsprings of Knowledge - Dorothy Leonard-Barton

If Only We Knew What We Know - Carla O'dell and C. Jackson Grayson

Started, but haven't finished yet:

The Discipline of Market Leaders - Michael Treacy and Fred Wiersema

Marketing Warfare - Jack Trout and Al Ries

Naked Statistics - Charles Wheelan

Wiki Management - Rod Collins

Antifragile - Nassim Nicholas Taleb

Capitalism: The Unknown Ideal - Ayn Rand

Fiction:

NOS4A2 - Joe Hill

The first four books in the Sandman series by Neil Gaiman

Innocence - Dean Koontz

Deeply Odd - Dean Koontz

Doctor Sleep - Stephen King

The Black List - Brad Thor

Most of The New Lovecraft Circle - an anthology of Lovecraft mythos stories by contemporary writers

And started re-reading Asimov's Foundation last night. I've read the original trilogy before, but this time I intend to read all seven books. But I'm starting with Foundation and going to the end, before going back to the prequels.

Nate Silver has a similar point[1], and claims that he has been successful by choosing fields that were not already saturated with excellent players. Baseball prediction was waiting to be cracked wide open with modern statistical techniques. It would now be much, much harder to make meaningful gains, you'd be competing with a host of competent firms who have years of experience and have long since claimed all the low-hanging fruit. The competition in online and even professional poker matches used to be much looser, until enough people started getting serious about it that the field narrowed to exclude merely above-average players.

To the OPs point, If you want to compete in a well-established academic field, you have to be really freaking good. Want to be an experimental physicist? You see the competition out there: they write books and appear on TV shows, or at least have tenure and get grant money. Are you really one of those guys or gals? On the other hand, there's interesting stuff out there that is just waiting for someone with a modicum of intellect and desire to revolutionize it. The challenge lies in finding (or creating) such a field and exploiting it at the right time. I think this can be done in academia or in business alike, business is just much more flexible and forgiving of mistakes.

[1] "The Signal and the Noise"

> Likewise, music has been around forever, but the chances of someone like Lorde, a 16 year old from New Zealand, seeing such success was much less likely before the Internet democratized music for listeners and creators alike.

You know, I see this argument a lot and I'm not buying it. To paraphrase Nate Silver in "The Signal and the Noise": just because the internet provides an exponential amount of information to the public doesn't mean everyone is smarter, because with every increase in total information there is an increase in bad information. The solution is basically better curation. Curation, in today's world, is still run by individuals and people in positions of power. Music, just like information, has seen an explosion in data points since the advent of the internet. All of this becomes democratized until it doesn't. In other words - these markets go from non-democratized (the CD world), to democratized (YouTube), and the back again (Spotify/Rdio). Cheaper access to art creation only means the market is more saturation. The signal to noise ratio (i.e. chance) stays the same, it just means a lot more noise exists.

Just finished Nate Silver's "The Signal and The Noise". Interesting collection of prediction case studies, seeing what can and can't be effectively predicted. Good for correcting black-and-white thinking.

Currently reading "Naples 44", diary of an intelligence officer there just after the US landings. The author is classically trained enough to immediately fall in love with the place despite the war, so it's a strange mix of lounging around Paestum and the stark effects of war.

Next in queue: pick one to re-read from Iain M Banks or Pratchett.

(Presumably Clausewitz is in your reading list somewhere as well; his book is unfinished but contains both good quotes to mine and real insight into how few military problems are about actual fighting itself.)

This is exactly the premise of Nate Silver's The Signal and the Noise. Basically the internet has created an explosion of information, but it's become increasingly harder to actually find the right information. So the general premise that "we should be smarter because we have the world's info at our fingertips" is counterbalanced by "there's equally, if not more, disinformation we need to wade through to get to that info"

I seem to recall that something along these lines for political punditry was discussed in "The Signal and the Noise" by that chap Nate Silver.

I think (but could well be mis-remembering) one of his conclusions was that since these pundits are for entertainment value rather than serious prediction, nobody in charge of putting them in front of the camera cares much.

I just read Silver's book "The Signal and the Noise" and I think this makes a lot of sense. Silver never was a political junkie, as all the NYT political journalists are. He was drawn into political statistical analysis almost by accident: he saw how "noisy" the political analysis were and knew that he could do something about it. In the book he clearly shows that baseball and poker are a lot more interesting subjects for him.