For those who are new to Bayesian Statistics and not to eager to dive into the maths right away I recommend Think Bayes [0]. It gives a nice introduction for those who know programming (Python). The ebook is available for free (see link).

[0]: https://greenteapress.com/wp/think-bayes/

I recommend Think Bayes by Allen Downey if you want to study more. It's a free book available online. http://www.greenteapress.com/thinkbayes/thinkbayes.pdf

Think Bayes and Python Data Science Handbook are a good starting point. Below is the list of free books to learn ML/AI

http://blog.paralleldots.com/data-scientist/list-must-read-b...

I've been working with both books and Think Bayes is more accessible (and its also free). I recommend going through it before getting to PyMC.

One huge reason why is that the author has implemented everything you use in Python in a way that enables you to read his code to more fully understand what's happening. I'm on chapter 5 or 6 and he hasn't even touched on MCMC yet which is most welcome.

Also reading The Signal and the Noise: Why So Many Predictions Fail — but Some Don't - Nate Silver

And also in progress:

Buried for Pleasure - Edmund Crispin

The Dying Trade - Peter Corris

Think Bayes - Allen B Downey

Think Bayes is the best probability book I've seen for people who know how to program. Definitely worth a read!

I took a step back and read Think Bayes by Downey and watched some of his youtube videos. Then Introduction to Bayesian Statistics by Bolstad is great once you're reading to deal with probabilities. Now I'm reading

* Building probabilistic graphical models with Python (Karkera)

* Mastering probabilistic graphical models using Python (Ankan)

* Probabilistic Graphical Models Principles and techniques (Koller)

Having skimmed (just started reading them) I'm very excited to continue to delve into them.

A relevant book is Think Bayes [1] by Allen B. Downey. It also uses only pure Python to solve simple problems of Bayesian inference. I like the simplicity of the way it communicates the ideas of Bayesian approach.

But the subject is not a duplicate of that one, as it seems to put more focus on sampling.

[1] http://greenteapress.com/wp/think-bayes/

This is the "online" (abridged) version of this book:

https://www.amazon.com/Bayesian-Methods-Hackers-Probabilisti...

There's a full version of another good book ("Think Bayes: Bayesian Statistics in Python") available as a PDF from the publisher here:

http://greenteapress.com/wp/think-bayes/

(related Amazon reviews)

https://www.amazon.com/Think-Bayes-Bayesian-Statistics-Pytho...

This is a good list of books. Unfortunately many of the links are broken? Probably just my luck, but the first few "with Sage" books I excitedly selected unfortunately 404'd.
I'll send an email.

> Moreover, the American Institute of Mathematics maintains a list of approved open-source textbooks.
https://aimath.org/textbooks/approved-textbooks/

I also like the (free) Green Tea Press books: Think Stats, Think Bayes, Think DSP, Think Complexity, Modeling and Simulation in Python,
Think Python 2e: How To Think Like a Computer Scientist
https://greenteapress.com/wp/

And IDK how many times I've recommended the book for the OCW "Mathematics for Computer Science" course: https://ocw.mit.edu/courses/electrical-engineering-and-compu...

There may be a newer edition than the 2017 version of the book:
https://courses.csail.mit.edu/6.042/spring17/mcs.pdf

I don't know if this qualifies as "Math" per se, but I just read a book on Bayesian statistical analysis that demonstrated in Python code how to accomplish certain things while also explaining the rationale.

It was very good and helped me "grok" Bayesian stuff in a way that I hadn't been able to purely by reading academic material.

I also learned a few things about Python from that book.
It is called Think Bayes.

Disclosure: I am in no way affiliated with this book, its author, or its publisher. I'm just a guy who wanted what it had to offer and found it very useful as an addition to my formal training.

As a sidenote, I would say that, while a lot of statistics is easier to understand with code than it is with the math; the math is very important.

Understanding the math is key to understanding the limitations of any given statistical procedure and to understanding the needed assumptions about a data set before any approach can be decided on.

We have a lot of issues in modern scientific thought that are based in people understanding code and not understanding math where people figure out how to run a statistical procedure on a dataset, but don't bother to understand if that procedure should be run on that particular data.

These kinds of errors happen from Climatology to Neuroscience. It's widespread.

I think they are fundamentally math principles, but I think it's fair if people disagree with me.

My tl;dr respsone is this: always take the hard way. You will benefit in the long run. Find a teacher or a website that will take you down the hard path. It will be good for you. Once you understand the concept, the code will be obvious.

I agree you'll definitely want to read Elements of Statistical Learning but there are a few more, namely Think Stats and Think Bayes.

Since no one has really said much about Bayes yet, I think it worth mentioning just how useful it is in DS and ML. A Bayesian approach makes a very good baseline and often one that is hard to beat.

If you're not particular fluent with Probability and Statistics now, let me suggest you add in Khan Academy (make sure to pick the CLEP version) and JBstatistics. Khan has the advantage of quizzes (so you're not just kidding yourself that you know the material). JBstatistics has the advantage of really good explanations. You'll probably want to watch Khan at x1.5 speed.

I found it helpful to read through Think Stats and Think Bayes before tackling a machine learning book.

[1] Think Stats: http://www.greenteapress.com/thinkstats/thinkstats.pdf

[2] Think Bayes: http://www.greenteapress.com/thinkbayes/thinkbayes.pdf

I'm really interested in knowing the prereqs I should have before picking up a book like this. Coming from a weak math background I find these books highly appealing but mildly intimidating. Also, could someone advise me on the preferred order of tackling the following Books?

1. Think Bayes

2. Think Stats

3. Programming Collective Intelligence by T.Segaran

Read Think Bayes from the author of Think Stats.