Take the example from the previous chapter: given someone’s age, gender, and body mass index, predict their life expectancy and the total health-care cost up to death. There are two completely different ways to build that predictor.
Traditional programming. A human analyst studies the problem and works out the rules — the algorithm that turns inputs into an output. They write those rules down explicitly (if age < 50 and gender = Female and BMI > 20, then life expectancy = 85 years and cost = 300k), test them against historical data, refine them where they’re wrong, and ship. The intelligence lives in the rules, and a person had to supply it.
Machine learning. Instead of writing the rules, you choose a model: a general set of formulas that depends on a set of free parameters, usually called weights. The model doesn’t know anything yet. You then train it — tune those weights — using known examples (the training set) so that its output reproduces the observed reality as faithfully as possible. You hold back a small portion of the data for a final test, go back and correct the model if needed, and ship. Nobody wrote the rules; they’re encoded in the weights the machine found through this optimization process.
The cleanest way to see the difference is to look at what goes in and what comes out.
Classical programming consumes rules + data and produces answers. Machine learning consumes data + answers and produces the rules.
One consequence is worth emphasizing, because it explains a lot of what follows: the same model, trained on a different training set, ends up with different weights and can describe a completely different problem. The formulas that predict life expectancy and health-care costs from age, gender, and BMI could, with different weights, predict whether a flower is a rose or a geranium from the characteristics of its petals. The structure is generic; the meaning is entirely in the values of the weights.
That is why, over the last couple of decades, for prediction problems machine learning has increasingly won out over programming based on explicit rules (so-called expert systems): computing power stopped being the bottleneck, data became available in quantities previously unimaginable, and letting the data write the rules scales in a way no analyst could match.
Machine learning is only one region of a bigger map. It is one of the ways to implement artificial intelligence, and now the most common one; deep learning is a particular class of machine learning; generative AI builds on deep learning to solve generation problems — and nearly everything we’ll cover is built on one piece of technology, the neural network, which is where this series goes next.