Before any talk of neural networks or transformers, there is one idea that has to be settled first, because the whole theory of machine learning rests on it: the prediction problem. Every machine-learning task can in fact be reduced to a single mathematical problem:
Given a set of N input variables, predict the value of M output variables.
That’s it. What changes from one application to the next is only what the inputs and outputs are — and that change of costume is what gives each task its own name.
A few examples from the world I work in:
- Given age, gender, body mass index, predict life expectancy and the total health-care cost up to death. The outputs are numbers on a continuous scale — this is regression.
- Given age, level of education, annual income, predict whether someone will be interested in a life insurance policy in the next twelve months. The output is a category — this is classification (or, when the groups aren’t labeled in advance, clustering).
- Given the words of a question, predict the best answer in natural language. The output is itself a sequence of words — this is generation, and it is exactly what a large language model does.
- Given an image, predict the best description of it — captioning.
- …
Regression, classification, generation, captioning, …: many tasks, one problem. Once you see that, the rest of this series is really the story of how machines learn to solve that basic problem effectively — how they turn “predict the output from the input” into something a computer can actually learn to do, rather than something a human has to program.
The next chapter takes up the life expectancy example again and asks the obvious question: how is machine learning any different from simply writing a program that solves the same prediction problem?