Overfitting

Overfitting is the case where a model fits details of its training data that new inputs do not share, and its predictions for new inputs are worse than for the training data. We detect overfitting by calculating the loss on examples that the training did not use.

4.1Fitting every example

A model with as many parameters as training examples can often fit every example exactly. For example, the following model adds a third parameter $c$ to the taxi model of Chapter 1: $$y = a \cdot x + b + c \cdot x^2$$ With $a = -2$, $b = 6$ and $c = 1$, it predicts $-2 + 6 + 1 = 5$, $-4 + 6 + 4 = 6$ and $-6 + 6 + 9 = 9$ for the rides of 1, 2 and 3 km, which are their exact prices. In this case, the loss of the curve $y = x^2 - 2x + 6$ is 0, and the loss of the best line $y = 2x + 8/3$ is 2/9 (see Section 1.1).

For any three rides of different distances, some values of $a$, $b$ and $c$ give the three prices exactly. Thus, the loss 0 says nothing about the rule behind the prices.

The line $y = 2x + 3$ through rides 1 and 3 predicts 7 for ride 2, but ride 2 cost 6. The best line spreads this difference over all rides (see Section 1.1), and the curve passes through ride 2. In this sense, the curve has learned the rides by heart: it treats the low price of one ride as a rule.

It turns out that large neural networks can do the same with many more examples. For example, networks that sort photos into 10 classes were trained on 50,000 photos, where each photo had a random class. The networks classified at least 99.3% of the training photos correctly, and on new photos they were right only as often as a random guess¹.

4.2Held-out data

Held-out data is examples with known outputs that the training does not use. Since the training cannot fit their details, we can use the loss on held-out data to measure the predictions for new inputs. For example, suppose that we keep two more rides out of the training: 1.5 km for 6 dollars and 2.5 km for 8 dollars. The following table shows the losses of the line and the curve:

The curve has the smaller training loss but about 5 times the held-out loss of the line. For example, it predicts $1.5^2 - 2 \cdot 1.5 + 6 = 5.25$ for the ride of 1.5 km, which is 0.75 too low. The following picture shows the rides, where the white points are the held-out rides:

Near 2 km, the curve runs below the line, because it dips to reach ride 2.

Overfitting shows as a training loss that is much smaller than the held-out loss. Note that the line has a smaller held-out loss than training loss. With only two held-out rides, this is chance: both of them lie on the line $y = 2x + 3$.

The held-out examples are usually chosen at random from all examples, so that they are like the inputs that the model will get. For example, if we held out only the longest rides, the held-out loss would measure long rides, which the training data does not contain.

4.3Early stopping

A model can also overfit during one training run. As an example, consider training the model $y = a \cdot x + b + c \cdot x^2$ on the three rides with gradient descent (see Chapter 2), with the learning rate 0.01 and the start $a = b = c = 0$. The following table shows the parameters and the two losses:

Both losses fall until step 559, where the model is near the line. After this, the training loss keeps falling and the held-out loss grows, because $c$ grows from 0.09 to 1 and the model turns into the curve. The following picture shows the two losses, where each mark on the horizontal axis is 10 times the step of the previous one, and the dot is step 559:

Early stopping is a technique that calculates the held-out loss during the training, keeps the parameters with the smallest one, and stops the training when this loss has not become smaller for a set number of steps². In the example, it keeps the parameters of step 559, whose held-out loss is 0.16.

Note that the line is still better. Its held-out loss is 0.11. Early stopping can only choose a point of the run, and this run never reaches the line: after step 0, $c$ is never below 0.078.

4.4Validation and test data

Early stopping uses the held-out loss to choose a step, and we can also use it to choose between the line and the curve or to choose the learning rate. Each choice favors parameters that happen to fit the held-out examples. Thus, the held-out loss of the final model is too small, like the training loss, but usually by less³.

For this reason, the held-out data is split into two parts. Validation data is held-out data that we use to make choices about the training. Test data is held-out data that we use only once, after all choices, to measure the final model. For example, the held-out rides are validation data, because early stopping chose step 559 by them.

As an example, consider 100 models that each answer a yes-or-no question correctly with probability 1/2, independently of each other, and suppose that we choose the model with the most correct answers on validation data of 10 questions. One model has at least 8 correct answers with probability 56/1024 ≈ 0.055, because 56 of the 1,024 ways to answer, all with the same probability, have 8 or more correct answers. The best of the 100 models has at least 8 with probability $1 - (1 - 56/1024)^{100} \approx 0.996$.

The chosen model almost always answers at least 80% of the validation questions correctly. On test data, it answers each question correctly with probability 1/2, like every other model.

In practice, the test data is set aside before any training, and about 20% of the other examples are often kept as validation data. The training then learns from only the remaining 80%. The results on a public test, i.e., published test data that many teams have used for many years, are too optimistic for the same reason, because the teams chose their models by these results.

4.5Contamination

Contamination is the case where examples of the test data are also in the training data. The test loss then measures how well the model remembers these examples, not its predictions for new inputs.

For example, a customer can send the same support ticket twice. If one copy is in the training data and the other in the test data, the model can answer the test copy from memory. For this reason, duplicates are removed before the data is split.

Exact copies can be found by comparing the texts. Near copies, such as the same ticket with another greeting, are harder to find. In some standard datasets of text, over 4% of the validation examples had a copy or a near copy in the training data⁴.

Language models are pretrained on text from the web (see Chapter 9), and the web contains the questions and answers of public tests.

We can write new test examples of the same kind and difficulty, which no training data contains. They show whether a model fits a public test better than new problems.

For example, researchers had people write 1,205 new grade-school math problems that mirror a public test. One model solved 44% of the public problems but only 36% of the new ones. Contamination explains part of such drops, but not all of them⁵.

4.6Language models

An epoch is one pass of the training through all its examples. Pretraining usually lasts at most a few epochs. For example, the pretraining of a model of 2020 read 44% of its largest set of web text once and Wikipedia 3.4 times on average⁶.

When the training reads a text for the first time, the loss on it is calculated before any step has used the text. If the data has no copy of the text, the training loss on it is also a held-out loss.

However, a model can still learn single texts by heart. Memorization is the case where a model writes a text of its training data word for word. For example, researchers got hundreds of texts of the training data out of a model of 2019, among them names, phone numbers and email addresses. Some of these texts appeared in only one document⁷.

We can measure memorization by giving a model the start of a training text and checking whether it writes the rest (see Chapter 7). In this way, a model memorizes more when the text repeats more, when the model is larger and when the given start is longer⁸. Removing duplicates from the training data made models write memorized text 10 times less often⁹.