Learning by Heart, Then Learning the Rule

Clock Arithmetic

At eight in the evening, seven hours from now is three in the morning: on a 24-hour clock, 20 + 7 = 3. Mathematicians call it addition modulo 24, and it is about as simple as a rule can be. Every question of this kind fits on a 24-by-24 board, one hour down the side, one along the top, and the answer in the square where they meet. Colour each answer by its hour and the board fills with diagonal stripes.

In 2022, researchers at OpenAI trained small neural networks on tables like this, showing them only some of the squares, to see whether they would find the rule or just remember the answers. Here a network far smaller than theirs does the same, in your browser, as you read.

All 576 questions of clock arithmetic · the dials behind show where the network keeps each hour

Learning by Heart

The network starts as a heap of random numbers: eight for each hour, about 2,600 more connecting them, and 64 “neurons” in the middle. It is shown half the board, 288 squares; the rest, marked with a pale ring, are held back to test it. Wrong answers are shown dimmed.

Within about a hundred training steps it gets every practice square right. On the held-back squares it does no better than chance. It has learnt the answers by heart, like a student who memorises last year’s exam paper without understanding the subject.

Walking Downhill

How does it learn at all? After each round, a single number scores how wrong it was. For each of its 2,776 numbers, training works out which way a small nudge would lower that score, and nudges them all a little that way. Picture someone walking down a hillside in fog, feeling for the slope underfoot: each training step is one step downhill. The chart keeps the record, amber for the practice questions and blue for the held-back ones.

There is one more rule, called weight decay. At every step each number is also shrunk a little towards zero, so that large numbers cost something. The dashed line on the chart shows how large the network’s numbers are in all. While it memorises, they grow.

The Jump

Training goes on. The practice questions are already right, so there is almost nothing left to learn from them, and for about a thousand steps the held-back squares stay wrong. Then, within a few hundred steps, nearly all of them come right, and the board fills with clean stripes.

The OpenAI team called this grokking. In one of their experiments, a network took a thousand times as many training steps to start answering unseen questions as it took to get its practice questions right. Ours is much quicker: it had the practice questions right after about 100 steps, and the unseen ones after about 1,400.

Clocks Inside

What changed? The network stores each hour as a list of eight numbers, and the dials behind the board show them flattened onto the two directions along which they vary most. At first the hours lie scattered. By the time of the jump they have settled into circles, evenly spaced, like the marks on a clock face. On each dial, every added hour turns the hand by the same step; the two dials turn by different steps.

To add two hours, the network in effect adds their angles, the way two turns of a clock hand add up, and reads the answer off where the hand comes to rest. In 2023, Neel Nanda and colleagues found the same trick in a small transformer trained on addition modulo 113, and Andrey Gromov worked out by hand the waves a network like this one needs. Nobody told the network about clocks. Training found them.

Why It Waited

Memorising takes many large numbers, a patch of them for every practice answer. The clocks do the job with fewer and smaller ones, and work for every question. Weight decay keeps charging for size, so even after the practice answers are all right, training keeps sliding towards the cheaper solution. The clocks were forming long before the jump; the jump is when they take over. After that the network sheds the numbers it used for memorising, and the dashed line falls.

Train the same network with no weight decay and in this run the held-back squares never come right (the red dotted line). Nanda’s team saw the same in their transformers, though other networks have grokked without it. In 2024 one team made grokking more than 50 times faster by amplifying the slow, steady part of each training step.