The Mark I Perceptron

Learning from examples

In 1957, at the Cornell Aeronautical Laboratory in Buffalo, New York, the psychologist Frank Rosenblatt proposed a “perceptron”: not a program, but a machine shown picture after picture, told right or wrong, adjusting itself until it sorted them correctly.

In 1958 he set out its structure in Psychological Review. Sensory units (S) take in the picture; association units (A) are wired to the sensory units at random; response units (R) add up the association units’ signals with weights and give the answer. Learning changes only that last layer of weights.

The Office of Naval Research paid for it, as Project PARA. The machine that was built was the Mark I.

Sensory units S → association units A (random, fixed) → response units R (adjustable weights)

“Embryo of computer”

In July 1958 the Office of Naval Research unveiled the perceptron. The Mark I was not yet built: the demonstration was a program on an IBM 704 computer. Fed punched cards, after 50 trials it learned to tell cards marked on the left from cards marked on the right.

The New York Times headline the next day read: “New Navy Device Learns By Doing; Psychologist Shows Embryo of Computer Designed to Read and Grow Wiser.” The Navy, the report said, expected it to walk, talk, see, write, reproduce itself and be conscious of its existence.

What the machine later did was far more specific, and far more limited.

July 1958: a simulation on an IBM 704, left from right marks after 50 trials

Four hundred photocells

The machine’s eye was a camera. A white letter on a black board, lit by two lamps; the lens threw it onto a 20 × 20 board of cadmium-sulphide photocells at the back of the camera, each cell’s resistance falling as its light rose. The image is upside down and reversed, which makes no difference to the machine.

At the top left of the cabinets the S-unit monitor shows, lamp for cell, what the photocells see; below it a door carries 400 manual switches, to set a picture by hand without the camera.

20 × 20 = 400 cadmium-sulphide cells · monitor of 400 lamps · 400 manual stimulus switches

Wired at random

Behind two doors is a large plugboard: the photocells’ jacks on one side, the association units’ on the other, and between them a hanging tangle of cords. Each association unit is connected to a number of photocells. In Rosenblatt’s design some connections push a unit towards firing (excitatory) and some hold it back (inhibitory); when the excitatory signal exceeds the inhibitory by its threshold, it fires.

Which cells went to which unit was set from a table of random numbers, so as not to build any intended bias into the machine. Once plugged, this layer stays as it is: learning never changes a single cord.

Photocells → association units: wired from a random number table · never changed by learning

Potentiometers that turn

The Mark I had 512 association units and 8 response units. The weight from an association unit to a response unit was a potentiometer, turned by a small electric motor.

A response unit adds up the weights of every association unit that is firing; if the sum is over its threshold its lamp lights, “yes”, and otherwise not. The eight response units can each stand for a letter, or together spell a binary code. The meter panel on the right shows the sums.

512 association units × 8 response units · each weight a motor-driven potentiometer

Turned only when wrong

In training, the operator shows the machine a picture and tells it the right answer. For every response unit that answered wrongly, the potentiometers from the association units firing at that moment each turn one step: up if it should have lit, down if it lit when it should not. Right answers move nothing, and units that are not firing move nothing.

This is the error-correction rule. In 1962 Novikoff proved that if any set of weights sorts every picture correctly, training by this rule makes only a finite number of mistakes: the machine is bound to get there.

Wrong: the firing units’ pots turn one step · right: nothing moves

Taken apart

Left to right: the board and its lamps, the camera on its pedestal, the S-unit monitor and switches, the plugboard, the columns of association units, the response panel, the meter panel. The two cabinets together are about 3.4 m wide, 2 m high and 64 cm deep.

The Smithsonian records it in that order: the cabinet marked MARK I at the top holds the plugboard, the one marked PERCEPTRON the potentiometers.

The camera, cut open

Cut open, the camera shows the lens, the bellows and the board of cells. The white letter lands on the board as an upside-down patch of light; the cells under it drop their resistance, and the signals run down the cable into the cabinets. Everything the machine does afterwards starts from these 400 numbers.

The lens makes an inverted image · 400 numbers are all the machine knows

What it learned

John Hay ran the experiments. He taught the Mark I to tell the letters E and X apart; even with the operator giving the wrong answer three times in ten on purpose, the machine ended up entirely right. It learned eight letters, large block capitals placed anywhere in its field of view.

All 26 letters in a fixed position it learned too, answering in a five-unit binary code; Rosenblatt wrote that this was about the limit of the Mark I’s capacity. The 26 letters in two typefaces at once came to about 85% at best.

E vs X: all correct · eight letters anywhere: learned · 26 letters, fixed position: learned · two typefaces: about 85%

What it could not learn

What one response unit does is draw a line through the association units’ signals. Whatever a line can separate, it is bound to learn; whatever it cannot, no turning of the pots will help. The simplest case is “exclusive or”: yes when exactly one of two spots is lit. A single unit wired straight to the two spots never learns it.

In their 1969 book Perceptrons, Minsky and Papert proved that for a question like “is the number of lit cells odd or even?” some association unit must see every point of the retina at once, and that telling whether a figure is all in one piece needs units that see more points as the retina grows. A Mark I whose units each see a limited number of points cannot learn them. Trained on parity, the machine still gets some of its lessons wrong after many passes, and new pictures come out right half the time, as good as guessing.

Separable by a line: bound to be learned · parity, connectedness: not with units that each see a few points

Today’s networks

Rosenblatt knew the trouble was the middle layer: random, and unable to learn. In 1986 Rumelhart, Hinton and Williams published back-propagation, passing the error back layer by layer so that the weights of every layer adjust. Today’s neural networks are trained that way. A unit is still a weighted sum through a threshold of sorts; there are tens or hundreds of layers, and the weights number hundreds of billions instead of thousands, held in memory rather than turned by motors.

The Mark I went to the Smithsonian in 1967, transferred by the Office of Naval Research; it is in the National Museum of American History, not currently on view. Rosenblatt drowned sailing on Chesapeake Bay in 1971, on his 43rd birthday.

1957 proposed · 1958 IBM 704 demonstration · 1960 Mark I manual · 1969 Perceptrons · 1986 back-propagation