Archimedes' Revenge: The Joys and Perils of Mathematics Book

I saw the following excerpt on Hacker News

The ILLIAC IV was a gigantic machine built in the 1970s at the University of Illinois. It had sixty-four processors, each as big as an upright piano because it was built before its time. In fact, they had to use a forklift to plug in the units. It took seven or eight years to build, and the machine was obsolete by the time it was finished. The university gave it to NASA, which promised to use it but had difficulty doing so.

That the project was too ambitious, given the available technology, was not what Minsky meant by the ILLIAC IV mistake. He told Hillis that the concept itself was flawed; it was a mistake, he said, to restrict all the sixty-four processors to doing exactly the same thing at the same time, an expanded electronic version of the Olympic event synchronized swimming. Minsky told Hillis that the processors should be able to operate independently.

"About a month later," Minsky recalls, "Danny came back to me and said, 'Well, I've decided to make the ILLIAC IV mistake.'" Hillis told Minsky that the problem was not in what the processors did but in how they communicated. He said that intraprocessor signals got stuck in traffic jams because, among other things, the connecting wires were restricted to two dimensions. Hillis recognized that a richer connection scheme was required, all the more so since he wanted to hook up not 64 processors but 65,536. The signal traffic would be equivalent to a telephone network serving 65,536 customers who placed a quarter of a billion calls per second. That was the chief technical problem Hillis faced in designing the Connection Machine.

In the end, Hillis and his co-workers at Thinking Machines settled on a three-dimensional architecture in which the processors were connected as if they formed a sixteen-dimensional cube. What this means is that each processor, although directly connected to only 16 others, was never more than sixteen steps away from any of the other 65,536 processors. Moreover, the possibility of a traffic jam was reduced because in sixteen dimensions there are numerous possible routes between any two processors. [1]

[1] From the book Archimedes' Revenge: The Joys and Perils of Mathematics by Paul Hoffman (New York: Ballantine Books, 1988), pp. 203--4.