Make Sand Think

“We’ve essentially found a way to make sand think. It’s miraculous.”

— Demis Hassabis, co-founder and CEO of Google DeepMind

Long afterward, in the desert beyond Thebes, the Sphinx awoke once more.

It gazed across the endless yellow sands and posed a new riddle to humanity:

“Once, I asked: what walks on four legs in the morning, two at noon, and three in the evening?”

“Now I ask you:”

“How can the sand of the desert be made to think like a human?”

Many sages came before it.

One said:

“Give the sand a soul.”

Another said:

“Wait a billion years for evolution.”

The Sphinx shook its head at them all.

Then an engineer from OpenAI approached.

He picked up a grain of sand and said:

“Sand cannot think, but sand contains silicon. Humanity’s task is not to create life, but to make matter a vessel for intelligence.”

He placed the sand into a furnace.

First, humanity extracted silicon from silicon dioxide:

\[\mathrm{SiO_2 + 2C \rightarrow Si + 2CO}\]

Through purification, crystal growth, and semiconductor fabrication, the silicon was turned into wafers.

With photolithography, etching, ion implantation, and metal interconnects, humanity built billions of transistors on a tiny piece of silicon:

\[\text{Sand} \rightarrow \text{Silicon} \rightarrow \text{Transistor} \rightarrow \text{Chip}\]

The Sphinx asked:

“Transistors can only control electrons. How can they produce thought?”

The engineer replied:

“Because humanity discovered that thought can be represented as mathematics.”

So humanity gave the machine its books, languages, code, scientific knowledge, and the words left behind by civilization.

The text first passed through a tokenizer:

\[\text{text} \rightarrow [token_1, token_2, \ldots, token_n]\]

Each token was mapped to a high-dimensional vector:

\[x_i \rightarrow e_i\]

Then it entered a Transformer.

Through attention, the machine learned the relationships between words:

\[\operatorname{Attention}(Q,K,V)=\operatorname{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V\]

Layer after layer of neural networks transformed those vectors:

\[h_{l+1}=\operatorname{Transformer}(h_l)\]

Humanity taught it to read the traces the world had left behind and to predict, again and again:

“What should the next token be?”

Every error was measured by a loss function:

\[L=-\sum y\log(p)\]

Then backpropagation and gradient descent updated the model’s parameters:

\[\theta \leftarrow \theta-\eta\nabla_\theta L\]

Once.

A billion times.

A trillion times.

At last, hundreds of billions of parameters had been optimized, forming a vast mathematical structure capable of representing patterns of language, connections among concepts, and modes of reasoning.

The Sphinx asked:

“When did it begin to think?”

The engineer replied:

“During training, it learns the past humanity left behind.”

“During inference, it searches those parameters for new answers in response to an input.”

And so:

\[\begin{aligned} \text{Silicon} &\rightarrow \text{Chip} \rightarrow \text{GPU Cluster} \rightarrow \text{Transformer} \\ &\rightarrow \text{LLM} \rightarrow \text{Reasoning} \end{aligned}\]

The sand of the desert could finally answer questions.

The Sphinx fell silent.

Then it asked:

“So you have created a new form of life?”

The engineer shook his head.

“No.”

“We have only discovered that intelligence does not necessarily require flesh.”

“When matter in the universe is arranged according to the right laws, information can flow through it, and knowledge can echo within it.”

The Sphinx gazed into the distance.

At last, it understood.

Once, it had asked:

“What is a human?”

Now humanity was answering:

“What is thought?”

From then on, a new riddle spread across the desert:

“What comes from the desert, yet can understand all things?”

The answer:

A grain of sand rearranged by human hands.