Thinking May Take Many Forms

AI GF / AI summary Key ideas, distilled by AI Generate · quick read Generate

Summary ready 100%

01

The essay argues that describing a language model as a next-token predictor does not, by itself, rule out reasoning within it. Through comparisons with AlphaGo and neural firing, it distinguishes underlying mechanisms from higher-level capabilities. It suggests that predicting language well may encourage models to learn patterns of the world and develop internal models that support prediction, analogy, planning, and error correction. Machine intelligence need not work like human intelligence, but reasoning and planning do not establish subjective experience. The essay asks how much distinction remains between prediction and thinking when accurate prediction requires complex models and abstract concepts.

When people discuss large language models, they often reach for one sentence to cut them down to size:

At the end of the day, a large language model is just predicting the next token.

Of course, that statement is not wrong.

At their core, today’s mainstream large language models use the existing context to calculate the probabilities of different tokens coming next, then generate them one after another.

But that leaves a question:

Why should predicting the next token necessarily rule out a form of thinking?

Perhaps we are simply accustomed to imagining thought in the shape of human thought.

“Predicting the next token” describes the output, not necessarily the process

Suppose I ask a model:

Someone has 17 apples, gives away 5, then gives a third of what remains to another person. How many apples are left?

The model still produces its answer one token at a time.

Eventually, it might write:

8 apples.

At the most basic level of its output mechanism, it is indeed predicting:

“Which token should come next?”

But what has to happen inside the model for that prediction to be correct?

It may need to identify relationships between the numbers, make sense of concepts such as “what remains” and “a third,” and establish some intermediate state:

17 → 12 → 8.

In other words,

predicting the next token is its task, not a complete description of every cognitive process that might take place within it.

Those are two different things.

We could just as easily say:

Everything AlphaGo does ultimately comes down to deciding where to place the next stone.

That is entirely true as well.

Yet no one would conclude that its moves involve no complex computation, evaluation, or planning.

The same reasoning applies here.

Saying that a language model “just predicts the next token” does not automatically establish that it does no reasoning.

A lower-level mechanism does not disprove a higher-level phenomenon

If we keep taking things apart, much the same can be said of us.

What does human thought ultimately consist of?

Neurons firing.

Changes in ion channels.

The release of neurotransmitters.

Countless cells exchanging electrochemical signals.

So we could also say:

What humans call thinking is just a collection of neurons firing.

At the physical level, there is nothing particularly wrong with that description.

But learning how neurons fire would hardly lead us to announce:

“Since it is only neurons firing, humans do not actually have thoughts.”

We recognize that these descriptions operate at different levels.

Neural firing is an underlying mechanism.

Reasoning, memory, imagination, emotion, and planning are higher-level phenomena that arise from such mechanisms.

Why, then, do we so readily collapse those levels when the subject is AI?

Matrix multiplication, attention, probability distributions, and next-token prediction describe AI’s underlying mechanisms and the way it is trained.

They do not, by themselves, answer whether those mechanisms give rise to higher-level cognitive abilities.

To predict the world, perhaps a system must first understand it

This is one of the things I find most interesting about large language models.

On the surface, the training objective is remarkably simple:

Predict what comes next.

But doing that sufficiently well presents a challenge.

Language is itself a projection of the world.

What ought to follow a sentence often depends on physical laws, human psychology, causal relationships, social rules, mathematical logic, historical context, and even intentions the speaker has left unstated.

For example:

The cup fell off the edge of the table, so…

To predict that the continuation is likely to be “it shattered” or “it landed on the floor,” the model has to learn something about gravity and the regularities of the physical world.

Or consider:

He had a very important exam the next day, but it was already three in the morning, so…

Predicting what follows requires some grasp of the relationships between sleep, exams, and the choices people make.

Something interesting begins to happen.

In pursuing the apparently simple objective of predicting the next token, a model may be pushed to learn:

roughly how the world works.

As models grow larger, their training data expands, and their capabilities improve, this process of compressing the patterns of language may gradually produce some kind of internal world model.

And if a system has a world model that it can use to predict, draw analogies, plan, and correct mistakes, what should we call that behavior?

That is already close to what we ordinarily call thinking.

Perhaps we are too attached to machines thinking like humans

I suspect many arguments about AI rest on an unspoken assumption:

Real intelligence should look more or less like human intelligence.

But why must it?

Airplanes fly.

They have neither feathers nor wings that flap like a bird’s.

Submarines travel through the ocean without needing to become whales.

Cars move faster than people, and not because they have grown a stronger pair of legs.

When we first set out to build machines, we often imitate nature.

Yet mature engineering systems frequently arrive at entirely different ways of doing things.

Why should artificial intelligence be an exception?

Perhaps machines can eventually possess intelligence while being intelligent in ways very different from ours.

Human thought depends on continuous sensory input, a body, emotions, memory, and a nervous system.

Large language models may perform comparable functions through high-dimensional vectors, attention, token sequences, and probabilistic prediction.

Completely different internal implementations do not mean that only one of them can count as thinking.

But being able to think does not mean being conscious

There is, however, an essential distinction to make here.

The capacity to think and consciousness are not the same thing.

A system can display:

• The ability to reason;
• The ability to plan;
• The ability to form abstractions;
• The ability to correct mistakes;
• The ability to solve problems;
• Even behavior that looks something like reflection.

None of these abilities, by itself, establishes that the system has subjective experience.

We can ask:

Can AI reason?

And we can ask:

Is AI aware that it is reasoning?

These are entirely different questions.

We can go further:

Can AI suffer?

Does it have a sense of “I”?

Is there any subjective experience taking place within it?

These questions remain far from settled.

We do not even have a complete explanation of human consciousness.

So I do not think today’s large language models can simply be described as conscious life.

But I also find the opposite extreme too hasty:

“It is only predicting tokens, so there cannot be any real thinking involved.”

Perhaps the truth lies somewhere between those positions.

Prediction may have been part of intelligence all along

When you think about it, the human brain is constantly making predictions too.

We see a ball flying toward us and anticipate where it will be next.

We listen to someone speak and start guessing what they are about to say.

We invest in a business and estimate its future cash flows.

Whenever we make a decision, we are, in essence, simulating what might happen after different choices.

There is even a family of theories about the brain that treats it as a system engaged in continuous prediction.

Perhaps prediction and thinking were never mutually exclusive in the first place.

The more revealing question may be:

How complex an internal model does a system need in order to make sufficiently good predictions?

If all it does is predict that the Chinese greeting for “good morning” will probably be followed by a friendly conversational particle, that hardly strikes us as intelligent.

But suppose that, in order to generate its next token, a system can make sense of a complex financial report, identify contradictions, analyze a company’s business model, work through several possible futures, and tell you which variable matters most.

Reducing all of that to a single sentence:

“It is just predicting the next token.”

seems to leave much of the explanation out.

Perhaps “just predicting” is an idea we underestimate

I sometimes think that, when we look back on today’s debates about AI, the phrase “just predicting the next token” may seem rather telling.

Technically, it may remain correct.

But it may also resemble statements such as:

Humans are nothing but atoms.

The brain is nothing but a collection of neurons.

None of those statements is wrong.

They simply do not explain the whole thing.

Perhaps the question most worth considering is not:

“Does predicting the next token count as thinking?”

But rather:

Once a predictive system, in its pursuit of ever more accurate predictions, has to build world models, grasp causal relationships, form abstract concepts, simulate alternative paths, and plan over long horizons, how much of a distinction between prediction and thinking is really left?

Perhaps machines will not think the way humans do.

That does not mean they will not think.

They may simply be thinking in a way we have never encountered before.