AI and neural intelligence: how much brain is in a model

Almost everyone deciding about AI in a company carries an idea of an artificial brain: something that learns from what you tell it, that remembers, that reasons. None of the three is true in the sense it appears to be, and all three get paid for in wrong decisions.

Where the word comes from

In 1943, McCulloch and Pitts proposed a mathematical model of a neuron: it takes several inputs, sums them with different weights, and fires if the total crosses a threshold. In 1958 Rosenblatt turned it into the perceptron, the first machine that adjusted those weights by itself from examples.

That is where the resemblance to biology ends. The name stuck because it described the inspiration, not the mechanism. A modern neural network has about as much to do with a brain as a plane has with a bird: the principle that was copied is real, everything else was abandoned because it worked better another way.

What a neural network actually does

Without metaphors and without mathematics: a neural network is an enormous function with millions of adjustable numbers, called weights. Something goes in — text converted into numbers — those numbers pass through layers of multiplications and sums, and a prediction comes out.

“Training” means exactly one thing: show it an example, compare its prediction with the correct answer, and nudge every weight slightly in the direction that reduces the error. Millions of times. There is no understanding in the process, there is gradient descent: rolling downhill across a surface of error.

A language model does that with one specific and seemingly silly task: predict the next chunk of text. What is remarkable — and this is honest, not marketing — is that doing that very well and at that scale produces something that looks a lot like reasoning. But underneath it is still prediction.

Where the brain metaphor costs you money

Believing it learns about your company

This is the most expensive one. A model does not accumulate knowledge about your business by being used a lot. If you want it to know your prices, your tone and who is who, that has to be handed to it on every request or written into a system that retrieves it. The feeling that “it knows me now” is scaffolding someone built, and if nobody built it, it does not exist.

Believing that getting it right means it understood

A model gets things wrong with exactly the same confidence it gets them right, and with better prose than yours. In a person, doubt shows in the voice. Here there is no signal. That is why the processes that work are the verifiable ones: not because the model is bad, but because you cannot tell its success from its failure by looking at its face.

Believing responsibility travels with the task

When you delegate to a person, you also delegate a share of the responsibility. When you delegate to a model, the responsibility stays entirely with you — only now it is spread between whoever built it, whoever did not review it, and whoever decided the level of autonomy. That is the conversation almost nobody has beforehand, and everybody has afterwards.

From model to agent: where this gets serious

A model predicts. An agent acts: it reads your email, writes to your CRM, messages a client, moves an invoice. The difference is not intelligence, it is permissions — and it is the only difference that matters to a company.

The moment a system acts, the useful question stops being “which model do you use” and becomes “how much can it do without asking”. And that question had no answer you could put in a contract, so I gave it one.

The Vergara Agentic Autonomy Scale measures exactly that: six levels from V0 to V6, three risk classes according to what the action can break, and seven conformance properties you can demand. It is published with a DOI, it is not a table in a blog post, and it works the same in both directions: demanding it from a vendor and not fooling yourself about your own system.

What to take away

• The word “neural” describes a 1943 inspiration, not the mechanism. Useful for explaining, not for deciding.
• A model does not learn by talking to you. Everything that looks like memory is context somebody resends.
• It gets things wrong in good prose, so the process has to be verifiable. That is not distrust, it is design.
• Responsibility is not delegated along with the task. Decide who answers before switching anything on.
• When it moves from predicting to acting, measure the autonomy. It is the only thing you can contract and check.

If what you want is the practical half — which process to touch first and how to know whether it worked — carry on to AI for business: what works and what is hype.