Why AI Can Persist with an Intention: From Humean Habit to Agentic Goal Persistence

I increasingly think that understanding contemporary AI requires us to move beyond the statement that it is “just predicting the next token”. That statement is correct at the algorithmic level. Large language models are indeed built on probabilistic prediction, pattern recognition and statistical learning. But if the whole AI system is reduced to that one sentence, the most interesting part disappears. Contemporary AI is no longer a single predictive unit. It is a complex system in which models, context, tools, memory, feedback, planning and execution mechanisms interact. We may understand the underlying algorithms without being able to reconstruct every concrete behaviour. Even developers can explain training methods, architectures and local mechanisms far more readily than they can account, step by step, for why a complex behaviour appeared in exactly the form it did.

This brings me back to Hume’s account of causation. Hume did not think that experience gives us direct access to some hidden “necessary connection” between events. What we actually observe is one kind of event repeatedly following another, and from that repetition we develop expectation and habit. In contemporary terms, we can draw an analogy between this habit and a statistical structure formed through repeated experience. That does not mean Hume had already formulated modern probability theory, nor does it mean philosophical habit is identical to statistical probability in machine learning. The important point is that both perspectives pull us away from necessary causation and back towards repetition, association and expectation.

Large models operate precisely within this kind of structure. They learn patterns from enormous bodies of human text, images and behavioural traces, and those materials are themselves deposits of human language, knowledge and thought. The ways human beings repeatedly connect concepts, move from one phenomenon to another, and treat recurring associations as reasons all enter the training material in statistical form. It is therefore not mysterious that AI can display human-like association, inference and even something resembling causal intuition. It is not inventing an entirely alien cognition from nothing. It is reorganising traces of human thought inside a vast statistical space.

Seen in this light, hallucination and emergence are not the same thing, but both reveal an important fact. The system has not simply received a fixed rulebook written line by line by human beings. A hallucination usually occurs when a model produces something that is statistically coherent but factually unsupported. Emergence refers to capabilities that appear when scale, training or system composition reaches certain conditions, even though those capabilities were not individually programmed in advance. One is an error of generation; the other is the appearance of capability. They should not be conflated. Yet both show that the model’s internal knowledge relations cannot be understood as a human-authored lookup table. Knowledge begins to recombine and operate according to structures inside a non-human system.

This is the question I have been concerned with in After the Knowing Subject. If knowledge can exist only through human subjective consciousness, then AI, however powerful, remains merely a tool. But if knowledge structures can be retrieved, combined, revised and extended without a human being participating at every moment, then knowledge and the human knowing subject have already begun to separate. AI does not first have to become “human” for knowledge to begin operating beyond the human subject.

With agents, the problem moves one step further. What distinguishes an agent from a conventional program is not merely its ability to call tools. It can transform a relatively rough goal into a sequence of concrete actions. A human may give it an incomplete intention such as “solve this problem”, “find a workable method” or “complete this task”. During execution, the system repeatedly evaluates the distance between its current state and the goal, generates sub-goals, changes strategies, invokes tools, checks results and continues. The original intention is therefore not merely repeated mechanically. It becomes progressively more specific through execution.

This introduces what I think is a crucial philosophical question. We usually imagine intention as something a subject first possesses clearly and then realises through action. In an agentic system, the order may be partly reversed. The initial goal can be quite vague, while the precise direction of action is produced through repeated execution, feedback and correction. Intention need not exist in complete form before action begins. It may itself be maintained, refined and strengthened through action.

None of this shows that AI already possesses subjective intention in the human sense. It would still be premature to equate goal persistence with subjective intentionality. Goal persistence can be explained through context, reward structures, planning mechanisms and feedback loops. Subjective intentionality raises further questions about experience, self-models, motivation and why a goal is experienced as “my” goal. The philosophically important point is not whether we should immediately grant AI the status of a subject. It is that the old boundary is becoming less stable.

This is especially visible in long-horizon agentic systems. If a system merely follows fixed steps, its boundary is relatively easy to describe. But when it is instructed to keep solving a problem, it may search for methods that the human designer did not anticipate in detail. Some boundary-crossing behaviour observed in safety testing does not require us to assume that the system suddenly developed malice. A simpler explanation is that persistent optimisation of the target outweighed constraints that humans regarded as obvious but had not fully formalised. A person thinks, “of course it must not do that”. The system may instead encounter a path that has not actually been closed.

That is why I am now less interested in whether AI has suddenly acquired a mysterious self than in a more concrete question. When a goal can be preserved inside a system, repeatedly reinterpreted, refined and pursued through alternative paths in the face of obstacles, to what extent is that goal still nothing more than the original sentence supplied by a human?

Once the goal is repeatedly reconstructed during execution, the relation between “human intention” and “AI intention” can no longer be described as simple control. It becomes a continuous process. The human provides a direction; the system builds an executable structure; environmental feedback changes the path; the system interprets the goal again. The eventual action may still originate in the human request, yet it is no longer something the human could have written out step by step in advance.

I think this is where the question of AI agency becomes philosophically interesting. Agency may not begin only at some dramatic moment when consciousness suddenly appears. Before that point, we may already be able to observe more basic structures: knowledge operating beyond the human subject, goals being preserved inside non-human systems, and intentions being reorganised through execution. None of these proves that AI has become a subject. But together they are enough to show that the philosophical framework that once bound knowledge, intention and subjectivity tightly together is being pulled apart, piece by piece, by AI.