From Natural Language to Executable Action — The Structural Problem in Agent Development

There is one basic and unavoidable problem in agent development, and it is also one of the hardest ones: how to translate natural language accurately into actions that a machine can execute. On the surface, this looks like an engineering problem. In reality, it involves language understanding, context management, operational constraints, system architecture, and even the abstract modeling of future human-agent interaction.

At the broadest level, natural language is not machine instruction. When people say something, it is often vague, elliptical, ambiguous, and sometimes mixed across languages. A single request may contain Chinese semantics, English keywords, contextual hints, default assumptions, and an unstated goal. It cannot simply be mapped directly to a computer operation. There must be an intermediate process of normalization, interpretation, constraint, and restructuring. In essence, this is an initial planning step, and today that step almost inevitably relies on large language models. Of course, it can also be supported by some direct recognition mechanisms, such as fast matching for fixed keywords, fixed patterns, or explicit commands. But when facing the real complexity of natural language, rules alone are not enough.

At present, many agent systems handle this problem by first converting natural language into a set of relatively coarse-grained callable capabilities, and then letting those capabilities carry out the actual work. This is one of the main lessons from systems like OpenClaw. It does not begin by imposing a fully formal specification at the strictest and lowest level of computer operations. Instead, it relies heavily on a higher-level packaging of capabilities, namely skills. Natural language is first interpreted as a call to a skill, and then the skill is responsible for grounding that into concrete operations. This approach has clear advantages. It is flexible, fast to extend, efficient to develop, and easier to adapt to open environments. But its limitations are just as clear, because accuracy at this stage depends heavily on how well the large model interprets the prompt and the skill. If the model understands well, the result is better. If the model interprets poorly, or if the skill itself is not defined rigorously enough, execution easily drifts.

This is also the point I have become more certain about. If an agent is to become truly practical, relying on the skill layer alone is not enough. It needs stronger structured context. At a higher level, the user’s request should be understood within a more targeted framework. In other words, the user’s sentence should not be treated as an isolated utterance. It should be treated as an action request within a certain project, a certain work item, a certain role division, and a certain behavioral mode. Only then does the model receive context that is not just a loose pile of information, but information that has already been bounded, organized, and partitioned. In that case its understanding becomes more stable, the plan it generates becomes more targeted, and the execution result becomes easier to control.

This is the fundamental reason why I introduced the two modes of personal and organization in SmallClaw. On the surface, they look like two usage modes. At a deeper level, both exist to make the model’s context more relevant and more bounded. Among them, the project layer is especially important. Many existing agent systems organize context more around sessions and tasks, that is, around the current conversation and the current task. That is reasonable in its own way, because it is more direct to implement and fits interaction patterns driven by single-turn or short-term tasks. But if the system is meant to support more complex, longer-running, and more realistic workflows, session and task alone are not enough. Different sessions need separation. Different work items also need separation. Different projects need even stronger separation. At the same time, different work items within the same project cannot be fully disconnected, because in reality they are related. What this produces is not a single layer of context, but a multi-layer structure of isolation and connection.

I am increasingly satisfied with this direction, not because it looks more complex, but because it is closer to reality. Real collaboration, whether in companies, schools, communities, or families, ultimately depends on the interaction and progression among projects, tasks, roles, and patterns of behavior. What I call an organization is not necessarily a traditional corporate org chart. It can also be a runtime structure in the AI era, a kind of abstract container that allows different goals, different responsibilities, and different workflows to coordinate with one another. If that judgment is correct, then project, work item, role, and behavior are not artificially added complexity. They are an abstract distillation of real collaborative structure.

At the current stage of development, these structures still need to be shown explicitly. We use development scripts, debugging interfaces, or other temporary tools to make projects, tasks, and roles visibly present, because we need to verify whether they are actually working, whether they are really improving the model’s understanding, and whether they are making execution more stable. But from the perspective of the final product, users do not actually need to see these internal mechanisms. What users really care about is simply what they want to do. They may have no background in project management, and may not even have the concept of a project at all. They may not understand structural terms like role or work item. But that does not prevent the system from quietly creating these structures in the background for their requests. The user simply gives a natural request, and the system automatically assigns context, establishes project affiliation, attaches relevant work items, binds the appropriate roles, and then completes the subsequent planning and execution. In other words, these structures are not there to educate the user. They are there to make the system itself smarter, more stable, and more orderly.

The key point here is not how beautifully the structure is displayed, but how thoroughly it is internalized. A truly mature agent should not make the user feel that they are operating a project management system, nor should it make the user strongly aware of how many roles, how many context layers, or how many intermediate reasoning steps are running in the background. The ideal state is that the user feels they are simply expressing an intention naturally, while the system has already completed the structural orchestration behind the scenes. This is very different from traditional software. Traditional software usually requires users to enter a module, understand a structure, and follow the logic of a form. If an agent is done well enough, it should absorb that complexity into the system itself instead of pushing it onto the user.

This also creates another requirement. The implementation architecture must be both stable and flexible. Large language models are still changing rapidly, and agent systems are far from settled. A boundary that makes sense today may not be the best boundary tomorrow. A skill design that works today may later be replaced by finer-grained operation graphs, richer multimodal input, or stronger planning ability. So the implementation cannot be rigid. It needs to preserve a certain level of abstract stability, for example the conceptual stability of project, work item, role, and behavior, while still allowing the underlying implementation to evolve with changes in model capability, channel capability, and system infrastructure. In the end, the architecture cannot be built only for today’s features. It must remain compatible with future development.

For that reason, this is no longer only a technical problem. It certainly has a technical side, but the deeper part is a highly abstract modeling problem. How should we understand future human-agent interaction? As a direct triggering of atomic operations one after another, or as a continuously advancing collaborative process supported by organizational structure? If it is the latter, then project, task, and role have not only engineering value, but theoretical value as well. They are not there just to complete a feature set. They express a judgment, namely that effective human-agent collaboration in the future will not consist of humans constantly issuing isolated commands, nor of AI acting freely without structure, but of goals being advanced continuously within a certain framework.

In this sense, this design direction is also connected with a larger question I have been thinking about for some time, namely that the subject does not possess an intrinsic exclusivity of its own. Many past systems implicitly assumed that the subject is the single user, a stable, continuous, centralized origin of command. But in agent systems we are beginning to see more clearly that what actually carries forward action is often not an isolated subject, but a structured system. The user, the roles, the projects, the tasks, the system memory, and the model’s reasoning together form a distributed unit of action. Subjectivity is no longer just a single point. It is more like an organized relation in operation. This is both a practical reality in technical implementation and a theoretical issue worth developing further.

So if we return to the original question, which is how to translate natural language accurately into machine-executable action, my conclusion has become relatively clear. It cannot rely only on stronger models. It cannot rely only on more skills. It cannot rely only on more elaborate prompts. The more reliable direction is to build a sufficiently stable and extensible structured layer between natural language and low-level action. This layer is not just a list of commands. It is a context system composed of projects, work items, roles, and patterns of behavior. The large model does not control everything directly. It works within that structure. The result is not only more accurate execution, but also a foundation for a more mature form of human-agent collaboration in the future.

That is where the real significance lies. This is not simply about making a product slightly easier to use. It is an attempt to answer a larger question: once AI truly enters everyday work and life, what kind of structure should we use to connect natural language, intelligent understanding, and real-world action? My judgment is that the answer will not be a pile of atomic operations. It will be a more layered, more bounded, and more realistic organizational structure, one that reflects how collaboration actually works. What SmallClaw is doing now is, in essence, a fairly concrete step in that direction.


Discover more from Geoffrey Chen

Subscribe to get the latest posts sent to your email.