Once AI Writes the Code, What Is the Programmer Still Responsible For?

Geoffrey Chen

After AI Enters the Workflow · Season One: “From Answering Questions to Participating in Work” · Article 7

A programmer asks an AI coding agent to add an export function to an existing application. The agent searches the repository, creates several files, updates a dependency and writes tests. The test suite passes. The interface appears to work.

A week later, a customer discovers that exported records can include fields hidden by the application’s normal permission checks. The code is syntactically sound and the new tests are green. The failure lies in a requirement that was never stated, a security boundary the agent did not infer and a test design that merely confirmed the generated implementation.

Who wrote the code? The agent generated most of the changed lines. Who is responsible for the defect? The answer cannot be found by counting keystrokes.

Software is a collection of promises: about what a system will do, what it will refuse to do, which data it will protect and how it will behave when conditions change. AI can produce an implementation. It cannot independently assume the professional and organisational commitments that make that implementation acceptable.

There is no single answer about AI coding productivity

Evidence about AI and programming does not support a universal percentage gain.

Controlled research associated with GitHub Copilot found that developers using the tool completed a defined coding task substantially faster than a control group. Microsoft Research, “The Impact of AI on Developer Productivity—Evidence from GitHub Copilot” Other research on high-skilled work has examined gains in task completion and differences among workers, reinforcing that assistance can be meaningful in suitable settings. Management Science, “The Effects of Generative AI on High-Skilled Work”

But task and setting matter. METR’s randomised study of experienced open-source developers working on their own mature repositories in early 2025 found that the available AI tools made participants slower on average, despite participants expecting a speed-up. The authors were careful to limit the conclusion to the tools, developers and conditions studied. METR, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity”

Tools improved after that study. In 2026, METR described changes to its experimental design after observing indications of productivity uplift under newer conditions, while also explaining methodological difficulties such as task selection and behaviour changes. METR, “We are Changing our Developer Productivity Experiment Design”

These findings are not contradictory once the unit of analysis is made explicit. Autocompleting a familiar function, completing a constrained exercise and modifying a large repository with hidden dependencies are different tasks. Developer experience, model generation, tool integration, repository quality and verification requirements all affect the result.

The right question is not “Does AI make programmers faster?” It is “Which parts of this software lifecycle become faster, which new costs appear, and what standard defines completion?”

Writing code is only one stage of software work

Before implementation, someone must decide what problem is being solved, identify users, establish constraints and resolve conflicts among requirements. During implementation, the change must fit architecture, conventions, dependencies and security boundaries. After implementation, it must be tested, reviewed, deployed, monitored and maintained.

AI can participate in all these stages, but generating code is the most visible because it produces a concrete artefact. That visibility can distort management. Lines of code are easy to count; avoided incidents, coherent architecture and accurate problem definition are harder to measure.

A system may produce more code while delivering less value. Generated abstractions can enlarge the maintenance surface. Duplicated logic can pass local tests but conflict with established design. A dependency added for convenience can introduce licensing, security or long-term support obligations. If the programmer accepts the change, those obligations become part of the organisation’s system whether or not the programmer typed the relevant lines.

Expertise moves towards direction and judgement

Anthropic’s research into how Claude Code is used in practice found patterns in which users delegate implementation while retaining varying degrees of direction, review and collaboration. It also examined how behaviour differs with user expertise. Anthropic, “How Claude Code is used in practice”

This suggests that expertise has not disappeared. It changes how a person constructs the task, supplies context, notices unexpected changes and evaluates trade-offs. A novice may be able to produce a working feature earlier than before, which is valuable. The same novice may also lack the repository knowledge needed to see that the feature duplicates an existing service or bypasses a control.

Experienced programmers are not valuable merely because they remember syntax. They recognise system boundaries, ask what is absent from a requirement, anticipate failure modes and connect local changes to operational consequences. AI can support those abilities, but fluent code can also create the impression that they are no longer necessary.

Six responsibilities remain with programmers and their organisations

First, problem responsibility. The team must establish that the requested change addresses the right problem. A precise implementation of a mistaken requirement is still a failure.

Second, context responsibility. The agent needs relevant architecture, conventions, data classifications and constraints without receiving unnecessary secrets or unrestricted access.

Third, design responsibility. Someone must judge whether the solution fits the system’s structure, whether complexity is justified and whether future maintainers can understand it.

Fourth, verification responsibility. Tests, review and security analysis must be capable of finding errors rather than merely confirming the path chosen by the generator.

Fifth, deployment responsibility. The organisation must control when code reaches users, monitor behaviour, manage rollback and communicate incidents.

Sixth, maintenance responsibility. Dependencies, documentation, technical debt and later corrections remain after the generation session has ended.

These are not necessarily carried by one individual programmer. They may be divided among product owners, architects, security specialists, reviewers and operators. But they must be allocated to real roles. The phrase “the AI wrote it” cannot serve as an unassigned seventh role.

Tests must validate requirements, not merely the generated answer

An agent asked to implement a feature can often generate tests that demonstrate its own interpretation of the feature. Those tests may be internally consistent and still miss the real requirement.

In the export example, tests might prove that the chosen fields appear in a file. They will not detect the privacy failure unless the test designer knows that permissions must be applied to export as well as display. The critical test comes from an independent understanding of the requirement.

This is why test generation and implementation generation should not collapse into a single unchecked process. Useful separation can come from pre-existing acceptance criteria, tests written before implementation, independent reviewers, property-based checks, security tools and production monitoring. Different checks should have different failure modes.

Agent evaluation guidance similarly stresses examining task outcomes and traces rather than trusting a final claim of success. Anthropic, “Demystifying evals for AI agents”

A green test suite is evidence only for what the suite actually tests. AI makes it more important, not less, to ask who defined those properties and what remains outside them.

Code ownership should follow adoption, not input

Teams sometimes treat generated code as if it arrived from an external author. That framing weakens review: the programmer feels less ownership of lines they did not type, while the organisation still deploys them under its name.

A better rule is that the person or team that adopts a change owns it in the operational sense. Adoption does not require memorising every character, but it requires enough understanding and evidence to justify merging, deploying and maintaining the change.

This standard already applies to copied examples, third-party libraries and contributions from colleagues. AI changes the speed and volume at which candidate code appears; it does not remove the adopter’s duty to assess provenance, licence, security, compatibility and maintainability.

That also means organisations must give programmers the time and authority required for ownership. If performance measures reward accepted volume while review is treated as delay, the organisation—not only the individual—has designed irresponsible adoption.

Conclusion: programmers are responsible for system commitments, not keystrokes

AI may generate a function, a test, a refactor or much of a small application. This can remove routine work and let programmers explore solutions faster. It does not convert software into a consequence-free text product.

The responsibility that remains is best expressed as follows:

A programmer’s professional responsibility is not to have typed every line. It is to ensure that the adopted system makes only commitments the team understands, can test, can operate and is prepared to maintain.

This shifts the centre of programming towards problem definition, architecture, independent testing, security, deployment control and maintenance. In some tasks, AI will reduce total effort. In others, reviewing generated changes may cost more than writing them directly. The balance will continue to change with tools and environments.

The stable principle is that generation does not equal acceptance. A system enters production because people and organisations decide that the evidence is sufficient. The author of the keystrokes may be unclear; the owner of that decision must not be.

Primary sources and further reading

Continue reading: Explore the After AI Enters the Workflow series.

Discover more from Geoffrey Chen

Subscribe to get the latest posts sent to your email.