When discussing AI in research and writing, we often hear that AI should only assist and that genuine originality must come from people. I think we first need to establish what we are assessing. Student learning, evaluations of researchers, and practical problem-solving do not all place the same demands on originality. Rejecting substantive AI contributions without distinguishing these settings conflates different questions.
Consider students first. Teachers usually assign work so that students can acquire knowledge through the process and demonstrate that they have learnt it. If a student is studying recursion and AI writes a function that runs correctly, this shows only that the student has obtained usable code. It does not establish an understanding of recursion. The teacher needs to ask why the function terminates, which inputs might cause problems, and how it should change if the conditions change. When students can explain, adapt and address these questions themselves, we have evidence that learning has taken place.
In this setting, “originality” often means independent work and thought. Students are not being asked to discover knowledge previously unknown to humanity. Solving a mathematical problem with an established answer can still be a valuable learning experience. Teaching needs to establish whether students have gone through and mastered the necessary thinking. AI therefore cannot replace all the work that an assessment is intended to examine.
This does not mean students cannot use AI. Finding information, checking facts, brainstorming, organising an argument, editing language and translating can all support learning. The boundaries depend on the learning objective. If an exercise specifically assesses translation skills, having AI produce the translation replaces the very work being assessed. We cannot permanently classify a particular use as “assistance” without considering what the student is learning.
Researchers’ promotion and capability assessments raise similar questions. A valuable result does not automatically establish that every contribution came from the person submitting it. Since the assessment concerns that person, we need to identify the questions they posed, the methods they selected, the key reasoning they contributed and how they checked their conclusions. AI may participate, but a human submission does not turn every novel contribution in the process into that person’s own.
Practical work has different requirements. When a company needs a feature implemented, its immediate concerns include whether the feature meets the requirements, whether the code is reliable and maintainable, and whether it works with the existing system. If AI-generated code has been checked and meets these requirements, there is no reason to make a programmer write it again merely to preserve evidence of manual work. The same code may be insufficient evidence of a student’s learning and entirely acceptable as part of an engineering delivery.
In these circumstances, giving AI more of the generation work need not be a bad thing. If it improves quality or efficiency on a particular task, or helps accomplish something previously difficult, there are grounds to encourage its use. We should assess the actual effect rather than downgrade a result simply because AI generated it.
However, a company encouraging AI coding, or even making it a job requirement, does not directly prove that AI produces better code than people. Companies may also be considering speed, cost and changes in working practices. Research on coding shows why the conditions matter. A study reported in 2025 combined randomised trials at three companies involving 4,867 developers. Developers given access to AI coding tools completed approximately 26% more tasks. This supports a productivity gain in those settings; it does not mean code quality improved by 26%.[1]
Another study reported that year involved 16 experienced open-source developers completing 246 tasks in projects they knew well. When they were permitted to use the AI tools available at the time, they took an average of 19% longer. Its sample and setting were limited, so this does not establish that AI coding has no value.[2] Taken together, the examples show why assessments need to be specific. They concern tools and working environments at particular points in time and cannot simply be treated as conclusions about all AI coding today. Models, tasks and forms of collaboration change, and our judgements need to respond to the evidence.
The reliability of generative AI also requires this approach. A Transformer-based language model producing a coherent explanation does not mean that the explanation has passed a rigorous logical check. It may still invent information, omit conditions or present invalid reasoning fluently. Retrieval, code execution, external tests and repeated checks can reduce errors under some conditions. But we cannot broadly claim that several rounds of agent checking have reduced hallucinations to a minimum. Repeated generation and agreement may simply repeat the same mistake.
Uncertainty in the generation process does not mean that its results must remain uncertain forever. AI-written code can undergo external testing, and an AI-proposed mathematical proof can be checked using formal tools. Producing a candidate answer and establishing that it is valid are different stages. Human intuition can also be wrong, yet we do not dismiss an idea that has subsequently passed rigorous verification merely because it began as an intuition.
Whatever the extent of AI involvement, I think one basic requirement should remain: anyone submitting work under their own name must understand what they submit and exercise judgement about the grounds for accepting it. “AI gave me this answer” is not sufficient to discharge an author’s or researcher’s responsibility.
This applies beyond the natural sciences. Facts, materials and arguments in the social sciences also need to be open to scrutiny. Is the cited survey real? Does its sample support the conclusion? How were the interviews conducted? Were contrary cases omitted? These are questions that can be pursued and checked. Interpretive and normative arguments likewise need clear concepts, premises, reasoning and value judgements. Different research may require different kinds of scrutiny, but that does not remove the need for evidence and argument.
For example, if AI writes an article about a social phenomenon, its author should do more than check whether the prose reads smoothly. They need to know which statements report facts, which interpret those facts, and which express their own value judgements. If someone proposes another explanation, can they say why they chose their interpretation and what evidence might change their mind? Surface completeness and fluency cannot answer these questions.
The same applies to software development. Even if AI produces most of a particular feature, the person submitting it needs to understand its key behaviour, inputs and outputs, exceptional cases, and relationships with other components. Especially where architecture, coordination between modules and requirements trade-offs are involved, passing tests for a local component does not establish that the whole system is correct. AI can participate in this work, and people need not generate every detail themselves. But whoever is responsible for delivery must be able to explain why the system is organised as it is, what has been verified and what remains uncertain.
Mathematics puts the issue in a stricter form. Many successful examples do not ordinarily prove a proposition about infinitely many objects. Nor does an apparently rigorous AI-generated proof automatically establish a theorem. We need to check the statement, premises and deductions, rather than substitute the model’s past success rate for verification of this proof.
Here we also need to distinguish verifiability from understanding. In a formal proof tool such as Lean, a logical kernel can check a proof against specified formal rules. Accepting the result also requires checking its axioms, the completeness of the proof, and whether the formal statement accurately expresses the original question.[3] This is a different kind of check from asking another conversational model to say, “I think it is correct.”
Suppose a proof is so long that no individual has read every line, but it has undergone reliable formal verification. The absence of step-by-step human understanding does not by itself reduce it to a probabilistic empirical law. Whether a proof is valid and whether people fully understand its reasoning are different questions. Formal verification can provide important grounds for accepting a conclusion without also providing an understanding of the proof’s structure or mathematical significance.
This is where the rational human oversight I am arguing for needs to apply. People need to know what the proposition says, which premises it uses, why the verification method supports the conclusion, and what assumptions that trust depends on. If a machine merely declares a result correct, while people cannot say what it has proved or why it should be believed, the machine’s affirmation is not sufficient grounds for acceptance.
Other fields may rely more on experiments and empirical checks, but this cannot simply mean “a sufficiently high accuracy rate is enough.” Whether a check is appropriate depends on what it measures, the conditions under which it applies, the cases it misses and the consequences of error. People may not need to know every internal step by which a model produces an output. They do need to understand the evidence and methods used to accept it. That understanding makes acceptance a reasoned judgement.
How can we tell whether someone has absorbed what AI has supplied? I think we can borrow a method from teaching: see whether they can teach it to someone else.
“Teaching it” cannot mean merely repeating the generated text. Other people may approach it from different angles, change the conditions, propose counterexamples or ask questions that were not anticipated. Someone who understands should be able to follow these questions, explain which judgements still hold and identify which require revision. Even when they cannot answer immediately, they should be able to locate the difficulty and explain what information is missing, rather than only asking AI to produce another response.
This does not require anyone to know everything. Teaching others is one useful test of understanding, not a sufficient guarantee of understanding or correctness. Someone may explain a mistaken idea very clearly, so explanatory ability still needs to be supported by evidence and verification.
When I say someone should be able to explain independently “even with AI removed,” I do not mean they must reproduce every detail from memory without references, code or verification tools. I mean they cannot depend entirely on AI to answer questions for them on the spot. They need to grasp the core concepts, key grounds and limits of the result, participate in discussion, respond to challenges and judge what to investigate or verify next.
Originality can then be assessed at two levels. At one level, we treat the person and AI together as the collaborative whole that produced the result, and ask whether it introduced new questions, ideas, methods or theories. The original contribution need not come entirely from the person, and AI participation in a key step does not invalidate the originality of the overall result. Of course, treating the person and AI together does not automatically make the result original. It still needs to be compared with existing knowledge and methods.
At the other level, we look inside the collaboration and identify the human contribution. Who posed the problem? Where did the initial idea come from? How was the key method developed? Which revisions came from the person, and which elements were generated by AI? These distinctions remain necessary. The originality of the overall result and the extent of the human author’s original contribution are different questions. Especially when assessing someone’s research capability, we cannot use the first judgement as a substitute for the second.
This is also why understanding must be distinguished from original contribution. A student who fully understands a teacher’s proof has acquired that knowledge, but has not thereby originated the proof. Even if someone thoroughly understands and absorbs an AI-proposed method, explains it independently and responds to challenges, these abilities alone do not make the method entirely their own original contribution. Mastering an existing method and proposing that method oneself test related but distinct abilities.
The difference between human and AI originality also needs to be understood through how results develop. A person’s new idea or solution may draw on years of accumulated knowledge, established habits of thought, a longstanding doubt or a question that has never received a satisfactory answer. These accumulated experiences and sustained questions participate in the original work, but the final sentences, argument or proposal may reveal little of that history. A person’s capacity for originality has sources and a developmental process built over time; a single generated piece of work cannot readily substitute for them. Comparing only the wording and form of the final result can obscure this background to human originality.
AI also relies on capabilities developed through training and may use context and persistently stored information. But these cannot simply be equated with an individual researcher’s accumulation of knowledge, habits of thought and sustained questioning. People and AI may propose equally valuable new methods through different processes. Emphasising this difference does not deny AI’s capacity to make original contributions. It means that similar final results do not make human and AI originality identical. Understanding and absorbing an AI-proposed method does not retroactively turn its original development into the person’s own creative process. The validity of a result, the submitter’s understanding and the attribution of original contributions remain connected questions that cannot substitute for one another.
When assessing personal capability and contribution, these questions need separate answers. In practical problem-solving, however, we need not make personal human generation a universal prerequisite. AI can move beyond organising information and editing language to generating code, proposing solutions and developing theoretical arguments. As its involvement grows, the requirements placed on people need to become more concrete. Anyone submitting work under their own name should be able to explain what has been established, on what grounds, what remains uncertain, and respond to substantive questions about the work. Under these conditions, substantial AI involvement is a form of collaboration we should allow. Where it demonstrably produces better results, it deserves encouragement and recognition.
References
[1] Cui et al., The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers. Microsoft Research, research page dated June 2025. The figures cited concern completed tasks, not changes in code quality.
[2] Becker et al., Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. METR, 10 July 2025. The results cited concern early-2025 tools in this experimental setting, not all current tools and tasks.
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
[3] Lean official documentation, Validating a Lean Proof. Explains formal proof checking, the axioms on which it relies, and checks on the meaning of the proposition.
https://lean-lang.org/doc/reference/latest/ValidatingProofs/
Sources checked on 14 October 2026.
Discover more from Geoffrey Chen
Subscribe to get the latest posts sent to your email.