AI生成的代码能运行，就可以使用了吗？ / If AI-Generated Code Runs, Is It Ready to Use?

Judgment in the Age of AI · Article 37

“It runs” proves only that the code did not immediately fail for one input. It does not establish correct edge cases, trustworthy dependencies, secure data handling, adequate performance or even that the real problem was solved.

Build a verification pyramid from the bottom

1. Understand: Explain each section; do not adopt what nobody can explain.
2. Static checks: Inspect formatting, types, dependencies, licences, known vulnerabilities and secrets.
3. Unit tests: Cover normal, empty, extreme, malformed and hostile inputs.
4. Integration tests: Exercise permissions, networks, databases and external services at real boundaries.
5. Human review: Have people who know the domain and security inspect the design, not merely the syntax.
6. Controlled release: Deploy narrowly with monitoring, rate limits and rollback.

A useful adversarial prompt

Do not rewrite the code yet. List five ways it could fail and design a test for each that could disprove the current implementation.

Asking the generating model to assess itself may expose obvious defects, but it is not independent verification. Use deterministic test tools and ensure that the person accepting responsibility has inspected critical paths.

References

• Cyber.gov.au: Guidelines for Secure AI System Development
• NIST: AI RMF Playbook