Judgment in the Age of AI · Article 6
Simply asking “Are you sure?” rarely helps. A model may restate the same answer in a more cautious tone without adding evidence. Useful uncertainty requires it to identify what is unknown, why it is unknown and what information could change the judgement.
Turn one question into four requested outputs
| Instead of asking | Request |
|---|---|
| “What is the answer?” | Give the answer and list the three facts on which it depends. |
| “How confident are you?” | Separate directly supported claims from inferences. |
| “Are you sure?” | Name the most likely failure points and one alternative explanation. |
| “Check again.” | Identify the external material required for verification. |
Why numerical confidence may mislead
A statement such as “90 per cent confident” may not come from a calibrated probability calculation. It can be another piece of generated language. Unless the system explains how confidence was measured, treat the percentage as a prompt for investigation, not a statistical guarantee. “Known, inferred and unknown” is usually more useful.
Try this prompt
Divide the answer into facts supported by sources, inferences from those facts, missing information, and evidence that could change the conclusion. Do not guess merely to appear complete.
Expressing uncertainty does not make an answer correct, but it reveals where verification should begin. A good answer does not hide its gaps.
References
Discover more from Geoffrey Chen
Subscribe to get the latest posts sent to your email.