How Do We Ask Questions and Exercise Judgment in the Age of AI?

When people discuss the abilities we still need in the age of AI, two familiar answers are that we must learn to ask questions and retain our judgment. Both make sense, but neither explains enough. AI can itself pose questions, draw conclusions and compare options, so we cannot simply reserve all questioning and judgment for humans. We need to ask why we raise a particular question, why we pursue that direction, and on what grounds we accept an answer. [1][2]

Someone with a background in mathematics may see a few scattered conditions, think of a possible result and wonder whether they are connected. The question draws on accumulated knowledge, perhaps alongside intuition, association and habits formed through years of practice. Many of our everyday questions arise this way. We do not have to overturn a whole body of knowledge every time we ask something.

Questioning also has a more elementary aspect. Before receiving any formal education, infants attend to unfamiliar things and explore events that violate their expectations. This does not mean they have no knowledge or experience. It does show that exploration need not wait until a mature theory has been mastered. Curiosity can make something worth finding out about even when it has no bearing on the task immediately at hand. [3]

In everyday life, curiosity and reasoning from existing knowledge are often intertwined. Knowing more may reveal more questions; having questions may make us willing to keep learning. We distinguish these elements to understand what questioning involves, not to label each question as coming either from curiosity or from logic. Whether AI could develop an experience of curiosity similar to ours can remain an open question. We do not have to settle it before discussing how to use AI today.

Developing the ability to ask questions therefore requires more than teaching people prompt templates. It is worth cultivating a willingness to notice what we do not understand, to pause when something contradicts our expectations, and to connect a present problem with what we already know. The arrival of AI has not made existing knowledge redundant. Sometimes it is precisely because we know a little about a subject that we can see what is wrong with an answer, or what else deserves investigation.

Yet a willingness to explore does not determine where to explore. Our purposes, interests and values enter here.

Suppose we ask AI to draw up a piano-learning plan. We first need to clarify what we hope to gain. Do we want to get through another piece as quickly as possible, understand harmony and improve our listening skills, or simply enjoy some music each day? These aims can coexist, but different priorities will produce different questions and plans. Asking only how to learn fastest can easily make speed the sole criterion, leaving the reason we wanted to learn piano out of the question altogether.

Familiar distinctions between truth and falsehood, correctness and error, good and evil, or beauty and ugliness can help us find different directions for questioning. They do not, however, express the same standard. When asking AI to edit an article, we might ask whether its factual claims have support, whether its reasoning holds, whether its wording is fair, and whether it reads naturally. Each question addresses something different. Beautiful writing does not establish factual reliability; a persuasive argument is not necessarily fair to others. Liking an answer certainly does not make it true.

Asking a better question often means bringing a previously overlooked criterion into view. After asking how to make something more effective, we can ask how effectiveness is measured, what might be sacrificed and who will be affected. AI can help identify these omissions and even point out conflicts between our original aims. Questioning does not end once we have explained the task clearly. It changes as our understanding develops.

The same applies to judgment. It does not take place only after AI has answered. Choosing what to ask already involves judging what matters. Reading an answer requires us to assess whether its grounds are sufficient and its conditions apply. Before acting, we also need to decide whether we are willing to accept the costs involved. An apparently good answer may reveal that our initial question was too narrow.

Some work can be delegated extensively to AI. When calculations, comparisons or preliminary organisation have clear objectives, readily checkable results and easily corrected errors, there is no need to insist that a person repeat everything. But accepting a particular answer still requires appropriate grounds. For factual claims, return to original sources wherever possible; for calculations, check the assumptions and results; for specialist questions, recognise that further verification may be needed. Asking the same AI again may produce another formulation, but it does not automatically provide independent verification.

When an answer bears on personal choices, aesthetic preferences, effects on others or responsibilities we must bear, an AI recommendation does not allow us to skip those considerations. AI can analyse the difference between two ways of expressing an idea, but being more likely to attract readers does not, by itself, entitle it to decide how an author wishes to write. A proposal may improve efficiency, yet who bears the cost and whether those affected have a say must also enter the decision.

Retaining human judgment does not mean assuming that humans will always be smarter than AI. We make mistakes too, and should be receptive to the counterexamples and reminders it offers. What matters is that people involved in a decision have the opportunity to understand, question and refuse, rather than clicking an approval button without understanding and then bearing all the responsibility. [4]

So when we tell students or AI users to learn to ask questions and improve their judgment, we need to go further. They need opportunities to use what they know to notice questions, explain why they care about them, and distinguish the different demands of facts, preferences and values. Once they have an answer, they should be able to examine its grounds, identify omissions and recognise when another person’s expertise is needed. This gives questioning and judgment something concrete to practise.

Sources

[1] Du, X., Shao, J., & Cardie, C. (2017). Learning to Ask: Neural Question Generation for Reading Comprehension. https://aclanthology.org/P17-1123/

[2] Trinh, T. H., et al. (2024). Solving olympiad geometry without human demonstrations. Nature. https://doi.org/10.1038/s41586-023-06747-5

[3] Stahl, A. E., & Feigenson, L. (2015). Observing the unexpected enhances infants’ learning and exploration. Science. https://doi.org/10.1126/science.aaa3799

[4] Santoni de Sio, F., & van den Hoven, J. (2018). Meaningful Human Control over Autonomous Systems: A Philosophical Account. https://doi.org/10.3389/frobt.2018.00015