<\/p>\n\n\n\n
By Geoffrey Chen<\/p>\n\n\n\n
My views on artificial intelligence have gone through several distinct stages. Because I studied computer science, I became interested in AI as an undergraduate in the 1980s. At that time, artificial intelligence was not a fashionable topic. It did not dominate news headlines, nor was it a focus of investor attention. It lived in a relatively quiet corner of computer science\u2014discussed seriously, yet almost in silence.<\/p>\n\n\n\n
Out of curiosity rather than any foresight about the future, I bought several books on artificial intelligence. What left the deepest impression on me was that I also learned the Lisp programming language\u2014entirely outside the formal curriculum. Lisp felt unfamiliar and abstract, yet extremely elegant. What set it apart from other programming languages was its ability to modify itself while it runs: code is data. If a machine can modify itself during execution, is that not a defining feature of life? At the time, I believed Lisp outlined a grand vision: intelligence could be built from symbols, rules, and recursive structures. In that era, it was easy to believe that the core of intelligence was essentially a problem of logic and representation.<\/p>\n\n\n\n
I came to Lisp through reading\u2014by going over its logic again and again\u2014until a quiet intuition began to take shape. If a program could take itself as an object of reflection, it would possess a rudimentary form of self-observation. Its homoiconicity, to a young person deeply in love with mathematics and logic, carried both a sense of life\u2019s first cry and a chilling hint of logical self-enclosure.<\/p>\n\n\n\n
What shocked me about Lisp was not only that it \u201cfelt alive,\u201d but that it crossed a formal boundary. A program could, for the first time, operate on itself. The chill I felt at that moment came from a more dangerous possibility\u2014when a system becomes coherent enough, we may read it as a kind of intention; and that \u201cseeming\u201d is easy for us to mistake as truth.<\/p>\n\n\n\n
That was the era of symbolic AI. People widely believed that machine intelligence should follow the path of human reason and formal logic. In 1969, Minsky and Papert offered a sharp critique of the perceptron\u2014especially pointing out key limits of single-layer structures in expressive power. This was not a final judgment on \u201cneural networks\u201d as a whole, but it did change academic confidence and funding trends for connectionism: academic legitimacy shifted toward symbolic logic that could be understood and derived, while statistical learning was increasingly treated as a side branch lacking rigor. True intelligence, at that time, was increasingly treated as a logical structure that could be explained and derived, rather than a black-box capability trained from statistical signals.<\/p>\n\n\n\n
For a long period after that, the development of artificial intelligence seemed unusually slow. Trying to describe how the world works using rational logic alone is extraordinarily complex\u2014almost impossible. Even if we assume such a complete model could exist in theory, at that time and for quite a long time, it was also hard to realize with the computing power available. Whether from theory or engineering practice, traditional AI gradually showed a bottleneck that was hard to cross.<\/p>\n\n\n\n
As research moved forward, people also began to realize that the bottleneck was not only a lack of computing power. It was rooted in human limits\u2014we do not truly have the ability to take a \u201cGod\u2019s-eye view\u201d and build a model of the whole world that is both complete and self-consistent.<\/p>\n\n\n\n
This is not only an engineering problem. More unsettling is that, even at the level of pure logic, we gradually realize that the world may not operate with the kind of \u201cnecessity\u201d we expect. What logical systems can express is always the relation between structures, not why the relation must be so. A program can exhaust all formal derivations, yet cannot answer a more basic question: why do these rules keep holding in reality?<\/p>\n\n\n\n
We can describe \u201cif A, then B\u201d in a model, but we cannot find within logic itself \u201cwhy A must lead to B.\u201d This lack does not appear as an error right away. It is more like a long-ignored blank\u2014covered in daily operation by the stability of experience, and temporarily hidden by successful predictions. But once we try to expand logical structures into a framework for explaining the entire world, this blank becomes impossible to avoid.<\/p>\n\n\n\n
It was in this background that the path of AI development gradually moved away from its original plan.<\/p>\n\n\n\n
Around 2010, this shift became clearer. The traditional symbolic path still moved slowly, while connectionism, long pushed to the margins, broke through when few expected it. As computing power rose, data grew, and training methods matured, deep learning began to show unprecedented ability in practice.<\/p>\n\n\n\n
Unlike earlier models that relied on clear rules and logical structure, this path does not try to understand how the world works first. Instead, through large-scale training, it keeps \u201capproaching\u201d the world in a statistical sense. Intelligence is no longer seen as the result of logical derivation, but more like an ability that emerges from complex systems.<\/p>\n\n\n\n
Until the last few years, I remained skeptical of connectionism. In my view, this kind of AI shows more of an outer performance of intelligence, rather than an inner process of logical reasoning. It looks as if it is \u201cunderstanding\u201d and \u201creasoning,\u201d but behind it there is no real grasp of causality or rational derivation.<\/p>\n\n\n\n
From the beginning, AI has not only been a computer problem; it is also a philosophical problem. How do we define and understand \u201cintelligence\u201d and \u201cconsciousness\u201d? These concepts, once highly abstract, have never been so directly and concretely tied to real, running technical systems as they are today.<\/p>\n\n\n\n
Here, David Hume\u2019s doubt about causality becomes sharp again. Hume pointed out that humans cannot directly sense the \u201cnecessity\u201d of causality in experience. What we observe is only the order of events in time and their closeness in space\u2014a regularity we keep noticing. Causality seems necessary not because it comes from an inner force of the world itself, but because it is a habit formed in the mind through long experience. When we ask ChatGPT why the sky is blue, its perfect explanation does not come from its understanding of Rayleigh scattering; it is more like a linguistic analogue of constant conjunction\u2014countless cases in which \u201cblue,\u201d \u201csky,\u201d and \u201cscattering\u201d recur together in its training data.<\/p>\n\n\n\n
This judgment is a hard, direct challenge to the modern tradition of reason. If causality is only habit, then natural scientific knowledge built on causal necessity loses its objective foundation. Immanuel Kant tried to rebuild the foundation of reason in this crisis: by introducing a priori structures of cognition, he repositioned causality as an a priori condition of experience\u2014not a law of the world itself, but a condition imposed by the understanding\u2014thus preserving necessity and universality for scientific knowledge. But at the same time, he also admitted that this necessity applies only to the world of phenomena; the world itself\u2014as the \u201cthing-in-itself\u201d\u2014still cannot be known.<\/p>\n\n\n\n
In that sense, the difficulty Hume revealed wasn\u2019t removed\u2014it was only displaced. The remainder that cannot be grasped by experience and concepts was taken out of the knowable world and sealed into the domain of the thing-in-itself. The ghost did not disappear; it was kept outside the boundary of reason. It no longer interferes with the daily operation of science, but continues to lie there in a form that is admitted yet cannot be handled.<\/p>\n\n\n\n
And today, the form of intelligence shown by large language models, and the semantic emergence we can\u2019t seem to avoid, again brings up deep doubt about the foundation of reason. These emergences are not system errors; they are results naturally produced when connectionism operates at scale. We not only do not avoid them\u2014we train models for them. The real problem is that we still find it hard to give a complete explanation of this emergence. This exposes a lack of clarity in connectionism at the level of outcomes,and also forces us to ask whether the logical derivation we treat as nearly mathematically strict is, at a deeper level, already built on stable habits in a statistical sense.<\/p>\n\n\n\n
Besides emergence, here is another more basic and more unsettling fact. Artificial intelligence itself does not know what \u201challucination\u201d is.<\/p>\n\n\n\n
\u201cHallucination\u201d is not a category a model can recognize. It is only a label humans put on certain outputs: these outputs are coherent in logic and correct in grammar, yet wrong in fact, or not related to the given input context\u2014when statistical coherence drifts away from truth and reference, we call it this. The machine is not a liar; it only faithfully carries out the goal we tell it to optimize\u2014even if we ourselves do not truly understand what the result of that optimization will be.<\/p>\n\n\n\n
What is truly unsettling is not that AI departs from reason, but that it carries out a form of reason too thoroughly\u2014a form that does not depend on real causal grounding. The explanations it produces are self-consistent in structure and restrained in language, yet they do not need real objects as support.<\/p>\n\n\n\n
The reason AI\u2019s hallucinations and emergences unsettle us is not that they deviate from reality, but that they come close to a fact we do not want to face: human reason itself is also, to a large extent, built on habit, repetition, and statistical stability.<\/p>\n\n\n\n
Modern civilization rests on a basic framework of reason made by formal logic, represented by the syllogism. But a question we cannot avoid is why humans can be sure that the \u201cmajor premise\u201d in a syllogism is valid\u2014whether its legitimacy comes from experience, intuition, social consensus, or some a priori assumption that can no longer be proven.<\/p>\n\n\n\n
For that reason, it\u2019s hard to reject large language models\u2019 statistical conclusions outright. Their inferences differ from traditional deduction, but it\u2019s far from clear that human reason has ever fully outgrown its own foundations in habit and statistical regularity.<\/p>\n\n\n\n
I would argue large models have not weakened the status of reason. They only reveal a long-ignored fact\u2014that from the beginning, reason has not been a necessary structure that can be described in a purely mathematical way.<\/p>\n\n\n\n
Against this background, a more basic question emerges\u2014an uncertain factor that has long moved along the boundary between theory and experience now seems to be entering concrete technical implementation.<\/p>\n\n\n\n
The famous \u201chard problem\u201d in the philosophy of mind is still one of the most disputed problems today. We still do not understand where self-consciousness comes from. Therefore, whether AI may have some form of self-consciousness cannot be simply avoided.<\/p>\n\n\n\n
My idea is that this uncertainty is like a variable that keeps drifting. In the past, it mainly stayed at the level of philosophical thinking; but today, it is entering AI systems and, for the first time, showing itself in an engineered, runnable form.<\/p>\n\n\n\n
We still cannot fully explain some surprising abilities shown by large language models. I am not unfamiliar with the basic principles of machine learning and neural networks, but even so, I still cannot answer why, when the depth and scale of a neural network reach a certain threshold, these abilities appear\u2014abilities that seem, in nature, quite different.<\/p>\n\n\n\n
Even Geoffrey Hinton, one of the central figures in deep learning, has increasingly framed this continuity in more radical terms. Between humans and machines, maybe there is no essential break\u2014this is intelligence, this is consciousness.<\/p>\n\n\n\n
I do not fully accept this view. Because what is at stake here is not only a dispute over definitions, but a deeper question: has this uncertainty that cannot be removed already truly entered contemporary AI systems?<\/p>\n\n\n\n
This ghost no longer lives in the fog of Kant\u2019s \u201cthing-in-itself.\u201d On chips and energy in data centers, it asks again, in a cold, statistical way: what is \u201chuman\u201d? We can no longer pretend it is abstract, because it has already entered the machinery of the real world.<\/p>\n\n\n\n
A more detailed and systematic account of the argument sketched here can be found in my book After the Knowing Subject.<\/em><\/p>","protected":false},"excerpt":{"rendered":" Artificial intelligence’s evolution began with Geoffrey Chen’s exploration of Lisp, a language that defied logic. In the 1980s, Chen was intrigued by AI’s potential for self-modification, questioning the boundaries between logic and life. Today, this echoes in neural networks’ emergence. What does this imply for future AI? Delve into the origins of AI’s captivating enigma.<\/p>","protected":false},"author":17897162,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"image","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_geoff_meta_description_en":"","_geoff_meta_description_zh":"","_wpcom_ai_launchpad_first_post":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"The Statistical Ghost in the Machine","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[212550,783694611],"tags":[],"class_list":["post-12050","post","type-post","status-publish","format-image","hentry","category-my-philosophy","category-philosophy-notes","post_format-post-format-image",""],"translatepress_titles":{"en_AU":"The Statistical Ghost in the Machine"},"jetpack_publicize_connections":[],"jetpack_likes_enabled":false,"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p1ar9H-38m","jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/geoffreychen.com\/zh\/wp-json\/wp\/v2\/posts\/12050","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/geoffreychen.com\/zh\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/geoffreychen.com\/zh\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/geoffreychen.com\/zh\/wp-json\/wp\/v2\/users\/17897162"}],"replies":[{"embeddable":true,"href":"https:\/\/geoffreychen.com\/zh\/wp-json\/wp\/v2\/comments?post=12050"}],"version-history":[{"count":1,"href":"https:\/\/geoffreychen.com\/zh\/wp-json\/wp\/v2\/posts\/12050\/revisions"}],"predecessor-version":[{"id":12051,"href":"https:\/\/geoffreychen.com\/zh\/wp-json\/wp\/v2\/posts\/12050\/revisions\/12051"}],"wp:attachment":[{"href":"https:\/\/geoffreychen.com\/zh\/wp-json\/wp\/v2\/media?parent=12050"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/geoffreychen.com\/zh\/wp-json\/wp\/v2\/categories?post=12050"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/geoffreychen.com\/zh\/wp-json\/wp\/v2\/tags?post=12050"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}