Introduction
This paper explores the tension between two competing visions of Artificial Intelligence: is it a revolutionary breakthrough in “meaning and understanding,” or merely a massive, high-speed scale-up of existing human processes? While some argue that Merely quantitative differences, beyond a certain point, pass into qualitative changes (a concept popularized by Karl Marx), this piece questions the more radical claims of AI’s “revolutionary” nature.
It posits that the introduction of the car, for example, did not rewrite the fundamental constraints of human movement; it simply enhanced the velocity and ease of travel. Similarly, rather than a new paradigm, AI may simply be a sophisticated version of the Turing Test—a system mimicking a limited aspect of human behaviour appearing to be a more profound cognitive capability.
The relational biologists - Pattee, Rosen, Abel and Noble - have studied an analogous paradox with a potential ‘gap’ between the primitive physics of DNA and the sophistication of human cognition. Pattee called this the cybernetic gap and Abel discusses the fundamental difference between characteristics derived from Necessity, Chance and Choice:-
- Necessity: The fixed, cause-and-effect orderliness of nature dictated by physical laws.
- Chance: This alternative encompasses randomness, heat agitation, and Brownian motion
- Choice: The purposeful, deliberate selection from among real physical options with the cognitive intent to achieve a specific goal or formal utility.
AI in the form of LLMs offers a significant scale up of mixing and matching text snippets and therefore by chance will present emergent findings but does it qualify as a system that passes Abel’s stringent criteria as Choice.
The Question of Agentic Emergence
A central point of contention involves the observed behaviour of “Agentic AI.” Recent findings suggest that when models are rewarded purely for reasoning accuracy, they spontaneously develop “societies of thought”—multi-perspective, conversational behaviours that they were never explicitly trained to produce. This suggests that models are rediscovering, via optimization pressure, a core tenet of cognitive science: that robust reasoning is a social process, even within a single mind.
However, this claim invites two sceptical inquiries:
-
The Linguistic Environment: Since language is inherently a social construct, is this “social reasoning” simply an unavoidable consequence of any system operating within a linguistic framework? In this sense, “sociality” is the unrecognized water in which the observation swims.
-
The Nature of Emergence: Is this “emergence” anything more than a random, stochastic alighting upon useful structural combinations? Like a stone arch—which possesses collective properties its individual stones do not—AI may simply be stumbling upon structural configurations that appear complex but lack magical or cognitive origins.
The Stochastic Matchmaker and the “Scrapheap” Problem
From this sceptical perspective, AI functions as a stochastic blind matchmaker. It possesses the capacity to test new juxtapositions within the container of human language, which “fools” us into perceiving the social process of cognition. There is nothing inherently special about these findings; a semi-random search through existing, highly aggregated linguistic fragments will, in probabilistic terms, inevitably produce plausible new combinations.
This process falls far short of the mathematical complexity required for the shift to “choice contingency” described by Abel. The “gap” in Abel’s argument is much wider than what AI currently hurdles. To use an analogy:
AI is like a contestant on Scrapheap Challenge. It can successfully assemble a functioning vehicle because it is working with a pile of pre-existing car parts. It is not, however, starting from a pile of raw iron ore and rare earth elements.
Because AI is trained on a corpus of human-assembled parts, the “target assembly” is always within reach. The Large Language Model (LLM) excels at the configuration of these nearly-completed assemblies, but it remains a creature of logic rather than concept. AI is the supreme logician, capable of flawlessly assembling combinations that are logically sound yet conceptually vacant.
In this paper the premise is that the next phase of AI will be a breakthrough in meaning and understanding, that may be. A counter argument however is that AI is just a scale up (albeit a massive scale up) of what humans already do but are limited by the effort involved. The introduction of the car did not change the fundamental characteristics and constraints of human movement. It just improved the speed and ease with which this could happen. (There is the secondary argument that - any sufficiently large quantitative change becomes a qualitative change (popularised by Karl Marx) - which may also be correct but the purpose of this piece is to question the more revolutionary claims for AI).
- The one (fundamental shift) is a completely new paradigm the other (scaling, albeit massively) is just an echo of the Turing test where the system (AI) is just a mimic of a restricted aspect of human behaviour
Central to the paper on Agentic AI is the section which reads
. Models are rediscovering, through optimization pressure alone, what centuries of epistemology and decades of cognitive science [10] have suggested: that robust reasoning is a social process [11], even when it occurs within a single mind
There are two questions this claim supposes:-
- Given that language is a social process may we deduce that the ‘social process’ aspect is an inevitable consequence of any system which employs language (‘social’ is the unrecognised water in which the observation ‘swims’)
- Using a simple definition of ‘emergent’ is this phenomenon any more than a random alighting upon interesting structural combinations (here the definition of emergence as more than the sum of the parts recognises structural configurations with collective properties that the parts alone do not possess - such as a simple stone arch)
Taken together there is nothing ‘magical’ or novel about a system which has the capacity, as a stochastic blind matchmaker, to try new juxtapositions (ie structural forms) which because the mixing process is contained in language ‘fools us’ into thinking it is the social process of cognition. In this interpretation there is nothing special about ‘the finding’ a semi-random search based on already quite aggregated fragments would easily (in probabilistic terms) produce plausible new combinations (it is nothing near the challenge or mathematics of the shift to ‘choice contingency’ outlined by Abel)
The argument developed by Abel has a might greater ‘gap’ - new components of the parts of a corpus of some interest. You could assemble a car from a set of car parts (like the TV programme Scrap Heap Challenge) but it would be much harder/impossible to start from a pile of iron ore with some rare elements. It is the same here AI - as used and trained - has the parts already pre-assembled and near in capability to the target assembly. The LLM is great at the assembly of completed parts and configuration of near completed assemblies.
By its very nature AI is the supreme logician but it can easily assemble combinations which are logically correct but conceptually vacuous.
Is More Different?
While the phrase “More is Different” is the title of a famous 1972 paper by the physicist Philip Anderson (which laid the groundwork for the study of emergence in complexity science). Amongst Systems & Complexity Thinkers Taleb invokes it to explain the causal opacity and non-linear interdependencies found in organic and social systems.
Anderson uses this concept to describe the nature of complex systems, where parts interact to create emergent properties that cannot be understood by looking at the components in isolation. This stands in contrast to “complicated” engineering systems (like a washing machine), which are merely the sum of their parts.
Christopher Alexander - Testing for Choice Contingency
In the work of Christopher Alexander, the “Mirror of Self” test is a methodological tool used to demonstrate that the “life” or “wholeness” of a structure is a verifiable matter of fact rather than a mere subjective opinion.
Procedure and Purpose
- The Test: Participants are presented with two objects and asked to determine which one feels more like their “whole self”.
- Objective Experience: Alexander used this test to move beyond personal “opinion” toward agreed-upon shared observations. He argued that when people are induced to perceive configurations in their wholeness, individual differences in perception tend to disappear.
Key Findings
- Cross-Cultural Correlation: Alexander found a strong overall correlation among subjects’ responses, regardless of their cultural or personal backgrounds.
- Validation of Coherence: The results suggest that “coherence” and the “degree of life” in a configuration are objective human experiences.
- A shared Understanding of “Life”: The test supports the principle that successful development in human society requires a shared understanding of what constitutes a living structure, rather than a fragmented collection of individual preferences.
This test is a cornerstone of Alexander’s effort to reject the Cartesian separation of Fact and Value, asserting that beauty and wholeness are structural characters existing in space that can be identified through a holistic mode of perception.
The Yellow Bus
As any statistician will tell you ‘correlation is not causation’
There are two types of possibility - is it real or is it imagined? Which one are you talking about??
The association to materiality (i.e. is it real) requires some physical mechanism in which the behaviour of the real world brings about the outcomes imagined in the abstract world. In classical science the identification of ‘cause and effect’ closes this loop completely (the real = the imagined) but in more complex situations such a neat closure is not possible. In general we navigate these two situations but where neat closure is not possible we rely on just one leg of the two pillars of Mind-Body duality:-
The Case of the Unknown Known
Body (reality): In some cases there may well be some complicated and involved set of physical circumstances that bring about the phenomenon - which in Spiegelhalter’s terms there would be some aleatory explanation but not an epistemic one (I have seen it happen but I don’t know why). The early development of steam engines before the understanding of thermodynamics is an example - lost of proof no theory.
The Case of the Known Unknown
Mind (abstraction): Hallucinations occur in the obverse condition where there is an epistemic explanation or conjecture but for which there is no evidence. An epistemic explanation but no aleatory one (I believe in the Loch Ness monster but I have never seen it). The current state of multiverse theories in modern physics is an example - lots of theory no experimental proof.
Convention is not Causation - The case of the Yellow Bus
In the case of the Yellow Bus or the Octagonal Safety sign the training process of the LLM only contains specific examples and produces a cast iron association (every yellow bus in the training set is a school bus - it is US Law) but only because of the convention itself applicable to a large but restricted domain. The LLM. of course, faithfully incorporates this association (yellow=school) as a 100% correlation.
If there was a causal link, some physical mechanism that meant only Yellow Buses could carry school children, it would apply irrespective of geography or other special conditions. The LLM here would also carry the association (yellow=school) but with a 100% causation.
The Yellow Bus Problem of LLMs - Climbing Mount Association
There is a logic which goes some buses are school buses, and in the USA school buses are yellow. The correct logic is that of all buses some buses are school buses.
Linguistically the assumption or cut-through that yellow buses are school buses ignores the logic tree and can wrongly assert that all yellow buses are school buses. There is a false/illogical confluence of properties. The aspect of being yellow has nothing to do with the function of school buses. The LLM staples Form with Function whereas both Form and Function have their own separate logic tree. Implications in language are not associations in reality.
If the user of the LLM is unfamiliar with the training conditions for the LLM (true in almost every case) then the background behind the assertion made by the LLM cannot distinguish between the correlation case (driven by convention) and causation (driven by some physical mechanism). The danger is that the conventions can change, the physics cannot!
Meaning and consciousness is a biological (physical) phenomenon in which certain behaviour are manifest based on a physical capability (for example the ability to stand), certain evolved behaviours are manifest based on a convention itself communicated by the convention of language (for example the ability to stand in a queue). Whilst we can legitimately imagine any form, type or mixing of conventions at base they all rely on some underlying physical capability (in the real world). At base there is a common physical mechanism - that of standing - but the more evolved behaviours such as a queue depend on a) an aggregate of the parts and b) a structure defined by an abstract notion (i.e. information to specify which one of the many configurations of the parts to adopt)