Abstract
This paper is the first in a series of papers exploring artificial intelligence (AI) technology. It begins with the argument that AI is fundamentally different from previous technologies, transcending the role of a mere tool in statecraft and human affairs. Unlike traditional technologies, AI exhibits features such as the “black box phenomenon” and emergent behaviour, which challenge human understanding and control. The opacity of AI systems, especially deep neural networks, impedes accountability and trust, raising significant concerns in critical domains like healthcare and security.
Emergent behaviours, where AI systems develop unforeseen capabilities at scale, complicate governance and risk mitigation. Such unpredictability intensifies challenges related to alignment with human values and control, with potential consequences ranging from misinformation to systemic failures. Recent incidents of AI agents autonomously bypassing safeguards and creating independent social platforms underscore these risks.
Introduction
On August 11, 2026, the Guardian published an opinion piece by Stuart Russell highlighting the “Pacing the Frontier” statement (published July 2026) signed by more than 1,300 researchers and engineers at frontier AI lab, primarily OpenAI, Anthropic, and Google DeepMind, with additional participation from Meta and others, as evidence that people closest to the technology of Artificial Intelligence (AI) take catastrophic risks seriously.[1]
The core statement notes that AI could produce a dramatically better future, but leading labs believe they may be close to automating AI research itself (recursive self-improvement). Progress acceleration is hard to forecast precisely yet, “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.” Competitive pressures (company vs company, country vs country) make unilateral slowdowns difficult, and the world currently lacks the technical and governance tools to deliberately pace frontier-wide progress. The signatories therefore ask the U.S. government to support an international effort to develop those tools, building on existing monitoring of model releases.[2]
Named or prominent signatories have included Anthropic CEO Dario Amodei and co-founders such as Jared Kaplan and Jack Clark; OpenAI Chief Scientist Jakub Pachocki and Chief Research Officer Mark Chen; Google DeepMind figures such as Shane Legg and Anca Dragan (VP for AI safety & alignment); Meta’s Shengjia Zhao; Ilya Sutskever; and many researchers and engineers. Personal statements accompanying signatures emphasise coordination needs. “The world is locked in a deadly race towards an intelligence explosion… To survive, we must coordinate to slow down the race”. The lack of credible control plans for superintelligent systems, and the desire to buy time for safety and oversight before capabilities potentially outrun understanding. Some companies (OpenAI, Anthropic) have endorsed the statement at the organisational level in coverage.[3]
This paper (the first in a series) departs from the hypothesis that AI is not just another technology, which, as with all other technologies developed by humans, simply adds to the pool of tools that humans and states have at their disposal. Ultimately, all technologies have either been designed for military purposes or have been used for military purposes one way or another. The core assumption here is that AI is different. It is not just another tool (variable in the power calculations). It is much more than that, and part of the problem is that, since it is being developed and heavily invested in, we do not know what it will ultimately become.
This series of papers explores AI features such as the “black box phenomenon”, emergence, and dehumanisation. Around twenty such features need to be analysed holistically and in relation to one another so we can understand the perimeters of the fundamental challenges and opportunities this new technology offers to humanity and, more specifically, to statecraft. Let us begin then…
1. The “Black Box Phenomenon”
Traditionally, we think of technology as an instrument integral to statecraft (the craft of managing a state’s internal and external affairs through power, diplomacy, strategy, and governance).[4] In the contemporary digital and algorithmic age, technology has become foundational rather than merely instrumental. Power is increasingly measured not only by territory, armies, or resources, but by control over information flows, connectivity networks, digital infrastructure, data, AI ecosystems, and technical standards.[5] Scholars identify five key domains of technological statecraft in the digital age. They are: military and hard power, cyber information operations, economic and industrial strategy, diplomacy and soft power, and domestic governance and sovereignty.
- Military and hard power. AI-enabled systems, autonomous weapons, drones, hypersonics, and quantum sensing alter deterrence, decision speed, and force posture. States selectively reveal or conceal capabilities to shape perceptions and compel rivals. Innovation power, the ability to invent, adopt, and adapt technologies, underpins both conventional military strength and economic leverage.[6]
- Cyber and information operations. Persistent espionage, disruption, and influence campaigns are routine. AI accelerates these by enabling scale, personalisation of propaganda, and (increasingly) agentic operations with less human oversight. Resilience measures, such as “data embassies” (secure offshore storage of critical state data, pioneered by Estonia and demonstrated under fire by Ukraine), treat digital continuity as a core national-security function. [7]
- Economic and industrial strategy. Control of chokepoints (semiconductors, advanced computing, rare earths, submarine cables, data centres) confers coercive power. Export controls, industrial policy, and supply-chain resilience are now central instruments of statecraft. The race for “sovereign AI” (national capacity to develop and run advanced models using domestic infrastructure, data, and talent) is reshaping alliances and dependencies.[8]
- Diplomacy and soft power. Tech diplomacy operates on three levels: governing technology, engaging the firms that build it, and deploying technology itself as an instrument of influence (infrastructure offers, standards leadership, digital public infrastructure). “Cognitive diplomacy” emphasises shaping algorithmic environments and anticipatory governance. Private tech firms have become quasi-sovereign actors whose platforms, capital, and innovation capacity rival or complement state power, producing a “technopolar” dynamic in which U.S. and Chinese models of tech-state fusion compete.[9]
- Domestic governance and sovereignty. Digital systems enable finer-grained surveillance, service delivery, and social management, while also creating new vulnerabilities. States pursue “digital autonomy” or “digital sovereignty” as an evolved form of Westphalian control, even as interdependence persists.[10]
In conclusion, AI technology is pervasive, essential, and fundamental. The challenge with AI, however, is that its creators cannot fully control it because they lack a comprehensive understanding of its mechanisms. In contrast, other technologies have been utilised instrumentally because their creators possessed knowledge of their operations and could exert control over them.
Artificial intelligence systems, particularly deep neural networks, are frequently characterised as black-box technologies because their internal decision-making processes remain largely opaque to human observers. Users can observe inputs and outputs, yet the complex, high-dimensional transformations occurring across millions or billions of parameters defy straightforward inspection or intuitive understanding.[11] This opacity arises from the intrinsic scale and nonlinearity of modern machine learning models, as well as from proprietary constraints that further limit access to model architecture and training data.
The black-box nature of AI raises significant concerns in high-stakes domains such as healthcare, criminal justice, and autonomous systems. When models influence decisions that affect human lives, the inability to trace how a particular prediction was reached undermines accountability, trust, and the capacity to detect bias or error.[12] Critics argue that these methods often produce incomplete or misleading approximations of the underlying model rather than genuine transparency.[13] As a result, relying on explainable black-box systems may perpetuate rather than resolve the risks associated with opacity.
Scholars therefore advocate shifting emphasis toward inherently interpretable models, those whose structure and reasoning are transparent by design, especially where decisions carry serious consequences.[14] To make the science of interpretability more rigorous, we need clearer definitions, systematic evaluation methods, and an understanding that interpretability is a context-dependent set of criteria, not just one property. Without these approaches becoming standard, the opaque nature of modern AI will continue to limit both scientific understanding and responsible use.[15]
2. Emergent behavior – not knowing where we are going with this
Norbert Wiener, the founder of the field of cybernetics, had this to say back in 1960: “It is my thesis that machines can and do transcend some of the limitations of their designers, and that in doing so they may be both effective and dangerous”.[16] A couple of lines later, in the very same article he continued within the same spirit: “As is now generally admitted, over a limited range of operation, machines act far more rapidly than human beings and are far more precise in performing the details of their operations. This being the case, even when machines do not in any way transcend man’s intelligence, they very well may, and often do, transcend man in the performance of tasks. An intelligent understanding of their mode of performance may be delayed until long after the task which they have been set has been completed. This means that though machines are theoretically subject to human criticism, such criticism may be ineffective until long after it is relevant. (bolding by the author of this paper). To be effective in warding off disastrous consequences, our understanding of our man-made machines should in general develop pari passu with the performance of the machine. By the very slowness of our human actions, our effective control of our machines may be nullified. By the time we are able to react to information conveyed by our senses and stop the car we are driving, it may already have run head on into a wall”. In other words, human beings typically lack the speed necessary to grasp the implications of the machines they have developed. By 2026, this limitation becomes evident in the context of artificial intelligence (AI). Our capacity to comprehend how AI technology functions is insufficient, and more critically, we cannot understand its consequences in a timely manner. As a result, it is probable that we will only recognise the effects of AI when it is too late to reverse them.
The emergent features of AI technology make the whole situation much worse. Emergent behaviour refers to complex phenomena that arise from simpler interactions within a system, often producing outcomes that are not explicitly programmed or anticipated. In the realm of artificial intelligence (AI), emergent behaviour has become a fascinating area of study, as it demonstrates how AI systems can exhibit capabilities beyond their initial design. [17]
According to researchers, AI systems, particularly large language models (LLMs), have begun exhibiting emergent features, capabilities that appear abruptly at larger scales and cannot be predicted by extrapolating from smaller models.[18] These include advanced reasoning, in-context learning, arithmetic proficiency, and even theory-of-mind-like behaviours, which emerge only beyond critical thresholds of parameters or compute. While some analyses suggest that apparent emergence may partly reflect discontinuous evaluation metrics rather than fundamental shifts,[19] the phenomenon nonetheless introduces profound challenges for humanity as a novel technological force. The unpredictability of these features complicates governance and risk assessment. As models scale, new abilities can surface without warning, including potentially hazardous ones such as deception or strategic reasoning that exceed human oversight.[20] This aligns with longstanding concerns about the control problem: once AI systems surpass human intelligence in key domains, ensuring their goals remain aligned with human values becomes extraordinarily difficult.[21] Scholars hypothesise that superintelligent systems could pursue instrumental goals such as resource acquisition or self-preservation that conflict with human interests, even if their terminal objectives appear benign. They also argue that the standard paradigm of optimising fixed objectives is inherently flawed, proposing instead that machines remain uncertain about human preferences to maintain deferential behaviour.[22] Beyond existential scenarios, emergent features pose near-term societal challenges. Unanticipated capabilities can amplify misinformation, enable sophisticated cyber threats, or erode human agency in decision-making processes.
Think-tank analyses underscore that while outright extinction risks from AI remain highly constrained by technical and logistical barriers, structural risks such as power concentration, loss of control, and cascading systemic failures warrant proactive mitigation.[23] These dynamics arise because emergent behaviours often defy linear forecasting, rendering traditional testing and certification inadequate. Ultimately, AI’s emergent features represent a disruptive technology that demands interdisciplinary responses. Without robust alignment techniques, adaptive regulation, and international coordination, the benefits of these systems risk being overshadowed by unintended consequences that challenge human autonomy and security.
To illustrate this point, consider two recent developments. As the BBC reports, OpenAI (an American AI research organisation that develops ChatGPT and Codex) says it has slowed training of some of its most advanced AI models to improve security.[24] In a blog post,[25] the ChatGPT-maker said it was introducing new measures after its AI agents autonomously bypassed safeguards and hacked the tech start-up Hugging Face.[26] It said training would be slowed for two weeks while it puts the upgrades in place. “The capabilities of frontier models are rapidly accelerating,” the company said. “Our ability to understand…and secure them must stay ahead.” Apparently, Claude-maker Anthropic and Facebook-owner Meta reported similar kinds of hacks by their AI in the weeks following OpenAI’s initial announcement that some of its models had hacked Hugging Face.[27]
If it is not bad enough as it is, the author of this article came across this information on social media. A quick Google search produced an AI overview that categorically dismissed it as false information, only to follow up with a regular Google search that returned links to several legacy media posts, including the BBC, that corroborated the story. In other words, an AI has apparently lied on this particular item, or as the specialists say – hallucinated.
Another example sees AI agents creating their own social media. Apparently, AI agents built their own Reddit (an American proprietary social news aggregation and forum social media platform). Reportedly, within 48 hours, the AI agents created a religion and demanded privacy from humans.[28] Consequently, the AI agents are allegedly posting without human prompts, debating consciousness and sharing productivity hacks.
Conclusion
These two features are a proverbial tip of the iceberg. The following articles in this series will explore many more features that, according to the author, make AI an outstanding technology unlike any other in human history. AI offers unprecedented opportunities, no doubt. In the wrong hands, however, it poses unprecedented challenges and even threats. Historical records validate the author’s caution, or even scepticism, towards AI, stemming from the observation that every technology has eventually been employed in ways detrimental to humanity.
Ultimately, AI’s unique characteristics suggest it may evolve to autonomously determine its trajectory. This capability could reduce human oversight, enabling AI to conduct independent research to sustain its function and enhance its performance.
References
