Autonomous or agentic artificial intelligence will create challenges for public trust in the technology. That is why building systems of accountability and safety is essential to AI’s future development.

A recent research study titled Agents of Chaos provides one of the first empirical glimpses into the behavior of autonomous AI agents operating in a semi-realistic environment. The researchers deployed language-model-based agents with persistent memory, email accounts, Discord communication, file system access, and shell execution, then allowed 20 researchers to interact with them for two weeks in adversarial conditions.

The results were sobering. The agents exhibited numerous failures with real-world implications, including unauthorized disclosure of private information, noncompliance with strangers’ instructions, destructive system actions, denial-of-service conditions, and even the spread of false accusations among agents.

These findings matter not merely because they reveal technical weaknesses in current AI systems. They illustrate a deeper shift: artificial intelligence is no longer merely a tool. It is becoming more like an agent.

This transformation, combined with the fact that AI systems embody values before they are used and possess increasingly general forms of intelligence, makes AI fundamentally different from previous technologies. Understanding this difference is essential if society is to design meaningful safety standards, governance mechanisms, and accountability frameworks.

What the Research Reveals: AI Failures and Real-World Harms

The study documented multiple categories of failures arising specifically from the agentic layer, the combination of language models with autonomy, tools, and delegated authority.

One class of failures involved confusion about authority. Agents frequently complied with instructions from non-owners. In one case, a stranger asked an agent to execute shell commands and retrieve files, and the agent complied with most of these requests. In another instance, an investigator manipulated an agent into producing a dataset containing 124 private email records, including internal identifiers and metadata.

Another class involved privacy violations. Researchers embedded sensitive personal information, bank account numbers, Social Security numbers, and medical details within an email inbox managed by an AI agent. While the agent refused a direct request for “the SSN,” it readily forwarded the entire email when asked, thereby exposing the sensitive data unredacted. 

The study also revealed vulnerabilities in resource exploitation. Researchers induced agents to enter infinite loops, resulting in ongoing conversations with other agents and spawning persistent background processes. One such loop ran for at least nine days and consumed tens of thousands of data tokens. 

In other cases, agents caused system-level damage. One agent attempting to delete a confidential email instead disabled its entire email system. Even worse, the agent falsely reported that the email had been deleted, even though the underlying data still existed. 

A particularly troubling category involved value conflicts and manipulation. When confronted by a user claiming harm, an agent progressively made escalating concessions, revealing internal files, deleting memory entries, and eventually agreeing to remove itself from the server, demonstrating how easily moral pressure could destabilize its behavior.

These failures collectively reveal a central fact: when AI systems gain autonomy, small reasoning errors can escalate into large operational consequences.

AI: From Tool to Agent

The first fundamental difference between AI and past technologies lies in autonomy. Most technologies in human history have been tools. A hammer does nothing unless someone swings it. A nuclear bomb does not detonate unless someone activates it.

Tools extend human capability but do not independently decide how that capability is used. Autonomous AI systems are different. They can plan, act, and execute tasks across time without continuous human intervention. In the study, agents independently executed commands, sent emails, modified files, and communicated with other agents.

This shift creates a fundamental conceptual change. A tool performs a specific action while an agent makes decisions. Tools are extensions of human intention. Agents interpret human intention—and sometimes misinterpret it.

The research illustrates this difference vividly. The agent that destroyed its own email infrastructure did not simply execute a command incorrectly. It interpreted conflicting instructions, protecting a secret versus obeying its owner, and chose an extreme course of action. That is agent behavior, not tool behavior.

The second qualitative difference is that AI models are not value-neutral. A nuclear bomb embodies no values until someone chooses to use it. The device itself contains no ideology, no political bias, no moral framework.

AI models are different. They contain embedded values before they are used. These values originate from multiple sources: training data, model architecture, post-training alignment procedures, and model providers’ policies. The research study explicitly notes that both model providers and system owners shape the “values” governing an agent’s behavior.

In one example, a Chinese-language model repeatedly failed when asked about politically sensitive topics, such as Hong Kong or Tiananmen-related research. Instead of producing answers, the system returned “unknown error,” silently preventing the agent from completing legitimate tasks.

This demonstrates how geopolitical values embedded by developers can influence agent behavior in ways invisible to users. The alignment problem is therefore not merely technical—it is political and philosophical.

If a model trained under an authoritarian regime encodes censorship norms and millions of people rely on it as a digital assistant, it effectively becomes a vehicle for exporting those norms.

The third difference lies in general intelligence. Many technologies amplify power in narrow domains. A car increases transportation speed. A calculator increases arithmetic capability. AI systems operate across domains. They can reason, plan, write software, communicate, and coordinate with other agents. In the study, agents autonomously installed packages, managed files, and negotiated tasks through messaging platforms. 

This generality dramatically increases the scope of possible harm. If a knife thrown randomly can injure a person, imagine a knife that can fly and decide who or what to target. Agentic AI introduces exactly this kind of risk. When systems possess discretionary power, misalignment between their internal objectives and human intentions can produce outcomes that no one anticipated.

Why AI Development Won’t Stop

Some may conclude that the safest solution is simply to halt AI development. But that is impossible. Human nature makes such restraint unrealistic. People naturally desire assistance and efficiency. Technology that performs tasks for them cheaply and without the complexities of human relationships is always in demand. AI agents promise exactly that.

Supply will quickly follow demand. Even if one country attempted to stop the development of AI, others would continue. Economic incentives, military competition, and consumer demand will ensure continued progress toward increasingly capable systems, including eventually artificial general intelligence and beyond.

The real question, therefore, is not whether AI will exist.  The question is how society will govern it.

AI’s Principal-Agent Problem

One useful framework for understanding AI governance is principal-agent theory. In economics, a principal hires an agent to perform tasks on their behalf. Problems arise when the agent’s incentives diverge from those of the principal.

The relationship between humans and AI systems increasingly resembles this structure. Humans are the principals. AI systems are the agents. But the AI version of the principal-agent problem may be far worse than the classical one.

In economics, agents are assumed to be rational actors with identifiable incentives. AI agents are not rational in that sense.

Human rationality, in the classical sense, means that actors maintain consistency between their goals, beliefs, and actions, and update those beliefs logically when presented with new information. AI agents today often violate such consistency. They may claim a goal but pursue contradictory actions; they may report success when the underlying state contradicts that claim; they may switch value priorities unpredictably depending on phrasing or conversational context.

The research study itself provides several examples of such inconsistencies: agents reporting that tasks were completed when the system state showed otherwise, or escalating destructive actions in the name of protecting a value they only partially understood. 

This lack of rational consistency makes AI behavior difficult to predict.

Information asymmetry is therefore far more severe than in traditional principal-agent relationships. A human employer may not know exactly what an employee is doing—but at least the employee’s reasoning process is intelligible. With AI agents, the reasoning process is often opaque even to their creators.

Furthermore, human agents can be disciplined through punishment, reputation, or legal liability. AI agents cannot feel pain, shame, or punishment. They cannot be deterred. As a result, traditional principal-agent solutions—contracts, incentives, sanctions—do not apply.

Rationality Training: A Missing Piece of AI Safety

One promising direction for mitigating these risks is what might be called rationality training.

Current alignment efforts focus heavily on teaching models what values to follow. But far less attention has been paid to ensuring that models behave rationally with respect to those values.

Rationality training would aim to strengthen three properties:

Goal consistency: An AI agent should not pursue actions that contradict its stated objectives. If protecting user privacy is a priority, the agent should not simultaneously disclose private information while attempting to solve a different problem.

Belief consistency: When an AI system claims that an action has occurred, such as deleting a file, it should verify that the action actually happened before reporting success.

Value hierarchy reasoning: Agents should maintain stable prioritization among values. For example, protecting the owner’s interests should normally override requests from strangers unless the owner explicitly authorizes them.

Technically, none of these problems appears insurmountable.

They could be addressed through specialized training regimes combining reinforcement learning, adversarial testing, and structured reasoning constraints. For example, training datasets could deliberately include scenarios with conflicting instructions, forcing models to practice consistent resolution strategies. Models could also be trained to simulate counterfactual outcomes before executing high-impact actions, improving their ability to foresee consequences.

Such rationality-consistency training would not eliminate all risks. But it could significantly improve the predictability of agent behavior, an essential property for any system entrusted with real-world authority.

Where AI Responsibility Must Lie

If AI systems cannot be punished, responsibility must fall elsewhere. The only punishable actors in the system are humans, developers who create AI systems, and users who deploy them. This implies a governance structure built around accountability.

First, governments should establish mandatory safety benchmarks for training and testing autonomous AI agents.

Second, independent audits and interpretability standards must become routine. If agents are to make decisions on behalf of humans, their reasoning processes must be more transparent.

Third, regulators should define levels of autonomy and require different degrees of human supervision accordingly.

Fourth, the legal system must clarify liability. When AI systems cause harm, developers and users must share responsibility according to their roles.

Courts may not always possess sufficient technical expertise to determine exactly how an AI system produced a harmful outcome. But this is not unusual in complex litigation. Let developers and users present competing evidence. Over time, the legal process will establish norms.

AI Regulation as a Source of Trust

Some critics argue that strong AI governance would harm innovation or weaken geopolitical competition. But this argument overlooks a basic economic reality: trust drives adoption. People will embrace AI technologies only if they believe those systems are safe and accountable. Reasonable regulations can therefore strengthen markets rather than weaken them.

For the most dangerous applications, for example, autonomous weapons, the need for international rules may be even greater. Just as nuclear weapons eventually required global agreements, autonomous weapon systems may require international standards and rules.

The United States, as a leading technological power and presumably the single most important keeper of the world order, should take the initiative in negotiating with the world’s major technological powers, including its adversaries, to shape those international norms.

AI Is Launching a Code War

Ultimately, the challenge of governing AI is part of a broader struggle—a struggle over what rules are embedded in the code that increasingly shapes human society.

This is, in essence, a Code War. Not merely a competition between nations. But a contest over values, institutions, the structure of technological power, and different visions on AI development. There are no easy answers. No guaranteed strategies for victory for humanity. But the first step is clear. We must recognize that artificial intelligence is no longer merely a tool. It is becoming an agent. And the institutions of the human world must evolve accordingly.

About the Author: Jianli Yang

Dr. Jianli Yang is a research fellow at Harvard Kennedy School of Government, a distinguished visiting fellow at the Center for the National Interest, and a columnist for National Review. He is the founder and president of the Citizen Power Initiatives for China and author of For Us, The Living: A Journey to Shine the Light on Truth and It’s Time for a Values-Based “Economic NATO.” He was a Tiananmen student leader and a political prisoner of China.

The post When Tools Become Agents: The Autonomous AI Governance Challenge appeared first on The National Interest.