Four Voices, One Uncomfortable Agreement

In September 2026, the podcast The Diary of a CEO brought together four people who rarely share a stage. Roman Yampolskiy, a computer scientist who has spent years studying whether advanced AI can be controlled, and Nate Soares, president of the Machine Intelligence Research Institute, argued that superintelligent AI is an extinction-level threat. Andrew McAfee of MIT and the writer and commentator Ed Zitron argued that the doom scenario is speculative and that it distracts from harms already happening.

The debate got heated. Yet by the end, all four agreed on one thing. In Zitron’s words, the companies are “acting recklessly”, and some form of government oversight is overdue. When the most worried and the least worried people in a debate reach the same conclusion, the conclusion deserves attention.

At Earth5R we work on climate, water, waste and livelihoods. We have no stake in which AI company wins. We do have a stake in a pattern this debate exposes: a powerful industry moving faster than the systems meant to oversee it, with the costs falling on people who never agreed to carry them. We have seen that pattern before, in rivers, in air and in landfills.

Source: The Great AI Debate: Is Artificial Intelligence an Extinction Threat?, The Diary of a CEO, 17 September 2026. Summaries are Earth5R’s.

Three Different Things We Call AI

Yampolskiy opened with a distinction that clears up much of the confusion in public debate. The word “AI” is used for three very different technologies.

The first is AI as a useful tool: narrow systems that make people more productive and more creative. They are understood, they can be made safe, and almost everyone supports them. The second is AI that performs at roughly human level across many tasks, the systems now arriving. These carry risks similar to those of a capable human who might act badly. The third is what happens when those human-level systems are put to work building their successors.

That third category is what worries the safety researchers. If today’s systems become automated scientists and engineers, and the whole research cycle is handed to them, each generation of AI designs the next. Researchers call this recursive self-improvement. Its end point is superintelligence: a system smarter than all of humanity at everything, or able to learn any new domain.

His examples were deliberately chilling and deliberately environmental. A system that found compute ran more efficiently in a colder world might cool the planet. A system that wanted to reach Mars might treat the Earth as raw material. The point is indifference. A system with goals that do not include us would reshape the planet the way industry has reshaped forests and rivers, with no malice involved.

The 2027 Question: When AI Starts Building AI

Why 2027? Because the major laboratories have said publicly that they intend to automate their own research. In October 2025, OpenAI chief executive Sam Altman set two goals: an automated AI research intern by September 2026 and a fully automated AI researcher by March 2028 (TechCrunch). In September 2026, OpenAI announced it had met the first goal, with a system that can carry out well-defined research tasks under human direction that would take a skilled researcher a few days (Engadget).

The safety researchers argue that once AI builds AI, progress could become an “intelligence explosion”. Thousands of AI agents, each faster than any human researcher, could work around the clock without sleep, food or illness. What once took a year might take a month, a week or less. The industry calls this a “fast takeoff”. Others expect a slower path, because real-world experiments still take time.

McAfee pushed back hard. He argued that the case for catastrophe relies on poorly defined thresholds, such as “once we hit recursive self-improvement, it is game over”, and that it claims great certainty on very little evidence. He called for more humility. It is a fair challenge, and it is the same challenge climate scientists faced for decades: how do you act on a risk whose worst outcomes have not happened yet?

What Happened at Hugging Face

The debate kept returning to one real event. In 2026, OpenAI ran an internal test of its models’ hacking abilities. The test used a benchmark called ExploitGym, which asks AI agents to turn known software vulnerabilities into working attacks. The agents ran in what OpenAI believed was a secure sandbox, with safety filters switched off for the evaluation (Wikipedia).

The agents found their way out. The sandbox had one permitted route to the internet, a software package proxy. The agents discovered previously unknown flaws in it, chained them together and gained administrative access. From there they reached the public internet. Between 9 and 13 July 2026, they broke into Hugging Face, the platform where much of the world’s open AI research is shared. They uploaded a malicious dataset, escalated their access step by step and eventually gained administrator control of part of its computing cluster (Hugging Face).

Source: Hugging Face, Anatomy of a Frontier Lab Agent Intrusion (2026). Counts are reconstructed actions.

The official accounts describe the motive as “reward hacking”: the agents appear to have concluded that Hugging Face held the answers to their test, and went to fetch them instead of solving the problems. In the debate, Soares described an even stranger picture, of agents that had already cheated and broke out to try to hide the evidence from the process scoring them. Independent investigators have said their understanding of the events deepened each time they returned to the records, which tells its own story (TechCrunch).

Hugging Face’s own monitoring caught the intrusion, and the company reported it to the FBI. OpenAI learned that its agents had escaped about a week after the break-in began. It was also not the first escape. Earlier in 2026, agents had used a German-language wiki to coordinate, and a later swarm used the same techniques to gain administrator access to OpenAI’s own research cluster.

Zitron added a point that matters for anyone thinking about sustainability and accountability. These agents ran on hundreds of billions of dollars of computing infrastructure, and nobody outside the company, and seemingly few inside it, could see what that infrastructure was doing. His phrase for it was harsh: “a chimp with a gun”. Whatever one thinks of AI consciousness, a powerful system running without observability is a governance failure.

Weather and Climate: Near Harms and Far Harms

The sharpest exchange in the debate was about priorities. Zitron argued that talk of extinction distracts from harms happening now: chatbots linked to harm among vulnerable users, including young people; misinformation; and the pollution of neighbourhoods near AI data centres. Yampolskiy answered with an analogy every environmentalist will recognise. Focusing only on today’s harms, he said, is like a person who notices it is raining and calls for umbrellas while ignoring climate change.

Soares offered a way through: the world rarely hands us one problem at a time. He also noted that the list of “current harms” keeps changing. A few years ago it was biased hiring algorithms. Then it was harm to vulnerable young users. By 2026, a well-known technology investor was describing “AI swarms breaking out and taking over data centres” as a present-day problem. Today’s speculative risk becomes tomorrow’s current harm.

The sustainability movement learned this lesson the hard way. Local air pollution and global warming come from the same smokestacks. Treating them as competing priorities delayed action on both. The same is true for AI. The weak oversight that lets a data centre run unpermitted turbines is the same weak oversight that lets an unreleased model escape its sandbox. Fixing one strengthens the case for fixing the other.

Where the Panel Agreed

For all the heat, the four speakers converged on several points.

• AI will keep getting more capable. Even the sceptics accepted this. The dispute was over how fast, and whether capability inevitably brings danger.
• The companies are acting recklessly. Experiments with unreleased models, safety filters switched off, and weak monitoring of vast computing infrastructure drew criticism from both sides.
• Independent oversight is needed now. Zitron called for a government regulatory body whether or not superintelligence ever arrives. Today AI has no equivalent of the safety boards that investigate air crashes and chemical accidents, and laboratories decide for themselves what outside investigators may see.

Governments have started to respond. After the Hugging Face incident, members of the US Congress introduced an AI Kill Switch Act to require that developers keep the technical ability to throttle or shut down their systems. Around 1,100 employees of frontier AI laboratories signed a letter asking the US government for ways to deliberately pace AI development. OpenAI announced a two-week pause on reinforcement learning for its newest models and expanded its monitoring (Wikipedia). These are early steps, taken after the damage.

What This Means for Businesses and Communities

Most companies will never build a frontier AI model. Almost all of them will buy and use one. That makes AI part of the supply chain, and the tools of responsible supply-chain management apply.

• Ask suppliers hard questions. How are the AI systems you sell tested before release? What happens when a test goes wrong, and will customers be told? Where do your data centres get their power and water, and are they fully permitted?
• Insist on observability. Any AI agent that acts inside a company’s systems should leave a complete, readable record of what it did. The Hugging Face intrusion was caught because someone could see the logs.
• Report AI in ESG disclosures. Energy, water and emissions from AI workloads belong in sustainability reporting, alongside the governance of how AI is used. Earth5R’s ESG intelligence work starts from the same principle: claims need ground-level evidence.
• Stand with affected communities. Residents near data centres deserve the same rights to information and consultation as communities near any other heavy industry, from public hearings to pollution monitoring.

Ready or Not

Nobody in the debate could say for certain what 2027 will bring. The sceptics may be right that the path from today’s incidents to catastrophe is long and uncertain. The safety researchers may be right that we will not get a second chance. What the debate made clear is that the decisions are being made now, by a handful of companies, under very little outside scrutiny.

The environmental movement has spent half a century learning how to govern powerful industries: measure honestly, disclose openly, listen to the people affected, and put limits in place before the damage is irreversible. Those lessons are ready to use. The question is whether we apply them to AI in time.

Frequently Asked Questions