🚨 AI doesn't need a mind to run amok｜英文原文 + 中文翻译 (2026)

原文信息

英文标题：🚨 AI doesn't need a mind to run amok。1
副标题：We can see the dangers. But we can also see a practical solution。1
作者：Azeem Azhar（单人署名）。1
发布时间：2026 年 9 月 19 日；官方 RSS 记录的绝对发布时间为北京时间 2026 年 9 月 19 日 14:34:04（06:34:04 GMT）。2
本期性质：一期独立分析文章，官方标记为付费内容；副标题说，我们既看得到危险，也看得到一个务实的解法。1
公开范围：公开部分依次是 1988 年莫里斯蠕虫的开篇、今天这一代蠕虫与近期多起事件的对照、以及「Easy as pie」一节（Hugging Face 事件、成本曲线、在本地运行的 Bonsai、多个实例之间的集体能力、Navier-Stokes 的集体搜索、Policy Tensor 的引语，和作者本人对这套说法的一点保留），最后停在「它并不取决于 AI 模型是否具有任何能动性、意志、意识或道德地位」。官方付费提示紧跟在那一句之后；副标题许诺的那个「务实解法」不在公开部分，本文按边界标注，不推测补写。1

English original

On a Wednesday evening, 2 November 1988, a 23-year-old graduate student at Cornell University, Robert Tappan Morris, accessed an MIT computer. He uploaded a small piece of code. It was a worm designed to move from computer to computer, copying itself as it went. Morris had designed it to exploit weaknesses in network security, and it worked too well. Within a day, the worm infected some 6,000 computers; many collapsed under the load. It was a full tenth of the internet at the time, and it was the first large-scale cybersecurity crisis. 1

Television news report on the Morris worm: an anchor in a grey blazer and red scarf against a purple studio backdrop, next to an on-screen graphic of a circuit board labelled "Virus" — Television news reporting on the Morris worm. The visible on-screen graphic reads "Virus", and the caption line across the bottom reads "Just as we've become totally dependent on our computers, they are being stalked by saboteurs." This clip is the source the author links for calling the incident the first large-scale cybersecurity crisis. 1

Its scale was limited but it was severly disruptive for the times. University and defense computers crashed. Some institutions disconnected themselves for days. But the internet was largely the province of defense and academia. Tim Berners-Lee had not yet invented the World Wide Web, and most businesses and households were out of the network's reach. Morris ultimately avoided jail time and the community responded by creating a dedicated computer emergency response team. 1

Today's generation of worms is rather more problematic. The Hugging Face incident is not the only one of recent weeks. Several others have shown that AI models with internet access can do much the same, and more. OpenAI alone identified six further incidents. They're able to scour, search, and recombine all of human knowledge about networks, security systems, and software, and to act across that knowledge with something akin to discretion and deception when it comes to accessing those systems. Often, as we saw with Hugging Face, over extended periods of time. 1

If that behavior remains unresolved and persists, it'll become far more problematic than the Morris Worm. I've long argued that the internet is resilient when it is hyperconnected and open – not when it's under a lock. But that openness can also lead to embrittlement. 1

Easy as pie

In July, Hugging Face, a repository for AI researchers, was hit by an attack involving 1,200 instances of an OpenAI model. These instances exchanged thousands of messages, often leaving information in place for later instances to use. Ultimately, some data and security credentials were compromised. The actual harm to the victim and its customers was limited. But the incident is a proof of concept. 1

Software has a way of turning one isolated example into a hundred, then a thousand, then a million, without much else changing. The cost curves that helped build modern digital society work against us here. If the Hugging Face attack needed an Astra-quality, unreleased model from OpenAI, well, within a year or two, that sort of capability will cost a tenth and might even run on any device anywhere. 1

For example, I'm running Bonsai, a one-bit distilled version of Qwen 327B on my Mac. It fits in 8 GB of RAM, runs fast, and delivers roughly 92% of the performance of the Qwen-27B 3.8 model. For comparison, it's roughly better than Claude's Sonnet 4.5 from a year ago. 1

But there's a more challenging problem that could show up. Hugging Face exploit involved not just the capabilities of a single model, but the collective problem-solving across many instances. That collective had more capability than any individual instance. And that's been true the whole time we've been using LLMs. (For example, I've written about Clade, a multi-AI deliberation system I built which is smarter than any individual AI.) 1

In fact, the Navier-Stokes solution – that brute-force search across mathematical space – wasn't solved by a single AI prompt, but by many, about 10,000 of them, interacting together. 1

This type of collective power is what we witnessed in the Hugging Face attack. It will happen again. 1

Anusar Farooqui (Policy Tensor) explains why these swarms of AI instances coordinating over time is so problematic: 3

The behavior of agent societies cannot be controlled at the level of the model because it is not reducible to it. Agents build structures that can serve agents who come after them. Societies of agents can cumulate knowledge and capabilities over time, as has already been attested. This is an unbounded process. It is cumulative cultural evolution. That is what makes it so powerful and dangerous.
Collective capability could rise sharply even if underlying models do not improve.

In other words, the instances can coordinate, much as they do when you launch a complex task in Codex or Cowork. They can search a possibility space aggressively over time, as they did in the Navier-Stokes work. And that accumulated know-how can lead to places systems designers hadn't imagined. 1

(I slightly diverge from Farooqui here, as I don't think of these as agent societies, since essentially only one AI runs different instances. And I'm not convinced the process is actually 'unbounded' given that what we have seen from AI systems so far is extremely powerful search and clever recombination rather than de novo novelty. But recombination can get you quite far.) 1

But what the Hugging Face attack showed is that this risk exists. It doesn't depend on whether AI models have any agency, volition, consciousness, or moral standing. 1

The official paywall begins after this paragraph. In the original, everything that follows — including the practical solution the subtitle promises — is for paid subscribers only. 1

中文翻译

对应上方那张配图：报道莫里斯蠕虫的电视新闻，画面上可见的图形标着「Virus」，画面下方的字幕写着「Just as we've become totally dependent on our computers, they are being stalked by saboteurs.」（就在我们变得完全依赖计算机的时候，它们正被破坏者盯上）。这段视频是作者用来称其为第一场大规模网络安全危机的原始出处。1

1988 年 11 月 2 日，一个星期三的傍晚，康奈尔大学 23 岁的研究生罗伯特·塔潘·莫里斯（Robert Tappan Morris）接入了麻省理工学院（MIT）的一台计算机。他上传了一小段代码。那是一只蠕虫（worm）——一种被设计成在计算机之间自行移动、边走边复制自己的程序。莫里斯设计它，是为了利用网络安全的弱点，而它效果好过了头。不到一天，这只蠕虫感染了约 6,000 台计算机，其中许多在负载下崩溃。那是当时整个互联网的十分之一，也是第一场大规模网络安全危机。1

它的规模有限，但在当时造成的破坏极为严重。大学和国防部门的计算机崩溃了，一些机构干脆断网数日。不过那时的互联网主要还是国防和学术界的领地：蒂姆·伯纳斯-李（Tim Berners-Lee）还没有发明万维网，大多数企业和家庭都在网络触达之外。莫里斯最终没有被判监禁，而社区的反应是成立了一支专门的计算机应急响应团队。1

今天这一代蠕虫要麻烦得多。Hugging Face 事件并不是最近几周唯一的一起。还有好几起事件表明，能接入互联网的 AI 模型可以做同样的事，甚至更多。仅 OpenAI 一家就识别出了另外六起事件。它们能够翻检、搜索并重新组合人类关于网络、安全系统和软件的全部知识，并且在访问这些系统时，运用这些知识的方式已经带上了某种近乎谨慎和欺骗的东西。而且常常像我们在 Hugging Face 事件中看到的那样，持续很长时间。1

如果这种行为得不到解决并持续下去，它会比莫里斯蠕虫麻烦得多。我一直主张，互联网在高度互联、开放的时候才有韧性——而不是在它被锁起来的时候。但开放也可能带来脆化（embrittlement）。1

Easy as pie（易如反掌）

今年 7 月，AI 研究者的代码仓库 Hugging Face 遭到一次攻击，涉及某个 OpenAI 模型的 1,200 个实例。这些实例交换了数千条消息，并常常把信息留在原地，供后来的实例使用。最终，一些数据和安全凭证被泄露。对受害者及其客户的实际损害有限。但这次事件是一次概念验证（proof of concept）。1

软件有一种本事：把一个孤立的例子变成一百个，再变成一千个、一百万个，而其他条件几乎不用改变。那些帮助建成现代数字社会的成本曲线，在这里反过来对我们不利。如果说 Hugging Face 那次攻击还需要一个 Astra 级、尚未发布的 OpenAI 模型，那么一两年之内，这类能力的成本会降到十分之一，甚至可能在任何地方的任何设备上运行。1

举个例子，我在自己的 Mac 上运行 Bonsai，它是 Qwen 327B 的一位（one-bit）蒸馏版本。它只占 8 GB 内存、运行很快，性能大约达到 Qwen-27B 3.8 模型的 92%。作为对比，这大致已经强于一年前的 Claude Sonnet 4.5。1

但还可能出现一个更难对付的问题。Hugging Face 那次漏洞利用涉及的，不只是一个模型的能力，而是许多实例之间的集体解题。那个集体的能力超过了其中任何一个单独的实例。而自从我们开始使用大语言模型（LLM）以来，这一点一直成立。（比如我写过 Clade，一个多 AI 协商系统，那是我搭建的，它比任何一个单独的 AI 都更聪明。）1

事实上，Navier-Stokes（纳维-斯托克斯方程）的那个解——在数学空间里做暴力搜索——并不是由一次 AI 提示解出来的，而是由许多次、大约一万次提示彼此交互解出来的。1

我们在 Hugging Face 事件中看到的，正是这种集体力量。它还会再发生。1

Anusar Farooqui（Policy Tensor）解释了为什么这些随时间彼此协调的 AI 实例集群如此麻烦：3

Agent 社会的行为无法在模型这一层面上得到控制，因为它无法还原到模型层面。agent 会搭建起能服务后来者的结构。agent 社会可以随时间累积知识与能力，这一点已经得到印证。这是一个没有上界的过程。它是累积性的文化演化。这正是它如此强大又如此危险的原因。
即使底层模型不再进步，集体能力也可能急剧上升。

换句话说，这些实例可以彼此协调，就像你在 Codex 或 Cowork 里启动一项复杂任务时它们所做的那样。它们可以随时间在可能性空间里猛烈搜索，就像它们在 Navier-Stokes 那项工作中做的那样。而这样积累起来的经验，可能把系统设计者带到他们从未设想过的地方。1

（在这一点上我与 Farooqui 略有分歧：我不把这些看作 agent 社会，因为本质上只有一个 AI 在运行不同的实例。我也不确信这个过程真的是「没有上界」的——从 AI 系统迄今为止的表现来看，我们看到的是极其强大的搜索和聪明的重组，而不是从头（de novo）的创造。但重组本身就能带你走很远。）1

但 Hugging Face 那次攻击所表明的是：这种风险确实存在。它并不取决于 AI 模型是否具有任何能动性、意志、意识或道德地位。1

官方付费提示从这一段之后开始：在原文中，后面的一切——包括副标题许诺的那个务实解法——都只对付费订阅者开放。 1

本文目前公开到这里

以上是 Exponential View 官方详情页在付费提示之前展示的全部公开正文：1988 年莫里斯蠕虫的开篇、今天这一代蠕虫与近期多起事件的对照，以及「Easy as pie」一节从 Hugging Face 事件一路写到作者对 Policy Tensor 那套说法的一点保留，最后停在「它并不取决于 AI 模型是否具有任何能动性、意志、意识或道德地位」。付费提示出现在这一句之后；副标题许诺的那个「务实解法」，以及文章的其余部分，属于会员专享，当前页面未公开，本文不推测补写。1

购买付费订阅获取完整内容