「宁可要知道自己何时不知道」：Zoubin Ghahramani 谈 AI 不确定性的数学

这期《Google DeepMind: The Podcast》是一场观点访谈。主持人 Hannah Fry 请来剑桥大学信息工程教授、Google DeepMind 研究副总裁 Zoubin Ghahramani，围绕一个听起来很抽象、却直接决定系统能不能落地的问题展开：机器怎样表达「我不确定」，以及为什么这件事比再堆一轮数据更难。

节目的主线很清楚。今天的大模型回答问题时常常语气笃定，错了也像很有把握；Zoubin 从 1980 年代末就在做另一条路线——用概率把不确定性写进智能系统。他用自动驾驶的长尾场景、校车被改成猎豹的对抗样本、大模型被一推就改口、天气集合预报和 AlphaFold 的置信度着色，把「正确性」和「置信度」拆开讲，并落到持续学习、能效和架构研究还缺什么。读者如果只想听一句「缩放够不够」，这期会明确站到「还需要架构与不确定性创新」一侧；如果你想听一位贝叶斯机器学习长期推动者如何把自己的主张接到 GenCast、AlphaFold 和 AGI 争论上，这约 45 分钟信息密度够高。

YouTube 导语把本期争论写成两边：一边指望纯缩放继续推高 AI；另一边像 Zoubin 这样，认为真正智能还需要架构创新，而改进机器的不确定性可能是缺失的一块。2

Zoubin Ghahramani 是英籍伊朗裔机器学习研究者，公开履历把学术与工业叠在一起：剑桥大学信息工程教授、圣约翰学院 Fellow；Google DeepMind 研究副总裁；此前曾任 Uber 首席科学家兼 AI 副总裁，并领导过 Google Brain，直至其与 DeepMind 合并。他在图模型、变分贝叶斯、非参数贝叶斯方法等领域有长期工作，2015 年当选英国皇家学会会士（FRS），2021 年获皇家学会 Milner Award。345

这与本期谈话一一对应。1980 年代末他就在宾夕法尼亚大学接触并行分布式处理与神经网络；1990 年代初转向概率与贝叶斯机器学习，并与 Geoffrey Hinton 合作过概率模型路线的深度学习尝试；2015 年在 Nature 发表 Probabilistic machine learning and artificial intelligence，主张智能的许多方面依赖对不确定性的谨慎概率表示。3 节目里他自称目前在 DeepMind 的职务表述是「frontier AI 联合负责人」之一；公开机构页仍以研究副总裁为主——摘要按公开页写职务，按节目原话保留他的自我介绍。

Hannah Fry 是节目固定主持人，官方播客页介绍她以数学家与广播人身份带听众进入 DeepMind 的研究现场。1

Zoubin 把中心论点压成一条因果链：

他把不确定性至少拆成两类。一类是世界本身的随机性，比如行人下一步往左还是往右——术语里常称 aleatoric uncertainty（偶然不确定性）。另一类是从没见过的场景：自动驾驶车见过大量路况，却可能很少见过冰雹里突然窜出的马；这时系统需要一种「自我觉察」——知道自己处在分布外，于是减速，而不是把马硬认成自行车。

数学上，他把多种不确定性都映射到概率，再用概率论规则去更新信念。区分类型之所以重要，是因为决策不同：对硬币正反这种纯随机，也许该放弃硬预测；对自己信念不足的情形，则该去收集信息。他补了一句计算机科学里的定义：1 bit 信息，就是把不确定性降到原来的一半。

认知科学这边，他用哥斯达黎加树叶沙沙作响「也许是美洲豹」的例子，说明感知是把感觉输入与先验信念结合——伦敦市中心通常不会有美洲豹。Kahneman 与 Tversky 的工作显示，人在有意识地报概率时往往很差；但在无意识的感知系统里，人又往往相当擅长，因为生存依赖它。婴儿不会写方程，却仍可用同一套形式系统去建模学习与决策。

Hannah 提醒：正确性与置信度常被一起塞进「不确定性」这个词里，其实要拆开。

Zoubin 举了十多年前就已广为人知的对抗样本：一张校车图，人眼仍看成校车，只改几个难以察觉的像素后，神经网络可以 99% 自信地输出「猎豹」，也可以改成任意其他类别。结论是：只追求答对不够，还要系统在错的时候别那么自信。

推到大模型，他的观察更日常：不论哪个模型，常常先自信作答；你一追问，它又可能改口。过度自信的系统难信任，过度自信的人也一样。他希望智能系统对自己的知识边界有感觉，并把这一点「烘焙」进系统。

谈到 AGI，他把谦逊与过度自信直接写成部署问题：知识可以堆很多，部署时若没有限度感，用户很难托付。

历史段是理解他立场的钥匙。

他强调：今天的大模型本身就是概率模型——预测下一 token 的概率。缺的不是「完全没用概率」，而是他所说的谨慎、显式的概率表示：信念概率散落在数十亿单元的激活里，你很难指出「模型认为某事为真的概率是多少」。系统更像在希望「数据够多就会自己学会表示概率」，而不是显式地算。

他对 Bayes' rule 的讲解用了侦探故事：先验里管家可疑；在食品储藏室发现凶器后，用似然更新，再归一化得到后验；后验变成下一轮先验，继续吸收证据。信息增益用 bit 计量。

同一条规则既可建模感知，也可建模学习：模型参数一开始不确定，数据来了就顺序更新。他希望 AI 像《星际迷航》里的 Data 那样理性，不要无证据地来回翻烧饼；理想情况下，系统在表示与操纵概率上甚至应强过人——就像计算器乘法应强过人。

对今天的大模型，他的判断很硬：在进步，但仍然不好。你问「你有多确信」，它会用下一 token 预测答你一句，却没有在显式计算 Bayes' rule；它可能在模仿网上大量推理痕迹，所以「有点像在装」。一推就道歉改口，说明内部信念并不连贯。幻觉在他看来是症状：有时创作故事需要「幻觉」，事实问题则需要 grounding；网上信息还常互相矛盾，系统最好能为任何陈述给出信念估计——「我们还没有」。

谈到 semantic entropy（语义熵，节目称由其学生提出的思路之一）：在产出 token 前，模型对下一 token 已有分布；分布熵低则尖、自信，熵高则散、不确定。用埃菲尔铁塔例子：巴黎应是大尖峰，纽约或拉斯维加斯的仿制品可有小质量。但 Zoubin 立刻泼冷水：若只靠数据共现来「读」不确定性，仍像用加减乘除例题去「装」一台计算器——没见过的数字可能就不会；他要的是真正会算、真正对世界推理的系统。

难在哪？计算难。他估计「理想理性系统」的配方 15–20 年前就大体齐了，但在所有可能上维护概率分布在计算上难以承受，于是领域转向大力训练数据。他说自己有一些重访想法，但「还没成形」。

离开纯语言模型后，Hannah 把话题转到已经更贝叶斯的前沿系统。

天气：DeepMind 的 GenCast 可做最长约 15 天的预报，节目中称约 8 分钟完成（相对传统超算上数小时的集合预报）。关键不只是神经网络，而是扩散模型在时间上操纵概率分布，并生成预报集合（ensemble）：对飓风路径给出一整团可能轨迹，新观测进来后再更新集合——这就是贝叶斯更新在工程上的样子。不确定性既来自混沌（蝴蝶效应），也来自传感器有限与模型假设可能错。官方博文与节目口径一致：GenCast 是面向地球球面几何的扩散模型，生成未来天气情景的概率分布，可并行生成集合成员，单条 15 天预报在单块 Cloud TPU v5 上约需 8 分钟。6

节目里提到用该模型追踪 Hurricane Melissa 一类热带风暴；这是嘉宾举例，摘要不把单次业务结果写成已独立核验的气象结论。

AlphaFold：结构预测用颜色标出模型有多确信。蛋白质从序列到折叠本身不确定：有的部位会晃；预测也不是实验数据。需要把分子位置的「预报云」表达出来。

Hannah 担心另一个极端：系统诚实到永远只说「我不知道」。Zoubin 的平衡尺是校准（calibration）：若说降雨概率 70%，那么所有你报 70% 的日子里，长期应约有 70% 真下了雨。对「人类何时首次登陆火星」这类一次性事件，贝叶斯仍允许用概率表达信念程度——不是只能对可重复事件报 50%。

沟通上，他举医疗 AI：医生看症状与检验时，系统必须把不确定性以可见形式交给人，因为这会改治疗方案。剑桥同事 David Spiegelhalter 长期研究如何把概率优雅传达给公众。他要的是与人互补、对「知与不知」诚实的系统，而不是让人依赖过度自信的黑箱。

Hannah 点出领域里的反对意见：有人认为数据堆够了就不必再操心不确定性。Zoubin 承认「他们也不全错」——模型已经很有用；但一拉到长尾，缺口就露出来。聊天机器人决策后果有限；自动驾驶与医疗诊断则必须把概率做对。

谈到 AGI，他把阵营简化成两派：一派只要更多数据与算力；一派认为还要新架构。他站第二派，同时承认第一派已带来巨大进展。他点名几块仍缺研究：

Hannah 半开玩笑：这听起来像发现了「魔法」——一行 Bayes' rule 同时带来谦逊、持续学习与数据效率。Zoubin 接话说「太好以至于不真实」：教科书早会写理想做法，但精确求解往往是 NP-hard，只能近似。现代系统可看成对理想的近似；算力已今非昔比——他本科用的 Connection Machine 甚至慢过口袋里的 Pixel 手机——再加上几十年近似方法，有机会把两边的优点合起来。

收尾句被主持人当作点题：对真正要紧的问题，他宁可要一个知道自己何时不知道的系统，也不要一个傲慢且过度自信的系统。

这期最值得带走的结构，不是一句口号，而是 Zoubin 反复使用的同一把尺：决策是否在不确定下发生、置信度是否与正确性分开、概率是否显式可更新、落地场景的后果是否足够重。尺的一边，他承认缩放已经交出巨大用处；另一边，他拒绝把「数据汤」里的下一 token 概率，直接当成系统对自己信念的诚实报告。

它提供的是一位长期做概率机器学习的研究者，如何用自动驾驶、天气、蛋白折叠和 AGI 争论，把自己的研究纲领讲成工程问题。读者可以用它决定：要不要再花完整 45 分钟听原片，并对照 GenCast 博文与 2015 年 Nature 综述，自己把主张钉到可核对材料上。7

以下为本期约 45 分钟访谈的完整逐字稿。英文来自官方 YouTube 字幕（保留字幕中的专名拼写原样）；中文为对照译文。按讲话人 Hannah Fry 与 Zoubin Ghahramani 分段，时间轴约每 10 分钟标注一次。官方字幕本身不带姓名；姓名分段依据节目主持/嘉宾身份与对话结构重建，交接处偶有粘连。时间轴按全文词量线性映射到片长，并对照官方章节锚点。2

Welcome to Google DeepMind, the podcast. Now, if you ask an AI a question, it will usually give you an absolute answer with unwavering authority, even if that answer turns out to be wrong. In fact, today's AI seems to be missing a fundamental human trait self-doubt. But long before the current wave of large language models, one academic researcher was trying to give machines a sense of their own limitations. Zoubin Ghahramani has spent the last 30 years pioneering a type of intelligence built on the mathematics of uncertainty. Today, as a professor at Cambridge and co-lead of frontier AI at Google DeepMind, zoubin finds himself at the heart of another interesting debate. On the one side are those who are hoping that Pure Scale will be the answer to ever improving AI, and on the other are those like zoubin, who believe that true intelligence requires innovations in architecture, and that improving machine uncertainty may be one of the missing pieces. Zubin, welcome to the podcast.

欢迎收听 Google DeepMind 播客。如果你向 AI 提问，它通常会带着不容置疑的权威给出一个斩钉截铁的答案，哪怕这个答案最终是错的。事实上，今天的 AI 似乎缺少人类一种基本特质：自我怀疑。但早在当前这波大语言模型兴起之前，就有一位学者在努力让机器拥有对自己局限的感知。Zoubin Ghahramani 过去 30 年一直在开拓一种建立在不确定性数学之上的智能。如今，作为剑桥大学教授、Google DeepMind 前沿 AI 联合负责人，他正处在另一场有意思的争论中心：一边希望纯靠规模扩展就能不断改进 AI；另一边像 Zoubin 这样的人则认为，真正的智能还需要架构上的创新，而改进机器的不确定性可能是缺失的一块。Zubin，欢迎来到节目。

Thank you.

谢谢。

If you were to distill it all down, I mean, your central thesis is that we need to have uncertainty in. I mean, give me the top line of it.

如果要把这一切浓缩下来——我是说，你的中心论点是我们需要把不确定性放进来。给我一句总括。

So yeah. Well if you think about intelligence, one of the most important parts of intelligence is decision making. Can't have an intelligent system that doesn't make decisions, from bacteria to animals to humans to robots decision making is really important. And if you want to make decisions in the real world, our perception is limited. So we are always uncertain about the state of the real world. And we need to make decisions under uncertainty. We can't know everything. We don't know everything from our senses. We can't predict the future. And so fundamentally, to build an intelligent system, you need a system that can represent uncertainty, that can update its uncertainty and then can use that to make good decisions under uncertainty.

好。如果你想智能，智能最重要的部分之一就是决策。不可能有一个不做决策的智能系统：从细菌到动物、人类到机器人，决策都很重要。而如果你要在真实世界里做决策，我们的感知是有限的。所以我们总是对真实世界的状态不确定，需要在不确定下做决定。我们不可能知道一切。我们无法从感官获知一切。我们也无法预测未来。因此，从根本上说，要建造智能系统，你需要一个能够表示不确定性、能够更新不确定性，并据此在不确定下做出好决策的系统。

Because actually, I mean, there's two different types of uncertainty, I guess, right. There's the uncertainty of just the inherent randomness of the world.

因为其实，不确定性大概有两种，对吧。一种是世界固有的随机性带来的不确定性。

Yeah there's a pedestrian in a normal, typical street, and you just don't know which way they're going to. They're going to turn.

对，比如普通街上有个行人，你就是不知道他会往哪边走。他会转向哪边。

But then there's the uncertainty of a scenario that you've never encountered before.

但还有一种是你从未遇到过的情境带来的不确定性。

Let me give an example from something that is becoming more and more of a reality in all our lives, which is self-driving cars. So when you're in a self-driving car, the self-driving car has been trained on lots and lots of data. It's seen many, many scenarios. But you can imagine that there is what's called the long tail of things that could happen. For example, the car may have not been trained in many instances of hailstorms, and it may not have been trained with horses suddenly jumping in front of the car in a hailstorm. And so essentially, what you really want from an intelligent system is a certain self-awareness. If we can use those terms, a self-awareness about its uncertainty. So it needs to be able to know the situation that it's in is something that is unusual or it hasn't seen before. And in the case of the self-driving car, for example, if it were to have a sense of its uncertainty, it would basically decide to slow down because it hasn't encountered that situation before. It's not confident that the horse isn't a bicycle or whatever it is. There are many different kinds of uncertainty, but the beauty of it is that from a mathematical point of view, we can boil it all down to probabilities. So we can map all these different forms of uncertainty onto probabilities, and then use the rules of probability theory to manipulate uncertainty, update your state of uncertainty, et cetera.

我举一个在我们生活中越来越真实的例子：自动驾驶汽车。当你坐在自动驾驶车里，这辆车已经在大量数据上训练过，见过很多很多场景。但你可以想象，还有所谓的「长尾」事件可能发生。比如，车可能很少在冰雹天气里训练过，也可能没训练过冰雹里突然有马跳到车前。所以，你真正希望智能系统具备的，是某种自我觉察——如果我们可以用这个词的话——对自己不确定性的自我觉察。它需要能够知道：自己所处的情境是异常的，或是以前没见过的。以自动驾驶为例，如果它能感知自己的不确定性，它基本上会决定减速，因为它没遇到过这种情况，也不确信那匹马不是一辆自行车或别的什么。不确定性有很多种，但美妙之处在于：从数学角度看，我们都能把它归结为概率。我们可以把这些不同形式的不确定性映射到概率上，再用概率论的规则去操纵不确定性、更新你的不确定状态，等等。

But how important is it that a machine can tell the different types of uncertainty apart.

那机器能不能区分不同类型的不确定性，这件事有多重要？

Yeah, I think it's important insofar that the different types of uncertainty may mean different decisions. So for example, if you have what's called aleatoric uncertainty, which is the randomness of a coin flip, which is the pedestrian going to turn or which ways the pedestrian going to turn might want to decide that you're going to give up on trying to predict because it is just random, whereas in other cases, your state of belief is uncertain and you would want to collect more information to. And in fact, that's the definition of information. So information a bit of information that we use in computer science is the reduction of your uncertainty by a factor of 2. That's what a bit is. And so collecting information is the way we reduce our uncertainty.

我认为重要之处在于：不同类型的不确定性可能意味着不同的决策。比如，如果你有所谓的偶然不确定性（aleatoric uncertainty），也就是抛硬币那种随机性——行人往哪边走——你可能决定放弃去预测，因为它就是随机的；而在另一些情形里，你的信念状态不确定，你就会想去收集更多信息。事实上，这就是信息的定义。我们在计算机科学里用的一比特信息，就是把你的不确定性降到原来的二分之一。那就是一比特。所以收集信息，是我们降低不确定性的方式。

How did these ideas of uncertainty map onto what humans are doing.

这些关于不确定性的想法，怎样映射到人类在做的事情上？

So I actually studied cognitive science. I studied computer science and cognitive science as an undergraduate. So I've always been interested in human intelligence as well as machine intelligence. And one of the really interesting things is that the field of cognitive science has really embraced these ideas of uncertainty and probabilities and so on, to try to understand both human perception and human decision making. So we go through our lives perceiving things. Perceiving the world is fundamentally an act of sensing something that is uncertain. I can't see the back of your head, but I can infer what the back of your head might look like from you. Hope it was there. I would hope it's there. And similarly, let's say I'm hiking in Costa Rica and suddenly there's the rustling of the leaves. It may be a Jaguar. And so my sense is through evolution, my senses have developed to take in perceptual information, taken my prior beliefs, because there are Jaguars in Costa Rica, or there wouldn't be Jaguars in the middle of London. And that process of combining information has been modeled by cognitive scientists and psychologists and neuroscientists through the language of probabilistic inference, basically. And also actually one of the interesting things is that we as humans are actually quite bad at estimating probabilities explicitly. If you ask somebody what is the probability of a certain event, they might get it wrong by orders of magnitude, for example. And, we have these fallacies of probabilistic inference and belief. Kahneman and Tversky showed that humans are actually quite bad at representations of uncertainty at a conscious level. But in our perceptual systems, unconsciously, we tend to be quite good about these things because our survival depends on them.

我本科其实学过认知科学，计算机科学和认知科学都学了。所以我一直对人类智能和机器智能都感兴趣。很有意思的一点是，认知科学领域真的拥抱了不确定性、概率这类想法，去理解人类感知与人类决策。我们一生都在感知事物。感知世界，本质上是在感知某种不确定的东西。我看不见你的后脑勺，但我可以从你推断后脑勺大概长什么样。希望它还在那儿。我希望它在。同样，假设我在哥斯达黎加徒步，突然树叶沙沙作响，那可能是美洲豹。于是通过进化，我的感官发展出了这样的能力：接收感知信息，并结合我的先验信念——因为哥斯达黎加有美洲豹，而伦敦市中心不会有。这种信息结合过程，已被认知科学家、心理学家和神经科学家用概率推断的语言来建模。还有一点很有意思：我们人类其实很不擅长显式地估计概率。如果你问某人某事件的概率，他们可能差上几个数量级。我们还有各种概率推断与信念上的谬误。Kahneman 和 Tversky 表明，人在意识层面的不确定性表征其实相当差。但在我们的感知系统里，在无意识层面，我们往往相当擅长这些事，因为生存依赖它们。

I'm thinking also about babies here. Toddlers and the way that they learn is a lot about their belief in what action will drive a particular outcome.

我也在想婴儿。幼儿学习的方式，很大程度上是关于他们相信什么行动会带来什么结果。

Yeah I mean, it's all implicit. Obviously babies don't know what probabilities are and they won't be able to write down any equations for you. But there are many cognitive scientists and psychologists who try to understand human learning, human perception, human decision making, using these same formalisms that we're using for AI systems.

对，这全是隐含的。婴儿显然不知道概率是什么，也写不出任何方程。但有很多认知科学家和心理学家，试图用我们给 AI 系统用的同一套形式系统，去理解人类学习、感知与决策。

I think there are a few things here we should probably tease apart before we get really into it, because there's a difference between correctness and confidence, which both sometimes come under the umbrella of uncertainty.

在真正深入之前，我觉得有几件事该先拆开，因为正确性与置信度是有差别的，而两者有时都被笼统放在「不确定性」这个伞下。

Yeah, absolutely. A great example of this is we use AI systems for image classification all the time. So you give it an image and it gives you an answer of what's in the image. And we can measure correctness, but we also want it to tell us how confident it is. And more than a decade ago, people discovered that you can take an image, for example, of a school bus, modify just a few pixels in that image in an imperceptible way, so a human being would look at it and say, well, that's an image of a school bus. You give it to the neural network and it confidently says, that's a cheetah. 99% that's a cheetah. And in fact, you could do it for any category. You could turn the school bus into a monkey or whatever in the eyes of the neural network. And so that sort of adversarial example shows us that it's not correctness that we care about alone. It's actually correctness and confidence. We don't want systems that can be overconfidently wrong. Yeah, or can be fooled, I guess. Or can be fooled. And I think our systems can still be fooled in that way.

对，完全同意。很好的例子是我们一直在用 AI 做图像分类：你给它一张图，它告诉你图里是什么。我们可以衡量正确性，但也希望它告诉我们有多确信。十多年前人们发现，你可以拿一张校车图，以人眼几乎察觉不到的方式改几个像素，人看还是校车；交给神经网络，它却自信地说：那是猎豹，99% 是猎豹。而且你可以改成任意类别，在神经网络眼里把校车变成猴子或其他什么。这类对抗样本告诉我们：我们关心的不只是正确性，而是正确性加上置信度。我们不想要那种错得还特别自信的系统。对，或者说容易被骗。或者会被骗。我认为我们的系统现在仍可能被那样骗。

These ideas of overconfidence and humility, are these going to end up being important as we start to build AGI.

过度自信与谦逊这些想法，在我们开始建造 AGI 时，会不会变得很重要？

Yeah, absolutely. I think we've advanced a great deal with systems that can acquire a lot of knowledge, but when we deploy them, for example, large language models, we've all encountered this. No matter what large language model you interact with, it will confidently give you an answer and then you question it. And maybe it will then flip-flop and give you a different answer. And so it's hard to trust the system that is overconfident. It's also hard to trust people that are overconfident. I think we would all intelligent systems to have a sense of their limits, their knowledge, and we need to bake that into our AI systems for sure.

绝对会。我们已经在能获取大量知识的系统上前进了很多，但部署时——比如大语言模型——我们都遇到过：无论你交互的是哪个大语言模型，它都会自信地给你一个答案；然后你一追问，它或许又会改口给出另一个答案。所以很难信任过度自信的系统。也很难信任过度自信的人。我认为我们都希望智能系统对自己的限度、对自己的知识有感觉，我们需要把这一点烘焙进 AI 系统。

Absolutely I know that you've been working in AI for a really long time, right. I mean, since the late 80s, basically. Yeah what was the field like then.

当然。我知道你做 AI 已经很久了，对吧。基本上从 80 年代末开始。当时这个领域是什么样的？

So I'm pretty boring in that I was interested in AI as a teenager. So when I was 14 or 15, I already wanted to work in AI. When I went to University, I needed a summer job, so I went to the head of the computer science department, this very famous computational linguist, Arvind Joshi at University of Pennsylvania. And I said, I'm here for the summer. I'm a first year student. I need a summer job. And what he said was that these two books that have come out, these were called parallel distributed processing. This was in 1986, the same year that the back propagation paper had come out that launched the whole neural network, kind of modern neural network revolution. And he gave me a summer job reading these books and explaining them to him, which was most wonderful summer job one could possibly have. If you're interested and curious about things. So I learned about neural networks back then. And you asked me what was it like in the mid to late 1980s. Well, the dominant paradigm in AI was expert systems. So people were looking at Rule based systems that would make decisions, and they were quite brittle. And neural networks, which were sort of modeled after the human brain, were actually much more flexible. And fundamentally, they could learn from data in a way that previous methods were not very good at learning from data. So that was really a revolution back in the 1980s. We talk about the transformer revolution and all that. But back in the mid 1980s, that was a real revolution. And it attracted people from cognitive science, from computer science, psychology, neuroscience, economics. And then I went on to write an undergraduate thesis on learning how to parse human natural language using neural networks. So it was one of the very earliest I didn't publish it, but if I'd published it, it would have been a very early paper on small language. Let's call them small language models, because my models were very small.

说起来挺无趣的：我十几岁就对 AI 感兴趣了。十四五岁时就已经想做 AI。上大学时我需要暑期工，就去找计算机系主任——宾夕法尼亚大学那位非常有名的计算语言学家 Aravind Joshi。我说，我暑假在这儿，大一新生，需要一份暑期工作。他说，有两本书刚出来，叫 Parallel Distributed Processing。那是 1986 年，也就是反向传播论文发表、启动整场现代神经网络革命的同一年。他给我的暑期工作就是读这些书，再讲给他听——如果你对事物好奇，这大概是能有的最棒的暑期工作了。于是我那时候就学了神经网络。你问我 1980 年代中后期是什么样：当时 AI 的主导范式是专家系统，人们在看会做决策的基于规则的系统，它们相当脆弱。而某种程度上模仿人脑的神经网络，其实灵活得多，而且从根本上能从数据中学习，这是此前方法不太擅长的。所以那在 1980 年代真是一场革命。我们现在谈 transformer 革命之类，但 1980 年代中期那次也是真正的革命，吸引了认知科学、计算机科学、心理学、神经科学、经济学的人。后来我写了本科论文，用神经网络学习解析人类自然语言。那是非常早期的工作之一——我没发表；如果发表了，大概会是很早的一篇关于「小语言」的论文。我们叫它们小语言模型吧，因为我的模型非常小。

Wait, what year was this.

等等，这是哪一年？

This is 1989.

1989 年。

And you're very much ahead of the game.

那你确实远远走在前面。

Yeah and we had and we actually had a parallel computer. So now people think about data centers and all sorts of GPUs and parallel computers. And we had the most sci-fi, beautiful, iconic parallel computer of the time, which was this connection machine, which was this cube with 65,000 little red lights blinking inside this kind of opaque cube because it had 65,000 processors. So I would sit there coding up in this parallel language, neural networks for natural language. And it was pretty revolutionary at the time because neural networks were at the counterculture. People were very enamored of the old school of AI. But then, OK, despite being ahead of the game by approximately, what, 40 years. Yeah then I messed it up. Yeah, well.

对，而且我们当时真的有一台并行计算机。现在人们想的是数据中心、各种 GPU 和并行计算机。我们当时有那台最科幻、最漂亮、最具标志性的并行机——Connection Machine：一个不透明立方体，里面有 6.5 万个闪烁的小红灯，因为它有 6.5 万个处理器。我就坐在那儿，用并行语言写自然语言的神经网络。那在当时相当革命，因为神经网络还属于反主流文化，人们很迷恋旧派 AI。然后——好吧，尽管大概领先了 40 年——接着我就搞砸了。嗯，好吧。

Well, what happened.

后来发生了什么？

No, no, what happened is actually really fascinating. So I was working in neural networks, and at the time, this was now the early 1990s. We felt like we understood how they worked. So neural networks are these amazing function approximators. We can feed them data. They can map from inputs to outputs, from x to y, from images to labels of images and so on. And remember, the data sets were very small at the time. So when I was writing my undergraduate thesis, the world wide web didn't even exist. Compute was also very limited, so we felt like we understood neural networks. We felt like their nice function approximators. But people started to uncover these beautiful relationships between neural networks and other ideas from probability theory, statistics, stochastic processes.

不，后来发生的事其实很迷人。我当时在做神经网络，那是 1990 年代初。我们觉得已经理解它们怎么工作了。神经网络是了不起的函数逼近器：喂数据，就能从输入映射到输出，从 x 到 y，从图像到图像标签，等等。记住，当时数据集非常小。我写本科论文时，万维网甚至还不存在。算力也很有限，所以我们觉得已经理解神经网络了，觉得它们就是很好的函数逼近器。但人们开始揭示神经网络与概率论、统计学、随机过程等其他想法之间那些漂亮的关系。

I mean, you actually gave up working on neural networks in favor of these probability questions. Yeah, absolutely. Was it annoying though. Yeah In the end, neural networks were the thing. And you were there so much earlier than anyone else.

我是说，你实际上放弃了神经网络，转向这些概率问题。对，完全是。那会不会觉得懊恼？因为到头来神经网络才是大热的东西，而你比几乎所有人都早那么多。

Yeah, well, I didn't. I did redeem myself. I spent a number of years working with Geoff Hinton. Of course, he went on to win the Nobel Prize for his work on neural networks. So we were attempting to do deep learning, but we were using probabilistic models rather than simple neural networks. We were overcomplicating things because we hadn't seen what happens with large scale data. And then I went off and worked on Bayesian machine learning and probabilistic models and all these things for many years. But I never totally dismissed neural networks, actually, because I knew that they work. It's just that they weren't from a research point of view, they weren't as interesting to me because the mathematics was at the time, we felt well understood.

嗯，我没有一直放弃。我后来扳回一城。我和 Geoff Hinton 共事过好些年。当然，他后来因神经网络工作拿了诺贝尔奖。我们当时试图做深度学习，但用的是概率模型，而不是简单的神经网络。我们把事情弄复杂了，因为还没见过大规模数据会带来什么。然后我离开去做贝叶斯机器学习、概率模型这些，做了很多年。但我从未完全否定神经网络，因为我知道它们有效。只是从研究角度看，它们当时对我没那么有意思，因为数学上我们觉得已经被理解了。

In 2015, though wrote this seminal nature paper, which appeared in the same issue as all of the foundational deep learning and reinforcement learning papers by Hinton. And in that paper you argue many aspects of intelligence depend crucially on the careful probabilistic representation of uncertainty. Yes I mean, has everyone heard you have they needed your Warning.

不过 2015 年，你写了那篇开创性的 Nature 论文，和 Hinton 那些奠基性的深度学习、强化学习论文发在同一期。你在文中主张：智能的许多方面，关键取决于对不确定性的谨慎概率表示。那么——大家都听到你的警告了吗？他们需要你的警告吗？

No, no, I think there are many people who understand that and believe that. I think that it depends on the level that you think about things. So actually, if you look at large language models, they are probabilistic models. They predict the probability of the next token or word given a sequence of previous tokens. So probabilities are at the heart of everything we do in machine learning. But what's missing is we're not really doing what I said, which is the careful representation of probabilities. We're actually hoping that the models represent probabilities, OK, because we've trained them on enough data, but not thinking about it in an explicit way. Yeah if you look in a giant neural network, you can't really find the explicit representation say, the probability that it thinks something or other. It's sort of spread out somehow over all the activations of billions of units in the neural network. And by contrast, I mean, you're more of AI guess, a Bayesian thinker.

不，我觉得很多人理解并相信这一点。这取决于你在哪个层面思考。其实，如果你看大语言模型，它们就是概率模型：在给定先前 token 序列时，预测下一个 token 或词的概率。所以概率就在我们机器学习所做一切的核心。但缺失的是，我们并没有真正在做我说的那种对概率的谨慎表示。我们其实是在希望：因为数据够多，模型会自己表示概率，而不是以显式方式去思考它。对，如果你盯着一个巨大的神经网络，你很难找到显式表示——比如它认为某事为真的概率——它好像以某种方式分散在网络里数十亿单元的激活上。相比之下，你更像是——我猜——一个贝叶斯思考者。

Yeah just explain for anybody who hasn't come across this before, just explain to us what that actually means.

那就给还没接触过的人解释一下：这到底是什么意思。

Yeah so Bayes' rule is this fascinating and very simple concept from probability theory. So before you observe something, before you get some evidence or data, you have what's called prior beliefs. You represent those with a probability distribution. So for example, think of a detective story like a whodunit. There's a number of suspects and you may have some prior beliefs about it's the Butler that did it or whatever.

Bayes' rule 是概率论里一个迷人又非常简单的概念。在你观察到什么、得到证据或数据之前，你有所谓的先验信念，用概率分布来表示。比如想一个侦探故事、谁是凶手：有若干嫌疑人，你可能有些先验——觉得是管家干的，或别的什么。

So he's looking suspicious.

所以他看起来可疑。

Yeah somebody looking suspicious or something. So you have some beliefs. You represent those with a probability distribution. And there are many, many reasons why probability theory is the right way of representing beliefs. There's whole, branches of mathematics that have proven that. And now you observe some evidence like the murder weapon is found in the pantry or something like that. And so you take your prior beliefs, multiply them by what's called the likelihood, the probability under each possible, culprit. And then you renormalize because probabilities have to sum to 1. And from that you get your posterior beliefs, your new state of knowledge. And through that evidence, by the way, you've gained information literally measured in bits, how much your uncertainty has decreased. And now if you get more evidence, you just take your current posterior probability distribution, which is now your new prior, and you repeat and rinse. You do it again. You get the new evidence, you update the probabilities, and so on and so forth. And through that application of Bayes' rule, we can model both perception. I open my eyes, I see something that I see more things, and I know it's not a Jaguar that's following me in Costa Rica, but you can also model what learning is. So learning is you have a model. The model has parameters. At the beginning you don't know what the parameters should be. You get some data and you update the model parameters sequentially through that data. Applying Bayes' rule. In theory, that's the beautiful model that I had been pushing forward as a model of learning because I guess I mean, on the one hand, this is a very elegant mathematical way to link together evidence and unknowns and knowns.

对，有人看起来可疑之类的。于是你有一些信念，用概率分布表示。有非常多理由说明概率论是表示信念的正确方式，数学里有整支分支证明过这一点。然后你观察到一些证据，比如凶器在食品储藏室被发现。于是你拿先验信念，乘以所谓的似然——在每个可能凶手假设下的概率——再归一化，因为概率必须加总为 1。由此得到后验信念，也就是你新的知识状态。通过这些证据，你获得了信息——字面意义上用比特计量——你的不确定性下降了多少。若再有更多证据，你就把当前后验当作新的先验，再重复一遍：拿到新证据，更新概率，如此往复。通过应用 Bayes' rule，我们可以建模感知：我睁开眼，看到一些东西，看到更多东西，知道在哥斯达黎加跟着我的不是美洲豹；也可以建模学习是什么。学习就是：你有一个模型，模型有参数，一开始不知道参数该是什么；你得到数据，就按数据顺序更新模型参数，应用 Bayes' rule。理论上，这就是我一直在推动的漂亮学习模型——因为一方面，这是一种非常优雅的数学方式，把证据、未知与已知连在一起。

But I guess actually, on an intuitive level, I mean, this sort of is the way that our brains work. I mean, going back to your example of the detective it's like, oh, I did think it was that person. But now this new evidence has come in and I've changed my mind. I mean, that's essentially a one sentence description of what Bayes' rule is doing.

但直觉层面，这其实也像我们大脑的工作方式。回到你的侦探例子：哦，我本来以为是那个人，但新证据来了，我改主意了。本质上，这就是用一句话描述 Bayes' rule 在做什么。

Exactly, yeah.

正是如此。

But this is a formal way to get AI to be able to do it.

而这是一种让 AI 也能这样做的形式化方式。

Exactly so we would like to build AI systems that accumulate knowledge and information over time. And we would like them to be rational sort of data from Star Trek is a very rational being. We don't want them to flip-flop around unpredictably based on no evidence. And, I mean, I would argue that ideally we would like them to be, even more rational than humans. I want my I just like I want my calculator to be really good at multiplying large numbers. I would actually our AI systems to be more rational, better at representing and manipulating probabilities than humans are.

正是。我们希望建造能随时间积累知识与信息的 AI 系统。我们希望它们理性——有点像《星际迷航》里的 Data，非常理性。我们不想让它们在没有证据时不可预测地来回改口。而且我认为，理想情况下我们希望它们甚至比人更理性。就像我希望计算器特别擅长乘大数一样，我其实希望我们的 AI 系统更理性，比人更擅长表示和操纵概率。

If we fast forward to today, I mean, the paper you wrote was a decade ago. Yeah how good, really are the AI systems that we're all used to playing around with. How good are they at representing uncertainty.

快进到今天——你那篇论文是十年前写的。我们现在天天玩的这些 AI 系统，在表示不确定性这件事上，到底有多好？

Yeah, they've been getting better, but they're not very good. And you can tell that because if you interact with a large language model and it asserts something, you can ask it, how confident are you. And it might say something back, but it's really doing next token prediction. It doesn't have an explicit representation of how confident it is. It's not calculating Bayes' rule. It's not calculating Bayes' rule, at least not explicitly. It may be because you've trained it on trillions of tokens of stuff on the web. It's mimicked a lot of other kinds of reasoning traces and so on. It's sort of faking it. And you can tell it's faking it, because then if you push back and you say something silly like, no, I think you're wrong, then it might respond, oh, sorry, yes, I was wrong. So it's not really coherent. It's not going to stand its ground. And our systems are getting better at factual grounding and things like that. But we haven't really nailed the idea of how one of these models should be able to represent probabilities over its beliefs.

它们在变好，但还不够好。你可以看出来：如果你和大语言模型交互，它断言了某事，你问它有多确信，它也许会回你一句，但它真正在做的是下一 token 预测。它没有关于自己有多确信的显式表示。它没有在算 Bayes' rule。至少不是显式地在算。也许因为你在网上数万亿 token 上训练过它，它模仿了大量其他推理痕迹之类的，有点像在装。你能看出它在装，因为你一反击，说句傻话比如「不，我觉得你错了」，它可能回应「哦对不起，是我错了」。所以它并不真正连贯，也不会坚持立场。我们的系统在事实 grounding 等方面在变好，但我们还没真正解决：这类模型该如何表示对自己信念的概率。

Because that, I mean, I feel like that would be a very useful feature for a large language model to be able to say, it's not sure why do they struggle with that so much.

因为我觉得那会是大语言模型非常有用的功能——能说出自己不确定。那它们为什么在这上面挣扎得这么厉害？

They struggle because the paradigm for training them hasn't prioritized that. We train them to be just really good at modeling the data. If the data involves a lot of human reasoning by many different humans with many different beliefs, then what you get is a soup. You get a mish mash of everything. But if we want to build, like I said, self-driving cars or robots, they need to be able to reason about the real world. They need to have an understanding of cause and effect. They need to have an understanding of their own uncertainty, and they need to use that to be able to act in the real world in a safe way.

它们挣扎，是因为训练范式没有把这件事放在优先位置。我们训练它们只是为了特别擅长建模数据。如果数据里包含许多不同人类、带着许多不同信念的推理，那你得到的就是一锅汤，一堆大杂烩。但如果我们想建造我说过的自动驾驶或机器人，它们需要能对真实世界推理，需要理解因果，需要理解自己的不确定性，并据此在真实世界里安全地行动。

But then, I mean, I'm just thinking about hallucinations here because that's part of this as well, right. There are some times where you just want there's a verifiable fact that it's getting wrong. Yeah and that all plays into this too.

不过我在想幻觉，因为这也是其中一部分，对吧。有些时候你就是想要——它把一个可核验的事实搞错了。对，那也全扯进这件事里。

Yeah I mean, hallucinations are a symptom. Of course, sometimes we want our models to hallucinate. So there's a tension between not hallucinating at all and not being creative. For example, if I want my large language model to write me a short story of that time that Albert Einstein went to the moon in a rocket, that's clearly a hallucination, but it's an act of creative writing, so it should be able to do that. It should be able to infer that that's what you want, infer your intent. If I ask it a factual question, it should try to be grounded. And of course, at Google, we think a lot about grounding our models in what's available. And even information on the web is often contradictory. And so you want to hedge your bets. So actually what you would like is a system that's able to tell you for any statement, some estimate of its belief or probability. I think that's what we want. We don't have it yet.

幻觉是一种症状。当然，有时我们希望模型会「幻觉」。所以在「完全不幻觉」和「不能创作」之间有张力。比如，如果我想让大语言模型写个短故事：爱因斯坦坐火箭去月球——那显然是幻觉，但它是创作，所以它应该能做。它应该能推断那是你想要的，推断你的意图。如果我问事实问题，它就应力求 grounded。当然，在 Google，我们很多时候在想如何把模型 grounding 到可获得的信息上。而网上信息也常常互相矛盾，所以你要两边下注。实际上你想要的是：对任何陈述，系统都能告诉你某种信念或概率估计。我认为那是我们想要的。我们还没有。

Know how might you build it if you wanted large language models to have uncertainty in there. How do you do it. Because there are some quite good ideas. I mean, thinking about semantic entropy here, which is one of your students came up with. Tell me a little bit about that. How might that work.

如果你想让大语言模型里面有不确定性，你会怎么建造？怎么做？因为有一些相当不错的想法。我想到语义熵（semantic entropy），是你学生提出的思路之一。跟我讲讲那可能怎么工作。

Yeah, I mean, I think you can take the internals of a particular model and try to infer from that is the degree of belief. So before it answers, before it produces a token, in a large language model, you actually have a probability distribution over all possible next tokens. And that probability distribution, the entropy of that distribution tells you something about the uncertainty. A low entropy distribution is very spiky, is very certain. A high entropy distribution is very spread out. It's very uncertain. And so there are ways of teasing from the internals of a model how certain it might be.

我觉得你可以从某个模型的内部去推断信念程度。在它回答之前、产出一个 token 之前，大语言模型实际上对所有可能的下一 token 有一个概率分布。这个分布的熵会告诉你一些关于不确定性的事。低熵分布很尖、很确定；高熵分布很散、很不确定。所以有办法从模型内部挤出它可能有多确定。

Yeah I guess I'm thinking here about an example, let's say the Eiffel Tower in Paris, right. Those two things would be an example of something that's quite spiky.

我在想一个例子，比如说巴黎的埃菲尔铁塔，对吧。这两样会是相当「尖」的例子。

Yeah, yeah if you ask it, where is the Eiffel tower. It should have a high probability over Paris. Although I believe there's one in New York as well, isn't there. There's a little Eiffel Tower Vegas as well. I think they've got. Maybe there's one in Vegas as well. So basically, depending on the context, you might have little bits of entropy on these other Vegas and New York as options. But Paris would have a big spike on it.

对，如果你问埃菲尔铁塔在哪，它在巴黎上应有很高概率。虽然我相信纽约也有一座，对吧。拉斯维加斯也有个小埃菲尔铁塔。我想他们有。也许拉斯维加斯也有。所以基本上，取决于语境，你可能在拉斯维加斯、纽约这些选项上有一点点熵，但巴黎会有一个大尖峰。

Is that the idea then, that in the data set, Paris and the Eiffel Tower appear near each other a lot. You have a lot of really big signal there, whereas I don't the Eiffel Tower and Marrakesh.

那想法是不是：在数据集里，巴黎和埃菲尔铁塔经常一起出现，信号很强；而我没有——埃菲尔铁塔和马拉喀什。

Yeah, that might have very low probability. The problem with that is that's faking it in the sense that you're relying on the data. I'll give you the analogy of a calculator. It's trying to build a calculator just by showing it examples of addition and multiplication. But imagine you never show it a particular number. Then it might not generalize to that particular number. And you don't want to fake a calculator. You want a calculator that actually calculates. You want it to actually reason about the world that it's in. That's what we want from our AI systems.

对，那可能概率非常低。问题在于，那有点像在装——因为你依赖的是数据。我用计算器打个比方：你试图只靠给它看加减乘除的例子来造一台计算器。但想象你从没给它看过某个数字，它可能就推不到那个数字。你不想要一台装出来的计算器。你想要一台真正会算的计算器。你想要它真正对所处的世界做推理。那才是我们希望 AI 系统做到的。

So why hasn't this been done yet. I mean, what is it that makes it so hard to do.

那为什么还没做成？是什么让这件事这么难？

It is genuinely hard because it's computationally hard. So essentially, I would argue we had all the ingredients of AI maybe 15 or 20 years ago. We kind of know how to build rational systems. We just thought like, well, representing probability distributions over every possible thing is computationally intractable. It would take giant supercomputers that we don't have and it would be incredibly slow. You'd be waiting for millions of years before you get the answer. So people abandoned that for let's go with just training from data. And I think we can revisit this idea. And I have ideas for how to do this, and I'm exploring them now. But it's not super well formed yet.

确实很难，因为计算很难。本质上，我会说大约 15 到 20 年前我们就有了 AI 的全部配料。我们大概知道怎么建造理性系统。只是我们觉得：在所有可能事物上表示概率分布，在计算上难以承受。那需要我们没有的巨型超算，而且会慢得离谱——你可能要等上百万年才得到答案。于是人们放弃了那条路，改成「就从数据训练吧」。我认为我们可以重访这个想法。我有一些做法上的点子，现在也在探索，但还没有成形得很清楚。

OK So moving away from the large language model, the transformer type system for a moment because I mean, there are other examples of really cutting edge artificial intelligence, which does handle uncertainty in more of this Bayesian way that you're describing. I'm thinking about weather forecasting here.

好，先离开大语言模型、transformer 这类系统一会儿。因为还有其他真正前沿的人工智能例子，是以你描述的更贝叶斯的方式处理不确定性的。我想到的是天气预报。

Yes I mean, that's really important in weather forecasting.

对，这在天气预报里非常重要。

Tell me a bit about that.

跟我讲讲。

Yeah so we've developed a whole series of state of the art weather forecasting models at Google DeepMind. And if you look at the GenCast model, very recent model, one of the key features it has, it can predict weather over 15 days. And it can do it very fast, much faster. It can do it in eight minutes rather than on a giant supercomputer for hours. And it's obviously using neural networks and things like that, but a key ingredient for getting this to work is that it. It uses a diffusion model so that already is the image generation models that already is manipulating probability distributions over time. But then it generates an ensemble of forecasts. So if you're trying to track a tropical storm like the hurricane, Hurricane Melissa, let's say which we did with this model, you have the data so far. And then you want to be able to forecast the track of this into the future, because your decisions depend on that, whether you evacuate a city or whether you call in emergency services and so on. And so it represents that with an ensemble of forecasts, it has a whole probability distribution over the possible tracks. You rerun the model over and over and over and then as you get more data. So a few hours later, you get more observations, you update that ensemble. And that is essentially applying these basic ideas from Bayesian updating to this sort of problem because I mean, the original weather forecasting models that came up here did not have this. Yeah, that was added on. And every time we add on these features, it makes the model better because you're essentially saying there is an inherent uncertainty in the way that weather works. You can't just run the model once and be like, oh, well, that's to be the weather tomorrow. Yeah, you have to do it lots of times and then work out a probability. Yeah, yeah, it's a combination of both that inherent randomness, a classic butterfly effect in weather and that it's chaotic. It's very hard to predict. But also there is an uncertainty just because we have limited numbers of sensors. So it's a system that represents his beliefs about the weather, whether it's the trajectory of this hurricane and its intensity, and then those beliefs get updated as you take in the sensor measurements and as time progresses.

我们在 Google DeepMind 开发了一系列最先进的天气预报模型。看 GenCast 这个很新的模型，关键特征之一是它可以预测大约 15 天的天气，而且非常快——大约八分钟，而不是在巨型超算上跑好几个小时。它显然也用神经网络之类，但让它工作起来的关键配料是：它用扩散模型——那已经是在时间上操纵概率分布的图像生成那类模型——然后生成一个预报集合（ensemble）。如果你要追踪像飓风 Melissa 这样的热带风暴——我们用这个模型做过——你有迄今为止的数据，然后想预报未来路径，因为决策依赖它：要不要疏散城市、要不要调集应急服务等等。于是它用预报集合来表示，对可能路径有一整团概率分布。你一遍又一遍重跑模型；几个小时后有更多观测，你就更新那个集合。这本质上就是把贝叶斯更新的基本想法用到这类问题上。因为最初那边出来的天气预报模型并没有这个——对，那是后来加上的。每次加上这些特征，模型都会更好，因为你本质上在说：天气运作方式里有内在不确定性。你不能只跑一次模型就说「哦，明天天气就是这样」。对，你得跑很多次，再得出概率。对，它既是那种内在随机性——天气里经典的蝴蝶效应、混沌、很难预测——也是因为传感器数量有限带来的不确定性。所以这是一个表示自己对天气信念的系统——飓风轨迹与强度——这些信念随着传感器测量进入、随着时间推进而被更新。

I think there's something quite delightfully counterintuitive about that you add in uncertainty to the system. Yeah and it makes the predictions more accurate.

我觉得有点美妙地反直觉：你把不确定性加进系统，预测反而更准。

Exactly yeah. I think it is definitely a key insight in I mean, it's not that you're making the system noisy in an arbitrary way that's not going to help you. But what you're doing is you're being honest about the fact that know your sensors are inaccurate. Sometimes you get faulty, faulty sensors, just like sometimes my ears are blocked and I can't hear very well. And sometimes your model assumptions are wrong. That is also a form of uncertainty. And so all of those different forms of uncertainty have to be represented somehow. Approximately we can't do it all exactly so that we can get better calibrated forecasts.

正是如此。我认为这绝对是关键洞见。不是说你任意把系统弄嘈杂——那样帮不上忙。你做的是诚实面对：传感器不准确，有时会坏，就像我有时耳朵堵住听不清；有时模型假设也是错的，那也是一种不确定性。所有这些不同形式的不确定性都必须以某种方式被表示——近似地，我们不可能全部精确做到——这样我们才能得到校准得更好的预报。

I think the other example that really manages to get this uncertainty idea is AlphaFold, where the protein prediction is color coded by how sure the model is that that's yeah, that's the correct folding.

另一个真正把不确定性做进去的例子是 AlphaFold：蛋白质预测用颜色标出模型有多确信那是正确的折叠。

Yeah, absolutely. And essentially you're fundamentally trying to solve an uncertain problem. You're going from a sequence to the folded structure. And the physics itself means that some parts of the protein are going to wiggle around more. So you don't know exactly where they are. And you also have just uncertainty because you've used the model to predict that it's not like experimental data. And so you need to be able to represent that uncertainty in the Cloud of forecasts of where the molecules are.

对，完全是。本质上你在解一个不确定的问题：从序列到折叠结构。物理本身意味着蛋白质有些部位会晃得更厉害，所以你不知道它们精确在哪。你也有不确定性，因为那是用模型预测的，不像实验数据。所以你需要能在分子位置的「预报云」里表示那种不确定性。

There is a danger here that you could go too far the other way. You could. You could end up with a model that was so honest about its uncertainty that it just sort of didn't ever really give you an answer. It just always said, I don't just hedging, hedging all the time. That would be pretty funny. But I think there's different degrees of I don't how do you get the balance right then.

这里有个危险：你可能走到另一个极端。你可能得到一个对自己不确定性诚实到从来不真正给答案的模型，永远只说「我不知道」，一直在两边下注。那会挺好笑。但「我不知道」也有不同程度——那平衡怎么拿捏？

Yeah, I think the balance is if there is a repeatable event in the real world, then you want to be calibrated in that. If I say the chance of rain is 0.7 or 70% then for all days that I've said that if I sum up over all those days, if I'm calibrated, then on 70% of those days, it actually rained on 30% It didn't rain. So that's the calibrated probability. Now, if you ask me a statement of uncertainty about something that is not a repeatable event. So for example, we may be uncertain about the first day a human being reaches Mars. So that's a date. It's either going to happen or it's not going to happen. And that date, when it happens, you'll be certain of it. That's the event that you can have probabilities over. You can have beliefs over. And it will resolve itself when it happens If it happens. And Bayes' also has a way of handling that where it's not just 50% It's not just yeah it doesn't. Yeah, absolutely. So what Bayesian statistics tells you is that it's perfectly valid. And in fact, the right thing to do to use probabilities to represent your degree of uncertainty about things that only happen once.

我认为平衡是：如果真实世界里有可重复事件，你就要校准。如果我说降雨概率是 0.7 或 70%，那么在所有我这样报过的日子里加总，如果我是校准的，那么大约 70% 的日子真的下了雨，30% 没下。那就是校准概率。如果你问我对一个不可重复事件的不确定性陈述——比如人类首次抵达火星的第一天——那是一个日期，要么发生要么不发生；到发生那天你就会确定。那是你可以对其持有概率、持有信念的事件，发生时会自我消解。Bayes' 也有处理方式，不只是 50%。对，完全是。贝叶斯统计告诉你：用概率表示你对只发生一次的事情的不确定程度，完全成立，而且事实上是正确做法。

How do you communicate that uncertainty in a way that actually means it adds value, rather than just has a human sort of nodding along, or taking cognitive shortcuts when the machine says it's really confident.

你怎样沟通这种不确定性，才能真正带来价值，而不是让人只是点点头，或在机器说自己很有把握时走认知捷径？

Yeah I think people have different reactions to AI systems. Some people are just very skeptical and will not believe anything the AI system tells it. Other people are going to end up being reliant. Let's say. Let's imagine a future where we've got not a very distant future, but imagine a future where we've got AI systems in a medical domain, for example, and you have doctors aided by an AI system looking at patients and symptoms and tests results and so on. It's a great example of why we need uncertainty. If the AI system says something, you really want it to convey its uncertainty, because that's literally what is going to determine your treatment plan or whether you take one decision or another. These could be life and death decisions. So it's absolutely essential that first of all, we don't become reliant on overconfident AI systems. But to be able to do that, we need our AI systems to be honest about their uncertainty and bring that uncertainty in a visible form to the human user so we can understand it. And I mean, my colleague at Cambridge, David spiegelhalter, has done a lot of amazing work on how to convey uncertainties and probabilities in elegant ways to the general public. And I think there are ways you can do that. You can visualize the answer and so on. So, I mean, I really believe that it's important to have AI systems that are doing complementary things that are additive to humans, that are helping people solve problems that we care about. And in order to do that want them to be honest about what they know and what they don't know.

我认为人们对 AI 系统反应不同。有些人非常怀疑，什么都不信；另一些人最终会依赖它。想象一个不那么遥远的未来：医疗领域有 AI 系统，医生在 AI 辅助下看病人、症状、检验结果等等。这是为什么我们需要不确定性的绝佳例子。如果 AI 说了什么，你真的希望它传达不确定性，因为那会直接决定治疗方案，决定你选 A 还是选 B。这些可能是生死决策。所以绝对关键的是：首先，我们不要依赖过度自信的 AI；而要做到这一点，需要 AI 对自己的不确定性诚实，并以可见形式把不确定性交给人类用户，让我们能理解。我剑桥的同事 David Spiegelhalter 在如何把不确定性与概率优雅传达给公众方面做了大量出色工作。我认为有办法做到，比如把答案可视化等等。我真心相信，重要的是让 AI 做与人类互补、对人类有增益的事，帮助人们解决我们在乎的问题；为此，希望它们对自己知道什么、不知道什么保持诚实。

It feels like this is really very critical that we get this right. Yeah it's such an important part of designing our collective future with AI.

感觉把这件事做对真的非常关键。它是我们设计与 AI 共处的集体未来里如此重要的一部分。

Yeah, I think it is really important. I mean, I don't want to take away from the fact that AI systems have been incredibly useful already. I think they're pretty good at some of these things. And we can do better. And they're open problems along the way. And that's one of the reasons we actually need more research, actually.

对，我认为确实很重要。我也不想抹杀：AI 系统已经非常有用了。它们在有些事情上相当不错。我们可以做得更好。一路上还有开放问题。这也是我们其实还需要更多研究的原因之一。

The thing is, I mean I'm sort of sitting here agreeing with you. You're sort of you're also a Bayesian thinker. So this is very much my philosophy. But not everybody does. No, no, not everyone agrees with you. I mean, there are some people who say, look, you just put in more data and then you don't need to worry about uncertainty because the model will know everything.

问题是，我坐在这儿基本同意你。你也是贝叶斯思考者，这很大程度上也是我的哲学。但不是所有人都这样。不，不是每个人都同意你。有些人会说：你只要塞进更多数据，就不必操心不确定性，因为模型会知道一切。

Yeah, I think that is a view that a lot of people have in the field, and they're not completely wrong. Just like I'm not completely right in that the models are actually pretty good at a lot of useful things. The problem is that when you stretch them in the long tail of unusual things, then you can uncover some gaps. And also, I think when the decisions if you're just interacting with a chatbot, the decisions may not be so consequential. But if we're trying to build self-driving cars that are reliable or medical AI systems that are helping us make diagnosis decisions, then we really do care about getting those probabilities right, because I guess when it comes to building AGI, to oversimplify it, two camps really, one which says you just need scale, you just need more data, more compute. Off you go. And then the other that's no, we actually need a new architecture to be able to do things better that we can't do at the moment.

对，我认为这是领域里很多人持有的观点，他们也不完全错。就像我也不完全对——模型在很多有用的事情上其实相当好。问题在于，当你把它们拉到异常事物的长尾上，就会露出一些缺口。还有，如果你只是和聊天机器人交互，决策可能没那么严重；但如果我们要造可靠的自动驾驶，或帮助诊断的医疗 AI，我们就真的在乎把那些概率做对。因为说到建造 AGI，过度简化一下，其实有两个阵营：一边说你只要规模，只要更多数据、更多算力，走你的；另一边说不，我们其实需要新架构，才能把现在做不好的事做得更好。

It sounds like you're very much in the second KMP.

听起来你非常站在第二阵营。

Yeah I think we've made a lot of progress in the first KMP. So of course, our systems are incredibly useful. They're used by billions of people every day. But that doesn't mean that we've run out of interesting things to discover. So I'll give you a few examples of things that I think are important areas of research. One example is continual learning. So the way we currently train our models and we means everybody in the field. We train a giant model and then we use it in products or we release it in various forms. And then a few months later, we train another giant model and so on. If you compare that to how humans and animals learn, we learn continuously. We're basically constantly getting a stream of data and constantly adapting our connections between our neurons and so on. And our AI systems are not really able to do that very well. They suffer from things like catastrophic forgetting and so on and so forth, which absolutely links back to the idea of Bayesian thinking that we were talking about earlier. Because the reason why humans, animals are able to do that is because we have this updating system in our minds of incorporating evidence with existing knowledge. Yeah so it turns out, if you think of learning from a strictly Bayesian point of view, you have your prior beliefs, you get a data point, you update them, you get a posterior, and you get another data point. You update them. It turns out that sort of Bayesian updating can do continual learning and does not suffer from catastrophic forgetting. And all these things in theory, actually, many of our attempts at doing continual learning in large neural networks are approximations of that Bayesian updating. So that's one. One area of research is continual learning. Another area where I think we may need breakthroughs is energy efficiency. So if you look at the power consumption of a human brain, it's about watts. I don't want to make an equivalence. It's a light bulb. Yeah, it's a light bulb. It's a rubbish light bulb. Yeah it's a nice, very energy efficient light bulb, let's say. If you compare that to training, a large language model in a big data center, it's orders of magnitude off. I don't want to compare a single brain to a large language model, because single brains involve the single lived experience of a human being. Large language models are basically giant soups of all of World knowledge of some kind, but they're definitely less efficient. But they're way less efficient. So we can certainly afford to do more research in more energy efficient learning. I also think that there may be new architectures that we need to discover. Basically, the two workhorses of modern AI are transformers and diffusion models. They're great, but there may be completely other evolutions of software and hardware, like novel hardware architectures that may involve very sparse neural networks of various kinds and oh, I'll give you another one. Are learning systems are incredibly data inefficient compared to human and animal learning. So again, I think Bayesian ideas can help us there

对，我认为我们在第一阵营已经取得很多进展。当然，我们的系统极其有用，每天被数十亿人使用。但这不意味着有意思的发现已经穷尽。我举几个我认为重要的研究方向。一个例子是持续学习（continual learning）。我们现在训练模型的方式——我说的「我们」是整个领域——是训练一个巨型模型，用到产品里或以各种形式发布；几个月后再训练另一个巨型模型，如此往复。对比人类和动物如何学习：我们是持续学习的，基本上不断接收数据流，不断调整神经元之间的连接等等。我们的 AI 系统其实不太能做好这件事。它们会有灾难性遗忘之类的问题。这绝对连回我们前面谈的贝叶斯思路。因为人类、动物能这样做，是因为我们心里有一套把证据与既有知识结合的更新系统。对，如果你严格从贝叶斯角度看学习：你有先验信念，来一个数据点就更新，得到后验；再来一个数据点再更新。这种贝叶斯更新可以做持续学习，理论上也不受灾难性遗忘之苦。事实上，我们许多在大型神经网络上做持续学习的尝试，都是对那种贝叶斯更新的近似。所以这是一块。另一块我认为可能需要突破的是能效。人脑功耗大概是瓦特量级——我不想简单划等号——像一只灯泡。对，像一只灯泡。一只糟糕的灯泡。对，或者说一只很好、很节能的灯泡。对比在大数据中心训练大语言模型，那是差了几个数量级。我不想拿单个大脑和大语言模型直接比，因为单个大脑是一个人的单一生活经验，大语言模型基本上是某种世界知识的巨型汤；但它们肯定更低效，而且低效得多。所以我们当然值得在更节能的学习上做更多研究。我也认为我们可能还需要发现新架构。现代 AI 的两大主力是 transformer 和扩散模型，它们很棒，但也许还会有完全不同的软硬件演进，比如涉及各种非常稀疏神经网络的新硬件架构。哦，再给你一个：我们的学习系统相对人类和动物学习，数据效率极低。所以我认为贝叶斯想法在那里也能帮上忙。

sort of sounds a bit like you've discovered a magic trick. This magic trick which embeds humility and uncertainty, offers the opportunity for continual learning and makes it data more efficient. Yeah and you can write a single line of Bayes' rule, is this a little bit too good to be true.

听起来有点像你发现了一个魔术：这个魔术嵌入谦逊与不确定性，提供持续学习的机会，还让数据更高效。而且你能写出一行 Bayes' rule——这是不是有点好得不像真的？

It's too good to be true, right. Obviously, we've known this for a long time. I think it's important to understand these ideas. So I think all students of machine learning should at least understand that it's possible to do these things. The magic trick comes with a big curse. The curse here is that to do all of this is computationally very slow. So essentially, if you look at textbooks in AI, they will explain to you how to do some of these things, but they will say, we can't do these things exactly because they're computationally hard problems. They're kind of NP complete or NP hard problems to solve. And so we need to approximate them somehow. And you could argue, well, we know how to build ideal AI systems using these concepts. Are modern AI systems are approximations to that. Can we have our cake and eat it too. Can we have the best of both worlds here.

太好以至于不真实，对吧。显然我们早就知道这些。我认为理解这些想法很重要，所以我觉得所有机器学习学生至少该明白：这些事在原则上是可能的。魔术带着一个大诅咒：把这一切做出来在计算上非常慢。本质上，如果你看 AI 教科书，它们会解释怎么做其中一些事，但会说我们无法精确做，因为那是计算难题，有点像 NP-complete 或 NP-hard。所以我们需要以某种方式近似。你也可以说：我们知道如何用这些概念建造理想 AI；现代 AI 是对它的近似。我们能不能鱼与熊掌兼得？能不能两边优点都要？

Because the thing is, what you just said there about well, it's very computationally expensive. It's very slow. They were saying that about neural networks in the 80s. Yeah you're not going to make that mistake twice.

因为你刚才说计算很贵、很慢——80 年代人们也这么说神经网络。你不会犯两次同样的错吧。

Yeah, yeah. I think we can revisit some of these ideas with the compute power that we have now. The giant state of the art supercomputer. Parallel connection machine computer that I used in my undergraduate years is actually slower than the Pixel phone that I have in my pocket. Computation is getting better, faster, and also we have decades of ideas on how to approximate these methods really efficiently. And so we have the tools. We just need to put them together and maybe come up with a few new ideas.

对对。我认为我们可以用现在的算力重访其中一些想法。我本科用的那台顶尖超算级并行 Connection Machine，其实比我口袋里的 Pixel 手机还慢。计算在变得更好、更快；我们也有几十年关于如何高效近似这些方法的想法。所以工具在手，我们只需要把它们拼起来，或许再想出一些新点子。

What really strikes me is everything we've discussed, they feel very human like qualities. The sort of ineffable characteristics of humility and honesty and uncertainty and doubt. Yeah I mean, did you imagine when you started all these years ago that this would be the thing that our systems are lacking, rather than just computational power or power for analysis.

真正打动我的是：我们讨论的这一切，都感觉非常像人类特质——谦逊、诚实、不确定性、怀疑这些难以言说的品质。你多年前开始时，有没有想过：我们的系统缺的会是这个，而不仅仅是算力或分析能力？

I mean, certainly I didn't really imagine we would be where we are now, because I think if you talk to any AI researcher, they will be saying that they're stunned by the rate of progress. But I also think that, although these are human qualities we want to add, they're also kind of fundamental qualities of intelligence systems. So I think we need to have a concept of intelligence that transcends humans because as I mentioned before, we have flaws in our reasoning. We're actually quite bad at making good, rational decisions under uncertainty in the real world. We will miscalculate probabilities or misestimate probabilities and so on. So I think if we build human centric AI systems, we work backwards from. Well, what do humans need. What are society's biggest problems. What are humanity's biggest problems. And what are the AI systems that we need to solve those things. And for I think, all problems that matter, I would rather have an AI system that knows when it doesn't know than an AI system that is arrogant and overconfident.

我当然没真正想象过我们会走到现在这一步，因为如果你问任何 AI 研究者，他们都会说自己被进步速度惊到了。但我也认为，虽然这些是我们想加入的人类品质，它们也是智能系统的某种根本品质。所以我认为我们需要一个超越人类的智能概念——因为如前所述，我们推理有缺陷，其实很不擅长在真实世界不确定下做良好理性决策，我们会算错或估错概率等等。所以如果我们建造以人为中心的 AI，我们从后往前推：人需要什么？社会最大的问题是什么？人类最大的问题是什么？我们需要什么样的 AI 来解决它们？而对所有真正要紧的问题，我宁可要一个知道自己何时不知道的 AI，也不要一个傲慢且过度自信的 AI。

What an amazing point to end on. Steven, thank you so much. That was brilliant.

用这句话收尾太棒了。Steven，非常感谢。太精彩了。

Thank you Hannah.

谢谢你，Hannah。

For years, we tried to build AI that was focused on accuracy. Building systems that crunch through enormous amounts of data to land an answer. And these things, they're astonishingly capable, of course, but they're also quite brittle in some ways. When they fail, they fail with total confidence. But by teaching AI to embrace uncertainty, it gives it something more human humility and honesty. And the wisdom to doubt. And this isn't a vulnerability. It doesn't make AI weaker. It grounds it in reality, making it a collaborator we can actually rely on.

多年来，我们试图建造专注于准确性的 AI：让系统啃掉海量数据，落到一个答案。这些东西当然能力惊人，但在某些方面也很脆。它们失败时，是带着十足把握失败的。而通过教 AI 拥抱不确定性，我们给了它更像人的东西：谦逊与诚实，以及怀疑的智慧。这不是弱点，不会让 AI 更弱；它把 AI 锚定在现实里，使它成为我们真正可以信赖的合作者。