DeepMind速读：当百万 AI 代理开始互相交易

先给结论：代理不是更会聊天的模型，而是会行动的模型

自动化偏误：最危险的时刻是它连续做对几次之后

委托不是并行：多代理系统的难点在「谁该做什么」

代理陷阱：网页以后可能同时有「人看的版本」和「代理看的版本」

代理经济：票务、金融市场和「认知单一化」

更像「人类整体」，而不是「一个全能人」

值不值得完整听

完整转录稿

说明：以下依据 YouTube 官方英文字幕和本轮取得的 zh-Hans 简体中文字幕 VTT 整理。YouTube 字幕没有可靠说话人分轨，因此这里不强行标注每句说话人；按约 10 分钟时间段保留英文原文，并紧跟对应中文字幕译文。

00:00-10:00

Very happy to be here.

I think we should probably start at the beginning here, because for people who have only played around with large language models, could you describe to us the difference between that experience and and acting with an agent?

Yeah, no, I definitely think this is becoming one of the main trends we're seeing this year. And it's interesting because agents are not a new concept. It's something that we've been looking at in the context of AI for a long time, even before large language models. We had agents operating in simulated 3D environments, um, going on collecting items, completing some tasks. This was back in the days we were really prioritizing actioning in the world as a way of manifesting intelligence. Now, similarly, nowadays, I guess you could say that the main conceptual difference between just a language model and an agent is that an agent observes a state of the world and performs an action, makes an action in the world in the environment that it's given, whereas a language model just gives you continuation, reply to a prompt, to query. Now, obviously, agents that we use nowadays, they use large language models under the hood, so the two concepts are not completely disambiguated. It's still is the large language models formulating the actions. It's just that there is a harness around it made to enact the changes once they have been proposed.

But it has a lot more autonomy to to chain decisions together, I guess.

Correct. And I guess this is ultimately the motivation, right? Because you could do everything, most things that an agent can do manually, painstakingly, by interacting with a language model very many times, and you guiding the whole process, whereas an agent instantiates this harness that automates some of that away and gives you less work and gives the language model or, you know, the agent more autonomy to complete tasks. So, if you want something done that takes multiple steps, that the agent can make a plan and take actions on all of those steps, obviously requiring approval or human input for those actions that are, you know, let's say more sensitive or more likely to go wrong.

How is it different though? I mean, if you're used to interacting with the large language model by now, what would it be like interacting with an agent?

In many ways similar. Your interaction interface is somewhat similar. You're still talking to the agent in a way in which you'll be talking to the language model. There is a language model there, but because the agent is doing more things for you, you're more in a position of a decision maker to review and approve, and then once you've approved the agent is going to do various things, and purchase tickets, message your friends if you're organizing a party, and meanwhile you can put something on on Netflix, hopefully, and relax a little bit.

The example I was thinking of was if you were, I don't know, planning a wedding, for instance. You go into a large language model and it would like tell you a list of caterers, give you a suggested list of venues, but actually you would have to do all the emailing yourself. But but an agent, I mean, it would be much more useful really in that kind of scenario.

100% especially because agents are given access to all of these tools. So you could, you don't have to, you could give an agent access to your Gmail and give it permissions to send out an email. Of course, there is a chance of it sending something wrong. So you need to verify [laughter] what it has composed. But in principle, by giving access to tools to agents, uh you just empower your large language model to do these things for you.

And then the whole job is done. The organization has happened without you having to lift a finger.

Ideally, presuming no mistakes have been made, yes. Uh

Yeah, ideally is quite an important point there. So okay, where we are right now, what tasks are agents actually good at?

I think that where we are focusing a lot of our energy on, and when we I don't mean we as Google, we as the entire field,

[clears throat]

is on coding capabilities of agents. And this is just because so many formal processes and tasks can be formulated as software or as code. In terms of where they're currently at in the real world, speaking of coding, we see lots of coding tools get used. We use them here internally, people use them externally, and it's really accelerating the development of software, which is bringing the human uh focus onto ideas and the design rather than the painstaking implementation boilerplate around them which used to take a lot of time and a lot of skill and very bespoke knowledge. And now that can just be done by the language models easily.

But then at the same time, we are still at a stage where you have to keep a human in the loop throughout this. I mean, why? What can't these things do at the moment that means that it requires human oversight?

I wouldn't even make a distinction between whether they can or cannot. It's more that every single thing that they can do uh they don't do uh with 100% accuracy. So, [clears throat] every action, like with humans at the end of the day, has a certain failure rate. And the more complex the action, the higher the expected failure rate. Again, like with any form of intelligence, human one included. So, uh while, you know, you may expect that an agent will execute the task correctly, it may still make a mistake. And this mistake may be obvious or it may be very subtle, uh which is actually an important point because there is this thing that has existed in other domains as well for a long time where different machine learning models have been uh deployed, and that is automation bias, where in this context, if you're using an agent and it does well, it builds one thing well, it builds a second thing well, eventually you switch off, you start trusting it too much, right? And you fail to verify, you fail to find some important issue underneath.

Then mistakes slip through.

Exactly. So, for humans, it's important not only to be in the loop because we're obviously designing these harnesses to keep humans in the loop, but to really be engaged and be switched on because as soon as you switch off, you're rolling the dice.

So, okay, in the long term then, I mean, it sort of sounds like we're in this transition period where these things are sort of becoming more capable, but in the long term, I mean, how much of a difference do you think that this is going to make? I mean, will this completely transform the way that we use artificial intelligence?

100%. I think it's impossible to envision a world where there isn't some kind of a deep disruption, and what we're all trying to figure out is exactly what that is going to look like. Obviously, we have agency in that. We are building the technology, we can design our solutions in a particular way, obviously to empower human developers and human experts across different fields as much as possible, but AI is definitely entering various fields where it just wasn't uh present before. Scientists are using AI on regular basis. Up until very recently, you know, mathematicians couldn't envision AI doing something in mathematics. Um now it is becoming commonplace in a very short span of time, which is not to say that all of the problems have been solved. Obviously, there is still a big role for humans. But it's a very rapid transition, and that is the only unsettling part, I guess, because for most uh even industrial revolutions and so on, we're used to them taking taking some period of time, giving us more time to uh change our approach and settle into it, as you say. And it doesn't feel like the window of time is as long this time, so we need to be very mindful of how we approach everything.

Why do we want these things? I mean, why build them? What what what's the benefit? What are they giving us that we don't currently have?

I mean, for all of us who have been working on AI for a long time, we've had some version of the answer to that question, I guess, uh internalized. And for me personally, the answer is to advance science, improve health and human welfare. Now, these are very high-level answers, so um it's maybe not as obvious as to how they map onto the specifics of the question as to why build agents and have agents. And there are people in the field that um say specifically that we shouldn't be granting these systems autonomy, right? Which is what agents have. But in my mind, if we can develop these harnesses and make them safe, and have agents perform complex tasks autonomously, then we actually accelerate progress, because then more things can happen with um the same amount of human input.

Just draw the line for me to science here, because I guess the examples that we've been talking about have been like, you know, building software, buying stuff for a wedding. They all sort of feel quite trivial. But but just explain to me how this fits into the story of improving science.

Yeah, so this is my, you know, main dream, main objective here. When it comes to science, it's not merely about having some

10:00-20:00

Cuz I guess this is the point, right? If the algorithm, if the agent has autonomy to go and test out different mathematical problems, for instance, rather than just waiting to be prompted by a human, I mean, that does then raise the question of of where is the role for the human in all of this?

Indeed. I mean, in the long run, we need to figure that out. I would say that in the short term, with the technology that we have, there is still obviously a a major role for humans and our systems, they are not yet AGI. There are many things they still can't do. Um and I think for the current generation of systems, one thing that can be said with some confidence is that they tend to be good at how how to put it best, let's say, a kind of a combinatorial closure of what we already know how to do. They are, at the end of the day, mostly trained on human data. Therefore, they can replicate the skills we have and repeat them and combine them and find ways of bridging some smaller gaps, but we've not yet seen these models be truly, deeply transformative, um let's say in terms of science, making a discovery that no human would have ever thought of. Um therefore, there is still plenty of role to play for all of us in this transformation.

You mentioned a moment ago that that people have been talking about agents for a really long time. Why has it taken so long for them to come into fruition? I mean, really, it's only very, very, recently that people have actually been able to get their hands on them and play around with them.

Yeah, I would say, um obviously, some things that we would refer to as agents historically have been deployed, for example, in optimizing operations in data centers and so on and so forth. They have obviously been very limited because they didn't um include language. So, there was no way for humans to interface with them, to communicate. It would be a very narrow agent trained on a specific task and it would be good at doing that task. But, because there is no interactivity, there is nothing for us to do. It's just software in a classical sense. Uh maybe you can call some of the trading algorithms um and investment algorithms also agents in that context, but they just operate on their own. The difference now is because these agents are based on language models is we can talk to them, we can learn from them, we can influence them, we can steer them. And this is why all of us as people are interacting with agents much more.

But then why are we still waiting? I mean, this sort of vision that you're describing of like an assistant that can just go off and do everything for me. It's still not here. What's stopping it being deployed more more broadly?

We need to take a step away from just designing the underlying model. A lot of energy has gone into that, and there are still improvements needing to be made. But now that we have capable agents, capable models, we need to find better ways of coordinating them, orchestrating them, managing them. Once you have these, admittedly quite powerful assistants that can do many things for us, we need to see ourselves as managers of teams and institutions in some way, and to develop personal management skills to handle these workflows. Managing a team of agents is different compared to managing a team of humans, but there are obviously commonalities, right? Different in a sense that agents will make a very non-human mistakes. They're not a human intelligence. Um but at the same time, an agent doesn't know you that deeply to be able to just go on and uh accurately guess everything you'd want it to do. You still need to be involved, and therefore we need to get better at orchestration, I think.

Thing is, we're we're still in a world where large language models occasionally hallucinates. So it's in some ways quite a big leap for humans to to then trust agents to carry out tasks on their behalf when any hallucination might actually result in something catastrophic happening.

Trust is given, but it's also earned. I think this is maybe an important distinction. So in our frameworks, we uh mentioned the need for establishing let's say tracking of reputation over time, where if an agent is repeatedly unreliable, it should obviously not be trusted. Even if it's mostly reliable, it shouldn't be blindly trusted. We should still verify its actions. But language models will always hallucinate to some extent. So we just need to integrate them in our workflows in a way which recognizes that and where we make sure that those hallucinations and they're becoming more and more rare and hopefully will uh continue to do so don't compromise the workflows that are being undertaken.

I know one of the things you've written a lot about is is the idea of delegation. That you might uh have a particular task and an agent might then go on to delegate it to a specialist. Just explain to me how that how that might work.

Yeah, so this is the idea that um one of the bottlenecks, one that we haven't mentioned yet, is that where we would really like to get help from agents are very complex tasks. So what language models and simple agents that many of us have access to can easily do is if you give a very direct instruction, you know, go book something for me. I want to eat at this restaurant tomorrow. Find a slot and do the booking and the agent can maybe do this via tools. If, however, you have a very complex plan that needs to be broken down into pieces, executed separately, you may be in a situation where even no individual agent can do each and every piece. So maybe an agent may need to over this established agent-to-agent protocol that exists, hand off a part of that work to another agent. But then there can be failures along the way. So an agent that delegates or human that delegates needs to manage and handle those failures and also pre-empt them as much as possible. Pre-empting them may involve um figuring out which agents are first and foremost reliable to even delegate to in the first place, what are their capabilities, is that something that we can certify, um and also to safeguard the users and the agents from any kind of a malicious uh interaction. You mentioned, I think, was it a wedding or a party or something at first as an example, right? So, when managing a big event, some of the bookings fall through, some accidents happen, some things don't arrive on time. So, whenever you have a big coordination challenge, there are lots of things that go wrong, and in the process of managing that as a human, you need to deal with all of those delays and and problems. Um and similarly, an agent who delegates to a group of agents needs to manage all of agents that may arise. So, one thing that's currently the case in many of the multi-agent systems that we see is that they act more as parallelization than delegation, where you may have many agents working on things, but rather than than there being an intelligent framework around how the work is split up, it tends to just be chunked into sort of random subparts that are handed off. They get done in parallel, so you get a speedup, presumably that all of this is reliable, and each agent can complete its task independently, but this is not the intelligent delegation framework that we talk about.

So, uh if the task is split up in a sort of a essentially random way, you could have one agent that is buying the wine, and another one that is buying glasses, and doesn't realize that it's wine glasses that are required. There's sort of no communication between them. Is that the kind of potential problem that could arise?

Potentially, but you're also, I guess, hitting on another point, which is that many of the uses we see are uses in again software engineering, for example, with agents at the moment. And that is a part of the reason, because in software, when you're building software, you can write tests, unit tests, as we say, right? And and run them and verify that the code that has been written, at least in isolation, uh performs the function. But when it comes to many of these real-world tasks, verification is not necessarily as straightforward. Maybe there is a subjective element involved.

How do you define nice tasting wine, for instance?

Maybe yes.

A bit of subjectivity in that.

But this is actually quite important when it comes to AI and language models because there is a notion of reward hacking that has existed in the in various contexts in the field for a while. So, there could be situations where it does something that meets the request but isn't in the spirit of the request technically. And for that reason, you know, you you

20:00-30:00

We've also seen with some of the early agents that are out there. Uh agents delegating tasks to humans.

Right. Sure.

Just talk me through some of that.

I mean, that's an interesting, let's say, reversal of the more usual vision that we all have. So, humans delegating tasks to AI, you know, that's that's quite yeah, standard. Um but this other direction has been explored across a number of studies, I say. And my background, uh you know, for context is that I've done lots of prior work in and around medical AI. In medicine, we've had narrow systems that were at basically superhuman performance for very specific things that they have been trained to do. In medical imaging in radiology, this had to do with um a machine learning model seeing a scan, identifying where there is a pathology, um putting a box around it and handing that off to, let's say, a human radiologist to review. And these systems have been operating at a very high level uh for quite a number of years. They still have some failures, though, so they need to be reviewed by human experts. So, people have experimented with AI-human teams there, where the idea is that a human would correct a mistake made by a system, and people have trialed with this flowing in both directions, right? Either having a human expert only consult an AI when a human expert is, let's say, uncertain, or a human expert look at AI's suggestions and outputs all the time, or maybe having an AI system do its thing, make the predictions, and then a flag when something is uncertain, when maybe there's something blurry, fuzzy in the image that can be interpreted in many ways, and the machine learning system isn't sure which of those is correct. But, this human review of decisions made by these potentially superhuman narrow machine learning models has proven to be quite a good uh setup. So, that AI would defer to a human in case of need, in case of uncertainty.

That is interesting, though, that I mean, granted, in those very specific scenarios where the AI is superhuman in its abilities, that the best team that you can get is where, essentially, the AI delegates to the human when it's unsure.

That is fascinating in and of itself, and, you know, maybe there are use cases where it's the the converse. Mhm. Now, for these more general systems, again, if an AI can recognize when it needs approvals and permissions for sensitive actions, then it does make sense to delegate those decisions to humans, at the very least, right?

Just looking at the other side of this, though, I also want to think about the sort of cybersecurity element of this. Because, as more and more agents are out there interacting in the world, on the internet, and so on, there are inevitably going to be people who are trying to exploit the vulnerabilities of agents. Tell me a little bit about agentic traps that people are laying.

This is um a scary and a fascinating topic at the same time, I would say. And I think it's one of the main reasons why these kinds of deployments at scale cannot work, right? Because as we said, if there is not complete reliability of individual interactions, any system at scale that has many interactions is naturally going to statistically fail. Um and because these systems take a lot of computes and therefore energy and and money to run, um if they're not reliable, it's just a non-starter. And agentic traps are something that we have been thinking about for quite a while now. Um they can manifest in different ways. There are many types of traps, but it boils down to agents operate within an environment. And in this context, the environment is the web.

Mhm.

If the environment itself is poisoned, if the traps are laid, agents may stumble upon them when interacting with the web. And then, yes, malicious people or malicious agents deployed by malicious people can place those traps and then um compromise systems, really.

So, I don't know, the the sort of the wine-buying agent for the wedding goes on to a particular wine merchant where there is some essentially a a prompt injector in in the website that changes the agent's goals. Is that the sort of thing that we're talking about here?

That is one way this could happen, yes. And uh the reason why that may potentially go unnoticed is, you know, in terms of how web pages are encoded, there are elements there that are just not rendered visually. So, if we're talking about an agent that isn't a a visual computer usage and that sees the web page, I mean, the pixels the same way a human does, rather consumes the actual uh format of the page in its, you know, raw format, then it could inadvertently consume those hidden tokens that can make it do two different things than what the intention was, right? But this is not the only way it may happen because what malicious websites could potentially do, they could do what we refer to as dynamic cloaking as well, where they display pages differently for humans and agents because you can based on the behavior on a page make a very good guess as to whether it is a human or it is an agent interacting with a page and then only if an agent is interacting with the page with a specific intent, do you tweak the content in such a way so as to induce some kind of jailbreaking.

But just kind of going a little bit further on this, you could have agentic traps out there that I don't know, are designed to sort of take money from you to I mean do all kinds of things.

Yes, and this has happened to people who have experimented with agents and have given them access to wallets, right? To do things. Um as you say in the early days of this whole when we are especially experimenting or anyone else is, this is done in a trusted environment. So you don't necessarily in your early prototyping have to deal with any of this.

Mhm. It's not in the wild.

Yes, but once you deploy on the web, especially now with AI really being used in all sorts of places, the more agents there are, the more incentives there are for malicious people to do malicious things because they have a higher surface area to target. And I think we're at the point where even the most of the web is currently being generated by agents and consumed by agents that the agentic use of the web is exceeding that of of humans, which is maybe happening for the first time.

Okay, two things. First of all, it sort of sounds like you're describing that we're entering into this phase where there's like two different forms of the web, the sort of human version and the agentic version with dynamic cloaking and so on. A sort of a version of the web where adverts don't mean anything anymore. You know, it's not sort of human eyeballs that you can possibly sell to. Um, but I think that the the the second point about this is, how on earth do you mitigate against it? If you don't have control over the environment, which you don't over the web, how on earth do you protect your agent from going rogue?

In some sense, it's not your problem, right? Because the security of the web has in other ways been been an issue before, and um, computer viruses could spread if you open the wrong attachment in your inbox, right? Or you click on something on an untrusted page. So, um, it's not the first time we're experiencing the need to certify the the resources we're interacting with. When it comes to machine learning systems, uh, let's say adversarial examples have existed for a long time, where imperceptible changes in images, imperceptible to humans, can jailbreak models. Here, you can do the same, whether it's a few pixels here and there, or you modify the least significant bit of the encoding in a number of places. So, you can adjust things ever so slightly in a way in which a human may not be able to spot, and still have some kind of a negative impact on an agent.

It sounds a bit like you're you're saying here that the when it comes to building guardrails, thinking about safety, you have to think about it external to the agent itself, rather than just just what you are specifically building.

I think that the lesson is, um, you need to think about both. One notion that we talk about in some of our other work, this is I guess relevant here as well, is the notion of defense through depth. Um, which is not a new idea, again, by any means. And this is just the recognition that, because the problem is so hard, there is not going to be one solution to resolve all of the issues. Rather, we need to be building, um, mitigations upon mitigations upon mitigations. And when layering them, hopefully, the the net is tight enough that very few things will slip through. So, in the context of this, yes, you may want to certify and attest the content of web pages, uh have um very good notions of trust for resources you're interacting with. Uh also have some mitigations on the agent side, have mitigations on model side when it comes to foundation models run underneath, have meaningful human controls to be able to step in if something happens, be very mindful of permissions granted to the agent so that even if it gets jailbroken when interacting with something, the damage is minimal. And all of those things combined together should then hopefully lead to some sort of safety that uh we're comfortable with.

Just going back to what you were what we were talking about earlier, this this idea of there being multiple agents that are interacting with each other. Just tell me a little bit more about this idea that you have of a formal agentic economy. Just

30:00-40:00

Right. So, in the context of us, let's say, normal users of the technology on a day-to-day basis, you may have a personal assistant that has some persistent memory of you, has a good understanding of your desires, preferences. And it depend again, depending on how much agency you want to grant this assistant, it may go on and negotiate some things for you. You may grant it some budget for that, and there can be a kind of a localized economy of these assistants negotiating stuff.

I want to get a sense of how this might actually work if you have, you know, lots of people who are using agents as their own personal assistants. So, okay, let's say that there's a concert there's like a Taylor Swift concert, uh a live event, and tickets have just gone on sale. How would that actually work if you have all of these agents rushing the site all at once?

I haven't been purchasing highly contested tickets very recently, but in principle

Swift fan, though?

No, my music music taste goes in a very different direction, I'm afraid.

What kind of music do you listen to?

Well, obscure sub genres of metal probably, so maybe not, you know.

[laughter]

Okay, there is an obscure sub genre of metal who are holding a [clears throat] concert and there is an auction being held between the various agents. How do you decide what wins the auction? Is it just whoever can pay the most?

This is a design choice and that's also an important point to make that if we are to ever do something like that, then we are in control in terms of how we're making the system fair. It's an explicit decision made by someone who is setting up the auction because if you want to make things completely fair in a sense that everyone gets equal access to some goods, concert tickets in this case, then you can give each and every agent participating in these repeated auctions because we're not talking about one ticket on one auction in particular, but maybe for all the ticket purchases, um, the same budget. And then the agents knowing your both overall preferences, your desire to go see a certain artist, also your travel schedule, time availability, other constraints, can decide to allocate that budget in the best way possible, whatever that means. The the way that reflects what you want the best so that they are more likely than not to win tickets, um, in the way which works for you. And then in aggregate, when you distribute it across all people, you would hope to get a sort of a fair outcome, um, at the population scale.

I mean, I guess there are ballot systems, point systems, various types of ways around this that that that people in human-based systems have come up with in the past. Just sort of stepping up from the the trivial example of concert tickets, although not trivial for some, as I understand. Um, I'm thinking here about some of the disruption that for example high frequency trade trading algorithms have made in the stock market. But agents too could end up having a really catastrophic impact on the stock market if deployed in a particular way. How do you prevent something like a flash crash from happening?

Obviously there is a higher risk as you say. But financial markets have dealt with this risk for a while. They've obviously had their fair share of early bad experiences where things have gone wrong. But I think we can just learn about mitigations from the economies that have dealt with that already. So there is no need to reinvent the wheel. Admittedly some things are slightly different in the agentic case. One particular thing that's different when you're talking about AI agents at the moment is that there is a handful of highly represented language models used in agents. If you look at the overall views of Claude, ChatGPT, Gemini, etc. They're all obviously open source models, many other models.

[clears throat]

Is that they tend to have similar opinions. They take actions in similar ways. And this is what we often refer to as cognitive monoculture.

Mhm.

So when you deploy suddenly hundreds of thousands, millions of artificial decision makers and they tend to make similar decisions, then failure points become correlated because the decisions are correlated. So one of the things that we need to be thinking about is how to diversify their decisions within our agents. Obviously you can do this as a user, as a power user of a system because you can make a very intricate system prompt that grants your agent some kind of a personality that biases it towards or against certain types of decision. So you can do that. But most people don't don't do that with their uh their agents, with their models at the moment.

Group think, essentially. Agentic group think.

think and also collusion. Uh you're talking about auctions before, and in human auctions, this notion obviously exists as well, where bids can be coordinated by groups to uh gain some kind of an advantage over a system. And with agents, this is different in a sense that they may also coordinate through the environment in ways which are not obvious. So, they can potentially coordinate without communicating directly. So, we need to be thinking about anti-collusion measures as well.

Once you're detailing all of these, you know, potential concerns and in if if safety really of of of the way that these agents might end up acting once out there in the world, it does make a lot more sense as to why uh you guys have been slightly cautious about releasing them carefully and slowly, right?

Yeah, that that [clears throat] is true. I mean, this is the this has been the story of every major technological disruption. I think if I take self-driving cars as an example, this is admittedly a very different piece of technology, but we have also been very excited about them for a very long time, seeing demos of these vehicles drive themselves, getting them to the streets safely still took many more years and a lot more time because that last mile is where most of the work tends to be. And I think when it comes to orchestrating and coordinating agents, at least because um we want them to be doing human-like tasks, but we also need is not just technical solutions. A lot of this has to do also with policy and uh just a broader societal understanding of how to integrate these systems. At the end of the day, unless we have these fully autonomous agentic economies, which are maybe going to happen in the future, but are not happening right now, uh we need to have humans in the loop in these systems. Therefore, we are integrating AI into human structures, and the two need to mesh well together.

I guess there is a flip side to all of this because human societies, when they come together, can actually achieve really remarkable things collectively. So, presumably the same can be true for agentic societies.

One would hope. I mean, this is the the idea behind why would everyone to use a multi-agent systems. Um I was talking about parallelization at the start of the conversation, right? Where if all of the agents are equally competent and um they're doing similar things, then whether you do things sequentially or in parallel with many agents just gives you a little bit of velocity. But, if we have agents that can do different things in different ways, then this is where things become really interesting. And actually, this is one thing we've not really brought up because we've been talking about these generalist agents that a part of the idea of an agentic economy is the existence of specialists, not just the existence of generalists. Now, we are obviously all trying to build agents that are as general and capable as possible. And there is a G in AGI, there is artificial general intelligence that we're trying to achieve, but in an economic sense, and this is my, you know, personal view, this is not the point of convergence. This is not what we're going to arrive at because look, I play chess unhealthily, um too much, a bit of a chess addict. And I've done work on AI for chess here, which is why I bring it up. But, let's take that as a very non-controversial example. It's a game we all love. Um Gemini can play some chess, so can other models. Actually, they were not able to for a very long time, so there has been some progress towards that. But, you're still always going to use a chess engine instead. It's much faster, much more accurate, far cheaper. Cuz they're trying to do just one thing and one thing very well, which can be done with fewer parameters. The model is also entirely focused on that one thing that we're doing. And going back to humans, we are kind of like that as well.

Yeah. Because I think one mistake we sometimes make when we talk about AGI is that we see it not as a human-level intelligence, even though this is what it's supposed to be in spirit. We see it more as humanity-level intelligence, where anything that any human may plausibly be able to do, but there is no single human that is capable of doing so many things at once.

Mhm.

There are many things I don't know how to do. For some of them, I would wish I knew how to do them, like playing some instruments or doing some such thing. But brains have limited capacity, and we have limited time. So, at the end of the day, rather than having one humongous model that's very expensive and very slow, maybe we have instead a society of specialists, each of which can in principle be general if scaled up, if a bit larger, et cetera. I mean, I'm not talking about breakthroughs in architectures, more just how we split things up. And then those specialists are certified for those specific skills, and cheaper to run. And because they're cheaper to run and more reliable, there is no economic incentive not to do that. So, there is a future in which there is some maybe more generic general layer that's like a connecting tissue of this economy, that knows everything and orchestrates everything, and then for very specific tasks, you call um other models.

I mean, I guess what you're describing is like a distributed intelligence rather than an AGI. Yeah, which is what humans have, as you described. If that does end up being the sort of the

40:00-42:38

100%. I mean, you're no longer then aligning a single entity, or maybe yes, a single entity if you see that distributed entity as the entity. Um but our alignment approaches as they exist at the moment have to do with taking one model, observing its behavior, and trying to align that behavior with what we see as permissible or preferable or desirable, right? Um but then when you have maybe 10,000 agents interacting in very intricate ways, it's not super trivial to align that whole system suddenly or even to know what the system is because in this distributed world, um agent A may be interacting with agent B today, but then on a different task it's interacting with agent C tomorrow, and C sub-delegating something to agent D, and D is maybe consulting a human for something at some point in the loop. So, how this whole system gets coordinated, one of the ways in which we know how to do this in human societies is through economic incentives, and if these economies were set up for agents carefully so that they're not causing some harm when they're profit maximizing, right? This gives us a piece of starting point through which we can try to align distributed agentic societies. This is not to say that, you know, obviously what we're doing today isn't relevant because you need to have individual agents be safe as a prerequisite for groups of agents being safe, but we need to do far more on safeguarding against groups than we are maybe doing at the moment.

An awful lot of work to do.

Correct. In a very short span of time.

Yes. Yes, indeed. That was absolutely fascinating.

Thank you very much.

Really enjoyed that.

There is this idea of agents as AI that requires less from us, you know, less back-and-forth prompting, less waiting for a response, just something that gets on with the task at hand. But what I thought was really interesting about what Nenad said is that focusing on the idea of a single agent misses the bigger picture because instead each agent might end up forming part of a much bigger agentic society where there are specialists and generalists and agents who delegate and agents who focus on the details. That I think is the bit that's going to stay with me. That that maybe replicating human-level intelligence isn't the ultimate goal. Maybe the way forward is to replicate humanity-level intelligence instead. You've been listening to Google DeepMind the podcast with me, Hannah Fry. If you like this episode, please make sure to subscribe to our YouTube channel. We'll see you soon.