先把产品当一个实验室：Dan Shipper 谈怎么同时探索前沿和交付｜Lenny's Podcast

他开场先说了一句玩笑

这一场要解决的问题：探索与交付互相拉扯

先弄明白几个词

为什么要在产品团队之外养一个小实验室

怎么把实验室跑好

一个具体案例：把主编的改稿变成一个智能体

从实验室回到主产品：Codex 那条路

这一场值不值得看

人物背景与外部对照

完整中英双语逐字转录稿

00:00 - 10:00

Okay. So, I'm going to talk about how to build products on a moving frontier. Um, or as I like to call it, the unreasonable ineffectiveness of business as usual during technology revolutions and what to do about it. So, the world just changed. The world just changed again. Uh, we had Fable 5.1 launched last week. We had Astra 6 launched last week. And these are sick models. Like look at this. This I made this like Battle of Waterloo 3D like historical re reproduction that's historically accurate with like a single prompt and it just churned for like four hours and then I got that like that's crazy.

中文：好的。所以，我将谈谈如何在不断变化的前沿构建产品。嗯，或者像我喜欢称呼的那样，在技术革命期间，常规业务的非理性低效以及如何应对它。所以，世界刚刚发生了变化。世界又一次发生了变化。呃，上周我们发布了 Fable 5.1。上周我们还发布了 Astra 6。这些都是非常棒的模型。看看这个。我用一个提示就制作了一个《水蓝之战 3D》的历史再现模型，完全历史准确，它运行了大约四个小时，然后我得到了这个结果，太疯狂了。

That is crazy. Someone else on my team made this uh sim agent simulation where he fed it a scientific paper about how agents interact and and how they get simulated together. And then he had like a thousand agents all running on his computer and like visualized by this after like just a couple prompts and putting a paper in. It's also now starting to do our video editing. We do a lot of videos and like Astra is actually good enough to go into Premiere and like do a bunch of editing and uh also fun fact the all of the animations in this deck and a lot of this deck was made by Astra.

中文：那太疯狂了。我团队里的另一个人做了这个模拟代理模拟，他输入了一篇关于代理如何互动以及如何一起被模拟的科学论文。然后他在自己的电脑上运行了大约一千个代理，并且通过几个提示和输入一篇论文就将其可视化出来了。它现在还开始参与我们的视频编辑。我们做很多视频，而 Astra 实际上已经足够好，可以进入 Premiere 进行大量编辑。顺便一提，这个演示文稿中的所有动画以及很多内容都是由 Astra 制作的。

Like I did not touch a lot I didn't touch the animate tab at all and we'll get into that in a sec. So this is crazy. What should you do? Do you keep your head down and focus if you're a product leader and you have a product team? That might work for a little while. Uh do you ask your customers what they want? Like maybe, but most your customers are not familiar with this stuff. They're actually looking to you uh to tell them what to do. Do you turn your B2B SAS app into a 3D multiplayer strategy game? Maybe. Maybe.

中文：就像我没有碰很多一样，我根本没有碰“动画”标签，我们一会儿会详细讲这个。这真是疯狂。你应该怎么做？如果你是产品负责人，并且有一个产品团队，你会低调专注吗？那可能暂时有效。你会问你的客户他们想要什么吗？也许吧，但大多数客户不熟悉这些东西。他们实际上是希望你来告诉他们该做什么。你会把你的 B2B SAS 应用变成一个 3D 多人战略游戏吗？也许吧，也许。

But it's hard. This is this is hard. This is a this is a big problem that I think everyone in this room faces. I face it. So the first thing to know, rule number one is never make any major life decisions within 30 days of a meditation retreat, a psychedelic experience, or your first encounter with a frontier model. All right, we're settled. We're settled. That's rule number one. So let's get into the problem, though. So what's the core problem? Why is this so hard? It feels like if you're running a product team, it feels like it's everyone's job to execute the road map at a high level and to stay at the frontier at the same time.

中文：但是这很难。这真的很难。这是一个——这是一个大问题，我认为在座的每个人都会面临。我也会面临。所以首先要知道的一件事，规则第一条是，绝不要在冥想静修、迷幻体验或者第一次接触前沿模型后的 30 天内做任何重大的生活决策。好了，我们确定了。我们确定了。那就是规则第一条。那么我们来说说问题本身。那么核心问题是什么？为什么这这么难？感觉就像你在领导一个产品团队一样，好像每个人的工作既要在高层次上执行路线图，又要同时保持在前沿。

And that's really really hard because these are very opposed ways of working. Uh if you want to explore the frontier, you're exploring. That's divergent. You're doing lots of different things. You're probably going to throw a lot of this stuff out. You're you're trying all these new models. You're doing demos. Exploiting is very different. You're converging. You are focusing. You're saying no to things. You're trying to execute against a road map that you've like you've planned out. And doing both of those. Uh you're you're pulling in the opposite direction.

中文：这真的非常非常困难，因为这些是非常对立的工作方式。呃，如果你想探索前沿，你就是在探索。这是发散的。你会做很多不同的事情。你可能会丢掉很多东西。你在尝试所有这些新模型。你在做演示。而利用则非常不同。你在收敛。你在集中注意力。你在对一些事情说不。你在尝试执行你已经规划好的路线图。而同时做这两者，呃，你就在往相反的方向拉。

Um so what should we do? Well, the thing I want to talk about in this talk is I my contention that you should run your product or like a research lab or you should add some research lab lab elements to your product or we're going to go through why you should make a lab, how to run a lab, and then what what like how to bring what works from the lab back into your main product. We're going to use examples from stuff we do at every and from some of the uh best companies in AI and how they do it. So let's start. Why should you make a lab?

中文：嗯，那么我们应该做什么呢？好吧，我在这次演讲中想要讨论的事情是，我的观点是你应该把你的产品当作一个研究实验室来运行，或者你应该在你的产品中加入一些实验室的元素，或者我们将讨论为什么你应该建立一个实验室，如何运行一个实验室，然后如何把实验室中有效的东西带回到你的主产品中。我们将使用我们在 Every 公司做的事情以及一些 AI 行业顶尖公司是如何做的例子。所以让我们开始吧。你为什么应该建立一个实验室？

The problem is as we said before right now everyone in your product or is doing both exploration and execution. The yellow circles are the execution, exploration is purple. You can see everyone in your or is doing both. And uh what what's really interesting though is there are some people in your or I guarantee it there are some people in your org who are doing way more exploring than others. And that's the really important to identify um because if you identify these people, they are your early adopters. We are your early adopters.

中文：正如我们之前所说的问题是，目前你们组织中的每个人都在同时进行探索和执行。黄色的圆圈代表执行，紫色代表探索。你可以看到你们组织中的每个人都在做两者。而且，有趣的是，我可以保证你们组织中确实有一些人比其他人做更多的探索。这一点非常重要，因为如果你能找到这些人，他们就是你的早期采用者。我们就是你的早期采用者。

And by the way, that animation and this whole all this stuff all built by Astra. Round of applause for Astra. Inc. incredible. Incred. I did not do any of this. So we are your early adopters. Um we love new things. We're already running Astra and Fable on the weekends to do our own personal projects or do things that are similar to uh to to explore things that we might bring to work. And because of that, there's a lot of things we might be excited about that aren't useful, but because of that, we're going to have a really good sense of what might what might your product become?

中文：顺便说一下，那段动画以及所有这些东西都是由 Astra 构建的。为 Astra 鼓掌。公司非常了不起。真了不起。我没有做任何这些。所以我们是你们的早期用户。嗯，我们喜欢新事物。我们已经在周末使用 Astra 和 Fable 来做个人项目，或者做一些类似的事情，以探索我们可能带到工作中的东西。正因为如此，有很多事情我们可能会觉得很兴奋，虽然它们并不实用，但也正因为如此，我们会对你的产品可能会变成什么有一个很好的感觉。

Because we're like living in the future already. The problem is that we can be a massive distraction. And um the so the the key question to to ask yourself if you're running a product or is how do I harness the early adopters in my team on my team and maybe even uh some of my some of my customers who are early adopters without distracting everybody else. And the solution that uh we found at every and that I think a lot of big a lot of really great companies are starting to do is make a labs team. On a labs team what you've started to do is separate concerns.

中文：因为我们就像已经生活在未来一样。问题是它可能会成为巨大的干扰。嗯，所以关键的问题是，如果你在管理一个产品，你需要问自己，我如何在我的团队中利用早期采用者，甚至可能还包括一些早期采用的客户，而不干扰其他人。而我们在 Every 找到的解决方案，我认为很多很棒的大公司也开始这样做，就是建立一个实验室团队。在实验室团队中，你开始做的是分离关注点。

Some people are in charge of improving and scaling what already works. They're they're on the product team. And some people are in charge of exploring what what what comes next, exploring the frontier, doing all these demos, trying these new models, all that kind of stuff. That allows you to get the best of both worlds. And I'll explain how that works and why. And if you're looking at this and you're thinking, "Oh, that's really expensive and I don't have the resources for that and all that kind of stuff." What is amazing about AI is it allows you to have a labs team of one.

中文：有些人负责改进和扩展已经有效的东西。他们在产品团队里。还有一些人负责探索接下来会出现的东西，探索前沿，做各种演示，尝试这些新模型，所有这些类型的事情。这让你能够获得两全其美的效果。我会解释这是如何运作的以及原因。如果你看这个，并且想，‘哦，这真的很昂贵，我没有那样的资源，诸如此类。’AI 的惊人之处在于，它允许你拥有一个人的实验室团队。

uh you can just have one person on your team whose job it is maybe even like right when a new model comes out to go explore all that stuff and come back and tell you what they learned. It actually does not require a lot of resources because all the people on your team are now so they have so many superpowers because they can just go off and have Astra and Fable do a bunch of stuff that it doesn't require a huge investment to do this. So again to sort of pull apart the difference between a labs team and a product team on a labs team your job is you're going to explore capabilities especially when new models drop to see what's now possible.

中文：呃，你可以只让你团队里有一个人，他的工作可能甚至是在新模型一出来的时候就去探索所有这些东西，然后回来告诉你他学到了什么。实际上这并不需要大量资源，因为你团队里的所有人现在都有很多超能力，他们可以去让 Astra 和 Fable 做很多事情，所以做这个并不需要巨大的投入。所以，再次想要区分实验室团队和产品团队的不同，对于实验室团队来说，你的工作就是去探索能力，尤其是在新模型发布时，看看现在有什么可能性。

You're going to run a ton of experiments in parallel and we'll get into that. But crucially, the expectation for a labs team is very different. On a labs team, you're going to expect to dispose of like 90% of what you make. You try it and you throw it away. Product team is very, very different. Your job on a product team is you're going to improve and scale the product. You want to deliver for existing customers. You want to make sure that they feel like they understand the product. It's coherent. You're not just throwing a bunch of garbage at them.

中文：你将会并行进行大量实验，我们稍后会详细讨论。但关键是，实验室团队的期望与众不同。在实验室团队中，你会期待丢弃大约 90% 的你所制作的东西。你尝试它，然后扔掉它。产品团队则非常非常不同。你在产品团队的工作是改进和扩展产品。你希望为现有客户提供服务。你希望确保他们觉得自己理解产品。它是连贯的。你不是在随便向他们扔一堆垃圾。

So for a product team, you want to expect to adopt about 10% of what the labs team makes or tries. And this is how they sort of start to work together. And there's a lot of really good examples of this starting to work in AI in a way that like we've had labs teams for a long time, but it's starting to work in a way that it has never ever worked before because of this technology. So a really good example is Anthropic Labs. Um, I'll be talking with uh some people at Anthropic a little bit later about this, but Anthropic Labs, Claude Code, like one of the biggest, most successful productivity products of all time, came out, came out of Anthropic Labs, a separate little group in charge of doing lots of experiments.

中文：所以对于一个产品团队来说，你应该预计采用实验室团队所做或尝试的约 10%的内容。这就是他们开始合作的方式。而在人工智能领域，有很多非常好的例子显示这种合作开始奏效，就像我们很久以来一直有实验室团队一样，但由于这项技术，它开始以一种前所未有的方式运作。一个非常好的例子是 Enthropic Labs。我稍后会与 Enthropic 的一些人谈谈这个问题，但 Enthropic Labs 的 Claude Code，作为有史以来最大、最成功的生产力产品之一，就是从 Enthropic Labs 产生的，这是一个负责进行大量实验的独立小组。

MCPs, Skills, Claude Design, all of these things came from a small group of people inside of Anthropic experimenting with things and then Anthropic investing more and more into the winners and the ones that worked. and there's like a thousand experiments that you never you've never seen. Let's say you're convinced now that this sort of structure gets the best gets you the best of both worlds. It it helps you to unleash your uh your early adopters without distracting the people on the product team who need to actually deliver on the road map.

中文：MCPS、技能、抓爪设计，这些东西都来源于 Enthropic 内部的一小群人进行实验，然后 Enthropic 不断地在那些成功的、有效的项目上投入更多。你从未见过大约有一千个实验。假设你现在已经确信这种结构能够让你获得两全其美的效果。它有助于释放早期采用者的潜力，同时不会分散产品团队成员的注意力，而这些人需要实际完成路线图上的任务。

The question is like how do you run a lab well? So the first thing that I've started to come to is you really want to use very very very small teams. For a long time, the uh the standard for team size was the the two pizza team. So that's like eight to 10 people. That's what Jeff Bezos came to. I think in the AI age, I I call it a two slice team. Like you want one or two people max. You can get so far so fast with one or two people that anything more, there's a lot of coordination overhead and there's differing visions and it just doesn't work.

中文：问题是如何有效地管理一个实验室？所以我最先想到的是，你真的想要使用非常非常非常小的团队。很长一段时间里，团队规模的标准是“两份披萨团队”，也就是大约八到十个人。这是杰夫·贝佐斯定下来的。我认为在人工智能时代，我称之为“两片披萨团队”，你最多只需要一到两个人。一到两个人可以走得非常快，非常远，而人多了之后，会有大量的协调工作和不同的愿景，这根本行不通。

One or two people is really really great. And I think the the composition that works for us and I I think is starting to work for other people is I call them pirates and architects. So, one person on the team is the pirate. They are slop cannons who are absolutely obsessed with with finding value. That's that's me. Like, I'm just out there just doing lots and lots of stuff and I'm throwing it away and it's all messy, but I'm going to find something really interesting. And then architects are people who are like looking at a messy system, like maybe something that I built that's completely violated and trying to help shape it into something that's valuable and beautiful and extensible.

中文：一两个人真的非常非常棒。我认为对我们有效的组合，我也认为开始对其他人有效的组合，我称之为海盗和建筑师。所以，团队中的一个人是海盗。他们是狂热分子，绝对痴迷于发现价值。那就是我。比如，我只是不断地尝试做很多很多事情，虽然会丢掉很多，整个过程很混乱，但我会找到一些真正有趣的东西。然后建筑师就是那种看着混乱系统的人，比如可能是我创建的完全杂乱无章的东西，尝试将其塑造成有价值、美观且可扩展的东西。

And pairing those types of people together, I think, is very, very powerful in AI. Um, the next best practice that I think is really important is dog fooding. The biggest thing that that matters on a labs team is making the feedback loop as tight as possible between making something and knowing if it's good. And the tightest feedback loop is making something for yourself. If you can do that, I would do it. If you can't, then it's really important to get a couple of early customers or early adopter type customers uh to help help you test with a tight feedback loop so you can iterate really really fast.

中文：我认为，将那种类型的人配对在一起，在人工智能领域是非常非常强大的。嗯，我认为下一个非常重要的最佳实践是“自用自测”（dogfooding）。在实验室团队中，最重要的一点是尽可能缩短从制作某物到知道它是否良好的反馈循环。而最紧密的反馈循环就是为自己制作东西。如果你能做到，我会建议你这么做。如果做不到，那么非常重要的是找到一些早期客户或早期采用者类型的客户，帮助你在紧密的反馈循环中进行测试，这样你就可以非常非常快地迭代。

Um and to use the the stuff you're building all the experiments ideally should be used for actual work that you actually have so you can tell is this useful or is it just new. It's a big thing you need to differentiate. Another best practice which I think is quite different from how you should probably run a product team is you should be building many experiments in parallel even if those experiments are trying to do the same thing. Try competing approaches to the problem. Uh it might look like a mess and sort of like a lack of coherence but everybody's going to have a little bit of a different take on how to solve a particular problem.

中文：嗯，要使用你正在构建的东西，所有的实验理想情况下都应该用于你实际的工作，这样你就可以判断它是否有用，还是仅仅新颖。这是一个你需要区分的重要点。另一个最佳实践，我认为这与如何管理产品团队可能大不相同，就是你应该并行构建很多实验，即使这些实验试图解决相同的问题。尝试不同的方法来解决问题。嗯，这可能看起来有些杂乱，似乎缺乏一致性，但每个人对如何解决特定问题都会有一些不同的看法。

And when capabilities move the frontier becomes very unknown. And so having people try the same thing from different perspectives helps you map that frontier and decide what's valuable and what's not. And then the last thing that I think is really important for doing labs is to figure out ways to make your experiments net positive in terms of ROI. So we're going to throw out 90% of what we make. How do we make even that 90% even the even the 90% that doesn't make it into the product eventually, how do we make that actually useful for us?

中文：当能力提升时，前沿领域变得非常未知。因此，让人们从不同角度尝试同样的事情，有助于你绘制出那片前沿区域的地图，并判断什么是有价值的，什么不是。我认为进行实验室工作最后一个非常重要的事情是找到方法，使你的实验在投资回报率方面是净正面的。我们会丢弃我们制作的 90%的东西。那么，即使是那最终没有进入产品的 90%，我们如何让它对我们真正有用呢？

10:00 - 20:00

So, one thing that we've done a lot at every that has worked really well is we turn them into external content. People love to see here's all the things that that we tried, here's what worked, here's what didn't. And then that brings customers to to use to read our stuff, use our products, all that kind of stuff. If that's possible, I highly recommend it. Another thing to do is use these experiments to feed an early adopter program. there are probably a lot of uh customers that you have that want to be closer, want to be in the fold, who are early adopters and this can be a value ad for them to get to work with you more closely.

中文：所以，我们在每一方面做过的一件非常有效的事情是，我们将它们变成外部内容。人们喜欢看到这里是我们尝试过的所有事情，这些有效，这些无效。然后，这会吸引客户来阅读我们的内容，使用我们的产品，所有这些。如果可能的话，我强烈推荐这样做。另一件可以做的事情是利用这些实验来支持早期使用者计划。你可能有很多客户希望更接近，想要加入这个圈子，他们是早期采用者，这对他们来说是一个增加价值的机会，可以更紧密地与你合作。

And then the last thing is like at base the labs team should be sharing what they learn about capabilities and what's now possible with the product team. So the product team can take that into account uh as they build things without getting distracted by having to go, you know, explore the frontier themselves. So let's say you've done that. you're uh you've decided to make a lab. You're starting to figure out uh like all the all the little best practice. Now, how do you get the stuff that you're making into from the lab into your product?

中文：最后一点是，基本上实验室团队应该与产品团队分享他们关于能力的学习成果以及现在产品可以实现的内容。这样产品团队在构建产品时就可以考虑这些因素，而不会因为必须自己去探索前沿而分心。假设你已经做到了这一点，你决定建立一个实验室，开始摸索各种小的最佳实践。那么，你如何把你在实验室中制作的东西转移到产品中去呢？

The thing you need to do is make a research pipeline where ideas start on the left and they start out as lab only and there are lots of little experiments and then you move them progressively from the left to the right and at some point you have a handoff with your product team where they start to get incorporated into the product. So to give you a sense of what this looks like, you've got the the labs team. They're running lots of lots and lots of experiments. Um, and most of them are not good, but a few of them might work.

中文：你需要做的是建立一个研究流程，在这个流程中，想法从左侧开始，并且最初只在实验室中进行，有很多小实验，然后你逐步将它们从左侧向右侧推进，在某个阶段你会和你的产品团队进行交接，他们开始将这些想法整合到产品中。为了让你了解这是什么样子，你有一个实验室团队。他们进行很多很多实验，大部分实验都不成功，但有少数可能有效。

And those become things that you start to test in real work. One of the things that we do all the time internally is we just let other people on the team adopt it and see if they like it. Uh, there are different configurations that work depending on what kind of customers you serve, but the first step for us is do other people on the team start to use it? If that's the case, it's often then ready to put in front of early customers and ready for the product teams to like really take a look at it and figure out where does this fit in our road map and um how might we use this?

中文：这些就成为了你开始在实际工作中测试的东西。我们内部一直做的一件事是，让团队中的其他人采用它，看看他们是否喜欢它。根据你服务的客户类型，不同的配置会有不同的效果，但对我们来说，第一步是看看团队中的其他人是否开始使用它？如果是这样，它通常就可以放在早期客户面前，并准备好让产品团队真正审视它，弄清楚它在我们的路线图中适合的位置，以及我们可能如何使用它。

And uh and even there not everything makes it but like at at some point uh the thing that you do will be ready to scale and release to customers. And we'll go through some examples of how that works. That's the uh the research pipeline. And I'll give you like a little bit of a concrete example. This is Kate, our editor-inchief, who I've been trying to automate for the last 3 years. um very lovingly. Um and Kate Kate is fantastic at a lot of things, but one of the things she's really good at is she has a very good taste for copy edits.

中文：而且，呃，即使在那里，并不是所有事情都会成功，但在某个时候，呃，你所做的事情将准备好进行扩展并向客户发布。我们将通过一些例子来说明这是如何运作的。这就是呃研究流程。我会给你一个具体的例子。这是凯特，我们的总编辑，我在过去三年里一直想要自动化她。嗯，非常有爱地。嗯，凯特，凯特在很多事情上都很出色，但她特别擅长的一件事是她对文案编辑有非常好的品味。

Um but as the team has grown, we're about 30 people now. Uh she can't do all of the copy edits herself. Um and we could hire someone, but it's really hard to get someone who has the level of taste that she has and train them and all that kind of stuff. So, what I've been trying to do is like figure out how do we make how do we make AI like help her with this so she can expand her impact in the or without having to spend additional time doing copy edits really late at night. And so Kate often wakes up to messages like this from me saying not important, but I downloaded every single one of your copy edits over the last 3 years and had Fable try to do a copy edit on the latest piece.

中文：嗯，但是随着团队的壮大，我们现在大约有 30 个人。呃，她不能一个人完成所有的文案编辑。嗯，我们可以雇一个人，但很难找到具备她这种品味水平的人，并且训练他们以及处理所有相关的事情。所以，我一直在尝试的是，想办法让 AI 帮助她，这样她就可以扩大她在工作中的影响力，而不必在深夜花额外时间做文案编辑。因此，凯特经常会收到我发给她这样的信息，虽然不重要，但我下载了过去三年你做的每一条文案编辑，并让 Fable 尝试对最新的作品进行文案编辑。

Um, and I've literally been doing stuff like this for a couple years and it just it just started to work where the capabilities are good enough that it's it's actually like uh Kate ready for Kate to actually use. So we move from we we call it Kate bench. So we're starting to move from okay we've done a bunch of experiments now there's one that's like starting to work now let's push it out into the or see if we can get some internal use. So we may we have a we have an every agent. It's a it's a it's a single agent for your entire company.

中文：嗯，我实际上已经做这类事情好几年了，它就是开始奏效了，现在它的功能已经足够好，实际上就像呃，Kate 可以真正使用。所以我们从，我们称之为 Kate 实验台，开始转变。也就是说，我们已经做了一堆实验，现在有一个开始有效了的实验，我们就尝试把它推出去，看看是否可以在内部使用。所以我们可能……我们有一个，每个代理。这是你整个公司的单一代理。

It helps your company get AI filled and use all of all of uh AI workflows. Agents every.TO. But uh we use it internally. And what what happens now is Kate is starting to be like okay at every do a Kate pass of this draft and it'll go in and it will literally file suggested changes like she would based on her historical edits and then improve over time. And this is something that I built just in my spare time that just it just sort of works. But it's like now we're starting to see that there's some value for the organization cuz she's starting to adopt it and starting to spread to other people and that's a really good sign.

中文：它帮助你的公司实现人工智能填充，并使用所有的人工智能工作流程。代理每个.TO。但我们内部使用它。现在发生的是，Kate 开始像这样：好，让每个执行一个 Kate 审阅这个草稿，它会进入系统，并会根据她过去的编辑提出建议性的修改，然后随着时间改进。这是我在业余时间构建的东西，它就是这样运作的。但现在我们开始看到它对组织有一些价值，因为她开始采用它，并开始传播给其他人，这是一个非常好的迹象。

So when we get to this um internal use stage, this is the time where I would then bring in an architect and I have one in particular who I work with on my team. His name is Yannik. He's fantastic. Um and Yannik is going to Yannik goes and takes this like messy thing that kind of works and makes it really great. So now we have a whole dashboard of okay like for each document how many uh how many suggestions were were accepted and then how much work is remaining uh for Kate to do after uh after we go in and do it. So it checks what are the edits she does after the the agent goes and does the edits.

中文：所以当我们进入这个内部使用阶段时，这正是我会请一个架构师介入的时候，我团队里有一个我特别合作的架构师。他的名字叫 Yannik。他非常棒。Yannik 会去处理这个有点乱但大致可行的东西，并把它做得非常好。所以现在我们有了一个完整的仪表板，例如对于每个文档，有多少建议被采纳，以及在我们介入处理之后，Kate 还剩下多少工作要做。它会检查代理做完修改后，她自己做了哪些编辑。

And you can see we're improving. We did uh she does 12% less work uh on these types of edits than she did uh the month before. and his job is to make it like a real system that we can improve and compound uh in an ongoing way. Now it's starting to work well enough that we're starting to think about okay how do we put this in front of early customers. Does this sort of process work for things beyond copy editing? I think it does but uh you know we've sort of taken it from it's only an experiment in the lab to something that we're ready to be like okay I think we can give this to customers.

中文：你可以看到我们在进步。我们在这些类型的编辑上，她比上个月少做了 12%的工作。而他的工作是把它变成一个真正的系统，我们可以不断地改进和积累。现在它开始运作得足够好了，我们开始考虑如何让早期客户使用它。这样的流程是否适用于文案编辑之外的事情？我认为可以，但你知道，我们已经把它从实验室里的实验，发展到可以说‘好吧，我认为我们可以把它交给客户’的阶段。

So a few best practices for how to continually push things uh through your lab pipeline. One thing that we do that I think is very important is we review the pipeline regularly. We have I have a tracker in notion. Every week on our all hands, everyone talks about what's going on in our pipeline. And I think that's crucially important because what you want to be able to do is you want your product team, the rest of the product team who's not playing around with everything to know here's what we think is interesting.

中文：所以关于如何持续推动实验室流程的一些最佳实践。我们做的一件我认为非常重要的事情是我们定期审核流程。我在 Notion 中有一个跟踪工具。每周的全员会议上，每个人都会谈论我们流程中正在进行的事情。我认为这是至关重要的，因为你希望的是让你的产品团队，其他没有参与所有实验的人，知道这是我们认为有趣的内容。

here's what's starting to move up the pipeline so that you can start to think about okay if this does actually prove out what are the implications for the product without having to then go like scramble to like do your own experiments and all that kind of stuff. It helps keep uh the lab and the pip and the product uh orgs in sync. Another really important thing is defining clear decision criteria for moving an idea through the pipeline and really making it an event when you do it. So some of the decision criteria like our big one is are people using it and are they coming back?

中文：这是正在开始推进的内容，这样你就可以开始考虑，如果这确实得到验证，对产品的影响是什么，而不必然后手忙脚乱去做你自己的实验等等。这有助于保持实验室、管道和产品组织的同步。另一个非常重要的事情是为将一个想法推进管道定义清晰的决策标准，并在推进时真正把它当作一个事件来处理。一些决策标准，比如我们最重要的一条是人们是否在使用它，以及他们是否会回来了？

Because what you're trying to what you're trying to do is use uh internal use as a proxy for value. The second big question is it 10x better than what currently exists. And this is actually a really interesting one a really interesting filter because what is new feels very exciting but the question is when it's a month later is it actually any better? It's hard to know unless you use time and usage over time as a filter. Um, and also is it affordable? Like maybe this works now, but is it could we actually serve it at scale for our customers?

中文：因为你正在尝试做的是将内部使用作为价值的代理。第二个大问题是它是否比现有的东西好十倍。这实际上是一个非常有趣的问题，一个非常有趣的筛选标准，因为新的东西感觉非常令人兴奋，但问题是一个月后它实际上是否更好？除非你使用时间和随时间的使用情况作为筛选，否则很难知道。而且，它是否负担得起？也许现在可行，但我们是否真的能够为我们的客户大规模提供？

Another really big big question. And and having that clear criteria allows you to make sure that you're you're being rigorous about how you do this. But the ultimate goal is that you um merge the winners into the main product. And this is this is this full cycle of labs to merging winners is something that is starting to happen really really well. Like we've had labs for a long time, but the and the idea of doing a self-d disruption of building something that then disrupts your own product has been around for a while, but it is actually starting to work now in AI because you can move so fast.

中文：另一个非常非常重要的问题。而且拥有清晰的标准可以让你确保在执行过程中保持严谨。但最终的目标是将获胜者整合到主产品中。这种从实验室到整合获胜者的完整循环，现在已经开始非常非常顺利地进行。我们已经有实验室很长时间了，而通过自我破坏来构建某些然后影响自己产品的想法已经存在了一段时间，但实际上现在在 AI 领域开始起作用，因为你可以移动得非常快。

I think a really good example is Codex. So, um, Codex built by a small team working outside of the main app. There were a bunch of different teams inside of OpenAI who were working on the future of coding at the same time as Codex was. They had a they did a bunch of different iterations on different form factors for what the future of coding would look like. Is it inside an IDE? Is it a CLI? All that kind of stuff. Um, but they launched their desktop app which uh I think is fantastic. They launched it in February 2026 and it it grew so fast that they eventually just merged it into ChatGPT and it became the foundation of ChatGPT.

中文：我认为一个非常好的例子是 Codex。嗯，Codex 是由一个在主应用之外工作的一个小团队开发的。在 Codex 开发的同时，OpenAI 内部还有很多不同的团队在研究编程的未来。他们对编程未来的不同形式进行了多次迭代。它会在 IDE 里面吗？还是在 CLI 上？等等。嗯，但他们推出了他们的桌面应用，我认为非常棒。他们在 2026 年 2 月推出了它，它增长得非常快，以至于最终他们将其并入了 ChatGPT，并成为了 ChatGPT 的基础。

they the this small little Codex team then just uh took over this 800 million daily active user app because they were so successful with this this way of experimenting. So I think that's the promise of doing something like this is that you can actually build the next version of your product while scaling the one that you already have and you can do it without going crazy. The way that you know this is working is you will welcome moments when new models drop and be excited about them instead of dreading them. And that is my talk.

中文：他们这个小小的 Codeex 团队然后就接管了这个每天有 8 亿活跃用户的应用，因为他们在这种实验方式上非常成功。所以我认为，做类似事情的承诺在于你实际上可以在扩大现有产品的同时，构建你的产品的下一版本，并且可以在不疯狂的情况下做到这一点。你知道这种方法有效的方式是，当新的模型发布时，你会欢迎这些时刻并感到兴奋，而不是害怕。这就是我的演讲内容。

Uh I'm Dan Shipper. I'm the co-founder CEO of Every You should check out Everywhere, the only subscription you need to say at the edge of AI. Thank you very much.

中文：呃，我是丹·希帕。我是 Every 的联合创始人兼首席执行官。你应该去看看 Everywhere，这是你在人工智能前沿所需要的唯一订阅。非常感谢。