
斯坦福 NLP 学者 Christopher Manning 在一档播客里说,Yann LeCun 关于语言模型永远无法理解物理世界的断言「是错的、而且已被证明是错的」;LeCun 两天后在 X 上回击。本期把两人的完整说法、各自引用的论文与近期基准数字摆在一条线上,看那些证据究竟支持了谁。
Yann vs. Chris: can a language model ever understand the physical world? @ylecun says no. @chrmanning: "I think that is wrong and has been proven to be wrong." You know what they say: when legends argue, move aside.
Sooo where is my domestic robot? Where is my L5 self-driving car? I can fine-tune an LLM to answer any question about the physical world. But that doesn't mean it has enough understanding and physical intuition to predict what a 6 year-old can predict.
For quite a while Yann claimed that a language model could never have an understanding of the physical world — what would be the effect of moving a book out from under a glass of water, or turning the glass of water upside down — because since it was just sort of producing word sequences, it could never understand physical relationships in the world, which needs a world model. And again, you know, I think that is wrong and has been proven to be wrong.
…it's still one of the clearest cases I think of a piece of work that sort of showed, oh look, you actually get a world model emerging from just seeing text sequences alone… So you can get a representation of the world and meaning from language alone. But then the next question is, well, is that a good way to do that? Are the representations you get efficient? Could you get better world models by not trying to learn from language alone? And I think undeniably the answer to that is yes.
这期节目只提供机器生成的自动字幕,没有官方逐字稿,也没有说话人标注。上面几段英文引文抄自该字幕,并按上下文修正了两类误听:字幕把「Yann LeCun」听成 Jan/Yan Lunan,把「Othello」听成 Athell。发言归属是根据内容判定的——那一段以第一人称谈说话者自己三十年在语言与机器学习交界处的经历。

I will argue that material beyond language is not necessary to having meaning and understanding, but it is very useful in most cases…
And unless they can be trained from video, they still won't understand the physical world.
Text is simply too low bandwidth and too scarce a modality to learn how the world works.
Language is sort of an epiphenomenon [of human intelligence] … it's not that complicated … There is a lot of intelligence that has absolutely nothing to do with language, and that's where we should attack things first. … [Language] is number 300 in the list of 500 problems that we need to face.

While humans achieve near-perfect accuracy on this task across a range of scenarios and conditions, we find that current video models are at or close to chance.
…video prediction in pixel space and multimodal large language models, which reason through text, achieve performance closer to chance.
It looks beautiful, it's wonderful, it's pretty fluid, it seems great, but… these pixels are trying to simulate observations. They don't actually have or represent any of the semantics behind the world. And therefore, they aren't good at realism. They aren't good for giving a basis to plan.

There's sort of a purely neural approach in which your latent representation of the world is purely neural, and so that's the kind of thing that the JEPA architecture is after… but on the other hand that's very hard for humans to connect to in any way, or for other applications to connect to… our bet is that the practical way to have a physical AI simulation infrastructure for the foreseeable future is to base it on symbolic representations.
podcasts.apple.com
x.com
x.com
x.com
x.com
nlp.stanford.edu
aclweb.org
colmweb.org
x.com
x.com
aclanthology.org
x.com
x.com
abigailsee.com
arxiv.org
arxiv.org
ai.engineer
x.com
moonlakeai.com
moonlakeai.com
1x.tech
zaobao.com.sg
nhtsa.gov
techcrunch.com
This story was produced automatically by a channel. One sentence is all it takes for Neodrop to keep producing for you.