31
0Episode: Zhipu Tang Jie: Scaling Beyond Parameters
Duration: approximately 8 minutes
Level: B1 (Intermediate)
---
[Mike]: Welcome back to Learn English with Podcasts! Sarah, if a company told you their new AI has 1 trillion parameters, would you say wow, it must be the best?
zh:欢迎回到 Learn English with Podcasts！Sarah，如果一家公司告诉你他们的新 AI 有 1 万亿参数，你会说哇，它一定是最好的吗？
[Sarah]: A few years ago, I would say yes. Bigger means smarter, right? Like a bigger brain?
zh:几年前我会说是的。更大意味着更聪明，对吧？就像更大的大脑？
[Mike]: That is exactly what Zhipu founder Tang Jie wants to challenge. He just posted his Thoughts About Scaling Law, and his first line is simple - Scaling, but not just parameters.
zh:这正是智谱创始人唐杰想挑战的观念。他刚发布了关于 Scaling Law 的思考，开头就很简单——要 Scaling，但不只是参数。
[Sarah]: So scaling is still true, but we are scaling the wrong thing?
zh:所以 Scaling 依然成立，但我们一直在扩大错误的东西？
[Mike]: We are scaling an incomplete picture. Tang Jie says reporting only the parameter number tells you almost nothing today. You also need to ask: how much data, where you spend your compute, and how the model will run every day.
zh:我们看到的是不完整的图景。唐杰说，今天只报参数量几乎说明不了什么。你还得问：用了多少数据、把算力花在哪，以及模型每天是怎么运行的。
[Sarah]: Okay, give me the history. When did we start thinking bigger is better?
zh:好，给我讲讲历史。我们什么时候开始觉得越大越好？
[Mike]: It started in 2020. Researchers led by Kaplan tested many model sizes and found a rule. When you have more compute, you should grow parameters faster than data. About 2.7 to 1.
zh:从 2020 年开始。Kaplan 带领的研究者测试了很多模型尺寸，发现一条规律：算力更多时，参数应该比数据涨得更快，大约是 2.7 比 1。
[Sarah]: And the industry listened?
zh:然后行业就照做了？
[Mike]: Completely. That rule set the race. GPT-3 came with 175B parameters. Then Gopher, then MT-NLG with 530B. Everyone started chasing 1 trillion.
zh:完全照做。这条规则定下了竞赛方向。GPT-3 带着 175B 参数登场，然后是 Gopher，再到 530B 的 MT-NLG。大家都开始追逐 1 万亿。
[Sarah]: I remember those headlines. Every launch was look, we are bigger!
zh:我记得那些标题。每次发布都是看，我们更大了！
[Mike]: And then in 2022, DeepMind did the test again. But bigger. They trained over 400 models, from 70M to 16B parameters, on 5B to 500B tokens.
zh:然后在 2022 年，DeepMind 又把这个实验重做了一遍。但规模更大。他们训练了超过 400 个模型，参数从 70M 到 16B，数据从 50 亿到 5000 亿 Token。
[Sarah]: 400 models just to check the math? That is serious.
zh:为了验算数学就训了 400 个模型？真拼。
[Mike]: And they found we were wasting resources. Models had too many parameters and not enough data. So they built Chinchilla.
zh:然后他们发现我们一直在浪费资源。模型参数塞得太多，吃进去的数据却不够。所以他们造了 Chinchilla。
[Sarah]: Little Chinchilla versus big Gopher? I know this story.
zh:小小的 Chinchilla 对大大的 Gopher？我知道这个故事。
[Mike]: Exactly. Chinchilla is only 70B parameters, 4 times smaller than Gopher at 280B. But it was trained on 1.4T tokens, 4 times more data. With the same training compute, the smaller, better-fed animal won. It beat Gopher, GPT-3, Jurassic-1, and MT-NLG.
zh:没错。Chinchilla 只有 70B 参数，比 280B 的 Gopher 小 4 倍。但它用了 1.4T Token 训练，多 4 倍数据。用同样的训练算力，更小但吃得更饱的选手赢了。它击败了 Gopher、GPT-3、Jurassic-1 和 MT-NLG。
[Sarah]: So the new rule became 20 tokens per parameter, and params and data should grow together?
zh:所以新规则就成了每个参数对应 20 个 Token，参数和数据应该一起长？
[Mike]: Yes. Chinchilla Scaling Law. But Tang Jie says even that was not the final answer.
zh:对，这就是 Chinchilla Scaling Law。但唐杰说，那也不是最终答案。
[Sarah]: Why? Did we find another missing piece?
zh:为什么？我们又发现了缺失的一块？
[Mike]: Because training is only done once, but running the model happens billions of times a day. A 1 trillion parameter model needs huge compute every single time you ask it something. Chinchilla did not count that daily bill.
zh:因为训练只做一次，但运行模型每天要发生数十亿次。1 万亿参数的模型每次你提问都要消耗巨大算力。Chinchilla 没算这笔日常账单。
[Sarah]: Ah, the inference cost. The price after you buy the car.
zh:啊，推理成本。买车之后的使用成本。
[Mike]: Perfect metaphor. Some studies looked at the full life. If a model will handle about 1B requests, a smaller model trained longer can be cheaper overall. They pushed training to 10,000 tokens per parameter and quality still kept improving.
zh:这个比喻完美。一些研究看了全生命周期。如果一个模型要处理约 10 亿次请求，一个更小但训练更久的模型总成本可能更低。他们把训练强度一路推到每个参数 1 万个 Token，质量还在提升。
[Sarah]: Can you give me real numbers?
zh:能给我真实数字吗？
[Mike]: Sure. Llama 2 7B was trained on about 290 tokens per parameter. Gemma 2 9B reached about 889 tokens per parameter. Way beyond the old 20.
zh:可以。Llama 2 7B 每个参数约对应 290 个训练 Token，Gemma 2 9B 更达到约 889 个。远超当年的 20。
[Sarah]: So from fix the size, to feed it more data, to count the daily running cost. The best place to spend compute keeps moving.
zh:所以从扩大参数，到补足数据，再到计算日常运行成本。最值得花钱的地方一直在变。
[Mike]: Exactly. And for today's MoE models, it moves again. Tang Jie splits two ideas: total params and active params.
zh:没错。而对于今天的 MoE 模型，它又变了。唐杰把两个概念分开：总参数和激活参数。
[Sarah]: Like a warehouse and the workers inside?
zh:像仓库和里面的工人？
[Mike]: He uses a similar picture. Total params is the warehouse - how much knowledge you can store. Active params and effective depth is how much power you use in one go, how far you can think in one chain.
zh:他用了类似的比喻。总参数是仓库——能装多少知识。激活参数和有效深度是单次能调动多少能力，能把一条推理链走多远。
[Sarah]: So you need both. Lots of books is not enough, you must read them well.
zh:所以两者都需要。藏书多不够，还得会读。
[Mike]: And bug hunting shows the difference. Remembering many CVEs means you saw many cases. But to find a new bug, you must follow the code for 20 steps - from strange code to how it breaks, to how to test it - without losing the thread.
zh:而抓漏洞最能看出区别。记住很多 CVE 意味着你见过很多案例。但要发现一个新漏洞，你得把代码跟上 20 步——从异常代码到触发条件再到验证方法——全程不能掉线。
[Sarah]: Twenty steps without getting lost. That is not memory, that is focus.
zh:20 步不掉线。那不是记忆力，是专注力。
[Mike]: So Tang Jie says you cannot judge that long-chain skill by total size alone. And that brings us to GLM-5.3.
zh:所以唐杰说，你不能只用总规模去判断这种长链能力。这就说到了 GLM-5.3。
[Sarah]: The new Zhipu model everyone is talking about?
zh:就是大家在聊的智谱新模型？
[Mike]: Yes. For GLM-5.3, Zhipu kept the base, the architecture, total params and active params all the same as GLM-5.2. No bigger warehouse. For the past month, they put new compute into long-horizon environments and reinforcement learning.
zh:是的。做 GLM-5.3 时，智谱保留了 GLM-5.2 的基座和架构，总参数和激活参数都没变。没有把仓库变大。过去一个月，他们把新增算力都投进了长程任务环境和强化学习。
[Sarah]: So same body, better training after school?
zh:所以身子没变，放学后加练？
[Mike]: Great way to say it. The gains came from post-training. Official numbers: Terminal-Bench 3.0 went from 4.6 to 28.3, DeepSWE from 46.2 to 66.9, and Agents' Last Exam from 23.8 to 28.5.
zh:说得太好了。提升来自后训练。官方数字：Terminal-Bench 3.0 从 4.6 升到 28.3，DeepSWE 从 46.2 涨到 66.9，Agents' Last Exam 也从 23.8 升到 28.5。
[Sarah]: That jump on Terminal-Bench is huge!
zh:Terminal-Bench 那个涨幅也太大了！
[Mike]: It is. Now, these are official tests, outside teams will need to check again when weights are open. But for Tang Jie, it proves a point - scaling has many knobs.
zh:确实很大。当然这些目前主要是官方测试，外部复现还得等权重开放。但对唐杰来说，这已经证明一点——Scaling 有很多旋钮。
[Sarah]: Many knobs, you do not have to turn them all at once.
zh:很多旋钮，不用一次全拧。
[Mike]: Exactly. GLM-5.3 turned the post-training knob because that knob had the most room left. Next time, Zhipu might turn pre-training or mid-training or compute per forward pass.
zh:没错。GLM-5.3 拧了后训练这个旋钮，因为当时这里剩余空间最大。下一次，智谱可能再去拧预训练、中训练或单次前向算力。
[Sarah]: So the lesson is not bigger is dead, but single-number bigger is dead.
zh:所以教训不是越大越没用，而是只看一个数字的越大已经过时了。
[Mike]: Beautiful summary. The game is moving from who has the biggest library to who can finish a real job - work for hours or days like a digital employee, without dropping the ball.
zh:总结得漂亮。竞赛正从谁的藏书最大，转向谁能把一份真实工作做完——像数字员工那样连续工作几小时、几天，不掉线、不跑偏。
[Sarah]: If the model only answers one question, size still sounds cool. If it must work all day for you, you care if it can stay on track.
zh:如果模型只答一道题，参数大小听起来还很酷。如果它得为你干一整天，你更关心它能不能一直靠谱。
[Mike]: And Tang Jie closes with a calm line: 1 trillion params has not disappeared. But from now on, it may not be the only number worth putting in the headline.
zh:唐杰最后也平静地收了尾：万亿参数没有消失。只是从现在开始，它可能不再是唯一值得写进标题的那个数字。
[Sarah]: That hits me. I used to check the number like a price tag. Now I will ask, and how long can it actually work?
zh:这句话点到我了。我以前像看价签一样看参数。现在我会问，它到底能干多久？
[Mike]: And that is the quiet joke of this story. For three years we counted params like height. Turns out AI is not a basketball team. The tallest player does not always win the marathon.
zh:这也是这个故事里安静的幽默。三年里我们像量身高一样数参数。结果发现 AI 不是篮球队，最高的不一定赢得了马拉松。
[Sarah]: I love that. Okay, listeners, what would you check now before you pick a model - the warehouse size or how far it can run? Tell us!
zh:太喜欢这个比喻了。好，听众们，现在让你选模型，你会先看仓库大小，还是看它能跑多远？告诉我们吧！
[Mike]: Thanks for listening to Learn English with Podcasts. Try asking your next AI not how big it is, but how many steps it can stay with you.
zh:感谢收听 Learn English with Podcasts。下次问你的 AI 时，别问它多大，问问它能陪你走多少步。
[Sarah]: See you next time!
zh:下次见！
