对于Kimi K3我还有话说…
作为RRLab的第一个项目RRLab-Bench在之前出过两期内容，但这一次我可能打脸了…
RRLab-Bench has published two issues before, but this time I might have to eat my words...
第二篇内容中Kimi K3的一些技术做了一些了解，知道这是一个基于深度思考为基础的长Agent任务为主要考虑的模型。作为一个开源的2.8万亿参数的模型，其整体表现不可能像我使用的时候的那种感受的。因为现阶段主流的比它低参数的模型和相近参数量的模型我都有使用过，最终感受不会这样。
带着这个疑问，我继续找寻问题的根源…
既然是深度思考为基础，整个Agent工作链中因为深度思考造成了输出时间和Token消耗数都直线上升。那么降低Thinking的级别呢？Kimi K3使用coding plan或者API调用时，Thinking级别有三层：默认是MAX（最大思考强度）、High（中等思考强度）、Low（最低思考强度）。关于这个思考深度强度问题，之前有了解：Kimi其实有计划做动态的思考强度。就是在整个Agent任务中，每一次Agent的思考深度按任务难度做动态调整。期待这个动态思考强度…
第一次的Bench测试所有的模型都是基于最大深度思考级别做的测试。当时，因为Opus5的测试结果实在离谱。所以只对Opus5降低思考级别重做了一次。而对于Kimi K3并没有修改思考深度来做测试。
在Pi Agent里把Kimi K3的Thinking Level调到Low（最低思考强度），然后试用了下…结果，完全改观了！K3在这个时候的工作效率比之前强很多，同时Token的消耗不像之前那样的夸张。至少我现在使用的coding plan可以让我很顺利的完成工作。
In the second issue I learned about K3's technical design — a model built primarily for deep-thinking-based long agent tasks. As an open-source 2.8-trillion-parameter model, its real-world performance shouldn't match my initial impressions. I've used smaller and similarly-sized models and the experience shouldn't be this poor.
With this doubt, I dug deeper into the root cause...
Since deep thinking is the foundation, the entire agent chain sees output time and token consumption spike accordingly. So what about lowering the thinking level? Kimi K3 has three thinking levels for coding plans and API calls: MAX (maximum intensity, default), High, Low. I'd previously heard Kimi plans dynamic thinking intensity — where each agent step adjusts thinking depth based on task difficulty. Looking forward to that...
The first Bench test ran all models at max thinking depth. Back then, because Opus 5's results were absurd, I only retested Opus 5 at a lower level. For K3, I never adjusted thinking depth.
I set K3's Thinking Level to Low in Pi Agent and tried it... The result was a complete reversal! K3's efficiency was far stronger, and token consumption no longer outrageous. At least my current coding plan lets me work smoothly now.
接下来，调整RRLab-Bench里的Kimi K3的思考强度级别。做第二次的模型效率测试…结果反转了！Grok4.5依然在效率和综合成本角度排第一，Kimi K3在这次测试时直接冲到第二名，直接超过了Deepseek V4 Pro。Deepseek V4 Pro唯一还占优势的是完成任务的成本比K3低…
Next: adjusted K3's thinking level in RRLab-Bench and ran a second round of efficiency tests... The results flipped! Grok 4.5 still ranked #1 on efficiency and total cost, but K3 shot to #2, surpassing DeepSeek V4 Pro. DeepSeek V4 Pro's only remaining edge: lower per-task cost than K3...
调整Thinking Level为Low后，完成任务的Agent总轮次降低了27%、总耗时降低了60%、Token消耗降低了67%。（RRLab-Bench测试时每题测试三次）
After setting Thinking Level to Low: total agent rounds dropped 27%, total time dropped 60%, token consumption dropped 67%. (RRLab-Bench runs each task 3 times.)
三个不同场景中，fix-bug的效率提高最多…
Among the three scenarios, fix-bug saw the biggest efficiency gain...
看下第二次测试后Kimi K3在9个模型中的位置：
K3's position among 9 models after the second test:
从成本和效率角度，Kimi K3在Thinking为Low状态下，已经进入了优势区间。
From a cost and efficiency perspective, K3 at Low thinking has entered the advantage zone.
完成任务的Token消耗数也已经是第一梯队
这是一篇更新自己错误的文章。尽管很早就开始做多模型协同，也知道各模型有自己的特长。但在横向比较测试中忽略了Thinking Level对模型输出质量、成本效率的影响。
一直主张使用国产模型，这是因为见多了闭源模型的傲慢和对开源模型的展望。之前是真的国产模型没有能顺利完成我的日常工作，但自从Deepseek V4的出现，之后的GLM5.2，直到Kimi K3的出现。一直希望国产模型有一个可以和Opus、GPT这些顶级大模型抗衡的出现，至少在我们平常人使用角度可以不再承受高昂的Token价格和被人掣肘的憋屈。现在有了…期待Kimi K3的动态思考模式的推出，到时在完成Agent任务的效率和成本上应该会有更好的表现。
Token consumption is now also in the top tier.
This is an article correcting my own mistake. Though I've been doing multi-model orchestration for a while and know each model has its strengths, I overlooked how Thinking Level impacts output quality, cost, and efficiency in cross-model comparisons.
I've always advocated for Chinese models, having seen enough of closed-source arrogance and believing in open-source's promise. Before, no Chinese model could smoothly handle my daily work. Then DeepSeek V4 appeared, followed by GLM 5.2, and now Kimi K3. I've always hoped for a Chinese model that could stand against Opus and GPT — at least so ordinary users don't have to endure sky-high token prices and being held hostage. Now, that model exists... Looking forward to K3's dynamic thinking mode — agent task efficiency and cost should see even better performance then.