Nice work! While I'm not sure whether this design is practical enough on real hardware, the sparsity in both token and channel dimensions for attention is very elegant.
Replying to @TianzheC
Nestor is more compute-efficient than a matched-sparsity MoE baseline in both pretraining and long-context continued pretraining.
Xinyu Yang retweeted
我不知道大家怎么理解ETH这件事。
我还认识一些TCS领域的人。有些老教授的此刻正在经历精神世界乃至大半生心血被时代巨浪瞬间重塑的极限冲击,这样说也毫不为过。自从数十年前,这一假设被提出以来,ETH 就一直是理论计算机科学中,用来证明某个问题到底有多难、算法下限在哪的最核心的基座之一。
插一句,其实还有SETH,强指数时间假设。如果ETH完蛋了,那SETH也完蛋了。ETH和SETH就是算法复杂度理论的“牛顿三定律”级别的基座。如果这些都是错误的,那很多技术护城河,就是纯废铁。
在学术界,有多少篇的论文都是建立在假设 ETH 为真的前提下展开推导的?
答案是,很多很多。推特已经有人做了论文的总结。ETH彻底证明错误将失去任何价值的论文。ETH这个大前提被 AI 证伪了,意味着成千上万篇基于它的复杂度分析、衍生证明、理论模型,在逻辑的源头上瞬间失效,彻底失去意义。
你现在和ChatGPT、Claude聊,它可能会假惺惺地安慰你,说这是科学的必由之路,是人到机器火炬的传递。是的,我承认它们说的在某种意义上没错,但是这是科学本身的递进本质。我从来不否定对于整体人类和科学进步来说,ETH错误确实是一场伟大的纠错,但对于被大厦崩塌砸中的每一个活生生的个体来说,那些数十年的通宵不眠、熬干的心血、甚至为此放弃的无数个人生活,在一夜之间沦为废纸,其代价就是成为AI时代的炮灰。
在科学的漫长进程中,这种大厦一夜倾覆的个体悲剧曾无数次上演。当爱因斯坦证明绝对时空观错误时,无数研究了一辈子经典力学的物理学家在一夜之间失去了方向;当以太假设被证伪时,那些为之付出大半生心血的科学家也曾陷入同样的虚无与绝望。现在,我们仿佛就在那个节点上。
不想传播焦虑,但这次我无话可说。
今年科学界最大的新闻,没有之一。OpenAI 的内部 AI 模型自己做出了三个科学难题的成果。我用我自己的理解说一下。
第一,Kakeya 猜想。这个问题研究波聚焦时会怎样。原来的猜想我记得是说最极端的聚焦只会持续很短时间。人类以前只在低维度证明过,OpenAI 说它的模型证明了所有维度的情况,因此顺带解决了 Kakeya 猜想和 Stein 限制猜想。纯逆天。
第二,指数时间假说 ETH 被推翻。这个太难以置信了。ETH 好像是说有一类逻辑难题不可能被比暴力穷举快很多地解决。而之前学界普遍相信它成立,几百篇论文都以它为前提,现在 AI 却证明它是错的。
这个太他喵逆天了,因为我之前是研究过一段时间理论计算机的。这几十年ETH几乎被当成公理在用。很多论文的结论都写成假设 ETH 成立,那么某某问题不可能在多少时间内解决。现在地基没了,这些不可能,全都要打问号。
最让我感觉匪夷所思的是 AI 证明的方式。据 Lean 证明的说明文档,它给出的是一个确定性算法,应该是对固定宽度的 k-CNF 公式,它能在亚指数时间、多项式空间内精确数出所有满足的赋值。但是,懂行的人应该都知道“数出有多少个解”通常比“判断有没有解”难得多,它连更难的那个都做到了,所以 ETH 只是顺带被推翻。
第三,杨-米尔斯、量子色动力学。AI 应该是给出了一套严格的数学证明,证明描述强核力的理论在数学上确实成立,其中包括纯杨-米尔斯理论的存在性,这是千禧年七大难题之一的一部分。我不太了解,但是只是知道一些浅层的理论,就不评论了。
最让我恐惧和惊骇的是第二个。今天晚上我看了又看,感觉这是地震级别。如果最终确认无误,这会是 AI 第一次独立推翻一个被广泛相信的核心猜想。说它是一个时代的分界点,也毫不夸张。
如果这三个都核查通过,这代表 AI 第一次在多个顶尖领域独立解决了几十年没人能解决的难题,影响史无前例。
Xinyu Yang retweeted
Replying to @wenhaocha1 @sun_hanchi
Tested K3.1 on 4 discriminative questions from Frontier CS 2 across different thinking budget and harnesses for an upcoming eval video.
Genuinely a fantastic benchmark! 🥳
World models that see only through the players' eyes forget what happens out of view.
OverSeer adds a few fixed cameras and generates every view together, so the world stays consistent over time and across players.
📄 arxiv.org/abs/2610.28554
🌐 runjiali-rl.github.io/OverSe…
🧵👇
Xinyu Yang retweeted
Kimi K3.1 Pro enters MathArena as the best open model (once they release the weights)! Its estimated overall performance is now similar to GPT-5.5 (April 24), putting the open frontier around 6 months behind on MathArena
Xinyu Yang retweeted
We're sorry for the rate limits and unnecessary quota consumption on the K3.1 Pro subscription plan. These were our issues, and we should have fixed them sooner.
Today, every subscriber on every plan gets a one-time usage reset, restoring 25% of their monthly quota (usage below 25% resets to zero), equal to one week of usage. Your usage bar will reflect this adjustment by end of day PST.
We've also adjusted the plan's parameters and are reducing unnecessary subagent calls and output length, so your quota goes further.
Thank you for your patience while we put this right.
卧槽,3090要可能要涨价了!!
因为老黄的刀法被人破解了
有人给3090解了个锁,同样限制在 360W,PyTorch 测试从 75 TFLOPS 跑到了 135 TFLOPS,提升约 80%!
你以为是超频,把功耗往上拉?
但他改的是显卡里控制运算速度的寄存器,这些寄存器平时有权限保护,正常改不了
他利用 GPU 启动固件的漏洞,赶在 NVIDIA 驱动加载之前拿到写入权限,把相关运算改成全速,再交给原版驱动使用
解开的主要是 FP16/BF16 输入、FP32 累加这类 AI 运算的速度限制 但测试算力多了 80%!!
4090 也有分支项目做到了:同类运算的指令吞吐,从约 165 TFLOPS 提升到 331 TFLOPS,接近翻倍
项目地址,附上👇
github.com/Zagitta/gpu-unloc…
Xinyu Yang retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
Xinyu Yang retweeted
DeepInfer set the bar. SGLang pushed it higher! 🚀
SGLang now matches DeepInfer performance on BOTH prefill and decode:
• Prefill: Up to 1.04× throughput on 4× GB300 and 4× B300
• Decode: Up to 7.4% throughput on 64× GB300
We also provide an end-to-end reproduction guide and a full serving stack to run it in production.
Xinyu Yang retweeted
I asked #KimiK31 to deconstruct a 911, and turn every component into cinema.
The result: 30 seconds of AI-generated visuals, precision motion graphics, and a pulse-built soundtrack.
Concept film, not affiliated with Porsche AG
Xinyu Yang retweeted
I have to ask people not to spread this slop
@scut_longhui explains their policy pretty candidly
I disagree with it (If datasets are conflicting and one has strictly richer signal for your purposes, discard the other and clearly state the provenance), but it is what it is.
🤯炸了!梁文峰的实力被Kimi用“图表技巧”藏起来了!同一组标准Kimi竟然偷换数据来凸显自己!
1️⃣Kimi官方把DeepSeek V4.1 Flash 1017画成几乎看不见的小红点,自己则高高在上。表面上比分数Kimi 65.7% vs DS 42.3%,真正被弱化的是成本,Kimi K3.1 Pro单任务成本大约是DeepSeek的20倍。
2️⃣更关键的是Kimi官方将DeepSeek的TB4.0数据给偷换了,来源不一致,表格用DeepSeek自己公布的42.3,图表却单独换了Artificial Analysis的数据。
DeepSeek开源、极便宜、迭代快;
有人选择用“小点”把它藏起来。
分数好看,成本效率才是落地关键。
#DeepSeek #Kimi #TerminalBench #梁文峰 #开源AI #成本效率 #V41Flash1017 #K31Pro
Replying to @uniartisan
Happy one Kimi year anniversary!
Xinyu Yang retweeted
Kimi K3.1 Pro is very good at 3D modeling.
I asked Kimi K3.1 to create a 3D website that pulls apart a robotic hand into 2,317 pieces. It did pretty well!
You can just build things.
Project link: form-01-score.ok.kimi.link/
Xinyu Yang retweeted
to prepare for an upcoming release (👀) we dug into the pretokenizer a little more. are people aware that hand-crafted regexes are part of every frontier LLM and doing some pretty heavy lifting?
some quick examples 🧵
Xinyu Yang retweeted
Kimi K3.1 Pro launches at 4th on the Artificial Analysis Intelligence Index scoring 55, making large gains in agentic performance; it will be the leading open-weights model once its weights are published
@Kimi_Moonshot's new reasoning model gains 11 points over Kimi K3 (max, 44). It trails only Claude Opus 5.5 (max with fallback, 58), Claude Sonnet 5.5 (max with fallback, 56), and Muse Wonder Full (max, 56) on the Intelligence Index.
Once the weights for K3.1 Pro are released it will be the highest-scoring open weights model and the gap between the leading open weights and proprietary models on the Intelligence Index will narrow from 12 points to 3.
Key results:
➤ Large gains in agentic performance: Kimi K3.1 Pro scores 3rd on GDPval-AA with 1,846 Elo, up 312 Elo from Kimi K3, behind only Claude Opus 5.5 (1,866 Elo) and Muse Wonder Full (1,857 Elo). Kimi K3.1 Pro also achieves the highest score of any model on Terminal-Bench 4.0. K3.1 Pro scores 67%, up from 13% for Kimi K3 and ahead of Claude Sonnet 5.5 (64%). AutomationBench-AA, which tests agentic SaaS workflows, rises from 58% to 73%.
➤ Gains in other evals are more moderate: Humanity's Last Exam rises from 47% to 52%, CritPt from 23% to 25%, and SciCode from 59% to 61%. AA-Omniscience Index rises from 20 to 30, as the hallucination rate falls from 53% to 44% and accuracy rises from 48% to 51%.
➤ Cost per Task roughly equal to Kimi K3 at $1.98 per Task: K3.1 Pro uses 55K output tokens per Intelligence Index task, 14% more than Kimi K3 at 48K, while its first-party token prices are 33% lower.
Additional model details:
➤ Context window: 1M tokens.
➤ Size: 2.8T total, 104B active parameters
➤ Input modalities: Text and image
➤ Weights and license: Weights currently not published, Moonshot is expected to release the weights soon after launch
➤ Pricing: $2/$10/$0.20 per 1M input/output/cache hit tokens, 33% lower than Kimi K3 on all three rates.
Congratulations to the team at @Kimi_Moonshot on the release!
Come give it a try 👀
Introducing Kimi K3.1 Pro: our most capable 3T-class open-weight model.
🔹 1M-token context, 2.8T total / 104B active params (MoE), Multimodal.
🔹 Full-project coding: games in Unreal, Unity & Godot.
🔹 Excels at parametric 3D modeling in Blender & CAD.
🔹 More reliable at finance work: Accurate modeling with hallucination down by 65%.
🔹 Matching the strongest proprietary models across coding, agentic and knowledge benchmarks.
Kimi K3.1 Pro and K3.1 Pro Highspeed (2–3× faster) are now live on Kimi.ai, Kimi Work, Kimi Code, and the Kimi API.
Open weights by Nov 4, 2026.
🔗 API: platform.kimi.ai
🔗 Tech blog: kimi.ai/blog/kimi-k3-1-pro






























