Stephane Kasriel retweeted
Introducing *HackBench* 🤡💀⏱️📊 ⏲️ 💪🔧 🔥
- Detecting & localizing reward hacking in autoresearch agents
- Frontier models fail
- Dataset on HuggingFace
- Recipe to collect natural & synthetic hacks
Blog post: facebookresearch.github.io/R…
🧵1/5
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
Stephane Kasriel retweeted
✨Happy to share the release of technical report for Muse Glimmer, an open-weights 30B dense LLM that we built with emphasis on local agentic use. Dig in for plots and numbers on:
🪜 Architecture and scaling ladders
📊 Scorecard across 39 (!) benchmarks
🌍 Multilingual evaluations with Muse Glimmer supporting 137 language varieties
🧱 Pretraining and mid-training, including checkpoint averaging
🎯 Post-training: SFT, on-policy distillation, and multi-objective RL
🛡️ Privacy and prompt injection robustness baked into post-training
🏭 Synthetic data generation pipeline
⚡ Inference optimizations: quantization and speculative decoding
Looking forward to the discussions and implementations this report sparks 🚀
Today we published a technical report on Muse Glimmer, our 30B open-weight model built for agentic tasks on consumer hardware.
It runs at practical speeds on a Mac or PC with a single consumer GPU at 4-bit precision, and leads comparably-sized open models on tool use and multi-step planning.
Read about the model architecture, distillation recipe, quantization and more: go.meta.me/4Bcvxol
Stephane Kasriel retweeted
this is what we in the business call framemogging
Deutsche Bank: Muse remains significantly ahead ChatGPT in terms of downloads and DAUs through 36 days since launch.
$META
Stephane Kasriel retweeted
We trained our Muse foundational models, including Muse Wonder, at Prometheus, our gigawatt-scale supercluster in New Albany, Ohio.
Across a supercluster with hundreds of thousands of GPUs, even one underperforming host can slow an entire training run. We engineered Prometheus for resilience at scale, achieving 85%+ goodput on pretraining, 90%+ on reinforcement learning and ~47K GPU hours mean time between interruptions during Muse Wonder training.
Take a look at the tech powering Prometheus: go.meta.me/4yXXApZ
Stephane Kasriel retweeted
A video is worth a thousand tokens
after an exciting week of launches (muse wonder [full] & muse spark open), i want to write a note of appreciation to the amazing research teams at meta superintelligence labs on reaching such a special milestone ♥️
we started MSL and TBD a little more than a year ago and set a goal of producing a frontier model within a year or so. given where llama4 landed, that was an incredibly audacious goal, and we knew it wouldn't be easy. many people told us it would be impossible. we knew it was unlikely, but still worth doing. a remarkable group of researchers and engineers joined the effort from inside and outside meta. every single person came in, picked up a shovel, and got to work with earnestness and urgency. nobody asked whose job it was.
we assembled a team of the most brilliant people i have ever had the opportunity to work with (and remember, i was an HRT intern 🙃). i am surrounded by ambitious, visionary, technically excellent, hard-working, pragmatic, thoughtful scientists, engineers, and product folks—people who will chase a weird loss spike past midnight just because they need to know why. these models are the product of their passion, hard work, and brilliance.
somewhat paradoxically, when we started TBD, we set a principle of no deadlines. we knew we had an astronomical amount of work ahead of us—across pretraining, post-training, RL, multimodal, infrastructure, safety, data, AND frontier research. there was no use setting arbitrary timelines—we just had to get to work, be thoughtful, and build the foundations of a frontier research effort in record time. and run after run, this team did exactly that.
our researchers are magicians. i can only describe building these models with this team as a wondrous journey of scientific discovery. i am constantly amazed by what we can accomplish with scaling. i will remember this past year for the rest of my life—the late nights, the broken runs, the evals that came back better than any of us dared to hope. watching this team create wonder after wonder has been a true and deep gift.
more than just the models we released this week, we have built a machine that builds new models. it often sounds like a cop-out to say labs can just turn the crank to keep scaling, but now i have seen it first-hand. sure, it's a lot of work and a careful process, but we have built a scalable system to keep advancing at the frontier, and i am really proud of how fast this team keeps making these models better.
lastly, i get to work with researchers who care deeply about how their work benefits humanity. it's easy to forget when everything is changing so fast, but the whole damn point is to make people's lives better. time and time again, i have seen our researchers, engineers, and product folks build our technology with care so it can truly be good for the world—whether that's building the best models for muse, thoughtfully open-sourcing strong models, or making sure the alignment and security of our systems are rock solid. this is a team of people who never forget there's a real person on the other side of every prompt.
thank you to everyone at MSL for being part of this crazy journey over the past year. i'm so grateful for every one of you, and our work is not done!
Stephane Kasriel retweeted
The attached blog post lays the groundwork for a major project in mathematics. I think the repercussions of this enterprise will be vast and significant. If we manage to bring all existing mathematics into a single unified library, it will open up the possibility of making mathematical exploration much more efficient. It should also have a deep impact on mathematical education. A robust library will make it possible to fully embrace the prospect of having an autonomous agent that can guide a human along whatever path is requested. (1/n)
xenaproject.wordpress.com/20…
This is a big deal. Can’t wait to see how the community uses this model and fine tunes it with their own data to build next generation AI applications.
Today we’re releasing Muse Spark Open, the most capable US open-weight model to date.
Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoning.
Key Model Specs:
→ 816B total / 42B active parameters
→ 1M token context window
→ Natively multimodal across text, image, video, audio
Stephane Kasriel retweeted
1/ muse spark open is the most capable US open-weight model to date.
816B total params, 42B active. 1M-token context. natively multimodal. built for coding and agentic work like long-horizon tasks, hard software engineering, multi-step reasoning.
america! 🇺🇲
Stephane Kasriel retweeted
Muse Spark Open -- weights releasing today -- runs great on new Mac Studios thanks to work with @johnternus and Apple.
Stephane Kasriel retweeted
Dear Secretary Bessent @SecScottBessent,
My name is Everett Stern. In 2010 I was a compliance officer at @HSBC. From a back office in New Castle, Delaware, I watched billions of dollars move to drug cartels, sanctioned regimes, and terrorist organizations. The enemies of the United States. I documented it, I escalated it, and when the bank would not act, I took it to the @FBI, the @CIA, and eventually the @SECGov.
The result was a $1.92 billion settlement. At the time, it was one of the largest in American history. It was also about five weeks of the bank’s profit. Not one banker was charged. Not one went to prison.
I paid a different price. I lost my career, my savings, and my place in the industry I had trained for. I was blacklisted. Every door in American finance closed at once. I slept on a camping cot in a 300 square foot apartment because I could not afford a bed, and I waited tables at @PFChangs to eat. I tell you this not for sympathy, but so you understand that I know exactly what it costs to do the right thing in American finance, and exactly how little the system does in return.
I made that sacrifice because I believed it would change something.
Yesterday, your department fined @BankofAmerica $1.2 billion for failing to report more than $5 billion in suspicious transactions, some of it tied by your own findings to Chinese laundering networks, narcotics, and the southwest border, alongside 475 apparent sanctions violations touching Russia, Iran, and global terrorism. The bank admitted it willfully violated the Bank Secrecy Act.
Fourteen years after HSBC. A bank with “America” in its name. And once again, not one individual charged.
Secretary Bessent, I want to say plainly what I believe you already know. The fines are not working.
A fine is paid by shareholders, absorbed by the balance sheet, and forgotten by the next earnings call. Bank of America reported earnings this morning. Nobody who set the alert thresholds, nobody who decided the 2024 cease and desist could wait, nobody who chose the backlog over the law will lose a single thing they value. Their lives are untouched. That is the entire reason the behavior repeats.
This is money laundering. This is blood money. Money that reaches a cartel buys fentanyl that kills Americans. Money that reaches a terrorist organization buys the next attack. This country has fought a war on drugs and a war on terror for decades, at enormous cost in lives and treasure, and all the while we have allowed the financing to flow through institutions we charter and regulate. Cut off the money and we win. We have never seriously tried.
There is also a question of equal justice that I cannot let go.
Before I walked into the FBI building to report HSBC, I stopped in front of the Great Seal on the wall and read the words engraved beneath it: “Justice is the defense of our liberty.” I believed that. I still want to. But every day in this country, people go to prison for crimes that are nothing next to moving money for the people who want Americans dead. A suit, a title, and a corporate badge should not be a shield from the consequences that apply to everyone else. If a street dealer goes to prison for what he sells, the banker who moved the cartel’s money should be standing next to him.
I also want to raise something that is rarely said in these conversations. Why has no one inside Bank of America come forward the way I did? The answer is that everyone in American banking knows what happened to me, and to the whistleblowers who came after. The last person who spoke up is the only policy anyone believes. That is a culture problem, and it starts with leadership, in the banks and in the agencies that regulate them.
This is not a political issue. It has spanned administrations of both parties. It is a national security issue, and I am asking you to treat it as one.
Three things would change the math overnight.
Refer individuals for criminal prosecution. Not the analyst in the queue. The executives who set the thresholds and chose the risk. One banker in handcuffs would do more than every fine in Treasury’s history combined.
End “too big to jail.” An institution cannot be too big to fail and too big to jail at the same time. If it is the first, it must be held to a higher standard, not a lower one.
And if Bank of America will not act as America’s bank, review its charter. A national charter is a privilege granted on behalf of the American people. It should be conditioned on not moving money for the people trying to kill them.
Secretary Bessent, you said yesterday that these actions send a clear message that even the biggest institutions are not above the law. With respect, the institution was never the problem. The people inside it are above the law, and they know it. Until that changes, I will keep writing letters like this one, and the next bank will keep paying the next fine.
I gave up everything I had to stop this. I am asking you to use the power you have to finish it.
Respectfully,
Everett A. Stern
HSBC Whistleblower
Founder and Intelligence Director, Tactical Rabbit
Stephane Kasriel retweeted
In our health workstream at Meta we did a project where we wanted to see what research papers medical experts could write using a Muse Wonder coding scaffold that could access a database of 5M electronic health records we licensed from a vendor. The goal is that with 5M patients in this database, AI could do research to answer arbitrary questions like “did people who take statin have higher rates of diabetes later in life”? The thing we were betting on here is on what happens when you put human experts, a strong coding model, and access to data all together
We worked with a small group of medical experts to help them use this AI tool and the punchline is that they all found that this setup enabled a much faster pace of health research. They generated ten manuscripts in around two weeks, with one researcher (Paddy Ssentongo) generating three. Throughout the process, they collaborated with each other to review manuscripts and write skill files to help our coding scaffold improve at doing research. The medical experts liked how quickly they could test different hypotheses and choose the best ones to continue working on
Health research is pretty unique in that it is super important to be able to interpret noisy data and have taste to know which variables are important, which was still a significant value add from human researchers. Given that medicine is one of the oldest fields in human history and it is ultimately about humans, keeping a substantial human component in this type of research is probably important. In the near future I see AI primarily as a tool to facilitate human-led research. With the right scaffold I believe health researchers can be 10x or even 100x more productive using AI tools that have appropriate access to the wealth of data that the world has collected about human health
One way to see this work is through the broader lens of AI for science. You can think of science as two parts: (1) collecting raw data about the world, and (2) synthesizing that raw data into understanding. In this project we already had existing data about the world and we aimed to accelerate part (2) using AI. Eventually, I imagine a world where we have such great AI tools that the time to synthesize raw data into understanding is driven towards zero. In that world, science will be substantially faster and most of the work will be focused on collecting the right raw data via physical experiments
We did this in a pretty short amount of time (which was the point) and I think the future of scientific research will be more faster and more exciting than ever
We’re exploring how AI can accelerate health science research.
While AI has made significant breakthroughs in formal sciences like mathematics, health data science depends on imperfect real-world data and consequential choices about cohorts, variables, and causal assumptions.
To explore how AI can assist health researchers, we built a human-led research scaffold powered by Muse Spark 1.3 and Muse Wonder with access to 5M retrospective de-identified electronic health records and publicly available data. A cohort of medical experts used the system to conduct original research, accelerate hypothesis generation, code execution, and manuscript drafting while maintaining full authority over scientific decisions.
Today, we’re sharing 10 research manuscripts produced through this exploration, spanning cardiology, oncology, infectious disease, emergency medicine, and more.
Read more in our research blog: go.meta.me/3U06wvD
Stephane Kasriel retweeted
We’re exploring how AI can accelerate health science research.
While AI has made significant breakthroughs in formal sciences like mathematics, health data science depends on imperfect real-world data and consequential choices about cohorts, variables, and causal assumptions.
To explore how AI can assist health researchers, we built a human-led research scaffold powered by Muse Spark 1.3 and Muse Wonder with access to 5M retrospective de-identified electronic health records and publicly available data. A cohort of medical experts used the system to conduct original research, accelerate hypothesis generation, code execution, and manuscript drafting while maintaining full authority over scientific decisions.
Today, we’re sharing 10 research manuscripts produced through this exploration, spanning cardiology, oncology, infectious disease, emergency medicine, and more.
Read more in our research blog: go.meta.me/3U06wvD
Stephane Kasriel retweeted
Muse Wonder is SOTA on legal reasoning.
–We want to help people understand their rights and make better decisions and close the legal literacy gap. Legal literacy gap is one of the main causes of the higher Gini index of a society. We would love to level the playing field and empower everyone to make the best decisions in their situation.
–Legal has been an awesome testbed for in-context learning. Laws are different in different countries / states and are ever changing. Instead of training the model to memorize all the laws and cases, we train the model to effectively retrieve relevant laws and cases and leverage them to learn in-context to provide most accurate and up-to-date results. Legal documents can be long and cumbersome, which give us the opportunity to evaluate and improve our long-context reasoning capabilities.
h/t @swarnaNLP @jaseweston, Tiara, Vivian, and all the amazing collaborators.
research.meta.ai/blog/muse-w…
Our first Ray-Ban Meta Audio glasses are on sale today! Open-ear audio has been the most popular feature on our AI glasses since day one, so we leaned into it and made a new line that delivers the best sound quality on the lightest and most comfortable frames we’ve ever made.
meta.com/blog/ray-ban-meta-a…
Stephane Kasriel retweeted
We just open-sourced an entire game studio.
Introducing GameDev OS by Scenario.
It gives your coding agent 9 specialists and 64 skills.
📁 GameDev OS
┃
┣ 📁 2D Artist
┃ ┣ 📁 Concept Sketches
┃ ┗ 📁 Sprites and VFX
┃
┣ 📁 3D Artist
┃ ┣ 📁 Image to 3D
┃ ┗ 📁 Rigging and Retargeting
┃
┣ 📁 Environment Artist
┃ ┣ 📁 Textures and Skyboxes
┃ ┗ 📁 Walkable 3D Worlds
┃
┣ 📁 Art Director
┃ ┣ 📁 One Character, Every Shot
┃ ┗ 📁 Quality Check Before It Ships
┃
┣ 📁 Sound Designer
┃ ┣ 📁 Music and Score
┃ ┗ 📁 Gunfire and SFX
┃
┣ 📁 Video Producer
┃ ┣ 📁 Cinematic Trailers
┃ ┗ 📁 Captions for Every Platform
┃
┣ 📁 Marketing Artist
┃ ┣ 📁 Store Art for Every Platform
┃ ┗ 📁 Ads From One Shot
┃
┣ 📁 Technical Director
┃ ┣ 📁 Workflows Imported From ComfyUI
┃ ┗ 📁 Custom Pipelines
┃
┗ 📁 Cost Manager
┣ 📁 Model Bake-Offs With Cost
┗ 📁 Team Budgets
Each specialist knows the major models for its job: Seedance, GPT, Meshy, Rodin and more.
And they work together.
2D Artist sketches the gun → 3D Artist builds it → Environment Artist builds the world around it → Sound Designer adds the gunfire → Video Producer cuts the trailer.
One agent. An entire game studio behind it.
Free to install. Runs on your agent.
Comment “OS” and I’ll send you the entire skill library.
Stephane Kasriel retweeted
Very excited to share our latest, frontier-class, model! It’s been a fun year of cool ideas, rigorous science, and a lot of hard work. Check it out and give us your feedback! research.meta.ai/blog/muse-w…
Stephane Kasriel retweeted
This model is very very good. I was struggling to find anything to throw at it that it couldn’t do, even when other top models failed. Meta is back.
Stephane Kasriel retweeted
Meta has released Muse Wonder and Muse Wonder Full, achieving Artificial Analysis Intelligence Index v4.3 scores of 52 and 56 at max reasoning effort - each a material gain over Muse Spark 1.3 (max) at 48, with a release gap of only 6 weeks
Muse Wonder is available for users on Meta’s first-party API, while Muse Wonder Full is currently only available for Muse Code users. Meta notes that each model is designed for personal and professional agent use
Congratulations to @AIatMeta, @finkd, and @alexandr_wang on the release!
Key Takeaways:
➤ Both models exhibit a material jump in overall Intelligence Index score. Muse Wonder Full (max) improves on Muse Spark 1.3 (max) across all ten Intelligence Index evaluations, gaining 8 points to 56. The largest gains were seen in agentic evaluations: Terminal-Bench 4.0 (+24 points, 33% to 57%), GDPval-AA v2.1 (+176 Elo, 1681 to 1857), and AA-Briefcase (+148 Elo, 1581 to 1729). Muse Wonder (max) gains 4 Intelligence Index points and lands at 52, with notable improvements vs. Muse Spark 1.3 (max) in Terminal-Bench 4.0 (33% to 52%) and AutomationBench-AA (58% to 68%)
➤ Muse Wonder Full (max) places in the top three on six of the ten Intelligence Index evaluations, and is particularly strong in agentic knowledge work and scientific reasoning. In agentic knowledge work, it narrowly leads on GDP.pdf (33%), ranks #2 on GDPval-AA v2.1 (1857 Elo), and #3 on both AA-Briefcase (1729 Elo) and AutomationBench-AA (71%). In scientific reasoning, it tops the CritPt leaderboard at 33% and ranks #3 on SciCode with 63%. This builds on the relative strengths of Muse Spark 1.3 (max), which also showed particular ability in these two domains
➤ Both models use fewer output tokens than Muse Spark 1.3 (max). Muse Wonder (max) and Muse Wonder Full (max) each use around 54k output tokens per Intelligence Index task, down ~10% from Muse Spark 1.3 (max, 60k). This usage is lower than similarly intelligent models Gemini 4 Argon (high, 62k) and Claude Fable 5.1 (max w/ fallback, 78k), but higher than GPT-6.1 Sol (max, 38k) and GPT-6 Astra (max, 27k). Muse Wonder Full (max) uses 16% fewer input tokens per Intelligence Index task than Muse Wonder (max) (6.44M vs 7.69M), meaning it is both more intelligent and more token-efficient
➤ Cost per Task for Muse Wonder (max) lands higher than most similarly intelligent models. Despite efficient output token use, premium API pricing ($4 per million input tokens, $20 per million output tokens) and high input token use (7.69M per task) push Muse Wonder (max)’s cost to $4.52 per Intelligence Index task. This is lower than Claude Fable 5.1 (max w/ fallback, $7.63), Claude Opus 5.5 (max w/ fallback, $5.98), and Claude Sonnet 5.5 (max w/ fallback, $5.46), but higher than GPT-6 Astra (max, $3.26), Gemini 4 Argon (high, $1.99), GPT-6.1 Sol (max, $0.72), and its predecessor Muse Spark 1.3 (max, $1.60). Per-token pricing is not currently available for Muse Wonder Full, so no Cost per Task figure can be estimated at this stage
➤ Both models gain significantly in Coding Agent performance. Muse Wonder Full achieves 68 on the Artificial Analysis Coding Agents Index at max effort in Muse Code, alongside Claude Code with Sonnet 5.5 (max, 68) and Opus 5.5 (max, 66). Muse Wonder (max) scores 63. These represent material gains over Muse Code with Muse Spark 1.3 (max), which scores 54
Other details for the two new models:
➤ Context windows: 1M tokens each, unchanged from Muse Spark 1.3
➤ Pricing: for Muse Wonder, $4 input / $0.20 cached input / $20 output per 1M tokens. Pricing per token is not yet announced for Muse Wonder Full
➤ Input modalities: text, image, video, and audio
➤ Output modalities: text
➤ Availability: Muse Wonder is available on Meta's first-party API and Muse Code at launch, while Muse Wonder Full is only available to Muse Code users

































