@NovaSkyAI

Building SkyRL at @BerkeleySky Join the Slack community: https://xitter.cf/t.co/mSO97T61vR

 Berkeley, California
github.com/NovaSky-AI/SkyRL
Joined January 2025
  • Tweets179
  • Following18
  • Followers2.9K
38Media
NovaSky retweeted
Bryan Lu
@yuzhe_lu4h
Great to see @modal launch this! You can also deploy a Tinker endpoint directly on AWS EC2 with SkyRL @NovaSkyAI for the same cost and flexibility benefits, and start agentic RL training with our AgentCore RL Toolkit! github.com/awslabs/agentcore…
Modal
@modal6h
Tinker changed how people do post-training. We're big fans, and so are many of the teams we work with. But as those teams scale up, they keep raising the same two problems: per-token pricing gets expensive quickly, and you’re limited to the models and features the provider decides to support. We think a good API deserves more than one implementation. Today we're releasing Spindle, an open-source, Tinker-compatible training server that runs in your Modal workspace. Your existing Tinker scripts run unchanged, and because you own the server, you pay for GPU time instead of tokens.
1
3
8
1,134
NovaSky retweeted
Qiuyang Mang
@MangQiuyang11h
FrontierCS 2 is our new benchmark for CS research & development. Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.
22
45
23
221
30,242
NovaSky retweeted
SkyPilot
@skypilot_org9h
Post-training a 4B model with RL to fly Tesla Roadster. It outperforms recent open models and even GPT-6 Astra in a storm. The key: have the model learn from the teachers first and then learn itself in practice. Trained with SkyRL (@NovaSkyAI) and vLLM (@vllm_project), orchestrated by SkyPilot's Job Group (@skypilot_org). Check it out ⬇️
Zhanghao Wu
@Michaelvll110h
Tesla's flying Roadster went viral. The real demo had been delayed two weeks "due to high winds." We asked: could the latest LLMs sit in the pilot's seat and fly the car through that wind? We built an RL environment for LLMs to fly it. In real time, GPT-6 Astra, DeepSeek, Muse Spark Open, Beam and Nemotron 3.5 Super VL landed 0% of kickflips. Even with unlimited time to think for each decision, they still did worse than a Qwen 4B model we post-trained with RL in a storm.
1
5
13
1,925
NovaSky retweeted
avi basnet
@avigyabasnet11h
Super excited to share some of the work I’ve been doing on supporting 1M token context training on SkyRL for GLM 5.3 Flash, 64x higher than the out of the box Megatron baseline! 🚀 This work was done in collaboration with the amazing folks at @LilaSciences! 🤩 Getting there took key algorithmic and systems optimizations in four main areas: 1) DeepSeek Sparse Attention: run DSA on a sparse kernel; chunk and shard its quadratic indexer. 2) Activation recompute and offloading: recompute activations in backward and offload saved inputs to CPU. 3) Context parallelism: split each sequence across GPUs, with new CP support for KDA and a fix for DSA. 4) Avoiding unnecessary precision upcasts: keep FP32 copies out of memory and off the network. For more details on our implementations and our results, check out our full technical blog here: blog.skyrl.ai/blog/1m-contex…! This work benefited from the contributions of many people along the way. Big thanks to @pcmoritz, @erictang000, @sumanthrh, @CyrusHakha, @richliaw from @anyscalecompute and Keith Truongcao, Tushar Khot, Vivek Subramanian, Semih Kara from Lila Sciences for all the help and guidance!
3
19
5
107
7,878
NovaSky retweeted
Zhanghao Wu
@Michaelvll110h
Check our blog: skypilot.ai/blog/flying-road… Trained with SkyRL (@NovaSkyAI), served with vLLM (@vllm_project), and orchestrated with SkyPilot (@skypilot_org)

Post-training a 4B model with RL to fly the Tesla Roadster 21% better than GPT-6 Astra

We RL-trained Qwen3-4B to fly a simulated flying Roadster. In storms, it beats DeepSeek, Muse, Beam, Nemotron and GPT-6 Astra, even when they get unlimited time to think.

skypilot.ai
5
8
270
NovaSky retweeted
Zhanghao Wu
@Michaelvll110h
Tesla's flying Roadster went viral. The real demo had been delayed two weeks "due to high winds." We asked: could the latest LLMs sit in the pilot's seat and fly the car through that wind? We built an RL environment for LLMs to fly it. In real time, GPT-6 Astra, DeepSeek, Muse Spark Open, Beam and Nemotron 3.5 Super VL landed 0% of kickflips. Even with unlimited time to think for each decision, they still did worse than a Qwen 4B model we post-trained with RL in a storm.
2
8
4
19
3,613
NovaSky retweeted
Reflection
@reflection_aiOct 21
Model customization recipes are live on our website and in the cookbook repo. Beam is now supported across RL and SFT stacks including @axolotl_ai, NeMo RL, NeMo AutoModel, Megatron Bridge (@NVIDIAAI), Miles (@radixark), SkyRL (@NovaSkyAI), and TRL (@huggingface). Get cooking 🧑‍🍳 developers.reflection.ai/cus…

Customize Beam - Reflection Developer Docs

Fine-tune and post-train Beam with the open-source training stacks that support it

developers.reflection.ai
5
9
5
121
7,491
First day 0 model support on SkyRL. Great work from @sumanthrh !
Sumanth Hegde
@sumanthrhOct 16

Day 0 support for Beam on SkyRL

Article · @sumanthrh
3
30
1,205
NovaSky retweeted
vLLM
@vllm_projectOct 16
🎉 Day-0 RL support for Beam in SkyRL @NovaSkyAI. LoRA, BF16 full fine-tuning and MXFP8 RL, with vLLM rollouts and Reflection’s Megatron-Bridge implementation. SkyRL reports +62% LoRA decode throughput from right-sizing expert adapters in vLLM: 276 → 447 tok/s/GPU at 32 concurrent requests, PP2×TP8. This is a decode result, not an end-to-end training speedup. R3 works with pipeline parallelism, and FP32 router weights are preserved during weight sync. Thanks to Sumanth Hegde and the SkyRL team! docs.skyrl.ai
1
6
8
1,086

Day 0 support for Beam on SkyRL

Article · @sumanthrh
13
5
40
4,029
Great to see this out! Datadog's new SOTA model for investigating production incidents. Their in-house training stack builds on top of SkyRL! 🚀🚀🚀
Datadog, Inc.
@datadoghqOct 15
Meet Scout 1.0, our first AI model built for investigating production incidents. Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier. We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence. Learn more and sign up for the preview: bit.ly/3TJknGO
2
6
24
1,251
NovaSky retweeted
Eric Tang@erictang000Oct 15
datadog ai research used SkyRL to post-train GLM 5.3 Flash to perform at the pareto frontier for their SRE eval @bthecohen and team were some of the earliest users of SkyRL at scale, really cool to see these results finally out! great tech report as well
Datadog, Inc.
@datadoghqOct 15
Meet Scout 1.0, our first AI model built for investigating production incidents. Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier. We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence. Learn more and sign up for the preview: bit.ly/3TJknGO
2
6
36
1,235
Datadog post-trained GLM-5.3 Flash via RL on real-incident sims. Matches frontier SRE performance at much lower cost. Their RL stack is built on top of SkyRL! Check out the technical report web-assets.dd-static.net/425…
Datadog, Inc.
@datadoghqOct 15
Meet Scout 1.0, our first AI model built for investigating production incidents. Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier. We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence. Learn more and sign up for the preview: bit.ly/3TJknGO
1
12
76
5,390
Post-trained with SkyRL!
Brian Armstrong
@brian_armstrongOct 11
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case). I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves. We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them: "/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started" I imagine we'll have some backend services that could be replaced entirely by this new architecture.
3
50
2,068
NovaSky retweeted
Richard Liaw@richliawOct 12
Post-trained with SkyRL @NovaSkyAI Served on Ray Serve LLM @raydistributed all on Anyscale Platform @anyscalecompute
Brian Armstrong
@brian_armstrongOct 11
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case). I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves. We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them: "/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started" I imagine we'll have some backend services that could be replaced entirely by this new architecture.
4
21
1,451
NovaSky retweeted
AfterQuery
@AfterQueryOct 8
Read more about our experiment using SkyRL here: afterquery.com/blog/improvin… To learn more about our tasks and reward design, reach out to us at [email protected]

AfterQuery - Improving Qwen3.8-27B-Medium on DeepSWE with GRPO

GRPO on 500 SWE agent training tasks created by AfterQuery raised Qwen3.8-27B’s DeepSWE pass@1 (avg 3) from 27.8% to 39.1%.

afterquery.com
2
7
3
28
4,060
Kimi K2.5-2.7 can be trained with RL on just 16 B300 GPUs (yes, just 2 nodes!) with full context Most post-training frameworks out there will tell you that you need 16 full nodes, and might not even give you full context Thanks to SkyRL maintainers for collab
Eric Tang@erictang000Oct 2
Excited to announce the release of SkyRL v0.4.0! Some highlights: - Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7 - Top-P sampler replay and optimized R3 data transfer - Full FP8/MXFP8 RL - Sharded P2P weight transfer Blog: blog.skyrl.ai/blog/skyrl-v0-… Details in 🧵
8
17
1
355
23,866
NovaSky retweeted
Eric Tang@erictang000Oct 2
Excited to announce the release of SkyRL v0.4.0! Some highlights: - Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7 - Top-P sampler replay and optimized R3 data transfer - Full FP8/MXFP8 RL - Sharded P2P weight transfer Blog: blog.skyrl.ai/blog/skyrl-v0-… Details in 🧵
2
32
3
190
31,980
NovaSky retweeted
Edward Hu
@edwardjhuSep 1
How does one RL post-train a 397B model for long-horizon knowledge work? 👩‍💼 We share every step we took to bring Qwen 3.5 397B from 16.1% Pass@1 to 27.3% on APEX-Agents using DPPO, including final models weights and the full training script🚀 This is the first of many works from Mercor Research on open model training research. Full blog: mercor.com/blog/training-fro… Source code: github.com/Mercor-Intelligen…
16
117
24
792
156,974
Worked with Mercor Research and the SkyRL team, training Qwen3.5-397B-A17B on APEX-Agents (long-horizon office work) off-the-shelf data with SkyRL, improving Pass@1 from 16% to 27%. The post is more of a practical field guide for what to do given an RL dataset, de-risking step by step: environment infra, harness fixes/optimizations, token-in-token-out, an overfitting run, small-model ablations, then the hero run. We were also delighted to see the gains transfer to OpenCode despite training on an MCP-based harness, and even to Terminal-Bench 2.1. Code, models, and eval traces are all open source. Hope it helps your next hillclimb!
6
26
2
264
21,641
Load more