
Building @HeyStreamGenie + @DiffractAgency. I live in the realm of HW and SW. Past me: @tastytrade, @wendys, @layeroneio, @streamlabs, @chipotletweets.
Ohio
Joined May 2010
- Tweets2K
- Following425
- Followers397
- Likes2.1K
Thom retweeted
Show up early in SF tech week. Space for 30 line of 400
Any folks in robot deployment, data capture, physical AI, etc. that’d like to meet up before tonight - let’s link!
Thom retweeted
Robotics has a data problem. Industry estimates put usable robot training footage at just ~100,000 hours, and every serious lab is racing to change that.
Join us for the first in a new SF series on robot learning, VLAs, world models, and the data engines behind physical AI.
We'll dig into how teams are sourcing and scaling human demonstration video to train VLAs, and where the next bottleneck really is.
Who this is for:
For robotics founders, VLA/world model builders, and anyone betting on physical AI.
Thom retweeted
Landed in SF for tech week!
Last night spent time testing some new methodologies with our so-101 and chatting through new site partners and deployment opportunities for data collection.
Thom also walked me through our latest sim environments we’re working through!
Let’s chat!
Thom retweeted
The first thing we did when we landed in SF was to buy a $7K robot arm.
Before we got funded by @ycombinator, we had been testing our policies on 3D-printed SO-101 arms. But there's a fundamental limit to the diversity of tasks you can perform when you have only have 5+1 DoF and limited payload strength.
Now with our YAM bimanual arms, we're able to run our cloud inference of VLAs and WAMs and test them on this standard robot embodiment. Plus, an added benefit is that there are many open-source datasets (e.g. by @allen_ai and teams) that we can use to train models.
The first of more robots to come!
@i2rt_robotics, @jerryhuang01
Thom retweeted
Twitch's Chief Product Officer was asked, live on stream, why the AI training toggle isn't opt-in.
He said: "If it was opt-in, nobody would opt in. That's honestly the answer."
Then he was asked whether Amazon has already trained on the archive. He said he doesn't know.
73 days. That's how long people spent upvoting a request on Twitch's own feedback board asking for opt-in. It's past 15,000 votes. It shipped opt-out anyway.
And here's the part nobody is connecting. Same week, a woman in Chennai is reportedly paid ₹250 an hour to strap a phone to her head and film her own housework, because a robotics company decided that footage was worth buying.
Two prices for human training data in one week. Zero, and ₹250 an hour.
The difference isn't the value of the data. It's whether anybody asked first.
Live on The Lab @thomswann and me, taking both of those apart nitter.gq/i/broadcasts/1AxRnZZEn…
Thom retweeted
Surprise first episode of the Lab. We're talking robotics, physical AI and the data behind training it all!
Come join us! nitter.gq/i/broadcasts/1oKMvvXyM…
Thom retweeted
🔥Call for contributors — building the world's largest open tactile dataset, Open-X-Tactile!
1/ 🖐️ Robotics can't be solved without touch!
2/ 👁️ Vision scales from the internet. Touch only grows from real contact — a robot can see an object, but not know if it truly grasped, pressed, or inserted it.
🚀 So we're building Open-X-Tactile (OXT): the "Open-X-Embodiment" of touch, toward the world's largest open tactile manipulation dataset — and we're calling for contributors worldwide⬇️
✦ Any sensor: visuotactile, array, force/torque, dexhand, gripper
✦ Unified representation — mixable, trainable, downloadable
✦ Open-Source & community-built
Until Now:
✦ ~400 tasks · ~26K trajectories · ~18M frames — and growing
✦ 37+ institution nodes (~21 labs + 16 industry teams) — and growing
✦ 15+ data sources · 12 sensor type
Got tactile data? Come build the foundation with us — any sensor, any scale welcome! 🦾
🌐Website: open-x-tactile.github.io/
🫧Slack: join.slack.com/t/open-x-tact…
📧 Contact: [email protected]
🛰️or WeChat: tx-leo-wz
🧵 1/n
Thom retweeted
LeRobot v0.6.0 is officially here: Imagine, Evaluate, Improve! 🤖🚀
We are closing the robot learning loop with massive upgrades for the open-source robotics community. From policies that imagine the future to a much leaner installation, here is what is new:
- 🌍 World Models: VLA-JEPA, LingBot-VA, and FastWAM help your policies anticipate the future.
- 👀 VLA Expansion: Welcome GR00T 1.7, MolmoAct2, EO-1, Multitask DiT, and EVO1.
- 🏅 Reward Models API: Track success seamlessly with Robometer and TOPReward.
- 🎯 Unified Evaluation: 6 new simulation benchmarks, all accessible via the lerobot-eval CLI.
- ☁️ And more: lerobot-rollout CLI for DAgger corrections, HF Jobs cloud training, up to 2x faster data loading, GUI - LeLab, many docs improvements
Ready to build the future of robotics? Dive into the full release notes here: huggingface.co/blog/lerobot-…
@ClementDelangue @Thom_Wolf
Thom retweeted
I still have remaining budget for this campaign closing EOM.
If you’re creating content around privacy, data, the World Cup, or your community is hyped for quarterfinals…
Comment below or DM me asap
Paid creator campaign for the World Cup 2026!
We're lining up more creators for a major privacy/VPN brand's World Cup push. Sponsored integrations in your own voice, built around one of two angles: watching the matches privately, or pulling up your home broadcast while you travel.
Who fits:
- You're creating content around the World Cup
- Engaged US / North America audience
- Sports, travel, tech, or strong general reach
- Can produce and post mid-June through July
If that's you (or someone you'd vouch for) drop a comment on how you're integrating the World Cup or DM. We're building a slate now, early replies get first look.
Thom retweeted
Rerun 0.34 is out
The headline: your coding agent can now see and drive the Rerun Viewer via MCP. And with the new headless mode, it doesn't even need a screen.
Also: sparse voxel grids, gamepad nav, a full robot-learning course. 🧵
Thom retweeted
Demonstrating the task to a robot!
Teleoperation used to serve purely for remote recovery or robot diagnostics.
Nowadays, that role has changed a bit.
Today, teleop is mainly used to collect high-quality demonstration data, making robotic models more robust and capable.
We remember seeing operators training Tesla's Optimus a few years back.
I'm curious how long teleoperation will remain one of the main ways these models get fed data.
Will teleop for super complex tasks become one of the most sought-after skills in the job market? Will we soon be hiring tons of teleoperators?
We might even be creating entirely new jobs built around this method of data collection.
Imagine a scenario a few years from now where being a top robot operator is the ultimate flex.
It'd be interesting to see a ranking built on data quality benchmarks from collected datasets.
I captured a tele-op g during @AutomateShow in Chicago and it was smoooooooooth! 😮💨
~~
♻️ Join the weekly robotics newsletter, and never miss any news → ziegler.substack.com
Today, we give robots a /skills library that self-evolves and compounds indefinitely! Introducing ASPIRE: a robot solving its 100th task is no longer as clueless as solving its first. Coding agents observe multimodal sensory traces from simulation and real robots, launch an evolutionary search over control programs, and distill the best know-how into an ever-expanding library.
ASPIRE is a new type of continual learning: "training" is skill refinement instead of gradient descent.
"Trained model" is a repo of sensorimotor skills instead of floating weights.
“Distributed training” is a panel of agents each practicing a different skill instead of sharded minibatches.
Here's the beauty: ASPIRE gives the tired terms "sim2real transfer" and "cross-embodiment transfer" a whole new meaning. Bridging the sim-to-real gap is notoriously brutal. An end-to-end policy has to swallow both the visual shift (sim looks toyish next to a real camera) and the subtle contact physics it never quite gets right. ASPIRE sidesteps the mess, because it doesn't ship pixels or weights across the gap, but ships the know-how. The robot still has to practice in the real world, not zero-shot, but it gets there way faster because it isn't rediscovering the strategy from scratch. Same for going single-arm to bimanual hardware, which usually requires new data and retraining from zero. ASPIRE achieves up to ~10x cut in "transfer learning” tokens (yes, tokens are the new unit of *training* compute ;)
Check out our gallery of 150+ tasks and 90+ skills the robots taught themselves, all on the website! Kind of wild that we can ship the "learned weights" as an HTML page rather than a GGUF. We'll open-source the full stack so your own robot library starts compounding from ours!
Deep dive in thread:
Thom retweeted
Welcome to Robot Park, where we’re building the future with Apollo 2. Robot Park is where Apollo learns today, getting the experience needed to make a difference tomorrow.
#Apptronik #Robotics #AI #Humanoids
Thom retweeted
The new YUBI paper is the closest thing out there to @GeneralistAI's UMI-like grippers.
UMI is better than teleoperation but has issues:
> Gripper is clunky and heavy ~900g
> Uses inaccurate visual SLAM
> Pistol trigger closing mechanism feels unnatural
YUBI:
> Uses Meta Quest for tracking
> Is fully open-source
> Closing gripper gives user haptic feedback
> Weighs 319g - a third of UMI
> Open-sourced an 8400h dataset
Data collection is ~2x faster than with regular UMI in their usability studies.
But what surprised me:
After collecting such a huge dataset, they only finetune a few π0.5 policies on minor subsets of the data as POC.
Been exploring datasets.bot lately, it's full of rich open datasets. I fine-tuned RF-DETR to track a robot's gripper and the object it's grasping, then pointed it at DROID, which has 76k+ real teleoperation episodes spanning hundreds of scenes and a dozen labs. Same model everywhere, no retraining, and honestly it held up better than I expected.
Thom retweeted
Are We Picking Up Laundry Yet?
There's something you should know about robot demos: whatever you see the robot doing in the demo is the absolute best thing it can do.
That's true of what I've shown you previously as well, but I’d like you to see what I see. This video is a continuous eval of a new model I’m working on.
In order for the Stringman to be a useful home appliance that actually saves you time, it needs to be able to pick up toys in a messy playroom and put them in some kind of container.
So far, what I have found is that I can train DiT models to grasp things if my training episodes consist purely of grasping things. I have tried episodes that consist of traveling to an object, picking it up, and taking it to its destination, but I found that the DiT model in LeRobot was mostly unsuccessful at learning cross-room traversals from the data I have collected.
So, for the cross-room traversal in the demo I showed, I relied on another model and some fairly straightforward point-to-point motion. That other model is what I've been calling the "target model" because it picks out targets on the floor. It requires its own set of training data, though it could possibly be swapped out for off-the-shelf solutions like YOLO.
For the past few weeks, I've been trying to answer the question of whether a more advanced DiT model can learn both target identification and cross-room traversal behavior.
I believed the amount of training data was a limiting factor, so I have collected 588 episodes that consist of locating an object, picking it up, and taking it to its destination. In that dataset, I stuck to a small set of prompts, such as "put laundry in the hamper," "put toys in the toy box," "put a ball in the ball pit," or "put trash in the trash can."
The other limiting factor was that the DiT model in LeRobot ignores all but the CLS token from the CLIP vision encoder. To improve the vision encoding and hopefully produce an embedding that contains more spatial data, I switched to the DINOv3 encoder and added a few layers of cross-attention that consume the spatial patches from DINOv3.
I retrained a model on this, and while it showed some promising results, it is still mostly useless, as you can see in the attached video.
For clothes, the model did learn to seek out and pick up certain articles of clothing, although it almost always goes for the same pair of gray pants. It did not learn to take the gray pants to the laundry hamper. It prefers to drop them in some random place in the room and then pick them up again.
Conversely, for toys, the model cannot seem to locate them. If placed directly over a toy, practically with the toy between its fingers, it will grasp it, pick it up, and carry it off to its destination. Unfortunately, it learned to overfit to the particular bin locations in my playroom.
So, in both cases, the model learned basic grasping and cross-room traversals, but never all of it in one usable package on any particular type of data.
Does this make it a useful cleaning robot? No, not yet. If hybridized with the point-to-point motion and the target-finding model, it is somewhat useful on laundry, though the target model is weak and needs a human to add or remove a target far too often.
There are currently six total deployed Stringman robots, scaling up to nine in the next week. Two owners have agreed to contribute data, so hopefully, if data volume is the limiting factor, we will slowly overcome it.
Large datasets of bimanual manipulation are available, but it’s not clear how they could be leveraged to improve the behavior of an overhead cable robot. I have tried fine tuning VLAs that have been pretrained on bimanual manipulation and it seems they pretty much immediately forget 99% of their training.
In the meantime, I'll continue to rely on the hybridized behavior and investigate other techniques, such as JEPA, RL, and creating simulated training data using Gaussian splat-based images.
Running a Mujoco simulation of Stringman is possible, though it doesn’t have a primitive for strings, so you’d have to either use chains or use the overlapping bowls trick. There isn’t a strong technical reason it hasn’t been done yet, it’s just a time and money constraint.
Thom retweeted
Over the last decade, we’ve built robots that can run marathons, harvest food, and even dance. But robots still aren’t very good at the simple tasks we use our hands for, like unpacking groceries or tying a shoelace. Most rely on simple grippers that aren’t much better than a claw machine.
Dexterous manipulation has long been one of the hardest unsolved problems in robotics, but humanoid robots need it to be useful in the real world, from warehouses to homes. The traditional approach to training robotic hands involves collecting data through teleoperated robots, but it’s expensive, slow, and unscalable.
@_jaku_xu and @JayLiStanford Li got to know this problem well while building humanoid robots at Tesla, where they had the idea to flip the paradigm: instead of having a human control a robot via teleoperation to train AI, they wanted to get better data from actual human hands.
With @proceptionAI, Jack and Jay have built the hand that robots have been waiting for: ProHand. They worked closely with hand surgeons to get the anatomy right, using the latest hardware breakthroughs like “soft,” skin-like sensors and finger actuation that mimics tendons. When they showed ProHand at Y Combinator Demo Day, everyone kept asking them whether there was a human underneath the table with their hand sticking out.
They’ve also built a data layer with ProGlove, a sensorized glove that humans can wear to collect real motion data to train ProHands — because the ProHand so closely resembles a human hand, a human can wear the same glove that covers the robotic hand (no robot in the loop required).
Proception is officially shipping ProHand and ProGlove to researchers and robotics companies today. I’m proud that @firstround got to lead their Seed Round, alongside BoxGroup and Y Combinator.
Thom retweeted
Physical AI is pretty hot these days, and there’s a lot of VC money flowing into the space.
The problem is, a lot of people still don’t know what Physical AI actually means or how it’s different from traditional robotics.
So I put together a simple one-page guide that explains:
• What Physical AI actually is
• How it’s different from traditional robotics
• The major building blocks of the ecosystem
• The terms you’ll keep hearing (VLA, World Models, Teleoperation, Sim-to-Real, and more)
If you’re curious about physcial ai this is for you.





















