Pierre Sermanet retweeted
We’re growing our team @UMA_Robots and building the infrastructure to take our in-house model to the next level.
If you have experience in large-scale training on massive multimodal data, please apply app.dover.com/jobs/uma.
(hiring RS/REs to work on Training/Data)
Pierre Sermanet retweeted
Our team scaled @Tesla Autopilot and Optimus, VL-JEPA at FAIR, RL post-training on Gemini, RL for locomotion.
Come work with us to push the frontier of AI in robotics!
Open roles in 🧵 -> Infra | Scientist | Engineer
Pierre Sermanet retweeted
At @UMA_Robots , we are convinced that latent-space world models will unlock scaling for robotics
1 position left for the roles
🧪 Research Scientist:
app.dover.com/apply/UMA/e223…
🚀 Research Engineer:
app.dover.com/apply/UMA/1bb6…
Pierre Sermanet retweeted
Excited by what we’ve achieved in the last 9 months.
We went from a model that could merely pick objects to one that can run autonomously for hours on challenging, high-precision tasks.
AI developed at @UMA_Robots
Single neural network, End to End
It operates robustly for hours, with precision, adapting to what it sees.
Pierre Sermanet retweeted
Starting with the fundamentals
Prototype Version 0
AI, Software, Hardware
A small team, 9 months
Designed and assembled in Paris at @UMA_Robots
Pierre Sermanet retweeted
Tl;dr After 10+ years pushing the boundaries of RL at @GoogleDeepMind , @Google Brain and then Gemini, I’m looping back to startup mode and joining @UMA_Robots to push the frontier of human-centric robotics. (1/4)🧵
Pierre Sermanet retweeted
One of the first AgiBot X2 humanoid robots just landed in our lab 🤖
I like its articulated neck, back handle, battery location, anthropomorphic wrist and white soft cover in a sort of Polyurethane foam.
Impressive delivery speed and support from the @AGIBOTofficial team!
Pierre Sermanet retweeted
I am starting a venture on top of LeRobot!
We’re at a pivotal time. AI is moving beyond digital to the physical world. Embodied AI will change our surroundings in ways we can barely imagine. This technology holds the potential to empower everyone. It must not be controlled by just a few.
This conviction led me to propose an ambitious open-source AI robotics project to Thom, Clem, and Julien back in 2024. Hugging Face, home to a community of millions of AI builders and a team of experts who brought us transformers, datasets, and the Hugging Face Hub, was the perfect place to launch LeRobot.
I’m incredibly grateful for all the support that allowed me to build LeRobot alongside an amazing team and community. In such a short time, we built one of the most adopted open-source robotics platforms, used by startups, universities, and research labs. It is helping countless people take their first steps in robotics. Together, we’ve even assembled the world’s largest open robotics dataset. And this is only the beginning for LeRobot!
Building on this momentum, I now feel the urgency to start something new on top of LeRobot. It will push the limit of what robots are capable of and commoditize them within society. Like LeRobot, it will start in Paris, leveraging its vibrant international AI scene. Stay tuned!
As LeRobot continues to expand, it’s now in the best possible hands with @AractingiMichel, @pepijn2233 and Steven Palma taking the lead. Watching the team deliver exceptional results over the last weeks has been one of the most rewarding experiences. Their creativity, dedication, and capability to ship fast is proving just how strong the team is today!
I am extremely grateful to the many people who contributed to making LeRobot at Hugging Face and within its powerful community. Many thanks to Thom, Clem, Julien, Simon, Rob, Michel, Pepijn, Steven, Gloria, Adil, Martino, Caroline, Marine, Mishig, Guillaume, Pablo, Lysandre, Arthur, Quentin, Florent, Brigitte, Victor, Marina, Mustafa, Francesco, Jess, Jade, Ville, Leo, Max, Julien, Alexander, Flavien, Raphael, Adina, Tao, Dana, Batu, Olivier, Matthieu, Eugene, Theo, Guilherme, Hynek, Loubna, Clémentine, Merve, Vaibhav, Anna, Jeff, Adrien, Emily, Johanne, Adrien and others. There are too many of you to be all named!
Thanks again and see you soon!!! :)
~ Remi
Pierre Sermanet retweeted
Excited to share what we've been cooking @GoogleDeepMind to bring Gemini into the physical world! 🦾🧠
Here is an uncut interaction of the Gemini Robotics VLA helping me clean up my desk. 🔊
Interacting with a robot through voice is the most surreal experience, and parallels the giddy excitement I had playing with early LLMs that is hard to put in words. You've got to try it to believe it!
Over the next few days, we'll be sharing more such interactions that go beyond shiny demos and expose raw capabilities of our models. Stay tuned for more raw videos!
Tech Report: storage.googleapis.com/deepm…
Blog: deepmind.google/discover/blo…
YouTube Playlist: youtube.com/playlist?list=PL…
Pierre Sermanet retweeted
Is language capable of representing low-level *motions* of a robot?
RT-Hierarchy learns an action hierarchy using motions described in language, like “move arm forward” or “close gripper” to improve policy learning.
📜: arxiv.org/abs/2403.01823
🏠: rt-hierarchy.github.io (1/10)
Pierre Sermanet retweeted
And I want to close with a Haiku.
Prompt in gray by @brian_ichter, and the completion written by PaLM-E-562B:
Pierre Sermanet retweeted
For one, “Let’s think step by step” comes to multimodal models!
Zero-shot chain-of-thought has been one of these emergent behaviors that has caught considerable interest in researching LLM capabilities…
With PaLM-E-562B, zero-shot visual chain-of-thought comes “included”.
Pierre Sermanet retweeted
What happens when we train the largest vision-language model and add in robot experiences?
The result is PaLM-E 🌴🤖, a 562-billion parameter, general-purpose, embodied visual-language generalist - across robotics, vision, and language.
Website: palm-e.github.io
Pierre Sermanet retweeted
Introducing RepNet, a model that counts repetitions in videos of *any* action
w @yusufaytar, @JonathanTompson, @psermanet and Andrew Zisserman
Paper: openaccess.thecvf.com/conten…
Project: sites.google.com/repnet
Video: youtube.com/watch?v=bvqg-ngu…
#CVPR2020 #computervision #deeplearning
Pierre Sermanet retweeted
How can we instruct robots in natural language?
Grounding Language in Play
Article: language-play.github.io
Pdf: arxiv.org/abs/2005.07648
with @psermanet
Here’s a video of a person typing commands to our agent in real time:
1/8 👇
Pierre Sermanet retweeted
😊Self-supervised learning opens up a huge opportunity for better utilizing unlabelled data while learning in a supervised learning manner. My latest post covers many interesting ideas of self-supervised learning tasks on images, videos & control problems: lilianweng.github.io/lil-log…
Pierre Sermanet retweeted
We've open sourced the first playroom from the learning from play project (learning-from-play.github.io). Check out github.com/google-research/g…!
Many thanks to Michael Wu, @Vikashplus.
Pierre Sermanet retweeted
I Now call it "self-supervised learning", because "unsupervised" is both a loaded and confusing term.
In self-supervised learning, the system learns to predict part of its input from other parts of it input. In... facebook.com/722677142/posts…






















