Co-Founder & Chief Scientist at UMA

Joined November 2017
Pierre Sermanet retweeted
We’re growing our team @UMA_Robots and building the infrastructure to take our in-house model to the next level. If you have experience in large-scale training on massive multimodal data, please apply app.dover.com/jobs/uma. (hiring RS/REs to work on Training/Data)
4
5
37
2,338
Pierre Sermanet retweeted
Our team scaled @Tesla Autopilot and Optimus, VL-JEPA at FAIR, RL post-training on Gemini, RL for locomotion. Come work with us to push the frontier of AI in robotics! Open roles in 🧵 -> Infra | Scientist | Engineer
9
21
334
27,534
At UMA, they flip and live 😁
15
20
3
154
12,623
Pierre Sermanet retweeted
At @UMA_Robots , we are convinced that latent-space world models will unlock scaling for robotics 1 position left for the roles 🧪 Research Scientist: app.dover.com/apply/UMA/e223… 🚀 Research Engineer: app.dover.com/apply/UMA/1bb6…
6
10
4
120
16,360
Pierre Sermanet retweeted
Excited by what we’ve achieved in the last 9 months. We went from a model that could merely pick objects to one that can run autonomously for hours on challenging, high-precision tasks.
AI developed at @UMA_Robots Single neural network, End to End It operates robustly for hours, with precision, adapting to what it sees.
8
27
2,785
Pierre Sermanet retweeted
Starting with the fundamentals Prototype Version 0 AI, Software, Hardware A small team, 9 months Designed and assembled in Paris at @UMA_Robots
87
121
51
892
248,702
Pierre Sermanet retweeted
Tl;dr After 10+ years pushing the boundaries of RL at @GoogleDeepMind , @Google Brain and then Gemini, I’m looping back to startup mode and joining @UMA_Robots to push the frontier of human-centric robotics. (1/4)🧵
17
32
3
274
24,697
Pierre Sermanet retweeted
One of the first AgiBot X2 humanoid robots just landed in our lab 🤖 I like its articulated neck, back handle, battery location, anthropomorphic wrist and white soft cover in a sort of Polyurethane foam. Impressive delivery speed and support from the @AGIBOTofficial team!
23
54
7
321
34,374
Pierre Sermanet retweeted
I recently moved from @teslasi to @UMA_Robots. 🧵(1/5)
🤖 Made with AI
23
14
3
100
33,972
Pierre Sermanet retweeted
I am starting a venture on top of LeRobot! We’re at a pivotal time. AI is moving beyond digital to the physical world. Embodied AI will change our surroundings in ways we can barely imagine. This technology holds the potential to empower everyone. It must not be controlled by just a few. This conviction led me to propose an ambitious open-source AI robotics project to Thom, Clem, and Julien back in 2024. Hugging Face, home to a community of millions of AI builders and a team of experts who brought us transformers, datasets, and the Hugging Face Hub, was the perfect place to launch LeRobot. I’m incredibly grateful for all the support that allowed me to build LeRobot alongside an amazing team and community. In such a short time, we built one of the most adopted open-source robotics platforms, used by startups, universities, and research labs. It is helping countless people take their first steps in robotics. Together, we’ve even assembled the world’s largest open robotics dataset. And this is only the beginning for LeRobot! Building on this momentum, I now feel the urgency to start something new on top of LeRobot. It will push the limit of what robots are capable of and commoditize them within society. Like LeRobot, it will start in Paris, leveraging its vibrant international AI scene. Stay tuned! As LeRobot continues to expand, it’s now in the best possible hands with @AractingiMichel, @pepijn2233 and Steven Palma taking the lead. Watching the team deliver exceptional results over the last weeks has been one of the most rewarding experiences. Their creativity, dedication, and capability to ship fast is proving just how strong the team is today! I am extremely grateful to the many people who contributed to making LeRobot at Hugging Face and within its powerful community. Many thanks to Thom, Clem, Julien, Simon, Rob, Michel, Pepijn, Steven, Gloria, Adil, Martino, Caroline, Marine, Mishig, Guillaume, Pablo, Lysandre, Arthur, Quentin, Florent, Brigitte, Victor, Marina, Mustafa, Francesco, Jess, Jade, Ville, Leo, Max, Julien, Alexander, Flavien, Raphael, Adina, Tao, Dana, Batu, Olivier, Matthieu, Eugene, Theo, Guilherme, Hynek, Loubna, Clémentine, Merve, Vaibhav, Anna, Jeff, Adrien, Emily, Johanne, Adrien and others. There are too many of you to be all named! Thanks again and see you soon!!! :) ~ Remi
99
83
19
758
124,375
Pierre Sermanet retweeted
Excited to share what we've been cooking @GoogleDeepMind to bring Gemini into the physical world! 🦾🧠 Here is an uncut interaction of the Gemini Robotics VLA helping me clean up my desk. 🔊 Interacting with a robot through voice is the most surreal experience, and parallels the giddy excitement I had playing with early LLMs that is hard to put in words. You've got to try it to believe it! Over the next few days, we'll be sharing more such interactions that go beyond shiny demos and expose raw capabilities of our models. Stay tuned for more raw videos! Tech Report: storage.googleapis.com/deepm… Blog: deepmind.google/discover/blo… YouTube Playlist: youtube.com/playlist?list=PL…
23
101
21
763
68,952
Is language capable of representing low-level *motions* of a robot? RT-Hierarchy learns an action hierarchy using motions described in language, like “move arm forward” or “close gripper” to improve policy learning. 📜: arxiv.org/abs/2403.01823 🏠: rt-hierarchy.github.io (1/10)
3
18
7
91
38,288
Pierre Sermanet retweeted
And I want to close with a Haiku. Prompt in gray by @brian_ichter, and the completion written by PaLM-E-562B:
2
9
1,238
Pierre Sermanet retweeted
For one, “Let’s think step by step” comes to multimodal models! Zero-shot chain-of-thought has been one of these emergent behaviors that has caught considerable interest in researching LLM capabilities… With PaLM-E-562B, zero-shot visual chain-of-thought comes “included”.
2
4
25
3,444
Pierre Sermanet retweeted
What happens when we train the largest vision-language model and add in robot experiences? The result is PaLM-E 🌴🤖, a 562-billion parameter, general-purpose, embodied visual-language generalist - across robotics, vision, and language. Website: palm-e.github.io
26
487
136
2,107
1,272,962
Pierre Sermanet retweeted
Introducing RepNet, a model that counts repetitions in videos of *any* action w @yusufaytar, @JonathanTompson, @psermanet and Andrew Zisserman Paper: openaccess.thecvf.com/conten… Project: sites.google.com/repnet Video: youtube.com/watch?v=bvqg-ngu… #CVPR2020 #computervision #deeplearning
15
116
24
459
Pierre Sermanet retweeted
How can we instruct robots in natural language? Grounding Language in Play Article: language-play.github.io Pdf: arxiv.org/abs/2005.07648 with @psermanet Here’s a video of a person typing commands to our agent in real time: 1/8 👇
9
52
5
225
Pierre Sermanet retweeted
😊Self-supervised learning opens up a huge opportunity for better utilizing unlabelled data while learning in a supervised learning manner. My latest post covers many interesting ideas of self-supervised learning tasks on images, videos & control problems: lilianweng.github.io/lil-log…
11
423
16
1,617
Pierre Sermanet retweeted
We've open sourced the first playroom from the learning from play project (learning-from-play.github.io). Check out github.com/google-research/g…! Many thanks to Michael Wu, @Vikashplus.
Pierre Sermanet retweeted
I Now call it "self-supervised learning", because "unsupervised" is both a loaded and confusing term. In self-supervised learning, the system learns to predict part of its input from other parts of it input. In... facebook.com/722677142/posts…
34
349
42
1,376