Yury Liavitski (@Heliocene) header
Yury Liavitski (@Heliocene) profile picture

human in the loop | eng lead building agents at đŸȘ„

Amsterdam
Joined February 2012
Photo by Yury Liavitski (@Heliocene)Photo by Yury Liavitski (@Heliocene)Photo by Yury Liavitski (@Heliocene)Photo by Yury Liavitski (@Heliocene)Photo by Yury Liavitski (@Heliocene)Photo by Yury Liavitski (@Heliocene)Photo by Yury Liavitski (@Heliocene)Photo by Yury Liavitski (@Heliocene)Photo by Yury Liavitski (@Heliocene)Photo by Yury Liavitski (@Heliocene)
Kids love creepypasta horror characters these days and one of the most popular monsters is Verity. It's a "personal helper friend" that gets obsessed with you and turns into a monster if you ignore it. Yeah, I think we might overestimate how much people want AI 😁 youtu.be/Ief7wnV8StY?is=tU1b

The responses to the "no AI" {for the code change that caused the outage} from non-devs is wild: many WANT to believe AI caused the outage I think many of us in the industry might be underestimating the hatred towards AI outside the tech bubble
1
60
If Huang can be replaced at nvda, what hope for the rest of us. All our past tricks of writing incomprehensible code, not documenting anything, creating and resolving P1 incidents are useless now
The sad part about Jensen Huang’s story is that, without him, NVIDIA would not have existed, whereas, counterfactually, replacing any other employee would have made no significant difference.
40
The only thing that hurts more than weightlifting at 42 is skipping weightlifting for a few weeks
1
48
Honestly I wish everyone did the same. Just say what you want to say guys!
29
Finally my training in OKRs will pay off
As models continue to improve + the “coding solved” meme becomes reality, what will differentiate you and your team will be less routine feature work and more about your ability to effectively translate product and business requirements into well-articulated verifiable metrics and outcomes which can be hill-climbed against
42
Excellent, we are in another iteration of "constraints liberate, liberties constrain". I really liked @poteto's talk btw, made some notes for my own project.
Today's experiment: If too much choice is bad for agents, what if you could create an EXTREMELY constrained version of the tools you like? I.e. the agent is allowed to use React, but all state must be in Zustand/useReducer? Strip out all the nuance, create a black hole of success, all through incredibly harsh lint rules. Thinking of calling it /constrain
39
I wonder why...
The responses to the "no AI" {for the code change that caused the outage} from non-devs is wild: many WANT to believe AI caused the outage I think many of us in the industry might be underestimating the hatred towards AI outside the tech bubble
1
127
This might just make me buy an iPhone again
Since Apple Music and Spotify designs are so boring, I built an iPhone Duo app called Duoamp that is a throwback to Winamp Shake it and a new Winamp-style skin appears
61
This work is turning me into a fat proxy
56
Can't stop laughing at this one. Expectation when joining a robotics startup: advanced algorithms, bleeding edge machine learning and engineering. Reality: pretending to be a little duck
This live demo of four Microducks with Jensen Huang went completely viral, and we're ecstatic about it. But there's also a lot of confusion, so I want to clarify what Microduck can and cannot do, and what it will realistically be able to do in the future. I count 60M+ views on this demo alone, based on the few big posts I see on social media, and everything has been buzzing. Even my own simple emotional design post on a newly created Instagram account got 1M+ views. We are selling Microducks at an incredible rate, even faster than during its historic launch. I've been trying to answer all the questions I see on our socials: people are asking if X or Y is possible on the robots, and this post aims to answer most of them. First of all, the demo you've seen was not fully autonomous. We had one engineer piloting each Microduck. The motions on the robots were trained with reinforcement learning, using a sim-to-real pipeline: walking, standing up after a fall, sitting, doing a somersault, roller skating. All of those are policies trained this way. What the pilots did during the live event was send commands: directions (move forward, rotate), head orientation (look here), quacks, and special actions (do the somersault, sit down). If you buy a Microduck, will you get an autonomous companion right out of the box? The current answer is no. But there is no impossible technical barrier to get there. I'm not sure how much time we'll have to work on this before shipping, but since everything is open source, I fully expect the community to build it, and to build it fast. Now let's understand what's already done, what will be easy to add, and what's an open problem. What you'll be able to do out of the box, and you can already do it in simulation right now, is train new policies with reinforcement learning. You could teach your duck to walk differently from the default gait we provide, to do a weird dance, or to walk on stilts. The community has been crazy creative with this, even though they only have the simulator. Note that you don't need reinforcement learning when the movements are simple. For example, the early emotional design I did is mostly coded movements plus pre-generated sounds. A simple mental model: use reinforcement learning when balance is part of the movement. If you try to code a cinematic walk by naively putting one foot in front of the other, your robot will realistically never walk, because you need to account for the dynamics of the movement, the inaccuracies of the sensors, the friction with the ground, and potential perturbations from the outside, like being pushed. So training new policies is definitely doable, but you can't train a policy to get a genuine long-term companion in your home (well, technically you could, but that's a whole other range of policy: a research-grade problem, and the resulting neural network would probably be huge and never able to run on such a small compute). Instead, you need a few more building blocks: voice recognition, facial detection/recognition, localization, some simple navigation, and a behavior layer, which is less technical and harder to describe. As a rule of thumb, a prototype of any of these is within reach of anyone if they have a modern coding agent and if the computation doesn't run on the robot (meaning the robot can reach the internet or an external computer and use external APIs). A rough prototype for each of the above can be a chill weekend project. As always, going from the prototype to a high-quality implementation is another story, but still doable for most if you put in the time and test a lot. However, having all of this run locally, on the onboard chip, I would call a very hard task. You'd need some very clever engineering to get a good enough version of everything working together. But I can't say it's impossible, because I've seen some crazy efficient implementations of all of the above. And here we don't need a super high success rate: it's okay if the duck sometimes does something a bit random. So I wouldn't be surprised if folks managed a fully local, cool, simple version of this autonomous companion. For example, you don't need Whisper-grade open-vocabulary transcription: being able to recognize a few keywords would already be great for interaction. One of the fastest ways to turn this robot into an autonomous companion would be to stream Microduck's sensors (the camera, the depth sensor, the IMU and the joint positions) to a computer, and use a software stack like the one we use for the conversation app on Reachy Mini: a local VAD, then a cloud speech-to-text model, then a call to a frontier LLM that decides on actions. Actions could be quacking, looking at things/faces/a direction, walking to a given position, sitting down, playing an emotion, kicking or doing a somersault. A SLAM algorithm would run in parallel so the robot can localize itself in your home, and you would prompt the LLM for a lifelike behavior: moving to random places, sometimes following people, and having emotional responses that fit what you're saying and the duck's environment. Of course someone is going to build a Tamagotchi-like layer on top of this, and it will be insane. How much demand is there for a software suite that would do all of this? I for one think it would be super cool. A word about the humans. We had five engineers backstage, and they are pretty much the core team behind Microduck. Their impact per head is absolutely off the charts. Antoine Pirrone piloted Graphite, Augustin Crampette Lavender (the rebel one), Matthieu Lapeyre Sky, Coralie Deplane Cream, and Pierre Rouanet was stressing over the Wi-Fi. Give them a follow! (Antoine @antoinepirrone, Augustin on LinkedIn, Matthieu @matth_lapeyre, Coralie @Drenaya_, Pierre on LinkedIn). I get visibility because I communicate a lot, but I mainly work on Reachy Mini (you might want to check it out: it's a very cool robot, and it ships much faster than Microduck :) ). My contribution to Microduck has been minimal: I'm mostly a first power user. My main added value at GTC was having hands big enough to carry four Microducks below the stage. Where I take pride is that Antoine and Coralie are former students and former interns of mine. I'm therefore entitled to 3% credit on anything (good) they do, for life. If they do something bad, however, it was clearly the fault of their parents. About the super smart engineering that can be done on board: Antoine just released two very cool blog posts (links below). The first is on odometry, the robot estimating its own movement from its own internal sensors. The second is about SLAM, which allows long-term localization: it adds the depth sensor to the odometry to get a stable position over time. Think of it as recognizing known landmarks in your home to know where the robot is. The crazy part is that all of this runs on the robot. So this gives me hope that we'll be able to do a lot of cool stuff on board.
90
What people call software factory is often a software sweatshop now Underdesigned software delivered quickly for the next demo day, and just as quickly left in a software landfill Factories plan ahead and control quality
I'd wager that 99% of companies do NOT need a "software factory." They just need to figure out WHAT to fix, and WHAT to build. This example - Substack's search has been poor since forever, never addressed, even though it's easy to fix now (in hours, max days) a good example
85
The first two are programmer jargon for decades now. The last two I have never heard. Granted I am not in sf
saying “isomorphic” and “orthogonal” now is almost normie in sf. you need to be using 2010s cuda compiler speak e.g., instead of saying “we need to align,” say “we need to reconverge” instead of saying “ultimately,” say “the postdominator.” this will maximize your insufferability
51
Exactly. Selling in an interview is a two way street. If you apply to a high profile startup, they have already done their part. Now it's your turn.
Replying to @GergelyOrosz
In general, to have any shot at having a decent career in tech, learn to be proactive even when it’s not as painfully obvious (like here) that you need to. Do your research before applying to a new place: you’ll learn a bunch if you do (and research w AI is much easier). If applying for a startup, build stuff that helps them, don’t just shoot an application (again, with AI if you know how to use it easier). Etc Or just don’t, and complain how the market is tough
 yea if you still work the same way you did in 2020
49
I might agree, but side projects are not just code, they could be hardware, crafts, sports, community work, politics etc. etc.
From a startup founder: “it’s now a red flag for me if I don’t see side projects on someone’s resume / application, because it signals lack of autonomy. Because it’s now so trivial to build anything on the side, vs before.”
1
107
Couldn't have said it better myself
When it comes to hiring, it’s easier to tame a reckless hustler than it is to instill that dawg in someone
122
I'm here to get rich and eat rice and beans. And I'm all out of rice and beans.
culture at anthropic as written by an MTS there.
1
187
Single responsibility microservices were always a bad pattern. With AI development they are now fully deadly. So much new surface for attacks, and the constant need for deployment coordination. But a whole generation of developers doesn't know another way.
1
85
There were signs, we just didn't see them
Day in the Life of a Senior SWE at Bloomberg
273
On YouTube a monologue of a guy who was fired from a "people manager" position. He says his job was to protect the team from noise, and his team underused AI because they were not given guidance on how to use it. Oh, buddy...
79
We brought Touch to the Mac! From the all-new iPad mini to the 14-inch MacBook Pro and iMac... and now MacBook Pro with PROLED XDR Display and Touch, creators have new tools to do their best work yet.
1
83