predicting state from t -> t+1 has always fascinated me. it’s obviously super exciting in robotics, but I’ve been just as interested in it for forecasting macroeconomic and geopolitical events. working with @haifengxu0 on @ProphetArena last year, it was interesting to see LLMs’ emerging forecasting abilities (and where they fell short). as in most domains, a big edge comes from information asymmetry, and how quickly you act on it. An obvious observation: prediction markets are thin and slow. financial markets are the opposite: a noisy but fast representation of the macro state of the world, encoding not only current beliefs but speculation from some of the world’s most sophisticated actors. pulling signal from this source is a fun challenge, and a big part of why I worked on this. in this paper, we build a foundation for learning high-quality embeddings of market state with LeJEPA (and SSL more broadly). totally agree with @randall_balestr that the geometry of market data and predicting in latent space deserve more investigation. I’m super excited to explore using these representations in larger systems for conditional, what-if forecasting on binary events. data and code are linked in the thread for anyone who wants to poke at it.

We also took a shot at sim2real. While it didn't end up using it for data collection, we did use it for evaluating policies. We recreated the scene by recording a few minutes of calibration video demonstrating the speed of various movements relative to their joystick position. In addition, we took pictures of a checkerboard calibration target in multiple orientations.