Anooj

Published biology can take weeks or months to turn into usable training data and new evals. Our Prima Daedalus agents do it up to 10× faster, and 89% lower estimated cost. They reproduce papers, beat published results, and find previously overlooked biology.

We introduce 𝘢𝘶𝘵𝘰𝘷𝘦𝘳𝘴𝘦 🌿🧫👽, a fast, evolvable description language for reinforcement learning environments, implemented in jax. First of many papers on this subject w/ @togelius. Our goal is to provide a general framework for open-ended learning, in which we can implement canonical gridworld-type games, discover new (often alien) ones, and train robust player agents, that can observe a set of environment dynamics and adapt their behavior accordingly. Notably, autoverse allows for the evolution of map layouts, rules and reward dynamics; other than the notion of an embodied "player" with the ability to edit tiles on the map, no semantics are hardcoded into the environment. In sum, we set out to create a substrate for open-ended learning that is: - capable of expressing a virtually limitless number of novel environments (via evolution of *all* aspects of the environment) - fast, allowing for research at a small/academic scale (thanks to GPU implementation of environment dynamics) - scalable, allowing for potentially unbounded environment complexity (by increasing the size of the map, number of rules, etc.) Below is an example of an environment evolved via open-ended RL, in which environments are continually evolved to provide learnable challenges for player agents. Here, the agent is riding the waves generated by the environment, keeping them alive as long as it can to maximize reward. (The RL player is depicted in blue, game rules and their application across the board are depicted on the right.)