Accelerated Understanding

Honored to follow in the footsteps of @JensenHuang, Jack Dongarra, Fran Allen and many other luminaries and be this year’s @Supercomputing (SC26) keynote speaker! The SC community has laid the foundations for scaling AI. At the same time, HPC workloads like scientific simulation are undergoing dramatic changes and are increasingly powered by AI. In my keynote, I will offer a glimpse of a new scaling law beyond LLMs, building AI that simulates and understands the physical world to invent and discover. This builds on my work at the intersection of AI and HPC over the last decade. Five years ago, I announced FourCastNet, the first AI high-resolution weather model, at an invited talk at SC21. Today, we have AI weather models running at weather agencies around the world and also making inroads into long-term climate modeling with AI. At SC22, I was part of the team that won the Gordon Bell special prize and released the first genome-scale model, the largest biological AI model at that time, which started the revolution in AI-driven design in biology. We have seen similar successes at @Caltech in our work across areas, like AI designing a new medical catheter that reduces bacterial contamination hundredfold or predicting plasma behaviour in nuclear fusion. In all these cases, AI does not model text but the physical world itself. This is essential for unlocking nature’s intelligence for scientific discovery. The defining driver of the AI revolution of the past few years has been universality and scale and that moment has now come for scientific simulation and physical understanding as well. I am excited to talk about what we have been working on at Accelerated Understanding @accelerated_u to drive this revolution and the progress it will enable. sc26.supercomputing.org/prog… @Supercomputing #SC26 #HPC

Celebrating @TIME #Time100AI at the SF Asian art museum

Our first podcast since the launch of our company Accelerated Understanding @accelerated_u last week. We go into what it means to have record-breaking 5 Trillion context length and why high-resolution 4D predictions are essential for physical understanding. We also talk about universality and achieving multiple areas of physical skill in the same model, and all the industries this will impact. What does self-improvement look like for physical AI? Watch the podcast to find out more. @bjenik @latentspacepod youtu.be/KS_IpnX7n9I

Was fun to be on the @latentspacepod podcast a few weeks ago to talk about AI for physical simulation and understanding. This pre-dates the public launch of @accelerated_u so I couldn't yet talk the exact details but hint at what happens if you scale some of these methods to Trillion parameter model sizes and fully 4D context lengths in the Trillions in a universal model. The podcast covers some of the ideas that provide the foundation for scaling. It also goes into the promise and successes that were possible even before going really big like building high resolution fully AI based weather models that are tens of thousands of times faster and as accurate as existing forecasts. We talk about predicting plasma behavior in fusion so quickly that one could take corrective action before something bad happens. And we cover how physical understanding doesn't just help with replacing experiments but lets us optimize design directly. We even briefly touch on the promise of combining those capabilities into one large model which I can now talk about more. Watch for yourself: youtu.be/79mIutht1f4?si=M3Wd…

Excited to see @Reuters cover the launch of our startup Accelerated Understanding. We are training large scale AI models that can simulate and understand physics to invent and discover. Our models understand the world directly in 4D (3D + time) and across physical phenomena. Going full 4D requires massive context length, we have pushed it to a Trillion in training and exceeding 5 Trillion at inference. AI giving you a bigger haystack of ideas doesn’t help. The bottleneck for new inventions and discoveries is shifting from ideas to the ability to test them. With AI that can simulate and understand physics we are directly attacking this bottleneck. People have been trying to do this for a while now, but usually by taking shortcuts. Narrow surrogates are great if you happen to have enough of precisely the right data and your design loop stays in distribution. Video models look fantastic but sweep physical accuracy under the rug, and some static world models cut out physics altogether. A lot of interesting physics isn’t visual. What does not cutting corners look like? Space stays 3D and you also have time: so 4D in total. You also need multiple physical modalities in the same model, not just things you can see. That’s what we’ve built. Scaling is the primary ingredient to make this work. To represent the world you need sufficient context, which in our case grows in 4 dimensions. Individual samples get so big they don’t fit into single accelerators or even full nodes anymore. We’ve developed architectural tricks to make it work. We’ve pushed our models to 1T parameters during large scale pre-training and are able to train at up to a Trillion context when needed and do inference exceeding 5 Trillion context without any sub-sampling or patching. Building on prior successes of AI weather forecasting, fusion simulation, design of medical devices, drugs and chips, we wanted to see if scale and universality can benefit AI for physical understanding. With our teams’ experience in large-scale infrastructure and model training we’ve been able to pull it off. reuters.com/business/ai-foun… acceleratedunderstanding.com… @accelerated_u @bjenik