Mauricio Ordóñez retweeted
Proud of what our team at Kingo is building. Solar-powered connectivity at Yaxhá is a major step toward modernizing park management and conservation infrastructure across Guatemala.
Kingo, @Swisscontact y @CONAPgt llevan internet @Starlink con energía solar a 10 parques nacionales. La primera instalación ya opera en Yaxhá, mejorando la gestión y habilitando pagos digitales. Pronto integraremos sensores para monitorear incendios, intrusiones y biodiversidad.
Mauricio Ordóñez retweeted
Open Sourcing Forge: 3D Gaussian splat rendering for web developers!
3DGS has become a dominant paradigm for differentiable rendering, combining high visual quality and real-time rendering. However, support for splatting on the web still lags behind its adoption in AI.
Mauricio Ordóñez retweeted
There is an alternate reality where Cray took their vector supercomputers, ditched FP64 calculations, and went with one FP32 pipe and a BF16 tensor core pipe. The same instruction set, memory architecture, and vector registers would have made a sweet deep learning machine, in many ways nicer than SIMT CUDA programming on GPUs. A Y-MP class machine like that could have delivered the AlexNet and DQN moments two decades earlier.
Even doing everything in FP64 with no architectural changes, a Cray-1 would have been the best machine in the world for neural networks. If @geoffreyhinton had access to one for early research, the case could have been made for the architectural modifications to 10x the performance.
Mauricio Ordóñez retweeted
Either I made an arithmetic mistake, or the risk factor from cycling is ridiculously small!
Compare to Rf for any other activity.
Mauricio Ordóñez retweeted
Replying to @endingwithali
Mauricio Ordóñez retweeted
Several years ago, when we started pursuing building our own chips, a lot of folks thought this was nuts. We heard a lot of the same refrains you often hear—why make this investment, why invest in a team and all the other fixed costs to develop your own chip when you can buy from other suppliers? And, while we knew we’d partner with those other companies for the foreseeable future, if your customers are telling you they’re thirsty for better price-performance, and you’re driven by what makes customers’ lives better and easier every day, you explore options to make it so.
We realized pretty quickly that designing our own chips was going to be the best path to delivering this value for customers. We were lucky to find and join forces with the amazing Annapurna Labs team, who started with a chip (named Nitro) that offloaded security, networking, and some other virtualization functions from our servers so customers could use more of the server than they could before. Then, that team built a generalized CPU chip, Graviton, which has been very popular and impactful for customers, before embarking on building custom AI chips—Trainium (for training) and Inferentia (for inference)—which are also off to a strong start.
Am very excited about our most recent chip releases at AWS re: Invent: Graviton4 and Trainium2. Graviton4 marks the fourth generation we’ve delivered in just five years (you can see the evolution from left to right in the image below), and it’s the most powerful and energy efficient multipurpose chip we have built to date. And with the surge of interest in generative AI, Trainium2 will help customers train their ML models faster, at a more advantaged price-performance. I’m really proud of the pace of innovation our teams are delivering on and what it is making possible for customers! aboutamazon.com/news/aws-rei…
Mauricio Ordóñez retweeted
Replying to @geoffreyhinton
One thing we know is that if future AI systems are built on the same blueprint as current Auto-Regressive LLMs, they may become highly knowledgeable but they will still be dumb.
They will still hallucinate, they will still be difficult to control, and they will still merely regurgitate stuff they've been trained on.
MORE IMPORTANTLY, they will still be unable to reason, unable to invent new things, or to plan actions to fulfill objectives.
And unless they can be trained from video, they still won't understand the physical world.
Future systems will *have* to use a different architecture capable of understanding the world, capable of reasoning, and capable of planning so as to satisfy a set of objectives and guardrails.
These objective-driven architectures will be safe and will remain under our control because *we* set their objectives and guardrails and they can't deviate from them.
They won't want to dominate us because they won't have any objective that drives them to dominate (unlike many living species, particularly social species like humans). In fact, guardrail objectives will prevent that.
They will be smarter than us but will remain under our control.
They will make *us* smarter.
The idea that smart AI systems will necessarily dominate humans is just wrong.
Instead of scaling current systems 100x, which will go nowhere, we need to make these Objective-Driven AI architectures work.
Mauricio Ordóñez retweeted
A cancer drug designed with the aid of #AlphaFold has already 'shipped' and is in clinical trials tinyurl.com/3bc7v8pf - but medical research takes time and protein structure is just one component. We're working on other aspects too @IsomorphicLabs to reimagine drug discovery
Mauricio Ordóñez retweeted
Indeed. Would love a coherent explanation of how we get from LLMs that confidently make up anything — the answers to simple math problems, the name of Albert Einstein’s cat etc etc etc — to 🤖DIE DIE DIE 🤖.
Mauricio Ordóñez retweeted
I've been learning RLHF (Reinforcement Learning From Human Feedback) for a while now (I know quite late to the party) and here are some of the best resources that I've come across that have really helped me understand it 👇
(PS: my annotated notes at the end)
A thread 🧵
Mauricio Ordóñez retweeted
I was playing with LEGOs with my daughter but then she worked out what I was doing and she was not impressed. An attempt at a (very poor) CMOS NOR-gate. The vertical red lines are polysilicon that provide the A and B inputs to the circuit. When polysilicon crosses n-type
Mauricio Ordóñez retweeted
Common Q: Can you train language model w diffusion?
Favorite A: read this post (the whole blog is excellent)
(Roughly speaking state of the art generative AI is either trained autoregressively or with diffusion. The underlying neural net usually a Transformer.)
New blog post about diffusion language models: benanne.github.io/2023/01/09…
Diffusion models have completely taken over generative modelling of perceptual signals -- why is autoregression still the name of the game for language modelling? And can we do anything about that?
Mauricio Ordóñez retweeted
LLMs like ChatGPT are becoming more fluent – how can we detect if something was written by a language model or a human?
We developed DetectGPT: a method for detecting if a passage was written by a particular language model.
Mauricio Ordóñez retweeted
🔥 New (1h56m) video lecture: "Let's build GPT: from scratch, in code, spelled out."
youtube.com/watch?v=kCc8FmEb…
We build and train a Transformer following the "Attention Is All You Need" paper in the language modeling setting and end up with the core of nanoGPT.
Mauricio Ordóñez retweeted
The impact of RL has been exactly as small as I had predicted.
The impact of Self-Supervised Learning has been even bigger that I had predicted.
Mauricio Ordóñez retweeted
Stable Diffusion with Core ML on Apple Silicon
@Apple releases optimizations to Core ML for #StableDiffusion in macOS 13.1 and iOS 16.2, along with code to get started with deploying to Apple Silicon devices. Wonder if iPhones will eventually come with SD.machinelearning.apple.com/re…






































