Werner Bogula (@Bogula) header
Werner Bogula (@Bogula) profile picture

True digital native and analogue lute player / currently AI enabler @ https://nitter.gq/t.co/HmWbJswrPd

Hamburg
Joined January 2009
Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)
Werner Bogula retweeted
This is insane! 😱 You can now train a 100-billion parameters LLM on Google Colab. Explanation and code below ⤵️⤵️⤵️ --- TL;DR --- Distributed training over the Internet has become operational with the release of the new version of the PETALS distributed training package. Join forces with people from around the world and receive immense computing power in return for your use! 🌐⚡️ --- Intro --- Distributed training over the internet has turned from a dream into a reality with the release of the new version of the Petals distributed training package. Upon connection, the package grants you access to immense computing power that allows you to do anything you can imagine. It doesn't even have to be your own computing power: Attaching below is a Colab notebook that trains a 65-billion-parameter model (and also supports 100 billion. Yes, models with a hundred billion parameters are accessible to you). 💻🚀 --- It surely takes years, doesn't it? --- No. For example, with a 65-billion-parameter inference, you'll get a speed of 5 tokens per second. ⏱️⚡️ --- How does it work? --- The concept is very simple. In LoRA-based training (My tweet about LoRA - nitter.gq/ItakGol/status/1…), the base model is the same for everyone and does not change until the end of training, so many people need it at the same time. With Petals, people load small parts of the model, and each executes their part in the computation of running the full model. All the headache and complex software engineering required to make this madness run smoothly and quickly happens inside the package. 🤯🔧 --- How complex? --- From your own perspective, simply run: AutoDistributedModel.from_pretrained --- Important! Privacy! --- Obviously, this infrastructure sends your data outside to the world, so it's crucial to take that into account. Your code is never sent outside. 🌐🔒 --- Links --- 1. Article: Petals: Collaborative Inference and Fine-tuning of Large Models - aclanthology.org/2023.acl-de… 2. See the speed for yourself: Distributed Chatbot - chat.petals.ml/ 3. Notebook for training a 65-billion-parameter model: colab.research.google.com/dr… 4. Code: github.com/bigscience-worksh… 5. Paper: arxiv.org/abs/2209.01188 6. Join the community: discord.com/invite/D9MwApKgW…
How to Fine-Tune a Large Language Model (LLM) on a Single GPU or Google Colab? 😊 LoRA (Low-Rank Adaptation)! 🚀 Recently, LoRA has been a hot topic in the context of fine-tuning LLMs. But what does LoRA really mean? Let's dive in 📚 1 / 18
22
314
11
1,427
372,435