Popular research papers https://nitter.gq/t.co/oYO7p6A5uq https://nitter.gq/t.co/H0iO5rcfdp https://nitter.gq/t.co/jo9PHiu6SQ https://nitter.gq/t.co/oMf3AGVvtJ
Joined June 2021
- Tweets2.8K
- Following4
- Followers1.2K
- Likes75
papers.bar retweeted
🥳 Our "Annotated Research Paper Implementations" Github repo reached 25,000 stars!
github.com/labmlai/annotated…
It has over 50 paper implementations. Thank you everyone for the support!
papers.bar retweeted
GeoV-9b language model
@gharik & @labml is releasing a 9 billion parameter LM with open licenses to be useful in a variety of settings. The current version is trained for ~69B tokens and they will continue training for 300B tokens.
📗 Google Colab notebook: colab.research.google.com/gi…
📀 Model weights: huggingface.co/GeoV/GeoV-9b
🖥 Code: github.com/geov-ai/geov
The model is been trained on c4en and english wikipedia datasets.
It has a context length of 2048 tokens, 32 transformer layers and embeddings of size 5120. The model uses gated GELU activations & rotary embeddings with relative distances.
We are open sourcing GeoV-9b, a 9 billion parameter causal language model designed and trained by @gharik
🖥 Code (apache 2.0): github.com/geov-ai/geov
📀 Model weights (bigscience-openrail-m): huggingface.co/GeoV/GeoV-9b
📗 Google Colab notebook: colab.research.google.com/gi…
🧶👇
papers.bar retweeted
We released an iPhone app to find trending AI research papers. It's faster and easier to use on the phone than papers.labml.ai website.
⬇️ Download the iPhone app: apps.apple.com/gb/app/papers…
papers.bar retweeted
A Statistician Teaches Deep Learning
ai.papers.bar/paper/6c854db5…
Deep learning (DL) has gained much attention and become increasingly popular in modern data science. Computer scientists led the way in developing deep...
🧵 👇
papers.bar retweeted
We made a list of @iclr_conf 2023 submissions. You can click on the papers to see their source code, videos, and comments on social media.
papers.labml.ai/lists/iclr_2…
We will update it to show the status when the reviews are out
papers.bar retweeted
Implementation/tutorial of stable diffusion with side-by-side notes
📝 Code & notes: nn.labml.ai/diffusion/stable…
🖥 Github: github.com/labmlai/annotated…
It has annotated model code; DDIM and DDPM sampling; and scripts to generate and in-paint.
🧶👇
papers.bar retweeted
Annotated @PyTorch implementation of the paper "ZeRO: Memory Optimization Towards Training A Trillion Parameter Models"
📝 Code+Notes nn.labml.ai/scaling/zero3/in…
📎 Paper papers.labml.ai/paper/d6ec6a…
It enables training large NNs that do not fit in a single device.
🧶👇
papers.bar retweeted
Annotated @PyTorch implementation of Nucleus Sampling (and some other sampling methods)
📝 Code nn.labml.ai/sampling/nucleus…
🖥 Github github.com/labmlai/annotated…
📎 Paper papers.labml.ai/paper/1904.0…
Nucleus sampling selects a nucleus (tokens with high prob.) and samples from them.
papers.bar retweeted
Multimodal Learning with Transformers: A Survey
ai.papers.bar/paper/f05c0f90…
Transformer is a promising neural network learner, and has achieved great success in various machine learning tasks. This paper presents a...
🧵 👇
papers.bar retweeted
Rotary Positional Embeddings with Relative distance (RoPER) by @gharik and @vpj
research.labml.ai/RoPER.html
They investigate adding relative positions of the tokens it pays attention to, to the values of the attention calculation.
papers.bar retweeted
Autoregressive Search Engines: Generating Substrings as Document Identifiers
📝 Highlighted PDF github.com/labmlai/annotated…
📎 Paper papers.labml.ai/paper/42ca84…
This paper by @MetaAI introduces SEAL.
🧶👇
papers.bar retweeted
Training Compute-Optimal Large Language Models
📝 Highlighted PDF github.com/labmlai/annotated…
📎 Paper papers.labml.ai/paper/918d4f…
@DeepMind train over 400 language models to find the optimal model size and number of training tokens for a given compute capacity.
🧶👇
papers.bar retweeted
PaLM: Scaling Language Modeling with Pathways
✏️ Highlighted PDF: github.com/labmlai/annotated…
📎 Paper: papers.labml.ai/paper/eb71aa…
PaLM is a 540B model by @GoogleAI. It achieves SOTA on a variety of tasks. The 83 page paper analyzes the model capabilities in detail.
🧶 👇
papers.bar retweeted
EvoJAX: Hardware-Accelerated Neuroevolution by @yujin_tang @alanyttian @hardmaru
📎papers.labml.ai/paper/204639…
EvoJAX is a scalable, general purpose, hardware-accelerated neuro-evolution toolkit. It utilizes GPU/TPU and is 10-20x faster than baselines for a range of tasks.
👇
papers.bar retweeted
LaMDA: Language Models for Dialog Applications by @GoogleAI
papers.labml.ai/paper/281378…
Fine tuned language model to get human level dialog quality in terms of quality (sensibleness/ specificity/ interestingness), safety, and factual grounding.
Here's how it works:
1/6 🧵👇
papers.bar retweeted
🥳 Excited to share our Chrome browser extension for papers.labml.ai 🎉
chrome.google.com/webstore/d…
It identifies research papers mentioned in websites you visit and shows a 2-line summary, availability code/videos/discussions, popularity on Twitter, and conferences.
🧵👇
papers.bar retweeted
We added a page that lists major AI conferences
papers.labml.ai/conferences
@iclr_conf @NeurIPSConf ICCV @CVPR @icmlconf @IJCAIconf @emnlpmeeting @aclmeeting @RealAAAI
🔎 Search within conferences and find paper related resources such as videos, code, and discussions
papers.bar retweeted
A ConvNet for the 2020s by @MetaAI & @berkeley_ai
📎 papers.labml.ai/paper/9f4eea…
They modify ResNet architecture to build a convolution-based model (ConvNeXt) that performs better than transformer-based vision models, whilst keeping the simplicity.
1/9
Summary:
🧵👇
papers.bar retweeted
1/5 A Neural Network Solves and Generates Mathematics Problems by Program Synthesis: Calculus, Differential Equations, ...
📎 papers.labml.ai/paper/212cd3…
Shows that @OpenAI Codex can be used to generate code that solves math problems by rephrasing the them as prompts.
🧵👇
papers.bar retweeted
How can we improve papers.labml.ai (trending machine learning / deep learning papers)
(DMs open)
🙏
