Founder of #DataScienceInitiative. #ML / #MLOps / #AI / #Data @Microsoft. Prev. Cloud Architect @Oracle. Opinions are my own.
London
Joined February 2011
- Tweets20.9K
- Following806
- Followers1.3K
- Likes2.8K
Fatos retweeted
AI agents are writing more code. How can developers ensure that code is correct?
Microsoft Research’s Proof-Driven Development (PDD) Kit explores a more rigorous approach: it translates a developer’s intent into a formal specification, then uses formal verification to prove that generated code meets it.
PDD Kit works with Verus for Rust and is integrated with @GitHubNext’s Chopin, bringing formal methods into an AI-assisted development workflow for teams.
See the work in action at @github Universe, Oct. 28–29, 2026.
Fatos retweeted
There are 2 career paths in AI right now:
The API Caller: Knows how to use an API. (Low leverage, first to be automated, $150k salary).
The Architect: Knows how to build the API. (High leverage, builds the tools, $500k+ salary).
Bootcamps train you to be an API Caller. This free 17-video Stanford course trains you to be an Architect.
It's CS336: Language Modeling from Scratch.
The syllabus is pure signal, no noise:
➡️ Data Collection & Curation (Lec 13-14)
➡️ Building Transformers & MoE (Lec 3-4)
➡️ Making it fast (Lec 5-8: GPUs, Kernels, Parallelism)
➡️ Making it work (Lec 10: Inference)
➡️ Making it smart (Lec 15-17: Alignment & RL)
Choose your path.
(I will put the playlist in the comments.)
♻️ Repost to save someone $$$ and a lot of confusion.
✔️ You can follow @techNmak, for more insights.
Fatos retweeted
Excited to release new repo: nanochat!
(it's among the most unhinged I've written).
Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.
It weighs ~8,000 lines of imo quite clean code to:
- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.
Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.
My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.
Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.
Fatos retweeted
I’m excited to share a project I’ve been working on for over a year, which I believe will fundamentally change our approach to language models.
We’ve designed a new architecture, which replaces the hidden state of an RNN with a machine learning model. This model compresses context through actual gradient descent on input tokens. We call our method “Test-Time-Training layers.”
TTT layers directly replace attention, and unlock linear complexity architectures with expressive memory, allowing us to train LLMs with millions (someday billions) of tokens in context.
Our instantiations, TTT-Linear and TTT-MLP, both match or beat the strongest Transformers and Mamba. Arxiv: arxiv.org/abs/2407.04620
Fatos retweeted
🚨 BREAKING: Microsoft launches a free Python library that converts ANY document to Markdown
Introducing Markitdown. Let me explain. 🧵
Fatos retweeted
🧵I’m excited to share not one but two research papers, written jointly by researchers from OpenAI and the @medialab at MIT. We try to answer the following question: How do interactions with AI chatbots affect people’s social and emotional well-being?
Fatos retweeted
Webinar announcement 📣
Find out what Machine Learning actually is and how to enter the world of AI.
Perfect for anyone considering this career path or is just curious!
Free tickets at alsauk.co.uk/events
Fatos retweeted
Here is a list of everything that happened in AI Agents this week 🧵
(save for later)
ALT AgentOps is the best tool for Agent Observability, monitoring, and compliance. Every Agent Needs AgentOps. Industry leading developer platform to test and debug AI agents. We built the tools so you don't have to. Start for Free Docs Powering thousands of engineers building reliable agents CrewAI Visualize Visually track events such as LLM calls, tools, and multi-agent interactions. Time Travel Debugging Rewind and replay agent runs with point in time precision. Debug and Audit Keep a full data trail of logs, errors, and prompt injection attacks from prototype to production. One SDK. Many integrations. Native integrations with the top agent frameworks srcsrcsrc Track spending Across multiple agents. GPT -4 Anthropic Research Base AgentFine-tuned Agent Token Counts Track, save, and monitor every token your agent sees. Cost Tracking Manage and vizualize agent spend with up-to-date price monitoring. Fine-tuning Fine-tune specialized LLMs up to 25x cheaper on saved completions.
Fatos retweeted
Excited to announce the release of Eureka, an open-source framework for evaluating and understanding large foundation models! 🌟
Eureka offers: 🔍In-depth analysis of 12 cutting-edge models 🧠 Multimodal & language capability testing beyond single-score reporting and rankings 📈 Insights into model strengths, weaknesses, determinism, and backward compatibility.
Join us in exploring the next AI frontier and contribute to open-source evaluations & insights!
Blog: aka.ms/eureka-ml-insights-bl…
Technical report: aka.ms/eureka-ml-insights-re…
Github: github.com/microsoft/eureka-…
Website: microsoft.github.io/eureka-m…
#ArtificialIntelligence #AI #LLM #ResponsibleAI #AISafety #AIFrontiers #MicrosoftResearch
Fatos retweeted
The important details you need to know for our 2024 Albanian Youth Empowerment Summit❤️
You 𝗺𝘂𝘀𝘁 sign up on our website to register a place 𝗯𝗲𝗳𝗼𝗿𝗲 𝗙𝗿𝗶𝗱𝗮𝘆, tickets are still available!
Any questions? DM us📩
#albanian #networking
Fatos retweeted
The Batch API now supports embedding and vision models. We’ve also increased the requests-per-batch limit from 10,000 to 50,000, and added the ability to manage batches from the dashboard. See our new getting started guide: platform.openai.com/docs/gui…
Fatos retweeted
Majorly improved GPT-4 Turbo model available now in the API and rolling out in ChatGPT.
GPT-4 Turbo with Vision is now generally available in the API. Vision requests can now also use JSON mode and function calling.
platform.openai.com/docs/mod…
Below are some great ways developers are building with vision. Drop yours in a reply 🧵
Fatos retweeted
Have you heard of the 12-Factor App?
The "12 Factor App" offers a set of best practices for building modern software applications. Following these 12 principles can help developers and teams in building reliable, scalable, and manageable applications.
Here's a brief overview of each principle:
1. Codebase:
Have one place to keep all your code, and manage it using version control like Git.
2. Dependencies:
List all the things your app needs to work properly, and make sure they're easy to install.
3. Config:
Keep important settings like database credentials separate from your code, so you can change them without rewriting code.
4. Backing Services:
Use other services (like databases or payment processors) as separate components that your app connects to.
5. Build, Release, Run:
Make a clear distinction between preparing your app, releasing it, and running it in production.
6. Processes:
Design your app so that each part doesn't rely on a specific computer or memory. It's like making LEGO blocks that fit together.
7. Port Binding:
Let your app be accessible through a network port, and make sure it doesn't store critical information on a single computer.
8. Concurrency:
Make your app able to handle more work by adding more copies of the same thing, like hiring more workers for a busy restaurant.
9. Disposability:
Your app should start quickly and shut down gracefully, like turning off a light switch instead of yanking out the power cord.
10. Dev/Prod Parity:
Ensure that what you use for developing your app is very similar to what you use in production, to avoid surprises.
11. Logs:
Keep a record of what happens in your app so you can understand and fix issues, like a diary for your software.
12. Admin Processes:
Run special tasks separately from your app, like doing maintenance work in a workshop instead of on the factory floor.
Over to you: Where do you think these principles can have the most impact in improving software development practices?
--
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/3KCnWXq
Fatos retweeted
VC-backed AI employee startups are a trend.
Here are some that raised in 2024 + total funding:
Software Engineer - Cognition ($21M+)
Software Engineer - Magic ($145M+)
Product Manager - Version Lens ($1.6M)
Data Scientist - TextQL ($4.1M)
Data Analyst - Fluent ($7.5M)
Chief of Staff - Mindy ($6M)
Universal Employee - Ema ($25M)
Financial Analyst - Finpilot ($4.5M)
Financial Analyst - Rogo ($7.5M)
Compliance - Norm Ai ($11.1M)
Receptionist - Arini ($0.5M)
Loan Officer - Casca ($3.9M)
Hotel Concierge - Runnr ($1.2M)
Designer - SevnAI ($0.5M)
Customer Support - Sierra ($110M)
Customer Support - Rasa ($70M)
Fatos retweeted
Amazing new work by Andreas Kopp! A new Solution Accelerator 🚀 designed to integrate advanced image generation capabilities while ensuring compliance and brand safety.
✅ Screen user-provided scene descriptions to filter out jailbreak attempts and NSFW content.
✅ Apply domain-specific rules like neutralizing competitor brands within images.
✅ Adjust scene prompts to align with specific themes and styles as required.
✅ Use AI models, such as DALL-E 3 or Stable Diffusion XL, for high-quality image creation.
✅ Post-generation checks to ensure images are free from NSFW content or unintended brand associations.
Repo github.com/Azure/gen-cv/tree…
Fatos retweeted
𝙁𝙧𝙚𝙚 𝙖𝙣𝙙 𝙊𝙥𝙚𝙣 𝙎𝙚𝙧𝙞𝙚𝙨 𝙩𝙤 𝙇𝙚𝙖𝙧𝙣 𝘽𝙡𝙤𝙘𝙠𝙘𝙝𝙖𝙞𝙣 𝙖𝙣𝙙 𝙎𝙢𝙖𝙧𝙩 𝘾𝙤𝙣𝙩𝙧𝙖𝙘𝙩 𝘿𝙚𝙫𝙚𝙡𝙤𝙥𝙢𝙚𝙣𝙩 📚𝙄𝙣 𝙋𝙧𝙞𝙨𝙝𝙩𝙞𝙣𝙖 🇽🇰 𝙖𝙣𝙙 𝙏𝙞𝙧𝙖𝙣𝙖 🇦🇱
Since last year I've been planning on doing a series of free lectures as a part of a course
Fatos retweeted
ML Papers of the Week (6K⭐️)
If you are looking for good recent LLM papers to read, you don't need to look too far.
We have been tracking the most popular, impressive, and trending ones in the "ML Papers of the Week" repo.
Our paper collection is used by thousands of students, researchers, and developers.
We use this list to keep up to date on all the latest developments in AI and LLMs @dair_ai. We have also implemented and combined ideas from a few of these papers to power some of the LLM services we provide for companies.
My recommendation for finding good papers is to narrow your search. While this is a bit tedious with the repo, we are working to make it even easier to find specific papers based on your interests and searches (more on this soon).
We have also converted the list of papers into a dataset for further analysis if that's useful for you.
Fatos retweeted
All you need to know about the Transformer architecture:
👉 How to structure the inputs
👉 Attention (Queries, Keys, Values)
👉 Position embeddings
👉 Residual connections, etc.
📺 youtu.be/ec9IQMiJBhs
Bonus: Overview of the difference between RNNs and transformers.
