Product Manager & AI Engineer working on Gen AI & ML. Opinions are my own, not my employer's. RT !=endorsement
Seattle, WA
Joined November 2021
- Tweets28.9K
- Following7.5K
- Followers4.1K
- Likes112K
iamrobotbear (bk) retweeted
@treetowntree and i created neuron lens: a database of every single neuron and attention head in qwen 3 8b and gemma 4 12b read using the R-lens (a J-lens variant) and what we call the "S-lens". you can find it at neuron-lens.com/, along with the tools we used to make it.
using the R-lens to read neurons' input and output projections turns out to be very effective at determining what a neuron does. we're able to legibly read what a majority of neurons do when fired: the lens qualitatively predicts the result of steering over 90% of the time!
the same trick lets us read attention heads' QK circuits and OV circuits, showing us both what they attend to and what they do with it. however, the R-lens has two important flaws: its accuracy degrades in early layers, and it can't separate the MLP block and the attention block.
we found ways around both these issues. we first modified the r-lens by fitting it on input tokens rather than output tokens to get the s-lens, which is accurate in early layers but inaccurate in late layers. we then fitted R- and S-lenses to the activations after attention but before the MLP to get "mid-lenses", which are much better at deciphering MLP inputs and attention outputs. using both of these techniques, we were able to interpret most of the models' layers.
iamrobotbear (bk) retweeted
a lot of decision models came out in the last two months. almost all of them work the same way. they read text, wait until the whole input has arrived, and then give you an answer
that works for a support ticket. it doesn’t work for a robot, a phone call, or a video feed, where the input never really ends and the decision has to happen while it is still coming in
so we spent half of our YC money training a model:) RightNow takes text, PDFs, images, voice, code, robot state and video, all in one model, and you can stream up to 1M tokens of context into it. it updates its decision as each piece arrives. on text it decides in 30 ms and we will make it less with our kernel optimization
we trained it with what we call Reflex Training. the current decision models read everything first and answer once. RightNow is trained to decide on whatever has arrived so far, to hold back until the evidence is there, and then commit. it serves on what we call Reflex Architecture which keeps a probability for every option and updates it as each part of the input comes in
we call this kind of model “System Zero”. system 1 is fast thinking and system 2 is slow reasoning. system zero is the step that comes before both, which is just deciding
we will publish a technical paper on Reflex Training and Reflex Architecture. the model itself will stay closed source and available via RightNow API
this is our first checkpoint and it’s still a little buggy. we launched it early because the fastest way to make it good is to have people use it:)
RightNow is out today. our first “System Zero” model
it streams in text, JSON, PDFs, images, voice, codebases, robot state and video, and returns a decision with a probability for every option while the input is still arriving
30 ms decisions on text. up to 1,024 questions in one call. output is free
system 1 is fast thinking. system 2 is slow reasoning. “System Zero” is the step before both -- the decision itself
this is our first checkpoint. there is a lot of work ahead, and we decided to ship early to get your feedback. more features and fixes are shipping this week, with more checkpoints after
rightnowai.co
iamrobotbear (bk) retweeted
Introducing *Switch*: a *harness framework* for *decision models* like Jev
I looked for this the other day and was surprised to not find anything like it
You obviously cannot use a decision model like an LLM in a coding agent harness like @pidotdev, because Jev cannot generate tool calls
But what if there was a framework that "generated the tool calls" for it programmatically, and Jev only had to pick out of that finite set of choices
That is what switch does. You pick a "domain" to implement the decision model in, and it provides a core API to define your state, and the functions that will derive decision model questions out of that state. Chess is a domain, as well as being a rubric judge for evals
It basically standardizes what I imagine a lot of people are already vibe-coding, to an abstract interface: calling a decision-model API through multiple steps, i.e. an agent loop over finite choices
Because of the finiteness of the output of a decision model, you cannot have a general pure decision model harness. Instead, you need to implement specific ones that generate the choices programmatically (or have an LLM generate the choices, but I find the former more interesting)
A primary use case I have for this is use as a judge in an eval. For example, At the end of a task run that involves a bunch of documents, most people currently follow this approach: extract text out of PDF/docx/md files, and dump them right into the context
But what if the visual formatting of the output is a judging criterion? Then you have to dump an image of every page in the input state. But what if you have 200 pages, and your question does not need looking at all of them? What if Jev could navigate through the haystack and find the info it needs?
I have noticed that Jev is not so good at chess nor being a Decision-Model-as-a-Judge. It sucks at chess, and it does not perform well enough to replace GPT 6.1 Sol as a judge in the domain I am working for
But switch is not only for Jev! It is for any decision model including the ones supported by the pi-ai package
So once you can generate enough data with a better model, you should be able to pick an open source one and fine-tune it on your use case to perform better! And the model should be trained in a harness like switch's to be especially performant and cost-efficient
I wanted to get this out because I find the idea very exciting, at least for benchmarking. Jev is currently not there, but decision models obviously will be there at some point, and we can take them there! There are also a lot of potential gains from picking a specific domain, and creating a looping agent that does a job for 100x cheaper!
Let me know if you have any ideas, I am quite keen to try this on new domains!
Repo: github.com/osolmaz/switch
iamrobotbear (bk) retweeted
Meet Pika H3 Flex, debuting today as the #1 open weight model on DesignArena’s t2v board.
It’s built for professional iteration, with exportable audio stems, resolutions up to 4k, and faster-than-realtime generation. It can produce a 15-second clip with soundtrack in as little as 7 seconds.
So, yeah. It’s kind of a flex 💅
iamrobotbear (bk) retweeted
Starting today, eligible ChatGPT Pro and Business subscribers can claim up to $500 per month in API credits to build apps and agents.
iamrobotbear (bk) retweeted
Meet hunch! Jev-like decisions in ~20ms without even needing a GPU, learning live from a teacher model.
Built on @GoogleGemma's EmbeddingGemma 2, it can works with any teacher model like @CloudFlare's Clef.
Multimodal, w/a systemone endpoint, slots into your existing stack 🧵
iamrobotbear (bk) retweeted
Introducing Vela 3.0 📡 Name it. Vela finds it.
Six open models (0.3B–27B): decisions + the spans you name. One call.
⚡ 380M: #1 Decision Index ≤500M
🎯 Beats GPT-5-mini at zero-shot classification
🔍 18 ms per span question
🤗 huggingface.co/collections/v…
#Jev #Decison
iamrobotbear (bk) retweeted
Watch this, understand it, and embrace what the future of reality will look like
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
iamrobotbear (bk) retweeted
When I said hunch could learn in real-time I wasn't kidding. It's like teaching a kid!
Behind the scenes, @Cloudflare’s Clef is helping refine the local model so it learns in real time, while @googlegemma’s EmbeddingGemma2 runs live on the CPU. 🧵
It's been a while since I showed any Jev stuff but I've been cooking on a way to use @GoogleDeepMind's EmbeddingGemma2 as an insanely fast real-time /v1/systemone endpoint (20ms on text, 100ms on images).
It's trainable in real-time, can distill an upstream teacher model, and is multi-modal by default. Hard to believe a @googlegemma model of this size can do so well with the right harness!
iamrobotbear (bk) retweeted
Introducing 𝙻𝙵𝙼𝟹: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence.
> 6 model instances, across text, audio, vision, and omni modalities, generative and decision models.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱: text, 256k context window.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱: vision-language model with video-input support for spatial understanding
> 𝙻𝙵𝙼𝟹-𝙰𝚞𝚍𝚒𝚘-𝟸𝙱: tool calls, voice chat, reasoning in 18 languages.
> 𝙻𝙵𝙼𝟹-𝙾𝚖𝚗𝚒-𝟸𝙱: an omni model combines vision, audio, and language.
> 𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input.
> fine-tune on 1 GPU.
> a new tri-hybrid architecture
> architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱 is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱 is #1 on the Pareto frontier of device intelligence 𝚙𝚒𝚙𝚎𝚝𝚝𝚎.𝚕𝚒𝚚𝚞𝚒𝚍.𝚊𝚒
Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device.
1/ 🧵
iamrobotbear (bk) retweeted
Genie 4 is quite an insane experience, generating game-like enviornments as you go. It isn't flawless but I can type "Neogothic brutalist city inhabited by robots and horses" & it creates one in a minute. Then I asked it to add dragons. Then a friendly otter NPC. All in realtime
iamrobotbear (bk) retweeted
Now that the dust has settled with Jev, here's where it works well and where it falls short
iamrobotbear (bk) retweeted
Introducing Reinforcement Learning from Collective Rewards (RLCR).
RLCR is a recipe that trains language models as agent organisations, which share a message board and workspace. We use it to train Avalon-M1-Mini (120B) and study its behaviour.
We find a number of interesting results, including:
🛠️ Learnt organisational hierarchies and divisions of labour
🕵️ Increased collective verification, through hires of verifiers and testers
👥 The rise of collective thinking, where agents think more about other agents and how to help them
We believe RLCR can provide a laboratory for studying collective intelligence and multi-agent safety.
Blog post and technical report 👇
Is it just me or is the @OpenAIDevs ChatGPT/Codex app been broken for side chat in the past ~1 week? @thsottiaux
iamrobotbear (bk) retweeted
Today, we announce 𝙻𝚒𝚚𝚞𝚒𝚍 𝚁𝙸𝙿𝙻 (real-time inference pipeline), our ultra low-latency inference engine built to put LFMs directly in the request path of real-time workloads like search, ads, fraud, recommendations, and payments.
> complete generated responses end-to-end in sub 50ms
> maintains low latency under continuous peak load
> LFMs get up to 20x faster on RIPL
> RIPL supports Multimodal LFMs
> RIPL supports d1, our decision models too!
RIPL optimizes the full path from request arrival to a complete generated response. As traffic rises, its performance advantage is greatest on the slowest requests. Let’s see some production use cases:
iamrobotbear (bk) retweeted
How on earth did this happen:
The Indian certificate authority (CA) eMudhra had one job (well, two):
1. Create secure, 384-bit private keys
2. Store these securely and do not leak them
Well, eMudhra chose private keys between 1 and 256 (the numbers!!) and called it a day. WTF
iamrobotbear (bk) retweeted
this is great deepdive into what a managed harness looks like! some quick thoughts while reading:
- many diff types of agents built on it. doesnt surprise me - a lot of agents look very similar inside these days
- why not use off the shelf? (1) model and provider agnostic), (2) co-build with product teams, (3) own observability stack. I very much agree with points 1 and 3!!! and is something we obviously emphasize at langchain
- "The simplest agent harnesses are truly just a while loop, but when you want to support a wide array of agentic products in production, the responsibility of the harness or framework expands quite a bit" - +100 to this, is why were investing in deepagents so much
- "Using the “sandbox-as-a-tool” paradigm" - the massive trend seems to be going this way (is how deepagents is architected)
- "Supporting non-blocking async agents as the default". we are working on this right now for deepagents! agree is the future
- "The Connectors Service is the standard interface through which agents at Ramp interact with data sources. It centralizes authentication, credential management, access controls, and tool execution behind one API. This allows us to build and maintain integrations once, and reuse them across products." Also very important! and something we're thinking a bunch about for managed deepagents
We at Ramp tried a bunch of agent harnesses and ended up building our own. Cloud-native, model-agnostic, tailored to our infrastructure and product.
And we published how we built it: builders.ramp.com/post/build…
iamrobotbear (bk) retweeted
BREAKING PROOF!! Continuity we perceive in consciousness is an illusion! We flip between conscious and unconscious states every 60 milliseconds and we recognize backward speech when played every 20 milliseconds. Listen to this!
Speech can survive being flipped backward every twenty milliseconds.
The same speech in backward pieces of 200 then 100 then 50 then 20 milliseconds.
The original comes last.
When do the words become clear?
A listening demo from @OrukLabs.
iamrobotbear (bk) retweeted
LLMs were just the beginning. Bullish on world models, and I think they are the next frontier in AI.
Amazing to see a dedicated world model for engineering this early. SKY-001 against GPT-6 Astra matches accuracy on 3D construction at about 150x lower cost per sample. On BIM, it runs over 100x faster.
🤝 Paid partnership
Introducing SKY-001, the first Engineering World Model.
On CAD and BIM tasks, it beats GPT-6 Astra and every frontier model we tested:
-> 51-125x faster
- > 150-210x cheaper
-> Higher IOU than all frontier models
SKY-001 lets you dream a physical object before you build it.
18 months of work. Built by the team out of Maluuba's deep learning lab (acq. by @MSRResearch).
The first ‘AI General Engineer’ will be the backbone for Autonomous Manufacturing.
iamrobotbear (bk) retweeted
if your attention span is cooked like mine skip to 57 seconds in and you'll immediately understand why this is a great app
you'll then not download it, go to do things the old way, and immediately revisit this tweet from your bookmarks to set it up
