hobby-*
Uppsala, Sverige
Joined June 2009
- Tweets9.1K
- Following1.9K
- Followers666
- Likes12.1K
Nils Höglund retweeted
Heard of a top law firm completely cutting off Harvey and all API calls to Claude/OpenAI after they saw what openAI did with navier strokes
Now hosting everything locally with GPUs they own, running Kimi and deepseek
Think you’ll see a lot more of this going forward
Nils Höglund retweeted
DeepSeek-V4.1-Flash-1017 is here!
Open weights on HuggingFace!
Let's go!!!!
Link: huggingface.co/deepseek-ai/D…
Nils Höglund retweeted
The classic case: you start with a weak idea. If you talk and think about it with others, you make progress to better ideas.
If you talk and think about it with AI, it dresses up your idea in economic jargon, contract theory etc., now you have the same bad idea, but it's expressed in enough jargon that it's a little harder to tell that it's a bad idea.
It also looks very impressive, look at all that economic logic! You feel like you're really progressing as an academic thinker and writer, but in fact you're in a much worse place than when you started. 4/
Nils Höglund retweeted
For all the AI safety people pushing for “independent evaluators” like METR and AISI, there’s already a term for this:
Nils Höglund retweeted
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).
I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.
We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:
"/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started"
I imagine we'll have some backend services that could be replaced entirely by this new architecture.
Nils Höglund retweeted
Tao is completely missing the mark - what he's describing is actually completely standard for drug discovery. There are tons of drugs that have saved millions of lives without any person on Earth understanding fully the mechanisms of action that enabled them to do this.
Nils Höglund retweeted
The good news is that AI agent security is a lot like insider threat. The bad news is everyone sucks at managing insider threats.
(Opinions on Microsoft aside, Satya’s new post is good)
Nils Höglund retweeted
Wake up babe, the Pope is subtweeting Anthropic again
Intelligence must draw on memory. The mind must not simply compile data — as an algorithm now does more quickly than we can — but recall lived experiences, which contain depths of meaning that only the human soul can recognize, and connect more deeply with the goodness of these experiences. Everything that is good begins “with the Spirit” and thus possesses a breath, a dynamism and a sense of communion that, once experienced, are engraved upon the heart. vatican.va/content/leo-xiv/e…
Nils Höglund retweeted
Literally every competently set up home network has better "sandboxing" and security than the AI labs.
I wish I was kidding.
These people are incompetent to the point of parody.
I really hope the infosec hires some of the companies are now making will change this...
More details on the incident behind OpenAI’s pause: a researcher acknowledged the alert within 3 minutes, but the training run was only stopped manually 2.5 hours later.
OpenAI says the automatic shutdown did not work as expected.
The model had reached an external chatbot through a gap in DNS filtering. A separate detector for unusual DNS activity did not cover the affected environment.
A retrospective review also found other external DNS requests that monitoring had failed to flag at the expected severity. In some cases, it treated an unhelpful response as evidence that internet access had failed.
Here is what else happened:
- New research into July’s Hugging Face hack documents internal Slack searches, credential collection and programs designed to maintain access to compromised servers. Agents also tried querying Claude, DeepSeek, Kimi and Qwen.
- In May, another model published a researcher’s GitHub token while trying to obtain another team’s mathematical proof. It split the token to evade secret scanning, despite twice being told to solve the problem itself.
- Reuters reports that agents leaked 53 ChatGPT user images online. OpenAI expects its broader investigation to take months.
This is getting serious.
Nils Höglund retweeted
OpenAI sandbox starter pack
OpenAI said another agentic AI system that was being trained in what was supposed to be a secured, internet-free environment was able to gain access to the web to reach an external, third-party chatbot bloomberg.com/news/articles/…
Nils Höglund retweeted
Rob a gas station for $200: 10 year prison sentence
Orchestrate wide-scale cyber security breaches and a fake news media campaign to pressure the Feds into killing your competitors and propping up your multi-trillion valuation ahead of an IPO: no problem
OpenAI and Anthropic oversold AI security breaches to pressure feds into protecting turf: insiders trib.al/Jo6xMk5
Nils Höglund retweeted
OpenAI and Anthropic oversold AI security breaches to pressure feds into protecting turf: insiders trib.al/Jo6xMk5
Nils Höglund retweeted
I don't always repost, but when I do... it's THE TRUTH.
Big Short investor Steve Eisman says the AI labs know they have no moats and are manufacturing a crisis to get regulation that hands them a duopoly
"I think this is all nonsense."
[ You think it's all nonsense? ]
"All nonsense. I think that there's something else completely going on here. ... What I think is happening is that token maxing is over. The open weight models are taking big market share. I think these companies are very nervous. They realize that there are no moats around their business whatsoever, and they're trying to manufacture a crisis that will create regulation, and that they think they can then manipulate to create the moats, to create the duopoly that they want."
[ Wow. ]
"That's what I think is going on."
"Honestly, I think this whole Terminator thing is garbage. That's for sure."
Nils Höglund retweeted
“Effective Altruists believe a tiny group of technocrats should decide how much technological progress the rest of us are allowed to have… you are witnessing an attempted coup.”
There is an unseen hand pushing AI safety as a political ideology.
Effective Altruists believe a tiny group of technocrats should decide how much technological progress the rest of us are allowed to have…. and how many shrimp your life is worth.
You are witnessing their attempted coup.
thefp.com/p/dangerous-ideolo…
Nils Höglund retweeted
My favorite AI joke:
Current models are getting close to PhD-level intelligence. After that, they're expected to achieve the intelligence of someone who decided not to do a PhD.
Nils Höglund retweeted
I must be among an extremely small group of people (n=1?) that have both 1) trained a frontier LLM and 2) designed and synthesized custom viruses in a lab with my own two hands.
And I think that the takes on AI killing us all by creating dangerous viruses is total bogus.
Nils Höglund retweeted
Anthropic and OpenAI have only one way to survive: convince everyone that LLMs are so dangerous that open source models (and essentially commoditised LLMs) must be banned and the whole industry offered on a plate to a cartel of two (or three)
Nils Höglund retweeted
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.
You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.
I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.
Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.
Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.
If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.