Generating variety is not intelligence. Overwhelming regulators with too much of it is a road to perdition.
As we appear to be relearning the hard way over the course of this year, most spectacularly (so far) only last week...😬
So this is how civilisation ends?
Generating variety is not intelligence. Overwhelming regulators with too much of it is a road to perdition.
Ashby 1960:
The processes that interest us, and about which dispute has raged, are those (whether carried out by brain or digital computer) in which the end-product shows evidence of high selection. For instance, of all the ways in which a bookful of letters might be arranged, one set was actually produced by one Shakespeare showing evidence of very intense selection. And a computer has emitted a string of digits corresponding exactly with the first thousand digits of π. Most practical activities show this aspect of selection as an essential component (as Sommerhoff has shown extensively in his Analytical Biology, 1950). Roughly, getting right answers implies selection.
3
Richard S O'Rourke retweeted
God did not solve the alignment problem. Why do we think we can?
Humans are cruel, vain, selfish, impulsive and frequently badly behaved. And yet, over centuries, we’ve built institutions that have made us considerably better. These institutions are imperfect. They allow for mistakes, disagreement and change. And … they allow us to discover that our inherited conceptions of goodness are wrong.
Believers of perfectly aligned AI promise machines that are completely good, know what we want, help us become who we wish to be and eventually eliminate the gap between our desires and our ideals. But without moral agency what are we? Do we lose something important about how goodness is discovered and created?
The strange virtues of sin, a history of policing, authoritarianism, hedonistic monsters and more in:
not to put you on the spot...
what say you @dsquareddigest ?
Replying to @BritishProgress
We made the front page of the Times with our killer stat that the UK pays, on average, 65 per cent more per unit of infrastructure than comparable countries.
Britain can build more and do more for less.
Congratulations to authors @Sam_Dumitriu, @Michael_J_Hil, @dc_lawrence and @pdmsero.
5
So this is how civilisation ends?
Generating variety is not intelligence. Overwhelming regulators with too much of it is a road to perdition.
Ashby 1960:
The processes that interest us, and about which dispute has raged, are those (whether carried out by brain or digital computer) in which the end-product shows evidence of high selection. For instance, of all the ways in which a bookful of letters might be arranged, one set was actually produced by one Shakespeare showing evidence of very intense selection. And a computer has emitted a string of digits corresponding exactly with the first thousand digits of π. Most practical activities show this aspect of selection as an essential component (as Sommerhoff has shown extensively in his Analytical Biology, 1950). Roughly, getting right answers implies selection.
Replying to @maxrumpf
The total volume of text in the world dwarfs the internet. What AI has written this year exceeds both.
The web's growth rate has fallen to the lowest level in a decade (+8%), while AI output is growing 9-15x annually.
The most valuable information AI creates won't be shared with everyone, but will feed an exploding corpus of private data.
Companies and individuals are now sitting on amounts of text previously reserved for governments and a handful of enterprises.
Ashby 1956 Design for an "intelligence-amplifier":
The Criterion of Intelligence
Let us first be clear about what we want. There is no intention here to inquire into the "real" nature of intelligence (whatever that may mean). The position is simple: we have problems and we want answers. We proceed then to ask, where are the answers to be found?
It has often been remarked that any random sequence, if long enough, will contain all the answers. Nothing prevents a child from doodling
cos^2 x + sin^2 x = 1
or a dancing mote in the sunlight from emitting the same message in Morse or a similar code. Let us be more definite. If each of the above 13 symbols might have been any one of 50 letters and elementary signs, then as 50^{13} is approximately 2^{73}, the equation can be given in coded form by 73 binary symbols. Now consider a cubic centimeter of air as a turmoil of colliding molecules. A particular molecule's turnings after collision, sometimes to the left and sometimes to the right, will provide a series of binary symbols, each 73 of which, on some given code, either will or will not represent the equation. A simple calculation from the known facts shows that the molecules in every cubic centimeter of air are emitting this sequence correctly over a hundred thousand times a second. The objection that "such things don't happen" cannot stand.
Doodling, then, or any other random activity, is capable of producing all that is required. What spoils the child's claim to be a mathematician is that he will doodle, with equal readiness, such forms as
cos^2 x + sin^2 x = 2 or c i ) x s i = n x 1
or any other variation. After the child has had some mathematical experience he will stop producing these other variations. He becomes not more but less productive: he becomes selective.
The close, indeed essential, relation between intelligence and selection is shown clearly if we examine the tests specially devised for its objective measurement. Take, for instance, those of the Terman and Merrill2 series for Year IV. In the first test the child is shown a picture of a common object and is asked to give its name. Out of all the words he knows he is asked to select one. In the second test, three model objects — motor-car, dog, showsic3 — are placed in a row and seen by the child; then all are hidden from him and a cover is placed over the dog; he is then shown motor-car, cover, shoe, and asked what is under the cover. Again his response is correct if, out of all possible words, he can select the appropriate one. Similarly the other tests, for all ages, evoke a response that is judged "correct" or "incorrect" simply by the subject's power of appropriate selection.
The same fact, that getting a solution implies selection, is shown with special clarity in the biological world. There the problems are all ultimately of how to achieve survival, and survival implies that the essential variables — the supply of food, water, etc. — are to be kept within physiological limits. The solutions to these problems are thus all selections from the totality of possibilities.
The same is true of the most important social and economic problems. What is wanted is often simple enough in aim — a way of ensuring food for all with an increasing population, or a way of keeping international frictions small in spite of provocations. In most of these problems the aim is the keeping of certain variables within assigned limits; and the problem is to find, amid the possibilities, some set of dynamic linkages that will keep the system both stable, and stable within those limits. Thus, finding the answer is again equivalent to achieving an appropriate selection.
The fact is that in admiring the productivity of genius our admiration has been misplaced. Nothing is easier than the generation of new ideas: with some suitable interpretation, a kaleidoscope, the entrails of a sheep, or a noisy vacuum tube will generate them in profusion. What is remarkable in the genius is the discrimination with which the possibilities are winnowed.
A possible method, then, is to use some random source for the generation of all the possibilities and to pass its output through some device that will select the answer. But before we proceed to make the device we must dispose of the critic who puts forward this well known argument: as the device will be made by some designer, it can select only what he has made it to select, so it can do no more than he can. Since this argument is clearly plausible, we must examine it with some care.
29
The total volume of text in the world dwarfs the internet. What AI has written this year exceeds both.
The web's growth rate has fallen to the lowest level in a decade (+8%), while AI output is growing 9-15x annually.
The most valuable information AI creates won't be shared with everyone, but will feed an exploding corpus of private data.
Companies and individuals are now sitting on amounts of text previously reserved for governments and a handful of enterprises.
Oh god! Not more AI psychosis…
@dsquareddigest
In my ongoing quest to find the limits of Opus 5.5, I directed it to audit my old scientific field, theoretical astrophysics. We indexed about 150,000 papers, audited 50,000 for issues, and did progressively more thorough checks on thousands more. We found nearly 1000 major errors that got cited many times, most are still uncorrected. This survey enabled me to estimate the efficacy of peer review for correcting errors (not very good) and to identify dozens of open questions that anyone with spare Claude Max tokens could answer today (.md context files included).
Bottom line: AI can speed up quantitative science by >1000x with no further model improvement required. But I'm seeing no evidence of this being pushed within universities, only academics complaining about labs dropping math proofs.
Among my fellow reformed academic high agency founders, however, several are speed running entire disciplines.
This survey has implications for how science will be run in the very near future. Hint: Academic publishing is dead. It was already a very stupid way to record and promulgate knowledge, and had been so for decades, even before all that publicly funded knowledge was locked behind paywall cartels. Now AIs can generate thousands of novel, correct papers per day, it is no longer meaningful for humans to consider a research paper a "unit of science" or to even bother much with the 99.9% of science which, up to this point, was necessary drudgery. Time to discover something like General Relativity or Evolution every week. LFG!!
Enjoy: science.caseyhandmer.com/phy…
2
311
I guess evidence of the important intelligence (appropriate selection) role played by peer review.
Certainly it can go wrong, as well documented in the philosophy and sociology of science, and like any process, it has an implicit error rate, à la Deming.
One of its critical roles is saving author’s blushes that might arise from irrational exuberance. Which is not a service offered by Twitter.
Not a man I like to quote but does appear apt: “Democracy is the worst form of government except for all the others that have been tried.”
Same would be true of Nietzsche: “In individuals, insanity is rare; but in groups, parties, nations and epochs, it is the rule.”
10
Richard S O'Rourke retweeted
The video of my talk, "What's in a name? From Norbert Wiener's Teleology to Artificial Intelligence" at SCiO yesterday is now up!
systemspractice.org/resource…
Richard S O'Rourke retweeted
Shannon and McCarthy's Automata Studies collection is mostly from Ratio Club and Macy meetings attendees.
McCarthy tried to rename it Towards Automata Studies. Shannon nixed that.
Ashby's Design for an Intelligence-Amplifier stands out
9 of 9. End Part 3.
all getting very cybernetic @TheUnjournaling @antlerboy
My new book, The Common Good Economy, argues that the goal is never enough on its own. Get the capital-labour relationship wrong, or the public-private one, or leave citizens out of the design, and the common good stays out of reach. Telos and polis. The goal and the community. The 'What' and the 'How'.
Spoke to @RoryStewartUK and @campbellclaret for @RestIsPolitics Leading about the new book, about Andy Burnham, and about why I set up @IIPP_UCL. Links in reply.
4
72
I like a flutter on the ggs every now and then. Anyone care to offer any wagers on what these people will be saying in 12 months time on such matters?
I'm looking for examples of what they might be saying. So in the case below, a very mundane example would be: well, I did say 1/3, so it wasn't likely to happen and things slowed down.
Odds: 10/1 on
A less mundane example would be: it wasn't going to happen after China took over Taiwan.
Odds: 10/1
Episode dropping tomorrow with @paulfchristiano.
He thinks there’s a 1 in 3 chance of extreme superintelligence within a year (absent any slowing down).
And he'll explain what labs and governments should do right now given that possibility.
1
68
Richard S O'Rourke retweeted
Norbert Wiener met, and was heavily influenced by William James?? 🤯
Thanks again @TheUnjournaling for all the wonderful historical research you do. Unravelling all of these connections feels really important at this moment in time
Richard S O'Rourke retweeted
I spent the evening with one of the most famous books regarding digital art. What stayed with me wasn’t just the images, but the words they used for them: “graphics,” “programming,” “systems,” “cybernetics.”Long before anyone talked about a market for digital art, artists and engineers were already inventing a language for a new way of seeing.
Are Ashby's “ideas as congenial to the technofascist as they are to the socialist”?
Skinner’s behaviourism versus Chomsky’s cognitive revolution and where Ashby might stand, and why it might matter.
Latest Substack on who is responsible for what agents do.
rsorourke.substack.com/p/are…
@benjaminjriley @dioscuri #ashbyancybernetics #cybernetics
or you could just call it how to operate an "intelligence-amplifier"
ashby.info/Ashby-Mechanisms_…
rsorourke.com/publications/2…
"prompt engineering" is just knowing what you want
"context engineering" is just knowing what's relevant
"harness engineering" is just knowing what to verify
"loop engineering" is just knowing when you're done
"goal engineering" is… just knowing what you want
4
71,390
Ashby:
1961, "Cybernetics Today"
Construction of a brain-like system
"cannot be begun until the question has been answered: What do you want?"
1970, "Analysis of the System to be Modeled". The primary question is
"not: What are my variables? but: What do I want?"
1972, "Setting Goals in Cybernetic Systems" opening paragraph:
"all these must start with the question 'What do you want?'"
2
7
337
I’ve been so unfair.
What would Heidegger say?
"We can affirm the unavoidable use of technical devices, and also deny them the right to dominate us, and so to warp, confuse, and lay waste our nature. [...] I would call this comportment toward technology which expresses 'yes' and at the same time 'no', by an old word, releasement toward things [Gelassenheit]."
— Heidegger, Gelassenheit (1959) translated by John M. Anderson and E. Hans Freund, Discourse on Thinking (1966)
Heidegger rejects the idea that changing your mind means simply swapping one rigid dogma for another. Instead, he advocates holding a simultaneous "yes" and "no" toward your tools, frameworks, and habits.
In strategic terms, this means you can fully commit to a plan, model, or belief while it serves the situation, yet remain detached enough to abandon it the moment reality shifts.
Alex Karp now says that Palantir is progressive, he’s always been a Democrat, Trump went too far on immigration and his primary commitment is to economic justice. LMAOOOOOO
1
1
78

























