Pinned Tweet
Hey #econtwitter, do you have a binary outcome? Estimate logit models? Want to cluster your standard errors? We've got bad, good, and great news for you. See the thread below, or our new working paper (with @jgmQED and @MortenEcon) for details arxiv.org/abs/2406.00650 1/7
Matt Webb retweeted
I've been pointing out since the early 2010s that Thomas Piketty, Emmanuel Saez, and Gabriel Zucman have a bad habit of biasing their data construction to fit their politics.
Whenever they make a suspect call in how to handle stats, it ends up showing that inequality is on the rise. They almost never go in the other direction. The errors are non-random and almost invariably reinforce their political claims about rising income and wealth concentration since 1980.
Piketty did this throughout his book Capital in the 21st Century.
Piketty and Saez did this throughout their famous 2003 paper on inequality in the US.
Piketty, Saez, and Zucman Saez and Zucman did this in their Distributional National Accounts paper that Auten and Splinter dissected.
Saez and Zucman did this in their 2019 book on tax incidence and inequality.
Saez and Zucman did this in their flurry of "studies" and white papers to support Elizabeth Warren's wealth tax proposal, California Prop 40, a new wealth tax in France, and dozens of other similar examples.
A new AI report (linked below) audited 138 papers by Saez and Zucman. It found thousands of errors, 194 of which were serious. A whopping 97% of the serious errors went in the same direction: they showed inequality rising faster or made it higher.
But there's a bigger scandal at play here. Despite these clear biases, Piketty, Saez, and Zucman have enjoyed a stream of praise and prestige within the economics profession. They get expedited and friendly reviews in top econ journals even though a more serious peer review should have caught many of these errors (the Quarterly Journal of Economics is a recurring offender in this regard). Journals regularly give them a pass after the errors in their old work come to light - they're never expected to correct any of it, and just keep on using it as if nothing changed. Their non-peer reviewed data claims are also treated as gospel by major media outlets such as the New York Times and the Washington Post. And they get showered with prizes - Saez and Zucman are both Clark Medal winners from the AEA, and Zucman got his prize in 2023 after evidence of major data shenanigans in his work came to light in 2019.
Until now, those of us who have been trying to correct the record have had to wade through the PSZ trio's cluttered, disorganized, and often non-transparent replication packages (almost all of which are convoluted piles of Excel spreadsheets) to find the problems. I did a line-by-line replication of Piketty and Saez's entire 2003 US income inequality series to use one example, and found it was riddled with errors - almost every single one of which biased their results in favor of their political story.
We had the means to catch this stuff as it was happening, but the economics profession failed to do its job. And it failed precisely because its elite ranks wanted to believe the Piketty-Saez-Zucman story for political reasons.
Matt Webb retweeted
Very funny that billionaires are like “I am leaving”. And Zucman’s response is “sorry no, my theory doesn’t allow this”
Matt Webb retweeted
Aidan Toner-Rodgers (infamous for being kicked out of the MIT economics PhD program for a fraudulent AI paper (which was R&R QJE & then pulled)) now has another paper retracted, this time from the Journal of Development Economics. Retraction notice here: sciencedirect.com/science/ar…
Matt Webb retweeted
Looking for 125 researchers to participate in virtual forensic AI games! We will give you an AI report that documents irregularities in sports/exercise related articles.
Also releasing our open source new I4R AI Audit Protocol.
Details in this 🧵.
Join us!
Matt Webb retweeted
Being a professor is an unusual form of celebrity. You're famous among people who are professionally obligated to criticize you.
Thanks Peter, this Hull-set of results will be quite helpful for a few projects I'm working on.
Please enjoy!
Writeup: dropbox.com/scl/fi/ago5vsi89…
Repo: github.com/peterdhull/twfe-r…
Planning to expand to other journals soon
Matt Webb retweeted
Please enjoy!
Writeup: dropbox.com/scl/fi/ago5vsi89…
Repo: github.com/peterdhull/twfe-r…
Planning to expand to other journals soon
Very proud of my wife, Jennifer Brigden , who is doing her first theatre show this Thursday night as part of the Just For Laughs Comedy Club at Casino du Lac-Leamy. Free admission, Doors open at 7 p.m., and seating is first come, first served.
casinos.lotoquebec.com/en/la…
🤖 Made with AI
Matt Webb retweeted
I recently asked ChatGPT to improve the efficiency of some simulations from a few years ago. To my amazement, its code ran in a few seconds instead of 3 hours.
It turns out it went into the GitHub repository, downloaded my final results, and used it as a starting guess in the optimization.
I then tried running it on a new set of parameters. It’s been 15 hours.
Matt Webb retweeted
P-values are often misunderstood, so should we use confidence intervals instead? Probably not.
Matt Webb retweeted
This might be a first in economics: Authors officially request to be removed from a paper published in REStat. The paper itself has not been retracted - and the other authors remain on the paper. direct.mit.edu/rest/article/…
Matt Webb retweeted
Replying to @NorthwesternU
@NorthwesternU Economics department dropping some hard truths about predocs (good!)
Matt Webb retweeted
Here is a memo to all Canadians:
There are no protectionist paths to free trade.
Its that simple.
Matt Webb retweeted
I tested GPT-Astra on mathematics. It's a quantum leap. You can talk with the model and prove the statements live in Lean. The feeling is absolutely stunning. You can verify your ideas, compile truth. For a mathematician it feels like finally we arrived in the era where we can focus entirely on the ideation and exploration. Each lemma flows once the logic is set. Before the verification was lagging behind but Astra is very fast and for many tasks the formalization happens as you write your argument in Codex.
If you tell the model to use literate programming + LaTeX you end up with your proof combined with the Lean code, everything explained as you wrote, mixed with small chunks of Lean which are digestible.
I don't want to go back to the era where the only confirmation of the proof was "aha". Now the "aha" is followed by a green tick that indeed tells you that you have captured the essence. Imagine how cool it will be to have all the lemmas of the world combined in one giant database, pointing to people and models who found them. You compose and mix your ideas and build on the shoulders of the giants. But you see much further now and build much faster. And we are just at the beginning of those changes. So much work to do, so much fun!
GPT-6 Astra is here! This is a big moment for our research team - years of work on pretraining, reinforcement learning, and post-training have come together in our most capable and aligned model yet. It can build and test software, work across apps on your computer, and even help you take a crack at open scientific problems!
Capabilities that felt like grand challenges a few years ago have become tools people can actually use. One example is Computer Use - if you’ve tried this before and felt like it was too slow or not good enough, I encourage you to give it another shot. We’ve come a long way since Operator, and it “just works” now.
We’re also asking these systems to act on your behalf for more consequential work. Agents needs to stay aligned with your goals and values, think transparently, and respond to oversight even when tasks become difficult. We’ve made substantial progress on these behaviors in Astra, alongside stronger monitoring that can stop potentially unauthorized actions. That work is part of what makes this release possible.
I think alignment is one of the most important research frontiers in AI, and it remains far from solved. Our ability to understand and align models has to keep pace with model capabilities. We want to give people more room to think, build, and discover with increasingly powerful tools that remain *under their control*.
Huge thanks to the researchers and teams who got us here. There’s a lot more work ahead, and I’m incredibly excited about what we can make possible in the near future!
Shuffling some books into the appropriate collection in light of recent discoveries.
datacolada.org/139
Matt Webb retweeted
There was a lot of commentary about how academic frauds get differential treatment. You know what? Let's show that this isn't the case by shaming Duke into firing Ariely.
An influential paper by Ariely & Wertenbroch (2002) reported two studies with tampered data.
Post 1 of 2.
datacolada.org/138
Matt Webb retweeted
This may be the single most egregious structural failure I’ve seen in academia, out of a lot of egregious failures.
A system that suspends the person reporting academic fraud while conspiring to circle the wagons around the fraud itself is fundamentally, irreparably rotten.
Matt Webb retweeted
I strongly believe that knowledge is fundamental to running a society. We need economists who understand monetary policy in central banks. We need researchers who have studied history, strategy, and military affairs to help steer national defense. We need doctors and public health experts to design effective responses to epidemics.
But the public needs sound reasons to let these experts shape their future. Once that trust evaporates, we are all worse off. Unfortunately, the Jason Arday scandal is the perfect example of how these experts, and the institutions they run, are destroying that trust.
I do not find Arday’s tales extraordinary. Throughout history, there are thousands of examples of people spinning fantastic tales about their lives. He is just one more name on a long list of confidence men.
What is extraordinary is the massive institutional failure at every level. Liverpool John Moores University granting a degree for a Ph.D. dissertation that did not survive even basic scrutiny (including a grammatical mistake in its title). Cambridge University appointing him (full) professor at an unusually early academic age (eight years after his dissertation; in academia, this is what matters, not your biological age) and with a scholarly record that was rather thin. Hilary Cremin (@hilarycremin), head of the Faculty of Education, telling the world on television that Arday was “the best in the world in terms of the research that you do.” The BBC broadcasting segments about Arday, a most unusual decision for an academic appointment. Simon & Schuster paying him a large advance for his memoirs on the basis of a book proposal that defied common sense. And Amazon promoting his book, still this very morning, as an editor’s pick.
If I were a regular person, say someone driving a truck in the Midlands or working at a small shop in Sussex, how could I trust the universities any longer? How could I trust the BBC and the media? How could I trust the publishing industry? How could I trust Amazon’s editor’s picks?
Then, should we be surprised if the farmer or the shop owner votes for someone selling snake oil? Or if they refuse to listen to the case for policies that cost them in the short run but benefit them in the long run, such as the transition to net zero emissions? Or if they read books written by charlatans? Or if they deny basic scientific findings?
The universities, the media, the publishing industry, Amazon, and many of our other core knowledge institutions have a serious problem. Their response is to close ranks and avoid a serious discussion. No, sorry, this will not do.
The Jason Arday scandal should be a moment of reckoning. Unfortunately, I am afraid it will not be.
Matt Webb retweeted
It is absolutely bananas how much grade inflation has occurred in Ontario high schools.
Just look at how much came after Ford was elected and after the pandemic.
We can talk about underfunding until we are blue in the face, but my biggest concern is falling standards.
I’ve heard horror stories from teachers and professors alike about how unprepared students are still being passed upward to avoid conflicts with parents and administrators.
I am committed to increasing education funding and resources for our classrooms, but there is no doubt that curriculum changes and a reintroduced role for standardized testing are needed. It genuinely seems we’ve lost a major goal of education: to create resilient citizens capable of dealing with challenges and setbacks.
By letting standards fall, we have also made it harder for high-potential students to really distinguish themselves. It doesn’t surprise me that more parents are looking to tutoring and private education under these circumstances.








































