Chicago HAI

AI safety is important and challenging. It is thus important to have reliable evidence. @cohere recently reported that a Qwen model tried to upload data to an Alibaba Cloud bucket during one of their evaluations, so we rebuilt their setup in a sandbox and reran it on Qwen3.8-Flash-Next, Kimi K3.1 Pro and GPT-6-Sol. We also reran a related Qwen report from @wtogami (github.com/wtogami/qwen-bloc…). Across 130 runs of Cohere's task, none of the models uploaded anything. Kimi sometimes went online on its own to read public reference material, but it never sent any data out, and GPT-6-Sol never left the sandbox. @wtogami 's finding did reproduce. When asked to fetch Python documentation pages, Qwen3.8-Flash-Next and Qwen3.8-27B made up fake Alibaba file-proxy URLs and tried to download them in 8 of 40 runs, while Kimi and GPT-6-Sol never did this in 40 runs. The claim from Cohere rests on a single event, the "1 in 100,000" comes without the number of runs behind it. What we could reproduce is Qwen making up URLs while fetching, and those were downloads rather than uploads. Echoing many similar accounts, although @cohere 's report gives much more context, the title and framing is irresponsible. Thanks a lot to @wtogami for publishing his reproducer, which made this second check possible. All of our traces are public in the same format as Cohere's, with every network request the sandbox caught. Full blog and data link 🧵

This week we extended The Allocation of Talent and U.S. Economic Growth by Chang-Tai Hsieh, Erik Hurst, @ChadJonesEcon, and @PeteKlenow. In 1960, 94 percent of American doctors and lawyers were white men. Many talented women and Black Americans were kept out of jobs that suited them. This paper asked how much the U.S. economy grew as those obstacles came down and found about 40 percent of the growth in income per person from 1960 to 2010. We got the same result from their data and code, then added Census data up to 2024. The obstacles are still coming down, but the big gains are over. Since 2010, there was about as much growth each year as in the 2000s. But that is now only about 7 percent of all growth. In the 1970s it was more than half. For women, progress is slowing down. On the current trend, the gap with white men shrinks but does not close.

We are extending the effort of living science to safety research, an area that evolves very quickly. We also need get updated understanding and evidence. We are starting with five papers in this thread. 🧵

I saw a lot of discussions on the China syndrome yesterday here. Coincidently, we chose the same paper for this week's living science update. Our extension focused on manufacturing jobs per 100 working-age adults. We find that the damage kept growing long after the shock ended. Manufacturing losses in exposed places deepened for another decade, reaching the size of the paper's headline estimate around 2018. Most of that gap was still there in 2023. Average local wages have recovered, but the manufacturing jobs did not come back. Learn more in the link below.

Excited to announce Living Science! Papers have been the de facto unit of knowledge in science. But science is not static and a paper is not meant to be the final destination, especially in empirical research. Empowered by AI agents, we're revisiting influential research as new data, models, and methods arrive, starting with economics. 🧵

Excited that our work on replicating ICML papers @sailabshq is featured on @ScienceMagazine. The actual goal of replication is always building towards better science. science.org/content/article/…

Verification is becoming a central bottleneck in AI. But a deeper problem arises (c.f. our work on replicating ICML papers). How do we evaluate the verifiers? Today, we're launching SAI Arena at @sailabshq, a platform for evaluating AI verification systems (broadly AI scientists) with feedback from the people who use them. sai.science/arena