Stephan ₿entyn 🇸🇻

btw anthropic's internal document on this literally said "we don't want it to be known that we are working on this.” it was called project panama. here's exactly what happened: 1: anthropic concluded that books were the cheapest way to build a world-class model because they gave claude curated facts, structured arguments, compelling stories, and writing “an editor would approve of.” 2: once anthropic decided it needed books at enormous scale, its first solution was piracy. it downloaded 7m+ books from online libraries including libgen. the judge later wrote that although anthropic had legal ways to buy them, it chose piracy to avoid what dario amodei called the “legal/practice/business slog.” 3: that piracy created a massive legal risk. so in february 2024, anthropic hired tom turvey, the former head of partnerships for google books, to find a legally safer way of obtaining “all the books in the world.” 4: turvey first contacted major publishers about licensing their catalogs. those attempts didn’t produce agreements, so anthropic chose a route that required no publisher permission: buying millions of physical books through distributors and used-book retailers. 5: within about a year, anthropic spent tens of millions acquiring and scanning millions of books, including many rare and 1/1 titles. one vendor proposal targeted 500,000 to 2 million books in six months. 6: to scan that many books within months, the vendors physically dismantled them. a hydraulic cutter removed each spine. the pages were trimmed to size, fed as loose sheets through high-speed industrial scanners, and converted into searchable PDFs. the paper remains were then sent for recycling. 7: these PDFs were fed into claude as training data. the complete collection became a private, searchable anthropic library that the company planned to “store forever.” the scans aren’t available to the public and were never open-sourced.