Proud to share TranscriptFormer - a generative model for single-cell transcriptomics, trained on 112 million cells across 12 species spanning 1.5 billion years of evolution. TranscriptFormer is distinct in being both generative and multi-species. Built entirely on raw gene expression data, the model learns to organize biological structure in an unsupervised manner and is trained without labels. It exhibits out-of-distribution generalization properties on common tasks like cell type prediction across vast evolutionary distances, and also facilitates exploration of cell atlas knowledge via prompting as a unified knowledge resource. It marks the transition from collecting datasets, for instance cell atlases via CELLxGENE, to interactive AI models of these datasets. Using that capability, we show how to use it as a virtual instrument for predicting transcription factor-cell type relationships, and we demonstrate its competitive performance against existing models on multiple benchmarks, including cell type annotation, even in zero-shot settings. This represents an early but important step toward building general-purpose generative foundation models for CZI’s virtual cell scientific grand challenge. Congratulations to CZI for our first in-house foundation model release. 📄 Read more:
Pre-print - bit.ly/3YnxzjB

Join us tonight from 7:30-9:30 PM for a CZI Happy Hour at #NeurIPS2024 in Vancouver! Register here: forms.gle/CWJFNWu9PYmpGRq17