Trillium Labs Launches to Study AI Self-Improvement in the Open

Nathan Lambert and Tom Zick have launched Trillium Labs, a nonprofit that will conduct AI research in a transparent way, publishing details of experiments for outside scientists to study and replicate. The organization launched today.

Trillium Labs will initially focus on post-training, recursive self-improvement (RSI), and reinforcement learning. It will also study how reinforcement learning shapes the character and behavior of AI models.

The launch comes amid ongoing debate about RSI. Earlier this month, an Anthropic researcher left the company and warned that RSI could pose an existential threat to humankind.

Lambert previously worked at Ai2, which has taken an unusually open approach to AI, and at Hugging Face, where he founded the American Truly Open Models initiative. Zick worked at Harvard University and helped Charles Schwab devise policies around responsible AI.

The two met over Zoom during the COVID-19 pandemic when both were graduate students at UC Berkeley. Their new venture joins a small but growing set of open AI efforts. Recently, Xiaomi published live details of a major training run involving one of its models, and researchers at Stanford are pretraining the AI model Marin in the open.