Ai2 releases Olmo-core 3 framework for faster LLM training
The Allen Institute for AI's open-source Olmo-core 3 boosts per-GPU throughput roughly 2.7x on a 47-billion-parameter MoE model, according to aitimes.com.
The Allen Institute for AI (Ai2) has released the open-source framework Olmo-core 3 for training large language models. According to aitimes.com, the toolkit raises per-GPU throughput by about 2.7x on a 47-billion-parameter mixture-of-experts (MoE) model compared with the previous FSDP-based system.
In a test using eight Nvidia B300 GPUs, the 47-billion-parameter MoE model processed 52,000 tokens per second per GPU.
Ai2 switched from Fully Sharded Data Parallelism (FSDP) to a Distributed Data Parallelism (DDP) based system for MoE training. Olmo-core 3 relies on row-wise expert parallelism, GPU-resident routing and grouped GEMM to cut data movement costs.
The institute also applied the MXFP8 low-precision format, which increased training throughput by roughly 21 percent.
Ai2 published the code and a technical report on GitHub.
Quick answers
What is Olmo-core 3?
It is an open-source framework from the Allen Institute for AI for training large language models, with optimizations targeting mixture-of-experts (MoE) training.
How much faster is Olmo-core 3?
Ai2 reports about 2.7x higher per-GPU throughput on a 47-billion-parameter MoE model versus the previous FSDP-based system; the MXFP8 low-precision format added about a 21 percent gain.
Where can I find the Olmo-core 3 code?
The code and a technical report were published on GitHub.