Microsoft Explores Composable Memory for AI Data Centers
Microsoft principal architect Samir Lazardnya says AI inference faces power and memory limits as the company studies pooled, on-demand memory for Azure.
Microsoft is exploring composable memory technology as a way to address memory shortages in AI data centers, principal architect Samir Lazardnya said at an AI Data Pipeline Forum. According to SiliconANGLE, Lazardnya introduced the company's interest in composable memory during the event.
Lazardnya said his technology exploration team is looking for technology that pools memory across servers for Azure-wide use and allocates it on demand.
He said AI inference faces two limits at once: power and memory. Memory is currently scarce, he added, and tying memory to unused tasks wastes a precious resource.
Lazardnya also said memory was once an appendage of CPUs or GPUs but is now an important resource in itself. Optical interconnect technology, he said, will help enable composable memory.
Quick answers
What is Microsoft exploring for AI data centers?
Microsoft is exploring composable memory technology to address memory shortages in AI data centers, according to principal architect Samir Lazardnya.
What limits did Lazardnya say AI inference faces?
He said AI inference faces two limits at once: power and memory.
What technology did Lazardnya say will help enable composable memory?
Lazardnya said optical interconnect technology will help enable composable memory.