LG Uplus shows AI token-efficiency results across models and chips

LG Uplus presented its research on AI token efficiency and outlined commercialization plans at a press study session, according to digitaltoday.co.kr. A paper tied to the work was accepted at a NeurIPS workshop.

The company said model routing kept answer quality at about 95% while cutting token costs by roughly 25%. In an internal coding agent, the same approach produced cost savings of about 30%.

Using LG Uplus's own AI model for coding work cut operating costs by about 90% compared with a commonly used model, the company said. Reducing a 35-billion-parameter model to 10 billion parameters held quality at 91% and raised token generation speed by 40%.

On the device side, an on-device AI model was slimmed from 2.4-3GB to about 800MB while reaching more than 95% of the performance of a cloud model used for comparison. In an EXAONE-based experiment, speculative decoding doubled generation from 80 to 160 tokens per second.

Chips and harness engineering

LG Uplus described a Chip-Mix technology that allocates GPUs from Nvidia and AMD, plus NPUs from FuriosaAI, to the prefill and decoding stages of inference. Its Harness Engineering effort reduced token usage by 38% for general tasks and 84% for coding tasks.

The company has been optimizing quality and cost in its ixi-O AI call agent using its own models, LG AI Research's EXAONE and Google Gemini. It has worked with FuriosaAI for more than two years to validate NPU use, and at the end of this year plans to commercialize "Modu's AI" in cooperation with Kakao. It said it intends to expand token optimization into the B2B market starting next year.