Vessel AI to Power Global Inference for Upstage's Solar Pro 4
Vessel AI said it will provide global inference infrastructure for Upstage's Solar Pro 4, with 2x to 3x faster processing and up to 2x cost savings per token.
Vessel AI announced it will provide global inference infrastructure for Upstage's large language model Solar Pro 4, according to digitaltoday.co.kr.
The company said it achieved an average processing speed increase of 2x to 3x compared with before the endpoint was adopted.
Vessel AI also reported up to 2x cost savings per token by improving cache hit rates.
To reach these results, the company applied internal cache hit rate improvements, model quantization and inference speed enhancement technologies.
Quick answers
What did Vessel AI announce?
Vessel AI announced it will provide global inference infrastructure for Upstage's large language model Solar Pro 4.
What performance gains did Vessel AI report?
Vessel AI reported an average processing speed increase of 2x to 3x and up to 2x cost savings per token through improved cache hit rates.