Together AI Open-Sources 'OSCAR' 2-Bit KV Cache Quantization System for LLMs

Together AI has open-sourced 'OSCAR,' a 2-bit KV cache quantization system for LLMs. It reduces memory usage by 8x while maintaining near-BF16 accuracy.

Mon May 25 2026

Together AI Open-Sources OSCAR System

Together AI has open-sourced OSCAR, a system for quantizing the KV (Key-Value) cache, a crucial component of Large Language Models (LLMs). OSCAR quantizes the KV cache to 2-bit precision, enabling an 8x reduction in memory usage. This technology is expected to significantly improve efficiency, especially in LLM serving environments that handle long contexts.

Enhanced Memory Efficiency and Accuracy

OSCAR dramatically reduces memory consumption while maintaining high accuracy, closely approaching that of BF16 (Brain Floating Point 16) precision. This advancement can help lower the operational costs of LLMs without compromising performance, making LLM services more accessible to a wider range of users. The open-source release is considered a significant step in increasing the accessibility of LLM technology and fostering related research and development.

*Source: MarkTechPost (2026-05-25)*

Tags: #AI #LLM #TogetherAI #open-source #tech