BREAKING: Google unveiled TurboQuant, a technique that aims to compress the working memory of language models without loss.

If it reaches production as the research suggests, it could alleviate one of the major bottlenecks in current AI: the cost and scarcity of memory needed to handle long contexts, agents, and massive inference workloads.
TurboQuant aims to reduce KV cache usage by up to six times and accelerate on chip processing by up to eight times without data loss.