option
Home
Flash News
Content
JohnYoung
JohnYoung
July 17, 2026

The Xiaohongshu team with PKU and SJTU developed HYPIC, the first system providing position-agnostic caching for hybrid attention large models. It caches transition operators for linear layers, uses seam windows for full attention, and enables segment-level parallelism. Testing showed 3.25x lower first token delay and 1.66x higher sustainable QPS versus prefix cache, with minimal quality loss.

The Xiaohongshu team with PKU and SJTU developed HYPIC, the first system providing position-agnostic caching for hybrid attention large models. It caches transition operators for linear layers, uses seam windows for full attention, and enables segment-level parallelism. Testing showed 3.25x lower first token delay and 1.66x higher sustainable QPS versus prefix cache, with minimal quality loss. The Xiaohongshu team with PKU and SJTU developed HYPIC, the first system providing position-agnostic caching for hybrid attention large models. It caches transition operators for linear layers, uses seam windows for full attention, and enables segment-level parallelism. Testing showed 3.25x lower first token delay and 1.66x higher sustainable QPS versus prefix cache, with minimal quality loss.
Comments (0)
0/300
OR