HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing

来源摘要

This work introduces Hybrid Sparse Attention (HySparse), a new architecture that interleaves each full attention layer with several sparse attention layers. While conceptually simple, HySparse strategically derives each sparse layer’s token selection and KV caches directly from t…

摘要由机器生成（来自来源站点），可能存在偏差； 本站不转载全文，请以原文为准。