random-features

RACE

RACE Attention: A Strictly Linear-Time Attention Layer for Training on Outrageously Large Contexts

RACE 用可微软哈希桶聚合质量与值，在固定桶预算下实现线性注意力；任务效果与单层极长序列扩展分别验证，归一化证明存在偏差漏项。