Role-Decoupled Attention Residuals: Separating Matching and Content Retrieval Across Depth

Summary

Depth-routing residual architectures allow Transformer layers to retrieve earlier representations instead of inheriting only the immediately preceding state.