Sparse Attention

Fibottention: Inceptive Visual Representation Learning with Diverse Attention Across Heads
Visual perception tasks are predominantly solved by Vision Transformer (ViT) architectures, which, despite their effectiveness, …
Cite Code arXiv