Theorem 5: Pure attention has a strict rank barrier that depth cannot overcome; only FFN width increases representational rank. Verified via SVD.
deep-learning attention-mechanism machine-learning-theory transformer-architecture mathematical-proof rank-barrier depth-width
-
Updated
Jun 20, 2026 - Python