Effective rank, RankMe, E1, CKA and anisotropy on transformer hidden states are determined by one direction. The exact identity, and the attention sink behind it.
Visualizing Modern LLM Mechanics, Loss Landscapes & HPC Topologies
Code for 'Exploring the Impact of a Transformer's Latent Space Geometry on Downstream Task Performance' (arXiv:2406.12159)
Com la tokenització fractura la morfologia catalana i si una segmentació conscient dels morfemes recupera la geometria. Provat en 3 llengües indoeuropees (català, castellà, anglès): el català es fragmenta ~1,7× més que l'anglès; forçar el tall morfèmic recupera la composicionalitat (robust a portadora i replicat en castellà).
Code for 'Reliable Measures of Spread in High Dimensional Latent Spaces' (ICML 2023)
An empirical and geometric analysis of Neural Collapse under different optimizers on CIFAR-10
Add a description, image, and links to the representation-geometry topic page so that developers can more easily learn about it.
To associate your repository with the representation-geometry topic, visit your repo's landing page and select "manage topics."