[Feature Request] Self-Transcendence (HUGE improvement over the idea of LayerSync)

[MeiYi-dev]

https://arxiv.org/abs/2601.07773

https://github.com/csslc/Self-Transcendence

Looks to be a much better alternative(?) to LayerSync.

[kabachuha]

Does it require two copies of the model (well, one of the copies is partial, but it's the same order of memory) to reside in VRAM at the same time? Also, it looks like is not only the trainable latent passed, but also both auxiliary conditional and unconditional latents, which then compute a sample. Looks disastrously memory-wise for big models. Maybe only helpful for models much smaller than the VAE (including its forward pass) and DINO.

[kabachuha]

A giant red flag is that in Table 1. they compare it to the other models after the model underwent additional VAE-based alignment, which increases the resources even more. This actually makes their method look much worse.

[MeiYi-dev]

A giant red flag is that in Table 1. they compare it to the other models after the model underwent additional VAE-based alignment, which increases the resources even more. This actually makes their method look much worse.

We already have some VAEs from REPA-E project https://huggingface.co/REPA-E

For large training runs, it will be immensely useful.