VAE embeddings as input for controlnet

[tinder-trideeprath]

Why does controlnet embedding use a small CNN model for embedding rather than using VAE embedding directly into the controlnet.

I believe for any condition the Condition embedding model is learning from scratch to basically map the image to the VAE space, if we just use VAE encoder we wouldn't need to do this additional model training.