follow-up from https://github.com/huggingface/transformers/pull/43794 we should add in our fixture a `set_seed` so we *always* set the same seed in all of the model tests to improve determinism. cc @Rocketknight1