tests/models/sam_hq/test_modeling_sam_hq.py::SamHQModelIntegrationTest may fail since a lot of cases are lack of set_seed()

[sywangyi]

System Info

transformers built from latest main.

Who can help?

@ydshieh
similar issue may occur in tests/models/sam/test_modeling_sam.py

see https://github.com/sywangyi/transformers/blob/main/src/transformers/models/sam_hq/modeling_sam_hq.py#L1077
randn() is used in positional_embedding, so you need to set_seed() before modeling loading to guarantee the stable output. or else accuracy check will fail.

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

pytest tests/models/sam_hq/test_modeling_sam_hq.py::SamHQModelIntegrationTest

Expected behavior

all pass in cuda.
actual: 11 failed, 3 passed,

[Rocketknight1]

I think this is a general issue - any test that doesn't load a pretrained model will randomly initialize weights, and so there will be test variability. This is the root of a lot of flaky tests for us.

However, I can confirm the SamHQ slow tests are failing for me too - if you can fix it by setting a seed, we'd welcome a PR!