Support devices and delegates parameters in android-perf workflow

[guangy10]

Summary:

The workflow can support parameters models, devices, delegates.
The workflow is able to translate devices to device-pool-arn and pass to the mobile_job for scheduling
The model is packed and uploaded to artifacts/{model}_{backend}/model.zip

Note: We should only expect the scheduling is working correctly by passing models, devices and delegates. The actual job may not finish successfully because the underlying model/delegate/device are not supported yet.

Test:

The workflow scheduling is working as expected: https://github.com/pytorch/executorch/actions/runs/10187260307

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/4484

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 73144c5 with merge base 4483bb6 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[facebook-github-bot]

@guangy10 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@guangy10 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[huydhn]

LGTM! Just a note that this setup means that when there are multiple models, all of them need to be exported successfully before the benchmark job could start. It is simpler but people might want more flexibility to stop models that fail to export while continue with models that are exported successfully. We can consider that feature later if there is an ask for it.

[guangy10]

LGTM! Just a note that this setup means that when there are multiple models, all of them need to be exported successfully before the benchmark job could start. It is simpler but people might want more flexibility to stop models that fail to export while continue with models that are exported successfully. We can consider that feature later if there is an ask for it.

Yes, that's a good point. We can address it later when we can support multiple models. It's easy to do but the tricky part is that we don't want it being a surprise if they scheduled to run 10 models but only 1 gets run at the end. We will need to surface the model export failures to users in a more explicit way other than log the failure in the CI, e.g. making it part of the final report or dumped file.

[facebook-github-bot]

@guangy10 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@guangy10 merged this pull request in 0caaf3f.