Port SBU dataset

[lezwon]

fixes #5349

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit 91bd94f (more details on the Dr. CI page):

---

• 1/1 failures introduced in this PR

---

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

binary_linux_wheel_py3.7_rocm4.3.1 (1/1)

Step: "packaging/build_wheel.sh" (full log | diagnosis details | 🔁 rerun)

https://repo.ius.io/7/x86_64/repodata/repomd.xm...- "Peer reports it experienced an internal error."

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=ius.skip_if_unavailable=true

failure: repodata/repomd.xml from ius: [Errno 256] No more mirrors to try.
https://repo.ius.io/7/x86_64/repodata/repomd.xml: [Errno 14] curl#35 - "Peer reports it experienced an internal error."


Exited with code exit status 1

---

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[lezwon]

@pmeier mind checking this PR and letting me know if I'm headed in the right direction?

[pmeier]

Argh, I knew there was a reason I marked SBU in the tracker issue. When I gave you the go I just saw that there is something to download and assumed I was mistaken before. That is on me, my bad.

In general your solution looks really good and works, but has one major downside: We need to re-download every image for every iteration. While we plan to support streaming datasets from the internet, we are not there yet. Thus, we need to download everything.

That could be achieved with a OnDiskCacheHolder, but that would mean we would only download at runtime. All current datasets download everything upfront and I would keep it that way for now.

My solution is to put a custom preprocess method onto the resource and download everything there.

[pmeier]

Looking good, thanks @lezwon! I have one simplification comment and one larger change for the mock data generation.

[pmeier]

This is a really cool idea and I'm definitely going to use this webiste for other things in the future 🚀 Unfortunately, we cannot have an actual download during mock data generation for two reasons:

1. Downloading these images takes quite some time and we want the tests to be fast.
2. Meta internal test system do not have access to the internet and thus would fail here.

I propose I send a patch for the test suite that allows us to also only generate the already preprocessed files. Thus, we only add a SBUCaptionedPhotoDataset that already includes test images. I'll ping you on the PR.

[pmeier]

See #5706.

[lezwon]

@pmeier I'll wait for the PR to get merged right? I can make the necessary changes after it.

[pmeier]

Yes, sorry for the delay. I'll try to get it merged soon.