Test Method Contains Randomness

[Adlai-Holler]

The method testReloadData on ASTableView contains a number of random variables. What's worse, it runs a loop of 100 iterations on the same table view, each with a new set of random data.

I get the appeal, but this is awful!! If this method fails on CI we have no idea why it failed and no way to reproduce the failure.

Our thrash testers save and print each configuration to avoid this issue.

What's worse, the failures are all unexpected (i.e. unhanded exceptions). So our options are:

• Easy: Ditch the method and remember never to add random elements willy-nilly in the future.
• Hard: Keep track of all random variables in e.g. an array of dictionaries. Add a mechanism to read this data structure from a JSON file to "replay" the test that failed. Add @try/@catch around the test body. If an exception is caught, print the JSON representation of the test config and then rethrow it.

[Adlai-Holler]

If ^ is fixed, we can go the easy route on this. That's what we should do.

[Adlai-Holler]

Our Pinterest CI is running our unit tests like 20 times in a row now, and so the failure conditions in here are popping up reliably now, and causing all the tests to fail.

If we go the easy route here, we should have someone enable the exception breakpoint, log all the random variables, and just run the test locally like 20 times and see if we hit the failure condition.

[hannahmbanana]

I'll disable this test until we can get it fixed.

[garrettmoon]

@Adlai-Holler since we haven't gotten a volunteer on this, can you take this on?

[hannahmbanana]

Consolidating this issue to #2436.