Provide a shortcut to run a test with original UA if bot protection is used

[tkadlec]

Feature Request Summary

In situations where some sort of bot mitigation is impacting the test results, provide a helpful shortcut to resubmit the test preserving the original User Agent String.

What's the motivation or use case for the change/feature?

Sometimes, bot mitigation will unfortunately flag WebPageTest agents as a bot and start a mitigation approach that negatively impacts the accuracy of our test results.

Here's an example of a test where we're flagged as a bot, and as a result, the server response is dramatically slowed down: https://www.webpagetest.org/result/210916_BiDcZY_7c9ec5eae9faaed8fde4791c60e64ea6/

Preserving the original user agent string fixes the issue: https://www.webpagetest.org/result/210916_BiDcPR_4e04cc325455440afa2b3093a69943c9/

Instead of requiring folks to know that's a possible issue, and to avoid having them dig into a problem that doesn't actually exist, providing a notification and button to quickly try a test with the original UA preserved would be a really helpful touch.

[scottjehl]

So, design TBD but one step here could be to provide the checkbox with a note about this occasional situation.

Some thoughts:

• Would it make sense to only show this when certain metrics, like TTFB are slower than a certain threshold? Say, a few seconds?
• From a technical standpoint, can we simply add a checkbox with a name of keepua and expect that to impact the new run's settings?
• It's a bit wordy. Maybe we could tuck it into an info details/summary to save space...

[scottjehl]

Preserving example markup here in case it's useful later.

<p><strong>Do your results appear unexpectedly slow?</strong> Some networks intentionally slow performance for bots. This may help:
<label><input type="checkbox" name="keepua"> Preserve original UA string for re-run
</label>
</p>

[tkadlec]

I like the checkbox approach, yeah.

Would it make sense to only show this when certain metrics, like TTFB are slower than a certain threshold? Say, a few seconds?

Yeah, definitely don't want this on every test, I don't think. I'd say if we can see that TTFB for each run of the tests is above ~3 seconds (something in that range).

From a technical standpoint, can we simply add a checkbox with a name of keepua and expect that to impact the new run's settings?

Yup, that should get the job done. Re-run already grabs the settings from the last run, so this just adds another parameter and away we go.

It's a bit wordy. Maybe we could tuck it into an info details/summary to save space...

+1

[rik]

Nice!

Nitpick: Please avoid using acronyms without a definition. "Preserve original User Agent string for re-run" should fit well here.

[scottjehl]

Thanks for the feedback. Sounds good all around!

[scottjehl]

Design TBD but, here's the markup for likely messaging for this spin-down: