Introduce --test-plan for more complex test plans

[jeremyandrews]

• introduce --test-plan and GooseDefault::TestPlan for configuring test plans
• closes Specify the traffic profile/shape? #365
• use [FromStr] to auto convert --test-plan "{users},{timespan};{users},{timespan}", where {users} must be an integer, ie "100", and {timespan} can be integer seconds or "30s", "20m", "3h", "1h30m", etc, to internal Vec<(usize, usize)> representation
• don't allow --test-plan together with --users, --startup-time, --hatch-rate or --run-time
• add page to https://book.goose.rs/ explaining how to configure test plans

From the new book page:

The following command tells Goose to start 10 users over 60 seconds and then to run for 5 minutes before shutting down:

$ cargo run --release -- -H http://local.dev/ --startup-time 1m --users 10 --run-time 5m

The exact same behavior can be defined with the following test plan:

$ cargo run --release -- -H http://local.dev/ --test-plan "10,1m;10,5m;0,0"

Another possibility when specifying a test plan is to add load spikes into otherwise steady load. For example, in the following example Goose starts 500 users over 5 minutes and lets it run with a couple of traffic spikes to 2,500 users:

$ cargo run --release -- -H http://local.dev/ --test-plan "500,5m;500,5m;2500,45;500,45;500,5m;2500,45;500,45;500,5m;0,0"

In the current implementation, a test plan can only Increase and Maintain, the load test will shut down any time it defines a Decrease. This will be resolved in a followup.

An example run:

% cargo run --release --example drupal_memcache -- --host http://apache.fosciana/ --test-plan "10,5;20,5;30,5;30,5;60,10;60,2;0,0"

Includes the follow overview in the metrics:

 === OVERVIEW ===
 ------------------------------------------------------------------------------
 Action       Started             Stopped           Elapsed    Users
 ------------------------------------------------------------------------------
 Increasing:  22-04-18 07:07:06 - 22-04-18 07:07:11 (00:00:05, 0 -> 10)
 Increasing:  22-04-18 07:07:11 - 22-04-18 07:07:17 (00:00:06, 10 -> 20)
 Increasing:  22-04-18 07:07:17 - 22-04-18 07:07:22 (00:00:05, 20 -> 30)
 Maintaining: 22-04-18 07:07:22 - 22-04-18 07:07:27 (00:00:05, 30)
 Increasing:  22-04-18 07:07:27 - 22-04-18 07:07:40 (00:00:13, 30 -> 60)
 Maintaining: 22-04-18 07:07:40 - 22-04-18 07:07:42 (00:00:02, 60)
 Decreasing:  22-04-18 07:07:42 - 22-04-18 07:07:42 (00:00:00, 0 <- 60)

 Target host: http://apache.fosciana/
 goose v0.16.0-dev
 ------------------------------------------------------------------------------

[slashrsm]

I tested and I can confirm that it works as expected (with the current limitations). Great job! This is not a small task!

It would be awesome to have some more automated tests for this as we're dealing with quite significant change.

I would also propose to create a list of follow-ups that are required and mark those that are release blocker. This way we'll avoid forgetting to fix something crucial.

[slashrsm]

PR # missing

[jeremyandrews]

Thanks, fixed!

[slashrsm]

I am wondering if this output really belongs in this file. Are these metrics or something else?

metrics.rs is already a huge file and we might want to consider splitting it at some point? Maybe this is a good time to start this discussion (actually doing it is out of scope for this PR of course).

[jeremyandrews]

This is the ASCII display of the metrics. I suppose it could be moved into report.rs if that's a better fit?

[slashrsm]

I would put test plan and its logic in a separate file. I would also include TestPlanStepAction and TestPlanHistory that are currently in metrics.rs.

[slashrsm]

Besides that I'd also add some test coverage.

[jeremyandrews]

There's very little logic here, but I suppose it's likely going to grow -- splitting it out works for me. (Especially as when I was writing this I could never remember which file it lived in).

And yes, it DEFINITELY needs tests! :)

[jeremyandrews]

I've split it into its own file now -- definitely the right call, thanks.

[slashrsm]

Awesome work! 🎉

[LionsAd]

RTBM - retro — great work 👍