Add additional graphs to the HTML report #394

slashrsm · 2021-11-24T16:07:49Z

Closes #246.

Graphs

Errors per second
Average response time per second
Active users per second
Active tasks per second

Screenshots

Errors

Response time

Active users

jeremyandrews

This is wonderful to see! I'm looking forward to getting this merged once my comments are addressed. There also seems to be a bug in the generation of the graph during startup, when startup is >=20 seconds (more or less):

I also would prefer to see the graphs mixed in with the relevant data, and graph first data next. So, show the requests per second graph, then show the requests break down, etc.

src/metrics.rs

jeremyandrews · 2021-11-30T13:44:08Z

src/metrics.rs

+        for set in self.task_sets.iter() {
+            tasks += set.tasks.len();
+        }
+        self.metrics.record_users_tasks_per_second(tasks as u32);


Why do we calculate this before merging in the received statistics? Shouldn't we merge statistics and then count?

Also, why does this happen here, instead of in the same place as where we record the other metrics for the graphs?

Why do we calculate this before merging in the received statistics? Shouldn't we merge statistics and then count?

I don't think this would change anything. This is simply counting tasks and users at this point and I don't think other stats would influence that.

Also, why does this happen here, instead of in the same place as where we record the other metrics for the graphs?

Because number of tasks do not need to be recorded for every request that we are collecting stats for; we need to do it just once.

This still feels like an illogical place to perform this function, especially if we don't actually care whether or not we receive the metrics (the purpose of this function).

Agreed. Moved it to GooseAttack::sync_metrics(). Will that work?

src/metrics.rs

…usize.

src/metrics.rs

…here was an error.

…c_metrics().

…usize.

…e request with the GooseRequestMetric.

src/metrics.rs

slashrsm · 2021-12-08T22:12:40Z

src/metrics.rs

-            expand_per_second_metric_array(&mut self.users_per_second, second, 0);
+            let last_user_count = match self.users_per_second.last() {
+                Some(last) => *last,
+                None => 0,
+            };
+            expand_per_second_metric_array(&mut self.users_per_second, second, last_user_count);
            self.users_per_second[second] = self.users;

-            expand_per_second_metric_array(&mut self.tasks_per_second, second, 0);
+            let last_task_count = match self.tasks_per_second.last() {
+                Some(last) => *last,
+                None => 0,
+            };
+            expand_per_second_metric_array(&mut self.tasks_per_second, second, last_task_count);


Users and tasks metric are not tied to a specific request so we still need to rely on Utc:now(). But this is OK since we will hit this immediately after a new user was spawned so the data should be accurate. The only thing we need to make sure is to carry the last count over the seconds we missed while the thread was sleeping to avoid gaps on the graph.

Tasks metrics has thread_user.started.elapsed().as_millis(), which is what is in the logs and is already accurate, why can't we use that?

And for users, why can't we also use thread_user.started?

I used GooseTaskMetric.elapsed, which also required me to move this code into receive_metric() again. But this time it is in the place where task metrics are received, so I think that it makes more sense.

src/metrics.rs

src/goose.rs

…h second on the graph request goes in.

…r users/tasks per second.

…cond vectors.

…f tasks that were executed each second.

jeremyandrews

This is fantastic! A couple small nits (in particular a style change that creates a lot of noise) then it's ready to merge. Very useful addition, thanks again!

CHANGELOG.md

src/metrics.rs

jeremyandrews · 2021-12-12T07:12:53Z

src/report.rs

+
+/// Helper function to build HTML charts powered by the
+/// [ECharts](https://echarts.apache.org) library.
+#[allow(clippy::too_many_arguments)]


I always aim to restructure how I call the function instead of selectively ignoring Clippy. Perhaps we can create an object to pass in, for example?

Makes sense. Done.

I ended up moving the function that generates markup onto the struct as well. This looks much nicer to me now.

src/metrics.rs

jeremyandrews · 2021-12-13T08:24:08Z

src/metrics.rs

@@ -2308,6 +2399,9 @@ impl GooseAttack {
                }
            }

+            // Record current users for users per second graph in HTML report.
+            self.metrics.record_users_per_second();


I still think this should be moved down to after we receive metrics, as otherwise we may not record a user until later than we should? It can be addressed in a follow, or let's talk through why this isn't relevant.

jeremyandrews

One small issue I'd like to talk through and address in a followup if it makes sense after we talk it through. Yay, merging!

tag1consulting#246: Add errors per second graph to the HTML report.

8821e12

slashrsm force-pushed the 246_additional_graphs branch from e06ea92 to 8821e12 Compare November 24, 2021 17:32

slashrsm added 2 commits November 24, 2021 19:00

tag1consulting#246: Add average request time graph to the HTML report.

b8abce3

tag1consulting#246: Add active users graph to the HTML report.

05a6eb4

slashrsm mentioned this pull request Nov 24, 2021

Test requests per second part of record_request_metric() #393

Open

tag1consulting#246: Add active tasks graph to the HTML report.

9494a35

slashrsm marked this pull request as ready for review November 24, 2021 19:43

slashrsm requested a review from jeremyandrews November 24, 2021 19:43

tag1consulting#246: Update CHANGELOG.md.

3e9fe17

jeremyandrews requested changes Nov 30, 2021

View reviewed changes

slashrsm added 2 commits December 6, 2021 14:31

tag1consulting#246: Move graphs in the relevant sections.

d7c72fb

tag1consulting#246: Change type of GooseMetrics::users_per_second to …

02347cd

…usize.

slashrsm force-pushed the 246_additional_graphs branch from e948168 to 5e96220 Compare December 6, 2021 21:31

jeremyandrews reviewed Dec 7, 2021

View reviewed changes

src/metrics.rs Outdated Show resolved Hide resolved

tag1consulting#246: Add more comments.

682fbdf

slashrsm force-pushed the 246_additional_graphs branch from 02cee74 to 66e5e53 Compare December 7, 2021 19:19

slashrsm requested a review from jeremyandrews December 7, 2021 20:31

slashrsm force-pushed the 246_additional_graphs branch from b157f45 to fa52a52 Compare December 7, 2021 21:07

jeremyandrews reviewed Dec 8, 2021

View reviewed changes

src/metrics.rs Outdated Show resolved Hide resolved

slashrsm added 4 commits December 8, 2021 21:26

tag1consulting#246: Use GooseRequestMetric::success to determine if t…

7c3edac

…here was an error.

tag1consulting#246: Collect user and task metrics in GooseAttack::syn…

7d827a9

…c_metrics().

tag1consulting#246: Add MovingAverage struct.

3af369d

tag1consulting#246: Change type of GooseMetrics::tasks_per_second to …

d944a5d

…usize.

slashrsm force-pushed the 246_additional_graphs branch from fa52a52 to d944a5d Compare December 8, 2021 21:26

slashrsm requested a review from jeremyandrews December 8, 2021 21:36

tag1consulting#246 and tag1consulting#386: Send the actual time of th…

bc80b91

…e request with the GooseRequestMetric.

slashrsm commented Dec 8, 2021

View reviewed changes

src/metrics.rs Outdated Show resolved Hide resolved

slashrsm commented Dec 8, 2021

View reviewed changes

src/metrics.rs Outdated Show resolved Hide resolved

jeremyandrews reviewed Dec 9, 2021

View reviewed changes

src/goose.rs Outdated Show resolved Hide resolved

slashrsm added 2 commits December 9, 2021 09:09

tag1consulting#246: Use GooseRequestMetric::elapsed to calculate whic…

dd542b6

…h second on the graph request goes in.

tag1consulting#246: Revert out of scope change.

70500a0

slashrsm force-pushed the 246_additional_graphs branch from 22fb7ac to 70500a0 Compare December 9, 2021 09:50

tag1consulting#246: Use GooseTaskMetric::elapsed to aggregate data fo…

499a084

…r users/tasks per second.

slashrsm requested a review from jeremyandrews December 9, 2021 19:05

slashrsm added 4 commits December 9, 2021 19:23

tag1consulting#246: Remove uneeded logic that fills task/users per se…

6f96383

…cond vectors.

tag1consulting#246: Improve function docs.

8079d0d

tag1consulting#246: Move the users graph into its own section.

834fd46

tag1consulting#246: Change tasks per second graph to display number o…

0cf043e

…f tasks that were executed each second.

jeremyandrews requested changes Dec 12, 2021

View reviewed changes

slashrsm added 2 commits December 12, 2021 08:13

tag1consulting#246: Remove unecessary change to the import from util.

f4bae2c

tag1consulting#246: Fix comments.

303787f

slashrsm force-pushed the 246_additional_graphs branch from 96bd5c7 to 9a45ab3 Compare December 12, 2021 10:29

slashrsm requested a review from jeremyandrews December 12, 2021 10:33

tag1consulting#246: Introduce graph struct.

9a2f685

slashrsm force-pushed the 246_additional_graphs branch from 9a45ab3 to 9a2f685 Compare December 12, 2021 10:48

jeremyandrews reviewed Dec 13, 2021

View reviewed changes

jeremyandrews approved these changes Dec 13, 2021

View reviewed changes

jeremyandrews merged commit 0dc4fc0 into tag1consulting:main Dec 13, 2021

jeremyandrews mentioned this pull request Dec 13, 2021

release 0.15.2 #402

Merged

This was referenced Dec 13, 2021

Check if users stats for graphs are recorded in the right place #405

Closed

Investigate spike or requests on transition between starting and started phase #386

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add additional graphs to the HTML report #394

Add additional graphs to the HTML report #394

slashrsm commented Nov 24, 2021 •

edited

Loading

jeremyandrews left a comment

jeremyandrews Nov 30, 2021

jeremyandrews Nov 30, 2021

slashrsm Dec 6, 2021

jeremyandrews Dec 7, 2021

slashrsm Dec 7, 2021

slashrsm Dec 8, 2021

jeremyandrews Dec 9, 2021

jeremyandrews Dec 9, 2021

slashrsm Dec 9, 2021

jeremyandrews left a comment

jeremyandrews Dec 12, 2021

slashrsm Dec 12, 2021

jeremyandrews Dec 13, 2021

jeremyandrews left a comment

Add additional graphs to the HTML report #394

Add additional graphs to the HTML report #394

Conversation

slashrsm commented Nov 24, 2021 • edited Loading

Graphs

Screenshots

Errors

Response time

Active users

jeremyandrews left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeremyandrews left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeremyandrews left a comment

Choose a reason for hiding this comment

slashrsm commented Nov 24, 2021 •

edited

Loading