feat: improve the latency and throughput formatting in the result table - #233
Conversation
| 'Latency average (ns)': `${mToNs(task.result.latency.mean).toFixed(2)} \xb1 ${task.result.latency.rme.toFixed(2)}%`, | ||
| 'Latency average (ns)': `${formatNumber(mToNs(task.result.latency.mean), 5, 2)} \xb1 ${task.result.latency.rme.toFixed(2)}%`, | ||
| // eslint-disable-next-line @typescript-eslint/no-non-null-assertion | ||
| 'Latency median (ns)': `${mToNs(task.result.latency.p50!).toFixed(2)}${Number.parseFloat(mToNs(task.result.latency.mad!).toFixed(2)) > 0 ? ` \xb1 ${mToNs(task.result.latency.mad!).toFixed(2)}` : ''}`, |
There was a problem hiding this comment.
Please keep the threshold condition here : when the median absolute deviation is upper zero, it's a sign of statistical significance issue during the benchmark and a valuable information.
You can use the same helper to display it.
There was a problem hiding this comment.
I'm not sure whether I get correctly what the absolute deviation is supposed to measure.
So far I've only seen it in the table when the number of samples was even. If the median for even number of samples is problematic, shouldn't the task be executed one more time instead of reporting a problem?
There was a problem hiding this comment.
It's totally unrelated to the number of samples. It's a robust statistical indicator of the statistical significance of the median value. And the only one for median. Like the standard deviation (or the absolute mean deviation) is for the average.
Adding a threshold to it is only way to reduce the information to display.
There was a problem hiding this comment.
Thanks for the explanation. After some debugging I understood what's going on. There is a bug in absoluteDeviation that makes mad exactly zero as opposed to a small positive number in most cases. I filed #234.
| : (convert?.(task) ?? { | ||
| 'Task name': task.name, | ||
| 'Latency average (ns)': `${mToNs(task.result.latency.mean).toFixed(2)} \xb1 ${task.result.latency.rme.toFixed(2)}%`, | ||
| 'Latency average (ns)': `${formatNumber(mToNs(task.result.latency.mean), 5, 2)} \xb1 ${task.result.latency.rme.toFixed(2)}%`, |
There was a problem hiding this comment.
I think you can also use the formaNumber helper for the standard deviation.
Displaying number only in the column would be better, but I do not think it's possible because of string only characters usage.
There was a problem hiding this comment.
For percentages less than 1000% the helper returns the same string as toFixed(2). I slightly prefer keeping toFixed(2) because of its simpler semantics.
commit: |
Make the result table a bit more compact, as per the discussion under issue #231
Example result table for
simple benchmark gc: