Skip to content

Conversation

@weillercarvalho
Copy link

What?

  • Fix --summary-trend-stats parsing so invalid stats are rejected immediately instead of silently discarded.
  • Update config_consolidation_test.go cases to expect the new CLI error path.

Why?

Users could pass malformed stats (e.g. p(101), coun, p() without any feedback; the CLI would accept them and the test would run
even though the stats were invalid. Surfacing the error when parsing flags prevents confusing, silent failures.

Checklist

  • I have performed a self-review of my code.
  • I have commented on my code, particularly in hard-to-understand areas.
  • I have added tests for my changes.
  • I have run linter and tests locally (make check) and all pass. (Ran go test ./internal/cmd; full make tests still fails on
    browser/DNS-dependent suites in this environment.)

Checklist: Documentation (only for k6 maintainers and if relevant)

Related PR(s)/Issue(s)

#5424

@weillercarvalho weillercarvalho requested a review from a team as a code owner November 18, 2025 14:22
@weillercarvalho weillercarvalho requested review from ankur22 and codebien and removed request for a team November 18, 2025 14:22
@CLAassistant
Copy link

CLAassistant commented Nov 18, 2025

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

@ankur22 ankur22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @weillercarvalho thanks for contributing a fix!

I wanted to understand the original issue, so as you suggested in the original issue, I:

  1. Created an empty test script using export default function () {}.
  2. Ran the test with k6 run empty.js --summary-trend-stats="med,avg,p(101)"

However the test didn't run and I was presented with:

ERRO[0000] invalid percentile trend stat value 'p(101)', provide a number between 0 and 100 
exit status 255

After applying the fix (which does indeed look correct) I get the same result though. What were you seeing before the fix?

@weillercarvalho
Copy link
Author

Here is a quick explanation:

Before the patch, the CLI flag parser (getOptions()) tried to validate --summary-trend-stats, but it checked the wrong error variable. So even clearly bad values like p(101) slipped through that first gate and only failed later, when k6 run reached getConsolidatedConfig() and the config consolidation stage re-ran the same validation. That’s why @ankur22 you already saw ERRO[0000] invalid percentile… before my change: the later stage has always rejected invalid stats.

The fix just moves the failure to the place it should have always happened: as soon as the flag is parsed. We now return the validation error directly from getOptions(), and the consolidation tests were updated to expect a CLI read error for malformed stats. So the final user-facing error message didn’t change; what changed is that the CLI no longer accepts bogus stats for a moment before rejecting them. This keeps the behavior consistent across all commands that call getOptions(), not only the ones that happen to run the full consolidation pipeline.

Copy link
Contributor

@ankur22 ankur22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for explaining @weillercarvalho! LGTM 🚀

@weillercarvalho weillercarvalho temporarily deployed to azure-trusted-signing November 19, 2025 17:27 — with GitHub Actions Inactive
Copy link
Contributor

@codebien codebien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weillercarvalho That's a good catch, thanks for contributing! 🙇 LGTM, I'll merge as soon as we have a green CI.

@weillercarvalho weillercarvalho temporarily deployed to azure-trusted-signing November 19, 2025 17:29 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants