Add CTST tests for storage usage reporting API by delthas · Pull Request #2340 · scality/Zenko

delthas · 2026-03-02T15:20:02Z

Summary

Add end-to-end CTST tests for the new GET /instance/{uuid}/reporting/usage Pensieve route
Tests cover: StorageManager happy path (200 + structure validation), unauthorized Keycloak user (403), multi-account presence, and metric accuracy (objectsTotal/bytesTotal strict equality)
Extend managementAPIRequest with optional username/password params for testing different Keycloak users
Extend prepareMetricsScenarios with objectSize/objectCount options for uploading non-empty test objects

bert-e · 2026-03-02T15:20:07Z

Hello delthas,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2026-03-02T15:20:40Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

delthas · 2026-03-02T15:30:59Z

Depends on https://github.com/scality/pensieve-api/pull/351 and https://github.com/scality/vault2/pull/189

solution/deps.yaml

tests/ctst/steps/reporting/storageUsageReporting.ts

tests/ctst/common/hooks.ts

tests/ctst/steps/reporting/storageUsageReporting.ts

tests/ctst/features/reporting/StorageUsageReporting.feature

bert-e · 2026-03-10T11:05:13Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following reviewers are expecting changes from the author, or must review again:

@DarkIsDude

bert-e · 2026-03-11T09:32:50Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following reviewers are expecting changes from the author, or must review again:

@DarkIsDude

tests/ctst/common/common.ts

tests/ctst/steps/reporting/storageUsageReporting.ts

tests/ctst/common/common.ts

.github/scripts/end2end/run-e2e-ctst.sh

tests/ctst/common/common.ts

tests/ctst/steps/reporting/storageUsageReporting.ts

tests/ctst/common/utils.ts

tests/ctst/features/reporting/StorageUsageReporting.feature

tests/ctst/steps/reporting/storageUsageReporting.ts

Replace the three scalar World keys (identityNameForScenario, identityTypeForScenario, accountNameForScenario) with a typed SavedIdentity[] array. This makes multiple identities naturally queryable and eliminates manual additionalAccountNames tracking. - Add SavedIdentity interface and getSavedIdentities/getSavedIdentity helpers - saveIdentityInformation now appends to the array - useSavedIdentity uses getSavedIdentity() (defaults to last entry) - Migrate direct getSaved reads in bucket-policies and iam-policies - Simplify "{int} additional accounts" step - Drop additional accounts After cleanup hook Issue: ZENKO-5202

Use the existing data_consumer Keycloak user (which has the DataConsumer role, not StorageManager) instead of introducing a dedicated norights user. This removes the KeycloakUsernameNoRights world parameter and reuses the existing DataConsumerUsername parameter. Issue: ZENKO-5202

Call prepareMetricsScenarios directly in the Before hook instead of going through an unnecessary indirection layer. Issue: ZENKO-5202

Add optional size parameter to putObject and uploadSetup so callers can pass the object size directly rather than routing it through the World saved-state dictionary. Issue: ZENKO-5202

Issue: ZENKO-5202

… groups Collapse permission scenarios into a single parameterized Scenario Outline with role/expectedStatus examples. Separate content tests (structure, multi-account, metrics) into their own group. Replace two hardcoded When steps with a single parameterized step using a role-to-username lookup. Issue: ZENKO-5202

Disable camelcase rule for role-to-username map keys (must match Keycloak usernames). Shorten step names and parameter names to stay within 120-char line limit. Issue: ZENKO-5202

DarkIsDude

Can you check the CI ?

The roleToWorldParam lookup resolved role names to ctst_-prefixed usernames (e.g. ctst_storage_manager) which are not valid for direct Keycloak token acquisition. Pass the role string directly instead, matching how managementAPIRequest works elsewhere. Issue: ZENKO-5202

tests/ctst/features/reporting/StorageUsageReporting.feature

The SavedIdentity refactor changed useSavedIdentity to read from an array, but getSavedIdentity returns undefined when no identity was saved. This crashed CountItems which calls useSavedIdentity via 'the operation finished without error' without prior identity setup. Match old behavior by returning early when no identity exists. Issue: ZENKO-5202

- Rename "role" to "persona" in feature file and step definitions - Replace hardcoded Keycloak usernames with World parameter keys - Add Given step for persona setup, decoupled from When action - Split into "tries to retrieve" (no status check) and "retrieves" (asserts 200) - Add generic "Then the http response code is {int}" step in common - Remove section comments from feature file Issue: ZENKO-5202

Kubernetes events expire after 1 hour by default. In CI, kind export logs often runs 1-2 hours after the test window, by which time the relevant events have already been garbage-collected. This was a concrete gap during the Azure archive flakiness investigation (PR #2340): the failing tests ran around 06:36 but kind export logs didn't execute until ~08:22 — nearly 2 hours later. All Kubernetes events from the failure window (pod scheduling, restarts, liveness probe failures, OOM kills) had expired and were unavailable for diagnosis. Bump --event-ttl to 4h via a kubeadm ClusterConfiguration patch in the Kind cluster config, ensuring events survive for the entire duration of CI jobs. Issue: ZENKO-5217

World parameter keys (StorageManagerUsername, DataConsumerUsername) resolve to ctst_-prefixed usernames which lack the instanceIds Keycloak attribute required by the management API. Use the base Keycloak usernames directly. Issue: ZENKO-5202

delthas · 2026-03-16T09:09:35Z

Can you check the CI ?

CI now passes

DarkIsDude

Congrats. Thanks for all the refactoring done. LGTM 🙏

bert-e · 2026-03-16T10:11:09Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

francoisferrand · 2026-03-17T19:08:34Z

solution/deps.yaml

  sourceRegistry: ghcr.io/scality
  image: pensieve-api
-  tag: 1.9.0
+  tag: 1.9.1


depending on timing, we may want to bump 1.9.2 and update VERSION (when rebasing before the merge) so we can make the release (c.f. https://scality.atlassian.net/browse/ZENKO-5210)
→ ping me if in doubt how to "optimize" rebuilds

francoisferrand · 2026-03-18T08:27:46Z

/after_pull_request=2357

bert-e · 2026-03-18T22:30:01Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following options are set: after_pull_request

scality deleted a comment from bert-e Mar 2, 2026

delthas force-pushed the improvement/ZENKO-5202/usage-reporting-route branch 5 times, most recently from 3f908f3 to eee9043 Compare March 4, 2026 21:06

delthas requested review from a team, SylvainSenechal and benzekrimaha March 4, 2026 21:08

delthas force-pushed the improvement/ZENKO-5202/usage-reporting-route branch 6 times, most recently from 42df742 to 1ef0c43 Compare March 5, 2026 18:39

delthas requested a review from DarkIsDude March 6, 2026 10:27

DarkIsDude requested changes Mar 9, 2026

View reviewed changes

scality deleted a comment from bert-e Mar 9, 2026

delthas force-pushed the improvement/ZENKO-5202/usage-reporting-route branch 2 times, most recently from 54b4acc to 4e692d3 Compare March 10, 2026 11:05

scality deleted a comment from bert-e Mar 10, 2026

francoisferrand requested a review from DarkIsDude March 11, 2026 09:30

delthas force-pushed the improvement/ZENKO-5202/usage-reporting-route branch from 4e692d3 to 0885b8e Compare March 11, 2026 09:32

scality deleted a comment from bert-e Mar 11, 2026

DarkIsDude reviewed Mar 11, 2026

View reviewed changes

francoisferrand reviewed Mar 12, 2026

View reviewed changes

delthas added 6 commits March 12, 2026 15:09

Remove prepareStorageUsageReportingScenarios wrapper function

20fabe4

Call prepareMetricsScenarios directly in the Before hook instead of going through an unnecessary indirection layer. Issue: ZENKO-5202

Pass objectSize directly to putObject instead of saving to World

ac81793

Add optional size parameter to putObject and uploadSetup so callers can pass the object size directly rather than routing it through the World saved-state dictionary. Issue: ZENKO-5202

Put async function on same line as When/Then step registration

3407bb0

Issue: ZENKO-5202

delthas requested review from DarkIsDude and francoisferrand March 12, 2026 16:34

Fix lint errors in storage usage reporting steps

12e1e09

Disable camelcase rule for role-to-username map keys (must match Keycloak usernames). Shorten step names and parameter names to stay within 120-char line limit. Issue: ZENKO-5202

DarkIsDude requested changes Mar 12, 2026

View reviewed changes

francoisferrand reviewed Mar 13, 2026

View reviewed changes

delthas added 2 commits March 13, 2026 10:13

delthas mentioned this pull request Mar 13, 2026

Increase Kind apiserver event TTL from 1h to 4h #2351

Merged

delthas requested review from DarkIsDude and francoisferrand March 16, 2026 09:09

DarkIsDude approved these changes Mar 16, 2026

View reviewed changes

francoisferrand approved these changes Mar 17, 2026

View reviewed changes

francoisferrand mentioned this pull request Mar 18, 2026

Bump pensieve-api & release 2.14.1 #2358

Merged

scality deleted a comment from bert-e Mar 18, 2026

bert-e merged commit 607e6f4 into development/2.14 Mar 19, 2026
27 of 28 checks passed

bert-e deleted the improvement/ZENKO-5202/usage-reporting-route branch March 19, 2026 02:27

Conversation

delthas commented Mar 2, 2026

Summary

Uh oh!

bert-e commented Mar 2, 2026

Hello delthas,

Uh oh!

bert-e commented Mar 2, 2026

Waiting for approval

Uh oh!

delthas commented Mar 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bert-e commented Mar 10, 2026

Waiting for approval

Uh oh!

bert-e commented Mar 11, 2026

Waiting for approval

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DarkIsDude left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

delthas commented Mar 16, 2026

Uh oh!

DarkIsDude left a comment

Choose a reason for hiding this comment

Uh oh!

bert-e commented Mar 16, 2026

Waiting for approval

Uh oh!

francoisferrand Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

francoisferrand commented Mar 18, 2026

Uh oh!

bert-e commented Mar 18, 2026

Waiting for approval

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants