Skip to content

[system_tests] Fix flaky IAST standalone tests via library_interface_timeout#6377

Draft
vlad-scherbich wants to merge 13 commits intomainfrom
vlad/unflake_test_asm_standalone
Draft

[system_tests] Fix flaky IAST standalone tests via library_interface_timeout#6377
vlad-scherbich wants to merge 13 commits intomainfrom
vlad/unflake_test_asm_standalone

Conversation

@vlad-scherbich
Copy link
Contributor

@vlad-scherbich vlad-scherbich commented Feb 24, 2026

Motivation

Flaky tests in test_asm_standalone.py::Test_IastStandalone_UpstreamPropagation_V2 (e.g. test_no_appsec_upstream__no_asm_event__is_kept_with_priority_1__from_0). See PROF-13796.

Root cause: traces sometimes weren't flushed to the agent before the scenario shut down the weblog. The tracer sends traces asynchronously; without enough delay, traces could still be buffered when containers stop.

Changes

  • iast_standalone scenario (utils/_context/_scenarios/__init__.py): Set library_interface_timeout=15 so the tracer has 15 seconds to flush traces before the weblog shuts down.
  • test_asm_standalone.py: Remove the per-setup _iast_standalone_wait_for_trace poll loop. Per-review feedback: avoid wait_for in setup functions (makes the scenario slow when many tests run). Use the global library_interface_timeout instead—one delay before shutdown achieves the same effect.

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@github-actions
Copy link
Contributor

github-actions bot commented Feb 24, 2026

CODEOWNERS have been resolved as:

tests/debugger/test_debugger_telemetry.py                               @DataDog/debugger @DataDog/system-tests-core
utils/_context/_scenarios/__init__.py                                   @DataDog/system-tests-core

@datadog-datadog-prod-us1-2

This comment has been minimized.

@vlad-scherbich vlad-scherbich changed the title attempt to fix flaky test_asm_standalone.py tests [system_tests] Fix flaky test_asm_standalone.py tests Feb 24, 2026
@vlad-scherbich vlad-scherbich marked this pull request as ready for review February 25, 2026 01:20
@vlad-scherbich vlad-scherbich requested review from a team as code owners February 25, 2026 01:20
@vlad-scherbich vlad-scherbich force-pushed the vlad/unflake_test_asm_standalone branch from 8f85efa to b048484 Compare February 25, 2026 17:55
@vlad-scherbich vlad-scherbich marked this pull request as draft February 25, 2026 19:33
@vlad-scherbich vlad-scherbich force-pushed the vlad/unflake_test_asm_standalone branch from 8ffe6ab to 1cdbc66 Compare February 26, 2026 14:37
@vlad-scherbich vlad-scherbich marked this pull request as ready for review February 26, 2026 16:19
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1cdbc66095

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@vlad-scherbich vlad-scherbich force-pushed the vlad/unflake_test_asm_standalone branch from 6e3aa92 to ef911ea Compare February 26, 2026 16:28
@vlad-scherbich vlad-scherbich enabled auto-merge (squash) February 26, 2026 16:28
@vlad-scherbich vlad-scherbich changed the title [system_tests] Fix flaky test_asm_standalone.py tests [system_tests] Fix flaky IAST standalone tests via library_interface_timeout Feb 27, 2026
@vlad-scherbich vlad-scherbich force-pushed the vlad/unflake_test_asm_standalone branch from ef911ea to ca1185b Compare February 27, 2026 14:06
@vlad-scherbich vlad-scherbich marked this pull request as draft February 27, 2026 15:14
auto-merge was automatically disabled February 27, 2026 15:14

Pull request was converted to draft

@vlad-scherbich vlad-scherbich force-pushed the vlad/unflake_test_asm_standalone branch from d216ed3 to 06e22f4 Compare February 27, 2026 17:29
@vlad-scherbich vlad-scherbich marked this pull request as ready for review February 27, 2026 20:34
@vlad-scherbich vlad-scherbich requested a review from a team as a code owner February 27, 2026 20:34
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 06e22f4288

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@vlad-scherbich vlad-scherbich marked this pull request as draft February 27, 2026 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants