Skip to content

APPSEC-61442 - SDK integration of SDS scans by AI Guard [system-tests]#6384

Open
obordeau wants to merge 6 commits intomainfrom
oceane.bordeau/create-ai-guard-sds-system-tests
Open

APPSEC-61442 - SDK integration of SDS scans by AI Guard [system-tests]#6384
obordeau wants to merge 6 commits intomainfrom
oceane.bordeau/create-ai-guard-sds-system-tests

Conversation

@obordeau
Copy link

@obordeau obordeau commented Feb 26, 2026

Motivation

Test that AI Guard sets SDS tags on spans from evaluator response.

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@github-actions
Copy link
Contributor

github-actions bot commented Feb 26, 2026

CODEOWNERS have been resolved as:

utils/build/docker/vcr/cassettes/aiguard/aiguard_evaluate_post_ee2b240f.json  @DataDog/k9-ai-guard @DataDog/system-tests-core
manifests/cpp_httpd.yml                                                 @DataDog/dd-trace-cpp
manifests/cpp_nginx.yml                                                 @DataDog/dd-trace-cpp
manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet
manifests/golang.yml                                                    @DataDog/dd-trace-go-guild
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
manifests/nodejs.yml                                                    @DataDog/dd-trace-js
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
manifests/python.yml                                                    @DataDog/apm-python @DataDog/asm-python
manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby
tests/ai_guard/test_ai_guard_sdk.py                                     @DataDog/k9-ai-guard @DataDog/system-tests-core

@obordeau obordeau changed the title Oceane.bordeau/create ai guard sds system tests APPSEC-61442 - SDK integration of SDS scans by AI Guard [system-tests] Feb 26, 2026
@obordeau obordeau marked this pull request as ready for review February 26, 2026 16:06
@obordeau obordeau requested review from a team as code owners February 26, 2026 16:06
@obordeau obordeau requested review from Anilm3, claponcet, jandro996, mabdinur, taegyunkim and ygree and removed request for a team February 26, 2026 16:06
Copy link
Member

@smola smola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly done, but we also need to add a new casette with a snapshot of the HTTP response from AI Guard for this request, like I did here: #6115

tbh I don't remember how the cassette update was triggered, but it's probably easier to figure out if you set python.yml temporarily set as supported for the latest version to then inspect the failure logs

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 91b6a65878

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@obordeau obordeau changed the title APPSEC-61442 - SDK integration of SDS scans by AI Guard [system-tests] [python@oceane.bordeau/add-sds-ai-guard-spans] APPSEC-61442 - SDK integration of SDS scans by AI Guard [system-tests] Feb 26, 2026
@obordeau obordeau marked this pull request as draft February 26, 2026 17:02
@obordeau obordeau changed the title [python@oceane.bordeau/add-sds-ai-guard-spans] APPSEC-61442 - SDK integration of SDS scans by AI Guard [system-tests] APPSEC-61442 - SDK integration of SDS scans by AI Guard [system-tests] Feb 26, 2026
@smola smola marked this pull request as ready for review February 27, 2026 08:49
assert len(sds) > 0, f"No 'sds' found in metastruct {ai_guard}"
for sd in sds:
assert _assert_key(sd, "rule_display_name")
assert _assert_key(sd, "rule_tag")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we assert a specific value here ? like
_assert_key(sd, "rule_tag", specific_value)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants