Skip to content

chore(xtest): Audit and local improvements together#390

Draft
dmihalcik-virtru wants to merge 5 commits intoopentdf:mainfrom
dmihalcik-virtru:chore/the-claudiest-day-tmux
Draft

chore(xtest): Audit and local improvements together#390
dmihalcik-virtru wants to merge 5 commits intoopentdf:mainfrom
dmihalcik-virtru:chore/the-claudiest-day-tmux

Conversation

@dmihalcik-virtru
Copy link
Member

  • feat(xtest): add tmux-based local test environment with modular scripts
  • feat(xtest): add audit log processing framework and test integration

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @dmihalcik-virtru, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly overhauls the local testing infrastructure for the xtest suite, introducing a new, highly structured approach to managing test environments and debugging. The changes aim to provide developers with a more reliable, consistent, and user-friendly experience when running and troubleshooting integration tests. By modularizing shell scripts and integrating a dedicated audit log analysis framework, the PR enhances the overall maintainability and diagnostic capabilities of the test harness.

Highlights

  • New Local Test Environment: Introduced a comprehensive tmux-based local test environment for xtest, enabling streamlined management of all services (Docker, Platform, KAS instances) through modular shell scripts. This includes dedicated scripts for starting, stopping, status checks, and log viewing.
  • Modular Shell Script Library: Developed a new modular shell script library (xtest/scripts/lib/) with 55 unit tests, providing reusable utilities for logging, platform detection, path resolution, health checks, service waiting, tmux management, KAS-specific configurations, and YAML manipulation. This library ensures consistent and robust scripting across the test suite.
  • KAS Audit Log Framework: Added a KAS audit log collection and assertion framework (xtest/audit_logs.py and xtest/fixtures/audit.py). This framework allows tests to capture KAS service logs in the background, mark timestamps, and assert on log contents, significantly enhancing debugging capabilities for KAS-related issues.
  • Updated KAS Naming Conventions: Refactored KAS instance naming in fixtures from generic 'value1', 'value2', 'attr', 'ns' to more descriptive 'alpha', 'beta', 'gamma', 'delta' to align with CI workflow naming and improve clarity.
  • EC-Wrapped TDF Support: Updated the manifest.schema.json to include 'ec-wrapped' as a valid key access object type, reflecting enhanced support for Elliptic Curve-based TDF wrapping algorithms.
  • Test Infrastructure Improvements: Refactored the skip_if_unsupported logic in tdfs.py for better clarity and removed the pytest-xdist dependency from pyproject.toml. The tmp_dir fixture in conftest.py was also simplified.
  • Updated LTS Versions: Adjusted the Long Term Support (LTS) versions for Go, Java, JavaScript SDKs, and the Platform in resolve-version.py.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Ignored Files
  • Ignored by pattern: .github/workflows/** (1)
    • .github/workflows/xtest.yml
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new audit logging framework for tests and refactors the local test environment to use a modular, tmux-based shell script system. The changes are extensive and well-structured. The new audit log framework is a great addition for improving test diagnostics, and the shell script refactoring improves maintainability. I've found a critical issue in a path helper function within the new script library and some minor path issues in the updated documentation. Overall, this is a significant improvement to the test infrastructure.

Comment on lines 31 to 35
get_platform_dir() {
local xtest_dir
xtest_dir="$(get_xtest_dir)"
echo "$(cd "$xtest_dir/../../platform" && pwd)"
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The get_platform_dir function appears to construct an incorrect path. It uses cd "$xtest_dir/../../platform", which resolves to a directory one level above the repository root. Assuming the platform directory is a sibling of the xtest directory within the repository, the correct relative path should be ../platform from the xtest directory.

Suggested change
get_platform_dir() {
local xtest_dir
xtest_dir="$(get_xtest_dir)"
echo "$(cd "$xtest_dir/../../platform" && pwd)"
}
get_platform_dir() {
local xtest_dir
xtest_dir="$(get_xtest_dir)"
echo "$(cd "$xtest_dir/../platform" && pwd)"
}


## Build, Test, and Development Commands
### Structure
- **Test Directory**: `tests/xtest/` - pytest-based integration tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This documentation contains a few incorrect paths that could confuse users:

  • On line 8, the test directory should be xtest/, not tests/xtest/.
  • Several example commands (e.g., on lines 347, 361, 362, 714) use paths like platform/opentdf-dev.yaml. Since test commands are typically run from the xtest/ directory, the correct relative path to the platform directory would be ../platform/.

@dmihalcik-virtru dmihalcik-virtru force-pushed the chore/the-claudiest-day-tmux branch from 7dfa6b5 to 41fc3fe Compare January 29, 2026 22:05
- Implement lmgmt Python CLI tool for managing local test environment
- Add service abstraction (Platform, KAS, Docker, Provisioner)
- Implement health checks and wait utilities
- Add structured log aggregation with filtering capabilities
- Include comprehensive pytest integration tests
- Update CI workflows for audit log collection
- Add comprehensive audit log test coverage for TDF roundtrip, ABAC, and policy tests
- Implement clock skew estimation for accurate timestamp comparison
- Increase audit log assertion timeout and report late-arriving logs
- Add integration tests for audit log collection in CI pipeline
- Make audit assertions required by default with override option
- Fix subprocess deadlock in otdfctl by properly handling stdout/stderr
- Capture stderr output for better debugging when CLI calls fail
- Add env command to lmgmt for outputting pytest environment variables
- Document lmgmt Python CLI as recommended tool for environment management
- Add quick reference commands for service management
- Update troubleshooting workflow to use lmgmt
- Include audit log assertions requirements and disabling procedures
- Add .shfmt configuration for shell script formatting
- Update .gitignore for generated files
- Update xtest dependencies and manifest schema
@dmihalcik-virtru dmihalcik-virtru force-pushed the chore/the-claudiest-day-tmux branch from e5921dd to deb3563 Compare February 6, 2026 15:56
@sonarqubecloud
Copy link

sonarqubecloud bot commented Feb 6, 2026

Quality Gate Failed Quality Gate failed

Failed conditions
1 Security Hotspot
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant