feat(xtest): add download-artifact.sh scripts for published SDK packages#386
Open
pflynn-virtru wants to merge 32 commits intomainfrom
Open
feat(xtest): add download-artifact.sh scripts for published SDK packages#386pflynn-virtru wants to merge 32 commits intomainfrom
pflynn-virtru wants to merge 32 commits intomainfrom
Conversation
Add AI assistant configuration files with project guidelines and development instructions. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
- Fix PR matrix bug: remove hardcoded 'main lts' default so DEFAULT_TAGS logic applies correctly (PRs now run 3 jobs instead of 6) - Add concurrency controls with github.run_id fallback to cancel superseded PR runs without affecting scheduled runs - Add dependency caching for npm, Go modules, and Maven repository (placed after SDK checkouts where lock files exist) Closes #377 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
The script was silently failing when pom.xml had no <properties> section
(e.g., Java SDK v0.7.5). The sed command to add <platform.branch> would
do nothing, but the replacement of branch=main with ${platform.branch}
would still run, causing Maven to pass the literal string to buf/git.
Now the branch replacement only runs if the property was successfully
added (or already existed).
Fixes #379
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Paul Flynn <pflynn@virtru.com>
…a.sh Java SDK versions need compatible platform protocol branches: - v0.7.5 uses protocol/go/v0.2.18, not main - Added get_platform_branch() function mapping versions to protocols - Mappings match those in resolve-version.py This fixes compilation errors when building older Java SDK versions against incompatible protocol definitions. Fixes #379 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
Update post-checkout-java.sh to: - Check if the CORRECT platform.branch value is set, not just any value - Replace existing incorrect values with the version-specific protocol branch - Add explicit return statement to get_platform_branch function This fixes Java SDK v0.7.5 (lts) builds that were failing because the pom.xml already had <platform.branch>main</platform.branch> defined, but needed <platform.branch>protocol/go/v0.2.18</platform.branch> for compatibility. Fixes #379 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
…xists When the pom.xml has no <properties> section (like Java SDK v0.7.5), directly replace `branch=main` with `branch=protocol/go/v0.2.18` instead of trying to use Maven property substitution which doesn't work. Fixes #379 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
Update LTS versions to more recent releases from summer 2025: - Go: 0.15.0 → 0.24.0 (2025-07-31) - Java: 0.7.5 → 0.9.0 (2025-07-14) - JS: 0.2.0 → 0.4.0 (2025-06-30) - Platform: 0.4.34 → 0.9.0 (2025-08-27) Also update PR runs to test main, latest, and lts versions (9 jobs total) to ensure comprehensive coverage. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
Don't overwrite platform.branch in pom.xml files that already have a valid protocol/go/v* value set. Only modify files that: - Are missing the platform.branch property - Have a specific mapping in the script (older versions) This fixes Java v0.9.0+ builds which already have the correct platform.branch defined in their pom.xml. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
- Update bufbuild/buf-setup-action to v1.50.0 - Use github.token instead of secrets.GITHUB_TOKEN - Set buf version to 1.56.0 - Remove BUF_INPUT_HTTPS env vars (secret doesn't exist in tests repo) This should fix the buf rate limiting issue by using proper authentication. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
- Replace buf-setup-action with buf-action (consolidated action) - Use setup_only: true to just install buf CLI - Add BUF_TOKEN secret for BSR rate limit authentication - Fixes BSR 'too many requests' errors during parallel Java builds Signed-off-by: Paul Flynn <pflynn@virtru.com>
- Grant `id-token: write` permission for GCP authentication - Add steps to authenticate with Google Cloud and retrieve BUF_TOKEN secret from Secret Manager - Update BUF action to use token from retrieved secrets
- Replace hardcoded GCP auth parameters with reusable workflow variables - Pin google-github-actions/auth and get-secretmanager-secrets actions to specific commits for better reproducibility
- Upgrade `google-github-actions/auth` to v3.0.0 - Use `secrets.GCP_WORKLOAD_IDENTITY` instead of deprecated vars - Update BUF_TOKEN secret reference to a specific project identifier
- Eliminate GCP authentication and secret retrieval steps - Update BUF_TOKEN reference to use `secrets.BUF_TOKEN` directly
Add pytest-xdist for parallel test execution to improve CI performance: - Add pytest-xdist>=3.6.1 dependency to pyproject.toml - Update tmp_dir fixture to create worker-specific directories, preventing file collisions between parallel test processes - Add -n auto --dist loadscope flags to pytest commands in xtest.yml The loadscope distribution groups tests by module/class, which aligns with the package-scoped fixtures used in the test suite. Expected impact: 40-60% reduction in test runtime. Closes #377 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
Add scripts to download published SDK artifacts from their respective package registries (npm, Maven Central, go install) instead of building from source. This enables testing against released versions. - Go: downloads otdfctl via `go install` - Java: downloads cmdline.jar from Maven Central - JS: installs @opentdf/ctl via npm Adds test-artifacts CI job to validate the download scripts work correctly. Closes #382 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
The Java cmdline.jar is not published to Maven Central - it must be built from source. Remove the Java test from CI and add explanatory comments to the download-artifact.sh script for future use. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
X-Test Failure Report |
Move SCRIPT_DIR resolution to the beginning of the script, before changing to DIST_DIR, so the relative path resolution works correctly. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
X-Test Failure Report |
Instead of building the full java-sdk from source, build only the cmdline module and let Maven resolve the SDK dependency from Maven Central. This is faster since we skip building the sdk module. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
X-Test Failure Report |
The -am flag was causing Maven to build the sdk module from source. Without it, Maven will fetch the released SDK from Maven Central. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
X-Test Failure Report |
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
X-Test Failure Report |
The enforcer plugin requires parent modules in the reactor, but we're only building cmdline. Skip the check with -Denforcer.skip=true. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
X-Test Failure Report |
For released versions (not HEAD), use the optimized download-artifact.sh scripts instead of building from source: - Go: uses `go install` for pre-built binary - JS: uses `npm install` for published package - Java: builds only cmdline module with released SDK from Maven Central HEAD versions continue to build from source as before. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
X-Test Failure Report |
1 similar comment
X-Test Failure Report |
- Add sdk-config.json with repo URLs, LTS versions, and artifact info - Update resolve-version.py to read from config and add 'version' field - The 'version' field is the clean version number (e.g., "0.4.0") - Update workflow to use 'version' instead of parsing 'release' field This fixes the JS build failure where release tag "sdk/v0.4.0" was being incorrectly passed to npm install. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
X-Test Failure Report |
1 similar comment
X-Test Failure Report |
The scripts change directories during execution, so relative paths for DIST_DIR no longer resolve correctly. Convert to absolute path after mkdir to ensure copies work from any directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
The resolve-versions job was checking out main branch instead of the PR branch, so it was using the old resolve-version.py without the 'version' field needed for download-artifact.sh. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
X-Test Failure Report |
X-Test Results✅ js-v0.11.6 |
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
- Java/JS HEAD builds now download from `dev` GitHub releases instead of building from source - Updated download-artifact.sh scripts to support `dev` version - Updated Java to 17/temurin distribution - Go remains unchanged (builds from source due to platform SDK dependency) Requires opentdf/java-sdk and opentdf/web-sdk to have dev-artifact workflows. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn@virtru.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
download-artifact.shscripts for each SDK to use published/released artifacts:otdfctlbinary viago install github.com/opentdf/otdfctl@v{VERSION}✅@opentdf/ctlfrom npm ✅cmdlinemodule using released SDK from Maven Central ✅-pl cmdlinewithout-amto fetch sdk dependency from Maven CentralAdd
test-artifactsCI job to validate the scripts work correctly.Test plan
test-artifactsjob passes:Closes #382
🤖 Generated with Claude Code