[Improve] Track Query Engine Modernization: TinkerPop 3.7.x, Groovy 4 and Java 17 #3069

Feature Description (功能描述)

Background

HugeGraph's core query engine has historically been built on an older runtime and query stack: Java 11, TinkerPop 3.5.x, and Groovy 3. While this foundation provides the essential graph query capabilities, it also limits long-term maintainability, security hardening, and access to newer runtime and language improvements.

The Groovy-based script execution path also relies on complex blacklist and whitelist mechanisms for sandboxing. This model is difficult to maintain and reason about when Groovy, TinkerPop, and the Java runtime evolve together. Modernizing the query engine is therefore both a dependency upgrade and a staged effort to establish clearer compatibility, testing, runtime, and security boundaries.

This issue tracks that modernization work. The TinkerPop 3.7.6 and Groovy 4.0.25 baseline has been merged; the current implementation phase is closing the compatibility gaps found after that merge. Java 17 implementation remains a follow-up stage.

Tracking Links

• Community planning note: https://hugegraph.feishu.cn/wiki/WoWpwSNZlidSHskAivccaQS9nig
• Merged baseline: refactor(server): upgrade TinkerPop to 3.7.6 and Groovy 4 hugegraph/hugegraph#142
• Current compatibility PR: fix: close 3.7.6 compatibility gaps hugegraph/hugegraph#163

Current Status — July 24, 2026

• Phase 1 baseline is complete. PR English docs perhaps? #142 merged the TinkerPop 3.7.6 and Groovy 4.0.25 baseline, including dependency alignment and the main serializer, predicate, traversal, remote-client, and test-suite adaptations.
• Phase 1.2 compatibility closure is in progress. PR mysql commit is invisible in other threads #163 connects the upstream Gherkin suite to HugeGraph and fixes transaction-visible count queries, unsupported TextP pushdown, GraphBinary ID encoding, typed GraphSON Path/Tree handling, dependency alignment, provider registration, and test lifecycle isolation.
• The latest PR mysql commit is invisible in other threads #163 head is 848ac8b. All 345 applicable memory Gherkin scenarios, the native memory/RocksDB new-step suites, and the functional CI matrix are passing. Server builds, Cluster, PD, Store, HStore, macOS, CodeQL, License, and dependency-review checks are green.
• One CI check is still red for an external access failure. dependency-check stopped before the project build because Maven Central returned HTTP 403 while resolving org.apache:apache:pom:23; this is not a TinkerPop compatibility regression.
• Java 17 is planned but implementation has not started. The implementation plan is ready, and the code migration remains gated on Phase 1.2 review and closure.

The remaining Phase 1.2 external validation is the full 345-scenario RocksDB Gherkin matrix, a slow-client stress smoke for Gremlin Server channel write-pause behavior, and acceptance against a future TinkerPop 3.7.7 release.

Current Direction

The merged 3.7.6 and Groovy 4 baseline remains the stable foundation for this modernization work. The immediate goal is to finish PR #163 with a reviewable compatibility contract and explicit provider limits. Java 17, Groovy sandbox review, broader distributed-runtime validation, benchmarks, release metadata, and TinkerPop 3.8 exploration remain separate follow-up workstreams.

What the Merged Baseline Changed

PR #142 established the main migration foundation:

• Upgraded the TinkerPop dependency line to 3.7.6.
• Upgraded Groovy-related dependencies to Groovy 4.0.25.
• Adapted code affected by TinkerPop package, API, traversal, predicate, and serializer changes.
• Updated HugeGraph-specific predicate and traversal integration.
• Preserved remote-client and serializer behavior across the upgraded dependency set.
• Normalized affected Cypher and remote response behavior.
• Updated tests and CI expectations for the 3.7.x dependency line.
• Kept PD, Store, and HStore dependency impact in scope.

PR #163 then closes compatibility gaps that were not visible in the earlier self-maintained suites:

• Runs applicable TinkerPop 3.7.6 Gherkin scenarios through a HugeGraph-specific provider lifecycle.
• Preserves uncommitted transaction data in optimized count queries.
• Keeps unsupported TextP predicates as local filters while retaining safe backend pushdown.
• Encodes HugeGraph IDs as standard GraphBinary wire primitives.
• Delegates typed GraphSON Path and Tree serialization to TinkerPop's versioned serializers.
• Removes the stale PD test dependency on gremlin-shaded:3.5.1 and guards version alignment.
• Hardens registration, process cleanup, scenario coverage floors, and provider lifecycle isolation in CI.

Planning Decisions

• Keep TinkerPop 3.7.6 and Groovy 4.0.25 as the first stable modernization baseline.
• Treat Server and Java Driver 3.7.6 as the supported compatibility pair; older driver lines are outside the current contract.
• Preserve TextP correctness through local filtering when backend pushdown is unavailable; this does not add text-index optimization.
• Keep existing provider limitations explicit instead of broadening test filters.
• Do not mix Java 17, Groovy sandbox redesign, TinkerPop 3.8, performance acceptance, and release metadata into the compatibility PR.

Workstream Status

• Baseline migration: TinkerPop 3.7.6 and Groovy 4.0.25 compatibility (PR English docs perhaps? #142).
• Phase 1.2 compatibility closure and external validation (PR mysql commit is invisible in other threads #163).
• Java 17 readiness and implementation: runtime, CI, Docker, and test coverage.
• Groovy sandbox review after the Groovy 4 migration.
• TinkerPop 3.8 readiness: delta map, compile spike, semantic regressions, provider-test plan, and acceptance gates.
• PD / Store / HStore distributed-runtime validation beyond the current CI coverage.
• Reproducible benchmark methodology and regression thresholds.
• Release metadata: dependency list, LICENSE, NOTICE, compatibility notes, and release notes.

Expected Next Steps

1. Complete review and the remaining external validation for PR mysql commit is invisible in other threads #163.
2. Close the Phase 1.2 compatibility boundary with explicit supported and deferred behavior.
3. Start the Java 17 implementation stage after mentor confirmation.
4. Continue the sandbox, distributed-runtime, benchmark, release, and TinkerPop 3.8 workstreams independently.