Skip to content

feat: Add a template-optimizer pipeline for REST-connector templates - #7359

Draft
vringar with Claude wants to merge 4 commits into
mainfrom
claude/add-template-optimizer-pipeline
Draft

feat: Add a template-optimizer pipeline for REST-connector templates#7359
vringar with Claude wants to merge 4 commits into
mainfrom
claude/add-template-optimizer-pipeline

Conversation

@Claude

@Claude Claude AI commented May 26, 2026

Copy link
Copy Markdown
Contributor

Adds an element-template-optimizer module that compacts conditional properties in generated Camunda element templates. REST-based templates frequently emit dozens of near-identical conditional properties per multi-operation API; collapsing them is a ~45% property-count reduction on real specs without any change in applied behaviour.

Closes #7357.

What the optimizer does

Four passes run in sequence, each taking an ElementTemplate and returning a new one:

  • merge-by-identity — groups properties by (binding, value, presentation) and collapses each group whose members differ only in their condition's discriminator value into one property with a oneOf condition over the union.
  • totalize — drops conditions that cover every value of their discriminator (currently scoped to Dropdown discriminators).
  • strength-reduce — rewrites singleton oneOf as equals.
  • reorder — emits properties in a stable canonical order (group → visibility → id) so regenerations produce review-friendly diffs.

How it's integrated

congen-cli's generate command runs the default pipeline before serializing each template, so generated templates always ship in their compacted form.

A standalone element-template-optimizer binary is also provided for one-shot cleanup of hand-maintained templates that bypass the generator (with --skip-passes, --dry-run, and a list-passes subcommand).

Branch layout

Three commits:

  1. Add element-template-optimizer module — the four passes, the Optimizer library entrypoint, the PropertyUtils rebuilder, polymorphism-aware deserializers for the sealed Property / PropertyBinding / PropertyCondition hierarchies, and the test suite.
  2. Add element-template-optimizer standalone CLIMain + OptimizerCommand.
  3. Wire element-template-optimizer into congen-cli generate command — runs the default pipeline unconditionally before serialization, with try/catch and a dedicated OPTIMIZATION_FAILED return code.

Behavioural equivalence

The core DSL is sealed without Jackson polymorphism annotations, so this module ships strict deserializers for Property, PropertyBinding, and PropertyCondition. They fail loud on missing or conflicting discriminators, non-string equals values, empty binding identifiers, unknown feel modes, and unknown generatedValue.type.

OptimizerPropertyTest enumerates 36 multi-operation template shapes (operations × shared params × unshared params × unconditional) and asserts XMLUnit-level BPMN equivalence per discriminator value by applying both the original and optimized templates via the npm-published @camunda/element-templates-cli. The test is gated by @EnabledIf; when CI=true or OPTIMIZER_REQUIRE_CLI=true is set and the CLI binary is missing, the gate is a hard failure rather than a silent skip.

Out of scope / follow-ups

  • TotalizePass currently considers only Dropdown properties as discriminators; a constant-valued hidden property would also be a tautological discriminator. Documented in the module README; can be widened in a follow-up.
  • A golden-file regression test on the existing OpenAPI / Postman generators would catch surprising behaviour changes on real templates beyond the parameterized equivalence suite.

@vringar vringar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@claude[agent] Left comments focussed on the overall structure. Don't assume the rest is good but focus on getting the architecture right first. Also how do we integrate this with the congen CLI. Note the PR you are likely to be blocked by.

Comment thread element-template-generator/optimizer/.jqwik-database Outdated
@vringar

vringar commented May 26, 2026

Copy link
Copy Markdown
Collaborator

#7354 is required for the CLI to be runable again. I'd like this pipeline to be a postprocessing step before the file gets written out to disk, so that for generated templates we only expose the compacted form. In this context none of the disable flags are relevant.

Copilot stopped work on behalf of vringar due to an error May 26, 2026 16:44
@Claude
Claude AI force-pushed the claude/add-template-optimizer-pipeline branch from 7cb1ef0 to 110dd18 Compare May 26, 2026 16:55
Copilot stopped work on behalf of vringar due to an error May 26, 2026 16:55
Copilot stopped work on behalf of vringar due to an error May 27, 2026 07:24
@vringar
vringar force-pushed the claude/add-template-optimizer-pipeline branch 4 times, most recently from 044bc94 to b3edb39 Compare May 27, 2026 15:07
@vringar vringar changed the title [WIP] Add a template-optimizer pipeline for REST-connector templates Add a template-optimizer pipeline for REST-connector templates May 27, 2026
@camunda camunda deleted a comment from Claude AI May 27, 2026
@camunda camunda deleted a comment from Claude AI May 27, 2026
@camunda camunda deleted a comment from Claude AI May 27, 2026
@camunda camunda deleted a comment from Claude AI May 27, 2026
@camunda camunda deleted a comment from Claude AI May 27, 2026
@camunda camunda deleted a comment from Claude AI May 27, 2026
@camunda camunda deleted a comment from Claude AI May 27, 2026
@camunda camunda deleted a comment from Claude AI May 27, 2026
@camunda camunda deleted a comment from Claude AI May 27, 2026
@camunda camunda deleted a comment from Claude AI May 27, 2026
@vringar
vringar marked this pull request as ready for review May 28, 2026 14:04
Copilot AI review requested due to automatic review settings May 28, 2026 14:04
@vringar
vringar requested a review from a team as a code owner May 28, 2026 14:04
@vringar
vringar requested a review from ztefanie May 28, 2026 14:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new element-template-optimizer Maven module under element-template-generator/ that compacts conditional properties in generated Camunda element templates via four sequential passes (merge-by-identity, totalize, strength-reduce, reorder) operating on the typed ElementTemplate IR. The optimizer is exposed both as a library entrypoint (Optimizer.defaultPipeline()), as a standalone CLI binary (element-template-optimizer), and is wired unconditionally into congen-cli's generate command before serialization. Strict polymorphic Jackson deserializers for the sealed Property / PropertyBinding / PropertyCondition hierarchies are added because the core DSL ships serializers only.

Changes:

  • New element-template-optimizer module: four passes, Optimizer entrypoint, PropertyUtils builder-based rebuilder, strict polymorphic deserializers, and a standalone CLI (OptimizerCommand + ListPassesCommand).
  • congen-cli generate runs Optimizer.defaultPipeline() after generation and before JSON-schema validation, with a new OPTIMIZATION_FAILED(4) return code.
  • Test coverage: per-pass unit tests, deserializer round-trip and rejection tests, optimizer-level pipeline tests, and a CI-gated end-to-end BPMN-equivalence suite using the npm @camunda/element-templates-cli.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated no comments.

Show a summary per file
File Description
element-template-generator/pom.xml Register new optimizer submodule.
element-template-generator/optimizer/pom.xml New module POM: Java 21, picocli + appassembler, slf4j-nop, xmlunit (test).
element-template-generator/optimizer/README.md Documents passes, reorder side effects, CLI usage, and equivalence-test gating.
optimizer/core/Pass.java Pass interface: id, apply, description.
optimizer/core/Optimizer.java Library entrypoint with default pipeline and defaultPipelineExcept that hard-fails on unknown ids.
optimizer/core/PropertyUtils.java Builder-based copy helpers (withId, withCondition, withValue, withoutCondition) with exhaustive sealed switch.
optimizer/core/TemplateLoader.java ObjectMapper that registers ElementTemplateModule + strict deserializers, plus load/save/toString helpers.
optimizer/core/PropertyDeserializer.java Discriminates on type, dispatches to concrete DSL builders, validates feel/optional/generatedValue/choices.
optimizer/core/PropertyConditionDeserializer.java Requires exactly one of equals/oneOf/isActive/allMatch; validates value types.
optimizer/core/PropertyBindingDeserializer.java Dispatches on binding type; rejects missing/empty identifier fields.
optimizer/pass/MergeByIdentityPass.java Groups by binding+value+presentation; merges equals/oneOf on same discriminator; refuses non-string equals; chooses neutral id via longest common suffix with collision disambiguation.
optimizer/pass/TotalizePass.java Drops a top-level equals/oneOf condition when its values cover every choice of its Dropdown discriminator.
optimizer/pass/StrengthReducePass.java Rewrites singleton oneOf as equals.
optimizer/pass/ReorderPass.java Sorts properties by group, then Hidden-first, then id.
optimizer/Main.java picocli bootstrap.
optimizer/command/OptimizerCommand.java CLI: positional input, -o, --dry-run, --skip-passes; distinct exit codes 1/2/3/4; list-passes subcommand.
optimizer/test/.../TestTemplates.java Test DSL helper factories for templates/properties/conditions/bindings.
optimizer/test/.../MergeByIdentityPassTest.java Coverage incl. collision and non-string-equals refusal.
optimizer/test/.../TotalizePassTest.java Coverage for full-coverage drop, partial keep, equals, unknown discriminator.
optimizer/test/.../StrengthReducePassTest.java Singleton vs. multi vs. no condition.
optimizer/test/.../ReorderPassTest.java Group→visibility→id ordering.
optimizer/test/.../core/OptimizerTest.java Default pipeline collapse, skip behavior, unknown-id rejection.
optimizer/test/.../core/TemplateLoaderTest.java Round-trip + strict rejection of malformed JSON.
optimizer/test/.../OptimizerPropertyTest.java CI-gated end-to-end equivalence via element-templates-cli over 36 parametric template shapes.
congen-cli/pom.xml Add element-template-optimizer dependency.
congen-cli/.../command/Generate.java Run Optimizer.defaultPipeline() after generation; map failures to OPTIMIZATION_FAILED.
congen-cli/.../ReturnCodes.java New OPTIMIZATION_FAILED(4).

@vringar vringar changed the title Add a template-optimizer pipeline for REST-connector templates feat: Add a template-optimizer pipeline for REST-connector templates May 28, 2026
@vringar
vringar force-pushed the claude/add-template-optimizer-pipeline branch from b3edb39 to def6ff0 Compare May 28, 2026 14:10
@CLAassistant

CLAassistant commented May 31, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@ztefanie ztefanie left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @vringar

Thanks for your contribution.

Some comments:

  • We want to have this optimizer as a separate module and do not want to apply it to the OOTB connectors for now
  • The optimizer is changing the ids of properties. Can you verify if this breaks migration to higher version of the element templates? In the team call we were assumed the bindings are used for this, but it would be good if you can verify changing ids is safe.
  • Can you please revert the reordering as this breaks conditions. Properties need to be declared before they can be used.

vringar added 3 commits July 2, 2026 15:52
A new sibling to the element-template validator that compacts conditional
properties in Camunda element templates into smaller, semantically-
equivalent forms. REST-based templates typically emit one identical
property per operation gated by a condition; collapsing those is a 30-45%
property-count reduction on real specs without any change in applied
behaviour.

The pipeline runs four passes in order:
  - merge-by-identity: groups properties by everything except condition
    and id, then collapses each group whose members differ only in their
    discriminator value into one property with a oneOf condition over
    the union.
  - totalize: drops conditions that cover every value of their
    discriminator (currently scoped to Dropdown discriminators).
  - strength-reduce: rewrites singleton oneOf as equals.
  - reorder: emits properties in a stable canonical order (group ->
    visibility -> id) so regenerations produce review-friendly diffs.

Public surface:
  - Optimizer.defaultPipeline() / defaultPipelineExcept(List<String>) —
    the library entrypoint for downstream consumers. Unknown skip ids
    are rejected loudly so a typo can't silently disable a pass.
  - Pass interface — id(), apply(template), description().
  - PropertyUtils — produces modified copies of Property objects via
    their public builders. The switch over the sealed Property
    hierarchy is exhaustive at compile time so adding a new subtype to
    core surfaces here as a compile error.
  - TemplateLoader — round-trips templates as JSON. The core DSL is
    sealed without Jackson polymorphism annotations so this module
    ships polymorphism-aware deserializers for Property,
    PropertyBinding, and PropertyCondition that fail loud on missing
    discriminators, conflicting discriminators, unknown enum values,
    and empty required identifiers.

Tests:
  - Per-pass unit tests cover the obvious positive and negative cases
    plus the id-collision and discriminator-value-coercion edge cases.
  - OptimizerTest covers the library entrypoint, including the
    unknown-pass-id validation.
  - TemplateLoaderTest covers the strict-deserializer paths
    (missing-type, empty binding field, multiple discriminators, etc.).
  - OptimizerPropertyTest is a JUnit ParameterizedTest enumerating 36
    multi-operation template shapes and asserting XMLUnit-level BPMN
    equivalence per discriminator value via the npm-published
    element-templates-cli. Gated by @EnabledIf; in CI a missing CLI
    binary is a hard failure so the equivalence gate cannot vanish
    silently behind a green build.
Adds a picocli-based command (element-template-optimizer) that loads a
single template file, runs the Optimizer pipeline over it, and either
rewrites the file in place or writes to the path passed via --output.

Flags:
  --output, -o      path to write the optimized template to (default:
                    rewrite the input file)
  --dry-run         report whether changes would be made, write nothing
  --skip-passes     comma-separated pass ids to leave out of the
                    pipeline. Unknown ids are rejected at parse time.

Subcommands:
  list-passes       prints the pass ids and one-line descriptions

Errors from TemplateLoader.load and Optimizer.optimize are reported as
single-line messages with dedicated exit codes (3, 4) instead of letting
Jackson stacks leak through picocli.

Packaged via appassembler-maven-plugin into target/appassembler/bin/
element-template-optimizer.
congen-cli's generate subcommand now runs every emitted template through
Optimizer.defaultPipeline() before serializing to disk. Generated
artifacts are always shipped in their compacted form; there is no flag.

The optimizer call is wrapped with try/catch and surfaces failures via
the new OPTIMIZATION_FAILED return code with a single-line error
message, matching the existing INPUT_PREPARATION_FAILED /
GENERATION_FAILED / JSON_SCHEMA_VALIDATION_FAILED pattern in this
command rather than letting an unexpected RuntimeException leak out as
a raw stack trace through picocli.
@vringar
vringar force-pushed the claude/add-template-optimizer-pipeline branch from def6ff0 to 904bd86 Compare July 2, 2026 13:56
@vringar
vringar requested a review from a team as a code owner July 2, 2026 13:56
@vringar
vringar requested review from sbuettner and removed request for a team July 2, 2026 13:56
@ztefanie

ztefanie commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

@vringar have you seen my comments above?

Comment 1: "Separate module, don't apply to OOTB connectors"

Half-addressed. The optimizer was correctly extracted into its own Maven module (element-template-generator/optimizer/). But the second half — "do not apply it to OOTB connectors" — was not addressed.
Generate.java:97-101 unconditionally runs Optimizer.defaultPipeline() on every congen generate invocation, which is exactly what generates OOTB connector templates. The README even says explicitly: "There is no
flag — compaction is the default." This directly contradicts the reviewer's instruction.


Comment 2: "Verify changing property IDs doesn't break template migration"

Not addressed at all. There is no mention of migration anywhere in the optimizer module — not in the README, not in tests, not in comments. The reviewer's specific concern (changing merged property IDs and whether
that's safe for version migration) has no response. Given the docs/connector-template-versioning.md file exists in the repo, there's clearly an established versioning story — but this PR makes no reference to it.

Some more comments:

  1. Wrong package namespace — All new source files use io.camunda.connector.optimizer.* instead of io.camunda.connector.generator.optimizer.. Every other module in element-template-generator/ lives under
    io.camunda.connector.generator.
    . This is a structural inconsistency across the entire module (AGENTS.md §1).
  2. xmlunit-core:2.11.0 version hardcoded in optimizer/pom.xml — Not present in any parent BOM. Per AGENTS.md §6, external dependency versions must be governed by in the parent POM.
  3. picocli / picocli-codegen version re-declared in the module — optimizer/pom.xml pins 4.7.7 locally instead of lifting it to the parent element-template-generator BOM. If congen-cli also uses picocli, the two
    will drift (AGENTS.md §6).
  4. ExitCode as a private inner enum — congen-cli uses a top-level ReturnCodes class. The optimizer buries its equivalent as a package-private inner type, making exit codes invisible to callers without opening the
    source.
  5. ListPassesCommand.call() returns bare 0 — Inconsistent with every other return site in the same file; should return ExitCode.SUCCESS.code.

@sbuettner
sbuettner removed their request for review July 9, 2026 09:38
@vringar
vringar marked this pull request as draft July 9, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a template-optimizer pipeline to compact REST-connector element templates

6 participants