Skip to content

feat(dsl): the meter-analyzer-config catalog, and stored-log content search - #117

Merged
wu-sheng merged 2 commits into
mainfrom
test/oap-bump-meter-catalog
Aug 8, 2026
Merged

feat(dsl): the meter-analyzer-config catalog, and stored-log content search#117
wu-sheng merged 2 commits into
mainfrom
test/oap-bump-meter-catalog

Conversation

@wu-sheng

@wu-sheng wu-sheng commented Aug 7, 2026

Copy link
Copy Markdown
Member

Why

Two capabilities the backend has and Horizon did not reach.

Agent-reported meter rules. OAP c59f7b4 (apache/skywalking#13969) brought meter-analyzer-config to parity with otel-rules: its rules load through the same pipeline, hot-update, and attach to a DSL debug session. Horizon's catalog list was three names, hardcoded in the debugger, the router, the sidebar and the BFF's attribution index — so the rule set analysing every agent's self-observability meters, the language runtimes, connection and thread pools was invisible.

Stored-log content search. The BFF already accepted keywordsOfContent and forwarded it to OAP; the Logs tab hardwired it to []. The comment justifying that named the stock H2 store, but the protocol answers the question directly — supportQueryLogsByKeywords, computed by the storage — and Horizon never asked. booster-ui does, and shows the field accordingly.

What

meter-analyzer-config is now a catalog like the others: a MAL · Meter page under DSL management with the same browse / edit / override / inactivate / revert flow, its rules in the live debugger's MAL picker, and Inspect attributing metrics to their rule file as MAL·Meter.

The catalog list is static, so the menu entry is always present. An OAP that doesn't know the catalog answers with its own reason, and the page now shows it — 400 invalid_catalog — catalog must be one of [lal, telegraf-rules, otel-rules, log-mal-rules] instead of a bare "could not load", which is the difference between "my backend is too old" and "the request failed". Everything around it degrades rather than breaking: the debugger's picker keeps offering the three catalogs that resolve, and Inspect's attribution fan-out drops the one that 400s. Verified by running this build against the previous pin.

The Logs tab gains a Content field — space-separated words the line must contain, AND-joined — that appears only on a storage backend that can answer it. On one that can't, the field is absent rather than disabled: OAP accepts the condition there and returns the unfiltered stream, so a search box would answer every query with "everything matched". The capability is read once per OAP and cached with the existing probe; asking is guarded by schema introspection so an older backend never sees an unknown field.

The fixture turns the Java agent's meter sender on (SW_METER_ACTIVE), which is what gives core a live MAL rule to sample. Three rule files in that catalog have a producer there — java-agent, plus HikariCP's datasource and Tomcat's thread_pool.

Validation

Cases run locally against the final tree: core 10/10, es 3/3, deployment 4/4. core and es both share the modified fixture, so both were re-run.

New e2e, each on the deployment that can prove it:

  • core — a MAL capture on meter-analyzer-config/java-agent through to the metric its output step materialised and a service=e2e-service-(provider|consumer) label off the opened sample group; and the Logs tab asserted to have no Content field on BanyanDB.
  • es — the Content field present, a needle no line carries emptying the stream, and clearing it bringing the rows back. A backend ignoring the condition returns the same page and cannot pass this.
  • a readiness check ahead of the browser proving MAL actually ran, so the project never opens a persistence cycle early.

Hand-checked in a browser against both the fixture and the public demo (which runs this exact OAP commit): the menu entry, 11 of 11 loaded rule files on the page and in the picker, a real capture, and honest degradation with the meter stream off — rules still browsable, session still installs, capture ends in "no MAL records from this node".

Preflight: type-check (6 packages), both builds, lint, unit tests (99 + 758 + 440, including 5 new for the capability probe), license 0 invalid, i18n complete across all 8 locales, bundled-template catalogs validated.

Note for review

specs/deployment/mal-debug.spec.ts is kept, with its header corrected — core now covers the same debugger path with value assertions, so whether that presence-only run still earns its place on the slowest stack is worth a second opinion.

…search

OAP c59f7b4 brought `meter-analyzer-config` to parity with otel-rules — its
rules hot-update and attach to a DSL debug session like any other MAL rule set.
Horizon reached it nowhere: the catalog list was three names, hardcoded in the
debugger, the router, the sidebar and the BFF's attribution index.

It is now a catalog like the others. DSL management gains a MAL · Meter page
with the same browse / edit / override / inactivate / revert flow, the live
debugger's MAL tab lists its rules, and Inspect attributes the metrics they
define to their rule file instead of leaving them unattributed. The catalogs
are static, so the menu entry is always there — an OAP that does not know the
catalog answers the page with its own reason ("catalog must be one of …"),
which the error state now shows rather than swallowing behind a bare "could not
load". The rest degrades with it: the debugger's picker keeps offering the
three catalogs that do resolve, and Inspect's fan-out drops the one that 400s.

Stored-log CONTENT search rides along, because it is the same shape of bug: the
BFF already accepted `keywordsOfContent` and forwarded it, the Logs tab hardwired
it empty, and the reason recorded in the comment (the stock H2 store) was not the
one the protocol offers. OAP answers `supportQueryLogsByKeywords` from its
storage — ElasticSearch yes, BanyanDB no — so the capability probe reads it and
the Content field appears only where it can work. On a backend that says no the
field is absent rather than present-and-ignored: OAP accepts the condition there
and returns the unfiltered stream, which reads as "everything matched".

e2e covers both, on the deployment that can prove each. `core` captures a MAL
run against `meter-analyzer-config/java-agent` — the Java agent's own
self-observability meters, which flow because the fixture now leaves the meter
sender on — and asserts the Logs tab has no Content field on BanyanDB. `es`
asserts the field is there and that a needle no line carries empties the stream
and clearing it brings the rows back. A readiness check ahead of the browser
proves MAL actually ran, so the project never opens a persistence cycle early.

Cases run locally against the final tree: core 10/10, es 3/3, deployment 4/4.
@wu-sheng wu-sheng added this to the 1.0.0 milestone Aug 7, 2026
@wu-sheng wu-sheng added the bug Something isn't working label Aug 7, 2026
@wu-sheng
wu-sheng requested a balanced review from Copilot August 7, 2026 15:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Horizon support for OAP’s meter-analyzer-config catalog and storage-aware stored-log content search, aligned with apache/skywalking#13969.

Changes:

  • Integrates agent meter rules across DSL management, live debugging, Inspect, routing, and permissions.
  • Probes OAP storage capability and conditionally exposes log-content filtering.
  • Adds documentation, translations, unit tests, and backend-specific E2E coverage.

Reviewed changes

Copilot reviewed 43 out of 43 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
test/e2e/script/env Pins the supporting OAP commit.
test/e2e/script/docker-compose/base-compose.yml Enables Java agent meter reporting.
test/e2e/playwright/specs/ui/signals.spec.ts Verifies BanyanDB hides content search.
test/e2e/playwright/specs/ui/dsl-debug.spec.ts Tests agent-meter MAL capture.
test/e2e/playwright/specs/es/log-content-search.spec.ts Tests Elasticsearch content filtering.
test/e2e/playwright/specs/deployment/mal-debug.spec.ts Updates collector-path coverage context.
test/e2e/cases/core/expected/has-agent-meter.yml Adds meter-readiness expectation.
test/e2e/cases/core/e2e.yaml Waits for analyzed agent meter data.
README.md Lists MAL Meter attribution support.
packages/api-client/src/types.ts Registers the new catalog.
packages/api-client/src/types.test.ts Verifies catalog membership.
packages/api-client/src/oap-info.ts Adds log-keyword capability typing.
packages/api-client/src/dsl-debugging.ts Enables the catalog for debugging.
packages/api-client/src/dsl-debugging.test.ts Verifies debugging vocabulary.
docs/operate/runtime-rules.md Documents the new runtime-rule catalog.
docs/operate/logs.md Documents conditional content search.
docs/operate/live-debugger.md Documents meter-rule debugging.
docs/operate/inspect.md Documents MAL Meter attribution.
docs/changelog/1.0.0.md Records both operator-facing features.
apps/ui/src/shell/useSidebarMenu.ts Adds the MAL Meter menu entry.
apps/ui/src/shell/useOapInfo.ts Exposes the capability to views.
apps/ui/src/shell/router/index.ts Routes the new catalog page.
apps/ui/src/layer/logs/LayerLogsView.vue Adds capability-gated content input.
apps/ui/src/i18n/locales/zh-CN.json Adds Chinese strings.
apps/ui/src/i18n/locales/pt.json Adds Portuguese strings.
apps/ui/src/i18n/locales/ko.json Adds Korean strings.
apps/ui/src/i18n/locales/ja.json Adds Japanese strings.
apps/ui/src/i18n/locales/fr.json Adds French strings.
apps/ui/src/i18n/locales/es.json Adds Spanish strings.
apps/ui/src/i18n/locales/en.json Adds source English strings.
apps/ui/src/i18n/locales/de.json Adds German strings.
apps/ui/src/features/operate/live-debug/DebugMal.vue Loads meter rules into the MAL picker.
apps/ui/src/features/operate/inspect/inspectTypes.ts Defines the MAL Meter source.
apps/ui/src/features/operate/inspect/InspectCatalogDrawer.vue Displays and filters MAL Meter rules.
apps/ui/src/features/operate/dsl/DslCatalogView.vue Surfaces detailed OAP catalog errors.
apps/ui/src/features/operate/dsl/catalog-labels.ts Adds the catalog display label.
apps/ui/src/features/admin/roles/RolesView.vue Includes the route in role coverage.
apps/ui/src/features/admin/roles/menuVisibility.test.ts Tests menu permission coverage.
apps/ui/src/api/client.ts Extends Inspect response typing.
apps/bff/src/logic/oap/capabilities.ts Probes storage keyword support.
apps/bff/src/logic/oap/capabilities.test.ts Tests capability detection and fallback.
apps/bff/src/logic/inspect/attribution.ts Attributes agent metrics to meter rules.
.github/workflows/e2e.yaml Updates E2E coverage documentation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apps/ui/src/features/operate/dsl/catalog-labels.ts Outdated
…fast

The catalog page headers expanded MAL as "Metrics Analysis Language" on all
five catalogs. Upstream calls it Meter Analysis Language — its own manual
heading, and what this repo's docs already said — so the pages disagreed with
the docs describing them. One name per concept: the labels, the two comments
that quote one, and the i18n key follow the upstream term now.

A keyword-support read that TIMED OUT was cached for the full five-minute
success TTL, so on ElasticSearch one slow reply hid the Content field for five
minutes. A failed read now expires on the short failure TTL the introspection
path already used; a false the storage actually answered stays cached, since
that only changes when OAP restarts.
@wu-sheng
wu-sheng merged commit 537ad5f into main Aug 8, 2026
10 checks passed
@wu-sheng
wu-sheng deleted the test/oap-bump-meter-catalog branch August 8, 2026 00:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants