Skip to content

security(kpi): constrain production log URL transport before collection #162

Description

@seonghobae

Fresh protected boundary

Protected main is c85d710804139c0697d7ef8fa47d02b1389e6d84. scripts/collect-kpi-logs.sh blob cd168d25a109ff8bd40ade126ac9641cfa675c73 currently executes:

curl -sS "${NOEMA_KPI_LOG_URL}" -o "${TARGET_FILE}"

without a protocol allowlist or HTTP failure semantics. NOEMA_KPI_LOG_URL is an operator input for production evidence collection, while the resulting bytes are retained and later treated as the candidate KPI evidence stream.

Fresh writer evidence: PR #105 is the active owner of scripts/collect-kpi-logs.sh; its live head is 19b49cb6cc2e072eafa37fae2170d7d1f2ccdfa3, not the stale head recorded in its body, and its changed-file set includes this collector. Do not start a parallel source writer while #105 is moving.

RCA

First failing boundary: operator-provided production-log locator -> curl transport -> retained KPI bytes.

Immediate cause: curl receives an unrestricted URL and does not use --fail/--fail-with-body or an explicit protocol policy.

Root cause: the collector treats “curl produced bytes” as transport success without declaring which transport schemes and HTTP outcomes are allowed for production evidence acquisition.

Security/reliability consequence: unsupported/local schemes accepted by curl or an HTTP error response can be written into the retained KPI path before later semantic gates reject it. That broadens file/network authority and weakens provenance clarity at the collection boundary. This issue does not claim such bytes can already satisfy the strict KPI final gate; it requires the collector itself to fail closed at its own authority boundary.

Materially distinct remedies / feasibility

  1. Constrain URL collection to reviewed HTTPS transport, reject unsupported schemes before curl, and require HTTP transport success — selected bounded direction after the active fix(kpi): bind strict provenance to exact log bytes #105 lineage settles. Use curl protocol controls as defense in depth rather than string-prefix checks alone.
  2. Allow HTTPS plus loopback HTTP — useful for local tests but inappropriate as a production evidence source contract unless tests inject a dedicated test-only path; do not broaden production semantics merely for fixtures.
  3. Remove URL collection and require only a separately authenticated archive/file handoff — stronger provenance separation but materially changes operator workflow and is larger than the current defect.
  4. Keep unrestricted curl and rely on downstream NDJSON/KPI validation — rejected because downstream semantic rejection does not justify unnecessary collector transport authority or HTTP-error acceptance.

Signed/presigned HTTPS URLs may legitimately contain query state. The repair must not log or retain the source URL/query and must not invent a generic “no query string” rule without an operator requirement.

Safe implementation order

  1. Let fix(kpi): bind strict provenance to exact log bytes #105 integrate, settle, or otherwise release ownership of scripts/collect-kpi-logs.sh.
  2. Refetch protected main, exact collector blob, open PR changed paths and writer evidence.
  3. Add deterministic test-first regressions proving unsupported protocols and HTTP error responses fail before successful provenance retention; tests must not use real credentials.
  4. Implement the smallest protocol/HTTP-status restriction while preserving successful HTTPS collection and fix(kpi): bind strict provenance to exact log bytes #105 exact-byte provenance semantics.
  5. Re-run full exact-head CI, configured 100% owned production statement/branch/function/line coverage, reviewer-ci and an eligible protected-base central Security Scan; keep formal approval separate.
  6. Perform protected-main operator acceptance after integration using non-secret test endpoints, and keep real 30-day production provenance under Noema 판매 가능 목표: 30일 운영 KPI 로그 및 provenance 확보 #3 separate.

Guardrails

Do not race #105, persist/log NOEMA_KPI_LOG_URL, add credential fallbacks, relax strict KPI provenance, claim upstream provider authentication, or promote synthetic transport tests to production KPI/acquisition evidence.

Related: #3, #27, #29, #104, #105, #110

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions