Skip to content

[Fleet incident] Restore the dedicated publication identity for hourly product development #227

Description

@seonghobae

Protected-main operational evidence

The latest protected-main Hourly NVIDIA NIM Product Development run 31596953227 at commit 18cac8dcab31dd53048d5d14ad01673b5b77d141 completed success, but every proposal, verification, and publication step was skipped.

The zero-PR gate observed:

  • zero open pull requests;
  • NIM_CONFIGURED=true;
  • MAINTAINER_APP_CLIENT_ID_CONFIGURED=false;
  • MAINTAINER_APP_PRIVATE_KEY_CONFIGURED=false;
  • terminal reason maintainer_app_unavailable.

This is a stable protected-main configuration blocker, not a model, queue, or repository-test failure.

Root-cause and identity boundary

Noema needs two different GitHub identities:

  1. the review-only Noema App that submits independent pull-request reviews; and
  2. a publication/maintainer identity that can push a verified proposal branch and open a Draft PR.

Reusing the review App for branch writes would destroy the independence required by branch policy and by the central reviewer architecture. Giving the model or verifier a write token is also prohibited. Therefore the publication capability is required-with-justification, but it must be supplied through a separately scoped identity rather than an invented PAT.

Feasible options

A. Configure the documented repository-scoped Maintainer GitHub App

Provide the existing workflow contract values:

  • variable NOEMA_MAINTAINER_APP_CLIENT_ID;
  • secret NOEMA_MAINTAINER_APP_PRIVATE_KEY.

The App installation must be limited to this repository with only metadata read, contents write, and pull-request write. It must have no review, merge, release, deployment, administration, secret, or organization-wide repository authority.

B. Migrate publication to a separate Maintainer App OIDC exchange

Prefer a short-lived repository-scoped installation token when a separately owned publication App and trusted exchange already exist. Bind repository, workflow ref/SHA, event, audience, expiry, and exact permission set. Do not reuse the cwl-noema-review OIDC token or depend on the unresolved review-envelope path for branch publication.

C. Same-repository GITHUB_TOKEN only with proven downstream-trigger architecture

Acceptable only if a regression proves that a Draft PR created under GITHUB_TOKEN receives the normal required exact-head CI, security, and independent-review workflows despite GitHub recursive-event suppression. A bare token substitution is not sufficient.

Rejected non-remedies

  • a PAT or renamed secret;
  • COPILOT_GITHUB_TOKEN;
  • granting write authority to the model/verifier;
  • reusing the review App for code publication;
  • weakening branch protection or independent-review requirements;
  • reporting a perpetual maintainer_app_unavailable no-op as operational product development;
  • a temporary self-modifying installer workflow.

Acceptance criteria

  1. select and document a currently feasible publication architecture and rollback;
  2. keep reviewer and maintainer identities cryptographically and operationally separate;
  3. test repository/permission/workflow binding and downstream CI triggering;
  4. exact-head CI/security/review evidence and normal protected-main integration;
  5. a real protected-main zero-PR run passes the current gate, executes NVIDIA/OpenCode, independently verifies the patch, and either records a verified no-mutation result or opens exactly one Draft PR whose normal exact-head workflows actually start;
  6. no publication token reaches the model or verifier;
  7. no PAT, COPILOT_GITHUB_TOKEN, broad secrets: inherit, or policy bypass.

This issue is part of the organization-wide Hourly Product Development fleet incident. Noema remains read-only to the fleet audit; its dedicated owner loop controls implementation. A real external Maintainer App installation/permission configuration is required unless an independently proven OIDC or downstream-trigger design removes the stored-key prerequisite.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions