Support queueing-service through an opt-in gRPC event sink

[MarcusSorealheis]

What and why

Add opt-in grpc.store_type: "event_sink" for BEP/origin forwarding to queueing-service. String-key digests currently advertise the key length, so the ByteStream size validator rejects event payloads with a different length. Event sinks use the exact payload length and retain the key hash as the identifier.

Reject acknowledgements that report a different committed size. Document the receiver's durable-append acknowledgement and stable event-key duplicate handling, including retries after a lost acknowledgement.

How was this verified?

On the refreshed branch, Bazel 9.1.1 passed all 13 gRPC-store tests and 11 configuration tests with Clippy and rustfmt enabled. Coverage includes payload/key length differences; rejection of reads, existence checks, digest keys, and unknown upload sizes; incomplete acknowledgements; and replay with the same upload identity, offset, and payload after a lost acknowledgement.

Regenerated the configuration reference and passed snippet, anchor, spelling, prose, and repository hooks. The local generated hook config uses an unsupported language label; the same hook commands passed using a temporary config that maps local executables to system.

Earlier integration validation sent actual BES requests through the built NativeLink server, the Go queueing-service receiver, a Kafka-compatible broker, and MongoDB. A simulated crash after projection left the broker offset uncommitted; restart replayed the event and persisted the terminal build state. That multi-service integration was not rerun for the acknowledgement follow-up. CI passed on commit 8565881, including Linux and Windows Cargo tests, both Bazel versions, ASAN, coverage, pre-commit, documentation checks, and integration tests.

Risk

Explicit opt-in; existing CAS/AC configuration keeps its behavior. This protocol requires a dedicated receiver that acknowledges the full payload only after a durable append and does not implement CAS content verification. The byte-count check cannot independently prove receiver durability. A receiver must handle duplicate events using the stable event key, including retries that use a new upload UUID. This does not provide exactly-once processing or a durable publisher recovery ledger, and it retains the origin publisher's bounded in-memory retry policy.

AI assistance

I had help from several OpenAI models with the implementation, tests, and documentation. I directed the scope, and the coding agent ran the local checks described above. The coding agent also reviewed the final diff and posted its findings on this PR.

---

This change is 

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
nativelink	Ready	Preview	Sep 30, 2026 6:42pm UTC
nativelink-aidm	Ready	Preview	Sep 30, 2026 6:42pm UTC

[MarcusSorealheis]

Completed an AI-assisted author-side review of the final diff at 8565881. No blocking findings.

Reviewed the event-sink configuration, StoreDriver operations, ByteStream upload/retry path, and the BEP/origin publisher call sites:

• Event-sink writes require a string key and an exact payload size. The resource retains the key hash while advertising the payload length; digest-key writes, reads, existence checks, and read batching are rejected on the event-publisher path.
• String-key events cannot enter the CAS compression path. Existing CAS/AC behavior remains unchanged by the new acknowledgement check.
• A negative, partial, or oversized committed byte count is rejected. BEP acknowledges its client only after the store update succeeds. A receiver still must make its own durable append before acknowledging; the byte count alone does not prove durability.
• The lost-ack regression verifies transport replay keeps the upload identity, offset, and payload. The BEP/origin publisher call sites also retain the event key across outer retries, which may create a new upload UUID. Receiver duplicate handling must therefore use the stable event key. Origin retries remain bounded in memory, with no new durable recovery ledger or exactly-once guarantee.

Validation: all 13 gRPC-store and 11 configuration tests passed locally with Bazel's Clippy/rustfmt checks, and documentation/hooks passed. Current CI is green, including Linux/Windows Cargo, both Bazel versions, ASAN, coverage, and integration tests. Both PR commits have valid GitHub signatures. The previously described multi-service receiver/broker/database experiment was not repeated for the acknowledgement follow-up.

The change is ready to leave draft. This is an author-side review, since the implementation and review were performed through the same account.