Apache Iggy Agent Guidelines

Apache Iggy (Incubating) is a persistent message streaming platform in Rust. Thread-per-core shared-nothing, io_uring + compio. Transports: QUIC, WebSocket, TCP (custom binary), HTTP (REST). SDKs: Rust, .NET, Java, Python, Go, C++, Node.js. A connectors subsystem ingests from / egresses to external systems via dlopened plugins.

Skills under .claude/skills/ are currently scoped to the connectors subsystem (core/connectors/). Load connectors-overview first for any change there. Other subsystems follow the repo-wide principles in this file.

Apache Iggy Agent Guidelines

STOP and ask the user before

Bumping iggy_connector_sdk MAJOR version or changing any FFI signature in core/connectors/sdk/src/{sink,source}.rs (breaks every pre-built plugin .so).
Modifying core/binary_protocol/ wire-format types tagged #[repr(C)] (breaks every running client).
Touching the state-save path (core/connectors/runtime/src/state.rs) in a way that changes the atomic-rename protocol (corruption risk).
Renaming default consumer groups, plugin path resolution rules, or any other operator-facing identifier.
Force-pushing to master, amending shipped commits, or skipping hooks with --no-verify / --no-gpg-sign.

Quick reference

# Verification order is enforced by CI (.github/actions/rust/pre-merge).
cargo fmt --all                                            # step 1
cargo sort --no-format --workspace                         # step 2 (--no-format mandatory)
cargo clippy --all-features --all-targets -- -D warnings   # step 3
cargo test -p <crate>                                      # step 4

# Other CI checks (mirror in pre-commit; pass --fix where supported)
./scripts/ci/taplo.sh             # TOML
./scripts/ci/markdownlint.sh      # Markdown
./scripts/ci/shellcheck.sh        # Shell
./scripts/ci/license-headers.sh   # Apache headers
./scripts/ci/trailing-whitespace.sh
./scripts/ci/trailing-newline.sh
./scripts/ci/sync-rust-version.sh
./scripts/ci/python-sdk-version-sync.sh
./scripts/ci/uv-lock-check.sh
./scripts/ci/binary-artifacts.sh --check
./scripts/ci/third-party-licenses.sh --validate --manifest <path>

# Run things
cargo run --bin iggy-server
cargo run --bin iggy -- <subcommand>
IGGY_CONNECTORS_CONFIG_PATH=... cargo run --bin iggy-connectors
./scripts/run-bdd-tests.sh
./scripts/run-benches.sh

Pre-commit hooks wire the same scripts. Install with pre-commit install (Python) or prek install (Rust drop-in) - both read .pre-commit-config.yaml.

Structure

iggy/
├── core/
│   ├── server/           Iggy server binary
│   ├── server-ng/        Next-gen server (Viewstamped Replication, WIP)
│   ├── sdk/              Rust client SDK
│   ├── cli/              iggy CLI
│   ├── connectors/       Connectors runtime + SDK + sinks/sources
│   ├── common/           Shared types (IggyDuration, IggyTimestamp, ...)
│   ├── binary_protocol/  Wire format (stable)
│   ├── configs/          Config plumbing + ConfigEnv derive
│   ├── ai/mcp/           MCP server
│   ├── bench/            Benchmark suite
│   ├── integration/      Cross-crate integration tests
│   ├── simulator/        Deterministic simulator
│   ├── consensus/        Raft (Miri-checked unsafe iobuf)
│   ├── journal/          WAL / journal
│   ├── message_bus/      In-process bus
│   ├── shard/            Shard runtime
│   ├── harness_derive/   #[iggy_harness] proc macro
│   └── metadata, partitions, server_common, configs_derive, clock, tools
├── foreign/              Java / .NET / Python / Go / C++ / Node.js SDKs
├── bdd/                  Cross-language BDD (Gherkin)
├── examples/             Per-SDK runnable examples
├── helm/                 Helm charts
├── web/                  Web UI
└── scripts/              CI + release + dev scripts

Where to look

Task	Location
Wire protocol	`core/binary_protocol/`
Server	`core/server/src/`
Next-gen server (WIP)	`core/server-ng/`
Rust client SDK	`core/sdk/src/`
Connectors	`core/connectors/` -> connector-* skills
Integration tests	`core/integration/tests/`
Cross-SDK BDD	`bdd/`
Benchmark suite	`core/bench/`

Skills

Connectors-scoped. Each SKILL.md has YAML frontmatter (name, description). Load connectors-overview first as router.

connectors-overview - router + universal connector rules
connector-runtime - FFI host, lifecycle, state, metrics
connector-sdk - traits, decoders/encoders, retry
connector-sink - sink plugin authoring
connector-source - source plugin authoring
connector-transform - transform authoring
connector-testing - unit + integration test patterns

Repo-wide principles

Apache 2.0 header on every new source file: Follow the comment style configured in licenserc.toml; common examples: .rs uses // ..., Cargo.toml and shell use # ....
Verification order: fmt --all -> sort --no-format --workspace -> clippy --all-features --all-targets -- -D warnings -> test. Plus taplo.sh --check. CI enforces all.
Always pass --no-format to cargo sort. Without it multi-line feature arrays flatten and taplo re-expands them, churning the workspace.
Avoid LLM-slop tells: em dashes, gratuitous semicolons, hedging narrative, trailing summaries. Defaults, not absolute bans. Same rule for code, comments, commits, PR descriptions.
Idiomatic Rust over free helpers. FromStr not parse_foo, Display not format_foo, From/TryFrom not to_foo_from_bar. Structured Err enums, not String.
Imports at the top. No inline use inside functions or blocks. Group: std, external crates, then crate::.
Precise names. No b, p, t, m. Match literal API field names as log labels.
Comments explain WHY, not WHAT. Default to no comment. Add one only when a hidden constraint or workaround would surprise a future reader. Never reference the current task / PR.
Never cargo install without authorization. Toolchain pinned in rust-toolchain.toml. CI installs additional cargo tools on demand.
No unwrap() / expect() on Results from external I/O outside tests.
tokio::sync::Mutex (not std::sync::Mutex) wherever a lock is held across .await. Hold locks briefly: lock, clone, drop guard, then do I/O.
Code reads top to bottom. Public consts and public fns first, helpers below, deeper helpers below those.
Zero clone in hot loops. &self, &str, &[T]. Vec::with_capacity(n) when size is known.
Forward-compatible config. New TOML fields use #[serde(default)] or Option<T>.

Tooling

Tool	Where
`cargo fmt`, `cargo sort`, `cargo clippy`	pre-commit + CI
`cargo machete`	CI `machete` task
`cargo nextest` + `cargo-rail`	CI `test-*` DAG-scoped
`cargo llvm-cov`	CI coverage
`cargo http-registry`	CI `verify-publish`
Miri	CI `miri` (`binary_protocol` + `consensus`)
`taplo`, `markdownlint-cli`, `shellcheck`	pre-commit + CI
`typos`	pre-commit
`cargo about` + `license-checker-rseidelsohn`	`scripts/ci/third-party-licenses.sh`
HawkEye / `license-headers.sh`	pre-commit + CI
`uv`, `ruff`	pre-commit (Python)
`golangci-lint`	pre-push (Go)

Testing

Unit: #[cfg(test)] mod tests at EOF of src/lib.rs (or per-module).
In-crate integration: <crate>/tests/.
Cross-crate (real infra): core/integration/tests/. Harness in core/integration/src/harness/ spawns real iggy-server (ServerHandle), connectors runtime (ConnectorsRuntimeHandle), MCP server (McpHandle), client (ClientHandle). #[iggy_harness] proc macro wires fixtures. Suites: cli/, cluster/, config_provider/, connectors/, data_integrity/, mcp/, sdk/, server/, state/, storage/. External backends inside connectors/ use testcontainers-modules. Per-test logs land in test_logs/.
Cross-SDK BDD: bdd/. Gherkin features against every SDK. Driver: scripts/run-bdd-tests.sh.

BDD test naming: given_X_when_Y_should_Z (3-part). Promote to 4-part given_X_when_Y_then_Z_should_W only with a distinct "then" intermediate. Be consistent inside a file.

Filter a single integration test by path:

cargo test -p integration -- connectors::runtime::benchmark::given_logging_format_json

Pitfalls

Docker is required for most integration tests. testcontainers-modules spawns containers per test.
cargo sort without --no-format will churn the workspace. Use the pre-commit hook.
Rust toolchain is pinned in rust-toolchain.toml. Older system cargo fails on transitive deps. prefer rustup so the pin auto-resolves.
test_logs/ grows quickly. Wipe between major refactors.
Miri only covers binary_protocol + consensus. Cannot emulate io_uring syscalls. do not try to expand Miri to crates that pull compio.
Integration crate has no --test target. Filter by test path inside the single mod.rs binary.

Local state directories

Created by server, connectors runtime, or tests. Gitignored. Safe to delete to reset:

local_data/ - server data
local_state/ - connectors runtime state files
test_logs/ - per-test logs + local_data snapshots
performance_results/ - benchmark output
target/ - cargo build

Commits, PRs, releases

Conventional commits with scope: feat(connectors): .... Subject <= 72 chars. Allowed types/scopes enforced by .github/workflows/pr-title.yml.
Body: why before what. No bullet lists of files changed.
Co-authored-by trailer when applicable.
Workspace + SDK versions sync via scripts/extract-version.sh and scripts/ci/{sync-rust,python-sdk}-version{,-sync}.sh.
Crate publish chain verified by scripts/verify-crates-publish.sh against cargo-http-registry.
Third-party license enumeration: scripts/ci/third-party-licenses.sh (ASF policy).

Discussion and support

Design: https://github.com/apache/iggy/discussions
Chat: https://discord.gg/apache-iggy
Docs: https://iggy.apache.org/docs/

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apache Iggy Agent Guidelines

Contents

STOP and ask the user before

Quick reference

Structure

Where to look

Skills

Repo-wide principles

Tooling

Testing

Pitfalls

Local state directories

Commits, PRs, releases

Discussion and support

FilesExpand file tree

AGENTS.md

Latest commit

History

AGENTS.md

File metadata and controls

Apache Iggy Agent Guidelines

Contents

STOP and ask the user before

Quick reference

Structure

Where to look

Skills

Repo-wide principles

Tooling

Testing

Pitfalls

Local state directories

Commits, PRs, releases

Discussion and support