|
| 1 | +# Agent API Architecture Overview |
| 2 | + |
| 3 | +The CortexBrain **Agent** is the user-space gRPC server that exposes the data collected by the eBPF kernel programs to clients such as [`cfcli`](../cfcli/overview.md). This page gives an overview of the service surface and the data flow behind it. The full message schema lives in [`core/api/protos/agent.proto`](https://github.com/CortexFlow/CortexBrain/blob/main/core/api/protos/agent.proto) in the repository. |
| 4 | + |
| 5 | +## What the Agent is |
| 6 | + |
| 7 | +The Agent is the `cortexflow_agent_api` crate (`core/api/` in the workspace). It is a [tonic](https://docs.rs/tonic/) gRPC server that: |
| 8 | + |
| 9 | +- Listens on `0.0.0.0:9090` (`core/api/src/main.rs`). |
| 10 | +- Registers the `agent.Agent` service together with a tonic **server reflection** service, so clients can enumerate the available RPCs at runtime - this is what `cfcli monitoring list` uses. |
| 11 | +- On startup, opens a set of **pinned BPF maps** from `/sys/fs/bpf/...` and spawns background reader tasks that drain the perf buffers into mpsc channels. |
| 12 | +- Serves the RPC methods (listed below) that drain those channels and return the data to the caller. |
| 13 | + |
| 14 | +The Agent is deployed as the `cortexflow-agent` Kubernetes service (ClusterIP, port 9090). The CLI reaches it via `kubectl port-forward svc/cortexflow-agent 9090:9090 -n cortexflow`. |
| 15 | + |
| 16 | +## Data flow |
| 17 | + |
| 18 | +The Agent is a **consumer and aggregator** of data produced by metrics service which load the eBPF programs and pin the maps |
| 19 | + |
| 20 | +### The BPF maps the Agent reads |
| 21 | + |
| 22 | +| Map (kernel name) | Pinned path | Producer | Userspace struct | Feeds RPC | |
| 23 | +|-------------------|-------------|----------|------------------|-----------| |
| 24 | +| `EventsMap` | `/sys/fs/bpf/maps/events_map` | Identity (`identity/src/map_handlers.rs`) | `PacketLog` | `ActiveConnections` | |
| 25 | +| `net_metrics` | `/sys/fs/bpf/trace_maps/net_metrics` | Metrics (`metrics/src/maps_handlers.rs`) | `NetworkMetrics` | `GetDroppedPacketsMetrics` | |
| 26 | +| `time_stamp_events` | `/sys/fs/bpf/trace_maps/time_stamp_events` | Metrics (`metrics/src/maps_handlers.rs`) | `TimeStampMetrics` | `GetLatencyMetrics` | |
| 27 | +| `Blocklist` | `/sys/fs/bpf/maps/blocklist_map` | Identity (seeded from the `cortexbrain-client-config` ConfigMap) | `[u8;4] ->[u8;4]` | `AddIpToBlocklist` / `CheckBlocklist` / `RmIpFromBlocklist` | |
| 28 | + |
| 29 | +!!! warning |
| 30 | + The Agent hardcodes the paths `/sys/fs/bpf/maps/...` and `/sys/fs/bpf/trace_maps/...` (`core/api/src/api.rs`), while the Identity and Metrics services pin under `${PIN_MAP_PATH}` which defaults to `/sys/fs/bpf/cortexbrain-identity-service/...`. This implies a deployment-time mount/symlink convention that is not captured in code. If you change the pin paths, update both sides. |
| 31 | + |
| 32 | +## The `agent.Agent` service |
| 33 | + |
| 34 | +The gRPC service is defined in `core/api/protos/agent.proto` (package `agent`, service name `Agent`). It exposes unary RPC methods. All methods are discoverable via server reflection. |
| 35 | + |
| 36 | +| RPC method | Request | Response | Description | |
| 37 | +|------------|---------|----------|-------------| |
| 38 | +| `ActiveConnections` | `RequestActiveConnections { optional string pod_ip }` | `ActiveConnectionResponse { string status; repeated ConnectionEvent events }` | Returns the connection events drained from `events_map`. Each `ConnectionEvent` carries `event_id` (the PID), `src_ip_port`, and `dst_ip_port`. | |
| 39 | +| `AddIpToBlocklist` | `AddIpToBlocklistRequest { optional string ip }` | `BlocklistResponse { string status; map<string,string> events }` | Inserts an IPv4 into the `Blocklist` BPF map. If `ip` is omitted, just returns the current blocklist. | |
| 40 | +| `CheckBlocklist` | `google.protobuf.Empty` | `BlocklistResponse` | Reads the entire `Blocklist` BPF map and returns it as a `map<ip, ip>`. | |
| 41 | +| `RmIpFromBlocklist` | `RmIpFromBlocklistRequest { string ip }` | `RmIpFromBlocklistResponse { string status; map<string,string> events }` | Removes an IPv4 from the `Blocklist` BPF map and returns the remaining entries. | |
| 42 | +| `GetLatencyMetrics` | `google.protobuf.Empty` | `LatencyMetricsResponse { string status; repeated LatencyMetric metrics; uint32 total_count; double average_latency_us; double min_latency_us; double max_latency_us }` | Returns TCP connection latency metrics (`delta_us` per event, plus aggregate stats) drained from `time_stamp_events`. | |
| 43 | +| `GetDroppedPacketsMetrics` | `google.protobuf.Empty` | `DroppedPacketsResponse { string status; repeated DroppedPacketMetric metrics; uint32 total_drops }` | Returns socket-level drop/error metrics (only entries where `sk_drops > 0`) drained from `net_metrics`. | |
| 44 | +| `GetTrackedVeth` | `google.protobuf.Empty` | `VethResponse { string status; repeated string veth_names; int32 tot_monitored_veth }` | Returns tracked veth names from recent events. | |
| 45 | +| `GetTrackedVethFromHashMap` | `google.protobuf.Empty` | `VethHashMapResponse { string status; map<string,string> veths }` | Returns tracked veth entries from the pinned eBPF hash map. | |
| 46 | + |
| 47 | +!!! note "Full message schema" |
| 48 | + The detailed field-by-field schema for every message (`ConnectionEvent`, `LatencyMetric`, `DroppedPacketMetric`, ...) lives in [`core/api/protos/agent.proto`](https://github.com/CortexFlow/CortexBrain/blob/main/core/api/protos/agent.proto). The [Integrated Metrics](metrics.md) page also describes the metric fields in tabular form. The [CLI Agent API page](../cfcli/Agent-API-Overview.md) has grpcurl examples. |
| 49 | + |
| 50 | +## How `cfcli` calls the Agent |
| 51 | + |
| 52 | +The CLI (`cli/`) is a tonic gRPC client. The mapping between `cfcli` commands and the Agent RPCs: |
| 53 | + |
| 54 | +| `cfcli` command | Agent RPC | |
| 55 | +|-----------------|-----------| |
| 56 | +| `cfcli monitoring list` | (server reflection - lists all `Agent` methods) | |
| 57 | +| `cfcli monitoring connections` | `ActiveConnections` | |
| 58 | +| `cfcli monitoring latencymetrics` | `GetLatencyMetrics` | |
| 59 | +| `cfcli monitoring droppedpackets` | `GetDroppedPacketsMetrics` | |
| 60 | +| `cfcli policy create-blocklist --flags <IP>` | `AddIpToBlocklist` | |
| 61 | +| `cfcli policy check-blocklist` | `CheckBlocklist` | |
| 62 | +| `cfcli policy remove-ip --flags <IP>` | `RmIpFromBlocklist` | |
| 63 | + |
| 64 | +## Relationship to the Identity service |
| 65 | + |
| 66 | +The **Identity** service (`core/src/components/identity/`) is the **producer** of the connection events and the blocklist. It: |
| 67 | + |
| 68 | +1. Loads the `conntracker` eBPF object. |
| 69 | +2. Extracts and pins `EventsMap` (->`events_map`), `Blocklist` (->`blocklist_map`), and other maps. |
| 70 | +3. Seeds the `Blocklist` map from the `cortexbrain-client-config` Kubernetes ConfigMap on startup. |
| 71 | +4. Attaches the TC classifier and kprobes that fill the perf buffers. |
| 72 | + |
| 73 | +The **Agent** never calls Identity directly - they communicate solely through the pinned BPF maps in `/sys/fs/bpf/`. The Agent is also a **mutator** of the `Blocklist` map that Identity owns: `AddIpToBlocklist` and `RmIpFromBlocklist` write to it at runtime, on top of the initial ConfigMap seeding. |
| 74 | + |
| 75 | +## Security notes |
| 76 | + |
| 77 | +The gRPC channel is currently **plaintext** HTTP/2: |
| 78 | + |
| 79 | +- The server binds to `0.0.0.0:9090` (`core/api/src/main.rs`, flagged with a `FIXME`). |
| 80 | +- The client connects to `http://127.0.0.1:9090` (`core/api/src/client.rs`, also flagged with a `FIXME`). |
0 commit comments