Implement Portal Infrastructure in ImixV2#1471
Merged
Conversation
- Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers.
Contributor
Author
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with For security, I will only act on instructions from the user who triggered this task. New to Jules? Learn more at jules.google/docs. |
- Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. - Updated `run_create_portal` to send initial registration message.
hulto
pushed a commit
that referenced
this pull request
Jan 3, 2026
* Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. - Updated `run_create_portal` to send initial registration message. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jan 3, 2026
* initial stubbing and renaming * feat: implement tavern/portals/stream package (#1462) This commit introduces a new package `stream` in `tavern/portals` which provides utilities for handling ordered streams of `portalpb.Mote` messages. Key features: - `payloadSequencer`: Handles atomic sequence ID generation and mote creation. - `OrderedWriter`: Wraps a sender function (like a gRPC stream Send) to automatically sequence and write messages. - `OrderedReader`: Wraps a receiver function (like a gRPC stream Recv) to reorder incoming messages, handling out-of-order delivery with configurable buffering and stale stream detection. This package is designed to support both client and server sides of the portal stream. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * refactor reader to use functional API * Implement PubSub Multiplexer (Mux) (#1466) * Implement PubSub Multiplexer (Mux) for Portals - Created `Mux` package in `tavern/internal/portals/mux`. - Implemented dual-mode operation: In-Memory (for dev) and GCP PubSub (for prod). - Implemented `CreatePortal` and `OpenPortal` lifecycle methods with resource provisioning. - Implemented `Publish` and `Subscribe` logic with local broadcasting (fast path) and global PubSub (slow path). - Added `HistoryBuffer` for message replay. - Added intelligent topic caching to handle `mempubsub` quirks and improve performance. - Added Prometheus metrics for observability. - Verified with comprehensive unit tests using `enttest` and `mempubsub`. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used `sync.RWMutex` for thread safety (`activeSubs`, `subscribers`, `histMu`). - **Observability:** Added Prometheus metrics for message counts. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, and Portal opening. * Refactor Mux and Address PR Feedback - **Structure:** Refactored `Mux` to use composed structs (`SubscriptionManager`, `SubscriberRegistry`, `TopicManager`, `HistoryManager`) for clearer locking semantics. - **Safety:** - Handled race conditions in `CreatePortal` and `OpenPortal` by re-checking state after lock acquisition. - Updated `teardown` logic to use `client.Portal.UpdateOneID` for reliability. - Standardized on `context.Background()` for shutdown operations to prevent context leaks. - **Features:** - Added `WithSubscriberBufferSize` to configure channel buffers. - Added `WithHistoryReplay` option to `Subscribe` for optional history. - Added `mux_messages_dropped_total` metric. - **Concurrency:** Moved global lock handling into granular manager structs to reduce contention. - **Correctness:** Fixed `CreatePortal` to use task-based lookup for dependencies and removed invalid `portalID` parameter usage in logic (though kept signature for now as per instructions). Tests passed. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used composed structs (`SubscriptionManager`, `SubscriberRegistry`) for granular locking. - **Observability:** Added Prometheus metrics for message counts and dropped messages. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, Portal opening, and Benchmarks. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * both grpc endpoints * stub out portal-stream package * Implement portal-stream crate (#1470) * Implement portal-stream crate with sequencer, reader, and writer logic - Added `implants/lib/portals/portal-stream` crate. - Implemented `PayloadSequencer` for atomic sequence ID generation. - Implemented `OrderedReader` for reordering incoming messages with timeout and buffer handling. - Implemented `OrderedWriter` for sequencing outgoing messages. - Added comprehensive unit tests for all components. - Added crate to `implants` workspace. * Switch to anyhow for error handling in portal-stream - Replaced `thiserror` with `anyhow` in `portal-stream`. - Updated `Cargo.toml` to use `anyhow` from workspace. - Updated `reader.rs` and tests to use `anyhow::Result` and `anyhow!`. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement SOCKS5 Proxy using tavern/portals/stream (#1467) * Implement SOCKS5 proxy with gRPC tunneling - Added bin/socks5/proxy.go implementing a SOCKS5 proxy server. - Implemented tunneling over stream.OrderedWriter/Reader. - Supported TCP CONNECT and UDP ASSOCIATE commands. - Implemented robust lifecycle management and cleanup. - Added benchmarks in bin/socks5/proxy_test.go demonstrating high throughput. * Address PR comments: Refactor writes, defaults, and shutdown tracking - Refactored raw `conn.Write` calls into named helper functions. - Changed default upstream port to 8000. - Added `sync.WaitGroup` to track connection lifecycle. - Added logging for dropped motes in dispatcher. - Defined `maxStreamBufferedMessages` constant. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement Portal Infrastructure in ImixV2 (#1471) * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. - Updated `run_create_portal` to send initial registration message. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * some changes * Add integration test * Added TRACE Bytes kind * added trace protos * Implement Trace Motes for Portals (#1473) * feat: implement end-to-end trace motes Implements application-level tracing for Portals infrastructure using the new `tracepb` definitions. * **CLI (`bin/socks5`)**: * Added `trace` subcommand to generate trace motes, send them to the server, and print a latency report. * Refactored `proxy.go` to support subcommands. * Added `addTraceEvent` helper for modifying trace motes. * **Server (`tavern`)**: * Instrumented `api_open_portal.go` and `api_create_portal.go` to inject trace events at key checkpoints (Recv, Pub, Sub, Send). * Created `trace_helper.go` to share event injection logic. * **Agent (`imixv2`)**: * Updated `run.rs` to intercept `BYTES_PAYLOAD_KIND_TRACE` motes. * Implemented logic to add `AGENT_RECV` and `AGENT_SEND` events and immediately echo the mote back. * added retry to trace --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * update flag to --portal * update buf size * Update portal API to send keepalive motes (#1472) - Added keepalive ticker to sendPortalInput loop in api_create_portal.go - Sends a BYTES_PAYLOAD_KIND_KEEPALIVE mote at regular intervals - Prevents connection timeouts similar to the reverse shell implementation Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * add tokio-console support for imixv2 * Fix SOCKS5 proxy cold start hang by avoiding BiLock on TcpStream (#1476) Replaced `tokio::io::split(stream)` with `stream.into_split()` in `implants/imixv2/src/portal/tcp.rs`. The former uses a `BiLock` which can cause deadlocks when the read and write halves are accessed concurrently in separate tasks, specifically causing the "cold start" hang where the initial payload might be blocked. `into_split()` returns owned halves that operate independently. Added a regression test `implants/imixv2/src/tests/repro_issue.rs` to verify the fix. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * socks proxy sends registration message now * socks5 trace must send registration message * Add E2E Portals Workflow and Playwright Test (#1478) Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * cargo fmt * remove spammy log line * fix some tests * add create_portal fake impl * revert some changes to grpc.rs * remove grpc explicit sizes * update socks5 proxy to support auth and portals to support gcp pubsub * oops * minor cleanup * fix tests * Fix e2e workflow * use env var for auth * wait for socks to start before continuing * Add benchmark tests for cryptocodec (#1485) Added a new test file `tavern/internal/cryptocodec/cryptocodec_bench_test.go` to measure the throughput of `Encrypt` and `Decrypt` methods in `CryptoSvc`. Includes benchmarks for both encryption and decryption operations using standard payload sizes. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Fix race condition in TestPortalIntegration (#1487) The `TestPortalIntegration` test was flaky and often hung because the Agent would publish messages to the portal before the User had successfully subscribed to the output topic. This resulted in messages being dropped (as evidenced by "message sent to topic with no subscribers" warnings from the in-memory pubsub) and the User reader blocking indefinitely. This commit replaces the arbitrary `time.Sleep` with a deterministic synchronization mechanism. The User now sends a "ping" message to the Agent immediately after opening the portal. The Agent waits to receive this ping before proceeding. This ensures that the User's portal connection (and thus the underlying pubsub subscription) is fully established before the Agent attempts to send any data. This reduces test execution time and eliminates the race condition. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * fix proxy upstream address parsing * minor fixes & cleanup * fix collapsible if * cargo fmt * fix portal workflow * Add Tavern User Guide for Authentication and API Token (#1501) * Add documentation for TAVERN_API_TOKEN and portal auth flow This commit adds a new user guide page for Tavern (`docs/_docs/user-guide/tavern.md`) detailing the purpose of `TAVERN_API_TOKEN`. It clarifies the distinction between this token and the web OAuth token and explains the "portal auth flow" for users SSH'd into remote environments (e.g., Kali VMs) where standard auth port forwarding is not feasible. * Update TAVERN_API_TOKEN docs: remove non-existent Portal Auth Flow Per code review feedback, the "Portal Auth Flow" feature does not exist yet. This commit removes that section from the documentation, leaving the explanation of what the token is and when to use it (SSH scenarios). --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * cleanup * Merge origin/main, resolve conflicts, and fix go generate (#1515) - Merged changes from origin/main, resolving conflicts in app.go, server.go, and mock transport. - Updated `golang.org/x/tools` to fix `ent` generation failure ("context without types"). - Re-ran `go generate ./...` to update generated protobuf and ent code. - Fixed compilation errors in tests due to renamed protobuf enum constants (ActiveTransport_TRANSPORT_HTTP1). Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Add offline filters (#1505) * feat(online-offline): Online Offline in progress * fix(online-offline): Change text * fix(query): Simplify query * fix(build): Build UI * fix(create-quest): Hide filter from create quest * Add Link entity and update CDN to use link-based file access (#1444) * Add Link entity and update CDN to use link-based file access This commit implements a new Link entity system for the CDN: - Add Link entity schema with: - path (unique, default UUIDv4) - active_before (timestamp, default epoch 0) - active_clicks (int, default 0) - edge to File entity (one-to-many relationship) - Update CDN upload handler to: - Create a new Link entity for each uploaded file - Return both file ID and link path in response - Add new CDN link download handler to: - Serve files using Link entity path instead of file name - Check active_clicks and active_before before serving - Decrement active_clicks when file is served - Return 404 if link is not active - Replace CDN route to use link-based downloads: - Changes /cdn/ endpoint from file name access to link path access - Removes ability to serve tome assets via direct file name - Maintains FetchAsset gRPC API for agent communication - Update dependencies: - Upgrade entgo.io/ent from v0.14.1 to v0.14.5 - Update related dependencies (atlas, sqlite3, tools) * go generate * Add mutations * Clarify docs * typeable random short string * Don't create link on upload * Resolve feedback --------- Co-authored-by: Claude <noreply@anthropic.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Squidli <32434695+cmp5987@users.noreply.github.com> Co-authored-by: Hulto <7121375+hulto@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>
nullmonk
pushed a commit
that referenced
this pull request
Jan 7, 2026
* initial stubbing and renaming * feat: implement tavern/portals/stream package (#1462) This commit introduces a new package `stream` in `tavern/portals` which provides utilities for handling ordered streams of `portalpb.Mote` messages. Key features: - `payloadSequencer`: Handles atomic sequence ID generation and mote creation. - `OrderedWriter`: Wraps a sender function (like a gRPC stream Send) to automatically sequence and write messages. - `OrderedReader`: Wraps a receiver function (like a gRPC stream Recv) to reorder incoming messages, handling out-of-order delivery with configurable buffering and stale stream detection. This package is designed to support both client and server sides of the portal stream. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * refactor reader to use functional API * Implement PubSub Multiplexer (Mux) (#1466) * Implement PubSub Multiplexer (Mux) for Portals - Created `Mux` package in `tavern/internal/portals/mux`. - Implemented dual-mode operation: In-Memory (for dev) and GCP PubSub (for prod). - Implemented `CreatePortal` and `OpenPortal` lifecycle methods with resource provisioning. - Implemented `Publish` and `Subscribe` logic with local broadcasting (fast path) and global PubSub (slow path). - Added `HistoryBuffer` for message replay. - Added intelligent topic caching to handle `mempubsub` quirks and improve performance. - Added Prometheus metrics for observability. - Verified with comprehensive unit tests using `enttest` and `mempubsub`. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used `sync.RWMutex` for thread safety (`activeSubs`, `subscribers`, `histMu`). - **Observability:** Added Prometheus metrics for message counts. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, and Portal opening. * Refactor Mux and Address PR Feedback - **Structure:** Refactored `Mux` to use composed structs (`SubscriptionManager`, `SubscriberRegistry`, `TopicManager`, `HistoryManager`) for clearer locking semantics. - **Safety:** - Handled race conditions in `CreatePortal` and `OpenPortal` by re-checking state after lock acquisition. - Updated `teardown` logic to use `client.Portal.UpdateOneID` for reliability. - Standardized on `context.Background()` for shutdown operations to prevent context leaks. - **Features:** - Added `WithSubscriberBufferSize` to configure channel buffers. - Added `WithHistoryReplay` option to `Subscribe` for optional history. - Added `mux_messages_dropped_total` metric. - **Concurrency:** Moved global lock handling into granular manager structs to reduce contention. - **Correctness:** Fixed `CreatePortal` to use task-based lookup for dependencies and removed invalid `portalID` parameter usage in logic (though kept signature for now as per instructions). Tests passed. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used composed structs (`SubscriptionManager`, `SubscriberRegistry`) for granular locking. - **Observability:** Added Prometheus metrics for message counts and dropped messages. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, Portal opening, and Benchmarks. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * both grpc endpoints * stub out portal-stream package * Implement portal-stream crate (#1470) * Implement portal-stream crate with sequencer, reader, and writer logic - Added `implants/lib/portals/portal-stream` crate. - Implemented `PayloadSequencer` for atomic sequence ID generation. - Implemented `OrderedReader` for reordering incoming messages with timeout and buffer handling. - Implemented `OrderedWriter` for sequencing outgoing messages. - Added comprehensive unit tests for all components. - Added crate to `implants` workspace. * Switch to anyhow for error handling in portal-stream - Replaced `thiserror` with `anyhow` in `portal-stream`. - Updated `Cargo.toml` to use `anyhow` from workspace. - Updated `reader.rs` and tests to use `anyhow::Result` and `anyhow!`. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement SOCKS5 Proxy using tavern/portals/stream (#1467) * Implement SOCKS5 proxy with gRPC tunneling - Added bin/socks5/proxy.go implementing a SOCKS5 proxy server. - Implemented tunneling over stream.OrderedWriter/Reader. - Supported TCP CONNECT and UDP ASSOCIATE commands. - Implemented robust lifecycle management and cleanup. - Added benchmarks in bin/socks5/proxy_test.go demonstrating high throughput. * Address PR comments: Refactor writes, defaults, and shutdown tracking - Refactored raw `conn.Write` calls into named helper functions. - Changed default upstream port to 8000. - Added `sync.WaitGroup` to track connection lifecycle. - Added logging for dropped motes in dispatcher. - Defined `maxStreamBufferedMessages` constant. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement Portal Infrastructure in ImixV2 (#1471) * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. - Updated `run_create_portal` to send initial registration message. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * some changes * Add integration test * Added TRACE Bytes kind * added trace protos * Implement Trace Motes for Portals (#1473) * feat: implement end-to-end trace motes Implements application-level tracing for Portals infrastructure using the new `tracepb` definitions. * **CLI (`bin/socks5`)**: * Added `trace` subcommand to generate trace motes, send them to the server, and print a latency report. * Refactored `proxy.go` to support subcommands. * Added `addTraceEvent` helper for modifying trace motes. * **Server (`tavern`)**: * Instrumented `api_open_portal.go` and `api_create_portal.go` to inject trace events at key checkpoints (Recv, Pub, Sub, Send). * Created `trace_helper.go` to share event injection logic. * **Agent (`imixv2`)**: * Updated `run.rs` to intercept `BYTES_PAYLOAD_KIND_TRACE` motes. * Implemented logic to add `AGENT_RECV` and `AGENT_SEND` events and immediately echo the mote back. * added retry to trace --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * update flag to --portal * update buf size * Update portal API to send keepalive motes (#1472) - Added keepalive ticker to sendPortalInput loop in api_create_portal.go - Sends a BYTES_PAYLOAD_KIND_KEEPALIVE mote at regular intervals - Prevents connection timeouts similar to the reverse shell implementation Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * add tokio-console support for imixv2 * Fix SOCKS5 proxy cold start hang by avoiding BiLock on TcpStream (#1476) Replaced `tokio::io::split(stream)` with `stream.into_split()` in `implants/imixv2/src/portal/tcp.rs`. The former uses a `BiLock` which can cause deadlocks when the read and write halves are accessed concurrently in separate tasks, specifically causing the "cold start" hang where the initial payload might be blocked. `into_split()` returns owned halves that operate independently. Added a regression test `implants/imixv2/src/tests/repro_issue.rs` to verify the fix. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * socks proxy sends registration message now * socks5 trace must send registration message * Add E2E Portals Workflow and Playwright Test (#1478) Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * cargo fmt * remove spammy log line * fix some tests * add create_portal fake impl * revert some changes to grpc.rs * remove grpc explicit sizes * update socks5 proxy to support auth and portals to support gcp pubsub * oops * minor cleanup * fix tests * Fix e2e workflow * use env var for auth * wait for socks to start before continuing * Add benchmark tests for cryptocodec (#1485) Added a new test file `tavern/internal/cryptocodec/cryptocodec_bench_test.go` to measure the throughput of `Encrypt` and `Decrypt` methods in `CryptoSvc`. Includes benchmarks for both encryption and decryption operations using standard payload sizes. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Fix race condition in TestPortalIntegration (#1487) The `TestPortalIntegration` test was flaky and often hung because the Agent would publish messages to the portal before the User had successfully subscribed to the output topic. This resulted in messages being dropped (as evidenced by "message sent to topic with no subscribers" warnings from the in-memory pubsub) and the User reader blocking indefinitely. This commit replaces the arbitrary `time.Sleep` with a deterministic synchronization mechanism. The User now sends a "ping" message to the Agent immediately after opening the portal. The Agent waits to receive this ping before proceeding. This ensures that the User's portal connection (and thus the underlying pubsub subscription) is fully established before the Agent attempts to send any data. This reduces test execution time and eliminates the race condition. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * fix proxy upstream address parsing * minor fixes & cleanup * fix collapsible if * cargo fmt * fix portal workflow * Add Tavern User Guide for Authentication and API Token (#1501) * Add documentation for TAVERN_API_TOKEN and portal auth flow This commit adds a new user guide page for Tavern (`docs/_docs/user-guide/tavern.md`) detailing the purpose of `TAVERN_API_TOKEN`. It clarifies the distinction between this token and the web OAuth token and explains the "portal auth flow" for users SSH'd into remote environments (e.g., Kali VMs) where standard auth port forwarding is not feasible. * Update TAVERN_API_TOKEN docs: remove non-existent Portal Auth Flow Per code review feedback, the "Portal Auth Flow" feature does not exist yet. This commit removes that section from the documentation, leaving the explanation of what the token is and when to use it (SSH scenarios). --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * cleanup * Merge origin/main, resolve conflicts, and fix go generate (#1515) - Merged changes from origin/main, resolving conflicts in app.go, server.go, and mock transport. - Updated `golang.org/x/tools` to fix `ent` generation failure ("context without types"). - Re-ran `go generate ./...` to update generated protobuf and ent code. - Fixed compilation errors in tests due to renamed protobuf enum constants (ActiveTransport_TRANSPORT_HTTP1). Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Add offline filters (#1505) * feat(online-offline): Online Offline in progress * fix(online-offline): Change text * fix(query): Simplify query * fix(build): Build UI * fix(create-quest): Hide filter from create quest * Add Link entity and update CDN to use link-based file access (#1444) * Add Link entity and update CDN to use link-based file access This commit implements a new Link entity system for the CDN: - Add Link entity schema with: - path (unique, default UUIDv4) - active_before (timestamp, default epoch 0) - active_clicks (int, default 0) - edge to File entity (one-to-many relationship) - Update CDN upload handler to: - Create a new Link entity for each uploaded file - Return both file ID and link path in response - Add new CDN link download handler to: - Serve files using Link entity path instead of file name - Check active_clicks and active_before before serving - Decrement active_clicks when file is served - Return 404 if link is not active - Replace CDN route to use link-based downloads: - Changes /cdn/ endpoint from file name access to link path access - Removes ability to serve tome assets via direct file name - Maintains FetchAsset gRPC API for agent communication - Update dependencies: - Upgrade entgo.io/ent from v0.14.1 to v0.14.5 - Update related dependencies (atlas, sqlite3, tools) * go generate * Add mutations * Clarify docs * typeable random short string * Don't create link on upload * Resolve feedback --------- Co-authored-by: Claude <noreply@anthropic.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Squidli <32434695+cmp5987@users.noreply.github.com> Co-authored-by: Hulto <7121375+hulto@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>
nullmonk
added a commit
that referenced
this pull request
Jan 7, 2026
commit 2a86d51 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Mon Jan 5 22:46:36 2026 -0500 Split stdlib into seperate impl files. (#1538) * Split stdlib into seperate impl files. * fmt commit 7ffd027 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Mon Jan 5 21:24:30 2026 -0500 Multi transport builds (#1517) * Add multi-callback * Add DSN configuration * Force lowercase * Auto grab pubkey with multi-callback using 0th element. * Update v1 * Update callback rotation * fmt * Remove agent callback_uris and idx in favor of config's * Remove callback_uri from transport init replaced by config * remove callback_uri direct ref * fmt * Cleanup warnings * Address code review feedback - Remove unused imports and variables in agent.rs - Switch to and_then syntax for cleaner Option chaining - Refactor parse_dsn to return Transport struct directly - Convert default parameters to const values - Bubble up errors instead of returning defaults - Use named enum variants instead of numeric constants in tests - Update all test cases to use TransportType enum 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Hulto <hulto@users.noreply.github.com> * fmt * Fix test_parse_callback_interval_valid test The test was failing because parse_callback_interval() now returns anyhow::Result<u64> instead of u64. Updated the test to properly unwrap the Result before comparison. Co-authored-by: Hulto <hulto@users.noreply.github.com> * Update enum name * Fix tests * Fix tests * Setup tests * fix? --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Hulto <hulto@users.noreply.github.com> commit 2d11111 Author: KCarretto <Kcarretto@gmail.com> Date: Sun Jan 4 23:32:42 2026 -0500 update devcontainer go version to 1.24.3 (#1532) commit 40e9e6d Author: KCarretto <Kcarretto@gmail.com> Date: Sun Jan 4 23:30:45 2026 -0500 [bug] Close Portals (#1530) * added close message * feat: add integration test for portal closure (#1531) Added `tavern/internal/portals/portal_close_test.go` to verify that when an agent's `CreatePortal` stream ends, the corresponding user `OpenPortal` streams are notified with a CLOSE mote and subsequently terminated. Implemented a retry mechanism with backoff for portal creation checks in the test setup. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * cleanup test --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> commit 2e1506c Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Mon Jan 5 02:38:56 2026 +0000 Refactor File entity to Asset (#1529) Renames the `File` entity to `Asset` across the entire codebase, including: - Ent schema definitions and relationships - Generated Ent code - GraphQL schema, resolvers, and queries - Business logic in C2, CDN, and Tomes packages - Test data and test cases This aligns the terminology to be more generic and suitable for various types of resources managed by the system. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> commit 42c30b9 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Sun Jan 4 19:30:21 2026 -0500 Update ZIG_VERSION to 0.16.0-dev.1859 commit f075895 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Sun Jan 4 19:11:52 2026 -0500 Update Go base image to version 1.24.0 commit cc831c2 Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Sun Jan 4 18:43:26 2026 -0500 fix(logo): Fix logo flashing (#1509) * fix(logo): Fix logo flashing * fix(z-index): Fix z-index of table commit 490b0cd Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Sun Jan 4 18:10:50 2026 -0500 Improve tome automation scheduling window (#1527) * Improve tome automation scheduling window Modified tome automation logic to queue tasks if their schedule falls within the upcoming beacon check-in interval window (now to now + interval), rather than just checking the current minute. This prevents beacons with long sleep intervals from missing scheduled tasks. Added `TestHandleTomeAutomation_IntervalWindow` to verify the fix and prevent regressions. Also fixed a missing sqlite3 driver import in `tome_automation_test.go`. * Refactor tome automation to use time.Duration Updated `handleTomeAutomation` signature to accept `time.Duration` for interval instead of `int`, addressing PR feedback. Updated call sites and tests to pass duration. * Refactor tome automation for lookahead and range support Updated `handleTomeAutomation` to support lookahead scheduling for standard cron expressions (queueing tasks scheduled within the upcoming beacon interval) while enforcing strict current-time matching for range-based schedules (e.g., `* 6-12 * * *`). Refactored method to accept `time.Duration` for interval. Added comprehensive tests for interval window logic and cron range behavior. Fixed missing sqlite3 driver in tests. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> commit ee68dea Author: Kirill Rudakov <simbiont666@gmail.com> Date: Mon Jan 5 00:21:14 2026 +0300 fix: fix #1516 (#1524) * fix: fix #1516 * fix: fixes second part of #1516 (type of ActiveTransport as enum variant name) --------- Co-authored-by: Kirill Rudakov <kirill.rudakov@gorparkovki.ru> commit 27c1744 Author: KCarretto <Kcarretto@gmail.com> Date: Sun Jan 4 16:17:50 2026 -0500 [feature] Tome automation (#1526) * added new fields for tome automation * Implement Tome automation in ClaimTasks API (#1525) * Implement Tome automation for ClaimTasks Refactored ClaimTasks to include logic for automatically queuing Quests/Tasks based on Tome triggers: - RunOnNewBeaconCallback - RunOnFirstHostCallback - RunOnSchedule (cron) Added Prometheus metrics for automation errors and implemented non-blocking error handling. * Refactor Tome automation into helper function Moved Tome automation logic from `ClaimTasks` to `handleTomeAutomation` to improve code readability and maintainability. * Add comprehensive tests for Tome automation Added `tome_automation_test.go` containing `TestHandleTomeAutomation` to verify: - Triggers: New Beacon, New Host, Schedule. - Constraints: Scheduled host restrictions (allow/deny). - Deduplication: Ensuring a tome is only queued once per callback even if multiple triggers match. This addresses PR feedback requesting unit/integration tests for the new `handleTomeAutomation` helper. * Refactor handleTomeAutomation to use early return Simplified the control flow in `handleTomeAutomation` by returning early on error, reducing nesting and improving readability. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> commit c20e22d Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Sat Jan 3 20:08:38 2026 -0500 UI improve table (#1519) * feat(table): Improve table layout * fix(styling): Change rows shown, add tests, fix styling commit 4d773c1 Author: KCarretto <Kcarretto@gmail.com> Date: Sat Jan 3 15:42:51 2026 -0500 update portals e2e to be used during merge queue (#1518) commit 605e830 Author: KCarretto <Kcarretto@gmail.com> Date: Sat Jan 3 14:14:38 2026 -0500 [feature] Portals! (#1484) * initial stubbing and renaming * feat: implement tavern/portals/stream package (#1462) This commit introduces a new package `stream` in `tavern/portals` which provides utilities for handling ordered streams of `portalpb.Mote` messages. Key features: - `payloadSequencer`: Handles atomic sequence ID generation and mote creation. - `OrderedWriter`: Wraps a sender function (like a gRPC stream Send) to automatically sequence and write messages. - `OrderedReader`: Wraps a receiver function (like a gRPC stream Recv) to reorder incoming messages, handling out-of-order delivery with configurable buffering and stale stream detection. This package is designed to support both client and server sides of the portal stream. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * refactor reader to use functional API * Implement PubSub Multiplexer (Mux) (#1466) * Implement PubSub Multiplexer (Mux) for Portals - Created `Mux` package in `tavern/internal/portals/mux`. - Implemented dual-mode operation: In-Memory (for dev) and GCP PubSub (for prod). - Implemented `CreatePortal` and `OpenPortal` lifecycle methods with resource provisioning. - Implemented `Publish` and `Subscribe` logic with local broadcasting (fast path) and global PubSub (slow path). - Added `HistoryBuffer` for message replay. - Added intelligent topic caching to handle `mempubsub` quirks and improve performance. - Added Prometheus metrics for observability. - Verified with comprehensive unit tests using `enttest` and `mempubsub`. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used `sync.RWMutex` for thread safety (`activeSubs`, `subscribers`, `histMu`). - **Observability:** Added Prometheus metrics for message counts. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, and Portal opening. * Refactor Mux and Address PR Feedback - **Structure:** Refactored `Mux` to use composed structs (`SubscriptionManager`, `SubscriberRegistry`, `TopicManager`, `HistoryManager`) for clearer locking semantics. - **Safety:** - Handled race conditions in `CreatePortal` and `OpenPortal` by re-checking state after lock acquisition. - Updated `teardown` logic to use `client.Portal.UpdateOneID` for reliability. - Standardized on `context.Background()` for shutdown operations to prevent context leaks. - **Features:** - Added `WithSubscriberBufferSize` to configure channel buffers. - Added `WithHistoryReplay` option to `Subscribe` for optional history. - Added `mux_messages_dropped_total` metric. - **Concurrency:** Moved global lock handling into granular manager structs to reduce contention. - **Correctness:** Fixed `CreatePortal` to use task-based lookup for dependencies and removed invalid `portalID` parameter usage in logic (though kept signature for now as per instructions). Tests passed. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used composed structs (`SubscriptionManager`, `SubscriberRegistry`) for granular locking. - **Observability:** Added Prometheus metrics for message counts and dropped messages. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, Portal opening, and Benchmarks. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * both grpc endpoints * stub out portal-stream package * Implement portal-stream crate (#1470) * Implement portal-stream crate with sequencer, reader, and writer logic - Added `implants/lib/portals/portal-stream` crate. - Implemented `PayloadSequencer` for atomic sequence ID generation. - Implemented `OrderedReader` for reordering incoming messages with timeout and buffer handling. - Implemented `OrderedWriter` for sequencing outgoing messages. - Added comprehensive unit tests for all components. - Added crate to `implants` workspace. * Switch to anyhow for error handling in portal-stream - Replaced `thiserror` with `anyhow` in `portal-stream`. - Updated `Cargo.toml` to use `anyhow` from workspace. - Updated `reader.rs` and tests to use `anyhow::Result` and `anyhow!`. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement SOCKS5 Proxy using tavern/portals/stream (#1467) * Implement SOCKS5 proxy with gRPC tunneling - Added bin/socks5/proxy.go implementing a SOCKS5 proxy server. - Implemented tunneling over stream.OrderedWriter/Reader. - Supported TCP CONNECT and UDP ASSOCIATE commands. - Implemented robust lifecycle management and cleanup. - Added benchmarks in bin/socks5/proxy_test.go demonstrating high throughput. * Address PR comments: Refactor writes, defaults, and shutdown tracking - Refactored raw `conn.Write` calls into named helper functions. - Changed default upstream port to 8000. - Added `sync.WaitGroup` to track connection lifecycle. - Added logging for dropped motes in dispatcher. - Defined `maxStreamBufferedMessages` constant. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement Portal Infrastructure in ImixV2 (#1471) * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. - Updated `run_create_portal` to send initial registration message. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * some changes * Add integration test * Added TRACE Bytes kind * added trace protos * Implement Trace Motes for Portals (#1473) * feat: implement end-to-end trace motes Implements application-level tracing for Portals infrastructure using the new `tracepb` definitions. * **CLI (`bin/socks5`)**: * Added `trace` subcommand to generate trace motes, send them to the server, and print a latency report. * Refactored `proxy.go` to support subcommands. * Added `addTraceEvent` helper for modifying trace motes. * **Server (`tavern`)**: * Instrumented `api_open_portal.go` and `api_create_portal.go` to inject trace events at key checkpoints (Recv, Pub, Sub, Send). * Created `trace_helper.go` to share event injection logic. * **Agent (`imixv2`)**: * Updated `run.rs` to intercept `BYTES_PAYLOAD_KIND_TRACE` motes. * Implemented logic to add `AGENT_RECV` and `AGENT_SEND` events and immediately echo the mote back. * added retry to trace --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * update flag to --portal * update buf size * Update portal API to send keepalive motes (#1472) - Added keepalive ticker to sendPortalInput loop in api_create_portal.go - Sends a BYTES_PAYLOAD_KIND_KEEPALIVE mote at regular intervals - Prevents connection timeouts similar to the reverse shell implementation Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * add tokio-console support for imixv2 * Fix SOCKS5 proxy cold start hang by avoiding BiLock on TcpStream (#1476) Replaced `tokio::io::split(stream)` with `stream.into_split()` in `implants/imixv2/src/portal/tcp.rs`. The former uses a `BiLock` which can cause deadlocks when the read and write halves are accessed concurrently in separate tasks, specifically causing the "cold start" hang where the initial payload might be blocked. `into_split()` returns owned halves that operate independently. Added a regression test `implants/imixv2/src/tests/repro_issue.rs` to verify the fix. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * socks proxy sends registration message now * socks5 trace must send registration message * Add E2E Portals Workflow and Playwright Test (#1478) Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * cargo fmt * remove spammy log line * fix some tests * add create_portal fake impl * revert some changes to grpc.rs * remove grpc explicit sizes * update socks5 proxy to support auth and portals to support gcp pubsub * oops * minor cleanup * fix tests * Fix e2e workflow * use env var for auth * wait for socks to start before continuing * Add benchmark tests for cryptocodec (#1485) Added a new test file `tavern/internal/cryptocodec/cryptocodec_bench_test.go` to measure the throughput of `Encrypt` and `Decrypt` methods in `CryptoSvc`. Includes benchmarks for both encryption and decryption operations using standard payload sizes. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Fix race condition in TestPortalIntegration (#1487) The `TestPortalIntegration` test was flaky and often hung because the Agent would publish messages to the portal before the User had successfully subscribed to the output topic. This resulted in messages being dropped (as evidenced by "message sent to topic with no subscribers" warnings from the in-memory pubsub) and the User reader blocking indefinitely. This commit replaces the arbitrary `time.Sleep` with a deterministic synchronization mechanism. The User now sends a "ping" message to the Agent immediately after opening the portal. The Agent waits to receive this ping before proceeding. This ensures that the User's portal connection (and thus the underlying pubsub subscription) is fully established before the Agent attempts to send any data. This reduces test execution time and eliminates the race condition. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * fix proxy upstream address parsing * minor fixes & cleanup * fix collapsible if * cargo fmt * fix portal workflow * Add Tavern User Guide for Authentication and API Token (#1501) * Add documentation for TAVERN_API_TOKEN and portal auth flow This commit adds a new user guide page for Tavern (`docs/_docs/user-guide/tavern.md`) detailing the purpose of `TAVERN_API_TOKEN`. It clarifies the distinction between this token and the web OAuth token and explains the "portal auth flow" for users SSH'd into remote environments (e.g., Kali VMs) where standard auth port forwarding is not feasible. * Update TAVERN_API_TOKEN docs: remove non-existent Portal Auth Flow Per code review feedback, the "Portal Auth Flow" feature does not exist yet. This commit removes that section from the documentation, leaving the explanation of what the token is and when to use it (SSH scenarios). --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * cleanup * Merge origin/main, resolve conflicts, and fix go generate (#1515) - Merged changes from origin/main, resolving conflicts in app.go, server.go, and mock transport. - Updated `golang.org/x/tools` to fix `ent` generation failure ("context without types"). - Re-ran `go generate ./...` to update generated protobuf and ent code. - Fixed compilation errors in tests due to renamed protobuf enum constants (ActiveTransport_TRANSPORT_HTTP1). Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Add offline filters (#1505) * feat(online-offline): Online Offline in progress * fix(online-offline): Change text * fix(query): Simplify query * fix(build): Build UI * fix(create-quest): Hide filter from create quest * Add Link entity and update CDN to use link-based file access (#1444) * Add Link entity and update CDN to use link-based file access This commit implements a new Link entity system for the CDN: - Add Link entity schema with: - path (unique, default UUIDv4) - active_before (timestamp, default epoch 0) - active_clicks (int, default 0) - edge to File entity (one-to-many relationship) - Update CDN upload handler to: - Create a new Link entity for each uploaded file - Return both file ID and link path in response - Add new CDN link download handler to: - Serve files using Link entity path instead of file name - Check active_clicks and active_before before serving - Decrement active_clicks when file is served - Return 404 if link is not active - Replace CDN route to use link-based downloads: - Changes /cdn/ endpoint from file name access to link path access - Removes ability to serve tome assets via direct file name - Maintains FetchAsset gRPC API for agent communication - Update dependencies: - Upgrade entgo.io/ent from v0.14.1 to v0.14.5 - Update related dependencies (atlas, sqlite3, tools) * go generate * Add mutations * Clarify docs * typeable random short string * Don't create link on upload * Resolve feedback --------- Co-authored-by: Claude <noreply@anthropic.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Squidli <32434695+cmp5987@users.noreply.github.com> Co-authored-by: Hulto <7121375+hulto@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> commit f74a1fb Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Sat Jan 3 13:42:00 2026 -0500 All the polling (#1508) * feat(#1207): Add global polling * fix(ui): Add polling visuals * fix(test): Add testing and rebuild * fix(unused): Remove unused code commit a456e40 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Fri Jan 2 21:51:52 2026 -0500 Add Link entity and update CDN to use link-based file access (#1444) * Add Link entity and update CDN to use link-based file access This commit implements a new Link entity system for the CDN: - Add Link entity schema with: - path (unique, default UUIDv4) - active_before (timestamp, default epoch 0) - active_clicks (int, default 0) - edge to File entity (one-to-many relationship) - Update CDN upload handler to: - Create a new Link entity for each uploaded file - Return both file ID and link path in response - Add new CDN link download handler to: - Serve files using Link entity path instead of file name - Check active_clicks and active_before before serving - Decrement active_clicks when file is served - Return 404 if link is not active - Replace CDN route to use link-based downloads: - Changes /cdn/ endpoint from file name access to link path access - Removes ability to serve tome assets via direct file name - Maintains FetchAsset gRPC API for agent communication - Update dependencies: - Upgrade entgo.io/ent from v0.14.1 to v0.14.5 - Update related dependencies (atlas, sqlite3, tools) * go generate * Add mutations * Clarify docs * typeable random short string * Don't create link on upload * Resolve feedback --------- Co-authored-by: Claude <noreply@anthropic.com> commit ac48265 Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Fri Jan 2 21:38:22 2026 -0500 Add offline filters (#1505) * feat(online-offline): Online Offline in progress * fix(online-offline): Change text * fix(query): Simplify query * fix(build): Build UI * fix(create-quest): Hide filter from create quest commit e9db24d Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Fri Jan 2 19:50:01 2026 -0500 Add host timing (#1502) commit 981df36 Author: KCarretto <Kcarretto@gmail.com> Date: Fri Jan 2 15:45:22 2026 -0500 fix ping message handling (#1500) commit c16a3c1 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Fri Jan 2 13:09:32 2026 -0500 Pb active transport (#1464) * Manual updates * go generated * Type * clippy isnt angry * todo * generate * Add extra field * Update docs. * Cleanup bad unwraps * Add extra to config and pass it through. * Fix proxy uri name. * Remove proxy_uri ref * Update docs. * Address PR review comments for active transport refactor (#1490) * Cleanup todos * go generate * fix e2e test * fmt commit 7115020 Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 12:36:12 2026 -0500 fix(graphql): fix foreign key violation in dropAllData (#1495) * fix(graphql): fix foreign key violation in dropAllData Reorder deletion steps in `DropAllData` mutation to ensure child entities (HostCredential, HostFile, HostProcess) are deleted before their parents (Task, Beacon, Host). Add missing `HostCredential` deletion. Add a regression test case `foreign_key.yml` to verify the fix. * fix(graphql): fix foreign key violation in dropAllData Reorder deletion steps in `DropAllData` mutation to ensure child entities (HostCredential, HostFile, HostProcess) are deleted before their parents (Task, Beacon, Host). Add missing `HostCredential` deletion. Add a regression test case `foreign_key.yml` to verify the fix. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> commit 92961af Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 17:23:46 2026 +0000 Fix imixv2 interval sync issue (#1494) * Fix: Force immediate server sync on interval change in imixv2 Previously, when the agent's callback interval was increased (e.g., 5s -> 60s), the agent would update its local sleep timer but wouldn't notify the server until the *next* callback. This caused the server to mark the beacon as dead/offline because it expected a check-in based on the old, shorter interval. This change modifies `set_callback_interval` to trigger an immediate `claim_tasks` call after updating the configuration. This ensures the server receives the updated beacon metadata (including the new interval) right away. Any tasks retrieved during this forced check-in are spawned for execution to prevent task loss. * Refactor claim_tasks logic into ImixAgent::process_job_request Per PR review feedback, the logic for claiming and spawning tasks has been moved from the `set_callback_interval` method and the standalone `process_tasks` function (in `run.rs`) into a unified `process_job_request` method on `ImixAgent`. This reduces code duplication and ensures consistent behavior for task processing. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> commit 5182fe0 Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 17:22:46 2026 +0000 Fix mixed Int/Float comparisons in eldritch-core (#1493) * Fix mixed Int/Float comparisons in eldritch-core Previously, mixed type comparisons (e.g., `Int` vs `Float`) fell back to ordering by type discriminant because they are different types. This caused `max(1, 0.8)` to return `0.8` because `Int` (2) < `Float` (3). This change adds a special case in `Value::cmp_helper` to handle `Int` vs `Float` and `Float` vs `Int` by converting the integer to a float and using `total_cmp`. This ensures numerical correctness for `max`, `min`, and general comparisons. Added regression test `tests/max_min_mixed.rs`. * Fix mixed Int/Float comparisons in eldritch-core Previously, mixed type comparisons (e.g., `Int` vs `Float`) fell back to ordering by type discriminant because they are different types. This caused `max(1, 0.8)` to return `0.8` because `Int` (2) < `Float` (3). This change adds a special case in `Value::cmp_helper` to handle `Int` vs `Float` and `Float` vs `Int` by converting the integer to a float and using `total_cmp`. This ensures numerical correctness for `max`, `min`, and general comparisons. Added regression test `tests/max_min_mixed.rs`. Ran `cargo fmt` to ensure code style compliance. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> commit b84d92f Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 12:18:04 2026 -0500 Configure CI for GitHub Merge Queues (#1489) commit b437b87 Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 12:09:29 2026 -0500 [AI] Test Coverage Expansion and Refinement (#1491) commit 651e377 Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 10:48:42 2026 -0500 Implement cursor context finder for LSP (#1439) commit beb1cab Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Thu Jan 1 22:50:13 2026 -0500 Cp add more filters (#1488) * fix(#979): Expand filtering on create quest for tome * fix(transport): Add transport filter and field tag * fix(enums): Add enum handling commit 9473a13 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Thu Jan 1 18:05:24 2026 -0500 Activate previously ignored tests (#1465) Remove #[ignore] attributes from three Rust tests that were previously disabled: - test_sys_exec_current_user: Tests exec functionality with current user - test_sys_exec_disown_no_defunct: Tests disowned processes don't become defunct - test_netstat_integration: Tests netstat functionality on systems with /proc All tests now pass successfully. Co-authored-by: Claude <noreply@anthropic.com> commit 3923f09 Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Thu Jan 1 15:04:41 2026 -0500 Cp fix small bugs (#1483) * fix(package): Fix package * fix(filter): Fix create quest filter by hostname or beacon * fix(bug): Fix spelling and submit warning * fix(spelling): Fix spelling issues * fix(build): Run npm build commit f22e181 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Thu Jan 1 14:12:14 2026 -0500 Add diverse transport types to test data (#1482) * Add diverse Transport types to test data
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implemented the portal infrastructure in
imixv2agent. This includes:create_portalmethod toTransporttrait and implemented it in the gRPC transport. This establishes a bi-directional stream with the C2.imixv2/src/portalmodule that handles the multiplexing of TCP, UDP, and Bytes messages over the single gRPC stream usingportal-streamfor ordering.run.rs: Main loop handling incoming motes and dispatching to handlers.tcp.rs: Handles TCP connections (connect, read, write).udp.rs: Handles UDP sockets.bytes.rs: Handles PING/Keepalive messages.ImixAgentto supportcreate_portalby spawning the portal runner as a subtask.create_portalineldritch-libpivotso it can be called from Eldritch scripts.portal-streamcrate to support asyncSenderfor non-blocking writes.PR created automatically by Jules for task 954376063208258928 started by @KCarretto