Commit 5da1515
authored
feat: Add async FDv2 data sources (#485)
## Overview
Part of the async Python SDK work (epic SDK-60). This is the first of
two stacked PRs
that add async FDv2 support. It adds the async FDv2 **data sources** and
the shared
support they need. The follow-up PR adds the async FDv2 **data system**
(coordinator)
and wires it into the async client.
This is experimental and not yet wired into a public code path on its
own; the async
client only builds an `AsyncFDv2` data system in the stacked follow-up
PR.
## What this PR adds
- `impl/datasourcev2/async_polling.py` — async polling data source and
its builders.
- `impl/datasourcev2/async_streaming.py` — async streaming data source
and its builder.
- `impl/integrations/test_datav2/async_test_data_sourcev2.py` plus a new
`TestDataV2.async_builder` property for use with the async FDv2 data
system.
## Shared refactors
To avoid duplicating parsing logic between the sync and async sources,
the payload
parsing and message handling are extracted into new shared modules that
both consume:
- `polling_common.py` — `polling_payload_to_changeset` /
`fdv1_polling_payload_to_changeset`,
moved out of `polling.py`. Sync `polling.py` now imports them (and
re-exports them).
- `streaming_common.py` — `process_message`, moved out of
`StreamingDataSource._process_message`.
Sync `streaming.py` now calls the shared function.
Two small fixes to existing async infrastructure that the async sources
depend on:
- `impl/datasource/async_status.py` — drop the read/write lock from
`AsyncDataSourceUpdateSinkImpl`; the single-threaded event loop does not
need it.
- `impl/aio/concurrency.py` — stop swallowing/re-raising
`CancelledError` separately in
`AsyncRepeatingTask`; let cancellation propagate normally.
## Testing
- `LD_SKIP_DATABASE_TESTS=1 uv run pytest
ldclient/testing/impl/datasourcev2/` — 115 passed.
- `ldclient/testing/integrations/test_test_data_sourcev2.py` — 34
passed.
- `make lint` (mypy, isort, pycodestyle) — clean.
Tracked internally: SDK-2869
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Overview**
> Adds **async FDv2 data sources** for the experimental async Python
SDK: `AsyncPollingDataSource` (with aiohttp requesters and FDv1 fallback
builders) and `AsyncStreamingDataSource` (SSE via `AsyncSSEClient`),
plus an async **TestDataV2** source exposed through
`TestDataV2.async_builder`.
>
> **Shared logic** is pulled into `polling_common.py` and
`streaming_common.py` so sync `polling.py` / `streaming.py` and the new
async modules share payload parsing, `map_polling_result`, FDv1 fallback
signaling, and stream error classification. Sync polling now sends the
selector as query param **`basis`** (was `selector`).
>
> Small supporting changes: **`AsyncDataSourceUpdateSinkImpl`** drops
its read/write lock; **`AsyncRepeatingTask`** no longer special-cases
`CancelledError` inside the action loop.
>
> Coverage includes new async polling/streaming tests and updates to
polling payload parsing tests after the move to common modules.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
161bb0f. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->1 parent 49e809f commit 5da1515
13 files changed
Lines changed: 2421 additions & 432 deletions
File tree
- ldclient
- impl
- aio
- datasourcev2
- datasource
- integrations/test_datav2
- integrations
- testing/impl/datasourcev2
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
242 | 242 | | |
243 | 243 | | |
244 | 244 | | |
245 | | - | |
246 | | - | |
247 | 245 | | |
248 | 246 | | |
249 | 247 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
7 | 6 | | |
8 | 7 | | |
9 | 8 | | |
| |||
23 | 22 | | |
24 | 23 | | |
25 | 24 | | |
26 | | - | |
27 | 25 | | |
28 | 26 | | |
29 | 27 | | |
30 | 28 | | |
31 | | - | |
32 | | - | |
| 29 | + | |
33 | 30 | | |
34 | 31 | | |
35 | 32 | | |
| |||
73 | 70 | | |
74 | 71 | | |
75 | 72 | | |
76 | | - | |
77 | | - | |
| 73 | + | |
78 | 74 | | |
79 | | - | |
80 | | - | |
| 75 | + | |
| 76 | + | |
81 | 77 | | |
82 | | - | |
83 | | - | |
| 78 | + | |
| 79 | + | |
84 | 80 | | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
90 | 86 | | |
91 | | - | |
| 87 | + | |
92 | 88 | | |
93 | 89 | | |
94 | 90 | | |
| |||
0 commit comments