perf: halve the database statements a flow run costs

[ogabrielluiz]

A flow run issued 27 SQL statements on the request-handling connection pool. It now issues 13.

Two PostgreSQL-only defects turned up on the way. Both silently destroy trace data, and both are reproduced against a real PostgreSQL 16.14 below.

Why this is worth more than the 52%

The halving is measured on a two-component flow. The change that matters is that a term which scaled with flow size is gone.

Persisting a trace cost 2 + 2N statements for N spans, where N is components plus LangChain callback spans:

flow	span statements before	after
2 components	12	2
20 components	~80	2
50-component agent flow	~200	2

The benefit therefore grows with exactly the flows that were already the slowest. Per-run database cost is now roughly flat regardless of graph size.

What that buys:

Round-trips. On PostgreSQL every statement is a network hop, so this is about 14 fewer per run on a small flow and far more on a large one. For a flow with an LLM call that is noise next to the model latency; for cheap flows and large graphs it is not.

Write volume. Writes per run went from 12 to 8, which is less WAL, less replication lag and less vacuum pressure.

Concurrency headroom. Checkouts did not change, but each one now holds its connection for fewer statements, so the same pool serves more concurrent runs before it saturates.

Export volume. Database spans were around 80% of exported span volume, so this cuts APM cost as a side effect rather than by hiding the spans.

Worth being explicit about what is not measured here: this counts statements and round-trips. It is not a latency or throughput benchmark, and any millisecond figure above is arithmetic from round-trip counts rather than an observation.

Why the statements were there

The largest block was the native tracer. NativeTracer._flush_to_database wrote its trace row and every span with await session.merge().

merge() has to SELECT a primary key before it can decide between INSERT and UPDATE. Every one of those SELECTs missed: the trace id is minted with uuid4 when the run starts, and span ids are uuid5 values derived from it, so the row cannot already exist. That cost 2 + 2N statements for N spans. A 50-component flow paid roughly 100 statements just to record itself.

It is now one INSERT ... ON CONFLICT DO UPDATE per table. 12 statements became 2, and span count no longer drives round-trips.

The scaling fix is literally that the database call moved out of the per-span loop. It used to be:

for span_data, span_uuid, parent_uuid in resolved:
    span = SpanTable(...)
    await session.merge(span)        # one SELECT plus one INSERT, per span

and it is now native.py:407-425, where the loop only builds objects and does no I/O:

spans = [
    SpanTable(...)
    for span_data, span_uuid, parent_uuid in resolved
]
await _upsert_rows(session, SpanTable, spans)

The loop that remains, in _upsert_rows at native.py:85-93, iterates chunks rather than rows:

chunk_size = max(1, 30000 // len(values[0]))    # about 2142 for a 14-column span row
for start in range(0, len(values), chunk_size):
    ...
    await session.exec(statement)

so the statement count went from N to ceil(N / 2142), which is 1 for any realistic flow.

The conflict clause is load-bearing rather than decorative. A paused HITL run flushes its partial spans (langflow/api/build.py, comment "merge is idempotent on resume") and re-flushes the same deterministic ids after resume, so those rows have to be overwritten. A plain add_all() would raise on the primary key.

The two PostgreSQL defects

Neither exists on SQLite, which is why no test caught them.

A repeated vertex loses the whole trace. Span ids are uuid5(namespace, f"{trace_id}-{component_id}"), keyed only on the component. A Loop component or a graph cycle traces the same vertex once per iteration, so one flush can carry the same primary key several times. PostgreSQL refuses to let a single ON CONFLICT DO UPDATE touch a row twice and raises CardinalityViolation, which rolls back the trace and every span with it. wait_for_flush swallows the error, so the run succeeds and the trace just disappears.

merge() tolerated this through the identity map. Fixed by collapsing rows on the primary key before sending, last write wins, which is what the merge() loop produced.

A flush past roughly 4681 spans fails entirely. PostgreSQL allows 65535 bind parameters per statement and a span row binds 14. merge() sent one statement per row and had no such ceiling. Fixed by chunking, sized from the column count rather than a constant.

Verified against PostgreSQL 16.14 with langflow's own models and its postgresql+psycopg driver:

1. duplicate span ids, WITH fix        PASS
2. duplicate span ids, WITHOUT fix     raised psycopg.errors.CardinalityViolation:
                                       ON CONFLICT DO UPDATE command cannot affect row a second time
3. 6000 spans, WITH fix                PASS
4. 6000 spans, WITHOUT fix             raised psycopg.OperationalError: number of parameters...
5. HITL re-flush overwrites            PASS

Three smaller reads

The post-commit message refresh. memory.py re-read every message it had just written. The session factory sets expire_on_commit=False, and no column on message is filled in by the database, so that SELECT returned exactly what the caller already held. All 18 columns have server_default=None, there is no autoincrement primary key, and the five defaults (id, timestamp, error, edit, is_output) are Python-side and already evaluated before the INSERT.

The job status read-back. update_job_status ran its UPDATE, then re-fetched the row by primary key to satisfy a Job | None return type. No caller binds the result, in src/ or in the tests. It returns bool from rowcount now, which carries the only part a caller could have used.

The separate user lookup. session.get(User, ...) immediately after the api key SELECT is now a joinedload. One statement instead of two, on every authenticated request rather than only on runs.

Per-run ledger

Two-component flow, ChatInput to ChatOutput.

statement	before	after
SELECT span	5	0
INSERT span	5	1
SELECT trace	1	0
INSERT trace	1	1
SELECT job	2	0
UPDATE job	2	2
INSERT job	1	1
SELECT message	2	1
INSERT message	1	1
UPDATE message	1	1
SELECT apikey	1	1
SELECT user	1	0
UPDATE apikey	1	1
SELECT flow	1	1
SELECT variable	1	1
SELECT memory_base	1	1
total	27	13

Pool checkouts stayed at 11. Cutting statements does not cut checkouts: checkouts track session scopes, not statements.

How to measure it

uv run pytest src/backend/tests/unit/test_db_calls_per_run.py -s

It counts with SQLAlchemy's own before_cursor_execute and pool checkout events rather than with OpenTelemetry spans. No exporter, no Docker, no demo stack, about 10 seconds.

Two corrections were needed before the number meant anything:

Two autouse fixtures in tests/conftest.py take the suite off the shipped configuration. disable_telemetry_writer forces the legacy synchronous write path and deactivate_tracing switches the internal trace and span tables off. Measured under those, a run looks like 30 statements with half of them vertex_build and transaction retention deletes, which is not what users pay. The harness overrides both.

transaction and vertex_build writes do not touch the request pool in production. The telemetry writer owns a dedicated engine and amortizes their retention, so they are correctly absent from these counts.

Running the suite on PostgreSQL

The client and async_session fixtures hardcoded SQLite, so nothing in the backend suite could exercise the backend most deployments actually run. Both now honour LANGFLOW_TEST_DATABASE_URL and fall back to SQLite when it is unset, so default behaviour is unchanged.

Both fixtures matter. Wiring only client leaves the tracer tests on SQLite, because they take async_session — which would have meant the tests guarding the two PostgreSQL defects were the ones never running on PostgreSQL. Pointing them at it immediately failed three that SQLite had been hiding: two issued PRAGMA foreign_keys=ON, a syntax error there, and one asserted that a malformed flow_id still persists its trace (see below). With those fixed, test_flush_collapses_repeated_builds_of_one_vertex now reproduces the real CardinalityViolation on PostgreSQL instead of only checking statement shape on SQLite.

Note the tradeoff on client: with the variable set, every test shares one database rather than getting a fresh file, so run it serially and expect row-counting suites to need a clean database. async_session is unaffected, since it creates and drops the schema per test.

docker run -d --name lf-pg -e POSTGRES_PASSWORD=lf -e POSTGRES_USER=lf -e POSTGRES_DB=lf \
  -p 55432:5432 postgres:16-alpine
LANGFLOW_TEST_DATABASE_URL="postgresql://lf:lf@localhost:55432/lf" uv run pytest ...

A run costs 11 to 13 statements and 10 checkouts on PostgreSQL, against 13 and 11 on SQLite. The statement count varies because the trace flush is an asyncio task that sometimes settles just after the response and lands outside the measurement window. The variance is in the measurement, not in the work.

What a run should cost

Ten statements, derived from what a run has to do rather than picked as a round number: read the api key, read the flow, read the global variables, check memory-base, insert the job row, write its terminal status, write the messages (2), write the trace and its spans (2).

Today's 13 is three above that floor: the redundant UPDATE job to in_progress, the pre-update SELECT message, and the UPDATE apikey usage counter.

What the floor does not contain is anything scaling with flow size. That was the real problem, and it is gone.

Deliberately left alone

The api key usage counters. This is the last per-request write before the flow starts. On PostgreSQL it takes a row lock held to commit, so every concurrent request presenting the same key serializes on one row. Deployments using one key per service make that the common case.

total_uses and last_used_at are read only by the API keys table in the frontend settings page. Nothing in authentication, rate limiting, or billing reads either, so they can be approximate. disable_track_apikey_usage already exists and defaults to False. Flipping that default or batching the write both work, but it changes visible product behaviour, so it is not this PR's call.

The 11 pool checkouts. Reducing them means merging session scopes, starting with the auth scope and the flow load, which run back to back in the same dependency chain. More invasive than anything here and it deserves its own measurement.

The latency tail. The 17% of checkouts over 50 ms was a SQLite contention property. On PostgreSQL the equivalent question is different and should be re-asked rather than re-run.

Tests

The seven tests in TestFlushToDatabase and TestFlushParentChildOrder asserted mock_session.merge.call_count, so they broke on a change that kept behaviour identical. They now assert on rows read back from a real session.

Two were added. test_flush_twice_upserts_instead_of_duplicating covers the HITL pause and resume shape. test_flush_collapses_repeated_builds_of_one_vertex covers the duplicate span id case, and it asserts on the shape of the statement rather than on the stored rows so that it fails on SQLite too. Verified RED-GREEN: it fails with the collapse removed.

On SQLite: tracing, test_messages.py, services/database, test_api_key.py, services/auth and the counter give 799 passed, 3 skipped. api/v2 and services/jobs give 319 passed. background_execution gives 107 passed.

A third PostgreSQL defect, pre-existing and not fixed here

Running the tracer tests on PostgreSQL surfaced one more, older than this change.

When a run has a malformed flow_id, the flush falls back to a sentinel uuid5, with the stated intent that "malformed flow_ids don't silently discard trace data". But trace.flow_id carries a foreign key to flow.id and the sentinel matches no row by construction. PostgreSQL enforces that immediately, so the INSERT raises, the session rolls back, and wait_for_flush swallows the error. The fallback does the opposite of what its comment claims.

merge() hit the same foreign key, so this predates the change and is left alone here. It is documented in test_flush_invalid_flow_id_logs_error_and_continues, whose persisted-row assertions now run only where foreign keys are off. Worth its own ticket.

Two known failures, both confirmed pre-existing by checking the changed source files out at the previous commit and reproducing them identically:

test_endpoints.py::test_concurrent_stream_run_with_input_type_chat errors on SQLite in a fixture.

Three row-count assertions in test_messages.py fail on PostgreSQL because the suite shares one database where each SQLite test gets a fresh file, so rows accumulate across tests. Worth fixing if PostgreSQL runs become routine.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 69c74c7c-3b1d-4d6d-a686-e242bbf63ce7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}