Skip to content

cnid: sqlite-default residual cleanup; dsi: raise the listen backlog to 128 - #3267

Open
andylemin wants to merge 1 commit into
mainfrom
perf-cnid-residuals-and-backlog
Open

cnid: sqlite-default residual cleanup; dsi: raise the listen backlog to 128#3267
andylemin wants to merge 1 commit into
mainfrom
perf-cnid-residuals-and-backlog

Conversation

@andylemin

@andylemin andylemin commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

CNID: sqlite-default residual cleanup

SQLite has been the default CNID scheme since PR #2265 ("Build and run netatalk without Berkeley DB") made meson select sqlite > mysql > dbd among the compiled backends — and all three compile by default, so DEFAULT_CNID_SCHEME, the runtime cnid scheme default, is already "sqlite" on any default build (afpd -v: "Default CNID backend: sqlite"). What remained was residue that still said dbd:

residue fix
docker entrypoint fallback: cnid scheme = ${AFP_CNID_BACKEND:-dbd} (two sites) :-sqlite — an unset AFP_CNID_BACKEND now matches what a stock source build already does
~30 CI test jobs (the ea:sys/ea:ad/UAM/AFP-version/login matrix) set no AFP_CNID_BACKEND and were unintentionally testing dbd via the entrypoint fallback they now inherit sqlite, the shipped default — only jobs that intentionally test a backend name it. The "AFP spec test (cnid:dbd)" job, which also relied on the fallback, now sets -e AFP_CNID_BACKEND=dbd explicitly
man page never stated which backend is the default the sqlite paragraph now says "This is the default backend when compiled in."
NEWS never recorded the default change entry added

Intentional backend coverage is unchanged: the cnid:mysql legs (Alpine + Debian) and the cnid:sqlite/perf legs set AFP_CNID_BACKEND explicitly, the VM spectest script exports dbd explicitly, and the cnid:dbd job is now pinned. Every other job had no business running dbd; this PR resolves that unintended coverage skew — the broad matrix now tests what ships.

DSI: raise the listen backlog to 128

DSI_TCPMAXPEND has been 20 since the CVS-era initial revision, with no config knob. A reconnect storm (default 200 max connections) overflows a backlog of 20: dropped SYNs and ~1s client retransmit delays. 128 matches the common SOMAXCONN floor, and the kernel silently caps the effective backlog at net.core.somaxconn, so no new option is warranted. No steady-state effect; the #ifndef guard stays for build-time overrides.

Testing

  • meson test (Alpine container, CI build flags): 3/3 suites, zero failures.
  • AFP spectest ea=sys and ea=ad legs run without AFP_CNID_BACKEND — i.e. on the new sqlite fallback end-to-end: both exit 0, zero failures.
  • Entrypoint verification: a container pinned to AFP_CNID_BACKEND=dbd generates cnid scheme = dbd and runs the cnid daemon; an env-less container generates cnid scheme = sqlite.
  • afpd -v in the built image reports sqlite as the default backend.

Outstanding issue

The netatalk master still starts cnid_metad whenever the dbd backend is merely compiled in, so an env-less (sqlite) container runs an idle cnid daemon that nothing talks to. Startup should be conditional on a volume actually using cnid scheme = dbd — tracked in #3268.

@andylemin
andylemin requested a review from a team August 24, 2026 03:47
@andylemin
andylemin requested a review from rdmark as a code owner August 24, 2026 03:47
…to 128

SQLite has been the default CNID scheme since PR #2265 made meson
select sqlite > mysql > dbd among the compiled backends. Residue
still said dbd: the docker entrypoint's AFP_CNID_BACKEND fallback,
and the man page and NEWS never stated the default. Align them; no
default changes here. The CI cnid:dbd spectest leg relied on the old
docker fallback for its backend, so it now sets AFP_CNID_BACKEND=dbd
explicitly -- intentional dbd and mysql coverage is unchanged.

Also raise DSI_TCPMAXPEND from 20 to 128. It has been 20 since the
CVS era, with no config knob; a reconnect storm (default 200 max
connections) overflows it: dropped SYNs and ~1s client retransmit
delays. 128 matches the common SOMAXCONN floor; the kernel silently
caps at net.core.somaxconn anyway, so no new option is warranted.
No steady-state effect.
@andylemin
andylemin force-pushed the perf-cnid-residuals-and-backlog branch from 65571f7 to 6aa912a Compare August 24, 2026 04:21
@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

📊 Performance Dashboard

Commit: 6aa912a5545b11444152ac6214ecce8b50421119

🔥 Spectest (AFP 3.4) - FlameGraph

Netatalk Code-time: 2.8% · Runtime: 62s · Stacks: 1017

🔥 Click the preview to open the interactive flamegraph (zoom + search).

Flamegraph preview

🔝 Top 10 leaf functions
Function Samples
[libsqlite3.so.3.53.4] 262403330
do_syscall_64 192943625
_raw_spin_unlock_irqrestore 175303065
__cp_end 87100265
copy_folio_from_iter_atomic 81587590
x64_sys_call 52921680
afpd 46306470
srso_alias_safe_ret 30870980
perf_syscall_enter 28665910
vfs_writev 27563375

📈 Speedtest (AFP 3.4) - PerfGraph

Speedtest throughput

Peak Read: 6830 MB/s (-6.1% vs hist avg 7271.0 MB/s; min 5963 / max 9526 over 25 PRs)
Peak Write: 1659 MB/s (+40.5% vs hist avg 1181.0 MB/s; min 225 / max 1808 over 25 PRs)

🔝 Throughputs per operation (vs. historical average)
Metric Current (MB/s) Cur Avg Δ% Hist avg Hist min Hist max
Read peak mean 6830 -6.1% 7271.0 5963 9526
Read avg mean 3838 -8.6% 4197.7 3524 5393
Read avg max 4204 -9.5% 4647.8 3812 5932
Write peak mean 1659 +40.5% 1181.0 225 1808
Write avg mean 542 +35.9% 398.8 91 563
Write avg max 956 +35.0% 708.3 224 962
Copy peak mean 2508 -5.3% 2649.7 2294 3353
Copy avg mean 1512 -3.5% 1567.2 1360 1954
Copy avg max 1635 -4.8% 1717.5 1515 2087
ServerCopy peak mean 3405 -8.2% 3711.2 3055 5090
ServerCopy avg mean 1862 -3.8% 1936.4 1604 2501
ServerCopy avg max 1904 -5.6% 2017.3 1675 2658

⏱️ Lantest (AFP 3.4) - LatencyGraph

Lantest latency

Avg total runtime: 3934 ms (-8.9% vs hist avg 4319.3 ms; min 2492 / max 5208 over 25 PRs)
Avg time per AFP op: 79 µs (-8.9% vs hist avg 86.7 µs; min 51 / max 105 over 25 PRs)

🐢 All operations (avg runtime, in test order, vs. historical average)
Metric Current (ms) Cur Avg Δ% Adj Δ% Hist avg Hist min Hist max
Writing one large file 35 -4.0% 36.4 26 41
Reading one large file 16 +4.2% 15.4 12 19
Creating 2000 files 485 -6.8% +1.8% 520.6 259 831
Create 2000 dirs tree (20×9×10) 469 -9.5% -0.9% 518.4 295 796
Open, write 1024 bytes, close 2000 files 335 -10.7% -2.0% 375.0 244 459
Open, read 1024 bytes, close 2000 files 293 -10.5% -1.9% 327.4 214 408
Copying 1000 files client-side (R+W) 519 -8.0% +0.7% 564.0 322 677
Copying 2000 files server-side 487 -8.2% +0.5% 530.4 212 727
Stat (lookup+getparams) 2000 files 190 -14.7% -6.0% 222.7 152 282
Enumerate dir with 2000 files 12 +43.5% +52.2% 8.36 3 14
Lock then unlock 2000 open forks 149 -8.7% +0.0% 163.1 118 199
Deleting 2000 files 321 -8.1% +0.5% 349.4 144 447
Byte-range lock/unlock 2000 ranges in one fork 156 -6.5% +2.2% 166.8 124 196
Directory cache hits (20 dirs x 100 files) 86 -13.4% -4.7% 99.3 67 122
Mixed cache operations (create/stat/enum/delete) on 500 files 205 -7.2% +1.5% 220.9 111 265
Deep path traversal (20 levels x 100 walks) 92 -12.9% -4.3% 105.7 73 131
Cache validation (500 files x 4 lookups) 84 -11.9% -3.3% 95.4 66 116

Run baseline: median op-test delta -8.7%, MAD 1.9%. Adj Δ% shifts each delta by the median; standouts ≥5% in bold. A large MAD means the run did not move uniformly — read the adjusted column with caution.

Performance trend

@andylemin

Copy link
Copy Markdown
Contributor Author

@rdmark ready for review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant