feat(elf): upgrade a stored symbol table to DWARF without --force

[krassx]

Closes #78. Server half: bugsee/bugsee-appserver#65, #66, #67 (all merged).

Problem

A library first uploaded as an AGP SYMBOL_TABLE (.so.sym) and later available with debug info shares one GNU build-id, so the server skipped the richer file (16004, no PUT). The only remedy was --force, which re-sends every library in the run — a poor fit for callers that upload a whole directory on every build (bugsee-android-gradle-plugin#10/#13).

Change

• Each native library declares what it can symbolicate: format_variant = dwarf | symtab, read from the file (debug info vs symbol table), never from its name. A library with neither declares nothing.
• Every --type elf metadata POST also sends replace_if_richer: true; the server compares with what it holds.
• Results (acceptance from the issue):
  • stored symtab, local dwarf → replaced, bytes transferred once, logged uploaded (upgraded SYMBOL_TABLE -> FULL) and counted as upgraded=N in the run summary
  • same file again, or a symtab when the server holds dwarf → 16004, no PUT, "already on server at the same or better level"
  • --force unchanged ("always replace"); it still declares the variant
• The 16004 reply can carry the stored variant (error.params.format_variant). A dwarf library skipped without the server saying what it holds means a server that predates this, and only then is the "re-run with --force" advice still printed (reworded). Against a current server it no longer appears, since it would be wrong.
• Wire/compat: both new Metadata fields are omitted from every other flow's body; an older server ignores them (strict schema), so nothing changes there. Outcome and presigned::upload are unchanged — the ELF flow uses a new upload_detailed returning {outcome, upgraded, stored}.
• --help for --force and the README --type elf section updated.

Rollout

Per CLAUDE.md this is a two-step rollout: the appserver change is already merged (staging deploys on merge; production is a manual deploy). Release this CLI, then bump the Gradle plugin's version floor. Until the server is on production, the new fields are ignored and behaviour is today's.

Test plan

• cargo fmt -- --check, cargo clippy --all-targets -- -D warnings, cargo test (657 unit + all integration)

• scripts/e2e_flows.py against a release build: ALL PASS (including the ELF-directory peak-RSS budget)

• New tests: variant classification on real fixtures (unstripped = dwarf, same library stripped of debug sections = symtab, same build-id), wire body per case incl. name/content disagreement, upgraded / stored-variant parsing (known, unknown, absent), upgrade → one PUT, richer-kept → zero PUTs and no --force advice, --force still sends overwrite

• Real uploads against the staging appserver (apidev.bugsee.com, appserver main incl. docs: validate READMEs against the code and published 0.7.12 #65-chore: license under MIT #67), release build of this branch, no --force anywhere. Fixture libsymbol1 (build-id bca64abf…4472), symbol-table and unstripped variants:

  Step	Result
  A. symbol table, first upload	uploaded=1; record ready (427 KB)
  B. same symbol table again	already on server … stored="symtab", uploaded=0 — no transfer
  C. unstripped over the stored symbol table	uploaded (upgraded SYMBOL_TABLE -> FULL), upgraded=1; afterwards exactly ONE ready record, the 1.8 MB DWARF one — the symbol-table record is gone
  D. same unstripped again	skipped, stored="dwarf", no transfer
  E. symbol table offered over the stored DWARF	skipped, stored="dwarf", no transfer (never downgrades)

  Left on that staging app: one ready ELF record, version cli-78-verify.

New fixture tests/fixtures/elf/libsymbol1.symtab.so (427 KB): libsymbol1.so with rust-objcopy --strip-debug.

🤖 Generated with Claude Code

[cursor]

Stale comment

Deep review — feat(elf): upgrade a stored symbol table to DWARF without --force

Closes #78. I reviewed the ELF identify/upload path, the shared presigned client (every format uses register), --type rust ELF (same format: elf documents), the Gradle directory-upload caller (#13 / #78), and the merged appserver ranking notes from #65/#67. CI is green. Locally: cargo test --test elf_upload (16/16) plus the new format_variant / upload_detailed unit tests — all pass.

The feature shape is right: content-based dwarf/symtab, other flows omit the new fields, Outcome/upload stay stable, and the old-server --force summary is still pinned by elf_force_overwrites_a_library_already_on_the_server.

There is one merge-blocking issue: this CLI asks the server to replace-if-richer for every native library, including files it labels symtab. At ship time every stored ELF record is unlabeled. That combination can downgrade a historical FULL upload.

Findings

1. P1 High — replace_if_richer: true on non-DWARF uploads vs unlabeled production records (src/cli/debug_files.rs ~1405, src/symbols/elf.rs format_variant)
2. P2 Medium — skip log always says “same or better level”, including when the server did not say what it holds (src/cli/debug_files.rs ~1428)

Overall risk: High

Merge recommendation: Do not merge until the P1 is fixed

Gate replace_if_richer on a local dwarf file (the only upgrade #78 asks for). If create-time outranks is numeric with missing < symtab < dwarf — which matches appserver #67 treating null as outranked — the first --type elf run after this ships can replace unlabeled FULL documents with a symbol table / stripped .so. That is silent loss of file:line. If the server already refuses non-dwarf incoming, the CLI is still requesting a replacement it does not need; omit the flag anyway and add a wire test that a .so.sym POST does not carry replace_if_richer.

Most important to fix

• Only send replace_if_richer: true when lib.format_variant() == Some("dwarf").
• Do not treat .dynsym as AGP SYMBOL_TABLE (symbolic’s has_symbols() is syms || dynsyms).

Residual (not blocking)

--type rust still omits both fields, so Linux/Android Rust ELF keeps creating unlabeled format: elf documents and still needs --force for stripped → unstripped. Same server identity as --type elf.

Positive

Content vs name is tested (FULL packed as .so.sym still sends dwarf). Other formats leave the new keys off the wire. --force still sends overwrite. Upgrade logging / upgraded=N / zero-PUT skip paths are pinned against a protocol-accurate mock.

  

_{Sent by Cursor Automation: Bugsee code review}

[cursor]

Deep review (re-review of 9c97b31)

Closes #78. I re-reviewed the ELF identify/upload path, the shared presigned client, --type rust ELF (same format: elf documents), the Gradle directory-upload caller, and the merged appserver ranking from #65/#67. CI is green (fmt, clippy, test, e2e on all platforms).

Both findings from d9b6645 are addressed.

Prior findings

1. P1 High — replace_if_richer: true on non-DWARF uploads vs unlabeled production records — fully addressed (rebuttal). outranks() only lets dwarf replace an unlabeled stored copy; symtab vs unlabeled is 16004, pinned by the appserver test “lets only dwarf replace a stored record that predates format_variant”. The unlabeled-FULL-then-.so.sym scenario is therefore the same skip as today. Gating the flag on dwarf would also be a regression: on a plain Android app the flag is what leaves the legacy find-or-update (no duplicate check), so an unchanged .so.sym would re-transfer on every build. The call-site comment in upload_one_so documents that.

2. P2 Medium — skip log always said “same or better” — fully addressed. AlreadyExists uses that line only when detail.stored is Some; otherwise it logs “already on server, skipped” and the run summary still carries the --force advice. Pinned by elf_upload_does_not_claim_same_or_better_when_the_server_does_not_say.

Findings

No remaining P0–P3.

Overall risk: Low

Merge recommendation: Approve

Most important issues to fix

None in this PR.

Residual, not blocking: --type rust ELF still omits both new fields, so Linux/Android Rust ELF keeps creating unlabeled format: elf documents and still needs --force for stripped → unstripped. .dynsym-only libraries are labelled symtab (symbolic’s has_symbols() is syms || dynsyms); that can miss a later true SYMBOL_TABLE upgrade, never a downgrade. Agreed as a follow-up.

Positive

Content vs name is tested (FULL packed as .so.sym still sends dwarf). Other formats leave the new keys off the wire. --force still sends overwrite. Upgrade logging / upgraded=N / zero-PUT skip paths are pinned against a protocol-accurate mock, including the old-server --force summary.

  

_{Sent by Cursor Automation: Bugsee code review}