Internal precompile binaries hook by BitcoinZavior · Pull Request #22 · bitcoindevkit/bdk-dart

BitcoinZavior · 2026-01-18T00:45:16Z

Precompiled Binaries Feature

Resolves #20
This PR introduces a comprehensive precompiled binaries system that significantly improves the developer experience by reducing build times and providing secure, signed artifacts.

Key Features

Secure Distribution

Ed25519-signed precompiled binaries with public key verification
Automated signing and verification pipeline
Secure artifact hosting via GitHub Releases

Performance Improvements

Skip local Rust compilation for supported platforms
Automatic fallback to local builds when precompiled binaries unavailable
Configurable modes: auto, always, or never

Developer Experience

Zero-configuration setup for most users
Comprehensive CLI tooling for maintainers
Detailed documentation and troubleshooting guides

Configuration

Users can configure precompiled binaries in their pubspec.yaml:
Defaulting to auto which most users would want.

bdk_dart:
  precompiled_binaries:
    mode: auto  # auto | always | never

reez · 2026-01-22T17:50:07Z

This is looking nice, added a bunch of comments/questions, and if you can run dart format to fix the ci failure too.

I’ll set up the signing secret later, let me know if any other maintainer setup is needed.

.github/workflows/ci.yml

.github/workflows/precompile_binaries.yml

reez · 2026-01-22T16:36:34Z

README.md

+```
+
+`mode` controls when the precompiled path is used:
+- `auto` prefers precompiled binaries when available, otherwise builds locally


auto is described as preferring precompiled, but the hook currently disables precompiled when rustup is present (UserOptions.defaultUsePrecompiledBinaries). Should we update this wording to reflect the rustup heuristic, or change the behavior to match the doc

I suggest updating the wording in README.md. It now explicitly states that auto prefers local builds when the Rust toolchain (rustup) is detected; otherwise, it uses precompiled binaries.

reez · 2026-01-22T16:38:57Z

docs/precompiled_binaries.md

+- CI builds and uploads precompiled binaries via `.github/workflows/precompile_binaries.yml`.
+- Artifacts are tagged by the crate hash and uploaded to a GitHub release.
+- Each binary is signed with an Ed25519 key; the public key is embedded in `pubspec.yaml`.
+- The build hook tries to download a verified binary first and falls back to a local build if needed.


The doc says ‘download verified binary first and fall back’; should we note that mode: always is intended to disable fallback (once that behavior is enforced), or explicitly call out the rustup heuristic for auto?

Added a “Mode behavior” section. It clarifies that always currently falls back but is intended to fail if a verified binary isn’t present, and reiterates how never operates.

Thanks for adding the Mode behavior section + rustup heuristic callout. One mismatch remains: docs say always falls back, but implementation now throws in always on any download/verify failure (no fallback). Can we update precompiled_binaries.md (always bullet + ‘If any step fails’ line) to reflect that always fails the build and only auto falls back?

lib/src/precompiled/artifacts_provider.dart

reez · 2026-01-22T16:57:04Z

lib/src/precompiled/cli/commands/precompile_binaries.dart

+    exitCode = 2;
+    return;
+  }
+  if (repository == null || repository.trim().isEmpty) {


--repository is required but never used in any gh release calls; should we pass --repo $repository to view/create/upload (or drop the flag if it’s not needed)?

Forgot to call it out earlier: --repository was left unused in the release commands by mistake, but we’ve since wired it back in. This lets us rerun the tool against another repo (e.g., for testing or experimentation) without editing the workflow. In production it still defaults to the current repo, so the rest of the release flow behaves as before.

--repository is wired for upload + asset check, but _ensureReleaseExists still calls gh release view without --repo, and gh release create arg order is wrong (--title consumes --repo, so $repository is treated as a positional asset/pattern and the command fails). (lib/src/precompiled/cli/commands/precompile_binaries.dart:394-406)

Can we:

add --repo repository to the release view in _ensureReleaseExists

reorder create args to --repo repository --title "Precompiled binaries $crateHash" …?

reez · 2026-01-22T16:58:41Z

lib/src/precompiled/cli/commands/precompile_binaries.dart

+          '--package',
+          cratePackage,
+          '--release',
+          '--locked',


We pass --locked to cargo, but there’s no committed native/Cargo.lock in this repo; cargo build --locked will fail without it. Should we add native/Cargo.lock or remove --locked?

Lock file added and .gitignore updated to allow lock file changes.

I still don’t see a Cargo.lock committed in native/ (maybe I’m missing it), but precompile-binaries always runs cargo with --locked (both the cargo ndk … build and cargo build paths). On a clean runner, --locked will fail if there’s no lockfile to lock against.

Should we either:

commit native/Cargo.lock (preferred for reproducible precompiled artifacts + stable crate hash), --or--

drop --locked?

reez · 2026-01-22T17:01:52Z

lib/src/precompiled/crate_hash.dart

+    final output = AccumulatorSink<Digest>();
+    final input = sha256.startChunkedConversion(output);
+
+    void addTextFile(File file) {


We concatenate lines from multiple files without a delimiter or filename marker which could (rarely) lead to hash collisions if file boundaries change. Should we consider prefixing each file with its path (or a separator) before hashing to make the hash more robust?

That scenario is extremely rare isn't it? we would have to arrange contents to collide via a prefix/suffix shift. I believe a lightweight fix ( a separator before its contents) should be sufficient.

Agree it’s rare, but right now we hash line contents with no delimiter / file boundary marker (and no newline between lines), so there are theoretical collisions if boundaries shift. Could we add a lightweight separator (e.g. prefix each file with its relative path + \n, and add \n between lines) before hashing?

Not blocking, but would make the crate hash strictly more robust.

I can also make a patch for this so you dont have to

lib/src/precompiled/precompiled_builder.dart

pubspec.yaml

BitcoinZavior · 2026-01-24T06:23:05Z

Thanks for the review 🧡 PR updated to address review comments and to provide more info.

Johnosezele · 2026-02-02T22:06:52Z

lib/src/precompiled/cli/commands/precompile_binaries.dart

+    '--title',
+    '--repo',
+    repository,
+    'Precompiled binaries $crateHash',


The args for the github release crate are misordered: --title is followed by --repo, so the title becomes --repo. The command will fail (or produce a release with the wrong metadata), aborting the whole precompile step. Title value should come immediately after --title, and --repo should be its own flag.

Suggested change

'--title',

'--repo',

repository,

'Precompiled binaries $crateHash',

'--title', 'Precompiled binaries $crateHash',

'--repo', repository,

+1

this is the same underlying issue as the earlier --repository thread

Johnosezele · 2026-02-02T23:18:02Z

lib/src/precompiled/artifacts_provider.dart

+
+    DownloadedArtifact? handleFailure(String message) {
+      if (requirePrecompiled) {
+        throw PrecompiledBinaryRequiredException(message);


The documentation wordings in precompiled_binaries.md says:
"always: Attempts to use precompiled binaries and falls back to local builds if download/verification fails. Future versions may disable fallback entirely for this mode."
This can be a bit misleading, because the code throws an exception when precompiled is required. Maybe we reword as:
"Throws an exception if download/verification fails: does not fall back"

reez · 2026-02-05T18:53:23Z

Thanks for the review 🧡 PR updated to address review comments and to provide more info.

great update! I resolved the threads that were addressed, and left a few open with follow-up questions.

also if there's any of the things I mentioned that you just want me to patch myself on this pr im happy to do that.

and then also CI is currently failing on the Format check step (dart format --output=none --set-exit-if-changed …). can you run dart format lib test example bdk_demo/lib bdk_demo/test and commit the result? (looks like lib/src/precompiled/artifacts_provider.dart and test/precompiled/options_test.dart still need formatting.)

BitcoinZavior · 2026-02-19T02:06:37Z

Thanks @reez @Johnosezele
More commits added to address previous comments.
Will review and address new comments soon.
Thanks for your feedback, really appreciate it 🧡

reez · 2026-02-19T16:50:15Z

Thanks @reez @Johnosezele More commits added to address previous comments. Will review and address new comments soon. Thanks for your feedback, really appreciate it 🧡

thanks to you for the PR! will review again when you give me the go ahead that its ready for final review

BitcoinZavior added 30 commits January 13, 2026 09:00

Add core utility functions for hex encoding and HTTP retry

b7fe450

Add utility functions and target triple mapping

85dfab4

Add options and configuration parsing

feb7f85

Add crate hash calculation

e126921

Add Rust toolchain detection

9d582c6

Add Cargo build integration

1e2df77

Add artifacts provider for download and verification

6c6f974

Add precompiled builder main integration

c19cef8

Add CLI support utilities

7dab321

Add CLI framework

224d288

Add CLI commands: targets and hash

bb8ca56

Add CLI commands: gen_key and sign

ac5f174

Add CLI command: precompile_binaries

8434ef4

Add build tool executable

e2781fe

Add tests for precompiled binaries

0d197a2

Update artifact host references in tests

d7d4a3e

Add documentation for precompiled binaries

675d6e9

Add CI workflow for precompiling binaries

e9051e6

Integrate precompile workflow into main CI

518cfa5

Update build hook to use precompiled builder

7728dda

Format crate_hash.dart code

c062e86

Add dependencies and configuration to pubspec.yaml

8576a9c

Update .gitignore and README documentation

fbf3f6f

Update artifact host to bitcoindevkit/bdk-dart

e73cf33

Format rust_toolchain.dart code

8ed8ef6

Format crate_hash_test.dart code

5edfa04

Remove trailing newlines from CLI files

c572e77

Remove trailing newline from target.dart

0087a44

update readme

4e3094e

Pin Rust toolchain to 1.85.1 in CI workflow

00b0daf

reez reviewed Jan 22, 2026

View reviewed changes

BitcoinZavior added 3 commits January 22, 2026 14:30

Add exception class for required precompiled binaries

8412bc8

Update precompiled binaries documentation intro

1f98a55

Add mode behavior documentation section

f6ee437

BitcoinZavior added 3 commits January 24, 2026 09:00

Update README precompiled binaries intro

c1446c1

Update README mode behavior descriptions

cdc2fc2

Add repository parameter to GitHub CLI commands

229cdeb

Johnosezele reviewed Feb 2, 2026

View reviewed changes

BitcoinZavior added 6 commits February 13, 2026 10:00

update readme

8a8e82d

docs: state that always mode throws on failure

54394f2

docs: trim trailing newline in example

fcff476

style: format PrecompiledBinaryRequiredException.toString

d137ff9

fix: correct GitHub release create argument order

16ac553

chore: add native Cargo.lock

b55cbce

Conversation

BitcoinZavior commented Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Precompiled Binaries Feature

Key Features

Configuration

Uh oh!

reez commented Jan 22, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

BitcoinZavior commented Jan 24, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Johnosezele Feb 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

reez commented Feb 5, 2026

Uh oh!

BitcoinZavior commented Feb 19, 2026

Uh oh!

reez commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

BitcoinZavior commented Jan 18, 2026 •

edited

Loading

Johnosezele Feb 2, 2026 •

edited

Loading

reez commented Feb 19, 2026 •

edited

Loading