Skip to content

Pin device repos#818

Merged
achow101 merged 23 commits intobitcoin-core:masterfrom
Sjors:2026/01/pin
Jan 29, 2026
Merged

Pin device repos#818
achow101 merged 23 commits intobitcoin-core:masterfrom
Sjors:2026/01/pin

Conversation

@Sjors
Copy link
Member

@Sjors Sjors commented Jan 19, 2026

Time to stop the whack-a-mole game.

Pin firmware and simulator versions to the latest releases known to work. We or the device makers can bump them as needed.

This approach seems more suitable for the lower pace of development on this repo.

Superseedes #812 (takes most of its commits)

Trezor switched to uv for dependency management
Recent versions of Speculos no longer support the Nano S. Additionally,
we build the legacy app for the Nano S separately now.
lwip is a submodule of many device firmware repos, but the default URLs
often fail to be cloned. Rewrite those URLs with git config to use the
Github mirror to avoid that issue.
Instead of listing out the dependencies required to install manually,
just install from speculos itself.
@Sjors Sjors force-pushed the 2026/01/pin branch 10 times, most recently from 1bea008 to 1f98eab Compare January 19, 2026 18:45
@Sjors
Copy link
Member Author

Sjors commented Jan 19, 2026

I'm not sure if pinning Ledger to 2.4.1 instead of 2.4.4 was actually necessary.

I reverted the Ledger changes in 0fe1dbb because these tests are still failing for me. It looks like they did pass on PR 812, though it's a mystery to me how.

@Sjors Sjors mentioned this pull request Jan 19, 2026
@Sjors
Copy link
Member Author

Sjors commented Jan 20, 2026

Ledger is green now, that's good. ColdCard is still very grey (timeout) and a couple of bitbox02 and jade tests timeout too.

@Sjors
Copy link
Member Author

Sjors commented Jan 20, 2026

Attemping an upgrade for ColdCard and a downgrade for Jade.

@Sjors Sjors force-pushed the 2026/01/pin branch 2 times, most recently from 38e4daf to ff11530 Compare January 20, 2026 12:38
@Sjors
Copy link
Member Author

Sjors commented Jan 20, 2026

ColdCard is green now, though mainly because I disabled the send test again.

The Jade tests for Wheel and SDist are timing out on the first test (test_backup). I'm adding a timeout to simulator launch so it fails faster. I'm also trying an older version.

The bitbox02 library tests timeout on Python 3.10 and 3.12 (but not 3.9 and 3.11). For now I just added a 5 minute timeout so it fails faster.

@Sjors
Copy link
Member Author

Sjors commented Jan 21, 2026

Although I couldn't reproduce the exact same error, based on local testing I suspect that fixing #817 will fix the Jade Sdist and Wheel builds. Our poetry.lock pinned an earlier version, but those two jobs are not restrained by that so they picked the newer "broken" version. Added a commit.

@Sjors
Copy link
Member Author

Sjors commented Jan 21, 2026

The error message mentioned in the commit description of a0acccb is a red herring. It was a classic case of CI only printing stderr if a test fails (if: failure()). I'll update the comment in a later push.

achow101 and others added 11 commits January 21, 2026 16:42
These don't seem to work anymore.
The getkeypool command can return an error dict instead of a list
when the underlying operation fails. Add assertions to catch this
early with a clear error message, rather than having the test fail
with a confusing JSONRPCException about wrong type passed to
importdescriptors.
This pins all device firmware versions to ensure reproducible CI builds.
Pin app-bitcoin-new to v2.4.4 (tag stax_1.9.0_2.4.4_sdk_...) since
newer versions may be incompatible with Speculos v0.25.10 which is
the last version supporting Python 3.9.

Use shallow clone (--depth 1) for both Ledger apps to speed up builds.
v5.4.3 causes test_backup to hang because it lacks the
'multiprocess simulator' fix merged in June 2025.
If the Jade emulator doesn't respond within 5 minutes, fail fast
with an error message instead of hanging until the job timeout.
Wrap the Bitbox02Client initialization and restore_device() call
in a thread with a 5 minute timeout. This prevents CI from hanging
for 45 minutes when the simulator startup blocks indefinitely.
In the Sdist and Wheel test environments, the Jade tests all timed out.

The other tests used a fixed version 5.6 of cbor2 which is why they
were not affected.

Fixes bitcoin-core#817
@Sjors
Copy link
Member Author

Sjors commented Jan 21, 2026

Hooray, that actually worked!

Pushing the branch again, this time dropping 0fe1dbb test: Re-enable disabled tests from #812 dropped (along with its revert). We can try to revive tests some other time.

I do still notice some occasional bitbox02 and ColdCard timeouts, but they don't seem to be systematic.

Ready for review.

@Sjors Sjors marked this pull request as ready for review January 21, 2026 15:47
@Sjors
Copy link
Member Author

Sjors commented Jan 21, 2026

Mmm, again two timed out bitbox02 jobs. Not sure what to do about those.

@Sjors
Copy link
Member Author

Sjors commented Jan 22, 2026

@benma any idea why these BitBox02 jobs time out? It seems to be a random selection of about two per push.

Example log: https://github.com/bitcoin-core/HWI/actions/runs/21216007510/job/61038515090?pr=818

2026-01-21T16:04:40.3083670Z CONFIRM TRANSACTION FEE SCREEN START
2026-01-21T16:04:40.3083895Z AMOUNT: 2.00000000 TBTC
2026-01-21T16:04:40.3084071Z FEE: 0.00005960 TBTC
2026-01-21T16:04:40.3084258Z CONFIRM TRANSACTION FEE SCREEN END
2026-01-21T16:04:40.3084483Z STATUS SCREEN START
2026-01-21T16:04:40.3084655Z TITLE: Transaction
2026-01-21T16:04:40.3084828Z confirmed
2026-01-21T16:04:40.3084981Z STATUS SCREEN END
2026-01-21T16:04:40.3085155Z Socket connection closed
2026-01-21T16:04:40.3085541Z Waiting for new clients, CTRL+C to shut down the simulator
2026-01-21T16:04:40.3085764Z 
2026-01-21T16:04:40.3085837Z Got signal, exiting
2026-01-21T16:04:40.3086014Z ok
2026-01-21T16:48:21.5925473Z context canceled

Note the 44 minute gap after "ok" before CI pulls the plug.

@achow101
Copy link
Member

It looks like the timeout is on stopping the simulator, so we could put a timeout on the wait() and then kill() the process if it fails to stop within the timeout.

@Sjors
Copy link
Member Author

Sjors commented Jan 23, 2026

BitBox02 tests are now all green.

@Sjors
Copy link
Member Author

Sjors commented Jan 27, 2026

#820 should reduce the occurance of spurious ColdCard failures.

@achow101
Copy link
Member

ACK cafc8b0

@achow101 achow101 merged commit 2ea8b78 into bitcoin-core:master Jan 29, 2026
232 of 233 checks passed
achow101 added a commit that referenced this pull request Jan 30, 2026
bc7f751 ci, coldcard: clone once (Sjors Provoost)

Pull request description:

  Based on #818.

ACKs for top commit:
  achow101:
    ACK bc7f751

Tree-SHA512: 44dc7e1aee33ece6b5438dcfa15aa3010657fc2f9b2f28f8c8706bff379ecd909b9e22483e40ec81c543845fcdaad14e8d9a2f09e8d2b12f24f08d3aa29c4230
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants