Simple TUF based downloader

[directionless]

Why

I've been wanting to make a download/fetch tool that validated against the TUF repo

Description

Because I wanted to support an in-memory TUF store, I didn't think the existing Library idiom was a good fit. Instead this is designed to replace the places we do simple fetches from dl.kolide.com

So this creates a simpleclient.Download does downloads as a one off. Playing around, fetching the TUF store is about 250ms, which seems okay, though I also plumbed in a way to reuse the metadata. The time savings on reusing the metastore are pretty small, I'm not it's really worthwhile.

seph@haskell launcher % ./build/launcher download --directory /tmp/o -debug --target osqueryd
{"time":"2026-03-01T16:29:54.240352Z","level":"INFO","msg":"launcher starting up","version":"1.31.6-16-gff62625d","revision":"ff62625d4fe52d5a9c07aa2f55fc3fff9e80336a"}
time=2026-03-01T16:29:54.504Z level=DEBUG msg="TUF metadata updated" duration=263.610666ms
time=2026-03-01T16:29:54.504Z level=DEBUG msg="target resolved" target_path=osqueryd/darwin/universal/osqueryd-5.21.0.tar.gz
time=2026-03-01T16:29:57.501Z level=DEBUG msg="target downloaded and verified" target_path=osqueryd/darwin/universal/osqueryd-5.21.0.tar.gz size=51361539 duration=2.997201792s
Downloaded and extracted osqueryd to: /tmp/o/osqueryd
seph@haskell launcher % ls /tmp/o                                               
osqueryd
seph@haskell launcher % ls /tmp/o/osqueryd 
osquery.app	osqueryd


# Reusing the TUF store:
seph@haskell launcher % ./build/launcher download --directory /tmp/o -debug --target osqueryd --tuf-store /tmp/t
{"time":"2026-03-01T16:30:46.300578Z","level":"INFO","msg":"launcher starting up","version":"1.31.6-16-gff62625d","revision":"ff62625d4fe52d5a9c07aa2f55fc3fff9e80336a"}
time=2026-03-01T16:30:46.547Z level=DEBUG msg="TUF metadata updated" duration=246.828625ms
time=2026-03-01T16:30:46.547Z level=DEBUG msg="target resolved" target_path=osqueryd/darwin/universal/osqueryd-5.21.0.tar.gz
time=2026-03-01T16:30:49.074Z level=DEBUG msg="target downloaded and verified" target_path=osqueryd/darwin/universal/osqueryd-5.21.0.tar.gz size=51361539 duration=2.527292333s
Downloaded and extracted osqueryd to: /tmp/o/osqueryd
seph@haskell launcher % ./build/launcher download --directory /tmp/o -debug --target osqueryd --tuf-store /tmp/t
{"time":"2026-03-01T16:30:52.50413Z","level":"INFO","msg":"launcher starting up","version":"1.31.6-16-gff62625d","revision":"ff62625d4fe52d5a9c07aa2f55fc3fff9e80336a"}
time=2026-03-01T16:30:52.620Z level=DEBUG msg="TUF metadata updated" duration=115.952958ms
time=2026-03-01T16:30:52.620Z level=DEBUG msg="target resolved" target_path=osqueryd/darwin/universal/osqueryd-5.21.0.tar.gz
time=2026-03-01T16:30:55.462Z level=DEBUG msg="target downloaded and verified" target_path=osqueryd/darwin/universal/osqueryd-5.21.0.tar.gz size=51361539 duration=2.842289792s
Downloaded and extracted osqueryd to: /tmp/o/osqueryd

[directionless]

This is the only non-whitespace change here, and I want to call it out.

Inside TUF we store our targets with both the bundle and the bare binary. I've setup this downloader to extract whatever we have in TUF, but for osquery, that's an extra directory.

I think this is fine here, but is there anywhere else it might break?

[RebeccaMahany]

Discussed with @directionless offline --

• We don't need this for spec file generation, but do want it to replace many other areas where we download binaries from dl.kolide.co (the osquery download touched in this PR, the packaging code that fetches binaries, many others)
• I was a little nervous about adding one more checklist to our long list for the go-tuf v2 upgrade
• We will revisit this PR and try it in go-tuf v2 instead!

[directionless]

Okay, I had some robots port this to go-tuf v2. An important call out, is that the robots tell me that go-tuf v2 depends on 0.7.0 of the library so we must update from 0.5.2. But also that there's no signature change, so we should be fine.

Code looks pretty clean, and I think we've got tests all over TUF. Honestly, seeing this downloader, I feel a little less scared of the eventual go-tuf 2 update.