Key off an external file's modtime

[dimo414]

I'm not sure how common/useful this would really be, but I can imagine wanting to use a file's modtime as an input to a command's cache key (alongside the cwd or environment). As a toy example:

$ bkt --modtime infrequent_updates.json --ttl=1d -- jq -f infrequent_updates.json '{foo: .bar}'

This assumes some occasional background job writes updates to infrequent_updates.json. We can generally rely on the cached data, so a long TTL is preferable, but we want to pick up updates as quickly as possible.

Users can accomplish this today using --scope like so:

$ bkt --scope="$(stat -c '%n:%Y' infrequent_updates.json)" --cwd --ttl=1d -- jq -f infrequent_updates.json '{foo: .bar}'

But this means launching two additional processes (the command substitution subprocess and stat) on each call. Doing a modtime check in-process would be much lighter weight.

[W1M0R]

Thanks for documenting your workaround @dimo414. I'm considering bkt for a simple make-like replacement that can perform builds based on a list of dependencies. Detecting whether the dependencies have changed (i.e. file modification or file hash) would be very helpful in this regard. Another option would be to specify the type of a positional argument (e.g. --0=file). bkt can then use that information to perform change detection via modtime, hash, etc.

[dimo414]

Thanks for the input! Extracting dependencies from the command's arguments is a really interesting idea, but getting it right+robust might be more heavyweight than bkt could reasonably implement. Have you taken a look at Bazel for this use-case? It's great for incremental builds, and way more flexible and powerful than make.

That said, please share anything you come up with using bkt! I'd be very curious to see it and it would definitely help inform this feature request.

[dimo414]

This works now and will be released as part of 0.6.0.