Boilerplate for community-contributed future backends

[wlandau]

I have a strong personal interest in contributing to new future backends, particularly clustermq (#204) and AWS Batch (#423, https://github.com/wlandau/targets/issues/152). It would be great if there were an automated way to create boilerplate structure (similar to usethis::create_package() or rstantools::rstan_create_package()) and allow devs like me to easily plug in what it means for a future to initialize, resolve, and return a value. Combined with future.tests, I think this could be a powerful way to encourage new community-contributed backends.

[HenrikBengtsson]

Looong story is that I've known since the beginning that I wanted to rewrite the internals of the Future Backend API, and the original solution became more and more of a technical debt, making the burden to add new features greater and greater. Given the large number of revdep package dependencies, it's been even harder to make this internal redesign, e.g. there were packages out there that relied on some unintended behavior. Over the years I've been working with package maintainers to remove such uses, I've added more internal sanity checks preventing this, and so on. I finally felt that I'm in a position to make the move, and without all the revdep packages this would not have been possible.

So, in the next release, I'll introduce this first major redesign. The built-in future backends are already using this new Future Backend API. I've verified that future is still backward compatible with package backends that implements the old style (e.g. future.batchtools, future.callr, future.mirai, civis). I've also verified that those backend packages can easily be rewritten using the new Future Backend API.

It will be much++ easier to write new backends with the Future Backend API.

I will roll out the next release with announcing this new API. I want to get some mileage out from it (e.g. end-user real-world validation - knock on wood), because finalizing it. But the major redesign work has now been done. It also allow me to ticking of boxes on my Future Roadmap in a steady go. I'm very optimistic, and I can feel the wheels are moving faster already ...

I'll keep this issue open until I've written up a vignette on the new Future Backend API.