feature: native execution

[ra0x3]

Background

• This is a sort've skunkworks idea
• Currently the indexer only supports WASM execution -- which is great for running untested code and allowing users to upload bundles of assets to a remote index operator in order to not have to run their own node
• However, WASM execution can be a little frustrating and unnecessary in some contexts:
  • If I'm not running untrusted code, I don't need that benefit of WASM
  • If I'm not packaging my indices as bundles and deploying them to a remote service, I don't need that benefit of WASM
  • There are so so many cool/interesting crates that simply don't compile to WASM (e.g., Redis, RocksDB, some websocket crates, etc)
• WASM is great for the current use case and we shouldn't change anything about WASM execution
  • WASM should still be the main frame of reference when thinking about the indexer
• However, if I'm a Rust dev that wants more custom functionality - I don't want to be limited to just WASM execution

Proposal

• I propose that we add a new method of execution "native"
• Native execution would be similar to WASM execution in that it is an execution method for indexing data, however there would be a few differences

Implementation

• Previously I tried making WASM/native almost exactly the same Add native execution #359
  • This was great in that the APIs looked the exact same to the dev
  • However, persisting data to the database becomes an issue using this method of native execution -- WASM execution caches the database connection in the WASM runtime, unlike native execution where we don't really have anywhere to stash the database connection
    • I'm open to revisiting this implementation if anyone has any ideas for it
• The current implementation I suggest is that users run "native execution" as a simple binary that registers executors
  • The Fuel indexer service is invoked via the fuel-indexer binary.
  • This binary just creates and registers executors via a supplied manifest file
  • Native execution would just require running your own binary where you create your own executors and register them -- super simple
• Native execution would not require any "new" functionality
  • The functionality for native execution already exists.
  • What we would do is merely show users how to user the indexer API in order to make the service execution run "natively"