Merge branch 'master' of github.com:FuelLabs/fuel-indexer into rashad…

…/feature/native-execution-1

Cargo.toml

@@ -3,23 +3,27 @@
 resolver = "2"
 members = [
    "examples/block-explorer/explorer-index",
+   "examples/hello-world/hello-bin",
+   "examples/hello-world/hello-index",
    "examples/simple-wasm/simple-wasm",
    "examples/native-execution/native-index",
    "examples/simple-wasm/simple",
-   "fuel-indexer-api-server",
-   "fuel-indexer-database",
-   "fuel-indexer-database/database-types",
-   "fuel-indexer-database/postgres",
-   "fuel-indexer-database/sqlite",
-   "fuel-indexer-lib",
-   "fuel-indexer-macros",
-   "fuel-indexer-plugin",
-   "fuel-indexer-schema",
-   "fuel-indexer-tests",
-   "fuel-indexer-tests/components/indices/fuel-indexer-test",
-   "fuel-indexer-tests/components/web",
-   "fuel-indexer-types",
-   "fuel-indexer",
+   "packages/fuel-indexer-api-server",
+   "packages/fuel-indexer-database",
+   "packages/fuel-indexer-database/database-types",
+   "packages/fuel-indexer-database/postgres",
+   "packages/fuel-indexer-database/sqlite",
+   "packages/fuel-indexer-lib",
+   "packages/fuel-indexer-macros",
+   "packages/fuel-indexer-metrics",
+   "packages/fuel-indexer-plugin",
+   "packages/fuel-indexer-schema",
+   "packages/fuel-indexer-tests",
+   "packages/fuel-indexer-tests/components/indices/fuel-indexer-test",
+   "packages/fuel-indexer-tests/components/web",
+   "packages/fuel-indexer-types",
+   "packages/fuel-indexer",
+   "plugins/forc-index",
 ]
 
 [profile.release]

README.md

@@ -26,5 +26,5 @@ DATABASE_URL=postgres://postgres@localhost bash scripts/run_migrations.bash
 ### Start the service
 
 ```bash
-cargo run --bin fuel-indexer -- --manifest fuel-indexer-tests/assets/fuel_indexer_test.yaml
+cargo run --bin fuel-indexer
 ```

config.yaml

@@ -32,3 +32,5 @@
 
 #   sqlite:
 #     path: database/sqlite/sqlite.db
+#
+# metrics: true

deployment/monitoring/fuel-indexer-service-monitor.yaml

@@ -0,0 +1,14 @@
+apiVersion: monitoring.coreos.com/v1
+kind: ServiceMonitor
+metadata:
+  name: fuel-indexer
+  namespace: ${k8s_namespace}
+  labels:
+    release: kube-prometheus
+spec:
+  selector:
+    matchLabels:
+      app: fuel-indexer
+  endpoints:
+  - path: /api/metrics 
+    port: http

docs/src/SUMMARY.md

@@ -10,14 +10,18 @@
     - [Configuration](./getting-started/configuration.md)
 - [Quickstart](./quickstart/index.md)
 - [Examples](./examples/index.md)
+  - [Hello World](./examples/hello-world.md)
   - [Block Explorer](./examples/block-explorer.md)
+- [Plugins](./plugins/index.md)
+  - [`forc index`](./plugins/forc-index.md)
 - [Components](./components/index.md)
   - [Assets](./components/assets/index.md)
     - [Manifest](./components/assets/manifest.md)
     - [Schema](./components/assets/schema.md)
     - [WASM Modules](components/assets/wasm.md)
   - [Database](./components/database/index.md)
     - [Data Types](./components/database/types.md)
+      - [ID Types](./components/database/ids.md)
     - [Foreign Keys](./components/database/foreign-keys.md)
   - [GraphQL](./components/graphql/index.md)
     - [Schema](./components/assets/schema.md)
@@ -27,4 +31,5 @@
   - [Indices](./components/indices/index.md)
     - [Registering Indices](./components/indices/registration.md)
 - [Indexing](./indexing/index.md)
+  - [Blocks and Transactions](./indexing/blocks-and-transactions.md)
   - [Receipts](./indexing/receipts.md)

docs/src/components/assets/manifest.md

@@ -11,6 +11,7 @@ graphql_schema: path/to/my/schema.graphql
 start_block: 1564
 module:
   wasm: path/to/my/wasm_module.wasm
+report_metrics: true
 ```
 
 ## `namespace`
@@ -47,3 +48,7 @@ module:
   - When specifying a `wasm` module, the provided path must lead to a compiled WASM binary.
 
 > Important: At this time, `wasm` is the preferred method of execution.
+
+## `report_metrics`
+
+- Whether or not to report Prometheus metrics to the Fuel backend

docs/src/components/assets/schema.md

@@ -1,8 +1,8 @@
 # GraphQL Schema
 
-GraphQL schema is a required component of the Fuel Indexer. When data is indexed into the database, the actual values that are persisted to the database will be values created using the data structures defined in the schema.
+The GraphQL schema is a required component of the Fuel indexer. When data is indexed into the database, the actual values that are persisted to the database will be values created using the data structures defined in the schema.
 
-In its most basic form, Fuel indexer GraphQL schema should have a `schema` definition that contains a defined query root. The rest of the implementation is up to you. Here's an example of a well-formed schema:
+In its most basic form, a Fuel indexer GraphQL schema should have a `schema` definition that contains a defined query root. The rest of the implementation is up to you. Here's an example of a well-formed schema:
 
 ```graphql
 schema {
@@ -26,7 +26,7 @@ type SecondThing {
 }
 ```
 
-The types you see above (e.g., `ID`, `UInt8`, etc) are Fuel abstractions that were created to more seamlessly integrate with the Fuel VM. A deeper explanation on these
-types can be found in [the Types section](./database/types.md)
+The types you see above (e.g., `ID`, `UInt8`, etc) are Fuel abstractions that were created to more seamlessly integrate with the Fuel VM and are not native to GraphQL. A deeper explanation on these
+types can be found in [the Types section](../database/types.md).
 
-> Important: These GraphQL schema types are not native to GraphQL.
+> Important: It is up to developers to manage their own unique IDs for each type, meaning that a data structure's `ID` field needs to be manually generated prior to saving it to the database. This generation can be as simple or complex as you want in order to fit your particular situation; the only requirement is that the developer implement their own custom generation. Examples can be found in the [Block Explorer](../../examples/block-explorer.md) and [Hello World](../../examples/hello-world.md) sections.

docs/src/components/assets/wasm.md

@@ -1,6 +1,6 @@
 # WASM Modules
 
-- WASM modules are compiled binaries that are registered into a Fuel indexer at runtime. The WASM bytes are read in by the indexer and _executors_ are created which will implement blocking calls the to the WASM runtime.
+- WebAssembly (WASM) modules are compiled binaries that are registered into a Fuel indexer at runtime. The WASM bytes are read in by the indexer and _executors_ are created which will implement blocking calls the to the WASM runtime.
 
 ## Usage
 
@@ -10,13 +10,17 @@ In order to compile a WASM module that you've written, you would merely:
 cd /my/index-lib && cargo build --release
 ```
 
-> Important:
->
-> 1. WASM modules are only used if the execution mode specified in your manifest file is `wasm`
->
-> 2. As of this writing, there is a small bug in newly built Fuel indexer WASM modules that produces a WASM runtime error due an errant upstream dependency. For now, a quick workaround requires using `wasm-snip` to remove the errant symbols from the WASM module. More info can be found in the related script [here](https://github.com/FuelLabs/fuel-indexer/blob/master/scripts/stripper.bash)
->
-> 3. Users on Apple Silicon macOS systems may experience trouble when trying to build WASM modules due to its `clang` binary not supporting WASM targets. If encountered, you can install a binary with better support from Homebrew (`brew install llvm`) and instruct `rustc` to leverage it by setting the following environment variables:
->
-> - `AR=/opt/homebrew/opt/llvm/bin/llvm-ar`
-> - `CC=/opt/homebrew/opt/llvm/bin/clang`
+## Notes on WASM
+
+There are a few points that Fuel indexer users should know when using WASM:
+
+1. WASM modules are only used if the execution mode specified in your manifest file is `wasm`.
+
+2. Developers should be aware of what things may not work off-the-shelf in a module: file I/O, thread spawning, and anything that depends on system libraries. This is due to the technological limitations of WASM as a whole; more information can be found [here](https://rustwasm.github.io/docs/book/reference/which-crates-work-with-wasm.html).
+
+3. As of this writing, there is a small bug in newly built Fuel indexer WASM modules that produces a WASM runtime error due to an errant upstream dependency. For now, a quick workaround requires the use of `wasm-snip` to remove the errant symbols from the WASM module. More info can be found in the related script [here](https://github.com/FuelLabs/fuel-indexer/blob/master/scripts/stripper.bash).
+
+4. Users on Apple Silicon macOS systems may experience trouble when trying to build WASM modules due to its `clang` binary not supporting WASM targets. If encountered, you can install a binary with better support from Homebrew (`brew install llvm`) and instruct `rustc` to leverage it by setting the following environment variables:
+
+- `AR=/opt/homebrew/opt/llvm/bin/llvm-ar`
+- `CC=/opt/homebrew/opt/llvm/bin/clang`

docs/src/components/database/foreign-keys.md

@@ -1,16 +1,15 @@
 # Foreign Keys
 
 - The Fuel indexer service supports foreign key constraints and relationships using a combination of GraphQL schema and a database (whether Postgres or SQLite).
-- There are two types of uses for foreign keys - _implicit_ and _explicit_
+- There are two types of uses for foreign keys - _implicit_ and _explicit_.
 
 > IMPORTANT:
 >
 > Implicit foreign keys do not require a `@join` directive. When using implicit foreign key references, merely add the referenced object as a field type (shown below). A lookup will automagically be done to add a foreign key constraint using this object's' `id` field.
 >
 > Note that implicit foreign key relationships _only_ use the `id` field on the referenced table. If you plan to use implicit foreign keys, the object being referenced _must_ have an `id` field.
 >
-> Explicit foreign keys on the other hand, _do_ require a `@join` directive. Explicit foreign key references work similarly to implicit foreign keys, however, when using explicit
-foreign key references, you must add a `@join` directive after your object type. This `@join` directive includes the field in your foreign object that you would like to reference (shown below).
+> In contrast, explicit foreign keys _do_ require a `@join` directive. Explicit foreign key references work similarly to implicit foreign keys; however, when using explicit foreign key references, you must add a `@join` directive after your object type. This `@join` directive includes the field in your foreign object that you would like to reference (shown below).
 
 - To demonstrate how the indexer uses GraphQL schema to resolve foreign key relationships, let's look at the following schema:
 

docs/src/components/database/ids.md

@@ -0,0 +1,18 @@
+# ID Types
+
+There are a few important things related to the use of IDs.
+
+> **Every GraphQL type defined in your schema file is required to have an id field.**
+>
+> - This field must be called `id`
+> - The type of this `id` field must by a `u64`
+>   - You typically want to use the `ID` type for these `id` fields
+>
+> **Why must every field have an ID?**
+>
+> Since the Fuel Indexer uses WASM runtimes to index events, an FFI is needed to call in and out of the runtime. When these calls out of the runtime are made, a pointer is passed back to the indexer service to indicate where the `id` of the type/object/entity being saved is.
+>
+> **Is this liable to change in the future?**
+>
+> Yes, ideally we'd like ID's to be of _any_ type, and we plan to work towards this in the future. 👍
+>

docs/src/components/database/index.md

@@ -10,3 +10,5 @@ The Fuel indexer uses [Postgres](https://github.com/docker-library/postgres/blob
   - How foreign keys are handled in GraphQL schema, Postgres, and SQLite
 - [Directives](./directives.md)
   - How GraphQL schema directives are translated into data-layer constraints
+- [⚠️ IDs](./ids.md)
+  - Explains some conventions surrounding the usage of `ID` types

docs/src/components/database/types.md

@@ -12,8 +12,8 @@ Below is a mapping of GraphQL schema types to their Postgres equivalents, refere
 | str[32] | AssetId | varchar(64) |
 | b256 | ContractId | varchar(64) |
 | str[32] | Salt | varchar(64) |
-| u32 | Int4 | integer |
-| u64 | Int8 | bigint |
+| u32 | UInt4 | integer |
+| u64 | UInt8 | bigint |
 | u64 | Timestamp | timestamp |
 | str[] | Blob | bytes |
 | str[32] | MessageId | varchar(64) |
@@ -40,7 +40,7 @@ type Event {
 }
 ```
 
-And finall, this GraphQL schema will generate the following Postgres schema:
+And finally, this GraphQL schema will generate the following Postgres schema:
 
 ```text
                                            Table "schema.event"