Skip to content

Commit

Permalink
[doc] Document our serialization manifest generator
Browse files Browse the repository at this point in the history
  • Loading branch information
@huitseeker
huitseeker committed Mar 2, 2022
1 parent 0da6369 commit 7cc5b38
Showing 1 changed file with 66 additions and 0 deletions.
66 changes: 66 additions & 0 deletions sui_core/tests/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
## What this test is about

This repository checks the conformance of our code to a BCS-compatible manifest of our serialized data formats.

It does this by running a manifest generator from the code (using serde-reflection) and checking the output has not changed.

If it has in a legitimate fashion (e.g. we update one of our main types), all that's left to do is to re-run the generator and check in the change.

Here are the references to the software above:
https://github.com/diem/bcs
https://github.com/novifinancial/serde-reflection

## Examples

In this example, we will update one of our core types (SuiError), and then update the manifest:

```
huitseeker@Garillots-MBP.localdomain➜~/tmp/fastnft(main)» git checkout main [7:40:40]
Already on 'main'
Your branch is up to date with 'origin/main'.
huitseeker@Garillots-MBP.localdomain➜~/tmp/fastnft(main)» ruplacer --subvert 'CertificateAuthorityReuse' 'CertificateAuthorityDuplicate' --go [8:42:33]
./sui_types/src/error.rs:103 - CertificateAuthorityReuse,
./sui_types/src/error.rs:103 + CertificateAuthorityDuplicate,
./sui_types/src/messages.rs:610 - SuiError::CertificateAuthorityReuse
./sui_types/src/messages.rs:610 + SuiError::CertificateAuthorityDuplicate
./sui_types/src/messages.rs:638 - SuiError::CertificateAuthorityReuse
./sui_types/src/messages.rs:638 + SuiError::CertificateAuthorityDuplicate
./sui_core/tests/staged/sui.yaml:390 - CertificateAuthorityReuse: UNIT
./sui_core/tests/staged/sui.yaml:390 + CertificateAuthorityDuplicate: UNIT
Performed 4 replacements on 196 matching files
```

Now our code is modified in a way that will make the format test fail: let's update the manifest.

```
huitseeker@Garillots-MBP.localdomain➜~/tmp/fastnft(main✗)» cd sui_core [8:43:38]
huitseeker@Garillots-MBP.localdomain➜tmp/fastnft/sui_core(main✗)» cargo -q run --example generate-format -- print > tests/staged/sui.yaml
```


Let's check that we pass the test again:
```
huitseeker@Garillots-MBP.localdomain➜tmp/fastnft/sui_core(main✗)» cargo test format 2>&1 |tail -n 40 [8:47:22]
Finished test [unoptimized + debuginfo] target(s) in 0.35s
Running unittests (/Users/huitseeker/tmp/fastnft/target/debug/deps/sui_core-5796871991341984)
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 52 filtered out; finished in 0.00s
Running tests/format.rs (/Users/huitseeker/tmp/fastnft/target/debug/deps/format-ecdfa91a67810be3)
running 1 test
Finished dev [unoptimized + debuginfo] target(s) in 0.20s
Running `target/debug/examples/generate-format test`
test test_format ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.48s
huitseeker@Garillots-MBP.localdomain➜tmp/fastnft/sui_core(main✗)» git status -s [8:47:38]
M tests/staged/sui.yaml
M ../sui_types/src/error.rs
M ../sui_types/src/messages.rs
```

1 comment on commit 7cc5b38

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bench results

�[0m�[0m�[1m�[32m Finished�[0m release [optimized] target(s) in 0.34s
�[0m�[0m�[1m�[32m Running�[0m target/release/bench
�[2m2022-03-02T18:03:31.141351Z�[0m �[32m INFO�[0m �[2mbench�[0m�[2m:�[0m Starting benchmark: TransactionsAndCerts
�[2m2022-03-02T18:03:31.141379Z�[0m �[32m INFO�[0m �[2mbench�[0m�[2m:�[0m Preparing accounts.
�[2m2022-03-02T18:03:31.141810Z�[0m �[32m INFO�[0m �[2mbench�[0m�[2m:�[0m Open database on path: "/tmp/DB_9FD737A4B7A9C7839D1A78D0703A86F600475786"
�[2m2022-03-02T18:03:36.196571Z�[0m �[32m INFO�[0m �[2mbench�[0m�[2m:�[0m Preparing transactions.
�[2m2022-03-02T18:03:44.748842Z�[0m �[32m INFO�[0m �[2msui_network::transport�[0m�[2m:�[0m Listening to TCP traffic on 127.0.0.1:9555
�[2m2022-03-02T18:03:45.750257Z�[0m �[32m INFO�[0m �[2mbench�[0m�[2m:�[0m Number of TCP connections: 2
�[2m2022-03-02T18:03:45.750290Z�[0m �[32m INFO�[0m �[2mbench�[0m�[2m:�[0m Set max_in_flight to 500
�[2m2022-03-02T18:03:45.750295Z�[0m �[32m INFO�[0m �[2mbench�[0m�[2m:�[0m Sending requests.
�[2m2022-03-02T18:03:45.756268Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m Sending TCP requests to 127.0.0.1:9555
�[2m2022-03-02T18:03:45.762163Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m Sending TCP requests to 127.0.0.1:9555
�[2m2022-03-02T18:03:46.609569Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 5000 packets
�[2m2022-03-02T18:03:47.381877Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 35000
�[2m2022-03-02T18:03:47.394282Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 35000
�[2m2022-03-02T18:03:47.461958Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 10000 packets
�[2m2022-03-02T18:03:48.318592Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 15000 packets
�[2m2022-03-02T18:03:49.077103Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 30000
�[2m2022-03-02T18:03:49.132259Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 30000
�[2m2022-03-02T18:03:49.171849Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 20000 packets
�[2m2022-03-02T18:03:50.031604Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 25000 packets
�[2m2022-03-02T18:03:50.812105Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 25000
�[2m2022-03-02T18:03:50.832698Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 25000
�[2m2022-03-02T18:03:50.883475Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 30000 packets
�[2m2022-03-02T18:03:51.738594Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 35000 packets
�[2m2022-03-02T18:03:52.493199Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 20000
�[2m2022-03-02T18:03:52.533783Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 20000
�[2m2022-03-02T18:03:52.593658Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 40000 packets
�[2m2022-03-02T18:03:53.446941Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 45000 packets
�[2m2022-03-02T18:03:54.164680Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 15000
�[2m2022-03-02T18:03:54.290159Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 15000
�[2m2022-03-02T18:03:54.303183Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 50000 packets
�[2m2022-03-02T18:03:55.157914Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 55000 packets
�[2m2022-03-02T18:03:55.922014Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 10000
�[2m2022-03-02T18:03:55.978459Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 10000
�[2m2022-03-02T18:03:56.018597Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 60000 packets
�[2m2022-03-02T18:03:56.875548Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 65000 packets
�[2m2022-03-02T18:03:57.610249Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 5000
�[2m2022-03-02T18:03:57.678531Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m In flight 500 Remaining 5000
�[2m2022-03-02T18:03:57.731975Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 70000 packets
�[2m2022-03-02T18:03:58.591540Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 75000 packets
�[2m2022-03-02T18:03:59.426005Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m Done sending TCP requests to 127.0.0.1:9555
�[2m2022-03-02T18:03:59.516377Z�[0m �[32m INFO�[0m �[2msui_core::authority_server�[0m�[2m:�[0m 127.0.0.1:9555 has processed 80000 packets
�[2m2022-03-02T18:03:59.562554Z�[0m �[32m INFO�[0m �[2msui_network::network�[0m�[2m:�[0m Done sending TCP requests to 127.0.0.1:9555
�[2m2022-03-02T18:03:59.562902Z�[0m �[32m INFO�[0m �[2mbench�[0m�[2m:�[0m Received 80000 responses.
�[2m2022-03-02T18:03:59.751372Z�[0m �[33m WARN�[0m �[2mbench�[0m�[2m:�[0m Completed benchmark for TransactionsAndCerts
Total time: 13812595us, items: 40000, tx/sec: 2895.907684254841

Please sign in to comment.