Added proving time and instructions

Author: hdvanegasm

test_suite/README.md

@@ -1,13 +1,33 @@
 # Test suite for zk-regex + Noir
 
+This test-suite is a tool for testing and benchmarking the zk-regex library module that generates Noir code to check if certain regex is matched.
+
 ## Requirements
 
 - Install zk-regex command following the instructions in the documentation.
 - Install Noir.
+- Install the `hyperfine` tool.
 
 ## How to run
 
-### Database
+To execute the testing and/or the benchmarking, you need to fill a JSON database that will be used for the testing and/or benchmarking. The testing and the benchmarking are two separate processes. This means that you may execute the testing without the benchmarking, or the benchmarking without the testing, or execute both the testing and the benchmarking. However, both processes will use the same database for their ends. It is important to add that if you want to execute just the testing or just the benchmarking, some of the fields are not mandatory as we will explain next. The basic structure for the database is as follows:
+
+```json
+{
+  "database": [
+    <sample>,
+    <sample>,
+    <sample>,
+  ]
+}
+
+```
+
+where `<sample>` is a JSON object with certain required fields. The instructions to fill the database are presented below.
+
+## Instructions for testing
+
+Each sample for testing can be one of two types: raw, or decomposed. "Raw" means that the regex is specified as just one string, while "decomposed" means that the regex is specified in fragments according to the zk-regex specification. Below you will find an example of how to specify a database with both types of specification. It is important to notice that the database can have samples with a mixture of both types: some samples may be "raw", and other samples may be "decomposed".
 
 ```json
 {
@@ -51,14 +71,63 @@
     ]
 }
 ```
-### Execute tests
 
+To execute the tests, you simply run.
+
+```bash
+RUST_LOG=info cargo run -- -t
+```
+
+## Instructions for benchmarking
+
+This tool allows you to benchmark the source code to evaluate the performance. The benchmarking requires to add additional flags and information to the database presented in the previous section. It is important to make clear that if you just want to execute the test, the modifications to the database associated with the benchmark **are not mandatory**.
+
+The benchmarking tool supports the following features:
+
+- If you want to benchmark all the samples, you can add the `"bench_all": true` to the database as follows:
+
+  ```json
+  {
+    "bench_all": true,
+    "database": [
+      <sample>,
+      <sample>,
+      <sample>,
+    ]
+  }
+  ```
+
+  This flag is optional.
+- You may want to execute benchmarks not for all the samples but for some of them. If you want to execute the benchmark for some of the samples but not for all, you can add the flag `"with_bench": true` to benchmark that sample. This flag is considered optional, and if you do not add it, the default value will be considered as `false`.
+- There are two types of benchmarking:
+  - *without time*: this mode of benchmarking means that the report will include just the gate-count of the circuit generated by the zk-regex library, but this mode does not measure the proving time. To execute the benchmarking in this mode you must execute the following command:
+
+    ```bash
+    RUST_LOG=info cargo run -- [OPTIONS] no-time
+    ```
+
+  - *with time*: this mode of benchmarking measures the gate-count as in the previous mode, and also measures the proving time of 5 executions of the Noir project. To execute this you need to add the field `"benchmark_str"` with a string that must successfully match the regex. This string will be used as the witness for the prove. If you **do not** want to benchmark the proving time, you can omit this field in the database. To run the benchmarking *with time*, you must execute the following command:
+
+    ```bash
+    RUST_LOG=info cargo run -- [OPTIONS] with-time
+    ```
+
+If you want to execute the benchmarking **without** the testing, you can omit the `-t` flag in the execution command as shown below:
+
+```bash
+RUST_LOG=info cargo run -- <no-time | with-time>
 ```
-$ RUST_LOG=info cargo run
+
+## Execution of testing and benchmarking simultaneously
+
+If you want to execute both the testing and the benchmarking you need to follow the instructions presented above for the benchmarking and the testing independently. Then you can execute the following command:
+
+```bash
+RUST_LOG=info cargo run -- -t <no-time | with-time>
 ```
 
 ## Limitations
 
-For some regexes the random sampling is not possible, because the sampling library is limited. For example the end anchor (`$`) is not supported. 
+For some regexes the random sampling is not possible, because the sampling library is limited. For example the end anchor (`$`) is not supported.
 
-Random sample testing for the `gen_substrs` setting is only support for `decomposed`. In the `raw` setting, the substrings are determined via a json file that contains the transition information. Determining what the substring parts are, would be quite involved since it requires building the DFA. 
+Random sample testing for the `gen_substrs` setting is only support for `decomposed`. In the `raw` setting, the substrings are determined via a json file that contains the transition information. Determining what the substring parts are, would be quite involved since it requires building the DFA.

test_suite/bench_result.csv

@@ -1,3 +1,2 @@
-acir_opcodes,circuit_size,regex,with_gen_substr
-114,17865,email was meant for @[a-z]+,true
-59,7808,1=(a|b) (2=(b|c)+ )+d,true
+acir_opcodes,circuit_size,regex,with_gen_substr,proving_time
+43,3175,(a|b)+c,false,0.18447697588

test_suite/decomposed.json

@@ -1 +1 @@
-{"parts":[{"is_public":false,"regex_def":"ab"},{"is_public":true,"regex_def":"cd"}]}
+{"parts":[{"is_public":false,"regex_def":"(\\r\\n|^)subject:"},{"is_public":true,"regex_def":"[^\\r\\n]+"},{"is_public":false,"regex_def":"\\r\\n"}]}

test_suite/execution_project/Prover.toml

@@ -1 +1 @@
-"input" = [109, 98, 45, 99, 99, 100, 99, 100, 100, 100, 100, 99, 100, 100, 99, 99]
+input = [97, 98, 97, 97, 98, 98, 97, 98, 97, 99]