Added new testcases that were added to circom tests. Concretely: a few new testcases that are added to the database file and a change in circuits and expansion of tests for both hardcoded test projects (from_addr and to_addr);

Also added more labels to the database file `regex_db_for_bench` so in the future they can be easily compared + explanation in the test_suite/README.

Author: ewynx

test_suite/README.md

@@ -2,6 +2,8 @@
 
 This test-suite is a tool for testing and benchmarking the zk-regex library module that generates Noir code to check if certain regex is matched.
 
+In addition, all the tests for which the circom implementation of zk-regex gets tested are implemented. A large part can be ran automatically through the automated testing/benchmarking functionality for which a database has to be provided (and `regex_db_for_bench.json` contains all the necessary testdata) and there are 2 testcases that have been implemented separately in `hardcoded_tests`.  The latter is necessary because for those tests multiple circuits have to be combined manually. 
+
 ## Requirements
 
 - Install zk-regex command following the instructions in the documentation.
@@ -126,8 +128,49 @@ If you want to execute both the testing and the benchmarking you need to follow
 RUST_LOG=info cargo run -- -t <no-time | with-time>
 ```
 
+## Circom testing compatibility
+
+The file `regex_db_for_bench.json` contains all testcases that the [circom implementation tests](https://github.com/zkemail/zk-regex/tree/main/packages/circom/tests) for and some additional ones. 
+
+The circom tests are contained in `.test.js` files and usually test for various circuits within a single file. In `regex_db_for_bench.json` the information is organized per circuit, but we've added information to link them back to the circom tests. If there's a specific circuit file it relates to, this is indicated with `circuit_name`, otherwise there is a reference in `test_name`. Examples: 
+```json
+"circuit_name": "asterisk1"
+
+"test_name": "simple_regex.test.js"
+``` 
+
+Furthermore, to relate the test inputs to the specific testcases, `circom_testname` is indicated for passing samples, and separately in an array for failing samples. E.g:
+
+```json
+      "samples_pass": [
+        { 
+          "input": "xb", 
+          "expected_substrings": [],
+          "circom_testname": "asterisk1 valid case 1"
+        },
+        { 
+          "input": "xab", 
+          "expected_substrings": ["a"],
+          "circom_testname": "asterisk1 valid case 2"
+        }
+      ],
+      "samples_fail": [
+        "xaaa",
+        "aaabx"
+      ],
+      "circom_testnames_fails": [
+        "asterisk1 invalid case 1",
+        "asterisk1 invalid case 2"
+      ]
+```
+
+These labels make it easier to verify whether all circom tests have been implemented. 
+
+Note that as mentioned in the introduction, there are a few tests that are implemented in `hardcoded_tests` as they combine multiple circuits, which cannot be done through the automated process. 
+
 ## Limitations
 
 For some regexes the random sampling is not possible, because the sampling library is limited. For example the end anchor (`$`) is not supported.
 
 Random sample testing for the `gen_substrs` setting is only support for `decomposed`. In the `raw` setting, the substrings are determined via a json file that contains the transition information. Determining what the substring parts are, would be quite involved since it requires building the DFA.
+

test_suite/bench_result.csv

@@ -1,2 +1,37 @@
-acir_opcodes,circuit_size,regex,with_gen_substr,proving_time
-43,3175,(a|b)+c,false,0.18447697588
+acir_opcodes,circuit_size,regex,with_gen_substr
+779,8828,"x[0-9]{2}-y{2,3}_z$",false
+801,19600,email was meant for @[a-z]+,false
+788,9603,1=(a|b) (2=(b|c)+ )+d,false
+781,4220,xa*b,false
+780,3451,ab*,false
+781,4220,a(x|y)*b,false
+782,4221,^a,false
+782,4221,^(a|b|c),false
+782,4221,(^|a)b+,false
+783,4990,(\n|^)x(a|b)+,false
+790,10373,(\n|^)x[^abc]+,false
+772,3445,a[bc]$,false
+775,4984,(\n|^)xa[bc]$,false
+787,8834,.,false
+789,10372,a.b,false
+820,34211,Latin-Extension=[¡-ƿ]+ Greek=[Ͱ-Ͽ]+ Cyrillic=[Ѐ-ӿ]+,false
+827,39594, Arabic=[؀-ۿ]+ Devanagari=[ऀ-ॿ]+ Hiragana&Katakana=[ぁ-ヿ]+,false
+786,8065,( )?(c|C)ode( )?(0|1|2|3|4|5|6|7|8|9|a|b|c|d|e|f)+,false
+790,11141,a:[^abcdefghijklmnopqrstuvwxyz\.]+\.,false
+787,8834,[^ab],false
+781,4220,a+b,false
+781,4220,a(b|c)+,false
+782,4989,a(bc)+,false
+784,6527,(12|345)+b,false
+781,4220,a?b,false
+783,5758,(1x?2)+b,false
+783,5758,12(a|b)?c,false
+782,4989,aba,false
+781,4220,a[ab],false
+802,20369,email was meant for @[a-zA-Z0-9_]+\.,false
+782,4989,[A-Za-z0-9!#$%&'*+=?\-\^_`{|}~./@]+@[A-Za-z0-9.\-]+,false
+782,4989,[A-Za-z0-9!#$%&'*+=?\-\^_`{|}~./]+@[A-Za-z0-9.\-@]+,false
+798,16525,(\r\n|^)message-id:<[A-Za-z0-9=@\.\+_-]+>\r\n,false
+800,18063,(\r\n|^)subject:[^\r\n]+\r\n,false
+813,28060,(\r\n|^)dkim-signature:([a-z]+=[^;]+; )+t=[0-9]+;,false
+814,28829,(\r\n|^)dkim-signature:([a-z]+=[^;]+; )+bh=[a-zA-Z0-9+/=]+;,false

test_suite/hardcoded_tests/from_addr/README.md

@@ -2,24 +2,24 @@
 
 This test for the circom code uses a circuit that includes 3 templates. The 3 templates have been generated with decomposed regex information (see `/decomposed`):
 - "from_all"
-- "email_addr_with_name"
+- "reversed_bracket"
 - "email_addr"
 
-They are combined as follows: "from_all" extracts a substring `s`. Then `s` is used as the input for both "email_addr_with_name" and "email_addr".
+They are combined as follows: "from_all" extracts a substring `s`. The reversed version of `s` is used as the input for "reversed_bracket" and the standard version as input for "email_addr".
 
-- "email_addr_with_name" matches an email address between `<>` 
+- "reversed_bracket" matches an email address between `<>`. Since the input is the reversed string, it take into account the last email that pops up between `<>`
 - "email_addr" only matches the email address
 
-If "email_addr_with_name" found an email address, that one is returned. Otherwise the email address found with "email_addr" is returned. If nothing was found, the match fails. 
+If "reversed_bracket" found an email address, that one is reversed and returned. Otherwise the email address found with "email_addr" is returned. If nothing was found, the match fails. 
 
 ## Recreate templates
 
 ```
 zk-regex decomposed -d decomposed/from_all.json --noir-file-path src/from_all.nr -g true
 
-zk-regex decomposed -d decomposed/email_addr_with_name.json --noir-file-path src/email_addr_with_name.nr -g true
+zk-regex decomposed -d decomposed/reversed_bracket.json --noir-file-path src/reversed_bracket.nr -g true
 
 zk-regex decomposed -d decomposed/email_addr.json --noir-file-path src/email_addr.nr -g true
 ```
 
-Note that `email_addr_with_name.nr` and `email_addr.nr` have to be adjusted to return a bool instead of fail the assertion if they don't match the regex. Otherwise we can't execute both functions and see which one returns a substring. 
+Note that `reversed_bracket.nr` and `email_addr.nr` have to be adjusted to return a bool instead of fail the assertion if they don't match the regex. Otherwise we can't execute both functions and see which one returns a substring.