[Investigate] REST API `/v2/blocks/{rnd}` returns invalid json

[algorandskiy]

Subject of the issue

If block contains app state change, it is incorrectly serialized to json - EvalDelta.LocalDeltas has non-quoted integer keys.
Seems like protocol.EncodeJSONStrict needs to be used.

Your environment

• Software version: algod -v

algod -v
8590155987
2.2.90323.master [master] (commit #e8a24ee9+)
go-algorand is licensed with AGPLv3.0

Steps to reproduce

Use newest pingpong with updated apps test (see #1700) to create blocks with local deltas. Then iterate over blocks and observe json decoding error.

goal network create -t test/testdata/nettemplates/TwoNodes50EachFuture.json -r ~/networks/two-fut -n two-fut

goal network start -r ~/networks/two-fut/
sleep 2
pingpong run -d ~/networks/two-fut/Primary --numapp 100 --appprogops 300 --numappoptin 10 --appproggk 10 --appproglk 10 --tps 1000 --rest 0  --refresh 1 --numaccounts 500 --duration 60 --minaccount 34000000
sleep 30
ROUND=$(curl -k http://$(cat ~/networks/two-fut/Primary/algod.net)/v2/status -H "X-Algo-API-Token: $(cat ~/networks/two-fut/Primary/algod.token)" | jq '.["last-round"]')
for i in $(seq 1 $ROUND); do curl -k http://$(cat ~/networks/two-fut/Primary/algod.net)/v2/blocks/$i -H "X-Algo-API-Token: $(cat ~/networks/two-fut/Primary/algod.token)" | python3 -m json.tool || break; done
goal network stop -r ~/networks/two-fut/
# goal network delete -r ~/networks/two-fut

Expected behaviour

No parsing error

Actual behaviour

Expecting property name enclosed in double quotes: line 11963 column 13 (char 397656)

[nullun]

I just stumbled upon this error myself. Whilst attempting to decode a response from testnet my code failed due to invalid JSON being returned. Here's a snippet which appears to be the problem "Expected STRING got NUMBER, line 3: 0: {

...
"dt": {
          "ld": {
            0: {
              "key-one": {
                "at": 2,
                "ui": 1
              },
              "key-two": {
                "at": 2,
                "ui": 1
              }
            }
          }
        },
...

I can't add much more info as the original post contained a good explanation. I'm just wondering if this is still on the radar as I'd like to use this functionality.

Edit:
Just found an example of someone elses application using local data on the testnet and getting the same error from the API.
https://testnet.algoexplorerapi.io/v2/blocks/14003785

Exact same issue here.

[fabrice102]

If you hit this issue, a workaround is to use the msgpack format, argument ?format=msgpack
The output will however be larger most likely as the msgpack format includes the certificate for the block.

For the Go SDK and the Python SDK, you need to use BlockRaw and block_raw respectively.
The Java and JS SDK use the msgpack format by default.

Hi @fabrice102 thanks for the workaround.
We have a working implementation using JSON, so using msgpack instead would require to re-write all the parsing logic. But we are considering a rework using the JS SDK indeed.

Are there any plans to fix the JSON issue though? The root cause is the keys of that "ld" object being numbers. This is not valid JSON per the spec, so converting these keys to strings would probably make sense.