Invalid JSON RPC response: ""

[ppoliani]

Description

Sending many transactions to the RPC endpoint doesn't always end up running smoothly. I have a script that concurrently sends 1000-5000 transactions. The challenge in this case is to get the nonce right; so I've implemented a local nonce tracker that periodically re-syncs with the remote nonce.

The issue is that some of my transactions return:

Invalid JSON RPC response: ""

I use web3.js to send the transactions and more specifically the following function:

web3.eth.sendSignedTransaction(signedTx)
  .once('transactionHash', onData)
  .on('error', onError));

I find it hard to understand that the meaning of that error. I understand that regarding the nonce management. three thing can go wrong:

1. Send a tx with a low nonce.
2. Send a tx with the same nonce as a previously submitted tx
3. Send a tx with a higher nonce than the current account nonce.

In the first situation, typically I should get Nonce too low. Second case should emit replacement transaction underpriced. Now regarding the third case, I'm not sure if it will cause Invalid JSON RPC response: "". I know that it will result in a nonce gap and some transactions will be stack in the transaction pool. The error message doesn't give me a hint if that's the case or if there is something else causing this issue.

Frequency: [What percentage of the time does it occur?]

Every time I send too many transactions in a concurrent way.

Versions (Add all that apply)

Software version:
besu/v1.3.8/linux-x86_64/oracle_openjdk-java-11

Java version: [java -version]

openjdk 11.0.2 2019-01-15
OpenJDK Runtime Environment (build 11.0.2+9-Debian-3bpo91)
OpenJDK 64-Bit Server VM (build 11.0.2+9-Debian-3bpo91, mixed mode, sharing
OS Name & Version:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
Kernel Version:
Linux staging1 4.4.0-171-generic #200-Ubuntu SMP Tue Dec 3 11:04:55 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

[roderik]

Historically this error is a sign your json rpc did not answer. are you sure you are not overloading the node or hitting some other os limits?

[ppoliani]

I tried to minimise the number of concurrent call to just two. I also checked what happens what happens when I use the same nonce.

I sent the following two transactions:

{ nonce: '0x15c8',
  gasPrice: '0x0',
  gasLimit: '0x24ada',
  to: '0xDA35f4cb13a6ea0144469b23C0fE2F025bD4663B',
  value: '0x0',
  data: '0x0'
}

{ nonce: '0x15c8',
  gasPrice: '0x0',
  gasLimit: '0x24ada',
  to: '0xDA35f4cb13a6ea0144469b23C0fE2F025bD4663B',
  value: '0x0'
  data:'0x0' 
}

The expected behaviour is to receive replacement transaction underpriced error, however I receive 400 Bard Request and an empty response instead.

I don't think a 400 bad request with an empty text is a very helpful response. There is not much information to know how to deal with the error.

I wonder if the fact that I use a fee free network causes this behaviour as you can see gasPrice is 0 so I wonder if that's make Besu believe that it's not an underpriced transaction. I'm just guessing here

[AbdelStark]

I tried to minimise the number of concurrent call to just two. I also checked what happens what happens when I use the same nonce.

I sent the following two transactions:

{ nonce: '0x15c8',
  gasPrice: '0x0',
  gasLimit: '0x24ada',
  to: '0xDA35f4cb13a6ea0144469b23C0fE2F025bD4663B',
  value: '0x0',
  data: '0x0'
}

{ nonce: '0x15c8',
  gasPrice: '0x0',
  gasLimit: '0x24ada',
  to: '0xDA35f4cb13a6ea0144469b23C0fE2F025bD4663B',
  value: '0x0'
  data:'0x0' 
}

The expected behaviour is to receive replacement transaction underpriced error, however I receive 400 Bard Request and an empty response instead.

I don't think a 400 bad request with an empty text is a very helpful response. There is not much information to know how to deal with the error.

I wonder if the fact that I use a fee free network causes this behaviour as you can see gasPrice is 0 so I wonder if that's make Besu believe that it's not an underpriced transaction. I'm just guessing here

Could you attach a sample of the configuration you are using including:

• command line options
• extract of the genesis file without any account information

[AbdelStark]

I tried to minimise the number of concurrent call to just two. I also checked what happens what happens when I use the same nonce.

I sent the following two transactions:

{ nonce: '0x15c8',
  gasPrice: '0x0',
  gasLimit: '0x24ada',
  to: '0xDA35f4cb13a6ea0144469b23C0fE2F025bD4663B',
  value: '0x0',
  data: '0x0'
}

{ nonce: '0x15c8',
  gasPrice: '0x0',
  gasLimit: '0x24ada',
  to: '0xDA35f4cb13a6ea0144469b23C0fE2F025bD4663B',
  value: '0x0'
  data:'0x0' 
}

The expected behaviour is to receive replacement transaction underpriced error, however I receive 400 Bard Request and an empty response instead.

I don't think a 400 bad request with an empty text is a very helpful response. There is not much information to know how to deal with the error.

I wonder if the fact that I use a fee free network causes this behaviour as you can see gasPrice is 0 so I wonder if that's make Besu believe that it's not an underpriced transaction. I'm just guessing here

@ppoliani If i run your scenario with the 2 transactions at the same nonce and a non zero gas price I don't get any error. The first and the second attempts return both a success response containing the transaction hash.

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": "0x8cdf5653a7847acee44dd93024d83ce56ccb696de614ac044bd57bf08638730e"
}

Now if i send a zero gas price transaction with defaults options iu get the following error:

{
  "jsonrpc": "2.0",
  "id": 1,
  "error": {
    "code": -32009,
    "message": "Gas price below configured minimum gas price"
  }
}

Which is the intended behaviour since the default minimum gas price is 1000 (can be changed with --min-gas-price, value expressed in WEI).

[ppoliani]

@abdelhamidbakhta Thanks for getting back.

I deployed my network on a K8s cluster. The pod start via the following command

             exec /opt/besu/bin/besu \
              --genesis-file=/configs/genesis.json \
              --rpc-http-authentication-enabled \
              --data-path=/var/besu/data \
              --rpc-ws-authentication-enabled \
              --rpc-http-authentication-credentials-file=/configs/creds.toml \
              --rpc-ws-authentication-credentials-file=/configs/creds.toml \
              --min-gas-price=0 \
              --rpc-http-enabled \
              --rpc-http-host=0.0.0.0 \
              --rpc-http-port=8545 \
              --rpc-http-cors-origins=${NODES_HTTP_CORS_ORIGINS} \
              --rpc-http-api=ETH,NET,CLIQUE,ADMIN,WEB3 \
              --graphql-http-enabled \
              --graphql-http-host=0.0.0.0 \
              --graphql-http-port=8547 \
              --graphql-http-cors-origins=${NODES_HTTP_CORS_ORIGINS} \
              --rpc-ws-enabled \
              --rpc-ws-host=0.0.0.0 \
              --rpc-ws-port=8546 \
              --metrics-enabled=true \
              --metrics-host=0.0.0.0 \
              --metrics-port=9545 \
              --host-whitelist=${NODES_HOST_WHITELIST} \
              --bootnodes=enode://${VALIDATOR1_PUBKEY}@${VALIDATOR1_SERVICE_HOST}:30303

Below is the content of the genesis file (modified to hide some parts)

    {
      "config": {
        "chainId": 1111,
        "constantinoplefixblock": 0,
        "contractSizeLimit": 2147483647,
        "clique": {
          "blockperiodseconds": 5,
          "epochlength": 30000
        }
      },
      "coinbase": "0x0000000000000000000000000000000000000000",
      "difficulty": "0x1",
      "gasLimit": "0x1fffffffffffff",
      "extraData": "0x0000000000000000000000000000000000000000000000000000000000000000<extra data>0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
      "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
      "nonce": "0x0",
      "timestamp": "0x0",
      "alloc": {
        "0xxxxxxxxxxxxxxxxxxx": {
          "balance": "0xad78ebc5ac6200000"
        }
      },
      "number": "0x0",
      "gasUsed": "0x0",
      "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000"
    }

[ppoliani]

@ppoliani If i run your scenario with the 2 transactions at the same nonce and a non zero gas price I don't get any error. The first and the second attempts return both a success response containing the transaction hash.

Now if i send a zero gas price transaction with defaults options iu get the following error:

This is not the case in my setup because I have explicitly created a free-gas network.
I don't believe this is an accurate behaviour though. You can't use the same nonce in any Ethereum based client. Most of the clients I worked for use a clear message error which is as I mentioned earlier is something along these lines

replacement transaction underpriced

In any case this is an error that should be reported by the rpc client.

[AbdelStark]

@ppoliani Ok you are right, the behaviour is confusing. I will work on it to align with other clients implementations.

[AbdelStark]

Actually with your scenario Geth would return (exact same transaction):

{
    "jsonrpc": "2.0",
    "id": 1,
    "error": {
        "code": -32000,
        "message": "known transaction: 6e5bd3806b90881721d5c92ccb8ec3abb9dbb3942e1513d10cb8e7e8a6015c6b"
    }
}

Right ?

[AbdelStark]

And with a gas price slightly higher but not sufficient to match the minimum price bump (or lower gas price):

{
    "jsonrpc": "2.0",
    "id": 1,
    "error": {
        "code": -32000,
        "message": "replacement transaction underpriced"
    }
}