Bug: etherscan api 403 when using getDefaultProvider

[thegostep]

GET https://api-goerli.etherscan.io/api?module=proxy&action=eth_getBlockByNumber&tag=latest&boolean=false&apikey=9D13ZE7XSBTJ94N9BNJ2MA33VMAY2YPIRB 403

[ricmoo]

Can you try again? I just tested it an it seems to have worked. Maybe Görli (Etherscans fleet) was down at the time?

If you still have issues, I’ll dig into this more.

[thegostep]

It still occurs, here is a screenshot

[ricmoo]

Thanks. I’ll look into it first thing. :)

[ricmoo]

Heya! Just did some experimenting and can't reproduce it yet, so I have a few questions. :)

• What version of ethers are you using?
• Are you using the version created in the dist/ folder of ethers? Or referencing the CDN version?
• Are you making a lot of requests? Are any earlier ones succeeding? I'm wondering if you are hitting the quota for your IP address with the default key? Can you also try registering your own Etherscan key and trying that?

[thegostep]

My quick fix was to remove etherscan provider for now - it is possible I am getting throttled so I'll have to check if creating my own api key fixes it

[ricmoo]

Quick question. What browser are you using and what is its user agent?

I spent the last few days debugging a similar issue with the browser tests failing, and I was finally able to determine the issue was the Cloudflare configurations sitting in-front of all the Etherscan testnet hosts (mainnet is fine) were overly aggressively blocking the User-Agent that the headless chrome reports. I fixed it by changing the User-Agent string.

[thegostep]

I was seeing this in Chrome. @PierrickGT may have more details.

[ricmoo]

Could it have been configured with a custom User-Agent (or maybe a plugin that tweaked it) or had an automated option enabled? Chrome with the --headless option for example changes the user-agent string to something that fails on testnets for Etherscan (works fine for mainnet).

Just a hypothesis, but trying to narow it down.

[ricmoo]

Closing as I don't believe this is an issue any longer? The EtherscanProvider now passes an empty API key along, which instructs their API to use IP throttling instead of API key throttling.

Thanks! :)

[HappyGroupHub]

I've the same problem before and got solved by doing this!
Add headers as well while doing the request.

url = YourApiCallingUrl
headers = {
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36'}
requests.get(url, headers=headers)

Etherscan eventually blocked request that without headers
But it seems only happening in testnets sometimes, which is weird..
But anyways! hopefully helps you out

[ricmoo]

Oh! That usually has to do with their configured Cloudflare heuristics for detecting a bot. This can often be addressed by acquiring your own API key.

It also changes frequently and it hard to predict what rules are in effect and when. :(