502 Bad Gateway, 0.32.3

[lifeBCE]

• Version:
  0.32.3

• Platform:
  All (Latest FF and Chome on Mac OS X, Latest FF and Chrome on Android)

• Subsystem:

Type:

Bug

Severity:

High

Description:

WebSocket connection to 'wss://lon-1.bootstrap.libp2p.io/' failed: Error during WebSocket handshake: Unexpected response code: 502

I get hundreds of these errors in the console when the node is just running. I also get the CORS issues and the preload stuff as well in this version but I don't think this is the same issue. This happens so often I have had to modify the config of the embedded node on startup and disable certain nodes though the problem changes from node to node so I am regularly re-adding and ignoring different bootstrap nodes.

Steps to reproduce the error:

• Load js-ipfs into browser session.
• Instantiate node and start
• Watch errors flow into console

[lifeBCE]

#1476 I think may need to be reopened. This problem seems to be growing. I cannot access anything on the IPFS network this morning through my app. None of the bootstrap nodes will connect.

[alanshaw]

ping @ipfs/infrastructure any ideas about this?

I know it's not super helpful but if you have a mission critical app you should consider setting up your own bootstrap node(s).

[lifeBCE]

@alanshaw I was considering that but unsure of how well that is supported. Should I remove all default at runtime and add my own or is there a more persisted way of accomplishing this?

I currently have logic such as this in a componentDidUpdate:

      this.props.node.bootstrap.rm('/dns4/nyc-2.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64');
      this.props.node.bootstrap.rm('/dns4/lon-1.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLMeWqB7YGVLJN3pNLQpmmEk35v6wYtsMGLzSr5QBU3');

Should I just expand on this and manually remove all default and then manually add my won each time?

[kyledrake]

I believe I have fixed the servers with the 502 errors and have a mitigation to prevent this particular issue from happening again. I'm going to close this as resolved, but feel free to keep commenting if you need to relay more information.

[kyledrake]

Actually, I'm still seeing it on some of the machines, reopened for now.

[kyledrake]

Alright, we should be good here. Re-open if you see this again.

[lifeBCE]

Still see the following (and no connectivity):

GET https://node0.preload.ipfs.io/api/v0/refs?r=true&arg=QmQMXh5gVvfa41Wvbbjdx2A8KG6reNaNj41KzTvxHBYez6 502 (Bad Gateway)

Access to fetch at 'https://node0.preload.ipfs.io/api/v0/refs?r=true&arg=QmQMXh5gVvfa41Wvbbjdx2A8KG6reNaNj41KzTvxHBYez6' from origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

I am using version 0.32.3 with default bootstrap config

After your reply here, I cleared browser cache, refreshed a few times and continue to see these errors and cannot access any data. Until this morning, I have always seen these errors but have stil been able to connect somehow.

[kyledrake]

@lifeBCE Can you run curl "https://node0.preload.ipfs.io/api/v0/refs?r=true&arg=QmQMXh5gVvfa41Wvbbjdx2A8KG6reNaNj41KzTvxHBYez6" and tell me if it says 502 bad gateway? You should not be getting a 502 for a CORS issue. That curl command works fine for me.

[kyledrake]

Ah, I see that it's failing sporadically. I'm taking a look.

[lifeBCE]

@kyledrake Running that curl yields the response list of refs without error. I have checked my production deployment and it is working great again, no errors. My local dev (localhost:3000) still has the intermittent issues but it will at least get some blocks through now. Looks like node0 will fail and node1 won't and then vise versa. Sometimes both still but all of this seems to only be in local dev.

UPDATE:
Looks like everything is good again in local dev as well. Thanks for looking into this.

[alanshaw]

Thanks for your help @kyledrake