Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the same endpoint for notifications and RPC requests #1053

Closed
alexvanin opened this issue Dec 24, 2021 · 1 comment · Fixed by #1170
Closed

Use the same endpoint for notifications and RPC requests #1053

alexvanin opened this issue Dec 24, 2021 · 1 comment · Fixed by #1170
Assignees
Labels
bug Something isn't working
Milestone

Comments

@alexvanin
Copy link
Contributor

NeoFS nodes have two separate pools of endpoint addresses:

  • to listen side chain events,
  • to send side chain RPC requests.
    Node might listen events from one endpoint but send RPC request to another. There is an issue with that approach.

Sidechain blocks are not distributed uniformly among RPC nodes. There is a case when NeoFS node receives notification from block which is not processed by other RPC node, where NeoFS sends requests.

Example:

addNewEpochAsyncNotificationHandler(c, func(ev event.Event) {
e := ev.(netmapEvent.NewEpoch).EpochNumber()
ni, err := c.netmapLocalNodeState(e)
if err != nil {
c.log.Error("could not update node state on new epoch",
zap.Uint64("epoch", e),
zap.String("error", err.Error()),
)
return
}
c.handleLocalNodeInfo(ni)

In this example NeoFS node tries to fetch new network map on NewEpoch event. If RPC node did not process block with new epoch yet, then fetch invocation will fail and new network map will not be processed by the node.

error   neofs-node/netmap.go:156        could not update node state on new epoch        {"epoch": 2529, "error": "could not perform test invocation (snapshotByEpoch): chain/client: contract execution finished with state FAULT; exception: at instruction 3593 (THROW): unhandled exception: \"incorrect diff\""}

To avoid such cases I propose to use single endpoint for listening and sending events. In case of errors, switch to another endpoint both in listener and in the client.

  1. Merge endpoint pools into single pool
morph:
  endpoint:
    - wss://rpc1.morph.fs.neo.org:40341/ws
    - wss://rpc2.morph.fs.neo.org:40341/ws
  1. Use web socket neo-go client to send requests

WSClient encapsulates ordinary client that we use in morph package.

  1. Drop multiclient component from RPC

  2. Provide seamless switch from one endpoint to another in runtime

In case of network errors during RPC requests or closed channel in subscriber, switch to another endpoint.

@alexvanin
Copy link
Contributor Author

I propose to do it in two steps.

  1. Use single pool of endpoints in config to fetch one web socket address for one client and one listener. In case of failures, do not anything. This kinda reverts Storage nodes do not switch between RPC endpoints #746

  2. Define components that will implement both Listener interface and necessary neo-go client functions. This component will use monitor routine to check health of nodes and switch between endpoints in case of failures like Storage nodes do not switch between RPC endpoints #746.

@carpawell carpawell added the blocked Can't be done because of something label Mar 9, 2022
@carpawell carpawell removed the blocked Can't be done because of something label Mar 31, 2022
aprasolova pushed a commit to aprasolova/neofs-node that referenced this issue Oct 19, 2022
Missed this in nspcc-dev#1170.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants