Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken pipe error seen in client (cli) when backend restarts and CLICON_SOCK is recreated #312

Closed
DeekshaBhandary opened this issue Mar 10, 2022 · 4 comments

Comments

@DeekshaBhandary
Copy link

step 1 : login to cli and execute any command (do not exit out of cli)
step 2. restart backend process in separate terminal
step 3: once server socket is recreated , go to the previous active cli session and execute any command
Broken pipe error is seen for the first time

supervisor@deeksha_latest: op> show config set
Mar 10 08:56:15: clicon_msg_send: 348: Config error: atomicio: Broken pipe
Mar 10 08:56:15: clicon_msg_send: write: Broken pipe len:0 msg:<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" username="supervisor" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="42"><get-config><source><running/></source><nc:filter nc:type="xpath" nc:select="/" xmlns="0"/></get-config></rpc>
Config error: atomicio: Broken pipe
supervisor@deeksha_latest: op> show config set
set system 
set system load-last-config true
supervisor@deeksha_latest: op> 
@olofhagsand
Copy link
Member

There are some complexities here.
If the cli has made some edits (in candidate) with or without locks and the backend is restarted, those changes in candidate are lost.
If the CLI is not aware that the backend has restarted and the candidate wiped (and possibly locks lost), I would say the CLI should exit or be notified in some way that the backend has restarted.
Maybe if the CLI is stateless (no ongoing edits) one could recover a backend restart.
What is the scenario in your case?

@DeekshaBhandary
Copy link
Author

backend has restarted and prior to that cli had some candidate changes.
i am not expecting the candidate changes to be recovered. is it possible for the backend to notify the cli in someway so that the client socket connection happens prior to sending any rpc to prevent broken pipe error

@olofhagsand
Copy link
Member

Being robust against pipe error is one thing and should be done. This is relatively new behavior and is a bug that should be fixed. I therefore add the Bug label now.

is it possible for the backend to notify the cli in someway so that the client socket connection happens prior to sending any rpc to prevent broken pipe error

Notifying: In the case the backend quits in a controlled way, yes , but it could also crash or in some other way be gone (eg the unix socket file is removed).

But what do you want the CLI to do in case it gets a notification that the backend has died / restarted? Especially in the case that it has candidate changes?
As a first step I can ensure the pipe error does not occur but that the CLI exits in a controlled way.
My question is what the behavior should be of the the CLI if it stays up after a backend restart?

olofhagsand added a commit that referenced this issue Mar 16, 2022
  * Note that edits to the candidate database or locks will be lost
  * To force the CLI to exit if backend restarts, undef `PROTO_RESTART_RECONNECT`
  * This is an effect of the fix of [Broken pipe error seen in client (cli) when backend restarts and CLICON_SOCK is recreated](#312), the CLI behavior on backend restart is changed.
* Added `eof` parameter to `clicon_rpc()` and `clicon_rpc1()` and error handling modified
@olofhagsand
Copy link
Member

Patch commit as follows:
CLI reconnects to backend if backend restarts with a warning

Please verify

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants