Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors: Ignore errors from async when quitting #1918

Merged

Conversation

har7an
Copy link
Contributor

@har7an har7an commented Nov 9, 2022

This fixes a bug where, upon successfully quitting zellij, the log would contain error messages from the async tasks associated with the PTYs about failing to send a message to the Screen.

The Situation

The errors look like this in the logs:

INFO   |zellij_client            | 2022-11-09 08:35:43.856 [main      ] [/var/home/har7an/.cargo/registry/src/github.com
-1ecc6299db9ec823/zellij-client-0.32.0/src/lib.rs:393]: Bye from Zellij! 
INFO   |zellij_server::wasm_vm   | 2022-11-09 08:35:43.856 [wasm      ] [/var/home/har7an/.cargo/registry/src/github.com
-1ecc6299db9ec823/zellij-server-0.32.0/src/wasm_vm.rs:326]: wasm main thread exits 
ERROR  |zellij_utils::errors::not| 2022-11-09 08:35:43.857 [async-std/runti] [/var/home/har7an/.cargo/registry/src/githu
b.com-1ecc6299db9ec823/zellij-utils-0.32.0/src/errors.rs:432]: Panic occured:
             thread: async-std/runtime
             location: At /var/home/har7an/.cargo/registry/src/github.com-1ecc6299db9ec823/zellij-server-0.32.0/src/term
inal_bytes.rs:124:14
             message: called `Result::unwrap()` on an `Err` value: failed to send message to screen

Caused by:
    0: Originating Thread(s)
       
    1: failed to send message to channel 

This is undesirable for a number of reasons:

  1. It fills the log with error messages even though we successfully terminated the application,
  2. It leaves the impression we have a bug and can't terminate the application properly
  3. The error message doesn't match the expectations with respect to the attached context messages
  4. The errors only turn up sporadically. They originate in the TerminalBytes::listen functions loop, and there is one per zellij pane. However, we do not get an error per zellij pane in the logs.

It turns out that 3 was actually a mistake in the implementation of the ToAnyhow trait for the SendError type. With 3 fixed the errors now look like this:

INFO   |zellij_client            | 2022-11-09 08:52:35.216 [main      ] [zellij-client/src/lib.rs:392]: Bye from Zellij! 
INFO   |zellij_server::wasm_vm   | 2022-11-09 08:52:35.216 [wasm      ] [zellij-server/src/wasm_vm.rs:328]: wasm main thread exits 
ERROR  |zellij_utils::errors::not| 2022-11-09 08:52:35.217 [async-std/runti] [zellij-utils/src/errors.rs:452]: Panic occured:
             thread: async-std/runtime
             location: At zellij-server/src/pty.rs:538:22
             message: Program terminates: a fatal error occured

Caused by:
    0: failed to run async task for terminal 6
    1: failed to listen for bytes from PTY
    2: failed to async-send to screen
    3: failed to send message to screen
    4: Originating Thread(s)
       
    5: failed to send message to channel

This also fixes 4, because now we get an error report like this for every pane that was open when we terminated zellij (e.g. with Ctrl+q). Also, the error "report" now matches the expectations of how it should look given the code, and it tells us what really went wrong.

The Cause

As the error message now correctly suggests, this error originates from failing to send a message to the Screen thread to render the screen. At the end of the TerminalBytes::listen function, after we have broken out of the main control loop, we send a final render instruction to the screen.

However, when quitting zellij, the individual threads currently don't adhere to a strict order when exiting and do not wait on each other. Therefore, the screen thread responsible for handling render requests may have exited before the individual pane tasks manage to send their final render requests. While this is technically an error, it is expected in the current state of the application.

Solving the problem

The problem is now solved by sending the final render instruction but ignoring the result. This may have side-effects for regular application operation, e.g. when exiting a single pane and the screen isn't re-rendered. However, this is likely solved by the next render instruction, which will likely happen shortly afterwards.

A long-term solution may include zellij entering a global "quitting" state, which can be checked for from any part of the code to handle specific (types of) errors gracefully. Alternatively, it may be possible to have the threads wait on each other when terminating the application so all IPC messages get to their destinations before closing IPC channels. However, this is out of scope for this PR.

@har7an har7an temporarily deployed to cachix November 9, 2022 08:44 Inactive
@har7an
Copy link
Contributor Author

har7an commented Nov 9, 2022

@imsnif @tlinford This fixes the error you mentioned yesterday where we get error messages in the log when quitting zellij normally. They should be gone now.

har7an added a commit to har7an/zellij that referenced this pull request Nov 10, 2022
don't log errors from async pane threads when quitting zellij
@har7an har7an force-pushed the errors/ignore-errors-from-async-when-quitting branch from 8d57231 to 9333312 Compare November 10, 2022 13:47
@har7an har7an temporarily deployed to cachix November 10, 2022 13:47 Inactive
impl for `SendError`. Previously we attached the context to `anyhow!`,
which is wrong (because it doesn't create an `Err` type itself) and
leads to strange behavior where the error seemingly is immediately
panicked upon.

Instead, Wrap `anyhow!` into an `Err()` and then attach the context to
that. This achieves the intended goal and doesn't lead to premature
termination.
which occurs when quitting zellij with the `Ctrl+q` keybinding. At the
end of the `listen` function we break out of a loop and send a final
`Render` instruction to the Screen. However, when quitting zellij as
mentioned above, the Screen thread is likely dead already and hence we
cannot send it any Instructions. This causes an error in the async tasks
of the panes that handle reading the PTY input.

If we leave the error unhandled, we will have error messages in the log
whenever we quit zellij, even though the application exited normally.
Hence, we now send the final `Render` instruction but do not care
whether it is sent successfully or not.

This is a "workaround" for the fact that we cannot tell whether the
application is quitting or not.
don't log errors from async pane threads when quitting zellij
@har7an har7an force-pushed the errors/ignore-errors-from-async-when-quitting branch from 9333312 to 9727945 Compare November 12, 2022 07:25
@har7an har7an temporarily deployed to cachix November 12, 2022 07:26 Inactive
@har7an har7an merged commit 342d162 into zellij-org:main Nov 12, 2022
@har7an har7an deleted the errors/ignore-errors-from-async-when-quitting branch November 12, 2022 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant