Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provided PipedInputStream should not be closed #3243

Closed
pdudits opened this issue Jun 15, 2021 · 3 comments · Fixed by #3245
Closed

Provided PipedInputStream should not be closed #3243

pdudits opened this issue Jun 15, 2021 · 3 comments · Fixed by #3245

Comments

@pdudits
Copy link
Contributor

pdudits commented Jun 15, 2021

As reported in #2497 and #1896 utilizing readingOut and readingErr to capture execution output looses last chunk of data.

This is caused by ExecWebSocketListener closing both provided PipedInputStream as well as producing PipedOutputStream.

When PipedOutputStream is closed, it only notifies connected PipedInputStream that no more data will be transferred.
PipedInputStream would still make the rest of its buffer available.

However as soon as close is invoked on PipedInputStream its buffer pointer is reset and any further attempt to read will cause an IOException("Pipe closed").

This causes data loss and exceptions thrown to client, and the workaround is to suppress call to close():

            PipedInputStream out = new PipedInputStream() {
                @Override
                public void close() throws IOException {
                    // k8s client errornously closes the stream when execution finishes.
                    // this prevents reading the rest of the buffer
                }
            };
            client.pods().withName(pod.getMetadata().getName())
                    .inContainer(container.getName())
                    .readingOutput(out)
                    .exec(command);
@rohanKanojia
Copy link
Member

@pdudits : Thanks for reporting this, Would it be possible for you to contribute a fix for this issue?

@rohanKanojia
Copy link
Member

@shawkins : Do you think we can take care of this issue in #3169 ?

@pdudits
Copy link
Contributor Author

pdudits commented Jun 15, 2021

I can try, should be ok as long as I don't hit major obstacles in building (for example, master doesn't seem to compile on 1.8.0_292 at first attempt).

Also in ExecWebSocketListener#inputStreamOrPipe a connect call seems to be missing that would connect the provided PipeOutputStream to input. (edit: I misread the code)

pdudits added a commit to pdudits/kubernetes-client that referenced this issue Jun 15, 2021
… loss

Closing provided stream is task of the client. This also caused data loss
when stream was closed before client would fully read available contents of the buffer.
pdudits added a commit to pdudits/kubernetes-client that referenced this issue Jun 15, 2021
… loss

Closing provided stream is task of the client. This also caused data loss
when stream was closed before client would fully read available contents of the buffer.
manusa pushed a commit to pdudits/kubernetes-client that referenced this issue Jun 21, 2021
… loss

Closing provided stream is task of the client. This also caused data loss
when stream was closed before client would fully read available contents of the buffer.
manusa pushed a commit that referenced this issue Jun 21, 2021
Closing provided stream is task of the client. This also caused data loss
when stream was closed before client would fully read available contents of the buffer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants