Prevent dangling cat-file calls (goroutine alternative)#19454
Merged
6543 merged 4 commits intoApr 22, 2022
Conversation
If an `os/exec.Command` is passed non `*os.File` as an input/output, go will create `os.Pipe`s and wait for their closure in `cmd.Wait()`. If the code following this is responsible for closing `io.Pipe`s or other handlers then on process death from context cancellation the `Wait` can hang. There are two possible solutions: 1. use `os.Pipe` as the input/output as `cmd.Wait` does not wait for these. 2. create a goroutine waiting on the context cancellation that will close the inputs. This PR provides the second option - which is a simpler change that can be more easily backported. Closes go-gitea#19448 Signed-off-by: Andrew Thornton <art27@cantab.net>
KN4CK3R
approved these changes
Apr 21, 2022
Contributor
|
I'm in favor of #19448 as it avoids the goroutine is more correct, however if that PR introduces performance problems we should revert back to this PR in which it still will use buffered I/O. |
Member
I'd like to retrive some metrics ... - but we should backport THIS |
Contributor
Author
|
I vote we should do this and in the meantime I can work on #19448 and try to make it cleaner. |
Gusted
approved these changes
Apr 21, 2022
Contributor
Author
|
OK ... one thing we might want to do which will prevent danglers for all calls to git: diff --git a/modules/git/command.go b/modules/git/command.go
index 3dd12e421..acc1ed66a 100644
--- a/modules/git/command.go
+++ b/modules/git/command.go
@@ -165,6 +165,30 @@ func (c *Command) Run(opts *RunOpts) error {
return err
}
+ closers := make([]io.Closer, 0, 3)
+ for _, pipe := range []interface{}{cmd.Stdout, cmd.Stdin, cmd.Stderr} {
+ if pipe == nil {
+ continue
+ }
+ if _, ok := pipe.(*os.File); ok {
+ continue
+ }
+
+ if closer, ok := pipe.(io.Closer); ok {
+ closers = append(closers, closer)
+ }
+ }
+
+ if len(closers) > 0 {
+ go func() {
+ <-ctx.Done()
+ cancel()
+ for _, closer := range closers {
+ _ = closer.Close()
+ }
+ }()
+ }
+
if opts.PipelineFunc != nil {
err := opts.PipelineFunc(ctx, cancel)
if err != nil { |
6543
pushed a commit
to 6543-forks/gitea
that referenced
this pull request
Apr 22, 2022
If an `os/exec.Command` is passed non `*os.File` as an input/output, go will create `os.Pipe`s and wait for their closure in `cmd.Wait()`. If the code following this is responsible for closing `io.Pipe`s or other handlers then on process death from context cancellation the `Wait` can hang. There are two possible solutions: 1. use `os.Pipe` as the input/output as `cmd.Wait` does not wait for these. 2. create a goroutine waiting on the context cancellation that will close the inputs. This PR provides the second option - which is a simpler change that can be more easily backported. Closes go-gitea#19448 Signed-off-by: Andrew Thornton <art27@cantab.net>
Member
|
-> #19466 |
zeripath
added a commit
that referenced
this pull request
Apr 22, 2022
) If an `os/exec.Command` is passed non `*os.File` as an input/output, go will create `os.Pipe`s and wait for their closure in `cmd.Wait()`. If the code following this is responsible for closing `io.Pipe`s or other handlers then on process death from context cancellation the `Wait` can hang. There are two possible solutions: 1. use `os.Pipe` as the input/output as `cmd.Wait` does not wait for these. 2. create a goroutine waiting on the context cancellation that will close the inputs. This PR provides the second option - which is a simpler change that can be more easily backported. Closes #19448 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net>
zjjhot
added a commit
to zjjhot/gitea
that referenced
this pull request
Apr 23, 2022
* giteaofficial/main: [skip ci] Updated translations via Crowdin Mark TemplateLoading error as "UnprocessableEntity" (go-gitea#19445) Prevent dangling cat-file calls (goroutine alternative) (go-gitea#19454)
AbdulrhmnGhanem
pushed a commit
to kitspace/gitea
that referenced
this pull request
Aug 24, 2022
If an `os/exec.Command` is passed non `*os.File` as an input/output, go will create `os.Pipe`s and wait for their closure in `cmd.Wait()`. If the code following this is responsible for closing `io.Pipe`s or other handlers then on process death from context cancellation the `Wait` can hang. There are two possible solutions: 1. use `os.Pipe` as the input/output as `cmd.Wait` does not wait for these. 2. create a goroutine waiting on the context cancellation that will close the inputs. This PR provides the second option - which is a simpler change that can be more easily backported. Closes go-gitea#19448 Signed-off-by: Andrew Thornton <art27@cantab.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
If an
os/exec.Commandis passed non*os.Fileas an input/output, gowill create
os.Pipes and wait for their closure incmd.Wait(). Ifthe code following this is responsible for closing
io.Pipes or otherhandlers then on process death from context cancellation the
Waitcanhang.
There are two possible solutions:
os.Pipeas the input/output ascmd.Waitdoes not wait for these.This PR provides the second option - which is a simpler change that can
be more easily backported.
Reference #19448
Fix #16113
Signed-off-by: Andrew Thornton art27@cantab.net