Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup ssh tunnels on cancel #10753

Closed
wants to merge 2 commits into from
Closed

Cleanup ssh tunnels on cancel #10753

wants to merge 2 commits into from

Conversation

m-lima
Copy link

@m-lima m-lima commented Mar 8, 2021

Details
In pkg/minikube/tunnel/kic/ssh_tunnel.go the interrupt signal is caught and it stops the LoadBalancerEmulator. However, it assumes that the interrupt will also propagate to the children ssh tunnels, which are running in a goroutine inside startConnections.
The interrupt, however, kills the goroutine and the blocking call which waits for the child process to finish. This leaves the ssh tunnels dangling.
On the other hand, if CTRL + C is sent on the same terminal as minikube tunnel and the clean-up routine included killing all ssh tunnels, the OS would error with "process already finished".

This leads to accumulation of processes and is likely to happen. Since minikube tunnel is a blocking command, users will tend to detach it from the terminal and kill it with a SIGINT (as proposed in #3647).

Approaches
There are two proposed approaches. This PR takes the approach of having the tunnel know if is running or not. The second approach is to have the parent of all connections managed and know which processes should be killed.

The first approach, proposed with this PR, also addresses #8511, by letting the process owner hold the knowledge if the process is running or not. Since the ssh tunnels are started inside startAndWait() on pkg/minikube/tunnel/kic/ssh_conn.go, and a blocking call to Wait() is made, the owner can know when the process is started, finished, or killed.

The second approach is more oriented towards the cleaning up of the ssh tunnels, and isolates changes to only pkg/minikube/tunnel/kic/ssh_tunnel.go. However it relies on sync. The second approach can be seen here: https://github.com/m-lima/minikube/tree/issue/10752-managed-kill

Output
With this PR, users will be notified of connections being closed (as opposed to the silent death that happens now). E.g:

$ minikube tunnel
🏃  Starting tunnel for service serviceA.
🏃  Starting tunnel for service serviceB.
🏃  Starting tunnel for service serviceC.
^C✋  Tunnel for service serviceB.
✋  Tunnel for service serviceA stopped.
✋  Tunnel for service serviceC stopped.

Also, when detaching from the terminal, all ssh tunnels will be reaped, also outputting to the terminal (if not redirected by the user). E.g:

$ minikube tunnel
🏃  Starting tunnel for service serviceA.
🏃  Starting tunnel for service serviceB.
🏃  Starting tunnel for service serviceC.

On another terminal:

kill -INT <PID>

Main terminal:

✋  Stopping tunnel for service serviceA.
✋  Stopping tunnel for service serviceB.
✋  Stopping tunnel for service serviceC.

fixes #10752

@k8s-ci-robot
Copy link
Contributor

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.


Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Mar 8, 2021
@k8s-ci-robot
Copy link
Contributor

Welcome @m-lima!

It looks like this is your first PR to kubernetes/minikube 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/minikube has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Mar 8, 2021
@k8s-ci-robot
Copy link
Contributor

Hi @m-lima. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: m-lima
To complete the pull request process, please assign afbjorklund after the PR has been reviewed.
You can assign the PR to them by writing /assign @afbjorklund in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Mar 8, 2021
@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@m-lima
Copy link
Author

m-lima commented Mar 9, 2021

/check-cla

1 similar comment
@m-lima
Copy link
Author

m-lima commented Mar 9, 2021

/check-cla

@medyagh
Copy link
Member

medyagh commented Mar 9, 2021

@m-lima thanks for this PR, do u mind checking if the Email u are using in the terminal is the same on that is Signed CLA in
https://identity.linuxfoundation.org/projects/cncf

@m-lima
Copy link
Author

m-lima commented Mar 9, 2021

@m-lima thanks for this PR, do u mind checking if the Email u are using in the terminal is the same on that is Signed CLA in
https://identity.linuxfoundation.org/projects/cncf

I used my github account to sign the CLA. Got an email confirmation on my email and it all looked good.
I assume that by checking my email in the terminal, you mean to check that the commits were made with the same email that was used for CLA, right? I just double checked that and it is correct.
I assumed that the problem with the CLA not validating was because of the migration that the linux foundation was reporting..

@medyagh
Copy link
Member

medyagh commented Mar 9, 2021

@m-lima
thats possible how rebasing the PR and force push the commits ? maybe that would help ?

@m-lima
Copy link
Author

m-lima commented Mar 9, 2021

@m-lima
thats possible how rebasing the PR and force push the commits ? maybe that would help ?

Sure I'll try that

@m-lima
Copy link
Author

m-lima commented Mar 9, 2021

@medyagh would it be helpful or harmful if I close this PR and try it again from scratch?

@m-lima m-lima closed this Mar 10, 2021
@m-lima m-lima deleted the issue/10752 branch March 10, 2021 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SSH tunnels left open after sending SIGINT to minikube tunnel
4 participants