Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade watchdog service to scheduled task #1951

Merged
merged 11 commits into from
Nov 12, 2024
Merged

Conversation

zackattack01
Copy link
Contributor

The current watchdog implementation installs a lightweight service to periodically check the launcher's service state, and restart if required. The main issue we've found with this in testing is that this service is subject to many of the same power management/sleep state transitions issues that impact the launcher service itself.

These changes alter this pattern to instead utilize a short-lived task that will perform the same state checking and restarts, driven instead by the windows task scheduler service. This is beneficial for a few reasons:

  • it gives us the flexibility to operate both on a timer, and as an event-driven task. the task has both time based (every 30 minutes) and power state transition based (1 minute after wake events) triggers
  • it offloads the event detection outside of launcher, which should be a more reliable methodology than depending on event subscriptions within a service that is also being heavily throttled immediately before or after these events take place

We still make use of much of the same log buffering, service restart, and controller + flag logic that was initially added for watchdog. This primarily includes the following changes:

  • swapping out the remove/install service methods for remove/install task methods. the new logic utilizes go-ole/oleutil to interface with the windows scheduler subsystems and handle all registration and configuration logic
  • removing the power event watcher and polling loops from the watchdog, which now just runs whichever checks and actions that we want to add, and exits
  • adding --install-task and --remove-task options to the watchdog subcommand, this was something I had wanted for the original watchdog and it should allow us to easily install or remove on demand while testing
  • ensuring that all service and task names used integrate the identifier set within our packaging logic - this was something that I missed for the service name in our original watchdog implementation, and it became very confusing for dual installs

@zackattack01 zackattack01 marked this pull request as ready for review November 11, 2024 21:29
// RunWatchdogTask is typically run as a check to determine the health of launcher and restart if required.
// it is installed as an exec action via windows scheduled task. e.g. C:\path\to\launcher.exe watchdog -config <path>.
// you can alternatively run this subcommand to install or remove the scheduled task via the --install-task or --remove-task flags
func RunWatchdogTask(systemSlogger *multislogger.MultiSlogger, args []string) error {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that this file was renamed and looks new but most of what has changed here is just the options parsing logic. the rest is a much simpler version (no power event watching, no polling) of what the service version did before

Copy link
Contributor

@RebeccaMahany RebeccaMahany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, just a couple questions

pkg/launcher/pkg_utils_windows.go Show resolved Hide resolved
ee/watchdog/controller_windows.go Outdated Show resolved Hide resolved
ee/watchdog/controller_windows.go Outdated Show resolved Hide resolved
ee/watchdog/controller_windows.go Show resolved Hide resolved
ee/watchdog/controller_windows.go Show resolved Hide resolved
ee/watchdog/controller_windows.go Show resolved Hide resolved
binDirBase = fmt.Sprintf(`C:\Program Files\Kolide\Launcher-%s\bin`, identifier)
launcherExeName = "launcher.exe"
default:
binDirBase = fmt.Sprintf(`/usr/local/%s/bin`, identifier)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh can you throw a quick comment here or in the docblock that this won't work for NixOS, just so we don't forget if we try to use it for Linux in the future?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh yep! or do you think it make more sense to just add this to pkg_utils_windows.go? it is only in use for these windows tasks currently

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a comment but happy to move this over if you think that's more appropriate

Copy link
Contributor

@RebeccaMahany RebeccaMahany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🔥

@RebeccaMahany RebeccaMahany added the features-improvements Features and Improvements label Nov 12, 2024
Copy link
Contributor

@James-Pickett James-Pickett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for documenting so well 🔥

@zackattack01 zackattack01 added this pull request to the merge queue Nov 12, 2024
Merged via the queue into main with commit bc44bcb Nov 12, 2024
29 checks passed
@zackattack01 zackattack01 deleted the zack/scheduled_tasks branch November 12, 2024 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
features-improvements Features and Improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants