Skip to content

feat: add spot termination handler #4176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Oct 10, 2024
Merged

Conversation

npalm
Copy link
Member

@npalm npalm commented Oct 9, 2024

Description

This PR adds a lambda to to log and metric spot termination based on the cloudtrail event BidEvictedEvent. The feature is experimental and disabled by default.

Future directions

The current implemenation only helps to make spot termination visible to an admin team. For the future we want to enrich a runner with information via tagging what job is active, This allows to let the termination handler also inform the user by adding a job annotation once a spot termination occurs.

Migration

No migration is required. By default the watcher is disabled.

  • logging for the watcher is changed
  • resources will be recreated for notification warning watcher

@npalm npalm marked this pull request as draft October 9, 2024 16:07
@npalm npalm changed the title Npalm/feature/spot termination handler feat: add spot termination handler Oct 9, 2024
@npalm npalm force-pushed the npalm/feature/spot-termination-handler branch from 6a0bf37 to af45ee3 Compare October 9, 2024 16:15
@npalm npalm added the enhancement New feature or request label Oct 9, 2024
@npalm npalm marked this pull request as ready for review October 9, 2024 18:15
mpas
mpas previously approved these changes Oct 10, 2024
Co-authored-by: Stuart Pearson <1926002+stuartp44@users.noreply.github.com>
@npalm npalm merged commit 8ba0a82 into main Oct 10, 2024
45 checks passed
@npalm npalm deleted the npalm/feature/spot-termination-handler branch October 10, 2024 09:06
npalm pushed a commit that referenced this pull request Oct 10, 2024
🤖 I have created a release *beep* *boop*
---


##
[5.17.0](philips-labs/terraform-aws-github-runner@v5.16.4...v5.17.0)
(2024-10-10)


### Features

* add spot termination handler
([#4176](https://github.com/philips-labs/terraform-aws-github-runner/issues/4176))
([8ba0a82](philips-labs/terraform-aws-github-runner@8ba0a82))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: forest-releaser[bot] <80285352+forest-releaser[bot]@users.noreply.github.com>
npalm added a commit that referenced this pull request Oct 21, 2024
## Problem

Output to termination watcher not updated in PR #4176

---------

Co-authored-by: philips-labs-pr|bot <philips-labs-pr[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants