Skip to content

Add scale sets - #376

Merged
gabriel-samfira merged 46 commits into
cloudbase:mainfrom
gabriel-samfira:scalesets
May 6, 2025
Merged

Add scale sets#376
gabriel-samfira merged 46 commits into
cloudbase:mainfrom
gabriel-samfira:scalesets

Conversation

@gabriel-samfira

@gabriel-samfira gabriel-samfira commented Apr 16, 2025

Copy link
Copy Markdown
Member

This PR adds scale set support to GARM.

Adding a new scale set is identical to adding a pool:

garm-cli scaleset add \
    --repo  30c6c4c7-e020-4203-b4f5-80f857b93238 \
    --provider-name yourProviderName \
    --image ubuntu:22.04 \
    --name test-scale-set \
    --flavor default \
    --enabled=true

Scalesets have the same commands that pools have:

ubuntu@garm:~$ garm-cli scaleset --help
Query information or perform operations on scale sets.

Usage:
  garm-cli scaleset [command]

Available Commands:
  add         Add scale set
  delete      Delete scale set by ID
  list        List scale sets
  show        Show details for a scale set
  update      Update one scale set

Flags:
  -h, --help   help for scaleset

Global Flags:
      --debug           Enable debug on all API calls
      --format string   Output format (table, json) (default "table")

Use "garm-cli scaleset [command] --help" for more information about a command.

Runners will appear along side the pool runners in:

garm-cli runner ls -a

Comment thread apiserver/controllers/instances.go Fixed
Comment thread apiserver/controllers/scalesets.go Fixed
Comment thread apiserver/controllers/scalesets.go Fixed
Comment thread apiserver/controllers/scalesets.go Fixed
@gabriel-samfira
gabriel-samfira force-pushed the scalesets branch 7 times, most recently from 0d1c0a1 to f099268 Compare May 1, 2025 19:30
@gabriel-samfira
gabriel-samfira marked this pull request as ready for review May 1, 2025 19:46
@gabriel-samfira

Copy link
Copy Markdown
Member Author

This still needs a lot of tests, but marking it as ready for review, as it should be fairly functional if anyone feels brave enough to test it.

@gabriel-samfira gabriel-samfira changed the title [WiP] Add scale sets Add scale sets May 1, 2025
This change moves the github client to a subpackage in utils
and adds the scaleset github client code.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
The locking logic was added to its own package as it may need to be used
by other parts of the code.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This adds the workers needed to start listening for scale set messages.
There is no handling of messages yet.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Close response body in scaleset client
* Wait for message listener loop to exit before attempting restart
* Add LastMessageID field to scaleset model and function to update it

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
It seems that Status is a string when you create a session, but a number
when you refresh it (?).

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
We will need to run various checks against the runners that are managed
by a scale set. The runners are updated by the DB watcher, so we should
always have an up to date view of their state. We can then confidently
monitor them without needing to query the DB.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Runners now get created and cleaned up in scale sets.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Fixes provider not spawning runners for newly added scale set

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Adds a periodic cleanup function that cross checks runners between github,
the provider and the GARM database. If an inconsistency is found, GARM will
attempt to fix it.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
You can have multiple scale sets with the same name, as long as
they live in different runner groups.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Split the main function into a couple of more functions
* Add credentials, entity, pool and scaleset cache
* add credentials worker that updates the cache

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
@gabriel-samfira

Copy link
Copy Markdown
Member Author

This has grown quite a bit. Merging this as is and will propose new PRs for additional tests and docs updates.

@gabriel-samfira
gabriel-samfira merged commit e49b35d into cloudbase:main May 6, 2025
@gabriel-samfira
gabriel-samfira deleted the scalesets branch May 6, 2025 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants