Repository navigation
feat: libvirt/KVM infrastructure backend as an alternative to DigitalOcean - #23
Merged
Merged
Conversation
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Inactive-pool cleanup, YAML-safe SSH-key rendering, and subnet validation need correction.
Review effort: Balanced
Findings: 3
Open (3)
What changed in this PR
Adds a backend-neutral infrastructure layer and local libvirt/KVM provisioning while retaining DigitalOcean as the default.
Changes:
- Introduces libvirt provisioning, cleanup, diagnostics, configuration, and CI.
- Refactors shared infrastructure concepts from droplets to nodes.
- Adds offline tests and operator documentation.
| File | Description |
|---|---|
.env.example |
Documents backend and libvirt settings. |
.github/workflows/e2e-libvirt.yml |
Adds self-hosted libvirt E2E workflow. |
AGENTS.md |
Documents libvirt development commands. |
Makefile |
Adds clean-libvirt. |
README.md |
Introduces local KVM usage. |
docs/libvirt-backend.md |
Documents setup, operation, and cleanup. |
pyproject.toml |
Generalizes the E2E marker. |
liquidmetal_at/config.py |
Adds backend selection and libvirt configuration. |
liquidmetal_at/logs.py |
Uses backend-neutral nodes. |
liquidmetal_at/infra/types.py |
Defines shared infrastructure types. |
liquidmetal_at/infra/backend.py |
Centralizes backend lifecycle dispatch. |
liquidmetal_at/infra/libvirt.py |
Implements libvirt provisioning and teardown. |
liquidmetal_at/infra/do.py |
Adapts DigitalOcean to shared types. |
liquidmetal_at/bootstrap/cloudinit.py |
Enables libvirt root SSH configuration. |
liquidmetal_at/bootstrap/host.py |
Bootstraps backend-neutral nodes. |
liquidmetal_at/bootstrap/brigade_cluster.py |
Uses shared infrastructure types. |
liquidmetal_at/bootstrap/templates/cloud_init.yaml.j2 |
Adds libvirt root SSH settings. |
liquidmetal_at/bootstrap/templates/provision_host.sh.j2 |
Supports automatic parent-interface selection. |
tests/conftest.py |
Uses the shared backend lifecycle. |
tests/battery/conftest.py |
Migrates battery fixtures to shared lifecycle. |
tests/battery/test_claim_release.py |
Uses node-based host counts. |
tests/battery/test_events.py |
Uses node-based host counts. |
tests/battery/test_lease_expiry.py |
Uses node-based host counts. |
tests/battery/test_pool_lifecycle.py |
Uses node-based host counts. |
tests/test_backend.py |
Tests backend dispatch and lifecycle. |
tests/test_cleanup.py |
Updates template rendering inputs. |
tests/test_config.py |
Tests libvirt configuration. |
tests/test_guest_agent_vsock.py |
Migrates host access to nodes. |
tests/test_infra_types.py |
Tests shared types and rendering. |
tests/test_libvirt.py |
Exercises libvirt behavior offline. |
tests/test_placement.py |
Migrates placement checks to nodes. |
tests/test_quorum.py |
Migrates quorum checks to nodes. |
tests/test_ssh_reachability.py |
Migrates SSH checks to nodes. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| users: | ||
| - name: root | ||
| ssh_authorized_keys: | ||
| - {{ root_ssh_key }} |
Comment on lines
+234
to
+238
| if not _SUBNET_PREFIX.match(subnet_prefix): | ||
| raise ConfigError( | ||
| f"LIBVIRT_SUBNET_PREFIX={subnet_prefix!r} invalid; give the first two " | ||
| "octets, e.g. 10.210" | ||
| ) |
Comment on lines
+433
to
+437
| if pool in _virsh(uri, run, "pool-list", "--name").split(): | ||
| _virsh(uri, run, "pool-refresh", pool) # console logs are created by QEMU, not virsh | ||
| for vol in _volumes(uri, pool, run): | ||
| if match(vol) and not vol.startswith(BASE_PREFIX): | ||
| remove("vol-delete", "--pool", pool, vol) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a second infrastructure backend so both e2e suites can run against local KVM virtual machines (system libvirt) instead of DigitalOcean droplets. Select it with
INFRA_BACKEND=libvirt. DigitalOcean stays the default.The aim is cheap, frequent runs on a bare-metal self-hosted GitHub runner, and a local dev loop on a workstation with
/dev/kvm. A full brigade run takes about 4 minutes locally.How
infra/types.pyholds neutralNode/Infratypes (Dropletis renamedNode,.dropletsbecomes.nodes).infra/backend.pydispatches onINFRA_BACKENDand owns the provision / collect-logs / teardown lifecycle that was duplicated in the two conftests.infra/libvirt.py), driven byvirshandvirt-install, no new Python dependencies:10.210.N.0/24, with a fixed DHCP lease per VM;lm-acceptancestorage pool;cloud_init.yaml.j2delivered as a NoCloud seed, plus a libvirt-only block enabling root SSH;artifacts/<run_id>/;make clean-libvirtto sweep leftovers (base images are kept).eth1on DO as before; on single-NIC libvirt hosts the host script uses the default-route interface. The unusedDISKtemplate variable is removed.INFRA_BACKENDandLIBVIRT_*knobs (see.env.example).DO_API_TOKENis only required for the DigitalOcean backend.e2e-libvirt.yml,workflow_dispatchonly, on[self-hosted, linux, x64, kvm], with asuiteinput (brigade / battery / both) and its own concurrency group.docs/libvirt-backend.mdcovers prerequisites for Ubuntu and Arch, nested virtualization, and the host-firewall caveat below.Things worth knowing
firewall_backend = "iptables". Docker'sFORWARD DROPpolicy can similarly break VM egress, so keep Docker off the runner.KEEP_INFRA_ON_FAILUREis set.RUN_IDmust be letters, digits,.and-, and must not start withbase.Testing
tests/test_libvirt.pydrives the provisioner through a stateful fake ofvirsh/virt-install), with and withoutINFRA_BACKEND=libvirtin the environment.make lintis clean.poolmgrdcame up; the run was stopped after the first test (xfail). The pool VM reached CREATED and then failed battery's guest-agent readiness check, which is the gap already described indocs/battery-known-gaps.md.DISK=line, and a DO regression run is planned after merge.e2e-libvirtworkflow itself, since the self-hosted runner does not exist yet.