Skip to content

Add a new option to allow vm recreation filter based on vm age#2656

Open
Alphasite wants to merge 3 commits intomainfrom
recreate-vm-age
Open

Add a new option to allow vm recreation filter based on vm age#2656
Alphasite wants to merge 3 commits intomainfrom
recreate-vm-age

Conversation

@Alphasite
Copy link

@Alphasite Alphasite commented Jan 28, 2026

What is this change about?

This should allow us to build in pervasive repaving of vms based on age. It will also allow users to resume failed recreate tasks without starting from the beginning again, which should allow for faster recovery after failure.

Please provide contextual information.

No public stories

What tests have you run against this PR?

Ran all unit tests. Working on testing on a running director (need to work around some cgroup issues).

How should this change be described in bosh release notes?

Add new argument to deploy endpoint to allow failed repaves to resume from where they failed.

Does this PR introduce a breaking change?

No.

Tag your pair, your PM, and/or team!

@selzoc

This should allow us to build in pervasive repaving of vms based on age.
@Alphasite
Copy link
Author

WIll undraft after some manual validation

@Alphasite
Copy link
Author

ok ran some manual validation on my local docker env:

Test Command Result
Test 1 deploy --recreate VM recreated
Test 2 recreate VM recreated
Test 3 recreate --vms-created-before="2020-01-01T00:00:00Z" VM NOT recreated
Test 4 recreate --vms-created-before="2030-01-01T00:00:00Z" VM recreated
Test 5 deploy --recreate --recreate-vms-created-before="2020-01-01T00:00:00Z" VM NOT recreatedrecreated
Test 6 deploy --recreate --recreate-vms-created-before="2030-01-01T00:00:00Z"  VM recreated
=== BOSH CLI Live Tests ===
Date: Thu Jan 29 19:48:54 PST 2026

=== Current VM State ===
{
  "job": "sanity",
  "vm_created_at": "2026-01-30T03:36:34Z"
}

========================================
TEST 1: Deploy without timestamp flag
========================================
Command: go run main.go deploy $MANIFEST -d sanity --recreate --non-interactive


Task 27 | 03:49:03 | Preparing deployment: Preparing deployment (00:00:02)
Task 27 | 03:49:05 | Preparing deployment: Rendering templates (00:00:01)
Task 27 | 03:49:06 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 27 | 03:49:06 | Updating instance sanity: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 27 | 03:49:06 | L executing pre-stop: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 27 | 03:49:07 | L executing drain: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 27 | 03:49:08 | L stopping jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 27 | 03:49:13 | L executing post-stop: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 27 | 03:49:58 | L installing packages: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 27 | 03:50:01 | L configuring jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 27 | 03:50:01 | L executing pre-start: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 27 | 03:50:02 | L starting jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 27 | 03:50:08 | L executing post-start: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary) (00:01:03)Exit code: 0

=== VM State after Test 1 ===
{
  "job": "sanity",
  "vm_created_at": "2026-01-30T03:49:18Z"
}

========================================
TEST 2: Recreate without timestamp
========================================
Command: go run main.go recreate -d sanity --non-interactive


Task 28 | 03:50:44 | Preparing deployment: Preparing deployment (00:00:00)
Task 28 | 03:50:44 | Preparing deployment: Rendering templates (00:00:00)
Task 28 | 03:50:44 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 28 | 03:50:44 | Updating instance sanity: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 28 | 03:50:44 | L executing pre-stop: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 28 | 03:50:44 | L executing drain: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 28 | 03:50:45 | L stopping jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 28 | 03:50:50 | L executing post-stop: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 28 | 03:51:34 | L installing packages: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 28 | 03:51:38 | L configuring jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 28 | 03:51:38 | L executing pre-start: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 28 | 03:51:39 | L starting jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 28 | 03:51:45 | L executing post-start: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary) (00:01:02)Exit code: 0

=== VM State after Test 2 ===
VM created at: 2026-01-30T03:50:56Z

========================================
TEST 3: Recreate with timestamp BEFORE VM creation
========================================
Using timestamp: 2020-01-01T00:00:00Z (before VM created at 2026-01-30T03:50:56Z)
Expected: VM should NOT be recreated
Command: go run main.go recreate -d sanity --vms-created-before="2020-01-01T00:00:00Z" --non-interactive


Task 29 | 03:52:08 | Preparing deployment: Preparing deployment (00:00:01)
Task 29 | 03:52:09 | Preparing deployment: Rendering templates (00:00:00)
Task 29 | 03:52:09 | Preparing package compilation: Finding packages to compile (00:00:00)Exit code: 0

VM created_at after test: 2026-01-30T03:50:56Z
RESULT: VM was NOT recreated (as expected)

=== VM State before Test 4 ===
VM created at: 2026-01-30T03:50:56Z

========================================
TEST 4: Recreate with timestamp AFTER VM creation
========================================
Using timestamp: 2030-01-01T00:00:00Z (after VM created at 2026-01-30T03:50:56Z)
Expected: VM SHOULD be recreated
Command: go run main.go recreate -d sanity --vms-created-before="2030-01-01T00:00:00Z" --non-interactive


Task 30 | 03:53:00 | Preparing deployment: Preparing deployment (00:00:00)
Task 30 | 03:53:00 | Preparing deployment: Rendering templates (00:00:01)
Task 30 | 03:53:01 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 30 | 03:53:01 | Updating instance sanity: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 30 | 03:53:01 | L executing pre-stop: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 30 | 03:53:01 | L executing drain: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 30 | 03:53:02 | L stopping jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 30 | 03:53:07 | L executing post-stop: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 30 | 03:53:49 | L installing packages: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 30 | 03:53:52 | L configuring jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 30 | 03:53:52 | L executing pre-start: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 30 | 03:53:53 | L starting jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 30 | 03:53:59 | L executing post-start: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary) (00:00:59)Exit code: 0

VM created_at after test: 2026-01-30T03:53:11Z
RESULT: VM WAS recreated (as expected)

========================================
TESTS COMPLETE
========================================
Completed at: Thu Jan 29 19:54:04 PST 2026

========================================
DEPLOY COMMAND TESTS WITH TIMESTAMP
========================================

=== VM State before Deploy Tests ===
VM created at: 2026-01-30T03:53:11Z

========================================
TEST 5: Deploy with timestamp BEFORE VM creation
========================================
Using timestamp: 2020-01-01T00:00:00Z (before VM created at 2026-01-30T03:53:11Z)
Expected: VM should NOT be recreated
Command: go run main.go deploy $MANIFEST -d sanity --recreate --recreate-vms-created-before="2020-01-01T00:00:00Z" --non-interactive


Task 31 | 03:55:42 | Preparing deployment: Preparing deployment (00:00:03)
Task 31 | 03:55:45 | Preparing deployment: Rendering templates (00:00:00)
Task 31 | 03:55:45 | Preparing package compilation: Finding packages to compile (00:00:00)Exit code: 0

VM created_at after test: 2026-01-30T03:53:11Z
RESULT: VM was NOT recreated (as expected)

=== VM State before Test 6 ===
VM created at: 2026-01-30T03:53:11Z

========================================
TEST 6: Deploy with timestamp AFTER VM creation
========================================
Using timestamp: 2030-01-01T00:00:00Z (after VM created at 2026-01-30T03:53:11Z)
Expected: VM SHOULD be recreated
Command: go run main.go deploy $MANIFEST -d sanity --recreate --recreate-vms-created-before="2030-01-01T00:00:00Z" --non-interactive


Task 32 | 03:56:10 | Preparing deployment: Preparing deployment (00:00:03)
Task 32 | 03:56:13 | Preparing deployment: Rendering templates (00:00:00)
Task 32 | 03:56:13 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 32 | 03:56:13 | Updating instance sanity: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 32 | 03:56:13 | L executing pre-stop: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 32 | 03:56:13 | L executing drain: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 32 | 03:56:14 | L stopping jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 32 | 03:56:19 | L executing post-stop: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 32 | 03:57:12 | L installing packages: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 32 | 03:57:16 | L configuring jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 32 | 03:57:16 | L executing pre-start: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 32 | 03:57:17 | L starting jobs: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary)
Task 32 | 03:57:22 | L executing post-start: sanity/df584f2e-e400-4deb-b243-1365b5fde48f (0) (canary) (00:01:10)Exit code: 0

VM created_at after test: 2026-01-30T03:56:24Z
RESULT: VM WAS recreated (as expected)

========================================
ALL TESTS COMPLETE
========================================
Completed at: Thu Jan 29 19:57:26 PST 2026

@Alphasite Alphasite marked this pull request as ready for review January 30, 2026 04:02
Copilot AI review requested due to automatic review settings February 4, 2026 00:03
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds functionality to filter VM recreation based on VM age by introducing a new recreate_vm_created_before parameter. This enables pervasive repaving of VMs based on age and allows users to resume failed recreate tasks without starting from the beginning.

Changes:

  • Added recreate_vm_created_before parameter to the deploy endpoint with RFC 3339 timestamp validation
  • Implemented age-based filtering logic in InstancePlan that compares VM creation time against the specified threshold
  • Added warning when future timestamps are provided (which would result in no VMs being filtered)
  • Ensured dirty instances (from failed updates) are always recreated regardless of age filter

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/bosh-director/lib/bosh/director/api/controllers/deployments_controller.rb Added RFC 3339 timestamp validation for recreate_vm_created_before parameter in two endpoints
src/bosh-director/lib/bosh/director/deployment_plan/planner.rb Added recreate_vm_created_before attribute to planner class
src/bosh-director/lib/bosh/director/deployment_plan/planner_factory.rb Passed recreate_vm_created_before option to planner initialization
src/bosh-director/lib/bosh/director/deployment_plan/assembler.rb Passed recreate_vm_created_before to instance plan factory
src/bosh-director/lib/bosh/director/deployment_plan/instance_plan_factory.rb Stored and passed recreate_vm_created_before to instance plans
src/bosh-director/lib/bosh/director/deployment_plan/instance_plan.rb Implemented should_recreate_based_on_vm_age? method with age filtering logic and updated recreation_requested? to use it
src/bosh-director/lib/bosh/director/jobs/update_deployment.rb Added warn_if_recreate_vm_created_before_is_future method to warn users about future timestamps
src/bosh-director/spec/unit/bosh/director/api/controllers/deployments_controller_spec.rb Added tests for valid and invalid timestamp handling
src/bosh-director/spec/unit/bosh/director/jobs/update_deployment_spec.rb Added tests for future timestamp warning logic
src/bosh-director/spec/unit/bosh/director/deployment_plan/instance_plan_spec.rb Added comprehensive tests for age-based filtering including edge cases
src/bosh-director/spec/unit/bosh/director/deployment_plan/instance_plan_factory_spec.rb Updated test expectations to include new parameter
src/bosh-director/spec/unit/bosh/director/deployment_plan/planner_factory_spec.rb Updated test expectations to include new parameter
src/bosh-director/spec/unit/bosh/director/deployment_plan/assembler_spec.rb Updated test expectations to include new parameter

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Alphasite Alphasite requested a review from selzoc February 5, 2026 00:18
@beyhan beyhan moved this from Inbox to Pending Review | Discussion in Foundational Infrastructure Working Group Feb 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Pending Review | Discussion

Development

Successfully merging this pull request may close these issues.

3 participants