Updated Endpoint Selector to pick the Cluster in Enabled state (in addition to Host state) by sureshanaparti · Pull Request #10757 · apache/cloudstack

sureshanaparti · 2025-04-21T09:42:58Z

Description

Current EndPoint selection checks for Host status Up and and resource_state Enabled, however there is a case where whole Cluster (where Host located is) can be disabled. This PR updates the EndPoint selection query to consider the clusters with allocation state 'Enabled' (in addition to current Host state check - status Up and and resource_state Enabled), which ensures the Host is chosen from Enabled cluster.

This includes changes of PR #10777 (targeted for 4.19.3).

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

…lection (in addition to Host state)

sureshanaparti · 2025-04-21T09:43:24Z

@blueorangutan package

blueorangutan · 2025-04-21T09:44:03Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2025-04-21T09:50:42Z

Codecov Report

Attention: Patch coverage is 0% with 18 lines in your changes missing coverage. Please review.

Project coverage is 16.14%. Comparing base (0af887b) to head (89fe76e).
Report is 61 commits behind head on 4.20.

Files with missing lines	Patch %	Lines
...e/cloudstack/storage/volume/VolumeServiceImpl.java	0.00%	11 Missing ⚠️
...e/driver/CloudStackPrimaryDataStoreDriverImpl.java	0.00%	7 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               4.20   #10757      +/-   ##
============================================
+ Coverage     16.02%   16.14%   +0.11%     
- Complexity    13127    13228     +101     
============================================
  Files          5652     5652              
  Lines        495994   496986     +992     
  Branches      60067    60218     +151     
============================================
+ Hits          79466    80214     +748     
- Misses       407668   407838     +170     
- Partials       8860     8934      +74

Flag	Coverage Δ
uitests	`4.00% <ø> (+<0.01%)`	⬆️
unittests	`16.99% <0.00%> (+0.12%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blueorangutan · 2025-04-21T11:04:19Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13121

borisstoyanov · 2025-04-22T10:28:59Z

@blueorangutan test

blueorangutan · 2025-04-22T10:30:03Z

@borisstoyanov a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-04-23T02:33:02Z

[SF] Trillian test result (tid-13082)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54961 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10757-t13082-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

weizhouapache · 2025-05-06T07:35:55Z

code lgtm

pod could be disabled as well.
it might be better to check if pod state is checked during resource allocation, etc

yadvr

LGTM didn't test it

yadvr · 2025-05-12T08:35:29Z

@sureshanaparti @borisstoyanov could you review Wei's remarks:

pod could be disabled as well.
it might be better to check if pod state is checked during resource allocation, etc

And advise if this is ready for merging?

borisstoyanov · 2025-05-12T09:38:17Z

@rohityadavcloud, Suresh will be syncing the latest fixes with this one, I'll doublecheck and LGTM.

- the pool id is persisted while creating the volume, when it fails the pool id is not reverted. On next create volume attempt, CloudStack couldn't find any suitable primary storage even there are pools available with enough capacity as the pool is already assigned to volume which is in Allocated state (and storage pool compatibility check fails). Ensure volume is not assigned to any pool if create volume fails (so the next creation job would pick the suitable pool).

sureshanaparti · 2025-05-12T11:11:55Z

code lgtm

pod could be disabled as well. it might be better to check if pod state is checked during resource allocation, etc

@weizhouapache disabled pods are checked during deployment plan, for volumes operations only zone disabled state is checked not pod state. so, there are chances that host in disabled pod (but cluster enabled) can be returned as an endpoint to perform operations.

sureshanaparti · 2025-05-12T11:13:17Z

@blueorangutan package

blueorangutan · 2025-05-12T11:14:03Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-05-12T12:15:26Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13343

borisstoyanov · 2025-05-12T12:42:48Z

@blueorangutan test

blueorangutan · 2025-05-12T12:44:03Z

@borisstoyanov a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-05-13T05:37:09Z

[SF] Trillian test result (tid-13276)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 57848 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10757-t13276-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

DaanHoogland

clgtm

borisstoyanov

LGTM

…dition to Host state) (apache#10757) * Consider the clusters with allocation state 'Enabled' for EndPoint selection (in addition to Host state) * Reset the pool id when create volume fails on the allocated pool - the pool id is persisted while creating the volume, when it fails the pool id is not reverted. On next create volume attempt, CloudStack couldn't find any suitable primary storage even there are pools available with enough capacity as the pool is already assigned to volume which is in Allocated state (and storage pool compatibility check fails). Ensure volume is not assigned to any pool if create volume fails (so the next creation job would pick the suitable pool). * endpoint check for resize * update the resize error through callback result instead of exception * logger fix

Consider the clusters with allocation state 'Enabled' for EndPoint se…

e458a41

…lection (in addition to Host state)

boring-cyborg bot added the component:storage label Apr 21, 2025

sureshanaparti added this to the 4.20.1 milestone Apr 21, 2025

sureshanaparti changed the title ~~Updated Endpoint Selector to pick the Cluster in Enabled state (in addition to Host state)~~ [WIP] Updated Endpoint Selector to pick the Cluster in Enabled state (in addition to Host state) Apr 21, 2025

sureshanaparti requested review from borisstoyanov and weizhouapache May 6, 2025 07:05

yadvr approved these changes May 9, 2025

View reviewed changes

sureshanaparti changed the title ~~[WIP] Updated Endpoint Selector to pick the Cluster in Enabled state (in addition to Host state)~~ Updated Endpoint Selector to pick the Cluster in Enabled state (in addition to Host state) May 9, 2025

sureshanaparti marked this pull request as ready for review May 9, 2025 08:48

sureshanaparti assigned borisstoyanov May 9, 2025

yadvr requested a review from DaanHoogland May 12, 2025 08:31

yadvr marked this pull request as draft May 12, 2025 10:10

sureshanaparti and others added 4 commits May 12, 2025 15:42

endpoint check for resize

4b07d00

update the resize error through callback result instead of exception

96fec77

logger fix

89fe76e

boring-cyborg bot added the component:primary-storage label May 12, 2025

DaanHoogland approved these changes May 13, 2025

View reviewed changes

borisstoyanov approved these changes May 13, 2025

View reviewed changes

yadvr marked this pull request as ready for review May 13, 2025 12:18

yadvr merged commit 52d9860 into apache:4.20 May 13, 2025
21 of 26 checks passed

yadvr deleted the end_point_selector_with_cluster_state branch May 13, 2025 12:18

Conversation

sureshanaparti commented Apr 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

sureshanaparti commented Apr 21, 2025

Uh oh!

blueorangutan commented Apr 21, 2025

Uh oh!

codecov bot commented Apr 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Apr 21, 2025

Uh oh!

borisstoyanov commented Apr 22, 2025

Uh oh!

blueorangutan commented Apr 22, 2025

Uh oh!

blueorangutan commented Apr 23, 2025

Uh oh!

weizhouapache commented May 6, 2025

Uh oh!

yadvr left a comment

Choose a reason for hiding this comment

Uh oh!

yadvr commented May 12, 2025

Uh oh!

borisstoyanov commented May 12, 2025

Uh oh!

sureshanaparti commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sureshanaparti commented May 12, 2025

Uh oh!

blueorangutan commented May 12, 2025

Uh oh!

blueorangutan commented May 12, 2025

Uh oh!

borisstoyanov commented May 12, 2025

Uh oh!

blueorangutan commented May 12, 2025

Uh oh!

blueorangutan commented May 13, 2025

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sureshanaparti commented Apr 21, 2025 •

edited

Loading

codecov bot commented Apr 21, 2025 •

edited

Loading

sureshanaparti commented May 12, 2025 •

edited

Loading