ui,api,server: template categorization based on os #10773

shwstppr · 2025-04-24T12:29:26Z

Description

Adds new interface for image selection (template/iso) for an instance in UI.
Old interface can still be used and it can be configured using UI configuration (config.json)

OS categories/Guest OS categories have been improved with ability to create new categories, delete an existing category, and marking a category as featured to allow it to show up in the UI in the image selection interface.

New APIs added:

addOsCategory
deleteOsCategory
updateOsCategory

APIs updated:

updateOsType
listTemplates
listOsCategories

Several improvements in UI especially related to forms - DeloyVM, ReinstallVM, CreateVnfAppliance, AddAutoscaleGroup.

DeployVM form can now be opened from template/ISO details view with query params.

Reorganized (removed and added some) OS categories to the following (in the same order):

1. Ubuntu
2. Debian
3. Fedora
4. CentOS
5. Rocky Linux
6. Alma Linux
7. Oracle
8. RedHat
9. SUSE
10. Windows
11. Other

Documentation PR: apache/cloudstack-documentation#500

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov · 2025-04-24T12:37:06Z

Codecov Report

Attention: Patch coverage is 57.63359% with 111 lines in your changes missing coverage. Please review.

Project coverage is 16.15%. Comparing base (5d28e66) to head (01a66bd).
Report is 24 commits behind head on 4.20.

Files with missing lines	Patch %	Lines
...in/java/com/cloud/server/ManagementServerImpl.java	67.36%	27 Missing and 4 partials ⚠️
.../command/admin/guest/DeleteGuestOsCategoryCmd.java	0.00%	15 Missing ⚠️
.../command/admin/guest/UpdateGuestOsCategoryCmd.java	40.00%	15 Missing ⚠️
...api/command/admin/guest/AddGuestOsCategoryCmd.java	36.84%	12 Missing ⚠️
...ain/java/com/cloud/storage/dao/GuestOSDaoImpl.java	0.00%	9 Missing ⚠️
...ain/java/com/cloud/api/query/QueryManagerImpl.java	0.00%	9 Missing ⚠️
...main/java/com/cloud/storage/GuestOSCategoryVO.java	41.66%	7 Missing ⚠️
.../java/com/cloud/storage/dao/VMTemplateDaoImpl.java	87.50%	2 Missing and 2 partials ⚠️
...tack/api/command/admin/guest/UpdateGuestOsCmd.java	50.00%	3 Missing ⚠️
...ck/api/command/user/template/ListTemplatesCmd.java	0.00%	3 Missing ⚠️
... and 2 more

Additional details and impacted files

@@             Coverage Diff              @@
##               4.20   #10773      +/-   ##
============================================
+ Coverage     16.13%   16.15%   +0.01%     
- Complexity    13215    13255      +40     
============================================
  Files          5652     5659       +7     
  Lines        496578   497389     +811     
  Branches      60134    60348     +214     
============================================
+ Hits          80139    80334     +195     
- Misses       407518   408124     +606     
- Partials       8921     8931      +10

Flag	Coverage Δ
uitests	`3.95% <ø> (-0.06%)`	⬇️
unittests	`17.01% <57.63%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

DaanHoogland · 2025-04-24T13:22:03Z

looks nice. would be great for end-users.

Adds new interface for image selection (template/iso) for an instance in UI. Old interface can still be used and it can be configured using UI configuration (config.json) OS categories/Guest OS categories have been improved with ability to create new categories, delete an existing category, and marking a category as featured to allow it to show up in the UI in the image selection interface. New APIs added: - addOsCategory - deleteOsCategory - updateOsCategory APIs updated: - updateOsType - listTemplates - listOsCategories Several improvements in UI especially related to forms - DeloyVM, ReinstallVM, CreateVnfAppliance, AddAutoscaleGroup. DeployVM form can now be opened from template details view with query params. Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2025-04-28T13:28:58Z

@blueorangutan package

blueorangutan · 2025-04-28T13:30:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-04-28T15:02:55Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13202

show only required elements when query params are passed Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2025-04-30T04:31:11Z

@blueorangutan package

blueorangutan · 2025-04-30T04:32:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-04-30T06:07:37Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13221

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Copilot

Pull Request Overview

This PR introduces an enhanced OS categorization interface for both the UI and API by adding new endpoints and updating existing ones to support operations such as adding, updating, and deleting OS categories. Key changes include:

New API commands for managing OS categories (add, update, delete).
Enhancements in the DAO and VO layers to support the new category properties such as featured and sortKey.
Updates to tests and response objects to incorporate the new fields.

Reviewed Changes

Copilot reviewed 55 out of 55 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
engine/schema/src/main/java/com/cloud/storage/dao/GuestOSDaoImpl.java	Added a new method to list IDs by category.
engine/schema/src/main/java/com/cloud/storage/GuestOSCategoryDaoImpl.java	Adjusted SQL count query for consistency.
engine/schema/src/main/java/com/cloud/storage/GuestOSCategoryVO.java	Included new columns (featured, sortKey) with their getters/setters.
api/.../*.java	Introduced and updated API commands and responses for managing OS categories.
api/src/main/java/org/apache/cloudstack/api/ApiConstants.java	Added new constant for ISO support.
api/src/main/java/com/cloud/server/ResourceTag.java	Added a new resource tag for Guest OS category.
api/src/main/java/org/apache/cloudstack/api/ResponseGenerator.java	Added response generation method for Guest OS categories.
api/src/main/java/org/apache/cloudstack/api/ManagementService.java	Declared new methods for OS category operations.

Copilot · 2025-04-30T11:35:22Z

api/src/main/java/org/apache/cloudstack/api/command/admin/guest/UpdateGuestOsCmd.java

@@ -71,7 +77,7 @@ public String getOsDisplayName() {
        return osDisplayName;
    }

-    public Map getDetails() {
+    public Map<String, String> getDetails() {


The getDetails() method initializes a new empty map and then checks if it is not empty, which will always return false, effectively ignoring the 'details' parameter intended to be processed. Also, remove the redundant semicolon in the map initialization ("new HashMap<>();;").

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2025-04-30T14:27:19Z

@blueorangutan package

blueorangutan · 2025-04-30T14:28:03Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-04-30T15:55:33Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13228

shwstppr · 2025-05-05T06:04:26Z

@blueorangutan test

blueorangutan · 2025-05-05T06:06:03Z

@shwstppr a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

ui/src/views/AutogenView.vue

Pearl1594

code lgtm

vishesh92

@shwstppr Some minor changes in the backend.
I am now reviewing changes in UI now.

api/src/main/java/org/apache/cloudstack/api/command/admin/guest/DeleteGuestOsCategoryCmd.java

api/src/main/java/org/apache/cloudstack/api/command/admin/guest/UpdateGuestOsCategoryCmd.java

api/src/main/java/org/apache/cloudstack/api/command/admin/guest/UpdateGuestOsCmd.java

engine/schema/src/main/java/com/cloud/storage/dao/GuestOSCategoryDaoImpl.java

server/src/main/java/com/cloud/server/ManagementServerImpl.java

server/src/test/java/com/cloud/server/ManagementServerImplTest.java

vishesh92 · 2025-05-05T07:21:22Z

api/src/main/java/org/apache/cloudstack/api/command/admin/guest/UpdateGuestOsCmd.java

@@ -50,7 +50,7 @@ public class UpdateGuestOsCmd extends BaseAsyncCmd {
    @Parameter(name = ApiConstants.ID, type = CommandType.UUID, entityType = GuestOSResponse.class, required = true, description = "UUID of the Guest OS")
    private Long id;

-    @Parameter(name = ApiConstants.OS_DISPLAY_NAME, type = CommandType.STRING, required = true, description = "Unique display name for Guest OS")


JFYI, this will require a change in go sdk here to ensure the SDK is backward compatible.

@vishesh92 created apache/cloudstack-go#109

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Related to apache/cloudstack#10773 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

vishesh92

@shwstppr some minor changes in UI.

ui/public/locales/en.json

Related to apache/cloudstack#10773 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Copilot

Pull Request Overview

This PR enhances the OS category support for image/template selection in the UI and improves the related API functionality. Key changes include new API endpoints for adding, updating, and deleting OS categories; introducing a featured flag and sort key to the OS category model; and updating tests and API responses to support these changes.

Reviewed Changes

Copilot reviewed 55 out of 55 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
engine/schema/src/main/java/com/cloud/storage/dao/GuestOSDaoImpl.java	Added new method to list guest OS IDs by category
engine/schema/src/main/java/com/cloud/storage/GuestOSCategoryVO.java	Introduced new fields "featured" and "sortKey" with corresponding accessors
api/src/main/java/org/apache/cloudstack/api/command/user/template/ListTemplatesCmd.java	Added osCategoryId parameter for template listing API
api/src/main/java/org/apache/cloudstack/api/command/user/guest/ListGuestOsCategoriesCmd.java	Extended API parameters (featured, iso, vnf, zoneId, arch) for listing OS categories
api/src/main/java/org/apache/cloudstack/api/command/admin/guest/UpdateGuestOsCmd.java	Updated API parameters and details processing with new osCategoryId parameter
api/src/main/java/org/apache/cloudstack/api/command/admin/guest/AddGuestOsCategoryCmd.java, UpdateGuestOsCategoryCmd.java, DeleteGuestOsCategoryCmd.java	New API commands for managing OS categories in the admin interface
api/src/main/java/org/apache/cloudstack/api/ResponseGenerator.java, ApiConstants.java, ManagementService.java	Updated to support the newly added guest OS category functionality

Comments suppressed due to low confidence (1)

api/src/main/java/org/apache/cloudstack/api/command/admin/guest/UpdateGuestOsCmd.java:53

The required attribute was removed from the OS_DISPLAY_NAME parameter. Verify if this change is intentional, as it might affect how updates to the guest OS are validated.

@Parameter(name = ApiConstants.OS_DISPLAY_NAME, type = CommandType.STRING, description = "Unique display name for Guest OS")

Copilot · 2025-05-05T07:56:20Z

engine/schema/src/main/java/com/cloud/storage/dao/GuestOSDaoImpl.java

@@ -152,4 +153,14 @@ public Pair<List<? extends GuestOS>, Integer> listGuestOSByCriteria(Long startIn
        return new Pair<>(result.first(), result.second());
    }



[nitpick] Consider adding a brief comment for the new listIdsByCategoryId method to clarify its purpose and usage in retrieving guest OS IDs by category.

Suggested change

/**

* Retrieves a list of guest OS IDs that belong to the specified category.

*

* @param categoryId the ID of the guest OS category

* @return a list of guest OS IDs

*/

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

vishesh92 · 2025-05-05T08:06:10Z

@shwstppr Found one bug in listTemplates command.
After setting showAllCategoryForModernImageSelection to true in the config.json, if you select the all category the request to the MS fails with the below error.

Unable to find on DB, due to: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')  AND template_view.account_type != 5  AND template_view.hypervisor_type='KVM' ' at line 1

Query params for the failed request

{
			"oscategoryid": "-1",
			"page": "1",
			"pageSize": "10",
			"zoneid": "cd313a1e-89f5-4d20-bfab-2ec2f41d259c",
			"account": "admin",
			"domainid": "22bb67fe-2976-11f0-93a0-1e0077000180",
			"templatefilter": "all",
			"details": "all",
			"showicon": "true",
			"isvnf": "false",
			"command": "listTemplates"
}

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2025-05-05T08:29:57Z

@vishesh92 thanks for the review and testing. I've addressed the review comments and fixed the listTemplates issue
@blueorangutan package

vishesh92

clgtm. did basic testing of the UI.

blueorangutan · 2025-05-05T08:32:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-05-05T09:41:09Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13259

boring-cyborg bot added component:api component:management-server component:marvin component:storage component:UI Python Warning... Python code Ahead! labels Apr 24, 2025

shwstppr changed the title ~~Deployvm improvements~~ ui,api,server: template categorization based on os Apr 24, 2025

shwstppr mentioned this pull request Apr 28, 2025

notes on guest os category changes and modern image selection apache/cloudstack-documentation#500

Draft

shwstppr force-pushed the deployvm-improvements branch from f9852f0 to 90294a6 Compare April 28, 2025 12:35

reorganize os categories

7fc3865

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fixes for isoid as queryparam

6d2c124

show only required elements when query params are passed Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

tests

eda7679

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr requested a review from vishesh92 April 30, 2025 08:54

shwstppr added 2 commits April 30, 2025 15:08

allow updating category for multiple os

f2e1b57

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix user template listing

14cf02a

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

vishesh92 requested a review from Copilot April 30, 2025 11:34

Copilot AI reviewed Apr 30, 2025

View reviewed changes

shwstppr added 3 commits April 30, 2025 17:08

cosistent

3ad581e

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

ui fixes

bcc64cc

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

filter catedories for vnf templates

138c704

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr requested a review from Pearl1594 May 5, 2025 06:09

Pearl1594 reviewed May 5, 2025

View reviewed changes

ui/src/views/AutogenView.vue Show resolved Hide resolved

Pearl1594 approved these changes May 5, 2025

View reviewed changes

vishesh92 requested changes May 5, 2025

View reviewed changes

add review comments

d9f37ce

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr added a commit to shwstppr/cloudstack-go that referenced this pull request May 5, 2025

compatibility: restore updateGuestOs params for <= 4.20.0

ae662a9

Related to apache/cloudstack#10773 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

vishesh92 reviewed May 5, 2025

View reviewed changes

ui/public/locales/en.json Outdated Show resolved Hide resolved

ui/public/locales/en.json Outdated Show resolved Hide resolved

ui/public/locales/en.json Outdated Show resolved Hide resolved

shwstppr mentioned this pull request May 5, 2025

compatibility: restore updateGuestOs params for <= 4.20.0 apache/cloudstack-go#109

Draft

shwstppr added a commit to shwstppr/cloudstack-go that referenced this pull request May 5, 2025

compatibility: restore updateGuestOs params for <= 4.20.0

b2094d4

Related to apache/cloudstack#10773 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

vishesh92 requested a review from Copilot May 5, 2025 07:55

Copilot AI reviewed May 5, 2025

View reviewed changes

address ui comments

6dbc2cc

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Pearl1594 added this to the 4.20.1 milestone May 5, 2025

Pearl1594 added this to ACS 4.20.1 May 5, 2025

Pearl1594 moved this to In Progress in ACS 4.20.1 May 5, 2025

fix issue with all category

01a66bd

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

vishesh92 approved these changes May 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ui,api,server: template categorization based on os #10773

ui,api,server: template categorization based on os #10773

shwstppr commented Apr 24, 2025 •

edited

Loading

codecov bot commented Apr 24, 2025 •

edited

Loading

DaanHoogland commented Apr 24, 2025

shwstppr commented Apr 28, 2025

blueorangutan commented Apr 28, 2025

blueorangutan commented Apr 28, 2025

shwstppr commented Apr 30, 2025

blueorangutan commented Apr 30, 2025

blueorangutan commented Apr 30, 2025

Copilot AI left a comment

Copilot AI Apr 30, 2025

shwstppr commented Apr 30, 2025

blueorangutan commented Apr 30, 2025

blueorangutan commented Apr 30, 2025

shwstppr commented May 5, 2025

blueorangutan commented May 5, 2025

Pearl1594 left a comment

vishesh92 left a comment

vishesh92 May 5, 2025

shwstppr May 5, 2025

shwstppr May 5, 2025

vishesh92 left a comment

Copilot AI left a comment

Copilot AI May 5, 2025

vishesh92 commented May 5, 2025

shwstppr commented May 5, 2025

vishesh92 left a comment

blueorangutan commented May 5, 2025

blueorangutan commented May 5, 2025

		@@ -152,4 +153,14 @@ public Pair<List<? extends GuestOS>, Integer> listGuestOSByCriteria(Long startIn
		return new Pair<>(result.first(), result.second());
		}

+    /**
+     * Retrieves a list of guest OS IDs that belong to the specified category.
+     *
+     * @param categoryId the ID of the guest OS category
+     * @return a list of guest OS IDs
+     */

ui,api,server: template categorization based on os #10773

Are you sure you want to change the base?

ui,api,server: template categorization based on os #10773

Conversation

shwstppr commented Apr 24, 2025 • edited Loading

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov bot commented Apr 24, 2025 • edited Loading

Codecov Report

DaanHoogland commented Apr 24, 2025

shwstppr commented Apr 28, 2025

blueorangutan commented Apr 28, 2025

blueorangutan commented Apr 28, 2025

shwstppr commented Apr 30, 2025

blueorangutan commented Apr 30, 2025

blueorangutan commented Apr 30, 2025

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Copilot AI Apr 30, 2025

Choose a reason for hiding this comment

shwstppr commented Apr 30, 2025

blueorangutan commented Apr 30, 2025

blueorangutan commented Apr 30, 2025

shwstppr commented May 5, 2025

blueorangutan commented May 5, 2025

Pearl1594 left a comment

Choose a reason for hiding this comment

vishesh92 left a comment

Choose a reason for hiding this comment

vishesh92 May 5, 2025

Choose a reason for hiding this comment

shwstppr May 5, 2025

Choose a reason for hiding this comment

shwstppr May 5, 2025

Choose a reason for hiding this comment

vishesh92 left a comment

Choose a reason for hiding this comment

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Copilot AI May 5, 2025

Choose a reason for hiding this comment

vishesh92 commented May 5, 2025

shwstppr commented May 5, 2025

vishesh92 left a comment

Choose a reason for hiding this comment

blueorangutan commented May 5, 2025

blueorangutan commented May 5, 2025

shwstppr commented Apr 24, 2025 •

edited

Loading

codecov bot commented Apr 24, 2025 •

edited

Loading