kvm: fix direct download template size by shwstppr · Pull Request #8093 · apache/cloudstack

shwstppr · 2023-10-13T09:40:27Z

Description

Fixes #8037

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Fixes apache#8037 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2023-10-13T09:41:07Z

@blueorangutan package

blueorangutan · 2023-10-13T09:42:04Z

@shwstppr a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

yadvr · 2023-10-13T10:22:24Z

utils/src/main/java/com/cloud/utils/storage/QCOW2Utils.java

        try {
            URL url = new URL(urlStr);
-            return getVirtualSizeFromInputStream(url.openStream());
+            boolean isCompressed = !urlStr.endsWith("qcow2");


Should we instaed check perhaps for compression extension such as tar.gz, bz2 etc? (what is somebody uploads an .img file which is a qcow2?)

@rohityadavcloud made the change to use UriUtils.COMMPRESSION_FORMATS.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

blueorangutan · 2023-10-13T10:49:49Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7353

DaanHoogland · 2023-10-13T11:27:41Z

@blueorangutan test rocky8 kvm-rocky8

blueorangutan · 2023-10-13T11:28:03Z

@DaanHoogland a [SF] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests

DaanHoogland

looks good

core/src/main/java/org/apache/cloudstack/direct/download/HttpsDirectTemplateDownloader.java

codecov · 2023-10-13T11:34:17Z

Codecov Report

Merging #8093 (9d09b24) into 4.18 (3e7f21a) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               4.18    #8093   +/-   ##
=========================================
  Coverage     13.06%   13.06%           
  Complexity     9109     9109           
=========================================
  Files          2720     2720           
  Lines        257526   257526           
  Branches      40150    40150           
=========================================
  Hits          33655    33655           
  Misses       219644   219644           
  Partials       4227     4227

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

blueorangutan · 2023-10-14T06:55:22Z

[SF] Trillian test result (tid-7969)
Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8
Total time taken: 47961 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8093-t7969-kvm-rocky8.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_03_deploy_vm_wrong_checksum	`Error`	42.89	test_templates.py
test_09_list_templates_download_details	`Failure`	0.07	test_templates.py

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2023-10-14T16:24:09Z

@blueorangutan package

blueorangutan · 2023-10-14T16:26:03Z

@shwstppr a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-10-14T17:24:27Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7361

shwstppr · 2023-10-14T17:29:17Z

@blueorangutan test

blueorangutan · 2023-10-14T17:30:03Z

@shwstppr a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2023-10-15T07:35:02Z

[SF] Trillian test result (tid-7973)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49262 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8093-t7973-kvm-centos7.zip
Smoke tests completed. 106 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	620.66	test_kubernetes_clusters.py
test_03_deploy_vm_wrong_checksum	`Error`	39.60	test_templates.py
test_09_list_templates_download_details	`Failure`	0.05	test_templates.py

* 4.18: test,refactor: fix test_project_resources cleanup (#8097) kvm: fix direct download template size (#8093)

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> (cherry picked from commit ba24a18) Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

kvm: fix direct download HTTP template size

e34ea1f

Fixes apache#8037 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr requested a review from nvazquez October 13, 2023 09:40

shwstppr requested review from DaanHoogland, harikrishna-patnala and kiranchavala October 13, 2023 09:42

yadvr reviewed Oct 13, 2023

View reviewed changes

compressed only for particular formats

dd8d358

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr changed the title ~~kvm: fix direct download HTTP template size~~ kvm: fix direct download template size Oct 13, 2023

DaanHoogland approved these changes Oct 13, 2023

View reviewed changes

core/src/main/java/org/apache/cloudstack/direct/download/HttpsDirectTemplateDownloader.java Outdated Show resolved Hide resolved

refactor

9d09b24

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

yadvr approved these changes Oct 16, 2023

View reviewed changes

yadvr added this to the 4.18.2.0 milestone Oct 16, 2023

DaanHoogland merged commit ba24a18 into apache:4.18 Oct 16, 2023

DaanHoogland deleted the fix-dd-templ-size branch October 16, 2023 12:52

DaanHoogland added a commit that referenced this pull request Oct 16, 2023

Merge release branch 4.18 to main

c9d2303

* 4.18: test,refactor: fix test_project_resources cleanup (#8097) kvm: fix direct download template size (#8093)

hulucc mentioned this pull request Oct 22, 2024

Http direct downloader can not get size of non compressed qcow2 #9835

Closed

GutoVeronezi mentioned this pull request Oct 29, 2024

Unable to deploy a vm from direct download templates on 4.18.2.x releases #9860

Closed

Conversation

shwstppr commented Oct 13, 2023

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

shwstppr commented Oct 13, 2023

Uh oh!

blueorangutan commented Oct 13, 2023

Uh oh!

yadvr Oct 13, 2023

Choose a reason for hiding this comment

Uh oh!

shwstppr Oct 13, 2023

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Oct 13, 2023

Uh oh!

DaanHoogland commented Oct 13, 2023

Uh oh!

blueorangutan commented Oct 13, 2023

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Oct 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Oct 14, 2023

Uh oh!

shwstppr commented Oct 14, 2023

Uh oh!

blueorangutan commented Oct 14, 2023

Uh oh!

blueorangutan commented Oct 14, 2023

Uh oh!

shwstppr commented Oct 14, 2023

Uh oh!

blueorangutan commented Oct 14, 2023

Uh oh!

blueorangutan commented Oct 15, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Oct 13, 2023 •

edited

Loading