Handle with VM snapshot events by RodrigoDLopez · Pull Request #4251 · apache/cloudstack

RodrigoDLopez · 2020-08-07T19:35:56Z

Description

On these days, when a VM snapshot is deleted, the cloudstack-usage still seeing that snapshot as active and continues billing this deleted snapshot. This PR proposes new methods to handle with vm_snapshot_create and vm_snapshot_delete events.
Also, add some unit tests to cover those new methods.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

How Has This Been Tested?

I don't know why the cloudstack-usage duplicate entries for vm_snapshot. I think this is a bug and I'll fix it in a future PR.
To run tests, I create a VM snapshot and used the cloudmonkey to list usageRecords type=25.

Created at 14:12

() 🐱 > list usagerecords startdate=2020-08-08 enddate=2020-08-08 type=25
{
  "count": 2,
  "usagerecord": [
    {
      "account": "admin",
      "description": "VMSnapshot usage Id: 1 (7b63f2f0-61fc-4034-8438-26e8989e3503)  for VM VM-ccb8b3d9-18aa-40c3-8e13-adde2de9b34b (ccb8b3d9-18aa-40c3-8e13-adde2de9b34b)",
      "enddate": "2020-08-08'T'14:13:31+00:00",
      "name": "i-2-5-VM_VS_20200808141157",
      "rawusage": "0.025106",
      "size": 214749184,
      "startdate": "2020-08-08'T'14:12:01+00:00",
      "usage": "0.025106 Hrs",
      "usagetype": 25,
      "virtualmachineid": "ccb8b3d9-18aa-40c3-8e13-adde2de9b34b",
    },
    {
      "account": "admin",
      "description": "VMSnapshot usage Id: 1 (7b63f2f0-61fc-4034-8438-26e8989e3503)  for VM VM-ccb8b3d9-18aa-40c3-8e13-adde2de9b34b (ccb8b3d9-18aa-40c3-8e13-adde2de9b34b)",
      "enddate": "2020-08-08'T'14:13:31+00:00",
      "name": "i-2-5-VM_VS_20200808141157",
      "rawusage": "0.025106",
      "size": 214749184,
      "startdate": "2020-08-08'T'14:12:01+00:00",
      "usage": "0.025106 Hrs",
      "usagetype": 25,
      "virtualmachineid": "ccb8b3d9-18aa-40c3-8e13-adde2de9b34b",
    }
  ]
}

Deleted at 14:18

() 🐱 > list usagerecords startdate=2020-08-08 enddate=2020-08-08 type=25
{
  "count": 4,
  "usagerecord": [
    {
      "account": "admin",
      "description": "VMSnapshot usage Id: 1 (7b63f2f0-61fc-4034-8438-26e8989e3503)  for VM VM-ccb8b3d9-18aa-40c3-8e13-adde2de9b34b (ccb8b3d9-18aa-40c3-8e13-adde2de9b34b)",
      "enddate": "2020-08-08'T'14:13:31+00:00",
      "name": "i-2-5-VM_VS_20200808141157",
      "rawusage": "0.025106",
      "size": 214749184,
      "startdate": "2020-08-08'T'14:12:01+00:00",
      "usage": "0.025106 Hrs",
      "usagetype": 25,
      "virtualmachineid": "ccb8b3d9-18aa-40c3-8e13-adde2de9b34b",
    },
    {
      "account": "admin",
      "description": "VMSnapshot usage Id: 1 (7b63f2f0-61fc-4034-8438-26e8989e3503)  for VM VM-ccb8b3d9-18aa-40c3-8e13-adde2de9b34b (ccb8b3d9-18aa-40c3-8e13-adde2de9b34b)",
      "enddate": "2020-08-08'T'14:13:31+00:00",
      "name": "i-2-5-VM_VS_20200808141157",
      "rawusage": "0.025106",
      "size": 214749184,
      "startdate": "2020-08-08'T'14:12:01+00:00",
      "usage": "0.025106 Hrs",
      "usagetype": 25,
      "virtualmachineid": "ccb8b3d9-18aa-40c3-8e13-adde2de9b34b",
    },
    {
      "account": "admin",
      "description": "VMSnapshot usage Id: 1 (7b63f2f0-61fc-4034-8438-26e8989e3503)  for VM VM-ccb8b3d9-18aa-40c3-8e13-adde2de9b34b (ccb8b3d9-18aa-40c3-8e13-adde2de9b34b)",
      "enddate": "2020-08-08'T'14:18:31+00:00",
      "name": "i-2-5-VM_VS_20200808141157",
      "rawusage": "0.083337",
      "size": 214749184,
      "startdate": "2020-08-08'T'14:13:31+00:00",
      "usage": "0.083337 Hrs",
      "usagetype": 25,
      "virtualmachineid": "ccb8b3d9-18aa-40c3-8e13-adde2de9b34b",
    },
    {
      "account": "admin",
      "description": "VMSnapshot usage Id: 1 (7b63f2f0-61fc-4034-8438-26e8989e3503)  for VM VM-ccb8b3d9-18aa-40c3-8e13-adde2de9b34b (ccb8b3d9-18aa-40c3-8e13-adde2de9b34b)",
      "enddate": "2020-08-08'T'14:18:31+00:00",
      "name": "i-2-5-VM_VS_20200808141157",
      "rawusage": "0.083337",
      "size": 214749184,
      "startdate": "2020-08-08'T'14:13:31+00:00",
      "usage": "0.083337 Hrs",
      "usagetype": 25,
      "virtualmachineid": "ccb8b3d9-18aa-40c3-8e13-adde2de9b34b",
    }
  ]
}

Create methods to handle with VM snapshot create and delete events Add unit tests to cover new code

RodrigoDLopez · 2020-08-10T23:28:15Z

@blueorangutan package

blueorangutan · 2020-08-10T23:29:02Z

@RodrigoDLopez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-08-11T00:22:26Z

Packaging result: ✔centos7 ✔debian. JID-1701

RodrigoDLopez · 2020-08-11T12:05:42Z

@blueorangutan test

yadvr · 2020-08-13T10:56:13Z

@blueorangutan package

blueorangutan · 2020-08-13T10:56:56Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-08-13T12:05:16Z

Packaging result: ✔centos7 ✔debian. JID-1740

yadvr · 2020-08-28T09:24:24Z

@blueorangutan package

blueorangutan · 2020-08-28T09:24:49Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-08-28T10:43:46Z

Packaging result: ✔centos7 ✖centos8 ✖debian. JID-1843

yadvr

LGTM, did not test it; travis tests LGTM

yadvr · 2020-09-09T23:11:16Z

@blueorangutan package

blueorangutan · 2020-09-09T23:12:34Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-09-10T00:14:02Z

Packaging result: ✔centos7 ✖centos8 ✖debian. JID-1946

DaanHoogland · 2020-09-18T14:26:22Z

@blueorangutan package

blueorangutan · 2020-09-18T14:27:36Z

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-09-18T15:26:03Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2043

yadvr · 2020-09-21T08:44:06Z

@blueorangutan test

blueorangutan · 2020-09-21T08:45:07Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

yadvr · 2020-09-23T05:40:13Z

@blueorangutan test

blueorangutan · 2020-09-23T05:41:13Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-09-23T20:23:42Z

Trillian test result (tid-2840)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 50708 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4251-t2840-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 82 look OK, 3 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_delete_kubernetes_supported_version	`Error`	1807.27	test_kubernetes_supported_versions.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	368.65	test_vpc_redundant.py
test_hostha_kvm_host_fencing	`Error`	178.49	test_hostha_kvm.py

DaanHoogland

code looks good but the double records are kind of a problem, does this happen in the old/current implementation as well?

DaanHoogland · 2020-09-29T09:21:27Z

I don't know why the cloudstack-usage duplicate entries for vm_snapshot. I think this is a bug and I'll fix it in a future PR.

Did you create an issue yet? please do so before merging. maybe these double records should be mentioned in release notes as well (cc @PaulAngus )

RodrigoDLopez · 2020-09-29T14:28:29Z

Hi @DaanHoogland
I saw this bug when I was fixing other issues in the 4.13.1.0 version. On this version, we can see duplicates records for vm_snapshot, I did not test this on master, but I saw the code. The duplicate records will occur on master as well.

As you requested, I will open an issue to report these duplicate records for vm_snapshot on cloud-usage

GabrielBrascher

Code LGTM

yadvr · 2020-10-12T10:21:20Z

@nvazquez can you review this? thanks.

nvazquez

Code looks good, just left a comment

usage/src/main/java/com/cloud/usage/UsageManagerImpl.java

DaanHoogland · 2020-10-15T11:30:40Z

@blueorangutan package

blueorangutan · 2020-10-15T11:31:39Z

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-10-15T12:03:59Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2187

DaanHoogland · 2020-10-15T12:40:26Z

@blueorangutan test

blueorangutan · 2020-10-15T12:41:38Z

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-10-16T00:52:08Z

Trillian test result (tid-2971)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 42028 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4251-t2971-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 83 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	338.39	test_vpc_redundant.py
test_hostha_kvm_host_fencing	`Error`	175.20	test_hostha_kvm.py

Handle with VM snapshot events

97f02cd

Create methods to handle with VM snapshot create and delete events Add unit tests to cover new code

RodrigoDLopez changed the title ~~Handle with VM snapshot events~~ [WIP] Handle with VM snapshot events Aug 7, 2020

GabrielBrascher added this to the 4.15.0.0 milestone Aug 7, 2020

GabrielBrascher added the status:work-in-progress label Aug 7, 2020

GabrielBrascher assigned RodrigoDLopez Aug 7, 2020

DaanHoogland marked this pull request as draft August 9, 2020 18:39

DaanHoogland changed the title ~~[WIP] Handle with VM snapshot events~~ Handle with VM snapshot events Aug 9, 2020

RodrigoDLopez marked this pull request as ready for review August 10, 2020 21:02

yadvr requested review from andrijapanicsb, borisstoyanov and nvazquez August 13, 2020 10:55

svenvogel added type:bug type:enhancement component:usage-server labels Aug 13, 2020

yadvr requested review from GabrielBrascher, weizhouapache and wido September 1, 2020 08:20

yadvr approved these changes Sep 1, 2020

View reviewed changes

yadvr requested a review from DaanHoogland September 29, 2020 07:10

DaanHoogland approved these changes Sep 29, 2020

View reviewed changes

GabrielBrascher approved these changes Sep 30, 2020

View reviewed changes

nvazquez reviewed Oct 13, 2020

View reviewed changes

usage/src/main/java/com/cloud/usage/UsageManagerImpl.java Outdated Show resolved Hide resolved

move if statement to be outside

724cd51

DaanHoogland merged commit d676ffa into apache:master Oct 16, 2020

RodrigoDLopez deleted the handle_with_vm_snapshot_event branch October 30, 2020 13:51

PaulAngus removed status:work-in-progress type:bug labels Nov 8, 2020

Conversation

RodrigoDLopez commented Aug 7, 2020 • edited by PaulAngus Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

How Has This Been Tested?

Uh oh!

RodrigoDLopez commented Aug 10, 2020

Uh oh!

blueorangutan commented Aug 10, 2020

Uh oh!

blueorangutan commented Aug 11, 2020

Uh oh!

RodrigoDLopez commented Aug 11, 2020

Uh oh!

yadvr commented Aug 13, 2020

Uh oh!

blueorangutan commented Aug 13, 2020

Uh oh!

blueorangutan commented Aug 13, 2020

Uh oh!

yadvr commented Aug 28, 2020

Uh oh!

blueorangutan commented Aug 28, 2020

Uh oh!

blueorangutan commented Aug 28, 2020

Uh oh!

yadvr left a comment

Choose a reason for hiding this comment

Uh oh!

yadvr commented Sep 9, 2020

Uh oh!

blueorangutan commented Sep 9, 2020

Uh oh!

blueorangutan commented Sep 10, 2020

Uh oh!

DaanHoogland commented Sep 18, 2020

Uh oh!

blueorangutan commented Sep 18, 2020

Uh oh!

blueorangutan commented Sep 18, 2020

Uh oh!

yadvr commented Sep 21, 2020

Uh oh!

blueorangutan commented Sep 21, 2020

Uh oh!

yadvr commented Sep 23, 2020

Uh oh!

blueorangutan commented Sep 23, 2020

Uh oh!

blueorangutan commented Sep 23, 2020

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Sep 29, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RodrigoDLopez commented Sep 29, 2020

Uh oh!

GabrielBrascher left a comment

Choose a reason for hiding this comment

Uh oh!

yadvr commented Oct 12, 2020

Uh oh!

nvazquez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DaanHoogland commented Oct 15, 2020

Uh oh!

blueorangutan commented Oct 15, 2020

Uh oh!

blueorangutan commented Oct 15, 2020

Uh oh!

DaanHoogland commented Oct 15, 2020

Uh oh!

blueorangutan commented Oct 15, 2020

Uh oh!

blueorangutan commented Oct 16, 2020

RodrigoDLopez commented Aug 7, 2020 •

edited by PaulAngus

Loading

DaanHoogland commented Sep 29, 2020 •

edited

Loading