Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding check if disk description is empty #46

Merged

Conversation

slavina-rumenova
Copy link
Contributor

@slavina-rumenova slavina-rumenova commented Nov 16, 2020

In order to prevent velero backups of being marked as PartialyFailed for disks, having no description set (as it is an optional argument), adding a check if the disk description is empty before parsing the description json. If empty only the velero tags are set on the snapshot.

Change fixes vmware-tanzu/velero#3149.

Signed-off-by: Slavina Sobadjieva slavina.rumenova@gmail.com

@slavina-rumenova slavina-rumenova changed the title Adding check if disk description is empty in issue #45 Adding check if disk description is empty Fixes #45 Nov 16, 2020
@slavina-rumenova slavina-rumenova changed the title Adding check if disk description is empty Fixes #45 Adding check if disk description is empty fixes #45 Nov 16, 2020
@slavina-rumenova slavina-rumenova changed the title Adding check if disk description is empty fixes #45 Adding check if disk description is empty Nov 16, 2020
Signed-off-by: Slavina Sobadjieva <slavina.rumenova@gmail.com>
Copy link
Contributor

@carlisia carlisia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This lgtm and thank you.

I wonder if it'd be useful to add a warning on LN 294 that the description is not being used.

@carlisia carlisia added the bug Something isn't working label Dec 17, 2020
@Helter-skelter42
Copy link

Any update on this one ? :)

@jenting jenting merged commit 330b845 into vmware-tanzu:main Feb 27, 2021
@jravetch
Copy link

We are running into this issue with disks that do not have a description. When can we expect a new release for the plugin?

@carlisia
Copy link
Contributor

I am going to push an image for the next release of this plugin (v1.2.0) tomorrow. It should still be compatible with the latest versions of Velero. There'll be a Velero RC for v1.6 this week.

@jravetch
Copy link

@carlisia Thank you! This commit e677e89 states that plugin 1.2.0 is only compatible with velero 1.6.0. Also can you add to the changelog the bug that is fixed by this PR?

@carlisia
Copy link
Contributor

carlisia commented Mar 16, 2021

The v1.2.0 image for this plugin is available.

@jravetch thanks for the callout, I'm making a note to add this to the release changelog. I haven't cut that yet, we are testing things first.

We would like for people to upgrade Velero as they upgrade their plugins but there is no known compatibility issues between the new v1.2.0 and the earlier versions of Velero.

@jravetch
Copy link

@carlisia I updated the deployment to use the new plugin version, but I am still getting the same error. Below is the output when I describe the latest backup run

Phase:  PartiallyFailed (run `velero backup logs velero-schedule-config-20210317194135` for more information)

Errors:    1             
Warnings:  0
                            
Namespaces:
  Included:  *                     
  Excluded:  kube-system, velero
                
Resources:
  Included:        *
  Excluded:        <none>
  Cluster-scoped:  auto                    
                                                                                                                                                                                                                                                                                                                                                          
Label selector:  <none>                                                                                              
                                                                                                                   
Storage Location:  default                                                                                                                                                                                                                                                                                                                                

Velero-Native Snapshot PVs:  auto

TTL:  720h0m0s

Hooks:  <none>

Backup Format Version:  1.1.0

Started:    2021-03-17 19:41:35 +0000 UTC
Completed:  2021-03-17 19:43:45 +0000 UTC

Expiration:  2021-04-16 19:41:35 +0000 UTC

Total items to be backed up:  9587
Items backed up:              9587

Velero-Native Snapshots:  14 of 14 snapshots completed successfully (specify --details for more information)

Why isn't the backup format version changed? Am I missing something?

@carlisia
Copy link
Contributor

Let me look into it. Why did you expect the backup format version to be different?

Also, let me see what your backup log says, please paste all of it for me.

@jravetch
Copy link

jravetch commented Mar 17, 2021

Oh I thought the format version was tied to the plugin version. Here's the output of the logs, grepping only for errors.

time="2021-03-17T19:42:43Z" level=error msg="unable to decode disk's description as JSON, so only applying Velero-assigned tags to snapshot" backup=velero/velero-schedule-config-20210317194135 cmd=/plugins/velero-plugin-for-gcp error="invalid character 'p' looking for beginning of value" logSource="/go/src/velero-plugin-for-gcp/velero-plugin-for-gcp/volume_snapshotter.go:283" pluginName=velero-plugin-for-gcp

Checking the description of the disk, it's not in JSON, just a string. So the plugin is still throwing the error and marking the backup as partially failed.

@carlisia
Copy link
Contributor

Just for sanity, what's the output of kubectl describe pod velero | grep Image:?

@jravetch
Copy link

    Image:          velero/velero-plugin-for-gcp:v1.2.0
    Image:          velero/velero:v1.5.3

@jravetch
Copy link

So i believe if this line is a string, but not JSON, the condition will still fail.

@carlisia
Copy link
Contributor

Unfortunately I didn't verify that a test be added for this new condition, which would've served as a double purpose of clarifying the intent of the change.

The conditions that determine if Velero tag will be applied or not are listed here:

https://github.com/vmware-tanzu/velero-plugin-for-gcp/blob/main/velero-plugin-for-gcp/volume_snapshotter_test.go#L93-L134

@jravetch
Copy link

jravetch commented Mar 17, 2021

So there is already a test case. Can we change the error log to just info instead?

log.Info("unable to decode disk's description as JSON, so only applying Velero-assigned tags to snapshot")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Backups marked as PartialyFailed when snapshotting a GCE disk without description
6 participants