restic: unable to restore "Local" Persistent Volumes since they're statically provisioned #2520
Description
What steps did you take and what happened:
I backed up a namespace with pods and PVC's which included Persistent Volumes with a "local" spec (which the documentation suggests is supported - https://github.com/vmware-tanzu/velero/blob/master/site/docs/master/restic.md #), and associated pods annotated with the volume names.
The backup appears to work - and as far as I can tell, Restic backs up the Persistent Volume.
However when attempting to restore the namespace the PV with "local" storage never seems to get restored. Sometimes Restic hangs for an hour before timing out waiting for the restore to complete ie:-
level=info msg="Waiting for all restic restores to complete" logSource="pkg/restore/restore.go:545" restore=velero/nginx-example3-20200507164146
level=error msg="unable to successfully complete restic restores of pod's volumes" error="timed out waiting for all PodVolumeRestores to complete"
Other times the restore completes without error - but no PV is brought down (though the namespace, pods and PVC's are restored - though in a forever "pending" state due to the missing PV)
In the "successful" restores - despite the PV being deleted from the cluster (along with namespace, pod deployments etc.) The logs seem to imply that restic/velero think the PV has already been restored and is thus skipped
time="2020-05-07T16:32:50Z" level=info msg="Skipping persistentvolumes/nginx-logs-volume because it's already been restored." logSource="pkg/restore/restore.go:910" restore=velero/nginx-example2-20200507163249
time="2020-05-07T16:32:50Z" level=info msg="Executing item action for persistentvolumeclaims" logSource="pkg/restore/restore.go:1030" restore=velero/nginx-example2-20200507163249
time="2020-05-07T16:32:50Z" level=info msg="Executing ChangeStorageClassAction" cmd=/velero logSource="pkg/restore/change_storageclass_action.go:63" pluginName=velero restore=velero/nginx-example2-20200507163249
time="2020-05-07T16:32:50Z" level=info msg="Done executing ChangeStorageClassAction" cmd=/velero logSource="pkg/restore/change_storageclass_action.go:74" pluginName=velero restore=velero/nginx-example2-20200507163249
time="2020-05-07T16:32:50Z" level=info msg="Resetting PersistentVolumeClaim nginx-example/nginx-logs for dynamic provisioning because its PV nginx-logs-volume has a reclaim policy of Delete" logSource="pkg/restore/restore.go:1103" restore=velero/nginx-example2-20200507163249
I have also tried to just restore the PV's on its own (with no preexisting pods or pod deployments) and this hasn't worked, both with a reclaim policy of Delete and Retain.
The reason I'm trying to use "local" storage is that this is a environment that is on prem and not in the cloud - the existing PV's use "hostpath" which restic doesnt support - so attempting to prove backups and restores using "local" - with a view of eventually changing the existing PV's to "local" from "hostpath" if this works.
For context - I also backed up a namespace which had a PV under it using "rook-ceph-block" storageclass - deleted all resources - and restored - and it worked fine, bringing the PV back along with associated data - so I can only surmise that restic isnt liking my PV's with "local" persistent volumes, or I am doing something wrong.
What did you expect to happen:
The Persistent volume(s) to restore along with data - and pods/PVC's to come up healthily.
Below is a quick demo of what I did.
velero install \
> --provider aws \
> --use-restic \
> --bucket cva-builder-velero-backups-test \
> --secret-file ./credentials-velero \
> --backup-location-config region=us-east-1 \
> --plugins velero/velero-plugin-for-aws:v1.0.1 \
> --snapshot-location-config region=us-east-1 \
> --use-volume-snapshots=true
kubectl get pods -n velero
NAME READY STATUS RESTARTS AGE
restic-469kh 1/1 Running 0 18s
restic-bclvc 1/1 Running 0 18s
velero-84b675ddc-p6ft7 1/1 Running 0 18s
kubectl apply -f nginx_example.yml
namespace/nginx-example created
persistentvolume/nginx-logs-volume created
persistentvolumeclaim/nginx-logs created
deployment.apps/nginx-deployment created
service/my-nginx created
kubectl get pv | grep nginx
nginx-logs-volume 10Gi RWO Delete Bound nginx-example/nginx-logs local-storage 4m40s
[centos@cva-builder:4.11.4-PR-1712-29 SANDBOX2 CVA-PR-1712 tmp]$ kubectl get pvc -n nginx-example
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
nginx-logs Bound nginx-logs-volume 10Gi RWO local-storage 4m53s
[centos@cva-builder:4.11.4-PR-1712-29 SANDBOX2 CVA-PR-1712 tmp]$ kubectl get pod -n nginx-example
NAME READY STATUS RESTARTS AGE
nginx-deployment-6fbd5f64c4-fps5c 1/1 Running 0 108s
kubectl get pv nginx-logs-volume -o yaml
apiVersion: v1
kind: PersistentVolume
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"PersistentVolume","metadata":{"annotations":{},"labels":{"app":"nginx","type":"local"},"name":"nginx-logs-volume"},"spec":{"accessModes":["ReadWriteOnce"],"capacity":{"storage":"10Gi"},"local":{"path":"/mnt/data"},"nodeAffinity":{"required":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"workerNodeName","operator":"In","values":["node-1"]}]}]}},"persistentVolumeReclaimPolicy":"Delete","storageClassName":"local-storage","volumeMode":"Filesystem"}}
pv.kubernetes.io/bound-by-controller: "yes"
creationTimestamp: "2020-05-08T08:56:36Z"
finalizers:
- kubernetes.io/pv-protection
labels:
app: nginx
type: local
name: nginx-logs-volume
resourceVersion: "298587"
selfLink: /api/v1/persistentvolumes/nginx-logs-volume
uid: d2b6957c-9109-11ea-a626-02021a19689b
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 10Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: nginx-logs
namespace: nginx-example
resourceVersion: "298572"
uid: d2bc87d4-9109-11ea-a626-02021a19689b
local:
path: /mnt/data
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: workerNodeName
operator: In
values:
- node-1
persistentVolumeReclaimPolicy: Delete
storageClassName: local-storage
volumeMode: Filesystem
status:
phase: Bound
kubectl get pod nginx-deployment-6fbd5f64c4-fps5c -n nginx-example -o yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2020-05-08T08:59:50Z"
generateName: nginx-deployment-6fbd5f64c4-
labels:
app: nginx
pod-template-hash: 6fbd5f64c4
name: nginx-deployment-6fbd5f64c4-fps5c
namespace: nginx-example
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: ReplicaSet
name: nginx-deployment-6fbd5f64c4
uid: d2c226da-9109-11ea-ac8a-028358a66085
resourceVersion: "299852"
selfLink: /api/v1/namespaces/nginx-example/pods/nginx-deployment-6fbd5f64c4-fps5c
uid: 4640dfa5-910a-11ea-ac8a-028358a66085
spec:
containers:
- image: nginx:1.7.9
imagePullPolicy: IfNotPresent
name: nginx
ports:
- containerPort: 80
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/log/nginx
name: nginx-logs
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-zmrr5
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: ip-10-165-138-230.ec2.internal
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: nginx-logs
persistentVolumeClaim:
claimName: nginx-logs
- name: default-token-zmrr5
secret:
defaultMode: 420
secretName: default-token-zmrr5
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2020-05-08T08:59:50Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2020-05-08T09:00:54Z"
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2020-05-08T09:00:54Z"
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2020-05-08T08:59:50Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: docker://735b18c8379ee6b452e95c6c2ab45d1c82a9086598300de7fd90374336bac13f
image: nginx:1.7.9
imageID: docker-pullable://nginx@sha256:e3456c851a152494c3e4ff5fcc26f240206abac0c9d794affb40e0714846c451
lastState: {}
name: nginx
ready: true
restartCount: 0
state:
running:
startedAt: "2020-05-08T09:00:54Z"
hostIP: 10.165.138.230
phase: Running
podIP: 130.174.242.199
qosClass: BestEffort
startTime: "2020-05-08T08:59:50Z"
kubectl get pvc -n nginx-example -o yaml
apiVersion: v1
items:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx-logs","namespace":"nginx-example"},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"50Mi"}},"storageClassName":"local-storage"}}
pv.kubernetes.io/bind-completed: "yes"
pv.kubernetes.io/bound-by-controller: "yes"
creationTimestamp: "2020-05-08T08:56:36Z"
finalizers:
- kubernetes.io/pvc-protection
labels:
app: nginx
name: nginx-logs
namespace: nginx-example
resourceVersion: "298592"
selfLink: /api/v1/namespaces/nginx-example/persistentvolumeclaims/nginx-logs
uid: d2bc87d4-9109-11ea-a626-02021a19689b
spec:
accessModes:
- ReadWriteOnce
dataSource: null
resources:
requests:
storage: 50Mi
storageClassName: local-storage
volumeMode: Filesystem
volumeName: nginx-logs-volume
status:
accessModes:
- ReadWriteOnce
capacity:
storage: 10Gi
phase: Bound
kind: List
metadata:
resourceVersion: ""
selfLink: ""
kubectl -n nginx-example annotate pod/nginx-deployment-6fbd5f64c4-fps5c backup.velero.io/backup-volumes=nginx-logs
pod/nginx-deployment-6fbd5f64c4-fps5c annotated
velero create backup nginx-example-backup-new --include-namespaces nginx-example --snapshot-volumes
Backup request "nginx-example-backup-new" submitted successfully.
Run `velero backup describe nginx-example-backup-new` or `velero backup logs nginx-example-backup-new` for more details.
velero backup describe nginx-example-backup-new --details
Name: nginx-example-backup-new
Namespace: velero
Labels: velero.io/storage-location=default
Annotations: <none>
Phase: Completed
Namespaces:
Included: nginx-example
Excluded: <none>
Resources:
Included: *
Excluded: <none>
Cluster-scoped: auto
Label selector: <none>
Storage Location: default
Snapshot PVs: true
TTL: 720h0m0s
Hooks: <none>
Backup Format Version: 1
Started: 2020-05-08 09:33:25 +0000 UTC
Completed: 2020-05-08 09:33:44 +0000 UTC
Expiration: 2020-06-07 09:33:25 +0000 UTC
Resource List:
apps/v1/Deployment:
- nginx-example/nginx-deployment
apps/v1/ReplicaSet:
- nginx-example/nginx-deployment-6fbd5f64c4
v1/Endpoints:
- nginx-example/my-nginx
v1/Event:
- nginx-example/nginx-deployment-6fbd5f64c4-2r7zk.160d01c6f35fac21
- nginx-example/nginx-deployment-6fbd5f64c4-2r7zk.160d01c6ffb7dd12
- nginx-example/nginx-deployment-6fbd5f64c4-2r7zk.160d01e397c7f57f
- nginx-example/nginx-deployment-6fbd5f64c4-2r7zk.160d020352295182
- nginx-example/nginx-deployment-6fbd5f64c4-fps5c.160d01f3d4be1543
- nginx-example/nginx-deployment-6fbd5f64c4-fps5c.160d02014a99421f
- nginx-example/nginx-deployment-6fbd5f64c4-fps5c.160d020281b1aab3
- nginx-example/nginx-deployment-6fbd5f64c4-fps5c.160d0202a4ed5f11
- nginx-example/nginx-deployment-6fbd5f64c4-fps5c.160d0202ae7423c0
- nginx-example/nginx-deployment-6fbd5f64c4.160d01c6b69e8011
- nginx-example/nginx-deployment-6fbd5f64c4.160d01f3d43d2119
- nginx-example/nginx-deployment.160d01c6b5b35eb3
- nginx-example/nginx-logs.160d01c6b399f74e
v1/Namespace:
- nginx-example
v1/PersistentVolume:
- nginx-logs-volume
v1/PersistentVolumeClaim:
- nginx-example/nginx-logs
v1/Pod:
- nginx-example/nginx-deployment-6fbd5f64c4-fps5c
v1/Secret:
- nginx-example/default-token-zmrr5
v1/Service:
- nginx-example/my-nginx
v1/ServiceAccount:
- nginx-example/default
Persistent Volumes: <none included>
Restic Backups:
Completed:
nginx-example/nginx-deployment-6fbd5f64c4-fps5c: nginx-logs
velero backup logs nginx-example-backup-new
time="2020-05-08T09:33:25Z" level=info msg="Setting up backup temp file" backup=velero/nginx-example-backup-new logSource="pkg/controller/backup_controller.go:494"
time="2020-05-08T09:33:25Z" level=info msg="Setting up plugin manager" backup=velero/nginx-example-backup-new logSource="pkg/controller/backup_controller.go:501"
time="2020-05-08T09:33:25Z" level=info msg="Getting backup item actions" backup=velero/nginx-example-backup-new logSource="pkg/controller/backup_controller.go:505"
time="2020-05-08T09:33:25Z" level=info msg="Setting up backup store" backup=velero/nginx-example-backup-new logSource="pkg/controller/backup_controller.go:511"
time="2020-05-08T09:33:25Z" level=info msg="Writing backup version file" backup=velero/nginx-example-backup-new logSource="pkg/backup/backup.go:213"
time="2020-05-08T09:33:25Z" level=info msg="Including namespaces: nginx-example" backup=velero/nginx-example-backup-new logSource="pkg/backup/backup.go:219"
time="2020-05-08T09:33:25Z" level=info msg="Excluding namespaces: <none>" backup=velero/nginx-example-backup-new logSource="pkg/backup/backup.go:220"
time="2020-05-08T09:33:25Z" level=info msg="Including resources: *" backup=velero/nginx-example-backup-new logSource="pkg/backup/backup.go:223"
time="2020-05-08T09:33:25Z" level=info msg="Excluding resources: <none>" backup=velero/nginx-example-backup-new logSource="pkg/backup/backup.go:224"
time="2020-05-08T09:33:40Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:40Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=pods
time="2020-05-08T09:33:40Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=pods
time="2020-05-08T09:33:40Z" level=info msg="Retrieved 1 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=pods
time="2020-05-08T09:33:40Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4-fps5c namespace=nginx-example resource=pods
time="2020-05-08T09:33:40Z" level=info msg="Executing custom action" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:330" name=nginx-deployment-6fbd5f64c4-fps5c namespace=nginx-example resource=pods
time="2020-05-08T09:33:40Z" level=info msg="Executing podAction" backup=velero/nginx-example-backup-new cmd=/velero logSource="pkg/backup/pod_action.go:51" pluginName=velero
time="2020-05-08T09:33:40Z" level=info msg="Adding pvc nginx-logs to additionalItems" backup=velero/nginx-example-backup-new cmd=/velero logSource="pkg/backup/pod_action.go:67" pluginName=velero
time="2020-05-08T09:33:40Z" level=info msg="Done executing podAction" backup=velero/nginx-example-backup-new cmd=/velero logSource="pkg/backup/pod_action.go:77" pluginName=velero
time="2020-05-08T09:33:40Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-logs namespace=nginx-example resource=persistentvolumeclaims
time="2020-05-08T09:33:40Z" level=info msg="Executing custom action" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:330" name=nginx-logs namespace=nginx-example resource=persistentvolumeclaims
time="2020-05-08T09:33:40Z" level=info msg="Executing PVCAction" backup=velero/nginx-example-backup-new cmd=/velero logSource="pkg/backup/backup_pv_action.go:49" pluginName=velero
time="2020-05-08T09:33:40Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-logs-volume namespace= resource=persistentvolumes
time="2020-05-08T09:33:40Z" level=info msg="Executing takePVSnapshot" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:400" name=nginx-logs-volume namespace= resource=persistentvolumes
time="2020-05-08T09:33:40Z" level=info msg="Skipping snapshot of persistent volume because volume is being backed up with restic." backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:418" name=nginx-logs-volume namespace= persistentVolume=nginx-logs-volume resource=persistentvolumes
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=persistentvolumeclaims
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=persistentvolumeclaims
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 1 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=persistentvolumeclaims
time="2020-05-08T09:33:42Z" level=info msg="Skipping item because it's already been backed up." backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:163" name=nginx-logs namespace=nginx-example resource=persistentvolumeclaims
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=persistentvolumes
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:127" resource=persistentvolumes
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=events
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 13 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4-2r7zk.160d01c6f35fac21 namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4-2r7zk.160d01c6ffb7dd12 namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4-2r7zk.160d01e397c7f57f namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4-2r7zk.160d020352295182 namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4-fps5c.160d01f3d4be1543 namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4-fps5c.160d02014a99421f namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4-fps5c.160d020281b1aab3 namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4-fps5c.160d0202a4ed5f11 namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4-fps5c.160d0202ae7423c0 namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4.160d01c6b69e8011 namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4.160d01f3d43d2119 namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment.160d01c6b5b35eb3 namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-logs.160d01c6b399f74e namespace=nginx-example resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=limitranges
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=limitranges
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=limitranges
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=services
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=services
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 1 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=services
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=my-nginx namespace=nginx-example resource=services
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=configmaps
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=configmaps
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=configmaps
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=resourcequotas
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=resourcequotas
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=resourcequotas
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=serviceaccounts
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=serviceaccounts
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 1 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=serviceaccounts
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=default namespace=nginx-example resource=serviceaccounts
time="2020-05-08T09:33:42Z" level=info msg="Executing custom action" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:330" name=default namespace=nginx-example resource=serviceaccounts
time="2020-05-08T09:33:42Z" level=info msg="Running ServiceAccountAction" backup=velero/nginx-example-backup-new cmd=/velero logSource="pkg/backup/service_account_action.go:77" pluginName=velero
time="2020-05-08T09:33:42Z" level=info msg="Done running ServiceAccountAction" backup=velero/nginx-example-backup-new cmd=/velero logSource="pkg/backup/service_account_action.go:120" pluginName=velero
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=nodes
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:127" resource=nodes
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=secrets
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=secrets
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 1 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=secrets
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=default-token-zmrr5 namespace=nginx-example resource=secrets
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=namespaces
time="2020-05-08T09:33:42Z" level=info msg="Getting namespace" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:184" namespace=nginx-example resource=namespaces
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-example namespace= resource=namespaces
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=endpoints
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=endpoints
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 1 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=endpoints
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/item_backupper.go:169" name=my-nginx namespace=nginx-example resource=endpoints
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=podtemplates
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=podtemplates
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=podtemplates
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:105" resource=replicationcontrollers
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=replicationcontrollers
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=replicationcontrollers
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=apiregistration.k8s.io/v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=apiregistration.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=apiservices
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=apiregistration.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:127" resource=apiservices
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:105" resource=controllerrevisions
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=controllerrevisions
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=controllerrevisions
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:105" resource=daemonsets
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=daemonsets
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=daemonsets
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:105" resource=statefulsets
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=statefulsets
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=statefulsets
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:105" resource=deployments
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=deployments
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 1 items" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=deployments
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment namespace=nginx-example resource=deployments.apps
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:105" resource=replicasets
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=replicasets
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 1 items" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=replicasets
time="2020-05-08T09:33:42Z" level=info msg="Backing up item" backup=velero/nginx-example-backup-new group=apps/v1 logSource="pkg/backup/item_backupper.go:169" name=nginx-deployment-6fbd5f64c4 namespace=nginx-example resource=replicasets.apps
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=events.k8s.io/v1beta1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=events.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=events
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it cohabitates and we've already processed it" backup=velero/nginx-example-backup-new cohabitatingResource1=events cohabitatingResource2=events.events.k8s.io group=events.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:148" resource=events
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=autoscaling/v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=autoscaling/v1 logSource="pkg/backup/resource_backupper.go:105" resource=horizontalpodautoscalers
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=autoscaling/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=horizontalpodautoscalers
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=autoscaling/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=horizontalpodautoscalers
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=batch/v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=batch/v1 logSource="pkg/backup/resource_backupper.go:105" resource=jobs
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=batch/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=jobs
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=batch/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=jobs
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=batch/v1beta1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=batch/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=cronjobs
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=batch/v1beta1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=cronjobs
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=batch/v1beta1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=cronjobs
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=certificates.k8s.io/v1beta1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=certificates.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=certificatesigningrequests
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=certificates.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:127" resource=certificatesigningrequests
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=networking.k8s.io/v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=networking.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=networkpolicies
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=networking.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=networkpolicies
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=networking.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=networkpolicies
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=policy/v1beta1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=policy/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=podsecuritypolicies
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=policy/v1beta1 logSource="pkg/backup/resource_backupper.go:127" resource=podsecuritypolicies
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=policy/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=poddisruptionbudgets
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=policy/v1beta1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=poddisruptionbudgets
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=policy/v1beta1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=poddisruptionbudgets
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=roles
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=roles
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=roles
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=clusterrolebindings
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:127" resource=clusterrolebindings
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=clusterroles
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:127" resource=clusterroles
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=rolebindings
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=rolebindings
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=rbac.authorization.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=rolebindings
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=storage.k8s.io/v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=storage.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=volumeattachments
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=storage.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:127" resource=volumeattachments
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=storage.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=storageclasses
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=storage.k8s.io/v1 logSource="pkg/backup/resource_backupper.go:127" resource=storageclasses
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=admissionregistration.k8s.io/v1beta1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=admissionregistration.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=mutatingwebhookconfigurations
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=admissionregistration.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:127" resource=mutatingwebhookconfigurations
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=admissionregistration.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=validatingwebhookconfigurations
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=admissionregistration.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:127" resource=validatingwebhookconfigurations
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=admissionregistration.k8s.io/v1alpha1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=admissionregistration.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:105" resource=initializerconfigurations
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=admissionregistration.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:127" resource=initializerconfigurations
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=apiextensions.k8s.io/v1beta1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=apiextensions.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=customresourcedefinitions
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=apiextensions.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:127" resource=customresourcedefinitions
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=scheduling.k8s.io/v1beta1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=scheduling.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=priorityclasses
time="2020-05-08T09:33:42Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=scheduling.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:127" resource=priorityclasses
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=coordination.k8s.io/v1beta1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=coordination.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=leases
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=coordination.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=leases
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=coordination.k8s.io/v1beta1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=leases
time="2020-05-08T09:33:42Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=cephnfses
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=cephnfses
time="2020-05-08T09:33:42Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=cephnfses
time="2020-05-08T09:33:42Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=cephblockpools
time="2020-05-08T09:33:42Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=cephblockpools
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=cephblockpools
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=cephfilesystems
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=cephfilesystems
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=cephfilesystems
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=cephclusters
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=cephclusters
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=cephclusters
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=cephobjectstoreusers
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=cephobjectstoreusers
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=cephobjectstoreusers
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=cephobjectstores
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=cephobjectstores
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=ceph.rook.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=cephobjectstores
time="2020-05-08T09:33:43Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:105" resource=prometheusrules
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=prometheusrules
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=prometheusrules
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:105" resource=servicemonitors
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=servicemonitors
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=servicemonitors
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:105" resource=prometheuses
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=prometheuses
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=prometheuses
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:105" resource=alertmanagers
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=alertmanagers
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=alertmanagers
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:105" resource=podmonitors
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=podmonitors
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=monitoring.coreos.com/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=podmonitors
time="2020-05-08T09:33:43Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=restores
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=restores
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=restores
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=schedules
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=schedules
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=schedules
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=podvolumebackups
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=podvolumebackups
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=podvolumebackups
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=podvolumerestores
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=podvolumerestores
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=podvolumerestores
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=resticrepositories
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=resticrepositories
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=resticrepositories
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=backups
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=backups
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=backups
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=downloadrequests
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=downloadrequests
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=downloadrequests
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=deletebackuprequests
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=deletebackuprequests
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=deletebackuprequests
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=serverstatusrequests
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=serverstatusrequests
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=serverstatusrequests
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=backupstoragelocations
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=backupstoragelocations
time="2020-05-08T09:33:43Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=backupstoragelocations
time="2020-05-08T09:33:43Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:105" resource=volumesnapshotlocations
time="2020-05-08T09:33:43Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=volumesnapshotlocations
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=velero.io/v1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=volumesnapshotlocations
time="2020-05-08T09:33:44Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=certmanager.k8s.io/v1alpha1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=certmanager.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:105" resource=certificates
time="2020-05-08T09:33:44Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=certmanager.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=certificates
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=certmanager.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=certificates
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=certmanager.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:105" resource=issuers
time="2020-05-08T09:33:44Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=certmanager.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=issuers
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=certmanager.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=issuers
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=certmanager.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:105" resource=clusterissuers
time="2020-05-08T09:33:44Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=certmanager.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:127" resource=clusterissuers
time="2020-05-08T09:33:44Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=nfs.rook.io/v1alpha1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=nfs.rook.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:105" resource=nfsservers
time="2020-05-08T09:33:44Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=nfs.rook.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=nfsservers
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=nfs.rook.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=nfsservers
time="2020-05-08T09:33:44Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=objectbucket.io/v1alpha1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=objectbucket.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:105" resource=objectbuckets
time="2020-05-08T09:33:44Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=objectbucket.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:127" resource=objectbuckets
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=objectbucket.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:105" resource=objectbucketclaims
time="2020-05-08T09:33:44Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=objectbucket.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=objectbucketclaims
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=objectbucket.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=objectbucketclaims
time="2020-05-08T09:33:44Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=snapshot.storage.k8s.io/v1alpha1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=snapshot.storage.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:105" resource=volumesnapshotcontents
time="2020-05-08T09:33:44Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=snapshot.storage.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:127" resource=volumesnapshotcontents
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=snapshot.storage.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:105" resource=volumesnapshots
time="2020-05-08T09:33:44Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=snapshot.storage.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=volumesnapshots
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=snapshot.storage.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=volumesnapshots
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=snapshot.storage.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:105" resource=volumesnapshotclasses
time="2020-05-08T09:33:44Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=snapshot.storage.k8s.io/v1alpha1 logSource="pkg/backup/resource_backupper.go:127" resource=volumesnapshotclasses
time="2020-05-08T09:33:44Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=rook.io/v1alpha2 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=rook.io/v1alpha2 logSource="pkg/backup/resource_backupper.go:105" resource=volumes
time="2020-05-08T09:33:44Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=rook.io/v1alpha2 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=volumes
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=rook.io/v1alpha2 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=volumes
time="2020-05-08T09:33:44Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=kubeless.io/v1beta1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=kubeless.io/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=cronjobtriggers
time="2020-05-08T09:33:44Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=kubeless.io/v1beta1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=cronjobtriggers
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=kubeless.io/v1beta1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=cronjobtriggers
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=kubeless.io/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=httptriggers
time="2020-05-08T09:33:44Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=kubeless.io/v1beta1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=httptriggers
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=kubeless.io/v1beta1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=httptriggers
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=kubeless.io/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=functions
time="2020-05-08T09:33:44Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=kubeless.io/v1beta1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=functions
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=kubeless.io/v1beta1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=functions
time="2020-05-08T09:33:44Z" level=info msg="Backing up group" backup=velero/nginx-example-backup-new group=extensions/v1beta1 logSource="pkg/backup/group_backupper.go:101"
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=podsecuritypolicies
time="2020-05-08T09:33:44Z" level=info msg="Skipping resource because it's cluster-scoped and only specific namespaces are included in the backup" backup=velero/nginx-example-backup-new group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:127" resource=podsecuritypolicies
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=networkpolicies
time="2020-05-08T09:33:44Z" level=info msg="Skipping resource because it cohabitates and we've already processed it" backup=velero/nginx-example-backup-new cohabitatingResource1=networkpolicies.extensions cohabitatingResource2=networkpolicies.networking.k8s.io group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:148" resource=networkpolicies
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=replicasets
time="2020-05-08T09:33:44Z" level=info msg="Skipping resource because it cohabitates and we've already processed it" backup=velero/nginx-example-backup-new cohabitatingResource1=replicasets.extensions cohabitatingResource2=replicasets.apps group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:148" resource=replicasets
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=deployments
time="2020-05-08T09:33:44Z" level=info msg="Skipping resource because it cohabitates and we've already processed it" backup=velero/nginx-example-backup-new cohabitatingResource1=deployments.extensions cohabitatingResource2=deployments.apps group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:148" resource=deployments
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=ingresses
time="2020-05-08T09:33:44Z" level=info msg="Listing items" backup=velero/nginx-example-backup-new group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:226" namespace=nginx-example resource=ingresses
time="2020-05-08T09:33:44Z" level=info msg="Retrieved 0 items" backup=velero/nginx-example-backup-new group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:240" namespace=nginx-example resource=ingresses
time="2020-05-08T09:33:44Z" level=info msg="Backing up resource" backup=velero/nginx-example-backup-new group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:105" resource=daemonsets
time="2020-05-08T09:33:44Z" level=info msg="Skipping resource because it cohabitates and we've already processed it" backup=velero/nginx-example-backup-new cohabitatingResource1=daemonsets.extensions cohabitatingResource2=daemonsets.apps group=extensions/v1beta1 logSource="pkg/backup/resource_backupper.go:148" resource=daemonsets
kubectl -n velero get podvolumebackups -l velero.io/backup-name=nginx-example-backup-new -o yaml
apiVersion: v1
items:
- apiVersion: velero.io/v1
kind: PodVolumeBackup
metadata:
annotations:
velero.io/pvc-name: nginx-logs
creationTimestamp: "2020-05-08T09:33:40Z"
generateName: nginx-example-backup-new-
generation: 3
labels:
velero.io/backup-name: nginx-example-backup-new
velero.io/backup-uid: f7226a8c-910e-11ea-ac8a-028358a66085
velero.io/pvc-uid: d2bc87d4-9109-11ea-a626-02021a19689b
name: nginx-example-backup-new-wc88w
namespace: velero
ownerReferences:
- apiVersion: velero.io/v1
controller: true
kind: Backup
name: nginx-example-backup-new
uid: f7226a8c-910e-11ea-ac8a-028358a66085
resourceVersion: "309164"
selfLink: /apis/velero.io/v1/namespaces/velero/podvolumebackups/nginx-example-backup-new-wc88w
uid: 002c9523-910f-11ea-ac8a-028358a66085
spec:
backupStorageLocation: default
node: ip-10-165-138-230.ec2.internal
pod:
kind: Pod
name: nginx-deployment-6fbd5f64c4-fps5c
namespace: nginx-example
uid: 4640dfa5-910a-11ea-ac8a-028358a66085
repoIdentifier: s3:s3.amazonaws.com/cva-builder-velero-backups-test/restic/nginx-example
tags:
backup: nginx-example-backup-new
backup-uid: f7226a8c-910e-11ea-ac8a-028358a66085
ns: nginx-example
pod: nginx-deployment-6fbd5f64c4-fps5c
pod-uid: 4640dfa5-910a-11ea-ac8a-028358a66085
pvc-uid: d2bc87d4-9109-11ea-a626-02021a19689b
volume: nginx-logs
volume: nginx-logs
status:
completionTimestamp: "2020-05-08T09:33:42Z"
path: /host_pods/4640dfa5-910a-11ea-ac8a-028358a66085/volumes/kubernetes.io~local-volume/nginx-logs-volume
phase: Completed
progress: {}
snapshotID: ffabaa99
startTimestamp: "2020-05-08T09:33:40Z"
kind: List
metadata:
resourceVersion: ""
selfLink: ""
kubectl delete deployment -n nginx-example --all
deployment.extensions "nginx-deployment" deleted
kubectl delete pv nginx-logs-volume
persistentvolume "nginx-logs-volume" deleted
kubectl delete namespace nginx-example
namespace "nginx-example" deleted
velero restore create --from-backup nginx-example-backup-new
Restore request "nginx-example-backup-new-20200508094438" submitted successfully.
Run `velero restore describe nginx-example-backup-new-20200508094438` or `velero restore logs nginx-example-backup-new-20200508094438` for more details.
velero restore describe nginx-example-backup-new-20200508094438 --details
Name: nginx-example-backup-new-20200508094438
Namespace: velero
Labels: <none>
Annotations: <none>
Phase: InProgress
Backup: nginx-example-backup-new
Namespaces:
Included: *
Excluded: <none>
Resources:
Included: *
Excluded: nodes, events, events.events.k8s.io, backups.velero.io, restores.velero.io, resticrepositories.velero.io
Cluster-scoped: auto
Namespace mappings: <none>
Label selector: <none>
Restore PVs: auto
Restic Restores:
New:
nginx-example/nginx-deployment-6fbd5f64c4-fps5c: nginx-logs
In this attempt - the restore hung for an hour before failing. It hung on the "waiting for all restic restores to complete"
time="2020-05-08T09:44:38Z" level=info msg="starting restore" logSource="pkg/controller/restore_controller.go:458" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Not including resource" groupResource=nodes logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Not including resource" groupResource=events logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Not including resource" groupResource=events.events.k8s.io logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Not including resource" groupResource=backups.velero.io logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Not including resource" groupResource=restores.velero.io logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Not including resource" groupResource=resticrepositories.velero.io logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Starting restore of backup velero/nginx-example-backup-new" logSource="pkg/restore/restore.go:394" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Restoring cluster level resource 'persistentvolumes'" logSource="pkg/restore/restore.go:779" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Getting client for /v1, Kind=PersistentVolume" logSource="pkg/restore/restore.go:821" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Dynamically re-provisioning persistent volume because it has a restic backup to be restored." logSource="pkg/restore/restore.go:992" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Restoring resource 'persistentvolumeclaims' into namespace 'nginx-example'" logSource="pkg/restore/restore.go:777" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Getting client for /v1, Kind=PersistentVolumeClaim" logSource="pkg/restore/restore.go:821" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing item action for persistentvolumeclaims" logSource="pkg/restore/restore.go:1030" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing AddPVFromPVCAction" cmd=/velero logSource="pkg/restore/add_pv_from_pvc_action.go:44" pluginName=velero restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Adding PV nginx-logs-volume as an additional item to restore" cmd=/velero logSource="pkg/restore/add_pv_from_pvc_action.go:66" pluginName=velero restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Skipping persistentvolumes/nginx-logs-volume because it's already been restored." logSource="pkg/restore/restore.go:910" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing item action for persistentvolumeclaims" logSource="pkg/restore/restore.go:1030" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing ChangeStorageClassAction" cmd=/velero logSource="pkg/restore/change_storageclass_action.go:63" pluginName=velero restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Done executing ChangeStorageClassAction" cmd=/velero logSource="pkg/restore/change_storageclass_action.go:74" pluginName=velero restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Resetting PersistentVolumeClaim nginx-example/nginx-logs for dynamic provisioning because its PV nginx-logs-volume has a reclaim policy of Delete" logSource="pkg/restore/restore.go:1103" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Attempting to restore PersistentVolumeClaim: nginx-logs" logSource="pkg/restore/restore.go:1136" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Restoring resource 'secrets' into namespace 'nginx-example'" logSource="pkg/restore/restore.go:777" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Getting client for /v1, Kind=Secret" logSource="pkg/restore/restore.go:821" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Attempting to restore Secret: default-token-zmrr5" logSource="pkg/restore/restore.go:1136" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Restoring resource 'serviceaccounts' into namespace 'nginx-example'" logSource="pkg/restore/restore.go:777" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Getting client for /v1, Kind=ServiceAccount" logSource="pkg/restore/restore.go:821" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing item action for serviceaccounts" logSource="pkg/restore/restore.go:1030" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing ServiceAccountAction" cmd=/velero logSource="pkg/restore/service_account_action.go:47" pluginName=velero restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Done executing ServiceAccountAction" cmd=/velero logSource="pkg/restore/service_account_action.go:78" pluginName=velero restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Attempting to restore ServiceAccount: default" logSource="pkg/restore/restore.go:1136" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Restoring resource 'pods' into namespace 'nginx-example'" logSource="pkg/restore/restore.go:777" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Getting client for /v1, Kind=Pod" logSource="pkg/restore/restore.go:821" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing item action for pods" logSource="pkg/restore/restore.go:1030" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing AddPVCFromPodAction" cmd=/velero logSource="pkg/restore/add_pvc_from_pod_action.go:44" pluginName=velero restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Adding PVC nginx-example/nginx-logs as an additional item to restore" cmd=/velero logSource="pkg/restore/add_pvc_from_pod_action.go:58" pluginName=velero restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Skipping persistentvolumeclaims/nginx-example/nginx-logs because it's already been restored." logSource="pkg/restore/restore.go:910" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing item action for pods" logSource="pkg/restore/restore.go:1030" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing item action for pods" logSource="pkg/restore/restore.go:1030" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing ResticRestoreAction" cmd=/velero logSource="pkg/restore/restic_restore_action.go:68" pluginName=velero restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Restic backups for pod found" cmd=/velero logSource="pkg/restore/restic_restore_action.go:94" pluginName=velero pod=nginx-example/nginx-deployment-6fbd5f64c4-fps5c restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Using image \"velero/velero-restic-restore-helper:v1.3.2\"" cmd=/velero logSource="pkg/restore/restic_restore_action.go:105" pluginName=velero pod=nginx-example/nginx-deployment-6fbd5f64c4-fps5c restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Done executing ResticRestoreAction" cmd=/velero logSource="pkg/restore/restic_restore_action.go:154" pluginName=velero restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Attempting to restore Pod: nginx-deployment-6fbd5f64c4-fps5c" logSource="pkg/restore/restore.go:1136" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Restoring resource 'replicasets.apps' into namespace 'nginx-example'" logSource="pkg/restore/restore.go:777" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Getting client for apps/v1, Kind=ReplicaSet" logSource="pkg/restore/restore.go:821" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Attempting to restore ReplicaSet: nginx-deployment-6fbd5f64c4" logSource="pkg/restore/restore.go:1136" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Restoring resource 'deployments.apps' into namespace 'nginx-example'" logSource="pkg/restore/restore.go:777" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Getting client for apps/v1, Kind=Deployment" logSource="pkg/restore/restore.go:821" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Attempting to restore Deployment: nginx-deployment" logSource="pkg/restore/restore.go:1136" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Restoring resource 'endpoints' into namespace 'nginx-example'" logSource="pkg/restore/restore.go:777" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Getting client for /v1, Kind=Endpoints" logSource="pkg/restore/restore.go:821" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Attempting to restore Endpoints: my-nginx" logSource="pkg/restore/restore.go:1136" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Restoring resource 'services' into namespace 'nginx-example'" logSource="pkg/restore/restore.go:777" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Getting client for /v1, Kind=Service" logSource="pkg/restore/restore.go:821" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Executing item action for services" logSource="pkg/restore/restore.go:1030" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:38Z" level=info msg="Attempting to restore Service: my-nginx" logSource="pkg/restore/restore.go:1136" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:41Z" level=info msg="Not including resource" groupResource=events logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:41Z" level=info msg="Not including resource" groupResource=nodes logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:41Z" level=info msg="Not including resource" groupResource=events.events.k8s.io logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:41Z" level=info msg="Not including resource" groupResource=resticrepositories.velero.io logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:41Z" level=info msg="Not including resource" groupResource=restores.velero.io logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:41Z" level=info msg="Not including resource" groupResource=backups.velero.io logSource="pkg/restore/restore.go:138" restore=velero/nginx-example-backup-new-20200508094438
time="2020-05-08T09:44:41Z" level=info msg="Waiting for all restic restores to complete" logSource="pkg/restore/restore.go:545" restore=velero/nginx-example-backup-new-20200508094438
Some resources get restored or attempt to be - but stay in pending state due to the absence of the PV.
kubectl get pods -n nginx-example
NAME READY STATUS RESTARTS AGE
nginx-deployment-6fbd5f64c4-fps5c 0/1 Pending 0 5m1s
kubectl get pvc -n nginx-example
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
nginx-logs Pending local-storage 5m11s
kubectl get pv | grep -i nginx
As mentioned previously other storage seems to backup and restore ok.
Can anyone shed any light as to what might be happening?
Something Ive noticed - in the getPodVolumeBackups command we have this
path: /host_pods/4640dfa5-910a-11ea-ac8a-028358a66085/volumes/kubernetes.io~local-volume/nginx-logs-volume
but when I exec into the restic pods I cannot see that path or any nginx-example references.
Looking at the restic repo though there is reference to nginx-example volume...
[centos@cva-builder:4.11.4-PR-1712-29 SANDBOX2 CVA-PR-1712 tmp]$ velero restic repo get -o yaml
apiVersion: velero.io/v1
kind: ResticRepository
metadata:
creationTimestamp: 2020-05-08T09:26:58Z
generateName: nginx-example-default-
generation: 3
labels:
velero.io/storage-location: default
velero.io/volume-namespace: nginx-example
name: nginx-example-default-htvtg
namespace: velero
resourceVersion: "308973"
selfLink: /apis/velero.io/v1/namespaces/velero/resticrepositories/nginx-example-default-htvtg
uid: 10bf1552-910e-11ea-ac8a-028358a66085
spec:
backupStorageLocation: default
maintenanceFrequency: 168h0m0s
resticIdentifier: s3:s3.amazonaws.com/cva-builder-velero-backups-test/restic/nginx-example
volumeNamespace: nginx-example
status:
lastMaintenanceTime: 2020-05-08T09:33:00Z
phase: Ready
**Environment:**
- Velero version (use `velero version`):
velero version
Client:
Version: v1.3.2
Git commit: 55a9914a3e4719fb1578529c45430a8c11c28145
Server:
Version: v1.3.2
- Velero features (use `velero client config get features`):
velero client config get feature
feature: <NOT SET>
- Kubernetes version (use `kubectl version`):
kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.8", GitCommit:"0c6d31a99f81476dfc9871ba3cf3f597bec29b58", GitTreeState:"clean", BuildDate:"2019-07-08T08:46:01Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.8", GitCommit:"0c6d31a99f81476dfc9871ba3cf3f597bec29b58", GitTreeState:"clean", BuildDate:"2019-07-08T08:38:54Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
- Kubernetes installer & version:
- Cloud provider or hardware configuration: **Test performed on AWS - but need "local" storage to emulate on prem none cloud environments**
- OS (e.g. from `/etc/os-release`):
cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7