I0407 09:45:31.459178 1 main.go:96] Version: v4.2.0 I0407 09:45:31.462499 1 connection.go:154] Connecting to unix:///csi/csi-provisioner.sock I0407 09:45:33.341868 1 connection.go:183] GRPC call: /csi.v1.Identity/GetPluginInfo I0407 09:45:33.341892 1 connection.go:184] GRPC request: {} I0407 09:45:33.349180 1 connection.go:186] GRPC response: {"name":"rbd.csi.ceph.com","vendor_version":"v3.5.1"} I0407 09:45:33.349307 1 connection.go:187] GRPC error: I0407 09:45:33.349361 1 main.go:154] CSI driver name: "rbd.csi.ceph.com" I0407 09:45:33.349383 1 common.go:111] Probing CSI driver for readiness I0407 09:45:33.349398 1 connection.go:183] GRPC call: /csi.v1.Identity/Probe I0407 09:45:33.349408 1 connection.go:184] GRPC request: {} I0407 09:45:33.350453 1 connection.go:186] GRPC response: {} I0407 09:45:33.350545 1 connection.go:187] GRPC error: I0407 09:45:33.350553 1 connection.go:183] GRPC call: /csi.v1.Controller/ControllerGetCapabilities I0407 09:45:33.350576 1 connection.go:184] GRPC request: {} I0407 09:45:33.352790 1 connection.go:186] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} I0407 09:45:33.352942 1 connection.go:187] GRPC error: I0407 09:45:33.353810 1 main.go:192] Start NewCSISnapshotSideCarController with snapshotter [rbd.csi.ceph.com] kubeconfig [] csiTimeout [1m0s] csiAddress [unix:///csi/csi-provisioner.sock] resyncPeriod [15m0s] snapshotNamePrefix [snapshot] snapshotNameUUIDLength [824635105368] I0407 09:45:33.356331 1 leaderelection.go:248] attempting to acquire leader lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com... I0407 09:45:33.376252 1 leaderelection.go:258] successfully acquired lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:45:33.376356 1 leader_election.go:212] new leader detected, current leader: ceph-csi-rbd-provisioner-6f6497f79c-l4v4r I0407 09:45:33.376581 1 leader_election.go:205] became leader, starting I0407 09:45:33.377068 1 reflector.go:219] Starting reflector *v1.VolumeSnapshotContent (15m0s) from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 I0407 09:45:33.377086 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 I0407 09:45:33.378148 1 snapshot_controller_base.go:115] Starting CSI snapshotter I0407 09:45:33.379785 1 reflector.go:219] Starting reflector *v1.VolumeSnapshotClass (15m0s) from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 I0407 09:45:33.379817 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:45:33.384786 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) E0407 09:45:33.385023 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:45:33.392606 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:45:34.718460 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:45:34.720767 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:45:34.938243 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:45:34.940403 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0407 09:45:37.021393 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:45:37.023058 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:45:37.220868 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:45:37.223693 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0407 09:45:38.413420 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:45:40.634023 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:45:40.637076 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0407 09:45:42.421836 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:45:42.423873 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:45:43.424563 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:45:48.039625 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:45:48.041429 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0407 09:45:48.433446 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:45:49.445180 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:45:49.446420 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:45:53.441567 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:45:58.451419 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:03.462717 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:04.694077 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:46:04.696389 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0407 09:46:08.476793 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:08.842144 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:46:08.844412 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:46:13.486802 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:18.500559 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:23.509380 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:28.521540 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:33.532112 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:38.542373 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:39.929982 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:46:39.931772 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:46:43.553160 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:48.561800 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:51.126135 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:46:51.127830 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0407 09:46:53.570173 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:46:58.578577 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:03.587881 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:08.596987 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:13.605999 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:18.617178 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:21.351934 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:47:21.353691 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:47:23.625818 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:28.635584 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:30.670533 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:47:30.672527 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0407 09:47:33.644680 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:38.655155 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:43.667597 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:48.679107 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:53.691332 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:47:58.702258 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:03.714482 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:05.420660 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:48:05.423096 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:48:08.725260 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:09.164560 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:48:09.167076 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0407 09:48:13.736001 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:18.746382 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:23.755378 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:28.766082 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:33.778277 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:38.790716 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:43.801421 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:44.216439 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:48:44.218930 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:48:48.812471 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:53.825050 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:58.833410 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:48:59.540619 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:48:59.541842 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0407 09:49:03.845555 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:49:08.854576 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:49:13.864736 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:49:18.875954 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0407 09:49:20.776703 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0407 09:49:20.778382 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0407 09:49:23.887058 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com