I0325 21:12:16.753940 1 main.go:96] Version: v4.2.0 I0325 21:12:16.755517 1 connection.go:154] Connecting to unix:///csi/csi-provisioner.sock I0325 21:12:23.766669 1 connection.go:183] GRPC call: /csi.v1.Identity/GetPluginInfo I0325 21:12:23.766687 1 connection.go:184] GRPC request: {} I0325 21:12:23.770140 1 connection.go:186] GRPC response: {"name":"rbd.csi.ceph.com","vendor_version":"v3.5.1"} I0325 21:12:23.770189 1 connection.go:187] GRPC error: I0325 21:12:23.770198 1 main.go:154] CSI driver name: "rbd.csi.ceph.com" I0325 21:12:23.770204 1 common.go:111] Probing CSI driver for readiness I0325 21:12:23.770215 1 connection.go:183] GRPC call: /csi.v1.Identity/Probe I0325 21:12:23.770219 1 connection.go:184] GRPC request: {} I0325 21:12:23.772698 1 connection.go:186] GRPC response: {} I0325 21:12:23.772805 1 connection.go:187] GRPC error: I0325 21:12:23.772815 1 connection.go:183] GRPC call: /csi.v1.Controller/ControllerGetCapabilities I0325 21:12:23.772819 1 connection.go:184] GRPC request: {} I0325 21:12:23.773763 1 connection.go:186] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} I0325 21:12:23.773897 1 connection.go:187] GRPC error: I0325 21:12:23.773907 1 main.go:192] Start NewCSISnapshotSideCarController with snapshotter [rbd.csi.ceph.com] kubeconfig [] csiTimeout [1m0s] csiAddress [unix:///csi/csi-provisioner.sock] resyncPeriod [15m0s] snapshotNamePrefix [snapshot] snapshotNameUUIDLength [824639815768] I0325 21:12:23.774997 1 leaderelection.go:248] attempting to acquire leader lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com... I0325 21:12:23.788157 1 leaderelection.go:258] successfully acquired lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:12:23.788253 1 leader_election.go:212] new leader detected, current leader: ceph-csi-rbd-provisioner-6f6497f79c-m54lr I0325 21:12:23.788304 1 leader_election.go:205] became leader, starting I0325 21:12:23.788507 1 reflector.go:219] Starting reflector *v1.VolumeSnapshotContent (15m0s) from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 I0325 21:12:23.788522 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 I0325 21:12:23.788783 1 snapshot_controller_base.go:115] Starting CSI snapshotter I0325 21:12:23.788842 1 reflector.go:219] Starting reflector *v1.VolumeSnapshotClass (15m0s) from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 I0325 21:12:23.788864 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:23.789891 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) E0325 21:12:23.790492 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:12:23.796249 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:12:25.123745 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:25.125528 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:12:25.343221 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:25.344948 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0325 21:12:27.426701 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:27.429184 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:12:27.624716 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:27.626379 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0325 21:12:28.805006 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:12:31.037063 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:31.039091 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0325 21:12:32.827738 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:32.829345 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:12:33.814667 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:12:38.441159 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:38.443361 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0325 21:12:38.822924 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:12:39.850262 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:39.851500 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:12:43.852916 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:12:48.861069 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:12:53.903595 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:12:55.095098 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:55.097149 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0325 21:12:58.912658 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:12:59.246321 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:12:59.247639 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:13:03.925276 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:08.934497 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:13.944961 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:18.953532 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:24.003811 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:29.012639 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:30.333645 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:13:30.335613 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:13:34.022950 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:39.030280 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:41.526741 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:13:41.528966 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0325 21:13:44.036957 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:49.048812 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:54.057872 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:13:59.065749 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:04.081481 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:09.090708 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:11.756149 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:14:11.758033 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:14:14.152318 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:19.162457 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:21.070842 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:14:21.072880 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0325 21:14:24.173368 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:29.181989 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:34.190726 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:39.201663 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:44.211232 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:49.218327 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:54.231609 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:55.824848 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:14:55.827041 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:14:59.240879 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:14:59.564906 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:14:59.567009 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0325 21:15:04.248235 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:09.259675 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:14.266488 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:19.294969 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:24.305077 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:29.570473 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:34.581826 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:34.620474 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:15:34.622591 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:15:39.590415 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:44.832389 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:49.839528 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:49.939637 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:15:49.941312 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotContent: failed to list *v1.VolumeSnapshotContent: the server could not find the requested resource (get volumesnapshotcontents.snapshot.storage.k8s.io) I0325 21:15:54.845487 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:15:59.852945 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:16:04.861851 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:16:09.868704 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com I0325 21:16:11.179308 1 reflector.go:255] Listing and watching *v1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117 E0325 21:16:11.180554 1 reflector.go:138] github.com/kubernetes-csi/external-snapshotter/client/v4/informers/externalversions/factory.go:117: Failed to watch *v1.VolumeSnapshotClass: failed to list *v1.VolumeSnapshotClass: the server could not find the requested resource (get volumesnapshotclasses.snapshot.storage.k8s.io) I0325 21:16:14.876657 1 leaderelection.go:278] successfully renewed lease kube-system/external-snapshotter-leader-rbd-csi-ceph-com