I0420 14:25:34.104071 1 cephcsi.go:191] Driver version: v3.11.0 and Git version: bc24b5eca87626d690a29effa9d7420cc0154a7a I0420 14:25:34.104197 1 cephcsi.go:223] Starting driver type: controller with name: rbd.csi.ceph.com I0420 14:25:34.105030 1 leaderelection.go:250] attempting to acquire leader lease kube-system/rbd.csi.ceph.com-kube-system... I0420 14:25:34.119920 1 leaderelection.go:260] successfully acquired lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:34.124577 1 reflector.go:289] Starting reflector *v1.PersistentVolume (9h32m3.535626437s) from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0420 14:25:34.124603 1 reflector.go:325] Listing and watching *v1.PersistentVolume from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0420 14:25:34.126113 1 reflector.go:351] Caches populated for *v1.PersistentVolume from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0420 14:25:34.127468 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:36.141218 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:38.150084 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:40.161234 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:42.171299 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:44.181091 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:46.189776 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:48.200132 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:50.211550 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:52.223594 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:54.234231 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:56.244968 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:25:58.254967 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:00.266721 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:02.277787 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:04.285211 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:06.299184 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:08.309899 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:10.319414 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:12.332507 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:14.341921 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:16.351236 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:18.360351 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:20.368728 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:22.378683 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:24.386724 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:26.395508 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:28.407627 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:30.417053 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:32.426292 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:34.436873 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:36.446628 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:38.459108 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:40.468939 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:42.478687 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:44.489680 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:46.500421 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:48.510791 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:50.523859 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:52.535347 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:54.545430 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:56.555967 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:26:58.569998 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:00.586595 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:02.595641 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:04.604473 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:06.614246 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:08.623722 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:10.635377 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:12.645169 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:14.653549 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:16.662858 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:18.677012 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:20.684724 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:22.692263 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:24.701949 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:26.716283 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:28.725986 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:30.735712 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:32.745602 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:34.755149 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:36.766538 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:38.774707 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:40.783536 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:42.794934 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:44.804300 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:46.812839 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:48.825536 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:50.834375 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:52.846944 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:54.856305 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:56.863230 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:27:58.873183 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:00.880472 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:02.888336 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:04.902454 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:06.911895 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:08.924890 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:10.936899 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:12.948616 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:14.956310 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:16.971484 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:18.982167 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:20.995282 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:23.026140 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:25.042641 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:27.055742 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:29.064349 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:31.073515 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:33.082451 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:35.091819 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:37.104184 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:39.114832 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:41.124303 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system [controller-runtime] log.SetLogger(...) was never called; logs will not be displayed. Detected at: > goroutine 97 [running]: > runtime/debug.Stack() > /usr/local/go/src/runtime/debug/stack.go:24 +0x5e > sigs.k8s.io/controller-runtime/pkg/log.eventuallyFulfillRoot() > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/log/log.go:60 +0xcd > sigs.k8s.io/controller-runtime/pkg/log.(*delegatingLogSink).WithValues(0xc0005c8dc0, {0xc000a17620, 0x6, 0x6}) > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/log/deleg.go:168 +0x49 > github.com/go-logr/logr.Logger.WithValues(...) > /go/src/github.com/ceph/ceph-csi/vendor/github.com/go-logr/logr/logr.go:332 > sigs.k8s.io/controller-runtime/pkg/controller.NewUnmanaged.func1(0xc00073ca00) > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/controller/controller.go:121 +0x1d7 > sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0005c5ea0, {0x27c0488, 0xc000826e10}, {0x1f93660?, 0xc00073c9e0?}) > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:306 +0x16a > sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0005c5ea0, {0x27c0488, 0xc000826e10}) > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266 +0x1af > sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2() > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227 +0x79 > created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 88 > /go/src/github.com/ceph/ceph-csi/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:223 +0x565 I0420 14:28:42.163086 1 reflector.go:289] Starting reflector *v1.Secret (10h3m8.673977495s) from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0420 14:28:42.163112 1 reflector.go:325] Listing and watching *v1.Secret from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0420 14:28:42.205196 1 reflector.go:351] Caches populated for *v1.Secret from sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:105 I0420 14:28:42.286514 1 omap.go:89] got omap values: (pool="kube", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-641ec87d-3ce4-4be4-9594-133b1c4f3d28:e58b6d0b-4647-439c-81d6-b95ab557dafa] I0420 14:28:42.288597 1 omap.go:89] got omap values: (pool="kube", namespace="", name="csi.volume.e58b6d0b-4647-439c-81d6-b95ab557dafa"): map[csi.imageid:37d9362983b8 csi.imagename:csi-vol-e58b6d0b-4647-439c-81d6-b95ab557dafa csi.volname:pvc-641ec87d-3ce4-4be4-9594-133b1c4f3d28 csi.volume.owner:default] I0420 14:28:42.360361 1 omap.go:89] got omap values: (pool="kube", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-641ec87d-3ce4-4be4-9594-133b1c4f3d28:e58b6d0b-4647-439c-81d6-b95ab557dafa] I0420 14:28:42.361509 1 omap.go:89] got omap values: (pool="kube", namespace="", name="csi.volume.e58b6d0b-4647-439c-81d6-b95ab557dafa"): map[csi.imageid:37d9362983b8 csi.imagename:csi-vol-e58b6d0b-4647-439c-81d6-b95ab557dafa csi.volname:pvc-641ec87d-3ce4-4be4-9594-133b1c4f3d28 csi.volume.owner:default] I0420 14:28:43.133104 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:45.143012 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:47.152866 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:49.161458 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:51.170786 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:53.179380 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:55.188990 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:57.199397 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:28:59.210991 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:01.221108 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:03.232190 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:05.242181 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:07.250660 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:09.259568 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:11.270429 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:13.280855 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:15.289934 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:17.303205 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:19.312808 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:21.320184 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:23.327209 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:25.335082 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:27.344117 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:29.353588 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:31.361284 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:33.371625 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system I0420 14:29:35.386448 1 leaderelection.go:281] successfully renewed lease kube-system/rbd.csi.ceph.com-kube-system