2026/03/20 05:48:03 maxprocs: Updating GOMAXPROCS=1: determined from CPU quota
2026-03-20 05:48:03.457223 I | rookcmd: starting Rook v1.15.9 with arguments '/usr/local/bin/rook ceph operator'
2026-03-20 05:48:03.457244 I | rookcmd: flag values: --enable-machine-disruption-budget=false, --help=false, --kubeconfig=, --log-level=INFO
2026-03-20 05:48:03.457246 I | cephcmd: starting Rook-Ceph operator
2026-03-20 05:48:03.552204 I | cephcmd: base ceph version inside the rook operator image is "ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable)"
2026-03-20 05:48:03.560306 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_NFS" = "false"
2026-03-20 05:48:03.560347 I | op-k8sutil: operator setting "ROOK_ENABLE_DISCOVERY_DAEMON" = "false"
2026-03-20 05:48:03.560384 I | op-k8sutil: operator setting "CSI_CEPHFS_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-cephfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-03-20 05:48:03.560395 I | op-k8sutil: operator setting "CSI_DISABLE_HOLDER_PODS" = "true"
2026-03-20 05:48:03.560409 I | op-k8sutil: operator setting "CSI_NFS_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-nfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n"
2026-03-20 05:48:03.560418 I | op-k8sutil: operator setting "CSI_RBD_ATTACH_REQUIRED" = "true"
2026-03-20 05:48:03.560424 I | op-k8sutil: operator setting "ROOK_CSI_DISABLE_DRIVER" = "false"
2026-03-20 05:48:03.560431 I | op-k8sutil: operator setting "ROOK_CSI_SNAPSHOTTER_IMAGE" = "registry.k8s.io/sig-storage/csi-snapshotter:v8.0.1"
2026-03-20 05:48:03.560438 I | op-k8sutil: operator setting "CSI_NFS_FSGROUPPOLICY" = "File"
2026-03-20 05:48:03.560448 I | op-k8sutil: operator setting "ROOK_CSI_ATTACHER_IMAGE" = "registry.k8s.io/sig-storage/csi-attacher:v4.6.1"
2026-03-20 05:48:03.560455 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_CEPHFS" = "false"
2026-03-20 05:48:03.560462 I | op-k8sutil: operator setting "CSI_ENABLE_CEPHFS_SNAPSHOTTER" = "true"
2026-03-20 05:48:03.560469 I | op-k8sutil: operator setting "CSI_ENABLE_METADATA" = "false"
2026-03-20 05:48:03.560475 I | op-k8sutil: operator setting "CSI_ENABLE_NFS_SNAPSHOTTER" = "true"
2026-03-20 05:48:03.560484 I | op-k8sutil: operator setting "CSI_PROVISIONER_PRIORITY_CLASSNAME" = "system-cluster-critical"
2026-03-20 05:48:03.560492 I | op-k8sutil: operator setting "ROOK_CSI_CEPH_IMAGE" = "quay.io/cephcsi/cephcsi:v3.12.3"
2026-03-20 05:48:03.560503 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_RBD" = "false"
2026-03-20 05:48:03.560512 I | op-k8sutil: operator setting "ROOK_CSI_REGISTRAR_IMAGE" = "registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.11.1"
2026-03-20 05:48:03.560527 I | op-k8sutil: operator setting "ROOK_OBC_WATCH_OPERATOR_NAMESPACE" = "true"
2026-03-20 05:48:03.560534 I | op-k8sutil: operator setting "CSI_CEPHFS_FSGROUPPOLICY" = "File"
2026-03-20 05:48:03.560541 I | op-k8sutil: operator setting "CSI_NFS_ATTACH_REQUIRED" = "true"
2026-03-20 05:48:03.560548 I | op-k8sutil: operator setting "CSI_PLUGIN_PRIORITY_CLASSNAME" = "system-node-critical"
2026-03-20 05:48:03.560556 I | op-k8sutil: operator setting "CSI_ENABLE_VOLUME_GROUP_SNAPSHOT" = "true"
2026-03-20 05:48:03.560563 I | op-k8sutil: operator setting "CSI_FORCE_CEPHFS_KERNEL_CLIENT" = "true"
2026-03-20 05:48:03.560569 I | op-k8sutil: operator setting "CSI_RBD_FSGROUPPOLICY" = "File"
2026-03-20 05:48:03.560591 I | op-k8sutil: operator setting "CSI_RBD_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-resizer\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-snapshotter\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-rbdplugin\n  resource:\n    requests:\n      memory: 512Mi\n    limits:\n      memory: 1Gi\n- name : csi-omap-generator\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-03-20 05:48:03.560614 I | op-k8sutil: operator setting "CSI_CEPHFS_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-resizer\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-snapshotter\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-cephfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-03-20 05:48:03.560620 I | op-k8sutil: operator setting "CSI_ENABLE_RBD_SNAPSHOTTER" = "true"
2026-03-20 05:48:03.560630 I | op-k8sutil: operator setting "CSI_ENABLE_TOPOLOGY" = "false"
2026-03-20 05:48:03.560641 I | op-k8sutil: operator setting "ROOK_CSI_RESIZER_IMAGE" = "registry.k8s.io/sig-storage/csi-resizer:v1.11.1"
2026-03-20 05:48:03.560647 I | op-k8sutil: operator setting "CSI_PROVISIONER_REPLICAS" = "2"
2026-03-20 05:48:03.560654 I | op-k8sutil: operator setting "ROOK_CEPH_COMMANDS_TIMEOUT_SECONDS" = "15"
2026-03-20 05:48:03.560660 I | op-k8sutil: operator setting "ROOK_CSI_IMAGE_PULL_POLICY" = "IfNotPresent"
2026-03-20 05:48:03.560666 I | op-k8sutil: operator setting "CSI_PLUGIN_ENABLE_SELINUX_HOST_MOUNT" = "false"
2026-03-20 05:48:03.560671 I | op-k8sutil: operator setting "ROOK_LOG_LEVEL" = "INFO"
2026-03-20 05:48:03.560676 I | op-k8sutil: operator setting "CSI_ENABLE_CSIADDONS" = "false"
2026-03-20 05:48:03.560681 I | op-k8sutil: operator setting "CSI_ENABLE_ENCRYPTION" = "false"
2026-03-20 05:48:03.560687 I | op-k8sutil: operator setting "CSI_GRPC_TIMEOUT_SECONDS" = "150"
2026-03-20 05:48:03.560696 I | op-k8sutil: operator setting "CSI_NFS_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-nfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n"
2026-03-20 05:48:03.560711 I | op-k8sutil: operator setting "CSI_RBD_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-rbdplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-03-20 05:48:03.560722 I | op-k8sutil: operator setting "ROOK_CEPH_ALLOW_LOOP_DEVICES" = "false"
2026-03-20 05:48:03.560734 I | op-k8sutil: operator setting "ROOK_CSIADDONS_IMAGE" = "quay.io/csiaddons/k8s-sidecar:v0.9.1"
2026-03-20 05:48:03.560740 I | op-k8sutil: operator setting "ROOK_CSI_PROVISIONER_IMAGE" = "registry.k8s.io/sig-storage/csi-provisioner:v5.0.1"
2026-03-20 05:48:03.560744 I | op-k8sutil: operator setting "CSI_CEPHFS_ATTACH_REQUIRED" = "true"
2026-03-20 05:48:03.560749 I | op-k8sutil: operator setting "CSI_ENABLE_HOST_NETWORK" = "true"
2026-03-20 05:48:03.560754 I | op-k8sutil: operator setting "CSI_ENABLE_OMAP_GENERATOR" = "false"
2026-03-20 05:48:03.560760 I | operator: watching all namespaces for Ceph CRs
2026-03-20 05:48:03.560856 I | operator: setting up schemes
2026-03-20 05:48:03.564101 I | operator: setting up the controller-runtime manager
2026-03-20 05:48:03.564640 I | ceph-cluster-controller: successfully started
2026-03-20 05:48:03.565083 I | ceph-cluster-controller: enabling hotplug orchestration
2026-03-20 05:48:03.565112 I | ceph-nodedaemon-controller: successfully started
2026-03-20 05:48:03.565128 I | ceph-block-pool-controller: successfully started
2026-03-20 05:48:03.565142 I | ceph-object-store-user-controller: successfully started
2026-03-20 05:48:03.565162 I | ceph-object-realm-controller: successfully started
2026-03-20 05:48:03.565172 I | ceph-object-zonegroup-controller: successfully started
2026-03-20 05:48:03.565181 I | ceph-object-zone-controller: successfully started
2026-03-20 05:48:03.565294 I | ceph-object-controller: successfully started
2026-03-20 05:48:03.565347 I | ceph-file-controller: successfully started
2026-03-20 05:48:03.565375 I | ceph-nfs-controller: successfully started
2026-03-20 05:48:03.565417 I | ceph-rbd-mirror-controller: successfully started
2026-03-20 05:48:03.565449 I | ceph-client-controller: successfully started
2026-03-20 05:48:03.565465 I | ceph-filesystem-mirror-controller: successfully started
2026-03-20 05:48:03.565492 I | operator: rook-ceph-operator-config-controller successfully started
2026-03-20 05:48:03.565506 I | ceph-csi: rook-ceph-operator-csi-controller successfully started
2026-03-20 05:48:03.565649 I | op-bucket-prov: rook-ceph-operator-bucket-controller successfully started
2026-03-20 05:48:03.565671 I | ceph-bucket-topic: successfully started
2026-03-20 05:48:03.565680 I | ceph-bucket-notification: successfully started
2026-03-20 05:48:03.565688 I | ceph-bucket-notification: successfully started
2026-03-20 05:48:03.565696 I | ceph-fs-subvolumegroup-controller: successfully started
2026-03-20 05:48:03.565744 I | blockpool-rados-namespace-controller: successfully started
2026-03-20 05:48:03.565795 I | ceph-cosi-controller: successfully started
2026-03-20 05:48:03.565814 I | operator: starting the controller-runtime manager
2026-03-20 05:48:03.785196 I | operator: rook-ceph-operator-config-controller done reconciling
2026-03-20 05:48:03.790738 I | ceph-csi: successfully created csi config map "rook-ceph-csi-config"
2026-03-20 05:48:03.795453 I | op-k8sutil: removing daemonset csi-rbdplugin if it exists
2026-03-20 05:48:03.797801 I | op-k8sutil: removing deployment csi-rbdplugin-provisioner if it exists
2026-03-20 05:48:03.804973 I | ceph-csi: successfully removed CSI Ceph RBD driver
2026-03-20 05:48:03.804989 I | op-k8sutil: removing daemonset csi-cephfsplugin if it exists
2026-03-20 05:48:03.806811 I | op-k8sutil: removing deployment csi-cephfsplugin-provisioner if it exists
2026-03-20 05:48:03.811272 I | ceph-csi: successfully removed CSI CephFS driver
2026-03-20 05:48:03.811289 I | op-k8sutil: removing daemonset csi-nfsplugin if it exists
2026-03-20 05:48:03.815658 I | op-k8sutil: removing deployment csi-nfsplugin-provisioner if it exists
2026-03-20 05:48:03.823268 I | ceph-csi: successfully removed CSI NFS driver
2026-03-20 05:48:25.439440 I | ceph-spec: adding finalizer "cephcluster.ceph.rook.io" on "ceph"
2026-03-20 05:48:25.443270 I | clusterdisruption-controller: deleted all legacy node drain canary pods
2026-03-20 05:48:25.444237 I | ceph-spec: adding finalizer "cephobjectstore.ceph.rook.io" on "ceph"
2026-03-20 05:48:25.464744 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-03-20 05:48:25.464768 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-03-20 05:48:25.464855 I | op-bucket-prov: ceph bucket provisioner launched watching for provisioner "openstack.ceph.rook.io/bucket"
2026-03-20 05:48:25.465216 I | op-bucket-prov: successfully reconciled bucket provisioner
I0320 05:48:25.465287       1 manager.go:135] "msg"="starting provisioner" "logger"="objectbucket.io/provisioner-manager" "name"="openstack.ceph.rook.io/bucket"
2026-03-20 05:48:25.466446 I | ceph-cluster-controller: reconciling ceph cluster in namespace "openstack"
2026-03-20 05:48:25.488376 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-03-20 05:48:25.488400 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-03-20 05:48:25.488417 I | ceph-spec: found the cluster info to connect to the external cluster. will use "client.admin" to check health and monitor status. mons=map[instance:0xc000ede180]
2026-03-20 05:48:25.488431 I | ceph-spec: detecting the ceph image version for image harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7...
2026-03-20 05:48:26.442479 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-03-20 05:48:26.442503 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-03-20 05:48:26.846445 I | op-k8sutil: removing daemonset csi-rbdplugin if it exists
2026-03-20 05:48:26.848888 I | op-k8sutil: removing deployment csi-rbdplugin-provisioner if it exists
2026-03-20 05:48:27.044208 I | ceph-csi: successfully removed CSI Ceph RBD driver
2026-03-20 05:48:27.044238 I | op-k8sutil: removing daemonset csi-cephfsplugin if it exists
2026-03-20 05:48:27.046831 I | op-k8sutil: removing deployment csi-cephfsplugin-provisioner if it exists
2026-03-20 05:48:27.244716 I | ceph-csi: successfully removed CSI CephFS driver
2026-03-20 05:48:27.244737 I | op-k8sutil: removing daemonset csi-nfsplugin if it exists
2026-03-20 05:48:27.246553 I | op-k8sutil: removing deployment csi-nfsplugin-provisioner if it exists
2026-03-20 05:48:27.446203 I | ceph-csi: successfully removed CSI NFS driver
2026-03-20 05:48:41.859469 I | ceph-spec: detected ceph image version: "18.2.7-0 reef"
2026-03-20 05:48:41.859497 I | ceph-cluster-controller: validating ceph version from provided image
2026-03-20 05:48:41.866798 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-03-20 05:48:41.866819 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-03-20 05:48:41.869252 I | cephclient: writing config file /var/lib/rook/openstack/openstack.config
2026-03-20 05:48:41.869391 I | cephclient: generated admin config in /var/lib/rook/openstack
2026-03-20 05:48:42.336111 E | cephver: external cluster ceph version is a major version higher "18.2.7-0 reef" than the local cluster "0.0.0-0 <unknown version>", consider upgrading
2026-03-20 05:48:42.812407 I | ceph-cluster-controller: cluster "openstack": version "18.2.7-0 reef" detected for image "harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7"
2026-03-20 05:48:42.865031 I | ceph-cluster-controller: creating "rook-config-override" configmap
2026-03-20 05:48:42.868271 I | ceph-cluster-controller: creating "rook-ceph-config" secret
2026-03-20 05:48:42.893639 I | ceph-cluster-controller: external cluster identity established
2026-03-20 05:48:42.893676 I | cephclient: getting or creating ceph auth key "client.csi-rbd-provisioner"
2026-03-20 05:48:43.681549 I | cephclient: getting or creating ceph auth key "client.csi-rbd-node"
2026-03-20 05:48:44.172591 I | cephclient: getting or creating ceph auth key "client.csi-cephfs-provisioner"
2026-03-20 05:48:44.669084 I | cephclient: getting or creating ceph auth key "client.csi-cephfs-node"
2026-03-20 05:48:45.188532 I | ceph-csi: created kubernetes csi secrets for cluster "openstack"
2026-03-20 05:48:45.198506 I | ceph-cluster-controller: successfully updated csi config map
2026-03-20 05:48:45.198538 I | cephclient: getting or creating ceph auth key "client.crash"
2026-03-20 05:48:45.793982 I | ceph-nodedaemon-controller: created kubernetes crash collector secret for cluster "openstack"
2026-03-20 05:48:45.796005 I | ceph-cluster-controller: enabling ceph mon monitoring goroutine for cluster "openstack"
2026-03-20 05:48:45.796039 I | ceph-cluster-controller: ceph status check interval is 1m0s
2026-03-20 05:48:45.796044 I | ceph-cluster-controller: enabling ceph status monitoring goroutine for cluster "openstack"
2026-03-20 05:48:46.517793 I | clusterdisruption-controller: all PGs are active+clean. Restoring default OSD pdb settings
2026-03-20 05:48:46.517818 I | clusterdisruption-controller: creating the default pdb "rook-ceph-osd" with maxUnavailable=1 for all osd
2026-03-20 05:48:46.572285 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:48:47.593362 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:48:48.295139 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:48:55.524194 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-03-20 05:48:55.524218 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-03-20 05:48:55.524229 I | ceph-spec: detecting the ceph image version for image harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7...
2026-03-20 05:48:57.008737 I | ceph-spec: detected ceph image version: "18.2.7-0 reef"
2026-03-20 05:48:58.313017 I | ceph-object-controller: reconciling object store deployments
2026-03-20 05:48:58.352459 I | ceph-object-controller: ceph object store gateway service running at 10.106.41.36
2026-03-20 05:48:58.352495 I | ceph-object-controller: reconciling object store pools
2026-03-20 05:49:01.307543 I | cephclient: reconciling replicated pool ceph.rgw.control succeeded
2026-03-20 05:49:01.307573 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.control"
2026-03-20 05:49:05.379118 I | cephclient: reconciling replicated pool ceph.rgw.meta succeeded
2026-03-20 05:49:05.379144 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.meta"
2026-03-20 05:49:09.431286 I | cephclient: reconciling replicated pool ceph.rgw.log succeeded
2026-03-20 05:49:09.431321 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.log"
2026-03-20 05:49:13.511953 I | cephclient: reconciling replicated pool ceph.rgw.buckets.index succeeded
2026-03-20 05:49:13.511988 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.buckets.index"
2026-03-20 05:49:17.590752 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:30} {StateName:unknown Count:2} {StateName:creating+peering Count:1}]"
2026-03-20 05:49:17.595667 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:49:18.480326 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:32} {StateName:unknown Count:8} {StateName:creating+peering Count:1}]"
2026-03-20 05:49:18.484968 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:49:18.589530 I | cephclient: reconciling replicated pool ceph.rgw.buckets.non-ec succeeded
2026-03-20 05:49:18.589565 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.buckets.non-ec"
2026-03-20 05:49:22.645591 I | cephclient: reconciling replicated pool ceph.rgw.otp succeeded
2026-03-20 05:49:22.645624 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.otp"
2026-03-20 05:49:26.711490 I | cephclient: reconciling replicated pool .rgw.root succeeded
2026-03-20 05:49:26.711518 I | cephclient: setting pool property "pg_num_min" to "8" on pool ".rgw.root"
2026-03-20 05:49:30.785798 I | cephclient: reconciling replicated pool ceph.rgw.buckets.data succeeded
2026-03-20 05:49:30.785832 I | ceph-object-controller: configuring object store "ceph"
2026-03-20 05:49:31.935474 I | ceph-object-controller: Object store "ceph": realm=ceph, zonegroup=ceph, zone=ceph
2026-03-20 05:49:32.311566 I | ceph-object-controller: committing changes to RGW configuration period for CephObjectStore "openstack/ceph"
2026-03-20 05:49:32.627226 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:49:32.806079 I | ceph-object-controller: configuration for object-store ceph is complete
2026-03-20 05:49:32.807841 I | ceph-object-controller: creating object store "ceph" in namespace "openstack"
2026-03-20 05:49:32.810214 I | cephclient: getting or creating ceph auth key "client.rgw.ceph.a"
2026-03-20 05:49:33.312091 I | ceph-object-controller: setting rgw config flags
2026-03-20 05:49:33.312129 I | ceph-object-controller: Configuring authentication with keystone
2026-03-20 05:49:33.312168 I | op-config: setting option "rgw_keystone_admin_project" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:33.697871 I | op-config: successfully set option "rgw_keystone_admin_project" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:33.697908 I | op-config: setting option "rgw_s3_auth_use_keystone" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:34.095038 I | op-config: successfully set option "rgw_s3_auth_use_keystone" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:34.095073 I | op-config: setting option "rgw_swift_account_in_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:34.477245 I | op-config: successfully set option "rgw_swift_account_in_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:34.477278 I | op-config: setting option "rgw_keystone_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:34.866455 I | op-config: successfully set option "rgw_keystone_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:34.866488 I | op-config: setting option "rgw_keystone_implicit_tenants" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:35.243493 I | op-config: successfully set option "rgw_keystone_implicit_tenants" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:35.243534 I | op-config: setting option "rgw_keystone_api_version" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:35.615350 I | op-config: successfully set option "rgw_keystone_api_version" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:35.615384 I | op-config: setting option "rgw_keystone_admin_domain" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:35.987656 I | op-config: successfully set option "rgw_keystone_admin_domain" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:35.987682 I | op-config: setting option "rgw_keystone_admin_user" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:36.360526 I | op-config: successfully set option "rgw_keystone_admin_user" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:36.360560 I | op-config: setting option "rgw_swift_versioning_enabled" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:36.735912 I | op-config: successfully set option "rgw_swift_versioning_enabled" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:36.735950 I | op-config: setting option "rgw_zonegroup" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:37.141283 I | op-config: successfully set option "rgw_zonegroup" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:37.141319 I | op-config: setting option "rgw_log_nonexistent_bucket" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:37.525456 I | op-config: successfully set option "rgw_log_nonexistent_bucket" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:37.525496 I | op-config: setting option "rgw_enable_usage_log" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:37.917009 I | op-config: successfully set option "rgw_enable_usage_log" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:37.917050 I | op-config: setting option "rgw_keystone_accepted_roles" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:38.320971 I | op-config: successfully set option "rgw_keystone_accepted_roles" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:38.321013 I | op-config: setting option "rgw_keystone_admin_password" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:38.719058 I | op-config: successfully set option "rgw_keystone_admin_password" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:38.719101 I | op-config: setting option "rgw_run_sync_thread" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:39.118191 I | op-config: successfully set option "rgw_run_sync_thread" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:39.118219 I | op-config: setting option "rgw_zone" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:39.511315 I | op-config: successfully set option "rgw_zone" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:39.511357 I | op-config: setting option "rgw_log_object_name_utc" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:39.891673 I | op-config: successfully set option "rgw_log_object_name_utc" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:49:39.892039 I | ceph-object-controller: object store "ceph" deployment "rook-ceph-rgw-ceph-a" created
2026-03-20 05:49:39.942813 I | ceph-object-controller: enabling rgw dashboard
2026-03-20 05:49:41.879447 I | ceph-object-controller: created object store "ceph" in namespace "openstack"
2026-03-20 05:49:41.880813 I | ceph-object-controller: setting the dashboard api secret key
2026-03-20 05:49:42.727632 I | ceph-object-controller: done setting the dashboard api secret key
2026-03-20 05:49:42.845392 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:49:48.107513 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:49:49.477412 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:50:19.173059 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:82} {StateName:unknown Count:31}]"
2026-03-20 05:50:19.178059 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:50:19.214558 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:50:19.944520 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:50:49.790757 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:50:50.893389 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:108} {StateName:clean+premerge+peered Count:1}]"
2026-03-20 05:50:50.897408 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:51:05.154470 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:51:20.245191 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:103} {StateName:peering Count:1}]"
2026-03-20 05:51:20.250129 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:51:21.383849 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:103} {StateName:peering Count:1}]"
2026-03-20 05:51:21.388374 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:51:51.275141 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:51:52.585437 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:51:52.817829 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:106} {StateName:activating Count:1}]"
2026-03-20 05:51:52.823271 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:52:21.764031 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:107} {StateName:unknown Count:18} {StateName:activating Count:1}]"
2026-03-20 05:52:21.768299 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:52:23.285619 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:52:38.548641 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:52:52.260270 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:119} {StateName:peering Count:2} {StateName:clean+premerge+peered Count:1}]"
2026-03-20 05:52:52.265462 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:52:54.293409 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:120} {StateName:peering Count:1}]"
2026-03-20 05:52:54.298410 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:53:22.718617 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:53:24.682557 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:53:24.994774 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:53:53.182650 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:53:55.992259 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:54:10.628065 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:54:23.653196 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:54:26.486196 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:54:54.117874 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:54:57.092921 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:54:57.682332 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:55:24.593775 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:55:28.152194 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:55:43.041325 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:55:55.065134 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:55:59.096219 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:56:25.521130 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:56:28.995380 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:56:29.572539 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:56:56.007809 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:57:00.496801 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:57:14.945724 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:57:26.491586 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:57:30.970674 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:57:56.982634 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:58:00.877772 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:58:01.991586 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:58:27.449455 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:58:32.469991 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:58:46.848675 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:58:57.935323 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:59:03.412407 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:59:28.433880 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:59:32.796493 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:59:33.877517 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0