2026/04/25 16:01:57 maxprocs: Updating GOMAXPROCS=1: determined from CPU quota
2026-04-25 16:01:57.349418 I | rookcmd: starting Rook v1.15.9 with arguments '/usr/local/bin/rook ceph operator'
2026-04-25 16:01:57.349445 I | rookcmd: flag values: --enable-machine-disruption-budget=false, --help=false, --kubeconfig=, --log-level=INFO
2026-04-25 16:01:57.349447 I | cephcmd: starting Rook-Ceph operator
2026-04-25 16:01:57.443776 I | cephcmd: base ceph version inside the rook operator image is "ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable)"
2026-04-25 16:01:57.450669 I | op-k8sutil: operator setting "ROOK_LOG_LEVEL" = "INFO"
2026-04-25 16:01:57.450708 I | op-k8sutil: operator setting "CSI_ENABLE_OMAP_GENERATOR" = "false"
2026-04-25 16:01:57.450720 I | op-k8sutil: operator setting "CSI_ENABLE_VOLUME_GROUP_SNAPSHOT" = "true"
2026-04-25 16:01:57.450740 I | op-k8sutil: operator setting "CSI_RBD_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-rbdplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-04-25 16:01:57.450749 I | op-k8sutil: operator setting "ROOK_CSI_IMAGE_PULL_POLICY" = "IfNotPresent"
2026-04-25 16:01:57.450757 I | op-k8sutil: operator setting "ROOK_CSI_PROVISIONER_IMAGE" = "registry.k8s.io/sig-storage/csi-provisioner:v5.0.1"
2026-04-25 16:01:57.450765 I | op-k8sutil: operator setting "CSI_DISABLE_HOLDER_PODS" = "true"
2026-04-25 16:01:57.450773 I | op-k8sutil: operator setting "CSI_ENABLE_NFS_SNAPSHOTTER" = "true"
2026-04-25 16:01:57.450781 I | op-k8sutil: operator setting "CSI_PROVISIONER_REPLICAS" = "2"
2026-04-25 16:01:57.450787 I | op-k8sutil: operator setting "CSI_RBD_FSGROUPPOLICY" = "File"
2026-04-25 16:01:57.450821 I | op-k8sutil: operator setting "CSI_RBD_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-resizer\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-snapshotter\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-rbdplugin\n  resource:\n    requests:\n      memory: 512Mi\n    limits:\n      memory: 1Gi\n- name : csi-omap-generator\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-04-25 16:01:57.450840 I | op-k8sutil: operator setting "ROOK_CSIADDONS_IMAGE" = "quay.io/csiaddons/k8s-sidecar:v0.9.1"
2026-04-25 16:01:57.450847 I | op-k8sutil: operator setting "ROOK_CSI_CEPH_IMAGE" = "quay.io/cephcsi/cephcsi:v3.12.3"
2026-04-25 16:01:57.450853 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_CEPHFS" = "false"
2026-04-25 16:01:57.450864 I | op-k8sutil: operator setting "CSI_ENABLE_HOST_NETWORK" = "true"
2026-04-25 16:01:57.450872 I | op-k8sutil: operator setting "ROOK_CSI_RESIZER_IMAGE" = "registry.k8s.io/sig-storage/csi-resizer:v1.11.1"
2026-04-25 16:01:57.450886 I | op-k8sutil: operator setting "CSI_NFS_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-nfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n"
2026-04-25 16:01:57.450912 I | op-k8sutil: operator setting "CSI_NFS_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-nfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n"
2026-04-25 16:01:57.450926 I | op-k8sutil: operator setting "ROOK_CEPH_COMMANDS_TIMEOUT_SECONDS" = "15"
2026-04-25 16:01:57.450933 I | op-k8sutil: operator setting "ROOK_CSI_DISABLE_DRIVER" = "false"
2026-04-25 16:01:57.450940 I | op-k8sutil: operator setting "ROOK_ENABLE_DISCOVERY_DAEMON" = "false"
2026-04-25 16:01:57.450953 I | op-k8sutil: operator setting "CSI_FORCE_CEPHFS_KERNEL_CLIENT" = "true"
2026-04-25 16:01:57.450960 I | op-k8sutil: operator setting "CSI_ENABLE_TOPOLOGY" = "false"
2026-04-25 16:01:57.450983 I | op-k8sutil: operator setting "CSI_CEPHFS_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-cephfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-04-25 16:01:57.450997 I | op-k8sutil: operator setting "CSI_ENABLE_ENCRYPTION" = "false"
2026-04-25 16:01:57.451005 I | op-k8sutil: operator setting "CSI_ENABLE_METADATA" = "false"
2026-04-25 16:01:57.451012 I | op-k8sutil: operator setting "CSI_ENABLE_RBD_SNAPSHOTTER" = "true"
2026-04-25 16:01:57.451018 I | op-k8sutil: operator setting "CSI_NFS_ATTACH_REQUIRED" = "true"
2026-04-25 16:01:57.451034 I | op-k8sutil: operator setting "CSI_RBD_ATTACH_REQUIRED" = "true"
2026-04-25 16:01:57.451048 I | op-k8sutil: operator setting "ROOK_CSI_ATTACHER_IMAGE" = "registry.k8s.io/sig-storage/csi-attacher:v4.6.1"
2026-04-25 16:01:57.451055 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_NFS" = "false"
2026-04-25 16:01:57.451065 I | op-k8sutil: operator setting "CSI_ENABLE_CSIADDONS" = "false"
2026-04-25 16:01:57.451080 I | op-k8sutil: operator setting "ROOK_CSI_REGISTRAR_IMAGE" = "registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.11.1"
2026-04-25 16:01:57.451087 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_RBD" = "false"
2026-04-25 16:01:57.451107 I | op-k8sutil: operator setting "CSI_CEPHFS_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-resizer\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-snapshotter\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-cephfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-04-25 16:01:57.451121 I | op-k8sutil: operator setting "CSI_PLUGIN_ENABLE_SELINUX_HOST_MOUNT" = "false"
2026-04-25 16:01:57.451129 I | op-k8sutil: operator setting "CSI_CEPHFS_FSGROUPPOLICY" = "File"
2026-04-25 16:01:57.451136 I | op-k8sutil: operator setting "CSI_ENABLE_CEPHFS_SNAPSHOTTER" = "true"
2026-04-25 16:01:57.451144 I | op-k8sutil: operator setting "CSI_GRPC_TIMEOUT_SECONDS" = "150"
2026-04-25 16:01:57.451152 I | op-k8sutil: operator setting "CSI_NFS_FSGROUPPOLICY" = "File"
2026-04-25 16:01:57.451160 I | op-k8sutil: operator setting "CSI_PLUGIN_PRIORITY_CLASSNAME" = "system-node-critical"
2026-04-25 16:01:57.451178 I | op-k8sutil: operator setting "CSI_PROVISIONER_PRIORITY_CLASSNAME" = "system-cluster-critical"
2026-04-25 16:01:57.451186 I | op-k8sutil: operator setting "ROOK_CEPH_ALLOW_LOOP_DEVICES" = "false"
2026-04-25 16:01:57.451193 I | op-k8sutil: operator setting "ROOK_CSI_SNAPSHOTTER_IMAGE" = "registry.k8s.io/sig-storage/csi-snapshotter:v8.0.1"
2026-04-25 16:01:57.451200 I | op-k8sutil: operator setting "CSI_CEPHFS_ATTACH_REQUIRED" = "true"
2026-04-25 16:01:57.451208 I | op-k8sutil: operator setting "ROOK_OBC_WATCH_OPERATOR_NAMESPACE" = "true"
2026-04-25 16:01:57.451217 I | operator: watching all namespaces for Ceph CRs
2026-04-25 16:01:57.451373 I | operator: setting up schemes
2026-04-25 16:01:57.455852 I | operator: setting up the controller-runtime manager
2026-04-25 16:01:57.456583 I | ceph-cluster-controller: successfully started
2026-04-25 16:01:57.457244 I | ceph-cluster-controller: enabling hotplug orchestration
2026-04-25 16:01:57.457280 I | ceph-nodedaemon-controller: successfully started
2026-04-25 16:01:57.457296 I | ceph-block-pool-controller: successfully started
2026-04-25 16:01:57.457316 I | ceph-object-store-user-controller: successfully started
2026-04-25 16:01:57.457350 I | ceph-object-realm-controller: successfully started
2026-04-25 16:01:57.457371 I | ceph-object-zonegroup-controller: successfully started
2026-04-25 16:01:57.457382 I | ceph-object-zone-controller: successfully started
2026-04-25 16:01:57.457501 I | ceph-object-controller: successfully started
2026-04-25 16:01:57.457558 I | ceph-file-controller: successfully started
2026-04-25 16:01:57.457590 I | ceph-nfs-controller: successfully started
2026-04-25 16:01:57.457639 I | ceph-rbd-mirror-controller: successfully started
2026-04-25 16:01:57.457685 I | ceph-client-controller: successfully started
2026-04-25 16:01:57.457711 I | ceph-filesystem-mirror-controller: successfully started
2026-04-25 16:01:57.457741 I | operator: rook-ceph-operator-config-controller successfully started
2026-04-25 16:01:57.457760 I | ceph-csi: rook-ceph-operator-csi-controller successfully started
2026-04-25 16:01:57.457979 I | op-bucket-prov: rook-ceph-operator-bucket-controller successfully started
2026-04-25 16:01:57.458015 I | ceph-bucket-topic: successfully started
2026-04-25 16:01:57.458027 I | ceph-bucket-notification: successfully started
2026-04-25 16:01:57.458040 I | ceph-bucket-notification: successfully started
2026-04-25 16:01:57.458054 I | ceph-fs-subvolumegroup-controller: successfully started
2026-04-25 16:01:57.458121 I | blockpool-rados-namespace-controller: successfully started
2026-04-25 16:01:57.458198 I | ceph-cosi-controller: successfully started
2026-04-25 16:01:57.458225 I | operator: starting the controller-runtime manager
2026-04-25 16:01:57.570620 I | operator: rook-ceph-operator-config-controller done reconciling
2026-04-25 16:01:57.603157 I | ceph-csi: successfully created csi config map "rook-ceph-csi-config"
2026-04-25 16:01:57.606052 I | op-k8sutil: removing daemonset csi-rbdplugin if it exists
2026-04-25 16:01:57.609321 I | op-k8sutil: removing deployment csi-rbdplugin-provisioner if it exists
2026-04-25 16:01:57.616162 I | ceph-csi: successfully removed CSI Ceph RBD driver
2026-04-25 16:01:57.616183 I | op-k8sutil: removing daemonset csi-cephfsplugin if it exists
2026-04-25 16:01:57.618061 I | op-k8sutil: removing deployment csi-cephfsplugin-provisioner if it exists
2026-04-25 16:01:57.625055 I | ceph-csi: successfully removed CSI CephFS driver
2026-04-25 16:01:57.625077 I | op-k8sutil: removing daemonset csi-nfsplugin if it exists
2026-04-25 16:01:57.627096 I | op-k8sutil: removing deployment csi-nfsplugin-provisioner if it exists
2026-04-25 16:01:57.632130 I | ceph-csi: successfully removed CSI NFS driver
2026-04-25 16:11:08.065766 I | ceph-spec: adding finalizer "cephcluster.ceph.rook.io" on "ceph"
2026-04-25 16:11:08.072778 I | ceph-spec: adding finalizer "cephobjectstore.ceph.rook.io" on "ceph"
2026-04-25 16:11:08.075070 I | clusterdisruption-controller: deleted all legacy node drain canary pods
2026-04-25 16:11:08.084103 I | ceph-cluster-controller: reconciling ceph cluster in namespace "openstack"
2026-04-25 16:11:08.084495 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-04-25 16:11:08.084508 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-04-25 16:11:08.084551 I | op-bucket-prov: ceph bucket provisioner launched watching for provisioner "openstack.ceph.rook.io/bucket"
2026-04-25 16:11:08.084837 I | op-bucket-prov: successfully reconciled bucket provisioner
I0425 16:11:08.084885       1 manager.go:135] "msg"="starting provisioner" "logger"="objectbucket.io/provisioner-manager" "name"="openstack.ceph.rook.io/bucket"
2026-04-25 16:11:08.104894 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-04-25 16:11:08.104918 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-04-25 16:11:08.104950 I | ceph-spec: found the cluster info to connect to the external cluster. will use "client.admin" to check health and monitor status. mons=map[instance:0xc001513230]
2026-04-25 16:11:08.104965 I | ceph-spec: detecting the ceph image version for image harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7...
2026-04-25 16:11:09.071820 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-04-25 16:11:09.071847 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-04-25 16:11:09.476830 I | op-k8sutil: removing daemonset csi-rbdplugin if it exists
2026-04-25 16:11:09.479477 I | op-k8sutil: removing deployment csi-rbdplugin-provisioner if it exists
2026-04-25 16:11:09.673614 I | ceph-csi: successfully removed CSI Ceph RBD driver
2026-04-25 16:11:09.673639 I | op-k8sutil: removing daemonset csi-cephfsplugin if it exists
2026-04-25 16:11:09.678362 I | op-k8sutil: removing deployment csi-cephfsplugin-provisioner if it exists
2026-04-25 16:11:09.874091 I | ceph-csi: successfully removed CSI CephFS driver
2026-04-25 16:11:09.874117 I | op-k8sutil: removing daemonset csi-nfsplugin if it exists
2026-04-25 16:11:09.879516 I | op-k8sutil: removing deployment csi-nfsplugin-provisioner if it exists
2026-04-25 16:11:10.080727 I | ceph-csi: successfully removed CSI NFS driver
2026-04-25 16:11:25.226483 I | ceph-spec: detected ceph image version: "18.2.7-0 reef"
2026-04-25 16:11:25.226513 I | ceph-cluster-controller: validating ceph version from provided image
2026-04-25 16:11:25.235051 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-04-25 16:11:25.235069 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-04-25 16:11:25.237317 I | cephclient: writing config file /var/lib/rook/openstack/openstack.config
2026-04-25 16:11:25.237396 I | cephclient: generated admin config in /var/lib/rook/openstack
2026-04-25 16:11:25.704065 E | cephver: external cluster ceph version is a major version higher "18.2.7-0 reef" than the local cluster "0.0.0-0 <unknown version>", consider upgrading
2026-04-25 16:11:26.186455 I | ceph-cluster-controller: cluster "openstack": version "18.2.7-0 reef" detected for image "harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7"
2026-04-25 16:11:26.208127 I | ceph-cluster-controller: creating "rook-config-override" configmap
2026-04-25 16:11:26.211379 I | ceph-cluster-controller: creating "rook-ceph-config" secret
2026-04-25 16:11:26.226938 I | ceph-cluster-controller: external cluster identity established
2026-04-25 16:11:26.226969 I | cephclient: getting or creating ceph auth key "client.csi-rbd-provisioner"
2026-04-25 16:11:26.703515 I | cephclient: getting or creating ceph auth key "client.csi-rbd-node"
2026-04-25 16:11:27.190072 I | cephclient: getting or creating ceph auth key "client.csi-cephfs-provisioner"
2026-04-25 16:11:27.669492 I | cephclient: getting or creating ceph auth key "client.csi-cephfs-node"
2026-04-25 16:11:28.177895 I | ceph-csi: created kubernetes csi secrets for cluster "openstack"
2026-04-25 16:11:28.200401 I | ceph-cluster-controller: successfully updated csi config map
2026-04-25 16:11:28.200432 I | cephclient: getting or creating ceph auth key "client.crash"
2026-04-25 16:11:29.146180 I | ceph-nodedaemon-controller: created kubernetes crash collector secret for cluster "openstack"
2026-04-25 16:11:29.148514 I | ceph-cluster-controller: enabling ceph mon monitoring goroutine for cluster "openstack"
2026-04-25 16:11:29.148559 I | ceph-cluster-controller: ceph status check interval is 1m0s
2026-04-25 16:11:29.148570 I | ceph-cluster-controller: enabling ceph status monitoring goroutine for cluster "openstack"
2026-04-25 16:11:29.260660 I | clusterdisruption-controller: all PGs are active+clean. Restoring default OSD pdb settings
2026-04-25 16:11:29.260688 I | clusterdisruption-controller: creating the default pdb "rook-ceph-osd" with maxUnavailable=1 for all osd
2026-04-25 16:11:29.271206 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:11:30.363133 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:11:31.482672 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:11:38.155107 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-04-25 16:11:38.155143 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-04-25 16:11:38.155182 I | ceph-spec: detecting the ceph image version for image harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7...
2026-04-25 16:11:41.784712 I | ceph-spec: detected ceph image version: "18.2.7-0 reef"
2026-04-25 16:11:43.071021 I | ceph-object-controller: reconciling object store deployments
2026-04-25 16:11:43.089124 I | ceph-object-controller: ceph object store gateway service running at 10.98.184.185
2026-04-25 16:11:43.089168 I | ceph-object-controller: reconciling object store pools
2026-04-25 16:11:46.213178 I | cephclient: reconciling replicated pool ceph.rgw.control succeeded
2026-04-25 16:11:46.213206 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.control"
2026-04-25 16:11:50.327293 I | cephclient: reconciling replicated pool ceph.rgw.meta succeeded
2026-04-25 16:11:50.327334 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.meta"
2026-04-25 16:11:54.466675 I | cephclient: reconciling replicated pool ceph.rgw.log succeeded
2026-04-25 16:11:54.466709 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.log"
2026-04-25 16:11:58.698005 I | cephclient: reconciling replicated pool ceph.rgw.buckets.index succeeded
2026-04-25 16:11:58.698046 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.buckets.index"
2026-04-25 16:11:59.762746 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:49} {StateName:unknown Count:8}]"
2026-04-25 16:11:59.767428 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:12:01.060158 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:49} {StateName:unknown Count:8}]"
2026-04-25 16:12:01.064143 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:12:02.788889 I | cephclient: reconciling replicated pool ceph.rgw.buckets.non-ec succeeded
2026-04-25 16:12:02.788934 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.buckets.non-ec"
2026-04-25 16:12:06.847381 I | cephclient: reconciling replicated pool ceph.rgw.otp succeeded
2026-04-25 16:12:06.847425 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.otp"
2026-04-25 16:12:11.466390 I | cephclient: reconciling replicated pool .rgw.root succeeded
2026-04-25 16:12:11.466417 I | cephclient: setting pool property "pg_num_min" to "8" on pool ".rgw.root"
2026-04-25 16:12:16.047882 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-25 16:12:16.597280 I | cephclient: reconciling replicated pool ceph.rgw.buckets.data succeeded
2026-04-25 16:12:16.597320 I | ceph-object-controller: configuring object store "ceph"
2026-04-25 16:12:17.232420 I | ceph-object-controller: Object store "ceph": realm=ceph, zonegroup=ceph, zone=ceph
2026-04-25 16:12:17.439234 I | ceph-object-controller: committing changes to RGW configuration period for CephObjectStore "openstack/ceph"
2026-04-25 16:12:17.811131 I | ceph-object-controller: configuration for object-store ceph is complete
2026-04-25 16:12:17.816877 I | ceph-object-controller: creating object store "ceph" in namespace "openstack"
2026-04-25 16:12:17.820539 I | cephclient: getting or creating ceph auth key "client.rgw.ceph.a"
2026-04-25 16:12:18.326100 I | ceph-object-controller: setting rgw config flags
2026-04-25 16:12:18.326128 I | ceph-object-controller: Configuring authentication with keystone
2026-04-25 16:12:18.326148 I | op-config: setting option "rgw_zonegroup" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:18.702614 I | op-config: successfully set option "rgw_zonegroup" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:18.702672 I | op-config: setting option "rgw_keystone_accepted_roles" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:19.097887 I | op-config: successfully set option "rgw_keystone_accepted_roles" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:19.097932 I | op-config: setting option "rgw_keystone_admin_domain" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:19.493438 I | op-config: successfully set option "rgw_keystone_admin_domain" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:19.493475 I | op-config: setting option "rgw_swift_account_in_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:19.912394 I | op-config: successfully set option "rgw_swift_account_in_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:19.912428 I | op-config: setting option "rgw_swift_versioning_enabled" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:20.335761 I | op-config: successfully set option "rgw_swift_versioning_enabled" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:20.335790 I | op-config: setting option "rgw_enable_usage_log" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:20.717494 I | op-config: successfully set option "rgw_enable_usage_log" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:20.717519 I | op-config: setting option "rgw_log_nonexistent_bucket" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:21.102950 I | op-config: successfully set option "rgw_log_nonexistent_bucket" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:21.102981 I | op-config: setting option "rgw_zone" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:21.466564 I | op-config: successfully set option "rgw_zone" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:21.466616 I | op-config: setting option "rgw_keystone_api_version" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:21.898686 I | op-config: successfully set option "rgw_keystone_api_version" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:21.898711 I | op-config: setting option "rgw_keystone_admin_project" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:22.287132 I | op-config: successfully set option "rgw_keystone_admin_project" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:22.287171 I | op-config: setting option "rgw_s3_auth_use_keystone" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:22.669093 I | op-config: successfully set option "rgw_s3_auth_use_keystone" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:22.669122 I | op-config: setting option "rgw_run_sync_thread" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:23.075387 I | op-config: successfully set option "rgw_run_sync_thread" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:23.075424 I | op-config: setting option "rgw_keystone_admin_password" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:23.452683 I | op-config: successfully set option "rgw_keystone_admin_password" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:23.452728 I | op-config: setting option "rgw_keystone_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:23.856020 I | op-config: successfully set option "rgw_keystone_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:23.856056 I | op-config: setting option "rgw_keystone_implicit_tenants" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:24.244044 I | op-config: successfully set option "rgw_keystone_implicit_tenants" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:24.244074 I | op-config: setting option "rgw_keystone_admin_user" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:24.657442 I | op-config: successfully set option "rgw_keystone_admin_user" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:24.657477 I | op-config: setting option "rgw_log_object_name_utc" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:25.091605 I | op-config: successfully set option "rgw_log_object_name_utc" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-25 16:12:25.091832 I | ceph-object-controller: object store "ceph" deployment "rook-ceph-rgw-ceph-a" created
2026-04-25 16:12:25.143836 I | ceph-object-controller: enabling rgw dashboard
2026-04-25 16:12:27.053728 I | ceph-object-controller: created object store "ceph" in namespace "openstack"
2026-04-25 16:12:27.054900 I | ceph-object-controller: setting the dashboard api secret key
2026-04-25 16:12:27.939884 I | ceph-object-controller: done setting the dashboard api secret key
2026-04-25 16:12:28.062898 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:12:30.253559 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:12:31.660633 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:13:00.704984 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:97} {StateName:peering Count:15}]"
2026-04-25 16:13:00.708981 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:13:02.341071 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-25 16:13:02.482797 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:97} {StateName:peering Count:15}]"
2026-04-25 16:13:02.486105 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:13:31.181840 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:115} {StateName:peering Count:1}]"
2026-04-25 16:13:31.185683 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:13:33.366136 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:116} {StateName:unknown Count:5} {StateName:creating+activating Count:3}]"
2026-04-25 16:13:33.370231 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:13:48.274476 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-25 16:14:01.660622 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:147} {StateName:peering Count:30} {StateName:unknown Count:24}]"
2026-04-25 16:14:01.664450 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:14:03.825923 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:180} {StateName:peering Count:15} {StateName:unknown Count:6}]"
2026-04-25 16:14:03.829685 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:14:32.150120 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:14:34.946761 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-25 16:14:35.168569 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:15:02.607202 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:190} {StateName:peering Count:1}]"
2026-04-25 16:15:02.611269 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:15:05.630047 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:189} {StateName:clean+premerge+peered Count:1}]"
2026-04-25 16:15:05.634877 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:15:20.913345 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-25 16:15:33.071253 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:186} {StateName:peering Count:1}]"
2026-04-25 16:15:33.076326 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:15:36.477556 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:16:03.550429 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:16:07.334529 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-25 16:16:07.408228 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:16:34.023383 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:16:38.442993 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:16:53.299290 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-25 16:17:04.517089 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:17:08.919258 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:17:34.976833 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:17:40.156785 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-25 16:17:40.437022 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:18:05.433283 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:18:10.926905 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:18:26.074212 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-25 16:18:35.921864 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-25 16:18:41.864438 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0