2026/03/20 05:49:11 maxprocs: Updating GOMAXPROCS=1: determined from CPU quota
2026-03-20 05:49:11.806463 I | rookcmd: starting Rook v1.15.9 with arguments '/usr/local/bin/rook ceph operator'
2026-03-20 05:49:11.806483 I | rookcmd: flag values: --enable-machine-disruption-budget=false, --help=false, --kubeconfig=, --log-level=INFO
2026-03-20 05:49:11.806486 I | cephcmd: starting Rook-Ceph operator
2026-03-20 05:49:11.915789 I | cephcmd: base ceph version inside the rook operator image is "ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable)"
2026-03-20 05:49:11.924916 I | op-k8sutil: operator setting "CSI_ENABLE_VOLUME_GROUP_SNAPSHOT" = "true"
2026-03-20 05:49:11.924956 I | op-k8sutil: operator setting "CSI_NFS_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-nfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n"
2026-03-20 05:49:11.924967 I | op-k8sutil: operator setting "ROOK_CSIADDONS_IMAGE" = "quay.io/csiaddons/k8s-sidecar:v0.9.1"
2026-03-20 05:49:11.924973 I | op-k8sutil: operator setting "CSI_ENABLE_OMAP_GENERATOR" = "false"
2026-03-20 05:49:11.924978 I | op-k8sutil: operator setting "CSI_ENABLE_METADATA" = "false"
2026-03-20 05:49:11.924986 I | op-k8sutil: operator setting "CSI_ENABLE_RBD_SNAPSHOTTER" = "true"
2026-03-20 05:49:11.924990 I | op-k8sutil: operator setting "CSI_NFS_ATTACH_REQUIRED" = "true"
2026-03-20 05:49:11.924994 I | op-k8sutil: operator setting "ROOK_CEPH_ALLOW_LOOP_DEVICES" = "false"
2026-03-20 05:49:11.924998 I | op-k8sutil: operator setting "ROOK_CSI_IMAGE_PULL_POLICY" = "IfNotPresent"
2026-03-20 05:49:11.925003 I | op-k8sutil: operator setting "ROOK_OBC_WATCH_OPERATOR_NAMESPACE" = "true"
2026-03-20 05:49:11.925008 I | op-k8sutil: operator setting "CSI_ENABLE_CSIADDONS" = "false"
2026-03-20 05:49:11.925012 I | op-k8sutil: operator setting "CSI_PLUGIN_ENABLE_SELINUX_HOST_MOUNT" = "false"
2026-03-20 05:49:11.925018 I | op-k8sutil: operator setting "CSI_PLUGIN_PRIORITY_CLASSNAME" = "system-node-critical"
2026-03-20 05:49:11.925022 I | op-k8sutil: operator setting "CSI_RBD_ATTACH_REQUIRED" = "true"
2026-03-20 05:49:11.925027 I | op-k8sutil: operator setting "CSI_RBD_FSGROUPPOLICY" = "File"
2026-03-20 05:49:11.925031 I | op-k8sutil: operator setting "CSI_ENABLE_ENCRYPTION" = "false"
2026-03-20 05:49:11.925036 I | op-k8sutil: operator setting "CSI_ENABLE_HOST_NETWORK" = "true"
2026-03-20 05:49:11.925041 I | op-k8sutil: operator setting "CSI_FORCE_CEPHFS_KERNEL_CLIENT" = "true"
2026-03-20 05:49:11.925046 I | op-k8sutil: operator setting "CSI_NFS_FSGROUPPOLICY" = "File"
2026-03-20 05:49:11.925051 I | op-k8sutil: operator setting "ROOK_CSI_ATTACHER_IMAGE" = "registry.k8s.io/sig-storage/csi-attacher:v4.6.1"
2026-03-20 05:49:11.925055 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_CEPHFS" = "false"
2026-03-20 05:49:11.925060 I | op-k8sutil: operator setting "ROOK_CSI_RESIZER_IMAGE" = "registry.k8s.io/sig-storage/csi-resizer:v1.11.1"
2026-03-20 05:49:11.925073 I | op-k8sutil: operator setting "CSI_CEPHFS_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-resizer\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-snapshotter\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-cephfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-03-20 05:49:11.925083 I | op-k8sutil: operator setting "CSI_PROVISIONER_REPLICAS" = "2"
2026-03-20 05:49:11.925087 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_RBD" = "false"
2026-03-20 05:49:11.925092 I | op-k8sutil: operator setting "ROOK_CSI_SNAPSHOTTER_IMAGE" = "registry.k8s.io/sig-storage/csi-snapshotter:v8.0.1"
2026-03-20 05:49:11.925096 I | op-k8sutil: operator setting "CSI_ENABLE_CEPHFS_SNAPSHOTTER" = "true"
2026-03-20 05:49:11.925100 I | op-k8sutil: operator setting "ROOK_CEPH_COMMANDS_TIMEOUT_SECONDS" = "15"
2026-03-20 05:49:11.925109 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_NFS" = "false"
2026-03-20 05:49:11.925115 I | op-k8sutil: operator setting "ROOK_CSI_PROVISIONER_IMAGE" = "registry.k8s.io/sig-storage/csi-provisioner:v5.0.1"
2026-03-20 05:49:11.925135 I | op-k8sutil: operator setting "CSI_RBD_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-resizer\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-snapshotter\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-rbdplugin\n  resource:\n    requests:\n      memory: 512Mi\n    limits:\n      memory: 1Gi\n- name : csi-omap-generator\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-03-20 05:49:11.925151 I | op-k8sutil: operator setting "CSI_CEPHFS_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-cephfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-03-20 05:49:11.925157 I | op-k8sutil: operator setting "CSI_DISABLE_HOLDER_PODS" = "true"
2026-03-20 05:49:11.925161 I | op-k8sutil: operator setting "CSI_ENABLE_NFS_SNAPSHOTTER" = "true"
2026-03-20 05:49:11.925167 I | op-k8sutil: operator setting "CSI_PROVISIONER_PRIORITY_CLASSNAME" = "system-cluster-critical"
2026-03-20 05:49:11.925172 I | op-k8sutil: operator setting "ROOK_CSI_CEPH_IMAGE" = "quay.io/cephcsi/cephcsi:v3.12.3"
2026-03-20 05:49:11.925178 I | op-k8sutil: operator setting "ROOK_CSI_REGISTRAR_IMAGE" = "registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.11.1"
2026-03-20 05:49:11.925183 I | op-k8sutil: operator setting "ROOK_ENABLE_DISCOVERY_DAEMON" = "false"
2026-03-20 05:49:11.925187 I | op-k8sutil: operator setting "CSI_CEPHFS_FSGROUPPOLICY" = "File"
2026-03-20 05:49:11.925192 I | op-k8sutil: operator setting "ROOK_LOG_LEVEL" = "INFO"
2026-03-20 05:49:11.925197 I | op-k8sutil: operator setting "CSI_ENABLE_TOPOLOGY" = "false"
2026-03-20 05:49:11.925201 I | op-k8sutil: operator setting "CSI_GRPC_TIMEOUT_SECONDS" = "150"
2026-03-20 05:49:11.925213 I | op-k8sutil: operator setting "CSI_NFS_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-nfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n"
2026-03-20 05:49:11.925232 I | op-k8sutil: operator setting "CSI_RBD_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-rbdplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-03-20 05:49:11.925237 I | op-k8sutil: operator setting "ROOK_CSI_DISABLE_DRIVER" = "false"
2026-03-20 05:49:11.925241 I | op-k8sutil: operator setting "CSI_CEPHFS_ATTACH_REQUIRED" = "true"
2026-03-20 05:49:11.925246 I | operator: watching all namespaces for Ceph CRs
2026-03-20 05:49:11.925944 I | operator: setting up schemes
2026-03-20 05:49:11.929765 I | operator: setting up the controller-runtime manager
2026-03-20 05:49:11.930289 I | ceph-cluster-controller: successfully started
2026-03-20 05:49:11.930866 I | ceph-cluster-controller: enabling hotplug orchestration
2026-03-20 05:49:11.930897 I | ceph-nodedaemon-controller: successfully started
2026-03-20 05:49:11.930913 I | ceph-block-pool-controller: successfully started
2026-03-20 05:49:11.930940 I | ceph-object-store-user-controller: successfully started
2026-03-20 05:49:11.930961 I | ceph-object-realm-controller: successfully started
2026-03-20 05:49:11.930979 I | ceph-object-zonegroup-controller: successfully started
2026-03-20 05:49:11.930990 I | ceph-object-zone-controller: successfully started
2026-03-20 05:49:11.931128 I | ceph-object-controller: successfully started
2026-03-20 05:49:11.931174 I | ceph-file-controller: successfully started
2026-03-20 05:49:11.931208 I | ceph-nfs-controller: successfully started
2026-03-20 05:49:11.931250 I | ceph-rbd-mirror-controller: successfully started
2026-03-20 05:49:11.931283 I | ceph-client-controller: successfully started
2026-03-20 05:49:11.931309 I | ceph-filesystem-mirror-controller: successfully started
2026-03-20 05:49:11.931342 I | operator: rook-ceph-operator-config-controller successfully started
2026-03-20 05:49:11.931363 I | ceph-csi: rook-ceph-operator-csi-controller successfully started
2026-03-20 05:49:11.931576 I | op-bucket-prov: rook-ceph-operator-bucket-controller successfully started
2026-03-20 05:49:11.931606 I | ceph-bucket-topic: successfully started
2026-03-20 05:49:11.931618 I | ceph-bucket-notification: successfully started
2026-03-20 05:49:11.931634 I | ceph-bucket-notification: successfully started
2026-03-20 05:49:11.931650 I | ceph-fs-subvolumegroup-controller: successfully started
2026-03-20 05:49:11.931722 I | blockpool-rados-namespace-controller: successfully started
2026-03-20 05:49:11.931794 I | ceph-cosi-controller: successfully started
2026-03-20 05:49:11.931820 I | operator: starting the controller-runtime manager
2026-03-20 05:49:12.349924 I | operator: rook-ceph-operator-config-controller done reconciling
2026-03-20 05:49:12.379803 I | ceph-csi: successfully created csi config map "rook-ceph-csi-config"
2026-03-20 05:49:12.382241 I | op-k8sutil: removing daemonset csi-rbdplugin if it exists
2026-03-20 05:49:12.386171 I | op-k8sutil: removing deployment csi-rbdplugin-provisioner if it exists
2026-03-20 05:49:12.392928 I | ceph-csi: successfully removed CSI Ceph RBD driver
2026-03-20 05:49:12.392945 I | op-k8sutil: removing daemonset csi-cephfsplugin if it exists
2026-03-20 05:49:12.394851 I | op-k8sutil: removing deployment csi-cephfsplugin-provisioner if it exists
2026-03-20 05:49:12.400940 I | ceph-csi: successfully removed CSI CephFS driver
2026-03-20 05:49:12.400956 I | op-k8sutil: removing daemonset csi-nfsplugin if it exists
2026-03-20 05:49:12.403039 I | op-k8sutil: removing deployment csi-nfsplugin-provisioner if it exists
2026-03-20 05:49:12.408815 I | ceph-csi: successfully removed CSI NFS driver
2026-03-20 05:49:33.859719 I | ceph-spec: adding finalizer "cephcluster.ceph.rook.io" on "ceph"
2026-03-20 05:49:33.866255 I | clusterdisruption-controller: deleted all legacy node drain canary pods
2026-03-20 05:49:33.867724 I | ceph-spec: adding finalizer "cephobjectstore.ceph.rook.io" on "ceph"
2026-03-20 05:49:33.878455 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-03-20 05:49:33.878506 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-03-20 05:49:33.878981 I | op-bucket-prov: ceph bucket provisioner launched watching for provisioner "openstack.ceph.rook.io/bucket"
2026-03-20 05:49:33.879706 I | op-bucket-prov: successfully reconciled bucket provisioner
I0320 05:49:33.879841       1 manager.go:135] "msg"="starting provisioner" "logger"="objectbucket.io/provisioner-manager" "name"="openstack.ceph.rook.io/bucket"
2026-03-20 05:49:33.880886 I | ceph-cluster-controller: reconciling ceph cluster in namespace "openstack"
2026-03-20 05:49:33.912063 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-03-20 05:49:33.912099 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-03-20 05:49:33.912116 I | ceph-spec: found the cluster info to connect to the external cluster. will use "client.admin" to check health and monitor status. mons=map[instance:0xc0017a6f00]
2026-03-20 05:49:33.912308 I | ceph-spec: detecting the ceph image version for image harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7...
2026-03-20 05:49:34.862281 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-03-20 05:49:34.862316 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-03-20 05:49:35.266545 I | op-k8sutil: removing daemonset csi-rbdplugin if it exists
2026-03-20 05:49:35.269175 I | op-k8sutil: removing deployment csi-rbdplugin-provisioner if it exists
2026-03-20 05:49:35.466251 I | ceph-csi: successfully removed CSI Ceph RBD driver
2026-03-20 05:49:35.466281 I | op-k8sutil: removing daemonset csi-cephfsplugin if it exists
2026-03-20 05:49:35.468908 I | op-k8sutil: removing deployment csi-cephfsplugin-provisioner if it exists
2026-03-20 05:49:35.668823 I | ceph-csi: successfully removed CSI CephFS driver
2026-03-20 05:49:35.668852 I | op-k8sutil: removing daemonset csi-nfsplugin if it exists
2026-03-20 05:49:35.672365 I | op-k8sutil: removing deployment csi-nfsplugin-provisioner if it exists
2026-03-20 05:49:35.866452 I | ceph-csi: successfully removed CSI NFS driver
2026-03-20 05:49:51.572170 I | ceph-spec: detected ceph image version: "18.2.7-0 reef"
2026-03-20 05:49:51.572214 I | ceph-cluster-controller: validating ceph version from provided image
2026-03-20 05:49:51.578986 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-03-20 05:49:51.579027 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-03-20 05:49:51.580767 I | cephclient: writing config file /var/lib/rook/openstack/openstack.config
2026-03-20 05:49:51.580864 I | cephclient: generated admin config in /var/lib/rook/openstack
2026-03-20 05:49:52.041139 E | cephver: external cluster ceph version is a major version higher "18.2.7-0 reef" than the local cluster "0.0.0-0 <unknown version>", consider upgrading
2026-03-20 05:49:53.489040 I | ceph-cluster-controller: cluster "openstack": version "18.2.7-0 reef" detected for image "harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7"
2026-03-20 05:49:53.715578 I | ceph-cluster-controller: creating "rook-config-override" configmap
2026-03-20 05:49:53.718047 I | ceph-cluster-controller: creating "rook-ceph-config" secret
2026-03-20 05:49:54.297710 I | ceph-cluster-controller: external cluster identity established
2026-03-20 05:49:54.297746 I | cephclient: getting or creating ceph auth key "client.csi-rbd-provisioner"
2026-03-20 05:49:55.081647 I | clusterdisruption-controller: all PGs are active+clean. Restoring default OSD pdb settings
2026-03-20 05:49:55.081683 I | clusterdisruption-controller: creating the default pdb "rook-ceph-osd" with maxUnavailable=1 for all osd
2026-03-20 05:49:55.534956 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:49:56.137120 I | cephclient: getting or creating ceph auth key "client.csi-rbd-node"
2026-03-20 05:49:56.722011 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:49:57.626875 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:49:57.861551 I | cephclient: getting or creating ceph auth key "client.csi-cephfs-provisioner"
2026-03-20 05:49:59.656684 I | cephclient: getting or creating ceph auth key "client.csi-cephfs-node"
2026-03-20 05:50:00.332424 I | ceph-csi: created kubernetes csi secrets for cluster "openstack"
2026-03-20 05:50:00.348342 I | ceph-cluster-controller: successfully updated csi config map
2026-03-20 05:50:00.348364 I | cephclient: getting or creating ceph auth key "client.crash"
2026-03-20 05:50:00.875714 I | ceph-nodedaemon-controller: created kubernetes crash collector secret for cluster "openstack"
2026-03-20 05:50:00.897209 I | ceph-cluster-controller: enabling ceph mon monitoring goroutine for cluster "openstack"
2026-03-20 05:50:00.897256 I | ceph-cluster-controller: ceph status check interval is 1m0s
2026-03-20 05:50:00.897262 I | ceph-cluster-controller: enabling ceph status monitoring goroutine for cluster "openstack"
2026-03-20 05:50:04.319489 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-03-20 05:50:04.319513 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-03-20 05:50:04.319525 I | ceph-spec: detecting the ceph image version for image harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7...
2026-03-20 05:50:06.762793 I | ceph-spec: detected ceph image version: "18.2.7-0 reef"
2026-03-20 05:50:07.994181 I | ceph-object-controller: reconciling object store deployments
2026-03-20 05:50:08.012497 I | ceph-object-controller: ceph object store gateway service running at 10.111.66.78
2026-03-20 05:50:08.012532 I | ceph-object-controller: reconciling object store pools
2026-03-20 05:50:11.445915 I | cephclient: reconciling replicated pool ceph.rgw.control succeeded
2026-03-20 05:50:11.445946 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.control"
2026-03-20 05:50:16.167809 I | cephclient: reconciling replicated pool ceph.rgw.meta succeeded
2026-03-20 05:50:16.167857 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.meta"
2026-03-20 05:50:20.746365 I | cephclient: reconciling replicated pool ceph.rgw.log succeeded
2026-03-20 05:50:20.746403 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.log"
2026-03-20 05:50:24.915590 I | cephclient: reconciling replicated pool ceph.rgw.buckets.index succeeded
2026-03-20 05:50:24.915630 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.buckets.index"
2026-03-20 05:50:26.062989 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:49} {StateName:unknown Count:5} {StateName:creating+peering Count:3}]"
2026-03-20 05:50:26.067560 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:50:27.513062 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:49} {StateName:unknown Count:5} {StateName:creating+peering Count:3}]"
2026-03-20 05:50:27.517649 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:50:30.011554 I | cephclient: reconciling replicated pool ceph.rgw.buckets.non-ec succeeded
2026-03-20 05:50:30.011584 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.buckets.non-ec"
2026-03-20 05:50:34.112078 I | cephclient: reconciling replicated pool ceph.rgw.otp succeeded
2026-03-20 05:50:34.112107 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.otp"
2026-03-20 05:50:38.194748 I | cephclient: reconciling replicated pool .rgw.root succeeded
2026-03-20 05:50:38.194779 I | cephclient: setting pool property "pg_num_min" to "8" on pool ".rgw.root"
2026-03-20 05:50:42.251882 I | cephclient: reconciling replicated pool ceph.rgw.buckets.data succeeded
2026-03-20 05:50:42.251928 I | ceph-object-controller: configuring object store "ceph"
2026-03-20 05:50:42.963134 I | ceph-object-controller: Object store "ceph": realm=ceph, zonegroup=ceph, zone=ceph
2026-03-20 05:50:43.205715 I | ceph-object-controller: committing changes to RGW configuration period for CephObjectStore "openstack/ceph"
2026-03-20 05:50:43.702594 I | ceph-object-controller: configuration for object-store ceph is complete
2026-03-20 05:50:43.705726 I | ceph-object-controller: creating object store "ceph" in namespace "openstack"
2026-03-20 05:50:43.709037 I | cephclient: getting or creating ceph auth key "client.rgw.ceph.a"
2026-03-20 05:50:44.185596 I | ceph-object-controller: setting rgw config flags
2026-03-20 05:50:44.185617 I | ceph-object-controller: Configuring authentication with keystone
2026-03-20 05:50:44.185631 I | op-config: setting option "rgw_zone" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:44.583797 I | op-config: successfully set option "rgw_zone" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:44.583834 I | op-config: setting option "rgw_keystone_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:44.973887 I | op-config: successfully set option "rgw_keystone_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:44.973923 I | op-config: setting option "rgw_keystone_admin_project" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:45.356721 I | op-config: successfully set option "rgw_keystone_admin_project" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:45.356797 I | op-config: setting option "rgw_keystone_admin_password" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:45.738465 I | op-config: successfully set option "rgw_keystone_admin_password" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:45.738493 I | op-config: setting option "rgw_s3_auth_use_keystone" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:46.331510 I | op-config: successfully set option "rgw_s3_auth_use_keystone" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:46.331539 I | op-config: setting option "rgw_run_sync_thread" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:47.131854 I | op-config: successfully set option "rgw_run_sync_thread" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:47.131887 I | op-config: setting option "rgw_log_object_name_utc" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:47.861927 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:50:47.959073 I | op-config: successfully set option "rgw_log_object_name_utc" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:47.959141 I | op-config: setting option "rgw_enable_usage_log" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:48.352098 I | op-config: successfully set option "rgw_enable_usage_log" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:48.352139 I | op-config: setting option "rgw_zonegroup" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:48.742375 I | op-config: successfully set option "rgw_zonegroup" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:48.742415 I | op-config: setting option "rgw_log_nonexistent_bucket" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:49.118269 I | op-config: successfully set option "rgw_log_nonexistent_bucket" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:49.118368 I | op-config: setting option "rgw_keystone_accepted_roles" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:49.497125 I | op-config: successfully set option "rgw_keystone_accepted_roles" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:49.497238 I | op-config: setting option "rgw_keystone_admin_domain" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:49.873115 I | op-config: successfully set option "rgw_keystone_admin_domain" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:49.873150 I | op-config: setting option "rgw_keystone_admin_user" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:50.298471 I | op-config: successfully set option "rgw_keystone_admin_user" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:50.298509 I | op-config: setting option "rgw_keystone_implicit_tenants" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:50.687324 I | op-config: successfully set option "rgw_keystone_implicit_tenants" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:50.687354 I | op-config: setting option "rgw_keystone_api_version" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:51.071176 I | op-config: successfully set option "rgw_keystone_api_version" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:51.071210 I | op-config: setting option "rgw_swift_account_in_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:51.464696 I | op-config: successfully set option "rgw_swift_account_in_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:51.464735 I | op-config: setting option "rgw_swift_versioning_enabled" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:51.858990 I | op-config: successfully set option "rgw_swift_versioning_enabled" (user "client.rgw.ceph.a") to the mon configuration database
2026-03-20 05:50:51.859391 I | ceph-object-controller: object store "ceph" deployment "rook-ceph-rgw-ceph-a" created
2026-03-20 05:50:51.904324 I | ceph-object-controller: enabling rgw dashboard
2026-03-20 05:50:53.728894 I | ceph-object-controller: created object store "ceph" in namespace "openstack"
2026-03-20 05:50:53.729499 I | ceph-object-controller: setting the dashboard api secret key
2026-03-20 05:50:54.566514 I | ceph-object-controller: done setting the dashboard api secret key
2026-03-20 05:50:54.693682 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:50:56.538450 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:50:57.983329 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:51:27.019280 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:51:28.464952 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:51:33.817304 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:51:57.517071 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:51:58.930186 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:52:19.761483 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:52:27.984026 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:101} {StateName:peering Count:1}]"
2026-03-20 05:52:27.988628 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:52:29.390405 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:101} {StateName:peering Count:1}]"
2026-03-20 05:52:29.394837 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:52:58.495686 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:103} {StateName:creating+peering Count:3}]"
2026-03-20 05:52:58.501273 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:52:59.858262 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:103} {StateName:creating+peering Count:3}]"
2026-03-20 05:52:59.863136 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:53:05.751185 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:53:28.971616 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:125} {StateName:peering Count:1}]"
2026-03-20 05:53:28.976356 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:53:30.326746 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:53:51.691292 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:53:59.457621 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:54:00.798779 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:54:29.971087 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:54:31.273721 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:54:37.604615 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:55:00.472262 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:55:01.734491 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:55:23.536291 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:55:30.943672 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:55:32.203501 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:56:01.407519 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:56:02.670257 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:56:09.481486 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:56:31.895985 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:56:33.152526 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:56:55.459581 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:57:02.355200 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:57:03.626997 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:57:32.837428 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:57:34.082986 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:57:41.404400 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:58:03.316709 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:58:04.557440 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:58:27.343346 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:58:33.784360 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:58:35.028941 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:59:04.266227 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:59:05.501434 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:59:13.280949 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 05:59:34.744120 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:59:35.974213 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 05:59:59.248175 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 06:00:05.203487 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:00:06.455860 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:00:35.679293 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:00:36.941771 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:00:45.220012 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 06:01:06.170894 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:01:07.411414 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:01:31.181597 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 06:01:36.660192 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:01:37.906404 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:02:07.160569 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:02:08.385348 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:02:17.169678 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-03-20 06:02:37.644473 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-03-20 06:02:38.862647 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0