2026/05/28 07:11:53 maxprocs: Updating GOMAXPROCS=1: determined from CPU quota
2026-05-28 07:11:53.368362 I | rookcmd: starting Rook v1.15.9 with arguments '/usr/local/bin/rook ceph operator'
2026-05-28 07:11:53.368386 I | rookcmd: flag values: --enable-machine-disruption-budget=false, --help=false, --kubeconfig=, --log-level=INFO
2026-05-28 07:11:53.368389 I | cephcmd: starting Rook-Ceph operator
2026-05-28 07:11:53.469410 I | cephcmd: base ceph version inside the rook operator image is "ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable)"
2026-05-28 07:11:53.477572 I | op-k8sutil: operator setting "CSI_CEPHFS_ATTACH_REQUIRED" = "true"
2026-05-28 07:11:53.477616 I | op-k8sutil: operator setting "CSI_CEPHFS_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-cephfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-05-28 07:11:53.477629 I | op-k8sutil: operator setting "CSI_ENABLE_NFS_SNAPSHOTTER" = "true"
2026-05-28 07:11:53.477645 I | op-k8sutil: operator setting "CSI_NFS_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-nfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n"
2026-05-28 07:11:53.477652 I | op-k8sutil: operator setting "CSI_PROVISIONER_PRIORITY_CLASSNAME" = "system-cluster-critical"
2026-05-28 07:11:53.477659 I | op-k8sutil: operator setting "ROOK_CSIADDONS_IMAGE" = "quay.io/csiaddons/k8s-sidecar:v0.9.1"
2026-05-28 07:11:53.477665 I | op-k8sutil: operator setting "ROOK_CSI_CEPH_IMAGE" = "quay.io/cephcsi/cephcsi:v3.12.3"
2026-05-28 07:11:53.477671 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_RBD" = "false"
2026-05-28 07:11:53.477677 I | op-k8sutil: operator setting "ROOK_CSI_PROVISIONER_IMAGE" = "registry.k8s.io/sig-storage/csi-provisioner:v5.0.1"
2026-05-28 07:11:53.477684 I | op-k8sutil: operator setting "ROOK_CSI_REGISTRAR_IMAGE" = "registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.11.1"
2026-05-28 07:11:53.477691 I | op-k8sutil: operator setting "ROOK_CSI_RESIZER_IMAGE" = "registry.k8s.io/sig-storage/csi-resizer:v1.11.1"
2026-05-28 07:11:53.477697 I | op-k8sutil: operator setting "ROOK_CSI_SNAPSHOTTER_IMAGE" = "registry.k8s.io/sig-storage/csi-snapshotter:v8.0.1"
2026-05-28 07:11:53.477703 I | op-k8sutil: operator setting "ROOK_LOG_LEVEL" = "INFO"
2026-05-28 07:11:53.477709 I | op-k8sutil: operator setting "CSI_CEPHFS_FSGROUPPOLICY" = "File"
2026-05-28 07:11:53.477715 I | op-k8sutil: operator setting "CSI_DISABLE_HOLDER_PODS" = "true"
2026-05-28 07:11:53.477720 I | op-k8sutil: operator setting "CSI_ENABLE_CEPHFS_SNAPSHOTTER" = "true"
2026-05-28 07:11:53.477732 I | op-k8sutil: operator setting "CSI_RBD_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-rbdplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-05-28 07:11:53.477739 I | op-k8sutil: operator setting "ROOK_CSI_IMAGE_PULL_POLICY" = "IfNotPresent"
2026-05-28 07:11:53.477744 I | op-k8sutil: operator setting "CSI_ENABLE_METADATA" = "false"
2026-05-28 07:11:53.477750 I | op-k8sutil: operator setting "ROOK_CEPH_ALLOW_LOOP_DEVICES" = "false"
2026-05-28 07:11:53.477756 I | op-k8sutil: operator setting "ROOK_CSI_DISABLE_DRIVER" = "false"
2026-05-28 07:11:53.477761 I | op-k8sutil: operator setting "ROOK_OBC_WATCH_OPERATOR_NAMESPACE" = "true"
2026-05-28 07:11:53.477767 I | op-k8sutil: operator setting "CSI_PLUGIN_PRIORITY_CLASSNAME" = "system-node-critical"
2026-05-28 07:11:53.477773 I | op-k8sutil: operator setting "CSI_RBD_ATTACH_REQUIRED" = "true"
2026-05-28 07:11:53.477779 I | op-k8sutil: operator setting "ROOK_CEPH_COMMANDS_TIMEOUT_SECONDS" = "15"
2026-05-28 07:11:53.477786 I | op-k8sutil: operator setting "ROOK_CSI_ATTACHER_IMAGE" = "registry.k8s.io/sig-storage/csi-attacher:v4.6.1"
2026-05-28 07:11:53.477794 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_CEPHFS" = "false"
2026-05-28 07:11:53.477833 I | op-k8sutil: operator setting "CSI_CEPHFS_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-resizer\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-snapshotter\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-cephfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-05-28 07:11:53.477868 I | op-k8sutil: operator setting "CSI_ENABLE_CSIADDONS" = "false"
2026-05-28 07:11:53.477882 I | op-k8sutil: operator setting "CSI_ENABLE_TOPOLOGY" = "false"
2026-05-28 07:11:53.477889 I | op-k8sutil: operator setting "CSI_NFS_FSGROUPPOLICY" = "File"
2026-05-28 07:11:53.477914 I | op-k8sutil: operator setting "CSI_NFS_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-nfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n"
2026-05-28 07:11:53.477933 I | op-k8sutil: operator setting "CSI_PLUGIN_ENABLE_SELINUX_HOST_MOUNT" = "false"
2026-05-28 07:11:53.477943 I | op-k8sutil: operator setting "CSI_ENABLE_ENCRYPTION" = "false"
2026-05-28 07:11:53.477951 I | op-k8sutil: operator setting "CSI_ENABLE_HOST_NETWORK" = "true"
2026-05-28 07:11:53.477960 I | op-k8sutil: operator setting "CSI_ENABLE_RBD_SNAPSHOTTER" = "true"
2026-05-28 07:11:53.477966 I | op-k8sutil: operator setting "CSI_GRPC_TIMEOUT_SECONDS" = "150"
2026-05-28 07:11:53.477975 I | op-k8sutil: operator setting "CSI_PROVISIONER_REPLICAS" = "2"
2026-05-28 07:11:53.477990 I | op-k8sutil: operator setting "CSI_RBD_FSGROUPPOLICY" = "File"
2026-05-28 07:11:53.477998 I | op-k8sutil: operator setting "CSI_ENABLE_VOLUME_GROUP_SNAPSHOT" = "true"
2026-05-28 07:11:53.478034 I | op-k8sutil: operator setting "CSI_RBD_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-resizer\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-snapshotter\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-rbdplugin\n  resource:\n    requests:\n      memory: 512Mi\n    limits:\n      memory: 1Gi\n- name : csi-omap-generator\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-05-28 07:11:53.478054 I | op-k8sutil: operator setting "ROOK_ENABLE_DISCOVERY_DAEMON" = "false"
2026-05-28 07:11:53.478062 I | op-k8sutil: operator setting "CSI_ENABLE_OMAP_GENERATOR" = "false"
2026-05-28 07:11:53.478071 I | op-k8sutil: operator setting "CSI_FORCE_CEPHFS_KERNEL_CLIENT" = "true"
2026-05-28 07:11:53.478080 I | op-k8sutil: operator setting "CSI_NFS_ATTACH_REQUIRED" = "true"
2026-05-28 07:11:53.478088 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_NFS" = "false"
2026-05-28 07:11:53.478098 I | operator: watching all namespaces for Ceph CRs
2026-05-28 07:11:53.478196 I | operator: setting up schemes
2026-05-28 07:11:53.481891 I | operator: setting up the controller-runtime manager
2026-05-28 07:11:53.482551 I | ceph-cluster-controller: successfully started
2026-05-28 07:11:53.482910 I | ceph-cluster-controller: enabling hotplug orchestration
2026-05-28 07:11:53.482929 I | ceph-nodedaemon-controller: successfully started
2026-05-28 07:11:53.482941 I | ceph-block-pool-controller: successfully started
2026-05-28 07:11:53.482959 I | ceph-object-store-user-controller: successfully started
2026-05-28 07:11:53.482977 I | ceph-object-realm-controller: successfully started
2026-05-28 07:11:53.482984 I | ceph-object-zonegroup-controller: successfully started
2026-05-28 07:11:53.482997 I | ceph-object-zone-controller: successfully started
2026-05-28 07:11:53.483126 I | ceph-object-controller: successfully started
2026-05-28 07:11:53.483153 I | ceph-file-controller: successfully started
2026-05-28 07:11:53.483176 I | ceph-nfs-controller: successfully started
2026-05-28 07:11:53.483212 I | ceph-rbd-mirror-controller: successfully started
2026-05-28 07:11:53.483232 I | ceph-client-controller: successfully started
2026-05-28 07:11:53.483246 I | ceph-filesystem-mirror-controller: successfully started
2026-05-28 07:11:53.483273 I | operator: rook-ceph-operator-config-controller successfully started
2026-05-28 07:11:53.483284 I | ceph-csi: rook-ceph-operator-csi-controller successfully started
2026-05-28 07:11:53.483453 I | op-bucket-prov: rook-ceph-operator-bucket-controller successfully started
2026-05-28 07:11:53.483473 I | ceph-bucket-topic: successfully started
2026-05-28 07:11:53.483483 I | ceph-bucket-notification: successfully started
2026-05-28 07:11:53.483492 I | ceph-bucket-notification: successfully started
2026-05-28 07:11:53.483506 I | ceph-fs-subvolumegroup-controller: successfully started
2026-05-28 07:11:53.483567 I | blockpool-rados-namespace-controller: successfully started
2026-05-28 07:11:53.483630 I | ceph-cosi-controller: successfully started
2026-05-28 07:11:53.483648 I | operator: starting the controller-runtime manager
2026-05-28 07:11:53.710805 I | operator: rook-ceph-operator-config-controller done reconciling
2026-05-28 07:11:53.732628 I | ceph-csi: successfully created csi config map "rook-ceph-csi-config"
2026-05-28 07:11:53.735468 I | op-k8sutil: removing daemonset csi-rbdplugin if it exists
2026-05-28 07:11:53.737193 I | op-k8sutil: removing deployment csi-rbdplugin-provisioner if it exists
2026-05-28 07:11:53.743143 I | ceph-csi: successfully removed CSI Ceph RBD driver
2026-05-28 07:11:53.743163 I | op-k8sutil: removing daemonset csi-cephfsplugin if it exists
2026-05-28 07:11:53.745075 I | op-k8sutil: removing deployment csi-cephfsplugin-provisioner if it exists
2026-05-28 07:11:53.750724 I | ceph-csi: successfully removed CSI CephFS driver
2026-05-28 07:11:53.750739 I | op-k8sutil: removing daemonset csi-nfsplugin if it exists
2026-05-28 07:11:53.753894 I | op-k8sutil: removing deployment csi-nfsplugin-provisioner if it exists
2026-05-28 07:11:53.758736 I | ceph-csi: successfully removed CSI NFS driver
2026-05-28 07:12:15.479838 I | ceph-spec: adding finalizer "cephcluster.ceph.rook.io" on "ceph"
2026-05-28 07:12:15.485267 I | ceph-spec: adding finalizer "cephobjectstore.ceph.rook.io" on "ceph"
2026-05-28 07:12:15.488317 I | clusterdisruption-controller: deleted all legacy node drain canary pods
2026-05-28 07:12:15.521681 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-05-28 07:12:15.521705 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-05-28 07:12:15.521788 I | op-bucket-prov: ceph bucket provisioner launched watching for provisioner "openstack.ceph.rook.io/bucket"
2026-05-28 07:12:15.522166 I | op-bucket-prov: successfully reconciled bucket provisioner
I0528 07:12:15.522230       1 manager.go:135] "msg"="starting provisioner" "logger"="objectbucket.io/provisioner-manager" "name"="openstack.ceph.rook.io/bucket"
2026-05-28 07:12:15.522858 I | ceph-cluster-controller: reconciling ceph cluster in namespace "openstack"
2026-05-28 07:12:15.535189 E | ceph-spec: failed to update cluster condition to {Type:Connecting Status:True Reason:ClusterConnecting Message:Attempting to connect to an external Ceph cluster LastHeartbeatTime:2026-05-28 07:12:15.522915407 +0000 UTC m=+22.195281069 LastTransitionTime:2026-05-28 07:12:15.522915307 +0000 UTC m=+22.195280969}. failed to update object "openstack/ceph" status: Operation cannot be fulfilled on cephclusters.ceph.rook.io "ceph": the object has been modified; please apply your changes to the latest version and try again
2026-05-28 07:12:15.539064 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-05-28 07:12:15.539092 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-05-28 07:12:15.539114 I | ceph-spec: found the cluster info to connect to the external cluster. will use "client.admin" to check health and monitor status. mons=map[instance:0xc001adaed0]
2026-05-28 07:12:15.539136 I | ceph-spec: detecting the ceph image version for image harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7...
2026-05-28 07:12:16.487037 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-05-28 07:12:16.487078 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-05-28 07:12:16.891031 I | op-k8sutil: removing daemonset csi-rbdplugin if it exists
2026-05-28 07:12:16.892918 I | op-k8sutil: removing deployment csi-rbdplugin-provisioner if it exists
2026-05-28 07:12:17.088303 I | ceph-csi: successfully removed CSI Ceph RBD driver
2026-05-28 07:12:17.088458 I | op-k8sutil: removing daemonset csi-cephfsplugin if it exists
2026-05-28 07:12:17.093664 I | op-k8sutil: removing deployment csi-cephfsplugin-provisioner if it exists
2026-05-28 07:12:17.289299 I | ceph-csi: successfully removed CSI CephFS driver
2026-05-28 07:12:17.289319 I | op-k8sutil: removing daemonset csi-nfsplugin if it exists
2026-05-28 07:12:17.291611 I | op-k8sutil: removing deployment csi-nfsplugin-provisioner if it exists
2026-05-28 07:12:17.489328 I | ceph-csi: successfully removed CSI NFS driver
2026-05-28 07:12:31.115660 I | ceph-spec: detected ceph image version: "18.2.7-0 reef"
2026-05-28 07:12:31.115681 I | ceph-cluster-controller: validating ceph version from provided image
2026-05-28 07:12:31.125488 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-05-28 07:12:31.125513 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-05-28 07:12:31.128077 I | cephclient: writing config file /var/lib/rook/openstack/openstack.config
2026-05-28 07:12:31.128150 I | cephclient: generated admin config in /var/lib/rook/openstack
2026-05-28 07:12:31.634622 E | cephver: external cluster ceph version is a major version higher "18.2.8-0 reef" than the local cluster "0.0.0-0 <unknown version>", consider upgrading
2026-05-28 07:12:32.130908 W | ceph-cluster-controller: image spec version 18.2.7-0 reef is lower than the running cluster version 18.2.8-0 reef, downgrading is not supported
2026-05-28 07:12:32.590086 I | ceph-cluster-controller: upgrading ceph cluster to "18.2.7-0 reef"
2026-05-28 07:12:32.590123 I | ceph-cluster-controller: cluster "openstack": version "18.2.7-0 reef" detected for image "harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7"
2026-05-28 07:12:32.761886 I | ceph-cluster-controller: creating "rook-config-override" configmap
2026-05-28 07:12:32.764879 I | ceph-cluster-controller: creating "rook-ceph-config" secret
2026-05-28 07:12:32.812434 I | ceph-cluster-controller: external cluster identity established
2026-05-28 07:12:32.812492 I | cephclient: getting or creating ceph auth key "client.csi-rbd-provisioner"
2026-05-28 07:12:33.350553 I | cephclient: getting or creating ceph auth key "client.csi-rbd-node"
2026-05-28 07:12:33.823412 I | cephclient: getting or creating ceph auth key "client.csi-cephfs-provisioner"
2026-05-28 07:12:34.320064 I | cephclient: getting or creating ceph auth key "client.csi-cephfs-node"
2026-05-28 07:12:34.838518 I | ceph-csi: created kubernetes csi secrets for cluster "openstack"
2026-05-28 07:12:34.847332 I | ceph-cluster-controller: successfully updated csi config map
2026-05-28 07:12:34.847358 I | cephclient: getting or creating ceph auth key "client.crash"
2026-05-28 07:12:35.325942 I | ceph-nodedaemon-controller: created kubernetes crash collector secret for cluster "openstack"
2026-05-28 07:12:35.407370 I | ceph-cluster-controller: enabling ceph mon monitoring goroutine for cluster "openstack"
2026-05-28 07:12:35.407410 I | ceph-cluster-controller: ceph status check interval is 1m0s
2026-05-28 07:12:35.407415 I | ceph-cluster-controller: enabling ceph status monitoring goroutine for cluster "openstack"
2026-05-28 07:12:36.680452 I | clusterdisruption-controller: all PGs are active+clean. Restoring default OSD pdb settings
2026-05-28 07:12:36.680505 I | clusterdisruption-controller: creating the default pdb "rook-ceph-osd" with maxUnavailable=1 for all osd
2026-05-28 07:12:36.737009 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:12:37.363245 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:12:45.582337 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-05-28 07:12:45.582366 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-05-28 07:12:45.582378 I | ceph-spec: detecting the ceph image version for image harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7...
2026-05-28 07:12:47.547864 I | ceph-spec: detected ceph image version: "18.2.7-0 reef"
2026-05-28 07:12:48.797128 I | ceph-object-controller: reconciling object store deployments
2026-05-28 07:12:48.815107 I | ceph-object-controller: ceph object store gateway service running at 10.104.133.208
2026-05-28 07:12:48.815139 I | ceph-object-controller: reconciling object store pools
2026-05-28 07:12:52.113207 I | cephclient: reconciling replicated pool ceph.rgw.control succeeded
2026-05-28 07:12:52.113257 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.control"
2026-05-28 07:12:56.170500 I | cephclient: reconciling replicated pool ceph.rgw.meta succeeded
2026-05-28 07:12:56.170550 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.meta"
2026-05-28 07:13:00.239524 I | cephclient: reconciling replicated pool ceph.rgw.log succeeded
2026-05-28 07:13:00.239565 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.log"
2026-05-28 07:13:04.313474 I | cephclient: reconciling replicated pool ceph.rgw.buckets.index succeeded
2026-05-28 07:13:04.313510 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.buckets.index"
2026-05-28 07:13:07.624494 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:25} {StateName:creating+peering Count:8}]"
2026-05-28 07:13:07.630793 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:13:08.092604 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:25} {StateName:creating+peering Count:8}]"
2026-05-28 07:13:08.098556 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:13:09.397073 I | cephclient: reconciling replicated pool ceph.rgw.buckets.non-ec succeeded
2026-05-28 07:13:09.397099 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.buckets.non-ec"
2026-05-28 07:13:13.444748 I | cephclient: reconciling replicated pool ceph.rgw.otp succeeded
2026-05-28 07:13:13.444784 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.otp"
2026-05-28 07:13:17.515901 I | cephclient: reconciling replicated pool .rgw.root succeeded
2026-05-28 07:13:17.515927 I | cephclient: setting pool property "pg_num_min" to "8" on pool ".rgw.root"
2026-05-28 07:13:22.258286 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:13:22.612495 I | cephclient: reconciling replicated pool ceph.rgw.buckets.data succeeded
2026-05-28 07:13:22.612551 I | ceph-object-controller: configuring object store "ceph"
2026-05-28 07:13:23.326256 I | ceph-object-controller: Object store "ceph": realm=ceph, zonegroup=ceph, zone=ceph
2026-05-28 07:13:23.536954 I | ceph-object-controller: committing changes to RGW configuration period for CephObjectStore "openstack/ceph"
2026-05-28 07:13:24.011488 I | ceph-object-controller: configuration for object-store ceph is complete
2026-05-28 07:13:24.015094 I | ceph-object-controller: creating object store "ceph" in namespace "openstack"
2026-05-28 07:13:24.018996 I | cephclient: getting or creating ceph auth key "client.rgw.ceph.a"
2026-05-28 07:13:24.521517 I | ceph-object-controller: setting rgw config flags
2026-05-28 07:13:24.521547 I | ceph-object-controller: Configuring authentication with keystone
2026-05-28 07:13:24.521571 I | op-config: setting option "rgw_zone" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:24.919236 I | op-config: successfully set option "rgw_zone" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:24.919265 I | op-config: setting option "rgw_keystone_accepted_roles" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:25.333006 I | op-config: successfully set option "rgw_keystone_accepted_roles" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:25.333046 I | op-config: setting option "rgw_log_nonexistent_bucket" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:25.746589 I | op-config: successfully set option "rgw_log_nonexistent_bucket" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:25.746620 I | op-config: setting option "rgw_keystone_admin_domain" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:26.164884 I | op-config: successfully set option "rgw_keystone_admin_domain" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:26.164927 I | op-config: setting option "rgw_keystone_admin_project" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:26.595362 I | op-config: successfully set option "rgw_keystone_admin_project" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:26.595400 I | op-config: setting option "rgw_swift_versioning_enabled" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:26.994411 I | op-config: successfully set option "rgw_swift_versioning_enabled" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:26.994441 I | op-config: setting option "rgw_keystone_admin_password" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:27.411668 I | op-config: successfully set option "rgw_keystone_admin_password" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:27.411710 I | op-config: setting option "rgw_s3_auth_use_keystone" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:27.807456 I | op-config: successfully set option "rgw_s3_auth_use_keystone" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:27.807510 I | op-config: setting option "rgw_swift_account_in_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:28.196769 I | op-config: successfully set option "rgw_swift_account_in_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:28.196810 I | op-config: setting option "rgw_log_object_name_utc" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:28.576087 I | op-config: successfully set option "rgw_log_object_name_utc" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:28.576125 I | op-config: setting option "rgw_enable_usage_log" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:28.963782 I | op-config: successfully set option "rgw_enable_usage_log" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:28.963819 I | op-config: setting option "rgw_keystone_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:29.349884 I | op-config: successfully set option "rgw_keystone_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:29.349912 I | op-config: setting option "rgw_keystone_implicit_tenants" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:29.754443 I | op-config: successfully set option "rgw_keystone_implicit_tenants" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:29.754479 I | op-config: setting option "rgw_keystone_admin_user" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:30.171918 I | op-config: successfully set option "rgw_keystone_admin_user" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:30.171959 I | op-config: setting option "rgw_run_sync_thread" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:30.570790 I | op-config: successfully set option "rgw_run_sync_thread" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:30.570825 I | op-config: setting option "rgw_zonegroup" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:30.967142 I | op-config: successfully set option "rgw_zonegroup" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:30.967180 I | op-config: setting option "rgw_keystone_api_version" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:31.338986 I | op-config: successfully set option "rgw_keystone_api_version" (user "client.rgw.ceph.a") to the mon configuration database
2026-05-28 07:13:31.339308 I | ceph-object-controller: object store "ceph" deployment "rook-ceph-rgw-ceph-a" created
2026-05-28 07:13:31.392185 I | ceph-object-controller: enabling rgw dashboard
2026-05-28 07:13:33.219472 I | ceph-object-controller: created object store "ceph" in namespace "openstack"
2026-05-28 07:13:33.220393 I | ceph-object-controller: setting the dashboard api secret key
2026-05-28 07:13:34.083163 I | ceph-object-controller: done setting the dashboard api secret key
2026-05-28 07:13:34.163984 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:13:38.530086 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:13:39.003300 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:14:08.192756 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:14:08.993670 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:14:09.483120 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:14:39.896860 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:108} {StateName:peering Count:2}]"
2026-05-28 07:14:39.901507 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:14:40.363777 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:108} {StateName:peering Count:2}]"
2026-05-28 07:14:40.370102 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:14:54.131077 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:15:10.389306 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:15:10.872020 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:105} {StateName:peering Count:1}]"
2026-05-28 07:15:10.876317 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:15:40.356982 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:15:41.361567 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:15:42.063320 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:16:11.828742 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:16:12.570112 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:16:26.291447 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:16:42.788404 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:16:43.722454 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:17:12.205144 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:17:13.326090 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:17:14.182628 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:17:44.056182 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:17:45.184890 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:17:58.146158 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:18:14.523266 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:18:15.645732 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:18:44.104713 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:18:44.988727 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:18:46.566821 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:19:15.522043 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:19:17.033729 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:19:30.063554 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:19:45.987674 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:19:47.983293 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:20:16.000649 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:20:16.474717 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:20:18.446938 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:20:46.948044 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:20:49.464807 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:21:01.923562 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:21:17.440898 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:21:19.941798 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:21:48.363855 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:21:48.398208 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:21:50.957287 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:22:18.869702 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:22:21.447181 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:22:34.284318 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:22:49.340050 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:22:52.369669 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:23:20.277753 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:23:20.717755 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:23:22.825896 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:23:50.757922 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:23:53.759134 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:24:06.634946 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:24:21.225435 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:24:24.292698 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:24:51.692006 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:24:52.642504 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:24:55.254350 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:25:22.185319 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:25:25.743694 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:25:38.593954 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:25:52.640079 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:25:56.672522 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:26:23.111622 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:26:24.643791 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:26:27.139686 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:26:53.575098 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:26:58.041280 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:27:10.599994 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:27:24.037596 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:27:28.506423 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:27:54.497784 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:27:56.579631 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:27:59.274621 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:28:24.964197 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:28:29.795810 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:28:42.514824 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:28:55.439431 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:29:00.577691 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:29:25.926547 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:29:28.503319 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:29:31.067704 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:29:56.401480 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:30:01.783181 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:30:14.457737 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:30:26.872687 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:30:32.273909 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:30:57.371849 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:31:00.394254 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:31:02.961323 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:31:27.857931 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:31:33.467634 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:31:46.316571 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-05-28 07:31:58.326031 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-05-28 07:32:04.080235 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0