I0407 09:39:35.856383 1 options.go:220] external host was not specified, using 199.204.45.240 I0407 09:39:35.857276 1 server.go:148] Version: v1.28.13 I0407 09:39:35.857324 1 server.go:150] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" I0407 09:39:36.181034 1 shared_informer.go:311] Waiting for caches to sync for node_authorizer I0407 09:39:36.189896 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. I0407 09:39:36.189933 1 plugins.go:161] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota. I0407 09:39:36.190288 1 instance.go:298] Using reconciler: lease I0407 09:39:36.868395 1 handler.go:275] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager W0407 09:39:36.868436 1 genericapiserver.go:744] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources. I0407 09:39:37.139905 1 handler.go:275] Adding GroupVersion v1 to ResourceManager I0407 09:39:37.140076 1 instance.go:709] API group "internal.apiserver.k8s.io" is not enabled, skipping. I0407 09:39:37.613270 1 instance.go:709] API group "resource.k8s.io" is not enabled, skipping. I0407 09:39:37.623691 1 handler.go:275] Adding GroupVersion authentication.k8s.io v1 to ResourceManager W0407 09:39:37.623810 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1beta1 because it has no resources. W0407 09:39:37.623836 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1alpha1 because it has no resources. I0407 09:39:37.624257 1 handler.go:275] Adding GroupVersion authorization.k8s.io v1 to ResourceManager W0407 09:39:37.624302 1 genericapiserver.go:744] Skipping API authorization.k8s.io/v1beta1 because it has no resources. I0407 09:39:37.625099 1 handler.go:275] Adding GroupVersion autoscaling v2 to ResourceManager I0407 09:39:37.625972 1 handler.go:275] Adding GroupVersion autoscaling v1 to ResourceManager W0407 09:39:37.626018 1 genericapiserver.go:744] Skipping API autoscaling/v2beta1 because it has no resources. W0407 09:39:37.626039 1 genericapiserver.go:744] Skipping API autoscaling/v2beta2 because it has no resources. I0407 09:39:37.627130 1 handler.go:275] Adding GroupVersion batch v1 to ResourceManager W0407 09:39:37.627178 1 genericapiserver.go:744] Skipping API batch/v1beta1 because it has no resources. I0407 09:39:37.627912 1 handler.go:275] Adding GroupVersion certificates.k8s.io v1 to ResourceManager W0407 09:39:37.627958 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1beta1 because it has no resources. W0407 09:39:37.627978 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1alpha1 because it has no resources. I0407 09:39:37.628500 1 handler.go:275] Adding GroupVersion coordination.k8s.io v1 to ResourceManager W0407 09:39:37.628548 1 genericapiserver.go:744] Skipping API coordination.k8s.io/v1beta1 because it has no resources. W0407 09:39:37.628613 1 genericapiserver.go:744] Skipping API discovery.k8s.io/v1beta1 because it has no resources. I0407 09:39:37.635459 1 handler.go:275] Adding GroupVersion discovery.k8s.io v1 to ResourceManager I0407 09:39:37.641719 1 handler.go:275] Adding GroupVersion networking.k8s.io v1 to ResourceManager W0407 09:39:37.641760 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1beta1 because it has no resources. W0407 09:39:37.641772 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1alpha1 because it has no resources. I0407 09:39:37.642554 1 handler.go:275] Adding GroupVersion node.k8s.io v1 to ResourceManager W0407 09:39:37.642579 1 genericapiserver.go:744] Skipping API node.k8s.io/v1beta1 because it has no resources. W0407 09:39:37.642586 1 genericapiserver.go:744] Skipping API node.k8s.io/v1alpha1 because it has no resources. I0407 09:39:37.643840 1 handler.go:275] Adding GroupVersion policy v1 to ResourceManager W0407 09:39:37.643863 1 genericapiserver.go:744] Skipping API policy/v1beta1 because it has no resources. I0407 09:39:37.646723 1 handler.go:275] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager W0407 09:39:37.646750 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources. W0407 09:39:37.646758 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. I0407 09:39:37.647445 1 handler.go:275] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager W0407 09:39:37.647469 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1beta1 because it has no resources. W0407 09:39:37.647476 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. I0407 09:39:37.649991 1 handler.go:275] Adding GroupVersion storage.k8s.io v1 to ResourceManager W0407 09:39:37.650008 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1beta1 because it has no resources. W0407 09:39:37.650013 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1alpha1 because it has no resources. I0407 09:39:37.651308 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager I0407 09:39:37.652566 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta2 to ResourceManager W0407 09:39:37.652580 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources. W0407 09:39:37.652585 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources. I0407 09:39:37.656311 1 handler.go:275] Adding GroupVersion apps v1 to ResourceManager W0407 09:39:37.656327 1 genericapiserver.go:744] Skipping API apps/v1beta2 because it has no resources. W0407 09:39:37.656331 1 genericapiserver.go:744] Skipping API apps/v1beta1 because it has no resources. I0407 09:39:37.657195 1 handler.go:275] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager W0407 09:39:37.657210 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources. W0407 09:39:37.657214 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources. I0407 09:39:37.657726 1 handler.go:275] Adding GroupVersion events.k8s.io v1 to ResourceManager W0407 09:39:37.657738 1 genericapiserver.go:744] Skipping API events.k8s.io/v1beta1 because it has no resources. I0407 09:39:37.689569 1 handler.go:275] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager W0407 09:39:37.689699 1 genericapiserver.go:744] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources. I0407 09:39:38.126974 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0407 09:39:38.127004 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0407 09:39:38.127067 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/etc/kubernetes/pki/apiserver.crt::/etc/kubernetes/pki/apiserver.key" I0407 09:39:38.127197 1 secure_serving.go:213] Serving securely on [::]:6443 I0407 09:39:38.127228 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" I0407 09:39:38.127275 1 controller.go:80] Starting OpenAPI V3 AggregationController I0407 09:39:38.127316 1 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/etc/kubernetes/pki/front-proxy-client.crt::/etc/kubernetes/pki/front-proxy-client.key" I0407 09:39:38.127329 1 system_namespaces_controller.go:67] Starting system namespaces controller I0407 09:39:38.127337 1 available_controller.go:423] Starting AvailableConditionController I0407 09:39:38.127349 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller I0407 09:39:38.127364 1 aggregator.go:164] waiting for initial CRD sync... I0407 09:39:38.127378 1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller I0407 09:39:38.127389 1 shared_informer.go:311] Waiting for caches to sync for cluster_authentication_trust_controller I0407 09:39:38.127409 1 apiservice_controller.go:97] Starting APIServiceRegistrationController I0407 09:39:38.127415 1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller I0407 09:39:38.127430 1 crdregistration_controller.go:111] Starting crd-autoregister controller I0407 09:39:38.127440 1 apf_controller.go:374] Starting API Priority and Fairness config controller I0407 09:39:38.127441 1 shared_informer.go:311] Waiting for caches to sync for crd-autoregister I0407 09:39:38.127480 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0407 09:39:38.127563 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0407 09:39:38.127687 1 gc_controller.go:78] Starting apiserver lease garbage collector I0407 09:39:38.128244 1 customresource_discovery_controller.go:289] Starting DiscoveryController I0407 09:39:38.128241 1 controller.go:78] Starting OpenAPI AggregationController I0407 09:39:38.128351 1 handler_discovery.go:412] Starting ResourceDiscoveryManager I0407 09:39:38.128377 1 controller.go:134] Starting OpenAPI controller I0407 09:39:38.128264 1 controller.go:116] Starting legacy_token_tracking_controller I0407 09:39:38.128429 1 controller.go:85] Starting OpenAPI V3 controller I0407 09:39:38.128433 1 shared_informer.go:311] Waiting for caches to sync for configmaps I0407 09:39:38.128483 1 naming_controller.go:291] Starting NamingConditionController I0407 09:39:38.128519 1 establishing_controller.go:76] Starting EstablishingController I0407 09:39:38.128547 1 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController I0407 09:39:38.128575 1 crd_finalizer.go:266] Starting CRDFinalizer I0407 09:39:38.128578 1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController I0407 09:39:38.128766 1 gc_controller.go:78] Starting apiserver lease garbage collector I0407 09:39:38.182032 1 shared_informer.go:318] Caches are synced for node_authorizer E0407 09:39:38.198070 1 controller.go:146] "Failed to ensure lease exists, will retry" err="namespaces \"kube-system\" not found" interval="200ms" I0407 09:39:38.227603 1 shared_informer.go:318] Caches are synced for crd-autoregister I0407 09:39:38.227678 1 aggregator.go:166] initial CRD sync complete... I0407 09:39:38.227695 1 autoregister_controller.go:141] Starting autoregister controller I0407 09:39:38.227708 1 cache.go:32] Waiting for caches to sync for autoregister controller I0407 09:39:38.227729 1 cache.go:39] Caches are synced for autoregister controller I0407 09:39:38.227834 1 apf_controller.go:379] Running API Priority and Fairness config worker I0407 09:39:38.227894 1 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process I0407 09:39:38.228013 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller I0407 09:39:38.228067 1 shared_informer.go:318] Caches are synced for cluster_authentication_trust_controller I0407 09:39:38.228268 1 cache.go:39] Caches are synced for AvailableConditionController controller I0407 09:39:38.228546 1 shared_informer.go:318] Caches are synced for configmaps I0407 09:39:38.228625 1 controller.go:624] quota admission added evaluator for: namespaces I0407 09:39:38.401268 1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io I0407 09:39:39.148936 1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000 I0407 09:39:39.154415 1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000 I0407 09:39:39.154435 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist. I0407 09:39:39.905246 1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io I0407 09:39:39.962065 1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io I0407 09:39:40.049287 1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"} W0407 09:39:40.059611 1 lease.go:263] Resetting endpoints for master service "kubernetes" to [199.204.45.240] I0407 09:39:40.061000 1 controller.go:624] quota admission added evaluator for: endpoints I0407 09:39:40.066642 1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io I0407 09:39:41.814193 1 controller.go:624] quota admission added evaluator for: serviceaccounts I0407 09:39:41.971536 1 controller.go:624] quota admission added evaluator for: deployments.apps I0407 09:39:41.986389 1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"} I0407 09:39:42.201991 1 controller.go:624] quota admission added evaluator for: daemonsets.apps I0407 09:39:46.960152 1 controller.go:624] quota admission added evaluator for: replicasets.apps I0407 09:39:47.058690 1 controller.go:624] quota admission added evaluator for: controllerrevisions.apps I0407 09:40:08.973585 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0407 09:40:09.006406 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0407 09:40:09.019863 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0407 09:40:09.041350 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0407 09:40:09.056918 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0407 09:40:09.072435 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0407 09:40:09.102251 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0407 09:40:09.102419 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0407 09:40:09.144942 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0407 09:40:09.145075 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0407 09:40:09.210023 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0407 09:40:09.210090 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0407 09:40:09.257896 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0407 09:40:09.264996 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0407 09:40:09.265051 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0407 09:40:09.421162 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0407 09:40:09.421226 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0407 09:40:09.433428 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:40:09.484933 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:40:09.529317 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:40:09.563229 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:40:09.578953 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:40:10.482347 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:40:10.615016 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:40:16.232460 1 controller.go:624] quota admission added evaluator for: jobs.batch I0407 09:40:17.415362 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0407 09:40:17.455322 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0407 09:40:17.477607 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0407 09:40:17.603914 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0407 09:40:17.723264 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0407 09:40:17.961440 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0407 09:40:18.135771 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0407 09:40:18.316660 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0407 09:40:18.543644 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0407 09:40:20.722807 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0407 09:40:21.152006 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager W0407 09:40:22.010252 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found E0407 09:40:22.010327 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found I0407 09:40:25.015040 1 controller.go:624] quota admission added evaluator for: ciliumendpoints.cilium.io I0407 09:40:33.186807 1 alloc.go:330] "allocated clusterIPs" service="envoy-gateway-system/envoy-gateway" clusterIPs={"IPv4":"10.104.129.76"} W0407 09:40:33.232494 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.104.129.76:9443: connect: connection refused E0407 09:40:33.232539 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.104.129.76:9443: connect: connection refused I0407 09:40:46.026142 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0407 09:40:46.041687 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0407 09:40:46.052002 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0407 09:40:46.073774 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager" clusterIPs={"IPv4":"10.101.254.66"} I0407 09:40:46.084330 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager-webhook" clusterIPs={"IPv4":"10.100.233.177"} I0407 09:40:46.086919 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0407 09:40:46.131463 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0407 09:40:46.175834 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0407 09:40:57.586755 1 controller.go:624] quota admission added evaluator for: certificates.cert-manager.io I0407 09:41:03.375556 1 controller.go:624] quota admission added evaluator for: certificaterequests.cert-manager.io I0407 09:41:17.682244 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-admission" clusterIPs={"IPv4":"10.102.63.253"} I0407 09:41:17.691068 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-metrics" clusterIPs={"IPv4":"10.111.167.145"} I0407 09:41:17.697538 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-defaultbackend" clusterIPs={"IPv4":"10.106.226.13"} I0407 09:41:17.709130 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller" clusterIPs={"IPv4":"10.101.112.68"} I0407 09:41:24.254985 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.273228 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.299495 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.323681 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.343121 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.360564 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.378287 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.395861 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.422241 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.436366 1 handler.go:275] Adding GroupVersion rabbitmq.com v1alpha1 to ResourceManager I0407 09:41:24.454203 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.474199 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:24.490945 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:25.828594 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:41:29.237309 1 controller.go:624] quota admission added evaluator for: networkpolicies.networking.k8s.io I0407 09:41:29.244188 1 controller.go:624] quota admission added evaluator for: poddisruptionbudgets.policy I0407 09:41:29.296474 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-messaging-topology-operator-webhook" clusterIPs={"IPv4":"10.104.210.35"} I0407 09:41:29.328212 1 controller.go:624] quota admission added evaluator for: issuers.cert-manager.io I0407 09:41:31.667754 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0407 09:41:31.698655 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0407 09:41:32.288963 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-10-0 to ResourceManager I0407 09:41:32.289012 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-11-0 to ResourceManager I0407 09:41:32.289053 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0407 09:41:38.813447 1 controller.go:624] quota admission added evaluator for: perconaxtradbclusters.pxc.percona.com I0407 09:41:41.405368 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-cluster-operator" clusterIPs={"IPv4":"10.98.131.33"} I0407 09:41:41.789856 1 controller.go:624] quota admission added evaluator for: statefulsets.apps I0407 09:41:42.050176 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy" clusterIPs={"IPv4":"10.106.170.199"} I0407 09:41:42.065623 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-replicas" clusterIPs={"IPv4":"10.104.41.113"} I0407 09:43:17.690887 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-metrics" clusterIPs={"IPv4":"10.97.92.175"} I0407 09:43:20.197684 1 alloc.go:330] "allocated clusterIPs" service="openstack/valkey" clusterIPs={"IPv4":"10.106.22.183"} I0407 09:43:20.204224 1 alloc.go:330] "allocated clusterIPs" service="openstack/valkey-metrics" clusterIPs={"IPv4":"10.108.59.51"} I0407 09:43:26.795713 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak" clusterIPs={"IPv4":"10.106.100.57"} I0407 09:43:26.808936 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak-metrics" clusterIPs={"IPv4":"10.104.147.82"} I0407 09:46:00.637698 1 controller.go:624] quota admission added evaluator for: ingresses.networking.k8s.io I0407 09:46:04.550034 1 handler.go:275] Adding GroupVersion nfd.k8s-sigs.io v1alpha1 to ResourceManager I0407 09:46:04.583329 1 handler.go:275] Adding GroupVersion nfd.k8s-sigs.io v1alpha1 to ResourceManager I0407 09:46:08.457224 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0407 09:46:08.463845 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0407 09:46:08.523324 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0407 09:46:08.535842 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0407 09:46:08.544076 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0407 09:46:08.554973 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0407 09:46:08.568406 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0407 09:46:10.114653 1 controller.go:624] quota admission added evaluator for: nodefeatures.nfd.k8s-sigs.io I0407 09:46:21.446316 1 controller.go:624] quota admission added evaluator for: passwords.secretgen.k14s.io I0407 09:47:00.879920 1 controller.go:624] quota admission added evaluator for: secrettemplates.secretgen.carvel.dev I0407 09:47:15.175580 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0407 09:47:15.328842 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:47:15.366640 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:47:15.367125 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:47:15.623297 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0407 09:47:15.891227 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:47:15.898665 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:47:16.047007 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0407 09:47:16.086873 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:47:16.189391 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:47:51.953305 1 trace.go:236] Trace[17816497]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:03b3bf3e-e7ee-4c1a-a410-3865811d6be5,client:10.0.0.235,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:47:51.147) (total time: 806ms): Trace[17816497]: ["GuaranteedUpdate etcd3" audit-id:03b3bf3e-e7ee-4c1a-a410-3865811d6be5,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 805ms (09:47:51.147) Trace[17816497]: ---"Txn call completed" 804ms (09:47:51.953)] Trace[17816497]: [806.130449ms] [806.130449ms] END I0407 09:47:58.121159 1 trace.go:236] Trace[445917008]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4513e3bf-fa70-4ab9-9b98-aebbfc429e17,client:10.0.0.191,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:47:53.121) (total time: 4999ms): Trace[445917008]: ["GuaranteedUpdate etcd3" audit-id:4513e3bf-fa70-4ab9-9b98-aebbfc429e17,key:/leases/openstack/rabbitmq-cluster-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 4999ms (09:47:53.121) Trace[445917008]: ---"Txn call failed" err:context canceled 4998ms (09:47:58.120)] Trace[445917008]: [4.999774498s] [4.999774498s] END E0407 09:47:58.121179 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 12.67µs, panicked: false, err: context canceled, panic-reason: E0407 09:47:58.121170 1 wrap.go:54] timeout or abort while handling: method=PUT URI="/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election?timeout=5s" audit-ID="4513e3bf-fa70-4ab9-9b98-aebbfc429e17" E0407 09:47:58.121428 1 timeout.go:142] post-timeout activity - time-elapsed: 219.455µs, PUT "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election" result: E0407 09:47:58.956746 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 20.852µs, panicked: false, err: context canceled, panic-reason: E0407 09:47:58.956893 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:47:58.958025 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:47:58.958101 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:47:58.959393 1 trace.go:236] Trace[511688435]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:a03693c0-6550-4aa3-8a6d-3e23dcbef07d,client:10.0.0.235,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:47:53.958) (total time: 5001ms): Trace[511688435]: ["GuaranteedUpdate etcd3" audit-id:a03693c0-6550-4aa3-8a6d-3e23dcbef07d,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (09:47:53.958) Trace[511688435]: ---"Txn call failed" err:context canceled 4997ms (09:47:58.956)] Trace[511688435]: [5.001271115s] [5.001271115s] END E0407 09:47:58.959661 1 timeout.go:142] post-timeout activity - time-elapsed: 2.751606ms, PUT "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0407 09:47:59.045923 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0407 09:47:59.046055 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:47:59.046118 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:47:59.046166 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:47:59.047879 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:47:59.047953 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:47:59.047977 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:47:59.047978 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:47:59.049060 1 trace.go:236] Trace[1612132928]: "Get" accept:application/json, */*,audit-id:dda8594f-8cda-453a-bcd3-0e2b91ce8297,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:54.046) (total time: 5002ms): Trace[1612132928]: [5.002417881s] [5.002417881s] END E0407 09:47:59.049087 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:47:59.049136 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0407 09:47:59.049168 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:47:59.049214 1 timeout.go:142] post-timeout activity - time-elapsed: 3.086026ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0407 09:47:59.050473 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:47:59.050479 1 trace.go:236] Trace[321146426]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:85fd6ac8-5e39-4302-9896-d52581208200,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:47:54.045) (total time: 5004ms): Trace[321146426]: [5.004563055s] [5.004563055s] END I0407 09:47:59.050570 1 trace.go:236] Trace[193546117]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:460cf7f2-8acd-4dc4-9158-2e59fc84a218,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:47:54.046) (total time: 5003ms): Trace[193546117]: [5.003873227s] [5.003873227s] END E0407 09:47:59.050636 1 timeout.go:142] post-timeout activity - time-elapsed: 4.723884ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0407 09:47:59.050789 1 timeout.go:142] post-timeout activity - time-elapsed: 4.750045ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0407 09:47:59.398158 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:47:59.398357 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:47:59.398235 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 148.443µs, panicked: false, err: context canceled, panic-reason: E0407 09:47:59.399700 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:47:59.399833 1 trace.go:236] Trace[1481203566]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e9e1a675-38d9-4056-a5cf-58067735ec7b,client:10.0.0.31,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:47:54.398) (total time: 5001ms): Trace[1481203566]: ["GuaranteedUpdate etcd3" audit-id:e9e1a675-38d9-4056-a5cf-58067735ec7b,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (09:47:54.398) Trace[1481203566]: ---"Txn call failed" err:context canceled 4998ms (09:47:59.398)] Trace[1481203566]: [5.001661344s] [5.001661344s] END E0407 09:47:59.400040 1 timeout.go:142] post-timeout activity - time-elapsed: 1.851935ms, PUT "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0407 09:47:59.730735 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:47:59.730785 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 13.73µs, panicked: false, err: context deadline exceeded, panic-reason: E0407 09:47:59.730802 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:47:59.732239 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:47:59.732333 1 trace.go:236] Trace[39190864]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:42137fed-8413-4e9e-927a-64e1db80cda0,client:10.0.0.213,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:47:54.731) (total time: 5000ms): Trace[39190864]: ["GuaranteedUpdate etcd3" audit-id:42137fed-8413-4e9e-927a-64e1db80cda0,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 5000ms (09:47:54.731) Trace[39190864]: ---"Txn call failed" err:context deadline exceeded 4998ms (09:47:59.730)] Trace[39190864]: [5.000962005s] [5.000962005s] END E0407 09:47:59.732480 1 timeout.go:142] post-timeout activity - time-elapsed: 1.765182ms, PUT "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election" result: E0407 09:47:59.760854 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:47:59.760999 1 trace.go:236] Trace[906561560]: "Update" accept:application/json, */*,audit-id:064ef5f7-887b-4689-b4e7-410aff57adfb,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (07-Apr-2026 09:47:52.757) (total time: 7003ms): Trace[906561560]: ["GuaranteedUpdate etcd3" audit-id:064ef5f7-887b-4689-b4e7-410aff57adfb,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (09:47:52.758) Trace[906561560]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (09:47:59.760)] Trace[906561560]: [7.003136711s] [7.003136711s] END E0407 09:48:01.345633 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:48:01.345837 1 trace.go:236] Trace[430834967]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:588b0ac5-9c9a-4145-beb0-80da282d65ad,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (07-Apr-2026 09:47:54.341) (total time: 7003ms): Trace[430834967]: ["GuaranteedUpdate etcd3" audit-id:588b0ac5-9c9a-4145-beb0-80da282d65ad,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (09:47:54.342) Trace[430834967]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (09:48:01.345)] Trace[430834967]: [7.0038808s] [7.0038808s] END E0407 09:48:02.754046 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:02.754163 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:02.755301 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:02.755348 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:02.756425 1 trace.go:236] Trace[1193552309]: "Get" accept:application/json, */*,audit-id:0f668e8c-ba20-45cf-9102-bcde9bdd36a1,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:48:01.763) (total time: 992ms): Trace[1193552309]: [992.618842ms] [992.618842ms] END E0407 09:48:02.756585 1 timeout.go:142] post-timeout activity - time-elapsed: 2.545831ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock" result: E0407 09:48:03.121600 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:03.121729 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:03.123696 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:03.123773 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:03.124954 1 trace.go:236] Trace[1116086467]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:d67e910a-be6f-4ceb-93bb-323695974efe,client:10.0.0.191,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:58.122) (total time: 5002ms): Trace[1116086467]: [5.002294238s] [5.002294238s] END E0407 09:48:03.125116 1 timeout.go:142] post-timeout activity - time-elapsed: 4.000056ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election" result: E0407 09:48:03.956342 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:03.956537 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:03.957718 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:03.957767 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:03.958924 1 trace.go:236] Trace[1473094268]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:7586d82f-29b3-4ffb-963d-77479d6a7921,client:10.0.0.235,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:58.957) (total time: 5001ms): Trace[1473094268]: [5.001177722s] [5.001177722s] END E0407 09:48:03.959172 1 timeout.go:142] post-timeout activity - time-elapsed: 2.957471ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0407 09:48:03.972794 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:48:03.973037 1 trace.go:236] Trace[1937440403]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:28002308-dd93-4e9b-941c-b68a5a55a359,client:199.204.45.240,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (07-Apr-2026 09:47:56.969) (total time: 7003ms): Trace[1937440403]: ["Create etcd3" audit-id:28002308-dd93-4e9b-941c-b68a5a55a359,key:/events/kube-system/kube-apiserver-instance.18a40a1dfab9a2e5,type:*core.Event,resource:events 7002ms (09:47:56.970) Trace[1937440403]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (09:48:03.972)] Trace[1937440403]: [7.003399929s] [7.003399929s] END E0407 09:48:03.982628 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:03.982708 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:03.984641 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:03.984722 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:03.985860 1 trace.go:236] Trace[1510653725]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:b0c6984d-18a2-483a-964c-54eed8a9232e,client:10.0.0.235,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:47:57.133) (total time: 6852ms): Trace[1510653725]: [6.852270387s] [6.852270387s] END E0407 09:48:03.986101 1 timeout.go:142] post-timeout activity - time-elapsed: 3.126975ms, GET "/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0" result: E0407 09:48:04.045617 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:04.045746 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:04.046507 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:04.046676 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:04.046701 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:04.046914 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:04.047806 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0407 09:48:04.047852 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:04.047864 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:04.047892 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout I0407 09:48:04.048096 1 trace.go:236] Trace[726216206]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:8c16a580-e7a4-4880-b50a-545dedf36cea,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:48:01.047) (total time: 3000ms): Trace[726216206]: [3.000271159s] [3.000271159s] END E0407 09:48:04.048210 1 timeout.go:142] post-timeout activity - time-elapsed: 2.608103ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0407 09:48:04.049295 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:04.049333 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:04.049372 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0407 09:48:04.049405 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:04.050552 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0407 09:48:04.050595 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:04.050616 1 trace.go:236] Trace[1768553856]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:b2e8f522-efa6-44af-93f3-2b1194ae461f,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:48:01.048) (total time: 3002ms): Trace[1768553856]: [3.002273127s] [3.002273127s] END I0407 09:48:04.050625 1 trace.go:236] Trace[2060045192]: "Get" accept:application/json, */*,audit-id:62e97cd3-2491-4716-b3b1-5aed33dc17cd,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:48:01.048) (total time: 3002ms): Trace[2060045192]: [3.002087473s] [3.002087473s] END E0407 09:48:04.050716 1 timeout.go:142] post-timeout activity - time-elapsed: 3.989847ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0407 09:48:04.050719 1 timeout.go:142] post-timeout activity - time-elapsed: 4.425297ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: I0407 09:48:04.051848 1 trace.go:236] Trace[1375309925]: "Get" accept:application/json, */*,audit-id:351badc4-bfac-436f-8737-ae1386aeae33,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:47:54.046) (total time: 10004ms): Trace[1375309925]: [10.004840564s] [10.004840564s] END E0407 09:48:04.051980 1 timeout.go:142] post-timeout activity - time-elapsed: 4.122499ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock" result: E0407 09:48:04.397283 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:04.397511 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:04.398449 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:04.398716 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:04.399956 1 trace.go:236] Trace[2025567980]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:a825cfdb-85d4-417f-9ac1-61f663b1f7a7,client:10.0.0.31,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:59.399) (total time: 5000ms): Trace[2025567980]: [5.000511836s] [5.000511836s] END E0407 09:48:04.400169 1 timeout.go:142] post-timeout activity - time-elapsed: 2.981741ms, GET "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: I0407 09:48:04.610486 1 trace.go:236] Trace[1121447948]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:f2acaa7a-557c-4a85-b684-f7c727a28051,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (07-Apr-2026 09:48:04.048) (total time: 562ms): Trace[1121447948]: ["GuaranteedUpdate etcd3" audit-id:f2acaa7a-557c-4a85-b684-f7c727a28051,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 562ms (09:48:04.048) Trace[1121447948]: ---"Txn call completed" 561ms (09:48:04.610)] Trace[1121447948]: [562.298076ms] [562.298076ms] END I0407 09:48:04.610519 1 trace.go:236] Trace[1768346580]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:46b45ccb-2754-43fe-9dad-3beaf5b6aff6,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:48:02.054) (total time: 2556ms): Trace[1768346580]: ["GuaranteedUpdate etcd3" audit-id:46b45ccb-2754-43fe-9dad-3beaf5b6aff6,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 2555ms (09:48:02.054) Trace[1768346580]: ---"Txn call completed" 2554ms (09:48:04.610)] Trace[1768346580]: [2.556167221s] [2.556167221s] END I0407 09:48:04.610674 1 trace.go:236] Trace[137198894]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:bdf6d23d-7953-43ab-a663-9ef4f891244e,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:47:58.035) (total time: 6575ms): Trace[137198894]: ["GuaranteedUpdate etcd3" audit-id:bdf6d23d-7953-43ab-a663-9ef4f891244e,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 6574ms (09:47:58.035) Trace[137198894]: ---"Txn call completed" 6573ms (09:48:04.610)] Trace[137198894]: [6.575039978s] [6.575039978s] END I0407 09:48:04.610880 1 trace.go:236] Trace[2090565055]: "Update" accept:application/json, */*,audit-id:e2b6b8a2-1269-4203-859b-6ae2d27fe1c3,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (07-Apr-2026 09:48:02.755) (total time: 1855ms): Trace[2090565055]: ["GuaranteedUpdate etcd3" audit-id:e2b6b8a2-1269-4203-859b-6ae2d27fe1c3,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1855ms (09:48:02.755) Trace[2090565055]: ---"Txn call completed" 1854ms (09:48:04.610)] Trace[2090565055]: [1.855645735s] [1.855645735s] END I0407 09:48:04.610894 1 trace.go:236] Trace[1417900718]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:a4d0a12e-0902-4977-ba96-945054e02692,client:10.0.0.213,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:59.731) (total time: 4878ms): Trace[1417900718]: ---"About to write a response" 4878ms (09:48:04.610) Trace[1417900718]: [4.878912308s] [4.878912308s] END I0407 09:48:04.610895 1 trace.go:236] Trace[665558411]: "Update" accept:application/json, */*,audit-id:dadd80f7-dd7c-44ca-8c6d-b6fe15c4c6c6,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (07-Apr-2026 09:48:04.048) (total time: 562ms): Trace[665558411]: ["GuaranteedUpdate etcd3" audit-id:dadd80f7-dd7c-44ca-8c6d-b6fe15c4c6c6,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 562ms (09:48:04.048) Trace[665558411]: ---"Txn call completed" 561ms (09:48:04.610)] Trace[665558411]: [562.46005ms] [562.46005ms] END I0407 09:48:04.611083 1 trace.go:236] Trace[1123428777]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:5c62eb3a-b5a5-4442-8f93-10999ede7b98,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (07-Apr-2026 09:48:01.348) (total time: 3262ms): Trace[1123428777]: ---"About to write a response" 3262ms (09:48:04.610) Trace[1123428777]: [3.262520016s] [3.262520016s] END I0407 09:48:04.611653 1 trace.go:236] Trace[137348367]: "Get" accept:application/json, */*,audit-id:6037a083-048a-45ba-b78e-247fa41a4e35,client:199.204.45.240,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-m9msq,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:47:54.404) (total time: 10206ms): Trace[137348367]: ---"About to write a response" 10205ms (09:48:04.610) Trace[137348367]: [10.206598845s] [10.206598845s] END I0407 09:48:04.611809 1 trace.go:236] Trace[1813669421]: "Update" accept:application/json, */*,audit-id:2e3141e4-2545-4a3f-9fe0-004b2a17f3b9,client:199.204.45.240,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:48:04.048) (total time: 563ms): Trace[1813669421]: ["GuaranteedUpdate etcd3" audit-id:2e3141e4-2545-4a3f-9fe0-004b2a17f3b9,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 563ms (09:48:04.048) Trace[1813669421]: ---"Txn call completed" 561ms (09:48:04.610)] Trace[1813669421]: [563.29711ms] [563.29711ms] END I0407 09:48:04.618836 1 trace.go:236] Trace[1771527507]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.240,type:*v1.Endpoints,resource:apiServerIPInfo (07-Apr-2026 09:48:00.071) (total time: 4546ms): Trace[1771527507]: ---"initial value restored" 4538ms (09:48:04.609) Trace[1771527507]: [4.546906218s] [4.546906218s] END I0407 09:48:04.623275 1 trace.go:236] Trace[923578495]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:afc293e7-b734-4f62-a488-b881ad66e53c,client:199.204.45.240,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events/kube-apiserver-instance.18a40a1dfab9a2e5,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (07-Apr-2026 09:48:03.975) (total time: 647ms): Trace[923578495]: ["GuaranteedUpdate etcd3" audit-id:afc293e7-b734-4f62-a488-b881ad66e53c,key:/events/kube-system/kube-apiserver-instance.18a40a1dfab9a2e5,type:*core.Event,resource:events 647ms (09:48:03.975) Trace[923578495]: ---"initial value restored" 634ms (09:48:04.610)] Trace[923578495]: [647.807266ms] [647.807266ms] END I0407 09:48:26.484238 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-grafana" clusterIPs={"IPv4":"10.96.20.149"} I0407 09:48:26.491010 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-operator" clusterIPs={"IPv4":"10.105.58.15"} I0407 09:48:26.498621 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus" clusterIPs={"IPv4":"10.106.115.30"} I0407 09:48:26.505104 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-alertmanager" clusterIPs={"IPv4":"10.109.223.180"} I0407 09:48:26.514226 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus-node-exporter" clusterIPs={"IPv4":"10.108.51.49"} I0407 09:48:26.521207 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-kube-state-metrics" clusterIPs={"IPv4":"10.98.139.149"} I0407 09:48:26.564687 1 controller.go:624] quota admission added evaluator for: alertmanagers.monitoring.coreos.com I0407 09:48:26.575823 1 controller.go:624] quota admission added evaluator for: prometheusrules.monitoring.coreos.com I0407 09:48:26.608485 1 controller.go:624] quota admission added evaluator for: podmonitors.monitoring.coreos.com I0407 09:48:26.620706 1 controller.go:624] quota admission added evaluator for: servicemonitors.monitoring.coreos.com I0407 09:48:26.641269 1 controller.go:624] quota admission added evaluator for: prometheuses.monitoring.coreos.com W0407 09:48:26.651337 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.651367 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.651376 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.651503 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.651626 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.651674 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.651694 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.651725 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.652271 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.652335 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.652378 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.652338 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.652796 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.652833 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.652901 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.652929 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.653180 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.653198 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.653428 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.653480 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.653562 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.653609 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.653637 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.653674 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.653765 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.653808 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.654349 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.654378 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.654405 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.654407 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused W0407 09:48:26.656280 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused E0407 09:48:26.656307 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.58.15:443: connect: connection refused I0407 09:48:38.001514 1 alloc.go:330] "allocated clusterIPs" service="monitoring/loki-gateway" clusterIPs={"IPv4":"10.108.183.224"} I0407 09:48:38.011202 1 alloc.go:330] "allocated clusterIPs" service="monitoring/loki" clusterIPs={"IPv4":"10.103.14.48"} I0407 09:48:42.291170 1 alloc.go:330] "allocated clusterIPs" service="monitoring/goldpinger" clusterIPs={"IPv4":"10.102.51.104"} I0407 09:48:45.145652 1 alloc.go:330] "allocated clusterIPs" service="monitoring/prometheus-pushgateway" clusterIPs={"IPv4":"10.100.19.26"} I0407 09:48:49.707964 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached" clusterIPs={"IPv4":"10.102.103.41"} I0407 09:48:50.544153 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached-metrics" clusterIPs={"IPv4":"10.106.2.110"} I0407 09:48:57.288308 1 controller.go:624] quota admission added evaluator for: rabbitmqclusters.rabbitmq.com I0407 09:48:57.451015 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-keystone" clusterIPs={"IPv4":"10.107.232.241"} I0407 09:49:48.080019 1 alloc.go:330] "allocated clusterIPs" service="openstack/keystone-api" clusterIPs={"IPv4":"10.111.64.200"} I0407 09:49:48.092355 1 controller.go:624] quota admission added evaluator for: cronjobs.batch I0407 09:52:50.557964 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-barbican" clusterIPs={"IPv4":"10.101.176.180"} I0407 09:53:23.834831 1 alloc.go:330] "allocated clusterIPs" service="openstack/barbican-api" clusterIPs={"IPv4":"10.97.30.244"} I0407 09:55:19.123971 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.141877 1 handler.go:275] Adding GroupVersion objectbucket.io v1alpha1 to ResourceManager I0407 09:55:19.141999 1 handler.go:275] Adding GroupVersion objectbucket.io v1alpha1 to ResourceManager I0407 09:55:19.160412 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.171951 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.179947 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.187918 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.198836 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.215259 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.223995 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.241416 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.261015 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.270885 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.283635 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.301514 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.380104 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.416645 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.461330 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:55:19.537082 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0407 09:56:00.476732 1 controller.go:624] quota admission added evaluator for: cephclusters.ceph.rook.io I0407 09:56:00.487907 1 controller.go:624] quota admission added evaluator for: cephobjectstores.ceph.rook.io I0407 09:56:33.714790 1 alloc.go:330] "allocated clusterIPs" service="openstack/rook-ceph-rgw-ceph" clusterIPs={"IPv4":"10.109.113.226"} I0407 09:56:35.835261 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-glance" clusterIPs={"IPv4":"10.101.153.109"} I0407 09:57:09.577480 1 alloc.go:330] "allocated clusterIPs" service="openstack/glance-api" clusterIPs={"IPv4":"10.102.143.127"} I0407 09:59:46.616953 1 alloc.go:330] "allocated clusterIPs" service="openstack/staffeln-api" clusterIPs={"IPv4":"10.109.38.48"}