I0407 09:34:29.961859 1 options.go:220] external host was not specified, using 199.204.45.74 I0407 09:34:29.962778 1 server.go:148] Version: v1.28.13 I0407 09:34:29.962815 1 server.go:150] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" I0407 09:34:30.196495 1 shared_informer.go:311] Waiting for caches to sync for node_authorizer I0407 09:34:30.202266 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. I0407 09:34:30.202283 1 plugins.go:161] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota. I0407 09:34:30.202486 1 instance.go:298] Using reconciler: lease I0407 09:34:30.517377 1 handler.go:275] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager W0407 09:34:30.517495 1 genericapiserver.go:744] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources. I0407 09:34:30.770428 1 handler.go:275] Adding GroupVersion v1 to ResourceManager I0407 09:34:30.770686 1 instance.go:709] API group "internal.apiserver.k8s.io" is not enabled, skipping. I0407 09:34:31.215932 1 instance.go:709] API group "resource.k8s.io" is not enabled, skipping. I0407 09:34:31.240056 1 handler.go:275] Adding GroupVersion authentication.k8s.io v1 to ResourceManager W0407 09:34:31.240087 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1beta1 because it has no resources. W0407 09:34:31.240095 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1alpha1 because it has no resources. I0407 09:34:31.242486 1 handler.go:275] Adding GroupVersion authorization.k8s.io v1 to ResourceManager W0407 09:34:31.242513 1 genericapiserver.go:744] Skipping API authorization.k8s.io/v1beta1 because it has no resources. I0407 09:34:31.243433 1 handler.go:275] Adding GroupVersion autoscaling v2 to ResourceManager I0407 09:34:31.244085 1 handler.go:275] Adding GroupVersion autoscaling v1 to ResourceManager W0407 09:34:31.244098 1 genericapiserver.go:744] Skipping API autoscaling/v2beta1 because it has no resources. W0407 09:34:31.244103 1 genericapiserver.go:744] Skipping API autoscaling/v2beta2 because it has no resources. I0407 09:34:31.245440 1 handler.go:275] Adding GroupVersion batch v1 to ResourceManager W0407 09:34:31.245454 1 genericapiserver.go:744] Skipping API batch/v1beta1 because it has no resources. I0407 09:34:31.246330 1 handler.go:275] Adding GroupVersion certificates.k8s.io v1 to ResourceManager W0407 09:34:31.246344 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1beta1 because it has no resources. W0407 09:34:31.246347 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1alpha1 because it has no resources. I0407 09:34:31.247158 1 handler.go:275] Adding GroupVersion coordination.k8s.io v1 to ResourceManager W0407 09:34:31.247170 1 genericapiserver.go:744] Skipping API coordination.k8s.io/v1beta1 because it has no resources. W0407 09:34:31.247208 1 genericapiserver.go:744] Skipping API discovery.k8s.io/v1beta1 because it has no resources. I0407 09:34:31.247694 1 handler.go:275] Adding GroupVersion discovery.k8s.io v1 to ResourceManager I0407 09:34:31.249245 1 handler.go:275] Adding GroupVersion networking.k8s.io v1 to ResourceManager W0407 09:34:31.249260 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1beta1 because it has no resources. W0407 09:34:31.249264 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1alpha1 because it has no resources. I0407 09:34:31.249680 1 handler.go:275] Adding GroupVersion node.k8s.io v1 to ResourceManager W0407 09:34:31.249692 1 genericapiserver.go:744] Skipping API node.k8s.io/v1beta1 because it has no resources. W0407 09:34:31.249696 1 genericapiserver.go:744] Skipping API node.k8s.io/v1alpha1 because it has no resources. I0407 09:34:31.250361 1 handler.go:275] Adding GroupVersion policy v1 to ResourceManager W0407 09:34:31.250374 1 genericapiserver.go:744] Skipping API policy/v1beta1 because it has no resources. I0407 09:34:31.251970 1 handler.go:275] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager W0407 09:34:31.251985 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources. W0407 09:34:31.251989 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. I0407 09:34:31.252445 1 handler.go:275] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager W0407 09:34:31.252459 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1beta1 because it has no resources. W0407 09:34:31.252462 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. I0407 09:34:31.254481 1 handler.go:275] Adding GroupVersion storage.k8s.io v1 to ResourceManager W0407 09:34:31.254496 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1beta1 because it has no resources. W0407 09:34:31.254500 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1alpha1 because it has no resources. I0407 09:34:31.255438 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager I0407 09:34:31.256399 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta2 to ResourceManager W0407 09:34:31.256413 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources. W0407 09:34:31.256416 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources. I0407 09:34:31.259593 1 handler.go:275] Adding GroupVersion apps v1 to ResourceManager W0407 09:34:31.259608 1 genericapiserver.go:744] Skipping API apps/v1beta2 because it has no resources. W0407 09:34:31.259613 1 genericapiserver.go:744] Skipping API apps/v1beta1 because it has no resources. I0407 09:34:31.260364 1 handler.go:275] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager W0407 09:34:31.260376 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources. W0407 09:34:31.260380 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources. I0407 09:34:31.260774 1 handler.go:275] Adding GroupVersion events.k8s.io v1 to ResourceManager W0407 09:34:31.260787 1 genericapiserver.go:744] Skipping API events.k8s.io/v1beta1 because it has no resources. I0407 09:34:31.293682 1 handler.go:275] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager W0407 09:34:31.293733 1 genericapiserver.go:744] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources. I0407 09:34:31.728868 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0407 09:34:31.728883 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0407 09:34:31.729087 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/etc/kubernetes/pki/apiserver.crt::/etc/kubernetes/pki/apiserver.key" I0407 09:34:31.729325 1 secure_serving.go:213] Serving securely on [::]:6443 I0407 09:34:31.729391 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" I0407 09:34:31.729442 1 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/etc/kubernetes/pki/front-proxy-client.crt::/etc/kubernetes/pki/front-proxy-client.key" I0407 09:34:31.729671 1 apf_controller.go:374] Starting API Priority and Fairness config controller I0407 09:34:31.729711 1 controller.go:80] Starting OpenAPI V3 AggregationController I0407 09:34:31.729714 1 gc_controller.go:78] Starting apiserver lease garbage collector I0407 09:34:31.729803 1 aggregator.go:164] waiting for initial CRD sync... I0407 09:34:31.729845 1 handler_discovery.go:412] Starting ResourceDiscoveryManager I0407 09:34:31.729874 1 controller.go:116] Starting legacy_token_tracking_controller I0407 09:34:31.729893 1 shared_informer.go:311] Waiting for caches to sync for configmaps I0407 09:34:31.730016 1 system_namespaces_controller.go:67] Starting system namespaces controller I0407 09:34:31.730055 1 controller.go:78] Starting OpenAPI AggregationController I0407 09:34:31.730290 1 customresource_discovery_controller.go:289] Starting DiscoveryController I0407 09:34:31.730371 1 available_controller.go:423] Starting AvailableConditionController I0407 09:34:31.730388 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller I0407 09:34:31.730473 1 controller.go:134] Starting OpenAPI controller I0407 09:34:31.730517 1 controller.go:85] Starting OpenAPI V3 controller I0407 09:34:31.730523 1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller I0407 09:34:31.730534 1 shared_informer.go:311] Waiting for caches to sync for cluster_authentication_trust_controller I0407 09:34:31.730544 1 naming_controller.go:291] Starting NamingConditionController I0407 09:34:31.730569 1 establishing_controller.go:76] Starting EstablishingController I0407 09:34:31.730575 1 gc_controller.go:78] Starting apiserver lease garbage collector I0407 09:34:31.730598 1 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController I0407 09:34:31.730632 1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController I0407 09:34:31.730650 1 crd_finalizer.go:266] Starting CRDFinalizer I0407 09:34:31.730697 1 apiservice_controller.go:97] Starting APIServiceRegistrationController I0407 09:34:31.730725 1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller I0407 09:34:31.730763 1 crdregistration_controller.go:111] Starting crd-autoregister controller I0407 09:34:31.730781 1 shared_informer.go:311] Waiting for caches to sync for crd-autoregister I0407 09:34:31.730568 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0407 09:34:31.730582 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0407 09:34:31.797508 1 shared_informer.go:318] Caches are synced for node_authorizer I0407 09:34:31.830454 1 cache.go:39] Caches are synced for AvailableConditionController controller I0407 09:34:31.830759 1 apf_controller.go:379] Running API Priority and Fairness config worker I0407 09:34:31.830774 1 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process I0407 09:34:31.830987 1 shared_informer.go:318] Caches are synced for crd-autoregister I0407 09:34:31.830990 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller I0407 09:34:31.831066 1 aggregator.go:166] initial CRD sync complete... I0407 09:34:31.831085 1 autoregister_controller.go:141] Starting autoregister controller I0407 09:34:31.831106 1 cache.go:32] Waiting for caches to sync for autoregister controller I0407 09:34:31.831115 1 cache.go:39] Caches are synced for autoregister controller I0407 09:34:31.831155 1 shared_informer.go:318] Caches are synced for cluster_authentication_trust_controller I0407 09:34:31.831430 1 controller.go:624] quota admission added evaluator for: namespaces I0407 09:34:31.831739 1 shared_informer.go:318] Caches are synced for configmaps I0407 09:34:31.852890 1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io I0407 09:34:32.822367 1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000 I0407 09:34:33.296319 1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000 I0407 09:34:33.296344 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist. I0407 09:34:33.870412 1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io I0407 09:34:33.922517 1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io I0407 09:34:34.053441 1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"} W0407 09:34:34.066692 1 lease.go:263] Resetting endpoints for master service "kubernetes" to [199.204.45.74] I0407 09:34:34.067883 1 controller.go:624] quota admission added evaluator for: endpoints I0407 09:34:34.072472 1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io I0407 09:34:35.557496 1 controller.go:624] quota admission added evaluator for: serviceaccounts I0407 09:34:35.567295 1 controller.go:624] quota admission added evaluator for: deployments.apps I0407 09:34:35.761634 1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"} I0407 09:34:35.852107 1 controller.go:624] quota admission added evaluator for: daemonsets.apps I0407 09:34:40.822220 1 controller.go:624] quota admission added evaluator for: replicasets.apps I0407 09:34:41.074091 1 controller.go:624] quota admission added evaluator for: controllerrevisions.apps I0407 09:35:00.321347 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0407 09:35:00.365729 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0407 09:35:00.392163 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0407 09:35:00.392211 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0407 09:35:00.407800 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0407 09:35:00.425672 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0407 09:35:00.432586 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0407 09:35:00.479830 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0407 09:35:00.502897 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0407 09:35:00.503092 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0407 09:35:00.557947 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0407 09:35:00.557986 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0407 09:35:00.597530 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0407 09:35:00.611586 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0407 09:35:00.611749 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0407 09:35:00.769153 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0407 09:35:00.769194 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0407 09:35:00.799521 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:35:00.891031 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:35:00.905441 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:35:00.937033 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:35:00.949240 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:35:01.759736 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:35:01.778382 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:35:01.906119 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0407 09:35:08.411514 1 controller.go:624] quota admission added evaluator for: jobs.batch I0407 09:35:09.270617 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0407 09:35:09.301825 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0407 09:35:09.307918 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0407 09:35:09.314736 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0407 09:35:09.839308 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0407 09:35:11.414209 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0407 09:35:11.541819 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0407 09:35:11.554497 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0407 09:35:12.047533 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0407 09:35:12.689646 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0407 09:35:12.872312 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager W0407 09:35:18.970893 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found E0407 09:35:18.970944 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found I0407 09:35:19.426420 1 controller.go:624] quota admission added evaluator for: ciliumendpoints.cilium.io I0407 09:35:27.978501 1 alloc.go:330] "allocated clusterIPs" service="envoy-gateway-system/envoy-gateway" clusterIPs={"IPv4":"10.106.70.205"} W0407 09:35:28.008406 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.106.70.205:9443: connect: connection refused E0407 09:35:28.008450 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.106.70.205:9443: connect: connection refused I0407 09:35:43.411233 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0407 09:35:43.432320 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0407 09:35:43.479871 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0407 09:35:43.502982 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager" clusterIPs={"IPv4":"10.100.108.146"} I0407 09:35:43.515615 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager-webhook" clusterIPs={"IPv4":"10.98.113.196"} I0407 09:35:43.518259 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0407 09:35:43.571995 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0407 09:35:43.602280 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager W0407 09:35:53.892109 1 dispatcher.go:225] Failed calling webhook, failing closed webhook.cert-manager.io: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": dial tcp 10.98.113.196:443: connect: connection refused I0407 09:35:58.891218 1 controller.go:624] quota admission added evaluator for: certificates.cert-manager.io I0407 09:36:05.090442 1 controller.go:624] quota admission added evaluator for: certificaterequests.cert-manager.io I0407 09:36:18.303570 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-admission" clusterIPs={"IPv4":"10.107.225.56"} I0407 09:36:18.311555 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-metrics" clusterIPs={"IPv4":"10.108.13.68"} I0407 09:36:18.321337 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-defaultbackend" clusterIPs={"IPv4":"10.101.155.199"} I0407 09:36:18.331563 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller" clusterIPs={"IPv4":"10.106.125.117"} I0407 09:36:24.810767 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:24.841834 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:24.861446 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:24.885150 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:24.907227 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:24.934776 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:24.946755 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:24.963783 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:24.985315 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:25.009199 1 handler.go:275] Adding GroupVersion rabbitmq.com v1alpha1 to ResourceManager I0407 09:36:25.022050 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:25.049020 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:25.068774 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:26.337512 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0407 09:36:30.482896 1 controller.go:624] quota admission added evaluator for: networkpolicies.networking.k8s.io I0407 09:36:30.542393 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-messaging-topology-operator-webhook" clusterIPs={"IPv4":"10.104.227.77"} I0407 09:36:30.581382 1 controller.go:624] quota admission added evaluator for: issuers.cert-manager.io I0407 09:36:32.767933 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0407 09:36:32.807340 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0407 09:36:33.431381 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-10-0 to ResourceManager I0407 09:36:33.431427 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-11-0 to ResourceManager I0407 09:36:33.431626 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0407 09:36:40.648883 1 controller.go:624] quota admission added evaluator for: perconaxtradbclusters.pxc.percona.com I0407 09:36:43.596191 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-cluster-operator" clusterIPs={"IPv4":"10.107.37.163"} I0407 09:36:43.974907 1 controller.go:624] quota admission added evaluator for: statefulsets.apps I0407 09:36:44.098168 1 controller.go:624] quota admission added evaluator for: poddisruptionbudgets.policy I0407 09:36:44.238276 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy" clusterIPs={"IPv4":"10.107.26.92"} I0407 09:36:44.259688 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-replicas" clusterIPs={"IPv4":"10.98.75.42"} I0407 09:36:50.712677 1 trace.go:236] Trace[143659128]: "Update" accept:application/json, */*,audit-id:11e2ef30-c63e-4e3d-a249-e8830a151dd9,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:36:50.090) (total time: 622ms): Trace[143659128]: ["GuaranteedUpdate etcd3" audit-id:11e2ef30-c63e-4e3d-a249-e8830a151dd9,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 621ms (09:36:50.090) Trace[143659128]: ---"Txn call completed" 621ms (09:36:50.712)] Trace[143659128]: [622.128383ms] [622.128383ms] END I0407 09:36:50.721737 1 trace.go:236] Trace[1877987828]: "Delete" accept:application/vnd.kubernetes.protobuf, */*,audit-id:ce9d6b77-73c6-4a3f-aad7-4baa99266329,client:10.0.0.186,protocol:HTTP/2.0,resource:deployments,scope:resource,url:/apis/apps/v1/namespaces/openstack/deployments/percona-xtradb-pitr,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:DELETE (07-Apr-2026 09:36:50.098) (total time: 623ms): Trace[1877987828]: [623.539053ms] [623.539053ms] END I0407 09:36:50.722428 1 trace.go:236] Trace[193484869]: "Get" accept:application/json, */*,audit-id:831d0ff3-7816-4170-9cac-6d6c4907771d,client:10.0.0.207,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/local-path-storage/pods/helper-pod-create-pvc-8ad598bd-d7c7-4d79-b93a-1b758caa3371,user-agent:local-path-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:36:50.122) (total time: 599ms): Trace[193484869]: ---"About to write a response" 599ms (09:36:50.721) Trace[193484869]: [599.842973ms] [599.842973ms] END I0407 09:36:50.723164 1 trace.go:236] Trace[1955280516]: "Update" accept:application/json, */*,audit-id:d5a67bc1-e114-40a0-8282-78b8dd61b34e,client:10.0.0.206,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:36:50.090) (total time: 632ms): Trace[1955280516]: ["GuaranteedUpdate etcd3" audit-id:d5a67bc1-e114-40a0-8282-78b8dd61b34e,key:/leases/openstack/rabbitmq-cluster-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 632ms (09:36:50.090) Trace[1955280516]: ---"Txn call completed" 631ms (09:36:50.723)] Trace[1955280516]: [632.845434ms] [632.845434ms] END I0407 09:38:16.446163 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-metrics" clusterIPs={"IPv4":"10.100.192.204"} I0407 09:38:18.524846 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached" clusterIPs={"IPv4":"10.109.206.34"} I0407 09:38:19.411921 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached-metrics" clusterIPs={"IPv4":"10.104.130.129"} I0407 09:38:27.401095 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak" clusterIPs={"IPv4":"10.99.18.238"} I0407 09:38:27.411858 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak-metrics" clusterIPs={"IPv4":"10.102.127.153"} I0407 09:39:08.400890 1 trace.go:236] Trace[2038916091]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:b368b304-29f6-4f93-aff7-6f9f6e589450,client:199.204.45.74,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/auth-system/events/keycloak-0.18a4099cf10db55c,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (07-Apr-2026 09:39:07.756) (total time: 644ms): Trace[2038916091]: ["GuaranteedUpdate etcd3" audit-id:b368b304-29f6-4f93-aff7-6f9f6e589450,key:/events/auth-system/keycloak-0.18a4099cf10db55c,type:*core.Event,resource:events 643ms (09:39:07.756) Trace[2038916091]: ---"Txn call completed" 641ms (09:39:08.400)] Trace[2038916091]: ---"Object stored in database" 641ms (09:39:08.400) Trace[2038916091]: [644.114843ms] [644.114843ms] END I0407 09:39:08.629214 1 trace.go:236] Trace[415546322]: "Get" accept:application/json, */*,audit-id:f9bce15d-b303-492d-bbcd-9e758db2ced1,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:39:08.080) (total time: 548ms): Trace[415546322]: ---"About to write a response" 548ms (09:39:08.629) Trace[415546322]: [548.369183ms] [548.369183ms] END I0407 09:39:08.629770 1 trace.go:236] Trace[1909572883]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:da30e1a5-942b-4c3d-8a06-e60737853d90,client:10.0.0.186,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:39:08.117) (total time: 511ms): Trace[1909572883]: ["GuaranteedUpdate etcd3" audit-id:da30e1a5-942b-4c3d-8a06-e60737853d90,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 511ms (09:39:08.118) Trace[1909572883]: ---"Txn call completed" 510ms (09:39:08.629)] Trace[1909572883]: [511.819851ms] [511.819851ms] END I0407 09:39:08.629860 1 trace.go:236] Trace[11353574]: "Get" accept:application/json, */*,audit-id:ca64323e-0952-4c3a-8650-f601a52acf58,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:39:07.950) (total time: 678ms): Trace[11353574]: ---"About to write a response" 678ms (09:39:08.629) Trace[11353574]: [678.95456ms] [678.95456ms] END I0407 09:39:08.629770 1 trace.go:236] Trace[1822199299]: "Get" accept:application/json, */*,audit-id:ff5c1e23-7567-4e5f-bf65-e9270221df3a,client:10.0.0.206,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:39:07.848) (total time: 781ms): Trace[1822199299]: ---"About to write a response" 781ms (09:39:08.629) Trace[1822199299]: [781.304047ms] [781.304047ms] END I0407 09:39:08.630612 1 trace.go:236] Trace[2089727167]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:929872d8-7726-4d40-aea1-63f0dea6ca24,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:39:07.843) (total time: 786ms): Trace[2089727167]: ---"About to write a response" 786ms (09:39:08.629) Trace[2089727167]: [786.981206ms] [786.981206ms] END I0407 09:40:57.289497 1 controller.go:624] quota admission added evaluator for: ingresses.networking.k8s.io I0407 09:40:59.188627 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0407 09:40:59.198130 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0407 09:40:59.221222 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0407 09:40:59.229675 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0407 09:40:59.239944 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0407 09:40:59.250701 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0407 09:40:59.262787 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0407 09:41:11.900979 1 controller.go:624] quota admission added evaluator for: passwords.secretgen.k14s.io I0407 09:41:51.691227 1 controller.go:624] quota admission added evaluator for: secrettemplates.secretgen.carvel.dev I0407 09:42:05.720450 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0407 09:42:05.890940 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:42:05.913677 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:42:05.929109 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:42:06.268608 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0407 09:42:06.504165 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:42:06.509403 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:42:06.583825 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0407 09:42:06.595152 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:42:06.832722 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0407 09:42:48.355481 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-kube-state-metrics" clusterIPs={"IPv4":"10.101.183.78"} I0407 09:42:48.372033 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus-node-exporter" clusterIPs={"IPv4":"10.103.151.119"} I0407 09:42:48.378838 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-operator" clusterIPs={"IPv4":"10.104.239.209"} I0407 09:42:48.387644 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-grafana" clusterIPs={"IPv4":"10.105.86.143"} I0407 09:42:48.401757 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus" clusterIPs={"IPv4":"10.105.187.8"} I0407 09:42:48.419366 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-alertmanager" clusterIPs={"IPv4":"10.106.108.215"} I0407 09:42:48.488532 1 controller.go:624] quota admission added evaluator for: alertmanagers.monitoring.coreos.com I0407 09:42:48.510792 1 controller.go:624] quota admission added evaluator for: prometheusrules.monitoring.coreos.com I0407 09:42:48.557277 1 controller.go:624] quota admission added evaluator for: podmonitors.monitoring.coreos.com I0407 09:42:48.574123 1 controller.go:624] quota admission added evaluator for: servicemonitors.monitoring.coreos.com I0407 09:42:48.599449 1 controller.go:624] quota admission added evaluator for: prometheuses.monitoring.coreos.com W0407 09:42:48.608668 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.608753 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.610648 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.610684 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.611775 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.611822 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.612491 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.612662 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.612764 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.612793 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.612788 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.612872 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.613813 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.613864 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.614245 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.614287 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.614717 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.614754 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.615109 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.615569 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.615275 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.615633 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.615850 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.615874 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.616691 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.616712 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.617585 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.617623 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.617680 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused W0407 09:42:48.618883 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.619457 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused E0407 09:42:48.619563 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.239.209:443: connect: connection refused I0407 09:43:02.939320 1 controller.go:624] quota admission added evaluator for: rabbitmqclusters.rabbitmq.com I0407 09:43:03.168597 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-keystone" clusterIPs={"IPv4":"10.103.140.82"} I0407 09:43:43.142377 1 alloc.go:330] "allocated clusterIPs" service="openstack/keystone-api" clusterIPs={"IPv4":"10.109.24.67"} I0407 09:43:43.210941 1 controller.go:624] quota admission added evaluator for: cronjobs.batch I0407 09:44:01.197291 1 trace.go:236] Trace[1884517422]: "Get" accept:application/json, */*,audit-id:7d3690b3-8247-40a0-b13b-f4e8d6b2243e,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:44:00.679) (total time: 518ms): Trace[1884517422]: ---"About to write a response" 517ms (09:44:01.197) Trace[1884517422]: [518.158881ms] [518.158881ms] END I0407 09:44:07.509580 1 trace.go:236] Trace[65664534]: "Update" accept:application/json, */*,audit-id:853e176c-de70-43b7-88db-3df2955e2309,client:10.0.0.149,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:44:06.114) (total time: 1395ms): Trace[65664534]: ["GuaranteedUpdate etcd3" audit-id:853e176c-de70-43b7-88db-3df2955e2309,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 1395ms (09:44:06.114) Trace[65664534]: ---"Txn call completed" 1394ms (09:44:07.509)] Trace[65664534]: [1.39538562s] [1.39538562s] END I0407 09:44:07.778247 1 trace.go:236] Trace[1306729862]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:4354f4ce-287e-4d8e-8b0b-72ffa0f4def4,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:44:06.424) (total time: 1353ms): Trace[1306729862]: ---"About to write a response" 1353ms (09:44:07.778) Trace[1306729862]: [1.353421788s] [1.353421788s] END I0407 09:44:07.778500 1 trace.go:236] Trace[322621932]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:fc573a9b-57ab-4f1c-9436-d74cc878e166,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:44:07.225) (total time: 553ms): Trace[322621932]: ---"About to write a response" 553ms (09:44:07.778) Trace[322621932]: [553.242623ms] [553.242623ms] END I0407 09:44:07.778532 1 trace.go:236] Trace[33749359]: "Get" accept:application/json, */*,audit-id:2e2d6129-17ab-4295-8486-e9c076616d4c,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:44:07.225) (total time: 553ms): Trace[33749359]: ---"About to write a response" 553ms (09:44:07.778) Trace[33749359]: [553.460577ms] [553.460577ms] END I0407 09:44:07.778734 1 trace.go:236] Trace[634774993]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:fef5ad8e-3526-44fa-9895-aab4e4ce7054,client:199.204.45.74,protocol:HTTP/2.0,resource:nodes,scope:resource,url:/api/v1/nodes/instance/status,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (07-Apr-2026 09:44:06.796) (total time: 982ms): Trace[634774993]: ["GuaranteedUpdate etcd3" audit-id:fef5ad8e-3526-44fa-9895-aab4e4ce7054,key:/minions/instance,type:*core.Node,resource:nodes 982ms (09:44:06.796) Trace[634774993]: ---"Txn call completed" 977ms (09:44:07.777)] Trace[634774993]: ---"Object stored in database" 978ms (09:44:07.777) Trace[634774993]: [982.52997ms] [982.52997ms] END I0407 09:44:07.778734 1 trace.go:236] Trace[549385676]: "Get" accept:application/json, */*,audit-id:045958b2-80e5-42ce-9947-c92123358c3e,client:10.0.0.211,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/openstack/jobs/keystone-db-sync,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:44:06.625) (total time: 1153ms): Trace[549385676]: [1.15359079s] [1.15359079s] END I0407 09:44:07.778898 1 trace.go:236] Trace[1844088842]: "Get" accept:application/json, */*,audit-id:b0f5b232-ec70-46aa-a876-b7db0e080538,client:10.0.0.206,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:44:06.866) (total time: 911ms): Trace[1844088842]: ---"About to write a response" 911ms (09:44:07.778) Trace[1844088842]: [911.916902ms] [911.916902ms] END I0407 09:44:07.778898 1 trace.go:236] Trace[1441787048]: "Get" accept:application/json, */*,audit-id:004460ae-e492-4702-9c2b-e47f89421d07,client:10.0.0.211,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/openstack/jobs/keystone-fernet-setup,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:44:06.624) (total time: 1153ms): Trace[1441787048]: [1.153506069s] [1.153506069s] END I0407 09:44:07.778934 1 trace.go:236] Trace[59268409]: "Get" accept:application/json, */*,audit-id:fca45551-0d6b-4d1c-af74-8073673aff1e,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:44:06.483) (total time: 1294ms): Trace[59268409]: ---"About to write a response" 1294ms (09:44:07.778) Trace[59268409]: [1.294942728s] [1.294942728s] END I0407 09:44:07.779047 1 trace.go:236] Trace[1015222444]: "Get" accept:application/json, */*,audit-id:a131fa66-62b1-4c0e-a466-8033376ee846,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:44:07.227) (total time: 551ms): Trace[1015222444]: ---"About to write a response" 550ms (09:44:07.778) Trace[1015222444]: [551.045455ms] [551.045455ms] END I0407 09:44:07.779264 1 trace.go:236] Trace[1434793871]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:a6b1c5c2-bd39-4b9e-928c-8a268e702461,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:44:06.226) (total time: 1552ms): Trace[1434793871]: ["GuaranteedUpdate etcd3" audit-id:a6b1c5c2-bd39-4b9e-928c-8a268e702461,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 1552ms (09:44:06.226) Trace[1434793871]: ---"Txn call completed" 1551ms (09:44:07.779)] Trace[1434793871]: [1.55249098s] [1.55249098s] END I0407 09:44:07.781452 1 trace.go:236] Trace[1684715754]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:ebb135b1-d3dd-4fe0-b3f9-2d8d5fbd58e5,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:44:06.115) (total time: 1666ms): Trace[1684715754]: ["GuaranteedUpdate etcd3" audit-id:ebb135b1-d3dd-4fe0-b3f9-2d8d5fbd58e5,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 1666ms (09:44:06.115) Trace[1684715754]: ---"Txn call completed" 1664ms (09:44:07.781)] Trace[1684715754]: [1.666226582s] [1.666226582s] END I0407 09:44:18.556398 1 trace.go:236] Trace[1902164415]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:a773facb-3586-49db-be53-a18f90a76858,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (07-Apr-2026 09:44:17.972) (total time: 583ms): Trace[1902164415]: ["GuaranteedUpdate etcd3" audit-id:a773facb-3586-49db-be53-a18f90a76858,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 583ms (09:44:17.972) Trace[1902164415]: ---"Txn call completed" 582ms (09:44:18.555)] Trace[1902164415]: [583.814199ms] [583.814199ms] END I0407 09:44:18.556850 1 trace.go:236] Trace[1045153523]: "Get" accept:application/json, */*,audit-id:9ef01e4c-d77d-4078-9573-0f3ce00fd4d0,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:44:17.972) (total time: 584ms): Trace[1045153523]: ---"About to write a response" 583ms (09:44:18.556) Trace[1045153523]: [584.081245ms] [584.081245ms] END I0407 09:44:18.556862 1 trace.go:236] Trace[2115573121]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:7c894b35-7f5a-41dd-b691-01a7f43c8ca7,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:44:17.957) (total time: 598ms): Trace[2115573121]: ["GuaranteedUpdate etcd3" audit-id:7c894b35-7f5a-41dd-b691-01a7f43c8ca7,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 598ms (09:44:17.957) Trace[2115573121]: ---"Txn call completed" 597ms (09:44:18.556)] Trace[2115573121]: [598.995837ms] [598.995837ms] END I0407 09:44:21.679499 1 trace.go:236] Trace[1064890750]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:81a0adb9-e252-4816-bc40-b7acd98eb5d7,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:44:20.816) (total time: 862ms): Trace[1064890750]: ---"About to write a response" 862ms (09:44:21.679) Trace[1064890750]: [862.702444ms] [862.702444ms] END I0407 09:44:21.679740 1 trace.go:236] Trace[2097462448]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:0fdeceed-79ce-4c57-abaa-8031e326eb96,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:44:20.768) (total time: 911ms): Trace[2097462448]: ---"About to write a response" 910ms (09:44:21.679) Trace[2097462448]: [911.014186ms] [911.014186ms] END I0407 09:44:21.680181 1 trace.go:236] Trace[236680409]: "Update" accept:application/json, */*,audit-id:b02fbb82-bd19-4666-9d58-58e85ee9aead,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:44:20.994) (total time: 685ms): Trace[236680409]: ["GuaranteedUpdate etcd3" audit-id:b02fbb82-bd19-4666-9d58-58e85ee9aead,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 685ms (09:44:20.995) Trace[236680409]: ---"Txn call completed" 683ms (09:44:21.679)] Trace[236680409]: [685.263414ms] [685.263414ms] END I0407 09:44:21.680231 1 trace.go:236] Trace[560250033]: "Get" accept:application/json, */*,audit-id:8a282920-17da-4c23-b541-faca73157df4,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:44:20.817) (total time: 862ms): Trace[560250033]: ---"About to write a response" 862ms (09:44:21.680) Trace[560250033]: [862.48088ms] [862.48088ms] END I0407 09:44:21.680527 1 trace.go:236] Trace[386343350]: "Get" accept:application/json, */*,audit-id:49745e12-f3f3-4111-8865-89e43e81476d,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:44:20.818) (total time: 861ms): Trace[386343350]: ---"About to write a response" 861ms (09:44:21.680) Trace[386343350]: [861.625125ms] [861.625125ms] END I0407 09:44:21.680757 1 trace.go:236] Trace[33789644]: "Get" accept:application/json, */*,audit-id:2a4dfb0b-f74f-43b0-8520-cfd54bcfd80f,client:10.0.0.206,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:44:20.817) (total time: 862ms): Trace[33789644]: ---"About to write a response" 862ms (09:44:21.680) Trace[33789644]: [862.898988ms] [862.898988ms] END I0407 09:44:21.680920 1 trace.go:236] Trace[289573915]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:94f6a4e8-4534-4112-87c0-308df8b4592e,client:10.0.0.186,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:44:20.745) (total time: 934ms): Trace[289573915]: ["GuaranteedUpdate etcd3" audit-id:94f6a4e8-4534-4112-87c0-308df8b4592e,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 934ms (09:44:20.746) Trace[289573915]: ---"Txn call completed" 933ms (09:44:21.680)] Trace[289573915]: [934.924036ms] [934.924036ms] END I0407 09:44:42.458666 1 trace.go:236] Trace[1778164834]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:19d46dbf-0e8f-4acc-a416-49622a7baedd,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:44:39.659) (total time: 2798ms): Trace[1778164834]: ["GuaranteedUpdate etcd3" audit-id:19d46dbf-0e8f-4acc-a416-49622a7baedd,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 2798ms (09:44:39.660) Trace[1778164834]: ---"Txn call completed" 2797ms (09:44:42.458)] Trace[1778164834]: [2.798763646s] [2.798763646s] END I0407 09:44:42.465734 1 trace.go:236] Trace[318361820]: "Update" accept:application/json, */*,audit-id:2004e202-ef35-4d07-a10c-c9adba4f57d1,client:10.0.0.149,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:44:40.028) (total time: 2436ms): Trace[318361820]: ["GuaranteedUpdate etcd3" audit-id:2004e202-ef35-4d07-a10c-c9adba4f57d1,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 2436ms (09:44:40.029) Trace[318361820]: ---"Txn call completed" 2435ms (09:44:42.465)] Trace[318361820]: [2.436781036s] [2.436781036s] END I0407 09:44:42.466652 1 trace.go:236] Trace[1906441174]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:bd468cdb-4990-418c-8a28-5a31aa3dfae2,client:199.204.45.74,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (07-Apr-2026 09:44:41.883) (total time: 583ms): Trace[1906441174]: ["Create etcd3" audit-id:bd468cdb-4990-418c-8a28-5a31aa3dfae2,key:/events/kube-system/kube-apiserver-instance.18a409f08eb79623,type:*core.Event,resource:events 582ms (09:44:41.884) Trace[1906441174]: ---"Txn call succeeded" 581ms (09:44:42.466)] Trace[1906441174]: [583.16085ms] [583.16085ms] END I0407 09:44:42.468653 1 trace.go:236] Trace[781574581]: "Get" accept:application/json, */*,audit-id:699c1ac4-2e19-48a6-b46f-655ca6c6ef2e,client:10.0.0.211,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/openstack/jobs/keystone-db-sync,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:44:40.009) (total time: 2458ms): Trace[781574581]: ---"About to write a response" 2458ms (09:44:42.468) Trace[781574581]: [2.458657031s] [2.458657031s] END I0407 09:44:42.468778 1 trace.go:236] Trace[1038169723]: "Get" accept:application/json, */*,audit-id:cb5fa572-cebc-4ce3-beec-b8627b121852,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:44:39.851) (total time: 2617ms): Trace[1038169723]: ---"About to write a response" 2617ms (09:44:42.468) Trace[1038169723]: [2.617586132s] [2.617586132s] END I0407 09:44:42.468829 1 trace.go:236] Trace[712275996]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:99538d8a-5c79-45d4-9105-f7b54cdf4bc2,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:44:41.217) (total time: 1251ms): Trace[712275996]: ["GuaranteedUpdate etcd3" audit-id:99538d8a-5c79-45d4-9105-f7b54cdf4bc2,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 1251ms (09:44:41.217) Trace[712275996]: ---"Txn call completed" 1250ms (09:44:42.468)] Trace[712275996]: [1.251277487s] [1.251277487s] END I0407 09:44:42.468910 1 trace.go:236] Trace[1367740100]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:6018ca52-c98c-407a-bcc5-8f018fca477d,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:44:39.854) (total time: 2614ms): Trace[1367740100]: ---"About to write a response" 2614ms (09:44:42.468) Trace[1367740100]: [2.614223447s] [2.614223447s] END I0407 09:44:42.468944 1 trace.go:236] Trace[493869753]: "Get" accept:application/json, */*,audit-id:e9e0205d-91f8-44b9-8357-6af052c0f0ec,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:44:39.856) (total time: 2612ms): Trace[493869753]: ---"About to write a response" 2612ms (09:44:42.468) Trace[493869753]: [2.612402877s] [2.612402877s] END I0407 09:44:42.469060 1 trace.go:236] Trace[481327285]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:9f5a150c-ca8d-4765-97b0-1ab15581a31e,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (07-Apr-2026 09:44:41.074) (total time: 1394ms): Trace[481327285]: ["GuaranteedUpdate etcd3" audit-id:9f5a150c-ca8d-4765-97b0-1ab15581a31e,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 1394ms (09:44:41.074) Trace[481327285]: ---"Txn call completed" 1393ms (09:44:42.468)] Trace[481327285]: [1.394799821s] [1.394799821s] END I0407 09:44:42.469089 1 trace.go:236] Trace[1616882363]: "Get" accept:application/json, */*,audit-id:43fcf695-1e15-4391-b95e-7947fb733f3e,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:44:39.767) (total time: 2701ms): Trace[1616882363]: ---"About to write a response" 2701ms (09:44:42.468) Trace[1616882363]: [2.701501365s] [2.701501365s] END I0407 09:44:42.469216 1 trace.go:236] Trace[1044104778]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:9bed6a8a-27f4-4284-836e-cccae464cc82,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:44:39.808) (total time: 2660ms): Trace[1044104778]: ---"About to write a response" 2660ms (09:44:42.469) Trace[1044104778]: [2.660952707s] [2.660952707s] END I0407 09:44:42.469440 1 trace.go:236] Trace[1325894028]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:467cdd70-3e5b-42dd-9eb3-6a73a18a65a4,client:10.0.0.186,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:44:39.739) (total time: 2729ms): Trace[1325894028]: ["GuaranteedUpdate etcd3" audit-id:467cdd70-3e5b-42dd-9eb3-6a73a18a65a4,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 2729ms (09:44:39.740) Trace[1325894028]: ---"Txn call completed" 2728ms (09:44:42.469)] Trace[1325894028]: [2.72952937s] [2.72952937s] END I0407 09:44:42.470154 1 trace.go:236] Trace[600874309]: "Get" accept:application/json, */*,audit-id:e7638271-3d48-4012-8e65-47f6ff3cb2a2,client:10.0.0.206,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:44:39.859) (total time: 2610ms): Trace[600874309]: ---"About to write a response" 2609ms (09:44:42.469) Trace[600874309]: [2.610119881s] [2.610119881s] END I0407 09:46:37.270561 1 alloc.go:330] "allocated clusterIPs" service="openstack/horizon-int" clusterIPs={"IPv4":"10.107.250.188"} I0407 09:46:37.281780 1 alloc.go:330] "allocated clusterIPs" service="openstack/horizon" clusterIPs={"IPv4":"10.102.155.155"} I0407 09:47:32.408700 1 trace.go:236] Trace[1877279296]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:64f0b88a-c491-4a1b-9c42-c272423113e4,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:47:27.409) (total time: 4999ms): Trace[1877279296]: ["GuaranteedUpdate etcd3" audit-id:64f0b88a-c491-4a1b-9c42-c272423113e4,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 4998ms (09:47:27.409) Trace[1877279296]: ---"Txn call failed" err:context deadline exceeded 4997ms (09:47:32.408)] Trace[1877279296]: [4.999201429s] [4.999201429s] END E0407 09:47:32.408769 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 10.77µs, panicked: false, err: context deadline exceeded, panic-reason: E0407 09:47:32.408951 1 wrap.go:54] timeout or abort while handling: method=PUT URI="/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io?timeout=5s" audit-ID="64f0b88a-c491-4a1b-9c42-c272423113e4" E0407 09:47:32.408992 1 timeout.go:142] post-timeout activity - time-elapsed: 5.9µs, PUT "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0407 09:47:32.410057 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 8.991µs, panicked: false, err: context canceled, panic-reason: E0407 09:47:32.410158 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:47:32.411327 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:47:32.411430 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:47:32.413327 1 trace.go:236] Trace[2103859957]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:7c356628-9d62-4613-8c1c-7df1d7286148,client:10.0.0.186,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:47:27.410) (total time: 5002ms): Trace[2103859957]: ["GuaranteedUpdate etcd3" audit-id:7c356628-9d62-4613-8c1c-7df1d7286148,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 5002ms (09:47:27.410) Trace[2103859957]: ---"Txn call failed" err:context canceled 4998ms (09:47:32.409)] Trace[2103859957]: [5.002959949s] [5.002959949s] END E0407 09:47:32.413599 1 timeout.go:142] post-timeout activity - time-elapsed: 3.705501ms, PUT "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0407 09:47:32.902584 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:47:32.902777 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:47:32.903999 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:47:32.904078 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:47:32.905159 1 trace.go:236] Trace[199878591]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:2679528f-3576-471b-91aa-6c488cface0c,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:47:27.902) (total time: 5002ms): Trace[199878591]: [5.002434655s] [5.002434655s] END E0407 09:47:32.905386 1 timeout.go:142] post-timeout activity - time-elapsed: 2.833309ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0407 09:47:33.019412 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:47:33.019474 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0407 09:47:33.019771 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:47:33.021150 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:47:33.021211 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0407 09:47:33.021234 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:47:33.022332 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:47:33.022399 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:47:33.022399 1 trace.go:236] Trace[65502925]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:b6a1e985-4a21-4d6d-aae2-1984d3c8ce5b,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:47:28.020) (total time: 5001ms): Trace[65502925]: [5.001732961s] [5.001732961s] END E0407 09:47:33.022589 1 timeout.go:142] post-timeout activity - time-elapsed: 2.850389ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: I0407 09:47:33.023633 1 trace.go:236] Trace[835765345]: "Get" accept:application/json, */*,audit-id:01662242-cdb2-482c-bad7-07534377f92a,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:28.019) (total time: 5004ms): Trace[835765345]: [5.004094283s] [5.004094283s] END E0407 09:47:33.023715 1 timeout.go:142] post-timeout activity - time-elapsed: 4.003274ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0407 09:47:33.338639 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:47:33.338965 1 trace.go:236] Trace[1619878174]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:078f8348-472b-4eac-8db7-96e3f3c405dc,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:47:26.335) (total time: 7002ms): Trace[1619878174]: ["GuaranteedUpdate etcd3" audit-id:078f8348-472b-4eac-8db7-96e3f3c405dc,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (09:47:26.336) Trace[1619878174]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (09:47:33.338)] Trace[1619878174]: [7.002954268s] [7.002954268s] END E0407 09:47:34.384155 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:47:34.384385 1 trace.go:236] Trace[121098097]: "Update" accept:application/json, */*,audit-id:cb24876e-0ef0-4445-b4d2-a7d31a1f8433,client:10.0.0.149,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:47:27.381) (total time: 7002ms): Trace[121098097]: ["GuaranteedUpdate etcd3" audit-id:cb24876e-0ef0-4445-b4d2-a7d31a1f8433,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (09:47:27.381) Trace[121098097]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (09:47:34.384)] Trace[121098097]: [7.002777735s] [7.002777735s] END E0407 09:47:34.646896 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:47:34.647279 1 trace.go:236] Trace[832942981]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:7807978d-01a7-454c-8848-c3cb06241077,client:199.204.45.74,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/openstack/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (07-Apr-2026 09:47:27.644) (total time: 7003ms): Trace[832942981]: ["Create etcd3" audit-id:7807978d-01a7-454c-8848-c3cb06241077,key:/events/openstack/horizon-77d98d48d9-vmjpc.18a40a1726547e96,type:*core.Event,resource:events 7002ms (09:47:27.644) Trace[832942981]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (09:47:34.646)] Trace[832942981]: [7.003136927s] [7.003136927s] END E0407 09:47:34.682512 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:47:34.682739 1 trace.go:236] Trace[1688302037]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:9038c82c-baae-46bc-bd4a-b172871b4e6d,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (07-Apr-2026 09:47:27.679) (total time: 7002ms): Trace[1688302037]: ["GuaranteedUpdate etcd3" audit-id:9038c82c-baae-46bc-bd4a-b172871b4e6d,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (09:47:27.680) Trace[1688302037]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (09:47:34.682)] Trace[1688302037]: [7.002759779s] [7.002759779s] END I0407 09:47:34.937881 1 trace.go:236] Trace[1152552082]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:d2ab34e3-e338-4b8e-9823-184189e72668,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:47:32.512) (total time: 2425ms): Trace[1152552082]: ["GuaranteedUpdate etcd3" audit-id:d2ab34e3-e338-4b8e-9823-184189e72668,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 2425ms (09:47:32.512) Trace[1152552082]: ---"Txn call completed" 2424ms (09:47:34.937)] Trace[1152552082]: [2.42570848s] [2.42570848s] END I0407 09:47:34.939077 1 trace.go:236] Trace[1078977440]: "Get" accept:application/json, */*,audit-id:b6506249-0ce1-46b5-98ad-9bab538e3352,client:10.0.0.133,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:33.971) (total time: 968ms): Trace[1078977440]: ---"About to write a response" 967ms (09:47:34.938) Trace[1078977440]: [968.021178ms] [968.021178ms] END I0407 09:47:34.939541 1 trace.go:236] Trace[1028122897]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:353e223a-132c-4cf0-b809-7a66d9d1f6dd,client:10.0.0.186,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:32.411) (total time: 2527ms): Trace[1028122897]: ---"About to write a response" 2527ms (09:47:34.939) Trace[1028122897]: [2.527944382s] [2.527944382s] END I0407 09:47:34.941341 1 trace.go:236] Trace[1349041516]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:5e26d8ad-6051-4196-80ca-d25a33d8a48f,client:10.0.0.186,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:47:27.051) (total time: 7889ms): Trace[1349041516]: ---"About to write a response" 7887ms (09:47:34.938) Trace[1349041516]: [7.889680207s] [7.889680207s] END I0407 09:47:34.941557 1 trace.go:236] Trace[479952677]: "Get" accept:application/json, */*,audit-id:fed104cc-2200-49a9-8086-da4cdfcfb69e,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:47:28.020) (total time: 6920ms): Trace[479952677]: ---"About to write a response" 6920ms (09:47:34.941) Trace[479952677]: [6.92081096s] [6.92081096s] END I0407 09:47:34.941814 1 trace.go:236] Trace[2127432401]: "Get" accept:application/json, */*,audit-id:ff5ae7fb-5335-43b7-9e61-2d27c47e29d8,client:10.0.0.206,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:28.018) (total time: 6922ms): Trace[2127432401]: ---"About to write a response" 6922ms (09:47:34.941) Trace[2127432401]: [6.922942789s] [6.922942789s] END I0407 09:47:34.942151 1 trace.go:236] Trace[184200268]: "Get" accept:application/json, */*,audit-id:a38300ca-5861-4559-a286-83a715a48f73,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:47:27.986) (total time: 6955ms): Trace[184200268]: ---"About to write a response" 6955ms (09:47:34.941) Trace[184200268]: [6.95568412s] [6.95568412s] END I0407 09:47:34.942628 1 trace.go:236] Trace[347201794]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:ecf860c5-c17f-4669-b33c-bbc12901aa00,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:47:33.341) (total time: 1601ms): Trace[347201794]: ["GuaranteedUpdate etcd3" audit-id:ecf860c5-c17f-4669-b33c-bbc12901aa00,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 1601ms (09:47:33.341) Trace[347201794]: ---"Txn call completed" 1599ms (09:47:34.941)] Trace[347201794]: [1.601253191s] [1.601253191s] END I0407 09:47:34.943456 1 trace.go:236] Trace[1959856825]: "Get" accept:application/json, */*,audit-id:c344c0fb-f079-44a7-a6f9-bb98a1607a68,client:10.0.0.47,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.12.17 (linux/amd64) cert-manager/37b853ff34a6c093e946c657c189f40413c0f628,verb:GET (07-Apr-2026 09:47:32.846) (total time: 2096ms): Trace[1959856825]: ---"About to write a response" 2096ms (09:47:34.943) Trace[1959856825]: [2.096732396s] [2.096732396s] END I0407 09:47:34.944751 1 trace.go:236] Trace[864516113]: "Get" accept:application/json, */*,audit-id:3dfbb97d-eaca-4400-a603-9a12954e8d68,client:10.0.0.149,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:34.386) (total time: 558ms): Trace[864516113]: ---"About to write a response" 558ms (09:47:34.944) Trace[864516113]: [558.14971ms] [558.14971ms] END I0407 09:47:34.945384 1 trace.go:236] Trace[873160442]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:5d347046-c510-4889-85ed-c29867a4d9fa,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:47:32.410) (total time: 2534ms): Trace[873160442]: ---"About to write a response" 2534ms (09:47:34.945) Trace[873160442]: [2.534837817s] [2.534837817s] END I0407 09:47:34.951186 1 trace.go:236] Trace[1550072864]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.74,type:*v1.Endpoints,resource:apiServerIPInfo (07-Apr-2026 09:47:34.089) (total time: 861ms): Trace[1550072864]: ---"initial value restored" 849ms (09:47:34.938) Trace[1550072864]: [861.320124ms] [861.320124ms] END I0407 09:48:06.074951 1 trace.go:236] Trace[1653425762]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:d422a1d2-6937-470f-8540-a2dbb06bacd8,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:48:01.074) (total time: 5000ms): Trace[1653425762]: ["GuaranteedUpdate etcd3" audit-id:d422a1d2-6937-470f-8540-a2dbb06bacd8,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 5000ms (09:48:01.074) Trace[1653425762]: ---"Txn call failed" err:context deadline exceeded 4999ms (09:48:06.074)] Trace[1653425762]: [5.000212314s] [5.000212314s] END E0407 09:48:06.074960 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 8.1µs, panicked: false, err: context deadline exceeded, panic-reason: E0407 09:48:06.075163 1 wrap.go:54] timeout or abort while handling: method=PUT URI="/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io?timeout=5s" audit-ID="d422a1d2-6937-470f-8540-a2dbb06bacd8" E0407 09:48:06.075199 1 timeout.go:142] post-timeout activity - time-elapsed: 3.74µs, PUT "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0407 09:48:06.078992 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:06.079019 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:06.079122 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 59.731µs, panicked: false, err: context canceled, panic-reason: E0407 09:48:06.080370 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:06.080426 1 trace.go:236] Trace[1868945772]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:1c56a64d-6a6f-419d-b239-f7ab7e85cddf,client:10.0.0.186,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:48:01.079) (total time: 5001ms): Trace[1868945772]: ["GuaranteedUpdate etcd3" audit-id:1c56a64d-6a6f-419d-b239-f7ab7e85cddf,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (09:48:01.079) Trace[1868945772]: ---"Txn call failed" err:context canceled 4999ms (09:48:06.079)] Trace[1868945772]: [5.00133816s] [5.00133816s] END E0407 09:48:06.080493 1 timeout.go:142] post-timeout activity - time-elapsed: 1.529161ms, PUT "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0407 09:48:06.105150 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0407 09:48:06.105268 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:06.106444 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:06.106503 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:06.107586 1 trace.go:236] Trace[249961404]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:7a7daaf7-65d1-43bc-afb8-e25ac9a66a73,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:48:01.105) (total time: 5002ms): Trace[249961404]: [5.002527685s] [5.002527685s] END E0407 09:48:06.107683 1 timeout.go:142] post-timeout activity - time-elapsed: 2.359952ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0407 09:48:06.150155 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0407 09:48:06.150278 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:06.151316 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:06.151442 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:06.152487 1 trace.go:236] Trace[1708726319]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:4592bb95-42ec-4410-ae99-af68ec8ce870,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:48:01.150) (total time: 5002ms): Trace[1708726319]: [5.002068738s] [5.002068738s] END E0407 09:48:06.152666 1 timeout.go:142] post-timeout activity - time-elapsed: 2.348212ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0407 09:48:06.157101 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:06.157204 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:06.158845 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:06.158898 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:06.160027 1 trace.go:236] Trace[741016199]: "Get" accept:application/json, */*,audit-id:a6820933-e98e-459f-a2da-643d6f5bdf7a,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:48:01.156) (total time: 5003ms): Trace[741016199]: [5.003398325s] [5.003398325s] END E0407 09:48:06.160147 1 timeout.go:142] post-timeout activity - time-elapsed: 3.150492ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0407 09:48:07.315279 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:48:07.315587 1 trace.go:236] Trace[312779531]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:69dd422c-ab53-4801-a4c7-8573654079de,client:199.204.45.74,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/horizon-77d98d48d9-8tvvv/status,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (07-Apr-2026 09:48:00.306) (total time: 7008ms): Trace[312779531]: ["GuaranteedUpdate etcd3" audit-id:69dd422c-ab53-4801-a4c7-8573654079de,key:/pods/openstack/horizon-77d98d48d9-8tvvv,type:*core.Pod,resource:pods 7008ms (09:48:00.307) Trace[312779531]: ---"Txn call failed" err:etcdserver: request timed out 7003ms (09:48:07.315)] Trace[312779531]: [7.008518748s] [7.008518748s] END E0407 09:48:07.513181 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:48:07.513457 1 trace.go:236] Trace[532757774]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:a62e0d15-37ab-48e8-b149-6acc0f4e8e5a,client:10.0.0.186,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:48:00.510) (total time: 7002ms): Trace[532757774]: [7.00245475s] [7.00245475s] END E0407 09:48:08.082500 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:48:08.082705 1 trace.go:236] Trace[2050416341]: "Update" accept:application/json, */*,audit-id:dd344ae5-4a52-4211-8eb4-2af60ee7aa5a,client:10.0.0.149,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:48:01.079) (total time: 7002ms): Trace[2050416341]: ["GuaranteedUpdate etcd3" audit-id:dd344ae5-4a52-4211-8eb4-2af60ee7aa5a,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (09:48:01.080) Trace[2050416341]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (09:48:08.082)] Trace[2050416341]: [7.002927845s] [7.002927845s] END E0407 09:48:09.681296 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:48:09.681666 1 trace.go:236] Trace[1391945010]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:32238655-021f-47a9-812a-97064980ca59,client:199.204.45.74,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/openstack/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (07-Apr-2026 09:48:02.678) (total time: 7002ms): Trace[1391945010]: ["Create etcd3" audit-id:32238655-021f-47a9-812a-97064980ca59,key:/events/openstack/horizon-77d98d48d9-vmjpc.18a40a1f4f1232be,type:*core.Event,resource:events 7002ms (09:48:02.679)] Trace[1391945010]: [7.002794114s] [7.002794114s] END E0407 09:48:11.074426 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:11.076600 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:11.076632 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:11.078676 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:11.078742 1 trace.go:236] Trace[1698008888]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:807d53e6-4024-44e5-aceb-26d3ff9f9db9,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:48:06.076) (total time: 5002ms): Trace[1698008888]: [5.002454274s] [5.002454274s] END E0407 09:48:11.078929 1 timeout.go:142] post-timeout activity - time-elapsed: 2.272591ms, GET "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0407 09:48:11.079076 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:11.080815 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:11.080843 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:11.081913 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:11.081985 1 trace.go:236] Trace[1657122252]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:a15b1454-ba14-4c6c-aec0-ffd4dc4834f5,client:10.0.0.186,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:48:06.079) (total time: 5002ms): Trace[1657122252]: [5.002201281s] [5.002201281s] END E0407 09:48:11.082114 1 timeout.go:142] post-timeout activity - time-elapsed: 3.048141ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0407 09:48:11.099784 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:11.099890 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:11.101098 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:11.101154 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:11.102666 1 trace.go:236] Trace[710314754]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4aa1e80a-da8e-4d8e-8562-7039dfb25a8a,client:10.0.0.186,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:48:07.527) (total time: 3575ms): Trace[710314754]: [3.575104795s] [3.575104795s] END E0407 09:48:11.103826 1 timeout.go:142] post-timeout activity - time-elapsed: 4.077096ms, GET "/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0" result: E0407 09:48:11.104863 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:11.104945 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:11.106063 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:11.106160 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:11.107203 1 trace.go:236] Trace[1988886134]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:8aa4456c-17eb-4696-bdec-1961f8e1ffe6,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:48:08.106) (total time: 3000ms): Trace[1988886134]: [3.00028023s] [3.00028023s] END E0407 09:48:11.107330 1 timeout.go:142] post-timeout activity - time-elapsed: 2.414943ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0407 09:48:11.114541 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:11.114779 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:11.115220 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:11.115755 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:11.116178 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:11.116377 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:11.118273 1 trace.go:236] Trace[934924261]: "Get" accept:application/json, */*,audit-id:46b2ae49-2171-425a-88df-899d74f75d7e,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:48:01.113) (total time: 10004ms): Trace[934924261]: [10.004926477s] [10.004926477s] END E0407 09:48:11.118295 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:11.118375 1 timeout.go:142] post-timeout activity - time-elapsed: 3.884152ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock" result: E0407 09:48:11.118401 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:11.119588 1 trace.go:236] Trace[160085918]: "Get" accept:application/json, */*,audit-id:34947564-3de7-4cd7-aa9a-789c3023a285,client:10.0.0.206,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:48:01.113) (total time: 10006ms): Trace[160085918]: [10.006012871s] [10.006012871s] END E0407 09:48:11.119699 1 timeout.go:142] post-timeout activity - time-elapsed: 4.444929ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election" result: E0407 09:48:11.123960 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:11.124020 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:11.125114 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:11.125159 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:11.126305 1 trace.go:236] Trace[601838809]: "Get" accept:application/json, */*,audit-id:5819b323-7837-46a1-9345-6330f88b8b82,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (07-Apr-2026 09:48:01.123) (total time: 10002ms): Trace[601838809]: [10.002370272s] [10.002370272s] END E0407 09:48:11.126416 1 timeout.go:142] post-timeout activity - time-elapsed: 2.430883ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock" result: E0407 09:48:11.151405 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:11.151513 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:11.152537 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:11.152603 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:11.153620 1 trace.go:236] Trace[136532994]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:2681b1d8-eebf-4a63-9e69-9841700f2655,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (07-Apr-2026 09:48:08.153) (total time: 3000ms): Trace[136532994]: [3.000494593s] [3.000494593s] END E0407 09:48:11.153689 1 timeout.go:142] post-timeout activity - time-elapsed: 2.369871ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0407 09:48:11.157259 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:11.157356 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:11.158917 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:11.158979 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:11.160361 1 trace.go:236] Trace[2100336503]: "Get" accept:application/json, */*,audit-id:1d798196-45a6-4c8b-969e-769f032bb5a2,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:48:08.158) (total time: 3001ms): Trace[2100336503]: [3.001893052s] [3.001893052s] END E0407 09:48:11.160521 1 timeout.go:142] post-timeout activity - time-elapsed: 3.329144ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0407 09:48:11.980222 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:48:11.980403 1 trace.go:236] Trace[117780077]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:9f4febd8-f2b5-45d7-b66e-1924734c9e36,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (07-Apr-2026 09:48:04.977) (total time: 7002ms): Trace[117780077]: ["GuaranteedUpdate etcd3" audit-id:9f4febd8-f2b5-45d7-b66e-1924734c9e36,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (09:48:04.977) Trace[117780077]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (09:48:11.980)] Trace[117780077]: [7.002506577s] [7.002506577s] END E0407 09:48:12.702825 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:48:12.703015 1 trace.go:236] Trace[292477699]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:93b484a8-9fbe-438c-86ce-626903416911,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:48:05.699) (total time: 7003ms): Trace[292477699]: ["GuaranteedUpdate etcd3" audit-id:93b484a8-9fbe-438c-86ce-626903416911,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (09:48:05.699) Trace[292477699]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (09:48:12.702)] Trace[292477699]: [7.003107709s] [7.003107709s] END E0407 09:48:12.720880 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0407 09:48:12.721049 1 trace.go:236] Trace[2065986550]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:56f2169a-d521-43e0-ba3a-df31d80b0712,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:48:05.718) (total time: 7002ms): Trace[2065986550]: ["GuaranteedUpdate etcd3" audit-id:56f2169a-d521-43e0-ba3a-df31d80b0712,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (09:48:05.718) Trace[2065986550]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (09:48:12.720)] Trace[2065986550]: [7.002800693s] [7.002800693s] END E0407 09:48:12.722059 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" E0407 09:48:13.463895 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0407 09:48:13.464035 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0407 09:48:13.465000 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0407 09:48:13.465159 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0407 09:48:13.467169 1 trace.go:236] Trace[577179731]: "Get" accept:application/json, */*,audit-id:3efb87fb-e0ba-4b85-8fb5-48b98bb0f2b6,client:10.0.0.149,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:48:08.085) (total time: 5381ms): Trace[577179731]: [5.381751955s] [5.381751955s] END E0407 09:48:13.467393 1 timeout.go:142] post-timeout activity - time-elapsed: 3.477777ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election" result: I0407 09:48:14.829864 1 trace.go:236] Trace[160036675]: "Update" accept:application/json, */*,audit-id:5c56a6ca-9932-4012-8c78-af9213d5588e,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (07-Apr-2026 09:48:11.124) (total time: 3705ms): Trace[160036675]: ["GuaranteedUpdate etcd3" audit-id:5c56a6ca-9932-4012-8c78-af9213d5588e,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 3704ms (09:48:11.124) Trace[160036675]: ---"Txn call completed" 3703ms (09:48:14.829)] Trace[160036675]: [3.705084371s] [3.705084371s] END I0407 09:48:14.830756 1 trace.go:236] Trace[1599714900]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:85873812-8910-4685-833d-bc2b1f0e62ab,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (07-Apr-2026 09:48:11.151) (total time: 3678ms): Trace[1599714900]: ["GuaranteedUpdate etcd3" audit-id:85873812-8910-4685-833d-bc2b1f0e62ab,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 3678ms (09:48:11.151) Trace[1599714900]: ---"Txn call completed" 3678ms (09:48:14.830)] Trace[1599714900]: [3.678944218s] [3.678944218s] END I0407 09:48:14.833958 1 trace.go:236] Trace[1820535590]: "Get" accept:application/json,audit-id:69d3f813-5422-4b58-921a-7c672c89ed85,client:199.204.45.74,protocol:HTTP/1.1,resource:namespaces,scope:resource,url:/api/v1/namespaces/monitoring,user-agent:OpenAPI-Generator/35.0.0/python,verb:GET (07-Apr-2026 09:48:13.673) (total time: 1160ms): Trace[1820535590]: ---"About to write a response" 1159ms (09:48:14.833) Trace[1820535590]: [1.160046331s] [1.160046331s] END I0407 09:48:14.834390 1 trace.go:236] Trace[1258087622]: "Update" accept:application/json, */*,audit-id:eccc74de-eea6-44af-a436-c254f822f5e3,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (07-Apr-2026 09:48:11.114) (total time: 3719ms): Trace[1258087622]: ["GuaranteedUpdate etcd3" audit-id:eccc74de-eea6-44af-a436-c254f822f5e3,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 3719ms (09:48:11.115) Trace[1258087622]: ---"Txn call completed" 3718ms (09:48:14.834)] Trace[1258087622]: [3.719437594s] [3.719437594s] END I0407 09:48:14.835097 1 trace.go:236] Trace[1789362823]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:6693367d-dadf-4f9d-9662-cfb8ef99f88f,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:48:12.723) (total time: 2111ms): Trace[1789362823]: ["GuaranteedUpdate etcd3" audit-id:6693367d-dadf-4f9d-9662-cfb8ef99f88f,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 2111ms (09:48:12.723) Trace[1789362823]: ---"Txn call completed" 2110ms (09:48:14.834)] Trace[1789362823]: [2.111905864s] [2.111905864s] END I0407 09:48:14.835448 1 trace.go:236] Trace[1887206833]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:538bbf7f-892f-4c04-9c6f-ac7ca4e26915,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (07-Apr-2026 09:48:12.705) (total time: 2130ms): Trace[1887206833]: ["GuaranteedUpdate etcd3" audit-id:538bbf7f-892f-4c04-9c6f-ac7ca4e26915,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 2130ms (09:48:12.705) Trace[1887206833]: ---"Txn call completed" 2128ms (09:48:14.834)] Trace[1887206833]: [2.130236711s] [2.130236711s] END I0407 09:48:14.835792 1 trace.go:236] Trace[1074845036]: "Get" accept:application/json, */*,audit-id:beceac8d-7ce4-4524-9a69-7385e5dc61cb,client:10.0.0.47,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.12.17 (linux/amd64) cert-manager/37b853ff34a6c093e946c657c189f40413c0f628,verb:GET (07-Apr-2026 09:48:04.972) (total time: 9863ms): Trace[1074845036]: ---"About to write a response" 9863ms (09:48:14.835) Trace[1074845036]: [9.863535181s] [9.863535181s] END I0407 09:48:14.836039 1 trace.go:236] Trace[1637246629]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:d4c11b3f-6f66-4a3c-a29e-121da70aead3,client:199.204.45.74,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/horizon-77d98d48d9-9j9dv,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (07-Apr-2026 09:48:07.320) (total time: 7515ms): Trace[1637246629]: ---"About to write a response" 7514ms (09:48:14.835) Trace[1637246629]: [7.515274371s] [7.515274371s] END I0407 09:48:14.836478 1 trace.go:236] Trace[1257468505]: "Get" accept:application/json, */*,audit-id:38fec1f6-a72a-4db4-af93-06688318d1f3,client:10.0.0.133,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (07-Apr-2026 09:48:04.985) (total time: 9851ms): Trace[1257468505]: ---"About to write a response" 9850ms (09:48:14.836) Trace[1257468505]: [9.851107902s] [9.851107902s] END I0407 09:48:14.837015 1 trace.go:236] Trace[1663647882]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4ebfe6b5-570b-453d-8fb2-d0734d1c06c7,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (07-Apr-2026 09:48:11.982) (total time: 2854ms): Trace[1663647882]: ---"About to write a response" 2853ms (09:48:14.836) Trace[1663647882]: [2.854394236s] [2.854394236s] END E0407 09:48:14.837163 1 controller.go:193] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"apiserver-f3tcohoifagyom4bbi4wgeu7te\": the object has been modified; please apply your changes to the latest version and try again" I0407 09:48:14.838107 1 trace.go:236] Trace[1346247555]: "Update" accept:application/json, */*,audit-id:56c72765-81a5-432a-828e-0de04830a587,client:199.204.45.74,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (07-Apr-2026 09:48:11.157) (total time: 3680ms): Trace[1346247555]: ["GuaranteedUpdate etcd3" audit-id:56c72765-81a5-432a-828e-0de04830a587,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 3679ms (09:48:11.158) Trace[1346247555]: ---"Txn call completed" 3675ms (09:48:14.834)] Trace[1346247555]: [3.680159574s] [3.680159574s] END I0407 09:48:14.841615 1 trace.go:236] Trace[1429880605]: "Create" accept:application/json, */*,audit-id:1825db80-267a-42d5-a60f-42355d3ab388,client:10.0.0.219,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/events,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format,verb:POST (07-Apr-2026 09:48:11.076) (total time: 3765ms): Trace[1429880605]: ["Create etcd3" audit-id:1825db80-267a-42d5-a60f-42355d3ab388,key:/events/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io.18a40a21438cb8ee,type:*core.Event,resource:events 3764ms (09:48:11.077) Trace[1429880605]: ---"TransformToStorage succeeded" 3750ms (09:48:14.827)] Trace[1429880605]: [3.765268912s] [3.765268912s] END I0407 09:48:14.845620 1 trace.go:236] Trace[365825091]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:72ca15cd-ac89-413d-b3ab-ee9bb44bcc51,client:199.204.45.74,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events/kube-apiserver-instance.18a409f08eb79623,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (07-Apr-2026 09:48:09.685) (total time: 5160ms): Trace[365825091]: ["GuaranteedUpdate etcd3" audit-id:72ca15cd-ac89-413d-b3ab-ee9bb44bcc51,key:/events/kube-system/kube-apiserver-instance.18a409f08eb79623,type:*core.Event,resource:events 5159ms (09:48:09.685) Trace[365825091]: ---"initial value restored" 5148ms (09:48:14.834)] Trace[365825091]: [5.160077085s] [5.160077085s] END I0407 09:48:14.849965 1 trace.go:236] Trace[1947632974]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.74,type:*v1.Endpoints,resource:apiServerIPInfo (07-Apr-2026 09:48:04.091) (total time: 10758ms): Trace[1947632974]: ---"initial value restored" 10735ms (09:48:14.827) Trace[1947632974]: [10.75810317s] [10.75810317s] END