I0411 02:12:08.978670 1 options.go:220] external host was not specified, using 199.204.45.44 I0411 02:12:08.979690 1 server.go:148] Version: v1.28.13 I0411 02:12:08.979733 1 server.go:150] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" I0411 02:12:09.464205 1 shared_informer.go:311] Waiting for caches to sync for node_authorizer I0411 02:12:09.474075 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. I0411 02:12:09.474100 1 plugins.go:161] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota. I0411 02:12:09.474329 1 instance.go:298] Using reconciler: lease I0411 02:12:09.516981 1 handler.go:275] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager W0411 02:12:09.517011 1 genericapiserver.go:744] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources. I0411 02:12:09.754021 1 handler.go:275] Adding GroupVersion v1 to ResourceManager I0411 02:12:09.754435 1 instance.go:709] API group "internal.apiserver.k8s.io" is not enabled, skipping. I0411 02:12:10.213333 1 instance.go:709] API group "resource.k8s.io" is not enabled, skipping. I0411 02:12:10.226055 1 handler.go:275] Adding GroupVersion authentication.k8s.io v1 to ResourceManager W0411 02:12:10.226082 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1beta1 because it has no resources. W0411 02:12:10.226088 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1alpha1 because it has no resources. I0411 02:12:10.226585 1 handler.go:275] Adding GroupVersion authorization.k8s.io v1 to ResourceManager W0411 02:12:10.226599 1 genericapiserver.go:744] Skipping API authorization.k8s.io/v1beta1 because it has no resources. I0411 02:12:10.227476 1 handler.go:275] Adding GroupVersion autoscaling v2 to ResourceManager I0411 02:12:10.228274 1 handler.go:275] Adding GroupVersion autoscaling v1 to ResourceManager W0411 02:12:10.228288 1 genericapiserver.go:744] Skipping API autoscaling/v2beta1 because it has no resources. W0411 02:12:10.228293 1 genericapiserver.go:744] Skipping API autoscaling/v2beta2 because it has no resources. I0411 02:12:10.230367 1 handler.go:275] Adding GroupVersion batch v1 to ResourceManager W0411 02:12:10.230395 1 genericapiserver.go:744] Skipping API batch/v1beta1 because it has no resources. I0411 02:12:10.231975 1 handler.go:275] Adding GroupVersion certificates.k8s.io v1 to ResourceManager W0411 02:12:10.232003 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1beta1 because it has no resources. W0411 02:12:10.232010 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1alpha1 because it has no resources. I0411 02:12:10.232670 1 handler.go:275] Adding GroupVersion coordination.k8s.io v1 to ResourceManager W0411 02:12:10.232681 1 genericapiserver.go:744] Skipping API coordination.k8s.io/v1beta1 because it has no resources. W0411 02:12:10.232708 1 genericapiserver.go:744] Skipping API discovery.k8s.io/v1beta1 because it has no resources. I0411 02:12:10.233162 1 handler.go:275] Adding GroupVersion discovery.k8s.io v1 to ResourceManager I0411 02:12:10.234545 1 handler.go:275] Adding GroupVersion networking.k8s.io v1 to ResourceManager W0411 02:12:10.234560 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1beta1 because it has no resources. W0411 02:12:10.234564 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1alpha1 because it has no resources. I0411 02:12:10.234955 1 handler.go:275] Adding GroupVersion node.k8s.io v1 to ResourceManager W0411 02:12:10.234966 1 genericapiserver.go:744] Skipping API node.k8s.io/v1beta1 because it has no resources. W0411 02:12:10.234970 1 genericapiserver.go:744] Skipping API node.k8s.io/v1alpha1 because it has no resources. I0411 02:12:10.235651 1 handler.go:275] Adding GroupVersion policy v1 to ResourceManager W0411 02:12:10.235663 1 genericapiserver.go:744] Skipping API policy/v1beta1 because it has no resources. I0411 02:12:10.236914 1 handler.go:275] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager W0411 02:12:10.236925 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources. W0411 02:12:10.236929 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. I0411 02:12:10.237277 1 handler.go:275] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager W0411 02:12:10.237288 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1beta1 because it has no resources. W0411 02:12:10.237291 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. I0411 02:12:10.238768 1 handler.go:275] Adding GroupVersion storage.k8s.io v1 to ResourceManager W0411 02:12:10.238780 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1beta1 because it has no resources. W0411 02:12:10.238783 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1alpha1 because it has no resources. I0411 02:12:10.239809 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager I0411 02:12:10.240609 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta2 to ResourceManager W0411 02:12:10.240621 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources. W0411 02:12:10.240624 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources. I0411 02:12:10.253687 1 handler.go:275] Adding GroupVersion apps v1 to ResourceManager W0411 02:12:10.253714 1 genericapiserver.go:744] Skipping API apps/v1beta2 because it has no resources. W0411 02:12:10.253720 1 genericapiserver.go:744] Skipping API apps/v1beta1 because it has no resources. I0411 02:12:10.254621 1 handler.go:275] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager W0411 02:12:10.254635 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources. W0411 02:12:10.254639 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources. I0411 02:12:10.255121 1 handler.go:275] Adding GroupVersion events.k8s.io v1 to ResourceManager W0411 02:12:10.255134 1 genericapiserver.go:744] Skipping API events.k8s.io/v1beta1 because it has no resources. I0411 02:12:10.295372 1 handler.go:275] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager W0411 02:12:10.295523 1 genericapiserver.go:744] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources. I0411 02:12:10.697931 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0411 02:12:10.697989 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0411 02:12:10.698204 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/etc/kubernetes/pki/apiserver.crt::/etc/kubernetes/pki/apiserver.key" I0411 02:12:10.698524 1 secure_serving.go:213] Serving securely on [::]:6443 I0411 02:12:10.698572 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" I0411 02:12:10.698822 1 apiservice_controller.go:97] Starting APIServiceRegistrationController I0411 02:12:10.698851 1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller I0411 02:12:10.698861 1 controller.go:78] Starting OpenAPI AggregationController I0411 02:12:10.698869 1 customresource_discovery_controller.go:289] Starting DiscoveryController I0411 02:12:10.698918 1 controller.go:80] Starting OpenAPI V3 AggregationController I0411 02:12:10.698981 1 aggregator.go:164] waiting for initial CRD sync... I0411 02:12:10.699007 1 crdregistration_controller.go:111] Starting crd-autoregister controller I0411 02:12:10.699006 1 controller.go:116] Starting legacy_token_tracking_controller I0411 02:12:10.699018 1 shared_informer.go:311] Waiting for caches to sync for crd-autoregister I0411 02:12:10.699022 1 shared_informer.go:311] Waiting for caches to sync for configmaps I0411 02:12:10.699037 1 handler_discovery.go:412] Starting ResourceDiscoveryManager I0411 02:12:10.699097 1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController I0411 02:12:10.699099 1 establishing_controller.go:76] Starting EstablishingController I0411 02:12:10.699132 1 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController I0411 02:12:10.698824 1 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/etc/kubernetes/pki/front-proxy-client.crt::/etc/kubernetes/pki/front-proxy-client.key" I0411 02:12:10.699197 1 naming_controller.go:291] Starting NamingConditionController I0411 02:12:10.699206 1 crd_finalizer.go:266] Starting CRDFinalizer I0411 02:12:10.699319 1 gc_controller.go:78] Starting apiserver lease garbage collector I0411 02:12:10.699364 1 controller.go:85] Starting OpenAPI V3 controller I0411 02:12:10.699478 1 apf_controller.go:374] Starting API Priority and Fairness config controller I0411 02:12:10.699564 1 system_namespaces_controller.go:67] Starting system namespaces controller I0411 02:12:10.699555 1 available_controller.go:423] Starting AvailableConditionController I0411 02:12:10.699603 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller I0411 02:12:10.699608 1 gc_controller.go:78] Starting apiserver lease garbage collector I0411 02:12:10.699993 1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller I0411 02:12:10.700034 1 shared_informer.go:311] Waiting for caches to sync for cluster_authentication_trust_controller I0411 02:12:10.700160 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0411 02:12:10.700006 1 controller.go:134] Starting OpenAPI controller I0411 02:12:10.700286 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0411 02:12:10.764340 1 shared_informer.go:318] Caches are synced for node_authorizer I0411 02:12:10.798986 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller I0411 02:12:10.799066 1 shared_informer.go:318] Caches are synced for configmaps I0411 02:12:10.799276 1 shared_informer.go:318] Caches are synced for crd-autoregister I0411 02:12:10.799405 1 aggregator.go:166] initial CRD sync complete... I0411 02:12:10.799449 1 autoregister_controller.go:141] Starting autoregister controller I0411 02:12:10.799457 1 cache.go:32] Waiting for caches to sync for autoregister controller I0411 02:12:10.799465 1 cache.go:39] Caches are synced for autoregister controller I0411 02:12:10.799733 1 apf_controller.go:379] Running API Priority and Fairness config worker I0411 02:12:10.799755 1 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process I0411 02:12:10.799779 1 cache.go:39] Caches are synced for AvailableConditionController controller I0411 02:12:10.800185 1 shared_informer.go:318] Caches are synced for cluster_authentication_trust_controller I0411 02:12:10.801576 1 controller.go:624] quota admission added evaluator for: namespaces I0411 02:12:10.842877 1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io I0411 02:12:11.705686 1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000 I0411 02:12:11.714518 1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000 I0411 02:12:11.714541 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist. I0411 02:12:12.294193 1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io I0411 02:12:12.344672 1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io I0411 02:12:12.421931 1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"} W0411 02:12:12.430490 1 lease.go:263] Resetting endpoints for master service "kubernetes" to [199.204.45.44] I0411 02:12:12.432261 1 controller.go:624] quota admission added evaluator for: endpoints I0411 02:12:12.439498 1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io I0411 02:12:14.338633 1 controller.go:624] quota admission added evaluator for: serviceaccounts I0411 02:12:14.361617 1 controller.go:624] quota admission added evaluator for: deployments.apps I0411 02:12:14.382102 1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"} I0411 02:12:14.557199 1 controller.go:624] quota admission added evaluator for: daemonsets.apps I0411 02:12:19.778546 1 controller.go:624] quota admission added evaluator for: replicasets.apps I0411 02:12:20.027074 1 controller.go:624] quota admission added evaluator for: controllerrevisions.apps I0411 02:12:41.543084 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0411 02:12:41.566270 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0411 02:12:41.611287 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0411 02:12:41.620809 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0411 02:12:41.646647 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0411 02:12:41.646691 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0411 02:12:41.666026 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0411 02:12:41.685157 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0411 02:12:41.707222 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0411 02:12:41.707337 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0411 02:12:41.733161 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0411 02:12:41.733235 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0411 02:12:41.788752 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0411 02:12:41.789050 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0411 02:12:41.849543 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0411 02:12:41.927734 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0411 02:12:42.113952 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0411 02:12:42.114043 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0411 02:12:42.158181 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0411 02:12:42.187216 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0411 02:12:42.219601 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0411 02:12:42.232710 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0411 02:12:42.979321 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0411 02:12:43.020139 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0411 02:12:43.086864 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0411 02:12:48.602100 1 controller.go:624] quota admission added evaluator for: jobs.batch I0411 02:12:50.456709 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0411 02:12:50.472634 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0411 02:12:50.481178 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0411 02:12:50.489803 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0411 02:12:50.511607 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0411 02:12:50.520783 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0411 02:12:50.534466 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0411 02:12:51.078980 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0411 02:12:52.098159 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0411 02:12:52.695082 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0411 02:12:53.084113 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager W0411 02:12:58.239939 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found E0411 02:12:58.239990 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found I0411 02:12:58.683729 1 controller.go:624] quota admission added evaluator for: ciliumendpoints.cilium.io I0411 02:13:07.745899 1 alloc.go:330] "allocated clusterIPs" service="envoy-gateway-system/envoy-gateway" clusterIPs={"IPv4":"10.102.250.233"} W0411 02:13:07.780442 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.102.250.233:9443: connect: connection refused E0411 02:13:07.780478 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.102.250.233:9443: connect: connection refused E0411 02:13:20.134922 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:20.134970 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:20.134976 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 20.261µs, panicked: false, err: context deadline exceeded, panic-reason: E0411 02:13:20.136484 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:20.136584 1 trace.go:236] Trace[722577248]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:4f5c1e59-7f26-41b7-907b-4dd88a80c9ab,client:199.204.45.44,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (11-Apr-2026 02:13:15.134) (total time: 5001ms): Trace[722577248]: ["GuaranteedUpdate etcd3" audit-id:4f5c1e59-7f26-41b7-907b-4dd88a80c9ab,key:/leases/kube-system/kube-controller-manager,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (02:13:15.134) Trace[722577248]: ---"Txn call failed" err:context deadline exceeded 4998ms (02:13:20.134)] Trace[722577248]: [5.001706615s] [5.001706615s] END E0411 02:13:20.136747 1 timeout.go:142] post-timeout activity - time-elapsed: 1.889424ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0411 02:13:20.370717 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0411 02:13:20.370886 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:20.372326 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:20.372374 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:20.373545 1 trace.go:236] Trace[2042963478]: "Get" accept:application/json, */*,audit-id:cc01f9bb-7d64-439c-8190-ada4ebc1a095,client:199.204.45.44,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (11-Apr-2026 02:13:15.370) (total time: 5002ms): Trace[2042963478]: [5.002512749s] [5.002512749s] END E0411 02:13:20.373751 1 timeout.go:142] post-timeout activity - time-elapsed: 3.036376ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0411 02:13:21.850767 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:21.850810 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:21.850848 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 10.49µs, panicked: false, err: context deadline exceeded, panic-reason: E0411 02:13:21.852135 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:21.852225 1 trace.go:236] Trace[1462021608]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:612de8a3-1703-4ff3-969b-0a12f6f5c572,client:10.0.0.166,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (11-Apr-2026 02:13:16.850) (total time: 5001ms): Trace[1462021608]: ["GuaranteedUpdate etcd3" audit-id:612de8a3-1703-4ff3-969b-0a12f6f5c572,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (02:13:16.851) Trace[1462021608]: ---"Txn call failed" err:context deadline exceeded 4998ms (02:13:21.850)] Trace[1462021608]: [5.001239416s] [5.001239416s] END E0411 02:13:21.852384 1 timeout.go:142] post-timeout activity - time-elapsed: 1.659807ms, PUT "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0411 02:13:21.871606 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:21.871676 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:21.872792 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:21.872840 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:21.874074 1 trace.go:236] Trace[993097806]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:aa0a5902-a54c-4397-9c65-80a0e43316b9,client:199.204.45.44,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (11-Apr-2026 02:13:16.871) (total time: 5002ms): Trace[993097806]: [5.002154402s] [5.002154402s] END E0411 02:13:21.874203 1 timeout.go:142] post-timeout activity - time-elapsed: 2.641165ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0411 02:13:22.137137 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0411 02:13:22.137292 1 trace.go:236] Trace[2113805535]: "Update" accept:application/json, */*,audit-id:4c70a2d9-26df-41af-afd8-76ac94edb0eb,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (11-Apr-2026 02:13:15.134) (total time: 7002ms): Trace[2113805535]: ["GuaranteedUpdate etcd3" audit-id:4c70a2d9-26df-41af-afd8-76ac94edb0eb,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (02:13:15.134) Trace[2113805535]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (02:13:22.137)] Trace[2113805535]: [7.002710846s] [7.002710846s] END E0411 02:13:22.651868 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0411 02:13:22.652033 1 trace.go:236] Trace[94187645]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:cb98f306-33c6-42c3-a8d4-03cb81d9affd,client:199.204.45.44,protocol:HTTP/2.0,resource:nodes,scope:resource,url:/api/v1/nodes/instance/status,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (11-Apr-2026 02:13:15.645) (total time: 7006ms): Trace[94187645]: ["GuaranteedUpdate etcd3" audit-id:cb98f306-33c6-42c3-a8d4-03cb81d9affd,key:/minions/instance,type:*core.Node,resource:nodes 7006ms (02:13:15.645) Trace[94187645]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (02:13:22.651)] Trace[94187645]: [7.006701141s] [7.006701141s] END E0411 02:13:23.153747 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0411 02:13:23.153973 1 trace.go:236] Trace[1660540019]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e0d72380-af0b-4b71-b441-e6d066d9c9a8,client:199.204.45.44,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (11-Apr-2026 02:13:16.151) (total time: 7002ms): Trace[1660540019]: ["GuaranteedUpdate etcd3" audit-id:e0d72380-af0b-4b71-b441-e6d066d9c9a8,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (02:13:16.151) Trace[1660540019]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (02:13:23.153)] Trace[1660540019]: [7.002780371s] [7.002780371s] END E0411 02:13:25.131638 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:25.131836 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:25.132012 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:25.133015 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:25.133097 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0411 02:13:25.133230 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:25.134731 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:25.134846 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:25.134870 1 trace.go:236] Trace[881771283]: "Get" accept:application/json, */*,audit-id:80758fb1-6006-4aac-9e90-4bad37ae85c2,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (11-Apr-2026 02:13:24.140) (total time: 994ms): Trace[881771283]: [994.298273ms] [994.298273ms] END E0411 02:13:25.135037 1 timeout.go:142] post-timeout activity - time-elapsed: 3.636222ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock" result: I0411 02:13:25.136031 1 trace.go:236] Trace[310115202]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:055cabeb-4d09-48c1-84ff-3dfb44ffcb7f,client:199.204.45.44,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (11-Apr-2026 02:13:22.136) (total time: 2999ms): Trace[310115202]: [2.999425311s] [2.999425311s] END E0411 02:13:25.136328 1 timeout.go:142] post-timeout activity - time-elapsed: 4.810136ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0411 02:13:25.156882 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:25.156954 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:25.157054 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:25.157122 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:25.157525 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:25.158124 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:25.158220 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:25.158301 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:25.158327 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:25.158655 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:25.159973 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:25.160132 1 trace.go:236] Trace[1045453871]: "List" accept:application/vnd.kubernetes.protobuf;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json,audit-id:acb17353-98de-4068-9cf1-6ace0dda87cd,client:199.204.45.44,protocol:HTTP/2.0,resource:ciliuml2announcementpolicies,scope:cluster,url:/apis/cilium.io/v2alpha1/ciliuml2announcementpolicies,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/metadata-informers,verb:LIST (11-Apr-2026 02:13:20.631) (total time: 4528ms): Trace[1045453871]: ["List(recursive=true) etcd3" audit-id:acb17353-98de-4068-9cf1-6ace0dda87cd,key:/cilium.io/ciliuml2announcementpolicies,resourceVersion:0,resourceVersionMatch:,limit:500,continue: 4528ms (02:13:20.631)] Trace[1045453871]: [4.52872444s] [4.52872444s] END E0411 02:13:25.160177 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:25.160198 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:25.160231 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0411 02:13:25.161069 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:25.161425 1 trace.go:236] Trace[824932866]: "List" accept:application/vnd.kubernetes.protobuf;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json,audit-id:451b730a-b039-4e88-ad90-9c0986eb5896,client:199.204.45.44,protocol:HTTP/2.0,resource:ciliumnodeconfigs,scope:cluster,url:/apis/cilium.io/v2alpha1/ciliumnodeconfigs,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/metadata-informers,verb:LIST (11-Apr-2026 02:13:20.566) (total time: 4595ms): Trace[824932866]: ["List(recursive=true) etcd3" audit-id:451b730a-b039-4e88-ad90-9c0986eb5896,key:/cilium.io/ciliumnodeconfigs,resourceVersion:0,resourceVersionMatch:,limit:500,continue: 4594ms (02:13:20.566)] Trace[824932866]: [4.595024906s] [4.595024906s] END E0411 02:13:25.161492 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:25.161522 1 trace.go:236] Trace[1826962952]: "List" accept:application/vnd.kubernetes.protobuf;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json,audit-id:ad0fa744-c821-4312-93ca-2d9d35865a9d,client:199.204.45.44,protocol:HTTP/2.0,resource:ciliumexternalworkloads,scope:cluster,url:/apis/cilium.io/v2/ciliumexternalworkloads,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/metadata-informers,verb:LIST (11-Apr-2026 02:13:20.613) (total time: 4548ms): Trace[1826962952]: ["List(recursive=true) etcd3" audit-id:ad0fa744-c821-4312-93ca-2d9d35865a9d,key:/cilium.io/ciliumexternalworkloads,resourceVersion:0,resourceVersionMatch:,limit:500,continue: 4548ms (02:13:20.613)] Trace[1826962952]: [4.54828416s] [4.54828416s] END E0411 02:13:25.161600 1 timeout.go:142] post-timeout activity - time-elapsed: 4.697212ms, GET "/apis/cilium.io/v2alpha1/ciliumnodeconfigs" result: E0411 02:13:25.161556 1 timeout.go:142] post-timeout activity - time-elapsed: 4.667511ms, GET "/apis/cilium.io/v2alpha1/ciliuml2announcementpolicies" result: E0411 02:13:25.161790 1 timeout.go:142] post-timeout activity - time-elapsed: 4.996411ms, GET "/apis/cilium.io/v2/ciliumexternalworkloads" result: I0411 02:13:25.162832 1 trace.go:236] Trace[1316624185]: "List" accept:application/vnd.kubernetes.protobuf;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json;as=PartialObjectMetadataList;g=meta.k8s.io;v=v1,application/json,audit-id:887e1eff-4d95-4fcd-9fa2-8d7f88e78870,client:199.204.45.44,protocol:HTTP/2.0,resource:ciliumpodippools,scope:cluster,url:/apis/cilium.io/v2alpha1/ciliumpodippools,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/metadata-informers,verb:LIST (11-Apr-2026 02:13:20.594) (total time: 4568ms): Trace[1316624185]: ["List(recursive=true) etcd3" audit-id:887e1eff-4d95-4fcd-9fa2-8d7f88e78870,key:/cilium.io/ciliumpodippools,resourceVersion:0,resourceVersionMatch:,limit:500,continue: 4568ms (02:13:20.594)] Trace[1316624185]: [4.568457448s] [4.568457448s] END E0411 02:13:25.163020 1 timeout.go:142] post-timeout activity - time-elapsed: 5.552566ms, GET "/apis/cilium.io/v2alpha1/ciliumpodippools" result: E0411 02:13:25.371309 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:25.371582 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:25.372484 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:25.373195 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:25.374429 1 trace.go:236] Trace[1041019643]: "Get" accept:application/json, */*,audit-id:f5f54a9c-8cac-4173-919d-14d56bac91bb,client:199.204.45.44,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (11-Apr-2026 02:13:22.373) (total time: 3000ms): Trace[1041019643]: [3.000481469s] [3.000481469s] END E0411 02:13:25.374663 1 timeout.go:142] post-timeout activity - time-elapsed: 3.58136ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0411 02:13:25.554004 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0411 02:13:25.554308 1 trace.go:236] Trace[1555972034]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:22c69d17-f741-4b53-90ba-52ee34f34ac3,client:199.204.45.44,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (11-Apr-2026 02:13:18.550) (total time: 7003ms): Trace[1555972034]: ["Create etcd3" audit-id:22c69d17-f741-4b53-90ba-52ee34f34ac3,key:/events/kube-system/coredns-7c96b6546b-t64k2.18a52ba0ff8b5928,type:*core.Event,resource:events 7002ms (02:13:18.551)] Trace[1555972034]: [7.003403548s] [7.003403548s] END E0411 02:13:26.850229 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:26.850390 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:26.851355 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:26.851701 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:26.852857 1 trace.go:236] Trace[841114237]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:ccff735c-205c-459b-b574-11d5f33519bb,client:10.0.0.166,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (11-Apr-2026 02:13:21.851) (total time: 5000ms): Trace[841114237]: [5.000878392s] [5.000878392s] END E0411 02:13:26.852973 1 timeout.go:142] post-timeout activity - time-elapsed: 2.849151ms, GET "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0411 02:13:26.871668 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:26.871749 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:26.872771 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:26.872826 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:26.873893 1 trace.go:236] Trace[1594269394]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:4a3e0826-08a6-4b09-a567-0022da1857c1,client:199.204.45.44,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (11-Apr-2026 02:13:23.873) (total time: 3000ms): Trace[1594269394]: [3.000473416s] [3.000473416s] END E0411 02:13:26.874177 1 timeout.go:142] post-timeout activity - time-elapsed: 2.536591ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0411 02:13:27.056731 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0411 02:13:27.056876 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:27.057908 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:27.057975 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0411 02:13:27.059198 1 trace.go:236] Trace[1688951072]: "Get" accept:application/json, */*,audit-id:87f7ed25-5f9b-4989-ba8d-0b196079fb8e,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (11-Apr-2026 02:13:17.056) (total time: 10002ms): Trace[1688951072]: [10.002683244s] [10.002683244s] END E0411 02:13:27.059344 1 timeout.go:142] post-timeout activity - time-elapsed: 2.576333ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock" result: I0411 02:13:28.669475 1 trace.go:236] Trace[617587188]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:12ec5b86-143d-4953-8432-89293b39a475,client:199.204.45.44,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (11-Apr-2026 02:13:26.872) (total time: 1796ms): Trace[617587188]: ["GuaranteedUpdate etcd3" audit-id:12ec5b86-143d-4953-8432-89293b39a475,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 1796ms (02:13:26.872) Trace[617587188]: ---"Txn call completed" 1795ms (02:13:28.669)] Trace[617587188]: [1.796802181s] [1.796802181s] END I0411 02:13:28.669497 1 trace.go:236] Trace[389209654]: "Update" accept:application/json, */*,audit-id:4aefc1eb-d8ba-431e-afc1-49c3d0aab9df,client:199.204.45.44,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (11-Apr-2026 02:13:25.373) (total time: 3296ms): Trace[389209654]: ["GuaranteedUpdate etcd3" audit-id:4aefc1eb-d8ba-431e-afc1-49c3d0aab9df,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 3295ms (02:13:25.373) Trace[389209654]: ---"Txn call completed" 3294ms (02:13:28.669)] Trace[389209654]: [3.296355941s] [3.296355941s] END I0411 02:13:28.669536 1 trace.go:236] Trace[1153354635]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:d95cbd8f-becb-4da5-81e4-bcb9ac0bf62e,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (11-Apr-2026 02:13:22.313) (total time: 6355ms): Trace[1153354635]: ["GuaranteedUpdate etcd3" audit-id:d95cbd8f-becb-4da5-81e4-bcb9ac0bf62e,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 6355ms (02:13:22.314) Trace[1153354635]: ---"Txn call completed" 6354ms (02:13:28.669)] Trace[1153354635]: [6.355596315s] [6.355596315s] END I0411 02:13:28.669585 1 trace.go:236] Trace[1836158716]: "Update" accept:application/json, */*,audit-id:16387711-a64c-4cc1-96c6-741e065201cd,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (11-Apr-2026 02:13:27.057) (total time: 1611ms): Trace[1836158716]: ["GuaranteedUpdate etcd3" audit-id:16387711-a64c-4cc1-96c6-741e065201cd,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1611ms (02:13:27.058) Trace[1836158716]: ---"Txn call completed" 1610ms (02:13:28.669)] Trace[1836158716]: [1.611678109s] [1.611678109s] END I0411 02:13:28.669954 1 trace.go:236] Trace[256992319]: "Update" accept:application/json, */*,audit-id:f3ad9ce3-f943-494c-96aa-b047468fb427,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (11-Apr-2026 02:13:25.133) (total time: 3536ms): Trace[256992319]: ["GuaranteedUpdate etcd3" audit-id:f3ad9ce3-f943-494c-96aa-b047468fb427,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 3535ms (02:13:25.134) Trace[256992319]: ---"Txn call completed" 3533ms (02:13:28.669)] Trace[256992319]: [3.536223595s] [3.536223595s] END I0411 02:13:28.670125 1 trace.go:236] Trace[2053031616]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:ea3a1a41-9b2d-4014-97a4-fb72ef3e543a,client:199.204.45.44,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (11-Apr-2026 02:13:23.155) (total time: 5514ms): Trace[2053031616]: ["GuaranteedUpdate etcd3" audit-id:ea3a1a41-9b2d-4014-97a4-fb72ef3e543a,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 5514ms (02:13:23.156) Trace[2053031616]: ---"Txn call completed" 5512ms (02:13:28.669)] Trace[2053031616]: [5.514230503s] [5.514230503s] END E0411 02:13:28.676407 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0411 02:13:28.676453 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0411 02:13:28.676464 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 8.18µs, panicked: false, err: context canceled, panic-reason: E0411 02:13:28.677884 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0411 02:13:28.678021 1 timeout.go:142] post-timeout activity - time-elapsed: 1.645947ms, POST "/api/v1/namespaces/kube-system/events" result: I0411 02:13:28.760769 1 trace.go:236] Trace[11648109]: "List(recursive=true) etcd3" audit-id:,key:/cilium.io/ciliumpodippools,resourceVersion:,resourceVersionMatch:,limit:10000,continue: (11-Apr-2026 02:13:20.593) (total time: 8166ms): Trace[11648109]: [8.166875214s] [8.166875214s] END I0411 02:13:28.760838 1 trace.go:236] Trace[1688746779]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:6965c0d1-b9ef-475b-8f81-db0305693162,client:199.204.45.44,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (11-Apr-2026 02:13:25.556) (total time: 3204ms): Trace[1688746779]: ["Create etcd3" audit-id:6965c0d1-b9ef-475b-8f81-db0305693162,key:/events/kube-system/kube-apiserver-instance.18a52ba13ef19949,type:*core.Event,resource:events 3203ms (02:13:25.556) Trace[1688746779]: ---"TransformToStorage succeeded" 3112ms (02:13:28.669) Trace[1688746779]: ---"Txn call succeeded" 91ms (02:13:28.760)] Trace[1688746779]: [3.204516225s] [3.204516225s] END I0411 02:13:28.761028 1 trace.go:236] Trace[1813265084]: "List(recursive=true) etcd3" audit-id:,key:/cilium.io/ciliumnodeconfigs,resourceVersion:,resourceVersionMatch:,limit:10000,continue: (11-Apr-2026 02:13:20.565) (total time: 8195ms): Trace[1813265084]: [8.195244203s] [8.195244203s] END I0411 02:13:28.761265 1 trace.go:236] Trace[1720649774]: "List(recursive=true) etcd3" audit-id:,key:/cilium.io/ciliuml2announcementpolicies,resourceVersion:,resourceVersionMatch:,limit:10000,continue: (11-Apr-2026 02:13:20.630) (total time: 8130ms): Trace[1720649774]: [8.1304867s] [8.1304867s] END I0411 02:13:28.761323 1 trace.go:236] Trace[1117603494]: "List(recursive=true) etcd3" audit-id:,key:/cilium.io/ciliumexternalworkloads,resourceVersion:,resourceVersionMatch:,limit:10000,continue: (11-Apr-2026 02:13:20.612) (total time: 8148ms): Trace[1117603494]: [8.148990971s] [8.148990971s] END I0411 02:13:28.761503 1 trace.go:236] Trace[1052580827]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:579a0eb7-539f-43f2-9a1c-623c8a7c99d1,client:199.204.45.44,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/pods/envoy-gateway-78446f96c9-kcwzb,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (11-Apr-2026 02:13:18.093) (total time: 10668ms): Trace[1052580827]: ---"About to write a response" 10667ms (02:13:28.760) Trace[1052580827]: [10.668014058s] [10.668014058s] END I0411 02:13:28.762161 1 trace.go:236] Trace[1788804095]: "Create" accept:application/json, */*,audit-id:19543bde-6c89-4c2d-b037-4c25f4eb303a,client:10.0.0.166,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/events,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format,verb:POST (11-Apr-2026 02:13:26.852) (total time: 1909ms): Trace[1788804095]: ["Create etcd3" audit-id:19543bde-6c89-4c2d-b037-4c25f4eb303a,key:/events/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io.18a52ba2ee5b1fad,type:*core.Event,resource:events 1907ms (02:13:26.854) Trace[1788804095]: ---"TransformToStorage succeeded" 1814ms (02:13:28.669) Trace[1788804095]: ---"Txn call succeeded" 92ms (02:13:28.761)] Trace[1788804095]: [1.909312108s] [1.909312108s] END I0411 02:13:28.807512 1 trace.go:236] Trace[650008651]: "Get" accept:application/json,audit-id:32cd6ed4-410b-43ed-99f3-36e3dd08b5e8,client:199.204.45.44,protocol:HTTP/2.0,resource:serviceaccounts,scope:resource,url:/api/v1/namespaces/local-path-storage/serviceaccounts/local-path-provisioner,user-agent:Helm/3.11.2,verb:GET (11-Apr-2026 02:13:28.259) (total time: 547ms): Trace[650008651]: [547.68475ms] [547.68475ms] END I0411 02:13:28.808331 1 trace.go:236] Trace[1354354708]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:ce28fb62-5fda-4f25-b62f-05609fbe2e86,client:199.204.45.44,protocol:HTTP/2.0,resource:nodes,scope:resource,url:/api/v1/nodes/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (11-Apr-2026 02:13:22.654) (total time: 6154ms): Trace[1354354708]: ---"About to write a response" 6153ms (02:13:28.807) Trace[1354354708]: [6.154209353s] [6.154209353s] END I0411 02:13:29.097356 1 trace.go:236] Trace[1301063474]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.44,type:*v1.Endpoints,resource:apiServerIPInfo (11-Apr-2026 02:13:22.418) (total time: 6678ms): Trace[1301063474]: ---"initial value restored" 6389ms (02:13:28.807) Trace[1301063474]: ---"Transaction prepared" 120ms (02:13:28.928) Trace[1301063474]: ---"Txn call completed" 169ms (02:13:29.097) Trace[1301063474]: [6.678608939s] [6.678608939s] END I0411 02:13:34.098843 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0411 02:13:34.110797 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0411 02:13:34.122668 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0411 02:13:34.143328 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager" clusterIPs={"IPv4":"10.96.209.243"} I0411 02:13:34.145871 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager-webhook" clusterIPs={"IPv4":"10.102.177.233"} I0411 02:13:34.179549 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0411 02:13:34.208724 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0411 02:13:34.243339 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0411 02:14:04.450931 1 controller.go:624] quota admission added evaluator for: certificates.cert-manager.io I0411 02:14:08.763407 1 controller.go:624] quota admission added evaluator for: certificaterequests.cert-manager.io I0411 02:14:22.200539 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-defaultbackend" clusterIPs={"IPv4":"10.105.204.82"} I0411 02:14:22.208103 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller" clusterIPs={"IPv4":"10.101.221.141"} I0411 02:14:22.216151 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-admission" clusterIPs={"IPv4":"10.110.7.136"} I0411 02:14:22.227174 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-metrics" clusterIPs={"IPv4":"10.107.204.107"} I0411 02:14:28.687755 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.706709 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.725613 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.745970 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.769096 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.789793 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.810984 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.823504 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.858496 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.869901 1 handler.go:275] Adding GroupVersion rabbitmq.com v1alpha1 to ResourceManager I0411 02:14:28.886528 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.904205 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:28.927525 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:30.267962 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0411 02:14:33.792325 1 controller.go:624] quota admission added evaluator for: networkpolicies.networking.k8s.io I0411 02:14:33.800552 1 controller.go:624] quota admission added evaluator for: poddisruptionbudgets.policy I0411 02:14:33.848623 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-messaging-topology-operator-webhook" clusterIPs={"IPv4":"10.101.55.152"} I0411 02:14:33.871843 1 controller.go:624] quota admission added evaluator for: issuers.cert-manager.io I0411 02:14:36.064036 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0411 02:14:36.103289 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0411 02:14:36.712388 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-10-0 to ResourceManager I0411 02:14:36.712450 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-11-0 to ResourceManager I0411 02:14:36.712494 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0411 02:14:43.233528 1 controller.go:624] quota admission added evaluator for: perconaxtradbclusters.pxc.percona.com I0411 02:14:48.877842 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-cluster-operator" clusterIPs={"IPv4":"10.105.81.60"} I0411 02:14:49.152036 1 controller.go:624] quota admission added evaluator for: statefulsets.apps I0411 02:14:49.302607 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy" clusterIPs={"IPv4":"10.99.27.65"} I0411 02:14:49.318245 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-replicas" clusterIPs={"IPv4":"10.100.88.81"} I0411 02:16:21.346667 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-metrics" clusterIPs={"IPv4":"10.97.46.158"} I0411 02:16:24.027486 1 alloc.go:330] "allocated clusterIPs" service="openstack/valkey-metrics" clusterIPs={"IPv4":"10.111.100.122"} I0411 02:16:24.037172 1 alloc.go:330] "allocated clusterIPs" service="openstack/valkey" clusterIPs={"IPv4":"10.99.254.118"} I0411 02:16:30.362930 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak" clusterIPs={"IPv4":"10.108.211.219"} I0411 02:16:30.371679 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak-metrics" clusterIPs={"IPv4":"10.99.245.6"} I0411 02:17:09.521065 1 trace.go:236] Trace[767191534]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:948bb4ee-09e2-40ad-af74-911fd2422241,client:10.0.0.166,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (11-Apr-2026 02:17:08.887) (total time: 633ms): Trace[767191534]: ["GuaranteedUpdate etcd3" audit-id:948bb4ee-09e2-40ad-af74-911fd2422241,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 633ms (02:17:08.887) Trace[767191534]: ---"Txn call completed" 633ms (02:17:09.520)] Trace[767191534]: [633.796197ms] [633.796197ms] END I0411 02:19:00.128330 1 controller.go:624] quota admission added evaluator for: ingresses.networking.k8s.io I0411 02:19:03.865834 1 handler.go:275] Adding GroupVersion nfd.k8s-sigs.io v1alpha1 to ResourceManager I0411 02:19:03.898482 1 handler.go:275] Adding GroupVersion nfd.k8s-sigs.io v1alpha1 to ResourceManager I0411 02:19:07.765514 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0411 02:19:07.778356 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0411 02:19:07.791682 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0411 02:19:07.804746 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0411 02:19:07.812897 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0411 02:19:07.826134 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0411 02:19:07.842276 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0411 02:19:09.549150 1 controller.go:624] quota admission added evaluator for: nodefeatures.nfd.k8s-sigs.io I0411 02:19:19.781831 1 controller.go:624] quota admission added evaluator for: passwords.secretgen.k14s.io I0411 02:19:58.732051 1 controller.go:624] quota admission added evaluator for: secrettemplates.secretgen.carvel.dev I0411 02:20:12.814044 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0411 02:20:12.986092 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0411 02:20:13.043471 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0411 02:20:13.043556 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0411 02:20:13.253269 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0411 02:20:13.540010 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0411 02:20:13.541852 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0411 02:20:13.647485 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0411 02:20:13.661791 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0411 02:20:13.788672 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0411 02:20:53.268065 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus" clusterIPs={"IPv4":"10.106.109.203"} I0411 02:20:53.283071 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-alertmanager" clusterIPs={"IPv4":"10.101.41.36"} I0411 02:20:53.293803 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus-node-exporter" clusterIPs={"IPv4":"10.105.95.69"} I0411 02:20:53.301841 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-grafana" clusterIPs={"IPv4":"10.106.28.33"} I0411 02:20:53.308722 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-kube-state-metrics" clusterIPs={"IPv4":"10.109.243.212"} I0411 02:20:53.318279 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-operator" clusterIPs={"IPv4":"10.104.66.104"} I0411 02:20:53.370472 1 controller.go:624] quota admission added evaluator for: alertmanagers.monitoring.coreos.com I0411 02:20:53.384471 1 controller.go:624] quota admission added evaluator for: prometheusrules.monitoring.coreos.com I0411 02:20:53.424997 1 controller.go:624] quota admission added evaluator for: podmonitors.monitoring.coreos.com I0411 02:20:53.433341 1 controller.go:624] quota admission added evaluator for: servicemonitors.monitoring.coreos.com I0411 02:20:53.451617 1 controller.go:624] quota admission added evaluator for: prometheuses.monitoring.coreos.com W0411 02:20:53.461163 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.461216 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.462335 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.462386 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.462627 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.462650 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.462715 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.462760 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.462916 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.462956 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.462979 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.463009 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.463056 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.463202 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.463315 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.463351 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.463963 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.463992 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.464017 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.464049 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.465249 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.465288 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.466285 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.466310 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.466379 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.466406 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.466331 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.466536 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.467323 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.467348 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused W0411 02:20:53.468691 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused E0411 02:20:53.468722 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.104.66.104:443: connect: connection refused I0411 02:21:03.897605 1 alloc.go:330] "allocated clusterIPs" service="monitoring/loki-gateway" clusterIPs={"IPv4":"10.102.178.14"} I0411 02:21:03.915134 1 alloc.go:330] "allocated clusterIPs" service="monitoring/loki" clusterIPs={"IPv4":"10.101.65.104"} I0411 02:21:08.077094 1 alloc.go:330] "allocated clusterIPs" service="monitoring/goldpinger" clusterIPs={"IPv4":"10.109.210.200"} I0411 02:21:10.763913 1 alloc.go:330] "allocated clusterIPs" service="monitoring/prometheus-pushgateway" clusterIPs={"IPv4":"10.109.19.104"} I0411 02:21:15.351194 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached" clusterIPs={"IPv4":"10.109.222.81"} I0411 02:21:16.418082 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached-metrics" clusterIPs={"IPv4":"10.103.105.33"} I0411 02:21:23.450889 1 controller.go:624] quota admission added evaluator for: rabbitmqclusters.rabbitmq.com I0411 02:21:23.612545 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-keystone" clusterIPs={"IPv4":"10.111.128.120"} E0411 02:22:00.142729 1 upgradeaware.go:425] Error proxying data from client to backend: write tcp 199.204.45.44:46342->199.204.45.44:10250: write: broken pipe I0411 02:22:14.179326 1 alloc.go:330] "allocated clusterIPs" service="openstack/keystone-api" clusterIPs={"IPv4":"10.108.18.205"} I0411 02:22:14.193836 1 controller.go:624] quota admission added evaluator for: cronjobs.batch I0411 02:25:16.098326 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-barbican" clusterIPs={"IPv4":"10.103.218.204"} I0411 02:25:49.354725 1 alloc.go:330] "allocated clusterIPs" service="openstack/barbican-api" clusterIPs={"IPv4":"10.99.189.170"} I0411 02:27:40.992426 1 handler.go:275] Adding GroupVersion objectbucket.io v1alpha1 to ResourceManager I0411 02:27:41.009076 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.022338 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.047520 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.047962 1 handler.go:275] Adding GroupVersion objectbucket.io v1alpha1 to ResourceManager I0411 02:27:41.058380 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.069243 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.082203 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.093076 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.100439 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.109929 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.129380 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.148471 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.168923 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.184876 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.223518 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.257607 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.298791 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:27:41.438099 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0411 02:28:21.998798 1 controller.go:624] quota admission added evaluator for: cephclusters.ceph.rook.io I0411 02:28:22.008570 1 controller.go:624] quota admission added evaluator for: cephobjectstores.ceph.rook.io I0411 02:28:54.921602 1 alloc.go:330] "allocated clusterIPs" service="openstack/rook-ceph-rgw-ceph" clusterIPs={"IPv4":"10.107.61.137"} I0411 02:28:57.199411 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-glance" clusterIPs={"IPv4":"10.105.74.60"} I0411 02:29:25.982404 1 alloc.go:330] "allocated clusterIPs" service="openstack/glance-api" clusterIPs={"IPv4":"10.101.182.134"} I0411 02:33:58.988343 1 alloc.go:330] "allocated clusterIPs" service="openstack/staffeln-api" clusterIPs={"IPv4":"10.102.158.127"} E0411 02:34:08.512442 1 watch.go:287] unable to encode watch object *v1.WatchEvent: http2: stream closed (&streaming.encoderWithAllocator{writer:responsewriter.outerWithCloseNotifyAndFlush{UserProvidedDecorator:(*metrics.ResponseWriterDelegator)(0xc03f0da270), InnerCloseNotifierFlusher:struct { httpsnoop.Unwrapper; http.ResponseWriter; http.Flusher; http.CloseNotifier; http.Pusher }{Unwrapper:(*httpsnoop.rw)(0xc03f0c8320), ResponseWriter:(*httpsnoop.rw)(0xc03f0c8320), Flusher:(*httpsnoop.rw)(0xc03f0c8320), CloseNotifier:(*httpsnoop.rw)(0xc03f0c8320), Pusher:(*httpsnoop.rw)(0xc03f0c8320)}}, encoder:(*versioning.codec)(0xc03d0e9180), memAllocator:(*runtime.Allocator)(0xc01e0f5638)})