I0428 16:50:21.464887       1 options.go:220] external host was not specified, using 199.204.45.205
I0428 16:50:21.465834       1 server.go:148] Version: v1.28.13
I0428 16:50:21.465894       1 server.go:150] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0428 16:50:21.891798       1 shared_informer.go:311] Waiting for caches to sync for node_authorizer
I0428 16:50:21.907342       1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0428 16:50:21.907367       1 plugins.go:161] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota.
I0428 16:50:21.907643       1 instance.go:298] Using reconciler: lease
I0428 16:50:21.957280       1 handler.go:275] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager
W0428 16:50:21.957300       1 genericapiserver.go:744] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources.
I0428 16:50:22.217360       1 handler.go:275] Adding GroupVersion  v1 to ResourceManager
I0428 16:50:22.217609       1 instance.go:709] API group "internal.apiserver.k8s.io" is not enabled, skipping.
I0428 16:50:22.700472       1 instance.go:709] API group "resource.k8s.io" is not enabled, skipping.
I0428 16:50:22.722273       1 handler.go:275] Adding GroupVersion authentication.k8s.io v1 to ResourceManager
W0428 16:50:22.722333       1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1beta1 because it has no resources.
W0428 16:50:22.722340       1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1alpha1 because it has no resources.
I0428 16:50:22.722902       1 handler.go:275] Adding GroupVersion authorization.k8s.io v1 to ResourceManager
W0428 16:50:22.722916       1 genericapiserver.go:744] Skipping API authorization.k8s.io/v1beta1 because it has no resources.
I0428 16:50:22.723777       1 handler.go:275] Adding GroupVersion autoscaling v2 to ResourceManager
I0428 16:50:22.724673       1 handler.go:275] Adding GroupVersion autoscaling v1 to ResourceManager
W0428 16:50:22.724695       1 genericapiserver.go:744] Skipping API autoscaling/v2beta1 because it has no resources.
W0428 16:50:22.724700       1 genericapiserver.go:744] Skipping API autoscaling/v2beta2 because it has no resources.
I0428 16:50:22.725866       1 handler.go:275] Adding GroupVersion batch v1 to ResourceManager
W0428 16:50:22.725881       1 genericapiserver.go:744] Skipping API batch/v1beta1 because it has no resources.
I0428 16:50:22.726736       1 handler.go:275] Adding GroupVersion certificates.k8s.io v1 to ResourceManager
W0428 16:50:22.726751       1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1beta1 because it has no resources.
W0428 16:50:22.726755       1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1alpha1 because it has no resources.
I0428 16:50:22.727303       1 handler.go:275] Adding GroupVersion coordination.k8s.io v1 to ResourceManager
W0428 16:50:22.727316       1 genericapiserver.go:744] Skipping API coordination.k8s.io/v1beta1 because it has no resources.
W0428 16:50:22.727401       1 genericapiserver.go:744] Skipping API discovery.k8s.io/v1beta1 because it has no resources.
I0428 16:50:22.727997       1 handler.go:275] Adding GroupVersion discovery.k8s.io v1 to ResourceManager
I0428 16:50:22.729784       1 handler.go:275] Adding GroupVersion networking.k8s.io v1 to ResourceManager
W0428 16:50:22.729799       1 genericapiserver.go:744] Skipping API networking.k8s.io/v1beta1 because it has no resources.
W0428 16:50:22.729803       1 genericapiserver.go:744] Skipping API networking.k8s.io/v1alpha1 because it has no resources.
I0428 16:50:22.730365       1 handler.go:275] Adding GroupVersion node.k8s.io v1 to ResourceManager
W0428 16:50:22.730378       1 genericapiserver.go:744] Skipping API node.k8s.io/v1beta1 because it has no resources.
W0428 16:50:22.730382       1 genericapiserver.go:744] Skipping API node.k8s.io/v1alpha1 because it has no resources.
I0428 16:50:22.731215       1 handler.go:275] Adding GroupVersion policy v1 to ResourceManager
W0428 16:50:22.731230       1 genericapiserver.go:744] Skipping API policy/v1beta1 because it has no resources.
I0428 16:50:22.733103       1 handler.go:275] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager
W0428 16:50:22.733118       1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources.
W0428 16:50:22.733123       1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
I0428 16:50:22.733639       1 handler.go:275] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager
W0428 16:50:22.733655       1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1beta1 because it has no resources.
W0428 16:50:22.733659       1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
I0428 16:50:22.735970       1 handler.go:275] Adding GroupVersion storage.k8s.io v1 to ResourceManager
W0428 16:50:22.735985       1 genericapiserver.go:744] Skipping API storage.k8s.io/v1beta1 because it has no resources.
W0428 16:50:22.735989       1 genericapiserver.go:744] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
I0428 16:50:22.737012       1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager
I0428 16:50:22.738073       1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta2 to ResourceManager
W0428 16:50:22.738087       1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources.
W0428 16:50:22.738092       1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources.
I0428 16:50:22.741284       1 handler.go:275] Adding GroupVersion apps v1 to ResourceManager
W0428 16:50:22.741299       1 genericapiserver.go:744] Skipping API apps/v1beta2 because it has no resources.
W0428 16:50:22.741303       1 genericapiserver.go:744] Skipping API apps/v1beta1 because it has no resources.
I0428 16:50:22.742067       1 handler.go:275] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager
W0428 16:50:22.742080       1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources.
W0428 16:50:22.742085       1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources.
I0428 16:50:22.742639       1 handler.go:275] Adding GroupVersion events.k8s.io v1 to ResourceManager
W0428 16:50:22.742653       1 genericapiserver.go:744] Skipping API events.k8s.io/v1beta1 because it has no resources.
I0428 16:50:22.779627       1 handler.go:275] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager
W0428 16:50:22.779653       1 genericapiserver.go:744] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources.
I0428 16:50:23.223210       1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt"
I0428 16:50:23.223267       1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt"
I0428 16:50:23.223450       1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/etc/kubernetes/pki/apiserver.crt::/etc/kubernetes/pki/apiserver.key"
I0428 16:50:23.223667       1 secure_serving.go:213] Serving securely on [::]:6443
I0428 16:50:23.223696       1 tlsconfig.go:240] "Starting DynamicServingCertificateController"
I0428 16:50:23.223755       1 apf_controller.go:374] Starting API Priority and Fairness config controller
I0428 16:50:23.223868       1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller
I0428 16:50:23.223880       1 shared_informer.go:311] Waiting for caches to sync for cluster_authentication_trust_controller
I0428 16:50:23.223969       1 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/etc/kubernetes/pki/front-proxy-client.crt::/etc/kubernetes/pki/front-proxy-client.key"
I0428 16:50:23.224128       1 apiservice_controller.go:97] Starting APIServiceRegistrationController
I0428 16:50:23.224140       1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
I0428 16:50:23.224186       1 gc_controller.go:78] Starting apiserver lease garbage collector
I0428 16:50:23.224220       1 controller.go:116] Starting legacy_token_tracking_controller
I0428 16:50:23.224231       1 shared_informer.go:311] Waiting for caches to sync for configmaps
I0428 16:50:23.224289       1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt"
I0428 16:50:23.224369       1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt"
I0428 16:50:23.224384       1 gc_controller.go:78] Starting apiserver lease garbage collector
I0428 16:50:23.224444       1 available_controller.go:423] Starting AvailableConditionController
I0428 16:50:23.224453       1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
I0428 16:50:23.224471       1 handler_discovery.go:412] Starting ResourceDiscoveryManager
I0428 16:50:23.223988       1 customresource_discovery_controller.go:289] Starting DiscoveryController
I0428 16:50:23.224489       1 controller.go:78] Starting OpenAPI AggregationController
I0428 16:50:23.224518       1 controller.go:80] Starting OpenAPI V3 AggregationController
I0428 16:50:23.224546       1 system_namespaces_controller.go:67] Starting system namespaces controller
I0428 16:50:23.224719       1 controller.go:134] Starting OpenAPI controller
I0428 16:50:23.224747       1 controller.go:85] Starting OpenAPI V3 controller
I0428 16:50:23.224767       1 naming_controller.go:291] Starting NamingConditionController
I0428 16:50:23.224968       1 establishing_controller.go:76] Starting EstablishingController
I0428 16:50:23.224988       1 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController
I0428 16:50:23.225002       1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController
I0428 16:50:23.225017       1 crd_finalizer.go:266] Starting CRDFinalizer
I0428 16:50:23.225069       1 crdregistration_controller.go:111] Starting crd-autoregister controller
I0428 16:50:23.225081       1 shared_informer.go:311] Waiting for caches to sync for crd-autoregister
I0428 16:50:23.224471       1 aggregator.go:164] waiting for initial CRD sync...
I0428 16:50:23.292575       1 shared_informer.go:318] Caches are synced for node_authorizer
E0428 16:50:23.304356       1 controller.go:146] "Failed to ensure lease exists, will retry" err="namespaces \"kube-system\" not found" interval="200ms"
I0428 16:50:23.323874       1 apf_controller.go:379] Running API Priority and Fairness config worker
I0428 16:50:23.323900       1 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process
I0428 16:50:23.324002       1 shared_informer.go:318] Caches are synced for cluster_authentication_trust_controller
I0428 16:50:23.324227       1 cache.go:39] Caches are synced for APIServiceRegistrationController controller
I0428 16:50:23.324501       1 cache.go:39] Caches are synced for AvailableConditionController controller
I0428 16:50:23.324567       1 shared_informer.go:318] Caches are synced for configmaps
I0428 16:50:23.325664       1 shared_informer.go:318] Caches are synced for crd-autoregister
I0428 16:50:23.325883       1 aggregator.go:166] initial CRD sync complete...
I0428 16:50:23.326027       1 autoregister_controller.go:141] Starting autoregister controller
I0428 16:50:23.326120       1 cache.go:32] Waiting for caches to sync for autoregister controller
I0428 16:50:23.326224       1 cache.go:39] Caches are synced for autoregister controller
I0428 16:50:23.326800       1 controller.go:624] quota admission added evaluator for: namespaces
I0428 16:50:23.349397       1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io
I0428 16:50:24.573500       1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000
I0428 16:50:24.860163       1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000
I0428 16:50:24.860204       1 storage_scheduling.go:111] all system priority classes are created successfully or already exist.
I0428 16:50:25.558335       1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0428 16:50:25.610043       1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
I0428 16:50:25.738481       1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"}
W0428 16:50:25.749635       1 lease.go:263] Resetting endpoints for master service "kubernetes" to [199.204.45.205]
I0428 16:50:25.751069       1 controller.go:624] quota admission added evaluator for: endpoints
I0428 16:50:25.757920       1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0428 16:50:26.576350       1 controller.go:624] quota admission added evaluator for: serviceaccounts
I0428 16:50:27.617551       1 controller.go:624] quota admission added evaluator for: deployments.apps
I0428 16:50:27.661217       1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"}
I0428 16:50:27.864193       1 controller.go:624] quota admission added evaluator for: daemonsets.apps
I0428 16:50:31.417979       1 controller.go:624] quota admission added evaluator for: replicasets.apps
I0428 16:50:31.772634       1 controller.go:624] quota admission added evaluator for: controllerrevisions.apps
I0428 16:50:53.335682       1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager
I0428 16:50:53.360207       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager
I0428 16:50:53.385934       1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager
I0428 16:50:53.394845       1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager
I0428 16:50:53.420151       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager
I0428 16:50:53.420575       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager
I0428 16:50:53.420691       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager
I0428 16:50:53.483803       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager
I0428 16:50:53.483841       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager
I0428 16:50:53.533403       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager
I0428 16:50:53.533475       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager
I0428 16:50:53.565225       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager
I0428 16:50:53.622878       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager
I0428 16:50:53.623018       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager
I0428 16:50:53.650784       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager
I0428 16:50:53.806486       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager
I0428 16:50:53.806574       1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager
I0428 16:50:53.815261       1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager
I0428 16:50:53.894889       1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager
I0428 16:50:53.909233       1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager
I0428 16:50:53.954323       1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager
I0428 16:50:53.967948       1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager
I0428 16:50:54.573404       1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager
I0428 16:50:54.578254       1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager
I0428 16:50:54.659042       1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager
I0428 16:51:00.503771       1 controller.go:624] quota admission added evaluator for: jobs.batch
I0428 16:51:02.231571       1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager
I0428 16:51:02.297100       1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager
I0428 16:51:02.304581       1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager
I0428 16:51:02.313133       1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager
I0428 16:51:02.320739       1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager
I0428 16:51:02.344539       1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager
I0428 16:51:02.558037       1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager
I0428 16:51:03.159794       1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager
I0428 16:51:04.145859       1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager
I0428 16:51:04.949574       1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager
I0428 16:51:05.200051       1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager
W0428 16:51:18.331041       1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found
E0428 16:51:18.331143       1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found
I0428 16:51:19.108201       1 controller.go:624] quota admission added evaluator for: ciliumendpoints.cilium.io
I0428 16:51:27.040112       1 alloc.go:330] "allocated clusterIPs" service="envoy-gateway-system/envoy-gateway" clusterIPs={"IPv4":"10.109.207.208"}
W0428 16:51:27.070790       1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.109.207.208:9443: connect: connection refused
E0428 16:51:27.070838       1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.109.207.208:9443: connect: connection refused
I0428 16:52:33.540378       1 alloc.go:330] "allocated clusterIPs" service="kube-system/ceph-csi-rbd-nodeplugin-http-metrics" clusterIPs={"IPv4":"10.96.50.40"}
I0428 16:52:33.552533       1 alloc.go:330] "allocated clusterIPs" service="kube-system/ceph-csi-rbd-provisioner-http-metrics" clusterIPs={"IPv4":"10.107.171.173"}
I0428 16:53:15.891068       1 trace.go:236] Trace[17413185]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:742ed4d2-d2bc-4cc9-8b3e-40a6a5689081,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Apr-2026 16:53:14.694) (total time: 1196ms):
Trace[17413185]: ["GuaranteedUpdate etcd3" audit-id:742ed4d2-d2bc-4cc9-8b3e-40a6a5689081,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 1196ms (16:53:14.694)
Trace[17413185]:  ---"Txn call completed" 1195ms (16:53:15.890)]
Trace[17413185]: [1.196430989s] [1.196430989s] END
I0428 16:53:15.891124       1 trace.go:236] Trace[1030675995]: "Get" accept:application/json, */*,audit-id:c8b80437-6f83-460e-b6d7-fe0f5b73b207,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/rbd.csi.ceph.com-kube-system,user-agent:cephcsi/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Apr-2026 16:53:13.802) (total time: 2088ms):
Trace[1030675995]: ---"About to write a response" 2088ms (16:53:15.890)
Trace[1030675995]: [2.088713026s] [2.088713026s] END
I0428 16:53:15.891325       1 trace.go:236] Trace[1271043080]: "Get" accept:application/json, */*,audit-id:05c088ec-409c-404a-86ae-af8d82edc15a,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:13.800) (total time: 2090ms):
Trace[1271043080]: ---"About to write a response" 2090ms (16:53:15.891)
Trace[1271043080]: [2.090829798s] [2.090829798s] END
I0428 16:53:15.891454       1 trace.go:236] Trace[2052910347]: "Get" accept:application/json, */*,audit-id:52a0c7d1-4dd3-42d1-820c-9545445b4377,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:14.339) (total time: 1551ms):
Trace[2052910347]: ---"About to write a response" 1551ms (16:53:15.891)
Trace[2052910347]: [1.551761767s] [1.551761767s] END
I0428 16:53:15.892054       1 trace.go:236] Trace[236372375]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:343e84a4-bf1c-4b55-974e-cb7a535329fe,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Apr-2026 16:53:13.797) (total time: 2093ms):
Trace[236372375]: ---"About to write a response" 2093ms (16:53:15.891)
Trace[236372375]: [2.093400203s] [2.093400203s] END
I0428 16:53:15.893143       1 trace.go:236] Trace[1667314505]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:952c4ed7-7110-4382-bdbf-4f9d3ed0886c,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Apr-2026 16:53:15.247) (total time: 645ms):
Trace[1667314505]: ---"About to write a response" 645ms (16:53:15.893)
Trace[1667314505]: [645.319379ms] [645.319379ms] END
I0428 16:53:15.893283       1 trace.go:236] Trace[1506661631]: "Get" accept:application/json, */*,audit-id:2bb14cb7-a29d-4283-9a18-f9ad026104e8,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Apr-2026 16:53:14.540) (total time: 1352ms):
Trace[1506661631]: ---"About to write a response" 1352ms (16:53:15.893)
Trace[1506661631]: [1.35290828s] [1.35290828s] END
I0428 16:53:15.893397       1 trace.go:236] Trace[279708466]: "Get" accept:application/json, */*,audit-id:6291bf34-a129-4d7e-b9ef-a627e935f444,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/external-resizer-rbd-csi-ceph-com,user-agent:csi-resizer/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:14.729) (total time: 1164ms):
Trace[279708466]: ---"About to write a response" 1164ms (16:53:15.893)
Trace[279708466]: [1.164353509s] [1.164353509s] END
I0428 16:53:18.948595       1 trace.go:236] Trace[1496100171]: "Get" accept:application/json, */*,audit-id:51860859-3192-4e90-a886-32e95cf133fe,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/external-attacher-leader-rbd-csi-ceph-com,user-agent:csi-attacher/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:17.454) (total time: 1494ms):
Trace[1496100171]: ---"About to write a response" 1494ms (16:53:18.948)
Trace[1496100171]: [1.494377867s] [1.494377867s] END
I0428 16:53:18.948668       1 trace.go:236] Trace[1646959425]: "Get" accept:application/json, */*,audit-id:4b6f8c88-ba22-4ba0-a5ed-742b4b7bd477,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/rbd-csi-ceph-com,user-agent:csi-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:17.797) (total time: 1150ms):
Trace[1646959425]: ---"About to write a response" 1150ms (16:53:18.948)
Trace[1646959425]: [1.150919584s] [1.150919584s] END
I0428 16:53:18.951768       1 trace.go:236] Trace[1058789030]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:fd97d70b-717f-4bf4-b479-26ef2e233ddc,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Apr-2026 16:53:17.893) (total time: 1058ms):
Trace[1058789030]: ["GuaranteedUpdate etcd3" audit-id:fd97d70b-717f-4bf4-b479-26ef2e233ddc,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 1057ms (16:53:17.893)
Trace[1058789030]:  ---"Txn call completed" 1056ms (16:53:18.951)]
Trace[1058789030]: [1.058055549s] [1.058055549s] END
I0428 16:53:19.544286       1 trace.go:236] Trace[2002119742]: "Get" accept:application/json, */*,audit-id:caa9eff3-83a9-4b9f-a31f-1feb55ce9a06,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:18.142) (total time: 1401ms):
Trace[2002119742]: ---"About to write a response" 1401ms (16:53:19.544)
Trace[2002119742]: [1.401573694s] [1.401573694s] END
I0428 16:53:19.544436       1 trace.go:236] Trace[1678635272]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:f39ec3c0-5a50-4c06-aeef-ea5d1d1303b5,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Apr-2026 16:53:18.096) (total time: 1447ms):
Trace[1678635272]: ["GuaranteedUpdate etcd3" audit-id:f39ec3c0-5a50-4c06-aeef-ea5d1d1303b5,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 1447ms (16:53:18.096)
Trace[1678635272]:  ---"Txn call completed" 1446ms (16:53:19.544)]
Trace[1678635272]: [1.447573801s] [1.447573801s] END
I0428 16:53:19.552393       1 trace.go:236] Trace[1126854026]: "Get" accept:application/json, */*,audit-id:f61772cf-901d-4085-889c-d366cd368216,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/rbd.csi.ceph.com-kube-system,user-agent:cephcsi/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Apr-2026 16:53:18.145) (total time: 1407ms):
Trace[1126854026]: ---"About to write a response" 1406ms (16:53:19.552)
Trace[1126854026]: [1.407009119s] [1.407009119s] END
I0428 16:53:19.552401       1 trace.go:236] Trace[149250432]: "Update" accept:application/json, */*,audit-id:e1156550-51d1-41cd-83f0-85d4ae46cb48,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/external-attacher-leader-rbd-csi-ceph-com,user-agent:csi-attacher/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Apr-2026 16:53:18.950) (total time: 601ms):
Trace[149250432]: ["GuaranteedUpdate etcd3" audit-id:e1156550-51d1-41cd-83f0-85d4ae46cb48,key:/leases/kube-system/external-attacher-leader-rbd-csi-ceph-com,type:*coordination.Lease,resource:leases.coordination.k8s.io 601ms (16:53:18.951)
Trace[149250432]:  ---"Txn call completed" 600ms (16:53:19.552)]
Trace[149250432]: [601.529738ms] [601.529738ms] END
I0428 16:53:19.552684       1 trace.go:236] Trace[1957985793]: "Get" accept:application/json, */*,audit-id:b5eaeb6f-6554-4924-95be-0cbb3f51e7cc,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Apr-2026 16:53:18.144) (total time: 1407ms):
Trace[1957985793]: ---"About to write a response" 1407ms (16:53:19.552)
Trace[1957985793]: [1.407939292s] [1.407939292s] END
I0428 16:53:19.552781       1 trace.go:236] Trace[252935715]: "Update" accept:application/json, */*,audit-id:b973b220-0794-4e35-9f2a-b0401abb7586,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/rbd-csi-ceph-com,user-agent:csi-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Apr-2026 16:53:18.950) (total time: 601ms):
Trace[252935715]: ["GuaranteedUpdate etcd3" audit-id:b973b220-0794-4e35-9f2a-b0401abb7586,key:/leases/kube-system/rbd-csi-ceph-com,type:*coordination.Lease,resource:leases.coordination.k8s.io 601ms (16:53:18.951)
Trace[252935715]:  ---"Txn call completed" 600ms (16:53:19.552)]
Trace[252935715]: [601.754002ms] [601.754002ms] END
I0428 16:53:19.553231       1 trace.go:236] Trace[388746999]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:a77099b9-b24c-4377-813f-e3027f2f0886,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Apr-2026 16:53:18.144) (total time: 1408ms):
Trace[388746999]: ---"About to write a response" 1407ms (16:53:19.552)
Trace[388746999]: [1.408801384s] [1.408801384s] END
I0428 16:53:19.553281       1 trace.go:236] Trace[108795711]: "Get" accept:application/json, */*,audit-id:b634f85b-ef23-46df-a1be-10dc0406928b,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:18.145) (total time: 1408ms):
Trace[108795711]: ---"About to write a response" 1407ms (16:53:19.553)
Trace[108795711]: [1.408160708s] [1.408160708s] END
I0428 16:53:19.553334       1 trace.go:236] Trace[1281474529]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:e6f6216a-3f3a-4cc8-89c9-0a2a826cd982,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Apr-2026 16:53:18.145) (total time: 1407ms):
Trace[1281474529]: ---"About to write a response" 1406ms (16:53:19.552)
Trace[1281474529]: [1.407589194s] [1.407589194s] END
E0428 16:53:30.670939       1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded
E0428 16:53:30.671231       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:30.672092       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:30.672233       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled
E0428 16:53:30.672297       1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded
E0428 16:53:30.672781       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
I0428 16:53:30.673843       1 trace.go:236] Trace[2060661133]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:6553f318-a7c4-4849-a47b-5bdd7e868766,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Apr-2026 16:53:25.670) (total time: 5003ms):
Trace[2060661133]: [5.003138868s] [5.003138868s] END
E0428 16:53:30.673998       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:30.674028       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:30.674119       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:30.674137       1 timeout.go:142] post-timeout activity - time-elapsed: 3.214941ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: <nil>
E0428 16:53:30.676140       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:30.676190       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0428 16:53:30.676225       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
I0428 16:53:30.677511       1 trace.go:236] Trace[507754662]: "Get" accept:application/json, */*,audit-id:9bd87f94-aea7-403e-a9d0-feb6c98aef06,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Apr-2026 16:53:25.672) (total time: 5005ms):
Trace[507754662]: [5.005027504s] [5.005027504s] END
I0428 16:53:30.677529       1 trace.go:236] Trace[1147416108]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:6e86d509-5372-45fe-a18b-323fa9a5795b,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Apr-2026 16:53:25.672) (total time: 5005ms):
Trace[1147416108]: [5.005295241s] [5.005295241s] END
E0428 16:53:30.677784       1 timeout.go:142] post-timeout activity - time-elapsed: 5.380314ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: <nil>
E0428 16:53:30.677998       1 timeout.go:142] post-timeout activity - time-elapsed: 5.823895ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: <nil>
E0428 16:53:31.977092       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:31.977132       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:31.977440       1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 244.637µs, panicked: false, err: context canceled, panic-reason: <nil>
E0428 16:53:31.978443       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
I0428 16:53:31.978690       1 trace.go:236] Trace[1121864829]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:c5c500cb-e483-468d-818c-14750ede9a5a,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Apr-2026 16:53:26.977) (total time: 5000ms):
Trace[1121864829]: ["GuaranteedUpdate etcd3" audit-id:c5c500cb-e483-468d-818c-14750ede9a5a,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 5000ms (16:53:26.977)
Trace[1121864829]:  ---"Txn call failed" err:context canceled 4998ms (16:53:31.977)]
Trace[1121864829]: [5.000940693s] [5.000940693s] END
E0428 16:53:31.978971       1 timeout.go:142] post-timeout activity - time-elapsed: 1.779095ms, PUT "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: <nil>
E0428 16:53:32.596660       1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out
I0428 16:53:32.596921       1 trace.go:236] Trace[238032325]: "Update" accept:application/json, */*,audit-id:c536defa-40fa-4d9e-8ea6-b6a94ee6c420,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Apr-2026 16:53:25.593) (total time: 7003ms):
Trace[238032325]: ["GuaranteedUpdate etcd3" audit-id:c536defa-40fa-4d9e-8ea6-b6a94ee6c420,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (16:53:25.593)
Trace[238032325]:  ---"Txn call failed" err:etcdserver: request timed out 7002ms (16:53:32.596)]
Trace[238032325]: [7.00347287s] [7.00347287s] END
E0428 16:53:35.589222       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled
E0428 16:53:35.589423       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:35.590558       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:35.590636       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
I0428 16:53:35.591938       1 trace.go:236] Trace[1189884678]: "Get" accept:application/json, */*,audit-id:578d0b31-d79a-45b4-b4a7-bad06bb8b925,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:34.600) (total time: 991ms):
Trace[1189884678]: [991.556532ms] [991.556532ms] END
E0428 16:53:35.592178       1 timeout.go:142] post-timeout activity - time-elapsed: 2.902132ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock" result: <nil>
E0428 16:53:35.670996       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled
E0428 16:53:35.671134       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:35.671534       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled
E0428 16:53:35.672674       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:35.672700       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled
E0428 16:53:35.672730       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:35.672706       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:35.673996       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
I0428 16:53:35.674145       1 trace.go:236] Trace[581886291]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:790b04a2-4b15-4d1f-9093-e11cf8fd7197,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Apr-2026 16:53:32.672) (total time: 3001ms):
Trace[581886291]: [3.001587165s] [3.001587165s] END
E0428 16:53:35.674225       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:35.674494       1 timeout.go:142] post-timeout activity - time-elapsed: 3.307713ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: <nil>
E0428 16:53:35.674680       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0428 16:53:35.675126       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:35.676269       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
I0428 16:53:35.676423       1 trace.go:236] Trace[693197392]: "Get" accept:application/json, */*,audit-id:07e664b0-fa6f-41c9-b100-2f0d35ae3978,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Apr-2026 16:53:32.674) (total time: 3001ms):
Trace[693197392]: [3.001973125s] [3.001973125s] END
I0428 16:53:35.676523       1 trace.go:236] Trace[584778760]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:138fb85f-fbad-47b5-8e75-3b01789d5690,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (28-Apr-2026 16:53:32.674) (total time: 3002ms):
Trace[584778760]: [3.002289422s] [3.002289422s] END
E0428 16:53:35.676740       1 timeout.go:142] post-timeout activity - time-elapsed: 4.989354ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: <nil>
E0428 16:53:35.676946       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled
E0428 16:53:35.676984       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled
E0428 16:53:35.677117       1 timeout.go:142] post-timeout activity - time-elapsed: 3.952729ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: <nil>
E0428 16:53:35.678031       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:35.678099       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:35.678675       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:35.679599       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
I0428 16:53:35.679800       1 trace.go:236] Trace[1608887317]: "Get" accept:application/json, */*,audit-id:714ac754-62b7-48e8-a9b1-d6dbb842a3c1,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/rbd.csi.ceph.com-kube-system,user-agent:cephcsi/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Apr-2026 16:53:25.675) (total time: 10004ms):
Trace[1608887317]: [10.00446428s] [10.00446428s] END
E0428 16:53:35.680020       1 timeout.go:142] post-timeout activity - time-elapsed: 3.370413ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/rbd.csi.ceph.com-kube-system" result: <nil>
E0428 16:53:35.679634       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:35.681946       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
I0428 16:53:35.681994       1 trace.go:236] Trace[1511464220]: "Get" accept:application/json, */*,audit-id:be957811-f0c7-4941-858a-e51d04cf3018,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:25.676) (total time: 10005ms):
Trace[1511464220]: [10.005586408s] [10.005586408s] END
E0428 16:53:35.682062       1 timeout.go:142] post-timeout activity - time-elapsed: 4.951534ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock" result: <nil>
E0428 16:53:36.152055       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled
E0428 16:53:36.152183       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:36.153268       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:36.153512       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
I0428 16:53:36.154530       1 trace.go:236] Trace[343379759]: "Get" accept:application/json, */*,audit-id:71f0a365-0e34-4d5b-aa82-976d755a78d5,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/external-resizer-rbd-csi-ceph-com,user-agent:csi-resizer/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:26.151) (total time: 10003ms):
Trace[343379759]: [10.003154818s] [10.003154818s] END
E0428 16:53:36.154767       1 timeout.go:142] post-timeout activity - time-elapsed: 2.643306ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/external-resizer-rbd-csi-ceph-com" result: <nil>
E0428 16:53:36.935887       1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out
I0428 16:53:36.936110       1 trace.go:236] Trace[469204651]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:5033ca41-f242-4237-b026-94d4fc18f6f3,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Apr-2026 16:53:29.932) (total time: 7003ms):
Trace[469204651]: ["GuaranteedUpdate etcd3" audit-id:5033ca41-f242-4237-b026-94d4fc18f6f3,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (16:53:29.932)
Trace[469204651]:  ---"Txn call failed" err:etcdserver: request timed out 7001ms (16:53:36.935)]
Trace[469204651]: [7.003795488s] [7.003795488s] END
E0428 16:53:36.936782       1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out"
E0428 16:53:36.977105       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled
E0428 16:53:36.977177       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0428 16:53:36.978675       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0428 16:53:36.978867       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
I0428 16:53:36.979967       1 trace.go:236] Trace[493544102]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:d775095d-c57f-4df1-94ab-1582d8e0c610,client:10.0.0.219,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (28-Apr-2026 16:53:31.978) (total time: 5001ms):
Trace[493544102]: [5.001341793s] [5.001341793s] END
E0428 16:53:36.980614       1 timeout.go:142] post-timeout activity - time-elapsed: 3.246471ms, GET "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: <nil>
E0428 16:53:37.178266       1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out
I0428 16:53:37.178818       1 trace.go:236] Trace[2135499907]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:104e0980-b745-4a81-b842-f75102362942,client:199.204.45.205,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (28-Apr-2026 16:53:30.174) (total time: 7003ms):
Trace[2135499907]: ["Create etcd3" audit-id:104e0980-b745-4a81-b842-f75102362942,key:/events/kube-system/kube-apiserver-instance.18aa9386cfd83693,type:*core.Event,resource:events 7003ms (16:53:30.175)
Trace[2135499907]:  ---"Txn call failed" err:etcdserver: request timed out 7002ms (16:53:37.178)]
Trace[2135499907]: [7.003903471s] [7.003903471s] END
E0428 16:53:37.290528       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled
I0428 16:53:37.290727       1 trace.go:236] Trace[503789022]: "Get" accept:application/json, */*,audit-id:1a90b7fa-2b11-4d06-8d40-dff57528f99c,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/external-snapshotter-leader-rbd-csi-ceph-com,user-agent:csi-snapshotter/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:26.173) (total time: 11116ms):
Trace[503789022]: [11.116991987s] [11.116991987s] END
E0428 16:53:37.291484       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/external-snapshotter-leader-rbd-csi-ceph-com" audit-ID="1a90b7fa-2b11-4d06-8d40-dff57528f99c"
E0428 16:53:37.291827       1 timeout.go:142] post-timeout activity - time-elapsed: 255.526µs, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/external-snapshotter-leader-rbd-csi-ceph-com" result: <nil>
I0428 16:53:37.303277       1 trace.go:236] Trace[1929658128]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:ec571324-0b2f-4913-bdb7-6b6d1ed8221a,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (28-Apr-2026 16:53:35.672) (total time: 1630ms):
Trace[1929658128]: ["GuaranteedUpdate etcd3" audit-id:ec571324-0b2f-4913-bdb7-6b6d1ed8221a,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 1630ms (16:53:35.672)
Trace[1929658128]:  ---"Txn call completed" 1629ms (16:53:37.303)]
Trace[1929658128]: [1.630640886s] [1.630640886s] END
I0428 16:53:37.303328       1 trace.go:236] Trace[1424642379]: "Update" accept:application/json, */*,audit-id:cac44177-33a3-482a-8cb2-e62f4a1955cf,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Apr-2026 16:53:35.677) (total time: 1625ms):
Trace[1424642379]: ["GuaranteedUpdate etcd3" audit-id:cac44177-33a3-482a-8cb2-e62f4a1955cf,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1625ms (16:53:35.678)
Trace[1424642379]:  ---"Txn call completed" 1623ms (16:53:37.303)]
Trace[1424642379]: [1.625397134s] [1.625397134s] END
I0428 16:53:37.303425       1 trace.go:236] Trace[1479302907]: "Get" accept:application/json, */*,audit-id:f96e045e-e7b5-4f22-a549-9dfcf88a7267,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/external-attacher-leader-rbd-csi-ceph-com,user-agent:csi-attacher/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:29.565) (total time: 7738ms):
Trace[1479302907]: ---"About to write a response" 7738ms (16:53:37.303)
Trace[1479302907]: [7.738262s] [7.738262s] END
I0428 16:53:37.303442       1 trace.go:236] Trace[789412265]: "Update" accept:application/json, */*,audit-id:b5526f59-5d9c-481a-86bf-c7918ef66228,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (28-Apr-2026 16:53:35.673) (total time: 1629ms):
Trace[789412265]: ["GuaranteedUpdate etcd3" audit-id:b5526f59-5d9c-481a-86bf-c7918ef66228,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1629ms (16:53:35.673)
Trace[789412265]:  ---"Txn call completed" 1628ms (16:53:37.303)]
Trace[789412265]: [1.629859705s] [1.629859705s] END
I0428 16:53:37.303445       1 trace.go:236] Trace[266099124]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:cbd19575-8e88-4baf-a3d3-77898de06887,client:199.204.45.205,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (28-Apr-2026 16:53:31.434) (total time: 5868ms):
Trace[266099124]: ["GuaranteedUpdate etcd3" audit-id:cbd19575-8e88-4baf-a3d3-77898de06887,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 5868ms (16:53:31.435)
Trace[266099124]:  ---"Txn call completed" 5867ms (16:53:37.303)]
Trace[266099124]: [5.868535094s] [5.868535094s] END
I0428 16:53:37.303280       1 trace.go:236] Trace[1746085035]: "Update" accept:application/json, */*,audit-id:67f0d3a1-eaa9-4cc9-8c58-688bb6c409f1,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (28-Apr-2026 16:53:35.591) (total time: 1711ms):
Trace[1746085035]: ["GuaranteedUpdate etcd3" audit-id:67f0d3a1-eaa9-4cc9-8c58-688bb6c409f1,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 1711ms (16:53:35.591)
Trace[1746085035]:  ---"Txn call completed" 1710ms (16:53:37.302)]
Trace[1746085035]: [1.711920761s] [1.711920761s] END
I0428 16:53:37.303607       1 trace.go:236] Trace[620615726]: "Get" accept:application/json, */*,audit-id:98a008d8-410a-44c8-b0f8-7de504d6d77f,client:10.0.0.183,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/rbd-csi-ceph-com,user-agent:csi-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (28-Apr-2026 16:53:29.563) (total time: 7739ms):
Trace[620615726]: ---"About to write a response" 7739ms (16:53:37.303)
Trace[620615726]: [7.739613942s] [7.739613942s] END
E0428 16:53:37.303743       1 controller.go:193] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"apiserver-f3tcohoifagyom4bbi4wgeu7te\": the object has been modified; please apply your changes to the latest version and try again"
I0428 16:53:37.318436       1 trace.go:236] Trace[1692418372]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.205,type:*v1.Endpoints,resource:apiServerIPInfo (28-Apr-2026 16:53:25.738) (total time: 11580ms):
Trace[1692418372]: ---"initial value restored" 11564ms (16:53:37.302)
Trace[1692418372]: [11.580133585s] [11.580133585s] END