Troubleshooting Managed Kubernetes
Symptoms you can hit with a cluster, their causes and the fix.
kubectl says you are not allowed, or asks for credentials
Cause: the token request was refused. 403 "You have no access entry on my-cluster. Ask
someone with eks:CreateAccessEntry to add one." means IAM allowed you but you have no access entry;
a plain 403 means IAM does not allow eks:AccessKubernetesApi on the cluster.
Fix: have someone give you an access entry (Give someone access), or an IAM
allow on eks:AccessKubernetesApi.
kubectl cannot reach the endpoint
Cause: your address is not in the cluster's allowed networks, or the public endpoint is off.
Fix: add your network on the Networking tab, or run kubectl from a machine in the cluster's network (Set API endpoint access).
A node group stays without nodes
Cause: the server type is sold out in the group's zones, or nodes keep failing. When it is sold
out, the cluster's page shows "Node group workers is not getting its nodes." and nodes are added
when it is back. After three failed nodes in a row the group's status is failed, the cluster's
page shows "Node group workers stopped: no nodes are being created.", and no machine is made.
Fix: change the group, for example its server type or zones, or allow a capacity fallback (Change a node group). Any change starts it again.
A node group upgrade is paused
Cause: a PodDisruptionBudget held a node's drain longer than the group's drainTimeoutMinutes,
and the group's forceAfterTimeout is off (with it on, the node's pods are removed anyway). The
group's message starts "Upgrade paused at node" and says what held the drain.
Fix: let the budget allow a pod to move (or turn on forceAfterTimeout), then choose Resume
upgrade (Upgrade a node group).
The control-plane upgrade is refused
Cause: an add-on's version does not support the next minor, a node group is too far behind, or
the control plane is not settled. The 409 message says which: it starts "Upgrade these add-ons
first", "Upgrade node group", or "The control plane is not settled". upgradeBlockers on the
cluster lists only the add-ons in the way.
Fix: update the add-on to the version named, upgrade the node group, or wait until every control-plane machine is ready.
A LoadBalancer Service stays pending
Cause: the Service has a non-TCP port, or an lb.ews.cloud/ annotation is invalid.
Fix: read the Service's events (kubectl describe service) and the load balancer's error on the
cluster's Networking tab (Expose a Service).
Add-ons or snapshots are refused on an older cluster
Cause: the control plane was built before managed add-ons or etcd backups.
Fix: upgrade the control plane to the next minor; it replaces the machines with ones that can.