Your EKS PVC is stuck Pending, and gp2 is a ghost

We hit this on our own product. Last week we ran an end-to-end test of Adjoint from a genuinely fresh machine: blank Ubuntu box, install the CLI, provision an EKS cluster into a test AWS account, seed a source database, clone it. The cluster came up in twelve minutes. Then we tried to do the most ordinary thing in Kubernetes — create a PersistentVolumeClaim for the pgbench seed job — and spent the next twenty minutes staring at this:

0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims

We built this cluster’s storage layer. We wrote the Terraform that installs the CSI driver. And we still walked into it, because the failure has three layers, every one of them reports as waiting rather than failing, and two of them are stock EKS behavior that will bite anyone standing up a cluster today. This post is the diagnosis we wish we could have searched for.

Layer one: there is no default StorageClass

The PVC in question was the boring kind that every Helm chart in existence writes — a size request and nothing else:

spec:
  accessModes: [ReadWriteOnce]
  resources:
    requests:
      storage: 20Gi

No storageClassName. On most clusters people have touched before, that works, because omitting the class means “use the cluster’s default StorageClass.” Our PVC sat in Pending with this event:

no persistent volumes available for this claim and no storage class is set

Here is the part that surprised us: since Kubernetes 1.30, a new EKS cluster has no default StorageClass at all. Up through 1.29, EKS marked its built-in gp2 class with the storageclass.kubernetes.io/is-default-class annotation. From 1.30 on, it does not — the class still exists, but nothing is the default. AWS documented the change; approximately nobody read it, because nothing breaks loudly. A PVC with no class on a cluster with no default is not an error condition in Kubernetes. It is a valid object waiting for an administrator to hand-create a matching PV — which is to say, on a modern cluster, waiting forever, by design.

So the first diagnostic is one command:

$ kubectl get sc
NAME   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      AGE
gp2    kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   14m

Look for (default) next to a name. On a fresh EKS ≥ 1.30 cluster, it is not there.

Layer two: the class you can see is a ghost

The obvious move from that kubectl get sc output is to set storageClassName: gp2 — it is the only class listed, EKS created it, how wrong can it be?

It cannot provision anything. gp2’s provisioner is kubernetes.io/aws-ebs, the in-tree EBS plugin — storage code that used to be compiled into Kubernetes itself. That plugin was deprecated in 1.17, silently translated to the CSI driver from 1.23 (CSI migration), and deleted from the codebase in 1.27. On any current cluster, kubernetes.io/aws-ebs is a forwarding address: Kubernetes rewrites it to ebs.csi.aws.com and hands the work to the EBS CSI driver.

Which is fine — if the EBS CSI driver is installed. EKS does not install it by default. It is an optional add-on, with its own IAM role to configure, and a fresh eksctl or Terraform cluster that didn’t explicitly ask for it doesn’t have it. So the built-in class forwards to a driver that isn’t there, and a PVC bound to it waits on:

waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com"
or manually created by system administrator

Read that event again. It does not say “no such provisioner.” It says the volume is on its way. Nothing will ever arrive, and nothing times out.

There is even a third layer of camouflage: gp2 uses volumeBindingMode: WaitForFirstConsumer, so until a pod actually schedules against the PVC, the only event you see is the entirely innocent waiting for first consumer to be created before binding. Three different waiting messages, zero errors, no volume.

The uncomfortable summary: every fresh EKS cluster ships exactly one StorageClass, it is not the default, and it does not work. It exists for backward compatibility with volumes from the in-tree era, but nothing about kubectl get sc tells you that. It is a museum piece displayed as if it were inventory.

Layer three: the class that works has a rule of its own

Our clusters don’t use EBS for database volumes at all — clones come from FSx for OpenZFS, because copy-on-write is the entire product. So the working class on an Adjoint cluster is fsx-openzfs, and pinning it got us a new failure, and this one at least had the decency to be explicit:

failed to provision volume with StorageClass "fsx-openzfs":
rpc error: code = InvalidArgument desc = resourceType Volume expects storage capacity to be 1Gi

The FSx for OpenZFS CSI driver treats the PVC’s storage request as a sentinel. Child volumes on an OpenZFS filesystem don’t have their own capacity — they draw from the parent filesystem’s pool, which every volume shares — so the driver demands the request be exactly 1Gi and refuses anything else. Our 20Gi, a perfectly reasonable ask for a 5-million-row pgbench database, was rejected outright. Changed to 1Gi, the volume bound in seconds and happily held ten times that much data.

That rule is documented in the driver, and we had even written it down ourselves during an earlier spike. We still tripped on it, which told us something: a constraint that contradicts a decade of PVC muscle memory (“request what you need”) has to be caught by the platform, not remembered by the user.

The diagnosis table

If your PVC is Pending on EKS, the event text tells you which layer you are in:

Event on the PVC	What it actually means	Fix
`no storage class is set`	No default StorageClass exists (all EKS ≥ 1.30 clusters). The PVC will wait forever.	Set `storageClassName` explicitly, or mark a working class as default.
`waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com"…`	The class forwards to the EBS CSI driver, which is not installed. Waits forever.	Install the `aws-ebs-csi-driver` add-on (with its IAM role), or use a class whose driver exists.
`waiting for first consumer to be created before binding`	Not a storage problem yet — `WaitForFirstConsumer` defers until a pod schedules.	Check the pod’s events; the real cause is behind this one.
`expects storage capacity to be 1Gi`	FSx for OpenZFS sentinel: child volumes must request exactly `1Gi`; real capacity comes from the filesystem.	Set `storage: 1Gi`. Size the parent filesystem instead.

The first two rows apply to any EKS cluster. The last is FSx-specific, but the shape of the lesson isn’t: kubectl get sc is the first command, and the provisioner column matters more than the name column. A class is only as real as the driver behind it.

What we changed

On our side this was a product bug, not a user error, and we shipped the fix the same week: Adjoint clusters now mark fsx-openzfs as the default StorageClass, so the boring chart-shaped PVC — no storageClassName, deployed by someone who has never read an FSx document — lands on the driver that actually works. The 1Gi sentinel is in the troubleshooting docs with the exact error text, because that string is what people search.

The general version, for anyone who hands EKS clusters to other people — a platform team, an operator, a product like ours: the output of kubectl get sc is part of your API. A cluster with no default class and one ghost class is technically functional and practically booby-trapped. Set a default that works, and consider deleting or renaming what doesn’t — the person who inherits the cluster will judge the classes by their names, because the names are all they can see.

Caveats

gp2 is not universally dead. If the EBS CSI add-on is installed, CSI migration forwards kubernetes.io/aws-ebs to it correctly and gp2 provisions volumes fine. The trap is specifically the default state of a fresh cluster: class present, driver absent.
EKS Auto Mode is different. Auto Mode clusters manage storage capability themselves and don’t have this shape of problem. Everything above describes standard EKS with self-managed or managed node groups.
Versions: observed on EKS 1.32 (Kubernetes 1.32) in August 2026. The no-default change applies to clusters created at 1.30 or later; the in-tree plugin removal landed in Kubernetes 1.27. Both are old enough now that “fresh cluster” and “affected cluster” are the same thing.
The twenty minutes was ours. The exact-1Gi rule was in our own spike notes and we hit it anyway. That is the strongest argument we know for fresh-host end-to-end tests: the maintainer’s environment remembers things the customer’s environment has never seen.

AWS: Use Kubernetes volume storage with Amazon EBS — the official page for the EBS CSI add-on, including the IAM setup that is the actual prerequisite for gp2-shaped classes to work. Notably, the page is about installing the driver; it does not lead with the fact that until you do, the built-in class is inert.
Kubernetes blog: Storage in-tree to CSI migration status — the canonical explanation of the migration machinery that turns kubernetes.io/aws-ebs into a forwarding address. Written mid-migration; the deletions it forecasts have long since happened.
Marcin Cuber: Amazon EKS upgrade journey from 1.29 to 1.30 — one of the few write-ups that flagged the default-StorageClass removal at the time. The change shipped in release notes, which is exactly where nobody who hits the symptom looks.

The cluster in this story is what adj cluster provision builds — Adjoint provisions it, seeds it, and now defaults its StorageClass correctly, in your own AWS account.