stagDB Engineering

Benchmarking FSx for OpenZFS for Postgres: throughput vs. provisioned IOPS

Fri, 07 Aug 2026 00:00:00 +0000

FSx for OpenZFS is sized along two independent dimensions, and it bills them separately. Provisioned throughput capacity is a fixed ladder of tiers — 64, 128, 256, 512, 1024 MB/s and up — and it is the expensive lever. Provisioned SSD IOPS is a much cheaper add-on above the baseline that comes with your storage. Writing a Terraform module for a Postgres workload requires picking both, and the console does not indicate which one a given database will be sensitive to.

We run Postgres clones for development environments on FSx for OpenZFS, so we had to answer this for our own default sizing. The question was narrow: at a fixed filesystem, which knob buys Postgres TPS — throughput or IOPS — and is the relation linear?

We measured it. The answer is lopsided enough that it changed how we size filesystems.

Method

One EKS cluster, one FSx for OpenZFS filesystem: 256 GiB, SINGLE_AZ_1, in us-east-2. One Postgres 16 database under CloudNativePG, its data directory on a volume from that filesystem, loaded with pgbench at scale 50 — 5,000,000 accounts, about 0.7 GiB logical.

The important detail is that we did not reprovision anything between tiers. FSx accepts throughput and IOPS changes in place via aws fsx update-file-system; each step applies non-disruptively over roughly 2–10 minutes while the filesystem stays mounted and the database stays up. So every row below is the same database, on the same volume, with the same cache state and the same pod — only the filesystem’s provisioned numbers moved. That removes the usual confound where each configuration gets a freshly loaded database with different physical layout.

At each tier we ran in-pod pgbench twice: select-only (-S) and the default TPC-B-like read/write script, each with 4 clients, 2 threads, 60 s.

Two honesty constraints, up front: one run per tier, 60 s each. These are not distributions. The effects below are large enough to survive that — a 2.5× gap does not come from run-to-run noise — but nothing here supports reasoning about a 5% difference.

Results

Tier (MB/s / IOPS)	select-only TPS	TPC-B TPS	TPC-B avg latency
128 / 1000	15,272	645	6.21 ms
256 / 1000	16,361	699	5.72 ms
512 / 1000	15,398	769	5.20 ms
1024 / 1000	15,770	834	4.80 ms
1024 / 3000	15,865	2,089	1.92 ms
128 / 3000	14,556	1,925	2.08 ms

The first four rows walk throughput across three doublings at fixed IOPS. The fifth changes only IOPS. The sixth is the interesting one: cheapest throughput tier, expensive IOPS.

Findings

1. Throughput scaling is not linear, and its effect is small.

8× the provisioned throughput, 128 → 1024 MB/s, bought +29% TPC-B TPS: 645 → 834. Three doublings of the expensive lever produced less than a third more throughput. The curve rises roughly logarithmically, consistent with marginally lower per-operation queueing rather than any bandwidth constraint being lifted.

That is the expected result once you account for what the workload actually moves. A pgbench TPC-B transaction touches a handful of pages and forces a WAL flush; a 0.7 GiB working set streams nowhere near 128 MB/s, let alone 1024. The read/write workload is bound by sync-write latency per operation, not by bandwidth, so additional bandwidth addresses a constraint that is not binding.

2. Provisioned SSD IOPS is the knob that matters.

3× the IOPS, 1000 → 3000, at fixed 1024 MB/s throughput took TPC-B from 834 → 2,089 TPS — 2.5× — and cut average latency from 4.80 ms to 1.92 ms. That is near-linear in IOPS, which is what a WAL-fsync-bound workload should look like: each commit needs a durable write to land, the rate of durable writes is what IOPS provisions, so TPS tracks it.

Compare the two levers directly. 8× on throughput: +29%. 3× on IOPS: +150%.

3. The cost-optimal shape is cheap throughput plus provisioned IOPS.

The last row is the most useful one for a sizing discussion. 128 MB/s — the lowest tier we tested, the default sizing — combined with 3000 IOPS delivered 1,925 TPS: 92% of the best result we measured, which needed 8× the throughput to add the remaining 8%.

On FSx for OpenZFS pricing, stepping throughput from 128 to 1024 MB/s is the expensive lever; 2,000 extra provisioned IOPS costs a small fraction of that. We are deliberately not printing dollar figures — AWS pricing varies by region and changes — but the ratio is substantial, and it points in the same direction as the performance data. The configuration that performs nearly best is also the less expensive one, which makes the trade-off unusually straightforward.

4. Read-heavy work is insensitive to both knobs.

Select-only sat at roughly 15–16k TPS across every tier, with average latency in the 0.24–0.28 ms range. The spread has no trend in it — the highest select-only number, 16,361 TPS, is at 256 MB/s, in the middle of the ladder, and the lowest is at the tier with the most IOPS.

The reason is that the hot set lives in Postgres shared buffers and the page cache, so the reads mostly never reach FSx. The pod’s CPU is the ceiling. For a read-heavy workload that fits in RAM, neither FSx knob is the bottleneck, and the choice between them has little effect either way.

What this means for sizing

For a write-oriented workload, the data supports scaling provisioned IOPS with expected concurrent write load while holding throughput at a low tier. That runs against the intuition that a “bigger” filesystem means more MB/s, and against what our own default sizing did before this run — we had been scaling throughput with the number of developers and pinning IOPS at 1000, which is the less effective allocation for anything TPC-B-shaped.

Throughput still matters, just not for OLTP. It is the right lever for streaming reads and bulk sequential work: restoring a dump, an initial data load, a large sequential scan that misses cache, a backup drain. Those genuinely move bytes at a rate a 128 MB/s tier will cap. A filesystem serving both — an OLTP database plus a nightly bulk load — is best sized with throughput chosen for the bulk window and IOPS for the transactional steady state; the two are independent enough to reason about separately.

Two practical notes. First, SINGLE_AZ_1 throughput is a fixed list of tiers, not a continuous dial: 64, 128, 256, 512, 1024, 2048, 3072, 4096 MB/s — a request for 200 is not possible, so sizing math needs to be rounded onto that ladder before any decision is made. Second, because updates apply in place and non-disruptively, this is an inexpensive experiment to repeat against a specific workload: a single afternoon of aws fsx update-file-system and pgbench produces a table like the one above for the database actually in use, which is more informative than our numbers.

One operational caution for anyone repeating it: an in-place FSx update puts the filesystem in an optimization window for several minutes, and we saw volume-creation calls against that filesystem stall for the duration. It is non-disruptive to running I/O, but resizing a filesystem is best avoided while anything depends on provisioning new volumes from it on a schedule.

Caveats

One 60 s run per tier. No variance estimate, no repeats, no warm-up discipline beyond the fact that the same database stayed hot across all six tiers. The 2.5× IOPS effect is far too large to be noise; the +29% throughput effect is directionally solid but its exact magnitude is not something we would defend to two significant figures.
Small hot set. At scale 50, roughly 0.7 GiB, the data fits comfortably in cache. That is the entire explanation for the flat read-side numbers, and a working set larger than RAM would change them substantially. It would not change the write-side finding: WAL fsync is a durability cost that a bigger cache does not remove, so the IOPS-over-throughput conclusion should hold — and get stronger — as the database grows.
One Postgres instance, no concurrent cluster load. The filesystem’s provisioned budget is shared across everything on it. These numbers describe one database with the whole budget to itself; a filesystem hosting many active databases splits it, and per-database TPS falls accordingly.
Modest client concurrency. 4 clients / 2 threads is a light load. Higher write concurrency will move the absolute TPS numbers and may change where the IOPS curve stops being near-linear; we did not find that ceiling.
SINGLE_AZ_1, us-east-2, July 2026. Deployment type, region, pricing, and FSx behavior can all change. MULTI_AZ_1 has different write-path characteristics that we did not measure at all.

Clone latency on FSx for OpenZFS: from 10 minutes to 16 seconds

Fri, 07 Aug 2026 00:00:00 +0000

A ZFS clone does not copy data. zfs clone writes a bit of metadata pointing a new dataset at an existing snapshot’s blocks, marks them copy-on-write, and returns. The cost is the same whether the snapshot is 10 MB or 10 TB — you only pay for blocks that later diverge. That property is the reason we picked ZFS: it makes it plausible to give every developer their own full-size copy of production data instead of a gutted 500-row fixture set.

AWS offers that property as a managed service. FSx for OpenZFS runs real OpenZFS, and its CSI driver provisions a volume from a snapshot with CopyStrategy=CLONE, sharing blocks rather than copying them. On paper: a developer asks for a clone of a production-size Postgres database and gets one in seconds, no matter how big it is.

We built that, and the block sharing behaved exactly as documented. The latency did not match the underlying primitive, and characterizing the gap is what the rest of this post is about.

Where the latency actually comes from

Our first milestone spike was deliberately small: EKS 1.30, one FSx for OpenZFS SINGLE_AZ_1 filesystem (256 GiB, 128 MB/s), and a ~10 GB golden snapshot — pgbench at scale 700, 10.98 GB logical. We cloned it into three developer namespaces at once, booted a CloudNativePG Postgres on each, and measured clone-to-Ready.

Three concurrent clones came back at 110 s, 177 s, and 238 s — a p50 of 177 s against a gate of 120 s, and a p95 of 238 s.

The storage half passed convincingly. Amplification for the golden plus three clones was 1.107× — 12.16 GB used where four independent full copies would have been about 44 GB. The blocks were genuinely shared, the cloned data directories byte-identical to the golden, and they replayed WAL cleanly after we killed the pods. Copy-on-write worked as advertised.

So the latency was not data movement, and we confirmed it was size-independent — nothing was being read or written. What we were waiting on was the managed control plane in front of ZFS. FSx’s CreateVolume API takes roughly 75 s for a plain provisioned volume and about 2 minutes when creating from a snapshot. That is the floor, and it is independent of the data involved.

The shape of those three numbers is the more consequential detail. 110, 177, 238 — rising with concurrency, spaced about a minute apart. That is not three requests each taking two minutes in parallel; it is the signature of a queue. We ran the test again at ten concurrent clones to characterize how the queue behaves at higher fan-out:

Concurrent clones	Clone-to-Ready (s)
K=3	110 / 177 / 238
K=10	129 / 198 / 269 / 319 / 381 / 428 / 490 / 550 / 611, plus one that had not finished at 10 min

The inter-completion gap is a near-constant ~60 s. FSx CreateVolume serializes at roughly one clone per minute, so the Nth developer to ask for a database waits about N minutes. At K=10 the p50 was 381 s and the p95 was 611 s.

That characteristic is a poor match for the demand pattern a development-environment tool sees. Developer demand for clones is not uniform; it arrives in a burst when a team starts work in the morning, which is precisely the regime in which a serialized API performs worst. Ten concurrent developers is a small team rather than a stress test, and under this API the last request in line waits about ten minutes for an operation the storage layer itself performs in milliseconds.

An unplanned natural control

We built the fix (below), redeployed, and ran a time-to-first-clone benchmark on a separate live cluster, against a smaller golden — prod-pg16-v3, 86 MB and 750k rows — measuring each Instance’s creationTimestamp to its Ready condition. The size difference does not matter here, and that is the point: the claim path moves no data at all, so its latency is independent of golden size, which is what makes these numbers comparable to the 10 GB measurements above.

Nine of the ten claims were fast, and one was not: a single claim raced a momentarily-empty pool, found nothing warm, and fell through to the on-demand path — a real CreateVolume against FSx. It returned in 600.3 s.

We could not have designed a cleaner control. Same cluster, same golden snapshot, same operator build, same measurement — one request took the old path and every other took the new one, roughly 37× apart, with no confounding variable except which code path it took.

It also reproduced the spike’s finding on a live cluster, months later, without our intending it: 600.3 s is what “serializes at ~1/min” predicts for a request landing behind a queue of pool refills. We excluded it from the percentiles — it has no pool-claim timestamp pair, so it is not a sample of what we were measuring — but it is the most informative number in the run.

The fix: moving the API call out of the hot path

If the bottleneck is a control-plane API you do not control, you cannot make it faster. What you can do is arrange not to be waiting on it while the user is.

So we pre-pay it. A pool controller keeps N clones per golden snapshot already provisioned and idle in a holding namespace, each created ahead of demand by the slow serialized path, on our schedule rather than a developer’s. When a developer runs adj clone, they do not create a clone — they claim one.

The claim path makes zero CreateVolume calls. We verified that directly in a separate spike, counting FSx volumes and CSI calls across a claim: the FSx volume count went 3 → 3, with 0 CSI CreateVolume calls, and the resulting Postgres came up Ready with the golden’s data intact.

What a claim does is move an existing volume between namespaces and convince Postgres to adopt it:

Patch the pre-warmed PV’s reclaim policy to Retain, so releasing it does not destroy the underlying FSx volume.
Delete the holding-namespace PVC. The PV goes to Released; the FSx volume survives.
Strip the PV’s claimRef, returning it to Available.
Create a PVC named <cluster>-1 in the developer’s namespace, bound by volumeName directly to that PV, carrying CloudNativePG’s data-PVC identity metadata — cnpg.io/pvcRole: PG_DATA, cnpg.io/nodeSerial: "1", and critically cnpg.io/pvcStatus: ready.
Create a CloudNativePG Cluster with no bootstrap stanza.

That last pair is the essential part. cnpg.io/pvcStatus: ready is what distinguishes an “adopt me” volume from one still initializing. With it present and no bootstrap stanza, CNPG runs neither initdb nor pg_basebackup — it logs Creating new Pod to reattach a PVC and starts Postgres directly on the existing data directory. No initdb pod is ever created.

We expected to need a specially prepared “cold” golden, since ours is taken as an online volume-snapshot backup and could in principle carry in-progress-backup state. It did not. pg_controldata on the clone showed Backup start location: 0/0, End-of-backup record required: no, Database cluster state: in production. The cloned directory is just a crash-consistent copy of a live PGDATA; Postgres ran ordinary automatic recovery, hit an end-of-recovery checkpoint, and came up. Our existing goldens were directly adoptable.

Measured over the 9-sample benchmark, the claim path gives:

Metric	Result	Gate
Operator time-to-first-clone, p50	16.0 s	< 120 s
Operator time-to-first-clone, p95	25.0 s	< 300 s
Operator min / max	15.0 s / 25.0 s
CLI wall-clock (user-visible)	16.4–26.4 s
n	9

Operator time is the Instance CR’s creationTimestamp to its Ready condition; the CLI wall-clock a developer experiences tracks it about 1 s higher. The remaining ~16 s is not FSx at all — it is the PV rebind (dominated by waiting on the PVC delete) plus CNPG starting a pod and running crash recovery. We moved the two-minute managed-API call out of the hot path, and what is left is Kubernetes and Postgres doing their normal work.

The constraint we did not remove

The pool does not make FSx faster. It moves the wait.

Refill runs at the same serialized rate we measured in the first place: filling a cold pool from 0 to 10 warm clones takes about 580 s, and refilling from 4 to 10 after a benchmark drained it takes about 348 s — both the same ~1 clone/min cadence.

So the honest description of what we built is a buffer. It absorbs a burst up to its warm size at seconds-latency, and beyond that it is bounded by the same API as before. The 600.3 s outlier is not an anomaly; it is what a claim looks like when demand outruns refill — the system’s designed failure mode, observed in practice.

That makes pool size the real operational knob, and it has to be set against expected concurrent-launch rate rather than headcount. A ten-person team that all starts at 9 a.m. needs a bigger warm pool than a thirty-person team spread across time zones. Sizing it from usage telemetry rather than a guess is work we still owe.

Takeaways

A managed control plane can dominate the cost of an operation that is nearly free at the storage layer. zfs clone is a metadata write; wrapped in CreateVolume it becomes a two-minute operation. When evaluating a managed version of a primitive you already understand, it is worth benchmarking the API rather than the primitive — the API is the part being purchased.
Concurrency behavior is more informative than single-shot latency. One clone at 110 s looked like a tuning problem. The same operation at K=10 was a 611 s p95 and an architectural constraint. Only the fan-out showed whether the design worked, and the indicator was the near-constant ~60 s gap between completions.
When an API outside your control is the bottleneck, the remaining lever is removing it from the hot path. We could not make CreateVolume faster, so we arranged for it to happen before anyone was waiting: pre-provision, then claim, with the claim itself kept genuinely cheap. Ours makes zero calls to the slow API, which is why it lands at ~16 s rather than at ~16 s plus an occasional two-minute provisioning call.

Caveats

n=9, and it is a burst benchmark, not sustained load. A real p95 for a burst. Sustained concurrency — many developers launching over a long window, contending on refill — is future work, and we expect it to be governed by the refill rate above rather than by the 16 s claim number.
The 600.3 s on-demand figure is a single observation. It is a clean natural experiment and it agrees with the independently measured ~1 clone/min serialization, but it is one sample, not a distribution.
Filesystem sizing: all numbers come from SINGLE_AZ_1 at 128 MB/s and 1000 IOPS. That matters less than it sounds for the headline result — the claim path moves no data, so the warm-claim numbers are throughput- and IOPS-independent and should hold on larger filesystems. It matters a great deal for what happens after a clone boots: in a separate test, one clone running pgbench alone hit 166 TPS, while ten clones writing concurrently got ~13–19 TPS each, because that budget is a single shared pool split across every clone on the filesystem.
This is FSx behavior as we measured it, June–July 2026, in us-east-2. AWS may have changed CreateVolume since. Anyone evaluating FSx for OpenZFS would be well served by re-running the fan-out test directly — it takes an afternoon, and it is the number that decides the design.

This pooled claim path is now the default way clones are created in Adjoint.

Copy-on-write development databases: ten clones at 1.12× the storage

Fri, 07 Aug 2026 00:00:00 +0000

To ship correct software, developers need data that behaves like production: real volumes, real distributions, real edge cases. The reason almost nobody gets that is arithmetic. A full copy of a multi-TB database costs real time and real money, multiplied by every developer who needs one. The naive approach is O(size × developers), and both terms grow.

So teams generally pick from four options, each of which is a version of the same concession:

What teams do instead	What it costs
Tiny seed or fixture data	The query that returns in 3 ms on 500 rows times out on 50 million. Cardinality-dependent bugs reach production because nothing before production had the cardinality.
One shared staging database	Contention and pollution. No isolation. One destructive migration or one long-running `ALTER TABLE` blocks everyone, so risky work gets scheduled instead of done.
Manual production dumps	Hours per copy, perpetually stale by the time it lands, full storage cost per copy, and unmanaged production data sitting on laptops.
Per-developer cloud copies (e.g. one RDS instance each)	Honest but expensive: at multi-TB × headcount the bill is the point of the exercise, and because refreshing costs the same again, nobody refreshes.

Each of these is a rational response to the multiplication. If you can remove the multiplication, all four choices stop being necessary.

What copy-on-write actually does

ZFS stores a dataset as a tree of blocks. A snapshot does not copy anything — it pins the current block tree so those blocks cannot be freed or overwritten in place. A clone is a new, writable dataset whose root points at that same pinned tree.

From there the accounting is simple. Reads on the clone follow shared blocks. A write on the clone allocates a new block and repoints the clone’s tree at it; the snapshot’s block is untouched, because the snapshot still needs it. Nobody copies the unchanged 99%.

So N clones of a base dataset cost:

base + Σ(divergence per clone)

not N × base. Two consequences fall out of that, and they are the whole argument. First, creating a clone is a metadata operation, so it does not get slower as the dataset gets bigger. Second, storage is charged for what each developer changes, not for what they have. A developer who clones a 2 TB database and runs a migration against three tables pays for three tables’ worth of rewritten blocks.

The question is whether that survives a managed cloud filesystem and a real Postgres on top of it. We measured it on AWS FSx for OpenZFS.

Measured

Our first substrate spike ran on EKS 1.30 with a single FSx for OpenZFS SINGLE_AZ_1 filesystem (256 GiB, 128 MB/s) in us-east-2. The base dataset was a ~10 GB golden — pgbench at scale 700, 10.98 GB logical — snapshotted and cloned into separate developer namespaces, with a CloudNativePG Postgres booted on each clone.

Golden + 3 clones: 1.107×. Total storage consumed was 12.16 GB, against roughly 44 GB for four independent full copies of the same data. That is the golden plus three clones, on the ~10 GB dataset described above.

Golden + 10 clones, before divergence: 1.115×. A second run on the same substrate and the same ~10 GB golden cloned ten times: 12.24 GB total, where eleven full copies would be about 121 GB. Per-clone overhead at rest was around 126 MB. These are two separate measurements — 1.107× is the 3-clone run, 1.115× the 10-clone run — and going from 3 clones to 10 barely moved the multiplier, because the multiplier is not a function of clone count.

With writes, amplification tracks divergence, as it must. After driving 120 s of heavy pgbench writes into each of the ten clones, total storage rose to 1.366× — about +2.76 GB, or roughly 276 MB of divergence per clone for that write burst. This is not a defect in the model; it is the model. You pay for changed blocks. Ten clones each carrying 276 MB of changes still cost about 8× less than eleven full copies.

The clones are real copies, not approximations. Data digests on the cloned Postgres data directories matched the golden’s byte for byte (be80ca2f…), and after killing the pods the clones replayed WAL cleanly and came back up. There is no consistency tax being hidden inside the storage number.

Amplification is divergence-driven and base-size-independent. Nothing in the mechanism reads the size of the base dataset. A 1 TB golden shares blocks exactly the way a 10 GB one does; what changes is the absolute size of the base you pay for once, not the multiplier.

What that buys as a workflow

Cheap clones only help if the lifecycle around them is cheap too, and the lifecycle is where a storage property turns into a way of working.

An admin curates a golden snapshot from a real source of truth — an RDS Postgres, say — and versions it. That golden is immutable, so every clone taken from v3 is the same v3, which is what makes a test run reproducible weeks later. Then each developer works against their own clone of it, in their own namespace, with four operations:

clone — take a private, full-size database from a golden version.
reset — throw the current one away and re-clone the same version. This is what makes destructive work safe: run the migration, drop the table, rewrite the index, and if it goes badly, discard the evidence in seconds instead of asking who else is on staging.
refresh — re-clone from a newer golden version when the base data has moved on.
destroy — reclaim it.

What makes this different from a dump is that reset and destroy are cheap enough to be reflexes. A copy that took two hours to produce becomes something you protect; a copy you can replace in seconds becomes something you use hard and throw away. Disposability is a behavior change, and it only exists if both the storage cost and the time cost are near zero.

The time cost is a separate engineering problem, and on FSx it was the harder one — the managed CreateVolume API serializes at roughly one clone per minute, which put the tenth concurrent clone about ten minutes out. With a pre-warmed pool in front of it, the measured p50 for a developer claiming a clone is 16.0 s. That work is written up separately in Clone latency on FSx for OpenZFS: from 10 minutes to 16 seconds.

The honest bill

Copy-on-write removes one specific cost — the per-copy multiplication of storage. It does not make infrastructure free, and it is worth being precise about what is left.

You still pay for the AWS infrastructure you run: the EKS cluster and the FSx for OpenZFS filesystem, including its provisioned throughput and IOPS. That is a real, recurring bill, and it has to be sized for the aggregate concurrent activity of your developers rather than per-developer — an FSx filesystem’s throughput and IOPS budget is a single shared pool.

On top of that you pay for one base dataset per golden version you keep, plus each developer’s divergence. Ten developers on one golden version is one base plus ten small deltas. Keeping four golden versions live is four bases — versions are not free, and retention policy is a real decision.

What you do not pay is a full copy per developer, and you do not ship production data to a third party to get any of it: the clones live on infrastructure in your own account.

Who this is not for

Two cases where this is the wrong tool, stated plainly.

Sub-GB databases. If a production database is a few hundred megabytes, pg_dump and restore takes a minute and costs nothing worth measuring. The argument above is about collapsing a multiplication; when the quantity being multiplied is small, there is nothing to collapse, and a plain copy is the better answer.

Regulated PII or PHI, until masking exists. Cloning gives every developer a byte-identical copy of production — which is the point, and also the problem if that data is regulated. Masking is not built yet, so clones today are raw. Where a compliance posture requires that developers never touch unmasked production data, namespace isolation does not address that requirement. This approach currently suits data that is acceptable to clone in full.

Caveats

The amplification numbers come from a ~10 GB golden. 1.107× is the golden + 3 clones run; 1.115× is the separate golden + 10 clones run. We have not measured the multiplier directly at 1 TB. The mechanism is size-independent and we expect it to hold, but that is an argument from how ZFS works, not a measurement at that scale.
The 1.12× in this post’s title is an illustrative model, not a measurement. It is a 500 GB golden plus ten developers each changing 6 GB — 560 GB, or about 1.12× — and it assumes pessimistically that every changed byte costs its full size. The measured anchors are the two above: 1.107× at 3 clones and 1.115× at 10, both on a ~10 GB golden. Your ratio depends entirely on how much your developers actually change.
Divergence dominates over time, and our divergence figures are short-window. The 276 MB per clone came from a 120 s pgbench write burst — a deliberately heavy synthetic write load over two minutes, not a measurement of what a developer accumulates over a working week. We do not have a measured sustained-use divergence number yet. A clone that lives for months under heavy writes converges toward the cost of a full copy. Copy-on-write defers storage cost; it is disposability — reset and refresh being cheap enough to use — that keeps the deferred cost from accumulating. TTLs and per-namespace quotas are load-bearing, not polish.
Storage efficiency is not performance. These clones share a filesystem’s provisioned throughput and IOPS. In a separate test, one clone running pgbench alone reached 166 TPS while ten writing concurrently got roughly 13–19 TPS each. Cheap storage does not mean free I/O, and the sizing question is independent.
FSx for OpenZFS, us-east-2, June–July 2026. Managed-service behavior changes. The ZFS property underneath is stable; the numbers around it are a snapshot of one provider at one point in time.
Postgres only, at the time of writing. The mechanic is substrate-agnostic in principle — it is a filesystem property — but everything above was measured with CloudNativePG.

Adjoint packages exactly this: clone, reset, refresh, and destroy across versioned golden snapshots.