Crate happenstance_postgres
Expand description
Postgres adapters for happenstance: an event store and a projection store.
§Status
The event store is implemented, and run rather than asserted: 107 of 107
gated tests pass against a live PostgreSQL 17.10, including the concurrency
family at CONTENDERS = 64. That is the first time an adapter in this
workspace has cleared that family against a store whose writers are not
serialised, which is the whole reason the crate is in the tree.
The projection store beside it is implemented too, and the crate carries
no todo!() and no #![allow(clippy::todo)] any more — the allow left with
the last stub, which is the contract it was written under.
It carries two projection stores, and they are not two ways of doing the
same thing. projection_store’s batch is an owned statement list replayed
at commit — the default an application reaches for, because it holds no
connection while apply runs. The skeleton
had declared sqlx::Transaction<'static, Postgres> instead, and that binding
could not be discharged while begin was total, synchronous and infallible;
a skeleton did not report it because todo!() has type ! and coerces to
anything. ADR-0062 moved begin and the probe seam, and
live_projection_store is that binding discharged: a batch that is a
live transaction, at the far end of PS-2’s batch-shape axis, passing the
suite there, and since ADR-0074 one that run_projection drives.
The crate shipped in 0.2.0. The release set was five, decided, and the
owner re-opened it on this crate’s evidence at e597c34; the manifest carries
no publish key. This paragraph said the opposite until the release pass,
which would have told a docs.rs reader that the crate they were reading was
unpublished.
What a dropped append future does here is answered in
event_store’s # Cancellation section, which ES-23 obliges this
adapter to carry. The short version, because a caller should not have to
click to learn it: the future cannot be cancelled and the batch may already
have committed.
§Why this crate exists
It is an instrument first and a target second. Every other adapter in the
workspace — MemoryEventStore, a RefCell store, rusqlite, a Durable
Object — assigns positions under a lock it holds until commit, so all four
satisfy the specification’s ES-10 (position order is visibility order) for
free, and none of them votes for it. Postgres is the one shape on the roadmap
that can violate ES-10 by construction: nextval() allocates outside the
transaction, so a writer that took position 99 can commit after a writer that
took position 100, and a reader that has already seen 100 then watches 99
appear beneath it. AppendCondition::after(100) evaluates 99 <= 100 and
reports no violation, so the consistency boundary silently stops enforcing.
This crate buys the invariant back rather than avoiding the shape, and the next section is what that costs the caller.
§What this store costs a caller
Read this before choosing the adapter. Everything below is a capability limit — a thing the store does not do — and not a defect to report, not a setting, and not something an operator can tune down.
ES-10 is bought with xid8 + pg_snapshot_xmin. Every row stamps the
transaction that wrote it, and every read admits only rows beneath
pg_snapshot_xmin(pg_current_snapshot()) — the transaction id below which
nothing can still be in flight. Writers are never serialised in the sense that
matters: this store takes no lock that makes disjoint boundaries queue behind
one another, which is the property it exists to have.
Most of the bill is paid on the read side, and there is a fourth item that
is not. The read-side costs are the three below. The write-side one is this:
an unconditional append runs at the pool’s default READ COMMITTED and pays
nothing extra, but a conditional append runs SERIALIZABLE, so two
conditional appends whose predicates overlap are adjudicated at commit and one
is aborted with 40001. This adapter re-runs the loser up to
SERIALISATION_ATTEMPTS times with exponential, capped, fully-jittered
backoff — so a writer already losing a serialisation fight can pay several
round trips and a fraction of a second of waiting before it gets its answer.
That is retry inside the adapter and is invisible to the caller as anything
but latency — until the budget runs out, when the last 40001 arrives as
AppendError::Busy: nothing was written, and deciding again is safe. The
constant’s own doc block records that its value was adopted
from the Neon lane rather than measured here, and that a re-measurement is
owed.
Three read-side things follow, and the crate does not soften them.
head() reports a visibility frontier, not max(position), and the
frontier trails the maximum. A head is a promise that nothing at or below
it will appear later; max(position) cannot make that promise here, because
it can name a row whose predecessors are still uncommitted. So head()
returns the highest position beneath the frontier, which legitimately sits
below the highest position this store has already assigned. The conformance
rule for a head asserts a bound rather than an equality for exactly this
reason.
Read-your-own-writes does not hold, and is not claimed. append
returning Ok(P) does not promise that the next head() is at or above
P, and a read issued immediately after a successful append may not contain
the event that append just wrote. Nothing is lost — the row is committed and
will become visible — but a workflow that writes and then reads back through
this store to confirm the write is not one this adapter serves.
Staleness is bounded by the longest open write transaction anywhere on the cluster. Not on this table and not in this database: any write transaction held open by anything connected to the same server holds the frontier where it is. A migration, a batch job, an idle-in-transaction connection or an unrelated tenant is enough, and it is a bound a consumer of this crate neither controls nor can necessarily observe.
What to budget for. Sub-millisecond when nothing else is holding a write transaction open, and the whole remaining duration of the longest one that is otherwise. The remainder of the holder, not a fraction of it.
Measured against this adapter, nine samples per cell: a median of 0.593 ms with no holder, and 4,799 ms behind a five-second held write transaction writing to its own unrelated table — while the same adapter with the visibility predicate removed is unaffected by the identical hold, at 0.595 ms. Same server, same hold, same load; only the predicate differs. So the figure to plan for is whatever your cluster’s longest write transaction is, and if you cannot bound that, you cannot bound this.
use happenstance_core::{Event, SendEventStore};
use happenstance_postgres::event_store::PostgresEventStore;
let written = store.append(events, None).await?;
// `written` is where the event landed. It is not a position `head()`
// promises to have reached, so this is a bound and never an equality —
// `head() == Some(written)` is the assertion a caller must not write.
assert!(store.head().await? <= Some(written));Three other mechanisms were measured and lost, and naming them once is
cheaper than having them re-proposed: a serialised sequence table (0.062 of
baseline throughput at 64 writers) and a constant advisory lock (0.033) are
both correct and both buy ES-10 by deleting the reason to reach for Postgres,
while a tag-keyed advisory lock is nearly free (0.935) and lost on the
invariant rather than on cost — it buys a per-boundary property where
ES-10 states a global one. references/adr/0024-position-visibility-mechanism.md
in this crate’s repository carries the argument and the measurements.
§After a restore, re-mint before the first append
This store’s identity, the StoreId half of every EventId it mints, is
one row that migration 1 writes once. A pg_restore, a point-in-time
recovery, a promoted replica that lost acknowledged writes, or any other copy
brings that row back together with an older position sequence, and the copy
would then re-issue EventIds it has already issued, which replication drops
silently (VT-6). In this order:
- Stop every process that appends to the copy.
- Re-mint the copy’s identity, with
PostgresEventStore::remint_identityor, from tooling that holds no store, the statement it runs:UPDATE store_meta SET v = uuid_send(gen_random_uuid()) WHERE k = 'store_id'; - Resume appends.
Nothing has to be restarted. No handle keeps a copy of the identity: every
append reads the row inside its own transaction, so a handle opened before
step 2 stamps the new identity from its next append on. Step 1 is still
first, because a writer that runs between the restore and the re-mint stamps
the old identity onto a position the rewound sequence hands out again.
Events already in the database keep the identity they were stamped with,
which is correct. Minting afresh on every open was considered and declined
(ADR-0086): a pool outlives a pg_restore, so it would not close the gap.
§Still open
- TLS. Deliberately not selected, and the reason is measured rather than
assumed.
sqlx’stls-rustlsfails the workspace’scargo denylicence allowlist — not onring, which isApache-2.0 AND ISCand passes, but onwebpki-roots, the Mozilla CA root bundle, which isCDLA-Permissive-2.0. A crate that connects to nothing does not need a TLS backend; a crate that connects to RDS or Neon does, and that is the point at which the allowlist has to either grow a data licence or the adapter has to take its roots from the platform trust store.
§The naive arm is not API
This crate carries a deliberately broken store: this adapter with the
visibility frontier taken out, which is the naive nextval() store ES-10
names in its Rejects:. It exists so that the rule this crate is built to
pass can be shown to reject something. It sits behind a rustc cfg,
happenstance_naive_arm, set only by a build’s own
RUSTFLAGS="-D warnings --cfg happenstance_naive_arm", and not behind a
Cargo feature. Until 0.4.0 it was the naive-arm feature, which any
manifest could turn on and docs.rs rendered. A cfg is out of every
manifest’s reach and every feature powerset, it is never built by docs.rs,
and nothing under it is covered by this crate’s semver promise.
§Not the Neon adapter
Neon’s serverless driver speaks one-shot HTTP to a /sql endpoint: no
connection, no interactive transaction, no cursor. Everything below — a
pool, a server-side DECLARE CURSOR, a Transaction held across awaits —
is unavailable there. That is happenstance-neon, a separate crate
implementing the bare EventStore, not a
feature of this one.
Re-exports§
pub use sqlx;pub use happenstance_core;pub use happenstance_sync;sync
Modules§
- error
- The failure modes a Postgres adapter actually has.
- event_
store event-store - Postgres-backed
SendEventStore. - live_
projection_ store projection-store - Postgres-backed
SendProjectionStorewhose batch is a live transaction. - migration
event-storeorprojection-store - The schema, and the one way it is applied.
- projection_
store projection-store - Postgres-backed
SendProjectionStore. - read_
stream event-store - The pooled-cursor read stream.