Skip to main content

Crate happenstance_postgres

Crate happenstance_postgres 

Expand description

Postgres adapters for happenstance: an event store and a projection store.

§Status

The event store is implemented, and run rather than asserted: 107 of 107 gated tests pass against a live PostgreSQL 17.10, including the concurrency family at CONTENDERS = 64. That is the first time an adapter in this workspace has cleared that family against a store whose writers are not serialised, which is the whole reason the crate is in the tree.

The projection store beside it is implemented too, and the crate carries no todo!() and no #![allow(clippy::todo)] any more — the allow left with the last stub, which is the contract it was written under.

It carries two projection stores, and they are not two ways of doing the same thing. projection_store’s batch is an owned statement list replayed at commit — the default an application reaches for, because it holds no connection while apply runs. The skeleton had declared sqlx::Transaction<'static, Postgres> instead, and that binding could not be discharged while begin was total, synchronous and infallible; a skeleton did not report it because todo!() has type ! and coerces to anything. ADR-0062 moved begin and the probe seam, and live_projection_store is that binding discharged: a batch that is a live transaction, at the far end of PS-2’s batch-shape axis, passing the suite there, and since ADR-0074 one that run_projection drives.

The crate shipped in 0.2.0. The release set was five, decided, and the owner re-opened it on this crate’s evidence at e597c34; the manifest carries no publish key. This paragraph said the opposite until the release pass, which would have told a docs.rs reader that the crate they were reading was unpublished.

What a dropped append future does here is answered in event_store’s # Cancellation section, which ES-23 obliges this adapter to carry. The short version, because a caller should not have to click to learn it: the future cannot be cancelled and the batch may already have committed.

§Why this crate exists

It is an instrument first and a target second. Every other adapter in the workspace — MemoryEventStore, a RefCell store, rusqlite, a Durable Object — assigns positions under a lock it holds until commit, so all four satisfy the specification’s ES-10 (position order is visibility order) for free, and none of them votes for it. Postgres is the one shape on the roadmap that can violate ES-10 by construction: nextval() allocates outside the transaction, so a writer that took position 99 can commit after a writer that took position 100, and a reader that has already seen 100 then watches 99 appear beneath it. AppendCondition::after(100) evaluates 99 <= 100 and reports no violation, so the consistency boundary silently stops enforcing.

This crate buys the invariant back rather than avoiding the shape, and the next section is what that costs the caller.

§What this store costs a caller

Read this before choosing the adapter. Everything below is a capability limit — a thing the store does not do — and not a defect to report, not a setting, and not something an operator can tune down.

ES-10 is bought with xid8 + pg_snapshot_xmin. Every row stamps the transaction that wrote it, and every read admits only rows beneath pg_snapshot_xmin(pg_current_snapshot()) — the transaction id below which nothing can still be in flight. Writers are never serialised in the sense that matters: this store takes no lock that makes disjoint boundaries queue behind one another, which is the property it exists to have.

Most of the bill is paid on the read side, and there is a fourth item that is not. The read-side costs are the three below. The write-side one is this: an unconditional append runs at the pool’s default READ COMMITTED and pays nothing extra, but a conditional append runs SERIALIZABLE, so two conditional appends whose predicates overlap are adjudicated at commit and one is aborted with 40001. This adapter re-runs the loser up to SERIALISATION_ATTEMPTS times with exponential, capped, fully-jittered backoff — so a writer already losing a serialisation fight can pay several round trips and a fraction of a second of waiting before it gets its answer. That is retry inside the adapter and is invisible to the caller as anything but latency — until the budget runs out, when the last 40001 arrives as AppendError::Busy: nothing was written, and deciding again is safe. The constant’s own doc block records that its value was adopted from the Neon lane rather than measured here, and that a re-measurement is owed.

Three read-side things follow, and the crate does not soften them.

head() reports a visibility frontier, not max(position), and the frontier trails the maximum. A head is a promise that nothing at or below it will appear later; max(position) cannot make that promise here, because it can name a row whose predecessors are still uncommitted. So head() returns the highest position beneath the frontier, which legitimately sits below the highest position this store has already assigned. The conformance rule for a head asserts a bound rather than an equality for exactly this reason.

Read-your-own-writes does not hold, and is not claimed. append returning Ok(P) does not promise that the next head() is at or above P, and a read issued immediately after a successful append may not contain the event that append just wrote. Nothing is lost — the row is committed and will become visible — but a workflow that writes and then reads back through this store to confirm the write is not one this adapter serves.

Staleness is bounded by the longest open write transaction anywhere on the cluster. Not on this table and not in this database: any write transaction held open by anything connected to the same server holds the frontier where it is. A migration, a batch job, an idle-in-transaction connection or an unrelated tenant is enough, and it is a bound a consumer of this crate neither controls nor can necessarily observe.

What to budget for. Sub-millisecond when nothing else is holding a write transaction open, and the whole remaining duration of the longest one that is otherwise. The remainder of the holder, not a fraction of it.

Measured against this adapter, nine samples per cell: a median of 0.593 ms with no holder, and 4,799 ms behind a five-second held write transaction writing to its own unrelated table — while the same adapter with the visibility predicate removed is unaffected by the identical hold, at 0.595 ms. Same server, same hold, same load; only the predicate differs. So the figure to plan for is whatever your cluster’s longest write transaction is, and if you cannot bound that, you cannot bound this.

use happenstance_core::{Event, SendEventStore};
use happenstance_postgres::event_store::PostgresEventStore;

let written = store.append(events, None).await?;

// `written` is where the event landed. It is not a position `head()`
// promises to have reached, so this is a bound and never an equality —
// `head() == Some(written)` is the assertion a caller must not write.
assert!(store.head().await? <= Some(written));

Three other mechanisms were measured and lost, and naming them once is cheaper than having them re-proposed: a serialised sequence table (0.062 of baseline throughput at 64 writers) and a constant advisory lock (0.033) are both correct and both buy ES-10 by deleting the reason to reach for Postgres, while a tag-keyed advisory lock is nearly free (0.935) and lost on the invariant rather than on cost — it buys a per-boundary property where ES-10 states a global one. references/adr/0024-position-visibility-mechanism.md in this crate’s repository carries the argument and the measurements.

§After a restore, re-mint before the first append

This store’s identity, the StoreId half of every EventId it mints, is one row that migration 1 writes once. A pg_restore, a point-in-time recovery, a promoted replica that lost acknowledged writes, or any other copy brings that row back together with an older position sequence, and the copy would then re-issue EventIds it has already issued, which replication drops silently (VT-6). In this order:

  1. Stop every process that appends to the copy.
  2. Re-mint the copy’s identity, with PostgresEventStore::remint_identity or, from tooling that holds no store, the statement it runs:
    UPDATE store_meta SET v = uuid_send(gen_random_uuid()) WHERE k = 'store_id';
  3. Resume appends.

Nothing has to be restarted. No handle keeps a copy of the identity: every append reads the row inside its own transaction, so a handle opened before step 2 stamps the new identity from its next append on. Step 1 is still first, because a writer that runs between the restore and the re-mint stamps the old identity onto a position the rewound sequence hands out again. Events already in the database keep the identity they were stamped with, which is correct. Minting afresh on every open was considered and declined (ADR-0086): a pool outlives a pg_restore, so it would not close the gap.

§Still open

  • TLS. Deliberately not selected, and the reason is measured rather than assumed. sqlx’s tls-rustls fails the workspace’s cargo deny licence allowlist — not on ring, which is Apache-2.0 AND ISC and passes, but on webpki-roots, the Mozilla CA root bundle, which is CDLA-Permissive-2.0. A crate that connects to nothing does not need a TLS backend; a crate that connects to RDS or Neon does, and that is the point at which the allowlist has to either grow a data licence or the adapter has to take its roots from the platform trust store.

§The naive arm is not API

This crate carries a deliberately broken store: this adapter with the visibility frontier taken out, which is the naive nextval() store ES-10 names in its Rejects:. It exists so that the rule this crate is built to pass can be shown to reject something. It sits behind a rustc cfg, happenstance_naive_arm, set only by a build’s own RUSTFLAGS="-D warnings --cfg happenstance_naive_arm", and not behind a Cargo feature. Until 0.4.0 it was the naive-arm feature, which any manifest could turn on and docs.rs rendered. A cfg is out of every manifest’s reach and every feature powerset, it is never built by docs.rs, and nothing under it is covered by this crate’s semver promise.

§Not the Neon adapter

Neon’s serverless driver speaks one-shot HTTP to a /sql endpoint: no connection, no interactive transaction, no cursor. Everything below — a pool, a server-side DECLARE CURSOR, a Transaction held across awaits — is unavailable there. That is happenstance-neon, a separate crate implementing the bare EventStore, not a feature of this one.

Re-exports§

pub use sqlx;
pub use happenstance_core;
pub use happenstance_sync;sync

Modules§

error
The failure modes a Postgres adapter actually has.
event_storeevent-store
Postgres-backed SendEventStore.
live_projection_storeprojection-store
Postgres-backed SendProjectionStore whose batch is a live transaction.
migrationevent-store or projection-store
The schema, and the one way it is applied.
projection_storeprojection-store
Postgres-backed SendProjectionStore.
read_streamevent-store
The pooled-cursor read stream.