Struct NeonEventStore
pub struct NeonEventStore<T> { /* private fields */ }event-store only.Expand description
An event store over Neon’s serverless /sql HTTP endpoint.
use happenstance_core::EventStore;
use happenstance_neon::{NeonConfig, NeonEventStore, NullTransport};
// Binding the *bare* flavour, per the contract crate's guidance.
fn takes_a_store<S: EventStore>(_store: &S) {}
let store = NeonEventStore::new(NullTransport::new(), NeonConfig::default());
takes_a_store(&store);Implementations§
§impl<T: SqlTransport> NeonEventStore<T>
impl<T: SqlTransport> NeonEventStore<T>
pub async fn remint_identity(&self) -> Result<StoreId, NeonError<T::Error>>
pub async fn remint_identity(&self) -> Result<StoreId, NeonError<T::Error>>
Replaces this store’s persisted identity with a freshly minted one, and returns it.
§When a deployment invokes this
On every Neon branch that is created, reset or restored, and after a
pg_restore or a point-in-time recovery, before the copy accepts its
first append. A branch is a copy-on-write clone of its parent, so it
carries the parent’s identity row together with an older position
sequence, and would then re-issue EventIds
the parent already issued, which a replication peer drops as already
seen, with no error anywhere. VT-6 permits this store’s mint-once only
because the deployment is documented to invoke a re-mint, and this is
that re-mint (ADR-0086). Minting afresh on every open was considered and
declined there: a Workers deployment builds a store per request.
Stop every writer to the copy, call this, then resume appends. Tooling
that makes branches and holds no store runs the same statement,
migration::remint_statement.
§Handles
This store holds no identity in Rust: every append reads the meta row inside its own statement. So every handle and clone, in this isolate or another, stamps the new identity on its next append, and nothing has to be rebuilt.
§What it does not change
Events already in the store keep the identity they were stamped with, which is correct: they were written by the incarnation being retired. A re-mint when no copy was made breaks nothing; it only starts a new origin for later events. It applies no migration, and refuses rather than minting into a schema migration 1 never ran on.
§Errors
NeonError::MissingIdentityif the meta table has nostore_idrow, which means migration 1 has not been applied. No row is created.NeonError::MalformedIdentityorNeonError::MalformedPayloadif the value read back is not sixteen bytes of base64.NeonError::Transport,NeonError::Http,NeonError::Sqlor a decoding variant as any round trip reports them. ATransportfailure may have committed. Calling again is the remedy, because a re-mint never makes an identity collide.
§Examples
use happenstance_neon::{NeonConfig, NeonError, NeonEventStore, SqlTransport};
// Every writer to the new branch is stopped.
let store = NeonEventStore::new(transport, NeonConfig::default());
let _new_identity = store.remint_identity().await?;
// Appends may resume, through this handle or any other.§impl<T> NeonEventStore<T>
impl<T> NeonEventStore<T>
pub const MAX_EVENT_DATA_LEN: usize = 131_072
pub const MAX_EVENT_DATA_LEN: usize = 131_072
The largest data payload this store accepts, in bytes of
Event::data — not of an encoded row, and not of data and
metadata together.
An adapter policy rather than a server limit, and the arithmetic behind it
is this adapter’s own rather than borrowed. A payload travels base64, so
it costs 4/3 of its length in the request body and again in any response
that reads it back; 131,072 bytes is 174,764 characters each way, against
MAX_RESPONSE_BYTES’s 64 MiB. Twice VT-21’s
65,536-byte floor, so
store_accepts_the_guaranteed_minimum_payload has headroom rather than
sitting exactly on the line.
It is deliberately not the 1 MiB happenstance-sqlite and
happenstance-postgres agree on. Those two are wire-efficient binary
protocols with no per-request body to buffer; this one buffers the whole
request and the whole response in memory as JSON at both ends, and a
ceiling that ignored that would be a number copied rather than derived.
pub const MAX_TAGS_PER_EVENT: usize = 128
pub const MAX_TAGS_PER_EVENT: usize = 128
The largest number of tags on one event this store accepts.
Twice VT-22’s floor of 64. Tags live in one text[] column, so a tag
costs no extra row and no extra statement; the cost is the GIN index’s,
which builds one entry per element.
pub const MAX_EVENTS_PER_BATCH: usize = 128
pub const MAX_EVENTS_PER_BATCH: usize = 128
The largest number of events this store accepts in one append.
VT-24’s floor exactly, and the one of the three where this adapter is
genuinely tighter than its siblings — happenstance-postgres states 256.
The batch is one JSON document held whole in memory at both ends and the
response holds one row per event, so the honest ceiling is low and stated
rather than high and discovered as a 413.
pub const SERIALISATION_ATTEMPTS: u32 = 8
pub const SERIALISATION_ATTEMPTS: u32 = 8
How many times a conditional append re-runs after a serialisation failure.
Re-derived against this endpoint rather than copied from
happenstance-postgres, because the critique of the plan was right that
a number justified by “the retry is only ever needed once in the
two-writer case” says nothing about a transport with a thousand times the
transaction latency and 64 contenders.
The one-boundary measurement says three, and it is the wrong
measurement. 64 simultaneous conditional appends on one boundary
against the live endpoint, six runs: exactly one commit and sixty-three
ConditionViolated every time, between 0 and 25 contenders seeing a
40001 at all, and no contender ever reaching a third attempt. The reason
is structural — the moment the winner commits, every retry sees the
winner’s row and answers ConditionViolated instead of racing again, so
the population drains in one round.
Disjoint boundaries do not drain, and that is what sets this number.
k_disjoint_boundaries_never_conflict runs twelve contenders
over four separate boundaries, and the SSI predicate lock is not per
boundary: with a small table the planner takes a sequential scan and the
lock is relation-wide, so every contender conflicts with every other
regardless of which boundary it is racing. A retry that finds no conflict
on its own boundary goes back into the same fight, and the population
drains one round at a time rather than all at once.
Measured against that rule, four runs per value:
| attempts | result |
|---|---|
| 3 | failed 2 of 4 — a contender exhausted and answered 40001 |
| 4 | 4 of 4 green |
| 5 | 4 of 4 green |
| 8 | 5 of 5 green |
Eight is double the smallest sufficient value rather than one above it,
because the observed floor sits one attempt away from a failure that
reproduced half the time — a margin of one on a measurement that noisy is
not a margin. It costs nothing when it is not needed: a retry happens only
after a 40001, and the retry itself is the backoff, at one round trip
each.
What it costs when it is needed, stated because “costs nothing” is only
half the ledger. There is no sleep between attempts, so the whole budget
is up to eight serial round trips to the endpoint before a 40001
surfaces — the one path in this store where a single append call can spend
eight network latencies. That is the price of the margin, and it is paid
only by a caller already losing a serialisation fight: the winner commits on
its first attempt, and every contender on the same boundary answers
ConditionViolated on its second rather than racing again. It is the
disjoint-boundary case above, where the relation-wide predicate lock puts a
retry straight back into the same fight, that can walk the budget.
A caller that would rather fail fast than wait has the shape available:
ConditionViolated and Sql(40001) are distinct outcomes, so a retry
policy of its own can sit above this one. What it cannot do is lower this
constant — it is pub to be read, and the number is the adapter’s.
Exhaustion is not silent: it surfaces as
AppendError::Busy(NeonError::Sql(…)) carrying SQLSTATE 40001, which is
a named, documented outcome rather than a ConditionViolated this adapter
invented. Until 0.4.0 it was AppendError::Store, which is exactly what
the failures at three looked like; ADR-0077 moved it, because it meets
all three parts of Busy’s contract by the endpoint’s guarantee: the
endpoint answered, with a 40001, so it ran the batch’s transaction
and aborted it whole — nothing was written, and the same call can succeed
once the conflicting writer has committed.
Only an answer can be Busy. A round trip that got no answer is
NeonError::Transport, which may have committed and stays Store
whatever caused it — a timeout looks transient and is exactly the
ambiguous outcome Busy’s contract forbids. And an answer proves
nothing was written only when it answers the only send of its
request, which is why SqlTransport::round_trip’s # Retries makes
a transparent re-send of a request that may have reached the endpoint a
MUST NOT rather than an implementor’s choice.
pub const fn new(transport: T, config: NeonConfig) -> Self
pub const fn new(transport: T, config: NeonConfig) -> Self
Builds a store over transport.
pub const fn config(&self) -> &NeonConfig
pub const fn config(&self) -> &NeonConfig
The table names and limits this store was built with.
Trait Implementations§
§impl<T: Clone> Clone for NeonEventStore<T>
impl<T: Clone> Clone for NeonEventStore<T>
§impl<T: Debug> Debug for NeonEventStore<T>
impl<T: Debug> Debug for NeonEventStore<T>
§impl<T: SqlTransport> EventStore for NeonEventStore<T>
impl<T: SqlTransport> EventStore for NeonEventStore<T>
§async fn append(
&self,
events: &[Event],
condition: Option<&AppendCondition>,
) -> Result<SequencePosition, AppendError<Self::Error>>
async fn append( &self, events: &[Event], condition: Option<&AppendCondition>, ) -> Result<SequencePosition, AppendError<Self::Error>>
Appends a batch and returns the position of its last event.
§What Ok(P) does and does not promise
The batch is committed at positions ending at P, and the condition held
when evaluated. Read-your-own-writes is not promised: the next
head may trail P, as the visibility frontier trails
assigned positions — by sub-milliseconds, a margin and not a guarantee.
§Ordering against reads (ES-11)
The first attempt waits for SqlTransport::reads_settled, so no write
leaves while a read this transport sent earlier is unanswered: up to one
read round trip when one is in flight, and nothing otherwise.
§Errors
AppendError::NoEvents for an empty batch, refused before the condition
is evaluated; AppendError::ExceedsStoreLimit for a batch over one of
this store’s stated ceilings, refused before anything reaches the wire;
AppendError::ConditionViolated when the condition found a matching
event, which is not an adapter failure; AppendError::Busy for a 40001
that survived SERIALISATION_ATTEMPTS
attempts, each of which the endpoint aborted whole, so nothing was
written; and AppendError::Store for anything else the transport or the
endpoint reports — a round trip that got no answer included, because it
may have committed.
§async fn head(&self) -> Result<Option<SequencePosition>, Self::Error>
async fn head(&self) -> Result<Option<SequencePosition>, Self::Error>
The highest position beneath this store’s visibility frontier, or
None when nothing is visible yet.
§What a frontier is, and why it is not max(position)
A head is a promise that nothing at or below it will appear later
(ES-10). max(position) cannot make that promise: nextval() allocates
outside the transaction, so the highest position assigned can name a row
whose predecessors are still in flight. Each row therefore stamps its
appending transaction’s xid8, and this reports the frontier beneath
which nothing can still be running.
The frontier legitimately trails the position
append just returned; ES-30’s
head_is_the_highest_visible_position asserts a bound rather than an
equality precisely to admit that.
§Errors
The adapter’s error if the round trip fails, or if a stored position cannot be decoded — which is a corrupt store and is deliberately not spelled the same way as an empty one.
§async fn contains_event_id(&self, id: EventId) -> Result<bool, Self::Error>
async fn contains_event_id(&self, id: EventId) -> Result<bool, Self::Error>
Whether this store holds the event id names.
§Held, not visible: ES-41’s reading, settled by ADR-0095
This asks whether the store holds an event, and under the frontier
mechanism that is a different question from whether read would yield it:
a committed row above the frontier is held but invisible. This method
answers true for a held-but-invisible row — it does not carry the
frontier predicate — which is the reading ES-41 requires. It agrees with
the append condition, which ES-25 evaluates over what the store holds, and
with VT-8’s uniqueness on the origin pair; it briefly disagrees with read,
and that disagreement resolves itself as the frontier advances.
contains_event_id_reports_a_held_row_above_the_frontier is the rule that
checks it, against a row this crate’s fixture lifts above any frontier.
§Errors
The adapter’s error if the round trip fails or the answer does not have the shape this adapter asked for.
§type Error = NeonError<<T as SqlTransport>::Error>
type Error = NeonError<<T as SqlTransport>::Error>
§fn read(
&self,
query: &Query,
options: ReadOptions,
) -> impl Stream<Item = Result<SequencedEvent, Self::Error>>
fn read( &self, query: &Query, options: ReadOptions, ) -> impl Stream<Item = Result<SequencedEvent, Self::Error>>
§impl<T: SqlTransport> IngestStore for NeonEventStore<T>
Available on crate feature sync only.
impl<T: SqlTransport> IngestStore for NeonEventStore<T>
sync only.§async fn store_id(&self) -> Result<StoreId, Self::Error>
async fn store_id(&self) -> Result<StoreId, Self::Error>
The identity the meta row holds, read on every call in one read-only
round trip. Nothing is cached, as nothing is for append (ADR-0086),
so a re-mint is answered by the next call on any handle.
§Errors
Transport, Http or Sql as any read reports them;
MissingIdentity when the meta row is absent; MalformedIdentity when
it is not sixteen bytes; MalformedPayload when it is not base64.
§type Error = NeonError<<T as SqlTransport>::Error>
type Error = NeonError<<T as SqlTransport>::Error>
§async fn ingest(
&self,
groups: &[IngestGroup<'_>],
) -> Result<Ingested, IngestError<Self::Error>>
async fn ingest( &self, groups: &[IngestGroup<'_>], ) -> Result<Ingested, IngestError<Self::Error>>
§impl<T: SqlTransport> WriteGroupStore for NeonEventStore<T>
Available on crate feature sync only.The log in the writes that made it (SY-39): what a runner pushes from and a
StoreSyncPeer serves a pull from. The bare flavour, the only one this
store has, as its IngestStore is; only that name is in scope here
(constraint 4).
impl<T: SqlTransport> WriteGroupStore for NeonEventStore<T>
sync only.The log in the writes that made it (SY-39): what a runner pushes from and a
StoreSyncPeer serves a pull from. The bare flavour, the only one this
store has, as its IngestStore is; only that name is in scope here
(constraint 4).