Module ingest
Expand description
The ingest port: how a foreign identity reaches a local store.
§The leak this exists to close
Deferring the sync port was said to “leak EventId and a tail seam back
into EventStore”. The reasoning is easy to follow and it is worth stating
before the counter-argument: a replicated event arrives carrying an identity
its origin store minted, that identity must be preserved rather than
re-minted, and the only operation that writes to a store is
EventStore::append — so append
grows a Option<EventId> parameter, every local caller pays for a concept it
never uses, and replication has reached into the contract crate.
The escape is coherence. Rust’s orphan rule says an implementation may be
written by the crate that defines the trait or by the crate that defines the
type, and by nobody else. Turn that around and it is a growth mechanism: a
store adapter can acquire a new capability from any crate that defines a
trait for it, and the crate that defines the store port never has to hear
about it. IngestStore is that trait, it lives here, and append keeps
its signature.
§Where the leak went
It shrank three times, and the last one closed it.
The value type. SequencedEvent
once carried a position and an event and nothing else, so there was nowhere
to put an identity a peer had minted. Phase 4 closed that: it carries
position, id, recorded_at and event, and this module speaks the same
EventId and
RecordedAt rather than placeholders of
them.
The write path. That left an identity with a place to sit and no door to
come in through. The only &self operation that adds to a store is
append, and append mints — happenstance-core states that no
store-assigned value is ever supplied by a caller through it, which is the
property that keeps the contract’s write path from having to distinguish “I
decided this” from “somebody else did and I am copying it”. This crate used
to write impl SendIngestStore for MemoryEventStore — local trait, foreign
type — and it compiled, and every body was todo!(), because coherence lets
a crate add a trait to a foreign type and never lets it reach inside one.
MemoryEventStore mints EventId::new(self.store_id, position) for every
event it writes, and nothing outside happenstance-core can make it do
otherwise.
Whose door it is. Read that last sentence the other way round and it is
the answer: the write path that accepts a foreign identity belongs to the
crate that owns the store’s internals, which is the adapter — and the
adapters’ schemas were built to take it. Each already keeps origin apart from
position: SQLite and the Durable Object insert with the origin columns NULL
and stamp local identity afterwards through an IS NULL marker that exists
so an ingested row is not restamped, and Postgres and Neon write the origin
pair explicitly under a unique index on it. So the ingest path need not be a
second write path beside append: it can be the adapter’s existing row
writer, generalised to take a per-row origin, with what is ingest-only small
enough to list — the deduplicating conflict clause, a per-row origin and
recorded time, and the absence of any condition evaluation (SY-1).
Four adapters run it, each behind its sync feature. SQLite: one INSERT,
differing only in values bound. Postgres sends append’s text plus a conflict
clause; Neon, its insert builder with that clause and a statement refusing an
unheld claim, in one request. The Durable Object runs append’s INSERT, and
stamps the origin by an UPDATE OR IGNORE, in one transactionSync a call.
See crates/happenstance-{postgres,neon,cloudflare}/src/event_store/ingest.rs.
The instrument is crates/happenstance-sqlite/src/ingest_spike.rs: this
trait implemented for SqliteEventStore, in the adapter’s own crate, over
the same private writer append uses. happenstance-core grows nothing for
it and no published signature changes. That spike is the run VT-10’s
falsifier names — “a store adapter cannot implement IngestStore without
duplicating append’s write path”, with SQLite as the instrument — and the
phase-17 record states what it found.
§What coherence still refuses
A third crate cannot write the impl. happenstance-sync-testkit, which
will define neither the trait nor the store, is barred from writing it on an
adapter’s behalf — see the compile_fail example on IngestStore. That is
not a problem for a conformance suite, which takes an implementation rather
than supplying one, but it does mean there is no blanket
impl<S: EventStore> IngestStore for S waiting to make this free — and after
the paragraphs above, no reason to want one: a blanket impl could only call
append, and append re-mints.
The in-memory store is the one this leaves without an implementation.
MemoryEventStore lives in the contract crate, so the adapter-owned answer
would put an ingest operation in happenstance-core — the letter VT-10
forbids even where it would be additive. Its oracle is MemoryIngestStore,
a store this crate owns, behind memory.
§Membership is not on this trait
It used to carry holds(&EventId). That question is
EventStore::contains_event_id,
which landed with the identity fields and answers it exactly, so a runner
binds S: EventStore + IngestStore and asks the store it already has. It is
also not a question ingest needs asked first:
deduplication happens inside the write, against the store’s uniqueness on
the origin pair (VT-8), and a probe-then-write would be the race that clause
forbids.
Structs§
- Ingest
Group - One atomic unit of an
ingest: foreign events, and the local compensation this side writes for them. - Ingested
- What one
ingestdid, summed across its groups.
Enums§
- Ingest
Error - Why an
ingestdid not land.
Traits§
- Ingest
Store - A store that can accept events another store already minted.
- Send
Ingest Store - A store that can accept events another store already minted.