Skip to main content

Module ingest

Module ingest 

Expand description

The ingest port: how a foreign identity reaches a local store.

§The leak this exists to close

Deferring the sync port was said to “leak EventId and a tail seam back into EventStore”. The reasoning is easy to follow and it is worth stating before the counter-argument: a replicated event arrives carrying an identity its origin store minted, that identity must be preserved rather than re-minted, and the only operation that writes to a store is EventStore::append — so append grows a Option<EventId> parameter, every local caller pays for a concept it never uses, and replication has reached into the contract crate.

The escape is coherence. Rust’s orphan rule says an implementation may be written by the crate that defines the trait or by the crate that defines the type, and by nobody else. Turn that around and it is a growth mechanism: a store adapter can acquire a new capability from any crate that defines a trait for it, and the crate that defines the store port never has to hear about it. IngestStore is that trait, it lives here, and append keeps its signature.

§Where the leak went

It shrank three times, and the last one closed it.

The value type. SequencedEvent once carried a position and an event and nothing else, so there was nowhere to put an identity a peer had minted. Phase 4 closed that: it carries position, id, recorded_at and event, and this module speaks the same EventId and RecordedAt rather than placeholders of them.

The write path. That left an identity with a place to sit and no door to come in through. The only &self operation that adds to a store is append, and append mints — happenstance-core states that no store-assigned value is ever supplied by a caller through it, which is the property that keeps the contract’s write path from having to distinguish “I decided this” from “somebody else did and I am copying it”. This crate used to write impl SendIngestStore for MemoryEventStore — local trait, foreign type — and it compiled, and every body was todo!(), because coherence lets a crate add a trait to a foreign type and never lets it reach inside one. MemoryEventStore mints EventId::new(self.store_id, position) for every event it writes, and nothing outside happenstance-core can make it do otherwise.

Whose door it is. Read that last sentence the other way round and it is the answer: the write path that accepts a foreign identity belongs to the crate that owns the store’s internals, which is the adapter — and the adapters’ schemas were built to take it. Each already keeps origin apart from position: SQLite and the Durable Object insert with the origin columns NULL and stamp local identity afterwards through an IS NULL marker that exists so an ingested row is not restamped, and Postgres and Neon write the origin pair explicitly under a unique index on it. So the ingest path need not be a second write path beside append: it can be the adapter’s existing row writer, generalised to take a per-row origin, with what is ingest-only small enough to list — the deduplicating conflict clause, a per-row origin and recorded time, and the absence of any condition evaluation (SY-1).

Four adapters run it, each behind its sync feature. SQLite: one INSERT, differing only in values bound. Postgres sends append’s text plus a conflict clause; Neon, its insert builder with that clause and a statement refusing an unheld claim, in one request. The Durable Object runs append’s INSERT, and stamps the origin by an UPDATE OR IGNORE, in one transactionSync a call. See crates/happenstance-{postgres,neon,cloudflare}/src/event_store/ingest.rs.

The instrument is crates/happenstance-sqlite/src/ingest_spike.rs: this trait implemented for SqliteEventStore, in the adapter’s own crate, over the same private writer append uses. happenstance-core grows nothing for it and no published signature changes. That spike is the run VT-10’s falsifier names — “a store adapter cannot implement IngestStore without duplicating append’s write path”, with SQLite as the instrument — and the phase-17 record states what it found.

§What coherence still refuses

A third crate cannot write the impl. happenstance-sync-testkit, which will define neither the trait nor the store, is barred from writing it on an adapter’s behalf — see the compile_fail example on IngestStore. That is not a problem for a conformance suite, which takes an implementation rather than supplying one, but it does mean there is no blanket impl<S: EventStore> IngestStore for S waiting to make this free — and after the paragraphs above, no reason to want one: a blanket impl could only call append, and append re-mints.

The in-memory store is the one this leaves without an implementation. MemoryEventStore lives in the contract crate, so the adapter-owned answer would put an ingest operation in happenstance-core — the letter VT-10 forbids even where it would be additive. Its oracle is MemoryIngestStore, a store this crate owns, behind memory.

§Membership is not on this trait

It used to carry holds(&EventId). That question is EventStore::contains_event_id, which landed with the identity fields and answers it exactly, so a runner binds S: EventStore + IngestStore and asks the store it already has. It is also not a question ingest needs asked first: deduplication happens inside the write, against the store’s uniqueness on the origin pair (VT-8), and a probe-then-write would be the race that clause forbids.

Structs§

IngestGroup
One atomic unit of an ingest: foreign events, and the local compensation this side writes for them.
Ingested
What one ingest did, summed across its groups.

Enums§

IngestError
Why an ingest did not land.

Traits§

IngestStore
A store that can accept events another store already minted.
SendIngestStore
A store that can accept events another store already minted.