Expand description
The conformance suite for happenstance-sync peers, and for the stores
they ingest into.
§Status: the peer, ingest, write-group, restore, watermark and convergence families
Published since #108 part E, with happenstance-sync, and grown rule by rule
through phase 13, which ran it against the network peers. It is the bar a peer or ingest adapter must
clear before it is considered to exist, exactly as happenstance-testkit is
for a store, and an adapter outside this workspace mounts it the same way.
Pin it exactly: a new rule is a minor of this crate’s number.
Publishing settled the two questions this paragraph used to defer. Its
version is the workspace’s, in lockstep until 1.0 — ADR-0066 §4 makes a
new rule a minor of its number, and lockstep keeps that true while the
workspace is 0.y. And happenstance-testkit’s RuleOutcome is part of
this crate’s public API, through every rule’s signature.
§Mounting it
happenstance_sync_testkit::sync_peer_conformance!(
happenstance_sync_testkit::MemorySyncFixture::new()
);The expression builds a SyncFixture: one isolated backing peer per
instance. It is evaluated inside the generated module, so a fixture type
defined beside the invocation is named super::MyFixture; the macro does not
glob-import its parent, because an unused glob would be a warning in every
crate that named its fixture by full path.
§What is checked
| Rule | Clause | What it rejects |
|---|---|---|
ingest_never_rejects | SY-1 | a peer that evaluates the travelling guard and refuses or drops a conflicting fact |
compensation_is_atomic_with_the_losing_event | SY-2 | a peer that splits a losing event from the compensation that rode with it |
event_id_order_is_identical_on_every_peer | SY-23 | a peer that dedupes by the highest origin position it has seen, and so drops an origin’s lower event that arrives late |
peer_declares_its_own_limits | SY-18 | a peer whose declared limits admit an event its ingest then refuses, or that declares below VT-21’s floor |
peer_takes_a_batch_at_its_declared_limits | SY-18 | a peer that refuses a push its own batch limits admit: more events, or more bytes, than it really takes |
push_envelope_preserves_group_boundaries | SY-30 | a peer that merges the groups of a page into one, or strips the guard a group was pushed with |
every_refusal_states_its_class | SY-36 | a peer whose errors all state one failure class, whatever the failure |
nothing_waiting_means_the_next_pull_is_empty | SY-37 | a peer that says nothing is waiting when its page ran out, not its log |
ack_never_confirms_a_non_durable_event | SY-31 | a peer that confirms an event before it is durable |
round_trip_preserves_the_whole_log | SY-28, SY-27 | a peer that withholds from its counterpart the events a type or tag filter matches, among those the rule’s log carries |
retention_gap_is_reported_not_silent | SY-32 | a peer that compacts and answers a token below its floor, or the start, with a strict subset in silence, or refuses it as anything but Reseed; skipped unless the fixture declares COMPACTION |
Each rule names a wrong implementation, and that implementation lives in
this crate’s tests/wrong_peers.rs — SY-18’s in tests/wrong_limits.rs,
over KvCappedFixture, its batch rule’s in tests/wrong_peers.rs,
SY-23’s in tests/wrong_orders.rs, SY-28’s in tests/wrong_scopes.rs,
SY-31’s across a reopen in tests/wrong_reopens.rs, and SY-32’s in
tests/wrong_retention.rs — where the rule is shown to fail it. A rule no
peer can fail is decorative. The mount also emits
every_declined_capability_is_stated_by_this_fixture, which fails a
fixture that leaves REOPEN at this crate’s default
declension (sync_declensions_are_stated).
No peer or ingest rule decodes a payload (SY-35), and no rule in
rules holds two peers (SY-8). rules says how the rules keep both. The model family
below is the one exception to the second, and its module says why.
§The model family
Behind the off-by-default proptest feature, and absent on wasm32 because
proptest does not build there. sync_model generates peer topologies of
three to six peers — chains whose ends never meet included — and drives them
from one seed through writes, partitions, lost acks, duplicated pushes and
delayed pages, then checks every peer at quiescence against a model of its
own. It opens one fixture per node, so the fixture expression must build
a fresh, isolated peer each time it is evaluated.
| Rule | Clause | What it rejects |
|---|---|---|
transitive_convergence_over_a_partial_mesh | SY-24, SY-4 | a peer that quarantines what it accepted, or never forwards it |
redelivery_of_an_accepted_group_is_a_no_op | SY-11 | a peer that dedupes per batch (SF-03), or miscounts its acknowledgement |
forwarding_order_is_arrival_order | SY-25 | a peer that re-sorts what it forwards |
The names are written out rather than linked, because the module they live
in does not exist without the feature. A failure names its seed in hex and
the smallest scenario it shrank to; tests/wrong_topologies.rs pins every
rule’s verdict against four wrong peers.
#[cfg(not(target_arch = "wasm32"))]
happenstance_sync_testkit::sync_model_conformance!(
happenstance_sync_testkit::MemorySyncFixture::new()
);§The ingest family
The other side of a peer: the store an
IngestStore writes into. A store adapter
that implements the port mounts it beside the event-store suite:
happenstance_sync_testkit::ingest_store_conformance!(
happenstance_sync_testkit::MemoryIngestFixture::new()
);The expression builds an IngestFixture, whose store is both an
EventStore and an IngestStore. The reference is
MemoryIngestFixture, over happenstance-sync’s MemoryIngestStore;
happenstance-sqlite’s spike and the Durable Object mount the family too.
| Rule | Clause | What it rejects |
|---|---|---|
ingest_preserves_origin_identity | VT-5 | a store that re-mints an ingested event |
ingest_preserves_the_recorded_time | VT-9 | a store that restamps an ingested event with its arrival time, keeps whole seconds, holds a time before 1970 as the epoch, holds times non-decreasing with position, binds one time per group, or clamps a future time to its own clock |
ingested_events_land_above_the_local_head | SY-5 | an order-preserving merge, which hides an event below a checkpoint already read past |
dedupe_reaches_identity_without_decoding | SY-12, SY-11, SY-3 | content-keyed dedupe, and a compensation written again on redelivery |
an_unheld_own_identity_stops_the_ingest | SY-38 | a dropped claim, a check of the batch’s head only, a group-by-group commit, a claim hidden by the call’s own compensation, and a refused held echo |
ingest_group_is_atomic_under_a_mid_group_fault | SY-30 | a group written in chunks without a transaction, a compensation committed apart, a fault answered Ok, and an armed fault that never fires |
ingest_validates_every_event_not_the_head | SY-18 | per-event ceilings checked on each group’s head only, a compensation never checked, and a ceiling off by one |
ingest_holds_the_call_to_its_tags_per_batch_ceiling | VT-36, SY-18 | a tags-per-batch ceiling held to each group apart, a compensation left out of the total, a ceiling off by one, and a total refused as per-event, Park rather than Split |
only_the_adjudicator_compensates | SY-7 | a compensation minted under an incarnation the store never read, so its log names an author nobody assigned |
directional_merge_rules_compose | SY-10 | a store with the merge rule built in, which answers a group whose travelling guard its log breaks with a reversal of its own, and so judges, as a spoke, what the hub sends; and one that re-mints its own claim when a peer’s claim on the seat arrives |
bulk_ingest_is_idempotent_in_bounded_round_trips | SY-14, VT-24 | a store that consults what it holds for the first 64 events of a call only and writes a held event past them again, and one that refuses a call of VT-24’s 128 events as over its batch limit |
Each wrong store lives in tests/wrong_ingest_stores.rs, where a test runs
the whole family against it and asserts it fails exactly its rule.
SY-7’s rule opens two stores, so a fixture must give each store it
opens a fresh incarnation, and the rule fails one that does not. It plays
the runner above them and holds the adjudicator assignment itself,
mounting happenstance-sync’s shipped runner::Adjudicate hook: a seat
rule on the adjudicator and runner::Accept on the other (#450). What it
holds the store to is the identity a compensation is filed under; that the
shipped runner writes none under Accept is happenstance-sync‘s own
tests’. Every peer adjudicating, the configuration SY-7 forbids, is run by
the clause’s falsification test in this crate’s own unit tests, which
records the two compensations it leaves for one conflict.
SY-10’s rule opens two stores for each of its two walks, hub and spoke and
peer to peer, under the same requirement. It plays a runner that mounts one
merge rule per direction of the edge through the same hook: the hub
adjudicates what the spoke pushes, and the spoke takes what the hub sends
unjudged. What it holds the store to is writing what each direction hands
it and nothing of its own. The hub’s rule run on the spoke as well is the
clause’s falsification test, in this crate’s unit tests, and the shipped
runner::Edge is held to its half in happenstance-sync’s own tests,
against wrong runners that each mount a rule where they should not. SY-33’s
rule, which must exempt SY-38’s refusal (ADR-0098), is not written yet; the
exemption is carried forward to it.
§Capabilities
Three rules need the fixture’s co-operation, and say so rather than pass
when they do not get it. SY-30’s needs
MID_GROUP_FAULT and
arm_mid_group_fault, because only
the adapter can fail its own write between two rows. SY-18’s reads
MAX_EVENT_DATA_LEN,
MAX_TAGS_PER_EVENT and
MAX_METADATA_LEN, each mirrored from
the adapter’s own constant. VT-36’s reads
MAX_TAGS_PER_BATCH, mirrored the
same way; only a store whose call has a tags wall of its own states one,
today happenstance-cloudflare’s. MemoryIngestFixture declines the fault in its own words and declares
no ceiling, so it reports all three rules skipped; the faithful store in
tests/wrong_ingest_stores.rs runs them, and happenstance-sqlite’s
ingest spike runs the first two. The mount also emits
every_declined_capability_is_stated_by_this_fixture, which fails a
fixture whose declension is this crate’s default
(ingest_declensions_are_stated).
§The write-group family
A store that a runner or a StoreSyncPeer
reads from must hand its log back in the writes that made it (SY-39), so a
relay forwards each write as one group with its guard (SY-30). The family
mounts on the same IngestFixture, whose store must also implement
WriteGroupStore:
happenstance_sync_testkit::write_group_store_conformance!(
happenstance_sync_testkit::MemoryIngestFixture::new()
);| Rule | Clause | What it rejects |
|---|---|---|
an_append_reads_back_as_one_group | SY-39 | a store that records no groups, and one that merges adjacent appends |
an_ingested_group_keeps_its_boundary_and_guard | SY-39, SY-30 | a store that drops an ingested group’s guard, or merges what one call ingested |
compensation_joins_its_group | SY-39, SY-2 | a store that records a compensation as a group of its own |
a_group_after_a_redelivery_is_named_by_its_first_landed_row | SY-39, SY-11 | a store that names a group by the first position it allocated, not the first row it wrote |
a_read_never_cuts_a_group | SY-39 | a page filled by event, a first group skipped for being too large, and a resume above a row position |
pages_chain_on_first_and_end_with_nothing | SY-39, SY-37 | a page its budget cut that says nothing is waiting |
Each wrong store lives in tests/wrong_write_group_stores.rs, where a test
runs the whole family against it and asserts the exact set it fails.
§The convergence family
Not an adapter’s bar but a projection’s: a fold that declares itself
convergent, by implementing
ConvergentProjection, is run under
two interleavings of one set of events from two origins, and its read
models must be byte-identical; and run over one arrival order numbered
twice, densely and offset with gaps, and its read models must again be
byte-identical. The rules are called directly over a
ConvergenceFixture; there is no mounting macro, because there is no
adapter to mount them beside and no capability to decline.
| Rule | Clause | What it rejects |
|---|---|---|
convergent_projection_is_interleaving_independent | SY-20 | unit-ledger, and any declared fold whose answer depends on arrival order |
convergent_projection_cannot_observe_local_position | SY-21 | a declared fold that writes its checkpoint, a local position, into its read model |
non_convergent_projections_are_excluded_not_failed | SY-22 | a check with no exclusion, which fails FIFO cost layers, and a check that does not really reorder |
A payload is decoded here, and only by the projection under test: a fold
reads its events, which is what makes it a fold. The check itself reads no
event at all: it compares the bytes of two read models. What the runner
reads to select and route events is their type and the projection’s scope
tags, which is what SY-35 requires. The wrong projections live in
tests/wrong_projections.rs.
StockOnHandFixture and CostLayersFixture are the reference cases:
one domain, a sum that converges and a FIFO queue that cannot.
§The restore family
VT-6’s restore half: a peer whose store is restored from a backup must not
hand back an identity it already issued to a different event. Mounted on a
RestoreFixture, which supplies what no peer port has, a writer of the
store’s own events, a backup and a restore, and runs the adapter’s
documented post-restore procedure (VT-6’s re-mint) as part of the restore:
happenstance_sync_testkit::restore_conformance!(
happenstance_sync_testkit::MemoryRestoreFixture::new()
);| Rule | Clause | What it rejects |
|---|---|---|
restored_peer_does_not_reissue_identities | VT-6 | a store whose identity survives a restore, so a new event takes an identity a lost one was issued; a restore that restores nothing; and a store that stamps its current incarnation on what it reads back, so a re-mint rewrites restored history |
A family of its own rather than a capability on SyncFixture: a peer
that issues no identity of its own, as MemorySyncPeer does not, has
nothing a restore could reissue, and every rule of the peer family is held
against peers built on it. MemoryRestoreFixture, a store peer over the
in-memory oracle that restores into a fresh incarnation, is the reference;
the wrong peers live in tests/wrong_restores.rs, and
happenstance-postgres mounts the family with its re-mint and runs the
same sequence without it as its negative control.
§The watermark family
A projection store’s bar, not a peer’s: the store a runner keeps its
per-peer confirmation watermark in, beside the read model whose rows
depend on it (SY-31). The rule takes the
ProjectionFixture a projection
adapter already mounts the projection suite with, and is called directly,
as the convergence family’s are:
#[tokio::test]
async fn watermark_advances_transactionally_with_a_read_model()
-> Result<(), happenstance_sync_testkit::Violation> {
happenstance_sync_testkit::watermark_rules::watermark_advances_transactionally_with_a_read_model(
async || happenstance_testkit::fixtures::MemoryProjectionFixture::new(),
)
.await?
.report("watermark_advances_transactionally_with_a_read_model");
Ok(())
}| Rule | Clause | What it rejects |
|---|---|---|
watermark_advances_transactionally_with_a_read_model | SY-31 | a store that refuses the reserved sync/ prefix, keeps a watermark in its handle, keys every peer’s watermark alike, or upserts one around the port’s regression check |
The rule plays the runner, with the shipped
Edge and its
Watermark, so what it holds the
store to is what that runner asks of it. The wrong stores live in
tests/wrong_watermark_stores.rs.
§Harnesses
The emitter is a parameter, as in the event-store family, so the suite never
decides which runtime a peer is tested on:
emit_sync_tokio! (the default),
emit_sync_blocking! — no runtime and no
Send bound, which is what a !Send peer runs under — and
emit_sync_wasm!. The ingest family has
emit_ingest_tokio! (the default),
emit_ingest_blocking! and
emit_ingest_wasm!, the Durable Object’s. The
write-group family has
emit_write_group_tokio! (the default),
emit_write_group_blocking! and
emit_write_group_wasm!. The restore
family has emit_restore_tokio! (the default)
and emit_restore_blocking!.
§Failure is a value
A rule returns Result<RuleOutcome, Violation> and an emitted test returns
Result<(), Violation>, so a breach reaches the test runner as an error
naming its clause. Violation says why this family does not panic as the
other two do.
§Features
proptest, off by default: the model family above. Absent on wasm32 whatever the feature says, becauseproptestdoes not build there.
Modules§
- convergence_
rules - The convergence rules (SY-20, SY-21, SY-22).
- ingest_
rules - The ingest rules. See the crate documentation for what each covers.
- restore_
rules - The restore rules. See the crate documentation for what each covers.
- rules
- The sync rules. See the crate documentation for what each covers.
- sync_
model proptest - The model family: seeded peer topologies, driven through push and pull from one seed, and checked at quiescence against a model of their own.
- watermark_
rules - The watermark rule. See the crate documentation for what it covers.
- write_
group_ rules - The write-group rules. See the crate documentation for what each covers.
Macros§
- emit_
ingest_ blocking - Emits one plain
#[test]per ingest rule, driven byhappenstance_testkit::block_on. - emit_
ingest_ tokio - Emits one
#[tokio::test]per ingest rule. - emit_
ingest_ wasm - Emits one
#[wasm_bindgen_test]per ingest rule. - emit_
restore_ blocking - Emits one plain
#[test]per restore rule, driven byhappenstance_testkit::block_on: no runtime and noSendbound. - emit_
restore_ tokio - Emits one
#[tokio::test]per restore rule. - emit_
sync_ blocking - Emits one plain
#[test]per sync rule, driven byhappenstance_testkit::block_on. - emit_
sync_ model_ blocking proptest - Emits one plain
#[test]per model rule, driven byhappenstance_testkit::block_on: no runtime and noSendbound. - emit_
sync_ model_ tokio proptest - Emits one
#[tokio::test]per model rule. - emit_
sync_ tokio - Emits one
#[tokio::test]per sync rule. - emit_
sync_ wasm - Emits one
#[wasm_bindgen_test]per sync rule. - emit_
write_ group_ blocking - Emits one plain
#[test]per write-group rule, driven byhappenstance_testkit::block_on: no runtime and noSendbound. - emit_
write_ group_ tokio - Emits one
#[tokio::test]per write-group rule. - emit_
write_ group_ wasm - Emits one
#[wasm_bindgen_test]per write-group rule. - for_
each_ convergence_ rule - Hands the complete convergence rule set to
$callback. - for_
each_ ingest_ store_ rule - Hands the complete ingest rule set to
$callback. - for_
each_ restore_ rule - Hands the complete restore rule set to
$callback. - for_
each_ sync_ model_ rule proptest - Hands the complete model rule set to
$callback. - for_
each_ sync_ peer_ rule - Hands the complete sync rule set to
$callback. - for_
each_ watermark_ rule - Hands the complete watermark rule set to
$callback. - for_
each_ write_ group_ rule - Hands the complete write-group rule set to
$callback. - ingest_
store_ conformance - Generates the ingest conformance suite for a store adapter.
- restore_
conformance - Generates the restore conformance suite for a peer adapter.
- sync_
model_ conformance proptest - Generates the model family for a peer adapter.
- sync_
peer_ conformance - Generates the sync conformance suite for a peer adapter.
- write_
group_ store_ conformance - Generates the write-group conformance suite for a store adapter.
Structs§
- Cost
Layers - The cost of every issue, consuming the oldest units received first.
- Cost
Layers Fixture CostLayersover the same two sites: the honest non-convergent case SY-22 excludes.- KvCapped
Fixture - A
SyncFixturewhose peer stores each event as one KV value of at mostKV_VALUE_CAPbytes: Turnstile’s peer D, the shape SY-18’s marker names. - KvCapped
Peer - One handle onto a
KvCappedFixture’s peer. - Memory
Ingest Fixture - The reference
IngestFixture, over [MemoryIngestStore]. - Memory
Ingest Handle - One handle onto a
MemoryIngestFixture’s store. - Memory
Restore Fixture - The reference
RestoreFixture: a [StoreSyncPeer] over a [MemoryIngestStore]. - Memory
Sync Fixture - The reference
SyncFixture, over [MemorySyncPeer]. - Memory
Sync Handle - One handle onto a
MemorySyncFixture’s peer. - Refusal
- One refusal a fixture drove its peer into, and the class it declares for it.
- Stock
OnHand - Units received and units issued, as two sums.
- Stock
OnHand Fixture StockOnHandover two sites: the declared case SY-20’s rule is proven against before it may fail anyone’s projection.- Violation
- A peer did something a sync rule forbids.
Enums§
- KvPeer
Error - How a
KvCappedPeerfails. - Stock
- A stock movement at one site: the reference cases’ domain event.
Constants§
- BYTE_
LOG_ REFUSED_ REASON - The reason SY-37’s rule reports when the peer refused its byte-cut log, and the count-cut log showed no page.
- COUNT_
LOG_ REFUSED_ REASON - The reason SY-37’s rule reports when the peer refused its count-cut log.
- KV_
VALUE_ CAP - The per-value cap of Turnstile peer D’s KV store: 128 KiB.
- LOG_
REFUSED - The
capabilitySY-37’s rule reports when the peer showed no page boundary and refused a push of one of its logs that its own declaration admits. - MAX_
PROVOKED_ BATCH_ BYTES - The largest
max_batch_bytesbatch_over_the_limitwill build a batch over: 64 MiB. - MAX_
PROVOKED_ BATCH_ EVENTS - The most events one push a rule builds will carry: 4,096.
- MAX_
PROVOKED_ TAGS - The most tags SY-18’s ingest rule puts on one event, whatever ceiling the fixture declares (H-13).
- MAX_
PROVOKED_ TAGS_ PER_ BATCH - The most tags VT-36’s ingest rule puts in one call, whatever ceiling the fixture declares (H-13).
- NO_
CONFIRMATION - The
capabilitySY-31’s rule reports when the peer’s acknowledgement confirmed none of what the rule pushed. - NO_
CONFIRMATION_ REASON - The reason SY-31’s rule reports when the peer confirmed none of what it pushed.
- NO_
HIDDEN_ CLAIM - The
capabilitySY-38’s rule reports when the store did not put the rule’s compensation where its arrangement needed it. - NO_
HIDDEN_ CLAIM_ REASON - The reason SY-38’s rule reports when it could not arrange a claim hidden by the call’s own compensation.
- NO_
PER_ EVENT_ LIMITS - The
capabilitySY-18’s ingest rule reports when the fixture declares no per-event ceiling it can exercise. - NO_
PER_ EVENT_ LIMITS_ REASON - The reason SY-18’s ingest rule reports when it exercised no ceiling.
- NO_
SECOND_ PAGE - The
capabilitySY-37’s rule reports when the peer never showed it a second page: the two declared limits that decide how much one pull carries. - NO_
SECOND_ PAGE_ REASON - The reason SY-37’s rule reports when the peer never showed it a second page.
- NO_
TAGS_ PER_ BATCH_ LIMIT - The
capabilityVT-36’s ingest rule reports when the fixture declares no tags-per-batch ceiling it can exercise: theIngestFixtureconstant it reads. - NO_
TAGS_ PER_ BATCH_ LIMIT_ REASON - The reason VT-36’s ingest rule reports when it exercised no ceiling.
- UNSTATED_
COMPACTION - What
SyncFixture::COMPACTIONsays when a fixture says nothing. - UNSTATED_
MID_ GROUP_ FAULT - What
IngestFixture::MID_GROUP_FAULTsays when a fixture says nothing. - UNSTATED_
SYNC_ REOPEN - What
SyncFixture::REOPENsays when a fixture says nothing.
Traits§
- Convergence
Fixture - One projection, the events two origins authored, and a way to read what it committed.
- Ingest
Fixture - One isolated backing store that both appends and ingests, and a way to reach it.
- Restore
Fixture - One isolated far side whose storage can be backed up and restored, and a way to reach it.
- Sync
Fixture - One isolated far side, and a way to reach it.
Functions§
- batch_
over_ the_ limit - A batch one byte over
limits.max_batch_bytes, built from events no larger thanlimits.max_event_bytes, so a peer that checks both refuses it for the batch and not for any one event. - ingest_
declensions_ are_ stated - CF-18’s machine-checked half for the ingest family: no capability
Fdeclines carries the reason it would inherit by saying nothing. - oversized_
push_ refusal - The oversized-push refusal, provoked on
peerand declared [FailureClass::Split]. - sync_
declensions_ are_ stated - CF-18’s machine-checked half for the sync-peer family, for
SyncFixture::REOPENonly:Fmust not decline it with the reason it would inherit by saying nothing.
Type Aliases§
- EventOf
- The domain event a fixture’s projection folds.
- Refusals
Of - What
SyncFixture::refusalsresolves to for a peerP. - StoreOf
- The store a fixture’s projection commits into.