Module read_stream
event-store only.Expand description
The pooled-cursor read stream.
§The constraint this type exists to reconcile
Two requirements pull in opposite directions, and neither is negotiable.
EventStore::read is not async:
the stream sits at the top level of the return type, because that is what
lets the Send flavour mark the stream Send rather than merely the
future that produces it (ADR-0001, ADR-0008). So read cannot await, and it
cannot acquire a connection.
sqlx’s fetch borrows its executor:
pub fn fetch<'e, 'c: 'e, E>(self, executor: E) -> BoxStream<'e, Result<DB::Row, Error>>
where E: Executor<'c, Database = DB>The returned stream is tied to a connection that, at read time, does not
exist yet. Writing the obvious thing — acquire, then hand the borrow to
fetch, then return the result — produces a self-referential value, and the
compiler says so:
error[E0597]: `conn` does not live long enough
|
| sqlx::query("SELECT position FROM event").fetch(&mut *conn)
| ------------------------------------------------------^^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `conn` is borrowed for `'1`
| }
| - `conn` dropped here while still borrowed§The shape that works
Do not hold a borrowed stream at all. Hold the owner and re-borrow it for the duration of each step:
readcaptures the pool and the query and returns immediately, in stateReadState::Unstarted. Nothing is acquired, so nothing is borrowed.- The first
poll_nextstarts a future that acquires a connection, opens a transaction andDECLAREs a server-side cursor. That future owns the [Transaction], so there is no borrow to outlive. - Every subsequent chunk is a
FETCH, run by a future that takes the cursor by value and hands it back with the rows. The&mutborrow of the transaction lives entirely inside oneawait, which is a borrow the compiler can see the end of.
That is why the state machine is written by hand instead of with
futures_util::stream::unfold: unfold would do, but it makes the type
unnameable, and this is precisely the type the phase-2 portfolio exists to
record.
§A server-side cursor, not keyset pagination
The cheaper-looking alternative — re-issuing
WHERE position > $last ORDER BY position LIMIT n per chunk against a
connection borrowed only for that statement — sidesteps the borrow problem
entirely and is wrong. The specification’s ES-11 requires one read to
be evaluated against a single consistent state of the store, fixed no later
than the first poll; a self-paginating read grows under the caller’s feet.
BEGIN ISOLATION LEVEL REPEATABLE READ plus DECLARE CURSOR buys that
snapshot from the server, which is the whole reason the transaction is held
open across the stream’s life rather than the connection alone.
The cost is honest and belongs in the capability table: a reader occupies a
pooled connection and an open transaction for as long as the caller holds
the stream. A slow consumer pins a connection and holds back VACUUM.
§ES-11, and a wrong conclusion that is worth recording
Both read_result_is_stable_under_concurrent_append and
query_items_share_one_snapshot pass. They did not always, and the reason
they now do is worth writing down, because the first diagnosis was wrong in a
way that would have cost a specification amendment.
ES-11 requires a read to be evaluated against a state fixed no later than the first poll. Both rules exploit the gap deliberately: poll once, append, then drain. With the cursor’s opening held as an inline future, that future makes progress only while someone is polling it — so the snapshot was taken during the drain, strictly after the append, and both rules failed five times in five runs.
The conclusion drawn from that was that an async driver cannot satisfy the clause at all, since its first poll can only start the round trip. That was wrong, and consistent failure is what made it look right.
What fixes it is that the work is handed to the runtime at the first
poll and every other operation is too. open_cursor is spawned here;
append is spawned through PostgresEventStore::on_runtime. Both land in
the same queue, the read’s task was enqueued first, and so the snapshot is
taken before the append runs. “Fixed no later than the first poll” becomes
true in the only sense available to an async adapter: the work was
committed to at that poll and no longer depends on the caller polling
again.
The bridge was not built for this. It was built because sqlx panics
without a runtime in thread-local scope and the conformance suite’s
concurrency family drives its contenders on raw threads. That it also
settles ES-11 is why the earlier diagnosis survived as long as it did: the
two failures looked structural because the thing that would have disproved
them had not been written yet.
Structs§
- PgRead
Stream - The stream
PostgresEventStore::readreturns.