Sync

coven syncs SQLite row changes between devices that share a store. Each store selects one signed write policy. MergeConcurrent appends each device's changesets to its own causal stream, merges concurrent edits column by column, and makes deletes win over concurrent edits. Serial activates every changeset and control operation through one global compare-and-swap head. The unit of exchange in either policy is one host transaction: its SQLite changeset becomes a Store package named by an exact signed commit.

The diagram below is the MergeConcurrent path.

Nothing in storage is overwritten. Publication creates exact immutable objects for a package, commit, and head, then reads each object back before advancing durable state. A puller records the exact hash at every materialized device sequence, not a sequence number that could disagree with the accepted commit.

Examples use the todos app (workspaces hold lists, lists hold todos, todos carry attachments and labels); Alice and Bob share the store.

This page covers how a local write reaches every device. Row-level gating (which rows stay local) has its own page, Local data; fresh-device bootstrap from a snapshot has its own page, Bootstrap.

Write policies

The host must pass one WritePolicy to Coven::builder(...).write_policy(...). The signed Store protocol root binds that choice permanently; open, join, and restore refuse a different expected policy before touching storage or local state.

The local SQLite database independently persists the same required policy. On first open, Coven creates its complete internal schema, policy row, and initialization marker in one SQLite transaction. Later writer and read-only opens require the marker and validate the requested policy; a missing policy, invalid marker, or different policy fails the open without recreating metadata.

• MergeConcurrent keeps one append-only commit stream per device. A commit names its exact predecessor and materialized dependency frontier. Devices may publish while offline, and pull merges independent branches.
• Serial keeps one append-only global stream selected by a signed mutable head. Publishing reads the head, builds a consecutive package of commits, and replaces the head only if its provider version still matches. Losing that comparison preserves the local writes as a PendingBranch; the host must explicitly discard it or rerun its intent against the current global state.

Opening either policy works without a provider. Running, joining, or restoring Serial sync requires the provider's separate coordination capability. CloudKit and AWS S3 provide it. A custom S3 endpoint requires the host's explicit CustomS3Serial::ConditionalPutAndStrongReads assertion; Google Drive, Dropbox, and OneDrive are refused.

Change capture

A missed write is a silent divergence: two devices disagree and nothing reports it. If capture meant the host reporting its own changes, every forgotten call site in every app would be such a miss. So coven owns the connections and records changes itself; a host write that skips the recording cannot happen, because the only connection that can write is the one capture is attached to.

The host opens the store once through Coven::builder(config).write_policy(...).synced_tables(...).migrations(...).open(), declaring its synced tables, and from then on runs all its writes through handle.sql(...). The writer connection lives on one dedicated thread (an actor). Each host transaction gets a SQLite session attached to every declared table. Its insert, update, and delete operations become one changeset; the session ends with that transaction. The host writes as usual, and there is no host-lent pointer to a connection coven does not own.

Each declaration also states its row identity. (table, id) names one logical row across every device: independently created rows use canonical UUIDv4 or UUIDv7 ids, while SharedKey tables intentionally merge equal application keys. Before the host transaction commits, coven validates introduced ids and records a primary-key change as deletion of the old identity plus insertion of the new one. The app rows, shared changeset, exact materialized dependency frontier, stable WriteId, affected row identities, and initial WriteStatus commit together or all roll back. handle.sql and handle.write return a WriteReceipt; separate successful calls never combine into one Store commit.

The set is not a tuning knob. With no tables declared the session attaches nothing and produces empty changesets forever, so init_sync_over_storage treats an empty set as a hard error and refuses to start.

Initialization also installs policy-shaped signed authorization. A new MergeConcurrent store publishes its self-signed Owner founder and causal head, then records the founder and complete accepted head floor. A new Serial store derives founder membership and key generation directly from the signed Store protocol root; it creates no causal membership stream or head. Both policies finish authorization before returning a runnable session. This applies to opaque and browsable homes; browsable changes visibility and blob paths, not authorization.

Reads

Reads don't need capture, and they don't get it. Two read paths exist, and both hold the invariant above the same way: they run on read-only SQLite connections, so a write through them is refused by SQLite itself — a read cannot bypass capture because it cannot write at all.

• In the host's own process: handle.sql_read(...). The full handle opens a read-only companion connection on the same WAL database, on its own thread; a pure read runs there, concurrent with the writer instead of queued behind it, with no session attached. Read-your-writes holds for committed writes: a sql_read after an awaited sql/write sees that data.
• From a second process (or a second handle): Coven::builder(config).write_policy(...).synced_tables(...).migrations(...).open_read_only() returns a CovenReadHandle — a same-store reader for something like a macOS File Provider extension that must serve reads while the app holds the full handle open. It takes no store lock and runs no migrations (it refuses a schema newer than its binary), and it exposes reads only: SQL, and blob reads that may fetch from the cloud into the device cache. WAL makes the coexistence safe: many readers, one writer, each read seeing the last committed state.

The write path polices itself: a handle.sql(...) transaction that changed no rows at all logs a warning — that is a pure read left on the write path; move it to sql_read. A write to a device-local (undeclared) table changes rows while capturing nothing, which is routine and silent.

The sync cycle

One background loop runs one cycle at a time. Each cycle loads durable state and:

1. Resolves authorization from causal membership heads for MergeConcurrent, or from the exact signed global head chain for Serial, then refreshes encryption-key and device-registration state.
2. Drains blob uploads and retries the oldest prepared Store write using its persisted exact bytes.
3. Completes ready row-gate transitions and pulls verified remote Store commits.
4. Prepares pending writes in policy order, uploads and verifies their referenced blobs, then appends and verifies their packages and commits. MergeConcurrent activates each with its device head; Serial activates one consecutive package with a conditional global-head replacement.
5. Applies remote commits whose policy-specific predecessors are fully materialized; each commit's rows, authorization state, and exact position advance atomically.
6. Flushes the register clock, durable file cleanup, acknowledgements, and blob deletion work.
7. Evaluates snapshot publication and reclamation against exact commit coverage.

A host transaction's capture session exists only inside that transaction, so a host write can land during any network operation without joining another write. Remote applies use the engine's apply path rather than the host transaction path and therefore never enter the local write ledger.

Under MergeConcurrent, when Alice edits a todo title, that call already leaves a durable pending write. Her loop creates encrypted exact objects at store-v1/candidates/<family>/packages/<alice-device>/<seq>/<hash>.pkg, store-v1/candidates/<family>/commits/<alice-device>/<seq>/<hash>.json, and store-v1/heads/<alice-device>/<seq>.json. Bob verifies Alice's head and commit, waits until the named dependencies are materialized, then atomically applies the package and records Alice's exact sequence and commit hash. The signed commit derives <family> from its Store, author registration, write identity, policy, sequence, and predecessor. Its candidate-object manifest must exactly equal the package and other candidate-exclusive objects reached by its closed body.

Under Serial, the same package and commit use the serial stream. The loop reads the signed global head with its provider version, prepares consecutive commits from that exact base, appends and verifies their immutable objects, then conditionally replaces the head. A peer follows only the exact predecessor chain selected by that head.

Push

A commit stream is only trustworthy if its sequence numbers never skip and never change meaning, even across a crash. The durable write record owns the changeset and dependency frontier from the host commit onward. Preparation assigns each write its policy-specific sequence and predecessor, constructs the exact signed commit and activation bytes, and persists them before any protocol append. A retry creates and reads back the same journaled exact objects.

Before an append, the write is Publishing. A storage or readback failure puts it back in Pending; the loop's reconnect and backoff policy owns the retry. A missing blob, a still-local user blob, invalid package data, or invalid Store protocol state becomes typed durable Blocked and holds later writes behind it. After the head is read back, one SQLite transaction records the exact PublishedPosition, advances the local materialized position, applies owned cleanup metadata, and clears the prepared bytes from that same write record. For Serial, a head version mismatch records the whole prepared local branch as Conflict; it never silently rebases or drops those host transactions.

The host lists blocked records with handle.blocked_writes(). After repairing the named prerequisite, handle.retry_blocked_write(&write_id) requeues the blocked records and wakes sync. One Serial retry covers every blocked member of that ordered branch, and preparation validates the complete active branch, including members that remained Pending. If the write must be abandoned, handle.discard_blocked_write(&write_id) atomically reverses it and every later unpublished write whose working rows depend on it. Discarded records remain queryable with terminal Resolved(Discarded) status and no longer participate in preparation.

A peer must never learn of a row whose file is not yet in the cloud. That ordering rides the gate, per root, not a global hold: the cycle publishes whatever the gate emits and never holds the whole changeset back while uploads drain. A root being made remote stays gated off (local-only) while its blobs upload. When the last upload lands, coven flips the gate on and breaks the drain, and the gate re-emits the root's full subtree in that same cycle. One slow upload therefore delays only its own root. The host's BlobTransitionObserver only reports progress and completion; coven, not the host, decides when to publish.

Pull

MergeConcurrent pull lists signed device heads and makes a commit ready only after its predecessor and every exact dependency are materialized. Serial pull opens the complete visible candidate set for the signed global-head slot, requires every copy under one semantic hash to open to identical bytes, rejects multiple valid hashes as a fork, and then verifies the complete predecessor chain selected by that authoritative head. Unreachable immutable commits are inert, and provider listing order never chooses a Serial winner. The same candidate-set rule applies to packages and other signed objects in either policy. For each ready commit, pull:

• parses the signed commit and checks its schema_version against the local Database::schema_version;
• verifies the commit, policy-specific activation head, package hash, and Ed25519 signatures;
• checks the author against the policy-shaped authorization state: the exact causal membership grant for MergeConcurrent, or the preceding global prefix for Serial;
• validates every row id under the table's declared identity mode; an invalid id holds that exact Store commit without changing rows or its materialized position, while other device chains continue;
• applies the package and exact materialized position in one SQLite transaction, advancing the clock past its stamps;
• downloads any CacheEager blobs it references into the cache.

The materialized ledger advances only after the package, bookkeeping, and required blob work succeed. A failed blob download leaves the exact position unmaterialized, so the commit is retried; the pull reports this through PullResult::asset_downloads_failed.

A provider or network failure while reading a candidate or blob is a transport failure and drives SyncLoopStatus::Offline. A verified blob whose plaintext does not match its signed hash is invalid content, and failure to create or write its local cache destination is a local filesystem failure. Those two categories hold or fail the affected work without changing the loop to Offline.

Failure isolation

Under MergeConcurrent, no single cloud object may stop more than its own device stream. Under Serial, every commit belongs to the one global stream, so a malformed or missing object stops that exact global position and all successors; the materialized position never skips it.

• A malformed package or commit holds that device's position and stops pulling that device for the cycle; every other stream proceeds.
• An invalid signature (forged or corrupt) does the same, and is surfaced as a held Store position so the host can warn.
• A commit whose verified author is not a write-capable member under the entry it is signed against (revoked, or a read-only Follower) is skipped and the materialized position advances past it, surfaced as unauthorized: the client must not stay stuck behind an author who will never become valid.
• An unparseable or forked head is reported against that device and cannot select an alternate candidate by listing order.

How edits merge

Applying a changeset is its own subject: the hybrid logical clock that orders edits, the column-level three-way premerge, remove-wins deletes, and the future-skew bound all live on the Merge page. The cycle's part is only when: each changeset is applied, and the clock advanced past its stamps, as it lands during pull.

Schema versioning

Devices upgrade at different times, so two schema versions are routinely live against one store; the version stamp is what lets them coexist instead of corrupting each other. Every outgoing Store commit carries the device's schema version: the top rung of the host's migration ladder, reported by Database::schema_version. Pull enforces it two ways:

• Hard floor. If the local version is below storage's min_schema_version, pull returns PullError::SchemaVersionTooOld and syncs nothing. Its Display is the message shown to the user: update the app to keep syncing. This is permanent until the user upgrades. The floor object is untrusted input, so it is honored only when signed by a current Owner; anything else is a freeze or downgrade attempt and is ignored.
• Per-changeset skip. A single changeset whose schema_version is above the local one is skipped (counted in PullResult::skipped_schema); the device leaves its materialized position where it is and stops pulling that device for the cycle. The position is deliberately not advanced, so once the app upgrades the next cycle re-fetches from that sequence and applies it.

How migrations, this version number, the min_schema_version floor, and snapshots fit together, with worked examples for additive vs. structural changes, is its own page: Schema evolution.

Lifecycle

CovenHandle owns the sync lifecycle. The host calls handle.connect_sync() once a provider is connected; the handle builds the cloud home and, if sync is enabled, spawns the loop. handle.stop_sync() stops the loop after the in-flight cycle but keeps the installed manager so handle.start_sync() can resume it; handle.disconnect_sync() additionally drops the manager and its cloud home. handle.is_syncing() reports whether the loop thread is running, and handle.sync_now() asks the loop to run a cycle now.

The keys the loop signs and encrypts with are resolved from custody at each sync start: the OS keyring by default, or whatever preset the store's key_custody selected before open() — see Keys for the presets and what each one protects against. Either way, the host names its keyring service once at startup with set_keyring_service, which also installs the platform keyring store (apple-native on macOS and iOS, android-native on Android, windows-native on Windows; a target with no bundled store errors). There is no environment-variable or dev-mode key path.

The loop runs on a dedicated OS thread with its own current-thread tokio runtime. Database access goes through async calls on the Database handle, so the loop holds nothing tied to a thread; the dedicated thread is for stack size (aws-sdk-s3's endpoint resolution recurses deeply enough to overflow the default secondary-thread stack in debug builds). The loop stores the current SyncLoopStatus in a watch channel; the host observes it with CovenHandle::subscribe_sync_status:

pub enum SyncLoopStatus {
    Offline,
    CheckingStorage,
    Publishing,
    Synchronized(SyncLoopSuccess),
    Conflict { success: SyncLoopSuccess, branch: PendingBranch },
    Blocked { success: SyncLoopSuccess, writes: Vec<PendingWrite> },
    Failed { error: String },
}

The receiver immediately contains the current value and survives loop restarts. Intermediate values may be coalesced, so Synchronized.row_changes is a refresh hint rather than a complete event stream. Failed carries a user-facing message for a whole-cycle failure. Synchronized, Conflict, and Blocked carry SyncLoopSuccess, including alerts, device activity, and applied row changes. Conflict additionally names the stale Serial branch that requires explicit discard or replacement. Blocked names writes whose typed prerequisite prevents publication.

Backoff

A failing cycle should slow its retries, and a healthy one should not delay a fresh edit. One exponential formula (30s · 2^n) drives the cycle wait. A successful cycle waits the base 30 seconds before the next run; each consecutive failure doubles the wait (60s, 120s, 240s), capped at 300 seconds. A success resets the count, and sync_now preempts the wait.

Provider and network transport errors leave writes retryable, set Offline, and recover through the loop. Remote content mismatch and local blob-filesystem errors are not connectivity failures; they remain typed failed or held work. A write whose own package, blob state, or Store protocol state is invalid is durable Blocked and requires retry_blocked_write after repair or discard_blocked_write; reconnect does not silently requeue it. The schema-too-old floor requires an app upgrade, and membership rejection means the device is no longer a write-capable member.