Sync

A device's writes commit locally. This page follows them from there to every other device: how they're published, in what order everyone applies them, what a sync pass does, and what the app sees while it runs. How a write merges into another device's rows is the Merge page.

The sequence

All of a store's history is one numbered sequence of objects in storage: <store>/seq/1, seq/2, seq/3, and so on. Each object is a slot, and a slot holds one of four things:

• writes: one device's queued writes, sent together in one pass;
• an admin entry: a membership or settings change, such as adding Carol;
• a key copy: a key sealed to one member (Encryption);
• a snapshot announcement: "a snapshot of this audience, covering slots up to n, is at this path" (Snapshots).

Every device applies slots in order, 1, 2, 3, and never skips one. Because everyone applies the same slots in the same order, everyone agrees on what happened and when. That one order decides who was a member when a write was made, which of two inserts of the same unique value came first, and when history can be deleted.

A device applies each slot in one database transaction together with "applied through this slot". If that transaction fails, both the data and the progress stay where they were.

Claiming a slot

To publish, a device claims the next slot: it creates seq/n+1 with a request the provider accepts only if nothing is there yet. Every supported provider has such a create-once request (Storage). If two devices try the same slot, exactly one wins.

Ana and Ben both have writes to send, and both have applied through slot 40. Both try to create slot 41. The provider stores Ana's and refuses Ben's. Ben reads slot 41, applies Ana's writes, and tries slot 42 with the same writes he had queued. Nobody's work is lost; Ben's lands one slot later.

A device claims slot n+1 only after it has applied slots 1 through n. No clock or deadline is involved: a device that comes back after a month catches up, then claims.

When the reply is lost

A claim can go wrong in a less obvious way: the request succeeds, but the reply never arrives. Ana retries, and the provider says the slot is taken (AlreadyExists). Taken by whom? Ana reads slot 41 and compares it with the exact bytes she sent. If they match, the slot is hers. Without that comparison she would claim slot 42 too, and her writes would land twice.

So before every create, a device records in its local journal what it is claiming: a claim number, the slot path, and the exact bytes. Each device numbers its claims 1, 2, 3, with one counter across every kind of slot. It settles its current claim before starting another, and after a crash or a lost reply it settles first: read the recorded slot, compare, and only then move on.

What the device finds at its recorded slot	What it does
its own bytes	treats the claim as landed, after checking the floor (below)
different bytes from another device	applies them, checks the work is still allowed, and claims the next slot
different bytes signed with its own device id	stops sending: another copy of this device exists (below)
nothing	retries the same path with the same bytes
a failed read	leaves the claim unsettled and tries later

The same bytes go out on every retry. Coven encrypts slots with nonces derived from their content (Encryption), so sealing the same writes again reproduces the same object, byte for byte.

The floor

Old slots are eventually deleted, once snapshots cover them (Snapshots). That creates a trap: once slot 41 is deleted, its path is empty again, and a create-once request for it can succeed. A request from long ago, delayed in the network, could land there and look like a real slot.

The defence is one record, the floor, at <store>/floor. It names the first slot still kept and, for each audience, the snapshot it relies on. A device reads the floor after reading slots and before applying them. Anything below the floor is not trusted; the device loads a snapshot instead. A device that has just claimed a slot checks the floor the same way before believing its claim. If the floor has passed the slot it was claiming, it settles the claim from the floor's snapshot, which records each device's last claim number and a hash of that slot's bytes.

A sync pass

A device syncs in passes. A pass does four things, in order:

1. Read forward. Read seq/<last applied + 1>, then the next, until one is missing. On a miss, read the floor. A device far behind finds the end quickly by checking n+1, n+2, n+4, … and narrowing the gap.
2. Apply. Check each slot's framing and complete signature, then apply it in one transaction with "applied through this slot". The floor read at the final miss also checks that nothing just read has been deleted.
3. Claim. Settle any unsettled claim, then send: at most one writes slot holding this device's queued writes, plus any admin operations the app asked for.
4. Effects. Do what the new state calls for: share or unshare provider access, replace keys a removed member had seen, take snapshots, and delete history the floor has passed.

A pass over a quiet store, with nothing new and nothing to send, costs two requests: the next slot, which is missing, and the floor. It lists nothing and writes nothing, whether the store has three devices or twenty, and 20,000 files or none. If Ana added one note, Ben's next pass makes three reads: her slot, the miss after it, and the floor.

A writes slot carries every write this device has queued and can send, in the order they were made. It has no size cap; a large one uploads in parts and becomes visible only when complete. Each single write must fit SQLite's 1 GB value limit, or the commit itself fails with DbError::TooLarge. Writes the app commits while a pass is sending wait for the next one. A queued write leaves the queue only once its slot is confirmed and applied.

Files don't travel in slots. A separate upload worker sends them on its own schedule, so a pass never waits for a file (Files).

Order

Each write records read_through, the last slot its device had applied when the write was made. That record never changes, even if the write is sent much later. A write's past is every write in slots up to its read_through, plus its own device's earlier writes.

Ana creates note 43 in slot 20. Ben reads through 20, then attaches a file to it. Whatever slot Ben's write lands in comes after 20 on every device, so every device has note 43 before the attachment. Applying slots in order is all it takes.

Ben and Carol both edit while at slot 20: their edits are concurrent, even if Ben's lands in slot 21 and Carol's in slot 30, because neither saw the other. Which value a cell keeps is decided by timestamps, not slot numbers (Merge).

Every write is checked against its author's view: its signature with the author's key as it stood at its read_through, and its author's membership and circles there. The app sees its own commits at once, before they claim a slot; the device shows the published slots with its local queue on top.

A stuck device stops

A device that can't apply a slot stops there. It may be missing a key, need an app update to read a newer format or schema, or have read damaged bytes. Until it can apply that slot it applies nothing after it, reloads nothing, and sends nothing: no writes, no files, no operations.

Stopping is deliberate. Skipping slot 80 could mean missing Ben's removal at 81 while still sending writes that removal should have cut off. Ben's laptop reads slot 41, a copy of a Gifts key it can't open; a key that arrives in a later slot can't get it past 41.

The app can still read its data and commit the local writes it is allowed to. The last pass is LastPass::Blocked with the slot and its reason: KeyUnavailable, UpdateRequired, or a Refused cause for damaged bytes. The sync state follows the reason: waiting to sync, on the slot (SyncWait::Blocked), for a key that hasn't arrived; update needed; or failed for damaged bytes. A slot whose bytes haven't all arrived is a failed read, retried; only a complete object that fails a check is refused.

Two things look like blocks but aren't:

• Circle parts this member can't read. A writes slot can carry parts for several audiences. Parts for circles this member isn't in are passed over without being opened (Circles).
• Writes refused by a removal. Writes from a device or member that land after their removal, or after their member's keys were replaced, have no effect. Every device consumes the slot, records WritesRefused naming the entry that decided it, and moves on (Sharing).

Waiting and retrying

A device sleeps until something can have changed: a write from the app, a changed prerequisite, or a retry that is due. While idle, it runs a pass every 30 seconds; a local write or handle.sync_now() asks for one at once. Requests that arrive during a pass, or during connection backoff, coalesce into the next one. Coven watches the device's network: when a usable network appears, every wait that followed an attempt with no answer from storage ends at once, and a pass runs. The app makes no call for it.

Automatic retries share one schedule of delays: 1, 2, 4, …, 256 seconds, then 300. The deadline is saved with the work, so reopening the app neither restarts the wait nor skips it, and a clock changed in between can shorten a wait but not stretch it past its full length. A provider's Retry-After can lengthen a wait, and it applies to explicit retries from the app too. A wait for an app action or an app update makes no requests at all.

When the person pulls to refresh, handle.retry_now().await? ends every ordinary wait and returns a future that resolves when the pass it runs ends. What a retry would do right now is the one row of the coven_retry_effect table: pass, cooling_down until cooling_until, failed for a failure only new credentials, an update or a reset can clear, or not_syncing. The same row decides the retry's answer, so an app that follows it with a live query, handle.subscribe(RetryEffect::read), and offers Retry Now only while it says pass offers it only when pressing it runs a pass.

Status and pending work

handle.start_sync() starts the loop; its first pass builds the provider client from the credentials in custody. Setting up storage starts it too. handle.stop_sync().await? tells waiting retries that sync isn't running, then returns once the active pass and file transfers have finished and the provider client is released, keeping keys, credentials and the storage location. A start_sync asked meanwhile starts after it. Neither touches what's in storage.

A status screen shows one sync state and when sync last finished. handle.sync_status().await? reads them, and handle.subscribe_sync_status() follows them: its first value is the current state, so the screen needs no placeholder. Coven names the states; the app picks the words:

pub struct SyncStatus {
    pub state: SyncState,
    pub last_synced: Option<SystemTime>, // when a pass last reached the end
}

pub enum SyncState {
    NotSynced,                    // no storage location recorded here
    Syncing,                      // a slot or a file is moving
    WaitingToSync(SyncWait),      // held back, and what holds it
    Synced,                       // nothing waits
    Offline(SyncCause),           // the last pass got no answer from storage
    OfflineWithUnsentChanges(SyncCause), // ... and a write or a file is queued
    SignedOut { provider: CloudProvider, cause: Option<SyncCause> },
    NoAccess(SyncCause),
    StorageFull(SyncCause),
    FolderMissing(SyncCause),
    ClockWrong(SyncCause),
    UpdateNeeded(SyncCause),
    Removed(SyncCause),
    NeedsReset(SyncCause),
    Failed(SyncCause),            // any other failure, or a file that can't upload
}

The state is computed in each read from what coven knows: whether storage is set up and its credentials held, how the last pass ended, what is moving, the send and upload queues, the network and the pending records. When several facts hold, the most urgent shows: the device's own standing first, then signing in, then what storage said of the account, then reachability, then failures, then activity. The cause is the last pass or the first pending record that names the state.

Synced means nothing waits: the last pass reached the end of the sequence and every file this device uploads is stored. A file queued with nothing moving waits to sync, and the state names why: sync stopped, a provider's cooldown, a slot that can't apply yet, the app's pause, a metered network while the app moves files only on unmetered ones, the file's own retry, or its turn. A pass held at a slot waiting for a key, or cut short by a cooldown, waits to sync too, though nothing is queued. Offline describes the last attempt, not the connection's history: Ana syncs, then loses Wi-Fi, and her next attempt is offline. A provider error that denies a required read shows no access to storage. One photo whose original went missing while the rest syncs shows failed, naming the photo, until it uploads.

Only the handle that runs sync has the state: a widget's read-only handle can't tell whether the app that runs sync is still running, so it has no sync_status, and its SQL can't read the rows the state comes from (StatementForbidden).

Anything that can't proceed has one record in handle.pending() (live: a query over the coven_pending view, with each record's error in its cause column), offline too. Each PendingRecord names its subject, such as a write, a slot, a file, an operation or the connection, and its first unmet condition as a typed PendingReason: "needs an app update", "key not available yet", "waiting for Ana's device to unshare", "source file moved". The record stays until that condition changes or the work settles; time passing never clears it. record.retry() says how it will retry: Automatic on the schedule above, AfterUpdate, AppAction (for example new credentials), or Never. record.cause is the text of the error behind it, written with the record by the run that met the error; opening clears it, and a failure an app call received leaves none. The coven_pending view gives it as cause (Coven's state in SQL).

If coven can't save a pending record, it stops syncing and fails database calls with DbError::StatePersistence until the store is reopened, so the app never sees an empty list that hides a lost update.

Another copy of the same device

Restoring a phone from a backup can leave two installations with one device id. Both would sign slots as the same device and number their claims alike, and each could mistake the other's slot for its own.

The first defence is custody: a device's id also lives in a device-only keychain entry, which a backup doesn't carry to another device (on Apple, it comes back only when a backup is restored onto the same device), and opening with a missing or different id there is refused. A copy that does open stops sending with SyncError::DeviceIdInUse as soon as history shows another installation using its id: a slot signed by its id at a path its journal never attempted, or different bytes signed by its id where its own claim should be.

Coven never resets a device on its own. The app can call CovenBuilder::reset_device(store, device_name, on_status), which loads the store from storage under a fresh device id, reporting how far it has come to on_status as a restore does, registers it as replacing the old one, and moves the old directory aside for inspection. A damaged database is reset the same way: any call whose statement meets the damage fails with DbError::DamagedDatabase, opening included. Unsent writes from the old installation are not imported; handle.device_reset() returns the stored notice with the old and new ids, so the app can tell the person that unsent edits may have been lost.