Cache

An uploaded file lives in storage, encrypted. To show a photo or play a song, a device needs its bytes on local disk. The cache is where those bytes live: each device keeps uploaded files, and pieces of them, within a size budget the app sets. This page covers the budget, pinning and progress. How a file is staged, uploaded and read is the Files page.

The cache holds only uploaded files, so everything in it can be fetched again. A file still waiting to upload is read from its source on this device: the user's original, or coven's own copy of app-provided bytes. The cache keeps the encrypted chunks as storage holds them, checked as they arrived; reads decrypt them.

The budget

Each namespace the app declares when it opens the store has a cache budget in bytes (Files). Each namespace evicts on its own, so a big namespace can't push out a small one:

vec![
    Namespace::new("audio", Provenance::UserProvided, 2 << 30),
    Namespace::new("covers", Provenance::AppProvided, 64 << 20),
]

The declared budget applies when the namespace is first declared. After that, sql.set_cache_budget("audio", bytes), a step of the app's write, changes it and evicts down to it in that write (handle.set_cache_budget does the same in a write of its own), and handle.get_cache_budget("audio") reads it. Declaring a namespace twice fails opening (SchemaError::DuplicateNamespace).

When a namespace is over budget, the least recently used files and chunks go first. Freeing space never fails a read. Pinned files don't count toward the budget: with 20 GB pinned in a namespace budgeted at 20 GB, the next track played from the cloud is still cached, up to 20 GB, and nothing pinned goes.

sql.evict_file(id), a step of the app's write, removes a stored file's copies from the cache, pinned or not, and handle.evict_file(id) does it in a write of its own. It never touches storage or a file waiting to upload; a later read downloads it again.

sql.clear_cache("audio"), also a step of the app's write, removes every unpinned file and chunk of a namespace. Pinned copies and the partial downloads pins left stay, as does every other namespace; handle.clear_cache("audio") does it in a write of its own. An undeclared namespace fails with DbError::UndeclaredNamespace.

Cached bytes last while a record on the device names them. When the last row naming a file goes, the write deleting its record removes the file's cached copies too, pinned or not. A file several records share, such as a moved row's new file and the one it was made over, keeps its cached copies while any of them is on the device. Making a file device-only takes them the same way. A download under way meanwhile keeps nothing: a read still returns its bytes but caches none, and a pin fails with DbError::NoLongerStored.

Reads fill the cache

A device caches a file when it reads it; a record arriving downloads nothing. Reading a stored file that isn't cached fetches only the chunks the read needs and keeps them, so reading the same range again costs nothing. Ben plays the first minute of a song on his laptop: those chunks are cached. He seeks to the chorus: only the chunks under the new position are fetched.

Pinning

The cache is evictable. Pinning keeps a file whole on the device whatever the budget, until the app unpins it, and its bytes don't count toward the budget: the gesture for "keep this for the flight".

let files = vec![handle.file_ref(track_file).await?];
let prepared = handle.prepare_pin(&files).await?; // downloads; pins nothing
handle.write(move |sql| {
    sql.pin(prepared)?; // every file whole on this device, as this write commits
    sql.execute("UPDATE albums SET offline = 1 WHERE id = ?1", [&album])?;
    Ok::<_, CovenError>(())
}).await?;

Preparing assembles a complete cache file from the chunks already cached and the ones it fetches, checks it against the record's content hash, and syncs it to disk; the write's pin step keeps it, so the pin and the app's own rows change together. handle.pin(&files) prepares and commits in a write of its own. A prepared pin no write commits keeps nothing, and a cancelled or crashed prepare leaves no half-downloaded file exempt from eviction. Only stored files are pinned: pinning a file still on one device only, this one included while it uploads, fails with FileReadError::DeviceOnly, and the app pins it once it is stored.

While the app moves files only on unmetered networks (set_unmetered_only, Files), preparing on a metered network waits before it asks storage for bytes, until the network is unmetered or the setting goes off; dropping the call cancels it. Reads don't wait: a song that isn't pinned still streams.

To keep a file here as it goes to the cloud, stage it with Place::CloudKeptHere, or move it with MoveFile::AppKeptHere or MoveFile::UserKeptHere. The write that switches it to the cloud pins it, and coven makes that pin itself, resuming after a restart: an app file's cache copy comes from coven's own copy of its bytes, with no request to storage, and a user's file downloads, since coven never reads an original for the cache. Its failures retry under the shared backoff, with pending records of subject PendingSubject::Keep.

sql.unpin(&ids), a step of the app's write, stops keeping them and removes their copies from the device in that same write, cached chunks included and whatever room the budget has, so the app can record the change in its own rows as it happens; a later read downloads the file again. handle.unpin(&ids) is the same step in a write of its own. To show which files are kept offline, files_pinned(files) answers with one Option<bool> per file id, None for a file this device has no record of, and subscribe_files_pinned keeps the answer current. The app's SQL reads the same answer from the coven_pins table, which lists every pinned file by id, to count or filter its own rows by it, and its local triggers can follow it (Coven's state in SQL).

What the device keeps

handle.local_bytes() measures what a store takes on the device: for each declared namespace its budget, the bytes of its cached chunks and headers (cached, which the budget counts) and of its pinned files (pinned, which it doesn't); the partial downloads stopped pins left (pinning); the store's own files (library); and, while a reset's old installation is still there, its size (before_reset, removed by handle.delete_discarded_devices()). handle.subscribe_local_bytes() gives a measurement at once and a new one after each change to a namespace's bytes or budget; reading a cached file or a partial download growing starts none.

Progress and failures

Pin progress is a PinProgress: files completed and requested, and bytes cached and total. It counts only checked bytes committed to the cache, never bytes merely received, so after a restart the first reads show the same progress, even before storage reconnects.

Reads and pins of one file share its cached ranges. A call that fails returns its typed error at once, and progress stays at what was committed. Only a pin's failure goes on the pending list, as the file's one record of its first unmet condition: a pin is a lasting wish for the bytes. A read's failure goes to its caller only, since nothing is left to finish once the caller has it; a player shows a song it couldn't fetch offline, and the pending list stays empty. Ben's laptop is pinning a song and has two checked ranges when Wi-Fi drops. The pending list reports the network failure, progress stays at those two ranges, and pinning again fetches the rest. Nothing retries a download on its own. A final content hash that doesn't match instead leaves a permanent refusal (Refused(ContentHash)) and publishes no completed file or pin.

Pending reason	When	Next
NoStorage	no storage connected
Storage { .. }	a network or provider failure	the next read or pin tries again
FileUnavailable(Lost)	storage confirms nothing at the path the record names	needs the app to act
FileUnavailable(Source(..))	the original of a file this device uploads moved or changed	the upload retries
Refused(Decryption), Refused(ContentHash)	a complete chunk or the whole file failed its check	not retried on the same bytes
Failed(Disk)	writing the local cache failed