niso-storage

From the repository

This page is crates/niso-storage/CONTRACT.md, included when the site is built. Edit it there.

Purpose

Ordered, transactional storage at the level IndexedDB needs: per origin and storage endpoint, named databases with a version, object stores and indexes, records ordered by encoded key, and transactions that commit or abort. It never sees JS values: keys arrive encoded so that memcmp orders them as IndexedDB orders keys (key), values as structured-serialization bytes. It ships three drivers, SQLite (the default, one file per database), memory (sessionStorage and tests) and the adapter for a host’s C driver, and the configuration that picks one (directory, driver, app origin).

niso’s document uses it through niso_core::storage, a re-export: the core’s IndexedDB and Web Storage, and the Dom setters behind niso_dom_set_storage_*, are its users until steps 9 and 10.

Public surface

• Rust:
  • Driver and its types: Endpoint, Mode, Durability, DriverError, DriverResult, DbId, TxnId, StoreId, IndexId, StoreInfo, IndexInfo, DatabaseInfo, Source, End, Range, Entry.
  • The drivers: sqlite::SqliteDriver, memory::MemoryDriver (with a quota), ffi::CDriver over ffi::NisoStorageDriver.
  • key::Key (encode, decode, from and to structured-clone values).
  • Configuration: Config (Directory, Driver), Storage (configure and set_app_origin, both InvalidState once in use; in_use, app_origin, driver, driver_for, session), default_app_origin, default_directory.
  • From<DriverError> for niso_base::Error.
  • Behind the testing feature, testing (below); behind contract-tests (which turns on testing), contract: suite and its checks (exercise, edges, names, beside, aborted_upgrade, deletes, all_or_nothing), snapshots, configuration, k.
• C: include/storage-driver.h and include/storage-config.h, its sections of the composed niso_runtime.h (niso_storage_driver and its structs, niso_dom_set_storage_driver, niso_dom_set_storage_directory, niso_dom_set_app_origin); ffi’s #[repr(C)] types are their Rust side. The setters stay in niso-capi, unchanged.

Host requirements

None: it needs no host. A user keeps one Storage (or one driver), on one thread, and makes the calls Driver documents in the order it documents: schema changes only in a “versionchange” transaction, reads and writes only in a transaction’s scope, at most one “readwrite” or “versionchange” transaction per database at a time, and read-only ones beside it only on stores it does not use.

May assume from dependencies

niso-base: Error, ErrorCode, Result, err. niso-structured-clone: Value, Record, Serialized, JsString, js_string, for key conversion.

Guarantees

What the contract suite checks, on the memory driver, the SQLite driver and both behind the C vtable (tests/contract.rs). suite runs every check that needs one driver; snapshots needs two on shared storage, and runs on SQLite and C over SQLite; configuration is of Storage and runs once.

1. A database opens at version 0 with no stores; a committed transaction’s schema, version, records, index entries and key generator are what the next open and transaction see; an aborted “readwrite” transaction’s records and generator are undone. exercise.
2. An aborted “versionchange” transaction leaves the version and the schema as they were (stores made, deleted, renamed; indexes deleted), records and index entries included. aborted_upgrade.
3. A transaction the caller aborts after one of its calls failed (a put, or the commit itself) leaves nothing behind. all_or_nothing, with FailingDriver making the failures. A driver’s own failure in the middle of its commit is not something the suite can cause; SQLite’s commit is one SQLite transaction.
4. Scans visit a store’s records, or an index’s (key, primary) entries, in memcmp order of the encoded keys, forwards or in reverse, within a range whose ends may be open and may name a (key, primary) pair; count agrees with the scan. exercise, edges.
5. Replacing or deleting records drops their index entries; clear empties a store and its indexes and no other store; delete_store takes the store and its indexes, and a store made again under the same name and id starts empty, with none of the deleted one’s indexes. exercise, edges, deletes.
6. A read-only transaction on one store runs beside a “readwrite” one on another, and the writer’s commit lands. beside.
7. Two drivers on shared storage: a read-only transaction begun before the other’s commit does not see it, and one begun after does. snapshots. One driver alone never shows this (the core never runs a reader beside a writer on the same store), and memory drivers share nothing, so it runs on SQLite only, directly and through the vtable.
8. databases lists committed versions per origin and endpoint; deleting one database keeps the others, names of any length work, and a new database takes a free place. names.
9. The driver and the app origin are settable until the first driver call, then InvalidState. configuration.
10. A write past the space a driver has fails with DriverError::quota, which the C adapter carries as NISO_QUOTA_EXCEEDED both ways. (tests/contract.rs, and the memory driver’s quota test.)
11. The C adapter (CDriver) lets a panic in scan’s visitor stop the scan and go on once the host’s driver has returned, never unwinding through the host’s C frames. (tests/contract.rs.)

SQLite alone also detects another document deleting or replacing a database’s file under an open one (its readers and writers then fail rather than read or lose data): unit tests in lib.rs, #[cfg(unix)], as they rely on unix file semantics; they are SQLite’s, not the contract’s.

Differences from architecture.md

• Configuration is a plain Storage value the document holds in a field, not an extension slot: niso-storage has no host (2.2), and the slot comes with the crates that install it (niso-indexeddb, niso-webstorage, steps 9 and 10; architecture.md 11).
• The Web Storage areas loaded so far stay in niso-core, beside Storage, until step 10.

Threads

Not Send: a driver lives on the document’s thread. SQLite drivers on different threads (several documents, workers) may share files; SQLite’s locking serializes them, and a read-only transaction reads one snapshot.

Performance and memory

The move changes no code path: the document calls the same drivers through one more field. The memory driver keeps everything in maps, up to its quota (256 MiB by default).

Fakes

memory::MemoryDriver is the fake driver, and exact. Behind the testing feature, so the shipped build leaves them out: testing::FailingDriver wraps a driver and fails its n-th fallible call (or every one from it), as a quota error or not, for dependents’ error paths; testing::vtable puts any driver behind the C vtable, as a host’s C driver is, its functions catching panics (the Internal status) and refusing bad mode, durability or endpoint bytes (the Type status).

Miri runs the contract suite’s memory and C-vtable tests (tests/check.sh): MIRIFLAGS=-Zmiri-ignore-leaks cargo +nightly miri test -p niso-storage --test contract. The SQLite tests are cfg_attr(miri, ignore), as SQLite is C.

Standalone use

MemoryDriver::default() or SqliteDriver::new(dir), then open, begin, create_store, put, scan and commit with keys from Key::encode.