Sync

A copy on the client, changes streaming from the server, optimistic writes. In fenecdb that is not a library — it is an endpoint and a transport, because the same engine already runs on both sides.

import { sync } from './fenec.js';

const db = await sync({
  url: 'http://127.0.0.1:8080',
  shapes: [{ collection: 'tasks', where: { status: 'open' }, key: 'key' }],
});
await db.ready();

// read: from the local copy, no network
const rows = await db.from('tasks').where('priority', '>=', 3).rows();

// live query: re-runs on every change
const stop = db.live(db.from('tasks').order('priority', 'desc'), draw);

// write: to the local copy first, instantly, then to the server
await db.from('tasks').insert({ title: 'new', status: 'open' });

The query code does not change. db.from(...) is the same builder producing the same FenecQL; the only difference is where it is bound.

The stream carries state, not transactions

The classic route is a transaction log: every write is an event and the subscriber replays them. fenecdb sends the current state of what changed instead — the stream is a clock, not a record. That is right for a replica and wrong for anyone who wanted an audit log.

A shape is a set, not a window

The whole database is in memory and the WASM32 address space is 4 GB, so a client cannot pull an entire collection. A subscription is a subset and the filter is applied on the server.

The shape condition uses the ?field=op.value form, so there is no or group and no function call. The reason is escaping: a free where= would turn into string concatenation, exactly what the builder avoids. Escaping goes to the transport via URLSearchParams and types go to the server, leaving no injection surface.

One collection per shape. The seed has to mean "this is the whole of this collection", or the cleanup step becomes ambiguous.

Optimistic writes

A write is three steps and the order matters: first gather what undoes the local change, then apply it locally, then send it to the server. If the server refuses, the local side rolls back fully — no network needed, the undo is already in hand.

Everything up to the first await is synchronous. An async function body runs synchronously until its first await, so the optimistic row is local before the caller awaits the promise. A single microtask in between would have broken the "visible instantly" promise.

const p = db.from('tasks').insert({ title: 'new', status: 'open' });
await db.from('tasks').where('title', 'new').first();  // already here
await p;                                               // server ack

Reconciliation by key

The id space belongs to the server and the client cannot know it in advance, so an optimistic row gets a temporary id starting at 2⁵² — far from the server's range and below Number.MAX_SAFE_INTEGER. What matches the two is a business key: when a row with the same key arrives from the subscription, the temporary one is dropped.

Update and delete need no key: they work by id, and reading the previous state is enough.

Live queries

There is no incremental maintenance and none is needed. The local side is an indexed database, so re-running the query from scratch is already under a millisecond — the differential dataflow that a Map-backed collection requires buys nothing here.

It would not even be correct for near. A single insert can change the entire top-k ordering, and there is no incremental HNSW maintenance that is both cheap and correct — the machinery would have to be disabled for fenecdb's headline feature.

Invalidation is at collection granularity: the change stream says which collections changed, the live queries bound to them re-run, and all of it coalesces into one frame. Anything finer, like intersecting id sets, would cost more than the local query itself.

Batches

await db.batch(async (t) => {
  await t.from('tasks').insert({ title: 'a', status: 'open' });
  await t.from('tasks').where('key', 'x').update({ status: 'closed' });
});

There are no transactions in fenecdb and this batch does not fake one. The statements run in order under a single write lock and stop at the first error. The gain is one round trip instead of N, and no other writer slipping in between.

The local side rolls back fully; the server side cannot. A batch that stops halfway reports how many statements were applied, because a silent error would permanently separate the client's optimistic state from the server's.

Multiple tabs share one stream

Every tab has its own wasm instance. If each opened its own subscription there would be N copies and N connections, so a single leader is elected with navigator.locks; the others receive the same batches over BroadcastChannel and apply them to their own local copies. The apply path is identical — only the transport differs.

Writes do not go through the leader. Every tab sends its own write straight to the server, so leadership concerns only the read stream: a leader dying does not stop writes, the lock passes to the next tab and it continues from its own cursor.

Turn it off with leader: false. Without navigator.locks, as in Node, it is off already.

Dropped connections and the horizon

A client returns with ?since=<cursor> and asks for no seed. If the cursor has fallen behind the ring, the server reseeds by itself rather than silently sending incomplete events.

--changes <n>    entries in the ring (default 4096, 24 bytes per entry)

That number is directly how far behind a subscriber may fall. At 100 writes per second, 4096 entries is roughly a 40 second window. With persist, the image and the cursor are written to IndexedDB, so a reopened tab is not reseeded from scratch — unless its cursor has fallen past the horizon.

When the server restarts, the horizon is set to the counter: a database loaded from a file has no history, only its present state. A cursor standing exactly there, after a quiet restart, gets an empty response; anything behind it is reseeded.

Cost

At the default --changes 4096 the ring is about 96 KB and costs one push per entry on the write path. Subscriptions do not poll — see the HTTP endpoint for why — and are counted separately from ordinary requests under --http-max-streams.

Limits

• A shape is not a security boundary. If a changed id does not match the shape at all, the subscriber sees it as a deletion. Deleting a row that is not there locally is harmless, but which ids changed leaks. Hiding rows needs a separate endpoint, or --http-read-only plus a token.
• bytes fields cannot be synced. In JSON they become number arrays with no way back. That is a limit of the HTTP surface, not something sync introduces.
• In a projected shape the other fields read as null. With select, the local row carries only those fields — which is all the client knows anyway.
• batch() does not nest and the calls inside it return no result: writes pile up and go in a single request.
• Rolling back needs no network, but the server has the last word. A rolled-back local state is overwritten by the server's in the next batch.

Persisting the replica

const db = await sync({
  url: 'https://api.example.com',
  shapes: [{ collection: 'tasks', where: { status: 'open' }, key: 'key' }],
  persist: 'tasks',           // the image and the cursor are both stored
});
await db.ready();

db.live(db.from('tasks').order('priority', 'desc'), draw);
await db.from('tasks').insert({ title: 'new', status: 'open' });