Limits

Nothing on this page is a roadmap item. These are consequences of the design, hard-coded ceilings, and behaviours it is cheaper to read about than to discover.

Out of scope

Deliberately not done: transactions, JOIN, subqueries, schema migration, multi-writer replication, SQL.

Replication exists in one direction only: the server is authoritative and clients read it. Multi-writer merging, CRDTs and a long-lived offline write queue are out of scope — that is what fits the single-writer model, and the other would be a different database.

There are deliberate gaps in the type system too:

• No decimal. float is binary floating point and unfit for money. The answer is an int in cents.
• No UUID type. text @hash is functionally enough.
• No nested objects. A list is homogeneous and an object value is refused on purpose. A field you want to filter on should be a field.

Fixed limits

Ceilings baked into the code. Exceeding the first two makes the query error, because a silently cut result is a wrong answer believed right.

Memory and scale

The whole database is in memory, not just the vector arena: the segment bytes sit in RAM in their on-file form. That is the other side of saying there is no page cache, and it has three consequences.

• Open peaks at about 2× the file. The file is read in one go, then copied into segments. Measured: a 30.5 MB vector-free file reaches 65.5 MB peak RSS.
• checkpoint and compact peak at about 3×. The whole image is produced in memory and written to a side file. On the same file: 95.8 MB.
• Open cost is proportional to data size, not constant. 100 000 × 128 with the graph in the file takes about 110 ms. In a process that runs fewer than five vector queries, SQLite gives the faster total.

The vector arena is the largest item: one million 768-dimension vectors is about 3 GB, or 1.5 GB with f16. In the browser the WASM32 address space of 4 GB is the upper bound, and the 2–3× factors above come out of it too, so the real ceiling is lower.

Database::memory_bytes() sums the segment bytes, the offset indexes, the vector arenas and the graph links from counters, and --max-memory uses it. It is not RSS: allocator slack, HNSW build buffers, upper-level neighbour allocations (~3% of level 0) and the transient peaks all sit outside it.

So the ceiling is an early warning, not a guarantee. A third of the container memory is a sensible start: it covers both this 60–75% ratio and compact's 3× peak.

Index build, open and maintenance

• The graph is derived data and is written only during snapshot, compact and checkpoint — never on the write path. Without it in the file it is rebuilt on open: 0.90 s every open at 20 000 × 32, against 0.01 s with it.
• Who writes a checkpoint: the fenec shell when leaving an interactive or piped session, and fenec-pg on the shutdown signal — both only if a vector index exists. A one-shot fenec file.fenec -c '...' does not. Neither does kill -9 or a cgroup OOM; the graph is then rebuilt on the next open, so what is lost is derived data.
• compact is a full rebuild, not a garbage collection. Even with zero dead bytes every index is built from scratch and writes block throughout: 20 000 × 32 takes 0.94 s, against 0.05 s on a vector-free text collection of the same size.
• A delete is a tombstone and the graph node stays in memory until compact. Updating a document also marks the old node deleted and adds a new one, so in collections whose vectors change, the graph grows until compaction.
• Documents inside one batch cannot see each other as neighbours. The batch is 1/16 of the graph size (64–512), so there is no measured recall loss — but very small collections fall to the serial path below 1024 vectors.
• The build is parallel within a batch, and because writing links is serial the speed-up is not linear in core count: about 1.9× on eight cores.
• File writes are buffered. If the process crashes before sync or checkpoint, the last writes can be lost. In embedded use calling sync is the caller's job.

Query behaviour

• Only @hash equalities in an and chain reach an index. >=, ~, has, in and anything under an or is a full scan: every matching row is decoded and evaluated. There is no text index.
• The cost of a filtered near is dominated by extracting the filter set, not by the vector search. Of the ~6.5 ms measured on 100 000 documents, nearly all is the filter scan — an unfiltered ANN is 0.2 ms. With a selective filter the exact-scan path is taken, so recall is 10/10.
• There is no top-k for order. The key of every matching row is extracted and sorted, then limit applies. The cost depends on the number of matches, not on limit.
• near and order cannot be used together; near determines the ordering by similarity.
• A query vector can be given three ways: a vector, a list, or pgvector's text form (near embed '[1,0,0]'), which is the natural one when typing by hand in psql. None of the three fails silently — unparsable text and a non-numeric component both error.
• put does not check that a vector is finite. A NaN or infinite component enters the index and makes distances meaningless; the check exists only on the fenec import path. In a vector<N, f16> field an out-of-range value silently becomes inf or drops to zero below about 6×10^-8.
• now() does not work in the browser: wasm32-unknown-unknown has no clock and the call errors. Pass the time as a parameter with Date.now().

fenec-pg

• No TLS. The password is protected by SCRAM but the data flows in the clear. Non-loopback plus unauthenticated listening is refused without --insecure.
• No transactions. BEGIN and COMMIT are accepted and do nothing. The single-writer model gives per-statement atomicity.
• A cancellation is seen while waiting on the lock and between batched statements. A running statement, such as a long compact, is not cut in the middle.
• A ceiling, not a pool. Every connection is an OS thread. --max-connections rejects the excess with 53300 rather than queueing it; put pgbouncer in front if you want a queue. When a thread cannot be spawned only that connection drops, and an accept error is not fatal either — though the server stops after 64 in a row.
• An idle session lives forever by default. --idle-timeout <s> applies both while waiting for the next message and in the middle of a half-received one: both are signs of a dropped connection.
• Session threads get an 8 MiB stack. thread::spawn's 2 MiB default leaves 2.7× headroom for a 512-level expression in release but is not enough in a debug build — and a stack overflow is not a catchable panic, it is the process calling abort. It is virtual space: with 100 idle connections the measured RSS is 5.2 MB, about 36 KiB per connection.
• The row count of Execute is ignored. The portal always runs to the end and no PortalSuspended is sent. Clients using fetch_size or a named cursor get the whole result at once.

fenec-http

• Not a separate process. It opens as a second listener via fenec-pg --http; there is no standalone binary, because two processes opening the same file would corrupt it.
• No TLS. Without a token it will not bind a non-loopback address. The token is a single shared secret: no user separation, no scopes, no expiry.
• Transfer-Encoding: chunked is unsupported (411). The body is capped at 64 MiB and the header block at 64 KiB; above that, 413 or 431.
• Pagination is limit and offset — no cursor, no Range header. The response is built in one piece, so a very large select holds that much memory on the server.
• A field whose name matches a clause name (select, order, limit, offset, count, where) cannot be filtered from the query string. POST /query and fenec-pg are unaffected.
• No ETag, no Last-Modified, no conditional requests. Responses are not marked cacheable.
• A subscription is one-way. The server sends changes over SSE and the client writes with ordinary POST, PATCH and DELETE. There is no WebSocket; for the same job it would demand framing, masking and ping/pong.
• A subscription refuses order, limit, offset and count with a 400: a shape is a set, not a window, and an incremental diff cannot express a moving one.
• A write to any collection wakes every subscriber. That is a necessity rather than noise: the counter and the ring are shared across collections, so a subscriber to a quiet collection that did not wake would keep its cursor in place and be reseeded once other writes overflowed the ring. Waking moves the cursor forward, and no empty batch is sent for a change that does not match the shape. The cost is one read lock and an empty scan.
• POST /batch is not atomic. It stops at the first error and reports how many were applied. There is no rollback.

Import

• One table becomes one collection. JOINs and multi-table migration are out of scope — there is no JOIN in fenecdb anyway.
• fenec import is in the default build but is a separate feature. A binary built with --no-default-features has no such subcommand, and says so before exiting.
• --where sees only the built-in functions; plugin functions cannot be used. The filter is evaluated without borrowing the target database, so that writing can happen in the same loop.
• A SQLite file with an unprocessed WAL is not read. The main file shows stale data and stopping beats silently returning it. Run sqlite3 <db> "PRAGMA wal_checkpoint(TRUNCATE)" first. WITHOUT ROWID tables are not read either.
• No TLS, and no md5 authentication: scram-sha-256 or cleartext.
• PostgreSQL timestamp(tz) values are truncated to milliseconds.
• A non-finite component in a vector column stops the import and it says which row. A NaN makes a distance incomparable; even if the index accepted it, the result would be meaningless.
• An import that stops halfway cannot be rolled back — there are no transactions.