Upgrade from 0.5 to 0.6

Upgrading the library is a normal package upgrade:

pip install --upgrade engrava

0.6 is a schema-changing minor upgrade, so do not roll it across old and new workers sharing one database file.

Before upgrading

Back up the database, then quiesce writers.

sqlite3 my-data.db "PRAGMA wal_checkpoint(TRUNCATE);"
cp my-data.db my-data.db.bak

If the database is in WAL mode and you do not checkpoint first, copy the database together with its -wal and -shm files. Stop the 0.5 workers, let one 0.6 process run the migration to completion, then start the rest.

engrava --db my-data.db migrate

The migration also runs on the first ensure_schema() call — automatically, if your application already makes that call. Opening the store through SqliteEngravaCore.from_config() makes it for you. The direct constructor, SqliteEngravaCore(conn, ...), does not: it only wires the object up, so an application on that path runs 0.6 against the old schema unless it awaits store.ensure_schema() itself, or you run the engrava migrate command above before starting it.

The full OSS upgrade guide has the longer backup and recovery notes: Upgrade Guide.

What changes in 0.6

The core schema advances from user_version = 18 to 20 in two additive steps, applied on first open. Neither drops a table or rewrites your content.

Behaviour changes to review

A 0.5 application can notice these without changing a line of its own code. The first four are the ones most likely to change what your code sees; the rest of this section covers the remainder.

Archived thoughts leave default retrieval. search_hybrid, recall, search_fts, and search_similar now exclude rows whose lifecycle_status is ARCHIVED. If you archive thoughts, expect fewer results than on 0.5. Pass include_archived=True for an archive-search call, or restore_thought(...) to return a thought to ACTIVE. list_thoughts and count_thoughts are unaffected and stay lifecycle-neutral. See Hybrid Search.

A wrong-dimension query vector raises. search_similar and the vector arm now reject a vector whose length differs from the store’s embedding dimension with VectorDimensionMismatchError, an EngravaError subclass. Code that caught the previous incidental numpy ValueError, or relied on a wrong-length all-zero vector returning [], must catch the typed error instead. The dimension check runs first, so degeneracy is only forgiven at the declared length: an all-zero or non-finite vector of the right length still returns a graceful empty result and increments vector_arm_degradation_count, while an empty [] is a length-0 mismatch and raises like any other wrong length. That applies wherever the store declares a dimension — through a configured vector backend or an embedding provider.

Inverted validity intervals raise. A record whose valid_from is after its valid_until was previously accepted silently. 0.6 refuses it at write, and refuses it again on read — but only where a stored row is turned back into a record, and that boundary has two edges worth knowing:

• One inverted row breaks the whole listing. list_thoughts() builds every matching row into a record in a single pass, so one bad row anywhere in the page raises ValidationError and the call returns nothing at all — not merely minus that row. count_thoughts() and get_thought() on a healthy id keep working, which is what makes the failure easy to misread as sporadic.
• Ranked retrieval still returns it. recall() and search_hybrid() yield (thought_id, score) pairs and never build the record, so an inverted row goes on competing for result slots. The error surfaces one step later, when you hydrate that id.

Equal bounds (a zero-length, instantaneous fact) and open None bounds remain valid. Nothing reports the offending rows for you, so list them before you upgrade:

SELECT thought_id FROM thought WHERE valid_from > valid_until;

A NULL on either side drops out of that comparison on its own.

Treat that as a candidate list, not a complete detector. It compares the two bounds as stored text. engrava normalises an offset-bearing timestamp to UTC but leaves a naive one exactly as written, so a store holding both forms can have the text order disagree with the real order — in either direction. A row that looks inverted may not be, and a genuinely inverted row may not appear. Re-check any row whose two bounds are written in different formats by parsing both as datetimes.

A row that is genuinely inverted has to be repaired before the reads that touch it will work: swap the two bounds if they were transposed, clear one to NULL if the interval was meant to be open, or delete the row. There is no migration step that does this for you — 0.6 refuses such rows rather than rewriting them. See Bi-temporal Model.

engrava gc refuses rather than stranding vectors. If the upgraded install dropped the vector extra, a gc pass that is about to physically delete rows on a database carrying a sqlite-vec index stops before deleting anything and exits 1, because removing the rows without removing their vectors would strand those vectors in the index. Reinstall as pip install 'engrava[vec]' and retry. A --dry-run, and a run with nothing to delete, are never refused.

A stored decay_multiplier of 0.0 now reads back as 0.0. On 0.5.x the edge column held the 0.0 you wrote, but the decode tested it for truthiness and handed back the 1.0 default. Because update_edge rebuilds the whole record from that read and writes every column back, any later change to such an edge — a new weight, or an invalidate_edge closing its valid-time interval — persisted 1.0 over the stored 0.0, unless that same call set decay_multiplier itself. 0.6 decodes the column on presence, so 0.0 round-trips as written. Upgrading does not restore an already-overwritten value. Where a 0.5.x update replaced a stored 0.0, the database now holds 1.0, and 0.6 reads that back faithfully. If you set 0.0 on any edge deliberately, check those edges after upgrading and set the value again where it now reads 1.0.

Malformed full-text syntax gets one safe retry. A failed expert MATCH expression is now re-normalised through the bare sanitising path and retried once before the FTS arm gives up, so a query that previously returned no keyword hits may now return some. Every failed first attempt increments the read-only fts_match_failure_count counter.

Configuration validation is uniform. Invalid values in supported config sections are now rejected consistently whether the store is built from YAML or through the corresponding typed construction path — both raise ConfigError. Fix an invalid legacy value rather than relying on a path that previously skipped validation.

Enabled hygiene gains conservative wall-clock guards. Memory hygiene is still off by default, but for a store that already enabled it in 0.5 the omitted new fields resolve to a seven-day minimum inactivity age before archival and a 30-day wall-clock restore window before permanent GC, and archival additionally requires an active usage-history signal in the candidate pool. Existing hygiene archives with no archived_at fail closed and are not auto-GC’d while the wall-clock window is active. Review the defaults in Configuration → hygiene_policy before the first 0.6 hygiene run.

Custom embedding providers must expose a public dimension

If you pass your own embedding provider, check that it exposes dimension as a public attribute or property. The protocol always required it, but 0.5 read it at exactly one site off the query path — so a provider that kept the value privately (self._dimension, with no public property) worked for as long as nothing called verify_embedding_model().

0.6 reads it before every vector search that has to ask the provider, and such a provider now raises EmbeddingProviderContractError naming the provider class and the missing member.

A configured sqlite-vec backend declares the dimension on its own vec0 table and is consulted first, so that configuration never reaches the provider read.

The fix is to expose the value:

@property
def dimension(self) -> int:
    return self._dimension

The check is lazy: constructing a store with such a provider still succeeds, and a store that never searches by vector is unaffected. Call verify_embedding_model() after construction if you would rather fail at startup — it raises the same typed error.

What is additive

The new edge-metadata, derived-record, cycle-provider, and transaction-time recency surfaces are additive and opt-in. Migrating a store does not enable automatic derivation or a cycle provider.

Cutover checklist

1. Back up the database (checkpoint the WAL first).
2. Stop the 0.5 workers.
3. Upgrade application dependencies to engrava>=0.6.
4. If you run the standalone MCP server, upgrade engrava-mcp to 0.6 in the same step — 0.5.x pins engrava<0.6 and will otherwise conflict. See MCP Server.
5. Run one 0.6 process, or engrava migrate, and let the migration complete.
6. Start the remaining 0.6 workers.
7. Re-check any code that catches ValueError around vector search, and any custom embedding provider’s dimension.
8. If you archive thoughts, decide where you now want include_archived=True.
9. If you ever set an edge’s decay_multiplier to 0.0, re-check those edges — a 0.5.x update may already have replaced the value with 1.0.

Coming from 0.4

0.4 → 0.5 is a separate hop — the MCP server moved out of engrava in that release — and it is still documented in Upgrade to 0.5. Run that upgrade first, then work through the checklist above.