Architecture

engrava is the memory database for AI agents — a Python library for storing, linking, searching, and evolving ideas. It is SQLite-first and designed to be embedded by larger cognitive systems.

Layer Model

Imports flow downward only:

+--------------------------------------------------+
|  CLI / MCP server / Consumer apps, scripts, ...  |
+--------------------------------------------------+
|  Extensions / Embeddings / MindQL                |
|  (dreaming, hooks, providers, query language)    |
+--------------------------------------------------+
|  Infrastructure                                  |
|  SqliteEngravaCore, schema, migrations           |
+--------------------------------------------------+
|  Domain                                          |
|  models, enums, protocols, exceptions            |
+--------------------------------------------------+

• Domain (src/engrava/domain/) — stdlib + Pydantic only. Frozen models, @runtime_checkable Protocols, zero infra imports.
• Infrastructure (src/engrava/infrastructure/) — SQLite implementation of domain protocols.
• Extensions (src/engrava/extensions/) — optional capabilities (dreaming, hooks) that depend on domain + infrastructure.
• MindQL (src/engrava/mindql/) — read-only query language.
• Embeddings (src/engrava/embeddings/) — pluggable embedding providers.
• CLI (src/engrava/cli/) — Click-based command-line interface.
• MCP server — a Model Context Protocol server shipped as the separate engrava-mcp package (not part of engrava core). Like the CLI it is a top-layer API consumer: it wraps engrava’s public async API over stdio so MCP clients (Claude Desktop, Cursor, …) can use a store. Install and run it independently with uvx engrava-mcp. See MCP server.

Core Components

SqliteEngravaCore

The primary store implementation. Provides:

• Thought CRUD (create, read, update, list, search)
• Edge CRUD (create, read, update, delete, traverse)
• Embedding storage and vector similarity search
• Full-text search (FTS5)
• Hybrid search (5-signal fusion — see below)
• Bi-temporal valid time — optional valid_from / valid_until bounds on thoughts and edges (a second time axis: when a fact is true, distinct from when it was recorded), queried via the four valid-time MindQL predicates, with invalidate_thought / invalidate_edge to close an interval without deleting. See Bi-temporal Model.
• Scoped retrieval — metadata filters (an AND of typed field predicates over a thought’s metadata) and a VisibilityQueryFilter for the “public-or-mine” pattern, both accepted by recall() and search_hybrid(). A visibility filter is a query refinement, not access control.
• Result collapse — collapse_key / collapse_max_per_unit keep one best-ranked row per caller-defined unit (a conversation, a document) and backfill the freed slots with deeper distinct units. A de-duplication convenience, not a filter.
• Action records — ActionRecord storage for what an agent did, alongside what it knows.
• Derived records — a producer hook can persist children from a stored thought; core writes them without re-entering on_store.
• Memory Hygiene — an off-by-default pass that archives inactive thoughts (ACTIVE → ARCHIVED, reversible via restore_thought) behind wall-clock guards, with permanent collection as a separate step.
• Hash-chain audit journal — off by default; when enabled, every mutation on thoughts and edges is recorded as a hash-linked entry. See Observability.
• Schema management and migrations

Hybrid Search (5-Signal Fusion)

search_hybrid() fuses five ranking signals:

final_score = w1*FTS + w2*Vector + w3*Recency + w4*Priority + w5*Graph

Disabled signals have their weight redistributed proportionally across the active ones. With neither FTS nor vector usable there is no arm to fuse, and search_hybrid() returns a query-less window instead, pre-ordered so that truncation keeps the freshest rows. See Hybrid Search for details.

Dreaming Extension

Periodic memory consolidation that:

1. Scores active thoughts via configurable signals.
2. Promotes qualifying thoughts to P1 priority.
3. Creates edges (ASSOCIATED, source=DREAMING) between promoted thoughts and their nearest neighbours.
4. Clusters + reflects — groups semantically related thoughts and creates ThoughtType.REFLECTION meta-thoughts with centroid embeddings and CONSOLIDATED_FROM edges.
5. Retires orphaned reflections — a REFLECTION whose source thoughts have all left ACTIVE summarises nothing live, so it is archived (ACTIVE → ARCHIVED, reversible). This sweep runs on every consolidation pass; it is gated neither by enable_reflections nor by Memory Hygiene.

Dreaming is a graph mutator and abstraction builder — each consolidation run can grow the thought graph with dream-discovered connections and create higher-order REFLECTION thoughts that aggregate clusters. Both feed into hybrid search, closing the dream → structure → retrieval loop.

See Dreaming for details.

Extension System

New behaviors plug in via EngravaHooksProtocol — on_store and on_retrieve transform thoughts through the lifecycle, and decay_function supplies the decay multiplier an enabled Memory Hygiene pass folds into its eviction score. Custom query commands are MindQLExtension entries carried on ExtensionManifest.mindql_extensions, or passed straight to MindQLExecutor; the hooks’ mindql_extension_registry() method is reserved and core never calls it. Application-level logic (planners, reasoners) belongs in consumers, not in engrava core. See Extensions.

Data Flow

   Ingest                    Dreaming                   Search
   ------                    --------                   ------
   create_thought() --+     run_consolidation()        search_hybrid()
   store_embedding()  |      |                          |
                      v      v                          v
              +-------------------+            +-----------------+
              |    SQLite DB      |            |  5-signal fusion |
              |  thoughts table   |<-----------|  FTS + Vec +    |
              |  edge table       |            |  Rec + Pri +    |
              |  embedding table  |            |  Graph          |
              +-------------------+            +-----------------+
                      ^
                      |
              Dream: ASSOCIATED edges on promotion
              Dream: REFLECTION thoughts from clusters

Upgrade Paths

Version-to-version upgrade notes live on their own pages rather than here: Upgrade to 0.5 and Upgrade to 0.6, which covers the user_version 18 to 20 core-schema migration. Earlier releases are recorded in the library’s own Upgrade Guide.