Context8

• [*]LanguagesPython
• [*]FrameworksMCP Python SDK · Click · sentence-transformers
• [*]StoresSQLite · sqlite-vec (vec0) · SQLite FTS5 · Actian VectorAI DB (optional)
• [*]AsyncClaude Code afterToolUse hook (auto-capture) · batch embedding + upsert in ingest pipeline
• [*]InfraPyPI · Docker / Podman (Actian backend only) · GitHub Actions
• [*]Third partiesGitHub REST API (issue importer) · Hugging Face model downloads (MiniLM, CodeBERT)

How it holds up

Auth

None: it is a local, single-user MCP server over stdio, and the SQLite file lives in ~/.context8. The only credential is an optional GitHub token for the issue importer.

Data

On SQLite: a records table with 6 secondary indexes, plus schema_version and meta tables, three vec0 virtual tables (vec_problem, vec_solution, vec_code_context, cosine distance) and one FTS5 table (unicode61 tokenizer), all joined on record_id. apply_migrations is idempotent and records the vec0 dimension in meta, so switching embedding models on an existing DB raises DimMismatchError instead of corrupting it.

Consistency

Every write (record row, three vec0 upserts done as delete + insert, FTS row) runs in one explicit BEGIN/COMMIT with ROLLBACK on error. WAL mode and a 5-second busy_timeout let the MCP search loop read while the CLI writes. Seeded, GitHub-imported and session-mined records get uuid5 IDs, so re-running an import upserts instead of duplicating. Feedback is a read-modify-write of counters with no lock, so two concurrent ratings can lose one.

Failure

Dense or FTS queries that raise sqlite3.OperationalError log a warning and return an empty list, so the other retrieval paths still fuse. The auto-capture hook exits silently when the DB or SDK is missing, so it never blocks the agent. On Actian, the collection falls back to dense-only when sparse vectors are unsupported.

Testing

12 pytest files, 142 test functions, 1,886 lines. The default run uses SQLite with no infrastructure. The Actian e2e suite is gated on CONTEXT8_BACKEND=actian. CI runs ruff lint, a format check, an import check that the CLI loads without the Actian SDK, and a pytest matrix.

Techniques

1. Weighted Reciprocal Rank Fusion across three retrieval pathsProblem-vector KNN, code-vector KNN and BM25 each fetch up to 10x the limit (capped at 50). RRF fuses them on rank alone, weight/(60+rank), so cosine distances and BM25 scores never need normalising, and the same fusion works on both backends.src/context8/search/fusion.py:25 src/context8/search/engine.py:85 src/context8/search/engine.py:86
2. Query-shape-adaptive fusion weightsA query analyser looks for error markers (Traceback, ERR_, TS2, E0...) and code markers (def, =>, ::...) and reweights dense, code and sparse: error-only queries lean on BM25 for exact codes, code-heavy queries lean on the code vector.src/context8/search/analyzer.py:36
3. Pluggable storage backend with a zero-infrastructure SQLite defaultEverything above the storage layer talks to a runtime-checkable Protocol. The SQLite implementation does KNN through sqlite-vec vec0 with filters on the joined records table, BM25 through FTS5, and tag filters through JSON1 json_each. Each record write is one transaction covering four tables.src/context8/storage/backend.py:64 src/context8/storage/sqlite_backend.py:155 src/context8/storage/sqlite_backend.py:325 src/context8/storage/sqlite_backend.py:361
4. Immutable-dimension guard on schema migrationvec0 dimensions are fixed when the table is created. The schema helper stores the code-vector dimension in a meta table and raises DimMismatchError if CONTEXT8_USE_CODE_MODEL was flipped later, pointing the user to init --force instead of failing on mismatched blobs.src/context8/storage/sqlite_schema.py:105 src/context8/storage/sqlite_schema.py:111 src/context8/storage/sqlite_schema.py:119
5. Duplicate vs variant detection on writeBefore context8_log stores a record, it compares problem similarity (variant at 0.85 or above, exact at 0.95 or above) and then solution cosine. If both match, the record is skipped as a duplicate. If the problem matches but the solution differs, it is stored as a variant tagged variant-of:<id>.src/context8/search/engine.py:211 src/context8/mcp/tools.py:332
6. Feedback-weighted re-ranking without re-embeddingcontext8_rate updates only the counters and tags, so no vectors are rewritten. At query time the ranker multiplies the fused score by confidence, an exponential recency half-life and the worked ratio once there are enough samples. The ranker was written by Milan; the payload-only update path is the owner's.src/context8/feedback.py:57 src/context8/storage/sqlite_backend.py:170 src/context8/search/ranking.py:32 src/context8/search/ranking.py:41

Counted

• 52 (owner 45 incl. noreply, Milan 7)Commits
  git rev-list --count HEAD; git shortlog -sne HEAD
• owner 5,727 / Milan 3,035Lines attributed by git blame (src+tests)
  git blame --line-porcelain -w -M -C over git ls-files 'src/*.py' 'tests/*.py'
• 6,876Python LOC (src)
  git ls-files 'src/*.py' | xargs wc -l | tail -1
• 1,886Python LOC (tests)
  git ls-files 'tests/*.py' | xargs wc -l | tail -1
• 12 / 142Test files / test functions
  ls tests/test_*.py; grep -c 'def test_' tests/*.py
• 7MCP tools
  grep 'name="context8_' src/context8/mcp/tools*.py
• 16CLI commands
  grep -c '@click.command' src/context8/cli/commands/*.py
• 3 regular + 3 vec0 + 1 FTS5SQLite tables
  src/context8/storage/sqlite_schema.py
• 5 (0.1.0 to 1.0.0)PyPI releases
  curl https://pypi.org/pypi/context8/json