txNick Babaei
Back

Live · v5.3.0Personal project · Mar 2026 – ongoing · solo

One memory for every agent I use.

Sparrow is a client-agnostic MCP server holding durable memories, skills and a RAG library — with Google Workspace, Microsoft, Slack and Linear as tools. It began as a self-directed deep dive into AI and LLM engineering: about six iterations of agent harness, chat UI and tools came and went, and the knowledge plane carried through every rewrite.

MCP tools across 11 domains
48
Search MRR, up from 0.774 (193 labeled searches)
0.802
Downtime moving to a VPS — no client re-registration
13 s
Idle server memory, down from ~450–500 MB
~120 MB

§01Architecture

Sparrow knows; clients do.

Clients connect over MCP to the Sparrow server on a VPS, which holds knowledge, search, connectors and a console, backed by SQLite, an embedding child process and a library indexer on the Mac.

clients

  • Claude Code
  • Codex
  • Gemini
  • ChatGPT
  • JAI

Streamable HTTP · OAuth 2.1 · Cloudflare Tunnel

on the Mac

Library indexer

Folder + Gmail sync, PDF/Word/PowerPoint/Excel extraction and Apple Vision OCR for scans — indexed on the Mac, uploaded to the server.

sparrow server — 1 vCPU / 2 GB VPS · systemd

Knowledge

Memories (facts), skills (procedures, in Twig notation) and a RAG library, with wiki-links kept acyclic.

Hybrid search

SQLite FTS5 + sqlite-vec merged by reciprocal-rank fusion; fails open to keyword-only.

Connectors

Google Workspace, Microsoft/Outlook, Slack and Linear as MCP tools.

Console

React 19 web console: connectors, OAuth clients, activity log, knowledge graph, library browser.

node:sqlite — memories · skills · library pieces · vectors · OAuth clients · retrieval log

Embedding process

EmbeddingGemma-300m, 4-bit ONNX via transformers.js. Forked on demand, exits after 15 idle minutes; a crash fails one text, not the server.

§02Iterations

About six iterations. One knowledge plane.

  1. Mar–Jun 2026

    Web app + VPS relay

    Replaced a Python FastAPI “Brain” server with TypeScript (−3,150 lines); Claude Agent SDK chat, tRPC, a Windows-11-style shell, connectors for Google Workspace, GitHub, Linear, Figma and WhatsApp. ~1,050 commits.

  2. May 2026

    30–60 s off every chat turn

    Replaced an Ollama history-compression step with the Agent SDK's session resume — net −412 lines.

  3. Jul 6–26

    Rewritten as an Electron agent app

    Tool loop, permission gate with Haiku JSON verdicts, terminal, ripgrep find, web search, Monaco explorer and git panel, browser tool, DMG packaging.

  4. Jul 28–29

    The split: a client-agnostic knowledge plane

    Sparrow became an MCP server (OAuth 2.1, SQLite) with a hygiene layer — enforced frontmatter, wiki-links, an acyclic link graph, knowledge_lint. The desktop was hard-cut onto it.

  5. Aug 2026

    Hybrid search and the Library

    FTS5 + sqlite-vec with reciprocal-rank fusion; a RAG corpus synced from folders and Gmail with document extraction and OCR.

  6. Sep 22

    Twig

    A controlled notation for skill bodies — 10 keywords, prose as guidance — with a linter enforced inside skill_save.

  7. Sep 25

    Onto a 1 vCPU / 2 GB VPS

    Same hostname, OAuth issuer, database and Cloudflare tunnel; systemd as an unprivileged user; 13 s of downtime.

  8. Sep 26–27

    Cut to server + console

    Removed the Electron app and six shared packages (~19.4k lines). CI deploys every merge to main through a least-privilege deploy user.

  9. Sep 27–28

    Chunked embeddings, model in its own process

    Content-defined pieces keyed by hash, so an edit re-embeds ~1 piece; the model moved to a child process that exits when idle.

§03Hard problems

What it taught me.

Context management

Keeping long agent threads inside a token budget without losing what matters led to sliding-window and priority-window context managers, per-tool staleness, compaction and residency — compared on token spend by an offline benchmark.

Knowing when to revert

An isolated stage-worker delegation runtime took 7 slices over two days, then was reverted: it was hard to trace, debug or backtrack, and it burned too many tokens.

A 300M-parameter model on 2 GB

The 8-bit build was OOM-killed and ran ~1 s per query; the 4-bit build runs at ~15 ms. Idle unloading freed almost nothing — ONNX Runtime only returns memory when its process exits — so the model moved into a child process.

Search past page one

One vector per document covered only its first 512 tokens (46% of documents run past it). Splitting into 800–1,200-character pieces took vector-only search for later text from 19 to 46 of 60 in the top 5 — MRR 0.203 → 0.532 on 417 documents.

§04Stack

TypeScript · Node 22 · Express · MCP SDK (streamable HTTP, OAuth 2.1) · node:sqlite · FTS5 · sqlite-vec · transformers.js · ONNX · React 19 · Vite · Tailwind 4 · d3-force · Swift (Apple Vision OCR) · systemd · Cloudflare Tunnel · GitHub Actions

Next: JAI, the agent that runs on top of it