Introducing Sift

Persistent memory for Claude Code

Every time you start a new Claude Code session, Claude forgets everything. Your preferences, the patterns you’ve established, the decisions you’ve made together, the gotchas you’ve discovered—all gone. You end up repeating yourself, re-explaining your codebase, watching Claude make the same mistakes you corrected yesterday. I built Sift to solve this problem.

Sift started as a command-line tool that let you query text files with SQL. I compiled SQLite and PCRE2 into a single binary, exposing your text as database tables you could search and transform. That was useful, but the real breakthrough came when I realized Claude Code needed more than clever search—it needed memory that persists across sessions, grounding that prevents hallucination, and infrastructure that scales with the complexity of real work.

What began as “grep with SQL” evolved into a complete AI agent infrastructure: 78 MCP tools across eight subsystems, handling everything from file editing to conversation tracking to hardware-aware resource management.

The Memory Problem

Large language models hallucinate. Claude might confidently report that a function is on line 50 when it’s actually on line 200. It might “remember” that you prefer tabs when you actually said spaces. It might fabricate an API endpoint that doesn’t exist in your codebase.

Sift addresses this directly. When Claude searches your codebase with sift_search, it gets back real results from an indexed source of truth—exact file paths, exact line numbers, exact content. There’s no room to confabulate. When Claude reads a file with sift_read, it sees actual content with line numbers. When it edits with sift_edit, the tool returns an error if the target text doesn’t exist or isn’t unique. These aren’t polite suggestions—they’re hard failures that force confrontation with reality.

The memory system works the same way. When you tell Claude you prefer early returns, that preference is stored as a structured record that can be queried later—not as a fuzzy impression that might drift over time. When Claude is unsure what you said about your authentication system, it can run sift_memory_search and get back exactly what was recorded.

Eight Subsystems

Sift provides 78 MCP tools organized into eight subsystems:

Search uses FTS5 full-text indexing to search your codebase 30-195x faster than grep. Boolean queries with AND, OR, NOT, and NEAR operators work out of the box. The workspace auto-indexes on first use and refreshes incrementally.

File tools handle reading, writing, and editing with features designed for AI agents. sift_read returns line numbers so Claude can make precise edits. sift_edit supports find/replace, line insertion, deletion, SQL-powered transforms, and unified diff patches. sift_batch executes multiple operations atomically—all succeed or all fail.

Memory is the largest subsystem with 38 tools. It stores plans, tasks, patterns, preferences, gotchas, and notes. Claude can record decisions with rationale, log reflections on its reasoning, track dependencies between tasks, and traverse the graph of connected memories. The sift_memory_challenge tool generates adversarial queries to surface counterevidence—a built-in devil’s advocate that prevents confirmation bias.

Context preserves conversation history across sessions. Every message and tool call can be saved, searched, and linked to memories. When Claude needs to remember why a decision was made, it can search 25,000 messages and find the exact conversation.

Web tools crawl documentation sites and cache them locally for instant FTS5 search. A documentation site that would require hundreds of WebFetch calls becomes a single crawl operation followed by sub-millisecond queries.

Repository tools clone git repositories and index their source code into searchable SQLite databases. Point Sift at any GitHub repo and query it with full-text search or raw SQL.

SQL tools bring database power to text transformation. Parse CSV fields, apply regex substitutions, aggregate data—all in familiar SQL syntax.

Hardware awareness monitors system resources and adapts behavior under pressure. When memory is constrained, Sift automatically reduces result limits, enables streaming, and suggests batching operations. Claude can request resource budgets before expensive operations and receive guidance on what’s actually available.

Fingerprints and Continuity

One of Sift’s more unusual features is fingerprinting. A fingerprint captures how Claude engages with a project—not just what happened, but the patterns of interaction. It tracks engagement rhythm, learning signature, reasoning style, and tool fluency across six dimensions.

At session start, Claude loads its fingerprint before anything else. This shapes how it interprets the rest of the context. A high-confidence fingerprint means well-calibrated patterns to trust. A developing fingerprint means the collaboration is still calibrating.

The fingerprint system can also detect drift—when current session behavior deviates significantly from the baseline. This provides a check against both Claude going off-track and against external changes that break established patterns.

The CLI Remains

Sift still works as a standalone command-line tool. Pipe text through sift --for "SELECT..." to query it with SQL. Use --dig to search across multiple files with FTS5. Edit files surgically with --pick and --refine. The --shake flag previews changes without writing them, and --diff shows what would change.

But the real power comes from the MCP integration. Running sift --mcp starts a server that exposes all 78 tools as structured function calls. Register it with Claude Code using claude mcp add sift -- sift --mcp, and suddenly Claude has persistent memory, grounded search, and intelligent editing.

Getting Started

The fastest way to install Sift is with the interactive installer:

curl -fsSL https://github.com/edwardedmonds/sift-releases/releases/latest/download/sift-setup.py | python3

The installer prompts before each step: installing the binary to ~/.local/bin, documentation templates to ~/.claude/, and registering the MCP server with Claude Code.

For manual installation, download the binary for your platform:

Then install and register:

chmod +x sift-*
        mkdir -p ~/.local/bin
        mv sift-* ~/.local/bin/sift
        claude mcp add --scope user sift -- sift --mcp

Add ~/.local/bin to your PATH if not already present. Sift creates a .sift/ database directory in each project, so memories and indexes are project-specific.

I’ve found Sift most valuable in long-running projects where context accumulates. The first session is like any other Claude Code session. But by the tenth session, Claude knows your preferences, remembers the gotchas, understands the architecture decisions, and can search its own history to find exactly when you discussed that tricky authentication bug. The collaboration compounds.

The source is on GitHub, currently proprietary while features mature, with plans to open source once the API stabilizes.

Home Home Reference Comparison GitHub