BM25 Keyword Search Skill

Fast full-text keyword search using BM25 scoring in the agent-memory system.

When to Use

When Not to Use

• Conceptual/semantic queries (use vector search)
• Synonym-heavy queries (use hybrid search)
• Current session context (already in memory)
• Time-based navigation (use TOC directly)

Quick Start

Prerequisites

memory-daemon status  # Check daemon
memory-daemon start   # Start if needed

Validation Checklist

Before presenting results:

• Daemon running: memory-daemon status returns "running"
• BM25 index available: teleport stats shows Status: Available
• Query returns results: Check for non-empty matches array
• Scores are reasonable: Higher BM25 = better keyword match

BM25 Search

Basic Usage

# Simple keyword search
memory-daemon teleport search "JWT token"

# Search with options
memory-daemon teleport search "authentication" \
  --top-k 10 \
  --target toc

# Phrase search (exact match)
memory-daemon teleport search "\"connection refused\""

Options

Output Format

BM25 Search: "JWT token"
Top-K: 10, Target: all

Found 4 results:
----------------------------------------------------------------------
1. [toc_node] toc:segment:abc123 (score: 12.45)
   JWT token validation and refresh handling...
   Time: 2026-01-30 14:32

2. [grip] grip:1738252800000:01JKXYZ (score: 10.21)
   The JWT library handles token parsing...
   Time: 2026-01-28 09:15

Index Statistics

memory-daemon teleport stats

Output:

BM25 Index Statistics
----------------------------------------
Status:        Available
Documents:     2847
Terms:         45,231
Last Indexed:  2026-01-30T15:42:31Z
Index Path:    ~/.local/share/agent-memory/tantivy
Index Size:    12.5 MB

Index Lifecycle Configuration

BM25 index lifecycle is controlled by configuration (Phase 16):

[teleport.bm25.lifecycle]
enabled = false  # Opt-in (append-only by default)
segment_retention_days = 30
grip_retention_days = 30
day_retention_days = 180
week_retention_days = 1825
# month/year: never pruned (protected)

[teleport.bm25.maintenance]
prune_schedule = "0 3 * * *"  # Daily at 3 AM
optimize_after_prune = true

Pruning Commands

# Check what would be pruned
memory-daemon admin prune-bm25 --dry-run

# Execute pruning per lifecycle config
memory-daemon admin prune-bm25

# Prune specific level
memory-daemon admin prune-bm25 --level segment --age-days 14

Index Administration

Rebuild Index

# Full rebuild from RocksDB
memory-daemon teleport rebuild --force

# Rebuild specific levels
memory-daemon teleport rebuild --min-level day

Index Optimization

# Compact index segments
memory-daemon admin optimize-bm25

Search Strategy

Decision Flow

User Query
    |
    v
+-- Contains exact terms/function names? --> BM25 Search
|
+-- Contains quotes "exact phrase"? --> BM25 Search
|
+-- Error message or identifier? --> BM25 Search
|
+-- Conceptual/semantic query? --> Vector Search
|
+-- Mixed or unsure? --> Hybrid Search

Query Syntax

Error Handling

Combining with TOC Navigation

After finding relevant documents via BM25 search:

# Get BM25 search results
memory-daemon teleport search "ConnectionTimeout"
# Returns: toc:segment:abc123

# Navigate to get full context
memory-daemon query node --node-id "toc:segment:abc123"

# Expand grip for details
memory-daemon query expand --grip-id "grip:..." --before 3 --after 3

Advanced: Tier Detection

The BM25 index is part of the retrieval tier system (Phase 17):

Check current tier:

memory-daemon retrieval status

See Command Reference for full CLI options.