How the AI Works

SmartDocs uses a graph-backed retrieval strategy inspired by Andrej Karpathy's LLM Wiki pattern. Here's how the pipeline works from end to end.

The Philosophy: Compile Once, Query Many

Traditional RAG (Retrieval-Augmented Generation) works like this:

User asks → chunk documents → find similar chunks → feed to LLM

The problem: the LLM re-derives understanding from scratch on every query. Ask the same question twice, and it does the same work twice. No accumulation. No compounding.

Karpathy's insight was different: build a persistent artifact once, then query it. Instead of retrieving raw document chunks, an LLM agent compiles source material into interlinked markdown files — a structured knowledge graph. The knowledge is compiled once and kept current. Queries are fast because the cross-references already exist, contradictions already flagged, synthesis already done.

SmartDocs adapts this pattern for documentation sites. Your documentation is the compiled artifact. We don't need an LLM to build it — you wrote it. What we need is:

1. A way to navigate the connections between pages
2. A way to find the right pages for any question
3. A way to feed full context to the LLM without chunking

The Graph: Build Time

When you run npm run build:graph, SmartDocs scans every .md file in your content/ directory and builds two files:

page-index.json

A catalog of every page with searchable metadata:

{
  "slug": "/guides/configuration",
  "title": "Configuration Guide",
  "headings": ["Environment Variables", "Providers", "Security"],
  "keywords": ["smartdocs", "api", "environment", "variables", "openai"],
  "description": "SmartDocs is configured through environment variables...",
  "wordCount": 121
}

graph.json

A link graph built from your cross-references:

{
  "/getting-started/quickstart": ["/guides/configuration"],
  "/guides/configuration": ["/api/reference"],
  "/": ["/getting-started/quickstart", "/guides/configuration", "/api/reference"]
}

Every [link](/other-page) in your markdown becomes a directed edge in the graph. This is the Karpathy insight applied: the cross-references are already there. SmartDocs reads them.

Retrieval: Query Time

When a reader asks a question, SmartDocs retrieves context in four steps:

Step 1: Keyword Scoring

The query is broken into terms. Each page is scored against those terms:

Step 2: Select Top Pages

The highest-scoring 1-2 pages are selected as starting points.

Step 3: Graph Traversal

SmartDocs follows the outbound links from those pages in graph.json. If "Configuration" links to "API Reference", the API page is pulled in as additional context. This is why cross-references matter — they tell SmartDocs which pages are related.

Step 4: Full Page Context

Instead of chunking pages into fragments, SmartDocs sends the entire markdown content of 3-5 pages to the LLM. Modern context windows (128K-200K tokens) handle this easily — a typical documentation page is 500-2000 words (~1000-4000 tokens).

Why no vector DB? Documentation pages are discrete units of knowledge. If a user asks about authentication, they want the authentication page — not a semantically-similar paragraph from a deployment guide. Keyword matching plus graph traversal is more precise than embedding-based retrieval for docs.

Search: Fuse.js

The built-in search bar (Cmd+K) uses Fuse.js, a lightweight fuzzy search library.

Unlike the AI chat, search runs entirely in the browser with zero API calls. When you type:

const fuse = new Fuse(pageIndex.pages, {
  keys: [
    { name: "title", weight: 3 },
    { name: "headings", weight: 2 },
    { name: "keywords", weight: 1 },
    { name: "description", weight: 0.5 },
  ],
  threshold: 0.4,
});

Fuse.js uses a Bitap algorithm for approximate string matching. "quikstart" still finds "Quickstart". "env var" matches "Environment Variables". Results appear instantly — no server round-trip needed.

The search is powered by the same page-index.json that the AI chat uses. One build artifact, two features.

LLM Call: Multi-Provider

SmartDocs supports any OpenAI-compatible API. The prompt structure:

You are SmartDocs, an AI documentation assistant.
Answer based SOLELY on the documentation below.

DOCUMENTATION:
--- PAGE: /guides/configuration (Configuration Guide) ---
[full markdown content]
--- PAGE: /api/reference (API Reference) ---
[full markdown content]

USER QUESTION: How do I set up the API key?

Answer helpfully and concisely. Cite source pages.

The LLM response streams back to the chat widget in real-time via Server-Sent Events (SSE). Each chunk updates the displayed text, creating the typewriter effect.

All LLM calls happen server-side in a Next.js API route. Your API key never leaves the server. The browser only sees the streaming response.

Architecture Diagram

┌─────────────────────────────────────────────────────────┐
│                    BUILD TIME                             │
│                                                           │
│  content/*.md ──► build-graph.js ──► .smartdocs/         │
│       │                               ├─ page-index.json │
│       │                               └─ graph.json      │
│       │                                                  │
│       └──────────► Markdoc ──► Static HTML               │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│                    QUERY TIME                             │
│                                                           │
│  User: "How do I configure the API key?"                  │
│       │                                                  │
│       ▼                                                  │
│  ┌─────────────┐     ┌──────────────┐                    │
│  │ Keyword     │────►│ Top 2 pages  │                    │
│  │ Scoring     │     │ by relevance  │                    │
│  └─────────────┘     └──────┬───────┘                    │
│                              │                           │
│                    ┌─────────▼─────────┐                 │
│                    │ Graph Traversal   │                 │
│                    │ Follow outbound   │                 │
│                    │ links for context │                 │
│                    └─────────┬─────────┘                 │
│                              │                           │
│                    ┌─────────▼─────────┐                 │
│                    │ Full pages as     │────► LLM ──►    │
│                    │ prompt context    │     (streaming) │
│                    │ (no chunking)      │                │
│                    └───────────────────┘                 │
└─────────────────────────────────────────────────────────┘

Why This Beats Traditional RAG for Docs

Inspiration: Andrej Karpathy's LLM Wiki — "The knowledge is compiled once and then kept current, not re-derived on every query. This is the key difference: the wiki is a persistent, compounding artifact."