Waymark Engine

Waymark Engine 🧭

Sub-millisecond AST structural & lexical BM25 code intelligence for AI coding agents.

-informational)

Ask your codebase a question in plain English. Get an exact answer — file, symbol, and line span — in milliseconds, without paying 10,000–50,000 tokens of blind context-window re-reading.

Built for AI coding agents: one-shot discovery, zero required background processes (no daemon required; optional resident daemon for warm <10ms lookups), no vector index, and strictly deterministic zero-embedding recall.

Core Principles

• Deterministic by Construction: Every query path is zero-randomness. Semantic recall uses pure lexical BM25 (@paragon-ux/capn-hook) and hard-refuses embedding stores (CAPN_NON_DETERMINISTIC_MODE).
• Fail-Closed Defense: When structural or literal tiers miss, the Discovery Junction emits a machine-readable recommendation or clean miss. It never guesses.
• Two-Verb MCP Interface: Consolidates all discovery and maintenance into waymark_ask and waymark_memory, slashing tool context overhead to <450 tokens (vs. 3,000+ for flat tool catalogs).
• Sub-Millisecond Speed: In-memory PrefixTrie and warm AST call graphs answer queries in <10ms with warm daemon IPC.

Architectural Division of Responsibilities

                               AI Coding Agent
                                      │
                       ┌──────────────┴──────────────┐
                       │                             │
                 waymark_ask                  waymark_memory
               (Read & Discover)             (Write & Consensus)
                       │                             │
       ┌───────────────┼───────────────┐             │
       ▼               ▼               ▼             ▼
    Tier 1          Tier 2          Tier 3        Tier 4
 AST Call Graph   Path Router    Fuzzy Matcher  Lexical BM25
(@paragon/codedb) (PrefixTrie)   (algo.go fzf)  (capn-hook)
       │               │               │             │
       └───────────────┼───────────────┘             │
                       ▼                             ▼
               Discovery Junction           Consensus SQLite
             (deterministic miss)           (5-facet memory)

Quickstart

Requires Node.js 22+.

# 1. Install globally or as a project dependency
npm install -g waymark-engine

# 2. Initialize the deterministic lexical store (.waymark/memory)
waymark memory init

# 3. Query the codebase using Verb 1 (waymark ask)
waymark ask "Who calls verifyHop?"            # Tier 1 exact structural call graph
waymark ask "verifyHop" --depth 2 --exclude-tests  # Bounded BFS excluding test files
waymark ask --symbols User,Service,ApiWorker  # Concurrent batch symbol definitions
waymark ask --path src/types.ts               # Single-file structured AST outline
waymark ask "verfyHop"                        # Discovery Junction recommendation
waymark ask "verifyHop" --plain               # Token-minimal LLM output (~16-38 tokens)

Live REPL Observer & Web Viewer

Launch the interactive REPL with real-time SSE telemetry and browser viewer on port 4141:

waymark repl --live --port 4141

Open the access URL printed by the REPL to observe query executions, tier breakdowns, and cache hits in a viewport-locked layout. Run /battery in the REPL to execute prompt evaluations. The URL contains a random session token; keep it private.

The live observer binds to 127.0.0.1, requires explicit telemetry opt-in with WAYMARK_TELEMETRY=1, and checks the session token on the dashboard and telemetry endpoints. It rejects mismatched Host or Origin headers and blocks framing. Persistent query/session logs are disabled by default. Do not expose the local viewer through a public interface.

Semantic Repo Map and Platform Limits

waymark memory bootstrap collects source excerpts as unverified drafts; it does not publish architectural claims or initialize a Capn store. Verify each claim against its cited code, then publish it explicitly with waymark memory chart. Subsystem-scoped discovery and maps are not supported; requests that include --subsystem or subsystem return UNSUPPORTED_SUBSYSTEM.

On Windows, WAYMARK_CODEDB_EXECUTABLE must name a native executable. .cmd and .bat shims are rejected with CODEDB_UNSUPPORTED_SHIM because Waymark never invokes external tools through a shell.

Language Support & Tier Fallbacks

• Single-File AST Outlines (waymark ask --outline --path <file>):
  • In-Process web-tree-sitter (Exact line + column spans): TypeScript (.ts, .tsx, .mts, .cts), JavaScript (.js, .jsx, .mjs, .cjs), Python (.py, including classes, functions, methods, and module-level variable assignments), JSON (.json, including config_block properties and config_item array-object scopes keyed by id/name/key/*_id with disjoint #index disambiguation on duplicate, conflicting, numeric, or #<digits> identifiers, failing closed with OUTLINE_LIMIT_EXCEEDED above 500 array elements), TOML (.toml), YAML (.yaml, .yml), Bash (.sh, .bash, .zsh), C# (.cs), Scala (.scala), Lua (.lua), Zig (.zig), Elixir (.ex, .exs), Solidity (.sol), Rust (.rs), Go (.go), Java (.java), C (.c, .h), C++ (.cpp, .hpp, .cc, .cxx), Ruby (.rb), Swift (.swift), Kotlin (.kt), and PHP (.php).
  • Native Markdown Parser (markdown-it): Markdown (.md, .markdown, .mdx) document structure (h1–h6 sections, headings, table blocks, table_row data rows as row:<first-cell> with (L<line>) disambiguation on repeated or pattern-colliding identifiers, and code_block fenced regions as fence:L<startLine>, all with inclusive full-line character spans).
  • Polyglot codedb outline Fallback: Secondary fail-safe for supported compiled languages (.rs, .go, .java, .c, .cpp, .rb, .swift, .kt, .php) if WASM grammar is unavailable in host environment.
  • Fail-Closed Boundary: Unsupported extensions (e.g. .dart due to ABI 15 incompatibility, .sql) and dialect extensions (.jsonc, .json5) fail closed with UNSUPPORTED_LANGUAGE.
• Repo-Wide Call Graph & Symbol Search (waymark ask "<symbol>"):
  • Indexed via @paragon-ux/codedb-core across .ts, .tsx, .js, .jsx, .py, .rs, .go, .java, .rb, .swift, .kt (plus partial .c/.cpp/.php), supplemented by in-process web-tree-sitter repo-wide indexing for .mjs, .cjs, .mts, .cts, and .toml.
• Language-Agnostic Tiers (Tiers 2, 3, and 4):
  • Tier 2 (PrefixTrie Path Router), Tier 3 (fzf Fuzzy Matcher), and Tier 4 (.waymark/memory BM25 Consensus Memory & 5-Facet Semantic Repo Map) operate across all repository files regardless of language.

Comprehensive Documentation

For complete technical references, architectural specifications, and benchmark analyses, visit our Zensical Documentation Site:

• 📖 Getting Started: Installation, CLI flags, background daemon, and the Live REPL Observer.
• 🏗️ 4-Tier Architecture: Deep dive into AST call graphs, literal paths, Junegunn Choi fzf, and BM25 consensus.
• 🔌 Model Context Protocol (MCP): Two-Verb interface (<450 tokens), setup for Google Antigravity, Claude Desktop, and Cursor.
• 💻 CLI & Command Registry: Full canonical command reference, waymark memory action catalog, and exit codes.
• 🧭 Examples: Recorded runs of the architectural-map, explore-subsystem, and bootstrap-semantic-map MCP prompts against this repository.
• 📊 Empirical Benchmarks: Steelmanned Aider Repo Map parity benchmark (gpt-oss-120b) and metrics schema.
• 📦 Library API & Integrity: TypeScript SDK, verifyHop, anchorForRange, and fail-closed invariant guards.

Local Documentation Server

Build or preview the documentation locally using Zensical:

# Build static site
py -3.11 -m zensical build

# Serve live preview on localhost:8000
py -3.11 -m zensical serve

License

MIT. See LICENSE for details.