Waymark Engine 🧭
Sub-millisecond AST structural & lexical BM25 code intelligence for AI coding agents.
Ask your codebase a question in plain English. Get an exact answer — file, symbol, and line span — in milliseconds, without paying 10,000–50,000 tokens of blind context-window re-reading.
Built for AI coding agents: one-shot discovery, zero required background processes (no daemon required; optional resident daemon for warm <10ms lookups), no vector index, and strictly deterministic zero-embedding recall.
Core Principles
- Deterministic by Construction: Every query path is zero-randomness. Semantic recall uses pure lexical BM25 (
@paragon-ux/capn-hook) and hard-refuses embedding stores (CAPN_NON_DETERMINISTIC_MODE). - Fail-Closed Defense: When structural or literal tiers miss, the Discovery Junction emits a machine-readable recommendation or clean miss. It never guesses.
- Two-Verb MCP Interface: Consolidates all discovery and maintenance into
waymark_askandwaymark_memory, slashing tool context overhead to <450 tokens (vs. 3,000+ for flat tool catalogs). - Sub-Millisecond Speed: In-memory
PrefixTrieand warm AST call graphs answer queries in<10mswith warm daemon IPC.
Architectural Division of Responsibilities
AI Coding Agent
│
┌──────────────┴──────────────┐
│ │
waymark_ask waymark_memory
(Read & Discover) (Write & Consensus)
│ │
┌───────────────┼───────────────┐ │
▼ ▼ ▼ ▼
Tier 1 Tier 2 Tier 3 Tier 4
AST Call Graph Path Router Fuzzy Matcher Lexical BM25
(@paragon/codedb) (PrefixTrie) (algo.go fzf) (capn-hook)
│ │ │ │
└───────────────┼───────────────┘ │
▼ ▼
Discovery Junction Consensus SQLite
(deterministic miss) (5-facet memory)
Quickstart
Requires Node.js 22+.
# 1. Install globally or as a project dependency
npm install -g waymark-engine
# 2. Initialize the deterministic lexical store (.waymark/memory)
waymark memory init
# 3. Query the codebase using Verb 1 (waymark ask)
waymark ask "Who calls verifyHop?" # Tier 1 exact structural call graph
waymark ask "verifyHop" --depth 2 --exclude-tests # Bounded BFS excluding test files
waymark ask --symbols User,Service,ApiWorker # Concurrent batch symbol definitions
waymark ask --path src/types.ts # Single-file structured AST outline
waymark ask "verfyHop" # Discovery Junction recommendation
waymark ask "verifyHop" --plain # Token-minimal LLM output (~16-38 tokens)
Live REPL Observer & Web Viewer
Launch the interactive REPL with real-time SSE telemetry and browser viewer on port 4141:
waymark repl --live --port 4141
Open the access URL printed by the REPL to observe query executions, tier breakdowns, and cache hits in a viewport-locked layout. Run /battery in the REPL to execute prompt evaluations. The URL contains a random session token; keep it private.
The live observer binds to 127.0.0.1, requires explicit telemetry opt-in with WAYMARK_TELEMETRY=1, and checks the session token on the dashboard and telemetry endpoints. It rejects mismatched Host or Origin headers and blocks framing. Persistent query/session logs are disabled by default. Do not expose the local viewer through a public interface.
Semantic Repo Map and Platform Limits
waymark memory bootstrap collects source excerpts as unverified drafts; it does not publish architectural claims or initialize a Capn store. Verify each claim against its cited code, then publish it explicitly with waymark memory chart. Subsystem-scoped discovery and maps are not supported; requests that include --subsystem or subsystem return UNSUPPORTED_SUBSYSTEM.
On Windows, WAYMARK_CODEDB_EXECUTABLE must name a native executable. .cmd and .bat shims are rejected with CODEDB_UNSUPPORTED_SHIM because Waymark never invokes external tools through a shell.
Language Support & Tier Fallbacks
- Single-File AST Outlines (
waymark ask --outline --path <file>):- In-Process
web-tree-sitter(Exact line + column spans): TypeScript (.ts,.tsx,.mts,.cts), JavaScript (.js,.jsx,.mjs,.cjs), Python (.py, including classes, functions, methods, and module-levelvariableassignments), JSON (.json, includingconfig_blockproperties andconfig_itemarray-object scopes keyed byid/name/key/*_idwith disjoint#indexdisambiguation on duplicate, conflicting, numeric, or#<digits>identifiers, failing closed withOUTLINE_LIMIT_EXCEEDEDabove 500 array elements), TOML (.toml), YAML (.yaml,.yml), Bash (.sh,.bash,.zsh), C# (.cs), Scala (.scala), Lua (.lua), Zig (.zig), Elixir (.ex,.exs), Solidity (.sol), Rust (.rs), Go (.go), Java (.java), C (.c,.h), C++ (.cpp,.hpp,.cc,.cxx), Ruby (.rb), Swift (.swift), Kotlin (.kt), and PHP (.php). - Native Markdown Parser (
markdown-it): Markdown (.md,.markdown,.mdx) document structure (h1–h6sections, headings,tableblocks,table_rowdata rows asrow:<first-cell>with(L<line>)disambiguation on repeated or pattern-colliding identifiers, andcode_blockfenced regions asfence:L<startLine>, all with inclusive full-line character spans). - Polyglot
codedb outlineFallback: Secondary fail-safe for supported compiled languages (.rs,.go,.java, , , , , , ) if WASM grammar is unavailable in host environment.
- In-Process
Comprehensive Documentation
For complete technical references, architectural specifications, and benchmark analyses, visit our Zensical Documentation Site:
- 📖 Getting Started: Installation, CLI flags, background daemon, and the Live REPL Observer.
- 🏗️ 4-Tier Architecture: Deep dive into AST call graphs, literal paths, Junegunn Choi
fzf, and BM25 consensus. - 🔌 Model Context Protocol (MCP): Two-Verb interface (<450 tokens), setup for Google Antigravity, Claude Desktop, and Cursor.
- 💻 CLI & Command Registry: Full canonical command reference,
waymark memoryaction catalog, and exit codes. - 🧭 Examples: Recorded runs of the
architectural-map,explore-subsystem, andbootstrap-semantic-mapMCP prompts against this repository. - 📊 Empirical Benchmarks: Steelmanned Aider Repo Map parity benchmark (
gpt-oss-120b) and metrics schema. - 📦 Library API & Integrity: TypeScript SDK,
verifyHop,anchorForRange, and fail-closed invariant guards.
Local Documentation Server
Build or preview the documentation locally using Zensical:
# Build static site
py -3.11 -m zensical build
# Serve live preview on localhost:8000
py -3.11 -m zensical serve
License
MIT. See LICENSE for details.
