A CVE ingestion and enrichment pipeline that builds a single normalised SQLite
database stitched from CVEProject, NVD, CISA KEV, EPSS, and OSV — plus a
Streamlit explorer for the result.
Ships as a fully populated SQLite DB on each release so you don't have to wait
hours for the initial ingest.
What's in it
CVE ingestion + enrichment pipeline with an MCP server for AI agents.
星數
38
工具
—
可用率
—
Full CVE corpus from the upstream CVEProject cvelistV5 zip
NVD bulk merge for CVSS metrics, CPE configurations, vuln status, and
references
CISA Known Exploited Vulnerabilities with date_added, due_date,
ransomware flag, and required action
EPSS exploit probability + percentile, batched
OSV package mapping per CVE
Streamlit explorer with ~70 pre-built views over the data
Resume-safe pipeline state — interruptions pick up where they left off
If you just want to query the data without running the pipeline:
# 1. Clone and pick the extras you need
git clone https://github.com/mez-0/vulnify.git
cd vulnify
uv sync --extra mcp # for agents / MCP server
# or: uv sync --extra explore # for the Streamlit dashboard
# or: uv sync --all-extras # for everything
# 2. Grab the latest release DB
gh release download -R mez-0/vulnify --pattern vulnify.db.zst
zstd -d vulnify.db.zst -o vulnify.db
# 3. Point .env at the DB
cat > .env <<EOF
VULNIFY_SQLITE_PATH=$(pwd)/vulnify.db
VULNIFY_SCHEMA_SQL_PATH=vulnify/db/schema.sql
EOF
# 4a. Use it from an agent (see "MCP server" below for full setup)
uv run vulnify-mcp </dev/null # smoke test — should start then exit on EOF
# 4b. Or browse it in the Streamlit explorer
uv run streamlit run explore/app.py
# 4c. Or just query the SQLite directly
sqlite3 vulnify.db "SELECT cve_id, summary FROM cve WHERE cve_id = 'CVE-2024-3094';"
Building the DB from scratch
uv sync
cp .env.example .env # edit paths as needed
uv run vulnify-gather
The first run takes several hours without an NVD API key (the public NVD
rate limit is ~5 requests / 30 seconds). With a key it's roughly 10x faster.
The pipeline writes phase watermarks to the pipeline_run table, so re-runs
are idempotent — NVD only fetches changes since the last watermark, KEV
re-syncs in full, EPSS only updates stale scores.
The exploit phases (Nuclei / Exploit-DB / Metasploit) fetch and cache their
corpora under ~/.vulnify/cache and watermark on corpus version (commit SHA /
content hash). They re-parse whenever cvelistV5 ingestion ran this gather — an
ingest cascade-deletes a re-touched CVE's artefacts, so they're healed on the
next run — and otherwise auto-skip an unmoved corpus. An explicit --skip-*
always wins. The tri-state exploit.public_poc / exploit.metasploit summary
bools are materialised from the artefacts: null until the contributing
sources have run, then true/false.
Configuration
All config lives in .env at the project root (loaded by vulnify.settings).
Variable
Required
Default
Purpose
VULNIFY_SQLITE_PATH
yes
vulnify.db
Where the SQLite DB lives. Empty/unset switches the CVEProject ingest to print-only mode.
VULNIFY_SCHEMA_SQL_PATH
yes
vulnify/db/schema.sql
DDL used to bootstrap a new DB.
VULNIFY_KEV_JSON_PATH
no
known_exploited_vulnerabilities.json
Where the CISA KEV catalog is cached. The KEV phase downloads it there and refreshes it once it is a day old.
VULNIFY_NVD_API_KEY
no
—
NVD API key. Without it you're rate-limited to ~5 req / 30s.
VULNIFY_CACHE_DIR
no
~/.vulnify/cache
Where the exploit corpora (Nuclei / Exploit-DB / Metasploit) are cached; each is re-downloaded once a day old.
Relative paths are resolved against the project root.
20 tables, all normalised, foreign keys enabled. Top-level CVE plus separate
tables for the high-cardinality bits (CPE matches, references, CVSS metrics,
exploit signals + artefacts, KEV listing, intel) so you can join cleanly.
cve — primary CVE record
vendor — deduped vendors (case/whitespace-normalised)
product — products owned by a vendor
cve_vendor — many-to-many with `role` (assigner, affected, source, …)
affected_product — CVE <-> product affected/not-affected with notes
version_range — per affected_product version constraints
cwe / cve_cwe — weakness taxonomy
cvss — every CVSS vector from any source, deduped on (source, version, vector)
cpe_match — NVD configuration leaf matches
exploit — exploit availability + maturity (tri-state summary bools)
exploit_artefact — per-CVE exploit evidence (Nuclei/Exploit-DB/Metasploit)
kev — CISA KEV listing fields per CVE
intel — generic threat-intel rows (kind/value/source)
intel_string_list — backing list values for intel rows
reference — URL references with trust + source
reference_tag — many-to-many for ref tag vocabulary
tag / cve_tag — CVE-level tags
pipeline_run — per-phase watermarks for resume
vulnify.mcp exposes the SQLite database to AI agents over the
Model Context Protocol. It's a FastMCP
server speaking stdio, so agents like Claude Code, Claude Desktop, and
Cursor spawn it as a subprocess on demand.
Full tool reference, per-agent setup, and troubleshooting:
docs/MCP.md.
Step 1 — set up the project
git clone https://github.com/mez-0/vulnify.git
cd vulnify
uv sync --extra mcp
Step 2 — get a SQLite database
Two options:
Option A — pre-built (recommended). Download the release asset and
decompress:
Full hydrated CVE record — CVSS, KEV, EPSS, exploits, references, CWEs, affected products, plus an artefacts evidence list.
search_cves(...)
Filter by vendor / product / cwe / kev_only / min_cvss / min_epss / year. The vendor filter resolves four ways (vendor edge, product owner, product name, CPE vendor slug) — see MCP.md.
list_vendors(query, limit)
Vendor names with CVE / product counts and their NVD CPE slugs. Use it to disambiguate a vendor term before trusting an empty result.
exploits_for(cve_id)
Concrete exploit artefacts (Nuclei / Exploit-DB / Metasploit) with source, stable id, url, and exact/parsed confidence.
references_for(cve_id, tag)
Reference URLs with their tags, optionally filtered to one tag (e.g. patch, exploit).
list_kev(since, limit)
Recent CISA KEV entries, newest first.
database_overview()
Totals, published-date range, last pipeline-run times.
search_cves_text(query, limit)
FTS5 free-text search over title / summary / technical details. AND / OR / NOT / quoted phrases / prefix* all supported.
All tools return JSON-serialisable dicts. Limits cap at 100 rows.
Example agent prompts
"What's CVE-2024-3094?"
"Show me KEV entries added in the last 60 days about Cisco."
"Find CVEs about container escape with CVSS ≥ 8."
"Which products have the most CVEs published in 2025?"
Troubleshooting
Tools don't show up in an already-running agent — MCP servers are
attached at session start, not when claude mcp add runs. Exit your
current Claude Code / Cursor session and start a fresh one — don't
/resume an old one, that re-reads the old tool list.
claude mcp list shows ✗ Failed to connect — first thing to
check is the exact Args: field with claude mcp get vulnify; a
copy-paste truncation that turns vulnify-mcp into vulnify-mc is
a common one. Smoke test the spawned command verbatim in a shell:
uv run --directory /path/to/vulnify vulnify-mcp </dev/null.
uv: command not found in the agent's process — agents inherit a
minimal PATH. Either use an absolute path to uv in the config
(command: "/home/you/.cargo/bin/uv") or symlink uv into
/usr/local/bin.
VULNIFY_SQLITE_PATH is unset at startup — your .env isn't being
picked up. The server resolves .env relative to the vulnify package
root; pass an absolute --directory in the agent config so it lands in
the right place.
Tool calls return nothing — confirm the DB has data with
uv run python -c "import sqlite3; print(sqlite3.connect('vulnify.db').execute('SELECT COUNT(*) FROM cve').fetchone())".
Logs disappearing — vulnify-mcp logs to stderr (so it doesn't
corrupt the MCP protocol on stdout). Most agent UIs surface stderr in
their MCP logs view; in Claude Code that's /mcp then pick the server.
Explorer
explore/app.py is a Streamlit dashboard with roughly 70 pre-built views over
the data: severity distributions, KEV listing lag, EPSS density, CWE
co-occurrence heatmaps, vendor / product pressure, CVSS vector breakdowns,
publication recency, exploit-in-the-wild monthly trends, and more.
uv sync --extra explore
uv run streamlit run explore/app.py
Views are organised into 14 tabs (Overview, Vendors, Trends, Vendor links,
Severity & CWE, Enrichment, Exploit landscape, KEV deep dive, CVSS vector,
Products & CPEs, References, Threat intel, Lifecycle, Pipeline health) and split
across two query modules: explore/data.py (overview / landing) and
explore/data_views.py (the per-chart queries). Full tab-by-tab breakdown and
how to add a view: docs/EXPLORE.md.
Tests
uv run pytest
Tests are hermetic — no network, no real DB. They cover:
SQLite store upsert + dedup behaviour
Schema migration + backfill
NVD bulk merge
Pipeline state watermarks
Enrichment resume logic
Explorer query shapes
MCP tool surface (structured + FTS5 search)
Releases
Each release ships:
vulnify.db.zst — the full SQLite DB, zstd-compressed. Includes the
FTS5 index pre-built from v2026.05.21.1 onwards, and the
product_cpe_vendor vendor index pre-built from v2026.07.25.1 onwards, so
vulnify-mcp is ready to serve immediately on first start rather than
spending ~40s building them. Decompress with zstd -d vulnify.db.zst.
schema.sql — the DDL, in case you only want the structure.
Tag scheme:
vYYYY.MM.DD — major snapshot, data captured on that date.
vYYYY.MM.DD.N — code patch against the same data snapshot
(e.g. v2026.05.21.1 is the 2026-05-21 data with the vulnify-mcp
server added).
The code version in pyproject.toml follows SemVer independently.