Open-source detection signatures for AI/ML attacks and breaches.
A vendor-neutral collection of Sigma, YARA, and Suricata rules for detecting compromise of LLM applications, MCP servers, ML supply chains, AI infrastructure, AI-powered insider threats, and RAG/vector database attacks.
A vendor-neutral collection of Sigma, YARA, and Suricata rules for detecting compromise of LLM applications, MCP servers, ML supply chains, AI infrastructure, AI-powered insider threats, and RAG/vector database attacks.
星标
129
工具
—
可用率
—
Browse it at depalmar.github.io/ai-dfir-toolkit — the
artifact catalog, the detection rules, the ATLAS/OWASP mappings, the case
studies, and the investigation guide, rebuilt from this repository on every
push so the page cannot drift from the content.
For the companion investigation guide with attack background, forensic artifacts, Mermaid attack-chain diagrams, and hands-on investigation procedures, see docs/ai-dfir-investigation-guide.md.
License: Apache 2.0.
Disclaimer
This is an independent personal project. It is not affiliated with, endorsed by, or produced on behalf of any employer. All research is based on public sources — published CVEs, vendor advisories, academic papers, and vendor-neutral security research. Contributions are welcome from the community.
Why this exists
Most existing detection content is either locked behind vendor SIEMs or scattered across blog posts. AI/ML attacks need detection coverage that spans:
This pack uses open standards only so the rules can be deployed in any modern detection stack via open Sigma/YARA/Suricata tooling.
Structure
ai-dfir-toolkit/
├── 01-llm-prompt-injection/ # Prompt injection, jailbreaks, indirect injection
├── 02-mcp-attacks/ # MCP tool poisoning, config tampering, rug pulls
├── 03-model-supply-chain/ # Pickle exploits, HuggingFace, dependency confusion
├── 04-ai-infrastructure/ # ShadowRay, Triton, MLflow, GPU abuse
├── 05-copilot-assistant-abuse/ # M365 Copilot, GitHub Copilot, Claude, Cursor
├── 06-rag-vector-db/ # Vector DB exposure, RAG poisoning
├── 07-runtime-ai-malware/ # Malware calling an LLM API at runtime
├── 08-agentic-orchestration/ # Agent-as-operator, AI service API as C2
├── 09-agent-memory-forensics/ # Memory as persistence, context poisoning
├── collectors/ # Read-only acquisition with chain of custody
├── playbooks/ # CACAO v2.0 response playbooks
├── artifacts/ # Machine-readable AI agent artifact catalog
├── skills/ # Agent skills for maintaining the catalog
├── docs/ # Investigation guide
├── tests/ # Sample events / test files
├── MAPPINGS.md # ATLAS + OWASP cross-reference
└── README.md
Each category directory contains a README.md describing the threats covered and rule files in their native format (.yml for Sigma, .yar for YARA, .rules for Suricata).
The rules are vendor-neutral Sigma — see the pySigma backends list to convert to any other SIEM query language.
Case sensitivity: the keyword-based rules (prompt injection, jailbreak,
system-prompt extraction, etc.) follow the Sigma convention that contains
matching is case-insensitive — attacker text varies in case, so the rules are
written in lowercase and rely on the backend to fold case. On Elastic,
ensure the matched content fields are mapped as analyzed text (the default
for string fields), not keyword: a keyword mapping matches
case-sensitively and will miss capitalized input. Do not try to force this
with the re|i modifier — Lucene's regex engine does not support the (?i)
flag and the resulting query is invalid.
YARA scanning
# Scan a model directory
yara -r ai-dfir-toolkit/03-model-supply-chain/*.yar /path/to/models/
# Scan an MCP config
yara ai-dfir-toolkit/02-mcp-attacks/mcp_tool_poisoning.yar \
~/Library/Application\ Support/Claude/claude_desktop_config.json
artifacts/ is a machine-readable catalog of the endpoint
artifacts left by AI coding agents, local LLM runtimes, agentic workflow
engines, and Model Context Protocol components — install paths, credential
storage, MCP configs, listening ports, process relationships, and registry keys,
each rated by forensic value and sourcing confidence, with collection priorities
for IR triage.
Eleven tool categories are covered: local LLM runtimes, coding agents and IDEs,
agent frameworks, coding-agent CLIs, workflow engines, browser agents,
computer-use agents, MCP hosts, cloud agents, code-completion tools, and
code-execution agents. Alongside them sit documented incident case studies, each
carrying its own provenance and recording analyst disagreement in a contested
field rather than resolving it.
Ships vendor-neutral Sigma detection rules and an osquery inventory pack, plus
generated feeds in four downstream formats:
Actor profiles: 20 profiles, covering attributed groups and
reporter-named clusters.
Case studies: 22, with 221 indicators.
Pinned frameworks: ATLAS 2026.09, ATT&CK 19.2, CISA KEV, and the OWASP
LLM and Agentic 2026 lists.
Each profile is derived from the cases and reported sightings that cite it. It
gives the techniques the actor uses, the rule in this repository that detects
each one, the catalogued tools the actor abused, and which of those tools'
artifacts to collect first. The site adds:
a cross-filtering Threat intel view: ATLAS matrix with coverage gaps,
victimology, timeline, technique heatmap, detection backlog, actor similarity
and clusters
a Maltego-style graph workbench
a collection plan that generates a self-contained triage script (PowerShell or
bash) or the equivalent Velociraptor / KAPE command
the framework pins refresh weekly and auto-merge when only reference data
changed
new public reporting arrives as intel-candidate issues for a person to
research
For current counts — catalogued tools, documented artifacts, credential
locations, MCP config paths, case studies, and the risk and confidence
breakdowns — see the Contents table in artifacts/README.md.
That table is generated by scripts/readme_counts.py and gated in CI, so it is
the one place those numbers are guaranteed current.
Coverage overview
70 rule files containing 161 individual signatures — nine attack-class categories
plus a cross-tool endpoint set:
Category
Files
Signatures
ATLAS Techniques
OWASP LLM 2026
OWASP Agentic
LLM Prompt Injection
8
10
T0051, T0054, T0029
LLM01, LLM02, LLM06, LLM08
ASI01
MCP Attacks
5
14
T0010, T0115.002, T0110, T0086
LLM02, LLM03, LLM04
ASI02, ASI03, ASI04, ASI05
Model Supply Chain
8
23
T0010, T0018, T0020
LLM04
ASI03, ASI04, ASI05
AI Infrastructure
9
31
T0011, T0049, T0029
LLM06
—
Copilot/Assistant Abuse
8
19
T0086, T0024
LLM02, LLM03, LLM04
ASI01, ASI02, ASI03, ASI04, ASI05
RAG / Vector DB
5
17
T0020
LLM01, LLM02, LLM09
ASI01, ASI06
Runtime AI-Malware
8
16
T0096, T0086
LLM01, LLM03
—
Agentic Orchestration & AI-Service C2
3
12
T0096, T0086, T0054
LLM03
ASI02
Agent Memory & Context Poisoning
2
5
T0080, T0080.000, T0086
LLM01, LLM02, LLM03
ASI06
Endpoint (cross-tool)
14
14
T0053, T0081, T0083, T0098
LLM02, LLM03, LLM04, LLM06
ASI02, ASI03, ASI04, ASI05
Total
70
161
The endpoint set lives in artifacts/detections/ and is
scoped to agent behaviour on a host rather than to one attack class, so it applies
across every tool in the artifact catalog. An osquery pack there answers the
inventory question ("which hosts have this") that the Sigma rules cannot.
Signature count includes multi-document Sigma YAML, multiple rule blocks inside a single YARA file, and multiple alert lines inside a single Suricata .rules file. One file often covers several related variants.
These rules are written to err toward signal over noise, but every environment is different. Each rule includes:
falsepositives: section listing known FP scenarios
level: field (low / medium / high / critical) — start with high+ in production
Tunable selectors so you can scope to specific orgs/users/namespaces
Recommended rollout:
Deploy to a test index/workspace for 7 days
Triage hits, tune selection and filter blocks
Promote to production with appropriate severity
Testing
The tests/ directory contains sample artifacts (malicious and benign) for validating rule correctness. Run:
cd tests/
./validate.sh
Expected result: 15 passes, 0 failures.
The suite hard-exits if the yara binary is missing rather than reporting a
clean run — absent tooling would otherwise score every "expect no matches"
assertion as a pass and the suite would look healthy while testing nothing.
Contributing
PRs welcome. Rule submission requirements:
Reference an attack technique — ATLAS ID, CVE, or published research
Include test data in tests/ — sample log line, file, or PCAP
Document false positives in the rule
Use lowercase, snake_case filenames matching the threat
OWASP Top 10 for LLM Applications 2026 — detection content maps to the 2026 list; eight of the ten IDs changed meaning from 2025, so an ID quoted from an older report names a different category here
You are free to use, modify, and redistribute these rules in commercial and non-commercial settings. Attribution appreciated but not required.
Catalog data under artifacts/catalog/, artifacts/case-studies/, and
artifacts/docs/api/ is licensed CC BY 4.0; see
artifacts/LICENSE-DATA. All other content,
including the scripts and schema under artifacts/, is under the repository's
Apache-2.0 licence.
Maintainer: Raymond DePalma — independent security researcher.
This is a personal project and is not affiliated with any employer.
A
Ai Dfir Toolkit
depalmar·129 个星标·
A vendor-neutral collection of Sigma, YARA, and Suricata rules for detecting compromise of LLM applications, MCP servers, ML supply chains, AI infrastructure, AI-powered insider threats, and RAG/vecto
A vendor-neutral collection of Sigma, YARA, and Suricata rules for detecting compromise of LLM applications, MCP servers, ML supply chains, AI infrastructure, AI-powered insider threats, and RAG/vector database attacks.
上次检查时间:2026-10-27。 A vendor-neutral collection of Sigma, YARA, and Suricata rules for detecting compromise of LLM applications, MCP servers, ML supply chains, AI infrastructure, AI-powered insider threats, and RAG/vector database attacks.