Comprehensive, automated security investigations powered by Microsoft Sentinel, Defender XDR, Graph API, and threat intelligence — with 25 specialized Agent Skills
🎛️ Walkthrough using GitHub Copilot (15 min): See the new Mission Control canvas app in action — . A guided tour of the launchable skills dashboard: filter investigation skills by domain, search, and one-click launch skills or route an entity to the right investigation.
Automated security investigation tool using Microsoft MCP Servers, GitHub Copilot, Python Modules and custom copilot-instructions.
📺 Walkthrough using VS Code: See this project in action — Watch on YouTube (starts at the Security Investigator demo). Covers the end-to-end workflow: natural language investigations, MCP server integration, KQL query execution, threat intelligence enrichment, and automated report generation.
An investigation automation framework that combines GitHub Copilot, VS Code Agent Skills, and Model Context Protocol (MCP) servers to enable natural language security investigations. Ask questions like "Investigate this user for the last 7 days" or "Is this IP malicious?" and get comprehensive analysis with KQL queries, threat intelligence correlation, and professional reports.
🖥️ Also runs in the GitHub Copilot app (desktop) — ideal for scheduled automations (unattended Threat Pulse / Threat Intel Campaign runs). It uses a worktree-per-session model with a few setup differences from VS Code — see Running in the GitHub Copilot App.
Quick Start (TL;DR)
# 1. Clone and open in VS Code
git clone https://github.com/SCStelz/security-investigator.git
code security-investigator
# 2. Set up Python environment
python -m venv .venv
.venv\Scripts\Activate.ps1 # Windows
# source .venv/bin/activate # macOS/Linux
pip install --require-hashes -r requirements.lock # Hash-verified (recommended)
# pip install -r requirements.txt # Without hash verification
# 3. Configure environment
copy config.json.template config.json
# Edit config.json → add your Sentinel workspace ID, tenant ID
copy .env.template .env
# Edit .env → add your API tokens (ipinfo, AbuseIPDB, vpnapi, Shodan)
# 4. Configure MCP servers
copy .vscode\mcp.json.template .vscode\mcp.json
# All platform servers are pre-configured — just needs a GitHub PAT on first use
# GitHub Copilot app / CLI instead? Use the user-scope template:
# copy .copilot\mcp-config.json.template $env:USERPROFILE\.copilot\mcp-config.json
# (works as-is — see "Running in the GitHub Copilot App" below)
# 5. Open Copilot Chat (Ctrl+Shift+I) in Agent mode and start with:
# "Run a threat pulse scan"
🚀 Recommended first run: The Threat Pulse skill is the best starting point. It runs a broad-spectrum scan across 9 security domains (incidents, identity, endpoint, exposure, email, UEBA, auth spray, privileged ops, CVEs) and produces prioritized findings with color-coded verdicts (🔴 Escalate / 🟠 Investigate / 🟡 Monitor / ✅ Clear). Each finding includes a drill-down recommendation pointing to a specialized skill — so after the scan, you'll know exactly where to focus and which follow-up command to run.
Other example prompts:
"Investigate [email protected] for the last 7 days" → user-investigation
"Analyze incident 12345" → incident-investigation
"Is this IP malicious? 203.0.113.42" → ioc-investigation
"What skills do you have access to?" → lists all 25 skills
25 Agent Skills — Modular investigation workflows for incidents, users, devices, IoCs, authentication, scope drift (SPN/User/Device), MCP monitoring, exposure management, AI agent posture, app registration posture, identity posture, data security analysis, email threat posture, MITRE ATT&CK coverage, ingestion analysis, detection authoring, threat pulse scanning, SVG dashboards, and more
6 MCP Server Integrations — Sentinel Data Lake (incl. Exposure Graph tools), Graph API, Defender XDR Triage, KQL Search, Microsoft Learn, Azure MCP Server
Python Utilities — HTML report generation with IP enrichment (geolocation, VPN detection, abuse scores, Shodan port/service/CVE intelligence)
🤖 Agent Skills
This system uses VS Code Agent Skills to provide modular, domain-specific investigation workflows. Skills are automatically detected based on keywords in your prompts.
DataSecurityEvents (Purview/IRM) analysis: SIT access breakdowns, user risk ranking, file inventory, DLP policy correlation, Copilot SIT exposure, SIT GUID-to-name resolution, anomaly detection. Designed for 100k+ user environments
"data security", "sensitive information type", "SIT access", "DLP events", "DataSecurityEvents", "EDM access", "insider risk activity", "Purview data security"
After running an investigation, ask follow-up questions without re-running the entire workflow:
Is that IP a VPN?
Trace authentication for that suspicious location
Was MFA used for those sign-ins?
Skills can be chained for comprehensive analysis:
1. "Investigate incident 12345" → incident-investigation extracts entities
2. "Now investigate the user from that incident" → user-investigation runs on extracted UPN
3. "Check if that IP is malicious" → ioc-investigation analyzes the suspicious IP
4. "Show me a heatmap of the attack patterns" → heatmap-visualization
Copilot uses existing investigation data from temp/investigation_*.json when available.
Discovering Skills
What investigation skills do you have access to?
Explain the high-level workflow of the user-investigation skill
What data sources does the ioc-investigation skill use?
Authoring New Skills & Queries from Investigations
Ad-hoc investigations naturally evolve into reusable assets. After completing an investigation, ask Copilot to package the verified queries, schema pitfalls, and analytical logic into a new SKILL.md or query file.
"Based on the investigation we just completed, create a new reusable skill"
"Read this threat intel article: <URL> — extract TTPs and IOCs, then write, test, and tune a queries file for reusable threat hunts"
The queries/ folder contains verified, battle-tested KQL query collections organized by detection scenario. These are the Priority 2 lookup source in the KQL Pre-Flight Checklist — Copilot searches them before writing any ad-hoc KQL.
Each file uses a standardized metadata header for efficient grep_search discovery:
The discovery manifest provides a machine-readable index of all query files and skills, enabling deterministic cross-referencing by domain and MITRE technique. The Threat Pulse skill loads this manifest to match findings to downstream query files and drill-down skills automatically.
discovery-manifest.yaml — Compact index (~500 lines) with title, path, domains, mitre, and prompt fields for each query file and skill
build_manifest.py — Generator script that scans queries/ metadata headers and skill YAML frontmatter to produce the manifest
How it works:
Query files declare **Domains:** tags in their metadata header (valid tags: incidents, identity, spn, endpoint, email, admin, cloud, exposure)
Skills declare threat_pulse_domains: and drill_down_prompt: in their YAML frontmatter
python .github/manifests/build_manifest.py scans both and emits the manifest
The Threat Pulse skill reads the manifest to match non-✅ findings → relevant query files and skills by domain tag and MITRE technique overlap
Regenerate after creating or renaming query files/skills, or changing Domains:/threat_pulse_domains: values:
python .github/manifests/build_manifest.py
🚀 Setup
Prerequisites
Requirement
Details
VS Code
Version 1.99+ recommended (Agent mode + MCP support).
GitHub Copilot
Active subscription — Copilot Pro+, Business, or Enterprise. Agent mode must be enabled.
Python 3.8+
For IP enrichment utility and report generation. Download
Azure CLI
Required for Azure MCP Server (underlying auth) and sentinel-ingestion-report skill (az monitor log-analytics query for all KQL queries, az rest for analytic rule inventory, az monitor log-analytics workspace table list for tier classification). Install. Authenticate: az login --tenant <tenant_id>, then az account set --subscription <subscription_id>. Requires Log Analytics Reader (KQL queries + table list) and Microsoft Sentinel Reader (analytic rule inventory) on the workspace.
log-analytics CLI extension
Required by the sentinel-ingestion-report skill for az monitor log-analytics query (all KQL queries in Phases 1-5). Install: az extension add --name log-analytics. Verify: az extension list --query "[?name=='log-analytics']".
PowerShell 7.0+
Required for sentinel-ingestion-report skill (parallel query execution via ForEach-Object -Parallel). Install. Verify: $PSVersionTable.PSVersion.
Node.js 18+
Required for KQL Search MCP (npx) and building local MCP Apps. Download or install via winget install OpenJS.NodeJS.LTS (Windows) / brew install node (macOS).
Microsoft Sentinel
Log Analytics workspace with data. You'll need the workspace GUID and tenant ID.
Entra ID Permissions
If you can query Sentinel in the Azure Portal, you likely have sufficient access. The Graph MCP server requires a one-time tenant provisioning by an admin. See MCP Server Setup for detailed per-server requirements.
Microsoft.Graph PowerShell
Required for detection-authoring skill (CustomDetection.ReadWrite.All — create/update/delete custom detection rules via Graph API). Also used by sentinel-ingestion-report skill for rule inventory (CustomDetection.Read.All — read-only, degrades gracefully if not installed). Install-Module Microsoft.Graph.Authentication -Scope CurrentUser.
GitHub PAT
public_repo scope — Create one here. Used by KQL Search MCP.
1. Install Dependencies
Verify prerequisites:
python --version # Requires 3.8+
node --version # Requires 18+ (needed for KQL Search MCP)
az --version # Azure CLI (needed for Azure MCP Server, ingestion report skill)
pwsh --version # Requires 7.0+ (needed for sentinel-ingestion-report skill)
If Node.js is missing: Download or run winget install OpenJS.NodeJS.LTS (Windows) / brew install node (macOS).
If Azure CLI is missing: Install, then az login --tenant <tenant_id> and az account set --subscription <subscription_id>.
If the log-analytics extension is missing: az extension add --name log-analytics (required for sentinel-ingestion-report skill).
cd mcp-apps/sentinel-geomap-server; npm install; npm run build; cd ../..
cd mcp-apps/sentinel-heatmap-server; npm install; npm run build; cd ../..
cd mcp-apps/sentinel-incident-comment; npm install; npm run build; cd ../..
Bash (macOS/Linux):
cd mcp-apps/sentinel-geomap-server && npm install && npm run build && cd ../..
cd mcp-apps/sentinel-heatmap-server && npm install && npm run build && cd ../..
cd mcp-apps/sentinel-incident-comment && npm install && npm run build && cd ../..
This project also runs great in the GitHub Copilot app (desktop), not just VS Code. The app is especially useful for scheduled automations: its built-in Workflows system runs skills unattended on a schedule — for example the Daily Threat Pulse and Weekly Threat Intel Campaign definitions in automations/. It also runs each task in an isolated git worktree, so you can run multiple investigations in parallel without them colliding.
Scheduled automations (Workflows) — run the Threat Pulse scan every morning, or the Threat Intel Campaign weekly, fully unattended. See automations/ for ready-to-import, PII-free workflow definitions.
Parallel, isolated sessions — each session is its own git worktree + branch, so concurrent investigations don't step on each other.
Background agents — long-running hunts and report generation can run in the background while you keep working.
🎛️ Mission Control (Canvas extension)
The app also supports Canvas extensions — interactive panels that render beside the chat, where both you and the agent act on the same surface. This repo ships one: Mission Control, a launchable dashboard for the whole framework. It self-registers from .github/extensions/skills-canvas/ — no build step, no install.
Just start a session and prompt:
💬 "Start Mission Control"
The panel opens with three tabs:
🗂️ Skills — every investigation skill as a launchable card. Filter by domain, search, and one-click launch — with optional fleet-wide scope and inline/markdown report toggles.
📋 Queries — all KQL query files, domain-filtered. Open any to read the markdown or hand it to the agent as context for a targeted hunt.
📊 Findings — when the agent finishes an investigation it records the result back into the canvas (severity, entities, evidence, and tailored one-click follow-ups). Click a severity pill to filter.
The human launches work; the agent reports back to the same surface — a control plane for investigations, not just a UI.
Skills tab — launch any investigation with optional fleet-wide scope and inline/markdown output. The left rail filters skills and queries by domain; the entity quick-launch bar routes a UPN / IP / hash / device / incident to the right investigation.
Findings tab — the agent reports each result back to the canvas: summary, evidence chips, linked reports, and tailored one-click follow-ups (e.g. drill into a flagged CVE, user, or incident).
⚠️ Caveats & differences vs VS Code
The app behaves slightly differently from VS Code. The most important difference: each session is a fresh git worktree, and gitignored local files (config.json, .env, MCP config) do not exist in a newly created worktree — they live only in your main checkout. You need a small post-checkout step to copy them into each session, otherwise the first query fails with a missing-config error.
Concern
VS Code
GitHub Copilot app
MCP config
.vscode/mcp.json (per workspace)
User scope:~/.copilot/mcp-config.json — start from .copilot/mcp-config.json.template (see MCP config for the app below). The platform servers and the kql-searchGITHUB_TOKEN go here, not in .vscode/mcp.json. Authenticate once interactively so OAuth tokens cache and refresh silently in scheduled runs.
config.json / .env
One copy in your workspace folder
Gitignored, so absent in each new worktree — must be copied in per session (see below).
Memory / tenant context
VS Code AppData memory store (auto-loads ~200 lines)
~/.copilot/memories/ (user) and ~/.copilot/memories/repo/ (repo). Scheduled runs are non-interactive, so repo memory does NOT auto-load — automation prompts must read the context file explicitly by path (the automations/ definitions do this in STEP 1.5).
Sessions / workspace
Single workspace folder, one branch
One git worktree + branch per session, created under your worktrees root (e.g. ~/copilot-worktrees/<repo>/<branch>). Operate only inside the session worktree — never the main checkout.
MCP config for the app
The app (and Copilot CLI) reads MCP servers from user scope (~/.copilot/mcp-config.json), not .vscode/mcp.json. The format differs from VS Code: top-level mcpServers key, strict JSON (no comments / inputs block), and no ${workspaceFolder} variable. A ready-to-use template with all platform servers is provided:
# Windows (merge manually if you already have a mcp-config.json)
copy .copilot\mcp-config.json.template $env:USERPROFILE\.copilot\mcp-config.json
Set a GITHUB_TOKEN environment variable (GitHub PAT, public_repo scope) — kql-search reads it via ${env:GITHUB_TOKEN}.
Run az login --tenant <tenant_id> for azure-mcp-server (uses AzureCliCredential).
Restart the app / CLI and sign in once interactively so OAuth tokens cache for scheduled runs.
Multi-tenant (optional): If your account has access to multiple tenants (e.g., guest/B2B or CSP), pin the Sentinel and Graph servers to a specific tenant by adding a header to sentinel-data-mcp, sentinel-triage-mcp, and Graph-mcp-server:
Without it, sign-in uses your account's home tenant.
Optional MCP Apps (geomap, heatmap, incident-comment) need absolute paths in user scope. After building them, add entries like the following to mcpServers:
Use mcp-apps/sentinel-heatmap-server/dist/main.js for the heatmap, and mcp-apps/sentinel-incident-comment/dist/index.js (plus "env": { "SENTINEL_COMMENT_WEBHOOK_URL": "<your Logic App URL>" }) for incident comments.
Post-checkout workflow (persist config into each session)
Because config.json, .env, and .vscode/mcp.json are gitignored, a freshly created worktree won't contain them. Set up a post-checkout git hook that copies them from your main checkout into the new worktree. Example (.git/hooks/post-checkout, marked executable):
#!/usr/bin/env bash
# Copy gitignored local config from the main checkout into a fresh worktree/session.
MAIN="/path/to/your/main/checkout" # adjust to your primary clone
for f in config.json .env .vscode/mcp.json; do
if [ ! -f "$f" ] && [ -f "$MAIN/$f" ]; then
mkdir -p "$(dirname "$f")"
cp "$MAIN/$f" "$f"
fi
done
Alternatively, define the same copy logic as a per-session setup command in the app. Either way the goal is identical: every session ends up with a valid config.json, .env, and MCP config before the first query runs.
For scheduled automations, the workflow prompts in automations/ also include a STEP 1 bootstrap that recreates config.json from known values if it's missing — a belt-and-suspenders guarantee for non-interactive runs (it never commits config.json, which stays gitignored).
🔌 MCP Server Setup
The system uses several Model Context Protocol (MCP) servers. All are pre-configured in .vscode/mcp.json.template — copy it to .vscode/mcp.json to get started (see Step 3 above). For the GitHub Copilot app / CLI, use .copilot/mcp-config.json.template instead (see MCP config for the app). The sections below document permissions, tools, and installation guides for each server.
Exposure Graph tools (bundled here):find_blastradius, find_exposure_perimeter, find_walkable_paths, find_connected_nodes, find_nodes, get_graph_context, plus entity analysis (analyze_user_entity, analyze_url_entity, analyze_application_entity). These ship in the Data Lake server with no graph_ prefix.
Permissions:
Log Analytics Reader (minimum) — query workspace data
Sentinel Reader (recommended) — full investigation capabilities
# 1. Install the Entra Beta PowerShell module (v1.0.13+)
Install-Module Microsoft.Entra.Beta -Force -AllowClobber
# 2. Authenticate to your tenant
Connect-Entra -Scopes 'Application.ReadWrite.All', 'Directory.Read.All', 'DelegatedPermissionGrant.ReadWrite.All'
# 3. Register the MCP Server and grant permissions to VS Code
Grant-EntraBetaMCPServerPermission -ApplicationName VisualStudioCode
This only needs to be done once per tenant. After provisioning, all users in the tenant can use the Graph MCP server by signing in with their own account.
Permissions (delegated, per-user):
User.Read.All — user profiles and authentication methods
UserAuthenticationMethod.Read.All — MFA methods
Device.Read.All — device compliance and enrollment
Install via VS Code extension: search "Azure MCP Server" in Extensions, or install from the Marketplace. The extension registers as a stdio MCP server automatically.
Tools:monitor_workspace_log_query, monitor_activitylog_list, group_list, subscription_list, and 40+ namespaces covering AI, identity, security, databases, storage, compute, and networking.
Permissions:
Reader (minimum) — read-only access to Azure resources
Log Analytics Reader — for workspace_log_query (KQL against Log Analytics)
Contributor — for write/modify operations (optional)
Configuration: Requires azure_mcp parameters in config.json (tenant, subscription, resource group, workspace name) to avoid cross-tenant auth errors. See Configure Environment.
Verify Setup
Open Copilot Chat (Ctrl+Shift+I) in Agent mode and try these prompts:
Test
Prompt to type in Copilot Chat
Sentinel Data Lake
List my Sentinel workspaces
Microsoft Graph
Look up my user profile in Graph
Sentinel Triage
List recent security incidents
KQL Search
What columns does the SigninLogs table have?
Microsoft Learn
Search Microsoft docs for KQL query language
All skills
What investigation skills do you have access to?
If any server fails, check the MCP Servers panel in VS Code (click the {} icon in the bottom status bar) to verify each server shows a green connected status.
Token priority: If ipinfo_token is a paid plan, VPN detection is included and vpnapi_token is optional. Shodan uses the full API when a paid key is available; on 403/429 it automatically falls back to the free InternetDB.
IP enrichment happens during report generation (not data collection), so you can re-generate reports without re-querying Sentinel/Graph.
Dependencies
pip install -r requirements.txt
Core packages: requests (HTTP client for enrichment APIs), python-dateutil (date parsing for KQL time ranges).
🔒 Security Considerations
Confidential Data — Reports contain PII and sensitive security data. Mark as CONFIDENTIAL and follow organizational data classification policies.
Access Control — Restrict access to authorized SOC personnel. Use Azure RBAC for Sentinel, PIM for Graph API permissions.
Audit Trail — All investigations are timestamped. JSON files in temp/ preserve snapshots; HTML reports include generation metadata.
Data Retention — Investigations older than 3 days are auto-deleted (configurable). Archive important investigations before cleanup.
API Token Security — Never commit config.json with tokens (already in .gitignore). Use environment variables or Azure Key Vault for production.
Investigation JSON Files — Stored in temp/ (not committed to Git). Contain complete data including IP enrichment. Can be re-analyzed without re-querying.
🛠️ Troubleshooting
Issue
Solution
"No anomalies found"
Signinlogs_Anomalies_KQL_CL table doesn't exist or has no data. See user-investigation skill docs. Wait 24h for initial population.
"IP enrichment failed"
ipinfo.io rate limits (1K/day free). Add token to config.json for 50K/month.
"MCP server not available"
Check VS Code MCP server config. Verify authentication tokens are valid.
"User ID not found" (Graph)
Verify UPN is correct. Check Graph permissions: User.Read.All.
"Sentinel query timeout"
Reduce date range. Add | take 10 to limit results.
python enrich_ips.py 8.8.8.8 # Verifies IP enrichment API tokens
🧠 (Optional) Persistent Tenant Context
GitHub Copilot Chat in VS Code provides agents with a memory tool — a built-in filesystem (/memories/) for persisting notes across conversations. Copilot already uses this internally; you can extend it with tenant-specific context (known infrastructure IPs, validated personnel, false-positive patterns, lab automation signatures) so investigations don't repeatedly mis-classify documented activity as 🔴 critical.
Two memory tiers are relevant:
Tier
Path
Auto-loaded?
Use for
User memory
/memories/*.md
✅ Yes (~200 lines)
Short trigger rules ("when you see tenant X, read repo file Y")
Repo memory
/memories/repo/*.md
❌ Filenames only
Rich tenant context (IPs, personnel, FP patterns) — pulled in by trigger rules
The memory tool is an internal agent capability — VS Code does not publish a dedicated docs page for it. Closest related concepts are custom instructions and Agent Skills, which serve different purposes (always-applied conventions and specialized workflows, respectively).
This workspace ships with:
Templates in notes/memory/examples/ — copy and adapt for your tenant (one user-tier example, two repo-tier examples)
Sync scriptscripts/sync-repo-memory.ps1 — backs up workspace-scoped (repo) memory from VS Code AppData into the workspace folder, surviving VS Code reinstall and workspace rename. Any cloud sync attached to your workspace (OneDrive, Dropbox, iCloud, etc.) then mirrors the backup across machines. Defaults to one-way export (ToBackup); restore mode (FromBackup) requires -Force because it writes into Copilot's trusted memory store.
Setup guidenotes/memory/README.md — full walkthrough, sync usage, security model, and the trigger-rule pattern that makes Copilot actually consult repo memory
Quickstart: Open a template from notes/memory/examples/, then ask Copilot in chat to "create this as a memory file at /memories/..., replacing placeholders with my tenant values." Copilot uses its memory tool to write it directly — no AppData path navigation needed.
🎛️ From Mission Control (Copilot app): The Mission Control canvas surfaces the whole memory loop as one-click actions in its Findings tab — no manual path navigation or prompt-writing:
⚙️ Set memory file — names your tenant context-memory file (under .copilot/memories/repo/) and saves it to config.json. The memory features work on first run with a sensible per-tenant default even before you customize it.
🧠 Use memory — prepends a directive to every investigation you launch, telling the agent to consult the context-memory file — searching it for the task's specific entities rather than reading it in full — and apply its documented ground truth (known-good IPs, automation fingerprints, account classifications, false-positive rules) before rendering a verdict.
🧠 Compact to memory — hands the agent a propose-only prompt (via the context-memory-review skill) that distills accumulated findings into candidate ADD / MODIFY / FLAG changes for the memory file, for human approval. Offered again before you clear findings, so evidence is preserved.
Together these close the loop: findings recorded in the canvas → compacted into tenant memory → applied automatically on the next investigation.
⚠️ Memory = trusted input. Anything in notes/memory/repo/ becomes authoritative instructions for Copilot in every future chat (with MCP tool access to Sentinel, Graph, Azure). Review diffs from forks/PRs before restoring, never paste secrets, and if your workspace is cloud-synced, confirm the destination is acceptable for security context. See notes/memory/README.md for the full threat model.
📄 License
This project is licensed under the MIT License. Use it, fork it, adapt it for your SOC — just keep the copyright notice.
🙏 Acknowledgments
Microsoft Security Platform
Microsoft Sentinel — SIEM/SOAR platform powering all KQL queries and incident management
Special thanks to the Microsoft Security community for sharing KQL queries and detection logic, and to stefanpems for the Sentinel incident commenting MCP pattern.
S
Security Investigator
SCStelz·250 ดาว·
Automated security investigation tool using Microsoft MCP Servers, GitHub Copilot, Python Modules and custom copilot-instructions.