AI Video Editor
A local, open-source CLI tool that automatically edits talking-head videos using AI. Point it at a raw recording and it removes silences, filler words ("um", "uh", "ну", "типа"), and failed takes (say "cut cut" to mark a restart). It uses Silero VAD for speech detection, Whisper large-v3 for transcription with word-level timestamps, and FFmpeg for frame-accurate assembly. Optionally generates a smart hook opener and YouTube chapter markers via LLM. Runs entirely on your machine — no cloud APIs required for the core pipeline. Designed to be invoked by AI agents (structured JSON output) or used as an MCP server in Claude Desktop.
Features
- Silence removal — Silero VAD strips dead air; short natural pauses are preserved
- Filler word removal — English and Russian filler words ("um", "uh", "ну", "типа", "эээ", ...)
- Restart detection — "cut cut" / "кат кат" removes the entire failed take; short isolated bursts (coughs, false starts) are auto-removed via VAD duration filtering
- Repeated sentence detection — auto-detects and cuts duplicate sentence starts
- Smart hook generation — OpenRouter LLM picks the best 8-second opener (optional)
- YouTube chapter markers — LLM generates timestamped chapters from the transcript (optional)
- Hardware encoding —
h264_videotoolboxon Apple Silicon for fast final encode - Configurable YAML pipeline — every threshold, model, and feature toggle is overridable
Requirements
- Python 3.11+
- FFmpeg 7+ (
brew install ffmpeg) - macOS with Apple Silicon (for VideoToolbox hardware encoding; falls back to libx264 elsewhere)
- ~4 GB RAM for Whisper
large-v3(use--whisper-model smallfor lighter machines)
Installation
cd Tools/ai-video-editor
uv venv
source .venv/bin/activate
uv pip install -e .
Quick Start
# Core editing only (no API calls)
ai-video-editor process video.mp4 --no-hook --no-chapters
# Full pipeline with smart hook opener + YouTube chapters (requires OPENROUTER_API_KEY)
ai-video-editor process video.mp4
Example output (stdout):
{
"status": "complete",
"input": "video.mp4",
"output_video": "video_edited.mp4",
"duration_original_sec": 154.6,
"duration_edited_sec": 145.9,
"segments_removed": 1,
"silence_removed_sec": 2.1,
"restarts_removed": 1,
"fillers_removed": 4
}
Progress events are emitted to stderr as JSON lines during processing.
CLI Commands
process — Edit a video
ai-video-editor process VIDEO [OPTIONS]
info — Inspect a video file
ai-video-editor info VIDEO
Prints codec, resolution, FPS, duration, and bitrate as JSON.
models — List Whisper model sizes
ai-video-editor models
Configuration
The default config lives in config.default.yml. Override any section with --config my.yml — your file is deep-merged over the defaults, so you only need to specify what changes.
Example override — use a smaller Whisper model and enable audio enhancement:
# my-config.yml
whisper:
model: small
audio:
enabled: true
ai-video-editor process video.mp4 --config my-config.yml
How It Works
The pipeline runs four sequential phases:
- Analysis — Extract audio → Silero VAD (speech segments) → Whisper transcription → edit decisions (remove short bursts, merge short gaps, remove restarts and fillers, apply padding)
- Assembly — Frame-accurate FFmpeg segment extraction → concat → optional audio enhancement → optional LUT color grade
- AI Enhancement — Smart hook selection and YouTube chapters via OpenRouter (skipped if
--no-hook --no-chaptersor no API key) - Encode — Final h264_videotoolbox (or libx264) encode with AAC audio, optimized for web playback
Claude Desktop (MCP Server)
The tool includes a built-in MCP server so Claude Desktop can use it as a native capability.
Setup:
- Add to your
claude_desktop_config.json(seeclaude_desktop_config.example.jsonfor the template):
"ai-video-editor": {
"command": "/path/to/ai-video-editor/.venv/bin/python",
"args": ["-m", "src.mcp_server"],
"cwd": "/path/to/ai-video-editor"
}
- Restart Claude Desktop.
- Optionally install
SKILL.mdas a capability for model selection guidance and workflow tips.
Claude gets three tools: process_video, video_info, and list_models.
Environment Variables
License
MIT — see LICENSE.
