Ai Video Editor

AI Video Editor

A local, open-source CLI tool that automatically edits talking-head videos using AI. Point it at a raw recording and it removes silences, filler words ("um", "uh", "ну", "типа"), and failed takes (say "cut cut" to mark a restart). It uses Silero VAD for speech detection, Whisper large-v3 for transcription with word-level timestamps, and FFmpeg for frame-accurate assembly. Optionally generates a smart hook opener and YouTube chapter markers via LLM. Runs entirely on your machine — no cloud APIs required for the core pipeline. Designed to be invoked by AI agents (structured JSON output) or used as an MCP server in Claude Desktop.

Features

• Silence removal — Silero VAD strips dead air; short natural pauses are preserved
• Filler word removal — English and Russian filler words ("um", "uh", "ну", "типа", "эээ", ...)
• Restart detection — "cut cut" / "кат кат" removes the entire failed take; short isolated bursts (coughs, false starts) are auto-removed via VAD duration filtering
• Repeated sentence detection — auto-detects and cuts duplicate sentence starts
• Smart hook generation — OpenRouter LLM picks the best 8-second opener (optional)
• YouTube chapter markers — LLM generates timestamped chapters from the transcript (optional)
• Hardware encoding — h264_videotoolbox on Apple Silicon for fast final encode
• Configurable YAML pipeline — every threshold, model, and feature toggle is overridable

Requirements

• Python 3.11+
• FFmpeg 7+ (brew install ffmpeg)
• macOS with Apple Silicon (for VideoToolbox hardware encoding; falls back to libx264 elsewhere)
• ~4 GB RAM for Whisper large-v3 (use --whisper-model small for lighter machines)

Installation

cd Tools/ai-video-editor
uv venv
source .venv/bin/activate
uv pip install -e .

Quick Start

# Core editing only (no API calls)
ai-video-editor process video.mp4 --no-hook --no-chapters

# Full pipeline with smart hook opener + YouTube chapters (requires OPENROUTER_API_KEY)
ai-video-editor process video.mp4

Example output (stdout):

{
  "status": "complete",
  "input": "video.mp4",
  "output_video": "video_edited.mp4",
  "duration_original_sec": 154.6,
  "duration_edited_sec": 145.9,
  "segments_removed": 1,
  "silence_removed_sec": 2.1,
  "restarts_removed": 1,
  "fillers_removed": 4
}

Progress events are emitted to stderr as JSON lines during processing.

CLI Commands

process — Edit a video

ai-video-editor process VIDEO [OPTIONS]

info — Inspect a video file

ai-video-editor info VIDEO

Prints codec, resolution, FPS, duration, and bitrate as JSON.

models — List Whisper model sizes

ai-video-editor models

Configuration

The default config lives in config.default.yml. Override any section with --config my.yml — your file is deep-merged over the defaults, so you only need to specify what changes.

Example override — use a smaller Whisper model and enable audio enhancement:

# my-config.yml
whisper:
  model: small
audio:
  enabled: true

ai-video-editor process video.mp4 --config my-config.yml

How It Works

The pipeline runs four sequential phases:

1. Analysis — Extract audio → Silero VAD (speech segments) → Whisper transcription → edit decisions (remove short bursts, merge short gaps, remove restarts and fillers, apply padding)
2. Assembly — Frame-accurate FFmpeg segment extraction → concat → optional audio enhancement → optional LUT color grade
3. AI Enhancement — Smart hook selection and YouTube chapters via OpenRouter (skipped if --no-hook --no-chapters or no API key)
4. Encode — Final h264_videotoolbox (or libx264) encode with AAC audio, optimized for web playback

Claude Desktop (MCP Server)

The tool includes a built-in MCP server so Claude Desktop can use it as a native capability.

Setup:

1. Add to your claude_desktop_config.json (see claude_desktop_config.example.json for the template):

"ai-video-editor": {
  "command": "/path/to/ai-video-editor/.venv/bin/python",
  "args": ["-m", "src.mcp_server"],
  "cwd": "/path/to/ai-video-editor"
}

2. Restart Claude Desktop.
3. Optionally install SKILL.md as a capability for model selection guidance and workflow tips.

Claude gets three tools: process_video, video_info, and list_models.

Environment Variables

License

MIT — see LICENSE.