Every frame of your product is a real capture, never a generated mockup.
Tell your AI coding agent to make a video of your project, and it does: a real browser
opens your real app, and the agent plans, narrates, scores and renders the result on
your machine.
Code-based video generation. An open-source video pipeline for AI coding agents: a real browser captures your real product, and your agent directs the video, rendered locally.
You ask your agent for a video. openvidstudio gives it the tools to actually make one:
It looks at your project, works out how to start it, and reads your brand (colours, fonts, logo).
It shows you a plan before it writes or renders anything: the story, the shots, the script, the music. You approve it or change it.
It captures your running app with a real browser (or a desktop window, a phone, a terminal), so the pixels are yours.
It narrates, scores and directs the footage: a voice, an original soundtrack, camera moves, transitions.
It renders locally and hands you the mp4, a poster frame and a description with chapters and credits.
Claude Code, Cursor, or any other client that speaks the
Model Context Protocol can drive it, through
forty-four tools (forty-five with the optional Higgsfield tier).
Why not just record the screen?
A screen recording shows what happened once. A directed video shows what matters. And the
usual alternatives each cost something: a timeline editor costs an afternoon, stock b-roll
shows a product that is not yours, and an AI video generator invents a product that does not exist.
openvidstudio sits in between. Nothing on screen is generated. It is your app, captured live,
then directed.
What it does differently
Real product, real pixels. The UI, the text and the layout are ground truth, not a mockup.
Your agent drives. No separate app and no keyframing by hand.
Directed, not recorded. Push-ins, depth, a cursor that arcs and settles, transitions that
hide the cut, and depth of field that is off by default so it never smears the text you are showing.
Music that does not tick.compose_score writes an original score for the video's arc, with
an exact beat grid and no drums by default (evenly repeated hits sound like a ticking clock
under a voiceover), and finalize_video warns if a track still ticks. find_music_bed searches
two public-domain catalogs (about 9,400 tracks) when you want a real track. No attribution
line, no content ID claim.
Not only web apps. A desktop window, an Android device or iOS Simulator, and a terminal
session are all capturable.
One edit costs one beat. Beats render to their own cached segments, so fixing a caption in a
five minute video re-renders ninety frames, not nine thousand.
Vertical when you need it. A project can be 1080x1920 from the start, with a Reels export,
instead of cropping a 16:9 video afterwards.
Quick start
openvidstudio runs as an MCP server. How you add it depends on where you use your agent.
Claude Code, the Claude desktop app's Code tab, and Cowork: install the plugin. It registers the
server and the four openvidstudio skills together, and it updates with one command.
The first start installs the server once (about 330 MB, about a minute, in the background). Details,
and what to do if you already added the server by hand, are in the plugin's README.
Claude Desktop (Chat tab), Cursor, VS Code: one command writes the right config file, then restart
the app. It never touches your other servers and keeps a backup.
On native Windows some clients need the command wrapped: "command": "cmd" with
"args": ["/c", "npx", "-y", "@openvidstudio/mcp-server"].
Forty-four tools should appear. If they do not, run preflight; it checks your machine and
names the fix for your platform.
To run from source instead (to contribute, or to try what has not been released), clone the repo and
build it, then point args at packages/mcp-server/dist/stdio.js with "command": "node":
git clone https://github.com/AnayDhawan/openvidstudio.git
cd openvidstudio
pnpm install
pnpm build
Then paste this to your agent, from inside the repo you want a video of:
make a demo video of this project with openvidstudio.
handle it end to end. work out what this project is and how to start it, get it
running, and use its own brand rather than your defaults. ask me anything you
need along the way, one question at a time, and show me the result when it is
done.
That is all you do. You do not have to know the port, read the tool list or put the steps in
order. The agent checks the machine, reads your palette and fonts, asks what the video should
cover, and shows you the whole plan before anything is written. Say yes and it captures, renders
and hands you the mp4.
What you need
What
Minimum
Needed for
Blocking
Node.js
20
Runs the server
yes
pnpm
9
Building from a clone
yes
ffmpeg
4.0
Sound effects, audio checks, pulling frames back out of a render
yes
Playwright
1.48
Resolvable inside the project the video is built in
yes
Chromium
whatever playwright install pulls
The browser capture actually drives
yes
Your app
running, reachable
Capture points at a real URL
yes
edge-tts
any current, Python 3.8+
Narration
no
numpy, scipy, soundfile, pyloudnorm, matplotlib
any current, Python 3.8+
compose_score, the original-score tool
no
preflight checks every row. Only the Node version is enforced numerically; the rest are presence
checks.
# Windows. Reopen the terminal after installing ffmpeg, winget only puts it on PATH for new shells.
winget install --id Gyan.FFmpeg -e
npm install playwright && npx playwright install chromium
pip install edge-tts
pip install numpy scipy soundfile pyloudnorm matplotlib # only for compose_score
# macOS
brew install ffmpeg
npm install playwright && npx playwright install chromium
pip install edge-tts
# Debian or Ubuntu
sudo apt update && sudo apt install -y ffmpeg
npm install playwright && npx playwright install chromium
pip install edge-tts
Remotion normally downloads its own Chromium build the first time it renders, separate from the
Playwright one above, which fails with no network. Set render.browserExecutable in
openvidstudio.config.json to the path preflight's chromium check reports, and render_video
reuses that browser instead.
Long renders and client timeouts
A full render can take many minutes, and an MCP client's own timeout (often 60 to 120 seconds) can
end the call long before the render does. Pass async: true to render_video: it starts the render
in the background and returns a job id, and job_status reports progress until the result is ready.
render_stills, validate_scenes with render: true and compose_score take tens of seconds to a
couple of minutes; if your client cuts them off, raise its tool timeout rather than retrying, since
a retry starts the work again. plan_production measures this machine's real render speed, so its
estimate is the number to plan with.
Director mode
For a launch video, a trailer, or anything that should look made rather than assembled, pick the
peak or mid creative level when the agent plans the production. It then works like a
director instead of filling in templates:
A brief first. The logline, the audience, the one feeling the video should leave, a story
device chosen for this product, an arc with stakes, the design rationale and the music plan. It
marks every shot as real or constructed. Nothing is written or rendered until you approve it.
A design system as code. Palette roles, a motif lifted from your brand, type and motion live
in one file that scenes and validators both read.
Music first. An original score is composed to the brief's sections and impact points, and
every cut lands on a downbeat. The agent cannot listen, so the tool reports numbers instead:
band balance, clicks, a ticking check, loudness per section.
Scenes written, not templated. The agent builds them from core's primitives and writes a new
component when the brief's motif needs one.
A review loop before any full render.render_stills renders several frames per beat, tiles
them into one labelled contact grid, and returns the numbers (cropping, text size, reading
time, contrast, safe area, dead frames). Look, fix, repeat, until the rubric in
CREATIVE.md passes.
Facts, not memory. Every tool's result is recorded and merged into facts.json. A number
shown on screen comes from there or from a capture.
Final assembly.finalize_video sets the loudness (-14 LUFS), writes a poster and a
size-capped share copy, and writes a description with chapters, credits and a disclosure of any
constructed sample content.
The three production modes are peak (full creative liberty), mid (strong defaults, mostly
authored scenes) and literal (exactly what you specified, templates welcome). The agent-facing
process is the openvidstudio-director skill, and the craft is written down in
CREATIVE.md.
Supernova mode (STEM lessons)
A separate kind of production for short animated lessons: one maths, physics or computer science concept in up to
two minutes, vertical, narrated, always in the same seven beats (hook, picture, set up, derive, use it, aha, close), with the
script written and approved first. init_project with mode: "supernova" makes a
project with the @openvidstudio/stem primitives (a glowing Spark and a one-per-video Nova, graphs, sans-serif equations
with a spoken form, force diagrams, array pictures), check_script lints the lesson script, and generate_narration
gives word timings so a scene can point at the word being said. The pieces are built and tested one by one; the voice
step needs edge-tts and a network, and no finished lesson has been published yet. The process is the supernova
skill and the design and voice are in SUPERNOVA.md.
The tools
Area
What it does
Key tools
Project setup
Scaffolds the project, checks the machine, plans the production
init_project · preflight · plan_production
Beats
Plans, validates and commits the script-and-shot skeleton every render is built from
plan_beats · validate_beats · write_beats_file
Capture
Browser, desktop, mobile and terminal capture of the real running app, zoom-compensated; a page can be split into depth layers
A README GIF, docs screenshots, store frames, a Reels file, or a 9:16 cut
export_rendition · reformat_vertical
Brand
Reads your repo's palette, fonts and logo
extract_brand
B-roll (optional)
Higgsfield atmosphere shots for what a screen cannot produce, never product UI
import_higgsfield_clip
TOOLS.md lists every tool with a one-line purpose, generated from the
server's own registry.
The capture engine on its own
packages/capture is zoom-compensated browser capture, interaction replay
and the desktop, mobile and terminal backends, with no dependency on Remotion, React or the video
pipeline. It is built to be used alone, if you want trustworthy captures of your app and are not
making a video. Install it alone with npm install @openvidstudio/capture.
Status
The pipeline runs end to end. An agent can go from a brief to a narrated mp4 without hand-writing
Remotion, and the director-mode run on a sample app passed 40 of 41 automated checks. Here is
where it is still rough:
The templates are plain. They are structurally correct, so a first render looks right
rather than good. Director mode is how you get past that, and the independent review of our
own sample video scored its craft 2 out of 5 before the cinematic camera work. That work has
landed; we have not yet measured it the same way.
No lesson has been published yet. Supernova, the mode for short STEM explainers, is built and
tested (see above), and none is out. The follow-up work is tracked in #107.
Both this repo and every scaffolded project also ship the official Remotion agent-skills bundle for
Claude Code, Cursor, Windsurf, OpenCode and GitHub Copilot, so whichever agent writes your scenes
has real Remotion knowledge to work from.
openvidstudio/
├── packages/
│ ├── capture/ # browser, desktop, mobile and terminal capture, no Remotion needed
│ ├── core/ # Remotion building blocks: camera, scenes, transitions, sound
│ ├── mcp-server/ # @openvidstudio/mcp-server: the 44-45 MCP tools an agent calls
│ ├── stem/ # @openvidstudio/stem: Supernova lesson primitives
│ └── docs/ # PLANNING · OVERVIEW · PIPELINE · STYLE · CREATIVE · CAPTURE · SCRIPT · API · NARRATION · TOOLS
├── templates/
│ └── default/ # the project init_project creates
├── evals/ # the director-mode eval: a sample app, a task, a scorer
├── brand/ # logo, wordmark, demo gif
└── .github/workflows/ # ci.yml, labeler.yml, stale.yml
Contributing
This project is built mostly by a couple of students, and contributions of every size are welcome:
a typo, a bug report with good steps, a scene template, a capture backend, a guide for the agent you
use. You do not need to know Remotion or video to help. Start with CONTRIBUTING.md,
look for good first issue,
and please follow our Code of Conduct. Security issues: see
SECURITY.md.
License
openvidstudio itself is Apache-2.0: free for individuals and companies of any size,
including this repo's own code, the MCP tools and the capture package.
It renders through Remotion, which is under Remotion's own license, not
Apache-2.0. Remotion is free to render with if you are an individual, a non-profit, evaluating it, or
a for-profit company with up to 3 employees. A larger for-profit company needs a paid Remotion
Company License (remotion.pro) to render with openvidstudio, the
same as it would to use Remotion directly. openvidstudio does not check for this; it is between you
and Remotion.
O
Openvidstudio
AnayDhawan·35 estrelas·
Code-based video generation. An open-source video pipeline for AI coding agents: a real browser captures your real product, and your agent directs the video, rendered locally.
Ainda não há lista de ferramentas. Servidores remotos recebem uma após a primeira verificação de saúde.
Perguntas frequentes
Perguntas
Como instalo Openvidstudio?
Copie nesta página a configuração stdio para Cursor, Claude Desktop, Claude Code ou VS Code. Ela usa a instalação do publicador, não um proxy do PlugMCP.
Openvidstudio precisa de uma chave de API?
Este registro não lista nenhuma variável de ambiente obrigatória.
Qual transporte Openvidstudio usa?
Openvidstudio roda localmente via stdio.
Quando Openvidstudio foi verificado pela última vez?
Última verificação: 2026-11-13. Code-based video generation. An open-source video pipeline for AI coding agents: a real browser captures your real product, and your agent directs the video, rendered locally.