# Video as Code

> Video as code treats a video as a versioned, declarative spec rendered to frames, making every cut reproducible, reviewable and safe for AI agents.

Source: https://metavert.io/video-as-code  
Published: 2026-10-20  
Updated: 2026-10-20

**Video as code** is the practice of describing a video as a declarative, versioned specification and letting a renderer turn that specification into frames. The spec says which source shots play and between which in and out points, what captions, graphics and sound cues sit on top, and how the mix is set. It replaces a proprietary project file built up from clicks on a timeline as the source of truth. The idea applies the logic of [docs as code](https://metavert.io/docs-as-code) and infrastructure as code to moving pictures: plain text that can be diffed, reviewed, validated and rendered again with the same result. As of October 2026 the term has no standards body or canonical definition. This page sets out a working one, because the practice has become the main way [AI agents](https://metavert.io/agentic-ai) are trusted to edit video.

### Where the Idea Comes From

Editors have described cuts as text for decades. The [edit decision list](https://metavert.io/edit-decision-list) (EDL), born in the linear tape-editing era, is a plain list of events: which reel, which source timecodes, where they land in the program. Final Cut Pro's XML formats and [OpenTimelineIO](https://github.com/AcademySoftwareFoundation/OpenTimelineIO) carry richer versions of the same thing. OpenTimelineIO describes itself as "an interchange format and API for editorial cut information" and is explicit that it is not a container for media. Those formats were built to move an edit between tools. They were never meant to be where the edit is authored.

Programmatic rendering supplied the other half. [Remotion](https://www.remotion.dev) ("make videos programmatically with React") renders React components to video frame by frame. Motion Canvas does the same with TypeScript generators, and [Manim](https://github.com/ManimCommunity/manim) does it in Python for mathematical animation. Cloud APIs such as Shotstack render JSON templates at scale. What changed in 2025 and 2026 is the author. [Language models](https://metavert.io/large-language-models) write structured text and components well, and they operate graphical timelines poorly. Once agents became capable editors, the spec stopped being an export format and became the working copy.

### Anatomy of a Video Spec

Implementations differ, but a mature video spec has the same parts:

- **Sources by identity.** Media is referenced by a content hash (usually SHA-256), not a file path, so the same spec renders the same footage on any machine. Content-addressed storage also makes deduplication and provenance cheap.
- **Clips.** An ordered list of source in and out points, with speed, framing (crop and reframe), transitions and split-edit audio offsets.
- **Layers.** Captions derived from the transcript's word timings, overlays from templates or components with validated parameters, and audio layers with gain, fades and ducking.
- **Output.** Size, frame rate, platform target and loudness target, so a 16:9 master and a 9:16 short are two outputs of one edit.
- **Rationale and evidence.** For agent-authored specs, each clip carries the reason it is there and the transcript line or frame that justified it, so a reviewer can audit the cut instead of trusting it.
- **A resolution step.** Human-friendly units (seconds) are resolved into integer frames before rendering, so one timing implementation serves both the preview and the final file.

A schema governs the whole document. That makes impossible edits, such as a speed outside the supported range or an overlay that points at a missing asset, fail validation before anything renders.

### Why It Matters for AI Agents

Video as code gives agent editing the same safety properties that version control gave [AI code generation](https://metavert.io/ai-code-generation):

- **Reviewable changes.** A revision is a diff ("moved two clips, trimmed one, removed words in one place") that a person can read before it applies. This is the basis of [human-in-the-loop](https://metavert.io/human-in-the-loop) editing and of the [plan-and-apply](https://metavert.io/plan-and-apply) pattern.
- **Reproducibility.** The same spec, media and renderer version produce the same output. Quality checks can then compare the rendered file with the plan instead of taking the agent's word for it (see [automated video QC](https://metavert.io/video-quality-control)).
- **Reversibility.** Revisions can be immutable, with branches for alternate cuts. Nothing is overwritten, so undoing a bad agent edit means pointing at an earlier revision. This is [non-destructive editing](https://metavert.io/non-destructive-editing) taken to its logical end.
- **Concurrency.** Two collaborators, or a person and an agent, can change the same project. A conflict can be detected per clip instead of resolved by last-write-wins.
- **Provenance.** Every frame traces back to a source asset, a template or a generated shot, which is what platform and regulatory disclosure rules increasingly require (see [AI content labeling](https://metavert.io/ai-content-labeling)).

### Video as Code in Practice

[MetaRush](https://metavert.io/metarush) implements this model end to end. It is an AI video creativity and editing harness, in private preview as of October 2026, that people direct through [Claude Code](https://metavert.io/claude-code). Edit specs are written in seconds. A Go engine resolves each one into a frame-accurate render plan that Remotion renders verbatim, so the dashboard preview and the delivered MP4 never disagree. Go structs are the schema's source of truth, and JSON Schema and TypeScript types are generated from them. Revisions are immutable and content-hashed, and branch heads move by compare-and-swap. A plan written against a stale revision is rebased automatically only if nothing it touched has changed; otherwise it is reported as a conflict. Every change is previewed and confirmed by a person before it applies. The same plan exports to OpenTimelineIO, FCPXML and Final Cut Pro 7 XML for finishing in a traditional editor.

Other tools take parts of the approach. JSON-to-video APIs apply it to templated, high-volume video such as personalized ads. Remotion projects apply it to motion graphics and data-driven video (see [motion graphics as code](https://metavert.io/motion-graphics-as-code)). Mainstream editors are moving toward it from the other side by exposing operations to agents through the [Model Context Protocol](https://metavert.io/model-context-protocol), as DaVinci Resolve 21.1 did in September 2026. Their project state, though, still lives in files and databases that were never designed to be diffed.

### Limits and Trade-offs

Video as code is not a replacement for hands-on craft. Rhythm, performance selection, color grading and complex compositing are often faster by direct manipulation, and most professional finishing still happens in a nonlinear editor (see [video as code vs nonlinear editing](https://metavert.io/compare/video-as-code-vs-nonlinear-editing)). Specs are also only as expressive as their schema. Interchange to a traditional editor loses whatever the target format cannot carry, such as reframing or custom graphics. And when parts of the spec are executable components rather than data, the render pipeline needs the same isolation as any system that runs AI-written code (see [agent sandbox](https://metavert.io/agent-sandbox)).

### Where It's Headed

Three trends point the same way. Editors are becoming agent-operable, which raises the value of a reviewable record of what an agent did. Generated shots are entering timelines next to camera footage, which makes per-clip provenance a requirement rather than a nicety. And interchange is maturing, so the likely division of labor is an agent-made first cut, kept as code, that a person finishes in the editor of their choice. The teams that treat their cuts as versioned specifications will be the ones that can let agents edit without losing track of what changed and why.

## Related Topics

- [Docs as Code](https://metavert.io/docs-as-code) — The same discipline applied to prose
- [Agentic Video Editing](https://metavert.io/agentic-video-editing) — AI agents operating an edit through tools
- [Edit Decision Lists and Timeline Interchange](https://metavert.io/edit-decision-list) — How cuts move between editors
- [Motion Graphics as Code](https://metavert.io/motion-graphics-as-code) — Graphics written as components instead of keyframes
- [Non-Destructive Editing](https://metavert.io/non-destructive-editing) — Edits stored as instructions over untouched media
- [Video as Code vs Nonlinear Editing](https://metavert.io/compare/video-as-code-vs-nonlinear-editing) — Spec-and-render versus timeline-and-clicks
- [MetaRush](https://metavert.io/metarush) — Video as code, implemented
- [Agent Harness](https://metavert.io/agent-harness) — The infrastructure that makes long agent tasks reliable

## Further Reading

- [OpenTimelineIO](https://github.com/AcademySoftwareFoundation/OpenTimelineIO) — Academy Software Foundation; the open interchange format for editorial cut information
- [Remotion documentation](https://www.remotion.dev/docs/) — rendering React components to video
- [Remotion Agent Skills](https://www.remotion.dev/docs/ai/skills) — best-practice skills for coding agents working on Remotion projects
- [Docs as Code](https://www.writethedocs.org/guide/docs-as-code/) — Write the Docs; the practice video as code borrows from
- [MetaRush](https://metavert.io/metarush) — video as code, implemented