Edit Outside the Editor: Why a Timeline Is Just a File an AI Can Write

A timeline looks like something you work in. It is actually something you can write down.

Every cut, every silence you remove, every B-roll clip and sound effect is a decision. Export a timeline from an editor as XML or JSON and you can see it: a list of clips, in-points, out-points, tracks and effects. It is a file full of decisions.

This post is about what follows from that, and where it stops working.

The slow way: driving the editor#

The obvious way to use AI in editing is to make it operate your editor. In the story I read, that went through two stages, and both were slow:

1. An extension for DaVinci Resolve, with the AI going in and editing there.
2. Premiere Pro, where even placing basic B-roll took a long time.

There are structural reasons this is slow, whatever tool you use:

• The editor is a bottleneck. An editor runs one project at a time. While the AI works in it, your computer is busy and so are you.
• The AI is doing a human's job, badly. It operates an interface designed for hands and eyes, one click at a time.
• It does not scale. Ten videos mean ten sessions, in sequence.

The realisation: the screen is just a view#

What you see on screen in an editor is a rendering of data. Underneath is a list of decisions:

• take this clip, from here to here
• cut this silence
• put this B-roll over this line
• add this sound effect at this moment

If that is all an edit is, you do not need the editor to make the decisions. You need it to show them to you.

The workflow#

Instead of operating an editor, give the AI what an editor has in their head, and ask for the result as a file:

1. A transcript of the footage, with timestamps.
2. Video understanding, so it knows what is on screen and where the pauses are.
3. Your editing rules, written down: how tight the cuts are, when to add B-roll, what music and sound effects to use.
4. Access to asset sources, such as a stock library like Pexels or Pixabay, or a generation service like Replicate, for B-roll.
5. A spending budget, so it cannot run up a bill.
6. One instruction: write the edit decision list to disk. Do not touch an editor.

The output is a timeline file. Then:

1. Open it in the editor and review.
2. Fix what is wrong, usually a handful of small changes.
3. Feed the corrections back into your written rules, so the next one is better.

In the story, the first video came out close to right, with a few tweaks needed. After repeating that loop a handful of times and tightening the rules, the output was good enough to run on a batch of 15 videos at once, with each opened in the editor one by one, only to review.

What the file looks like#

There is no single standard for the shape. This is an illustration, not a format any editor reads directly:

{
  "source": "interview-raw.mp4",
  "cuts": [
    { "in": "00:00:02.40", "out": "00:00:11.80" },
    { "in": "00:00:14.10", "out": "00:00:29.50" }
  ],
  "broll": [
    { "at": "00:00:05.00", "duration": 2.5, "file": "clinic-exterior.mp4", "source": "pexels" }
  ],
  "sfx": [
    { "at": "00:00:11.80", "file": "whoosh.wav" }
  ]
}

In practice you would write to a format an editor can import:

Check your editor's current import options before choosing one, since support changes between versions.

Why this can scale#

An editor handles one video at a time. A file does not. Once the decisions live in files:

• There is no editor in the loop. Nothing needs a window open or a screen to click on.
• Work can run in parallel. Many videos can be processed at once, limited by your API limits and budget instead of by one application.
• Your machine stays free. The heavy lifting is not happening in a program you are also trying to use.

The creator's claim is that an agency can multiply output several times over this way, particularly for short-form content, because a client's videos tend to follow one repeatable pattern. I have not measured that, so treat it as their claim, but the logic is sound: repeatable content is exactly where written rules beat manual work.

Where it breaks#

This is the part most write-ups skip.

• Review is not optional. The AI will place B-roll that does not fit and cut a word you wanted. A human has to look at every video, and that time is real. Fifteen videos still means fifteen reviews.
• It only suits patterned content. A client's weekly short-form videos are a good fit. A film with creative decisions in every scene is not.
• Your rules are the product. The quality comes from the instruction set you refine over repeated rounds. A vague set gives vague edits, quickly and in bulk.
• Spending needs a hard stop. Handing an AI API keys and a budget is only safe with limits you do not control from inside the prompt. Use a separate key, set a spending cap at the provider, and make the pipeline stop when it is reached. I built the same kind of budget counter in a different project, for exactly this reason.
• Asset licences matter. Check the licence of every stock or generated clip before it goes into client work.
• A bad rule multiplies. A mistake in your instructions is repeated across the whole batch, so test on one video first.

How I would try it#

I have not built this, so here is how I would test it before trusting it:

1. Pick one client whose videos follow a clear pattern, and write down the rules a human editor already follows.
2. Run the pipeline on one video and time the review against a normal edit.
3. Tighten the rules and repeat until the corrections are small.
4. Only then run a batch, with a budget cap, and review every output.
5. Compare total time, including review, with doing it by hand.

If the review time does not fall, the workflow is not worth it for that kind of content, and you should find out on video one rather than video fifteen.

The takeaway#

The useful idea is not a tool. It is that a timeline is data. Once you accept that, the question changes from "how do I get AI to use my editor?" to "which of my decisions can I write down?" The editor then becomes the place where you check the work.

For the editing side, see 10 video editing tips for reels that hold attention, and for working with Claude through tools, connecting Claude to a content calendar. If you need video editing done, see the video editing portfolio.