Extending Montaj

Custom Steps

Any executable that follows the output convention is a valid Montaj step. Language agnostic — Python, bash, Node, or compiled binary.

Creating a Step

1. Scaffold

montaj create-step <name>
# Creates steps/<name>.py and steps/<name>.json in the current directory

2. Write the Script

steps/my_step.py:

#!/usr/bin/env python3
import argparse, os, sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "lib"))
from common import fail, require_file, check_output, run

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--input", required=True)
    parser.add_argument("--out")
    args = parser.parse_args()

    require_file(args.input)
    out = args.out or args.input.replace(".mp4", "_mystep.mp4")

    run(["ffmpeg", "-y", "-i", args.input, ..., out])
    check_output(out)
    print(out)   # stdout = result path

if __name__ == "__main__":
    main()

3. Write the Schema

steps/my_step.json:

{
  "name": "my_step",
  "description": "One sentence: what it does and when to use it.",
  "params": [
    { "name": "input",  "type": "string",  "required": true,  "description": "Source video file" },
    { "name": "out",    "type": "string",  "required": false, "description": "Output path" }
  ]
}

The schema is what the agent and MCP server use to discover and call steps — keep the description agent-readable.

4. Done

The step is automatically available via:

• montaj step my-step --input clip.mp4 (CLI)
• POST /api/steps/my_step (HTTP API)
• MCP tool (if running montaj mcp)
• Workflow builder UI sidebar

No registration, no config changes. Discovered automatically.

Output Contract

• Success → print result path (or JSON) to stdout, exit 0
• Error → print JSON {"error": "code", "message": "..."} to stderr, exit 1
• Never print progress to stdout — use stderr

Step Scopes

Steps can live in three locations:

Resolution order: project-local → user-global → built-in.

Installing to User-Global

montaj step install <path>
# Copies a step into ~/.montaj/steps/ and confirms the prefix to use

Testing

import pytest
from conftest import run_step, assert_file_output

def test_my_step_basic(test_video):
    proc = run_step("my_step.py", "--input", str(test_video))
    assert_file_output(proc)

Validation

montaj validate step steps/my_step.json
# Validates the schema against the step spec

Custom Workflows

Workflows live in workflows/. Copy any bundled workflow, adjust the step sequence and dependencies, and it becomes available immediately.

Creating a Workflow

Via CLI

montaj workflow new my-brand
# Scaffolds workflows/my-brand.json

Via UI

Open the Workflows tab in the browser UI and use the visual node graph to drag, connect, and configure steps.

Manually

Create a JSON file in workflows/:

{
  "name": "my-brand",
  "description": "Custom editing pipeline for brand content.",
  "steps": [
    { "id": "probe",      "uses": "montaj/probe" },
    { "id": "silence",    "uses": "montaj/waveform_trim", "foreach": "clips" },
    { "id": "transcribe", "uses": "montaj/transcribe",    "foreach": "clips", "needs": ["silence"] },
    { "id": "fillers",    "uses": "montaj/rm_fillers",    "foreach": "clips", "needs": ["transcribe"] },
    { "id": "concat",     "uses": "montaj/concat",                           "needs": ["fillers"] },
    { "id": "resize",     "uses": "montaj/resize",                           "needs": ["concat"], "params": { "ratio": "16:9" } }
  ]
}

Step Fields

Mixing Step Scopes

A workflow can reference steps from any scope:

{
  "steps": [
    { "id": "probe",    "uses": "montaj/probe" },
    { "id": "brand-wm", "uses": "user/brand-watermark" },
    { "id": "hook",     "uses": "./steps/viral-hook-detector" }
  ]
}

Using Custom Workflows

montaj run ./clips --workflow my-brand --prompt "clean edit for YouTube"

Or from the UI: select the workflow in the Upload view dropdown.

Validation

montaj validate workflow workflows/my-brand.json

Tips

• Start by forking a bundled workflow that's close to what you need
• Use foreach: "clips" for steps that should run on each clip independently
• Set needs carefully — it defines the dependency graph and parallelism
• Keep params minimal — only override values that differ from step defaults
• The agent will deviate from the workflow when the editing prompt calls for it

Custom Overlays

Overlays are React/JSX components that get rendered frame-by-frame by Puppeteer and composited into the final video. The agent typically writes these, but you can author them manually too.

Component Structure

An overlay is a JSX file that exports a default component. It receives animation globals and renders at 1080 x 1920 design resolution.

Available Globals

interpolate(frame, inputRange, outputRange)

Maps the current frame to an output value. Linear interpolation between keyframes.

const opacity = interpolate(frame, [0, 15, 45, 60], [0, 1, 1, 0]);
// Fades in over 15 frames, holds for 30, fades out over 15

spring({ frame, fps, config })

Physics-based easing for natural-feeling animations.

const scale = spring({
  frame,
  fps,
  config: { mass: 1, stiffness: 100, damping: 10 }
});

Adding to project.json

Overlays live in tracks[1+]:

{
  "tracks": [
    [],
    [
      {
        "id": "ov-hook",
        "type": "overlay",
        "src": "/abs/path/to/overlays/hook.jsx",
        "props": { "text": "She built an AI employee" },
        "start": 0.0,
        "end": 3.0
      }
    ]
  ]
}

Preview

Use the Overlays tab in the UI for live preview:

1. Select a JSX file
2. It compiles and renders at 1080 x 1920 in the browser
3. File watcher auto-recompiles on save
4. Iterate rapidly: edit → save → see result

The browser preview uses @babel/standalone for in-browser transpilation. It is an approximation — the render output (via Puppeteer) is the definitive result.

UI Positioning

In the review phase, users can drag and resize overlays. This sets:

• offsetX / offsetY — position offset as % of frame size
• scale — uniform scale multiplier

These are applied as CSS transforms on the component container by the render engine. The JSX component itself does not need to handle them.

Since v3.2.0, the same offsetX/offsetY/scale fields also drive video-clip reframe — selecting a video clip in the timeline activates drag/resize handles directly on the preview canvas, writing to the same three fields on the clip. The renderer applies them in order: source crop → scale → position offset.

Supported Item Types

Tracks can contain three types:

Embedding the Editor (@bycrux/editor)

The video and carousel editors are published as the @bycrux/editor npm package. A host embeds either editor by supplying a concrete EditorAdapter — an object that resolves every contract verb against the host's own transport (HTTP, RPC, filesystem, etc.).

Montaj's own reference adapter lives at montaj_assets/ui/src/app/editor/montajAdapter.ts.

Required Adapter Methods

These methods must be present for the editor to function:

Optional Video-Editor Methods

These methods unlock specific editor features when provided:

Key <VideoEditor> Props

Contributing

Development Setup

git clone https://github.com/theSamPadilla/montaj
cd montaj

# Python (CLI, steps, server)
python -m venv venv && source venv/bin/activate
pip install -e ".[test]"

# Editor package (@bycrux/editor — host-agnostic video + carousel editors)
cd montaj_assets/editor && npm install

# UI host (consumes editor via a file: link)
cd montaj_assets/ui && npm install

# Render engine
cd montaj_assets/render && npm install

System deps: ffmpeg, ffprobe, whisper.cpp (with at least ggml-base.en.bin).

Running Tests

make test          # run the full suite
make test-fast     # skip slow/ffmpeg-heavy tests
pytest tests/steps/test_probe.py   # single file

Tests require ffmpeg. Whisper-dependent tests use a fake binary fixture — no model download needed.

Adding a Step

1. Write steps/my_step.py — the executable
2. Write steps/my_step.json — the schema
3. Write tests/steps/test_my_step.py — the test
4. Done. Automatically available via CLI, HTTP, and MCP.

Adding a Workflow

Copy workflows/overlays.json, adjust the step sequence and needs dependencies. The file is immediately available.

Adding a Connector

1. Read the vendor's official docs (source of truth)
2. Create connectors/<vendor>.py for a new vendor, or add a function to an existing connector
3. Create steps/<verb>_<noun>.py + .json for the agent-facing step
4. Add CLI command in cli/commands/<verb>_<noun>.py
5. Add unit tests
6. Update the connectors documentation table

Adding a Shared Enum

Shared enums between Python and TypeScript live in schema/enums.yaml.

1. Edit schema/enums.yaml
2. Run codegen: python3 scripts/gen_types.py
3. Commit both the YAML and the generated files

CI runs python3 scripts/gen_types.py && git diff --exit-code and fails if generated files are stale.

Rules

• Never hand-edit files with the GENERATED FROM header
• The codegen is dev-only — not run at install or deploy time
• Generated files are committed artifacts

PR Guidelines

• Keep PRs focused — one feature or fix per PR
• New steps need a test and a schema
• Run make test before opening a PR
• If unsure about a direction, open an issue first

License

MIT — see LICENSE.