UI Guide

The Montaj UI is an optional browser-based interface that wraps the entire pipeline. It does not replace the CLI: every action in the UI maps to a CLI command. The full pipeline works headlessly without it.

montaj serve   # starts local server + opens http://localhost:3000

Overview

The Modes

For video projects:

1. Upload: drop clips, write a prompt, select a workflow, hit Run
2. Live View: watch the agent build the edit in real time via SSE
3. Review: adjust the timeline, captions, and overlays when the agent finishes
4. Render: trigger the final render to MP4

For carousel projects:

1. Intake: name + prompt + aspect ratio + drop reference assets
2. Pending screen: copy a one-line message to your agent; watch its log lines stream as it builds slides
3. Canvas editor: slide grid, drag-to-reposition with snap guides, double-click overlays to edit text
4. Render modal: full-screen gallery of every PNG with a download-as-zip button

Tabs

The UI has four top-level tabs:

How montaj serve Works

montaj serve is a thin local HTTP + SSE server: the bridge between the browser and the filesystem.

montaj serve
  ├── POST /api/run              → receives clips + prompt + workflow, starts pipeline
  ├── GET  /api/projects         → list projects and their status
  ├── GET  /api/projects/:id/stream  → SSE stream of project.json changes
  └── file watcher               → watches workspace/ for project.json writes → SSE

The agent polls serve, serve does not notify the agent. The agent writes directly to disk. montaj serve watches. Every write immediately pushes to the browser.

Key Design Decisions

• montaj serve is thin: no business logic, just file watching, SSE, and process spawning
• Filesystem is the source of truth: agent writes project.json to disk, serve watches, browser reflects
• No Puppeteer in the browser: the preview plays back through the editor's own canvas engine (falling back to native <video> when it can't) with CSS-positioned overlays; frame-by-frame Puppeteer rendering happens only in the render pipeline
• project.json is the only state: edits apply optimistically in the browser and are written back to disk through a queued save, so there is nothing to remember to save
• Every UI action has a CLI equivalent: the UI is a layer on top of the CLI

Tech Stack

The UI is built with Vite + React + Tailwind CSS. Source code is split across two packages in the Montaj repository.

montaj_assets/editor/ (@bycrux/editor): host-agnostic video and carousel editors. Contains the timeline, preview player, cut engine, version history panel, render modal, and VideoSourceCropModal. Has no knowledge of the Montaj HTTP transport; consumers supply a concrete EditorAdapter.

montaj_assets/editor/
  src/
    video/                      # Video editor (timeline, preview, cut engine)
    carousel/                   # Carousel editor (slide canvas, property panel)
    crop/
      VideoSourceCropModal.tsx  # Source-crop UI
    state/                      # Editor state management
    lib/                        # Shared utilities

montaj_assets/ui/: the Montaj host shell. Wires @bycrux/editor to the Montaj HTTP transport via montajAdapter.ts and provides upload/live-view, project list, and the workflow NodeGraph.

montaj_assets/ui/
  src/
    app/
      editor/
        EditorPage.tsx          # Editor tab wiring
        montajAdapter.ts        # Concrete EditorAdapter for Montaj
    components/
      NodeGraph.tsx             # Workflow builder
    lib/                        # Host utilities (SSE, project I/O)

Upload, Live View & Review

The Editor tab follows a three-phase flow: upload your clips, watch the agent work in real time, then review and adjust the result.

Upload

Drop clips, write a prompt, select a workflow, hit Run.

• Drag-and-drop clip upload (or file picker)
• Free-form prompt textarea: "tight cuts, remove filler, 9:16 for Reels"
• Workflow selector: choose from available workflows (native + custom)
• Run → POST /api/run to montaj serve → pipeline starts immediately

The upload form adapts to the selected workflow's project type. broll shows three zones instead of one, Footage (raw video the agent selects shots from), Assets (optional images and screenshots usable as shots), and Voiceover (a single audio or video file, with only its audio used). The submit button reads "Assemble b-roll". See B-Roll.

Live View

As the agent works, the UI updates in real time.

• montaj serve watches project.json for any file change
• Every write the agent makes: trim points added, clips reordered, captions cleaned, pushes to the browser via SSE
• Timeline rerenders on each update
• Preview player reflects the current state of the edit
• You watch the edit take shape as the agent builds it

Review

When the agent marks the project draft, the UI surfaces it for human adjustment.

Available Controls

• Full timeline with clip, caption, and overlay tracks
• Preview player: canvas-rendered playback (falling back to native <video> when needed) with CSS-positioned overlays
• Captions render live in the preview player during playback, z-indexed above the video layer, styled to match the final render (word-by-word, karaoke, pop, subtitle, highlight-box, outline, or clean) so you see the caption style as the video plays
• Caption editor: click to edit text inline, drag to retime
• Overlay editor: add/remove/reposition title cards, lower thirds
• Prompt bar: modify the prompt and re-run the agent

Skipping Review

Review is optional, click Render directly from live view if the first pass is good enough.

Render

Triggers the render pass. Progress streams back via SSE. Final MP4 lands in the project workspace directory.

Preview vs. Render

The preview and the render pipeline share one timeline-resolution package for timing, ordering, source selection, and geometry, so what plays in the browser and what renders agree on those specifics. Overlays and captions are the exception: the browser still draws them as CSS-positioned elements while the render still rasterizes them through a separate Puppeteer pipeline, so the two are not guaranteed to match pixel-for-pixel. See Editor for the full picture.

Carousel Editor

Carousel projects use a dedicated editor. There's no timeline, no live view in the video sense. The flow is:

Intake (/projects/new)

The carousel form takes name, profile, prompt (required), aspect ratio, and a drop zone for reference assets. Assets dropped here are uploaded to the workspace at submit and surface in project.assets[]. Hit "Create carousel" and you land on the pending screen.

Pending Screen

The center of the editing area shows:

• A bold "Message your agent to start" headline
• A subtitle: "Nothing will happen automatically. Copy this and send it to your agent."
• A blue-bordered card with a copy-to-clipboard button. The text resolves to "There is a new project pending: "<name>". Please see @<root-skill-path> and start. Talk to me if you run into questions."

Once the agent starts working and emits log lines via POST /api/projects/{id}/log, the same area swaps to a live status readout: spinner + "N assets attached. Agent is working:" + the latest log line in a blue mono pill. The right rail keeps showing the asset library throughout.

When the agent flips status off pending, the canvas takes over.

Canvas

[Slide grid]  |  [Canvas + hint]  |  [Property panel]
                                    [Asset library]

• Slide grid: drag to reorder, hover for duplicate / delete, click to select.
• Canvas: direct manipulation of the selected slide. Drag elements to move, resize from the eight handles, rotate from the handle above the element.
• Snap guides: pink lines appear when an element gets within 2.5% of the slide's center axes or any of the four edges. Rotation snaps to 0°/90°/180°/270° within ±5°.
• In-place text editing: double-click an overlay with a text prop to edit it directly on the canvas; commit with Cmd+Enter, cancel with Esc.
• Property panel: exact x/y/w/h/rotation fields, base color picker for the slide, per-prop editors for overlay props.
• Refresh button: top-left of the canvas area. Re-fetches project.json from disk if you bypassed SSE.
• Render button: top-right. Flips status to final, opens the render modal.

A small line below the canvas: "Drag elements to reposition. Ask the agent for any other changes." The canvas is for polish; structural edits go through the agent.

Render Modal (Carousel)

Streams the carousel renderer's log lines while running. On completion, opens a full-screen overlay:

• Left: a grid of every slide as a clickable thumbnail (each opens the full-res PNG in a new tab)
• Right: "Render complete · N slides ready", the absolute output dir, Download all (.zip) button, Close

The zip endpoint (GET /api/projects/{id}/render-zip) bundles every PNG as <project-name>-slides.zip and excludes manifest.json from the archive (it stays on disk for tooling).

Timeline and Caption Editing

Timeline editing, clip properties, caption editing, and overlay editing, trimming and cuts, reframe, source crop, multi-select, undo/redo, version history, and export: all now live on their own page. See Editor for the full reference.

Workflow Builder

The Workflows tab provides a visual node graph UI for building and editing workflows. Inspired by n8n, it lets you construct editing pipelines visually.

Interface

Sidebar                    Canvas
─────────────────────────────────────────────────────
Native steps:              ┌──────────┐
  probe                    │  probe   ├──► ┌─────────────┐
  rm_fillers               └──────────┘    │  rm_fillers │
  waveform_trim                            └──────┬──────┘
  transcribe                                      │
  jump_cut                               click to configure:
  crop_spec                              sensitivity: [====|  ] 0.8
  resize                                 words: [um, uh, hmm]  + add
  caption
  ...

Custom steps:
  viral-hook-detector
  b-roll-inserter
  + New step

Building a Workflow

1. Drag steps from the sidebar onto the canvas
2. Connect nodes to define data flow (edges = needs dependencies)
3. Click a node to configure its params, controls are rendered from the step's JSON schema
4. Invalid connections (type mismatch) are rejected visually
5. Save → writes workflows/<name>.json to disk
6. Run → executes the workflow against the current clips

Step Discovery

The sidebar shows all available steps across all three scopes:

• Native steps: built into Montaj
• User-global steps: from ~/.montaj/steps/
• Project-local steps: from ./steps/ in the current project

Custom steps appear automatically: no registration needed. Adding steps/my-step.py + steps/my-step.json makes it appear in the sidebar.

Node Configuration

Clicking a node opens its configuration panel. The controls are dynamically generated from the step's JSON schema:

• String params → text inputs
• Number params → number inputs or sliders
• Enum params → dropdowns
• Boolean params → toggles

Default values from the workflow file are pre-populated. Changes here update the params field in the workflow JSON.

CLI Equivalent

montaj workflow list              # list all available workflows
montaj workflow new <name>        # scaffold a new workflow file
montaj workflow edit <name>       # open in the node graph UI
montaj workflow run <name> ./clips --prompt "..."

Output Format

The node graph saves directly to the standard workflow JSON format:

{
  "name": "my-workflow",
  "description": "Custom editing pipeline",
  "steps": [
    { "id": "probe", "uses": "montaj/probe" },
    { "id": "silence", "uses": "montaj/waveform_trim", "foreach": "clips" },
    { "id": "transcribe", "uses": "montaj/transcribe", "needs": ["silence"] }
  ]
}

The saved file is immediately available to montaj run --workflow <name>.

Overlay Preview

The Overlays tab provides a live preview environment for custom JSX overlay components.

How It Works

1. Select any overlay JSX file from the current project or global overlays
2. The overlay is compiled and rendered at 1080 x 1920 (design resolution), scaled to fit the viewport
3. A file watcher via SSE recompiles and rerenders automatically on every save
4. Compile errors are displayed inline

Preview Pipeline

When montaj serve is running, the UI previews overlays and captions live in the browser via ui/src/lib/overlay-eval.ts:

1. The JSX file is fetched from the filesystem
2. Transpiled in-browser by @babel/standalone
3. Called directly on every animation frame

This is an approximation: font rendering and CSS compositing differ slightly from the Puppeteer render environment. The render output is what matters.

Real-Time Iteration

The file watcher makes this a rapid iteration loop:

Edit JSX in your editor → save → UI recompiles → preview updates instantly

No manual refresh needed. The SSE connection detects the file change and triggers recompilation.

Component Globals

Overlay components have access to these globals at render time:

• frame: current frame number
• fps: frames per second (from project settings)
• props: arbitrary data from the overlay item in project.json
• interpolate(frame, inputRange, outputRange): map frame number to any value
• spring({ frame, fps, config }): physics-based easing (mass, stiffness, damping)

Design Resolution

Overlays are always designed and rendered at 1080 x 1920 regardless of the output resolution. The render pipeline upscales at compose time (e.g., 2x for 4K output at 2160 x 3840).