Overlays

Overlays sit on top of a scene: who is speaking, what is said, where in the video we are. They take their times from the container they are shown in (ui.seconds() and ui.duration), so the same lower third works in any scene or Sequence, and the ones that leave do so before the container ends.

Component	Props	What it does
Appear	Appear::new(ui, from, center, children), then until, rise (60), scale_from (0.92), enter, exit, rate_func	children rise, grow about center and fade in at from seconds, and leave upward at until (or the end of the container); inside, time starts at from
LowerThird	name, role, x, y, name_style, role_style, name_fill, role_fill, bar, bar_width (6), background: Option<Color>, min_width, padding, delay, stagger, run_time, exit, rate_func	an accent bar grows, then the name and the role slide out from behind it; all of it leaves in reverse order before the end. With background, a plate as wide as the longer line
Captions	file, x, y, width, style, fill, align, video_time (true)	the cue of a WebVTT file that plays now, wrapped into width; cue times are video seconds, or container seconds with video_time: false
WordCaptions	pages: Vec<TikTokPage>, x, y, width, max_lines (2), style (title), fill, highlight (accent), pop (1.12), fade (0.08 s), uppercase, pill: Option<Color>, pill_opacity, padding, entrance (PageEntrance::Pop, Fade, Rise, None), enter, exit	TikTok-style captions: the page that plays now, centred on (x, y), the word being said in highlight and grown by pop, its neighbours moved aside; a page that needs more than max_lines lines shrinks. Times are milliseconds of the container
CaptionStack	items: Vec<Caption> (Caption::new(at, tag, line)), x, y, gap, tag_style, line_style, tag_fill, line_fill, tag_spacing, rise, enter, leave, rate_func	timed captions without a file: each rises in at its at and leaves upward while the next comes in; the last stays
ChapterList	chapters: Vec<(f32, Cow<str>)>, x, y, width, row_height, gap, style, fill, active_fill, highlight, radius, spring	an agenda whose highlight springs to the chapter playing now
Annotation	target, label, text, color, style, letter_spacing, stroke_width, delay, run_time, rate_func	a dot on target, an elbow line to label and the text at its end; the line draws on, the text fades in
ProgressBar	x, y, width, height (8), radius, track, fill, progress: Option<f32>	how far the container is (None), or a given value from 0 to 1
Check	x, y, size, color, stroke_width (5), delay, run_time, rate_func	a check mark that draws itself on

Gallery scenes: Appear, LowerThird, Captions, CaptionStack, ChapterList, Annotation, ProgressBar, Check.

Appear 2.5 sPlayUnmuteCopy render command

LowerThird 3 sPlayUnmuteCopy render command

Captions 3 sPlayUnmuteCopy render command

CaptionStack 3 sPlayUnmuteCopy render command

ChapterList 3 sPlayUnmuteCopy render command

Annotation 2 sPlayUnmuteCopy render command

ProgressBar 2 sPlayUnmuteCopy render command

Check 2 sPlayUnmuteCopy render command

Annotation is the HUD callout with a line; Callout (under Shapes) is the speech bubble. Captions has no glow of its own: wrap it in a <g filter=..> with the Glow decoration.

use fframes_kit::prelude::*;

fn annotations<'a>(ui: &mut Ui<'a, '_, '_>) -> Svgr<'a> {
    let appear = Appear::new(ui, 0.5, (960.0, 540.0), children(|ui| {
        ui.show(Check { x: 960.0, y: 540.0, size: 80.0, ..ui.defaults() })
    }));
    let appear = ui.show(appear);
    let note = ui.show(Annotation { target: (960.0, 540.0), label: (1200.0, 400.0), text: "Ready".into(), ..ui.defaults() });
    let captions = ui.show(CaptionStack {
        items: vec![Caption::new(0.0, "01", "First"), Caption::new(2.0, "02", "Next")],
        x: 200.0, y: 800.0, ..ui.defaults()
    });
    svgr!(<g>{appear}{note}{captions}</g>)
}

A speaker introduced over the first four seconds of a scene, with captions from a file under it and the video’s progress at the bottom:

use fframes_kit::prelude::*;

fn speaker<'a>(ui: &mut Ui<'a, '_, '_>) -> Svgr<'a> {
    let intro = Sequence { from: 0.5, duration: Some(4.0), children: children(|ui| {
        ui.show(LowerThird { name: "Ada Lovelace".into(), role: "Analyst".into(), ..ui.defaults() })
    }) };
    let intro = ui.show(intro);
    let captions = ui.show(Captions { file: "talk.vtt", y: 980.0, ..ui.defaults() });
    let progress = ui.show(ProgressBar { y: 1064.0, height: 4.0, ..ui.defaults() });
    svgr!(<g>{intro}{captions}{progress}</g>)
}

let list = ChapterList {
    chapters: vec![(0.0, "Intro".into()), (5.0, "Demo".into())],
    ..ChapterList::default()
};
assert_eq!(list.active(6.0), 1);

Word captions and voiceover timing

Word timings are data you bring. Nothing here speaks, records or recognises speech.

• Caption files.
  • CaptionWord { text, start_ms, end_ms, timestamp_ms, confidence } is Remotion’s Caption.
  • text::parse_srt and text::parse_vtt read the cues of a subtitle file: one CaptionWord each, trimmed, with confidence 1.
  • text::serialize_srt(&lines) writes one cue per line of words.
  • text::from_whisper_json reads the words of openai-whisper (segments[].words[] or words[], in seconds) and of whisper.cpp (transcription[] with offsets in milliseconds; run it with -ml 1 for one word per entry). Transcript words keep their leading space: " world".
• Pages. overlay::tiktok_pages(&words, combine_within_ms) is createTikTokStyleCaptions. A word that starts with a space starts a new page once the page has run for combine_within_ms. Subtitle cues have no leading space, so their pages break only between voiceover blocks.
• Voiceover. VoiceoverScript is manim-voiceover’s timing.
  • A block is VoiceoverBlock::new(text, audio, words): a recorded file and its words.
  • <bookmark mark="A"/> in the text marks a moment. It is timed at the first word after it, counted in letters, so a transcript that splits words differently still lines up.
  • VoiceoverScript::in_sequence(blocks, start, gap) plays the blocks one after the other; VoiceoverScript::new plays them at their own start.
  • It gives:
    • block(i): start, duration and end;
    • bookmark("A"), time_until_bookmark("A", t) and remaining(i, t);
    • subcaptions(): the words in container time, for tiktok_pages;
    • audio_tracks(), for fn audio;
    • sequence(i, children): a Sequence as long as the block. For a Series, give an item block(i).duration.

Times are seconds of the container the script is used in (milliseconds for the words). Give the video’s fn audio the tracks of a script that runs on video time. Gallery scenes: WordCaptions and Voiceover.

WordCaptions 3.5 sPlayUnmuteCopy render command

Voiceover 3.5 sPlayUnmuteCopy render command

use fframes_kit::{overlay::tiktok_pages, prelude::*, text::from_whisper_json};

/// Narration with a dot that appears at its bookmark and the words under it.
struct Narrated {
    script: VoiceoverScript<'static>,
    pages: Vec<TikTokPage>,
}

impl Narrated {
    fn new(transcript: &str) -> Self {
        let words = from_whisper_json(transcript).unwrap_or_default();
        let block = VoiceoverBlock::new("Here is <bookmark mark=\"dot\"/>a dot.", "voice.wav", words);
        let script = VoiceoverScript::in_sequence(vec![block], 0.5, 0.0);
        let pages = tiktok_pages(&script.subcaptions(), 1200.0);
        Self { script, pages }
    }

    fn draw<'a>(&self, ui: &mut Ui<'a, '_, '_>) -> Svgr<'a> {
        let at = self.script.bookmark("dot").unwrap_or(0.0);
        let opacity = motion::progress(ui.seconds(), at, at + 0.3);
        let captions = ui.show(WordCaptions { pages: self.pages.clone(), ..ui.defaults() });
        svgr!(<g><circle cx="960" cy="480" r="80" fill="#4f8cff" opacity={opacity} />{captions}</g>)
    }

    fn audio(&self) -> AudioMap<'static> {
        AudioMap::from(self.script.audio_tracks())
    }
}

let narrated = Narrated::new(
    r#"{"words": [{"word": " Here", "start": 0.0, "end": 0.3}, {"word": " is", "start": 0.3, "end": 0.45},
                  {"word": " a", "start": 0.6, "end": 0.7}, {"word": " dot.", "start": 0.7, "end": 1.1}]}"#,
);
assert_eq!(narrated.script.bookmark("dot"), Some(1.1));
assert_eq!(narrated.pages.len(), 1);
let _ = narrated.audio();

API

Components are plain structs. Set the fields you need and fill the rest with ..ui.defaults(): the defaults below come from the current theme (theme. values are fields of Theme).

Appear

Shows its children from from seconds on: they rise rise units into place, grow from scale_from about center and fade in over enter seconds. They leave the same way up over exit seconds, ending at until seconds, or at the end of the container (ui.duration) when until is None. Inside, time starts at from, as in a Sequence.

Field	Type	Default	Description
from	f32	—	Seconds after the container’s start when the children appear; inside, their time starts at from.
until	Option<f32>	—	Seconds when the children are gone, after leaving over exit; None ends with the container.
center	(f32, f32)	—	The point the content scales about, in canvas units.
rise	f32	—	Pixels the children rise into place as they enter, and sink as they leave.
scale_from	f32	—	Scale the children grow from as they enter, 1 for no growth.
enter	f32	—	Seconds the children take to fade, rise and grow in.
exit	f32	—	Seconds the children take to leave before until or the end of the container; 0 cuts them off.
rate_func	RateFunc	—	Easing of the entrance over enter seconds.
children	Children	—	What it shows (a closure taking the Ui).

Methods:

• fn new(ui: &Ui, from: f32, center: (f32, f32), children: Children) -> Self: Times and motion from the theme: motion.duration.enter and exit, a 60 unit rise, scaling from 0.92.
• fn state(&self, t: f32, end: Option<f32>) -> (f32, f32): (entered, left), both from 0 to 1, at t seconds of a container that ends at end.

Source

LowerThird

A name and a role next to an accent bar, the caption of a speaker. The bar grows first, then the name and the role slide in from behind it, stagger seconds apart, and all of it leaves in reverse order before the end of the container. (x, y) is the top left of the bar. With background, a plate as wide as the longer line sits behind the text (without a measurable font it is min_width wide).

Field	Type	Default	Description
name	Cow<str>	Cow::Borrowed("")	The speaker’s name, the larger first line.
role	Cow<str>	Cow::Borrowed("")	The role or subtitle under the name; empty for none.
x	f32	96.0	Left edge of the bar, in pixels of the canvas.
y	f32	820.0	Top of the bar, in pixels of the canvas.
name_style	TextStyle	theme.typography.heading	The font of the name: family, size in pixels and weight.
role_style	TextStyle	theme.typography.body	The font of the role: family, size in pixels and weight.
name_fill	Color	theme.colors.title	Colour of the name.
role_fill	Color	theme.colors.muted	Colour of the role.
bar	Color	theme.colors.accent	Colour of the accent bar.
bar_width	f32	6.0	Width of the accent bar in pixels.
background	Option<Color>	None	Colour of the plate behind the text; None for no plate.
min_width	f32	400.0	Least width in pixels of the text area on the plate, and its width when the font cannot be measured.
padding	f32	theme.spacing.md	Space between the bar and the text, and around the text on the plate.
delay	f32	0.0	Seconds after the container’s start before the bar begins to grow.
stagger	f32	theme.motion.stagger.loose	Seconds between the starts of the bar, the name and the role.
run_time	f32	theme.motion.duration.enter	Seconds each part takes to come in.
exit	f32	theme.motion.duration.exit	Seconds of the exit, which ends with the container; 0 never leaves.
rate_func	RateFunc	theme.motion.easing.enter	Easing of each part coming in.

Methods:

• fn height(&self) -> f32: Height of the bar: the two lines and the padding around them.

Source

Captions

The cue of a subtitle (.vtt) file that plays now, wrapped into width pixels. Cue times are seconds of the video, or of the container with video_time: false. A missing file draws nothing (and inspect reports it).

Field	Type	Default	Description
file	&str	""	A subtitle file of the video’s media.
x	f32	160.0	Left edge of the caption box, in pixels of the canvas.
y	f32	900.0	Baseline of the first line, in pixels of the canvas; later lines go below.
width	f32	1600.0	Width in pixels the cue wraps into; it is aligned inside this width.
style	TextStyle	theme.typography.heading	The font: family, size in pixels and weight.
fill	Color	theme.colors.caption	Colour of the text.
align	TextAlign	TextAlign::Center	How the lines sit inside width: left, center or right.
video_time	bool	true	true reads cue times as seconds of the whole video; false as seconds of the container.

Source

WordCaptions

TikTok-style captions: the page of pages that plays now, centred on (x, y) and wrapped into width pixels, with the word being said in highlight, scaled by pop. Times are milliseconds of the container. A page that needs more than max_lines lines is shrunk until it fits. Between pages and after the last one nothing is drawn.

Field	Type	Default	Description
pages	Vec<TikTokPage>	Vec::new()	The pages of words, each with its own timing, as made by tiktok_pages; one shows at a time.
x	f32	960.0	Horizontal center of the caption, in pixels of the canvas.
y	f32	860.0	Vertical center of the caption block, in pixels of the canvas.
width	f32	1400.0	Width in pixels the words wrap into.
max_lines	usize	2	Most lines a page may take; a longer page is shrunk until it fits.
style	TextStyle	theme.typography.title	The font: family, size in pixels and weight.
fill	Color	theme.colors.caption	Colour of the words that are not being said.
highlight	Color	theme.colors.accent	Colour of the word being said.
pop	f32	1.12	The size of the word being said, 1 for none.
fade	f32	0.08	Seconds a word takes to light up and to fade back.
uppercase	bool	false	true writes every word in capitals.
pill	Option<Color>	None	A rounded box behind the page, in this colour.
pill_opacity	f32	0.75	Opacity of the pill, 0 transparent to 1 opaque.
padding	f32	24.0	Space between the text and the edge of the pill.
entrance	PageEntrance	PageEntrance::Pop	How a page comes in over enter seconds.
enter	f32	0.15	Seconds of the entrance.
exit	f32	0.1	Seconds the page takes to fade out before the next one starts, or the last one ends.

Source

CaptionStack

Timed captions without a subtitle file: each Caption rises rise units into place at its at seconds and leaves upward while the next one comes in. The last stays. (x, y) is the baseline of the tag; the line is gap pixels below it.

Field	Type	Default	Description
items	Vec<Caption>	Vec::new()	The cards, in order of at; each shows from its at seconds.
x	f32	96.0	Left edge of the tag and line, in pixels of the canvas.
y	f32	860.0	Baseline of the tag in pixels of the canvas; the line sits gap below.
gap	f32	88.0	Pixels from the tag’s baseline to the line’s baseline.
tag_style	TextStyle	theme.typography.caption	The font of the small tag: family, size in pixels and weight.
line_style	TextStyle	theme.typography.heading	The font of the line: family, size in pixels and weight.
tag_fill	Color	theme.colors.accent	Colour of the tag.
line_fill	Color	theme.colors.title	Colour of the line.
tag_spacing	f32	4.0	Extra space between letters of the tag, in pixels.
rise	f32	40.0	Pixels a card rises into place as it enters (and about three quarters of that as it leaves).
enter	f32	theme.motion.duration.enter	Seconds a card takes to fade and rise in.
leave	f32	theme.motion.duration.exit	Seconds before the next caption’s at that this one starts to leave.
rate_func	RateFunc	theme.motion.easing.enter	Easing of a card rising into place.

Source

ChapterList

A list of chapters with a highlight that springs to the chapter playing now (an agenda, the steps of a tutorial). Each chapter is (start seconds, title), in order; before the first start the highlight is on the first row. (x, y) is the top left of the list.

Field	Type	Default	Description
chapters	Vec<(f32, Cow<str>)>	Vec::new()	Each chapter as (start in seconds, title), in order; the highlight moves to a chapter when it starts.
x	f32	96.0	Left edge of the list, in pixels of the canvas.
y	f32	240.0	Top of the first row, in pixels of the canvas.
width	f32	720.0	Width of the highlight bar in pixels.
row_height	f32	72.0	Height of one row, and of the highlight, in pixels.
gap	f32	12.0	Pixels between two rows.
style	TextStyle	theme.typography.body	The font: family, size in pixels and weight.
fill	Color	theme.colors.muted	Colour of the titles that are not playing.
active_fill	Color	theme.colors.title	Colour of the title of the chapter playing now.
highlight	Color	theme.colors.surface	Colour of the bar behind the current chapter.
radius	f32	theme.radius.md	Corner radius of the highlight bar in pixels.
spring	(f32, f32)	(130.0 / 0.45, 20.0 / 0.45)	(stiffness, damping) of the highlight’s mass-1 spring.

Methods:

• fn active(&self, t: f32) -> usize: The chapter playing at t.
• fn highlight_row(&self, t: f32) -> f32: The top of the highlight at t, in rows from the first (between two rows while it moves).

Source

Annotation

An annotation: a dot on the target, an elbow line to label (first across, then up or down) and the text at its end. The line draws on over run_time seconds from delay; the text fades in over the last 40 %. (The shape called Callout is the Remotion speech bubble, so this one has another name.)

Field	Type	Default	Description
target	(f32, f32)	(0.0, 0.0)	The point the dot marks, in pixels of the canvas.
label	(f32, f32)	(0.0, 0.0)	Where the line ends and the text sits, in pixels of the canvas; the text goes right of it when it is right of target, else left.
text	Cow<str>	Cow::Borrowed("")	The words at the end of the line.
color	Color	theme.colors.accent	Colour of the dot, line and text.
style	TextStyle	theme.typography.caption	The font: family, size in pixels and weight.
letter_spacing	f32	2.0	Extra space between letters of the text, in pixels.
stroke_width	f32	theme.stroke.thin	Width of the line and the ring around the dot, in pixels.
delay	f32	0.0	Seconds after the container’s start before it begins.
run_time	f32	theme.motion.duration.slow	Seconds the line takes to draw on; the text fades in over the last 40 %.
rate_func	RateFunc	theme.motion.easing.standard	Easing of the drawing over its run time.

Methods:

• fn draw(self, progress: f32) -> Svgr: The annotation progress (0 to 1) of the way drawn.

Source

ProgressBar

A bar that fills from the left: with progress: None it shows how far the container is through ui.duration (a video progress bar), otherwise the given value from 0 to 1. The filled part is never thinner than half a pixel, so it is visible from the start.

Field	Type	Default	Description
x	f32	96.0	Left edge of the bar in pixels of the canvas.
y	f32	1000.0	Top of the bar in pixels of the canvas.
width	f32	1728.0	Full width of the track in pixels.
height	f32	8.0	Height of the bar in pixels.
radius	f32	0.0	Corner radius of the track and the filled part in pixels; 0 for square corners.
track	Color	theme.colors.surface	Colour of the unfilled track behind the bar.
fill	Color	theme.colors.accent	Colour of the filled part.
progress	Option<f32>	None	How full the bar is, 0 to 1; None follows how far the container is through ui.duration.

Source

Check

A check mark that draws itself on from delay seconds over run_time. (x, y) is the start of the short stroke; the mark is 27 by 20 units times size.

Field	Type	Default	Description
x	f32	0.0	Start of the short stroke, in pixels of the canvas.
y	f32	0.0	Start of the short stroke, in pixels of the canvas.
size	f32	1.0	Scale of the mark, which is 27 by 20 pixels at 1.
color	Color	theme.colors.accent	Colour of the mark.
stroke_width	f32	5.0	Width of the mark’s line in pixels.
delay	f32	0.0	Seconds after the container’s start before it begins.
run_time	f32	theme.motion.duration.base	Seconds the animation takes.
rate_func	RateFunc	theme.motion.easing.standard	Easing of the drawing over its run time.

Source