Complete Workflow: Script to Synchronized Scene ```
VOICEOVER-SCRIPT.md β voiceover.py β public/audio/ β Remotion composition
β β β β
Scene narration Generate MP3 Audio files component
with durations per scene with timing synced to scenes
```
Step 1: Generate Per-Scene Audio Use the toolkit's voiceover tool to generate audio for each scene:
```bash
# Generate voiceover files for each scene
python tools/voiceover.py --scene-dir public/audio/scenes --json
# Output:
# public/audio/scenes/
# βββ scene-01-title.mp3
# βββ scene-02-problem.mp3
# βββ scene-03-solution.mp3
# βββ manifest.json (durations for each file)
```
The manifest.json contains timing info:
```json
{
"scenes": [
{ "file": "scene-01-title.mp3", "duration": 4.2 },
{ "file": "scene-02-problem.mp3", "duration": 12.8 },
{ "file": "scene-03-solution.mp3", "duration": 15.3 }
],
"totalDuration": 32.3
}
```
Step 2: Use Audio in Remotion Composition ```tsx
// src/Composition.tsx
import { Audio, staticFile, Series, useVideoConfig } from 'remotion';
// Import scene components
import { TitleSlide } from './scenes/TitleSlide';
import { ProblemSlide } from './scenes/ProblemSlide';
import { SolutionSlide } from './scenes/SolutionSlide';
// Scene durations (from manifest.json, converted to frames at 30fps)
const SCENE_DURATIONS = {
title: Math.ceil(4.2 * 30), // 126 frames
problem: Math.ceil(12.8 * 30), // 384 frames
solution: Math.ceil(15.3 * 30), // 459 frames
};
export const MainComposition: React.FC = () => {
return (
<>
{/ Scene sequence /}
{/ Audio track - plays continuously across all scenes /}
{/ Optional: Background music at lower volume /}
);
};
```
Step 3: Per-Scene Audio (Alternative) For more control, add audio to each scene individually:
```tsx
// src/scenes/ProblemSlide.tsx
import { Audio, staticFile, useCurrentFrame } from 'remotion';
export const ProblemSlide: React.FC = () => {
const frame = useCurrentFrame();
return (
slide styles / }}>
The Problem
{/ Scene content /}
{/ Audio starts when this scene starts (frame 0 of this sequence) /}
);
};
```
Syncing Visuals to Voiceover Calculate scene duration from audio, not the other way around:
```tsx
// src/config/timing.ts
import manifest from '../../public/audio/scenes/manifest.json';
const FPS = 30;
// Convert audio durations to frame counts
export const sceneDurations = manifest.scenes.reduce((acc, scene) => {
const name = scene.file.replace(/^scene-\d+-/, '').replace('.mp3', '');
acc[name] = Math.ceil(scene.duration * FPS);
return acc;
}, {} as Record);
// Usage in composition:
//
```
Audio Timing Patterns ```tsx
import { Audio, Sequence, interpolate, useCurrentFrame } from 'remotion';
// Fade in audio
export const FadeInAudio: React.FC<{ src: string; fadeFrames?: number }> = ({
src,
fadeFrames = 30
}) => {
const frame = useCurrentFrame();
const volume = interpolate(frame, [0, fadeFrames], [0, 1], {
extrapolateRight: 'clamp',
});
return
};
// Delayed audio start
export const DelayedAudio: React.FC<{ src: string; delayFrames: number }> = ({
src,
delayFrames
}) => (
);
// Usage:
//
//
```
Voiceover + Demo Video Sync When a scene has both voiceover and demo video:
```tsx
import { Audio, OffthreadVideo, staticFile, useVideoConfig } from 'remotion';
export const DemoScene: React.FC = () => {
const { durationInFrames, fps } = useVideoConfig();
// Calculate playback rate to fit demo into voiceover duration
const demoDuration = 45; // seconds (original demo length)
const sceneDuration = durationInFrames / fps; // seconds (from voiceover)
const playbackRate = demoDuration / sceneDuration;
return (
<>
src={staticFile('demos/feature-demo.mp4')}
playbackRate={playbackRate}
/>
);
};
```
Error Handling ```tsx
import { Audio, staticFile, delayRender, continueRender } from 'remotion';
import { useEffect, useState } from 'react';
export const SafeAudio: React.FC<{ src: string }> = ({ src }) => {
const [handle] = useState(() => delayRender());
const [audioReady, setAudioReady] = useState(false);
useEffect(() => {
const audio = new window.Audio(src);
audio.oncanplaythrough = () => {
setAudioReady(true);
continueRender(handle);
};
audio.onerror = () => {
console.error(Failed to load audio: ${src});
continueRender(handle); // Continue without audio rather than hang
};
}, [src, handle]);
if (!audioReady) return null;
return
};
```
Toolkit Command: /generate-voiceover The /generate-voiceover command handles the full workflow:
```
/generate-voiceover
Reads VOICEOVER-SCRIPT.md Extracts narration for each scene Generates audio via ElevenLabs API Saves to public/audio/scenes/ Creates manifest.json with durations Updates project.json with timing info ```
|