AI video dubbing that follows the picture
Voices, sound effects and music from one description
Most AI voice tools read text into a void and leave you to line it up with the edit. Seed Audio 1.5 watches the video. Describe who speaks, what they say, what the room sounds like and what the music does, and it returns one soundtrack timed to what is on screen.
You write the lines. The model performs them, with effects and music, in time with the picture.
Three ways to dub
ByteDance documents three video modes. Pick by how much control you need over the voice.
Video and a description
The model reads the scene and voices it from your text: the lines, the type of voice, the effects and the music bed.
Video, description and a voice
Add a reference clip and the lines are spoken in that voice. The video sets the timing, the reference sets the timbre.
Translate what is already there
If the video already has speech in English or Chinese, translate it instead and keep the speaker's own voice.
Where dubbing earns its place
AI-generated footage
Clips from video models often arrive silent or with a guessed soundtrack. Dub them with the dialogue and atmosphere the scene actually needs.
Bad location sound
ByteDance lists improving low-quality audio as a use case: re-perform the scene cleanly instead of fighting the noise.
Short dramas and series
Keep a character's voice the same from episode to episode by reusing the same reference clip.
Ads and promos
Time the voice-over, the product sound and the music sting to the frame, then change the language for the next market.
How to dub a video with Seed Audio 1.5
Prepare the cut
MP4 or MOV, 4 to 360 seconds, 480p to 1080p, up to 200 MB. Dub the final edit: if the cut changes, the timing changes.
Write what is heard
Lines in quotes, a short description of each voice, then the effects and the music. The model does not invent dialogue, so everything that should be said goes in the prompt.
Pin timing where it matters
Use timestamps like [2.0s:6.5s] for lines and effects that must land on a cut. Leave the rest to the model.
Add a voice reference if needed
Up to six clips of 2 to 30 seconds each. Use the same clip every time a character speaks.
Prompt examples
ByteDance's prompt guide recommends: what is heard, how it sounds, and what must not be there.
Two people on a rainy street
Outdoor street at night, steady rain on umbrellas and a passing car on wet asphalt. A young woman with a clear British voice, exasperated then laughing, says: “I cannot believe I let you talk me into this.” A man with a warm baritone, apologetic but smiling, answers: “I said it might rain. I did not say the sky would take it personally.” No music.
Each speaker gets a one-line description and an emotion before the line.
A product shot with exact timing
[0.0s:15.0s] Light electronic music under the whole clip. [0.4s:1.0s] A crisp click as the lid opens. [1.5s:5.0s] A confident adult male voice, General American accent, says: “Built for the days that do not end at five.” [12.0s:15.0s] The music lifts and resolves.
Timestamps keep the click on the frame where the lid opens.
Before you start
- The model does not write the script. A video with no description in the prompt gets no planned dialogue.
- It makes audio only. Mouth movement on screen is not changed, so dialogue works best on wide shots, backs of heads, narration and animation.
- One request is up to 6 minutes of audio; dub longer films scene by scene.
- Use voice references only with the speaker's permission.
AI video dubbing FAQ
What is AI video dubbing?
Generating the soundtrack of a video with AI: dialogue, voice-over, sound effects and music, timed to what happens on screen. With Seed Audio 1.5 you describe the sound and the model produces it in one pass.
Does Seed Audio 1.5 write the dialogue for me?
No. ByteDance is explicit that it does not dub automatically. You provide the lines and a description of the sound; the model performs and times them.
Can the dub use a specific voice?
Yes. Add a reference clip of 2 to 30 seconds and the lines are spoken in that voice, while the video controls the timing. Up to six references can be used in one request.
Can I dub in languages other than English?
Yes. Seed Audio 1.5 speaks 30 languages, from Japanese and Korean to Polish, Turkish and Hindi.
How is dubbing different from video translation?
Dubbing creates a new soundtrack from your description. Video translation takes speech that already exists in English or Chinese and re-voices it in another language with the same voice.
Related
Give your scene the sound it needs
Write the lines, describe the room, and let the picture set the timing.
Dub a video