Lip-sync on every plan
Measure loudness
The voice audio is measured every 10 milliseconds.
Open the mouth
Louder moments open the mouth wider; silence closes it.
Shape it
Vowels in the line choose round (O, U) or wide (A, E) shapes, and letters such as m, b and p briefly close the lips.
Why it needs no AI
Everything comes from two things StoryToon already has: the sound of the line and its text. The letters are spread over the length of the line with a simple timing rule, so no speech recognition and no machine-learning model is needed.
With your own recorded voice
Lip-sync also follows recordings made with your microphone, so a recorded line animates the mouth just like a computer voice. Because the mouth opening only needs the loudness of the sound, it works for any language and any accent.
The mouth shapes
On every frame lip-sync picks one of these drawings: quiet moments return to rest, letters such as m, b, p, f, v and th choose the matching shape, and the vowels in the line choose A/I, E, O or U.
| Mouth shape | Sounds it shows |
|---|---|
| Rest | Silence, pauses between words |
| A/I | Open vowels, as in "cat" and "bite" |
| E | As in "bed" and "see" |
| O | Rounded, as in "go" |
| U | Small and rounded, as in "you" |
| M/B/P | Lips pressed together |
| F/V | Top teeth on the lower lip |
| L/TH | Tongue visible behind the teeth |
Frequently asked questions
How many mouth shapes does StoryToon use?
8, including the closed rest shape.
Does lip-sync work in Urdu or Arabic?
Yes. Lip-sync follows the loudness of any audio, in any language, including your own recordings.
Does the rest of StoryToon need AI?
No. Everything works without AI. StoryToon has optional AI features, off by default, that can write or improve a script with your own key, but lip-sync never uses them.