AI Lip Sync Test: Seedance 2.5, Wan 3.0 and InfiniteTalk

👁 129 reads

I tested AI lip-sync on a Polish song, using the same section of music and the same character frame for each model. Seedance 2.5 produced the image I liked best, but scored lowest for matching the singer’s mouth movements to the vocals. Wan 3.0 was a faster route to usable playback, though I hit a duration limit. Local InfiniteTalk achieved the highest measured score, at the cost of a longer wait.

Same song section, same character

I chose a 20-second section of a Polish song whose title translates as “When I Play My Own Way,” from 14.5 to 34.5 seconds. This covers the second chorus, the drums coming in and the start of the verse. That meant I wasn’t judging only a quiet, sustained vowel. The material changes in energy, making it easier to see whether the face responds to the singing or simply performs an eye-catching animation. The song and test material were in Polish.

I started with the same character frame, prepared locally in Z-Image Turbo. For the API versions, I used an audio reference through the TAKT engine. The local version used InfiniteTalk through WanVideoWrapper, with the Wan 2.1 I2V 14B fp8 base and lightx2v at 6 steps. This is a comparison of specific configurations, not every capability of these models.

I planned to compare the full 20 seconds, but Wan 3.0 did not complete a request of that length. So the first 15 seconds became the common measurement window. I’ve left results from the longer section out of the shared ranking. Otherwise, I’d be comparing models on material with different content, and the table would look more reliable than the method warranted.

How I measured lip sync with MediaPipe

In the lipsync_measurement.py script, I compared mouth-opening patterns detected by MediaPipe with the loudness of the vocals separated by Demucs. I searched for an offset within ±0.5 seconds. A higher correlation meant that the patterns matched more closely under this method. This helped distinguish a general facial response to the voice from the impression that a character was “singing something.”

This measurement has limits. Loudness does not describe the exact mouth shapes required for every Polish sound. Lip closure, consonant articulation and natural-looking expressions need additional assessment by watching the clip. So I don’t interpret correlation as an automatic pronunciation score or proof that viewers will believe the character performed the whole song.

I also recorded face detection separately. InfiniteTalk and Wan had a reported rate of 100%, while Seedance ranged from 69% to 76%. Lower face availability makes the measurement less complete and limits direct conclusions. I don’t speculate about why some detections were missing, because the report does not establish a cause. I consider this metric alongside viewing the footage, rather than treating it as a verdict on its own.

Searching for an offset also means the result describes how well the patterns match after alignment. It does not prove that a file downloaded from the generator starts with its audio perfectly timed. When assembling a finished shot, I check the fixed offset separately and see whether the match holds across later phrases. Shifting the whole audio track may improve the start, but it cannot replace accurate mouth animation throughout the rest of the song.

Results, costs and render times

The times are for the clips received: the local version and Seedance produced 20-second clips, while Wan produced 15 seconds. The shared duration applies to the synchronisation measurement column. These amounts are the costs of this test, not a price list that applies on any given day. I haven’t added an electricity cost because I didn’t measure one.

The output settings also differed. InfiniteTalk produced 832×480 at 25 fps, Wan 1280×720 at 30 fps, and Seedance 1280×720 at 24 fps. You’ll need to account for those differences when editing. In this comparison, the higher resolution did not translate into better mouth-to-song matching.

Seedance: best-looking footage, but an audio issue

I liked Seedance’s lighting, composition and overall finished-shot look best. That’s a visual judgement, separate from the synchronisation measurement. An attractive frame can draw your attention away from the mouth, especially when you watch without the original song. Once I matched the image with the actual vocals, the visual advantage wasn’t enough.

The model returned its own version of the audio track. Its correlation with the original was 0.12. For this material, replacing the generated audio with the original was necessary, taking into account an offset of about 47 ms noted in my records. I wouldn’t automatically apply that offset to other clips. The model’s audio should not replace the song master just because it came with the video.

Wan 3.0: useful playback, but capped at 15 seconds in this test

Wan scored higher than Seedance for correlation and cost less in this test. The most practical issue came up before I could assess the image: a 20-second request ended with the generation_failed message, without a charge. Only 15 seconds worked. That’s how I interpret the limit in this test: a restriction in the API workflow I tried, not evidence about every possible integration.

If you’re planning a music video, I’d prepare shorter sections and places to cut between shots. I wouldn’t assume that a longer phrase will work just because the duration field accepts the requested value. Generating the image successfully and matching the clip boundaries to the music are separate tasks to account for before submitting another request.

InfiniteTalk leads the measurement, while V2V offers another option

Local InfiniteTalk achieved the highest correlation over the shared section. For the full 20 seconds, its score was 0.424. I got a lower resolution and a longer processing time, but it was a stronger starting point for a task focused on singing a specific section. That doesn’t mean every word was articulated perfectly.

I also tested combining Seedance with local InfiniteTalk in V2V mode. At 2 of 4 steps, with audio_scale set to 1.5, correlation increased from 0.153 to 0.366. Face detection was 77%, and the extra stage took 656 seconds. The result was 832×480, so the improvement in playback came with a change in image settings.

This workflow didn’t beat standalone InfiniteTalk in the measurement, but it let me reuse an image generated earlier through the API. A later pass with SeedVR2 7B fp8 upscaled it to 1872×1080 in an additional 1034 seconds. I see this as a way to finish a chosen shot. Upscaling is not evidence of another improvement in synchronisation.

Frequently asked questions

Which model would I choose for singing in Polish?

Based on this test, I’d choose InfiniteTalk if playback matters most. I’d consider Wan for shorter shots when I needed a quicker result. I’d use Seedance for images good enough to justify extra mouth correction.

Does the highest correlation mean perfect lip sync?

No. I measured how mouth opening relates to vocal loudness, not whether every sound was pronounced correctly. The result helps compare these tests, but it doesn’t replace watching the clip with the original mix.

Is local generation free?

The report records no generation fee. Electricity, hardware and setup time still have costs. Without measuring electricity use, I can’t give a total cost or promise savings for any particular number of clips.

I cover model pricing and access in my articles on Seedance and Wan 3.0, and shot planning for a full video in my article on Kling 4.0 for music videos.

Kinetyra