Software for measuring/correcting lip sync audio delay?

[Robbie303]

I have several movies where the audio needs to be delayed.
Finding the correct number of milliseconds that the audio in every one of those movies needs to be delayed is a time-consuming job.
For example, some movies need to be delayed by 300 milliseconds, while other movies need to be delayed by 100 milliseconds.
Is there some software or plugin for Hybrid that compares the video with the audio, and then measures what the lip sync delay should be?
AI software that compares the movements of the mouth with the audio?


Simply playing the movie in VLC and then kind of guessing the delay, trying and repeat is what I am doing now, but again, that simple method is quite time-consuming...
What methods are you using to correctly measure the audio delay?

[Selur]

Sadly, there is no plugin I'm aware of.
Vapoursynth, does support audio, but sadly there are no waveform plugins or similar.
(I hoped a
For Avisynth there is waveform plugin, but I never really cared about implementing support for it.
=> your best bet is some NLE like Davinci Resolve or similar.

Cu Selur

[Selur]

I just remembered, maybe https://gitlab.com/EleonoreMizo/faveworm could be used:

core.std.LoadPlugin(path="%FILTERPATH%/SourceFilter/BestSource/BestSource.dll")
# bs.AudioSource(string source[, int track = -1, int adjustdelay = -1, int threads = 0, bint enable_drefs = False, bint use_absolute_path = False, float drc_scale = 0, int cachemode = 1, string cachepath, int cachesize = 100, bint showprogress = True, maxdecoders = 0])
# see: https://github.com/vapoursynth/bestsource
audio = core.bs.AudioSource(source="PATH TO YOUR INPUT", track = -1, adjustdelay = -2) # select first audio track to no adjustment

core.std.LoadPlugin(path="%FILTERPATH%/Support/faveworm.dll")
scope = core.fw.scope(clip=clip, audio=audio, mode=3, sweep=0.20, beam_size=1.5)
import misc

clip = misc.Overlay(clip, scope, opacity=0.5)

not really sure how atm.

[Selur]

Downloading https://gitlab.com/EleonoreMizo/faveworm and extracting the 64bit .dll it into Hybrid/64bit/vsfilters/Support/ and replacing Hybrid/64bit/vsscripts/misc.py with https://github.com/Selur/VapoursynthScri...er/misc.py
One can use:

import misc
# load audio
core.std.LoadPlugin(path="%FILTERPATH%/SourceFilter/BestSource/BestSource.dll")
# bs.AudioSource(string source[, int track = -1, int adjustdelay = 40, int threads = 0, bint enable_drefs = False, bint use_absolute_path = False, float drc_scale = 0, int cachemode = 1, string cachepath, int cachesize = 100, bint showprogress = True, maxdecoders = 0])
# track= -1 <> select first audio track; adjustdelay = -2 <> 'Ignore file delay metadata'
# see: https://github.com/vapoursynth/bestsource
audio = core.bs.AudioSource(source="%INPUTFILE%", track = -1, adjustdelay = -1)

delay_ms = 0 # Here you can set custom delay values
audio = misc.DelayAudio(audio, delay_ms=delay_ms)

# rendering of an analog oscilloscope to display audio waveforms
core.std.LoadPlugin(path="%FILTERPATH%/Support/faveworm.dll")
sweep = clip.fps_den / clip.fps_num # 1/fps
scope = core.fw.scope(clip=clip, audio=audio, mode=2, sweep=sweep, beam_size=3, y_gain=3.0, y_ofs=-0.3)  # doubles the visual height of the waveform

# overlay scope on video and display multiple frames instead of just one
compareFrameCount = 3  # number of frame to show
clip = misc.Overlay(clip, scope, opacity=0.5)
clip = misc.ShowFramesAround(clip, count=compareFrameCount)

in a 'Custom'-section to see what different delays do.

delay_ms=0:

delay_ms=-20:

delay_ms=20:


adjust

• delay_ms to test different delays
  
• compareFrameCount to see more frames
  

consideration that each frame is only 1000ms/fps long, so
24fps => 1000 / 24 ≈ 41,67 ms
25fps => 1000 / 25 = 40 ms
30fps => 1000 / 30 ≈ 33,33 ms
60fps => 1000 / 60 ≈ 16,67 ms
so depending on the delay you might want to use higher compareFrameCount values.


Maybe this helps a bit, to figure out the delay of a source.

Cu Selur