This forum uses cookies
This forum makes use of cookies to store your login information if you are registered, and your last visit if you are not. Cookies are small text documents stored on your computer; the cookies set by this forum can only be used on this website and pose no security risk. Cookies on this forum also track the specific topics you have read and when you last read them. Please confirm whether you accept or reject these cookies being set.

A cookie will be stored in your browser regardless of choice to prevent you being asked this question again. You will be able to change your cookie settings at any time using the link in the footer.

Audio not fully sync after feeding through Hybrid
#1
(Was originally meant to be posted around the beginning of April but lost motivation to do it until now.)

The problem I been getting when doing a job is that the audio and video aren't in sync whether the output container is .mkv and .avi. It less than a second apart but it is noticeable. The video is 50 mins long, as shown on the two .mp4 videos in the Google Drive folder, the sync gap widen the longer. When I cut out a 15 second segment from the 50 mins long video and do the same Hybrid job as both a .mkv output and .avi output, the sync gap is shorten although still not in sync and the offsets are both different. The offset of the filtered .mkv sample and the .avi sample varies.

I test this offset by opening two VLC windows load up the original and the filtered, then I and clicking on the scrub head to a random section on the video, then alt tab to the other VLC video and click on the scrub head on the other video. This means I clicked on the exact part, time and frame of the video without moving my cursor. (There a demonstration of it in the Drive link.)


There are 4 videos demonstrating this problem, one showing a hoatzin munching on a leaf. On the original .avi, this scene happened after the narrator stopped talking for a moment. On the edited .mkv version that was fed through Hybrid (also happens if you do the output container in .avi), you can see the narrator is still talking while the hoatzin is munching on the leaf.

Another demonstration is at the end of the documentary where the broadcaster is announcing the upcoming programs while the credit roll. On the original .avi one, the video and audio are playing normally as it abruptly stop due to the fact the VHS tape has reach the end of that recording. On the .mkv version, the video freezes but the audio continues playing until the end, fully showing that there is a offset between the video and audio when filtered and edited through Hybrid.

It even happens on a smaller scale on VirtualDub where I made a basic filter edit in the software. What's the point of doing filters on these VHS captures if it going to end up out of sync? It seem like when I solve one problem, another one occurs.

Google Drive link: https://drive.google.com/file/d/1WjTFbBX...sp=sharing
Reply
#2
Async issues usually happen if there is something wrong with the source (audio or video), something is interfering or similar hard to reproduce issues.
You no providing any detailed information doesn't really help the matter.

If you remux the file with mkvtoolnix is the result still sync? (assuming the file is in a format supported by mkvtoolnix)
If it is sync, does progressing that output work better?
If it is not, does extracting audio and video stream produce proper output streams?

Cu Selur
Reply
#3
What details do you need? I tried to explain it as much as I can link demonstrations via google drive. I did another test where the job didn't include the audio, from the looks of it, the audio isn't causing the problem, it the video itself. It was a dodgy tape with lots of cuts and not very stable picture wise. I still don't know why any form of editing will cause to do this and I'm at a dead end. I know I said this before but if I manage to get this sync problem solved I should be able to do jobs on Hybrid nice and smooth, fingers crossed.

Quote:If you remux the file with mkvtoolnix is the result still sync? (assuming the file is in a format supported by mkvtoolnix)

How do you do this?
Reply
#4
Quote:How do you do this?
Assuming mkvtoolnix does support the input format. (not sure, you will have to test)
Download mkvtoolnix, start mkvtoolinx (https://www.fosshub.com/MKVToolNix.html), start mkvtoolnix-gui, drag&drop the input file, start muxing.


Quote:What details do you need?
a. reset Hybrid (Config->Defaults->Reset->All->Reset selected)
b. write a step-by-step what you clip to get the problem.
c. provide a sample clip which combined with the steps allow to reproduce the problem.

Cu Selur
Reply
#5
ffmpeg uses -async 1 to solve this problem,but I don't know how to configurate -async 1 and vsync in Hybird.
Reply
#6
Quote:ffmpeg uses -async 1 to solve this problem,but I don't know how to configurate -async 1 and vsync in Hybird.
You can't since that would require audio&video handling at the same time which Hybrid can not do.


Cu Selur
Reply
#7
Quote:If you remux the file with mkvtoolnix is the result still sync? (assuming the file is in a format supported by mkvtoolnix)
If it is sync, does progressing that output work better?
If it is not, does extracting audio and video stream produce proper output streams?


I downloaded the tool, not exactly sure what to do next? Do I use the unedited capture as the source file or the edited capture as the source file? What purpose does this make?


Video is x264 with Audio set to "passthrough all". 
> set container to .mkv 
> Encoding mode is "constant rate factor (1-pass) with quantization factor set to 16 with AVC Profile/Level set to High and 4.2 
> Picture is cropped 8 on both sides and 6 on top side and resize to 9 - lanczos 1440 x 1178 via avisynth 
> deinterlaced via QTGMC (Vapoursynth) Fast with bob ticked 
> Tick TemporalDegrain in DeGrain 
> Tick Spotless and set "Temporal radius" to 3 in DeNoise 
> Tick FineDeHalo and set "Radius(x/y)" to "3/3" in DeHalo 
> Tick Stab and set "Range" to "3" and "Shift(max X/Y)" to "8/8"
Reply
#8
Quote:I downloaded the tool, not exactly sure what to do next?
Extract install the tool, when like I wrote:
Quote:... start mkvtoolnix-gui, drag&drop the input file, start muxing.

Quote:Do I use the unedited capture as the source file or the edited capture as the source file?
The sync input you would otherwise feed to Hybrid.

Quote:What purpose does this make?
The purpose is to understand what is causing your sync issues.

Quote:Video is x264 with Audio set to "passthrough all".
> set container to .mkv
> Encoding mode is "constant rate factor (1-pass) with quantization factor set to 16 with AVC Profile/Level set to High and 4.2
> Picture is cropped 8 on both sides and 6 on top side and resize to 9 - lanczos 1440 x 1178 via avisynth
> deinterlaced via QTGMC (Vapoursynth) Fast with bob ticked
> Tick TemporalDegrain in DeGrain
> Tick Spotless and set "Temporal radius" to 3 in DeNoise
> Tick FineDeHalo and set "Radius(x/y)" to "3/3" in DeHalo
> Tick Stab and set "Range" to "3" and "Shift(max X/Y)" to "8/8"
Okay, reading that I'm confused. You write you use Avisynth for resizing, but Vapoursynth for deinterlacing, that does not make sense.
Hybrid can only use one or the other at the same time. According to the debug output, Vapoursynth is used.
Tried to reproduce the issue with a HuffYUV file I got here, but could not.

Looking at the debug output, here's what Hybrid does during the job processing:
  • Starting subjob 2022-04-24@21_00_11_2310_01_audio:
    Here Hybrid extracts the audio using:
    "C:\Program Files\Hybrid\64bit\ffmpeg.exe" -y -threads 8 -i "F:\VHS\Broken Silence - Waste Not Want Not.avi" -map 0:1 -vn -sn -ac 2 -ar 48000 -acodec pcm_s16le -f wav -map_metadata -1 -metadata encoding_tool="Hybrid 2022.03.26.1" "C:\Users\kakeh\AppData\Local\Temp\iId_1_aid_1_2022-04-24@21_00_11_2310_01.wav"
    during which ffmpeg reports:
    [avi @ 00000196bb7e2540] non-interleaved AVI
    Guessed Channel Layout for Input Stream #0.1 : stereo
    Input #0, avi, from 'F:\VHS\Broken Silence - Waste Not Want Not.avi':
      Duration: 00:25:32.44, start: 0.000000, bitrate: 58074 kb/s
      Stream #0:0: Video: huffyuv (HFYU / 0x55594648), yuv422p, 720x576, 56547 kb/s, 25 fps, 25 tbr, 25 tbn
      Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
    Stream mapping:
      Stream #0:1 -> #0:0 (pcm_s16le (native) -> pcm_s16le (native))
    Press [q] to stop, [?] for help
    Output #0, wav, to 'C:\Users\kakeh\AppData\Local\Temp\iId_1_aid_1_2022-04-24@21_00_11_2310_01.wav':
      Metadata:
        encoding_tool   : Hybrid 2022.03.26.1
        ISFT            : Lavf59.20.100
      Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
        Metadata:
          encoder         : Lavc59.25.100 pcm_s16le
    and
    2022-04-24@21_00_11_2310_01_audio finished after 00:05:39.921
    which:
    created C:\Users\kakeh\AppData\Local\Temp\iId_1_aid_1_2022-04-24@21_00_11_2310_01.wav (280.598 MB)
  • starting new SubJobHandler for: 2022-04-24@21_00_11_2310_02_index:
    during which the indexing file for the Source filter is created using:
    "C:\Program Files\Hybrid\64bit\vsfilters\SourceFilter\FFMS2\ffmsindex.exe" -f -t 0 "F:\VHS\Broken Silence - Waste Not Want Not.avi" "C:\Users\kakeh\AppData\Local\Temp\2022-04-24@21_00_11_2310.ffindex"
  • starting new SubJobHandler for: 2022-04-24@21_00_11_2310_03_create:
    during which the Vapousynth script (C:\Users\kakeh\AppData\Local\Temp\encodingTempSynthSkript_2022-04-24@21_00_11_2310.vp) for the video filterering is created.
    # Imports
    import os
    import sys
    import ctypes
    # Loading Support Files
    Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
    import vapoursynth as vs
    # getting Vapoursynth core
    core = vs.core
    # Import scripts folder
    scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
    sys.path.insert(0, os.path.abspath(scriptPath))
    # Loading Plugins
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/CTMF/CTMF.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/HQDN3D/libhqdn3d.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/DePan.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libtemporalmedian.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3m.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/temporalsoften.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
    core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
    # Import scripts
    import rescued
    import lostfunc
    import SpotLess
    import havsfunc
    # source: 'F:\VHS\Broken Silence - Waste Not Want Not.avi'
    # current color space: YUV422P8, bit depth: 8, resolution: 720x576, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: bottom field first
    # Loading source using FFMS2
    clip = core.ffms2.Source(source="F:/VHS/Broken Silence - Waste Not Want Not.avi",cachefile="C:/Users/kakeh/AppData/Local/Temp/2022-04-24@21_00_11_2310.ffindex",format=vs.YUV422P8,alpha=False)
    # Setting color matrix to 470bg.
    clip = core.std.SetFrameProps(clip, _Matrix=5)
    clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
    clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
    # Setting color range to TV (limited) range.
    clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
    # making sure frame rate is set to 25
    clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
    # setting field order to what QTGMC should assume (bottom field first)
    clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=1)
    # Deinterlacing using QTGMC
    clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=False) # new fps: 50
    # make sure content is preceived as frame based
    clip = core.std.SetFieldBased(clip, 0)
    clip = SpotLess.SpotLess(clip=clip, radT=3)
    # stabilizing using Stab
    clip = lostfunc.Stab(clp=clip,range=3,mirror=0,dxmax=8,dymax=8)
    # cropping the video to 704x570
    clip = core.std.CropRel(clip=clip, left=8, right=8, top=6, bottom=0)
    # removing grain using TemporalDegrain
    clip = rescued.TemporalDegrain(inpClip=clip)
    clip = havsfunc.FineDehalo(clip, rx=3.0, ry=3.0)
    clip = core.fmtc.resample(clip=clip, kernel="lanczos", w=1440, h=1166, interlaced=False, interlacedd=False)
    # adjusting output color from: YUV422P16 to YUV420P8 for x264Model
    clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited")
    # set output frame rate to 50.000fps
    clip = core.std.AssumeFPS(clip=clip, fpsnum=50, fpsden=1)
    # Output
    clip.set_output()
    Seeing that you use an input file created by VirtualDub and that FFMS2 is used as a source filter you could try whether enabling "Filtering->Vapoursynth->Misc->Source->Prefer AviSource for .avi input" changes anything.
  • starting new SubJobHandler for: 2022-04-24@21_00_11_2310_04_video:
    during which the video processing takes place using:
    "C:\Program Files\Hybrid\64bit\Vapoursynth\vspipe.exe" "C:\Users\kakeh\AppData\Local\Temp\encodingTempSynthSkript_2022-04-24@21_00_11_2310.vpy" - -c y4m | "C:\Program Files\Hybrid\64bit\x264.exe" --preset veryfast --crf 16.00 --profile high --level 4.2 --ref 3 --direct auto --b-adapt 0 --sync-lookahead 24 --qcomp 0.50 --rc-lookahead 40 --qpmax 51 --partitions i4x4,p8x8,b8x8 --no-fast-pskip --subme 5 --aq-mode 0 --vbv-maxrate 62500 --vbv-bufsize 78125 --sar 1:1 --non-deterministic --range tv --colormatrix bt470bg --demuxer y4m --input-range tv --fps 50/1 --output-depth 8 --output "C:\Users\kakeh\AppData\Local\Temp\2022-04-24@21_00_11_2310_04.264" -
    2022.04.24 - 21:05:57_Windows 11 Version 2009 (64bit)_2022.03.26.1 - level 9: tokenizeAndStart(1): "C:\Program Files\Hybrid\64bit\x264.exe" --preset veryfast --crf 16.00 --profile high --level 4.2 --ref 3 --direct auto --b-adapt 0 --sync-lookahead 24 --qcomp 0.50 --rc-lookahead 40 --qpmax 51 --partitions i4x4,p8x8,b8x8 --no-fast-pskip --subme 5 --aq-mode 0 --vbv-maxrate 62500 --vbv-bufsize 78125 --sar 1:1 --non-deterministic --range tv --colormatrix bt470bg --demuxer y4m --input-range tv --fps 50/1 --output-depth 8 --output "C:\Users\kakeh\AppData\Local\Temp\2022-04-24@21_00_11_2310_04.264" -
    which finishes with:
    x264 [info]: frame P:19877 Avg QP:22.22  size: 22859
    x264 [info]: frame B:55636 Avg QP:25.35  size:  2726
    x264 [info]: consecutive B-frames:  2.6%  0.0%  0.8% 96.6%
    x264 [info]: mb I  I16..4: 39.1% 48.0% 12.9%
    x264 [info]: mb P  I16..4: 26.4%  0.0%  4.5%  P16..4: 36.3% 13.8%  4.5%  0.0%  0.0%    skip:14.5%
    x264 [info]: mb B  I16..4:  1.0%  0.0%  0.1%  B16..8: 14.1%  2.2%  0.1%  direct: 1.5%  skip:81.0%  L0:46.4% L1:46.4% BI: 7.1%
    x264 [info]: 8x8 transform intra:4.8% inter:47.8%
    x264 [info]: direct mvs  spatial:100.0% temporal:0.0%
    x264 [info]: coded y,uvDC,uvAC intra: 17.0% 20.7% 1.7% inter: 4.5% 5.8% 0.0%
    x264 [info]: i16 v,h,dc,p: 41% 32% 13% 14%
    x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 35% 19%  5%  4%  3%  6%  4%  7%
    x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 33% 15%  5%  7%  6%  9%  4%  5%
    x264 [info]: i8c dc,h,v,p: 61% 21% 15%  3%
    x264 [info]: Weighted P-Frames: Y:9.4% UV:0.6%
    x264 [info]: ref P L0: 71.6% 22.4%  6.0%
    x264 [info]: ref B L0: 92.8%  6.4%  0.9%
    x264 [info]: ref B L1: 98.4%  1.6%
    x264 [info]: kb/s:3338.79

    encoded 76256 frames, 26.93 fps, 3338.79 kb/s
    and created:
    created C:\Users\kakeh\AppData\Local\Temp\2022-04-24@21_00_11_2310_04.264 (607.021 MB)
  • 2022-04-24@21_00_11_2310_05_cleanUp:[/code]
    durint which "C:\Users\kakeh\AppData\Local\Temp\encodingTempSynthSkript_2022-04-24@21_00_11_2310.vpy" is deleted
  • starting new SubJobHandler for: 2022-04-24@21_00_11_2310_06_cleanUp:
    during which "C:\Users\kakeh\AppData\Local\Temp\2022-04-24@21_00_11_2310.ffindex" is deleted.[/b]
  • starting new SubJobHandler for: 2022-04-24@21_00_11_2310_07_create:
    during which the global tag file for mkvmerge (C:\Users\kakeh\AppData\Local\Temp\mkvtags_2022-04-24@21_00_11_2310__07.xml) is created.
  • starting new SubJobHandler for: 2022-04-24@21_00_11_2310_08_muxing:
    during which mkvmerge is used to multiplex the audio&video using:
    "C:\Program Files\Hybrid\64bit\mkvmerge.exe" --ui-language en -o "F:\VHS\2022-04-24@21_00_11_2310__07.mkv" --global-tags "C:\Users\kakeh\AppData\Local\Temp\mkvtags_2022-04-24@21_00_11_2310__07.xml" -d 0 --default-track 0:yes --default-duration "0:50/1fps" --aspect-ratio-factor 0:1/1 --fourcc 0:MP4V --no-chapters --compression -1:none --forced-track 0:yes --no-audio --no-subtitles "C:\Users\kakeh\AppData\Local\Temp\2022-04-24@21_00_11_2310_04.264" --default-track 0:yes --forced-track 0:no -a 0 --compression -1:none --no-video --no-subtitles --no-chapters "C:\Users\kakeh\AppData\Local\Temp\iId_1_aid_1_2022-04-24@21_00_11_2310_01.wav"
    2022-04-24@21_00_11_2310_08_muxing finished after 00:00:30.183
    created F:\VHS\2022-04-24@21_00_11_2310__07.mkv (888.211 MB)
  • starting new SubJobHandler for: 2022-04-24@21_00_11_2310_09_cleanUp:
    during which "C:\Users\kakeh\AppData\Local\Temp\mkvtags_2022-04-24@21_00_11_2310__07.xml" is deleted
  • starting new SubJobHandler for: 2022-04-24@21_00_11_2310_10_cleanUp:
    during which "C:\Users\kakeh\AppData\Local\Temp\2022-04-24@21_00_11_2310_04.264" is deleted
  • starting new SubJobHandler for: 2022-04-24@21_00_11_2310_11_cleanUp:
    during which "C:\Users\kakeh\AppData\Local\Temp\iId_1_aid_1_2022-04-24@21_00_11_2310_01.wav" is deleted[


Hybrid does not detect any errors or warnings from the tools used, so the synch problem you notice either:
  • is created during the audio extraction (during which parts of the audio would need to get lost),
  • the video filtering/encoding (during which parts of the video would need to get lost),
  • using the audio&video multiplexing (during which audio&video do not get properly synched), or
  • the auio&video are not asynch and it's an issue with the playback.
To test which might be the issue you could tell Hybrid to not delete the temp files. (Config->Containers->General Settings->Keep intermediate files)
And then compare:
  • the extracted audio with the audio of the input source (fed to Hybrid) to see whether the unprocessed source audio and the audio that got distracted differ.
  • the reencoded video with the video of the input source (fed to Hybrid) to see whether the frame count is correct.
  • You could try whether remuxing the source to mkv before processing with Hybrid or causing Hybrid to use another source filter does change anything. (this is what the remuxing with mkvtoolnix and the switchting to AviSource which I both mentioned earlier would be for.)



Cu Selur
Reply
#9
Quote:Okay, reading that I'm confused. You write you use Avisynth for resizing, but Vapoursynth for deinterlacing, that does not make sense.

The crop style is Avisynth.

-

Okay so I did a test where I tick prefer avisource for .avi input and although it still not sync but I think it has reduce the severity. I also remux the .avi to .mkv using the software you suggested and there still is sync issues but in different places, I can't determine how far apart the delay is (about 9-10 frames apart). It hard to describe, like in some area of the video they are sync and other places they aren't. I did the edit job with the remux .mkv file through Hybrid and that also was out of sync slightly, it seem like anything I do result in at least one frame out of sync.

I have compared the raw .wav audio to the edited video and they seem to be in sync so somewhere in the process, the video frame gets lost in the process.

Edit:
I have also noticed if I do a job with it only doing deinterlacing, it adds additional frames. When loading the original .avi file in Hybrid, the frame count shows 38,311 frames. While after doing the job to convert it to .mkv and deinterlaced it loses 183 frames (38,128 frames in total). I did the same test but with I tick prefer avisource for .avi input and now shows 38,311 meaning no frames has been lost, however it is still slightly out of sync and lost about 4 frames even though it has the same amount of frames as the original .avi HuffYUV video. I inserted the debug of said edited video below.
Reply
#10
To clear things up, the frame count shown in Hybrid is the frame count which either is from the container tags if they are present or a calculation from frame rate and time, so it's normal that it's not totally accurate since it's a often an estimation.
Problem is I can't reproduce this here at all. :/

Quote:I also remux the .avi to .mkv using the software you suggested and there still is sync issues but in different places,
Okay. If even a simple remux of the source produces an sync output, then something is 'off' with the source file and Hybrid really might not be a good tool to handle that source.

Remuxing the source with ffmpeg in a command line to see warnings&go might help.
Try remuxing using:
"PATH TO Hybrid/64bit/ffmpeg.exe" -c copy -vsync 2 -i "PATH to source" "PATH to output file"
and see:
a. whether this shows any warnings during the process
b. produces a sync output file

Cu Selur
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)