[HELP] MKV (H.264/ACC) for Plex audio sync issues

[evillink]

Hello everyone, I’ve been trying out Hybrid for the past 3 months and found it to be awesome to have so many tools integrated into one place.

My goal is to transfer all of my family’s VHS to DVD, I have capture the master file in VirtualDub using huffyuv as the compression method, the resulting AVI is then processed using Avidemux to create a compliant DVD file and everything works as expected, I get a really good end product with audio and video in sync.

My other task was to convert the master file to an H.264 for my Plex server and this is where things get a bit weird. I tried Avidemux and even though the resulting file has no audio video sync problems, the deinterlacing algorithms available are not really good, at least not compared to QTGMC script. So, Hybrid to the rescue, not only does it allow me to encode to H.264 and AAC, add my chapters, but I can run QTGMC with bob using VapourSynth or AviSynth.

Problem is, in the resulting MKV file, the audio is progressively out of sync. An hour into the video the audio lags about 1s, I really don’t know what’s causing the problem. I tried encoding the video alone with audio passthrough, not applying any filtering, different encoding modes (2pass/CRF) all under the same profile Hight 4.1 Slow and every time the result is the same audio sync problem.

The only way I’ve managed to complete my task is by using Hybrid to apply QTGMC filter and output to FFvHuff with audio passthrough (doubling the file size from 30 to 60 GiB) and then use AviDemux to do the actual H.264 encoding, resulting in a synced mkv file.

Question is… why do I get this issue in Hybrid?

Hopefully someone can help shed some light on why is this happening.

Additional information

Hybrid 2020.02.17.1

x264 --preset slow --crf 18.00 --profile high --level 4.1 --sync-lookahead 12 --vbv-maxrate 62500 --vbv-bufsize 78125 --sar 20:11 --qpfile GENERATED_QP_FILE --non-deterministic --range tv --colormatrix bt470bg --demuxer raw --input-res 352x480 --input-csp i420 --input-range tv --input-depth 8 --fps 30000/1001 --output-depth 8 --output "C:\Users\f\AppData\Local\Temp\clip.264" –

MediaInfo for the original AVI file

General
Complete name                            : C:\Users\f\Videos\tape01.avi
Format                                   : AVI
Format/Info                              : Audio Video Interleave
Format profile                           : OpenDML
File size                                : 28.0 GiB
Duration                                 : 2 h 2 min
Overall bit rate                         : 32.7 Mb/s
Writing library                          : VirtualDub build 35491/release
 
Video
ID                                       : 0
Format                                   : HuffYUV
Format version                           : Version 2
Codec ID                                 : HFYU
Duration                                 : 2 h 2 min
Bit rate                                 : 31.2 Mb/s
Width                                    : 352 pixels
Height                                   : 480 pixels
Display aspect ratio                     : 0.733
Frame rate                               : 29.970 (30000/1001) FPS
Standard                                 : NTSC
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Bit depth                                : 8 bits
Scan type                                : Interlaced
Bits/(Pixel*Frame)                       : 6.152
Stream size                              : 26.7 GiB (95%)
 
Audio
ID                                       : 1
Format                                   : PCM
Format settings                          : Little / Signed
Codec ID                                 : 1
Duration                                 : 2 h 2 min
Bit rate mode                            : Constant
Bit rate                                 : 1 536 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Bit depth                                : 16 bits
Stream size                              : 1.32 GiB (5%)
Alignment                                : Aligned on interleaves
Interleave, duration                     : 33  ms (1.00 video frame)
Interleave, preload duration             : 500  ms

Please, read the 'Infos needed to fix&reproduce bugs,..'-sticky before you post about a problem.Please, read the 'Infos needed to fix&reproduce bugs,..'-sticky before you post about a problem.

[Selur]

At least I would need a sample file which allows me to reproduce the issue to figure out what is causing the issue and how to work around it.

Cu Selur

[evillink]

At least I would need a sample file which allows me to reproduce the issue to figure out what is causing the issue and how to work around it.

Cu Selur

Thank you for fast response Selur, I could provide you with a sample clip cause the entire file is 30GiB but I don't know if the issue will appear if the clip is too short, as it begins to be noticeable the longer the file is.

In any case I will prepare a clip and upload it so you can try and reproduce the error.

Thank you for your time.

[Selur]

No problem, send me a link via PM and I'll look into it after work.

Cu Selur

[evillink]

No problem, send me a link via PM and I'll look into it after work.

Cu Selur

Good morning Selur, I’m quite sorry for the delay of my response but it’s been kind of hectic these past days, I also wanted to run a few more test to try and make sure this same issue happen on other sources, which it did.

Like I said before my source file is a 30GiB 352x480 Huffyuv AVI capture using VirtualDub, and since its not possible for me to upload that file what I did was cut two clips one with just random footage (Tik.avi) and the second one with a person talking (Tok.avi).

Then I combined the two files Tik.avi (x2) + Tok.avi (x1) until I got different length videos and ran them through the same process on Hybrid  MKV[(x264 CRF 20), (AAC LC FDK) and (QTGMC Bob), (PAR 20:11)]  and for short clips 5, 10, 15, 20, 30, 45 minutes the audio pretty much stays in-sync.

I then created a 30GiB file following the same pattern two Tik.avi and one Tok.avi, repeated until I got about 2 hours of footage (the actual length of my VHS tapes) and that’s then the problem appears, you can see how the parts when the person speaks slowly goes out of sync towards the end of the video.

I uploaded the two files to: https://www.dropbox.com/sh/df38hmdkwahyl...qGjaa?dl=0

Thanks for your support and dedication to Hybrid.

PS.
System specs

AMD Ryzen 5 PRO 2400GE (Radeon Vega Graphics) 3.20GHz
16GiB RAM
256GiB NVMe
Win 10 Pro 64-bit (1903)

[Selur]

How did you append the files?
Did you make sure to create OpenDML avi files, otherwise anything beyond 2GB will make problems?

[evillink]

How did you append the files?
Did you make sure to create OpenDML avi files, otherwise anything beyond 2GB will make problems?

I tried two different ways with the same result. The first was using VirtualDub to append AVI segment and then just saved the AVI with direct stream copy for both audio and video. I did the same using AviDemux Append and direct copy to Avi Muxer selecting OpenDML.

[Selur]

I'll try to reproduce this later today.

Cu Selur

[Selur]

While encoding a 34gb file I created by appending these two files I thought I found the problem.
Those files seem to be vfr and interlaced which is a bad combi.
Looking at the time codes of the file Toki.avi file:

# timecode format v2
timecode  displaytime
  0 + 33
  33 + 34
  67 + 33
  100 + 33
  133 + 34
  167 + 33
  200 + 34
  234 + 33
  267 + 33
  300 + 34
  334 + 33
  367 + 33
  400 + 34
  434 + 33
  467 + 34
  501 + 33
  534 + 33
  567 + 34
  601 + 33
  634 + 33
  667 + 34
  701 + 33
  734 + 33
  767 + 34
  801 + 33
  834 + 34
  868 + 33
  901 + 33
....

(added the display time of each frame)
Everything seems to be okay since in average one would still get 33.6ms per frame (= 3000/1001 fps), but my guess is that your source might have larger glitches in the time codes which might be what causes the problem. Since the source isn't flagged as vfr Hybrid assumes it to be cfr and uses the average fps.
So the obvious solution would be to tell Hybrid to simply use the time codes of the input and be done with that. Problem is that as soon as you change the frame count that isn't possible any more. So whenever you bob or interpolate to another frame rate and thus change the frame count the input frame count would be ignored by Hybrid.
Since Hybrid handles audio and video separately it's not really capable of compensating this delay FFmpeg or Handbrake based tool might be better suited for this, but my guess is that anything that is Vapoursynth or Avisynth based won't be able to bob and keep the content sync.

So now after encoding the file I looked at the output and the output plays sync here. (I looked at the woman speaking at the end of the clip.)
Here's a link:
https://drive.google.com/open?id=1mWwA1x...iWCZzIEi0r
to the 4,4GB file I encoded.

Some glitches in the time codes might still be the cause of your sync issues, but with the given files I'm not able to reproduce the issue. So my recommendation would be to try a Handbrake or FFmpeg based tool which encodes Audio&Video at the same time and thus can to better error handling.

Cu Selur

Ps.: To be sure that you can use the same tools I used I'll send you a link via PM to my current dev version that I used to encode the file.

[evillink]

I’m finally back, it took me a while to try and understand and test out everything you pointed out. I’m far from an expert on many of the topics you clearly have profound knowledge on.

First, I checked my source and capture another VHS to the same container and same compression (AVI/Huffyuv) and they are in fact interlaced cfr and not vfr. Does AVI container even allow vfr? I was under the assumption it didn’t.

Second, I used ffmpeg and ffprobe to get the timecodes for each frame (I did it by hand on the Tik and Tok files  by copying and pasting form VirtualDub status bar and it was a horrible experience) and the results were same ones you found, each frame is around 33.3667 [33-34], no major jumps or glitches. Is there a better approach to extract timecodes?

What I did found is that I have some in-between shots that caused frame drops during capture, that lead me to believe that was the cause of all problems, but:

• The master AVI file plays in-sync even with those issues.
  
• The sample files don’t have any frame drops and when I appended them together on a file over 2h the audio drifts when encoded to H.264 with or without any kind of filtering (simple or bob deinterlace) but no so when they are left in their original AVI container.
  
• I can apply QTGMC bob on Hybrid with no audio drift problems, but it has to be kept as an FFvHuff AVI which I can later encode to H.264 using Handbreak, AviDemux or ffmpeg… but why? Why does AVI works, isn’t audio and video being handled separately like with MKV H.264?
  

I downloaded the file you help me encode an it sounds great, what parameters did you use to encode it? which de-interlacing algorithm was used? Cause no matter what I chose I always get audio drifting on 2 hours video, even with the sample files.

I really appreciate your help Selur