[HELP] Audio not-in-sync after processing

[s_noopy]

• What Hybrid version did you use?(Windows/Mac/Linux 32/64bit + versions number)
  Windows, 2024-06-16
  
• What did happen? If an error message came up what did it exactly say?
  No error. Audio is no longer in sync with the video after processing
  
• What did you do to get to the bug? (best thing is a detailed step-by-step guide so that I can reproduce the problem)
  1. Took a footage from a DV camera to a computer using Firewire interface
  2. Merged files using Lossless Cut into an .mkv
  3. Processed the merged files using Hybrid with  a. deinterlacing QTGMC  b. recoding audio to AAC (tried all different codecs, keeping the input sample rate, even tried no recoding but a passthrough to .mkv container in PCM)4. Observed output to be progressively out-of-sync
  
• Debug output would be helpful:
  Attached
  
• a small sample, which allows the reproduction of the problem would be nice
  Attached a single scene from the original files
  I may not have checked some checkbox to get the properly synced output but it is the first case I found. Footage I have captured from VHS has no sync issues with the same Hybrid processing. It may be a bug though since the input is correctly synced but output not.

[Selur]

Update to latest version.
Not sure what to do with the sample since:
a. you wrote that you fed Hybrid with an mkv (the attached is a mov)
b. the sample is too short (800ms) to see any sync issues.

Reading that you cut the file before processing it with Hybrid: Does the problem also happen if you process the uncut file with Hybrid?

Looking at the debug output, I noticed, that the MediaInfo output reports:

Delay                                    : 0
Delay                                    : 00:00:00.000
Delay                                    : 00:00:00:00
Delay                                    : 00:00:00.000 (00:00:00:00)
Delay, origin                            : Container
Delay, origin                            : Container
Delay_Original                           : 1520
Delay_Original                           : 1 s 520 ms
Delay_Original                           : 1 s 520 ms
Delay_Original                           : 1 s 520 ms
Delay_Original                           : 00:00:01.520
Delay_Original                           : 00:00:01:13
Delay_Original                           : 00:00:01.520 (00:00:01:13)
Delay_Original_DropFrame                 : No
Delay_Original_Source                    : Stream
Time code of first frame                 : 00:00:01:13

So container reports a delay of 0ms while the video stream itself report 1520ms, which seems strange.
The Audio stream reports a delay of 40ms to the video and will turn into -3ms delay due to the aac encoder delay.
This is the only delay Hybrid cares about and later uses:

mkvmerge --ui-language en -o "C:\Users\Darek\Documents\2024-10-01@23_48_30_7510__07.mkv" --global-tags "C:\Users\Darek\AppData\Local\Temp\mkvtags_2024-10-01@23_48_30_7510__07.xml" -d 0 --default-track 0:yes --color-matrix-coefficients 0:5 --color-range 0:1 --color-transfer-characteristics 0:1 --color-primaries 0:5 --track-name 0:"Core Media Video" --default-duration "0:50/1fps" --aspect-ratio-factor 0:16/15 --fourcc 0:MP4V --no-chapters --compression -1:none --forced-track 0:yes --field-order 0:0 --no-audio --no-subtitles "C:\Users\Darek\AppData\Local\Temp\2024-10-01@23_48_30_7510_05.264" --track-name 0:"Core Media Audio" --default-track 0:yes --sync 0:-3 --forced-track 0:no -a 0 --compression -1:none --no-video --no-subtitles --no-chapters "C:\Users\Darek\AppData\Local\Temp\iId_2_aid_0_DELAY_-3ms_2024-10-01@23_48_30_7510_02.aac" --track-order 0:1,1:2

The used scripts and calls look fine to me, can't see any issues, error messages or warnings.
Only thing that I noticed it that Hybrid assumes the output to have 172612 frames, but x264 reports to have encoded 177516 frames.(which is ~98 seconds), so my suspicion is that there is some problem decoding the source.


A few things come to mind that you can test:

• if you set Video&Audio processing to passthrough and just remux the file, is it still sync?
  
• if the Vapoursynth Preview shows the expected number of frames (I suspect not). Does this change if you use another decoder? (FFMS2 or Bestsource?)
  
• if you use your setting and additionally enable "Config->Input->Decoding->CFR output", does that help with the sync issue?
  
• if you use some elaborate 'monster' antivirus suite, try whether it helps to disable it.
  

If that does not help, share a sample of the source that you feed into Hybrid, which allows reproducing the issue. (Test with that sample before sharing it. You can upload the file, for example, to GoogleDrive or similar and share a link here or via pm.)

Cu Selur

[s_noopy]

Not sure what to do with the sample since:
a. you wrote that you fed Hybrid with an mkv (the attached is a mov)
b. the sample is too short (800ms) to see any sync issues.

I was hoping that maybe from the source material you would be able to see anything interesting. Mov to mkv conversion was made by the Lossless Cut program which from my knowledge does not re-encode streams but only repackages them. I used Lossless Cut to merge all the individual scenes to a single file to process with a single job. The merged file was in .mkv container which did not exhibit sync problems. Correct me if I am wrong or there is some way to feed 100 mov files into Hybrid so it would output a single file.

Interestingly enough DV codec is not acceptable in .mkv container according to Hybrid but Lossless Cut did it without batting an eye... I could try merging into .mov container

if you use your setting and additionally enable "Config->Input->Decoding->CFR output", does that help with the sync issue?

Not if I can tell

if you use some elaborate 'monster' antivirus suite, try whether it helps to disable it.

Windows Defender only

if you set Video&Audio processing to passthrough and just remux the file, is it still sync?

Nope, it wen't out of sync

if the Vapoursynth Preview shows the expected number of frames (I suspect not). Does this change if you use another decoder? (FFMS2 or Bestsource?)

Input file frame count: 88758
Vapoursynth Preview frame count: 177516
Output file with audio sync problem frame count: 177335 (roughly 4 seconds missing which is ~out-of-sync value at the end of the video I experience)
Output file with Frame Interpolation for 50 FPS frame count: 172573 frames (I tried that to see if maybe it would help)

I couldn't find the place to change the decoder, could you point me to it?
I'll try to prepare a sample but it may take a while since I have time only at evenings/nights to do stuff on the computer.

Edit: Passthrough, 50 FPS interpolation and checking preview frame count where made on updated Hybrid version (2024-09-29)

[Selur]

Mov to mkv conversion was made by the Lossless Cut program which from my knowledge does not re-encode streams but only repackages them.

If you cut on non-key frames reencoding is done, but only for the affected gops.

I used Lossless Cut to merge all the individual scenes to a single file to process with a single job.

afaik. LosslessCut uses ffmpeg, so the source might be vfr. (MediaInfo only analyses the first xy timecode to decide whether a source is vfr or not, so on merged content you can not reply on it.

The merged file was in .mkv container which did not exhibit sync problems.

okay.

Correct me if I am wrong or there is some way to feed 100 mov files into Hybrid so it would output a single file.

No there is not.

Interestingly enough DV codec is not acceptable in .mkv container according to Hybrid but Lossless Cut did it without batting an eye...

Yes, Hybrid does not support dv in mkv since mkvmerge does not.

I could try merging into .mov container

That will probably have the same issue, but you can try.

Nope, it wen't out of sync

Okay, then it's not a general extracting and muxing problem and the source is probably cfr, since Hybrid will ignore time codes if the source isn't detected as vfr or it's told to extract them.

Input file frame count: 88758
Vapoursynth Preview frame count: 177516
Output file with audio sync problem frame count: 177335 (roughly 4 seconds missing which is ~out-of-sync value at the end of the video I experience)

So, the sync issue is just at the end, starting at a specific point, or do you just notice a sync problem at the end?

I couldn't find the place to change the decoder, could you point me to it?

The settings under Filtering->Vapoursynth->Misc->Source, determine the source filter.
You can check the Vapoursynth Script View to see which source filter / decoder is used.

I'll try to prepare a sample but it may take a while since I have time only at evenings/nights to do stuff on the computer.

Ho hurry, from me, but make sure that the sample allows to reproduce the problem.

Cu Selur

[Bartoloni]

just found that my latest 4 jobs are out of sync... using MKV as container and AVI source from PAL VHS.
i have not changed anything on my settings.
now i'm rendering again one of the clip after unsinstalling hybrid, deleting folder on roaming... and reinstalling the NON-DEV version.
1 hours and i can say if there is again an audio de-sync.

[Selur]

1 hours and i can say if there is again an audio de-sync.

5 hours later,... no debug output, so not much to go on.
Usual causes for asynch:

• the source filter does not decode all frames (due to incompatibility or bug in either the filter or the source).
  Check the preview. Is the frame count 'correct'? Are there decoding problems? Can the whole clip be decoded?
  If there are problems, try changing the source filter.
  
• audio can't be extracted correctly. If you extract just the audio is it complete? If you extract and reencode the audio, is it still complete?
  
• some problem during the muxing. If you tell Hybrid to keep intermediate files are the separated audio&video streams okay?
  
• missing time codes? If the source if vfr and not detected as such, output will be asynch. (yes, even .avi can be vfr)
  
• some tool is interfering. Is anything running while the encoding takes place that could interfer? (like antivirus scans etc.)
  
• a bug in one of the tools or a hardware problem.
  
• some decoding problem. (since you didn't change your settings, this is unlikely)
  

Cu Selur

[Bartoloni]

sorry i was not responding because the final output was full audio and only half of the video footage
on 2nd  attempt switching to MP4 contaimner (using ffmpeg and not mp4box) has again the desync on audio (on 45min of clip.. 2 seconds of delay on final output ... audio ends before video)
on 3d attemp i removed "prefer bestsource" and again the audio displacement
now i'm rendering using MKV container, disabling "prefer bestsource" (waiting for the end)

the problem is that i have 4 different tapes converted with the audio desync (2 from Video8 and 2 from VHS-C) i don't think this is a corruption on the original uncompressed AVI.. Potplayer is able to play the clip until the end witrhout issues.. .and same with VLC.

also loseless cut is showing the entire clip.

OK.. just ended the MKV version with bestsource disabled!

clip end with 4 seconds of silence. and audio is desync by 4 seconds

www.wcn.it/deb.txt

[Selur]

The debug output you linked to starts 2024.10.16
I looked at it only starting 2024.10.31, please make sure to not upload old DebugOutputs.
According to the debug output the source has 68559@25i frames by deinterlacing the output should have 137118 frames.
Vapoursynth uses LWLibavSource as source filter:
x264 reported:
"encoded 137024 frames, 115.30 fps, 7448.90 kb/s"
so the difference is 94 frames which is ~2 seconds (@50fps).
My guess is that libav how LWLibavSource and BestSource use it has problems with the file, so using FFms2 or AviSource (assuming you have a 64bit vfw codec installed on the system) as source filters and checking the frame count might help with the video.
Also test the other points I mentioned before if switching the source filter does not help.

According to ffmpeg the extracted audio is 00:45:40.41 mediainfo reported 45 min 42 s 357 ms on the input.
Seems like ffmpeg too has problems with the file.
Not sure how to fix this.

No clue, why these files cause problems.

If you open the file with mkvtoolnix gui and remux it:
a. is it still sync
b. if you open the file then in Hybrid (and deinterlace), is the frame count correct?


Cu Selur

Ps.: as a general side note: you might want to move your temp folder to a dedicated folder and add an exemption for it in your antivirus solution.

[Bartoloni]

i will try thank you!
antiivirus was removed using github defender removal script...
i will try to record the AVI using official blackmagic tool.. maybe is something related to the output of OBS (that is also using default SDK blackmagic code)

have a nice day

[Selur]

Can't OBS capture to mkv as container?
Windows Defender should not cause issues. (at least it does not here)